mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
44 KiB
44 KiB
Building Knowledge Graphs
Overview
This notebook demonstrates how to build knowledge graphs from entities and relationships using Semantica's graph building modules. You'll learn to use GraphBuilder and EntityResolver.
Documentation: API Reference
Learning Objectives
- Use
GraphBuilderto construct knowledge graphs - Use
EntityResolverto resolve entity conflicts Note: For deduplication, use thesemantica.deduplicationmodule.
Installation
Install Semantica from PyPI:
pip install semantica
# Or with all optional dependencies:
pip install semantica[all]
Step 1: Build Knowledge Graph
Construct a knowledge graph from entities and relationships.
In [1]:
!pip install semantica
Requirement already satisfied: semantica in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (0.0.5) Requirement already satisfied: numpy>=1.21.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.26.4) Requirement already satisfied: pandas>=1.3.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (2.1.4) Requirement already satisfied: scikit-learn>=1.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.6.1) Requirement already satisfied: spacy>=3.4.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (3.8.11) Requirement already satisfied: transformers>=4.20.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (4.53.2) Requirement already satisfied: torch>=1.12.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (2.2.1) Requirement already satisfied: sentence-transformers>=2.2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (3.2.1) Requirement already satisfied: rdflib>=6.2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (7.4.0) Requirement already satisfied: networkx>=2.8.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (3.5) Requirement already satisfied: matplotlib>=3.5.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (3.10.1) Requirement already satisfied: seaborn>=0.11.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.13.2) Requirement already satisfied: plotly>=5.10.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (5.17.0) Requirement already satisfied: requests>=2.28.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (2.32.5) Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (4.12.3) Requirement already satisfied: lxml>=4.9.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (6.0.2) Requirement already satisfied: pypdf2>=2.10.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (3.0.1) Requirement already satisfied: python-docx>=0.8.11 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.1.2) Requirement already satisfied: openpyxl>=3.0.10 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (3.1.5) Requirement already satisfied: pillow>=9.2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (10.4.0) Requirement already satisfied: librosa>=0.9.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.11.0) Requirement already satisfied: opencv-python>=4.6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (4.9.0.80) Requirement already satisfied: faiss-cpu>=1.7.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.9.0) Requirement already satisfied: weaviate-client>=3.15.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (4.18.1) Requirement already satisfied: qdrant-client>=1.3.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.12.2) Requirement already satisfied: neo4j>=5.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (6.0.3) Requirement already satisfied: falkordb>=1.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.2.2) Requirement already satisfied: pymongo>=4.2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (4.15.4) Requirement already satisfied: sqlalchemy>=1.4.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (2.0.23) Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (2.9.9) Requirement already satisfied: pymysql>=1.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.1.2) Requirement already satisfied: redis>=4.3.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (6.4.0) Requirement already satisfied: celery>=5.2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (5.3.4) Requirement already satisfied: kafka-python>=2.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (2.3.0) Requirement already satisfied: pulsar-client>=3.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (3.3.0) Requirement already satisfied: pika>=1.3.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.3.2) Requirement already satisfied: boto3>=1.24.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.36.0) Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (12.27.1) Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (2.18.2) Requirement already satisfied: pydantic>=1.10.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (2.12.3) Requirement already satisfied: click>=8.1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (8.2.1) Requirement already satisfied: rich>=12.5.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (13.7.1) Requirement already satisfied: tqdm>=4.64.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (4.67.1) Requirement already satisfied: pyyaml>=6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (6.0.1) Requirement already satisfied: toml>=0.10.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.10.2) Requirement already satisfied: python-dotenv>=0.20.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.1.1) Requirement already satisfied: loguru>=0.6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.7.3) Requirement already satisfied: structlog>=22.1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (24.4.0) Requirement already satisfied: prometheus-client>=0.14.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.18.0) Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.38.0) Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.38.0) Requirement already satisfied: opentelemetry-instrumentation in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.59b0) Requirement already satisfied: fastapi>=0.78.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.120.4) Requirement already satisfied: uvicorn>=0.18.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.38.0) Requirement already satisfied: pytest>=7.1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (7.4.3) Requirement already satisfied: pytest-cov>=3.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (7.0.0) Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (0.21.1) Requirement already satisfied: black>=22.6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (23.11.0) Requirement already satisfied: isort>=5.10.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (5.12.0) Requirement already satisfied: flake8>=4.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (7.3.0) Requirement already satisfied: mypy>=0.971 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (1.17.1) Requirement already satisfied: pre-commit>=2.19.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from semantica) (4.4.0) Requirement already satisfied: azure-core>=1.30.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0) Requirement already satisfied: cryptography>=2.1.4 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3) Requirement already satisfied: typing-extensions>=4.6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1) Requirement already satisfied: isodate>=0.6.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2) Requirement already satisfied: six>=1.11.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0) Requirement already satisfied: soupsieve>1.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5) Requirement already satisfied: mypy-extensions>=0.4.3 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from black>=22.6.0->semantica) (1.0.0) Requirement already satisfied: packaging>=22.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from black>=22.6.0->semantica) (24.2) Requirement already satisfied: pathspec>=0.9.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from black>=22.6.0->semantica) (0.11.2) Requirement already satisfied: platformdirs>=2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from black>=22.6.0->semantica) (3.11.0) Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from boto3>=1.24.0->semantica) (1.36.26) Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from boto3>=1.24.0->semantica) (1.0.1) Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from boto3>=1.24.0->semantica) (0.11.3) Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0) Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20) Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from celery>=5.2.0->semantica) (4.2.1) Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from celery>=5.2.0->semantica) (5.5.3) Requirement already satisfied: vine<6.0,>=5.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from celery>=5.2.0->semantica) (5.1.0) Requirement already satisfied: click-didyoumean>=0.3.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from celery>=5.2.0->semantica) (0.3.1) Requirement already satisfied: click-repl>=0.2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from celery>=5.2.0->semantica) (0.3.0) Requirement already satisfied: click-plugins>=1.1.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from celery>=5.2.0->semantica) (1.1.1) Requirement already satisfied: tzdata>=2022.7 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from celery>=5.2.0->semantica) (2025.2) Requirement already satisfied: colorama in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from click>=8.1.0->semantica) (0.4.6) Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1) Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\users\mohd kaif\appdata\roaming\python\python311\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40) Requirement already satisfied: cffi>=1.12 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1) Requirement already satisfied: pycparser in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22) Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from fastapi>=0.78.0->semantica) (0.46.2) Requirement already satisfied: annotated-doc>=0.0.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from fastapi>=0.78.0->semantica) (0.0.3) Requirement already satisfied: annotated-types>=0.6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pydantic>=1.10.0->semantica) (0.7.0) Requirement already satisfied: pydantic-core==2.41.4 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pydantic>=1.10.0->semantica) (2.41.4) Requirement already satisfied: typing-inspection>=0.4.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pydantic>=1.10.0->semantica) (0.4.2) Requirement already satisfied: anyio<5,>=3.6.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0) Requirement already satisfied: idna>=2.8 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10) Requirement already satisfied: sniffio>=1.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1) Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from flake8>=4.0.0->semantica) (0.7.0) Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from flake8>=4.0.0->semantica) (2.14.0) Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from flake8>=4.0.0->semantica) (3.4.0) Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0) Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0) Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1) Requirement already satisfied: google-resumable-media>=2.7.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2) Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0) Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0) Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8) Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0) Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2) Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1) Requirement already satisfied: rsa<5,>=3.1.4 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9) Requirement already satisfied: charset_normalizer<4,>=2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from requests>=2.28.0->semantica) (3.4.0) Requirement already satisfied: certifi>=2017.4.17 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from requests>=2.28.0->semantica) (2025.8.3) Requirement already satisfied: pyasn1>=0.1.3 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1) Requirement already satisfied: audioread>=2.1.9 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (3.0.1) Requirement already satisfied: numba>=0.51.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (0.61.2) Requirement already satisfied: scipy>=1.6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (1.15.2) Requirement already satisfied: joblib>=1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (1.3.2) Requirement already satisfied: decorator>=4.3.0 in c:\users\mohd kaif\appdata\roaming\python\python311\site-packages (from librosa>=0.9.0->semantica) (5.1.1) Requirement already satisfied: soundfile>=0.12.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (0.13.1) Requirement already satisfied: pooch>=1.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (1.8.2) Requirement already satisfied: soxr>=0.3.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1) Requirement already satisfied: lazy_loader>=0.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (0.4) Requirement already satisfied: msgpack>=1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from librosa>=0.9.0->semantica) (1.1.0) Requirement already satisfied: win32-setctime>=1.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from loguru>=0.6.0->semantica) (1.1.0) Requirement already satisfied: contourpy>=1.0.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2) Requirement already satisfied: cycler>=0.10 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0) Requirement already satisfied: kiwisolver>=1.3.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8) Requirement already satisfied: pyparsing>=2.3.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0) Requirement already satisfied: pytz in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from neo4j>=5.0.0->semantica) (2024.2) Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0) Requirement already satisfied: et-xmlfile in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0) Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0) Requirement already satisfied: zipp>=0.5 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0) Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0) Requirement already satisfied: tenacity>=6.2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from plotly>=5.10.0->semantica) (8.5.0) Requirement already satisfied: cfgv>=2.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0) Requirement already satisfied: identify>=1.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15) Requirement already satisfied: nodeenv>=0.11.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1) Requirement already satisfied: virtualenv>=20.10.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6) Requirement already satisfied: wcwidth in c:\users\mohd kaif\appdata\roaming\python\python311\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9) Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pymongo>=4.2.0->semantica) (2.4.2) Requirement already satisfied: iniconfig in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pytest>=7.1.0->semantica) (2.0.0) Requirement already satisfied: pluggy<2.0,>=0.12 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from pytest>=7.1.0->semantica) (1.5.0) Requirement already satisfied: coverage>=7.10.6 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0) Requirement already satisfied: grpcio>=1.41.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0) Requirement already satisfied: grpcio-tools>=1.41.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3) Requirement already satisfied: httpx>=0.20.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1) Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1) Requirement already satisfied: pywin32>=226 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311) Requirement already satisfied: setuptools in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0) Requirement already satisfied: httpcore==1.* in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9) Requirement already satisfied: h11>=0.16 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0) Requirement already satisfied: h2<5,>=3 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0) Requirement already satisfied: hyperframe<7,>=6.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1) Requirement already satisfied: hpack<5,>=4.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0) Requirement already satisfied: markdown-it-py>=2.2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from rich>=12.5.0->semantica) (3.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from rich>=12.5.0->semantica) (2.19.2) Requirement already satisfied: mdurl~=0.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2) Requirement already satisfied: threadpoolctl>=3.1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0) Requirement already satisfied: huggingface-hub>=0.20.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2) Requirement already satisfied: filelock in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from transformers>=4.20.0->semantica) (3.16.1) Requirement already satisfied: regex!=2019.12.17 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from transformers>=4.20.0->semantica) (2024.11.6) Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from transformers>=4.20.0->semantica) (0.21.4) Requirement already satisfied: safetensors>=0.4.3 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from transformers>=4.20.0->semantica) (0.5.3) Requirement already satisfied: fsspec>=2023.5.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0) Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (3.0.12) Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (1.0.5) Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (1.0.15) Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (2.0.13) Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (3.0.12) Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (8.3.10) Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (1.1.3) Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (2.5.2) Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (2.0.10) Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (0.4.3) Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (0.20.0) Requirement already satisfied: jinja2 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from spacy>=3.4.0->semantica) (3.1.6) Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3) Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5) Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0) Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0) Requirement already satisfied: wrapt in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2) Requirement already satisfied: greenlet!=0.4.17 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3) Requirement already satisfied: sympy in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.12.0->semantica) (1.13.3) Requirement already satisfied: distlib<1,>=0.3.7 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7) Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0) Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0) Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0) Requirement already satisfied: MarkupSafe>=2.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3) Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\mohd kaif\appdata\local\programs\python\python311\lib\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)
WARNING: Ignoring invalid distribution ~gno (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages) WARNING: Ignoring invalid distribution ~lotly (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages) WARNING: Ignoring invalid distribution ~ython-socketio (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages) WARNING: Ignoring invalid distribution ~gno (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages) WARNING: Ignoring invalid distribution ~lotly (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages) WARNING: Ignoring invalid distribution ~ython-socketio (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages) WARNING: Ignoring invalid distribution ~gno (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages) WARNING: Ignoring invalid distribution ~lotly (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages) WARNING: Ignoring invalid distribution ~ython-socketio (C:\Users\Mohd Kaif\AppData\Local\Programs\Python\Python311\Lib\site-packages)
In [4]:
from semantica.kg import GraphBuilder
from semantica.semantic_extract import NERExtractor, RelationExtractor
builder = GraphBuilder()
ner_extractor = NERExtractor()
relation_extractor = RelationExtractor()
text = "Apple Inc. is a technology company. Tim Cook is the CEO of Apple Inc. Apple Inc. is headquartered in Cupertino, California."
entities_list = ner_extractor.extract(text)
relationships_list = relation_extractor.extract(text, entities_list)
entities = []
for i, entity in enumerate(entities_list[:5], 1):
entities.append({
"id": f"e{i}",
"type": entity.label,
"name": entity.text,
"properties": {}
})
relationships = []
for i, rel in enumerate(relationships_list[:3], 1):
relationships.append({
"source": f"e{1}",
"target": f"e{i+1}",
"type": rel.predicate,
"properties": {}
})
knowledge_graph = builder.build(entities, relationships)
print(f"Built knowledge graph with {len(knowledge_graph.get('entities', []))} entities")
print(f"Relationships: {len(knowledge_graph.get('relationships', []))}")In [7]:
from semantica.kg import EntityResolver
entity_resolver = EntityResolver()
resolved_entities = entity_resolver.resolve_entities(entities)
print(f"Original entities: {len(entities)}")
print(f"Resolved entities: {len(resolved_entities)}")Original entities: 5 Resolved entities: 4
In [13]:
from semantica.deduplication import DuplicateDetector, EntityMerger, MergeStrategy
# Detect duplicates
detector = DuplicateDetector(similarity_threshold=0.8)
duplicate_groups = detector.detect_duplicate_groups(knowledge_graph.get('entities', []))
# Merge duplicates
merger = EntityMerger()
merge_operations = merger.merge_duplicates(
knowledge_graph.get('entities', []),
strategy=MergeStrategy.KEEP_MOST_COMPLETE
)
deduplicated_entities = [op.merged_entity for op in merge_operations]
print(f"Original entities: {len(knowledge_graph.get('entities', []))}")
print(f"Deduplicated entities: {len(deduplicated_entities)}")
Original entities: 4 Deduplicated entities: 0