{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/01_Welcome_to_Semantica.ipynb)\n",
    "\n",
    "Semantica is a **semantic intelligence and knowledge engineering framework**. It helps you:\n",
    "\n",
    "- Build **knowledge graphs** from unstructured and semi-structured data\n",
    "- Create a unified **semantic layer** on top of diverse data sources\n",
    "- Power **GraphRAG**, AI agents, and multi-agent systems with structured knowledge\n",
    "- Incorporate **temporal and quality-aware reasoning** into your applications\n",
    "\n",
    "### Core Capabilities\n",
    "\n",
    "- **Universal ingestion**: Files, web, feeds, databases, repositories, streams\n",
    "- **Rich parsing**: PDFs, Office documents, HTML, JSON, CSV, images, code\n",
    "- **Normalization**: Cleaning, language detection, entity normalization, date/number standardization\n",
    "- **Semantic extraction**: Named entities, relationships, events, semantic networks\n",
    "- **Knowledge graph construction**: Property graphs from entities and relations\n",
    "- **Embeddings and vector search**: Text and graph embeddings, hybrid retrieval\n",
    "- **Reasoning and ontology**: Rule-based inference, ontology generation and validation\n",
    "- **Visualization and analytics**: Graph visualizations and quality metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Who Is Semantica For?\n",
    "\n",
    "- **AI/ML engineers** building GraphRAG systems, agents, and tools that need long-term memory\n",
    "- **Data engineers** orchestrating semantic enrichment pipelines over large, heterogeneous datasets\n",
    "- **Knowledge engineers and ontologists** designing and maintaining formal knowledge structures\n",
    "- **Researchers and analysts** creating domain knowledge graphs from documents and data feeds\n",
    "- **Product and platform teams** embedding semantic intelligence into applications and services"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Architecture Overview\n",
    "\n",
    "Semantica is organized as three conceptual layers and multiple concrete modules.\n",
    "\n",
    "### Layers\n",
    "\n",
    "- **Input Layer**\n",
    "  - Connects to files, web pages, APIs, databases, email, feeds, repositories, and streams\n",
    "  - Normalizes these different sources into a unified internal representation\n",
    "\n",
    "- **Semantic Layer**\n",
    "  - Performs parsing, cleaning, semantic extraction, graph construction, embeddings, and reasoning\n",
    "  - This is where **unstructured data becomes structured knowledge**\n",
    "\n",
    "- **Output Layer**\n",
    "  - Exposes knowledge graphs, embeddings, ontologies, and analytics\n",
    "  - Integrates with vector stores, graph databases, and downstream applications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🧩 Semantica Modules Reference\n",
    "\n",
    "Semantica is modular by design. Here is a comprehensive guide to all available modules, grouped by functionality.\n",
    "\n",
    "### 📥 Ingestion & Parsing\n",
    "Modules that handle raw data input and structure.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`ingest`** | **Data Ingestion**<br>Connects to data sources. | • File, Web, Feed, Stream ingestion<br>• DB, Email, Repo, MCP support |\n",
    "| **`parse`** | **Document Parsing**<br>Parses raw content into structures. | • PDF, HTML, JSON, CSV, Excel<br>• Image & Code parsing |\n",
    "\n",
    "### ⚙️ Data Processing\n",
    "Modules that clean, normalize, and split data.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`normalize`** | **Data Normalization**<br>Cleans and standardizes text. | • Text cleaning & Language detection<br>• Entity, Date, Number normalization |\n",
    "| **`split`** | **Chunking**<br>Splits documents for RAG. | • Recursive character splitting<br>• Semantic & Token-based splitting |\n",
    "\n",
    "### 🧠 Extraction & Enrichment\n",
    "Modules that extract meaning, structure, and vectors from raw data.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`semantic_extract`** | **Information Extraction**<br>Extracts entities and relations. | • NER & Relation Extraction<br>• Event & Semantic Network detection |\n",
    "| **`context`** | **Agent Memory**<br>Manages state for AI agents. | • Long-term memory & history<br>• Context graph & RAG integration |\n",
    "\n",
    "### 🕸️ Knowledge Graph Core\n",
    "Modules for building, refining, and resolving knowledge graphs.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`kg`** | **Graph Construction**<br>Builds and analyzes graphs. | • Graph Building & Analysis<br>• Validation & Entity Resolution |\n",
    "| **`conflicts`** | **Conflict Resolution**<br>Resolves data contradictions. | • Source reliability scoring<br>• Truth discovery algorithms |\n",
    "| **`deduplication`** | **Entity Resolution**<br>Merges duplicate entities. | • Similarity-based blocking<br>• Clustering & Canonicalization |\n",
    "\n",
    "### 💾 Storage & Retrieval\n",
    "Modules for persisting and querying data.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`embeddings`** | **Vector Embeddings**<br>Generates semantic vectors. | • Text & Graph embeddings<br>• Multi-provider support (OpenAI, etc.) |\n",
    "| **`vector_store`** | **Vector Database**<br>Stores and searches vectors. | • Similarity search & Filtering<br>• Hybrid search (Vector + Keyword) |\n",
    "| **`graph_store`** | **Property Graph Store**<br>Persists graph data. | • Neo4j, FalkorDB adapters<br>• Cypher query support |\n",
    "| **`triplet_store`** | **RDF Store**<br>Persists semantic triplets. | • SPARQL endpoints<br>• BlazeGraph, Jena, Virtuoso adapters |\n",
    "\n",
    "### 🔎 Reasoning & Analysis\n",
    "Modules for deriving new knowledge and evaluating quality.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`reasoning`** | **Reasoner Facade**<br>Unified interface for inference. | • Datalog/Rule-based inference<br>• Forward/Backward chaining |\n",
    "| **`ontology`** | **Ontology Management**<br>Manages schema and definitions. | • Ontology generation from data<br>• Validation & Evolution |\n",
    "| **`visualization`** | **Visual Analytics**<br>Visualizes graphs and metrics. | • 2D/3D Graph visualization<br>• Interactive plots & dashboards |\n",
    "| **`evals`** | **Evaluation**<br>Benchmarks pipeline quality. | • RAG & Graph quality metrics<br>• Ground truth comparison |\n",
    "\n",
    "### 🛠️ Orchestration & Utils\n",
    "Modules for managing the framework and workflows.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`core`** | **Framework Core**<br>Main entry point and config. | • Lifecycle management<br>• Plugin system & Configuration |\n",
    "| **`pipeline`** | **Workflow Orchestration**<br>Manages complex flows. | • DAG execution & Retries<br>• Error handling & Observability |\n",
    "| **`seed`** | **Data Seeding**<br>Initializes knowledge bases. | • Taxonomy & Ontology seeding<br>• Reference data loading |\n",
    "| **`export`** | **Data Export**<br>Exports data to files. | • JSON, CSV, RDF, GEXF export<br>• Report generation |\n",
    "| **`utils`** | **Utilities**<br>Common helper functions. | • Logging, Async, Hashing<br>• Text processing helpers |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Core Concepts (High-Level)\n",
    "\n",
    "- **Knowledge graph**\n",
    "  - Nodes represent entities such as people, organizations, locations, events, or concepts\n",
    "  - Edges represent relationships such as `works_for`, `located_in`, `founded_by`\n",
    "  - Properties capture attributes and metadata such as timestamps, sources, and confidence\n",
    "\n",
    "- **Entities and relationships**\n",
    "  - Entities are extracted from text and data using NER\n",
    "  - Relationships connect entities and are extracted using pattern-based, model-based, or LLM-based methods\n",
    "\n",
    "- **Embeddings**\n",
    "  - Numerical vectors that encode semantic meaning of text or graph structures\n",
    "  - Used for semantic search, clustering, and similarity-based retrieval\n",
    "\n",
    "- **GraphRAG**\n",
    "  - Combines vector search with graph traversal\n",
    "  - Uses both embeddings and graph structure to retrieve rich, context-aware information\n",
    "\n",
    "- **Ontology**\n",
    "  - A formal model of classes, relationships, and constraints in a domain\n",
    "  - Used to standardize meaning, enable reasoning, and integrate heterogeneous data\n",
    "\n",
    "- **Quality and governance**\n",
    "  - Quality metrics (completeness, consistency, accuracy, coverage)\n",
    "  - Conflict detection and resolution at the knowledge graph level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation\n",
    "\n",
    "You can install Semantica from PyPI. In this notebook, we use a pip cell so it can run in local Jupyter or Colab.\n",
    "\n",
    "Equivalent shell commands:\n",
    "\n",
    "```bash\n",
    "pip install semantica\n",
    "pip install semantica[all]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Configuration\n",
    "\n",
    "Semantica uses configuration for API keys, embedding providers, and knowledge graph options. The example below mirrors a typical configuration while staying simple enough for a notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'api_keys:\\n  openai: your_key_here\\n  anthropic: your_key_here\\nembedding:\\n  provider: openai\\n  model: text-embedding-3-large\\n  dimensions: 3072\\nknowledge_graph:\\n  backend: networkx\\n  temporal: true\\n'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.core import Config\n",
    "config = Config()\n",
    "print(config.to_yaml())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome to Semantica\n",
    "\n",
    "**Open Source Framework for Semantic Layer & Knowledge Engineering**\n",
    "\n",
    "Semantica is a Python framework for transforming raw, messy, multi-source data into **semantic layers** and **knowledge graphs** that are ready to power GraphRAG, AI agents, multi-agent systems, and analytical applications.\n",
    "\n",
    "This notebook is an executable introduction. It combines:\n",
    "\n",
    "- High-level explanation of what Semantica is and why it exists\n",
    "- A structured tour of the architecture and key modules\n",
    "- Small, runnable code snippets that show the end-to-end flow\n",
    "\n",
    "**You should use this notebook to understand the big picture, not to learn every API in depth.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)\n",
      "Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
      "Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.1.4)\n",
      "Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
      "Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
      "Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
      "Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
      "Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
      "Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
      "Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.5)\n",
      "Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
      "Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
      "Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
      "Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
      "Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.12.3)\n",
      "Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
      "Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
      "Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
      "Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
      "Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
      "Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
      "Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
      "Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
      "Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
      "Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
      "Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
      "Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
      "Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
      "Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
      "Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
      "Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
      "Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
      "Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
      "Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
      "Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
      "Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
      "Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
      "Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
      "Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
      "Requirement already satisfied: pydantic>=1.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
      "Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
      "Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (13.7.1)\n",
      "Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
      "Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
      "Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
      "Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
      "Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
      "Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
      "Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.18.0)\n",
      "Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
      "Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
      "Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.59b0)\n",
      "Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
      "Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
      "Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
      "Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
      "Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
      "Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
      "Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
      "Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
      "Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
      "Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
      "Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
      "Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
      "Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1)\n",
      "Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
      "Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
      "Requirement already satisfied: soupsieve>1.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
      "Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
      "Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
      "Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
      "Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (3.11.0)\n",
      "Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
      "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
      "Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
      "Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
      "Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
      "Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
      "Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
      "Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
      "Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
      "Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
      "Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
      "Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
      "Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
      "Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
      "Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
      "Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
      "Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
      "Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
      "Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (2.41.4)\n",
      "Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.4.2)\n",
      "Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
      "Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
      "Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
      "Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
      "Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
      "Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
      "Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
      "Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
      "Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
      "Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
      "Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
      "Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
      "Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
      "Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
      "Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2)\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
      "Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
      "Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
      "Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
      "Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
      "Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
      "Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
      "Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
      "Requirement already satisfied: decorator>=4.3.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.9.0->semantica) (5.1.1)\n",
      "Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
      "Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
      "Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
      "Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
      "Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
      "Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
      "Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
      "Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
      "Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
      "Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
      "Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0)\n",
      "Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0)\n",
      "Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0)\n",
      "Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
      "Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
      "Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
      "Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
      "Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6)\n",
      "Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
      "Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pymongo>=4.2.0->semantica) (2.4.2)\n",
      "Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
      "Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
      "Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
      "Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
      "Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
      "Requirement already satisfied: httpx>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1)\n",
      "Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
      "Requirement already satisfied: pywin32>=226 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311)\n",
      "Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
      "Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9)\n",
      "Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0)\n",
      "Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
      "Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
      "Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
      "Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
      "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (2.19.2)\n",
      "Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
      "Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
      "Requirement already satisfied: huggingface-hub>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2)\n",
      "Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (3.16.1)\n",
      "Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
      "Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.21.4)\n",
      "Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0)\n",
      "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
      "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
      "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
      "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
      "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
      "Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
      "Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
      "Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
      "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
      "Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
      "Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
      "Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
      "Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
      "Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
      "Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
      "Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
      "Requirement already satisfied: wrapt in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
      "Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from torch>=1.12.0->semantica) (1.13.3)\n",
      "Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
      "Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
      "Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0)\n",
      "Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
      "Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
     ]
    }
   ],
   "source": [
    "!pip install -U semantica\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What Is Semantica?\n",
    "\n",
    "Semantica is a **semantic intelligence and knowledge engineering framework**. It helps you:\n",
    "\n",
    "- Build **knowledge graphs** from unstructured and semi-structured data\n",
    "- Create a unified **semantic layer** on top of diverse data sources\n",
    "- Power **GraphRAG**, AI agents, and multi-agent systems with structured knowledge\n",
    "- Incorporate **temporal and quality-aware reasoning** into your applications\n",
    "\n",
    "### Core Capabilities\n",
    "\n",
    "- **Universal ingestion**: Files, web, feeds, databases, repositories, streams\n",
    "- **Rich parsing**: PDFs, Office documents, HTML, JSON, CSV, images, code\n",
    "- **Normalization**: Cleaning, language detection, entity normalization, date/number standardization\n",
    "- **Semantic extraction**: Named entities, relationships, events, semantic networks\n",
    "- **Knowledge graph construction**: Property graphs from entities and relations\n",
    "- **Embeddings and vector search**: Text and graph embeddings, hybrid retrieval\n",
    "- **Reasoning and ontology**: Rule-based inference, ontology generation and validation\n",
    "- **Visualization and analytics**: Graph visualizations and quality metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Who Is Semantica For?\n",
    "\n",
    "- **AI/ML engineers** building GraphRAG systems, agents, and tools that need long-term memory\n",
    "- **Data engineers** orchestrating semantic enrichment pipelines over large, heterogeneous datasets\n",
    "- **Knowledge engineers and ontologists** designing and maintaining formal knowledge structures\n",
    "- **Researchers and analysts** creating domain knowledge graphs from documents and data feeds\n",
    "- **Product and platform teams** embedding semantic intelligence into applications and services"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Architecture Overview\n",
    "\n",
    "Semantica is organized as three conceptual layers and multiple concrete modules.\n",
    "\n",
    "### Layers\n",
    "\n",
    "- **Input Layer**\n",
    "  - Connects to files, web pages, APIs, databases, email, feeds, repositories, and streams\n",
    "  - Normalizes these different sources into a unified internal representation\n",
    "\n",
    "- **Semantic Layer**\n",
    "  - Performs parsing, cleaning, semantic extraction, graph construction, embeddings, and reasoning\n",
    "  - This is where **unstructured data becomes structured knowledge**\n",
    "\n",
    "- **Output Layer**\n",
    "  - Exposes knowledge graphs, embeddings, ontologies, and analytics\n",
    "  - Integrates with vector stores, graph databases, and downstream applications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🧩 Semantica Modules Reference\n",
    "\n",
    "Semantica is modular by design. Here is a comprehensive guide to all available modules, grouped by functionality.\n",
    "\n",
    "### 📥 Ingestion & Parsing\n",
    "Modules that handle raw data input and structure.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`ingest`** | **Data Ingestion**<br>Connects to data sources. | • File, Web, Feed, Stream ingestion<br>• DB, Email, Repo, MCP support |\n",
    "| **`parse`** | **Document Parsing**<br>Parses raw content into structures. | • PDF, HTML, JSON, CSV, Excel<br>• Image & Code parsing |\n",
    "\n",
    "### ⚙️ Data Processing\n",
    "Modules that clean, normalize, and split data.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`normalize`** | **Data Normalization**<br>Cleans and standardizes text. | • Text cleaning & Language detection<br>• Entity, Date, Number normalization |\n",
    "| **`split`** | **Chunking**<br>Splits documents for RAG. | • Recursive character splitting<br>• Semantic & Token-based splitting |\n",
    "\n",
    "### 🧠 Extraction & Enrichment\n",
    "Modules that extract meaning, structure, and vectors from raw data.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`semantic_extract`** | **Information Extraction**<br>Extracts entities and relations. | • NER & Relation Extraction<br>• Event & Semantic Network detection |\n",
    "| **`context`** | **Agent Memory**<br>Manages state for AI agents. | • Long-term memory & history<br>• Context graph & RAG integration |\n",
    "\n",
    "### 🕸️ Knowledge Graph Core\n",
    "Modules for building, refining, and resolving knowledge graphs.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`kg`** | **Graph Construction**<br>Builds and analyzes graphs. | • Graph Building & Analysis<br>• Validation & Entity Resolution |\n",
    "| **`conflicts`** | **Conflict Resolution**<br>Resolves data contradictions. | • Source reliability scoring<br>• Truth discovery algorithms |\n",
    "| **`deduplication`** | **Entity Resolution**<br>Merges duplicate entities. | • Similarity-based blocking<br>• Clustering & Canonicalization |\n",
    "\n",
    "### 💾 Storage & Retrieval\n",
    "Modules for persisting and querying data.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`embeddings`** | **Vector Embeddings**<br>Generates semantic vectors. | • Text & Graph embeddings<br>• Multi-provider support (OpenAI, etc.) |\n",
    "| **`vector_store`** | **Vector Database**<br>Stores and searches vectors. | • Similarity search & Filtering<br>• Hybrid search (Vector + Keyword) |\n",
    "| **`graph_store`** | **Property Graph Store**<br>Persists graph data. | • Neo4j, FalkorDB adapters<br>• Cypher query support |\n",
    "| **`triplet_store`** | **RDF Store**<br>Persists semantic triplets. | • SPARQL endpoints<br>• BlazeGraph, Jena, Virtuoso adapters |\n",
    "\n",
    "### 🔎 Reasoning & Analysis\n",
    "Modules for deriving new knowledge and evaluating quality.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`reasoning`** | **Reasoner Facade**<br>Unified interface for inference. | • Datalog/Rule-based inference<br>• Forward/Backward chaining<br>• Automated explanation generation |\n",
    "| **`ontology`** | **Ontology Management**<br>Manages schema and definitions. | • Ontology generation from data<br>• Validation & Evolution |\n",
    "| **`visualization`** | **Visual Analytics**<br>Visualizes graphs and metrics. | • 2D/3D Graph visualization<br>• Interactive plots & dashboards |\n",
    "| **`evals`** | **Evaluation**<br>Benchmarks pipeline quality. | • RAG & Graph quality metrics<br>• Ground truth comparison |\n",
    "\n",
    "### 🛠️ Orchestration & Utils\n",
    "Modules for managing the framework and workflows.\n",
    "\n",
    "| Module | Description | Key Capabilities |\n",
    "| :--- | :--- | :--- |\n",
    "| **`core`** | **Framework Core**<br>Main entry point and config. | • Lifecycle management<br>• Plugin system & Configuration |\n",
    "| **`pipeline`** | **Workflow Orchestration**<br>Manages complex flows. | • DAG execution & Retries<br>• Error handling & Observability |\n",
    "| **`seed`** | **Data Seeding**<br>Initializes knowledge bases. | • Taxonomy & Ontology seeding<br>• Reference data loading |\n",
    "| **`export`** | **Data Export**<br>Exports data to files. | • JSON, CSV, RDF, GEXF export<br>• Report generation |\n",
    "| **`utils`** | **Utilities**<br>Common helper functions. | • Logging, Async, Hashing<br>• Text processing helpers |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Core Concepts (High-Level)\n",
    "\n",
    "- **Knowledge graph**\n",
    "  - Nodes represent entities such as people, organizations, locations, events, or concepts\n",
    "  - Edges represent relationships such as `works_for`, `located_in`, `founded_by`\n",
    "  - Properties capture attributes and metadata such as timestamps, sources, and confidence\n",
    "\n",
    "- **Entities and relationships**\n",
    "  - Entities are extracted from text and data using NER\n",
    "  - Relationships connect entities and are extracted using pattern-based, model-based, or LLM-based methods\n",
    "\n",
    "- **Embeddings**\n",
    "  - Numerical vectors that encode semantic meaning of text or graph structures\n",
    "  - Used for semantic search, clustering, and similarity-based retrieval\n",
    "\n",
    "- **GraphRAG**\n",
    "  - Combines vector search with graph traversal\n",
    "  - Uses both embeddings and graph structure to retrieve rich, context-aware information\n",
    "\n",
    "- **Ontology**\n",
    "  - A formal model of classes, relationships, and constraints in a domain\n",
    "  - Used to standardize meaning, enable reasoning, and integrate heterogeneous data\n",
    "\n",
    "- **Quality and governance**\n",
    "  - Quality metrics (completeness, consistency, accuracy, coverage)\n",
    "  - Conflict detection and resolution at the knowledge graph level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation\n",
    "\n",
    "You can install Semantica from PyPI. In this notebook, we use a pip cell so it can run in local Jupyter or Colab.\n",
    "\n",
    "Equivalent shell commands:\n",
    "\n",
    "```bash\n",
    "pip install semantica\n",
    "pip install semantica[all]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Configuration\n",
    "\n",
    "Semantica uses configuration for API keys, embedding providers, and knowledge graph options. The example below mirrors a typical configuration while staying simple enough for a notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'api_keys:\\n  openai: your_key_here\\n  anthropic: your_key_here\\nembedding:\\n  provider: openai\\n  model: text-embedding-3-large\\n  dimensions: 3072\\nknowledge_graph:\\n  backend: networkx\\n  temporal: true\\n'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "from pathlib import Path\n",
    "\n",
    "os.environ[\"SEMANTICA_API_KEY\"] = \"your_openai_key\"\n",
    "os.environ[\"SEMANTICA_EMBEDDING_PROVIDER\"] = \"openai\"\n",
    "os.environ[\"SEMANTICA_MODEL_NAME\"] = \"gpt-4\"\n",
    "\n",
    "config_text = \"\"\"api_keys:\n",
    "  openai: your_key_here\n",
    "  anthropic: your_key_here\n",
    "embedding:\n",
    "  provider: openai\n",
    "  model: text-embedding-3-large\n",
    "  dimensions: 3072\n",
    "knowledge_graph:\n",
    "  backend: networkx\n",
    "  temporal: true\n",
    "\"\"\"\n",
    "Path(\"config.yaml\").write_text(config_text, encoding=\"utf-8\")\n",
    "Path(\"config.yaml\").read_text(encoding=\"utf-8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup: Create Sample Data\n",
    "\n",
    "First, let's create a small sample document to work with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Created sample document at welcome_docs\\apple.txt\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "docs_dir = Path(\"welcome_docs\")\n",
    "docs_dir.mkdir(exist_ok=True)\n",
    "text_path = docs_dir / \"apple.txt\"\n",
    "text_content = (\n",
    "    \"Apple Inc. was founded by Steve Jobs, Steve Wozniak and Ronald Wayne in\"\n",
    "    \" Cupertino, California.\"\n",
    ")\n",
    "text_path.write_text(text_content, encoding=\"utf-8\")\n",
    "print(f\"Created sample document at {text_path}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Minimal End-to-End Pipeline\n",
    "\n",
    "The next example shows how to explicitly use several modules in sequence. This mirrors the architecture discussed earlier:\n",
    "\n",
    "1. Ingest a directory of documents\n",
    "2. Parse them into structured documents\n",
    "3. Normalize text\n",
    "4. Extract entities and relationships\n",
    "5. Build and analyze a knowledge graph\n",
    "6. Create embeddings and store them in a vector store\n",
    "7. Run a hybrid semantic search query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\sentence_transformers\\cross_encoder\\CrossEncoder.py:13: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)\n",
      "  from tqdm.autonotebook import tqdm, trange\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CommunityDetector</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>VectorStore</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>HybridSearch</td><td>-</td><td>3.15s</td></tr><tr><td>✅</td><td>Semantica is embedding</td><td>💾 embeddings</td><td>TextEmbedder</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is visualizing</td><td>📈 visualization</td><td>KGVisualizer</td><td>-</td><td>0.17s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>OntologyGenerator</td><td>-</td><td>0.04s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>ClassInferrer</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>PropertyGenerator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>-</td><td>0.01s</td></tr></table></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Graph is empty or has no edges, returning 0 communities\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.ingest import FileIngestor\n",
    "from semantica.parse import DocumentParser\n",
    "from semantica.normalize import TextNormalizer\n",
    "from semantica.semantic_extract import NERExtractor, RelationExtractor\n",
    "from semantica.kg import GraphBuilder, GraphAnalyzer\n",
    "from semantica.embeddings import EmbeddingGenerator\n",
    "from semantica.vector_store import VectorStore, HybridSearch\n",
    "\n",
    "ingestor = FileIngestor()\n",
    "documents = ingestor.ingest(str(docs_dir))\n",
    "\n",
    "parser = DocumentParser()\n",
    "parsed_docs = parser.parse(documents)\n",
    "\n",
    "normalizer = TextNormalizer()\n",
    "normalized_docs = normalizer.normalize(parsed_docs)\n",
    "\n",
    "ner = NERExtractor()\n",
    "entities = ner.extract(normalized_docs)\n",
    "rel_extractor = RelationExtractor()\n",
    "relationships = rel_extractor.extract(normalized_docs, entities)\n",
    "\n",
    "builder = GraphBuilder()\n",
    "kg = builder.build(entities, relationships)\n",
    "analyzer = GraphAnalyzer()\n",
    "metrics = analyzer.analyze(kg)\n",
    "\n",
    "emb_generator = EmbeddingGenerator()\n",
    "embeddings = emb_generator.generate_embeddings(documents, data_type=\"text\")\n",
    "\n",
    "vec_store = VectorStore()\n",
    "vec_store.store(embeddings, documents, metadata={})\n",
    "hybrid = HybridSearch(vec_store)\n",
    "search_results = hybrid.search(\"Apple founders\", top_k=3)\n",
    "len(search_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualization\n",
    "\n",
    "Semantica includes a powerful visualization module. Here we create an interactive network graph from the knowledge graph built above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hoverinfo": "none",
         "line": {
          "color": "#888",
          "width": 1
         },
         "mode": "lines",
         "type": "scatter",
         "x": [],
         "y": []
        },
        {
         "hoverinfo": "text",
         "marker": {
          "color": [
           "#45B7D1",
           "#4ECDC4",
           "#4ECDC4",
           "#FFA07A",
           "#FFA07A",
           "#FF6B6B"
          ],
          "line": {
           "color": "white",
           "width": 2
          },
          "opacity": 0.9,
          "size": [
           10,
           10,
           10,
           10,
           10,
           10
          ]
         },
         "mode": "markers+text",
         "text": [
          "<b>Apple Inc.</b><br>Type: ORG",
          "<b>Steve Jobs</b><br>Type: PERSON",
          "<b>Ronald Wayne</b><br>Type: PERSON",
          "<b>Cupertino</b><br>Type: GPE",
          "<b>California</b><br>Type: GPE",
          "<b>94</b><br>Type: CARDINAL"
         ],
         "textposition": "top center",
         "type": "scatter",
         "x": [
          -0.511077786663866,
          -0.5032568450956351,
          0.4844459936195711,
          -0.9048170496441226,
          0.4347056877840527,
          1
         ],
         "y": [
          0.8619642746036202,
          -0.8465963477111661,
          -0.9884258994576173,
          0.019420814915733102,
          0.9747682870772485,
          -0.021131129427818373
         ]
        }
       ],
       "layout": {
        "hovermode": "closest",
        "margin": {
         "b": 20,
         "l": 5,
         "r": 5,
         "t": 40
        },
        "showlegend": false,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Knowledge Graph Network"
        },
        "xaxis": {
         "showgrid": false,
         "showticklabels": false,
         "zeroline": false
        },
        "yaxis": {
         "showgrid": false,
         "showticklabels": false,
         "zeroline": false
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from semantica.visualization import KGVisualizer\n",
    "\n",
    "# Create a visualizer instance\n",
    "viz = KGVisualizer(layout=\"force\", color_scheme=\"vibrant\")\n",
    "\n",
    "# Generate an interactive network visualization\n",
    "# This returns a Plotly figure object that renders in the notebook\n",
    "fig = viz.visualize_network(kg, output=\"interactive\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ontology Generation\n",
    "\n",
    "You can also automatically generate an ontology (a formal model of your domain) from the extracted entities and relationships."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Org', 'Person', 'Gpe']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.ontology import OntologyGenerator\n",
    "\n",
    "generator = OntologyGenerator(base_uri=\"https://example.org/ontology/\")\n",
    "\n",
    "# Generate ontology from the extracted data\n",
    "ontology = generator.generate_ontology({\n",
    "    \"entities\": entities,\n",
    "    \"relationships\": relationships\n",
    "})\n",
    "\n",
    "# View inferred classes\n",
    "[cls[\"name\"] for cls in ontology.get(\"classes\", [])[:5]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced: Data Splitting and Chunking\n",
    "\n",
    "For RAG applications, splitting documents into smaller chunks is essential. Semantica provides a `split` module for this purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original documents: 1\n",
      "Generated chunks: 1\n"
     ]
    }
   ],
   "source": [
    "from semantica.split import TextSplitter\n",
    "\n",
    "splitter = TextSplitter(chunk_size=100, chunk_overlap=20)\n",
    "chunks = splitter.split_documents(documents)\n",
    "\n",
    "print(f\"Original documents: {len(documents)}\")\n",
    "print(f\"Generated chunks: {len(chunks)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced: Reasoning and Inference\n",
    "\n",
    "The `reasoning` module allows you to derive new facts from existing knowledge using logic rules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inferred 0 new facts\n"
     ]
    }
   ],
   "source": [
    "from semantica.reasoning import Reasoner\n",
    "\n",
    "# Simple rule: If X founded Y, then X works_for Y\n",
    "rule = \"\"\"\n",
    "IF (?x founded ?y) THEN (?x works_for ?y)\n",
    "\"\"\"\n",
    "\n",
    "reasoner = Reasoner()\n",
    "reasoner.add_rule(rule)\n",
    "inferred_facts = reasoner.infer_facts(kg)\n",
    "\n",
    "print(f\"Inferred {len(inferred_facts)} new facts\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced: Export and Persistence\n",
    "\n",
    "You can save your knowledge graph to disk or export it to standard formats like CSV, JSON, or RDF."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "GraphExporter.export() missing 1 required positional argument: 'file_path'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[12], line 4\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21;01msemantica\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mexport\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m GraphExporter\n\u001b[0;32m      3\u001b[0m exporter \u001b[38;5;241m=\u001b[39m GraphExporter()\n\u001b[1;32m----> 4\u001b[0m \u001b[43mexporter\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mexport\u001b[49m\u001b[43m(\u001b[49m\u001b[43mkg\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mjson\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43moutput_path\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mknowledge_graph.json\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m      5\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGraph exported to knowledge_graph.json\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "\u001b[1;31mTypeError\u001b[0m: GraphExporter.export() missing 1 required positional argument: 'file_path'"
     ]
    }
   ],
   "source": [
    "from semantica.export import GraphExporter\n",
    "\n",
    "exporter = GraphExporter()\n",
    "exporter.export(kg, format=\"json\", output_path=\"knowledge_graph.json\")\n",
    "print(\"Graph exported to knowledge_graph.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the Core `Semantica` Class\n",
    "\n",
    "For more complex systems, you can work directly with the `Semantica` core class and a configuration object. This gives you access to lifecycle management, plugin registration, and orchestration helpers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.core import Semantica, ConfigManager\n",
    "\n",
    "config_manager = ConfigManager()\n",
    "config = config_manager.load_from_file(\"config.yaml\")\n",
    "\n",
    "framework = Semantica(config=config)\n",
    "framework.initialize()\n",
    "\n",
    "kb_result = framework.build_knowledge_base(\n",
    "    sources=[str(docs_dir)],\n",
    "    embeddings=True,\n",
    "    graph=True,\n",
    ")\n",
    "\n",
    "framework.shutdown()\n",
    "sorted(kb_result.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Where to Go Next\n",
    "\n",
    "- Run the notebooks under `cookbook/introduction` for focused module overviews\n",
    "- Explore `cookbook/use_cases` for domain-specific end-to-end workflows\n",
    "- Read the **Core Concepts** documentation for deeper theory and best practices"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
