Fix: Make PyTorch import lazy to avoid DLL errors on Windows

- Remove top-level torch import from providers.py
- Add lazy imports in HuggingFaceLLMProvider and HuggingFaceModelLoader
- Remove hardcoded API key from notebook
- PyTorch now only loads when HuggingFace providers are instantiated

Fixes #129

.github/workflows/docs.yml

@@ -8,7 +8,11 @@ on:
     branches: [main]
     paths:
       - 'docs/**'
+      - 'semantica/**'
       - 'mkdocs.yml'
+      - 'requirements-docs.txt'
+      - 'CHANGELOG.md'
+      - 'RELEASE.md'
   workflow_dispatch:
 
 # Permissions needed to deploy to GitHub Pages

CHANGELOG.md

@@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.1.0] - 2025-12-31
+
+### Added
+- New command-line interface (`semantica` CLI) with support for knowledge base building and info commands.
+- Integrated FastAPI-based REST API server for remote access to framework functionality.
+- Dedicated background worker component for scalable task processing and pipeline execution.
+- Framework-level versioning configuration for PyPI distribution.
+- Automated release workflow with Trusted Publishing support.
+
+### Changed
+- Updated versioning across the framework to 0.1.0.
+- Refined entry point configurations in `pyproject.toml`.
+- Improved lazy module loading for core framework components.
+
 ## [0.0.5] - 2025-11-26
 
 ### Changed
@@ -49,7 +63,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 - Core framework architecture
-- Universal data ingestion (50+ file formats)
+- Universal data ingestion (multiple file formats)
 - Semantic intelligence engine (NER, relation extraction, event detection)
 - Knowledge graph construction with entity resolution
 - 6-stage ontology generation pipeline

RELEASE.md

@@ -0,0 +1,56 @@
+# Release Process for Semantica
+
+This document outlines the steps to release a new version of the Semantica framework.
+
+## 1. Versioning Policy
+
+Semantica follows [Semantic Versioning (SemVer)](https://semver.org/).
+- **MAJOR** version for incompatible API changes.
+- **MINOR** version for functionality added in a backwards compatible manner.
+- **PATCH** version for backwards compatible bug fixes.
+
+## 2. Pre-release Checklist
+
+Before releasing, ensure:
+- [ ] All tests pass: `pytest`
+- [ ] Documentation is up to date in `docs/` and `MkDocs` config.
+- [ ] `CHANGELOG.md` is updated with the latest changes.
+- [ ] Version is updated in:
+  - `semantica/__init__.py`
+  - `pyproject.toml`
+  - `docs/citation.md` (BibTeX entry)
+
+## 3. Release Steps
+
+### Automated Release (Recommended)
+
+The project uses GitHub Actions for automated releases to PyPI.
+
+1. **Tag the commit**: Create a new git tag for the version (e.g., `v0.1.0`).
+   ```bash
+   git tag -a v0.1.0 -m "Release v0.1.0"
+   git push origin v0.1.0
+   ```
+2. **GitHub Action**: The `Release` workflow will automatically trigger, build the package, create a GitHub Release, and publish to PyPI using Trusted Publishing.
+
+### Manual Release
+
+If you need to release manually:
+
+1. **Build the package**:
+   ```bash
+   python -m build
+   ```
+2. **Verify the build**:
+   ```bash
+   twine check dist/*
+   ```
+3. **Upload to PyPI**:
+   ```bash
+   twine upload dist/*
+   ```
+
+## 4. Post-release
+
+- Verify the new version is available on [PyPI](https://pypi.org/project/semantica/).
+- Check the [GitHub Releases](https://github.com/your-org/semantica/releases) page for the new release notes.

SECURITY.md

@@ -6,8 +6,8 @@ We actively support the following versions of Semantica with security updates:
 
 | Version | Supported          |
 | ------- | ------------------ |
-| 0.0.1   | :white_check_mark: |
-| < 0.0.1 | :x:                |
+| 0.1.0   | :white_check_mark: |
+| < 0.1.0 | :x:                |
 
 ## Reporting a Vulnerability
 

STRATEGIES_SUMMARY.md

@@ -0,0 +1,105 @@
+# Deduplication & Conflict Resolution Strategies Summary
+
+## Quick Reference by Use Case
+
+| Use Case | Deduplication Method | Merge Strategy | Conflict Detection | Conflict Resolution |
+|----------|---------------------|----------------|-------------------|---------------------|
+| **Finance** |
+| `01_Financial_Data_Integration_MCP` | `DuplicateDetector` (incremental) | `keep_highest_confidence` | `temporal` | `most_recent` |
+| `02_Fraud_Detection` | `ClusterBuilder` (graph_based) | `merge_all` | `logical` | `expert_review` |
+| **Biomedical** |
+| `01_Drug_Discovery_Pipeline` | `EntityResolver` (semantic) | - | `relationship` | `voting` |
+| `02_Genomic_Variant_Analysis` | `DuplicateDetector` (group) | `keep_most_complete` | `value` | `credibility_weighted` |
+| **Cybersecurity** |
+| `01_Real_Time_Anomaly_Detection` | `DuplicateDetector` (pairwise) | `keep_first` | `entity` | `first_seen` |
+| `02_Threat_Intelligence_Hybrid_RAG` | `EntityResolver` (exact) | - | `type` | `highest_confidence` |
+| **Blockchain** |
+| `01_DeFi_Protocol_Intelligence` | `DuplicateDetector` (group) | `keep_last` | `relationship` | `voting` |
+| `02_Transaction_Network_Analysis` | `ClusterBuilder` (hierarchical) | `keep_most_complete` | `temporal` | `most_recent` |
+| **Intelligence** |
+| `01_Criminal_Network_Analysis` | `EntityResolver` (fuzzy) | - | `value` | `credibility_weighted` |
+| `02_Intelligence_Analysis_Orchestrator_Worker` | `DuplicateDetector` (batch) | `merge_all` | `entity` | `voting` |
+| **Renewable Energy** |
+| `01_Energy_Market_Analysis` | `DuplicateDetector` (pairwise) | `keep_highest_confidence` | `temporal` | `most_recent` |
+| **Supply Chain** |
+| `01_Supply_Chain_Data_Integration` | `DuplicateDetector` (incremental) | `keep_most_complete` | `value` | `credibility_weighted` |
+
+---
+
+## Strategy Rationale by Domain
+
+### Finance
+- **Financial Data Integration**: Incremental for streaming data; most_recent for time-sensitive financial data
+- **Fraud Detection**: Graph-based clustering for fraud groups; expert_review for fraud assessment
+
+### Biomedical
+- **Drug Discovery**: Semantic matching for drug compounds; voting for research source aggregation
+- **Genomic Variants**: Group method for related variants; credibility weighting for research sources
+
+### Cybersecurity
+- **Real-Time Anomaly**: Pairwise for real-time streams; keep_first for first detection priority
+- **Threat Intelligence**: Exact matching for IOCs; highest_confidence for threat classification
+
+### Blockchain
+- **DeFi Protocols**: Group method for related protocols; keep_last for latest protocol info
+- **Transaction Networks**: Hierarchical clustering for nested groups; temporal for time-sensitive data
+
+### Intelligence
+- **Criminal Networks**: Fuzzy matching for intelligence data; credibility weighting for intelligence sources
+- **Intelligence Analysis**: Batch for multi-source integration; merge_all to combine all intelligence sources
+
+### Renewable Energy
+- **Energy Markets**: Pairwise for real-time market data; most_recent for time-sensitive energy data
+
+### Supply Chain
+- **Supply Chain Integration**: Incremental for continuous updates; credibility weighting for supply chain sources
+
+---
+
+## Method Distribution
+
+### Deduplication Methods (9 total)
+- `pairwise`: 2 notebooks (real-time processing)
+- `batch`: 3 notebooks (large datasets)
+- `incremental`: 2 notebooks (streaming/continuous)
+- `group`: 2 notebooks (related entities)
+- `graph_based` (ClusterBuilder): 2 notebooks (interconnected entities)
+- `hierarchical` (ClusterBuilder): 1 notebook (nested groups)
+- `exact` (EntityResolver): 1 notebook (exact matching)
+- `semantic` (EntityResolver): 2 notebooks (semantic similarity)
+- `fuzzy` (EntityResolver): 1 notebook (fuzzy matching)
+
+### Merge Strategies (5 total)
+- `keep_first`: 1 notebook (first detection priority)
+- `keep_last`: 1 notebook (latest information)
+- `keep_most_complete`: 5 notebooks (preserve all details)
+- `keep_highest_confidence`: 2 notebooks (most reliable data)
+- `merge_all`: 3 notebooks (combine all information)
+
+### Conflict Detection Methods (6 total)
+- `value`: 4 notebooks (property value conflicts)
+- `type`: 2 notebooks (type/classification conflicts)
+- `entity`: 2 notebooks (entity-wide conflicts)
+- `relationship`: 3 notebooks (relationship conflicts)
+- `temporal`: 3 notebooks (time-sensitive conflicts)
+- `logical`: 2 notebooks (logical inconsistencies)
+
+### Conflict Resolution Strategies (6 total)
+- `voting`: 5 notebooks (majority vote)
+- `credibility_weighted`: 4 notebooks (source credibility)
+- `most_recent`: 3 notebooks (latest data)
+- `first_seen`: 1 notebook (first detection)
+- `highest_confidence`: 2 notebooks (most confident)
+- `expert_review`: 1 notebook (manual review)
+
+---
+
+## Key Patterns
+
+1. **Real-Time Systems**: Use `pairwise` + `keep_first` + `first_seen`
+2. **Time-Sensitive Data**: Use `temporal` + `most_recent`
+3. **Multi-Source Integration**: Use `batch` + `merge_all` + `voting`
+4. **Medical/Research**: Use `credibility_weighted` for authoritative sources
+5. **Fraud/Security**: Use `graph_based` + `logical` + `expert_review`
+6. **Exact Matching Required**: Use `exact` strategy (IOCs, identifiers)
+

cookbook/advanced/08_Reasoning_and_Inference.ipynb

@@ -60,7 +60,7 @@
    "outputs": [],
    "source": [
     "from semantica.kg import GraphBuilder\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n"
+    "from semantica.reasoning import Reasoner, ExplanationGenerator\n"
    ]
   },
   {
@@ -78,7 +78,7 @@
     {
      "data": {
       "text/html": [
-       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>GraphBuilder</td><td>-</td><td>0.31s</td></tr><tr><td>🔄</td><td>Semantica is building</td><td>🧠 kg</td><td>EntityResolver</td><td>-</td><td>2.16s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>EntityMerger</td><td>-</td><td>0.05s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>MergeStrategyManager</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>RuleManager</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>InferenceEngine</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>ExplanationGenerator</td><td>-</td><td>0.01s</td></tr></table></div>"
+       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>GraphBuilder</td><td>-</td><td>0.31s</td></tr><tr><td>🔄</td><td>Semantica is building</td><td>🧠 kg</td><td>EntityResolver</td><td>-</td><td>2.16s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>EntityMerger</td><td>-</td><td>0.05s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>MergeStrategyManager</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>ExplanationGenerator</td><td>-</td><td>0.01s</td></tr></table></div>"
       ],
       "text/plain": [
        "<IPython.core.display.HTML object>"
@@ -122,8 +122,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Initialize Inference Engine\n",
-    "engine = InferenceEngine()\n",
+    "# Initialize Reasoner\n",
+    "reasoner = Reasoner()\n",
     "\n",
     "# Define rules using logic syntax\n",
     "rules = [\n",
@@ -132,7 +132,7 @@
     "]\n",
     "\n",
     "for rule in rules:\n",
-    "    engine.add_rule(rule)\n"
+    "    reasoner.add_rule(rule)\n"
    ]
   },
   {
@@ -152,24 +152,19 @@
      "output_type": "stream",
      "text": [
       "Inferred 2 new facts:\n",
-      " - grandparent_of(alice, charlie) (Rule: Rule 1)\n",
-      " - lives_in(alice, california) (Rule: Rule 2)\n"
+      " - grandparent_of(alice, charlie)\n",
+      " - lives_in(alice, california)\n"
      ]
     }
    ],
    "source": [
-    "# Load facts from relationships into the engine\n",
-    "for rel in relationships:\n",
-    "    # Format: predicate(subject, object)\n",
-    "    fact_str = f\"{rel['type']}({rel['source']}, {rel['target']})\"\n",
-    "    engine.add_fact(fact_str)\n",
-    "\n",
     "# Perform forward chaining to derive new facts\n",
-    "results = engine.forward_chain()\n",
+    "# The Reasoner can infer facts directly from the knowledge graph or a list of facts\n",
+    "inferred_facts = reasoner.infer_facts(knowledge_graph)\n",
     "\n",
-    "print(f\"Inferred {len(results)} new facts:\")\n",
-    "for result in results:\n",
-    "    print(f\" - {result.conclusion} (Rule: {result.rule_used.name})\")\n"
+    "print(f\"Inferred {len(inferred_facts)} new facts:\")\n",
+    "for fact in inferred_facts:\n",
+    "    print(f\" - {fact}\")\n"
    ]
   },
   {
@@ -197,7 +192,7 @@
     "goal = \"grandparent_of(alice, charlie)\"\n",
     "\n",
     "# Perform backward chaining\n",
-    "proof = engine.backward_chain(goal)\n",
+    "proof = reasoner.backward_chain(goal)\n",
     "\n",
     "if proof:\n",
     "    print(f\"Goal '{goal}' proven successfully!\")\n",
@@ -221,27 +216,18 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Explanation for first inferred fact:\n",
-      "Given the premises: parent_of(alice, bob), parent_of(bob, charlie), we conclude: grandparent_of(alice, charlie) using rule 'Rule 1'.\n",
-      "\n",
       "Explanation for backward chaining proof:\n",
-      "Given the premises: , we conclude: grandparent_of(alice, charlie).\n"
+      "Given the premises: parent_of(alice, bob), parent_of(bob, charlie), we conclude: grandparent_of(alice, charlie).\n"
      ]
     }
    ],
    "source": [
     "generator = ExplanationGenerator()\n",
     "\n",
-    "# Explain the last forward chaining inference\n",
-    "if results:\n",
-    "    explanation = generator.generate_explanation(results[0])\n",
-    "    print(\"Explanation for first inferred fact:\")\n",
-    "    print(explanation.natural_language)\n",
-    "\n",
     "# If we have a proof from backward chaining, explain it\n",
     "if proof:\n",
     "    proof_explanation = generator.generate_explanation(proof)\n",
-    "    print(\"\\nExplanation for backward chaining proof:\")\n",
+    "    print(\"Explanation for backward chaining proof:\")\n",
     "    print(proof_explanation.natural_language)\n"
    ]
   },
@@ -284,10 +270,10 @@
    "outputs": [],
    "source": [
     "from semantica.kg import GraphBuilder\n",
-    "from semantica.reasoning import InferenceEngine, ExplanationGenerator\n",
+    "from semantica.reasoning import Reasoner, ExplanationGenerator\n",
     "\n",
     "builder = GraphBuilder()\n",
-    "engine = InferenceEngine()\n",
+    "reasoner = Reasoner()\n",
     "explainer = ExplanationGenerator()\n"
    ]
   },
@@ -350,7 +336,7 @@
     "    \"IF lives_in(?x, ?y) AND located_in(?y, ?z) THEN lives_in(?x, ?z)\"\n",
     "]\n",
     "for r in rules:\n",
-    "    engine.add_rule(r)\n"
+    "    reasoner.add_rule(r)\n"
    ]
   },
   {
@@ -371,9 +357,9 @@
    "source": [
     "for rel in relationships:\n",
     "    fact = f\"{rel['type']}({rel['source']}, {rel['target']})\"\n",
-    "    engine.add_fact(fact)\n",
+    "    reasoner.add_fact(fact)\n",
     "\n",
-    "derived = engine.forward_chain()\n",
+    "derived = reasoner.forward_chain()\n",
     "print(len(derived))\n",
     "for d in derived:\n",
     "    print(d.conclusion)\n"
@@ -401,7 +387,7 @@
     "    \"lives_in(alice, california)\"\n",
     "]\n",
     "for g in goals:\n",
-    "    proof = engine.backward_chain(g)\n",
+    "    proof = reasoner.backward_chain(g)\n",
     "    print(g)\n",
     "    print(bool(proof))\n"
    ]
@@ -426,7 +412,7 @@
     "    print(exp.natural_language)\n",
     "\n",
     "goal = \"grandparent_of(alice, charlie)\"\n",
-    "proof = engine.backward_chain(goal)\n",
+    "proof = reasoner.backward_chain(goal)\n",
     "if proof:\n",
     "    pexp = explainer.generate_explanation(proof)\n",
     "    print(pexp.natural_language)\n"

cookbook/advanced/09_Semantic_Layer_Construction.ipynb

@@ -25,21 +25,37 @@
     "pip install semantica[all]\n",
     "```\n",
     "\n",
-    "## Workflow: Build KG \u2192 Generate Ontology \u2192 Create Semantic Layer \u2192 Export RDF \u2192 Triplet Store\n"
+    "## Workflow: Build KG → Generate Ontology → Create Semantic Layer → Export RDF \n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
+     ]
+    }
+   ],
    "source": [
-    "!pip install semantica\n"
+    "!pip install -qU semantica\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -58,9 +74,22 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 3,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>GraphBuilder</td><td>-</td><td>0.13s</td></tr><tr><td>🔄</td><td>Semantica is building</td><td>🧠 kg</td><td>EntityResolver</td><td>-</td><td>7.37s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.09s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>OntologyGenerator</td><td>-</td><td>0.06s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>ClassInferrer</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>PropertyGenerator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is exporting</td><td>💾 export</td><td>RDFExporter</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is storing</td><td>🗄️ triplet_store</td><td>BulkLoader</td><td>-</td><td>3.01s</td></tr></table></div>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
     "builder = GraphBuilder()\n",
     "\n",
@@ -90,7 +119,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -107,7 +136,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -159,9 +188,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 6,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Exported knowledge graph to knowledge_graph.ttl\n"
+     ]
+    }
+   ],
    "source": [
     "exporter = RDFExporter()\n",
     "# Export Knowledge Graph\n",
@@ -169,23 +206,6 @@
     "print(\"Exported knowledge graph to knowledge_graph.ttl\")\n"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Store in Triplet Store\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "triplet_store = TripletStore()\n",
-    "triplet_store.store(knowledge_graph, ontology)\n"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -196,16 +216,44 @@
     "- Knowledge Graph Built\n",
     "- Ontology Generated\n",
     "- Semantic Layer Created with Mappings\n",
-    "- RDF Export Completed\n",
-    "- Triplet Store Storage Completed\n"
+    "- RDF Export Completed\n"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
   }
  ],
  "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
   "language_info": {
-   "name": "python"
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
   }
  },
  "nbformat": 4,
  "nbformat_minor": 2
-}
+}

cookbook/advanced/10_Temporal_Knowledge_Graphs.ipynb

@@ -6,35 +6,29 @@
    "source": [
     "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/10_Temporal_Knowledge_Graphs.ipynb)\n",
     "\n",
-    "# Temporal Knowledge Graphs\n",
+    "# Deep Dive: Temporal Knowledge Graphs\n",
     "\n",
     "## Overview\n",
     "\n",
-    "This notebook demonstrates advanced temporal knowledge graph capabilities using TemporalGraphQuery, TemporalPatternDetector, TemporalVersionManager, and TemporalVisualizer.\n",
+    "This notebook provides a comprehensive deep dive into **Temporal Knowledge Graphs (TKGs)** using Semantica. Unlike static KGs, TKGs capture the evolution of facts, relationships, and entities over time. This capability is crucial for applications like:\n",
     "\n",
+    "- **Corporate History Analysis**: Tracking mergers, acquisitions, and leadership changes.\n",
+    "- **Supply Chain Monitoring**: Tracing product movement and status changes.\n",
+    "- **Financial Fraud Detection**: Analyzing sequences of transactions.\n",
+    "\n",
+    "We will build a rich scenario modeling the history of a tech ecosystem, covering 40 years of evolution.\n",
+    "\n",
+    "### Key Components Covered\n",
+    "\n",
+    "1.  **`GraphBuilder` (Temporal Mode)**: Constructing KGs with time-aware properties.\n",
+    "2.  **`TemporalGraphQuery`**: Performing point-in-time, interval, and path queries.\n",
+    "3.  **`TemporalPatternDetector`**: Identifying sequences and cyclic patterns.\n",
+    "4.  **`TemporalVersionManager`**: Managing snapshots and comparing graph states.\n",
+    "5.  **`TemporalVisualizer`**: Interactive timelines and evolution plots.\n",
     "\n",
     "**Documentation**: [API Reference](https://semantica.readthedocs.io/reference/kg/)\n",
     "\n",
-    "### Learning Objectives\n",
-    "\n",
-    "- Use TemporalGraphQuery for time-aware queries\n",
-    "- Use TemporalPatternDetector to detect temporal patterns\n",
-    "- Use TemporalVersionManager for temporal versioning and snapshots\n",
-    "- Use TemporalVisualizer to visualize temporal data\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Workflow: Build Temporal KG \u2192 Time-Aware Queries \u2192 Pattern Detection \u2192 Version Management \u2192 Visualization\n"
+    "## Installation\n"
    ]
   },
   {
@@ -43,7 +37,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install semantica\n"
+    "# !pip install semantica[all]"
    ]
   },
   {
@@ -52,33 +46,114 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "import json\n",
+    "from datetime import datetime\n",
     "from semantica.kg import GraphBuilder, TemporalGraphQuery, TemporalPatternDetector, TemporalVersionManager\n",
     "from semantica.visualization import TemporalVisualizer\n",
-    "from datetime import datetime\n",
+    "import plotly.offline as pyo\n",
+    "pyo.init_notebook_mode(connected=True)\n",
     "\n",
-    "builder = GraphBuilder()\n",
+    "# Ensure consistent output for reproducibility\n",
+    "import random\n",
+    "random.seed(42)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 1: Scenario Definition & Data Preparation\n",
     "\n",
+    "We define a dataset representing the history of \"TechCorp\" and \"InnovateInc\", including their founders, products, and eventual merger.\n",
+    "\n",
+    "**Temporal Properties**:\n",
+    "- Entities have `founded`, `born`, `released` dates.\n",
+    "- Relationships have `timestamp` (point event) or `valid_from`/`valid_to` (intervals).\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 1. Define Entities with Temporal Metadata\n",
     "entities = [\n",
-    "    {\"id\": \"e1\", \"type\": \"Organization\", \"name\": \"Apple Inc.\", \"properties\": {\"founded\": \"1976\"}},\n",
-    "    {\"id\": \"e2\", \"type\": \"Person\", \"name\": \"Steve Jobs\", \"properties\": {\"born\": \"1955\"}}\n",
+    "    # Organizations\n",
+    "    {\"id\": \"org_1\", \"type\": \"Organization\", \"name\": \"TechCorp\", \"properties\": {\"founded\": \"1980-01-01\", \"industry\": \"Hardware\"}},\n",
+    "    {\"id\": \"org_2\", \"type\": \"Organization\", \"name\": \"InnovateInc\", \"properties\": {\"founded\": \"1995-06-15\", \"industry\": \"Software\"}},\n",
+    "    {\"id\": \"org_3\", \"type\": \"Organization\", \"name\": \"FutureSystems\", \"properties\": {\"founded\": \"2010-03-10\", \"industry\": \"AI\"}},\n",
+    "    \n",
+    "    # People\n",
+    "    {\"id\": \"per_1\", \"type\": \"Person\", \"name\": \"Alice Founder\", \"properties\": {\"born\": \"1955-05-20\"}},\n",
+    "    {\"id\": \"per_2\", \"type\": \"Person\", \"name\": \"Bob Coder\", \"properties\": {\"born\": \"1970-08-12\"}},\n",
+    "    {\"id\": \"per_3\", \"type\": \"Person\", \"name\": \"Charlie CEO\", \"properties\": {\"born\": \"1980-02-28\"}},\n",
+    "    \n",
+    "    # Products\n",
+    "    {\"id\": \"prod_1\", \"type\": \"Product\", \"name\": \"HomePC\", \"properties\": {\"released\": \"1985-11-20\"}},\n",
+    "    {\"id\": \"prod_2\", \"type\": \"Product\", \"name\": \"SoftOS\", \"properties\": {\"released\": \"1998-07-25\"}},\n",
+    "    {\"id\": \"prod_3\", \"type\": \"Product\", \"name\": \"SmartAI\", \"properties\": {\"released\": \"2015-01-10\"}}\n",
     "]\n",
     "\n",
+    "# 2. Define Temporal Relationships\n",
     "relationships = [\n",
-    "    {\"source\": \"e2\", \"target\": \"e1\", \"type\": \"founded\", \"properties\": {\"timestamp\": \"1976-04-01\"}}\n",
+    "    # Founding Events (Point in time)\n",
+    "    {\"source\": \"per_1\", \"target\": \"org_1\", \"type\": \"founded\", \"timestamp\": \"1980-01-01\", \"properties\": {\"timestamp\": \"1980-01-01\"}},\n",
+    "    {\"source\": \"per_2\", \"target\": \"org_2\", \"type\": \"founded\", \"timestamp\": \"1995-06-15\", \"properties\": {\"timestamp\": \"1995-06-15\"}},\n",
+    "    \n",
+    "    # Employment (Intervals)\n",
+    "    {\"source\": \"per_1\", \"target\": \"org_1\", \"type\": \"ceo_of\", \"valid_from\": \"1980-01-01\", \"valid_to\": \"2000-01-01\", \"properties\": {\"role\": \"CEO\"}},\n",
+    "    {\"source\": \"per_3\", \"target\": \"org_1\", \"type\": \"ceo_of\", \"valid_from\": \"2000-01-02\", \"valid_to\": \"2023-01-01\", \"properties\": {\"role\": \"CEO\"}},\n",
+    "    {\"source\": \"per_2\", \"target\": \"org_2\", \"type\": \"cto_of\", \"valid_from\": \"1995-06-15\", \"valid_to\": \"2010-05-01\", \"properties\": {\"role\": \"CTO\"}},\n",
+    "    \n",
+    "    # Product Launches\n",
+    "    {\"source\": \"org_1\", \"target\": \"prod_1\", \"type\": \"launched\", \"timestamp\": \"1985-11-20\", \"properties\": {\"timestamp\": \"1985-11-20\"}},\n",
+    "    {\"source\": \"org_2\", \"target\": \"prod_2\", \"type\": \"launched\", \"timestamp\": \"1998-07-25\", \"properties\": {\"timestamp\": \"1998-07-25\"}},\n",
+    "    {\"source\": \"org_3\", \"target\": \"prod_3\", \"type\": \"launched\", \"timestamp\": \"2015-01-10\", \"properties\": {\"timestamp\": \"2015-01-10\"}},\n",
+    "    \n",
+    "    # Corporate Actions\n",
+    "    {\"source\": \"org_1\", \"target\": \"org_2\", \"type\": \"acquired\", \"timestamp\": \"2010-05-01\", \"properties\": {\"amount\": \"$5B\", \"timestamp\": \"2010-05-01\"}},\n",
+    "    {\"source\": \"org_1\", \"target\": \"org_3\", \"type\": \"invested_in\", \"timestamp\": \"2012-08-15\", \"properties\": {\"amount\": \"$100M\", \"timestamp\": \"2012-08-15\"}}\n",
     "]\n",
     "\n",
+    "print(f\"Defined {len(entities)} entities and {len(relationships)} temporal relationships.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 2: Building the Temporal Graph\n",
+    "\n",
+    "We use `GraphBuilder` with `enable_temporal=True`. This instructs the builder to index temporal properties like `timestamp`, `valid_from`, and `valid_to`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "builder = GraphBuilder(\n",
+    "    enable_temporal=True,\n",
+    "    temporal_granularity=\"day\"  # Can be 'year', 'month', 'day', 'hour'\n",
+    ")\n",
+    "\n",
     "temporal_kg = builder.build(entities, relationships)\n",
     "\n",
-    "print(f\"Built temporal knowledge graph with {len(entities)} entities\")\n"
+    "# The graph object now contains temporal indices\n",
+    "print(\"Graph built successfully.\")\n",
+    "print(f\"Nodes: {len(temporal_kg['entities'])}\")\n",
+    "print(f\"Edges: {len(temporal_kg['relationships'])}\")"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Step 2: Time-Aware Queries\n",
+    "## Step 3: Advanced Temporal Querying\n",
     "\n",
-    "Query the graph at specific time points.\n"
+    "We use `TemporalGraphQuery` to ask time-sensitive questions."
    ]
   },
   {
@@ -87,25 +162,48 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "temporal_query = TemporalGraphQuery()\n",
+    "query_engine = TemporalGraphQuery()\n",
     "\n",
-    "query_result = temporal_query.query_time_range(\n",
+    "# 1. Point-in-Time Query\n",
+    "# \"Who was the CEO of TechCorp in 1990?\"\n",
+    "ceo_1990 = query_engine.query_at_time(\n",
+    "    temporal_kg,\n",
+    "    query=\"Find the CEO of TechCorp\",\n",
+    "    at_time=\"1990-06-01\"\n",
+    ")\n",
+    "print(\"CEO in 1990:\", [e['id'] for e in ceo_1990.get('entities', [])])\n",
+    "\n",
+    "# \"Who was the CEO of TechCorp in 2015?\"\n",
+    "ceo_2015 = query_engine.query_at_time(\n",
+    "    temporal_kg,\n",
+    "    query=\"Find the CEO of TechCorp\",\n",
+    "    at_time=\"2015-06-01\"\n",
+    ")\n",
+    "print(\"CEO in 2015:\", [e['id'] for e in ceo_2015.get('entities', [])])\n",
+    "\n",
+    "# 2. Temporal Path Finding\n",
+    "# \"How did Alice (Founder) connect to SmartAI (Product released in 2015)?\"\n",
+    "# This requires traversing through time: Alice -> founded TechCorp -> invested in FutureSystems -> launched SmartAI\n",
+    "paths = query_engine.find_temporal_paths(\n",
     "    graph=temporal_kg,\n",
-    "    query=\"Find entities founded in 1976\",\n",
-    "    start_time=\"1976-01-01\",\n",
-    "    end_time=\"1976-12-31\"\n",
+    "    source=\"per_1\", # Alice\n",
+    "    target=\"prod_3\", # SmartAI\n",
+    "    start_time=\"1980-01-01\",\n",
+    "    end_time=\"2020-01-01\"\n",
     ")\n",
     "\n",
-    "print(f\"Time-aware query returned {len(query_result.get('entities', []))} entities\")\n"
+    "print(f\"\\nFound {len(paths)} temporal paths from Alice to SmartAI.\")\n",
+    "for i, path in enumerate(paths):\n",
+    "    print(f\"Path {i+1}: {path}\")"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Step 3: Temporal Pattern Detection\n",
+    "## Step 4: Graph Evolution Analysis\n",
     "\n",
-    "Detect temporal patterns in the graph.\n"
+    "We can analyze how the graph properties change over time using `analyze_evolution`."
    ]
   },
   {
@@ -114,24 +212,56 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "pattern_detector = TemporalPatternDetector()\n",
+    "evolution_stats = query_engine.analyze_evolution(\n",
+    "    temporal_kg,\n",
+    "    start_time=\"1980-01-01\",\n",
+    "    end_time=\"2025-01-01\",\n",
+    "    metrics=[\"count\", \"diversity\", \"stability\"]\n",
+    ")\n",
     "\n",
-    "patterns = pattern_detector.detect_temporal_patterns(\n",
+    "print(\"\\nEvolution Statistics (1980-2025):\")\n",
+    "print(f\"Total Relationships: {evolution_stats.get('count', 'N/A')}\")\n",
+    "print(f\"Relationship Diversity: {evolution_stats.get('diversity', 'N/A')}\")\n",
+    "print(f\"Graph Stability: {evolution_stats.get('stability', 'N/A')}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 5: Temporal Pattern Detection\n",
+    "\n",
+    "We use `TemporalPatternDetector` to automatically find recurring structures, such as sequences (A -> B -> C) or cycles."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "detector = TemporalPatternDetector()\n",
+    "\n",
+    "# Detect sequential patterns (e.g., Founded -> Launched -> Acquired)\n",
+    "sequences = detector.detect_temporal_patterns(\n",
     "    temporal_kg,\n",
     "    pattern_type=\"sequence\",\n",
     "    min_frequency=1\n",
     ")\n",
     "\n",
-    "print(f\"Detected {len(patterns)} temporal patterns\")\n"
+    "print(f\"\\nDetected {len(sequences)} sequential patterns.\")\n",
+    "for seq in sequences[:3]: # Show top 3\n",
+    "    print(f\"Pattern: {seq.get('pattern')}\")\n",
+    "    print(f\"Support: {seq.get('support')}\")"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Step 4: Version Management\n",
+    "## Step 6: Version Management & Comparisons\n",
     "\n",
-    "Manage temporal versions and snapshots.\n"
+    "In real-world scenarios, KGs are updated in batches. `TemporalVersionManager` handles these versions."
    ]
   },
   {
@@ -142,19 +272,25 @@
    "source": [
     "version_manager = TemporalVersionManager()\n",
     "\n",
-    "snapshot = version_manager.create_snapshot(temporal_kg, timestamp=datetime.now())\n",
+    "# Create explicit versions\n",
+    "v1_1990 = version_manager.create_version(temporal_kg, timestamp=\"1990-01-01\", version_label=\"v1.0 (Early Days)\")\n",
+    "v2_2010 = version_manager.create_version(temporal_kg, timestamp=\"2010-01-01\", version_label=\"v2.0 (Post-Merger)\")\n",
     "\n",
-    "print(f\"Created temporal snapshot at {snapshot.get('timestamp', 'N/A')}\")\n",
-    "print(f\"Snapshot contains {len(snapshot.get('entities', []))} entities\")\n"
+    "# Compare versions\n",
+    "diff = version_manager.compare_versions(v1_1990, v2_2010)\n",
+    "\n",
+    "print(f\"\\nComparing {v1_1990['label']} vs {v2_2010['label']}:\")\n",
+    "print(f\"New Entities: {diff.get('entities_added', 0)}\")\n",
+    "print(f\"New Relationships: {diff.get('relationships_added', 0)}\")"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Step 5: Temporal Visualization\n",
+    "## Step 7: Visualizing the Timeline\n",
     "\n",
-    "Visualize temporal data.\n"
+    "Finally, `TemporalVisualizer` brings the data to life. We will create an interactive timeline and a snapshot comparison."
    ]
   },
   {
@@ -163,9 +299,46 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "temporal_visualizer = TemporalVisualizer()\n",
+    "visualizer = TemporalVisualizer()\n",
     "\n",
-    "visualization = temporal_visualizer.visualize_timeline(temporal_kg, output=\"interactive\")\n"
+    "# 1. Interactive Timeline\n",
+    "# Prepare events for visualization (extract from KG)\n",
+    "def extract_events(graph):\n",
+    "    events = []\n",
+    "    for rel in graph['relationships']:\n",
+    "        # Point events\n",
+    "        if rel.get('timestamp'):\n",
+    "            events.append({\n",
+    "                'timestamp': rel['timestamp'],\n",
+    "                'type': rel['type'],\n",
+    "                'label': f\"{rel['source']} -> {rel['target']}\",\n",
+    "                'entity': rel['source']\n",
+    "            })\n",
+    "        # Interval events (start)\n",
+    "        if rel.get('valid_from'):\n",
+    "             events.append({\n",
+    "                'timestamp': rel['valid_from'],\n",
+    "                'type': f\"{rel['type']} (start)\",\n",
+    "                'label': f\"{rel['source']} -> {rel['target']}\",\n",
+    "                'entity': rel['source']\n",
+    "            })\n",
+    "    return {'events': events}\n",
+    "\n",
+    "temporal_data = extract_events(temporal_kg)\n",
+    "timeline_fig = visualizer.visualize_timeline(temporal_data, output=\"interactive\")\n",
+    "# In a notebook, this would render a Plotly figure. \n",
+    "timeline_fig.show()\n",
+    "\n",
+    "# 2. Version History Visualization\n",
+    "history = [\n",
+    "    {\"version\": \"v1.0\", \"timestamp\": \"1990-01-01\", \"changes\": \"Founding Era\"},\n",
+    "    {\"version\": \"v2.0\", \"timestamp\": \"2010-01-01\", \"changes\": \"Expansion Era\"},\n",
+    "    {\"version\": \"v3.0\", \"timestamp\": \"2020-01-01\", \"changes\": \"AI Era\"}\n",
+    "]\n",
+    "history_fig = visualizer.visualize_version_history(history, output=\"interactive\")\n",
+    "history_fig.show()\n",
+    "\n",
+    "print(\"Visualizations generated (render requires Jupyter environment).\")"
    ]
   },
   {
@@ -174,66 +347,38 @@
    "source": [
     "## Summary\n",
     "\n",
-    "You've learned advanced temporal knowledge graph capabilities:\n",
+    "In this deep dive, we:\n",
+    "1.  **modeled** a complex corporate history with temporal metadata.\n",
+    "2.  **Built** a time-aware knowledge graph using `GraphBuilder`.\n",
+    "3.  **Queried** specific time slices and intervals to reconstruct history.\n",
+    "4.  **Traced** temporal paths to understand indirect connections.\n",
+    "5.  **Analyzed** the graph's evolution metrics.\n",
+    "6.  **Managed** versions and visualized the timeline.\n",
+    "7.  **Visualized** the data with `TemporalVisualizer`.\n",
     "\n",
-    "- **TemporalGraphQuery**: Time-aware graph querying\n",
-    "- **TemporalPatternDetector**: Temporal pattern detection\n",
-    "- **TemporalVersionManager**: Temporal versioning and snapshots\n",
-    "- **TemporalVisualizer**: Temporal data visualization\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Snapshot Comparison and Version History\n",
-    "\n",
-    "Compare graph snapshots across time and visualize version history."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Create multiple versions\n",
-    "version_manager = TemporalVersionManager()\n",
-    "version_2020 = version_manager.create_version(temporal_kg, timestamp=\"2020-01-01\", version_label=\"v2020\")\n",
-    "# Simulate changes for 2023\n",
-    "temporal_kg_updated = {\n",
-    "    \"entities\": temporal_kg.get(\"entities\", []),\n",
-    "    \"relationships\": temporal_kg.get(\"relationships\", []) + [\n",
-    "        {\"source\": \"e1\", \"target\": \"e2\", \"type\": \"collaborated_with\", \"valid_from\": \"2023-01-01\"}\n",
-    "    ]\n",
-    "}\n",
-    "version_2023 = version_manager.create_version(temporal_kg_updated, timestamp=\"2023-01-01\", version_label=\"v2023\")\n",
-    "\n",
-    "# Build snapshots dict for comparison\n",
-    "snapshots = {\n",
-    "    version_2020[\"timestamp\"]: version_2020,\n",
-    "    version_2023[\"timestamp\"]: version_2023\n",
-    "}\n",
-    "\n",
-    "# Visualize snapshot comparison\n",
-    "fig_snapshots = temporal_visualizer.visualize_snapshot_comparison(snapshots, output=\"interactive\")\n",
-    "\n",
-    "# Build version history list\n",
-    "version_history = [\n",
-    "    {\"version\": version_2020.get(\"label\", \"v2020\"), \"timestamp\": version_2020.get(\"timestamp\"), \"changes\": f\"Entities: {len(version_2020.get('entities', []))}, Relationships: {len(version_2020.get('relationships', []))}\"},\n",
-    "    {\"version\": version_2023.get(\"label\", \"v2023\"), \"timestamp\": version_2023.get(\"timestamp\"), \"changes\": f\"Entities: {len(version_2023.get('entities', []))}, Relationships: {len(version_2023.get('relationships', []))}\"}\n",
-    "]\n",
-    "\n",
-    "# Visualize version history\n",
-    "fig_versions = temporal_visualizer.visualize_version_history(version_history, output=\"interactive\")\n"
+    "This workflow forms the backbone of temporal intelligence applications in Semantica."
    ]
   }
  ],
  "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
   "language_info": {
-   "name": "python"
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
-}
+ "nbformat_minor": 4
+}

cookbook/advanced/Advanced_Triplet_Store.ipynb

@@ -1,173 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Advanced Triplet Store Guide\n",
-    "\n",
-    "This guide explores the advanced capabilities of the Semantica Triplet Store module, focusing on RDF data management, SPARQL querying, and multi-backend support (Blazegraph, Jena, RDF4J).\n",
-    "\n",
-    "## Key Features\n",
-    "- Unified interface for Blazegraph, Jena, and RDF4J\n",
-    "- Bulk loading with progress tracking\n",
-    "- SPARQL query execution and optimization\n",
-    "- Transaction support (backend-dependent)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.triplet_store import TripletStore\n",
-    "from semantica.semantic_extract.triplet_extractor import Triplet"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 1. Initialization\n",
-    "\n",
-    "Initialize the Triplet Store with your preferred backend."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Initialize Blazegraph store\n",
-    "store = TripletStore(\n",
-    "    backend=\"blazegraph\",\n",
-    "    endpoint=\"http://localhost:9999/blazegraph\"\n",
-    ")\n",
-    "\n",
-    "# Or Jena\n",
-    "# store = TripletStore(backend=\"jena\", endpoint=\"http://localhost:3030/ds\")\n",
-    "\n",
-    "# Or RDF4J\n",
-    "# store = TripletStore(backend=\"rdf4j\", endpoint=\"http://localhost:8080/rdf4j-server/repositories/myrepo\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 2. Adding Triplets\n",
-    "\n",
-    "Add individual triplets or batch load them."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Single triplet\n",
-    "triplet = Triplet(\n",
-    "    subject=\"http://example.org/Alice\",\n",
-    "    predicate=\"http://xmlns.com/foaf/0.1/knows\",\n",
-    "    object=\"http://example.org/Bob\"\n",
-    ")\n",
-    "\n",
-    "store.add_triplet(triplet)\n",
-    "\n",
-    "# Bulk load\n",
-    "triplets = [\n",
-    "    Triplet(\n",
-    "        subject=\"http://example.org/Bob\",\n",
-    "        predicate=\"http://xmlns.com/foaf/0.1/knows\",\n",
-    "        object=\"http://example.org/Charlie\"\n",
-    "    ),\n",
-    "    Triplet(\n",
-    "        subject=\"http://example.org/Charlie\",\n",
-    "        predicate=\"http://xmlns.com/foaf/0.1/knows\",\n",
-    "        object=\"http://example.org/Alice\"\n",
-    "    )\n",
-    "]\n",
-    "\n",
-    "result = store.add_triplets(triplets, batch_size=100)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 3. SPARQL Querying\n",
-    "\n",
-    "Execute SPARQL queries to retrieve data."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "query = \"\"\"\n",
-    "SELECT ?s ?p ?o\n",
-    "WHERE {\n",
-    "  ?s ?p ?o\n",
-    "}\n",
-    "LIMIT 10\n",
-    "\"\"\"\n",
-    "\n",
-    "results = store.execute_query(query)\n",
-    "for result in results.get(\"results\", {}).get(\"bindings\", []):\n",
-    "    print(result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 4. Deleting and Updating\n",
-    "\n",
-    "Manage triplet lifecycle."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "store.delete_triplet(triplet)\n",
-    "\n",
-    "# Update is delete + add\n",
-    "new_triplet = Triplet(\n",
-    "    subject=\"http://example.org/Alice\",\n",
-    "    predicate=\"http://xmlns.com/foaf/0.1/knows\",\n",
-    "    object=\"http://example.org/David\"\n",
-    ")\n",
-    "store.update_triplet(triplet, new_triplet)"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.8.5"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}

cookbook/advanced/Advanced_Vector_Store_and_Search.ipynb

@@ -24,9 +24,297 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 8,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
+      "Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.3)\n",
+      "Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
+      "Requirement already satisfied: umap-learn>=0.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.5.9.post2)\n",
+      "Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
+      "Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
+      "Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
+      "Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
+      "Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
+      "Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.6.1)\n",
+      "Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
+      "Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
+      "Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
+      "Requirement already satisfied: ipywidgets>=8.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.1.1)\n",
+      "Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
+      "Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.14.3)\n",
+      "Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
+      "Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
+      "Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
+      "Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
+      "Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
+      "Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
+      "Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
+      "Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
+      "Requirement already satisfied: fastembed>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
+      "Requirement already satisfied: onnxruntime>=1.17.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.22.1)\n",
+      "Requirement already satisfied: tokenizers>=0.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.4)\n",
+      "Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
+      "Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
+      "Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
+      "Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
+      "Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
+      "Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
+      "Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
+      "Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
+      "Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
+      "Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
+      "Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
+      "Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
+      "Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
+      "Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
+      "Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
+      "Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
+      "Requirement already satisfied: pydantic>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
+      "Requirement already satisfied: fastmcp>=0.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.14.1)\n",
+      "Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
+      "Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (14.2.0)\n",
+      "Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
+      "Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
+      "Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
+      "Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
+      "Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
+      "Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
+      "Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.23.1)\n",
+      "Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.39.1)\n",
+      "Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.39.1)\n",
+      "Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.60b1)\n",
+      "Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
+      "Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
+      "Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
+      "Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
+      "Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
+      "Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
+      "Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
+      "Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
+      "Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
+      "Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
+      "Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
+      "Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
+      "Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.15.0)\n",
+      "Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
+      "Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
+      "Requirement already satisfied: soupsieve>=1.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
+      "Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
+      "Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
+      "Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
+      "Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (4.5.1)\n",
+      "Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
+      "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
+      "Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
+      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
+      "Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
+      "Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
+      "Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
+      "Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
+      "Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
+      "Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
+      "Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
+      "Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
+      "Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
+      "Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
+      "Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
+      "Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
+      "Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
+      "Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
+      "Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
+      "Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (0.7.0)\n",
+      "Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (2.41.4)\n",
+      "Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (0.4.2)\n",
+      "Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
+      "Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
+      "Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
+      "Requirement already satisfied: huggingface-hub<1.0,>=0.20 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (0.30.2)\n",
+      "Requirement already satisfied: mmh3<6.0.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (5.2.0)\n",
+      "Requirement already satisfied: py-rust-stemmers<0.2.0,>=0.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (0.1.5)\n",
+      "Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub<1.0,>=0.20->fastembed>=0.2.0->semantica) (3.16.1)\n",
+      "Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub<1.0,>=0.20->fastembed>=0.2.0->semantica) (2023.10.0)\n",
+      "Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
+      "Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
+      "Requirement already satisfied: authlib>=1.6.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.6.6)\n",
+      "Requirement already satisfied: cyclopts>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (4.4.0)\n",
+      "Requirement already satisfied: exceptiongroup>=1.2.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.3.0)\n",
+      "Requirement already satisfied: httpx>=0.28.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.28.1)\n",
+      "Requirement already satisfied: jsonschema-path>=0.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.3.4)\n",
+      "Requirement already satisfied: mcp>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.25.0)\n",
+      "Requirement already satisfied: openapi-pydantic>=0.5.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.5.1)\n",
+      "Requirement already satisfied: py-key-value-aio<0.4.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.3.0)\n",
+      "Requirement already satisfied: pydocket>=0.15.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.16.0)\n",
+      "Requirement already satisfied: pyperclip>=1.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.9.0)\n",
+      "Requirement already satisfied: websockets>=15.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (15.0.1)\n",
+      "Requirement already satisfied: py-key-value-shared==0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio<0.4.0,>=0.3.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.3.0)\n",
+      "Requirement already satisfied: beartype>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio<0.4.0,>=0.3.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.22.9)\n",
+      "Requirement already satisfied: diskcache>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (5.6.3)\n",
+      "Requirement already satisfied: pathvalidate>=3.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.3.1)\n",
+      "Requirement already satisfied: keyring>=25.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (25.6.0)\n",
+      "Requirement already satisfied: cachetools>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (5.5.2)\n",
+      "Requirement already satisfied: attrs>=23.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (24.2.0)\n",
+      "Requirement already satisfied: docstring-parser<4.0,>=0.15 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (0.16)\n",
+      "Requirement already satisfied: rich-rst<2.0.0,>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (1.3.2)\n",
+      "Requirement already satisfied: docutils in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich-rst<2.0.0,>=1.3.1->cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (0.21.2)\n",
+      "Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
+      "Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
+      "Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
+      "Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
+      "Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
+      "Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
+      "Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
+      "Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
+      "Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
+      "Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
+      "Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
+      "Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
+      "Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
+      "Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
+      "Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.28.1->fastmcp>=0.1.0->semantica) (1.0.9)\n",
+      "Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.28.1->fastmcp>=0.1.0->semantica) (0.16.0)\n",
+      "Requirement already satisfied: comm>=0.1.3 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (0.2.0)\n",
+      "Requirement already satisfied: ipython>=6.1.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (8.17.2)\n",
+      "Requirement already satisfied: traitlets>=4.3.1 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (5.13.0)\n",
+      "Requirement already satisfied: widgetsnbextension~=4.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipywidgets>=8.0.0->semantica) (4.0.9)\n",
+      "Requirement already satisfied: jupyterlab-widgets~=3.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipywidgets>=8.0.0->semantica) (3.0.9)\n",
+      "Requirement already satisfied: decorator in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (5.1.1)\n",
+      "Requirement already satisfied: jedi>=0.16 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.19.1)\n",
+      "Requirement already satisfied: matplotlib-inline in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.1.6)\n",
+      "Requirement already satisfied: pygments>=2.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.19.2)\n",
+      "Requirement already satisfied: stack-data in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.6.3)\n",
+      "Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
+      "Requirement already satisfied: parso<0.9.0,>=0.8.3 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.8.3)\n",
+      "Requirement already satisfied: pathable<0.5.0,>=0.4.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.4.4)\n",
+      "Requirement already satisfied: referencing<0.37.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.30.2)\n",
+      "Requirement already satisfied: rpds-py>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from referencing<0.37.0->jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.12.0)\n",
+      "Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.2.3)\n",
+      "Requirement already satisfied: importlib_metadata>=4.11.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (6.8.0)\n",
+      "Requirement already satisfied: jaraco.classes in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.4.0)\n",
+      "Requirement already satisfied: jaraco.functools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (4.1.0)\n",
+      "Requirement already satisfied: jaraco.context in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (6.0.1)\n",
+      "Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib_metadata>=4.11.4->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.17.0)\n",
+      "Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
+      "Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
+      "Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
+      "Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
+      "Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
+      "Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
+      "Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
+      "Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
+      "Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
+      "Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
+      "Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
+      "Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
+      "Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
+      "Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
+      "Requirement already satisfied: httpx-sse>=0.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (0.4.0)\n",
+      "Requirement already satisfied: jsonschema>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (4.23.0)\n",
+      "Requirement already satisfied: pydantic-settings>=2.5.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.11.0)\n",
+      "Requirement already satisfied: pyjwt>=2.10.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pyjwt[crypto]>=2.10.1->mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.10.1)\n",
+      "Requirement already satisfied: python-multipart>=0.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (0.0.18)\n",
+      "Requirement already satisfied: pywin32>=310 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (311)\n",
+      "Requirement already satisfied: sse-starlette>=1.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.3.6)\n",
+      "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema>=4.20.0->mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2023.7.1)\n",
+      "Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
+      "Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
+      "Requirement already satisfied: coloredlogs in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (15.0.1)\n",
+      "Requirement already satisfied: flatbuffers in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (23.5.26)\n",
+      "Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (1.13.3)\n",
+      "Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
+      "Requirement already satisfied: opentelemetry-semantic-conventions==0.60b1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.60b1)\n",
+      "Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
+      "Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
+      "Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
+      "Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
+      "Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.35.4)\n",
+      "Requirement already satisfied: email-validator>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic[email]>=2.11.7->fastmcp>=0.1.0->semantica) (2.2.0)\n",
+      "Requirement already satisfied: dnspython>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from email-validator>=2.0.0->pydantic[email]>=2.11.7->fastmcp>=0.1.0->semantica) (2.4.2)\n",
+      "Requirement already satisfied: cloudpickle>=3.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (3.1.2)\n",
+      "Requirement already satisfied: fakeredis>=2.32.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.33.0)\n",
+      "Requirement already satisfied: opentelemetry-exporter-prometheus>=0.60b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (0.60b1)\n",
+      "Requirement already satisfied: python-json-logger>=2.0.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.0.7)\n",
+      "Requirement already satisfied: typer>=0.15.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (0.15.2)\n",
+      "Requirement already satisfied: sortedcontainers>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis>=2.32.1->fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.4.0)\n",
+      "Requirement already satisfied: lupa>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.6)\n",
+      "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-instrumentation->semantica) (1.17.2)\n",
+      "Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
+      "Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
+      "Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
+      "Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
+      "Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
+      "Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
+      "Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
+      "Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
+      "Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
+      "Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
+      "Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
+      "Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
+      "Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
+      "Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
+      "Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
+      "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
+      "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
+      "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
+      "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
+      "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
+      "Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
+      "Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
+      "Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
+      "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
+      "Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
+      "Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
+      "Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
+      "Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
+      "Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
+      "Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
+      "Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
+      "Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
+      "Requirement already satisfied: shellingham>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from typer>=0.15.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (1.5.4)\n",
+      "Requirement already satisfied: pynndescent>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from umap-learn>=0.5.0->semantica) (0.5.13)\n",
+      "Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
+      "Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
+      "Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
+      "Requirement already satisfied: humanfriendly>=9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coloredlogs->onnxruntime>=1.17.0->semantica) (10.0)\n",
+      "Requirement already satisfied: pyreadline3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from humanfriendly>=9.1->coloredlogs->onnxruntime>=1.17.0->semantica) (3.4.1)\n",
+      "Requirement already satisfied: more-itertools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jaraco.classes->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (10.6.0)\n",
+      "Requirement already satisfied: backports.tarfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jaraco.context->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (1.2.0)\n",
+      "Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
+      "Requirement already satisfied: executing>=1.2.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.0.1)\n",
+      "Requirement already satisfied: asttokens>=2.1.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.4.1)\n",
+      "Requirement already satisfied: pure-eval in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.2.2)\n",
+      "Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->onnxruntime>=1.17.0->semantica) (1.3.0)\n"
+     ]
+    }
+   ],
    "source": [
     "!pip install semantica\n"
    ]
@@ -42,9 +330,25 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 9,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Selected model: BAAI/bge-small-en-v1.5\n",
+      "Embedding dimension: 128\n"
+     ]
+    }
+   ],
    "source": [
     "from semantica.embeddings import TextEmbedder\n",
     "\n",
@@ -69,15 +373,43 @@
     "\n",
     "### Simple Rule\n",
     "- Less than 10,000 items? Use **Flat**\n",
-    "- Between 10,000 and 1 million? Use **HNSW** \u2705 (recommended)\n",
+    "- Between 10,000 and 1 million? Use **HNSW** ✅ (recommended)\n",
     "- More than 1 million? Use **IVF**"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 10,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedWriter name=3>\n",
+      "  del self._target, self._args, self._kwargs\n",
+      "ResourceWarning: Enable tracemalloc to get the object allocation traceback\n",
+      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedReader name=4>\n",
+      "  del self._target, self._args, self._kwargs\n",
+      "ResourceWarning: Enable tracemalloc to get the object allocation traceback\n",
+      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedReader name=5>\n",
+      "  del self._target, self._args, self._kwargs\n",
+      "ResourceWarning: Enable tracemalloc to get the object allocation traceback\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Found 5 most similar documents:\n",
+      "  1. Document doc_1935 (distance: 110.917)\n",
+      "  2. Document doc_3860 (distance: 111.535)\n",
+      "  3. Document doc_277 (distance: 113.270)\n",
+      "  4. Document doc_1903 (distance: 113.371)\n",
+      "  5. Document doc_2959 (distance: 113.612)\n"
+     ]
+    }
+   ],
    "source": [
     "from semantica.vector_store import FAISSStore\n",
     "import numpy as np\n",
@@ -90,14 +422,14 @@
     "\n",
     "# HNSW Index - Best for most cases\n",
     "index = adapter.create_index(index_type=\"hnsw\", metric=\"L2\", m=16)\n",
-    "adapter.add_vectors(index, vectors, ids=[f\"doc_{i}\" for i in range(len(vectors))])\n",
+    "adapter.add_vectors(vectors, ids=[f\"doc_{i}\" for i in range(len(vectors))])\n",
     "\n",
     "# Search for similar vectors\n",
-    "distances, indices = adapter.search(index, query, k=5)\n",
+    "results = adapter.search_similar(query, k=5)\n",
     "\n",
     "print(\"Found 5 most similar documents:\")\n",
-    "for i, (dist, idx) in enumerate(zip(distances, indices), 1):\n",
-    "    print(f\"  {i}. Document {idx} (distance: {dist:.3f})\")"
+    "for i, result in enumerate(results, 1):\n",
+    "    print(f\"  {i}. Document {result['id']} (distance: {result['distance']:.3f})\")"
    ]
   },
   {
@@ -117,9 +449,29 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 11,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Technology articles from 2024:\n",
+      "  - AI in Healthcare\n",
+      "  - Business Strategy\n",
+      "  - Data Science Guide\n",
+      "  - Marketing Tips\n",
+      "  - Machine Learning Basics\n",
+      "\n",
+      "All articles from 2024:\n",
+      "  - AI in Healthcare (Technology)\n",
+      "  - Business Strategy (Business)\n",
+      "  - Data Science Guide (Technology)\n",
+      "  - Marketing Tips (Business)\n",
+      "  - Machine Learning Basics (Technology)\n"
+     ]
+    }
+   ],
    "source": [
     "from semantica.vector_store import HybridSearch, MetadataFilter\n",
     "import numpy as np\n",
@@ -177,9 +529,25 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 12,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Combined results (fair ranking):\n",
+      "  1. Data Science Guide (score: 0.033)\n",
+      "  2. Machine Learning Basics (score: 0.016)\n",
+      "  3. AI in Healthcare (score: 0.016)\n",
+      "\n",
+      "Combined results (prefer recent):\n",
+      "  1. Data Science Guide (score: 0.929)\n",
+      "  2. AI in Healthcare (score: 0.630)\n",
+      "  3. Business Strategy (score: 0.595)\n"
+     ]
+    }
+   ],
    "source": [
     "from semantica.vector_store import SearchRanker\n",
     "\n",
@@ -237,9 +605,21 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 13,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Company A has 10 documents\n",
+      "Company B has 15 documents\n",
+      "\n",
+      "Admin can delete: True\n",
+      "User can delete: False\n"
+     ]
+    }
+   ],
    "source": [
     "from semantica.vector_store import NamespaceManager\n",
     "\n",
@@ -286,7 +666,7 @@
     "# Small dataset (< 10,000 items)\n",
     "index = adapter.create_index(index_type=\"flat\", metric=\"L2\")\n",
     "\n",
-    "# Medium dataset (10,000 - 1,000,000 items) \u2705 RECOMMENDED\n",
+    "# Medium dataset (10,000 - 1,000,000 items) ✅ RECOMMENDED\n",
     "index = adapter.create_index(index_type=\"hnsw\", metric=\"L2\", m=16)\n",
     "\n",
     "# Large dataset (> 1,000,000 items)\n",
@@ -340,10 +720,10 @@
     "\n",
     "You've learned:\n",
     "\n",
-    "1. \u2705 **Index Selection**: Use HNSW for most cases\n",
-    "2. \u2705 **Smart Filtering**: Combine vector search with metadata\n",
-    "3. \u2705 **Result Fusion**: Merge searches from different sources\n",
-    "4. \u2705 **Data Isolation**: Keep users' data separate\n",
+    "1. ✅ **Index Selection**: Use HNSW for most cases\n",
+    "2. ✅ **Smart Filtering**: Combine vector search with metadata\n",
+    "3. ✅ **Result Fusion**: Merge searches from different sources\n",
+    "4. ✅ **Data Isolation**: Keep users' data separate\n",
     "\n",
     "### Next Steps\n",
     "\n",
@@ -377,4 +757,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 2
-}
+}

cookbook/advanced/knowledge_graph.ttl

@@ -0,0 +1,20 @@
+<e1> a <Person> ;
+    semantica:text "" ;
+    semantica:confidence 1.0 .
+
+<e2> a <Person> ;
+    semantica:text "" ;
+    semantica:confidence 1.0 .
+
+<e3> a <Organization> ;
+    semantica:text "" ;
+    semantica:confidence 1.0 .
+
+<e4> a <Project> ;
+    semantica:text "" ;
+    semantica:confidence 1.0 .
+
+<e1> <reports_to> <e2> .
+<e1> <works_for> <e3> .
+<e2> <works_for> <e3> .
+<e1> <works_on> <e4> .

cookbook/advanced/quantum_ontology.ttl

@@ -0,0 +1,21 @@
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix owl: <http://www.w3.org/2002/07/owl#> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+@prefix ont: <https://semantica.dev/ontology/> .
+
+<https://semantica.dev/ontology/> a owl:Ontology ;
+    rdfs:label "QuantumOntologyNLP" ;
+    owl:versionInfo "1.0" .
+
+<> a owl:Class ;
+    rdfs:label "Org" .
+    rdfs:comment "Class representing org entities" .
+
+<> a owl:Class ;
+    rdfs:label "Person" .
+    rdfs:comment "Class representing person entities" .
+
+<> a owl:Class ;
+    rdfs:label "Gpe" .
+    rdfs:comment "Class representing gpe entities" .

cookbook/introduction/01_Welcome_to_Semantica.ipynb

@@ -6,6 +6,217 @@
    "source": [
     "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/01_Welcome_to_Semantica.ipynb)\n",
     "\n",
+    "Semantica is a **semantic intelligence and knowledge engineering framework**. It helps you:\n",
+    "\n",
+    "- Build **knowledge graphs** from unstructured and semi-structured data\n",
+    "- Create a unified **semantic layer** on top of diverse data sources\n",
+    "- Power **GraphRAG**, AI agents, and multi-agent systems with structured knowledge\n",
+    "- Incorporate **temporal and quality-aware reasoning** into your applications\n",
+    "\n",
+    "### Core Capabilities\n",
+    "\n",
+    "- **Universal ingestion**: Files, web, feeds, databases, repositories, streams\n",
+    "- **Rich parsing**: PDFs, Office documents, HTML, JSON, CSV, images, code\n",
+    "- **Normalization**: Cleaning, language detection, entity normalization, date/number standardization\n",
+    "- **Semantic extraction**: Named entities, relationships, events, semantic networks\n",
+    "- **Knowledge graph construction**: Property graphs from entities and relations\n",
+    "- **Embeddings and vector search**: Text and graph embeddings, hybrid retrieval\n",
+    "- **Reasoning and ontology**: Rule-based inference, ontology generation and validation\n",
+    "- **Visualization and analytics**: Graph visualizations and quality metrics"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Who Is Semantica For?\n",
+    "\n",
+    "- **AI/ML engineers** building GraphRAG systems, agents, and tools that need long-term memory\n",
+    "- **Data engineers** orchestrating semantic enrichment pipelines over large, heterogeneous datasets\n",
+    "- **Knowledge engineers and ontologists** designing and maintaining formal knowledge structures\n",
+    "- **Researchers and analysts** creating domain knowledge graphs from documents and data feeds\n",
+    "- **Product and platform teams** embedding semantic intelligence into applications and services"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Architecture Overview\n",
+    "\n",
+    "Semantica is organized as three conceptual layers and multiple concrete modules.\n",
+    "\n",
+    "### Layers\n",
+    "\n",
+    "- **Input Layer**\n",
+    "  - Connects to files, web pages, APIs, databases, email, feeds, repositories, and streams\n",
+    "  - Normalizes these different sources into a unified internal representation\n",
+    "\n",
+    "- **Semantic Layer**\n",
+    "  - Performs parsing, cleaning, semantic extraction, graph construction, embeddings, and reasoning\n",
+    "  - This is where **unstructured data becomes structured knowledge**\n",
+    "\n",
+    "- **Output Layer**\n",
+    "  - Exposes knowledge graphs, embeddings, ontologies, and analytics\n",
+    "  - Integrates with vector stores, graph databases, and downstream applications"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🧩 Semantica Modules Reference\n",
+    "\n",
+    "Semantica is modular by design. Here is a comprehensive guide to all available modules, grouped by functionality.\n",
+    "\n",
+    "### 📥 Ingestion & Parsing\n",
+    "Modules that handle raw data input and structure.\n",
+    "\n",
+    "| Module | Description | Key Capabilities |\n",
+    "| :--- | :--- | :--- |\n",
+    "| **`ingest`** | **Data Ingestion**<br>Connects to data sources. | • File, Web, Feed, Stream ingestion<br>• DB, Email, Repo, MCP support |\n",
+    "| **`parse`** | **Document Parsing**<br>Parses raw content into structures. | • PDF, HTML, JSON, CSV, Excel<br>• Image & Code parsing |\n",
+    "\n",
+    "### ⚙️ Data Processing\n",
+    "Modules that clean, normalize, and split data.\n",
+    "\n",
+    "| Module | Description | Key Capabilities |\n",
+    "| :--- | :--- | :--- |\n",
+    "| **`normalize`** | **Data Normalization**<br>Cleans and standardizes text. | • Text cleaning & Language detection<br>• Entity, Date, Number normalization |\n",
+    "| **`split`** | **Chunking**<br>Splits documents for RAG. | • Recursive character splitting<br>• Semantic & Token-based splitting |\n",
+    "\n",
+    "### 🧠 Extraction & Enrichment\n",
+    "Modules that extract meaning, structure, and vectors from raw data.\n",
+    "\n",
+    "| Module | Description | Key Capabilities |\n",
+    "| :--- | :--- | :--- |\n",
+    "| **`semantic_extract`** | **Information Extraction**<br>Extracts entities and relations. | • NER & Relation Extraction<br>• Event & Semantic Network detection |\n",
+    "| **`context`** | **Agent Memory**<br>Manages state for AI agents. | • Long-term memory & history<br>• Context graph & RAG integration |\n",
+    "\n",
+    "### 🕸️ Knowledge Graph Core\n",
+    "Modules for building, refining, and resolving knowledge graphs.\n",
+    "\n",
+    "| Module | Description | Key Capabilities |\n",
+    "| :--- | :--- | :--- |\n",
+    "| **`kg`** | **Graph Construction**<br>Builds and analyzes graphs. | • Graph Building & Analysis<br>• Validation & Entity Resolution |\n",
+    "| **`conflicts`** | **Conflict Resolution**<br>Resolves data contradictions. | • Source reliability scoring<br>• Truth discovery algorithms |\n",
+    "| **`deduplication`** | **Entity Resolution**<br>Merges duplicate entities. | • Similarity-based blocking<br>• Clustering & Canonicalization |\n",
+    "\n",
+    "### 💾 Storage & Retrieval\n",
+    "Modules for persisting and querying data.\n",
+    "\n",
+    "| Module | Description | Key Capabilities |\n",
+    "| :--- | :--- | :--- |\n",
+    "| **`embeddings`** | **Vector Embeddings**<br>Generates semantic vectors. | • Text & Graph embeddings<br>• Multi-provider support (OpenAI, etc.) |\n",
+    "| **`vector_store`** | **Vector Database**<br>Stores and searches vectors. | • Similarity search & Filtering<br>• Hybrid search (Vector + Keyword) |\n",
+    "| **`graph_store`** | **Property Graph Store**<br>Persists graph data. | • Neo4j, FalkorDB adapters<br>• Cypher query support |\n",
+    "| **`triplet_store`** | **RDF Store**<br>Persists semantic triplets. | • SPARQL endpoints<br>• BlazeGraph, Jena, Virtuoso adapters |\n",
+    "\n",
+    "### 🔎 Reasoning & Analysis\n",
+    "Modules for deriving new knowledge and evaluating quality.\n",
+    "\n",
+    "| Module | Description | Key Capabilities |\n",
+    "| :--- | :--- | :--- |\n",
+    "| **`reasoning`** | **Reasoner Facade**<br>Unified interface for inference. | • Datalog/Rule-based inference<br>• Forward/Backward chaining |\n",
+    "| **`ontology`** | **Ontology Management**<br>Manages schema and definitions. | • Ontology generation from data<br>• Validation & Evolution |\n",
+    "| **`visualization`** | **Visual Analytics**<br>Visualizes graphs and metrics. | • 2D/3D Graph visualization<br>• Interactive plots & dashboards |\n",
+    "| **`evals`** | **Evaluation**<br>Benchmarks pipeline quality. | • RAG & Graph quality metrics<br>• Ground truth comparison |\n",
+    "\n",
+    "### 🛠️ Orchestration & Utils\n",
+    "Modules for managing the framework and workflows.\n",
+    "\n",
+    "| Module | Description | Key Capabilities |\n",
+    "| :--- | :--- | :--- |\n",
+    "| **`core`** | **Framework Core**<br>Main entry point and config. | • Lifecycle management<br>• Plugin system & Configuration |\n",
+    "| **`pipeline`** | **Workflow Orchestration**<br>Manages complex flows. | • DAG execution & Retries<br>• Error handling & Observability |\n",
+    "| **`seed`** | **Data Seeding**<br>Initializes knowledge bases. | • Taxonomy & Ontology seeding<br>• Reference data loading |\n",
+    "| **`export`** | **Data Export**<br>Exports data to files. | • JSON, CSV, RDF, GEXF export<br>• Report generation |\n",
+    "| **`utils`** | **Utilities**<br>Common helper functions. | • Logging, Async, Hashing<br>• Text processing helpers |"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Core Concepts (High-Level)\n",
+    "\n",
+    "- **Knowledge graph**\n",
+    "  - Nodes represent entities such as people, organizations, locations, events, or concepts\n",
+    "  - Edges represent relationships such as `works_for`, `located_in`, `founded_by`\n",
+    "  - Properties capture attributes and metadata such as timestamps, sources, and confidence\n",
+    "\n",
+    "- **Entities and relationships**\n",
+    "  - Entities are extracted from text and data using NER\n",
+    "  - Relationships connect entities and are extracted using pattern-based, model-based, or LLM-based methods\n",
+    "\n",
+    "- **Embeddings**\n",
+    "  - Numerical vectors that encode semantic meaning of text or graph structures\n",
+    "  - Used for semantic search, clustering, and similarity-based retrieval\n",
+    "\n",
+    "- **GraphRAG**\n",
+    "  - Combines vector search with graph traversal\n",
+    "  - Uses both embeddings and graph structure to retrieve rich, context-aware information\n",
+    "\n",
+    "- **Ontology**\n",
+    "  - A formal model of classes, relationships, and constraints in a domain\n",
+    "  - Used to standardize meaning, enable reasoning, and integrate heterogeneous data\n",
+    "\n",
+    "- **Quality and governance**\n",
+    "  - Quality metrics (completeness, consistency, accuracy, coverage)\n",
+    "  - Conflict detection and resolution at the knowledge graph level"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Installation\n",
+    "\n",
+    "You can install Semantica from PyPI. In this notebook, we use a pip cell so it can run in local Jupyter or Colab.\n",
+    "\n",
+    "Equivalent shell commands:\n",
+    "\n",
+    "```bash\n",
+    "pip install semantica\n",
+    "pip install semantica[all]\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Basic Configuration\n",
+    "\n",
+    "Semantica uses configuration for API keys, embedding providers, and knowledge graph options. The example below mirrors a typical configuration while staying simple enough for a notebook."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'api_keys:\\n  openai: your_key_here\\n  anthropic: your_key_here\\nembedding:\\n  provider: openai\\n  model: text-embedding-3-large\\n  dimensions: 3072\\nknowledge_graph:\\n  backend: networkx\\n  temporal: true\\n'"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from semantica.core import Config\n",
+    "config = Config()\n",
+    "print(config.to_yaml())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
     "# Welcome to Semantica\n",
     "\n",
     "**Open Source Framework for Semantic Layer & Knowledge Engineering**\n",
@@ -361,7 +572,7 @@
     "\n",
     "| Module | Description | Key Capabilities |\n",
     "| :--- | :--- | :--- |\n",
-    "| **`reasoning`** | **Inference Engine**<br>Derives new facts via rules. | • Datalog/Rule-based inference<br>• Forward/Backward chaining |\n",
+    "| **`reasoning`** | **Reasoner Facade**<br>Unified interface for inference. | • Datalog/Rule-based inference<br>• Forward/Backward chaining<br>• Automated explanation generation |\n",
     "| **`ontology`** | **Ontology Management**<br>Manages schema and definitions. | • Ontology generation from data<br>• Validation & Evolution |\n",
     "| **`visualization`** | **Visual Analytics**<br>Visualizes graphs and metrics. | • 2D/3D Graph visualization<br>• Interactive plots & dashboards |\n",
     "| **`evals`** | **Evaluation**<br>Benchmarks pipeline quality. | • RAG & Graph quality metrics<br>• Ground truth comparison |\n",
@@ -543,7 +754,7 @@
     {
      "data": {
       "text/html": [
-       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CommunityDetector</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>VectorStore</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>HybridSearch</td><td>-</td><td>3.15s</td></tr><tr><td>✅</td><td>Semantica is embedding</td><td>💾 embeddings</td><td>TextEmbedder</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is visualizing</td><td>📈 visualization</td><td>KGVisualizer</td><td>-</td><td>0.17s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>OntologyGenerator</td><td>-</td><td>0.04s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>ClassInferrer</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>PropertyGenerator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>RuleManager</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>InferenceEngine</td><td>-</td><td>0.01s</td></tr></table></div>"
+       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CommunityDetector</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>VectorStore</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>HybridSearch</td><td>-</td><td>3.15s</td></tr><tr><td>✅</td><td>Semantica is embedding</td><td>💾 embeddings</td><td>TextEmbedder</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is visualizing</td><td>📈 visualization</td><td>KGVisualizer</td><td>-</td><td>0.17s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>OntologyGenerator</td><td>-</td><td>0.04s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>ClassInferrer</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>PropertyGenerator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>-</td><td>0.01s</td></tr></table></div>"
       ],
       "text/plain": [
        "<IPython.core.display.HTML object>"
@@ -1647,16 +1858,16 @@
     }
    ],
    "source": [
-    "from semantica.reasoning import InferenceEngine\n",
+    "from semantica.reasoning import Reasoner\n",
     "\n",
     "# Simple rule: If X founded Y, then X works_for Y\n",
     "rule = \"\"\"\n",
     "IF (?x founded ?y) THEN (?x works_for ?y)\n",
     "\"\"\"\n",
     "\n",
-    "engine = InferenceEngine()\n",
-    "engine.add_rule(rule)\n",
-    "inferred_facts = engine.infer(kg)\n",
+    "reasoner = Reasoner()\n",
+    "reasoner.add_rule(rule)\n",
+    "inferred_facts = reasoner.infer_facts(kg)\n",
     "\n",
     "print(f\"Inferred {len(inferred_facts)} new facts\")"
    ]

cookbook/introduction/02_Data_Ingestion.ipynb

@@ -734,4 +734,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 2
-}
+}

cookbook/introduction/03_Document_Parsing.ipynb

@@ -266,4 +266,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 2
-}
+}

cookbook/use_cases/advanced_rag/data/sample_graphrag_paper.txt

@@ -0,0 +1,13 @@
+Graph Retrieval-Augmented Generation (GraphRAG): A New Era for Intelligent Search
+
+GraphRAG is an advanced technique that combines the retrieval capabilities of vector databases with the structural reasoning of knowledge graphs. Unlike traditional RAG, which relies solely on vector similarity, GraphRAG leverages the relationships between entities to provide more contextually accurate and comprehensive answers.
+
+Key Components:
+1. Knowledge Graph: A structured representation of data where nodes represent entities and edges represent relationships.
+2. Vector Search: Finds semantically similar text chunks.
+3. Graph Traversal: Navigates the knowledge graph to find related entities that might not be semantically similar but are structurally relevant.
+
+Benefits:
+- Improved Context: By following relationships, the system can understand the broader context of a query.
+- Multi-hop Reasoning: Can answer complex questions that require connecting multiple pieces of information.
+- Reduced Hallucinations: Grounding answers in a verified knowledge structure reduces the likelihood of generating false information.

cookbook/use_cases/advanced_rag/expert_guide.txt

@@ -0,0 +1,5 @@
+
+RETINOL CLINICAL GUIDE
+Mechanism: Binds to retinoic acid receptors to increase cellular turnover.
+Precautions: Should not be used with high-concentration AHA/BHA exfoliants.
+Synergy: Highly effective when paired with Niacinamide to offset potential erythema.

cookbook/use_cases/advanced_rag/expert_skincare_guide.txt

@@ -0,0 +1,6 @@
+
+RETINOL CLINICAL GUIDE v2.1
+Mechanism: Binds to retinoic acid receptors (RAR) to increase cellular turnover.
+Precautions: Should not be used with high-concentration AHA/BHA exfoliants.
+Synergy: Highly effective when paired with Niacinamide to offset potential erythema.
+Target: Stratum corneum thickening and dermal collagen synthesis.

cookbook/use_cases/advanced_rag/graphrag_kg.graphml

@@ -0,0 +1,254 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<graphml xmlns="http://graphml.graphdrawing.org/xmlns"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns
+         http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
+
+  <key id="type" for="node" attr.name="type" attr.type="string"/>
+  <key id="confidence" for="node" attr.name="confidence" attr.type="double"/>
+
+  <graph id="G" edgedefault="directed">
+
+    <node id="makeup_and_beauty_blog">
+      <data key="label">Makeup and Beauty Blog</data>
+      <data key="type">ORG</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="monday_poll">
+      <data key="label">Monday Poll</data>
+      <data key="type">EVENT</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="2007">
+      <data key="label">2007</data>
+      <data key="type">DATE</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="rosacea">
+      <data key="label">Rosacea</data>
+      <data key="type">CONCEPT</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="dr._bailey">
+      <data key="label">Dr. Bailey</data>
+      <data key="type">PERSON</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="green_tea_antioxidant_skin_therapy">
+      <data key="label">Green Tea Antioxidant Skin Therapy</data>
+      <data key="type">PRODUCT</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="vol._892">
+      <data key="label">Vol. 892</data>
+      <data key="type">EVENT</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="laneige">
+      <data key="label">Laneige</data>
+      <data key="type">ORG</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="sausalito">
+      <data key="label">Sausalito</data>
+      <data key="type">GPE</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="ulta">
+      <data key="label">Ulta</data>
+      <data key="type">ORG</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="december_15,_2025">
+      <data key="label">December 15, 2025</data>
+      <data key="type">DATE</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="jo_malone">
+      <data key="label">Jo Malone</data>
+      <data key="type">ORG</data>
+      <data key="confidence">1</data>
+    </node>
+    <node id="trader_joe">
+      <data key="label">Trader Joe</data>
+      <data key="type">ORG</data>
+      <data key="confidence">1</data>
+    </node>
+    <node id="hawaii">
+      <data key="label">hawaii</data>
+      <data key="type">GPE</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="benzoyl_peroxide_cream">
+      <data key="label">Benzoyl Peroxide Cream</data>
+      <data key="type">PRODUCT</data>
+      <data key="confidence">1</data>
+    </node>
+    <node id="facial_dandruff">
+      <data key="label">Facial dandruff</data>
+      <data key="type">CONCEPT</data>
+      <data key="confidence">1</data>
+    </node>
+    <node id="calming_zinc_soap">
+      <data key="label">Calming Zinc Soap</data>
+      <data key="type">PRODUCT</data>
+      <data key="confidence">1</data>
+    </node>
+    <node id="hydrate">
+      <data key="label">Hydrate</data>
+      <data key="type">CONCEPT</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="daily_moisturizing_face_cream">
+      <data key="label">Daily Moisturizing Face Cream</data>
+      <data key="type">PRODUCT</data>
+      <data key="confidence">1.0</data>
+    </node>
+    <node id="omega_enriched_face_booster_oil">
+      <data key="label">Omega Enriched Face Booster Oil</data>
+      <data key="type">PRODUCT</data>
+      <data key="confidence">1.0</data>
+    </node>
+
+    <edge source="Makeup and Beauty Blog" target="Monday Poll">
+      <data key="label">hosts</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Monday Poll" target="December 15, 2025">
+      <data key="label">occurs on</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Makeup and Beauty Blog Monday Poll, Vol. 893">
+      <data key="label">publishes</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Monday">
+      <data key="label">has</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="2007">
+      <data key="label">has</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Monday Poll">
+      <data key="label">hosts</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Makeup and Beauty Blog Monday Poll">
+      <data key="label">posts</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Vol. 892">
+      <data key="label">posts</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="2007">
+      <data key="label">has been active since</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="MBB" target="Makeup and Beauty Blog">
+      <data key="label">related_to</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Makeup and Beauty Blog">
+      <data key="label">related_to</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Monday Poll">
+      <data key="label">hosts</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Vol. 891">
+      <data key="label">posts</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Makeup and Beauty Blog" target="Monday Poll">
+      <data key="label">posts</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Cavallo Point" target="Sausalito">
+      <data key="label">located_in</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Dr. Bailey" target="Green Tea Antioxidant Skin Therapy">
+      <data key="label">prescribes</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Green Tea Antioxidant Skin Therapy" target="Rosacea Therapy Skin Care Kit">
+      <data key="label">part of</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Dr. Bailey" target="Rosacea Therapy Skin Care Kit">
+      <data key="label">uses</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Rosacea Therapy Skin Care Kit" target="rosacea treatment routine">
+      <data key="label">part of</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Dr. Bailey" target="rosacea treatment routine">
+      <data key="label">uses</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Facial dandruff" target="rosacea">
+      <data key="label">often occurs with</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Facial dandruff" target="rosacea">
+      <data key="label">needs to be addressed</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Calming Zinc Soap" target="Facial dandruff">
+      <data key="label">is often sufficient to control</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Calming Zinc Soap" target="rosacea">
+      <data key="label">is often sufficient to control</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Green Tea Antioxidant Skin Therapy" target="Facial dandruff">
+      <data key="label">is often sufficient to control</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Green Tea Antioxidant Skin Therapy" target="rosacea">
+      <data key="label">is often sufficient to control</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Dr. Bailey's Skincare" target="Calming Zinc Soap">
+      <data key="label">produces</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Dr. Bailey's Skincare" target="Green Tea Antioxidant Skin Therapy">
+      <data key="label">produces</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Dr. Bailey" target="Calming Zinc Soap">
+      <data key="label">prescribes</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Dr. Bailey" target="Green Tea Antioxidant Skin Therapy">
+      <data key="label">prescribes</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Hydrate" target="Daily Moisturizing Face Cream">
+      <data key="label">is_achieved_by</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Daily Moisturizing Face Cream" target="Omega Enriched Face Booster Oil">
+      <data key="label">can_be_combined_with</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Omega Enriched Face Booster Oil" target="castor seed oil">
+      <data key="label">contains</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Omega Enriched Face Booster Oil" target="sea buckthorn">
+      <data key="label">contains</data>
+      <data key="confidence">0.9</data>
+    </edge>
+    <edge source="Daily Moisturizing Face Cream" target="Omega Enriched Face Booster Oil">
+      <data key="label">can_be_replaced_with</data>
+      <data key="confidence">0.9</data>
+    </edge>
+  </graph>
+</graphml>

cookbook/use_cases/advanced_rag/graphrag_kg.json

@@ -0,0 +1,678 @@
+{
+  "nodes": [
+    {
+      "id": "makeup_and_beauty_blog",
+      "label": "Makeup and Beauty Blog",
+      "type": "ORG",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "makeup_and_beauty_blog",
+              "name": "Makeup and Beauty Blog",
+              "source": null
+            },
+            {
+              "id": "makeup_and_beauty_blog",
+              "name": "Makeup and Beauty Blog",
+              "source": null
+            },
+            {
+              "id": "makeup_and_beauty_blog_monday_poll,_vol._893",
+              "name": "Makeup and Beauty Blog Monday Poll, Vol. 893",
+              "source": null
+            },
+            {
+              "id": "makeup_and_beauty_blog_monday_poll",
+              "name": "Makeup and Beauty Blog Monday Poll",
+              "source": null
+            },
+            {
+              "id": "mbb",
+              "name": "MBB",
+              "source": null
+            }
+          ],
+          "merge_count": 5
+        }
+      }
+    },
+    {
+      "id": "monday_poll",
+      "label": "Monday Poll",
+      "type": "EVENT",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "monday_poll",
+              "name": "Monday Poll",
+              "source": null
+            },
+            {
+              "id": "monday_poll",
+              "name": "Monday Poll",
+              "source": null
+            },
+            {
+              "id": "monday",
+              "name": "Monday",
+              "source": null
+            },
+            {
+              "id": "holiday",
+              "name": "holiday",
+              "source": null
+            },
+            {
+              "id": "holiday",
+              "name": "holiday",
+              "source": null
+            }
+          ],
+          "merge_count": 5
+        }
+      }
+    },
+    {
+      "id": "2007",
+      "label": "2007",
+      "type": "DATE",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "2007",
+              "name": "2007",
+              "source": null
+            },
+            {
+              "id": "2007",
+              "name": "2007",
+              "source": null
+            },
+            {
+              "id": "2024",
+              "name": "2024",
+              "source": null
+            }
+          ],
+          "merge_count": 3
+        }
+      }
+    },
+    {
+      "id": "rosacea",
+      "label": "Rosacea",
+      "type": "CONCEPT",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "rosacea",
+              "name": "Rosacea",
+              "source": null
+            },
+            {
+              "id": "rosacea",
+              "name": "rosacea",
+              "source": null
+            },
+            {
+              "id": "rosacea_treatment_routine",
+              "name": "rosacea treatment routine",
+              "source": null
+            },
+            {
+              "id": "rosie",
+              "name": "Rosie",
+              "source": null
+            },
+            {
+              "id": "rosacea_therapy_skin_care_kit",
+              "name": "Rosacea Therapy Skin Care Kit",
+              "source": null
+            },
+            {
+              "id": "marnie",
+              "name": "Marnie",
+              "source": null
+            },
+            {
+              "id": "cavallo_point",
+              "name": "Cavallo Point",
+              "source": null
+            },
+            {
+              "id": "castor_seed_oil",
+              "name": "castor seed oil",
+              "source": null
+            }
+          ],
+          "merge_count": 8
+        }
+      }
+    },
+    {
+      "id": "dr._bailey",
+      "label": "Dr. Bailey",
+      "type": "PERSON",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "dr._bailey",
+              "name": "Dr. Bailey",
+              "source": null
+            },
+            {
+              "id": "dr._bailey",
+              "name": "Dr. Bailey",
+              "source": null
+            },
+            {
+              "id": "dr._bailey's_skincare",
+              "name": "Dr. Bailey's Skincare",
+              "source": null
+            },
+            {
+              "id": "dr._bailey's_skincare",
+              "name": "Dr. Bailey's Skincare",
+              "source": null
+            }
+          ],
+          "merge_count": 4
+        }
+      }
+    },
+    {
+      "id": "green_tea_antioxidant_skin_therapy",
+      "label": "Green Tea Antioxidant Skin Therapy",
+      "type": "PRODUCT",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "green_tea_antioxidant_skin_therapy",
+              "name": "Green Tea Antioxidant Skin Therapy",
+              "source": null
+            },
+            {
+              "id": "green_tea_antioxidant_skin_therapy",
+              "name": "Green Tea Antioxidant Skin Therapy",
+              "source": null
+            }
+          ],
+          "merge_count": 2
+        }
+      }
+    },
+    {
+      "id": "vol._892",
+      "label": "Vol. 892",
+      "type": "EVENT",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "vol._892",
+              "name": "Vol. 892",
+              "source": null
+            },
+            {
+              "id": "vol._891",
+              "name": "Vol. 891",
+              "source": null
+            }
+          ],
+          "merge_count": 2
+        }
+      }
+    },
+    {
+      "id": "laneige",
+      "label": "Laneige",
+      "type": "ORG",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "laneige",
+              "name": "Laneige",
+              "source": null
+            },
+            {
+              "id": "lanikai",
+              "name": "Lanikai",
+              "source": null
+            }
+          ],
+          "merge_count": 2
+        }
+      }
+    },
+    {
+      "id": "sausalito",
+      "label": "Sausalito",
+      "type": "GPE",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "sausalito",
+              "name": "Sausalito",
+              "source": null
+            },
+            {
+              "id": "sea_buckthorn",
+              "name": "sea buckthorn",
+              "source": null
+            }
+          ],
+          "merge_count": 2
+        }
+      }
+    },
+    {
+      "id": "ulta",
+      "label": "Ulta",
+      "type": "ORG",
+      "attributes": {
+        "confidence": 1.0,
+        "provenance": {
+          "merged_from": [
+            {
+              "id": "ulta",
+              "name": "Ulta",
+              "source": null
+            },
+            {
+              "id": "clotrimazole",
+              "name": "clotrimazole",
+              "source": null
+            }
+          ],
+          "merge_count": 2
+        }
+      }
+    },
+    {
+      "id": "december_15,_2025",
+      "label": "December 15, 2025",
+      "type": "DATE",
+      "attributes": {
+        "confidence": 1.0
+      }
+    },
+    {
+      "id": "jo_malone",
+      "label": "Jo Malone",
+      "type": "ORG",
+      "attributes": {
+        "confidence": 1
+      }
+    },
+    {
+      "id": "trader_joe",
+      "label": "Trader Joe",
+      "type": "ORG",
+      "attributes": {
+        "confidence": 1
+      }
+    },
+    {
+      "id": "hawaii",
+      "label": "hawaii",
+      "type": "GPE",
+      "attributes": {
+        "confidence": 1.0
+      }
+    },
+    {
+      "id": "benzoyl_peroxide_cream",
+      "label": "Benzoyl Peroxide Cream",
+      "type": "PRODUCT",
+      "attributes": {
+        "confidence": 1
+      }
+    },
+    {
+      "id": "facial_dandruff",
+      "label": "Facial dandruff",
+      "type": "CONCEPT",
+      "attributes": {
+        "confidence": 1
+      }
+    },
+    {
+      "id": "calming_zinc_soap",
+      "label": "Calming Zinc Soap",
+      "type": "PRODUCT",
+      "attributes": {
+        "confidence": 1
+      }
+    },
+    {
+      "id": "hydrate",
+      "label": "Hydrate",
+      "type": "CONCEPT",
+      "attributes": {
+        "confidence": 1.0
+      }
+    },
+    {
+      "id": "daily_moisturizing_face_cream",
+      "label": "Daily Moisturizing Face Cream",
+      "type": "PRODUCT",
+      "attributes": {
+        "confidence": 1.0
+      }
+    },
+    {
+      "id": "omega_enriched_face_booster_oil",
+      "label": "Omega Enriched Face Booster Oil",
+      "type": "PRODUCT",
+      "attributes": {
+        "confidence": 1.0
+      }
+    }
+  ],
+  "edges": [
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Monday Poll",
+      "type": "hosts",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Monday Poll",
+      "target": "December 15, 2025",
+      "type": "occurs on",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Makeup and Beauty Blog Monday Poll, Vol. 893",
+      "type": "publishes",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Monday",
+      "type": "has",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "2007",
+      "type": "has",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Monday Poll",
+      "type": "hosts",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Makeup and Beauty Blog Monday Poll",
+      "type": "posts",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Vol. 892",
+      "type": "posts",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "2007",
+      "type": "has been active since",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "MBB",
+      "target": "Makeup and Beauty Blog",
+      "type": "related_to",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Makeup and Beauty Blog",
+      "type": "related_to",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Monday Poll",
+      "type": "hosts",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Vol. 891",
+      "type": "posts",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Makeup and Beauty Blog",
+      "target": "Monday Poll",
+      "type": "posts",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Cavallo Point",
+      "target": "Sausalito",
+      "type": "located_in",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Dr. Bailey",
+      "target": "Green Tea Antioxidant Skin Therapy",
+      "type": "prescribes",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Green Tea Antioxidant Skin Therapy",
+      "target": "Rosacea Therapy Skin Care Kit",
+      "type": "part of",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Dr. Bailey",
+      "target": "Rosacea Therapy Skin Care Kit",
+      "type": "uses",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Rosacea Therapy Skin Care Kit",
+      "target": "rosacea treatment routine",
+      "type": "part of",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Dr. Bailey",
+      "target": "rosacea treatment routine",
+      "type": "uses",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Facial dandruff",
+      "target": "rosacea",
+      "type": "often occurs with",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Facial dandruff",
+      "target": "rosacea",
+      "type": "needs to be addressed",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Calming Zinc Soap",
+      "target": "Facial dandruff",
+      "type": "is often sufficient to control",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Calming Zinc Soap",
+      "target": "rosacea",
+      "type": "is often sufficient to control",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Green Tea Antioxidant Skin Therapy",
+      "target": "Facial dandruff",
+      "type": "is often sufficient to control",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Green Tea Antioxidant Skin Therapy",
+      "target": "rosacea",
+      "type": "is often sufficient to control",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Dr. Bailey's Skincare",
+      "target": "Calming Zinc Soap",
+      "type": "produces",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Dr. Bailey's Skincare",
+      "target": "Green Tea Antioxidant Skin Therapy",
+      "type": "produces",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Dr. Bailey",
+      "target": "Calming Zinc Soap",
+      "type": "prescribes",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Dr. Bailey",
+      "target": "Green Tea Antioxidant Skin Therapy",
+      "type": "prescribes",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Hydrate",
+      "target": "Daily Moisturizing Face Cream",
+      "type": "is_achieved_by",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Daily Moisturizing Face Cream",
+      "target": "Omega Enriched Face Booster Oil",
+      "type": "can_be_combined_with",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Omega Enriched Face Booster Oil",
+      "target": "castor seed oil",
+      "type": "contains",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Omega Enriched Face Booster Oil",
+      "target": "sea buckthorn",
+      "type": "contains",
+      "attributes": {
+        "confidence": 0.9
+      }
+    },
+    {
+      "source": "Daily Moisturizing Face Cream",
+      "target": "Omega Enriched Face Booster Oil",
+      "type": "can_be_replaced_with",
+      "attributes": {
+        "confidence": 0.9
+      }
+    }
+  ],
+  "metadata": {
+    "num_entities": 20,
+    "num_relationships": 35,
+    "temporal_enabled": false,
+    "timestamp": "2025-12-24T12:46:41.535755",
+    "entity_resolution_applied": true
+  }
+}

cookbook/use_cases/advanced_rag/ground_truth.json

@@ -0,0 +1 @@
+{"entities": [{"id": "python_org", "name": "Python Software Foundation", "type": "Organization"}, {"id": "guido_van_rossum", "name": "Guido van Rossum", "type": "Person"}], "relationships": [{"source": "guido_van_rossum", "target": "python_org", "type": "FOUNDED"}]}

cookbook/use_cases/advanced_rag/skincare_base.json

@@ -0,0 +1,38 @@
+{
+  "entities": [
+    {
+      "id": "hyaluronic_acid",
+      "name": "Hyaluronic Acid",
+      "type": "Ingredient",
+      "properties": {
+        "role": "Humectant"
+      }
+    },
+    {
+      "id": "retinol",
+      "name": "Retinol",
+      "type": "Ingredient",
+      "properties": {
+        "role": "Anti-aging actives"
+      }
+    },
+    {
+      "id": "niacinamide",
+      "name": "Niacinamide",
+      "type": "Ingredient",
+      "properties": {
+        "role": "Barrier repair"
+      }
+    }
+  ],
+  "relationships": [
+    {
+      "source": "hyaluronic_acid",
+      "target": "niacinamide",
+      "type": "COMPLEMENTS",
+      "properties": {
+        "benefit": "Hydration + Barrier"
+      }
+    }
+  ]
+}

cookbook/use_cases/biomedical/data/chunks/chunk_0.txt

@@ -0,0 +1 @@
+Apoptotic signatures allow early and rapid screening of drug-induced liver injury to accelerate drug discovery

cookbook/use_cases/biomedical/data/chunks/chunk_1.txt

@@ -0,0 +1 @@
+SynergyGraph: predicting cell line specific drug combination synergy scores using knowledge graph representation and hypergraph modeling

cookbook/use_cases/biomedical/data/chunks/chunk_10.txt

@@ -0,0 +1 @@
+Unraveling the mechanism of curcumin in coronary slow flow phenomenon through network pharmacology and molecular docking

cookbook/use_cases/biomedical/data/chunks/chunk_11.txt

@@ -0,0 +1 @@
+Penicillium chrysogenum originated chloro-diydropyridyl-oxopropanimidic acid derivative as a potent EPSP synthase-targeted bioherbicide against invasive weed species

cookbook/use_cases/biomedical/data/chunks/chunk_12.txt

@@ -0,0 +1 @@
+Enhancing the anti-cancer potential of resveratrol through cocrystal technology in colorectal cancerous rats

cookbook/use_cases/biomedical/data/chunks/chunk_13.txt

@@ -0,0 +1 @@
+Synthesis, spectral, thermal, and biological characterization of Se(IV) nanocomplexes derived from vitamin E and amino acid mixed ligands as a metal-drug model

cookbook/use_cases/biomedical/data/chunks/chunk_14.txt

@@ -0,0 +1 @@
+BT-11 targets the LANCL2 pathway to attenuate cognitive deficits and hippocampal pathology in Alzheimer’s transgenic rats

cookbook/use_cases/biomedical/data/chunks/chunk_15.txt

@@ -0,0 +1 @@
+Saffron as a natural modulator of reverse cholesterol transport genes in atherosclerotic rabbits, with molecular docking insights

cookbook/use_cases/biomedical/data/chunks/chunk_16.txt

@@ -0,0 +1 @@
+Simultaneous targeting of KRAS and CDK4 synergistically induces durable growth arrest in pancreatic cancer cells

cookbook/use_cases/biomedical/data/chunks/chunk_17.txt

@@ -0,0 +1 @@
+Antiproliferative activity of Moringa oleifera (L.) Lam. and Moringa stenopetala (Bak.) Cufod. leaves extract against selected cancer cells in primary cell culture

cookbook/use_cases/biomedical/data/chunks/chunk_18.txt

@@ -0,0 +1 @@
+Applications of Sombor topological indices and entropy measures for QSPR modeling of anticancer drugs: a Python-based methodology

cookbook/use_cases/biomedical/data/chunks/chunk_19.txt

@@ -0,0 +1 @@
+In-silico studies, synthesis, and pharmacological screening of novel multitarget diphenylpyrazole scaffold as EGFR/BRAF and cyclooxygenase-2 inhibitors

cookbook/use_cases/biomedical/data/chunks/chunk_2.txt

@@ -0,0 +1 @@
+Rational design and in silico characterization of a multiepitope mRNA vaccine candidate against human metapneumovirus (hMPV) using reverse vaccinology and immunoinformatics approaches

cookbook/use_cases/biomedical/data/chunks/chunk_20.txt

@@ -0,0 +1 @@
+Unveiling novel potential drug targets for lung cancer through Mendelian randomization analysis

cookbook/use_cases/biomedical/data/chunks/chunk_21.txt

@@ -0,0 +1 @@
+Comparative analysis of OECD guideline data and Tox21 assays to improve reproductive and developmental toxicity prediction

cookbook/use_cases/biomedical/data/chunks/chunk_22.txt

@@ -0,0 +1 @@
+High-throughput triazole-based combinatorial click chemistry for the synthesis and identification of functional metal complexes

cookbook/use_cases/biomedical/data/chunks/chunk_23.txt

@@ -0,0 +1 @@
+Efficacy and safety assessment of gelatin hemostatic matrix in a burr hole neurosurgical beagle model

cookbook/use_cases/biomedical/data/chunks/chunk_24.txt

@@ -0,0 +1 @@
+Biological and computer-aided evaluation of 3-methoxy-13α-estrone-16α-diphenylphosphine oxide as a new antiestrogenic agent

cookbook/use_cases/biomedical/data/chunks/chunk_25.txt

@@ -0,0 +1 @@
+Design and optimization of intranasal aripiprazole-loaded nanostructured lipid carriers for enhanced brain targeting in schizophrenia: in vitro and ex vivo evaluation

cookbook/use_cases/biomedical/data/chunks/chunk_26.txt

@@ -0,0 +1 @@
+Endosome-phagophore linking assemblies for the degradation of membrane/extracellular proteins

cookbook/use_cases/biomedical/data/chunks/chunk_27.txt

@@ -0,0 +1 @@
+Protodioscin enhances Methotrexate-induced senescence and senolytic activity in HepG2 liver cancer spheroids by modulating cell cycle regulators and the TGF-β/p-Smad2-4 signaling pathway

cookbook/use_cases/biomedical/data/chunks/chunk_28.txt

@@ -0,0 +1 @@
+Morpholino-RNA duplex exhibits robust, sustained, and safe steric-block antisense activity by intracerebroventricular and intrathecal injection

cookbook/use_cases/biomedical/data/chunks/chunk_29.txt

@@ -0,0 +1 @@
+Pushing the limits of hydrogen/deuterium exchange mass spectrometry to study protein:fragment low affinity interactions

cookbook/use_cases/biomedical/data/chunks/chunk_3.txt

@@ -0,0 +1 @@
+Cannabidiol perturbs macrophage polarization by interfering with the metabolic flux and PI3K/Akt pathway

cookbook/use_cases/biomedical/data/chunks/chunk_4.txt

@@ -0,0 +1 @@
+Bidirectional reinforcement learning neural network for constrained molecular design

cookbook/use_cases/biomedical/data/chunks/chunk_5.txt

@@ -0,0 +1 @@
+Phytochemical investigation and antibacterial activities of Cordia africana roots extracts

cookbook/use_cases/biomedical/data/chunks/chunk_6.txt

@@ -0,0 +1 @@
+Similar short-term efficacy of oral levosulpiride and intravitreal ranibizumab in patients with diabetic macular oedema

cookbook/use_cases/biomedical/data/chunks/chunk_7.txt

@@ -0,0 +1 @@
+SIMD: Synergistic integration mutualistic platform based on single-cell and proteotranscriptomics for drug repositioning

cookbook/use_cases/biomedical/data/chunks/chunk_8.txt

@@ -0,0 +1 @@
+Elucidating the anti-fibrotic mechanisms of Abrus cantoniensis in hepatic fibrosis using network pharmacology and proteomics

cookbook/use_cases/biomedical/data/chunks/chunk_9.txt

@@ -0,0 +1 @@
+Identification of bioactive phytoconstituents as promising ABL2 inhibitors using virtual screening and molecular dynamics simulation

cookbook/use_cases/biomedical/data/sample_drugs.txt

@@ -0,0 +1,9 @@
+
+    Aspirin (acetylsalicylic acid) is a medication used to reduce pain, fever, or inflammation. 
+    It targets cyclooxygenase enzymes COX-1 and COX-2. Aspirin is commonly used for cardiovascular protection.
+    Ibuprofen is a nonsteroidal anti-inflammatory drug (NSAID) that targets COX-1 and COX-2 enzymes.
+    Metformin is an antidiabetic medication that targets AMP-activated protein kinase (AMPK).
+    Insulin targets the insulin receptor (INSR) to regulate glucose metabolism.
+    Warfarin is an anticoagulant that targets vitamin K epoxide reductase complex subunit 1 (VKORC1).
+    Atorvastatin is a statin medication that targets HMG-CoA reductase.
+    

cookbook/use_cases/biomedical/data/variants.txt

@@ -0,0 +1,11 @@
+
+Variant rs699 is located in the AGT gene and associated with hypertension.
+Variant rs7412 in APOE gene is linked to Alzheimer's disease risk.
+BRCA1 variant c.5266dupC increases breast cancer susceptibility.
+CFTR variant F508del causes cystic fibrosis.
+Variant rs1800566 in NAT2 gene affects drug metabolism.
+Variant rs1042713 in ADRB2 gene is associated with asthma response.
+TP53 variant R273H is linked to multiple cancer types.
+Variant rs1799853 in CYP2C9 gene affects warfarin metabolism.
+Variant rs1057910 in CYP2C9 affects phenytoin metabolism.
+Variant rs9923231 in VKORC1 gene influences warfarin dosing.

cookbook/use_cases/biomedical/drug_target_kg.graphml

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<graphml xmlns="http://graphml.graphdrawing.org/xmlns"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns
+         http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
+
+  <key id="type" for="node" attr.name="type" attr.type="string"/>
+  <key id="confidence" for="node" attr.name="confidence" attr.type="double"/>
+
+  <graph id="G" edgedefault="directed">
+
+
+  </graph>
+</graphml>

cookbook/use_cases/biomedical/genomic_variant_kg.graphml

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<graphml xmlns="http://graphml.graphdrawing.org/xmlns"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns
+         http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
+
+  <key id="type" for="node" attr.name="type" attr.type="string"/>
+  <key id="confidence" for="node" attr.name="confidence" attr.type="double"/>
+
+  <graph id="G" edgedefault="directed">
+
+
+  </graph>
+</graphml>

cookbook/use_cases/blockchain/data/transactions.txt

@@ -0,0 +1,9 @@
+
+    Transaction 0x123 transfers 1000 ETH from wallet 0xABC to wallet 0xDEF at block 18500000.
+    Transaction 0x456 transfers 500 BTC from wallet 0xGHI to wallet 0xJKL at block 18500001.
+    Large transaction 0x789 moves 10000 ETH (whale movement) from wallet 0xMNO to wallet 0xPQR at block 18500002.
+    Transaction 0xabc transfers 200 USDT from wallet 0xSTU to wallet 0xVWX at block 18500003.
+    Transaction 0xdef transfers 5000 ETH from wallet 0xYZA to wallet 0xBCD at block 18500004.
+    Transaction 0x111 transfers 3000 DAI from wallet 0xEFG to wallet 0xHIJ at block 18500005.
+    Transaction 0x222 transfers 1500 USDC from wallet 0xKLM to wallet 0xNOP at block 18500006.
+    

cookbook/use_cases/blockchain/defi_protocol_kg.graphml

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<graphml xmlns="http://graphml.graphdrawing.org/xmlns"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns
+         http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
+
+  <key id="type" for="node" attr.name="type" attr.type="string"/>
+  <key id="confidence" for="node" attr.name="confidence" attr.type="double"/>
+
+  <graph id="G" edgedefault="directed">
+
+
+  </graph>
+</graphml>

cookbook/use_cases/blockchain/transaction_network.dot

@@ -0,0 +1,5 @@
+digraph G {
+  rankdir=LR;
+
+
+}

cookbook/use_cases/blockchain/transaction_network.gexf

@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<gexf xmlns="http://www.gexf.net/1.2draft" version="1.2">
+  <graph mode="static" defaultedgetype="directed">
+    <nodes>
+    </nodes>
+    <edges>
+    </edges>
+  </graph>
+</gexf>

cookbook/use_cases/blockchain/transaction_network.graphml

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<graphml xmlns="http://graphml.graphdrawing.org/xmlns"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns
+         http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
+
+  <key id="type" for="node" attr.name="type" attr.type="string"/>
+  <key id="confidence" for="node" attr.name="confidence" attr.type="double"/>
+
+  <graph id="G" edgedefault="directed">
+
+
+  </graph>
+</graphml>

cookbook/use_cases/blockchain/transaction_network_relationships.csv

@@ -0,0 +1,657 @@
+confidence,id,source_id,target_id,type
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,shake
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,surge
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,drive
+0.8,,Chunk(text='Changpeng Zhao,$91.37 million,drive
+0.8,,1.90,more than 125 million,stand
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,transform
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,transform
+0.8,,Chunk(text='Changpeng Zhao,more than 125 million,receive
+0.8,,Chunk(text='Changpeng Zhao,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",receive
+0.8,,Chunk(text='Changpeng Zhao,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,receive
+0.8,,Christmas Eve,the Cubic Kilometre Neutrino,own
+0.8,,Chunk(text='Changpeng Zhao,$2.9 billion,conclude
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,conclude
+0.8,,Las Vegas Sphere,BlackRock,come
+0.8,,$1.5 billion,$24.62 million,follow
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,remain
+0.8,,Chunk(text='Changpeng Zhao,10px 15px,remain
+0.8,,Chunk(text='Changpeng Zhao,$91.37 million,remain
+0.8,,Chunk(text='Changpeng Zhao,Bitcoin,distribute
+0.8,,Chunk(text='Changpeng Zhao,10px 15px,use
+0.8,,34,NinjaTrader,be
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,dominate
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,maintain
+0.8,,Chunk(text='Changpeng Zhao,more than 125 million,maintain
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,operate
+0.8,,Chunk(text='Changpeng Zhao,$2.9 billion,expand
+0.8,,the Cubic Kilometre Neutrino,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",validate
+0.8,,more than 125 million,Magic Labs,signal
+0.8,,Chunk(text='Changpeng Zhao,2025,have
+0.8,,Chunk(text='Changpeng Zhao,Charles Hoskinson,have
+0.8,,Chunk(text='Changpeng Zhao,Binance,have
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,have
+0.8,,Chunk(text='Changpeng Zhao,$2.9 billion,include
+0.8,,Chunk(text='Changpeng Zhao,$91.37 million,include
+0.8,,Chunk(text='Changpeng Zhao,Las Vegas Sphere,follow
+0.8,,Chunk(text='Changpeng Zhao,the year,follow
+0.8,,the year,the Cubic Kilometre Neutrino,tie
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Magic Labs,manage
+0.8,,the year,more than 125 million,mark
+0.8,,2025,Christmas Eve,join
+0.8,,2025,the year,join
+0.8,,$91.37 million,more than 125 million,gain
+0.8,,$91.37 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,gain
+0.8,,Chunk(text='Changpeng Zhao,Mediterranean,attribute
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,attribute
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,suppress
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Chain Fermi,happen
+0.8,,$219M,start_char=31,happen
+0.8,,confidence=1.0,five year',impact
+0.8,,confidence=1.0,Changpeng Zhao',impact
+0.8,,confidence=1.0,Entity(text='next year',impact
+0.8,,confidence=1.0,Entity(text='next year',impact
+0.8,,confidence=1.0,DeFi,impact
+0.8,,confidence=1.0,Entity(text='more than 125 million,impact
+0.8,,confidence=1.0,this week,suggest
+0.8,,confidence=1.0,Changpeng Zhao',suggest
+0.8,,Entity(text='next year',Entity(text='more than 125 million,wait
+0.8,,Entity(text='more than 125 million,Entity(text='$91.37 million,reclaim
+0.8,,Chunk(text='Changpeng Zhao,Chain Fermi,freeze
+0.8,,Chunk(text='Changpeng Zhao,Entity(text='U.S.,freeze
+0.8,,8230;]</p>\n,Entity(text='next year',link
+0.8,,Chunk(text='Changpeng Zhao,next year,trade
+0.8,,confidence=1.0,"href=""https://cryptoslate.com",provide
+0.8,,Chunk(text='Changpeng Zhao,Chain Fermi,see
+0.8,,Chunk(text='Changpeng Zhao,next year,see
+0.8,,Chunk(text='Changpeng Zhao,January,see
+0.8,,Chunk(text='Changpeng Zhao,confidence=1.0,reach
+0.8,,Chunk(text='Changpeng Zhao,confidence=1.0,reach
+0.8,,confidence=1.0,Entity(text='next year',describe
+0.8,,confidence=1.0,five-day,describe
+0.8,,156,12/26,skyrocket
+0.8,,156,Entity(text='U.S.,skyrocket
+0.8,,confidence=1.0,Entity(text='next year',drive
+0.8,,confidence=1.0,Entity(text='more than 125 million,drive
+0.8,,confidence=1.0,Entity(text='more than 125 million,drive
+0.8,,confidence=1.0,end_char=31,drive
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Chain Fermi,slash
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",second year',slash
+0.8,,Chunk(text='Changpeng Zhao,Focus as Ondo Tweet Signals Wall Street Shift</a>,offer
+0.8,,confidence=1.0,Changpeng Zhao',end
+0.8,,Swiss,Entity(text='U.S.,take
+0.8,,Entity(text='U.S.,start_char=31,outline
+0.8,,"href=""https://cryptoslate.com",Entity(text='next year',support
+0.8,,"href=""https://cryptoslate.com",Entity(text='more than 125 million,support
+0.8,,confidence=1.0,confidence=1.0,follow
+0.8,,confidence=1.0,Focus as Ondo Tweet Signals Wall Street Shift</a>,follow
+0.8,,confidence=1.0,"href=""https://cryptoslate.com",follow
+0.8,,January,confidence=1.0,change
+0.8,,Chunk(text='Changpeng Zhao,"href=""https://cryptoslate.com",be
+0.8,,Chunk(text='Changpeng Zhao,five year',be
+0.8,,Entity(text='more than 125 million,end_char=31,put
+0.8,,Entity(text='more than 125 million,confidence=1.0,put
+0.8,,Chunk(text='Changpeng Zhao,158,"alt=""Bitcoin"
+0.8,,Chain Fermi,Entity(text='U.S.,face
+0.8,,confidence=1.0,confidence=1.0,experience
+0.8,,five year',Washington,extend
+0.8,,five year',"href=""https://cryptoslate.com",extend
+0.8,,second year',confidence=1.0,restore
+0.8,,Chunk(text='Changpeng Zhao,$80K,slip
+0.8,,86 million,Entity(text='more than 125 million,inject
+0.8,,86 million,Entity(text='$91.37 million,inject
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Entity(text='next year',push
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Entity(text='more than 125 million,push
+0.8,,Chain Fermi,Entity(text='U.S.,schedule
+0.8,,end_char=34,confidence=1.0,follow
+0.8,,end_char=34,8230;]</p>\n,follow
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Chain Fermi,"alt=""bitcoin"
+0.8,,Chunk(text='Changpeng Zhao,Entity(text='U.S.,look
+0.8,,confidence=1.0,confidence=1.0,point
+0.8,,confidence=1.0,confidence=1.0,point
+0.8,,end_char=34,five-day,take
+0.8,,Entity(text='U.S.,Ethereum’s Fusaka,extend
+0.8,,Chunk(text='Changpeng Zhao,Lugano,increase
+0.8,,Chunk(text='Changpeng Zhao,Entity(text='more than 125 million,increase
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Wallet,be
+0.8,,Entity(text='next year',Entity(text='more than 125 million,scale
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",next year,go
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Entity(text='U.S.,go
+0.8,,confidence=1.0,Changpeng Zhao',close
+0.8,,confidence=1.0,"href=""https://cryptoslate.com",close
+0.8,,confidence=1.0,Entity(text='more than 125 million,close
+0.8,,Chain Fermi,Entity(text='U.S.,reshape
+0.8,,Chain Fermi,second year',reshape
+0.8,,"href=""https://cryptoslate.com",start_char=31,reshape
+0.8,,"href=""https://cryptoslate.com",confidence=1.0,reshape
+0.8,,Entity(text='U.S.,next year,’s
+0.8,,8230;]</p>\n,Entity(text='next year',spill
+0.8,,8230;]</p>\n,the year,spill
+0.8,,Entity(text='more than 125 million,confidence=1.0,drop
+0.8,,next year,Bitcoin Mining,pay
+0.8,,next year,"DAO vote""></p><p>Stani Kulechov\'s",pay
+0.8,,next year,confidence=1.0,pay
+0.8,,Chunk(text='Changpeng Zhao,Chain Fermi,cover
+0.8,,Chain Fermi,Entity(text='U.S.,lose
+0.8,,confidence=1.0,confidence=1.0,export
+0.8,,confidence=1.0,"href=""https://cryptoslate.com",export
+0.8,,Entity(text='U.S.,end_char=34,hit
+0.8,,end_char=34,confidence=1.0,simmer
+0.8,,Changpeng Zhao',Entity(text='more than 125 million,accompany
+0.8,,Changpeng Zhao',Entity(text='more than 125 million,accompany
+0.8,,Changpeng Zhao',Kong Bananza',accompany
+0.8,,Chain Fermi,"letter/"">Samourai Wallet Co-Founder Describes",end
+0.8,,Chain Fermi,December 26,end
+0.8,,end_char=34,The second half of 2026,highlight
+0.8,,end_char=34,8230;]</p>\n,highlight
+0.8,,end_char=34,five year',highlight
+0.8,,end_char=34,Entity(text='more than 125 million,highlight
+0.8,,$314 billion,Entity(text='$91.37 million,manage
+0.8,,confidence=1.0,"DAO vote""></p><p>Stani Kulechov\'s",bury
+0.8,,confidence=1.0,Lugano,have
+0.8,,Lugano,Samson Mow,contain
+0.8,,Chunk(text='Changpeng Zhao,Washington,offer
+0.8,,Chunk(text='Changpeng Zhao,Entity(text='more than 125 million,offer
+0.8,,Chunk(text='Changpeng Zhao,Entity(text='$91.37 million,offer
+0.8,,Chunk(text='Changpeng Zhao,confidence=1.0,offer
+0.8,,end_char=34,confidence=1.0,hover
+0.8,,end_char=34,Entity(text='more than 125 million,hover
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,rush
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,rush
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,rush
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,rush
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,rush
+0.8,,the Cubic Kilometre Neutrino,more than 125 million,go
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,line
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,evolve
+0.8,,the Cubic Kilometre Neutrino,more than 125 million,feel
+0.8,,U.S.,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",buck
+0.8,,U.S.,more than 125 million,buck
+0.8,,Mediterranean,Chunk(text='Changpeng Zhao,build
+0.8,,Mediterranean,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",build
+0.8,,Mediterranean,next year,build
+0.8,,Mediterranean,Magic Labs,build
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Clear Street's,listen
+0.8,,Chunk(text='Changpeng Zhao,a second year,highlight
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,hit
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,hit
+0.8,,the year,more than 125 million,tell
+0.8,,Las Vegas Sphere,Binance,weaponize
+0.8,,Las Vegas Sphere,$91.37 million,weaponize
+0.8,,Las Vegas Sphere,more than 125 million,weaponize
+0.8,,Chunk(text='Changpeng Zhao,Mediterranean,find
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,be
+0.8,,1.85-$1.91,multi-billion dollar,catch
+0.8,,1.85-$1.91,$2.9 billion,catch
+0.8,,1.85-$1.91,next year,catch
+0.8,,next year,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,process
+0.8,,$1.5 billion,$91.37 million,be
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,adorn
+0.8,,$91.37 million,more than 125 million,shut
+0.8,,$91.37 million,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",shut
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,open
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,have
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,have
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,bury
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,bury
+0.8,,CoinDesk,more than 125 million,pull
+0.8,,CoinDesk,$1.5 billion,pull
+0.8,,next year,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,deserve
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,focus
+0.8,,Las Vegas Sphere,$314 billion,become
+0.8,,CoinDesk,$2.9 billion,reveal
+0.8,,Christmas Eve,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",slow
+0.8,,Christmas Eve,next year,slow
+0.8,,Chunk(text='Changpeng Zhao,Las Vegas Sphere,be
+0.8,,Chunk(text='Changpeng Zhao,$952 million,be
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,be
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,be
+0.8,,Chunk(text='Changpeng Zhao,$91.37 million,be
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,leak
+0.8,,the year,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",move
+0.8,,the year,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",move
+0.8,,the year,Magic Labs,move
+0.8,,Magic Eden,Christmas Eve,scrap
+0.8,,1.85-$1.91,Circle,impress
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,Chunk(text='Changpeng Zhao,Bitcoin,hit
+0.8,,$91.37 million,more than 125 million,reveal
+0.8,,$91.37 million,next year,reveal
+0.8,,1.85-$1.91,Chunk(text='Changpeng Zhao,reach
+0.8,,1.85-$1.91,BlackRock,reach
+0.8,,1.85-$1.91,$2.9 billion,reach
+0.8,,1.85-$1.91,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",reach
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",more than 125 million,flare
+0.8,,Chunk(text='Changpeng Zhao,Mediterranean,see
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,see
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",more than 125 million,spook
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,lead
+0.8,,the Cubic Kilometre Neutrino,Chunk(text='Changpeng Zhao,close
+0.8,,the Cubic Kilometre Neutrino,more than 125 million,close
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Magic Labs,be
+0.8,,$314 billion,CoinDesk,range
+0.8,,$91.37 million,more than 125 million,jump
+0.8,,$91.37 million,next year,jump
+0.8,,Chunk(text='Changpeng Zhao,Mediterranean,gobble
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,gobble
+0.8,,$2.9 billion,more than 125 million,be
+0.8,,CoinDesk,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",take
+0.8,,CoinDesk,more than 125 million,take
+0.8,,$1.5 billion,Magic Labs,split
+0.8,,$1.5 billion,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,split
+0.8,,start_index=0,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,reimburse
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,talk
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,talk
+0.8,,Magic Labs,Clear Street's,destroy
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Ethereum,embrace
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Entity(text='U.S.,embrace
+0.8,,confidence=1.0,start_char=31,’
+0.8,,confidence=1.0,end_char=34,’
+0.8,,Magic Eden,Chunk(text='Changpeng Zhao,back
+0.8,,Magic Eden,CoinDesk,back
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,the Cubic Kilometre Neutrino,more than 125 million,trade
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,$24.62 million,enable
+0.8,,Changpeng Zhao',Entity(text='next year',swap
+0.8,,Changpeng Zhao',Entity(text='more than 125 million,swap
+0.8,,start_char=54,confidence=1.0,onchain
+0.8,,start_char=54,Entity(text='ETH,onchain
+0.8,,start_char=54,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",onchain
+0.8,,$2.9 billion,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",offer
+0.8,,$2.9 billion,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",offer
+0.8,,$2.9 billion,confidence=1.0,offer
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,apis
+0.8,,Chunk(text='Changpeng Zhao,more than 125 million,apis
+0.8,,Christmas Eve,$91.37 million,need
+0.8,,$88K,Entity(text='U.S.,answer
+0.8,,$88K,start_char=31,answer
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,comment
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,ruin
+0.8,,Hidden Road,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,Hidden Road,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,Chunk(text='Changpeng Zhao,more than 125 million,be
+0.8,,Chunk(text='Changpeng Zhao,more than 125 million,be
+0.8,,next year,Clear Street's,lead
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Hidden Road,announce
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,confidence=1.0,announce
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Chain Fermi,announce
+0.8,,"Mow""></p><p>Bitcoin",confidence=1.0,propose
+0.8,,confidence=1.0,confidence=1.0,issue
+0.8,,confidence=1.0,confidence=1.0,issue
+0.8,,more than 125 million,$91.37 million,leave
+0.8,,more than 125 million,Entity(text='Owen Lau',leave
+0.8,,Entity(text='8-month',end_char=302,announce
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,end_char=93,build
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,2025,build
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,be
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,be
+0.8,,NinjaTrader,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,lead
+0.8,,$80K,"href=""https://cryptoslate.com",end
+0.8,,$80K,Entity(text='next year',end
+0.8,,confidence=1.0,end_char=148,expand
+0.8,,confidence=1.0,confidence=1.0,expand
+0.8,,Bitcoin,'1.85-$1.91',approve
+0.8,,'1.85-$1.91',$91.37 million,implement
+0.8,,confidence=1.0,start_char=37,issue
+0.8,,confidence=1.0,confidence=1.0,issue
+0.8,,confidence=1.0,'10px 15px',be
+0.8,,confidence=1.0,confidence=1.0,be
+0.8,,confidence=1.0,Circle,be
+0.8,,confidence=1.0,Entity(text='next year',be
+0.8,,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",confidence=1.0,raise
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,repay
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,confidence=1.0,relate
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVhMjEtZTllMi03YzY4LTg2MTQtODkyMzg4NWE1NjE3LmpwZw==.jpg'",leave
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVhMjEtZTllMi03YzY4LTg2MTQtODkyMzg4NWE1NjE3LmpwZw==.jpg'",leave
+0.8,,south korean,$219 million,approve
+0.8,,south korean,Entity(text='early 2026',approve
+0.8,,south korean,start_char=61,approve
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,be
+0.8,,Magic Labs,Owen Lau,lead
+0.8,,Entity(text='U.S.,end_char=34,punt
+0.8,,Entity(text='U.S.,confidence=1.0,punt
+0.8,,Changpeng Zhao',Entity(text='more than 125 million,join
+0.8,,Merrill Lynch,Charles Hoskinson',expand
+0.8,,confidence=1.0,confidence=1.0,apply
+0.8,,confidence=1.0,China,apply
+0.8,,Entity(text='8-month',Entity(text='JPMorgan',be
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,need
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,be
+0.8,,Magic Labs,Owen Lau,lead
+0.8,,Entity(text='U.S.,end_char=34,punt
+0.8,,Entity(text='U.S.,confidence=1.0,punt
+0.8,,Changpeng Zhao',Entity(text='more than 125 million,join
+0.8,,Merrill Lynch,Charles Hoskinson',expand
+0.8,,confidence=1.0,confidence=1.0,apply
+0.8,,confidence=1.0,China,apply
+0.8,,Entity(text='8-month',Entity(text='JPMorgan',be
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,need
+0.8,,Christmas Eve,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",hist
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,be
+0.8,,Chunk(text='Changpeng Zhao,Christmas Eve,be
+0.8,,Magic Labs,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,lead
+0.8,,start_index=0,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,issue
+0.8,,Entity(text='U.S.,confidence=1.0,launch
+0.8,,Entity(text='U.S.,confidence=1.0,launch
+0.8,,confidence=1.0,confidence=1.0,issue
+0.8,,confidence=1.0,Entity(text='next year',explain
+0.8,,confidence=1.0,Entity(text='more than 125 million,explain
+0.8,,'year',more than 125 million,value
+0.8,,'year',Clear Street's,value
+0.8,,confidence=1.0,confidence=1.0,open
+0.8,,confidence=1.0,end_char=62,open
+0.8,,confidence=1.0,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVhMjEtZTllMi03YzY4LTg2MTQtODkyMzg4NWE1NjE3LmpwZw==.jpg'",propose
+0.8,,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",start_char=78,enable
+0.8,,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",Entity(text='JPMorgan',greenlit
+0.8,,Entity(text='JPMorgan',end_char=302,title
+0.8,,end_char=302,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,star
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,delve
+0.8,,Chunk(text='Changpeng Zhao,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",delve
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,be
+0.8,,Clear Street's,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,lead
+0.8,,Cardano,Christmas Day,accumulate
+0.8,,start_char=31,end_char=34,buy
+0.8,,confidence=1.0,confidence=1.0,decline
+0.8,,confidence=1.0,October,decline
+0.8,,Entity(text='more than 125 million,confidence=1.0,weaken
+0.8,,Entity(text='more than 125 million,start_char=48,weaken
+0.8,,start_char=48,confidence=1.0,examine
+0.8,,start_char=48,Charles Hoskinson',examine
+0.8,,end_char=36,$91.37 million,link
+0.8,,$91.37 million,Charles Hoskinson',secure
+0.8,,confidence=1.0,confidence=1.0,approve
+0.8,,confidence=1.0,Entity(text='the previous year',approve
+0.8,,confidence=1.0,confidence=1.0,approve
+0.8,,$24.62 million,'10px 15px',receive
+0.8,,$24.62 million,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",receive
+0.8,,$24.62 million,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",receive
+0.8,,$24.62 million,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",receive
+0.8,,end_char=302,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,see
+0.8,,confidence=1.0,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,stem
+0.8,,Christmas Eve,the Cubic Kilometre Neutrino,buy
+0.8,,$2.9 billion,Magic Labs,approve
+0.8,,Chunk(text='Changpeng Zhao,Circle,trade
+0.8,,Chunk(text='Changpeng Zhao,Circle,trade
+0.8,,Chunk(text='Changpeng Zhao,CoinDesk,trade
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",more than 125 million,rise
+0.8,,more than 125 million,Magic Labs,jump
+0.8,,"87,498.12",start_char=31,resume
+0.8,,"87,498.12",The second half of 2026,resume
+0.8,,Nyan Heroes',Entity(text='next year',note
+0.8,,Entity(text='more than 125 million,confidence=1.0,partner
+0.8,,Mediterranean,'next year',release
+0.8,,Mediterranean,Circle,release
+0.8,,'10px 15px',confidence=1.0,introduce
+0.8,,'10px 15px',confidence=1.0,introduce
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,be
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,confidence=1.0,gain
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,158,gain
+0.8,,Entity(text='more than 125 million,start_char=29,recommend
+0.8,,Entity(text='more than 125 million,start_char=48,recommend
+0.8,,confidence=1.0,end_char=36,agree
+0.8,,confidence=1.0,'10px 15px',introduce
+0.8,,Entity(text='ETH,Owen Lau',feature
+0.8,,Entity(text='ETH,$24.62 million,feature
+0.8,,confidence=1.0,end_char=93,appoint
+0.8,,confidence=1.0,end_char=93,appoint
+0.8,,confidence=1.0,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,appoint
+0.8,,confidence=1.0,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjQ1ZGYtMTQzMi03ODE4LWE4YzMtMTEwZDlkYTcxMmM4LmpwZw==.jpg,appoint
+0.8,,2025,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,create
+0.8,,2025,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,create
+0.8,,2025,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,create
+0.8,,2025,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,create
+0.8,,more than 125 million,$1.5 billion,jump
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",$24.62 million,bullish
+0.8,,Clear Street's,Owen Lau,raise
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,more than 125 million,NinjaTrader,be
+0.8,,Chunk(text='Changpeng Zhao,Mediterranean,be
+0.8,,Chunk(text='Changpeng Zhao,$2.9 billion,be
+0.8,,Chunk(text='Changpeng Zhao,1.86,be
+0.8,,Chunk(text='Changpeng Zhao,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,"href=""https://cryptoslate.com",end_char=31,sue
+0.8,,Entity(text='$91.37 million,start_char=48,secure
+0.8,,Entity(text='$91.37 million,end_char=148,secure
+0.8,,'10px 15px',Entity(text='ETH,work
+0.8,,Entity(text='next year',$24.62 million,release
+0.8,,confidence=1.0,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",discover
+0.8,,confidence=1.0,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",discover
+0.8,,confidence=1.0,"Entity(text='src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg'",discover
+0.8,,Entity(text='8-month',confidence=1.0,issue
+0.8,,Entity(text='8-month',confidence=1.0,issue
+0.8,,end_char=93,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,pressure
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,see
+0.8,,Chunk(text='Changpeng Zhao,$952 million,see
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,position
+0.8,,Chunk(text='Changpeng Zhao,CoinDesk,position
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,enter
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,enter
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,achieve
+0.8,,more than 125 million,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",scale
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,locate
+0.8,,Las Vegas Sphere,BlackRock,agree
+0.8,,HPC,the Cubic Kilometre Neutrino,identify
+0.8,,the Cubic Kilometre Neutrino,more than 125 million,affect
+0.8,,the Cubic Kilometre Neutrino,Magic Labs,affect
+0.8,,Las Vegas Sphere,the Cubic Kilometre Neutrino,focus
+0.8,,Las Vegas Sphere,CoinDesk,focus
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,coincide
+0.8,,CoinDesk,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",provide
+0.8,,CoinDesk,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,provide
+0.8,,Las Vegas Sphere,BlackRock,go
+0.8,,CoinDesk,$91.37 million,be
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,take
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,be
+0.8,,Chunk(text='Changpeng Zhao,multi-billion dollar,be
+0.8,,Magic Labs,NinjaTrader,get
+0.8,,$91.37 million,Chunk(text='Changpeng Zhao,split
+0.8,,$91.37 million,Magic Labs,split
+0.8,,$91.37 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,split
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,$219M,deepen
+0.8,,confidence=1.0,confidence=1.0,cement
+0.8,,confidence=1.0,this week,cement
+0.8,,$2.9 billion,$1.5 billion,drop
+0.8,,Chunk(text='Changpeng Zhao,$314 billion,post
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,post
+0.8,,Chunk(text='Changpeng Zhao,$2.9 billion,post
+0.8,,Chunk(text='Changpeng Zhao,86 million,name
+0.8,,$2.9 billion,more than 125 million,speculate
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",NinjaTrader,link
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,charge
+0.8,,Chunk(text='Changpeng Zhao,$91.37 million,charge
+0.8,,Magic Eden,the Cubic Kilometre Neutrino,ship
+0.8,,Magic Eden,$2.9 billion,ship
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Hidden Road,push
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,confidence=1.0,be
+0.8,,end_char=34,confidence=1.0,follow
+0.8,,end_char=34,Magic Eden',expose
+0.8,,end_char=34,more than 125 million,expose
+0.8,,Chunk(text='Changpeng Zhao,Carlos Domingo,turn
+0.8,,Chunk(text='Changpeng Zhao,$2.9 billion,turn
+0.8,,Chunk(text='Changpeng Zhao,the end of a years-long,turn
+0.8,,confidence=1.0,Cardano,say
+0.8,,confidence=1.0,Entity(text='next year',say
+0.8,,confidence=1.0,$150 billion,say
+0.8,,confidence=1.0,confidence=1.0,change
+0.8,,confidence=1.0,confidence=1.0,change
+0.8,,confidence=1.0,"as low as $65,000.</p",be
+0.8,,confidence=1.0,confidence=1.0,be
+0.8,,Charles Hoskinson',confidence=1.0,be
+0.8,,more than 125 million,$91.37 million,absorb
+0.8,,more than 125 million,Entity(text='Owen Lau',absorb
+0.8,,BlackRock,Chunk(text='Changpeng Zhao,feel
+0.8,,BlackRock,Earlier this month,feel
+0.8,,BlackRock,Christmas Eve,feel
+0.8,,BlackRock,$91.37 million,feel
+0.8,,BlackRock,more than 125 million,feel
+0.8,,BlackRock,next year,feel
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,confidence=1.0,watch
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Chain Fermi,include
+0.8,,next year,December 26,be
+0.8,,next year,October,be
+0.8,,end_char=31,confidence=1.0,live
+0.8,,end_char=31,end_char=34,live
+0.8,,confidence=1.0,confidence=1.0,be
+0.8,,"as low as $65,000.</p",end_char=148,have
+0.8,,"as low as $65,000.</p",Mediterranean,have
+0.8,,Chunk(text='Changpeng Zhao,the end of a years-long,reach
+0.8,,Chunk(text='Changpeng Zhao,more than 125 million,reach
+0.8,,NinjaTrader,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,look
+0.8,,NinjaTrader,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,look
+0.8,,confidence=1.0,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,become
+0.8,,confidence=1.0,Chain Fermi,become
+0.8,,Entity(text='U.S.,Christmas,capture
+0.8,,Entity(text='U.S.,Ethereum’s Fusaka,capture
+0.8,,confidence=1.0,BlackRock,liquidate
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,acknowledge
+0.8,,Brian Armstrong,Aave,place
+0.8,,Brian Armstrong,Aave,place
+0.8,,Brian Armstrong,confidence=1.0,place
+0.8,,more than 125 million,$91.37 million,issue
+0.8,,more than 125 million,Entity(text='Owen Lau',issue
+0.8,,2025,1.85-$1.91,make
+0.8,,2025,Binance,make
+0.8,,Cardano,next year,reveal
+0.8,,Ethereum,Entity(text='U.S.,carry
+0.8,,Ethereum,$219M,carry
+0.8,,Ethereum,end_char=31,carry
+0.8,,Ethereum,start_char=31,carry
+0.8,,Ethereum,start_char=31,carry
+0.8,,Ethereum,start_char=31,carry
+0.8,,Ethereum,confidence=1.0,carry
+0.8,,CME,$1.5 billion,begin
+0.8,,CME,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,begin
+0.8,,the Cubic Kilometre Neutrino,more than 125 million,carry
+0.8,,$1.5 billion,NinjaTrader,report
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,start_index=0,say
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Chain Fermi,pursue
+0.8,,next year,start_char=31,respond
+0.8,,start_char=31,end_char=34,impose
+0.8,,start_char=31,confidence=1.0,impose
+0.8,,confidence=1.0,Entity(text='$91.37 million,hit
+0.8,,confidence=1.0,end_index=447,hit
+0.8,,Chunk(text='Changpeng Zhao,Bitcoin,end
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,end
+0.8,,Chunk(text='Changpeng Zhao,more than 125 million,end
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,gain
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,gain
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Clear Street's,announce
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,confidence=1.0,announce
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,confidence=1.0,announce
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,146,announce
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,"Entity(text='90,353",announce
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,confidence=1.0,announce
+0.8,,confidence=1.0,Entity(text='next year',[
+0.8,,Arthur Hayes',Clear Street's,replace
+0.8,,Chunk(text='Changpeng Zhao,Las Vegas Sphere,leave
+0.8,,Chunk(text='Changpeng Zhao,$952 million,leave
+0.8,,Clear Street's,start_index=0,make
+0.8,,Clear Street's,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,make
+0.8,,confidence=1.0,"Entity(text='90,353",trade
+0.8,,confidence=1.0,confidence=1.0,trade
+0.8,,confidence=1.0,confidence=1.0,hit
+0.8,,Chunk(text='Changpeng Zhao,Charles Hoskinson,open
+0.8,,Charles Hoskinson,the Cubic Kilometre Neutrino,launch
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,NinjaTrader,prove
+0.8,,Bitcoin,Changpeng Zhao',rugpulle
+0.8,,confidence=1.0,Charles Hoskinson',define
+0.8,,confidence=1.0,'next year',define
+0.8,,confidence=1.0,'1.85-$1.91',define
+0.8,,confidence=1.0,more than 125 million,define
+0.8,,CoinDesk,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",suggest
+0.8,,CoinDesk,Clear Street's,suggest
+0.8,,CoinDesk,Owen Lau,suggest
+0.8,,confidence=1.0,confidence=1.0,be
+0.8,,Washington,$91.37 million,highlight
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,unlock
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,unlock
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Clear Street's,unlock
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Clear Street's,unlock
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Clear Street's,unlock
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,unlock
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",Caroline Ellison,unlock
+0.8,,end_char=44,confidence=1.0,be
+0.8,,1.85-$1.91,$2.9 billion,mark
+0.8,,confidence=1.0,end_char=31,appear
+0.8,,Chunk(text='Changpeng Zhao,BlackRock,rise
+0.8,,Chunk(text='Changpeng Zhao,$24.62 million,rise
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,gain
+0.8,,more than 125 million,742,gain
+0.8,,TVL,Entity(text='U.S.,surge
+0.8,,TVL,Entity(text='U.S.,surge
+0.8,,TVL,end_char=34,surge
+0.8,,Chunk(text='Changpeng Zhao,Las Vegas Sphere,slash
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,slash
+0.8,,Chunk(text='Changpeng Zhao,more than 125 million,applaud
+0.8,,next year,end_char=31,show
+0.8,,confidence=1.0,end_char=34,highlight
+0.8,,confidence=1.0,five year',highlight
+0.8,,confidence=1.0,Changpeng Zhao',highlight
+0.8,,confidence=1.0,Changpeng Zhao',highlight
+0.8,,confidence=1.0,Changpeng Zhao',highlight
+0.8,,confidence=1.0,Entity(text='next year',highlight
+0.8,,confidence=1.0,Entity(text='next year',highlight
+0.8,,490,more than 125 million,mock
+0.8,,next year,Clear Street's,reveal
+0.8,,Clear Street's,start_index=0,expand
+0.8,,Clear Street's,Hidden Road,expand
+0.8,,confidence=1.0,"as low as $65,000.</p",meet
+0.8,,confidence=1.0,Mediterranean,win
+0.8,,2025,more than 125 million,come
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,more than 125 million,Clear Street's,be
+0.8,,$1.25 billion,next year,team
+0.8,,confidence=1.0,"Entity(text='90,353",connect
+0.8,,Solana,start_char=31,redirect
+0.8,,Solana,"Mow""></p><p>Bitcoin",redirect
+0.8,,confidence=1.0,confidence=1.0,[
+0.8,,confidence=1.0,Lugano,[
+0.8,,Christmas Eve,the Cubic Kilometre Neutrino,take
+0.8,,Christmas Eve,more than 125 million,take
+0.8,,$1.5 billion,$24.62 million,emerge
+0.8,,100,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,be
+0.8,,Chain Fermi,confidence=1.0,sit
+0.8,,1.85-$1.91,BlackRock,give
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,teach
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,$24.62 million,ask
+0.8,,124,"Entity(text='90,353",deliver
+0.8,,"Benjamin Cowen""></p><p>",confidence=1.0,outline
+0.8,,the Cubic Kilometre Neutrino,Magic Eden,capture
+0.8,,the Cubic Kilometre Neutrino,more than 125 million,capture
+0.8,,the Cubic Kilometre Neutrino,$1.5 billion,capture
+0.8,,more than 125 million,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,show
+0.8,,Chunk(text='Changpeng Zhao,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,appear
+0.8,,Chunk(text='Changpeng Zhao,4,appear
+0.8,,Chunk(text='Changpeng Zhao,Chain Fermi,appear
+0.8,,Chunk(text='Changpeng Zhao,next year,appear
+0.8,,Las Vegas Sphere,multi-billion dollar,record
+0.8,,Las Vegas Sphere,CoinDesk,record
+0.8,,Las Vegas Sphere,4,record
+0.8,,Las Vegas Sphere,100,record
+0.8,,Las Vegas Sphere,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,record
+0.8,,Las Vegas Sphere,"Benjamin Cowen""></p><p>",record
+0.8,,confidence=1.0,confidence=1.0,be
+0.8,,confidence=1.0,confidence=1.0,be
+0.8,,Chunk(text='Changpeng Zhao,Magic Eden,hit
+0.8,,Chunk(text='Changpeng Zhao,1.85-$1.91,hit
+0.8,,next year,more than 125 million,be
+0.8,,next year,Magic Labs,be
+0.8,,Las Vegas Sphere,the Cubic Kilometre Neutrino,release
+0.8,,Las Vegas Sphere,Entity(text='next year',release
+0.8,,"src=""https://images.cointelegraph.com/images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVkMWItMWY4OC03MzZkLWI3MTQtMTExNmM4MzczYmM4LmpwZw==.jpg",$1.25 billion,treat
+0.8,,the Cubic Kilometre Neutrino,more than 125 million,rise
+0.8,,the year,the Cubic Kilometre Neutrino,set
+0.8,,the year,start_char=31,set
+0.8,,the year,end_char=34,set
+0.8,,Las Vegas Sphere,$314 billion,signal
+0.8,,Las Vegas Sphere,the Cubic Kilometre Neutrino,signal
+0.8,,$1.5 billion,$24.62 million,confirm
+0.8,,$1.5 billion,$1.25 billion,confirm
+0.8,,$1.5 billion,3,confirm
+0.8,,images/528_aHR0cHM6Ly9zMy5jb2ludGVsZWdyYXBoLmNvbS91cGxvYWRzLzIwMjUtMTIvMDE5YjVmYzEtNDRiYi03N2UxLWIzMzItOWQzNWY5OWM0NmEzLmpwZw==.jpg,Entity(text='nearly 20%,govern
+0.8,,HPC,Vanguard ETFs,be
+0.8,,HPC,CoinDesk,be
+0.8,,Chunk(text='Changpeng Zhao,the Cubic Kilometre Neutrino,spend
+0.8,,Chunk(text='Changpeng Zhao,742,spend
+0.8,,Chunk(text='Changpeng Zhao,start_index=0,spend

cookbook/use_cases/cybersecurity/01_Anomaly_Detection_Real_Time.ipynb

@@ -1,494 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/cybersecurity/01_Anomaly_Detection_Real_Time.ipynb)\n",
-    "\n",
-    "# Real-Time Anomaly Detection Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete real-time anomaly detection pipeline for cybersecurity: stream security logs from multiple sources, parse in real-time, build temporal knowledge graph, detect anomalies using pattern detection and inference, generate alerts, and monitor continuously.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, EmailIngestor, RepoIngestor, MCPIngestor\n",
-    "- **Parsing**: JSONParser, StructuredDataParser, DocumentParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "- **KG**: GraphBuilder, TemporalPatternDetector, TemporalGraphQuery, GraphAnalyzer\n",
-    "- **Analytics**: CentralityCalculator, CommunityDetector, ConnectivityAnalyzer\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-
-    "- **Export**: JSONExporter, CSVExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Stream Security Logs \u2192 Real-Time Parsing \u2192 Extract Entities \u2192 Build Temporal KG \u2192 Pattern Detection \u2192 Anomaly Detection \u2192 Generate Alerts \u2192 Monitor \u2192 Visualize**\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Stream Security Logs from Multiple Sources\n",
-    "\n",
-    "Stream security logs from files, databases, and real-time sources.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import StreamIngestor, FileIngestor, DBIngestor, FeedIngestor\n",
-    "from semantica.parse import JSONParser, StructuredDataParser, DocumentParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "from semantica.kg import GraphBuilder, TemporalPatternDetector, TemporalGraphQuery, GraphAnalyzer\n",
-    "from semantica.kg import CentralityCalculator, CommunityDetector, ConnectivityAnalyzer\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "from semantica.export import JSONExporter, CSVExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "import time\n",
-    "from datetime import datetime, timedelta\n",
-    "from collections import deque\n",
-    "\n",
-    "stream_ingestor = StreamIngestor()\n",
-    "file_ingestor = FileIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "feed_ingestor = FeedIngestor()\n",
-    "\n",
-    "# Real streaming sources configuration\n",
-    "stream_sources = [\n",
-    "    {\n",
-    "        \"type\": \"kafka\",\n",
-    "        \"topic\": \"security_logs\",\n",
-    "        \"bootstrap_servers\": [\"localhost:9092\"],\n",
-    "        \"consumer_config\": {\"group_id\": \"semantica_security_monitor\"}\n",
-    "    },\n",
-    "    {\n",
-    "        \"type\": \"rabbitmq\",\n",
-    "        \"queue\": \"security_events\",\n",
-    "        \"connection_url\": \"amqp://user:password@localhost:5672/\"\n",
-    "    }\n",
-    "]\n",
-    "\n",
-    "# Real database connection for security logs\n",
-    "db_connection_string = \"postgresql://user:password@localhost:5432/security_logs_db\"\n",
-    "db_query = \"SELECT * FROM security_events WHERE timestamp > NOW() - INTERVAL '1 hour' ORDER BY timestamp DESC LIMIT 1000\"\n",
-    "\n",
-    "# Real security feed URLs for threat intelligence\n",
-    "security_feeds = [\n",
-    "    \"https://www.cisa.gov/news.xml\",\n",
-    "    \"https://www.us-cert.gov/ncas/alerts.xml\"\n",
-    "]\n",
-    "\n",
-    "json_parser = JSONParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "document_parser = DocumentParser()\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "# Real-world streaming security log format (simulating real-time stream)\n",
-    "security_log_stream_file = os.path.join(temp_dir, \"security_log_stream.json\")\n",
-    "stream_logs = [\n",
-    "    {\n",
-    "        \"timestamp\": (datetime.now() - timedelta(minutes=5)).isoformat(),\n",
-    "        \"source_ip\": \"192.168.1.50\",\n",
-    "        \"destination_ip\": \"10.0.0.100\",\n",
-    "        \"event_type\": \"normal_traffic\",\n",
-    "        \"bytes_sent\": 1024,\n",
-    "        \"bytes_received\": 2048,\n",
-    "        \"protocol\": \"TCP\",\n",
-    "        \"port\": 80\n",
-    "    },\n",
-    "    {\n",
-    "        \"timestamp\": (datetime.now() - timedelta(minutes=4)).isoformat(),\n",
-    "        \"source_ip\": \"203.0.113.100\",\n",
-    "        \"destination_ip\": \"10.0.0.100\",\n",
-    "        \"event_type\": \"suspicious_connection\",\n",
-    "        \"bytes_sent\": 5000000,\n",
-    "        \"bytes_received\": 1000,\n",
-    "        \"protocol\": \"TCP\",\n",
-    "        \"port\": 443\n",
-    "    },\n",
-    "    {\n",
-    "        \"timestamp\": (datetime.now() - timedelta(minutes=3)).isoformat(),\n",
-    "        \"source_ip\": \"192.168.1.50\",\n",
-    "        \"destination_ip\": \"10.0.0.100\",\n",
-    "        \"event_type\": \"normal_traffic\",\n",
-    "        \"bytes_sent\": 512,\n",
-    "        \"bytes_received\": 1024,\n",
-    "        \"protocol\": \"UDP\",\n",
-    "        \"port\": 53\n",
-    "    },\n",
-    "    {\n",
-    "        \"timestamp\": (datetime.now() - timedelta(minutes=2)).isoformat(),\n",
-    "        \"source_ip\": \"198.51.100.50\",\n",
-    "        \"destination_ip\": \"10.0.0.100\",\n",
-    "        \"event_type\": \"port_scan\",\n",
-    "        \"bytes_sent\": 100,\n",
-    "        \"bytes_received\": 0,\n",
-    "        \"protocol\": \"TCP\",\n",
-    "        \"port\": 22\n",
-    "    },\n",
-    "    {\n",
-    "        \"timestamp\": (datetime.now() - timedelta(minutes=1)).isoformat(),\n",
-    "        \"source_ip\": \"203.0.113.100\",\n",
-    "        \"destination_ip\": \"10.0.0.100\",\n",
-    "        \"event_type\": \"data_exfiltration\",\n",
-    "        \"bytes_sent\": 10000000,\n",
-    "        \"bytes_received\": 500,\n",
-    "        \"protocol\": \"TCP\",\n",
-    "        \"port\": 443\n",
-    "    }\n",
-    "]\n",
-    "\n",
-    "with open(security_log_stream_file, 'w') as f:\n",
-    "    json.dump(stream_logs, f, indent=2)\n",
-    "\n",
-    "# Simulate streaming by processing logs in batches\n",
-    "log_stream = deque(stream_logs)\n",
-    "file_objects = file_ingestor.ingest_file(security_log_stream_file, read_content=True)\n",
-    "\n",
-    "# Parse streaming logs\n",
-    "parsed_stream = json_parser.parse(security_log_stream_file)\n",
-    "\n",
-    "print(f\"Streaming security logs initialized\")\n",
-    "print(f\"Ingested {len([file_objects]) if file_objects else 0} log stream files\")\n",
-    "print(f\"Parsed {len(parsed_stream.data) if parsed_stream and parsed_stream.data else 0} log entries\")\n",
-    "print(f\"Stream ready for real-time processing\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Real-Time Parsing and Entity Extraction\n",
-    "\n",
-    "Parse streaming logs in real-time and extract security entities.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "event_detector = EventDetector()\n",
-    "triplet_extractor = TripletExtractor()\n",
-    "\n",
-    "# Real-time processing loop (simulated)\n",
-    "security_entities = []\n",
-    "stream_relationships = []\n",
-    "detected_events = []\n",
-    "\n",
-    "# Process logs in real-time batches\n",
-    "for log_entry in parsed_stream.data if parsed_stream and parsed_stream.data else []:\n",
-    "    if isinstance(log_entry, dict):\n",
-    "        log_text = f\"{log_entry.get('event_type', '')} from {log_entry.get('source_ip', '')} to {log_entry.get('destination_ip', '')} on port {log_entry.get('port', '')}\"\n",
-    "        \n",
-    "        entities = ner_extractor.extract(log_text)\n",
-    "        relationships = relation_extractor.extract(log_text, entities)\n",
-    "        events = event_detector.detect_events(log_text)\n",
-    "        \n",
-    "        security_entities.append({\n",
-    "            \"id\": log_entry.get(\"source_ip\", \"\"),\n",
-    "            \"type\": \"IP_Address\",\n",
-    "            \"name\": log_entry.get(\"source_ip\", \"\"),\n",
-    "            \"properties\": {\n",
-    "                \"timestamp\": log_entry.get(\"timestamp\", \"\"),\n",
-    "                \"source\": \"stream\"\n",
-    "            }\n",
-    "        })\n",
-    "        security_entities.append({\n",
-    "            \"id\": log_entry.get(\"destination_ip\", \"\"),\n",
-    "            \"type\": \"IP_Address\",\n",
-    "            \"name\": log_entry.get(\"destination_ip\", \"\"),\n",
-    "            \"properties\": {\n",
-    "                \"timestamp\": log_entry.get(\"timestamp\", \"\"),\n",
-    "                \"source\": \"stream\"\n",
-    "            }\n",
-    "        })\n",
-    "        security_entities.append({\n",
-    "            \"id\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"type\": \"Security_Event\",\n",
-    "            \"name\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"properties\": {\n",
-    "                \"timestamp\": log_entry.get(\"timestamp\", \"\"),\n",
-    "                \"bytes_sent\": log_entry.get(\"bytes_sent\", 0),\n",
-    "                \"bytes_received\": log_entry.get(\"bytes_received\", 0),\n",
-    "                \"protocol\": log_entry.get(\"protocol\", \"\"),\n",
-    "                \"port\": log_entry.get(\"port\", 0)\n",
-    "            }\n",
-    "        })\n",
-    "        \n",
-    "        stream_relationships.append({\n",
-    "            \"source\": log_entry.get(\"source_ip\", \"\"),\n",
-    "            \"target\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"type\": \"triggered\",\n",
-    "            \"properties\": {\"timestamp\": log_entry.get(\"timestamp\", \"\")}\n",
-    "        })\n",
-    "        stream_relationships.append({\n",
-    "            \"source\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"target\": log_entry.get(\"destination_ip\", \"\"),\n",
-    "            \"type\": \"targeted\",\n",
-    "            \"properties\": {\"timestamp\": log_entry.get(\"timestamp\", \"\")}\n",
-    "        })\n",
-    "        \n",
-    "        detected_events.extend(events)\n",
-    "\n",
-    "print(f\"Real-time processing complete\")\n",
-    "print(f\"Extracted {len(security_entities)} security entities\")\n",
-    "print(f\"Extracted {len(stream_relationships)} relationships\")\n",
-    "print(f\"Detected {len(detected_events)} events\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Build Temporal Knowledge Graph\n",
-    "\n",
-    "Build and continuously update temporal knowledge graph from streaming data.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "temporal_pattern_detector = TemporalPatternDetector()\n",
-    "temporal_query = TemporalGraphQuery()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "\n",
-    "# Build temporal KG from streaming data\n",
-    "temporal_kg = builder.build(security_entities, stream_relationships)\n",
-    "\n",
-    "# Analyze graph structure in real-time\n",
-    "metrics = graph_analyzer.compute_metrics(temporal_kg)\n",
-    "centrality_calculator = CentralityCalculator()\n",
-    "community_detector = CommunityDetector()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "\n",
-    "centrality_result = centrality_calculator.calculate_degree_centrality(temporal_kg)\n",
-    "centrality_scores = centrality_result.get('centrality', {})\n",
-    "communities = community_detector.detect_communities(temporal_kg)\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(temporal_kg)\n",
-    "\n",
-    "print(f\"Built temporal knowledge graph from stream\")\n",
-    "print(f\"  Entities: {len(temporal_kg.get('entities', []))}\")\n",
-    "print(f\"  Relationships: {len(temporal_kg.get('relationships', []))}\")\n",
-    "print(f\"  Graph density: {metrics.get('density', 0):.3f}\")\n",
-    "print(f\"  Communities: {len(communities)}\")\n",
-    "print(f\"  Central entities: {len([e for e, score in centrality_scores.items() if score > 0])}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Real-Time Pattern Detection\n",
-    "\n",
-    "Detect temporal patterns and anomalies in real-time.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Detect temporal patterns\n",
-    "temporal_patterns = temporal_pattern_detector.detect_temporal_patterns(\n",
-    "    temporal_kg,\n",
-    "    pattern_type=\"anomaly\",\n",
-    "    min_frequency=1\n",
-    ")\n",
-    "\n",
-    "# Real-time anomaly detection using inference\n",
-    "inference_engine = InferenceEngine()\n",
-    "rule_manager = RuleManager()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-    "\n",
-    "# Define real-time anomaly detection rules\n",
-    "inference_engine.add_rule(\"IF bytes_sent > 1000000 AND bytes_received < 1000 THEN potential_data_exfiltration\")\n",
-    "inference_engine.add_rule(\"IF event_type is port_scan AND port is 22 THEN ssh_brute_force\")\n",
-    "inference_engine.add_rule(\"IF multiple events from same source_ip in short time THEN suspicious_activity\")\n",
-    "\n",
-    "# Add facts from streaming logs\n",
-    "for log_entry in parsed_stream.data if parsed_stream and parsed_stream.data else []:\n",
-    "    if isinstance(log_entry, dict):\n",
-    "        inference_engine.add_fact({\n",
-    "            \"source_ip\": log_entry.get(\"source_ip\", \"\"),\n",
-    "            \"event_type\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"bytes_sent\": log_entry.get(\"bytes_sent\", 0),\n",
-    "            \"bytes_received\": log_entry.get(\"bytes_received\", 0),\n",
-    "            \"port\": log_entry.get(\"port\", 0),\n",
-    "            \"timestamp\": log_entry.get(\"timestamp\", \"\")\n",
-    "        })\n",
-    "\n",
-    "inferred_anomalies = inference_engine.forward_chain()\n",
-    "\n",
-    "# Real-time anomaly scoring\n",
-    "real_time_anomalies = []\n",
-    "for log_entry in parsed_stream.data if parsed_stream and parsed_stream.data else []:\n",
-    "    if isinstance(log_entry, dict):\n",
-    "        anomaly_score = 0\n",
-    "        reasons = []\n",
-    "        \n",
-    "        if log_entry.get(\"bytes_sent\", 0) > 1000000:\n",
-    "            anomaly_score += 5\n",
-    "            reasons.append(\"Unusually large data transfer\")\n",
-    "        \n",
-    "        if log_entry.get(\"event_type\") in [\"port_scan\", \"data_exfiltration\"]:\n",
-    "            anomaly_score += 4\n",
-    "            reasons.append(\"High-risk event type\")\n",
-    "        \n",
-    "        if log_entry.get(\"bytes_sent\", 0) > log_entry.get(\"bytes_received\", 0) * 100:\n",
-    "            anomaly_score += 3\n",
-    "            reasons.append(\"Asymmetric traffic pattern\")\n",
-    "        \n",
-    "        if anomaly_score >= 3:\n",
-    "            real_time_anomalies.append({\n",
-    "                \"source_ip\": log_entry.get(\"source_ip\", \"\"),\n",
-    "                \"destination_ip\": log_entry.get(\"destination_ip\", \"\"),\n",
-    "                \"event_type\": log_entry.get(\"event_type\", \"\"),\n",
-    "                \"severity\": \"high\" if anomaly_score >= 5 else \"medium\",\n",
-    "                \"score\": anomaly_score,\n",
-    "                \"reasons\": reasons,\n",
-    "                \"timestamp\": log_entry.get(\"timestamp\", \"\")\n",
-    "            })\n",
-    "\n",
-    "print(f\"Detected {len(temporal_patterns)} temporal patterns\")\n",
-    "print(f\"Inferred {len(inferred_anomalies)} anomalies from rules\")\n",
-    "print(f\"Identified {len(real_time_anomalies)} real-time anomalies\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Generate Real-Time Alerts\n",
-    "\n",
-    "Generate and send alerts for detected anomalies.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-
-    "json_exporter = JSONExporter()\n",
-    "csv_exporter = CSVExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-    "quality_score = quality_assessor.assess_overall_quality(temporal_kg)\n",
-    "\n",
-    "# Generate alerts\n",
-    "alerts = []\n",
-    "for anomaly in real_time_anomalies:\n",
-    "    alert = {\n",
-    "        \"alert_id\": f\"alert_{anomaly['source_ip']}_{int(time.time())}\",\n",
-    "        \"severity\": anomaly[\"severity\"],\n",
-    "        \"source_ip\": anomaly[\"source_ip\"],\n",
-    "        \"destination_ip\": anomaly[\"destination_ip\"],\n",
-    "        \"event_type\": anomaly[\"event_type\"],\n",
-    "        \"score\": anomaly[\"score\"],\n",
-    "        \"reasons\": anomaly[\"reasons\"],\n",
-    "        \"timestamp\": anomaly[\"timestamp\"],\n",
-    "        \"status\": \"active\"\n",
-    "    }\n",
-    "    alerts.append(alert)\n",
-    "\n",
-    "# Export alerts\n",
-    "json_exporter.export_knowledge_graph(temporal_kg, os.path.join(temp_dir, \"realtime_kg.json\"))\n",
-    "csv_exporter.export_entities(security_entities, os.path.join(temp_dir, \"realtime_entities.csv\"))\n",
-    "\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Real-time anomaly detection identified {len(real_time_anomalies)} anomalies\",\n",
-    "    \"total_events\": len(parsed_stream.data) if parsed_stream and parsed_stream.data else 0,\n",
-    "    \"anomalies\": len(real_time_anomalies),\n",
-    "    \"alerts\": len(alerts),\n",
-    "    \"quality_score\": quality_score.get('overall_score', 0),\n",
-    "    \"high_severity\": len([a for a in alerts if a.get('severity') == 'high'])\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"Generated {len(alerts)} real-time alerts\")\n",
-    "print(f\"High severity alerts: {len([a for a in alerts if a.get('severity') == 'high'])}\")\n",
-    "print(f\"Report length: {len(report)} characters\")\n",
-    "print(f\"Graph quality score: {quality_score.get('overall_score', 0):.3f}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 6: Real-Time Monitoring and Visualization\n",
-    "\n",
-    "Monitor security events in real-time and visualize results.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "kg_visualizer = KGVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(temporal_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(temporal_kg, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(temporal_kg, output=\"interactive\")\n",
-    "\n",
-    "print(f\"Real-time monitoring active\")\n",
-    "print(f\"Monitoring {len(temporal_kg.get('entities', []))} entities in real-time\")\n",
-    "print(f\"Active alerts: {len(alerts)}\")\n",
-    "print(f\"Total modules used: 20+\")\n",
-    "print(f\"Pipeline complete: Stream Logs \u2192 Real-Time Parse \u2192 Extract \u2192 Temporal KG \u2192 Pattern Detection \u2192 Anomaly Detection \u2192 Alerts \u2192 Monitor \u2192 Visualize\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/cybersecurity/02_Incident_Analysis.ipynb

@@ -1,474 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/cybersecurity/02_Incident_Analysis.ipynb)\n",
-    "\n",
-    "# Incident Analysis Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete security incident analysis pipeline: ingest security logs from multiple sources (files, databases, streams), parse structured and unstructured logs, extract security entities, build knowledge graph, analyze relationships, detect anomalies, and generate incident reports.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, RepoIngestor, EmailIngestor, MCPIngestor\n",
-    "- **Parsing**: JSONParser, XMLParser, StructuredDataParser, DocumentParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "- **KG**: GraphBuilder, GraphAnalyzer, ConnectivityAnalyzer, CentralityCalculator\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "- **Quality**: ProvenanceTracker\n",
-    "- **Export**: JSONExporter, RDFExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, AnalyticsVisualizer, TemporalVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Multiple Security Sources \u2192 Parse Logs \u2192 Extract Security Entities \u2192 Build Incident KG \u2192 Analyze Relationships \u2192 Detect Anomalies \u2192 Generate Reports \u2192 Visualize**\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Ingest Security Logs from Multiple Sources\n",
-    "\n",
-    "Ingest security logs from files, databases, streams, and threat intelligence feeds.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import FileIngestor, DBIngestor, StreamIngestor, FeedIngestor\n",
-    "from semantica.parse import JSONParser, XMLParser, StructuredDataParser, DocumentParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "from semantica.kg import GraphBuilder, GraphAnalyzer, ConnectivityAnalyzer, CentralityCalculator\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-
-    "from semantica.kg import ProvenanceTracker\n",
-    "from semantica.export import JSONExporter, RDFExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, AnalyticsVisualizer, TemporalVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "file_ingestor = FileIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "stream_ingestor = StreamIngestor()\n",
-    "feed_ingestor = FeedIngestor()\n",
-    "\n",
-    "json_parser = JSONParser()\n",
-    "xml_parser = XMLParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "document_parser = DocumentParser()\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "# Real-world security log formats\n",
-    "security_logs_json = os.path.join(temp_dir, \"security_logs.json\")\n",
-    "security_logs_data = [\n",
-    "    {\n",
-    "        \"timestamp\": (datetime.now() - timedelta(hours=2)).isoformat(),\n",
-    "        \"source_ip\": \"192.168.1.100\",\n",
-    "        \"destination_ip\": \"10.0.0.50\",\n",
-    "        \"event_type\": \"failed_login\",\n",
-    "        \"user\": \"admin\",\n",
-    "        \"severity\": \"medium\",\n",
-    "        \"message\": \"Multiple failed login attempts detected\"\n",
-    "    },\n",
-    "    {\n",
-    "        \"timestamp\": (datetime.now() - timedelta(hours=1)).isoformat(),\n",
-    "        \"source_ip\": \"203.0.113.45\",\n",
-    "        \"destination_ip\": \"10.0.0.50\",\n",
-    "        \"event_type\": \"port_scan\",\n",
-    "        \"severity\": \"high\",\n",
-    "        \"message\": \"Port scanning activity detected from external IP\"\n",
-    "    },\n",
-    "    {\n",
-    "        \"timestamp\": (datetime.now() - timedelta(minutes=30)).isoformat(),\n",
-    "        \"source_ip\": \"192.168.1.100\",\n",
-    "        \"destination_ip\": \"10.0.0.75\",\n",
-    "        \"event_type\": \"data_exfiltration\",\n",
-    "        \"user\": \"user123\",\n",
-    "        \"severity\": \"critical\",\n",
-    "        \"message\": \"Large data transfer detected to external server\"\n",
-    "    }\n",
-    "]\n",
-    "\n",
-    "with open(security_logs_json, 'w') as f:\n",
-    "    json.dump(security_logs_data, f, indent=2)\n",
-    "\n",
-    "# XML format security events (common in SIEM systems)\n",
-    "security_events_xml = os.path.join(temp_dir, \"security_events.xml\")\n",
-    "xml_content = \"\"\"<?xml version=\"1.0\"?>\n",
-    "<security_events>\n",
-    "    <event>\n",
-    "        <timestamp>2024-01-15T14:30:00</timestamp>\n",
-    "        <source_ip>172.16.0.10</source_ip>\n",
-    "        <destination_ip>10.0.0.50</destination_ip>\n",
-    "        <event_type>malware_detection</event_type>\n",
-    "        <severity>high</severity>\n",
-    "        <description>Malware signature detected in file transfer</description>\n",
-    "    </event>\n",
-    "    <event>\n",
-    "        <timestamp>2024-01-15T15:00:00</timestamp>\n",
-    "        <source_ip>192.168.1.200</source_ip>\n",
-    "        <destination_ip>10.0.0.50</destination_ip>\n",
-    "        <event_type>unauthorized_access</event_type>\n",
-    "        <severity>critical</severity>\n",
-    "        <description>Unauthorized access attempt to restricted resource</description>\n",
-    "    </event>\n",
-    "</security_events>\"\"\"\n",
-    "\n",
-    "with open(security_events_xml, 'w') as f:\n",
-    "    f.write(xml_content)\n",
-    "\n",
-    "# Ingest from files\n",
-    "file_objects = file_ingestor.ingest_file(security_logs_json, read_content=True)\n",
-    "file_objects_xml = file_ingestor.ingest_file(security_events_xml, read_content=True)\n",
-    "\n",
-    "# Parse structured logs\n",
-    "parsed_json = json_parser.parse(security_logs_json)\n",
-    "parsed_xml = xml_parser.parse(security_events_xml)\n",
-    "\n",
-    "# Real security intelligence feed URLs\n",
-    "security_feeds = [\n",
-    "    \"https://www.cisa.gov/news.xml\",  # CISA Security Advisories\n",
-    "    \"https://www.us-cert.gov/ncas/alerts.xml\",  # US-CERT Alerts\n",
-    "    \"https://feeds.feedburner.com/SecurityWeek\",  # Security Week\n",
-    "    \"https://www.darkreading.com/rss.xml\"  # Dark Reading\n",
-    "]\n",
-    "\n",
-    "threat_feed_list = []\n",
-    "for feed_url in security_feeds:\n",
-    "    threat_feed = feed_ingestor.ingest_feed(feed_url)\n",
-    "    if threat_feed:\n",
-    "        threat_feed_list.append(threat_feed)\n",
-    "        print(f\"  Ingested feed: {feed_url}\")\n",
-    "\n",
-    "print(f\"Ingested {len([file_objects]) if file_objects else 0} JSON log files\")\n",
-    "print(f\"Ingested {len([file_objects_xml]) if file_objects_xml else 0} XML event files\")\n",
-    "print(f\"Parsed {len(parsed_json.data) if parsed_json and parsed_json.data else 0} JSON log entries\")\n",
-    "print(f\"Parsed {len(parsed_xml.elements) if parsed_xml else 0} XML event elements\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Extract Security Entities and Relationships\n",
-    "\n",
-    "Extract security entities (IPs, users, events) and relationships from parsed logs.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "event_detector = EventDetector()\n",
-    "triplet_extractor = TripletExtractor()\n",
-    "\n",
-    "all_entities = []\n",
-    "all_relationships = []\n",
-    "all_events = []\n",
-    "\n",
-    "# Extract from JSON logs\n",
-    "if parsed_json and parsed_json.data:\n",
-    "    for log_entry in parsed_json.data:\n",
-    "        if isinstance(log_entry, dict):\n",
-    "            log_text = f\"{log_entry.get('event_type', '')} from {log_entry.get('source_ip', '')} to {log_entry.get('destination_ip', '')}: {log_entry.get('message', '')}\"\n",
-    "            \n",
-    "            entities = ner_extractor.extract(log_text)\n",
-    "            all_entities.extend(entities)\n",
-    "            \n",
-    "            relationships = relation_extractor.extract(log_text, entities)\n",
-    "            all_relationships.extend(relationships)\n",
-    "            \n",
-    "            events = event_detector.detect_events(log_text)\n",
-    "            all_events.extend(events)\n",
-    "\n",
-    "# Extract from XML events\n",
-    "if parsed_xml and parsed_xml.elements:\n",
-    "    for elem in parsed_xml.elements:\n",
-    "        if hasattr(elem, 'text') and elem.text:\n",
-    "            entities = ner_extractor.extract(elem.text)\n",
-    "            all_entities.extend(entities)\n",
-    "            \n",
-    "            relationships = relation_extractor.extract(elem.text, entities)\n",
-    "            all_relationships.extend(relationships)\n",
-    "\n",
-    "# Build structured entities from log data\n",
-    "security_entities = []\n",
-    "for log_entry in parsed_json.data if parsed_json and parsed_json.data else []:\n",
-    "    if isinstance(log_entry, dict):\n",
-    "        security_entities.append({\n",
-    "            \"id\": log_entry.get(\"source_ip\", \"\"),\n",
-    "            \"type\": \"IP_Address\",\n",
-    "            \"name\": log_entry.get(\"source_ip\", \"\"),\n",
-    "            \"properties\": {\"source\": \"security_logs\"}\n",
-    "        })\n",
-    "        security_entities.append({\n",
-    "            \"id\": log_entry.get(\"destination_ip\", \"\"),\n",
-    "            \"type\": \"IP_Address\",\n",
-    "            \"name\": log_entry.get(\"destination_ip\", \"\"),\n",
-    "            \"properties\": {\"source\": \"security_logs\"}\n",
-    "        })\n",
-    "        if log_entry.get(\"user\"):\n",
-    "            security_entities.append({\n",
-    "                \"id\": log_entry.get(\"user\", \"\"),\n",
-    "                \"type\": \"User\",\n",
-    "                \"name\": log_entry.get(\"user\", \"\"),\n",
-    "                \"properties\": {\"source\": \"security_logs\"}\n",
-    "            })\n",
-    "        security_entities.append({\n",
-    "            \"id\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"type\": \"Security_Event\",\n",
-    "            \"name\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"properties\": {\n",
-    "                \"severity\": log_entry.get(\"severity\", \"\"),\n",
-    "                \"timestamp\": log_entry.get(\"timestamp\", \"\"),\n",
-    "                \"message\": log_entry.get(\"message\", \"\")\n",
-    "            }\n",
-    "        })\n",
-    "\n",
-    "incident_relationships = []\n",
-    "for log_entry in parsed_json.data if parsed_json and parsed_json.data else []:\n",
-    "    if isinstance(log_entry, dict):\n",
-    "        incident_relationships.append({\n",
-    "            \"source\": log_entry.get(\"source_ip\", \"\"),\n",
-    "            \"target\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"type\": \"triggered\",\n",
-    "            \"properties\": {\"timestamp\": log_entry.get(\"timestamp\", \"\")}\n",
-    "        })\n",
-    "        incident_relationships.append({\n",
-    "            \"source\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"target\": log_entry.get(\"destination_ip\", \"\"),\n",
-    "            \"type\": \"targeted\",\n",
-    "            \"properties\": {\"timestamp\": log_entry.get(\"timestamp\", \"\")}\n",
-    "        })\n",
-    "\n",
-    "print(f\"Extracted {len(security_entities)} security entities\")\n",
-    "print(f\"Extracted {len(incident_relationships)} incident relationships\")\n",
-    "print(f\"Detected {len(all_events)} security events\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Build Incident Knowledge Graph\n",
-    "\n",
-    "Build a knowledge graph from security entities and relationships.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "centrality_calculator = CentralityCalculator()\n",
-    "provenance_tracker = ProvenanceTracker()\n",
-    "\n",
-    "incident_kg = builder.build(security_entities, incident_relationships)\n",
-    "\n",
-    "# Track provenance\n",
-    "for entity in security_entities:\n",
-    "    provenance_tracker.track_entity(entity.get(\"id\"), entity.get(\"properties\", {}).get(\"source\", \"unknown\"), entity)\n",
-    "\n",
-    "# Analyze graph structure\n",
-    "metrics = graph_analyzer.compute_metrics(incident_kg)\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(incident_kg)\n",
-    "centrality_result = centrality_calculator.calculate_degree_centrality(incident_kg)\n",
-    "centrality_scores = centrality_result.get('centrality', {})\n",
-    "\n",
-    "print(f\"Built incident knowledge graph\")\n",
-    "print(f\"  Entities: {len(incident_kg.get('entities', []))}\")\n",
-    "print(f\"  Relationships: {len(incident_kg.get('relationships', []))}\")\n",
-    "print(f\"  Graph density: {metrics.get('density', 0):.3f}\")\n",
-    "print(f\"  Connected components: {len(connectivity.get('components', []))}\")\n",
-    "print(f\"  Central entities: {len([e for e, score in centrality_scores.items() if score > 0])}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Analyze Relationships and Detect Anomalies\n",
-    "\n",
-    "Analyze security relationships and detect anomalous patterns.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "inference_engine = InferenceEngine()\n",
-    "rule_manager = RuleManager()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-
-    "\n",
-    "# Define security rules\n",
-    "inference_engine.add_rule(\"IF event_type is port_scan AND severity is high THEN potential_intrusion\")\n",
-    "inference_engine.add_rule(\"IF event_type is data_exfiltration AND severity is critical THEN data_breach\")\n",
-    "inference_engine.add_rule(\"IF multiple failed_login events from same source_ip THEN brute_force_attack\")\n",
-    "\n",
-    "# Add facts from security events\n",
-    "for log_entry in parsed_json.data if parsed_json and parsed_json.data else []:\n",
-    "    if isinstance(log_entry, dict):\n",
-    "        inference_engine.add_fact({\n",
-    "            \"event_type\": log_entry.get(\"event_type\", \"\"),\n",
-    "            \"severity\": log_entry.get(\"severity\", \"\"),\n",
-    "            \"source_ip\": log_entry.get(\"source_ip\", \"\")\n",
-    "        })\n",
-    "\n",
-    "# Run inference\n",
-    "inferred_threats = inference_engine.forward_chain()\n",
-    "\n",
-    "# Detect anomalies based on patterns\n",
-    "anomalies = []\n",
-    "for log_entry in parsed_json.data if parsed_json and parsed_json.data else []:\n",
-    "    if isinstance(log_entry, dict):\n",
-    "        anomaly_score = 0\n",
-    "        reasons = []\n",
-    "        \n",
-    "        if log_entry.get(\"severity\") == \"critical\":\n",
-    "            anomaly_score += 5\n",
-    "            reasons.append(\"Critical severity event\")\n",
-    "        \n",
-    "        if log_entry.get(\"event_type\") in [\"data_exfiltration\", \"unauthorized_access\"]:\n",
-    "            anomaly_score += 4\n",
-    "            reasons.append(\"High-risk event type\")\n",
-    "        \n",
-    "        if log_entry.get(\"severity\") == \"high\" and log_entry.get(\"event_type\") == \"port_scan\":\n",
-    "            anomaly_score += 3\n",
-    "            reasons.append(\"Port scanning detected\")\n",
-    "        \n",
-    "        if anomaly_score >= 3:\n",
-    "            anomalies.append({\n",
-    "                \"event\": log_entry.get(\"event_type\", \"\"),\n",
-    "                \"source_ip\": log_entry.get(\"source_ip\", \"\"),\n",
-    "                \"severity\": log_entry.get(\"severity\", \"\"),\n",
-    "                \"score\": anomaly_score,\n",
-    "                \"reasons\": reasons,\n",
-    "                \"timestamp\": log_entry.get(\"timestamp\", \"\")\n",
-    "            })\n",
-    "\n",
-    "# Detect conflicts in security data\n",
-
-    "\n",
-    "print(f\"Analyzed security relationships\")\n",
-    "print(f\"Inferred {len(inferred_threats)} potential threats\")\n",
-    "print(f\"Detected {len(anomalies)} anomalies\")\n",
-
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Generate Incident Reports\n",
-    "\n",
-    "Generate comprehensive incident analysis reports.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-
-    "json_exporter = JSONExporter()\n",
-    "rdf_exporter = RDFExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-
-    "\n",
-    "json_exporter.export_knowledge_graph(incident_kg, os.path.join(temp_dir, \"incident_kg.json\"))\n",
-    "rdf_exporter.export_knowledge_graph(incident_kg, os.path.join(temp_dir, \"incident_kg.rdf\"))\n",
-    "\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Security incident analysis identified {len(anomalies)} anomalies and {len(inferred_threats)} potential threats\",\n",
-    "    \"total_events\": len(parsed_json.data) if parsed_json and parsed_json.data else 0,\n",
-    "    \"anomalies\": len(anomalies),\n",
-    "    \"threats\": len(inferred_threats),\n",
-    "    \"quality_score\": 0.95,\n",
-    "    \"critical_events\": len([e for e in anomalies if e.get('severity') == 'critical'])\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"Report length: {len(report)} characters\")\n",
-    "print(f\"Graph quality score: 0.950\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 6: Visualize Security Incidents\n",
-    "\n",
-    "Visualize incident knowledge graph and security patterns.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "kg_visualizer = KGVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(incident_kg, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(incident_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(incident_kg, output=\"interactive\")\n",
-    "\n",
-    "print(f\"Total modules used: 20+\")\n",
-    "print(f\"Pipeline complete: Multiple Security Sources \u2192 Parse Logs \u2192 Extract Entities \u2192 Build KG \u2192 Analyze \u2192 Detect Anomalies \u2192 Reports \u2192 Visualize\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/cybersecurity/03_Threat_Correlation.ipynb

@@ -1,439 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/cybersecurity/03_Threat_Correlation.ipynb)\n",
-    "\n",
-    "# Threat Correlation Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete threat correlation pipeline for cybersecurity: ingest threat feeds from multiple sources, extract IOCs, build temporal knowledge graph, correlate threats, detect campaigns, and generate reports.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, RepoIngestor, EmailIngestor, MCPIngestor\n",
-    "- **Parsing**: XMLParser, StructuredDataParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector\n",
-    "- **KG**: GraphBuilder, TemporalGraphQuery, TemporalPatternDetector, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "- **Reasoning**: InferenceEngine, ExplanationGenerator\n",
-    "- **Quality**: ProvenanceTracker\n",
-    "- **Export**: RDFExporter, ReportGenerator\n",
-    "- **Visualization**: AnalyticsVisualizer, TemporalVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Multiple Threat Feeds → Parse → Extract IOCs → Build Temporal KG → Correlate Threats → Detect Campaigns → Generate Reports → Visualize**\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Ingest Threat Feeds\n",
-    "\n",
-    "Ingest threat intelligence from multiple sources.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import FileIngestor, FeedIngestor, DBIngestor, WebIngestor\n",
-    "from semantica.parse import XMLParser, StructuredDataParser, JSONParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector\n",
-    "from semantica.kg import GraphBuilder, TemporalGraphQuery, TemporalPatternDetector, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "from semantica.reasoning import InferenceEngine, ExplanationGenerator\n",
-    "from semantica.kg import ProvenanceTracker\n",
-    "from semantica.conflicts import ConflictDetector\n",
-    "from semantica.export import RDFExporter, ReportGenerator\n",
-    "from semantica.visualization import AnalyticsVisualizer, TemporalVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "file_ingestor = FileIngestor()\n",
-    "feed_ingestor = FeedIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "web_ingestor = WebIngestor()\n",
-    "xml_parser = XMLParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "json_parser = JSONParser()\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "# Real threat intelligence feed URLs\n",
-    "threat_feeds = [\n",
-    "    \"https://www.cisa.gov/news.xml\",  # CISA Security Advisories\n",
-    "    \"https://www.us-cert.gov/ncas/alerts.xml\",  # US-CERT Alerts\n",
-    "    \"https://feeds.feedburner.com/SecurityWeek\",  # Security Week\n",
-    "    \"https://www.darkreading.com/rss.xml\"  # Dark Reading\n",
-    "]\n",
-    "\n",
-    "# Real database connection pattern (PostgreSQL example)\n",
-    "db_connection_string = \"postgresql://user:password@localhost:5432/threat_intel_db\"\n",
-    "db_query = \"SELECT ioc, ioc_type, timestamp, severity, source FROM threat_indicators WHERE timestamp > NOW() - INTERVAL '7 days'\"\n",
-    "\n",
-    "# Real web API endpoints for threat intelligence\n",
-    "threat_apis = [\n",
-    "    \"https://api.github.com/repos/mitre/cti/contents/enterprise-attack/attack-pattern\",  # MITRE ATT&CK\n",
-    "    \"https://www.virustotal.com/vtapi/v2/domain/report\",  # VirusTotal API (requires API key)\n",
-    "    \"https://api.shodan.io/shodan/host/search\"  # Shodan API (requires API key)\n",
-    "]\n",
-    "\n",
-    "# Ingest from real RSS feeds\n",
-    "feed_data_list = []\n",
-    "for feed_url in threat_feeds:\n",
-    "    feed_data = feed_ingestor.ingest_feed(feed_url)\n",
-    "    if feed_data:\n",
-    "        feed_data_list.append(feed_data)\n",
-    "        print(f\"  Ingested feed: {feed_url}\")\n",
-    "        print(f\"  Items: {len(feed_data.items) if hasattr(feed_data, 'items') else 0}\")\n",
-    "\n",
-    "# Ingest from web APIs (example with public API)\n",
-    "web_content = web_ingestor.ingest_url(\"https://api.github.com/repos/mitre/cti\")\n",
-    "if web_content:\n",
-    "    print(f\"  Ingested web content\")\n",
-    "\n",
-    "# Database ingestion pattern (would connect to real database)\n",
-    "# Example: Export from threat intelligence database\n",
-    "db_data = db_ingestor.export_table(\n",
-    "    connection_string=db_connection_string,\n",
-    "    table_name=\"threat_indicators\",\n",
-    "    limit=1000\n",
-    ")\n",
-    "print(f\"  Query pattern: {db_query}\")\n",
-    "# Simulate database structure for demonstration if needed\n",
-    "if not db_data:\n",
-    "    db_data = {\n",
-    "        \"data\": [\n",
-    "            {\"ioc\": \"192.168.1.100\", \"ioc_type\": \"IP\", \"timestamp\": datetime.now().isoformat(), \"severity\": \"high\", \"source\": \"threat_feed\"},\n",
-    "            {\"ioc\": \"malicious-domain.com\", \"ioc_type\": \"Domain\", \"timestamp\": datetime.now().isoformat(), \"severity\": \"medium\", \"source\": \"threat_feed\"}\n",
-    "        ]\n",
-    "    }\n",
-    "\n",
-    "# Parse feed data\n",
-    "parsed_feeds = []\n",
-    "for feed_data in feed_data_list:\n",
-    "    if hasattr(feed_data, 'items'):\n",
-    "        for item in feed_data.items[:10]:  # Process first 10 items\n",
-    "            parsed_feeds.append({\n",
-    "                \"title\": item.title if hasattr(item, 'title') else \"\",\n",
-    "                \"description\": item.description if hasattr(item, 'description') else \"\",\n",
-    "                \"published\": item.published if hasattr(item, 'published') else \"\",\n",
-    "                \"link\": item.link if hasattr(item, 'link') else \"\"\n",
-    "            })\n",
-    "\n",
-    "parsed_db = structured_parser.parse_json(json.dumps(db_data)) if db_data else None\n",
-    "\n",
-    "print(f\"\\n📊 Ingestion Summary:\")\n",
-    "print(f\"  Feeds ingested: {len(feed_data_list)}\")\n",
-    "print(f\"  Feed items processed: {len(parsed_feeds)}\")\n",
-    "print(f\"  Database records: {len(db_data.get('data', [])) if db_data else 0}\")\n",
-    "print(f\"  Web sources: 1\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Extract IOCs\n",
-    "\n",
-    "Extract Indicators of Compromise (IOCs) from threat feeds.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "event_detector = EventDetector()\n",
-    "\n",
-    "all_threat_texts = []\n",
-    "if parsed_xml and parsed_xml.elements:\n",
-    "    for elem in parsed_xml.elements:\n",
-    "        if hasattr(elem, 'text') and elem.text:\n",
-    "            all_threat_texts.append(elem.text)\n",
-    "\n",
-    "for db_record in parsed_db.get(\"data\", threat_db_data):\n",
-    "    threat_text = f\"IOC: {db_record.get('ioc', '')} Type: {db_record.get('type', '')} Severity: {db_record.get('severity', '')}\"\n",
-    "    all_threat_texts.append(threat_text)\n",
-    "\n",
-    "all_entities = []\n",
-    "all_relationships = []\n",
-    "all_events = []\n",
-    "\n",
-    "for text in all_threat_texts:\n",
-    "    entities = ner_extractor.extract(text)\n",
-    "    all_entities.extend(entities)\n",
-    "    \n",
-    "    relationships = relation_extractor.extract(text, entities)\n",
-    "    all_relationships.extend(relationships)\n",
-    "    \n",
-    "    events = event_detector.detect_events(text)\n",
-    "    all_events.extend(events)\n",
-    "\n",
-    "print(f\"Extracted {len(all_entities)} IOCs\")\n",
-    "print(f\"Extracted {len(all_relationships)} relationships\")\n",
-    "print(f\"Detected {len(all_events)} events\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Build Temporal Knowledge Graph\n",
-    "\n",
-    "Build a temporal knowledge graph from extracted IOCs and relationships.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "\n",
-    "threat_entities = []\n",
-    "for i, entity in enumerate(all_entities[:10], 1):\n",
-    "    threat_entities.append({\n",
-    "        \"id\": f\"ioc_{i}\",\n",
-    "        \"type\": entity.get(\"type\", \"IOC\"),\n",
-    "        \"name\": entity.get(\"text\", entity.get(\"entity\", \"\")),\n",
-    "        \"properties\": {\"timestamp\": datetime.now().isoformat()}\n",
-    "    })\n",
-    "\n",
-    "threat_relationships = []\n",
-    "for i, rel in enumerate(all_relationships[:5], 1):\n",
-    "    threat_relationships.append({\n",
-    "        \"source\": f\"ioc_{i}\",\n",
-    "        \"target\": f\"ioc_{i+1}\",\n",
-    "        \"type\": rel.get(\"type\", \"related_to\"),\n",
-    "        \"properties\": {\"timestamp\": datetime.now().isoformat()}\n",
-    "    })\n",
-    "\n",
-    "threat_kg = builder.build(threat_entities, threat_relationships)\n",
-    "\n",
-    "print(f\"Built temporal knowledge graph\")\n",
-    "print(f\"  Entities: {len(threat_kg.get('entities', []))}\")\n",
-    "print(f\"  Relationships: {len(threat_kg.get('relationships', []))}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Correlate Threats\n",
-    "\n",
-    "Correlate threats using temporal queries and inference.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "temporal_query = TemporalGraphQuery()\n",
-    "pattern_detector = TemporalPatternDetector()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "inference_engine = InferenceEngine()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-    "\n",
-    "start_time = (datetime.now() - timedelta(days=7)).isoformat()\n",
-    "end_time = datetime.now().isoformat()\n",
-    "\n",
-    "temporal_results = temporal_query.query_time_range(\n",
-    "    graph=threat_kg,\n",
-    "    query=\"Find threats in the last 7 days\",\n",
-    "    start_time=start_time,\n",
-    "    end_time=end_time\n",
-    ")\n",
-    "\n",
-    "patterns = pattern_detector.detect_temporal_patterns(\n",
-    "    threat_kg,\n",
-    "    pattern_type=\"sequence\",\n",
-    "    min_frequency=1\n",
-    ")\n",
-    "\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(threat_kg)\n",
-    "\n",
-    "inference_engine.add_rule(\"IF IOC has high severity AND IOC is related to another IOC THEN potential_campaign\")\n",
-    "for entity in threat_entities[:3]:\n",
-    "    if entity.get(\"properties\", {}).get(\"severity\") == \"high\":\n",
-    "        inference_engine.add_fact({\"ioc\": entity.get(\"id\"), \"severity\": \"high\"})\n",
-    "\n",
-    "correlations = inference_engine.forward_chain()\n",
-    "\n",
-    "print(f\"Temporal query returned {len(temporal_results.get('entities', []))} entities\")\n",
-    "print(f\"Detected {len(patterns)} temporal patterns\")\n",
-    "print(f\"Connectivity: {connectivity.get('is_connected', False)}\")\n",
-    "print(f\"Inferred {len(correlations)} correlations\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Detect Campaigns\n",
-    "\n",
-    "Detect threat campaigns using graph analysis and inference.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "campaigns = []\n",
-    "\n",
-    "if len(patterns) > 0:\n",
-    "    campaigns.append({\n",
-    "        \"campaign_id\": \"campaign_1\",\n",
-    "        \"description\": \"Detected threat campaign based on temporal patterns\",\n",
-    "        \"iocs\": [e.get(\"id\") for e in threat_entities[:3]],\n",
-    "        \"severity\": \"high\",\n",
-    "        \"patterns\": len(patterns)\n",
-    "    })\n",
-    "\n",
-    "if correlations:\n",
-    "    campaigns.append({\n",
-    "        \"campaign_id\": \"campaign_2\",\n",
-    "        \"description\": \"Detected campaign from inference correlations\",\n",
-    "        \"iocs\": [e.get(\"id\") for e in threat_entities[:2]],\n",
-    "        \"severity\": \"medium\",\n",
-    "        \"correlations\": len(correlations)\n",
-    "    })\n",
-    "\n",
-    "print(f\"Detected {len(campaigns)} threat campaigns\")\n",
-    "for campaign in campaigns:\n",
-    "    print(f\"  Campaign: {campaign['campaign_id']} - Severity: {campaign['severity']}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 6: Quality Assessment and Provenance\n",
-    "\n",
-    "Assess graph quality and track provenance.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "quality_assessor = KGQualityAssessor()\n",
-    "provenance_tracker = ProvenanceTracker()\n",
-    "conflict_detector = ConflictDetector()\n",
-    "\n",
-    "quality_score = quality_assessor.assess_overall_quality(threat_kg)\n",
-    "\n",
-    "for entity in threat_entities:\n",
-    "    provenance_tracker.track_entity(entity.get(\"id\"), \"threat_feed\", entity)\n",
-    "\n",
-    "conflicts = conflict_detector.detect_value_conflicts(threat_entities, \"name\")\n",
-    "\n",
-    "print(f\"Graph quality score: {quality_score.get('overall_score', 0):.3f}\")\n",
-    "print(f\"Tracked provenance for {len(threat_entities)} entities\")\n",
-    "print(f\"Detected {len(conflicts)} conflicts\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 7: Generate Reports\n",
-    "\n",
-    "Generate threat intelligence reports.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "rdf_exporter = RDFExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-    "rdf_exporter.export_knowledge_graph(threat_kg, os.path.join(temp_dir, \"threats.rdf\"))\n",
-    "\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Threat correlation analysis detected {len(campaigns)} campaigns\",\n",
-    "    \"iocs\": len(threat_entities),\n",
-    "    \"campaigns\": campaigns,\n",
-    "    \"quality_score\": quality_score.get('overall_score', 0)\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"Report length: {len(report)} characters\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 8: Visualize Results\n",
-    "\n",
-    "Visualize threat correlation results.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(threat_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(threat_kg, output=\"interactive\")\n",
-    "\n",
-    "print(f\"Total modules used: 20+\")\n",
-    "print(f\"Pipeline complete: Multi-source ingestion → Extraction → Temporal KG → Correlation → Campaign Detection → Quality → Reports → Visualization\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/cybersecurity/04_Threat_Intelligence_Hybrid_RAG.ipynb

@@ -1,598 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/cybersecurity/04_Threat_Intelligence_Hybrid_RAG.ipynb)\n",
-    "\n",
-    "# Threat Intelligence Hybrid RAG Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete threat intelligence hybrid RAG pipeline: ingest threat intelligence from multiple sources (files, web, feeds), extract threat entities, build knowledge graph, generate embeddings, set up hybrid search (vector + temporal KG), and query threats using advanced RAG.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, DBIngestor, MCPIngestor\n",
-    "- **Parsing**: JSONParser, XMLParser, HTMLParser, DocumentParser, MCPParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "- **KG**: GraphBuilder, TemporalGraphQuery, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "- **Embeddings**: EmbeddingGenerator, TextEmbedder\n",
-    "- **Vector Store**: VectorStore, HybridSearch\n",
-    "- **Context**: ContextRetriever, ContextGraphBuilder\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "- **Export**: JSONExporter, RDFExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Multi-Source Threat Intel (Files, Web, Feeds, MCP) \u2192 Parse \u2192 Extract Entities \u2192 Build KG \u2192 Generate Embeddings \u2192 Vector Store \u2192 Hybrid RAG Setup \u2192 Query Threats \u2192 Generate Reports \u2192 Visualize**\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Multi-Source Threat Intelligence Ingestion\n",
-    "\n",
-    "Ingest threat intelligence from files, web sources, and feeds.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import FileIngestor, WebIngestor, FeedIngestor, DBIngestor, MCPIngestor, ingest_mcp\n",
-    "from semantica.parse import JSONParser, XMLParser, HTMLParser, DocumentParser, MCPParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "from semantica.kg import GraphBuilder, TemporalGraphQuery, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "from semantica.embeddings import EmbeddingGenerator, TextEmbedder\n",
-    "from semantica.vector_store import VectorStore, HybridSearch\n",
-    "from semantica.context import ContextRetriever, ContextGraphBuilder\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "from semantica.export import JSONExporter, RDFExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "file_ingestor = FileIngestor()\n",
-    "web_ingestor = WebIngestor()\n",
-    "feed_ingestor = FeedIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "mcp_ingestor = MCPIngestor()\n",
-    "\n",
-    "json_parser = JSONParser()\n",
-    "xml_parser = XMLParser()\n",
-    "html_parser = HTMLParser()\n",
-    "document_parser = DocumentParser()\n",
-    "mcp_parser = MCPParser()\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "# Real-world threat intelligence formats\n",
-    "threat_intel_json = os.path.join(temp_dir, \"threat_intel.json\")\n",
-    "threat_data = [\n",
-    "    {\n",
-    "        \"threat_id\": \"APT-001\",\n",
-    "        \"name\": \"Advanced Persistent Threat Group 1\",\n",
-    "        \"description\": \"State-sponsored APT group targeting financial institutions\",\n",
-    "        \"iocs\": [\"192.168.1.100\", \"malicious-domain.com\", \"hash_abc123\"],\n",
-    "        \"tactics\": [\"initial_access\", \"persistence\", \"exfiltration\"],\n",
-    "        \"timestamp\": (datetime.now() - timedelta(days=7)).isoformat(),\n",
-    "        \"severity\": \"high\"\n",
-    "    },\n",
-    "    {\n",
-    "        \"threat_id\": \"APT-002\",\n",
-    "        \"name\": \"Ransomware Campaign\",\n",
-    "        \"description\": \"Large-scale ransomware campaign targeting healthcare sector\",\n",
-    "        \"iocs\": [\"198.51.100.50\", \"ransomware-domain.net\", \"hash_def456\"],\n",
-    "        \"tactics\": [\"initial_access\", \"execution\", \"impact\"],\n",
-    "        \"timestamp\": (datetime.now() - timedelta(days=3)).isoformat(),\n",
-    "        \"severity\": \"critical\"\n",
-    "    },\n",
-    "    {\n",
-    "        \"threat_id\": \"APT-003\",\n",
-    "        \"name\": \"Phishing Campaign\",\n",
-    "        \"description\": \"Sophisticated phishing campaign using social engineering\",\n",
-    "        \"iocs\": [\"203.0.113.75\", \"phishing-site.org\", \"hash_ghi789\"],\n",
-    "        \"tactics\": [\"initial_access\", \"collection\"],\n",
-    "        \"timestamp\": (datetime.now() - timedelta(days=1)).isoformat(),\n",
-    "        \"severity\": \"medium\"\n",
-    "    }\n",
-    "]\n",
-    "\n",
-    "with open(threat_intel_json, 'w') as f:\n",
-    "    json.dump(threat_data, f, indent=2)\n",
-    "\n",
-    "# XML format threat intelligence (STIX format)\n",
-    "threat_intel_xml = os.path.join(temp_dir, \"threat_intel.xml\")\n",
-    "xml_content = \"\"\"<?xml version=\"1.0\"?>\n",
-    "<threat_intelligence>\n",
-    "    <indicator>\n",
-    "        <id>IOC-001</id>\n",
-    "        <type>IP</type>\n",
-    "        <value>172.16.0.50</value>\n",
-    "        <threat_type>malware</threat_type>\n",
-    "        <timestamp>2024-01-15T10:00:00</timestamp>\n",
-    "    </indicator>\n",
-    "    <indicator>\n",
-    "        <id>IOC-002</id>\n",
-    "        <type>Domain</type>\n",
-    "        <value>suspicious-domain.com</value>\n",
-    "        <threat_type>phishing</threat_type>\n",
-    "        <timestamp>2024-01-15T11:00:00</timestamp>\n",
-    "    </indicator>\n",
-    "</threat_intelligence>\"\"\"\n",
-    "\n",
-    "with open(threat_intel_xml, 'w') as f:\n",
-    "    f.write(xml_content)\n",
-    "\n",
-    "# Ingest from files\n",
-    "file_objects_json = file_ingestor.ingest_file(threat_intel_json, read_content=True)\n",
-    "file_objects_xml = file_ingestor.ingest_file(threat_intel_xml, read_content=True)\n",
-    "\n",
-    "# Parse threat intelligence\n",
-    "parsed_json = json_parser.parse(threat_intel_json)\n",
-    "parsed_xml = xml_parser.parse(threat_intel_xml)\n",
-    "\n",
-    "# Real threat intelligence feed URLs\n",
-    "threat_intel_feeds = [\n",
-    "    \"https://www.cisa.gov/news.xml\",  # CISA Security Advisories\n",
-    "    \"https://www.us-cert.gov/ncas/alerts.xml\",  # US-CERT Alerts\n",
-    "    \"https://feeds.feedburner.com/SecurityWeek\",  # Security Week\n",
-    "    \"https://www.darkreading.com/rss.xml\",  # Dark Reading\n",
-    "    \"https://krebsonsecurity.com/feed/\"  # Krebs on Security\n",
-    "]\n",
-    "\n",
-    "threat_feed_list = []\n",
-    "for feed_url in threat_intel_feeds:\n",
-    "    threat_feed = feed_ingestor.ingest_feed(feed_url)\n",
-    "    if threat_feed:\n",
-    "        threat_feed_list.append(threat_feed)\n",
-    "        print(f\"  Ingested feed: {feed_url}\")\n",
-    "        print(f\"  Items: {len(threat_feed.items) if hasattr(threat_feed, 'items') else 0}\")\n",
-    "\n",
-    "# Real web sources for threat intelligence\n",
-    "threat_web_sources = [\n",
-    "    \"https://api.github.com/repos/mitre/cti\",  # MITRE ATT&CK Framework\n",
-    "    \"https://www.cisa.gov/known-exploited-vulnerabilities-catalog\",  # CISA KEV Catalog\n",
-    "    \"https://nvd.nist.gov/vuln/search\"  # NIST NVD\n",
-    "]\n",
-    "\n",
-    "web_content_list = []\n",
-    "for web_url in threat_web_sources[:1]:  # Process first URL\n",
-    "    web_content = web_ingestor.ingest_url(web_url)\n",
-    "    if web_content:\n",
-    "        web_content_list.append(web_content)\n",
-    "        print(f\"  Ingested web source: {web_url}\")\n",
-    "\n",
-    "# Optional: Ingest from MCP server\n",
-    "# Users can bring their own threat intelligence MCP server via URL\n",
-    "mcp_threat_data = []\n",
-    "# Connect to threat intelligence MCP server via URL\n",
-    "# Example: http://localhost:8000/mcp or https://api.example.com/threat-mcp\n",
-    "threat_mcp_url = \"http://localhost:8000/mcp\"  # Replace with your MCP server URL\n",
-    "\n",
-    "mcp_ingestor.connect(\n",
-    "    \"threat_mcp_server\",\n",
-    "    url=threat_mcp_url,\n",
-    "    headers={\n",
-    "        \"Authorization\": \"Bearer your_token\",\n",
-    "        \"X-API-Key\": \"your_api_key\"\n",
-    "    } if \"api.example.com\" in threat_mcp_url else {}\n",
-    ")\n",
-    "\n",
-    "# Ingest threat indicators from MCP server\n",
-    "mcp_data = mcp_ingestor.ingest_resources(\n",
-    "    \"threat_mcp_server\",\n",
-    "    resource_uris=[\"resource://threats/feed\", \"resource://vulnerabilities/database\"]\n",
-    ")\n",
-    "mcp_threat_data.extend(mcp_data)\n",
-    "print(f\"  Ingested MCP resources: {len(mcp_data)}\")\n",
-    "\n",
-    "# Or use tool-based ingestion to query threat indicators\n",
-    "tool_data = mcp_ingestor.ingest_tool_output(\n",
-    "    \"threat_mcp_server\",\n",
-    "    tool_name=\"query_threat_indicators\",\n",
-    "    arguments={\n",
-    "        \"indicator_type\": \"IP\",\n",
-    "        \"date_range\": {\n",
-    "            \"start\": (datetime.now() - timedelta(days=7)).isoformat(),\n",
-    "            \"end\": datetime.now().isoformat()\n",
-    "        }\n",
-    "    }\n",
-    ")\n",
-    "if tool_data:\n",
-    "    mcp_threat_data.append(tool_data)\n",
-    "    print(f\"  Retrieved tool data\")\n",
-    "\n",
-    "# Parse MCP responses and merge with existing threat data\n",
-    "for mcp_item in mcp_threat_data:\n",
-    "    parsed_mcp = mcp_parser.parse_response(mcp_item, response_type=\"json\")\n",
-    "    if isinstance(parsed_mcp, dict):\n",
-    "        if \"threat_indicators\" in parsed_mcp:\n",
-    "            # Merge threat indicators from MCP\n",
-    "            if parsed_json and parsed_json.data:\n",
-    "                parsed_json.data.extend(parsed_mcp.get(\"threat_indicators\", []))\n",
-    "            else:\n",
-    "                parsed_json.data = parsed_mcp.get(\"threat_indicators\", [])\n",
-    "        elif \"threat_id\" in parsed_mcp:\n",
-    "            # Single threat indicator\n",
-    "            if parsed_json and parsed_json.data:\n",
-    "                parsed_json.data.append(parsed_mcp)\n",
-    "            else:\n",
-    "                parsed_json.data = [parsed_mcp]\n",
-    "        print(f\"  Parsed MCP item\")\n",
-    "\n",
-    "mcp_ingestor.disconnect(\"threat_mcp_server\")\n",
-    "print(f\"  Disconnected from MCP server\")\n",
-    "\n",
-    "print(f\"Ingested {len([file_objects_json]) if file_objects_json else 0} JSON threat intelligence files\")\n",
-    "print(f\"Ingested {len([file_objects_xml]) if file_objects_xml else 0} XML threat intelligence files\")\n",
-    "print(f\"Parsed {len(parsed_json.data) if parsed_json and parsed_json.data else 0} JSON threat entries\")\n",
-    "print(f\"Parsed {len(parsed_xml.elements) if parsed_xml else 0} XML indicator elements\")\n",
-    "print(f\"MCP server sources: {len(mcp_threat_data)}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Extract Threat Intelligence Entities\n",
-    "\n",
-    "Extract threat entities, IOCs, and relationships from threat intelligence data.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "event_detector = EventDetector()\n",
-    "triplet_extractor = TripletExtractor()\n",
-    "\n",
-    "threat_entities = []\n",
-    "threat_relationships = []\n",
-    "all_documents = []\n",
-    "\n",
-    "# Extract from JSON threat intelligence\n",
-    "if parsed_json and parsed_json.data:\n",
-    "    for threat in parsed_json.data:\n",
-    "        if isinstance(threat, dict):\n",
-    "            threat_text = f\"{threat.get('name', '')}: {threat.get('description', '')}\"\n",
-    "            all_documents.append(threat_text)\n",
-    "            \n",
-    "            threat_entities.append({\n",
-    "                \"id\": threat.get(\"threat_id\", \"\"),\n",
-    "                \"type\": \"Threat_Actor\",\n",
-    "                \"name\": threat.get(\"name\", \"\"),\n",
-    "                \"properties\": {\n",
-    "                    \"description\": threat.get(\"description\", \"\"),\n",
-    "                    \"severity\": threat.get(\"severity\", \"\"),\n",
-    "                    \"timestamp\": threat.get(\"timestamp\", \"\")\n",
-    "                }\n",
-    "            })\n",
-    "            \n",
-    "            for ioc in threat.get(\"iocs\", []):\n",
-    "                threat_entities.append({\n",
-    "                    \"id\": ioc,\n",
-    "                    \"type\": \"IOC\",\n",
-    "                    \"name\": ioc,\n",
-    "                    \"properties\": {\n",
-    "                        \"threat_id\": threat.get(\"threat_id\", \"\"),\n",
-    "                        \"timestamp\": threat.get(\"timestamp\", \"\")\n",
-    "                    }\n",
-    "                })\n",
-    "                threat_relationships.append({\n",
-    "                    \"source\": threat.get(\"threat_id\", \"\"),\n",
-    "                    \"target\": ioc,\n",
-    "                    \"type\": \"uses\",\n",
-    "                    \"properties\": {\"timestamp\": threat.get(\"timestamp\", \"\")}\n",
-    "                })\n",
-    "            \n",
-    "            for tactic in threat.get(\"tactics\", []):\n",
-    "                threat_entities.append({\n",
-    "                    \"id\": tactic,\n",
-    "                    \"type\": \"Tactic\",\n",
-    "                    \"name\": tactic,\n",
-    "                    \"properties\": {}\n",
-    "                })\n",
-    "                threat_relationships.append({\n",
-    "                    \"source\": threat.get(\"threat_id\", \"\"),\n",
-    "                    \"target\": tactic,\n",
-    "                    \"type\": \"employs\",\n",
-    "                    \"properties\": {}\n",
-    "                })\n",
-    "\n",
-    "# Extract from XML indicators\n",
-    "if parsed_xml and parsed_xml.elements:\n",
-    "    for elem in parsed_xml.elements:\n",
-    "        if hasattr(elem, 'text') and elem.text:\n",
-    "            entities = ner_extractor.extract(elem.text)\n",
-    "            threat_entities.extend(entities)\n",
-    "\n",
-    "print(f\"Extracted {len(threat_entities)} threat intelligence entities\")\n",
-    "print(f\"Extracted {len(threat_relationships)} threat relationships\")\n",
-    "print(f\"Collected {len(all_documents)} threat intelligence documents\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Build Threat Intelligence Knowledge Graph\n",
-    "\n",
-    "Build knowledge graph from threat entities and relationships.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "temporal_query = TemporalGraphQuery()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "\n",
-    "threat_kg = builder.build(threat_entities, threat_relationships)\n",
-    "\n",
-    "# Analyze graph structure\n",
-    "metrics = graph_analyzer.compute_metrics(threat_kg)\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(threat_kg)\n",
-    "\n",
-    "print(f\"Built threat intelligence knowledge graph\")\n",
-    "print(f\"  Entities: {len(threat_kg.get('entities', []))}\")\n",
-    "print(f\"  Relationships: {len(threat_kg.get('relationships', []))}\")\n",
-    "print(f\"  Graph density: {metrics.get('density', 0):.3f}\")\n",
-    "print(f\"  Connected components: {len(connectivity.get('components', []))}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Generate Embeddings and Setup Vector Store\n",
-    "\n",
-    "Generate embeddings from threat intelligence documents and store in vector database.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "embedding_generator = EmbeddingGenerator()\n",
-    "text_embedder = TextEmbedder()\n",
-    "vector_store = VectorStore()\n",
-    "hybrid_search = HybridSearch()\n",
-    "\n",
-    "# Generate embeddings for threat intelligence documents\n",
-    "embeddings = embedding_generator.generate_embeddings(all_documents, data_type=\"text\")\n",
-    "\n",
-    "# Prepare metadata for vector store\n",
-    "metadata = []\n",
-    "for i, doc in enumerate(all_documents):\n",
-    "    metadata.append({\n",
-    "        \"id\": f\"doc_{i}\",\n",
-    "        \"text\": doc,\n",
-    "        \"source\": \"threat_intelligence\"\n",
-    "    })\n",
-    "\n",
-    "# Store vectors\n",
-    "vector_ids = vector_store.store_vectors(embeddings, metadata)\n",
-    "\n",
-    "print(f\"Generated embeddings for {len(all_documents)} documents\")\n",
-    "print(f\"Stored {len(vector_ids)} vectors in vector store\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Setup Hybrid RAG (Vector + Temporal KG)\n",
-    "\n",
-    "Setup hybrid search combining vector similarity and temporal knowledge graph queries.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "context_retriever = ContextRetriever()\n",
-    "context_graph_builder = ContextGraphBuilder()\n",
-    "\n",
-    "# Setup context retriever with KG and vector store\n",
-    "context_retriever = ContextRetriever(\n",
-    "    knowledge_graph=threat_kg,\n",
-    "    vector_store=vector_store\n",
-    ")\n",
-    "\n",
-    "print(f\"  Knowledge graph: {len(threat_kg.get('entities', []))} entities\")\n",
-    "print(f\"  Vector store: {len(vector_ids)} vectors\")\n",
-    "print(f\"  Context retriever initialized\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 6: Query Threats Using Hybrid RAG\n",
-    "\n",
-    "Query threat intelligence using hybrid search (vector + temporal KG).\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Query examples\n",
-    "queries = [\n",
-    "    \"What are the latest APT threats?\",\n",
-    "    \"Find threats targeting financial institutions\",\n",
-    "    \"What IOCs are associated with ransomware?\"\n",
-    "]\n",
-    "\n",
-    "query_results = []\n",
-    "\n",
-    "for query in queries:\n",
-    "    # Generate query embedding\n",
-    "    query_embedding = text_embedder.embed_text(query)\n",
-    "    \n",
-    "    # Vector search\n",
-    "    vector_results = vector_store.search_vectors(query_embedding, k=3)\n",
-    "    \n",
-    "    # Temporal KG query\n",
-    "    start_time = (datetime.now() - timedelta(days=30)).isoformat()\n",
-    "    end_time = datetime.now().isoformat()\n",
-    "    \n",
-    "    temporal_results = temporal_query.query_time_range(\n",
-    "        graph=threat_kg,\n",
-    "        query=query,\n",
-    "        start_time=start_time,\n",
-    "        end_time=end_time\n",
-    "    )\n",
-    "    \n",
-    "    # Hybrid search using context retriever\n",
-    "    context_results = context_retriever.retrieve(\n",
-    "        query=query,\n",
-    "        top_k=3,\n",
-    "        use_graph_expansion=True\n",
-    "    )\n",
-    "    \n",
-    "    query_results.append({\n",
-    "        \"query\": query,\n",
-    "        \"vector_results\": len(vector_results),\n",
-    "        \"temporal_results\": len(temporal_results.get('entities', [])),\n",
-    "        \"context_results\": len(context_results) if context_results else 0\n",
-    "    })\n",
-    "\n",
-    "# Inference for threat analysis\n",
-    "inference_engine = InferenceEngine()\n",
-    "rule_manager = RuleManager()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-    "\n",
-    "inference_engine.add_rule(\"IF severity is critical AND tactics includes exfiltration THEN high_priority_threat\")\n",
-    "inference_engine.add_rule(\"IF threat targets financial AND uses initial_access THEN financial_apt\")\n",
-    "\n",
-    "for threat in parsed_json.data if parsed_json and parsed_json.data else []:\n",
-    "    if isinstance(threat, dict):\n",
-    "        inference_engine.add_fact({\n",
-    "            \"threat_id\": threat.get(\"threat_id\", \"\"),\n",
-    "            \"severity\": threat.get(\"severity\", \"\"),\n",
-    "            \"tactics\": threat.get(\"tactics\", [])\n",
-    "        })\n",
-    "\n",
-    "threat_insights = inference_engine.forward_chain()\n",
-    "\n",
-    "print(f\"Processed {len(queries)} threat intelligence queries\")\n",
-    "for result in query_results:\n",
-    "    print(f\"  Query: '{result['query']}' - Vector: {result['vector_results']}, Temporal: {result['temporal_results']}, Context: {result['context_results']}\")\n",
-    "print(f\"Generated {len(threat_insights)} threat insights from inference\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": []
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-
-    "json_exporter = JSONExporter()\n",
-    "rdf_exporter = RDFExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-
-    "\n",
-    "json_exporter.export_knowledge_graph(threat_kg, os.path.join(temp_dir, \"threat_kg.json\"))\n",
-    "rdf_exporter.export_knowledge_graph(threat_kg, os.path.join(temp_dir, \"threat_kg.rdf\"))\n",
-    "\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Threat intelligence analysis identified {len(threat_entities)} entities and {len(threat_insights)} insights\",\n",
-    "    \"threats_analyzed\": len(parsed_json.data) if parsed_json and parsed_json.data else 0,\n",
-    "    \"iocs\": len([e for e in threat_entities if e.get(\"type\") == \"IOC\"]),\n",
-    "    \"insights\": len(threat_insights),\n",
-    "    \"quality_score\": 0.95,\n",
-    "    \"critical_threats\": len([t for t in parsed_json.data if isinstance(t, dict) and t.get(\"severity\") == \"critical\"]) if parsed_json and parsed_json.data else 0\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"Report length: {len(report)} characters\")\n",
-    "print(f\"Graph quality score: 0.950\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 8: Visualize Threat Intelligence\n",
-    "\n",
-    "Visualize threat intelligence knowledge graph and relationships.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "kg_visualizer = KGVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(threat_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(threat_kg, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(threat_kg, output=\"interactive\")\n",
-    "\n",
-    "print(f\"Total modules used: 20+\")\n",
-    "print(f\"Pipeline complete: Multi-Source Threat Intel \u2192 Parse \u2192 Extract \u2192 Build KG \u2192 Embeddings \u2192 Vector Store \u2192 Hybrid RAG \u2192 Query \u2192 Reports \u2192 Visualize\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/cybersecurity/05_Threat_Intelligence_Integration.ipynb

@@ -1,560 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/cybersecurity/05_Threat_Intelligence_Integration.ipynb)\n",
-    "\n",
-    "# Threat Intelligence Integration Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates how to integrate Python/FastMCP MCP servers as data sources for threat intelligence ingestion. Connect to threat intelligence MCP servers via URL, ingest threat feeds, vulnerability data, and security events, then build a threat intelligence knowledge graph.\n",
-    "\n",
-    "**IMPORTANT**: This implementation supports ONLY Python-based MCP servers and FastMCP servers. Users can bring their own Python/FastMCP MCP servers via URL connections.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: MCPIngestor, ingest_mcp, WebIngestor, FeedIngestor\n",
-    "- **Parsing**: MCPParser, JSONParser, XMLParser, StructuredDataParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "- **KG**: GraphBuilder, TemporalGraphQuery, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "- **Embeddings**: EmbeddingGenerator, TextEmbedder\n",
-    "- **Vector Store**: VectorStore, HybridSearch\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "- **Export**: JSONExporter, RDFExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Connect to Threat Intel MCP Server \u2192 Ingest Threat Data via MCP \u2192 Parse MCP Responses \u2192 Extract Threat Entities \u2192 Build Threat KG \u2192 Generate Embeddings \u2192 Hybrid RAG \u2192 Analyze Threats \u2192 Generate Reports \u2192 Visualize**\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Connect to Threat Intelligence MCP Server\n",
-    "\n",
-    "Connect to a Python/FastMCP MCP server that provides threat intelligence data via URL. The MCP server can expose resources (threat feeds, vulnerability databases) and tools (threat queries, IOC checks).\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import MCPIngestor, ingest_mcp\n",
-    "from semantica.parse import MCPParser, JSONParser, XMLParser, StructuredDataParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "from semantica.kg import GraphBuilder, TemporalGraphQuery, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "from semantica.embeddings import EmbeddingGenerator, TextEmbedder\n",
-    "from semantica.vector_store import VectorStore, HybridSearch\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "from semantica.export import JSONExporter, RDFExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "# Initialize MCP ingestor\n",
-    "mcp_ingestor = MCPIngestor()\n",
-    "\n",
-    "# Connect to threat intelligence MCP server via URL\n",
-    "# Replace with your actual MCP server URL\n",
-    "# Example: http://localhost:8000/mcp or https://api.example.com/threat-mcp\n",
-    "threat_mcp_url = \"http://localhost:8000/mcp\"\n",
-    "\n",
-    "# Connect to MCP server with authentication (if required)\n",
-    "mcp_ingestor.connect(\n",
-    "    \"threat_server\",\n",
-    "    url=threat_mcp_url,\n",
-    "    headers={\n",
-    "        \"Authorization\": \"Bearer your_token\",\n",
-    "        \"X-API-Key\": \"your_api_key\"\n",
-    "    } if \"api.example.com\" in threat_mcp_url else {}\n",
-    ")\n",
-    "\n",
-    "# List available resources (threat feeds, vulnerability databases)\n",
-    "resources = mcp_ingestor.list_available_resources(\"threat_server\")\n",
-    "print(f\"\\n\ud83d\udcca Available Resources ({len(resources)}):\")\n",
-    "for resource in resources[:5]:  # Show first 5\n",
-    "    print(f\"  - {resource.uri}: {resource.name}\")\n",
-    "    if resource.description:\n",
-    "        print(f\"    {resource.description[:80]}...\")\n",
-    "\n",
-    "# List available tools (threat queries, IOC checks)\n",
-    "tools = mcp_ingestor.list_available_tools(\"threat_server\")\n",
-    "print(f\"\\n\ud83d\udd27 Available Tools ({len(tools)}):\")\n",
-    "for tool in tools[:5]:  # Show first 5\n",
-    "    print(f\"  - {tool.name}: {tool.description or 'No description'}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Ingest Threat Intelligence Data from MCP Server\n",
-    "\n",
-    "Ingest threat feeds, vulnerability data, and security events using both resource-based and tool-based methods.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Initialize parsers\n",
-    "mcp_parser = MCPParser()\n",
-    "json_parser = JSONParser()\n",
-    "xml_parser = XMLParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "\n",
-    "threat_data = []\n",
-    "\n",
-    "# Method 1: Resource-based ingestion\n",
-    "# Ingest from MCP resources (threat feeds, vulnerability databases)\n",
-    "threat_feeds = mcp_ingestor.ingest_resources(\n",
-    "    \"threat_server\",\n",
-    "    resource_uris=[\"resource://threats/feed\", \"resource://vulnerabilities/database\"]\n",
-    ")\n",
-    "\n",
-    "for item in threat_feeds:\n",
-    "    threat_data.append(item)\n",
-    "    print(f\"  Ingested resource: {item}\")\n",
-    "\n",
-    "# Method 2: Tool-based ingestion\n",
-    "# Call MCP tools to retrieve data dynamically\n",
-    "# Example: Query threat indicators\n",
-    "threat_indicators = mcp_ingestor.ingest_tool_output(\n",
-    "    \"threat_server\",\n",
-    "    tool_name=\"query_threat_indicators\",\n",
-    "    arguments={\n",
-    "        \"indicator_type\": \"IP\",\n",
-    "        \"date_range\": {\n",
-    "            \"start\": (datetime.now() - timedelta(days=7)).isoformat(),\n",
-    "            \"end\": datetime.now().isoformat()\n",
-    "        }\n",
-    "    }\n",
-    ")\n",
-    "\n",
-    "if threat_indicators:\n",
-    "    threat_data.append(threat_indicators)\n",
-    "    print(f\"  Retrieved threat indicators\")\n",
-    "\n",
-    "# Example: Check IOC (Indicators of Compromise)\n",
-    "ioc_check = mcp_ingestor.ingest_tool_output(\n",
-    "    \"threat_server\",\n",
-    "    tool_name=\"check_ioc\",\n",
-    "    arguments={\n",
-    "        \"ioc_type\": \"hash\",\n",
-    "        \"ioc_value\": \"abc123def456\"\n",
-    "    }\n",
-    ")\n",
-    "\n",
-    "if ioc_check:\n",
-    "    threat_data.append(ioc_check)\n",
-    "    print(f\"  Retrieved IOC check results\")\n",
-    "\n",
-    "# Sample threat intelligence data (if MCP server is not available)\n",
-    "if not threat_data:\n",
-    "    sample_data = {\n",
-    "        \"threat_indicators\": [\n",
-    "            {\n",
-    "                \"indicator_id\": \"TI001\",\n",
-    "                \"indicator_type\": \"IP\",\n",
-    "                \"indicator_value\": \"192.168.1.100\",\n",
-    "                \"threat_type\": \"malware\",\n",
-    "                \"severity\": \"high\",\n",
-    "                \"timestamp\": (datetime.now() - timedelta(days=1)).isoformat(),\n",
-    "                \"source\": \"ThreatFeed1\"\n",
-    "            },\n",
-    "            {\n",
-    "                \"indicator_id\": \"TI002\",\n",
-    "                \"indicator_type\": \"domain\",\n",
-    "                \"indicator_value\": \"malicious.example.com\",\n",
-    "                \"threat_type\": \"phishing\",\n",
-    "                \"severity\": \"medium\",\n",
-    "                \"timestamp\": (datetime.now() - timedelta(hours=12)).isoformat(),\n",
-    "                \"source\": \"ThreatFeed2\"\n",
-    "            },\n",
-    "            {\n",
-    "                \"indicator_id\": \"TI003\",\n",
-    "                \"indicator_type\": \"hash\",\n",
-    "                \"indicator_value\": \"abc123def456\",\n",
-    "                \"threat_type\": \"ransomware\",\n",
-    "                \"severity\": \"critical\",\n",
-    "                \"timestamp\": datetime.now().isoformat(),\n",
-    "                \"source\": \"ThreatFeed1\"\n",
-    "            }\n",
-    "        ],\n",
-    "        \"vulnerabilities\": [\n",
-    "            {\n",
-    "                \"cve_id\": \"CVE-2024-0001\",\n",
-    "                \"description\": \"Remote code execution vulnerability\",\n",
-    "                \"severity\": \"critical\",\n",
-    "                \"affected_products\": [\"Product A\", \"Product B\"],\n",
-    "                \"published_date\": (datetime.now() - timedelta(days=5)).isoformat()\n",
-    "            }\n",
-    "        ]\n",
-    "    }\n",
-    "    threat_data.append(sample_data)\n",
-    "    print(f\"  Loaded {len(sample_data['threat_indicators'])} threat indicators\")\n",
-    "    print(f\"  Loaded {len(sample_data['vulnerabilities'])} vulnerabilities\")\n",
-    "\n",
-    "print(f\"\\n\ud83d\udcca Total threat intelligence data items ingested: {len(threat_data)}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Parse Threat Intelligence Data\n",
-    "\n",
-    "Parse the threat intelligence data received from MCP server responses.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "parsed_threat_data = []\n",
-    "\n",
-    "# Parse MCP responses\n",
-    "for data_item in threat_data:\n",
-    "    # Parse MCP response (handles JSON, XML, text, binary)\n",
-    "    if isinstance(data_item, dict):\n",
-    "        parsed_item = data_item\n",
-    "    else:\n",
-    "        parsed_item = mcp_parser.parse_response(data_item, response_type=\"json\")\n",
-    "    \n",
-    "    parsed_threat_data.append(parsed_item)\n",
-    "    print(f\"  Parsed data item\")\n",
-    "\n",
-    "# Extract threat indicators and vulnerabilities\n",
-    "threat_indicators = []\n",
-    "vulnerabilities = []\n",
-    "\n",
-    "for item in parsed_threat_data:\n",
-    "    if isinstance(item, dict):\n",
-    "        if \"threat_indicators\" in item:\n",
-    "            threat_indicators.extend(item[\"threat_indicators\"])\n",
-    "        elif \"indicator_id\" in item:\n",
-    "            threat_indicators.append(item)\n",
-    "        elif \"vulnerabilities\" in item:\n",
-    "            vulnerabilities.extend(item[\"vulnerabilities\"])\n",
-    "        elif \"cve_id\" in item:\n",
-    "            vulnerabilities.append(item)\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Extract Threat Entities and Relationships\n",
-    "\n",
-    "Extract threat entities (indicators, vulnerabilities, threat actors) and relationships from MCP data.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "event_detector = EventDetector()\n",
-    "triplet_extractor = TripletExtractor()\n",
-    "\n",
-    "threat_entities = []\n",
-    "threat_relationships = []\n",
-    "\n",
-    "# Extract from threat indicators\n",
-    "for indicator in threat_indicators:\n",
-    "    if isinstance(indicator, dict):\n",
-    "        indicator_id = indicator.get(\"indicator_id\", \"\")\n",
-    "        indicator_type = indicator.get(\"indicator_type\", \"\")\n",
-    "        threat_type = indicator.get(\"threat_type\", \"\")\n",
-    "        source = indicator.get(\"source\", \"\")\n",
-    "        \n",
-    "        # Threat Indicator entity\n",
-    "        threat_entities.append({\n",
-    "            \"id\": indicator_id,\n",
-    "            \"type\": \"ThreatIndicator\",\n",
-    "            \"name\": indicator_id,\n",
-    "            \"properties\": {\n",
-    "                \"indicator_type\": indicator_type,\n",
-    "                \"indicator_value\": indicator.get(\"indicator_value\", \"\"),\n",
-    "                \"threat_type\": threat_type,\n",
-    "                \"severity\": indicator.get(\"severity\", \"\"),\n",
-    "                \"timestamp\": indicator.get(\"timestamp\", \"\"),\n",
-    "                \"source\": source\n",
-    "            }\n",
-    "        })\n",
-    "        \n",
-    "        # Threat Type entity\n",
-    "        if threat_type:\n",
-    "            threat_entities.append({\n",
-    "                \"id\": threat_type,\n",
-    "                \"type\": \"ThreatType\",\n",
-    "                \"name\": threat_type,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            threat_relationships.append({\n",
-    "                \"source\": indicator_id,\n",
-    "                \"target\": threat_type,\n",
-    "                \"type\": \"classified_as\",\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "        \n",
-    "        # Source entity\n",
-    "        if source:\n",
-    "            threat_entities.append({\n",
-    "                \"id\": source,\n",
-    "                \"type\": \"ThreatSource\",\n",
-    "                \"name\": source,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            threat_relationships.append({\n",
-    "                \"source\": indicator_id,\n",
-    "                \"target\": source,\n",
-    "                \"type\": \"reported_by\",\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "\n",
-    "# Extract from vulnerabilities\n",
-    "for vuln in vulnerabilities:\n",
-    "    if isinstance(vuln, dict):\n",
-    "        cve_id = vuln.get(\"cve_id\", \"\")\n",
-    "        \n",
-    "        # Vulnerability entity\n",
-    "        threat_entities.append({\n",
-    "            \"id\": cve_id,\n",
-    "            \"type\": \"Vulnerability\",\n",
-    "            \"name\": cve_id,\n",
-    "            \"properties\": {\n",
-    "                \"description\": vuln.get(\"description\", \"\"),\n",
-    "                \"severity\": vuln.get(\"severity\", \"\"),\n",
-    "                \"published_date\": vuln.get(\"published_date\", \"\")\n",
-    "            }\n",
-    "        })\n",
-    "        \n",
-    "        # Affected products\n",
-    "        for product in vuln.get(\"affected_products\", []):\n",
-    "            threat_entities.append({\n",
-    "                \"id\": product,\n",
-    "                \"type\": \"Product\",\n",
-    "                \"name\": product,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            threat_relationships.append({\n",
-    "                \"source\": cve_id,\n",
-    "                \"target\": product,\n",
-    "                \"type\": \"affects\",\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "\n",
-    "# Remove duplicates\n",
-    "seen_entities = set()\n",
-    "unique_entities = []\n",
-    "for entity in threat_entities:\n",
-    "    entity_key = (entity[\"id\"], entity[\"type\"])\n",
-    "    if entity_key not in seen_entities:\n",
-    "        seen_entities.add(entity_key)\n",
-    "        unique_entities.append(entity)\n",
-    "\n",
-    "threat_entities = unique_entities\n",
-    "\n",
-    "print(f\"  - Threat Indicators: {len([e for e in threat_entities if e['type'] == 'ThreatIndicator'])}\")\n",
-    "print(f\"  - Vulnerabilities: {len([e for e in threat_entities if e['type'] == 'Vulnerability'])}\")\n",
-    "print(f\"  - Threat Types: {len([e for e in threat_entities if e['type'] == 'ThreatType'])}\")\n",
-    "print(f\"  - Sources: {len([e for e in threat_entities if e['type'] == 'ThreatSource'])}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Build Threat Intelligence Knowledge Graph\n",
-    "\n",
-    "Build a temporal knowledge graph from the extracted threat entities and relationships.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "temporal_query = TemporalGraphQuery()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "\n",
-    "# Build knowledge graph\n",
-    "threat_kg = builder.build(threat_entities, threat_relationships)\n",
-    "\n",
-    "# Analyze graph structure\n",
-    "metrics = graph_analyzer.compute_metrics(threat_kg)\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(threat_kg)\n",
-    "\n",
-    "print(f\"  Entities: {len(threat_kg.get('entities', []))}\")\n",
-    "print(f\"  Relationships: {len(threat_kg.get('relationships', []))}\")\n",
-    "print(f\"  Graph density: {metrics.get('density', 0):.3f}\")\n",
-    "print(f\"  Connectivity: {connectivity.get('connected_components', 0)} components\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 6: Generate Embeddings and Set Up Hybrid RAG\n",
-    "\n",
-    "Generate embeddings for threat intelligence data and set up hybrid search (vector + temporal KG).\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Generate embeddings\n",
-    "embedding_generator = EmbeddingGenerator()\n",
-    "text_embedder = TextEmbedder()\n",
-    "\n",
-    "# Generate embeddings for threat entities\n",
-    "threat_texts = []\n",
-    "for entity in threat_entities:\n",
-    "    if entity.get(\"type\") == \"ThreatIndicator\":\n",
-    "        text = f\"{entity.get('properties', {}).get('indicator_value', '')} {entity.get('properties', {}).get('threat_type', '')} {entity.get('properties', {}).get('description', '')}\"\n",
-    "        threat_texts.append(text)\n",
-    "\n",
-    "embeddings = embedding_generator.generate_embeddings(threat_texts)\n",
-    "\n",
-    "# Set up vector store\n",
-    "vector_store = VectorStore()\n",
-    "vector_store.add_embeddings(threat_texts, embeddings)\n",
-    "\n",
-    "# Set up hybrid search (vector + temporal KG)\n",
-    "hybrid_search = HybridSearch()\n",
-    "hybrid_search.setup(vector_store, threat_kg)\n",
-    "\n",
-    "\n",
-    "# Inference engine for threat analysis\n",
-    "inference_engine = InferenceEngine()\n",
-    "rule_manager = RuleManager()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-    "\n",
-    "# Threat analysis rules\n",
-    "inference_engine.add_rule(\"IF severity(critical) AND threat_type(ransomware) THEN immediate_response_required\")\n",
-    "inference_engine.add_rule(\"IF severity(high) AND indicator_type(IP) THEN block_ip\")\n",
-    "\n",
-    "# Add facts from threat data\n",
-    "for indicator in threat_indicators:\n",
-    "    if isinstance(indicator, dict):\n",
-    "        inference_engine.add_fact({\n",
-    "            \"indicator_id\": indicator.get(\"indicator_id\", \"\"),\n",
-    "            \"severity\": indicator.get(\"severity\", \"\"),\n",
-    "            \"threat_type\": indicator.get(\"threat_type\", \"\"),\n",
-    "            \"indicator_type\": indicator.get(\"indicator_type\", \"\")\n",
-    "        })\n",
-    "\n",
-    "# Generate threat insights\n",
-    "threat_insights = inference_engine.forward_chain()\n",
-    "\n",
-    "print(f\"  Threat insights: {len(threat_insights)}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 7: Export and Visualize\n",
-    "\n",
-    "Export the threat intelligence knowledge graph and generate visualizations.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import tempfile\n",
-    "import os\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "json_exporter = JSONExporter()\n",
-    "rdf_exporter = RDFExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-    "# Export knowledge graph\n",
-    "json_exporter.export_knowledge_graph(threat_kg, os.path.join(temp_dir, \"threat_kg.json\"))\n",
-    "rdf_exporter.export_knowledge_graph(threat_kg, os.path.join(temp_dir, \"threat_kg.rdf\"))\n",
-    "\n",
-    "# Generate report\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Threat intelligence integration from MCP server identified {len(threat_insights)} insights\",\n",
-    "    \"threat_indicators\": len([e for e in threat_entities if e['type'] == 'ThreatIndicator']),\n",
-    "    \"vulnerabilities\": len([e for e in threat_entities if e['type'] == 'Vulnerability']),\n",
-    "    \"threat_types\": len([e for e in threat_entities if e['type'] == 'ThreatType']),\n",
-    "    \"insights\": len(threat_insights)\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"  JSON: {os.path.join(temp_dir, 'threat_kg.json')}\")\n",
-    "print(f\"  RDF: {os.path.join(temp_dir, 'threat_kg.rdf')}\")\n",
-    "\n",
-    "# Visualize\n",
-    "kg_visualizer = KGVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(threat_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(threat_kg, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(threat_kg, output=\"interactive\")\n",
-    "\n",
-    "\n",
-    "# Cleanup: Disconnect from MCP server\n",
-    "mcp_ingestor.disconnect(\"threat_server\")\n",
-    "print(\"  Disconnected from MCP server\")\n",
-    "\n",
-    "print(f\"\ud83d\udcca Total modules used: 20+\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/cybersecurity/06_Vulnerability_Tracking.ipynb

@@ -1,155 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/cybersecurity/06_Vulnerability_Tracking.ipynb)\n",
-    "\n",
-    "# Vulnerability Tracking Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete vulnerability tracking pipeline: ingest CVE data from multiple real sources (NVD, CVE feeds, security databases), build temporal knowledge graph, correlate vulnerabilities, predict impact, and generate vulnerability reports.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, RepoIngestor, EmailIngestor, MCPIngestor\n",
-    "- **Parsing**: JSONParser, XMLParser, StructuredDataParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "- **KG**: GraphBuilder, TemporalGraphQuery, TemporalPatternDetector, GraphAnalyzer\n",
-    "- **Analytics**: CentralityCalculator, CommunityDetector, ConnectivityAnalyzer\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-
-    "- **Export**: JSONExporter, RDFExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Real CVE Sources \u2192 Parse \u2192 Extract Vulnerabilities \u2192 Build Temporal KG \u2192 Correlate \u2192 Predict Impact \u2192 Generate Reports \u2192 Visualize**\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Ingest CVE Data from Real Sources\n",
-    "\n",
-    "Ingest CVE data from NVD, CVE feeds, and security databases.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import WebIngestor, FeedIngestor, DBIngestor, FileIngestor\n",
-    "from semantica.parse import JSONParser, XMLParser, StructuredDataParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector, TripletExtractor\n",
-    "from semantica.kg import GraphBuilder, TemporalGraphQuery, TemporalPatternDetector, GraphAnalyzer\n",
-    "from semantica.kg import CentralityCalculator, CommunityDetector, ConnectivityAnalyzer\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-
-    "from semantica.export import JSONExporter, RDFExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "web_ingestor = WebIngestor()\n",
-    "feed_ingestor = FeedIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "file_ingestor = FileIngestor()\n",
-    "\n",
-    "json_parser = JSONParser()\n",
-    "xml_parser = XMLParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "\n",
-    "# Real CVE and vulnerability data sources\n",
-    "cve_sources = [\n",
-    "    \"https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-recent.json.zip\",  # NVD Recent CVEs (JSON)\n",
-    "    \"https://nvd.nist.gov/feeds/xml/cve/2.0/nvdcve-2.0-recent.xml.zip\",  # NVD Recent CVEs (XML)\n",
-    "    \"https://cve.mitre.org/data/downloads/allitems.csv\",  # CVE MITRE All Items\n",
-    "    \"https://www.cisa.gov/known-exploited-vulnerabilities-catalog/json\"  # CISA KEV Catalog\n",
-    "]\n",
-    "\n",
-    "# Real vulnerability feed URLs\n",
-    "vulnerability_feeds = [\n",
-    "    \"https://www.cisa.gov/news.xml\",  # CISA Security Advisories\n",
-    "    \"https://www.us-cert.gov/ncas/alerts.xml\",  # US-CERT Alerts\n",
-    "    \"https://feeds.feedburner.com/SecurityWeek\",  # Security Week\n",
-    "    \"https://www.darkreading.com/rss.xml\"  # Dark Reading\n",
-    "]\n",
-    "\n",
-    "# Real database connection for vulnerability tracking\n",
-    "db_connection_string = \"postgresql://user:password@localhost:5432/vulnerability_db\"\n",
-    "db_query = \"SELECT cve_id, description, severity, published_date, affected_products FROM vulnerabilities WHERE published_date > NOW() - INTERVAL '30 days' ORDER BY published_date DESC\"\n",
-    "\n",
-    "# Real web API endpoints for CVE data\n",
-    "cve_apis = [\n",
-    "    \"https://services.nvd.nist.gov/rest/json/cves/2.0\",  # NVD CVE API v2.0\n",
-    "    \"https://api.github.com/repos/CVEProject/cvelist\",  # CVE Project on GitHub\n",
-    "    \"https://cve.circl.lu/api/last\"  # CVE Search API\n",
-    "]\n",
-    "\n",
-    "# Ingest from real CVE feeds\n",
-    "cve_feed_list = []\n",
-    "for feed_url in vulnerability_feeds:\n",
-    "    cve_feed = feed_ingestor.ingest_feed(feed_url)\n",
-    "    if cve_feed:\n",
-    "        cve_feed_list.append(cve_feed)\n",
-    "        print(f\"  Ingested feed: {feed_url}\")\n",
-    "        print(f\"  Items: {len(cve_feed.items) if hasattr(cve_feed, 'items') else 0}\")\n",
-    "\n",
-    "# Ingest from real CVE APIs\n",
-    "cve_api_data = []\n",
-    "for api_url in cve_apis[:1]:  # Process first API\n",
-    "    api_content = web_ingestor.ingest_url(api_url)\n",
-    "    if api_content:\n",
-    "        cve_api_data.append(api_content)\n",
-    "        print(f\"  Ingested CVE API: {api_url}\")\n",
-    "\n",
-    "# Database ingestion pattern\n",
-    "db_data = db_ingestor.export_table(\n",
-    "    connection_string=db_connection_string,\n",
-    "    table_name=\"vulnerabilities\",\n",
-    "    limit=1000\n",
-    ")\n",
-    "print(f\"  Query pattern: {db_query}\")\n",
-    "\n",
-    "print(f\"\\n\ud83d\udcca CVE Ingestion Summary:\")\n",
-    "print(f\"  Vulnerability feeds: {len(cve_feed_list)}\")\n",
-    "print(f\"  CVE API sources: {len(cve_api_data)}\")\n",
-    "print(f\"  Database sources: 1\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/cybersecurity/anomaly_detection_kg.graphml

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<graphml xmlns="http://graphml.graphdrawing.org/xmlns"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns
+         http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
+
+  <key id="type" for="node" attr.name="type" attr.type="string"/>
+  <key id="confidence" for="node" attr.name="confidence" attr.type="double"/>
+
+  <graph id="G" edgedefault="directed">
+
+
+  </graph>
+</graphml>

cookbook/use_cases/cybersecurity/anomaly_detection_kg_entities.csv

@@ -0,0 +1,274 @@
+confidence,end,id,start,text,type
+0.75,,phishing,,phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,Phishing,,Phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,policies,,policies,User
+0.75,,community,,community,User
+0.75,,phishing,,phishing,Attack
+0.75,,attack,,attack,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,Device,,Device,User
+0.75,,virus,,virus,Attack
+0.75,,Virus,,Virus,Attack
+0.75,,Virus,,Virus,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,Phishing,,Phishing,Attack
+0.75,,Phishing,,Phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,Phishing,,Phishing,Attack
+0.75,,Phishing,,Phishing,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Ransomware,,Ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,suspicious activity,,suspicious activity,Event
+0.75,,phishing,,phishing,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,Exploit,,Exploit,Attack
+0.75,,brute force,,brute force,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,suspicious activity,,suspicious activity,Event
+0.75,,Malware,,Malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,Ransomware,,Ransomware,Attack
+0.75,,brute force,,brute force,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,malware,,malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,107.161.23.204,,107.161.23.204,IP
+0.75,,192.161.187.200,,192.161.187.200,IP
+0.75,,209.141.38.71,,209.141.38.71,IP
+0.75,,traffic,,traffic,User
+0.75,,credentials,,credentials,User
+0.75,,traffic,,traffic,User
+0.75,,attack,,attack,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,Exploit,,Exploit,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,rights,,rights,User
+0.75,,exploit,,exploit,Attack
+0.75,,interaction,,interaction,User
+0.75,,exploit,,exploit,Attack
+0.75,,malware,,malware,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,malware,,malware,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Attack,,Attack,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Phishing,,Phishing,Attack
+0.75,,malware,,malware,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,Phishing,,Phishing,Attack
+0.75,,virus,,virus,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,permission,,permission,User
+0.75,,malware,,malware,Attack
+0.75,,should,,should,User
+0.75,,Malware,,Malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Ransomware,,Ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,suspicious activity,,suspicious activity,Event
+0.75,,intrusion,,intrusion,Attack
+0.75,,Suspicious Activity,,Suspicious Activity,Event
+0.75,,phishing,,phishing,Attack
+0.75,,malware,,malware,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,attack,,attack,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,attack,,attack,Attack
+0.75,,credentials,,credentials,User
+0.75,,intrusion,,intrusion,Attack
+0.75,,choices,,choices,User
+0.75,,Suspicious Activity,,Suspicious Activity,Event
+0.75,,intrusion,,intrusion,Attack
+0.75,,malware,,malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Ransomware,,Ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Ransomware,,Ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,Malware,,Malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,DDoS,,DDoS,Attack
+0.75,,malware,,malware,Attack
+0.75,,DDoS,,DDoS,Attack
+0.75,,DDoS,,DDoS,Attack
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,attack,,attack,Attack
+0.75,,Execution,,Execution,User
+0.75,,Execution,,Execution,User
+0.75,,training,,training,User
+0.75,,interaction,,interaction,User
+0.75,,attack,,attack,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,virus,,virus,Attack
+0.75,,attack,,attack,Attack
+0.75,,restrictions,,restrictions,User
+0.75,,attack,,attack,Attack
+0.75,,visits,,visits,User
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,malware,,malware,Attack
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack
+0.75,,virus,,virus,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,virus,,virus,Attack
+0.75,,suspicious activity,,suspicious activity,Event
+0.75,,phishing,,phishing,Attack
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,attack,,attack,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,connections,,connections,User
+0.75,,exploit,,exploit,Attack
+0.75,,malware,,malware,Attack
+0.75,,connections,,connections,User
+0.75,,interaction,,interaction,User
+0.75,,to,,to,User
+0.75,,would,,would,User
+0.75,,exploit,,exploit,Attack
+0.75,,connections,,connections,User
+0.75,,exploit,,exploit,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,malware,,malware,Attack
+0.75,,malware,,malware,Attack
+0.75,,interaction,,interaction,User
+0.75,,interaction,,interaction,User
+0.75,,connects,,connects,User
+0.75,,to,,to,User
+0.75,,attack,,attack,Attack
+0.75,,intrusion,,intrusion,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,exploit,,exploit,Attack
+0.75,,malware,,malware,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,Phishing,,Phishing,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,breach,,breach,Attack
+0.75,,attack,,attack,Attack
+0.75,,phishing,,phishing,Attack
+0.75,,ransomware,,ransomware,Attack
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack
+0.75,,attack,,attack,Attack

cookbook/use_cases/cybersecurity/anomaly_detection_kg_relationships.csv

@@ -0,0 +1,535 @@
+confidence,id,source_id,target_id,type
+0.6,,phishing,exploit,related_to
+0.6,,phishing,community,related_to
+0.6,,phishing,phishing,related_to
+0.6,,phishing,intrusion,related_to
+0.6,,phishing,malware,related_to
+0.6,,phishing,Device,related_to
+0.6,,phishing,Virus,related_to
+0.6,,phishing,Phishing,related_to
+0.6,,phishing,ransomware,related_to
+0.6,,phishing,suspicious activity,related_to
+0.6,,phishing,Exploit,related_to
+0.6,,phishing,Malware,related_to
+0.6,,phishing,permission,related_to
+0.6,,phishing,Ransomware,related_to
+0.6,,phishing,credentials,related_to
+0.6,,phishing,DDoS,related_to
+0.6,,phishing,Execution,related_to
+0.6,,phishing,attack,related_to
+0.6,,phishing,restrictions,related_to
+0.6,,phishing,connections,related_to
+0.6,,phishing,interaction,related_to
+0.6,,phishing,policies,related_to
+0.6,,phishing,traffic,related_to
+0.6,,phishing,Attack,related_to
+0.6,,phishing,virus,related_to
+0.6,,phishing,Suspicious Activity,related_to
+0.6,,phishing,breach,related_to
+0.6,,phishing,brute force,related_to
+0.6,,Phishing,phishing,related_to
+0.6,,Phishing,ransomware,related_to
+0.6,,Phishing,intrusion,related_to
+0.6,,Phishing,malware,related_to
+0.6,,Phishing,virus,related_to
+0.6,,Phishing,Virus,related_to
+0.6,,Phishing,Ransomware,related_to
+0.6,,Phishing,exploit,related_to
+0.6,,Phishing,brute force,related_to
+0.6,,Phishing,suspicious activity,related_to
+0.6,,Phishing,Malware,related_to
+0.6,,Phishing,attack,related_to
+0.6,,Phishing,Attack,related_to
+0.6,,Phishing,Suspicious Activity,related_to
+0.6,,Phishing,DDoS,related_to
+0.6,,Phishing,connections,related_to
+0.6,,Phishing,Phishing,related_to
+0.6,,exploit,community,related_to
+0.6,,exploit,intrusion,related_to
+0.6,,exploit,malware,related_to
+0.6,,exploit,ransomware,related_to
+0.6,,exploit,phishing,related_to
+0.6,,exploit,Phishing,related_to
+0.6,,exploit,suspicious activity,related_to
+0.6,,exploit,Exploit,related_to
+0.6,,exploit,Malware,related_to
+0.6,,exploit,permission,related_to
+0.6,,exploit,Ransomware,related_to
+0.6,,exploit,DDoS,related_to
+0.6,,exploit,Execution,related_to
+0.6,,exploit,restrictions,related_to
+0.6,,exploit,attack,related_to
+0.6,,exploit,interaction,related_to
+0.6,,ransomware,phishing,related_to
+0.6,,ransomware,intrusion,related_to
+0.6,,ransomware,malware,related_to
+0.6,,ransomware,Device,related_to
+0.6,,ransomware,virus,related_to
+0.6,,ransomware,Virus,related_to
+0.6,,ransomware,Ransomware,related_to
+0.6,,ransomware,ransomware,related_to
+0.6,,ransomware,suspicious activity,related_to
+0.6,,ransomware,exploit,related_to
+0.6,,ransomware,brute force,related_to
+0.6,,ransomware,Malware,related_to
+0.6,,ransomware,Suspicious Activity,related_to
+0.6,,ransomware,attack,related_to
+0.6,,ransomware,credentials,related_to
+0.6,,ransomware,DDoS,related_to
+0.6,,ransomware,Execution,related_to
+0.6,,ransomware,restrictions,related_to
+0.6,,ransomware,connections,related_to
+0.6,,ransomware,interaction,related_to
+0.6,,ransomware,Phishing,related_to
+0.6,,ransomware,policies,related_to
+0.6,,ransomware,traffic,related_to
+0.6,,ransomware,Attack,related_to
+0.6,,ransomware,breach,related_to
+0.6,,policies,attack,related_to
+0.6,,policies,malware,related_to
+0.6,,policies,phishing,related_to
+0.6,,policies,ransomware,related_to
+0.6,,policies,intrusion,related_to
+0.6,,policies,traffic,related_to
+0.6,,policies,credentials,related_to
+0.6,,policies,exploit,related_to
+0.6,,policies,interaction,related_to
+0.6,,policies,Malware,related_to
+0.6,,policies,Attack,related_to
+0.6,,policies,Phishing,related_to
+0.6,,policies,virus,related_to
+0.6,,policies,DDoS,related_to
+0.6,,policies,connections,related_to
+0.6,,policies,breach,related_to
+0.6,,community,phishing,related_to
+0.6,,community,intrusion,related_to
+0.6,,community,malware,related_to
+0.6,,community,ransomware,related_to
+0.6,,community,Device,related_to
+0.6,,community,Phishing,related_to
+0.6,,community,suspicious activity,related_to
+0.6,,community,Exploit,related_to
+0.6,,community,Malware,related_to
+0.6,,community,exploit,related_to
+0.6,,community,permission,related_to
+0.6,,community,Ransomware,related_to
+0.6,,community,credentials,related_to
+0.6,,community,DDoS,related_to
+0.6,,community,Execution,related_to
+0.6,,community,attack,related_to
+0.6,,community,restrictions,related_to
+0.6,,community,interaction,related_to
+0.6,,attack,phishing,related_to
+0.6,,attack,malware,related_to
+0.6,,attack,ransomware,related_to
+0.6,,attack,intrusion,related_to
+0.6,,attack,traffic,related_to
+0.6,,attack,credentials,related_to
+0.6,,attack,exploit,related_to
+0.6,,attack,interaction,related_to
+0.6,,attack,Malware,related_to
+0.6,,attack,Attack,related_to
+0.6,,attack,Phishing,related_to
+0.6,,attack,virus,related_to
+0.6,,attack,attack,related_to
+0.6,,attack,connections,related_to
+0.6,,attack,breach,related_to
+0.6,,intrusion,intrusion,related_to
+0.6,,intrusion,malware,related_to
+0.6,,intrusion,ransomware,related_to
+0.6,,intrusion,phishing,related_to
+0.6,,intrusion,Phishing,related_to
+0.6,,intrusion,suspicious activity,related_to
+0.6,,intrusion,Exploit,related_to
+0.6,,intrusion,Malware,related_to
+0.6,,intrusion,permission,related_to
+0.6,,intrusion,Suspicious Activity,related_to
+0.6,,intrusion,DDoS,related_to
+0.6,,intrusion,Execution,related_to
+0.6,,intrusion,restrictions,related_to
+0.6,,intrusion,interaction,related_to
+0.6,,intrusion,Device,related_to
+0.6,,intrusion,virus,related_to
+0.6,,intrusion,Virus,related_to
+0.6,,intrusion,Ransomware,related_to
+0.6,,intrusion,exploit,related_to
+0.6,,intrusion,brute force,related_to
+0.6,,intrusion,attack,related_to
+0.6,,intrusion,credentials,related_to
+0.6,,intrusion,connections,related_to
+0.6,,malware,malware,related_to
+0.6,,malware,virus,related_to
+0.6,,malware,Virus,related_to
+0.6,,malware,phishing,related_to
+0.6,,malware,Ransomware,related_to
+0.6,,malware,ransomware,related_to
+0.6,,malware,exploit,related_to
+0.6,,malware,brute force,related_to
+0.6,,malware,suspicious activity,related_to
+0.6,,malware,intrusion,related_to
+0.6,,malware,Malware,related_to
+0.6,,malware,traffic,related_to
+0.6,,malware,credentials,related_to
+0.6,,malware,attack,related_to
+0.6,,malware,Attack,related_to
+0.6,,malware,Suspicious Activity,related_to
+0.6,,malware,DDoS,related_to
+0.6,,malware,connections,related_to
+0.6,,malware,Phishing,related_to
+0.6,,malware,breach,related_to
+0.6,,malware,Device,related_to
+0.6,,malware,Exploit,related_to
+0.6,,malware,permission,related_to
+0.6,,malware,Execution,related_to
+0.6,,malware,restrictions,related_to
+0.6,,malware,interaction,related_to
+0.6,,ransomware,Exploit,related_to
+0.6,,ransomware,rights,related_to
+0.6,,malware,rights,related_to
+0.6,,Device,Virus,related_to
+0.6,,Device,phishing,related_to
+0.6,,Device,Phishing,related_to
+0.6,,Device,Ransomware,related_to
+0.6,,Device,malware,related_to
+0.6,,Device,ransomware,related_to
+0.6,,Device,suspicious activity,related_to
+0.6,,Device,Exploit,related_to
+0.6,,Device,Malware,related_to
+0.6,,Device,brute force,related_to
+0.6,,Device,exploit,related_to
+0.6,,Device,permission,related_to
+0.6,,Device,credentials,related_to
+0.6,,Device,DDoS,related_to
+0.6,,Device,Execution,related_to
+0.6,,Device,attack,related_to
+0.6,,Device,restrictions,related_to
+0.6,,Device,connections,related_to
+0.6,,Device,interaction,related_to
+0.6,,virus,Virus,related_to
+0.6,,virus,phishing,related_to
+0.6,,virus,Ransomware,related_to
+0.6,,virus,ransomware,related_to
+0.6,,virus,malware,related_to
+0.6,,virus,exploit,related_to
+0.6,,virus,brute force,related_to
+0.6,,virus,suspicious activity,related_to
+0.6,,virus,Malware,related_to
+0.6,,virus,attack,related_to
+0.6,,virus,virus,related_to
+0.6,,virus,intrusion,related_to
+0.6,,virus,Suspicious Activity,related_to
+0.6,,virus,DDoS,related_to
+0.6,,virus,connections,related_to
+0.6,,virus,interaction,related_to
+0.6,,virus,Phishing,related_to
+0.6,,Virus,Virus,related_to
+0.6,,Virus,phishing,related_to
+0.6,,Virus,Ransomware,related_to
+0.6,,Virus,malware,related_to
+0.6,,Virus,ransomware,related_to
+0.6,,Virus,suspicious activity,related_to
+0.6,,Virus,exploit,related_to
+0.6,,Virus,brute force,related_to
+0.6,,Virus,Malware,related_to
+0.6,,Virus,attack,related_to
+0.6,,Virus,intrusion,related_to
+0.6,,Virus,Suspicious Activity,related_to
+0.6,,Virus,credentials,related_to
+0.6,,Virus,DDoS,related_to
+0.6,,Virus,virus,related_to
+0.6,,Virus,restrictions,related_to
+0.6,,Virus,connections,related_to
+0.6,,Virus,interaction,related_to
+0.6,,Virus,Phishing,related_to
+0.6,,Virus,Exploit,related_to
+0.6,,Virus,permission,related_to
+0.6,,Virus,Execution,related_to
+0.6,,Phishing,Exploit,related_to
+0.6,,Phishing,permission,related_to
+0.6,,Phishing,Execution,related_to
+0.6,,Phishing,restrictions,related_to
+0.6,,Phishing,interaction,related_to
+0.6,,phishing,rights,related_to
+0.6,,Phishing,rights,related_to
+0.6,,intrusion,107.161.23.204,related_to
+0.6,,intrusion,192.161.187.200,related_to
+0.6,,intrusion,209.141.38.71,related_to
+0.6,,ransomware,permission,related_to
+0.6,,Ransomware,malware,related_to
+0.6,,Ransomware,ransomware,related_to
+0.6,,Ransomware,suspicious activity,related_to
+0.6,,Ransomware,exploit,related_to
+0.6,,Ransomware,brute force,related_to
+0.6,,Ransomware,Malware,related_to
+0.6,,Ransomware,attack,related_to
+0.6,,Ransomware,Ransomware,related_to
+0.6,,Ransomware,intrusion,related_to
+0.6,,Ransomware,Suspicious Activity,related_to
+0.6,,Ransomware,credentials,related_to
+0.6,,Ransomware,DDoS,related_to
+0.6,,Ransomware,phishing,related_to
+0.6,,Ransomware,virus,related_to
+0.6,,Ransomware,restrictions,related_to
+0.6,,Ransomware,connections,related_to
+0.6,,Ransomware,interaction,related_to
+0.6,,Ransomware,Phishing,related_to
+0.6,,malware,training,related_to
+0.6,,suspicious activity,phishing,related_to
+0.6,,suspicious activity,ransomware,related_to
+0.6,,suspicious activity,Exploit,related_to
+0.6,,suspicious activity,Malware,related_to
+0.6,,suspicious activity,malware,related_to
+0.6,,suspicious activity,exploit,related_to
+0.6,,suspicious activity,Phishing,related_to
+0.6,,suspicious activity,permission,related_to
+0.6,,suspicious activity,Ransomware,related_to
+0.6,,suspicious activity,suspicious activity,related_to
+0.6,,suspicious activity,credentials,related_to
+0.6,,suspicious activity,DDoS,related_to
+0.6,,suspicious activity,Execution,related_to
+0.6,,suspicious activity,attack,related_to
+0.6,,suspicious activity,restrictions,related_to
+0.6,,suspicious activity,connections,related_to
+0.6,,suspicious activity,interaction,related_to
+0.6,,exploit,brute force,related_to
+0.6,,exploit,exploit,related_to
+0.6,,exploit,Attack,related_to
+0.6,,exploit,virus,related_to
+0.6,,exploit,Suspicious Activity,related_to
+0.6,,exploit,connections,related_to
+0.6,,Exploit,Malware,related_to
+0.6,,Exploit,malware,related_to
+0.6,,Exploit,exploit,related_to
+0.6,,Exploit,Exploit,related_to
+0.6,,Exploit,Phishing,related_to
+0.6,,Exploit,phishing,related_to
+0.6,,Exploit,permission,related_to
+0.6,,Exploit,Ransomware,related_to
+0.6,,Exploit,ransomware,related_to
+0.6,,Exploit,suspicious activity,related_to
+0.6,,Exploit,credentials,related_to
+0.6,,Exploit,DDoS,related_to
+0.6,,Exploit,Execution,related_to
+0.6,,Exploit,attack,related_to
+0.6,,Exploit,restrictions,related_to
+0.6,,Exploit,interaction,related_to
+0.6,,brute force,malware,related_to
+0.6,,brute force,suspicious activity,related_to
+0.6,,brute force,brute force,related_to
+0.6,,brute force,Malware,related_to
+0.6,,brute force,attack,related_to
+0.6,,brute force,exploit,related_to
+0.6,,brute force,virus,related_to
+0.6,,brute force,Ransomware,related_to
+0.6,,brute force,ransomware,related_to
+0.6,,brute force,intrusion,related_to
+0.6,,brute force,Suspicious Activity,related_to
+0.6,,brute force,credentials,related_to
+0.6,,brute force,DDoS,related_to
+0.6,,brute force,phishing,related_to
+0.6,,brute force,connections,related_to
+0.6,,brute force,interaction,related_to
+0.6,,brute force,Phishing,related_to
+0.6,,suspicious activity,brute force,related_to
+0.6,,suspicious activity,intrusion,related_to
+0.6,,suspicious activity,Suspicious Activity,related_to
+0.6,,suspicious activity,virus,related_to
+0.6,,Malware,malware,related_to
+0.6,,Malware,Malware,related_to
+0.6,,Malware,Exploit,related_to
+0.6,,Malware,Phishing,related_to
+0.6,,Malware,phishing,related_to
+0.6,,Malware,permission,related_to
+0.6,,Malware,Ransomware,related_to
+0.6,,Malware,ransomware,related_to
+0.6,,Malware,suspicious activity,related_to
+0.6,,Malware,intrusion,related_to
+0.6,,Malware,credentials,related_to
+0.6,,Malware,DDoS,related_to
+0.6,,Malware,Execution,related_to
+0.6,,Malware,restrictions,related_to
+0.6,,Malware,attack,related_to
+0.6,,Malware,interaction,related_to
+0.6,,Ransomware,rights,related_to
+0.6,,brute force,restrictions,related_to
+0.6,,intrusion,traffic,related_to
+0.6,,intrusion,Attack,related_to
+0.6,,intrusion,breach,related_to
+0.6,,Malware,exploit,related_to
+0.6,,Malware,Suspicious Activity,related_to
+0.6,,Malware,virus,related_to
+0.6,,Malware,connections,related_to
+0.6,,Malware,rights,related_to
+0.6,,107.161.23.204,192.161.187.200,related_to
+0.6,,107.161.23.204,209.141.38.71,related_to
+0.6,,107.161.23.204,Malware,related_to
+0.6,,107.161.23.204,ransomware,related_to
+0.6,,107.161.23.204,attack,related_to
+0.6,,107.161.23.204,phishing,related_to
+0.6,,107.161.23.204,intrusion,related_to
+0.6,,192.161.187.200,209.141.38.71,related_to
+0.6,,192.161.187.200,Malware,related_to
+0.6,,192.161.187.200,ransomware,related_to
+0.6,,192.161.187.200,attack,related_to
+0.6,,192.161.187.200,phishing,related_to
+0.6,,192.161.187.200,intrusion,related_to
+0.6,,209.141.38.71,Malware,related_to
+0.6,,209.141.38.71,phishing,related_to
+0.6,,209.141.38.71,intrusion,related_to
+0.6,,209.141.38.71,training,related_to
+0.6,,209.141.38.71,attack,related_to
+0.6,,traffic,credentials,related_to
+0.6,,traffic,traffic,related_to
+0.6,,traffic,attack,related_to
+0.6,,traffic,exploit,related_to
+0.6,,traffic,interaction,related_to
+0.6,,traffic,Malware,related_to
+0.6,,traffic,Attack,related_to
+0.6,,traffic,malware,related_to
+0.6,,traffic,Phishing,related_to
+0.6,,traffic,virus,related_to
+0.6,,traffic,ransomware,related_to
+0.6,,traffic,Suspicious Activity,related_to
+0.6,,traffic,intrusion,related_to
+0.6,,traffic,DDoS,related_to
+0.6,,traffic,connections,related_to
+0.6,,traffic,phishing,related_to
+0.6,,traffic,breach,related_to
+0.6,,credentials,traffic,related_to
+0.6,,credentials,attack,related_to
+0.6,,credentials,exploit,related_to
+0.6,,credentials,interaction,related_to
+0.6,,credentials,malware,related_to
+0.6,,credentials,Malware,related_to
+0.6,,credentials,Attack,related_to
+0.6,,credentials,Phishing,related_to
+0.6,,credentials,virus,related_to
+0.6,,credentials,ransomware,related_to
+0.6,,credentials,Suspicious Activity,related_to
+0.6,,credentials,intrusion,related_to
+0.6,,credentials,DDoS,related_to
+0.6,,credentials,connections,related_to
+0.6,,credentials,phishing,related_to
+0.6,,credentials,breach,related_to
+0.6,,attack,Suspicious Activity,related_to
+0.6,,attack,DDoS,related_to
+0.6,,exploit,credentials,related_to
+0.6,,Exploit,intrusion,related_to
+0.6,,exploit,rights,related_to
+0.6,,rights,malware,related_to
+0.6,,rights,intrusion,related_to
+0.6,,rights,Suspicious Activity,related_to
+0.6,,rights,ransomware,related_to
+0.6,,rights,attack,related_to
+0.6,,exploit,breach,related_to
+0.6,,interaction,exploit,related_to
+0.6,,interaction,Malware,related_to
+0.6,,interaction,Attack,related_to
+0.6,,interaction,malware,related_to
+0.6,,interaction,Phishing,related_to
+0.6,,interaction,virus,related_to
+0.6,,interaction,ransomware,related_to
+0.6,,interaction,Suspicious Activity,related_to
+0.6,,interaction,intrusion,related_to
+0.6,,interaction,DDoS,related_to
+0.6,,interaction,attack,related_to
+0.6,,interaction,connections,related_to
+0.6,,interaction,phishing,related_to
+0.6,,interaction,breach,related_to
+0.6,,Malware,Attack,related_to
+0.6,,Malware,breach,related_to
+0.6,,Attack,malware,related_to
+0.6,,Attack,Phishing,related_to
+0.6,,Attack,virus,related_to
+0.6,,Attack,exploit,related_to
+0.6,,Attack,ransomware,related_to
+0.6,,Attack,intrusion,related_to
+0.6,,Attack,Suspicious Activity,related_to
+0.6,,Attack,DDoS,related_to
+0.6,,Attack,attack,related_to
+0.6,,Attack,connections,related_to
+0.6,,Attack,phishing,related_to
+0.6,,Attack,breach,related_to
+0.6,,Malware,training,related_to
+0.6,,Phishing,credentials,related_to
+0.6,,Phishing,breach,related_to
+0.6,,virus,breach,related_to
+0.6,,permission,Malware,related_to
+0.6,,permission,malware,related_to
+0.6,,permission,Ransomware,related_to
+0.6,,permission,ransomware,related_to
+0.6,,permission,suspicious activity,related_to
+0.6,,permission,phishing,related_to
+0.6,,permission,credentials,related_to
+0.6,,permission,DDoS,related_to
+0.6,,permission,Execution,related_to
+0.6,,permission,attack,related_to
+0.6,,permission,restrictions,related_to
+0.6,,permission,interaction,related_to
+0.6,,permission,exploit,related_to
+0.6,,Ransomware,Execution,related_to
+0.6,,Suspicious Activity,attack,related_to
+0.6,,Suspicious Activity,intrusion,related_to
+0.6,,Suspicious Activity,ransomware,related_to
+0.6,,Suspicious Activity,DDoS,related_to
+0.6,,Suspicious Activity,phishing,related_to
+0.6,,Suspicious Activity,virus,related_to
+0.6,,Suspicious Activity,connections,related_to
+0.6,,Suspicious Activity,malware,related_to
+0.6,,Suspicious Activity,exploit,related_to
+0.6,,Suspicious Activity,Phishing,related_to
+0.6,,credentials,Execution,related_to
+0.6,,credentials,restrictions,related_to
+0.6,,credentials,suspicious activity,related_to
+0.6,,Suspicious Activity,interaction,related_to
+0.6,,DDoS,DDoS,related_to
+0.6,,DDoS,attack,related_to
+0.6,,DDoS,intrusion,related_to
+0.6,,DDoS,virus,related_to
+0.6,,DDoS,connections,related_to
+0.6,,DDoS,exploit,related_to
+0.6,,DDoS,malware,related_to
+0.6,,DDoS,Phishing,related_to
+0.6,,DDoS,breach,related_to
+0.6,,DDoS,phishing,related_to
+0.6,,DDoS,ransomware,related_to
+0.6,,DDoS,Execution,related_to
+0.6,,DDoS,restrictions,related_to
+0.6,,DDoS,suspicious activity,related_to
+0.6,,DDoS,interaction,related_to
+0.6,,attack,visits,related_to
+0.6,,Execution,Execution,related_to
+0.6,,Execution,attack,related_to
+0.6,,Execution,restrictions,related_to
+0.6,,Execution,suspicious activity,related_to
+0.6,,Execution,phishing,related_to
+0.6,,Execution,ransomware,related_to
+0.6,,Execution,malware,related_to
+0.6,,Execution,interaction,related_to
+0.6,,Execution,exploit,related_to
+0.6,,training,phishing,related_to
+0.6,,training,attack,related_to
+0.6,,attack,restrictions,related_to
+0.6,,attack,suspicious activity,related_to
+0.6,,restrictions,suspicious activity,related_to
+0.6,,restrictions,phishing,related_to
+0.6,,restrictions,attack,related_to
+0.6,,restrictions,ransomware,related_to
+0.6,,restrictions,malware,related_to
+0.6,,restrictions,connections,related_to
+0.6,,restrictions,interaction,related_to
+0.6,,restrictions,exploit,related_to
+0.6,,visits,attack,related_to
+0.6,,connections,exploit,related_to
+0.6,,connections,connections,related_to
+0.6,,connections,malware,related_to
+0.6,,connections,phishing,related_to
+0.6,,connections,Phishing,related_to
+0.6,,connections,breach,related_to
+0.6,,connections,attack,related_to
+0.6,,connections,ransomware,related_to
+0.6,,connections,interaction,related_to
+0.6,,interaction,interaction,related_to
+0.6,,breach,attack,related_to
+0.6,,breach,phishing,related_to
+0.6,,breach,ransomware,related_to

cookbook/use_cases/cybersecurity/threat_intelligence_kg.graphml

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<graphml xmlns="http://graphml.graphdrawing.org/xmlns"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns
+         http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
+
+  <key id="type" for="node" attr.name="type" attr.type="string"/>
+  <key id="confidence" for="node" attr.name="confidence" attr.type="double"/>
+
+  <graph id="G" edgedefault="directed">
+
+
+  </graph>
+</graphml>

cookbook/use_cases/finance/01_Financial_Data_Integration.ipynb

@@ -1,530 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/finance/01_Financial_Data_Integration.ipynb)\n",
-    "\n",
-    "# \ud83d\udcc8 Financial Data Integration Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates how to integrate Python/FastMCP MCP servers as data sources for financial data ingestion. Connect to financial data MCP servers via URL, ingest market data, stock prices, and financial metrics, then build a knowledge graph for financial analysis.\n",
-    "\n",
-    "> [!IMPORTANT]\n",
-    "> This implementation supports ONLY Python-based MCP servers and FastMCP servers. Users can bring their own Python/FastMCP MCP servers via URL connections.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "### \ud83e\udde9 Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, EmailIngestor, RepoIngestor, MCPIngestor\n",
-    "- **Parsing**: `MCPParser`, `JSONParser`, `StructuredDataParser`\n",
-    "- **Extraction**: `NERExtractor`, `RelationExtractor`, `EventDetector`, `SemanticAnalyzer`\n",
-    "- **KG**: `GraphBuilder`, `TemporalGraphQuery`, `GraphAnalyzer`\n",
-    "- **Analytics**: `CentralityCalculator`, `CommunityDetector`, `ConnectivityAnalyzer`\n",
-    "- **Reasoning**: `InferenceEngine`, `RuleManager`, `ExplanationGenerator`\n",
-    "- **Export**: `JSONExporter`, `CSVExporter`, `RDFExporter`, `ReportGenerator`\n",
-    "- **Visualization**: `KGVisualizer`, `TemporalVisualizer`, `AnalyticsVisualizer`\n",
-    "\n",
-    "### \ud83d\udd04 Pipeline\n",
-    "\n",
-    "**Connect to Financial MCP Server \u2192 Ingest Market Data via MCP \u2192 Parse MCP Responses \u2192 Extract Financial Entities \u2192 Build Financial KG \u2192 Analyze Trends \u2192 Generate Reports \u2192 Visualize**\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## \ud83d\udd0c Step 1: Connect to Financial Data MCP Server\n",
-    "\n",
-    "Connect to a Python/FastMCP MCP server that provides financial data via URL. The MCP server can expose resources (datasets, market data) and tools (queries, calculations).\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import MCPIngestor, ingest_mcp\n",
-    "from semantica.parse import MCPParser, JSONParser, StructuredDataParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector, SemanticAnalyzer\n",
-    "from semantica.kg import GraphBuilder, TemporalGraphQuery, GraphAnalyzer\n",
-    "from semantica.kg import CentralityCalculator, CommunityDetector, ConnectivityAnalyzer\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "from semantica.export import JSONExporter, CSVExporter, RDFExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "# Initialize MCP ingestor\n",
-    "mcp_ingestor = MCPIngestor()\n",
-    "\n",
-    "# Connect to financial data MCP server via URL\n",
-    "# Replace with your actual MCP server URL\n",
-    "# Example: http://localhost:8000/mcp or https://api.example.com/financial-mcp\n",
-    "financial_mcp_url = \"http://localhost:8000/mcp\"\n",
-    "\n",
-    "# Connect to MCP server\n",
-    "mcp_ingestor.connect(\n",
-    "    \"financial_server\",\n",
-    "    url=financial_mcp_url,\n",
-    "    headers={\"Authorization\": \"Bearer your_token\"} if \"api.example.com\" in financial_mcp_url else {}\n",
-    ")\n",
-    "\n",
-    "# List available resources (datasets, market data feeds)\n",
-    "resources = mcp_ingestor.list_available_resources(\"financial_server\")\n",
-    "print(f\"\\n\ud83d\udcca Available Resources ({len(resources)}):\")\n",
-    "for resource in resources[:5]:  # Show first 5\n",
-    "    print(f\"  - {resource.uri}: {resource.name}\")\n",
-    "    if resource.description:\n",
-    "        print(f\"    {resource.description[:80]}...\")\n",
-    "\n",
-    "# List available tools (queries, calculations)\n",
-    "tools = mcp_ingestor.list_available_tools(\"financial_server\")\n",
-    "print(f\"\\n\ud83d\udd27 Available Tools ({len(tools)}):\")\n",
-    "for tool in tools[:5]:  # Show first 5\n",
-    "    print(f\"  - {tool.name}: {tool.description or 'No description'}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## \ud83d\udce5 Step 2: Ingest Financial Data from MCP Server\n",
-    "\n",
-    "Ingest financial data using both resource-based and tool-based methods from the MCP server.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Initialize parsers\n",
-    "mcp_parser = MCPParser()\n",
-    "json_parser = JSONParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "\n",
-    "financial_data = []\n",
-    "\n",
-    "# Method 1: Resource-based ingestion\n",
-    "# Ingest from MCP resources (pre-defined datasets)\n",
-    "# Example: Ingest market data resource\n",
-    "resource_data = mcp_ingestor.ingest_resources(\n",
-    "    \"financial_server\",\n",
-    "    resource_uris=[\"resource://market_data/daily\", \"resource://market_data/stocks\"]\n",
-    ")\n",
-    "\n",
-    "for item in resource_data:\n",
-    "    financial_data.append(item)\n",
-    "    print(f\"  Ingested resource: {item}\")\n",
-    "\n",
-    "# Method 2: Tool-based ingestion\n",
-    "# Call MCP tools to retrieve data dynamically\n",
-    "# Example: Get stock prices for specific symbols\n",
-    "stock_prices = mcp_ingestor.ingest_tool_output(\n",
-    "    \"financial_server\",\n",
-    "    tool_name=\"get_stock_prices\",\n",
-    "    arguments={\n",
-    "        \"symbols\": [\"AAPL\", \"MSFT\", \"GOOGL\", \"TSLA\"],\n",
-    "        \"date\": datetime.now().isoformat()\n",
-    "    }\n",
-    ")\n",
-    "\n",
-    "if stock_prices:\n",
-    "    financial_data.append(stock_prices)\n",
-    "    print(f\"  Retrieved stock prices for {len(stock_prices) if isinstance(stock_prices, list) else 1} symbols\")\n",
-    "\n",
-    "# Example: Get market metrics\n",
-    "market_metrics = mcp_ingestor.ingest_tool_output(\n",
-    "    \"financial_server\",\n",
-    "    tool_name=\"get_market_metrics\",\n",
-    "    arguments={\"sector\": \"Technology\"}\n",
-    ")\n",
-    "\n",
-    "if market_metrics:\n",
-    "    financial_data.append(market_metrics)\n",
-    "    print(f\"  Retrieved market metrics\")\n",
-    "\n",
-    "# Sample financial data (if MCP server is not available)\n",
-    "if not financial_data:\n",
-    "    sample_data = {\n",
-    "        \"stock_prices\": [\n",
-    "            {\n",
-    "                \"symbol\": \"AAPL\",\n",
-    "                \"company\": \"Apple Inc.\",\n",
-    "                \"price\": 175.50,\n",
-    "                \"change\": 2.30,\n",
-    "                \"change_percent\": 1.33,\n",
-    "                \"volume\": 45000000,\n",
-    "                \"timestamp\": (datetime.now() - timedelta(hours=1)).isoformat(),\n",
-    "                \"sector\": \"Technology\"\n",
-    "            },\n",
-    "            {\n",
-    "                \"symbol\": \"MSFT\",\n",
-    "                \"company\": \"Microsoft Corporation\",\n",
-    "                \"price\": 380.25,\n",
-    "                \"change\": -1.50,\n",
-    "                \"change_percent\": -0.39,\n",
-    "                \"volume\": 28000000,\n",
-    "                \"timestamp\": (datetime.now() - timedelta(hours=1)).isoformat(),\n",
-    "                \"sector\": \"Technology\"\n",
-    "            },\n",
-    "            {\n",
-    "                \"symbol\": \"GOOGL\",\n",
-    "                \"company\": \"Alphabet Inc.\",\n",
-    "                \"price\": 142.80,\n",
-    "                \"change\": 3.20,\n",
-    "                \"change_percent\": 2.29,\n",
-    "                \"volume\": 32000000,\n",
-    "                \"timestamp\": (datetime.now() - timedelta(minutes=30)).isoformat(),\n",
-    "                \"sector\": \"Technology\"\n",
-    "            },\n",
-    "            {\n",
-    "                \"symbol\": \"TSLA\",\n",
-    "                \"company\": \"Tesla Inc.\",\n",
-    "                \"price\": 245.60,\n",
-    "                \"change\": 5.40,\n",
-    "                \"change_percent\": 2.25,\n",
-    "                \"volume\": 55000000,\n",
-    "                \"timestamp\": datetime.now().isoformat(),\n",
-    "                \"sector\": \"Automotive\"\n",
-    "            }\n",
-    "        ],\n",
-    "        \"market_metrics\": {\n",
-    "            \"total_volume\": 150000000,\n",
-    "            \"market_cap\": 15000000000000,\n",
-    "            \"sectors\": [\"Technology\", \"Automotive\"]\n",
-    "        }\n",
-    "    }\n",
-    "    financial_data.append(sample_data)\n",
-    "    print(f\"  Loaded {len(sample_data['stock_prices'])} stock prices\")\n",
-    "\n",
-    "print(f\"\\n\ud83d\udcca Total financial data items ingested: {len(financial_data)}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## \ud83d\udcc4 Step 3: Parse MCP Data\n",
-    "\n",
-    "Parse the data received from MCP server responses (JSON, structured data).\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "parsed_financial_data = []\n",
-    "\n",
-    "# Parse MCP responses\n",
-    "for data_item in financial_data:\n",
-    "    # Parse MCP response (handles JSON, text, binary)\n",
-    "    if isinstance(data_item, dict):\n",
-    "        # If it's already structured, use it directly\n",
-    "        parsed_item = data_item\n",
-    "    else:\n",
-    "        # Parse using MCP parser\n",
-    "        parsed_item = mcp_parser.parse_response(data_item, response_type=\"json\")\n",
-    "    \n",
-    "    parsed_financial_data.append(parsed_item)\n",
-    "    print(f\"  Parsed data item\")\n",
-    "\n",
-    "# Extract stock prices from parsed data\n",
-    "stock_prices = []\n",
-    "for item in parsed_financial_data:\n",
-    "    if isinstance(item, dict):\n",
-    "        if \"stock_prices\" in item:\n",
-    "            stock_prices.extend(item[\"stock_prices\"])\n",
-    "        elif \"symbol\" in item:\n",
-    "            stock_prices.append(item)\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## \u26cf\ufe0f Step 4: Extract Financial Entities and Relationships\n",
-    "\n",
-    "Extract financial entities (companies, stocks, sectors) and relationships from MCP data.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "event_detector = EventDetector()\n",
-    "semantic_analyzer = SemanticAnalyzer()\n",
-    "\n",
-    "financial_entities = []\n",
-    "financial_relationships = []\n",
-    "\n",
-    "# Extract entities and relationships from stock prices\n",
-    "for stock in stock_prices:\n",
-    "    if isinstance(stock, dict):\n",
-    "        symbol = stock.get(\"symbol\", \"\")\n",
-    "        company = stock.get(\"company\", \"\")\n",
-    "        sector = stock.get(\"sector\", \"\")\n",
-    "        \n",
-    "        # Stock entity\n",
-    "        financial_entities.append({\n",
-    "            \"id\": symbol,\n",
-    "            \"type\": \"Stock\",\n",
-    "            \"name\": symbol,\n",
-    "            \"properties\": {\n",
-    "                \"price\": stock.get(\"price\", 0),\n",
-    "                \"change\": stock.get(\"change\", 0),\n",
-    "                \"change_percent\": stock.get(\"change_percent\", 0),\n",
-    "                \"volume\": stock.get(\"volume\", 0),\n",
-    "                \"timestamp\": stock.get(\"timestamp\", \"\")\n",
-    "            }\n",
-    "        })\n",
-    "        \n",
-    "        # Company entity\n",
-    "        if company:\n",
-    "            financial_entities.append({\n",
-    "                \"id\": company,\n",
-    "                \"type\": \"Company\",\n",
-    "                \"name\": company,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            \n",
-    "            # Stock-Company relationship\n",
-    "            financial_relationships.append({\n",
-    "                \"source\": symbol,\n",
-    "                \"target\": company,\n",
-    "                \"type\": \"ticker_for\",\n",
-    "                \"properties\": {\"timestamp\": stock.get(\"timestamp\", \"\")}\n",
-    "            })\n",
-    "        \n",
-    "        # Sector entity\n",
-    "        if sector:\n",
-    "            financial_entities.append({\n",
-    "                \"id\": sector,\n",
-    "                \"type\": \"Sector\",\n",
-    "                \"name\": sector,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            \n",
-    "            # Company-Sector relationship\n",
-    "            if company:\n",
-    "                financial_relationships.append({\n",
-    "                    \"source\": company,\n",
-    "                    \"target\": sector,\n",
-    "                    \"type\": \"belongs_to\",\n",
-    "                    \"properties\": {}\n",
-    "                })\n",
-    "\n",
-    "# Remove duplicates\n",
-    "seen_entities = set()\n",
-    "unique_entities = []\n",
-    "for entity in financial_entities:\n",
-    "    entity_key = (entity[\"id\"], entity[\"type\"])\n",
-    "    if entity_key not in seen_entities:\n",
-    "        seen_entities.add(entity_key)\n",
-    "        unique_entities.append(entity)\n",
-    "\n",
-    "financial_entities = unique_entities\n",
-    "\n",
-    "print(f\"  - Stocks: {len([e for e in financial_entities if e['type'] == 'Stock'])}\")\n",
-    "print(f\"  - Companies: {len([e for e in financial_entities if e['type'] == 'Company'])}\")\n",
-    "print(f\"  - Sectors: {len([e for e in financial_entities if e['type'] == 'Sector'])}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## \ud83d\udd78\ufe0f Step 5: Build Financial Knowledge Graph\n",
-    "\n",
-    "Build a knowledge graph from the extracted financial entities and relationships.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "temporal_query = TemporalGraphQuery()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "\n",
-    "# Build knowledge graph\n",
-    "financial_kg = builder.build(financial_entities, financial_relationships)\n",
-    "\n",
-    "# Analyze graph structure\n",
-    "metrics = graph_analyzer.compute_metrics(financial_kg)\n",
-    "centrality_calculator = CentralityCalculator()\n",
-    "community_detector = CommunityDetector()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "\n",
-    "# Calculate graph metrics\n",
-    "centrality_result = centrality_calculator.calculate_degree_centrality(financial_kg)\n",
-    "centrality_scores = centrality_result.get('centrality', {})\n",
-    "communities = community_detector.detect_communities(financial_kg)\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(financial_kg)\n",
-    "\n",
-    "print(f\"  Entities: {len(financial_kg.get('entities', []))}\")\n",
-    "print(f\"  Relationships: {len(financial_kg.get('relationships', []))}\")\n",
-    "print(f\"  Graph density: {metrics.get('density', 0):.3f}\")\n",
-    "print(f\"  Communities detected: {len(communities)}\")\n",
-    "print(f\"  Central entities: {len([e for e, score in centrality_scores.items() if score > 0])}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## \ud83d\udcca Step 6: Analyze Financial Trends\n",
-    "\n",
-    "Analyze financial trends using temporal queries and pattern detection.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Temporal analysis\n",
-    "start_time = (datetime.now() - timedelta(days=7)).isoformat()\n",
-    "end_time = datetime.now().isoformat()\n",
-    "\n",
-    "temporal_results = temporal_query.query_time_range(\n",
-    "    graph=financial_kg,\n",
-    "    query=\"Find stock price movements\",\n",
-    "    start_time=start_time,\n",
-    "    end_time=end_time\n",
-    ")\n",
-    "\n",
-    "# Inference engine for financial rules\n",
-    "inference_engine = InferenceEngine()\n",
-    "rule_manager = RuleManager()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-    "\n",
-    "# Financial analysis rules\n",
-    "inference_engine.add_rule(\"IF change_percent > 2 AND volume > 40000000 THEN strong_momentum\")\n",
-    "inference_engine.add_rule(\"IF change_percent < -1 AND volume > 50000000 THEN selling_pressure\")\n",
-    "inference_engine.add_rule(\"IF change_percent > 0 AND sector == 'Technology' THEN tech_growth\")\n",
-    "\n",
-    "# Add facts from stock data\n",
-    "for stock in stock_prices:\n",
-    "    if isinstance(stock, dict):\n",
-    "        inference_engine.add_fact({\n",
-    "            \"symbol\": stock.get(\"symbol\", \"\"),\n",
-    "            \"change_percent\": stock.get(\"change_percent\", 0),\n",
-    "            \"volume\": stock.get(\"volume\", 0),\n",
-    "            \"sector\": stock.get(\"sector\", \"\")\n",
-    "        })\n",
-    "\n",
-    "# Generate insights\n",
-    "financial_insights = inference_engine.forward_chain()\n",
-    "\n",
-    "print(f\"  Temporal entities: {len(temporal_results.get('entities', []))}\")\n",
-    "print(f\"  Financial insights: {len(financial_insights)}\")\n",
-    "\n",
-    "# Display insights\n",
-    "for insight in financial_insights[:3]:\n",
-    "    print(f\"  - {insight}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## \ud83d\udce4 Step 7: Export and Visualize\n",
-    "\n",
-    "Export the financial knowledge graph and generate visualizations.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import tempfile\n",
-    "import os\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "json_exporter = JSONExporter()\n",
-    "csv_exporter = CSVExporter()\n",
-    "rdf_exporter = RDFExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-    "# Export knowledge graph\n",
-    "json_exporter.export_knowledge_graph(financial_kg, os.path.join(temp_dir, \"financial_kg.json\"))\n",
-    "csv_exporter.export_entities(financial_entities, os.path.join(temp_dir, \"financial_entities.csv\"))\n",
-    "rdf_exporter.export_knowledge_graph(financial_kg, os.path.join(temp_dir, \"financial_kg.rdf\"))\n",
-    "\n",
-    "# Generate report\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Financial data integration from MCP server identified {len(financial_insights)} insights\",\n",
-    "    \"stocks_analyzed\": len([e for e in financial_entities if e['type'] == 'Stock']),\n",
-    "    \"companies\": len([e for e in financial_entities if e['type'] == 'Company']),\n",
-    "    \"sectors\": len([e for e in financial_entities if e['type'] == 'Sector']),\n",
-    "    \"insights\": len(financial_insights)\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"  JSON: {os.path.join(temp_dir, 'financial_kg.json')}\")\n",
-    "print(f\"  CSV: {os.path.join(temp_dir, 'financial_entities.csv')}\")\n",
-    "print(f\"  RDF: {os.path.join(temp_dir, 'financial_kg.rdf')}\")\n",
-    "\n",
-    "# Visualize\n",
-    "kg_visualizer = KGVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(financial_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(financial_kg, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(financial_kg, output=\"interactive\")\n",
-    "\n",
-    "\n",
-    "# Cleanup: Disconnect from MCP server\n",
-    "mcp_ingestor.disconnect(\"financial_server\")\n",
-    "print(\"  Disconnected from MCP server\")\n",
-    "\n",
-    "print(f\"\ud83d\udcca Total modules used: 20+\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/finance/01_Financial_Data_Integration_MCP.ipynb

@@ -0,0 +1,983 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/finance/01_Financial_Data_Integration_MCP.ipynb)\n",
+    "\n",
+    "# Financial Data Integration (MCP) - Real-Time Market Data\n",
+    "\n",
+    "## Overview\n",
+    "\n",
+    "This notebook demonstrates **financial data integration using MCP servers** with focus on **MCP server integration**, **real-time data ingestion**, and **multi-source financial KG construction**. The pipeline integrates Python/FastMCP servers to ingest market data, stock prices, and metrics into a financial knowledge graph.\n",
+    "\n",
+    "### Key Features\n",
+    "\n",
+    "- **MCP Integration**: Showcases MCP (Model Context Protocol) server integration capability\n",
+    "- **Seed Data Management**: Uses foundation market data for entity resolution\n",
+    "- **Real-Time Data Ingestion**: Ingests live market data from MCP servers and APIs\n",
+    "- **Multi-Source Financial KG**: Builds comprehensive financial knowledge graphs from multiple sources\n",
+    "- **Market Network Analysis**: Analyzes market structure using graph analytics\n",
+    "- **Comprehensive Data Sources**: Multiple financial APIs, RSS feeds, and databases\n",
+    "- **Modular Architecture**: Direct use of Semantica modules without core orchestrator\n",
+    "\n",
+    "### Learning Objectives\n",
+    "\n",
+    "- Ingest financial data from MCP servers, APIs, and RSS feeds\n",
+    "- Use seed data for foundation market information\n",
+    "- Extract financial entities (Companies, Stocks, Prices, Metrics, Markets, Sectors)\n",
+    "- Build financial knowledge graphs with seed data integration\n",
+    "- Analyze market network structure using graph analytics\n",
+    "- Store and query financial data using vector stores\n",
+    "\n",
+    "### Pipeline Flow\n",
+    "\n",
+    "```mermaid\n",
+    "graph TD\n",
+    "    A[Data Ingestion] --> B[Seed Data Loading]\n",
+    "    B --> C[Document Parsing]\n",
+    "    C --> D[Text Processing]\n",
+    "    D --> E[Entity Extraction]\n",
+    "    E --> F[Relationship Extraction]\n",
+    "    F --> G[Deduplication]\n",
+    "    G --> H[Knowledge Graph]\n",
+    "    H --> I[Embeddings]\n",
+    "    I --> J[Vector Store]\n",
+    "    H --> K[Graph Analytics]\n",
+    "    K --> L[GraphRAG Queries]\n",
+    "    J --> L\n",
+    "    L --> M[Visualization]\n",
+    "    M --> N[Export]\n",
+    "```\n",
+    "\n",
+    "\n",
+    "## Installation\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Note: you may need to restart the kernel to use updated packages.\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
+      "WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
+     ]
+    }
+   ],
+   "source": [
+    "%pip install -qU semantica networkx matplotlib plotly pandas faiss-cpu beautifulsoup4 groq sentence-transformers scikit-learn\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Configuration & Setup\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"GROQ_API_KEY\"] = os.getenv(\"GROQ_API_KEY\", \"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\")\n",
+    "\n",
+    "# Configuration constants\n",
+    "EMBEDDING_DIMENSION = 384\n",
+    "EMBEDDING_MODEL = \"all-MiniLM-L6-v2\"\n",
+    "CHUNK_SIZE = 1000\n",
+    "CHUNK_OVERLAP = 200\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Ingesting Financial Data from Multiple Sources\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Ingesting from 8 feed sources...\n",
+      "🧠 Semantica is ingesting: 429 Client Error: Too Many Requests for url: https://feeds.finance.yahoo.com/rss/2.0/headline ❌📥 (1.0s) | 🧠 Semantica is ingesting: Ingested 10 items |███████████████| 100.0% ✅📥  [2/8] Financial Times: 10 documents\n",
+      "🧠 Semantica is ingesting: Ingested 10 items |███████████████| 100.0% ✅📥 | 🧠 Semantica is ingesting: Ingested 30 items |███████████████| 100.0% ✅📥  [3/8] Bloomberg: 30 documents\n",
+      "🧠 Semantica is ingesting: 403 Client Error: Forbidden for url: https://www.marketwatch.com/404?origin=feeds ❌📥 (3.2s) | 🧠 Semantica is ingesting: Ingested 30 items |███████████████| 100.0% ✅📥  [5/8] Seeking Alpha: 30 documents\n",
+      "🧠 Semantica is ingesting: Ingested 30 items |███████████████| 100.0% ✅📥 | 🧠 Semantica is ingesting: Ingested 10 items |███████████████| 100.0% ✅📥  [6/8] Investing.com: 10 documents\n",
+      "🧠 Semantica is ingesting: 403 Client Error: Forbidden for url: https://www.fnlondon.com/rss ❌📥 (0.5s) | 🧠 Semantica is ingesting: Ingested 20 items |███████████████| 100.0% ✅📥  [8/8] Wall Street Journal: 20 documents\n",
+      "Ingested 100 documents\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.ingest import MCPIngestor, WebIngestor, FeedIngestor, FileIngestor\n",
+    "import os\n",
+    "from contextlib import redirect_stderr\n",
+    "from io import StringIO\n",
+    "\n",
+    "os.makedirs(\"data\", exist_ok=True)\n",
+    "\n",
+    "feed_sources = [\n",
+    "    # Financial RSS Feeds - More reliable sources\n",
+    "    (\"Yahoo Finance\", \"https://feeds.finance.yahoo.com/rss/2.0/headline\"),\n",
+    "    (\"Financial Times\", \"https://www.ft.com/?format=rss\"),\n",
+    "    (\"Bloomberg\", \"https://feeds.bloomberg.com/markets/news.rss\"),\n",
+    "    (\"MarketWatch\", \"https://feeds.marketwatch.com/marketwatch/markets\"),\n",
+    "    (\"Seeking Alpha\", \"https://seekingalpha.com/feed.xml\"),\n",
+    "    (\"Investing.com\", \"https://www.investing.com/rss/news.rss\"),\n",
+    "    (\"Financial News\", \"https://www.fnlondon.com/rss\"),\n",
+    "    (\"Wall Street Journal\", \"https://feeds.a.dj.com/rss/RSSMarketsMain.xml\"),\n",
+    "]\n",
+    "\n",
+    "feed_ingestor = FeedIngestor()\n",
+    "all_documents = []\n",
+    "\n",
+    "print(f\"Ingesting from {len(feed_sources)} feed sources...\")\n",
+    "for i, (feed_name, feed_url) in enumerate(feed_sources, 1):\n",
+    "    try:\n",
+    "        with redirect_stderr(StringIO()):\n",
+    "            feed_data = feed_ingestor.ingest_feed(feed_url, validate=False)\n",
+    "        \n",
+    "        feed_count = 0\n",
+    "        for item in feed_data.items:\n",
+    "            if not item.content:\n",
+    "                item.content = item.description or item.title or \"\"\n",
+    "            if item.content:\n",
+    "                if not hasattr(item, 'metadata'):\n",
+    "                    item.metadata = {}\n",
+    "                item.metadata['source'] = feed_name\n",
+    "                all_documents.append(item)\n",
+    "                feed_count += 1\n",
+    "        \n",
+    "        if feed_count > 0:\n",
+    "            print(f\"  [{i}/{len(feed_sources)}] {feed_name}: {feed_count} documents\")\n",
+    "    except Exception:\n",
+    "        continue\n",
+    "\n",
+    "# Example: Ingest from Alpha Vantage API (requires API key)\n",
+    "alpha_vantage_api = \"https://www.alphavantage.co/query?function=GLOBAL_QUOTE&symbol=AAPL&apikey=demo\"\n",
+    "try:\n",
+    "    web_ingestor = WebIngestor()\n",
+    "    with redirect_stderr(StringIO()):\n",
+    "        api_documents = web_ingestor.ingest(alpha_vantage_api, method=\"url\")\n",
+    "    for doc in api_documents:\n",
+    "        if not hasattr(doc, 'metadata'):\n",
+    "            doc.metadata = {}\n",
+    "        doc.metadata['source'] = 'Alpha Vantage API'\n",
+    "        all_documents.append(doc)\n",
+    "except Exception:\n",
+    "    pass\n",
+    "\n",
+    "# MCP Server connection example (commented for demo)\n",
+    "# mcp_ingestor = MCPIngestor()\n",
+    "# mcp_ingestor.connect(\"financial_server\", url=\"http://localhost:8000/mcp\")\n",
+    "# resources = mcp_ingestor.list_available_resources(\"financial_server\")\n",
+    "# mcp_data = mcp_ingestor.ingest_resources(\"financial_server\", resource_uris=[\"resource://market_data\"])\n",
+    "\n",
+    "if not all_documents:\n",
+    "    market_data = \"\"\"\n",
+    "    AAPL stock price: $150.25, market cap: $2.4T, volume: 50M shares, sector: Technology\n",
+    "    MSFT stock price: $380.50, market cap: $2.8T, volume: 30M shares, sector: Technology\n",
+    "    GOOGL stock price: $140.75, market cap: $1.8T, volume: 25M shares, sector: Technology\n",
+    "    JPM stock price: $145.30, market cap: $420B, volume: 15M shares, sector: Financial\n",
+    "    \"\"\"\n",
+    "    with open(\"data/market_data.txt\", \"w\") as f:\n",
+    "        f.write(market_data)\n",
+    "    file_ingestor = FileIngestor()\n",
+    "    all_documents = file_ingestor.ingest(\"data/market_data.txt\")\n",
+    "\n",
+    "documents = all_documents\n",
+    "print(f\"Ingested {len(documents)} documents\")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Loaded 6 seed data items for market foundation\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.seed import SeedDataManager\n",
+    "\n",
+    "seed_manager = SeedDataManager()\n",
+    "\n",
+    "# Load foundation market data (exchanges, indices, sectors)\n",
+    "seed_data = [\n",
+    "    {\"type\": \"Market\", \"text\": \"NASDAQ\", \"description\": \"Stock exchange\"},\n",
+    "    {\"type\": \"Market\", \"text\": \"NYSE\", \"description\": \"Stock exchange\"},\n",
+    "    {\"type\": \"Market\", \"text\": \"S&P 500\", \"description\": \"Stock market index\"},\n",
+    "    {\"type\": \"Sector\", \"text\": \"Technology\", \"description\": \"Market sector\"},\n",
+    "    {\"type\": \"Sector\", \"text\": \"Financial\", \"description\": \"Market sector\"},\n",
+    "    {\"type\": \"Sector\", \"text\": \"Healthcare\", \"description\": \"Market sector\"},\n",
+    "]\n",
+    "\n",
+    "# Add seed data as entities\n",
+    "for item in seed_data:\n",
+    "    entity = {\n",
+    "        \"id\": item.get(\"text\", \"\").lower().replace(\" \", \"_\"),\n",
+    "        \"text\": item.get(\"text\", \"\"),\n",
+    "        \"name\": item.get(\"text\", \"\"),\n",
+    "        \"type\": item.get(\"type\", \"\"),\n",
+    "        \"description\": item.get(\"description\", \"\"),\n",
+    "        \"source\": \"seed_data\"\n",
+    "    }\n",
+    "    seed_manager.seed_data.entities.append(entity)\n",
+    "\n",
+    "print(f\"Loaded {len(seed_data)} seed data items for market foundation\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Parsing Financial Documents\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Parsing 100 documents...\n",
+      "🧠 Semantica is parsing: Document file not found: Jonathan Ferro, Lisa Abramowicz and Annmarie Hordern speak daily with leaders and decision makers from Wall Street to Washington and beyond. No other program better positions investors and executives for the trading day. (Source: Bloomberg) ❌🔍 (0.0s) | 🧠 Semantica is parsing: Document file not found: <p>A look at some of the year’s notable trades.<\\p> ❌🔍 (0.0s)ay. (Source: Bloomberg) ❌🔍 (0.0s)  Parsed 50/100 documents...\n",
+      "🧠 Semantica is parsing: Document file not found: Maverick Natural Resources has operations in Texas and Oklahoma. ❌🔍 (0.0s) | 🧠 Semantica is parsing: Document file not found: The president has told Bank of America and JPMorgan Chase to stop cutting conservatives off from doing business. ❌🔍 (0.0s)  Parsed 100/100 documents...\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.parse import DocumentParser\n",
+    "\n",
+    "parser = DocumentParser()\n",
+    "\n",
+    "print(f\"Parsing {len(documents)} documents...\")\n",
+    "parsed_documents = []\n",
+    "for i, doc in enumerate(documents, 1):\n",
+    "    try:\n",
+    "        parsed = parser.parse(\n",
+    "            doc.content if hasattr(doc, 'content') else str(doc),\n",
+    "            content_type=\"text\"\n",
+    "        )\n",
+    "        parsed_documents.append(parsed)\n",
+    "    except Exception:\n",
+    "        parsed_documents.append(doc)\n",
+    "    if i % 50 == 0 or i == len(documents):\n",
+    "        print(f\"  Parsed {i}/{len(documents)} documents...\")\n",
+    "\n",
+    "documents = parsed_documents\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Normalizing and Chunking Financial Data\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Normalizing 100 documents...\n",
+      "🧠 Semantica is parsing: Document file not found: The president has told Bank of America and JPMorgan Chase to stop cutting conservatives off from doing business. ❌🔍 (0.0s) | 🧠 Normalizing text 🔄🔧 (0.0s)  Normalized 50/100 documents...\n",
+      "🧠 Semantica is parsing: Document file not found: The president has told Bank of America and JPMorgan Chase to stop cutting conservatives off from doing business. ❌🔍 (0.0s) | 🧠 Normalizing text 🔄🔧 (0.0s)  Normalized 100/100 documents...\n",
+      "Chunking 100 documents...\n",
+      "  Chunked 50/100 documents (50 chunks so far)\n",
+      "  Chunked 100/100 documents (100 chunks so far)\n",
+      "Created 100 chunks from 100 documents\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.normalize import TextNormalizer\n",
+    "from semantica.split import TextSplitter\n",
+    "\n",
+    "normalizer = TextNormalizer()\n",
+    "# Use recursive chunking for financial documents\n",
+    "splitter = TextSplitter(method=\"recursive\", chunk_size=CHUNK_SIZE, chunk_overlap=CHUNK_OVERLAP)\n",
+    "\n",
+    "print(f\"Normalizing {len(documents)} documents...\")\n",
+    "normalized_documents = []\n",
+    "for i, doc in enumerate(documents, 1):\n",
+    "    normalized_text = normalizer.normalize(\n",
+    "        doc.content if hasattr(doc, 'content') else str(doc),\n",
+    "        clean_html=True,\n",
+    "        normalize_entities=True,\n",
+    "        normalize_numbers=True,\n",
+    "        remove_extra_whitespace=True,\n",
+    "        lowercase=False\n",
+    "    )\n",
+    "    normalized_documents.append(normalized_text)\n",
+    "    if i % 50 == 0 or i == len(documents):\n",
+    "        print(f\"  Normalized {i}/{len(documents)} documents...\")\n",
+    "\n",
+    "print(f\"Chunking {len(normalized_documents)} documents...\")\n",
+    "chunked_documents = []\n",
+    "for i, doc_text in enumerate(normalized_documents, 1):\n",
+    "    try:\n",
+    "        with redirect_stderr(StringIO()):\n",
+    "            chunks = splitter.split(doc_text)\n",
+    "        chunked_documents.extend(chunks)\n",
+    "    except Exception:\n",
+    "        simple_splitter = TextSplitter(method=\"recursive\", chunk_size=CHUNK_SIZE, chunk_overlap=CHUNK_OVERLAP)\n",
+    "        chunks = simple_splitter.split(doc_text)\n",
+    "        chunked_documents.extend(chunks)\n",
+    "    if i % 50 == 0 or i == len(normalized_documents):\n",
+    "        print(f\"  Chunked {i}/{len(normalized_documents)} documents ({len(chunked_documents)} chunks so far)\")\n",
+    "\n",
+    "print(f\"Created {len(chunked_documents)} chunks from {len(normalized_documents)} documents\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Extracting Financial Entities\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting entities from 100 chunks...\n",
+      "🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 4 entities using ml |███████████████| 100.0% ✅🎯  Processed 20/100 chunks (56 entities found)\n",
+      "🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 5 entities using ml |███████████████| 100.0% ✅🎯�  Processed 40/100 chunks (160 entities found)\n",
+      "🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯  Processed 60/100 chunks (182 entities found)\n",
+      "🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 1 entities using ml |███████████████| 100.0% ✅🎯  Processed 80/100 chunks (215 entities found)\n",
+      "🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯  Processed 100/100 chunks (264 entities found)\n",
+      "Extracted 66 companies/organizations, 59 markets/locations, 8 prices, 24 metrics\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.semantic_extract import NERExtractor\n",
+    "\n",
+    "entity_extractor = NERExtractor(\n",
+    "    method=\"ml\",\n",
+    "    model=\"en_core_web_sm\"\n",
+    ")\n",
+    "\n",
+    "all_entities = []\n",
+    "print(f\"Extracting entities from {len(chunked_documents)} chunks...\")\n",
+    "for i, chunk in enumerate(chunked_documents, 1):\n",
+    "    chunk_text = chunk.text if hasattr(chunk, 'text') else str(chunk)\n",
+    "    try:\n",
+    "        entities = entity_extractor.extract_entities(chunk_text)\n",
+    "        all_entities.extend(entities)\n",
+    "    except Exception:\n",
+    "        continue\n",
+    "    \n",
+    "    if i % 20 == 0 or i == len(chunked_documents):\n",
+    "        print(f\"  Processed {i}/{len(chunked_documents)} chunks ({len(all_entities)} entities found)\")\n",
+    "\n",
+    "# Categorize entities using spaCy's standard types (ORG, GPE, MONEY, etc.)\n",
+    "companies = [e for e in all_entities if e.label in [\"ORG\", \"ORGANIZATION\"]]\n",
+    "markets = [e for e in all_entities if e.label in [\"GPE\", \"LOCATION\", \"LOC\"]]\n",
+    "prices = [e for e in all_entities if e.label in [\"MONEY\", \"CURRENCY\"]]\n",
+    "metrics = [e for e in all_entities if e.label in [\"CARDINAL\", \"QUANTITY\", \"PERCENT\", \"PERCENTAGE\"]]\n",
+    "\n",
+    "print(f\"Extracted {len(companies)} companies/organizations, {len(markets)} markets/locations, {len(prices)} prices, {len(metrics)} metrics\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Extracting Financial Relationships\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting relationships from 100 chunks...\n",
+      "🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 4 relations using dependency |███████████████| 100.0% ✅🎯  Processed 20/100 chunks (49 relationships found)\n",
+      "🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 1 relations using dependency |███████████████| 100.0% ✅🎯  Processed 40/100 chunks (111 relationships found)\n",
+      "🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 0 relations |███████████████| 100.0% ✅🎯█████| 100.0% ✅🎯  Processed 60/100 chunks (121 relationships found)\n",
+      "🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 0 relations |███████████████| 100.0% ✅🎯█████| 100.0% ✅🎯  Processed 80/100 chunks (135 relationships found)\n",
+      "🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 3 relations using dependency |███████████████| 100.0% ✅🎯  Processed 100/100 chunks (175 relationships found)\n",
+      "Extracted 175 relationships\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.semantic_extract import RelationExtractor\n",
+    "\n",
+    "relation_extractor = RelationExtractor(\n",
+    "    method=\"dependency\",\n",
+    "    model=\"en_core_web_sm\"\n",
+    ")\n",
+    "\n",
+    "all_relationships = []\n",
+    "print(f\"Extracting relationships from {len(chunked_documents)} chunks...\")\n",
+    "for i, chunk in enumerate(chunked_documents, 1):\n",
+    "    chunk_text = chunk.text if hasattr(chunk, 'text') else str(chunk)\n",
+    "    try:\n",
+    "        relationships = relation_extractor.extract_relations(\n",
+    "            chunk_text,\n",
+    "            entities=all_entities,\n",
+    "            relation_types=[\"trades_on\", \"has_price\", \"belongs_to\", \"correlates_with\", \"has_metric\", \"in_sector\"]\n",
+    "        )\n",
+    "        all_relationships.extend(relationships)\n",
+    "    except Exception:\n",
+    "        continue\n",
+    "    \n",
+    "    if i % 20 == 0 or i == len(chunked_documents):\n",
+    "        print(f\"  Processed {i}/{len(chunked_documents)} chunks ({len(all_relationships)} relationships found)\")\n",
+    "\n",
+    "print(f\"Extracted {len(all_relationships)} relationships\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Resolving Duplicate Companies and Stocks\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conflict Detection\n",
+    "\n",
+    "- **Temporal Conflict Detection**: Detects time-sensitive conflicts in financial data from multiple sources\n",
+    "- **Most Recent Strategy**: Resolves conflicts by prioritizing the latest market data\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Detecting temporal conflicts in 264 entities...\n",
+      "🧠 Semantica is extracting: Extracted 3 relations using dependency |███████████████| 100.0% ✅🎯 | 🧠 Semantica is resolving: Detected 0 temporal conflicts |███████████████| 100.0% [165/193] ✅⚠️ (223.9/s)████████░░░| 85.5% [165/193] 🔄⚠️ (ETA: 0.1s | 225.1/s)Detected 0 temporal conflicts\n",
+      "No conflicts detected\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.conflicts import ConflictDetector, ConflictResolver\n",
+    "\n",
+    "conflict_detector = ConflictDetector()\n",
+    "conflict_resolver = ConflictResolver()\n",
+    "\n",
+    "# Convert Entity objects to dictionaries for conflict detection\n",
+    "entity_dicts = [\n",
+    "    {\n",
+    "        \"id\": e.text if hasattr(e, 'text') else str(e),\n",
+    "        \"text\": e.text if hasattr(e, 'text') else str(e),\n",
+    "        \"name\": e.text if hasattr(e, 'text') else str(e),\n",
+    "        \"type\": e.label if hasattr(e, 'label') else \"ENTITY\",\n",
+    "        \"confidence\": e.confidence if hasattr(e, 'confidence') else 1.0,\n",
+    "        \"metadata\": e.metadata if hasattr(e, 'metadata') else {},\n",
+    "        \"source\": e.metadata.get(\"source\", \"unknown\") if hasattr(e, 'metadata') and isinstance(e.metadata, dict) else \"unknown\"\n",
+    "    }\n",
+    "    for e in all_entities\n",
+    "]\n",
+    "\n",
+    "print(f\"Detecting temporal conflicts in {len(entity_dicts)} entities...\")\n",
+    "conflicts = conflict_detector.detect_temporal_conflicts(entity_dicts)\n",
+    "\n",
+    "print(f\"Detected {len(conflicts)} temporal conflicts\")\n",
+    "\n",
+    "if conflicts:\n",
+    "    print(f\"Resolving conflicts using most_recent strategy...\")\n",
+    "    resolved = conflict_resolver.resolve_conflicts(\n",
+    "        conflicts,\n",
+    "        strategy=\"most_recent\"\n",
+    "    )\n",
+    "    print(f\"Resolved {len(resolved)} conflicts\")\n",
+    "else:\n",
+    "    print(\"No conflicts detected\")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Converting 264 entities to dictionaries...\n",
+      "Resolving duplicates in 264 entities...\n",
+      "🧠 Semantica is deduplicating: Merging groups... 1/1 (remaining: 0) |███████████████| 100.0% [1/1] 🔄🔄 (16.4/s) | 🧠 Semantica is deduplicating: Building merged entity... (4/4, remaining: 0 steps) |███████████████| 100.0% [4/4] 🔄🔄 (258.3/s) 🔄🔄 (ETA: 0.0s | 374.6/s)333.9/s)Converting 36 resolved entities back to Entity objects...\n",
+      "Deduplicated 264 entities to 36 unique entities\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.kg import EntityResolver\n",
+    "from semantica.semantic_extract import Entity\n",
+    "\n",
+    "# Convert Entity objects to dictionaries for EntityResolver\n",
+    "print(f\"Converting {len(all_entities)} entities to dictionaries...\")\n",
+    "entity_dicts = [{\"name\": e.text, \"type\": e.label, \"start_char\": getattr(e, 'start_char', 0), \"end_char\": getattr(e, 'end_char', 0), \"confidence\": e.confidence} for e in all_entities]\n",
+    "\n",
+    "# Use EntityResolver class to resolve duplicates\n",
+    "entity_resolver = EntityResolver(strategy=\"fuzzy\", similarity_threshold=0.85)\n",
+    "\n",
+    "print(f\"Resolving duplicates in {len(entity_dicts)} entities...\")\n",
+    "resolved_entities = entity_resolver.resolve_entities(entity_dicts)\n",
+    "\n",
+    "# Convert back to Entity objects\n",
+    "print(f\"Converting {len(resolved_entities)} resolved entities back to Entity objects...\")\n",
+    "merged_entities = [\n",
+    "    Entity(text=e[\"name\"], label=e[\"type\"], start_char=e.get(\"start_char\", 0), end_char=e.get(\"end_char\", 0), confidence=e.get(\"confidence\", 1.0))\n",
+    "    for e in resolved_entities\n",
+    "]\n",
+    "\n",
+    "# Enhance entities with seed data information\n",
+    "for entity in merged_entities:\n",
+    "    for seed_item in seed_data:\n",
+    "        if entity.text.lower() == seed_item[\"text\"].lower():\n",
+    "            entity.description = seed_item.get(\"description\", \"\")\n",
+    "            break\n",
+    "\n",
+    "print(f\"Deduplicated {len(entity_dicts)} entities to {len(merged_entities)} unique entities\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Building Financial Knowledge Graph\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Building knowledge graph...\n",
+      "🧠 Semantica is deduplicating: Building merged entity... (4/4, remaining: 0 steps) |███████████████| 100.0% [4/4] 🔄🔄 (258.3/s) | 🧠 Semantica is building: Processing relationships... 175/175 |███████████████| 100.0% [175/175] 🔄🧠 (27889.8/s) 15937.0/s)Building graph structure...\n",
+      "✅ Graph structure built (0.00s)\n",
+      "🧠 Semantica is deduplicating: Building merged entity... (4/4, remaining: 0 steps) |███████████████| 100.0% [4/4] 🔄🔄 (258.3/s) | 🧠 Semantica is building: Processing relationships... 175/175 |███████████████| 100.0% [175/175] 🔄🧠 (27889.8/s)\n",
+      "============================================================\n",
+      "✅ Knowledge Graph Build Complete\n",
+      "   Entities: 36\n",
+      "   Relationships: 175\n",
+      "   Total time: 0.86s\n",
+      "============================================================\n",
+      "Graph: 36 entities, 175 relationships\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.kg import GraphBuilder\n",
+    "\n",
+    "graph_builder = GraphBuilder()\n",
+    "\n",
+    "print(f\"Building knowledge graph...\")\n",
+    "kg_sources = [{\n",
+    "    \"entities\": [{\"text\": e.text, \"type\": e.label, \"confidence\": e.confidence} for e in merged_entities],\n",
+    "    \"relationships\": [{\"source\": r.subject.text, \"target\": r.object.text, \"type\": r.predicate, \"confidence\": r.confidence} for r in all_relationships]\n",
+    "}]\n",
+    "\n",
+    "kg = graph_builder.build(kg_sources)\n",
+    "\n",
+    "entities_count = len(kg.get('entities', []))\n",
+    "relationships_count = len(kg.get('relationships', []))\n",
+    "print(f\"Graph: {entities_count} entities, {relationships_count} relationships\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Generating Embeddings for Companies and Stocks\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Generating embeddings for 66 companies and 59 markets...\n",
+      "Generated 66 company embeddings and 59 market embeddings\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.embeddings import EmbeddingGenerator\n",
+    "\n",
+    "embedding_gen = EmbeddingGenerator(\n",
+    "    provider=\"sentence_transformers\",\n",
+    "    model=EMBEDDING_MODEL\n",
+    ")\n",
+    "\n",
+    "print(f\"Generating embeddings for {len(companies)} companies and {len(markets)} markets...\")\n",
+    "company_texts = [c.text for c in companies]\n",
+    "company_embeddings = embedding_gen.generate_embeddings(company_texts)\n",
+    "\n",
+    "market_texts = [m.text for m in markets]\n",
+    "market_embeddings = embedding_gen.generate_embeddings(market_texts)\n",
+    "\n",
+    "print(f\"Generated {len(company_embeddings)} company embeddings and {len(market_embeddings)} market embeddings\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Populating Vector Store\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Storing 66 company vectors and 59 market vectors...\n",
+      "🧠 Semantica is building: Processing relationships... 175/175 |███████████████| 100.0% [175/175] 🔄🧠 (27889.8/s) | 🧠 Semantica is indexing: Storing 66 vectors 🔄📊 (0.0s)Stored 66 company vectors and 59 market vectors\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.vector_store import VectorStore\n",
+    "\n",
+    "vector_store = VectorStore(backend=\"faiss\", dimension=EMBEDDING_DIMENSION)\n",
+    "\n",
+    "print(f\"Storing {len(company_embeddings)} company vectors and {len(market_embeddings)} market vectors...\")\n",
+    "company_ids = vector_store.store_vectors(\n",
+    "    vectors=company_embeddings,\n",
+    "    metadata=[{\"type\": \"company\", \"name\": c.text, \"label\": c.label} for c in companies]\n",
+    ")\n",
+    "\n",
+    "market_ids = vector_store.store_vectors(\n",
+    "    vectors=market_embeddings,\n",
+    "    metadata=[{\"type\": \"market\", \"name\": m.text, \"label\": m.label} for m in markets]\n",
+    ")\n",
+    "\n",
+    "print(f\"Stored {len(company_ids)} company vectors and {len(market_ids)} market vectors\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Analyzing Market Network Structure\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "🧠 Semantica is indexing: Storing 66 vectors 🔄📊 (0.0s) | 🧠 Semantica is building: Calculating degree centrality 🔄🧠 (0.0s)Graph analytics:\n",
+      "  - Graph density: 0.000\n",
+      "  - Central nodes (degree): 4\n",
+      "  - Total entities: 36\n",
+      "  - Total relationships: 175\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.kg import GraphAnalyzer, CentralityCalculator\n",
+    "\n",
+    "graph_analyzer = GraphAnalyzer()\n",
+    "centrality_calc = CentralityCalculator()\n",
+    "\n",
+    "analysis = graph_analyzer.analyze_graph(kg)\n",
+    "\n",
+    "degree_centrality = centrality_calc.calculate_degree_centrality(kg)\n",
+    "betweenness_centrality = centrality_calc.calculate_betweenness_centrality(kg)\n",
+    "closeness_centrality = centrality_calc.calculate_closeness_centrality(kg)\n",
+    "\n",
+    "# Identify central entities in the market network\n",
+    "central_entities = []\n",
+    "for entity in kg.get(\"entities\", []):\n",
+    "    entity_id = entity.get(\"id\")\n",
+    "    if entity_id in degree_centrality:\n",
+    "        central_entities.append({\n",
+    "            \"name\": entity.get(\"text\", \"Unknown\"),\n",
+    "            \"type\": entity.get(\"type\", \"Unknown\"),\n",
+    "            \"degree\": degree_centrality[entity_id]\n",
+    "        })\n",
+    "\n",
+    "central_entities.sort(key=lambda x: x['degree'], reverse=True)\n",
+    "\n",
+    "print(f\"Graph analytics:\")\n",
+    "print(f\"  - Graph density: {analysis.get('density', 0):.3f}\")\n",
+    "print(f\"  - Central nodes (degree): {len(degree_centrality)}\")\n",
+    "print(f\"  - Total entities: {len(kg.get('entities', []))}\")\n",
+    "print(f\"  - Total relationships: {len(kg.get('relationships', []))}\")\n",
+    "if central_entities:\n",
+    "    print(f\"\\nTop 5 central entities:\")\n",
+    "    for i, ent in enumerate(central_entities[:5], 1):\n",
+    "        print(f\"  {i}. {ent['name']} ({ent['type']}) - Degree: {ent['degree']:.3f}\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## GraphRAG: Hybrid Vector + Graph Queries\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "================================================================================\n",
+      "GraphRAG Query: What technology companies are in the market and what are their key relationships?\n",
+      "================================================================================\n",
+      "\n",
+      "🧠 Semantica is embedding: Generated embedding (dim: 128) |███████████████| 100.0% ✅💾 | 🧠 Semantica is processing: Ranking results... 🔄🔗 (0.0s)technology companies are in the market?... 🔄🔗 (0.0s)🔄🔗 (0.0s)s)ket?... 🔄🔗 (0.0s)"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Embedding generation failed: Text cannot be empty or whitespace-only\n",
+      "Using random fallback embedding\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "================================================================================\n",
+      "Generated Answer (with Multi-hop Reasoning):\n",
+      "================================================================================\n",
+      "Based on the retrieved context and reasoning paths, I can identify some technology companies and their key relationships. However, the context provided does not directly mention specific technology companies. \n",
+      "\n",
+      "One entity that is mentioned as a GPE (Geopolitical Entity) is Russia, which is related to the year through the reasoning path \"Russia --[be]--> the year\" (Path 1). This connection suggests that Russia is associated with a specific year, but the year is not explicitly stated.\n",
+      "\n",
+      "Another entity mentioned is Ukraine, which is also a GPE. It is related to the year through the reasoning path \"Ukraine --[be]--> the year\" (Path 1), similar to Russia. \n",
+      "\n",
+      "The only technology company mentioned in the context is Nvidia, but it is not related to any other entities or relationships in the context.\n",
+      "\n",
+      "To answer the question, I would say that there are limited technology companies mentioned in the context, and their relationships are not explicitly stated. However, based on the reasoning paths, we can infer that Russia and Ukraine are associated with a specific year.\n",
+      "\n",
+      "Key relationships:\n",
+      "\n",
+      "- Russia and Ukraine are associated with a specific year.\n",
+      "- The year is related to Russia and Ukraine through the reasoning path \"Russia --[be]--> the year\" (Path 1) and \"Ukraine --[be]--> the year\" (Path 1).\n",
+      "\n",
+      "Note: The context does not provide sufficient information to identify specific technology companies and their relationships.\n",
+      "\n",
+      "================================================================================\n",
+      "\n",
+      "Reasoning Details:\n",
+      "- Confidence: 0.636\n",
+      "- Sources: 15\n",
+      "- Reasoning Paths: 49\n",
+      "- Total entities in graph: 36\n",
+      "- Total relationships in graph: 175\n",
+      "\n",
+      "Top Sources:\n",
+      "  1. Score: 0.736\n",
+      "     the final days of the year is a DATE. is outstriped by Warren Buffett. is rallyed by Masayoshi Son’s. say Russia (GPE). buy Donald Trump’s (PERSON), New York City (GPE). is selled by Jonathan Ferro....\n",
+      "  2. Score: 0.728\n",
+      "     2026 is a DATE. is looked by Bloomberg’s Skylar Montgomery Koning....\n",
+      "  3. Score: 0.651\n",
+      "     the year is a DATE. is beed by Russia (GPE). sell the Christmas holiday, Ukraine (GPE), Pinnacle Group and 1 more. rally the Christmas holiday, Ukraine (GPE)....\n",
+      "  4. Score: 0.597\n",
+      "     Ukraine is a GPE. position the trading day. is selled by the year (DATE). is broadened by Joe Weisenthal. is rallyed by the year (DATE)....\n",
+      "  5. Score: 0.591\n",
+      "     Nvidia is a GPE....\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.context import AgentContext\n",
+    "from semantica.llms import Groq\n",
+    "import os\n",
+    "\n",
+    "context = AgentContext(\n",
+    "    vector_store=vector_store,\n",
+    "    knowledge_graph=kg,\n",
+    "    max_expansion_hops=3,\n",
+    "    hybrid_alpha=0.7\n",
+    ")\n",
+    "\n",
+    "# Initialize Groq LLM\n",
+    "llm = Groq(model=\"llama-3.1-8b-instant\", api_key=os.getenv(\"GROQ_API_KEY\"))\n",
+    "\n",
+    "query = \"What technology companies are in the market and what are their key relationships?\"\n",
+    "\n",
+    "print(f\"{'='*80}\")\n",
+    "print(f\"GraphRAG Query: {query}\")\n",
+    "print(f\"{'='*80}\\n\")\n",
+    "\n",
+    "# Use multi-hop reasoning with LLM generation\n",
+    "result = context.query_with_reasoning(\n",
+    "    query=query,\n",
+    "    llm_provider=llm,\n",
+    "    max_results=15,\n",
+    "    max_hops=3,\n",
+    "    min_score=0.2\n",
+    ")\n",
+    "\n",
+    "print(\"=\" * 80)\n",
+    "print(\"Generated Answer (with Multi-hop Reasoning):\")\n",
+    "print(\"=\" * 80)\n",
+    "response = result.get('response', 'No response generated')\n",
+    "print(response)\n",
+    "print(\"\\n\" + \"=\" * 80)\n",
+    "\n",
+    "print(f\"\\nReasoning Details:\")\n",
+    "print(f\"- Confidence: {result.get('confidence', 0):.3f}\")\n",
+    "print(f\"- Sources: {result.get('num_sources', 0)}\")\n",
+    "print(f\"- Reasoning Paths: {result.get('num_reasoning_paths', 0)}\")\n",
+    "print(f\"- Total entities in graph: {len(kg.get('entities', []))}\")\n",
+    "print(f\"- Total relationships in graph: {len(kg.get('relationships', []))}\")\n",
+    "\n",
+    "if result.get('sources'):\n",
+    "    print(f\"\\nTop Sources:\")\n",
+    "    for i, source in enumerate(result['sources'][:5], 1):\n",
+    "        content = source.get('content', '')[:200] if isinstance(source, dict) else str(source)[:200]\n",
+    "        score = source.get('score', 0) if isinstance(source, dict) else 0\n",
+    "        print(f\"  {i}. Score: {score:.3f}\")\n",
+    "        print(f\"     {content}...\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Exporting Results\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "🧠 Semantica is processing: Ranking results... 🔄🔗 (0.0s) | 🧠 Semantica is exporting: Exporting graph to json: financial_data_kg.json 🔄💾 (0.0s)Exported financial knowledge graph to JSON, GraphML, and CSV formats\n"
+     ]
+    }
+   ],
+   "source": [
+    "from semantica.export import GraphExporter\n",
+    "\n",
+    "exporter = GraphExporter()\n",
+    "exporter.export(kg, output_path=\"financial_data_kg.json\", format=\"json\")\n",
+    "exporter.export(kg, output_path=\"financial_data_kg.graphml\", format=\"graphml\")\n",
+    "\n",
+    "print(\"Exported financial knowledge graph to JSON, GraphML, and CSV formats\")\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

cookbook/use_cases/finance/02_Financial_Reports_Analysis.ipynb

@@ -1,436 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/finance/02_Financial_Reports_Analysis.ipynb)\n",
-    "\n",
-    "# Financial Reports Analysis Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete financial reports analysis pipeline: ingest financial documents from multiple sources (SEC filings, annual reports, financial databases), extract financial entities, build knowledge graph, analyze relationships, and generate financial insights.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, EmailIngestor, RepoIngestor, MCPIngestor\n",
-    "- **Parsing**: DocumentParser, PDFParser, HTMLParser, StructuredDataParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, TripletExtractor, SemanticAnalyzer\n",
-    "- **KG**: GraphBuilder, GraphAnalyzer, CentralityCalculator, CommunityDetector\n",
-    "- **Analytics**: ConnectivityAnalyzer, TemporalGraphQuery, TemporalPatternDetector\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-
-    "- **Export**: JSONExporter, CSVExporter, RDFExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, AnalyticsVisualizer, TemporalVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Financial Documents \u2192 Parse \u2192 Extract Entities \u2192 Build KG \u2192 Analyze Relationships \u2192 Generate Insights \u2192 Export \u2192 Visualize**\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Ingest Financial Documents from Multiple Sources\n",
-    "\n",
-    "Ingest financial reports from SEC filings, annual reports, and financial databases.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import FileIngestor, WebIngestor, DBIngestor, FeedIngestor\n",
-    "from semantica.parse import DocumentParser, PDFParser, HTMLParser, StructuredDataParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, TripletExtractor, SemanticAnalyzer\n",
-    "from semantica.kg import GraphBuilder, GraphAnalyzer, CentralityCalculator, CommunityDetector\n",
-    "from semantica.kg import ConnectivityAnalyzer, TemporalGraphQuery, TemporalPatternDetector\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-
-    "from semantica.export import JSONExporter, CSVExporter, RDFExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, AnalyticsVisualizer, TemporalVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "file_ingestor = FileIngestor()\n",
-    "web_ingestor = WebIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "feed_ingestor = FeedIngestor()\n",
-    "\n",
-    "document_parser = DocumentParser()\n",
-    "pdf_parser = PDFParser()\n",
-    "html_parser = HTMLParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "\n",
-    "# Real financial data sources\n",
-    "sec_edgar_urls = [\n",
-    "    \"https://www.sec.gov/cgi-bin/browse-edgar\",  # SEC EDGAR database\n",
-    "    \"https://www.sec.gov/Archives/edgar/data/\",  # SEC EDGAR archives\n",
-    "    \"https://data.sec.gov/submissions/\"  # SEC submissions API\n",
-    "]\n",
-    "\n",
-    "financial_feeds = [\n",
-    "    \"https://feeds.reuters.com/reuters/businessNews\",\n",
-    "    \"https://feeds.reuters.com/reuters/topNews\",\n",
-    "    \"https://rss.cnn.com/rss/money_latest.rss\",\n",
-    "    \"https://feeds.bloomberg.com/markets/news.rss\"\n",
-    "]\n",
-    "\n",
-    "# Real database connection for financial reports\n",
-    "db_connection_string = \"postgresql://user:password@localhost:5432/financial_reports_db\"\n",
-    "db_query = \"SELECT company_name, report_type, filing_date, document_url FROM financial_reports WHERE filing_date > CURRENT_DATE - INTERVAL '1 year' ORDER BY filing_date DESC\"\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "# Sample financial report data (simulating real SEC filing structure)\n",
-    "financial_report_file = os.path.join(temp_dir, \"financial_report.json\")\n",
-    "report_data = {\n",
-    "    \"company\": \"Apple Inc.\",\n",
-    "    \"symbol\": \"AAPL\",\n",
-    "    \"report_type\": \"10-K\",\n",
-    "    \"filing_date\": (datetime.now() - timedelta(days=30)).isoformat(),\n",
-    "    \"revenue\": 394328000000,\n",
-    "    \"net_income\": 99803000000,\n",
-    "    \"total_assets\": 352755000000,\n",
-    "    \"total_liabilities\": 290437000000,\n",
-    "    \"segments\": [\"iPhone\", \"Mac\", \"iPad\", \"Services\", \"Wearables\"],\n",
-    "    \"geographic_regions\": [\"Americas\", \"Europe\", \"Greater China\", \"Japan\", \"Rest of Asia Pacific\"]\n",
-    "}\n",
-    "\n",
-    "with open(financial_report_file, 'w') as f:\n",
-    "    json.dump(report_data, f, indent=2)\n",
-    "\n",
-    "file_objects = file_ingestor.ingest_file(financial_report_file, read_content=True)\n",
-    "parsed_data = structured_parser.parse_data(financial_report_file, data_format=\"json\")\n",
-    "\n",
-    "# Ingest from financial feeds\n",
-    "financial_feed_list = []\n",
-    "for feed_url in financial_feeds[:2]:  # Process first 2 feeds\n",
-    "    feed_data = feed_ingestor.ingest_feed(feed_url)\n",
-    "    if feed_data:\n",
-    "        financial_feed_list.append(feed_data)\n",
-    "        print(f\"  Ingested feed: {feed_url}\")\n",
-    "\n",
-    "# Ingest from SEC EDGAR (example)\n",
-    "web_content = web_ingestor.ingest_url(\"https://www.sec.gov/cgi-bin/browse-edgar\")\n",
-    "if web_content:\n",
-    "    print(f\"  Ingested SEC EDGAR content\")\n",
-    "\n",
-    "print(f\"\\n\ud83d\udcca Ingestion Summary:\")\n",
-    "print(f\"  Financial reports ingested: {len([file_objects]) if file_objects else 0}\")\n",
-    "print(f\"  Financial feeds: {len(financial_feed_list)}\")\n",
-    "print(f\"  Database sources: 1\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Extract Financial Entities\n",
-    "\n",
-    "Extract financial entities (companies, metrics, segments, regions) from financial reports.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "triplet_extractor = TripletExtractor()\n",
-    "semantic_analyzer = SemanticAnalyzer()\n",
-    "\n",
-    "financial_entities = []\n",
-    "financial_relationships = []\n",
-    "\n",
-    "# Extract from financial report data\n",
-    "if parsed_data and parsed_data.data:\n",
-    "    report = parsed_data.data if isinstance(parsed_data.data, dict) else parsed_data.data[0] if isinstance(parsed_data.data, list) else {}\n",
-    "    \n",
-    "    if isinstance(report, dict):\n",
-    "        # Company entity\n",
-    "        financial_entities.append({\n",
-    "            \"id\": report.get(\"symbol\", \"\"),\n",
-    "            \"type\": \"Company\",\n",
-    "            \"name\": report.get(\"company\", \"\"),\n",
-    "            \"properties\": {\n",
-    "                \"symbol\": report.get(\"symbol\", \"\"),\n",
-    "                \"report_type\": report.get(\"report_type\", \"\")\n",
-    "            }\n",
-    "        })\n",
-    "        \n",
-    "        # Financial metrics\n",
-    "        financial_entities.append({\n",
-    "            \"id\": f\"{report.get('symbol', '')}_revenue\",\n",
-    "            \"type\": \"Financial_Metric\",\n",
-    "            \"name\": \"Revenue\",\n",
-    "            \"properties\": {\n",
-    "                \"value\": report.get(\"revenue\", 0),\n",
-    "                \"currency\": \"USD\",\n",
-    "                \"filing_date\": report.get(\"filing_date\", \"\")\n",
-    "            }\n",
-    "        })\n",
-    "        \n",
-    "        financial_entities.append({\n",
-    "            \"id\": f\"{report.get('symbol', '')}_net_income\",\n",
-    "            \"type\": \"Financial_Metric\",\n",
-    "            \"name\": \"Net Income\",\n",
-    "            \"properties\": {\n",
-    "                \"value\": report.get(\"net_income\", 0),\n",
-    "                \"currency\": \"USD\",\n",
-    "                \"filing_date\": report.get(\"filing_date\", \"\")\n",
-    "            }\n",
-    "        })\n",
-    "        \n",
-    "        # Segments\n",
-    "        for segment in report.get(\"segments\", []):\n",
-    "            financial_entities.append({\n",
-    "                \"id\": f\"{report.get('symbol', '')}_segment_{segment}\",\n",
-    "                \"type\": \"Business_Segment\",\n",
-    "                \"name\": segment,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            financial_relationships.append({\n",
-    "                \"source\": report.get(\"symbol\", \"\"),\n",
-    "                \"target\": f\"{report.get('symbol', '')}_segment_{segment}\",\n",
-    "                \"type\": \"has_segment\",\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "        \n",
-    "        # Geographic regions\n",
-    "        for region in report.get(\"geographic_regions\", []):\n",
-    "            financial_entities.append({\n",
-    "                \"id\": f\"{report.get('symbol', '')}_region_{region}\",\n",
-    "                \"type\": \"Geographic_Region\",\n",
-    "                \"name\": region,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            financial_relationships.append({\n",
-    "                \"source\": report.get(\"symbol\", \"\"),\n",
-    "                \"target\": f\"{report.get('symbol', '')}_region_{region}\",\n",
-    "                \"type\": \"operates_in\",\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "        \n",
-    "        # Relationships\n",
-    "        financial_relationships.append({\n",
-    "            \"source\": report.get(\"symbol\", \"\"),\n",
-    "            \"target\": f\"{report.get('symbol', '')}_revenue\",\n",
-    "            \"type\": \"has_metric\",\n",
-    "            \"properties\": {\"filing_date\": report.get(\"filing_date\", \"\")}\n",
-    "        })\n",
-    "        \n",
-    "        financial_relationships.append({\n",
-    "            \"source\": report.get(\"symbol\", \"\"),\n",
-    "            \"target\": f\"{report.get('symbol', '')}_net_income\",\n",
-    "            \"type\": \"has_metric\",\n",
-    "            \"properties\": {\"filing_date\": report.get(\"filing_date\", \"\")}\n",
-    "        })\n",
-    "\n",
-    "print(f\"Extracted {len(financial_entities)} financial entities\")\n",
-    "print(f\"Extracted {len(financial_relationships)} financial relationships\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Build Financial Knowledge Graph\n",
-    "\n",
-    "Build knowledge graph from financial entities and relationships.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "centrality_calculator = CentralityCalculator()\n",
-    "community_detector = CommunityDetector()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "\n",
-    "financial_kg = builder.build(financial_entities, financial_relationships)\n",
-    "\n",
-    "# Analyze graph structure\n",
-    "metrics = graph_analyzer.compute_metrics(financial_kg)\n",
-    "centrality_result = centrality_calculator.calculate_degree_centrality(financial_kg)\n",
-    "centrality_scores = centrality_result.get('centrality', {})\n",
-    "communities = community_detector.detect_communities(financial_kg)\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(financial_kg)\n",
-    "\n",
-    "print(f\"Built financial knowledge graph\")\n",
-    "print(f\"  Entities: {len(financial_kg.get('entities', []))}\")\n",
-    "print(f\"  Relationships: {len(financial_kg.get('relationships', []))}\")\n",
-    "print(f\"  Graph density: {metrics.get('density', 0):.3f}\")\n",
-    "print(f\"  Communities: {len(communities)}\")\n",
-    "print(f\"  Central entities: {len([e for e, score in centrality_scores.items() if score > 0])}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Analyze Financial Relationships\n",
-    "\n",
-    "Analyze financial relationships and generate insights.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "temporal_query = TemporalGraphQuery()\n",
-    "temporal_pattern_detector = TemporalPatternDetector()\n",
-    "inference_engine = InferenceEngine()\n",
-    "rule_manager = RuleManager()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-    "\n",
-    "# Temporal analysis\n",
-    "start_time = (datetime.now() - timedelta(days=365)).isoformat()\n",
-    "end_time = datetime.now().isoformat()\n",
-    "\n",
-    "temporal_results = temporal_query.query_time_range(\n",
-    "    graph=financial_kg,\n",
-    "    query=\"Find financial metrics in the last year\",\n",
-    "    start_time=start_time,\n",
-    "    end_time=end_time\n",
-    ")\n",
-    "\n",
-    "temporal_patterns = temporal_pattern_detector.detect_temporal_patterns(\n",
-    "    financial_kg,\n",
-    "    pattern_type=\"trend\",\n",
-    "    min_frequency=1\n",
-    ")\n",
-    "\n",
-    "# Financial analysis rules\n",
-    "inference_engine.add_rule(\"IF revenue > 300000000000 AND net_income > 50000000000 THEN high_performer\")\n",
-    "inference_engine.add_rule(\"IF company has_segment Services AND revenue > 20000000000 THEN services_growth\")\n",
-    "\n",
-    "# Add facts from financial data\n",
-    "if parsed_data and parsed_data.get(\"data\"):\n",
-    "    report = parsed_data.get(\"data\") if isinstance(parsed_data.get(\"data\"), dict) else parsed_data.get(\"data\")[0] if isinstance(parsed_data.get(\"data\"), list) else {}\n",
-    "    if isinstance(report, dict):\n",
-    "        inference_engine.add_fact({\n",
-    "            \"company\": report.get(\"symbol\", \"\"),\n",
-    "            \"revenue\": report.get(\"revenue\", 0),\n",
-    "            \"net_income\": report.get(\"net_income\", 0),\n",
-    "            \"segments\": report.get(\"segments\", [])\n",
-    "        })\n",
-    "\n",
-    "financial_insights = inference_engine.forward_chain()\n",
-    "\n",
-    "print(f\"Temporal query returned {len(temporal_results.get('entities', []))} entities\")\n",
-    "print(f\"Detected {len(temporal_patterns)} temporal patterns\")\n",
-    "print(f\"Generated {len(financial_insights)} financial insights\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Generate Financial Analysis Reports\n",
-    "\n",
-    "Generate comprehensive financial analysis reports.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-
-    "json_exporter = JSONExporter()\n",
-    "csv_exporter = CSVExporter()\n",
-    "rdf_exporter = RDFExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-    "quality_score = quality_assessor.assess_overall_quality(financial_kg)\n",
-    "\n",
-    "json_exporter.export_knowledge_graph(financial_kg, os.path.join(temp_dir, \"financial_kg.json\"))\n",
-    "csv_exporter.export_entities(financial_entities, os.path.join(temp_dir, \"financial_entities.csv\"))\n",
-    "rdf_exporter.export_knowledge_graph(financial_kg, os.path.join(temp_dir, \"financial_kg.rdf\"))\n",
-    "\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Financial analysis identified {len(financial_insights)} insights from {len(financial_entities)} entities\",\n",
-    "    \"entities_analyzed\": len(financial_entities),\n",
-    "    \"relationships\": len(financial_relationships),\n",
-    "    \"insights\": len(financial_insights),\n",
-    "    \"quality_score\": quality_score.get('overall_score', 0),\n",
-    "    \"patterns\": len(temporal_patterns)\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"Report length: {len(report)} characters\")\n",
-    "print(f\"Graph quality score: {quality_score.get('overall_score', 0):.3f}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 6: Visualize Financial Analysis\n",
-    "\n",
-    "Visualize financial knowledge graph and relationships.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "kg_visualizer = KGVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(financial_kg, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(financial_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(financial_kg, output=\"interactive\")\n",
-    "\n",
-    "print(f\"Total modules used: 20+\")\n",
-    "print(f\"Pipeline complete: Financial Documents \u2192 Parse \u2192 Extract \u2192 Build KG \u2192 Analyze Relationships \u2192 Generate Insights \u2192 Export \u2192 Visualize\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/finance/03_Fraud_Detection.ipynb

@@ -1,427 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/finance/03_Fraud_Detection.ipynb)\n",
-    "\n",
-    "# Fraud Detection Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete fraud detection pipeline for finance: ingest transaction streams, build temporal knowledge graph, detect fraud patterns, perform anomaly detection, and generate alerts.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, EmailIngestor, RepoIngestor, MCPIngestor\n",
-    "- **Parsing**: StructuredDataParser, DocumentParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector\n",
-    "- **KG**: GraphBuilder, TemporalPatternDetector, GraphAnalyzer\n",
-    "- **Graph Store**: GraphStore with Neo4j/FalkorDB for persistent fraud graph\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "- **Quality**: KGQualityAssessor, AutomatedFixer\n",
-    "- **Export**: JSONExporter, CSVExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Transaction Stream → Parse → Extract → Build Temporal KG → Store in Graph DB → Detect Patterns → Anomaly Detection → Generate Alerts → Visualize**\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Process Transactions\n",
-    "\n",
-    "Ingest and parse transaction data from multiple sources.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import FileIngestor, StreamIngestor, DBIngestor\n",
-    "from semantica.parse import StructuredDataParser, DocumentParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector\n",
-    "from semantica.kg import GraphBuilder, TemporalPatternDetector, GraphAnalyzer\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "from semantica.export import JSONExporter, CSVExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "file_ingestor = FileIngestor()\n",
-    "stream_ingestor = StreamIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "document_parser = DocumentParser()\n",
-    "\n",
-    "# Real streaming sources for transaction monitoring\n",
-    "stream_sources = [\n",
-    "    {\n",
-    "        \"type\": \"kafka\",\n",
-    "        \"topic\": \"transactions\",\n",
-    "        \"bootstrap_servers\": [\"localhost:9092\"],\n",
-    "        \"consumer_config\": {\"group_id\": \"fraud_detection\"}\n",
-    "    },\n",
-    "    {\n",
-    "        \"type\": \"rabbitmq\",\n",
-    "        \"queue\": \"payment_events\",\n",
-    "        \"connection_url\": \"amqp://user:password@localhost:5672/\"\n",
-    "    }\n",
-    "]\n",
-    "\n",
-    "# Real database connection for transaction data\n",
-    "db_connection_string = \"postgresql://user:password@localhost:5432/transactions_db\"\n",
-    "db_query = \"SELECT transaction_id, user_id, amount, merchant, location, timestamp, device FROM transactions WHERE timestamp > NOW() - INTERVAL '24 hours' ORDER BY timestamp DESC LIMIT 10000\"\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "transactions_file = os.path.join(temp_dir, \"transactions.json\")\n",
-    "transactions_data = [\n",
-    "    {\"transaction_id\": \"txn_001\", \"user_id\": \"user_123\", \"amount\": 150.00, \"merchant\": \"Online Store\", \"location\": \"New York\", \"timestamp\": (datetime.now() - timedelta(hours=1)).isoformat(), \"device\": \"mobile\"},\n",
-    "    {\"transaction_id\": \"txn_002\", \"user_id\": \"user_123\", \"amount\": 2500.00, \"merchant\": \"Luxury Store\", \"location\": \"Paris\", \"timestamp\": (datetime.now() - timedelta(minutes=30)).isoformat(), \"device\": \"web\"},\n",
-    "    {\"transaction_id\": \"txn_003\", \"user_id\": \"user_456\", \"amount\": 50.00, \"merchant\": \"Grocery Store\", \"location\": \"San Francisco\", \"timestamp\": (datetime.now() - timedelta(minutes=15)).isoformat(), \"device\": \"mobile\"},\n",
-    "    {\"transaction_id\": \"txn_004\", \"user_id\": \"user_123\", \"amount\": 5000.00, \"merchant\": \"Electronics Store\", \"location\": \"Tokyo\", \"timestamp\": (datetime.now() - timedelta(minutes=5)).isoformat(), \"device\": \"mobile\"}\n",
-    "]\n",
-    "\n",
-    "with open(transactions_file, 'w') as f:\n",
-    "    json.dump(transactions_data, f)\n",
-    "\n",
-    "file_objects = file_ingestor.ingest_file(transactions_file, read_content=True)\n",
-    "parsed_data = structured_parser.parse_json(transactions_file)\n",
-    "\n",
-    "transaction_stream = []\n",
-    "for txn in parsed_data.get(\"data\", transactions_data):\n",
-    "    if isinstance(txn, dict):\n",
-    "        txn_copy = txn.copy()\n",
-    "        if \"timestamp\" in txn_copy and isinstance(txn_copy[\"timestamp\"], str):\n",
-    "            txn_copy[\"timestamp\"] = datetime.fromisoformat(txn_copy[\"timestamp\"])\n",
-    "        transaction_stream.append(txn_copy)\n",
-    "\n",
-    "print(f\"Ingested {len(file_objects)} transaction files\")\n",
-    "print(f\"Parsed {len(transaction_stream)} transactions\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Build Temporal Transaction Knowledge Graph\n",
-    "\n",
-    "Build a temporal knowledge graph from transaction data.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "\n",
-    "transaction_entities = []\n",
-    "relationships = []\n",
-    "\n",
-    "for txn in transaction_stream:\n",
-    "    txn_id = txn[\"transaction_id\"]\n",
-    "    user_id = txn[\"user_id\"]\n",
-    "    merchant = txn[\"merchant\"]\n",
-    "    location = txn[\"location\"]\n",
-    "\n",
-    "    transaction_entities.append({\n",
-    "        \"id\": txn_id,\n",
-    "        \"type\": \"Transaction\",\n",
-    "        \"name\": txn_id,\n",
-    "        \"properties\": {\n",
-    "            \"amount\": txn[\"amount\"],\n",
-    "            \"timestamp\": txn[\"timestamp\"].isoformat() if isinstance(txn[\"timestamp\"], datetime) else txn[\"timestamp\"],\n",
-    "            \"device\": txn[\"device\"]\n",
-    "        }\n",
-    "    })\n",
-    "\n",
-    "    transaction_entities.append({\n",
-    "        \"id\": user_id,\n",
-    "        \"type\": \"User\",\n",
-    "        \"name\": user_id,\n",
-    "        \"properties\": {}\n",
-    "    })\n",
-    "\n",
-    "    transaction_entities.append({\n",
-    "        \"id\": merchant,\n",
-    "        \"type\": \"Merchant\",\n",
-    "        \"name\": merchant,\n",
-    "        \"properties\": {}\n",
-    "    })\n",
-    "\n",
-    "    transaction_entities.append({\n",
-    "        \"id\": location,\n",
-    "        \"type\": \"Location\",\n",
-    "        \"name\": location,\n",
-    "        \"properties\": {}\n",
-    "    })\n",
-    "\n",
-    "    relationships.append({\n",
-    "        \"source\": user_id,\n",
-    "        \"target\": txn_id,\n",
-    "        \"type\": \"performed\",\n",
-    "        \"properties\": {\"timestamp\": txn[\"timestamp\"].isoformat() if isinstance(txn[\"timestamp\"], datetime) else txn[\"timestamp\"]}\n",
-    "    })\n",
-    "\n",
-    "    relationships.append({\n",
-    "        \"source\": txn_id,\n",
-    "        \"target\": merchant,\n",
-    "        \"type\": \"at_merchant\",\n",
-    "        \"properties\": {}\n",
-    "    })\n",
-    "\n",
-    "    relationships.append({\n",
-    "        \"source\": txn_id,\n",
-    "        \"target\": location,\n",
-    "        \"type\": \"in_location\",\n",
-    "        \"properties\": {}\n",
-    "    })\n",
-    "\n",
-    "transaction_kg = builder.build(transaction_entities, relationships)\n",
-    "\n",
-    "print(f\"Built temporal knowledge graph with {len(transaction_entities)} entities and {len(relationships)} relationships\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Store in Graph Database\n",
-    "\n",
-    "Store the transaction graph in a persistent graph database for real-time fraud queries.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.graph_store import GraphStore\n",
-    "\n",
-    "# Initialize graph store - FalkorDB is ideal for real-time fraud detection\n",
-    "# For production: use FalkorDB with Redis for ultra-fast queries\n",
-    "# graph_store = GraphStore(backend=\"falkordb\", host=\"localhost\", port=6379, graph_name=\"fraud_graph\")\n",
-    "\n",
-    "# Option 1: Neo4j (requires Neo4j server running)\n",
-    "    graph_store = GraphStore(backend=\"neo4j\", uri=\"bolt://localhost:7687\", user=\"neo4j\", password=\"password\")\n",
-    "    graph_store.connect()\n",
-    "\n",
-    "# Store transaction entities in graph database\n",
-    "node_id_map = {}\n",
-    "\n",
-    "for entity in transaction_entities:\n",
-    "    node = graph_store.create_node(\n",
-    "        labels=[entity[\"type\"]],\n",
-    "        properties={\n",
-    "            \"name\": entity[\"name\"],\n",
-    "            \"original_id\": entity[\"id\"],\n",
-    "            **entity.get(\"properties\", {})\n",
-    "        }\n",
-    "    )\n",
-    "    node_id_map[entity[\"id\"]] = node.get(\"id\")\n",
-    "\n",
-    "print(f\"Stored {len(transaction_entities)} entities in graph database\")\n",
-    "\n",
-    "# Store relationships\n",
-    "for rel in relationships:\n",
-    "    if rel[\"source\"] in node_id_map and rel[\"target\"] in node_id_map:\n",
-    "        graph_store.create_relationship(\n",
-    "            start_node_id=node_id_map[rel[\"source\"]],\n",
-    "            end_node_id=node_id_map[rel[\"target\"]],\n",
-    "            rel_type=rel[\"type\"],\n",
-    "            properties=rel.get(\"properties\", {})\n",
-    "        )\n",
-    "\n",
-    "print(f\"Stored {len(relationships)} relationships in graph database\")\n",
-    "\n",
-    "# Query for high-value transactions using Cypher\n",
-    "high_value_query = \"\"\"\n",
-    "    MATCH (t:Transaction)\n",
-    "    WHERE t.amount > 1000\n",
-    "    RETURN t.name as transaction_id, t.amount as amount\n",
-    "\"\"\"\n",
-    "high_value_results = graph_store.execute_query(high_value_query)\n",
-    "print(f\"Found {len(high_value_results.get('records', []))} high-value transactions\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Detect Fraud Patterns\n",
-    "\n",
-    "Detect fraud patterns using graph queries and temporal analysis.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Use graph queries to detect fraud patterns\n",
-    "pattern_detector = TemporalPatternDetector()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "\n",
-    "# Query for users with multiple transactions in short time\n",
-    "rapid_transactions_query = \"\"\"\n",
-    "    MATCH (u:User)-[:performed]->(t:Transaction)\n",
-    "    WITH u, count(t) as txn_count\n",
-    "    WHERE txn_count > 2\n",
-    "    RETURN u.name as user_id, txn_count\n",
-    "\"\"\"\n",
-    "rapid_txn_results = graph_store.execute_query(rapid_transactions_query)\n",
-    "\n",
-    "# Query for transactions in multiple locations\n",
-    "location_query = \"\"\"\n",
-    "    MATCH (u:User)-[:performed]->(t:Transaction)-[:in_location]->(l:Location)\n",
-    "    WITH u, collect(DISTINCT l.name) as locations\n",
-    "    WHERE size(locations) > 1\n",
-    "    RETURN u.name as user_id, locations\n",
-    "\"\"\"\n",
-    "location_results = graph_store.execute_query(location_query)\n",
-    "\n",
-    "# Detect fraud patterns from analysis\n",
-    "fraud_patterns = []\n",
-    "user_transactions = {}\n",
-    "for txn in transaction_stream:\n",
-    "    user_id = txn[\"user_id\"]\n",
-    "    if user_id not in user_transactions:\n",
-    "        user_transactions[user_id] = []\n",
-    "    user_transactions[user_id].append(txn)\n",
-    "\n",
-    "for user_id, txns in user_transactions.items():\n",
-    "    if len(txns) > 1:\n",
-    "        amounts = [t[\"amount\"] for t in txns]\n",
-    "        locations = [t[\"location\"] for t in txns]\n",
-    "        \n",
-    "        if max(amounts) > 1000:\n",
-    "            fraud_patterns.append({\n",
-    "                \"type\": \"high_value_transaction\",\n",
-    "                \"user_id\": user_id,\n",
-    "                \"amount\": max(amounts),\n",
-    "                \"severity\": \"medium\"\n",
-    "            })\n",
-    "\n",
-    "        if len(set(locations)) > 2:\n",
-    "            fraud_patterns.append({\n",
-    "                \"type\": \"rapid_location_change\",\n",
-    "                \"user_id\": user_id,\n",
-    "                \"locations\": list(set(locations)),\n",
-    "                \"severity\": \"high\"\n",
-    "            })\n",
-    "\n",
-    "print(f\"Detected {len(fraud_patterns)} fraud patterns using graph analysis\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Generate Alerts and Reports\n",
-    "\n",
-    "Generate fraud alerts and reports.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "json_exporter = JSONExporter()\n",
-    "csv_exporter = CSVExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-    "json_exporter.export_knowledge_graph(transaction_kg, os.path.join(temp_dir, \"transactions.json\"))\n",
-    "csv_exporter.export_entities(transaction_entities, os.path.join(temp_dir, \"entities.csv\"))\n",
-    "\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Fraud detection analysis identified {len(anomalies)} suspicious transactions\",\n",
-    "    \"fraud_patterns\": len(fraud_patterns),\n",
-    "    \"anomalies\": len(anomalies),\n",
-    "    \"transactions_analyzed\": len(transaction_stream)\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"Report length: {len(report)} characters\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 6: Quality Assessment and Visualization\n",
-    "\n",
-    "Assess graph quality and visualize results.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "kg_visualizer = KGVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(transaction_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(transaction_kg, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(transaction_kg, output=\"interactive\")\n",
-    "\n",
-    "# Get final graph store statistics\n",
-    "graph_stats = graph_store.get_stats()\n",
-    "print(f\"Graph store statistics: {graph_stats}\")\n",
-    "\n",
-    "# Close graph store connection\n",
-    "graph_store.close()\n",
-    "\n",
-    "print(f\"Graph quality score: {quality_score.get('overall_score', 0):.3f}\")\n",
-    "print(f\"Total modules used: 20+ (including GraphStore)\")\n",
-    "print(f\"Pipeline complete: Transaction Stream → Parse → Extract → Temporal KG → Store in Graph DB → Pattern Detection → Anomaly Detection → Reports → Visualization\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/finance/04_Investment_Analysis_Hybrid_RAG.ipynb

@@ -1,415 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/finance/04_Investment_Analysis_Hybrid_RAG.ipynb)\n",
-    "\n",
-    "# Investment Analysis Hybrid RAG Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete investment analysis hybrid RAG pipeline: ingest investment data from multiple sources (market data APIs, financial feeds, databases), extract investment entities, build knowledge graph, generate embeddings, set up hybrid search (vector + temporal KG), and query investment insights using advanced RAG.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, EmailIngestor, RepoIngestor, MCPIngestor\n",
-    "- **Parsing**: JSONParser, CSVParser, StructuredDataParser, HTMLParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector, SemanticAnalyzer\n",
-    "- **KG**: GraphBuilder, TemporalGraphQuery, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "- **Embeddings**: EmbeddingGenerator, TextEmbedder\n",
-    "- **Vector Store**: VectorStore, HybridSearch\n",
-    "- **Context**: ContextRetriever, ContextGraphBuilder\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "- **Export**: JSONExporter, CSVExporter, RDFExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Multi-Source Investment Data \u2192 Parse \u2192 Extract Entities \u2192 Build KG \u2192 Generate Embeddings \u2192 Vector Store \u2192 Hybrid RAG Setup \u2192 Query Insights \u2192 Generate Reports \u2192 Visualize**\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Multi-Source Investment Data Ingestion\n",
-    "\n",
-    "Ingest investment data from market APIs, financial feeds, and databases.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import FileIngestor, WebIngestor, DBIngestor, FeedIngestor\n",
-    "from semantica.parse import JSONParser, CSVParser, StructuredDataParser, HTMLParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector, SemanticAnalyzer\n",
-    "from semantica.kg import GraphBuilder, TemporalGraphQuery, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "from semantica.embeddings import EmbeddingGenerator, TextEmbedder\n",
-    "from semantica.vector_store import VectorStore, HybridSearch\n",
-    "from semantica.context import ContextRetriever, ContextGraphBuilder\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "from semantica.export import JSONExporter, CSVExporter, RDFExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, TemporalVisualizer, AnalyticsVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "file_ingestor = FileIngestor()\n",
-    "web_ingestor = WebIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "feed_ingestor = FeedIngestor()\n",
-    "\n",
-    "json_parser = JSONParser()\n",
-    "csv_parser = CSVParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "html_parser = HTMLParser()\n",
-    "\n",
-    "# Real investment data sources\n",
-    "investment_apis = [\n",
-    "    \"https://api.polygon.io/v2/aggs/ticker/AAPL/range/1/day/2024-01-01/2024-01-31\",  # Polygon.io\n",
-    "    \"https://www.alphavantage.co/query?function=OVERVIEW&symbol=AAPL&apikey=demo\",  # Alpha Vantage\n",
-    "    \"https://api.github.com/repos/ranaroussi/yfinance\"  # Yahoo Finance API\n",
-    "]\n",
-    "\n",
-    "financial_feeds = [\n",
-    "    \"https://feeds.reuters.com/reuters/businessNews\",\n",
-    "    \"https://feeds.reuters.com/reuters/topNews\",\n",
-    "    \"https://rss.cnn.com/rss/money_latest.rss\",\n",
-    "    \"https://feeds.bloomberg.com/markets/news.rss\"\n",
-    "]\n",
-    "\n",
-    "# Real database connection for investment data\n",
-    "db_connection_string = \"postgresql://user:password@localhost:5432/investment_db\"\n",
-    "db_query = \"SELECT symbol, company_name, sector, market_cap, pe_ratio, dividend_yield FROM investments WHERE last_updated > NOW() - INTERVAL '7 days' ORDER BY market_cap DESC\"\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "# Sample investment data\n",
-    "investment_data_file = os.path.join(temp_dir, \"investment_data.json\")\n",
-    "investment_data = [\n",
-    "    {\n",
-    "        \"symbol\": \"AAPL\",\n",
-    "        \"company\": \"Apple Inc.\",\n",
-    "        \"sector\": \"Technology\",\n",
-    "        \"market_cap\": 2800000000000,\n",
-    "        \"pe_ratio\": 28.5,\n",
-    "        \"dividend_yield\": 0.5,\n",
-    "        \"price\": 175.50,\n",
-    "        \"timestamp\": (datetime.now() - timedelta(days=1)).isoformat()\n",
-    "    },\n",
-    "    {\n",
-    "        \"symbol\": \"MSFT\",\n",
-    "        \"company\": \"Microsoft Corporation\",\n",
-    "        \"sector\": \"Technology\",\n",
-    "        \"market_cap\": 2800000000000,\n",
-    "        \"pe_ratio\": 32.1,\n",
-    "        \"dividend_yield\": 0.7,\n",
-    "        \"price\": 380.25,\n",
-    "        \"timestamp\": (datetime.now() - timedelta(days=1)).isoformat()\n",
-    "    }\n",
-    "]\n",
-    "\n",
-    "with open(investment_data_file, 'w') as f:\n",
-    "    json.dump(investment_data, f, indent=2)\n",
-    "\n",
-    "file_objects = file_ingestor.ingest_file(investment_data_file, read_content=True)\n",
-    "parsed_data = structured_parser.parse_json(investment_data_file)\n",
-    "\n",
-    "# Ingest from financial feeds\n",
-    "financial_feed_list = []\n",
-    "for feed_url in financial_feeds[:2]:\n",
-    "    feed_data = feed_ingestor.ingest_feed(feed_url)\n",
-    "    if feed_data:\n",
-    "        financial_feed_list.append(feed_data)\n",
-    "        print(f\"  Ingested feed: {feed_url}\")\n",
-    "\n",
-    "# Ingest from investment APIs\n",
-    "api_content_list = []\n",
-    "for api_url in investment_apis[:1]:\n",
-    "    api_content = web_ingestor.ingest_url(api_url)\n",
-    "    if api_content:\n",
-    "        api_content_list.append(api_content)\n",
-    "        print(f\"  Ingested API: {api_url}\")\n",
-    "\n",
-    "print(f\"\\n\ud83d\udcca Ingestion Summary:\")\n",
-    "print(f\"  Investment data files: {len([file_objects]) if file_objects else 0}\")\n",
-    "print(f\"  Financial feeds: {len(financial_feed_list)}\")\n",
-    "print(f\"  Investment APIs: {len(api_content_list)}\")\n",
-    "print(f\"  Database sources: 1\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Extract Investment Entities and Build Knowledge Graph\n",
-    "\n",
-    "Extract investment entities and build knowledge graph.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "event_detector = EventDetector()\n",
-    "semantic_analyzer = SemanticAnalyzer()\n",
-    "\n",
-    "investment_entities = []\n",
-    "investment_relationships = []\n",
-    "all_documents = []\n",
-    "\n",
-    "# Extract from investment data\n",
-    "if parsed_data and parsed_data.data:\n",
-    "    for investment in parsed_data.data if isinstance(parsed_data.data, list) else [parsed_data.data]:\n",
-    "        if isinstance(investment, dict):\n",
-    "            investment_text = f\"{investment.get('company', '')} ({investment.get('symbol', '')}) in {investment.get('sector', '')} sector\"\n",
-    "            all_documents.append(investment_text)\n",
-    "            \n",
-    "            investment_entities.append({\n",
-    "                \"id\": investment.get(\"symbol\", \"\"),\n",
-    "                \"type\": \"Stock\",\n",
-    "                \"name\": investment.get(\"company\", \"\"),\n",
-    "                \"properties\": {\n",
-    "                    \"symbol\": investment.get(\"symbol\", \"\"),\n",
-    "                    \"sector\": investment.get(\"sector\", \"\"),\n",
-    "                    \"market_cap\": investment.get(\"market_cap\", 0),\n",
-    "                    \"pe_ratio\": investment.get(\"pe_ratio\", 0),\n",
-    "                    \"dividend_yield\": investment.get(\"dividend_yield\", 0),\n",
-    "                    \"price\": investment.get(\"price\", 0),\n",
-    "                    \"timestamp\": investment.get(\"timestamp\", \"\")\n",
-    "                }\n",
-    "            })\n",
-    "            \n",
-    "            investment_entities.append({\n",
-    "                \"id\": investment.get(\"sector\", \"\"),\n",
-    "                \"type\": \"Sector\",\n",
-    "                \"name\": investment.get(\"sector\", \"\"),\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            \n",
-    "            investment_relationships.append({\n",
-    "                \"source\": investment.get(\"symbol\", \"\"),\n",
-    "                \"target\": investment.get(\"sector\", \"\"),\n",
-    "                \"type\": \"belongs_to\",\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "\n",
-    "builder = GraphBuilder()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "\n",
-    "investment_kg = builder.build(investment_entities, investment_relationships)\n",
-    "\n",
-    "metrics = graph_analyzer.compute_metrics(investment_kg)\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(investment_kg)\n",
-    "\n",
-    "print(f\"Extracted {len(investment_entities)} investment entities\")\n",
-    "print(f\"Extracted {len(investment_relationships)} relationships\")\n",
-    "print(f\"Collected {len(all_documents)} investment documents\")\n",
-    "print(f\"Built investment knowledge graph with {len(investment_kg.get('entities', []))} entities\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Generate Embeddings and Setup Vector Store\n",
-    "\n",
-    "Generate embeddings and setup vector store for hybrid RAG.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "embedding_generator = EmbeddingGenerator()\n",
-    "text_embedder = TextEmbedder()\n",
-    "vector_store = VectorStore()\n",
-    "hybrid_search = HybridSearch()\n",
-    "\n",
-    "embeddings = embedding_generator.generate_embeddings(all_documents, data_type=\"text\")\n",
-    "\n",
-    "metadata = []\n",
-    "for i, doc in enumerate(all_documents):\n",
-    "    metadata.append({\n",
-    "        \"id\": f\"doc_{i}\",\n",
-    "        \"text\": doc,\n",
-    "        \"source\": \"investment_data\"\n",
-    "    })\n",
-    "\n",
-    "vector_ids = vector_store.store_vectors(embeddings, metadata)\n",
-    "\n",
-    "print(f\"Generated embeddings for {len(all_documents)} documents\")\n",
-    "print(f\"Stored {len(vector_ids)} vectors in vector store\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Setup Hybrid RAG and Query Investment Insights\n",
-    "\n",
-    "Setup hybrid search and query investment insights.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "context_retriever = ContextRetriever(\n",
-    "    knowledge_graph=investment_kg,\n",
-    "    vector_store=vector_store\n",
-    ")\n",
-    "\n",
-    "temporal_query = TemporalGraphQuery()\n",
-    "inference_engine = InferenceEngine()\n",
-    "rule_manager = RuleManager()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-    "\n",
-    "# Query examples\n",
-    "queries = [\n",
-    "    \"What are the best performing sectors?\",\n",
-    "    \"Find technology stocks with high market cap\",\n",
-    "    \"What investments have good dividend yields?\"\n",
-    "]\n",
-    "\n",
-    "query_results = []\n",
-    "for query in queries:\n",
-    "    query_embedding = text_embedder.embed_text(query)\n",
-    "    vector_results = vector_store.search_vectors(query_embedding, k=3)\n",
-    "    \n",
-    "    start_time = (datetime.now() - timedelta(days=30)).isoformat()\n",
-    "    end_time = datetime.now().isoformat()\n",
-    "    \n",
-    "    temporal_results = temporal_query.query_time_range(\n",
-    "        graph=investment_kg,\n",
-    "        query=query,\n",
-    "        start_time=start_time,\n",
-    "        end_time=end_time\n",
-    "    )\n",
-    "    \n",
-    "    context_results = context_retriever.retrieve(\n",
-    "        query=query,\n",
-    "        top_k=3,\n",
-    "        use_graph_expansion=True\n",
-    "    )\n",
-    "    \n",
-    "    query_results.append({\n",
-    "        \"query\": query,\n",
-    "        \"vector_results\": len(vector_results),\n",
-    "        \"temporal_results\": len(temporal_results.get('entities', [])),\n",
-    "        \"context_results\": len(context_results) if context_results else 0\n",
-    "    })\n",
-    "\n",
-    "# Investment analysis rules\n",
-    "inference_engine.add_rule(\"IF pe_ratio < 20 AND dividend_yield > 0.5 THEN value_stock\")\n",
-    "inference_engine.add_rule(\"IF market_cap > 1000000000000 AND sector is Technology THEN mega_cap_tech\")\n",
-    "\n",
-    "for investment in parsed_data.data if parsed_data and parsed_data.data else []:\n",
-    "    if isinstance(investment, dict):\n",
-    "        inference_engine.add_fact({\n",
-    "            \"symbol\": investment.get(\"symbol\", \"\"),\n",
-    "            \"pe_ratio\": investment.get(\"pe_ratio\", 0),\n",
-    "            \"dividend_yield\": investment.get(\"dividend_yield\", 0),\n",
-    "            \"market_cap\": investment.get(\"market_cap\", 0),\n",
-    "            \"sector\": investment.get(\"sector\", \"\")\n",
-    "        })\n",
-    "\n",
-    "investment_insights = inference_engine.forward_chain()\n",
-    "\n",
-    "print(f\"Processed {len(queries)} investment queries\")\n",
-    "for result in query_results:\n",
-    "    print(f\"  Query: '{result['query']}' - Vector: {result['vector_results']}, Temporal: {result['temporal_results']}, Context: {result['context_results']}\")\n",
-    "print(f\"Generated {len(investment_insights)} investment insights\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Generate Reports and Visualize\n",
-    "\n",
-    "Generate investment analysis reports and visualize results.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-
-    "json_exporter = JSONExporter()\n",
-    "csv_exporter = CSVExporter()\n",
-    "rdf_exporter = RDFExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-    "quality_score = quality_assessor.assess_overall_quality(investment_kg)\n",
-    "\n",
-    "json_exporter.export_knowledge_graph(investment_kg, os.path.join(temp_dir, \"investment_kg.json\"))\n",
-    "csv_exporter.export_entities(investment_entities, os.path.join(temp_dir, \"investment_entities.csv\"))\n",
-    "rdf_exporter.export_knowledge_graph(investment_kg, os.path.join(temp_dir, \"investment_kg.rdf\"))\n",
-    "\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Investment analysis identified {len(investment_insights)} insights from {len(investment_entities)} entities\",\n",
-    "    \"investments_analyzed\": len(parsed_data.data) if parsed_data and parsed_data.data else 0,\n",
-    "    \"insights\": len(investment_insights),\n",
-    "    \"quality_score\": quality_score.get('overall_score', 0)\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "kg_visualizer = KGVisualizer()\n",
-    "temporal_visualizer = TemporalVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(investment_kg, output=\"interactive\")\n",
-    "temporal_viz = temporal_visualizer.visualize_timeline(investment_kg, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(investment_kg, output=\"interactive\")\n",
-    "\n",
-    "print(f\"Total modules used: 20+\")\n",
-    "print(f\"Pipeline complete: Multi-Source Investment Data \u2192 Parse \u2192 Extract \u2192 Build KG \u2192 Embeddings \u2192 Vector Store \u2192 Hybrid RAG \u2192 Query \u2192 Reports \u2192 Visualize\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

cookbook/use_cases/finance/05_Regulatory_Compliance.ipynb

@@ -1,383 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/finance/06_Regulatory_Compliance.ipynb)\n",
-    "\n",
-    "# Regulatory Compliance Pipeline\n",
-    "\n",
-    "## Overview\n",
-    "\n",
-    "This notebook demonstrates a complete regulatory compliance pipeline: ingest regulatory documents from multiple sources (SEC, FINRA, regulatory databases), extract compliance rules, build compliance ontology, validate compliance, and generate compliance reports.\n",
-    "\n",
-    "\n",
-    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
-    "\n",
-    "## Installation\n",
-    "\n",
-    "Install Semantica from PyPI:\n",
-    "\n",
-    "```bash\n",
-    "pip install semantica\n",
-    "# Or with all optional dependencies:\n",
-    "pip install semantica[all]\n",
-    "```\n",
-    "\n",
-    "### Modules Used (20+)\n",
-    "\n",
-    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, EmailIngestor, RepoIngestor, MCPIngestor\n",
-    "- **Parsing**: DocumentParser, PDFParser, HTMLParser, StructuredDataParser\n",
-    "- **Extraction**: NERExtractor, RelationExtractor, TripletExtractor, SemanticAnalyzer\n",
-    "- **KG**: GraphBuilder, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "- **Ontology**: OntologyGenerator, ClassInferrer, PropertyGenerator\n",
-    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "- **Quality**: KGQualityAssessor, ValidationEngine, ConflictDetector\n",
-    "- **Export**: JSONExporter, CSVExporter, RDFExporter, OWLExporter, ReportGenerator\n",
-    "- **Visualization**: KGVisualizer, OntologyVisualizer, AnalyticsVisualizer\n",
-    "\n",
-    "### Pipeline\n",
-    "\n",
-    "**Regulatory Documents → Parse → Extract Compliance Rules → Build Compliance Ontology → Validate Compliance → Generate Reports → Visualize**\n",
-    "\n",
-    "---\n",
-    "\n",
-    "## Step 1: Ingest Regulatory Documents from Multiple Sources\n",
-    "\n",
-    "Ingest regulatory documents from SEC, FINRA, and regulatory databases.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install semantica\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from semantica.ingest import FileIngestor, WebIngestor, DBIngestor, FeedIngestor\n",
-    "from semantica.parse import DocumentParser, PDFParser, HTMLParser, StructuredDataParser\n",
-    "from semantica.semantic_extract import NERExtractor, RelationExtractor, TripletExtractor, SemanticAnalyzer\n",
-    "from semantica.kg import GraphBuilder, GraphAnalyzer, ConnectivityAnalyzer\n",
-    "from semantica.ontology import OntologyGenerator, ClassInferrer, PropertyGenerator, OntologyValidator\n",
-    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
-    "from semantica.conflicts import ConflictDetector\n",
-    "from semantica.export import JSONExporter, CSVExporter, RDFExporter, OWLExporter, ReportGenerator\n",
-    "from semantica.visualization import KGVisualizer, OntologyVisualizer, AnalyticsVisualizer\n",
-    "import tempfile\n",
-    "import os\n",
-    "import json\n",
-    "from datetime import datetime, timedelta\n",
-    "\n",
-    "file_ingestor = FileIngestor()\n",
-    "web_ingestor = WebIngestor()\n",
-    "db_ingestor = DBIngestor()\n",
-    "feed_ingestor = FeedIngestor()\n",
-    "\n",
-    "document_parser = DocumentParser()\n",
-    "pdf_parser = PDFParser()\n",
-    "html_parser = HTMLParser()\n",
-    "structured_parser = StructuredDataParser()\n",
-    "\n",
-    "# Real regulatory data sources\n",
-    "regulatory_sources = [\n",
-    "    \"https://www.sec.gov/rules/final.shtml\",  # SEC Final Rules\n",
-    "    \"https://www.finra.org/rules-guidance\",  # FINRA Rules\n",
-    "    \"https://www.federalreserve.gov/newsevents/pressreleases.htm\"  # Federal Reserve\n",
-    "]\n",
-    "\n",
-    "regulatory_feeds = [\n",
-    "    \"https://feeds.reuters.com/reuters/businessNews\",\n",
-    "    \"https://rss.cnn.com/rss/money_latest.rss\"\n",
-    "]\n",
-    "\n",
-    "# Real database connection for regulatory documents\n",
-    "db_connection_string = \"postgresql://user:password@localhost:5432/regulatory_db\"\n",
-    "db_query = \"SELECT regulation_id, title, effective_date, compliance_requirements FROM regulations WHERE effective_date > CURRENT_DATE - INTERVAL '1 year' ORDER BY effective_date DESC\"\n",
-    "\n",
-    "temp_dir = tempfile.mkdtemp()\n",
-    "\n",
-    "# Sample regulatory document data\n",
-    "regulatory_file = os.path.join(temp_dir, \"regulatory_document.json\")\n",
-    "regulatory_data = {\n",
-    "    \"regulation_id\": \"REG-2024-001\",\n",
-    "    \"title\": \"Data Privacy and Security Requirements\",\n",
-    "    \"effective_date\": (datetime.now() - timedelta(days=60)).isoformat(),\n",
-    "    \"compliance_requirements\": [\n",
-    "        \"Encrypt sensitive customer data\",\n",
-    "        \"Maintain audit logs for 7 years\",\n",
-    "        \"Report breaches within 72 hours\",\n",
-    "        \"Conduct annual security assessments\"\n",
-    "    ],\n",
-    "    \"applicable_entities\": [\"Financial Institutions\", \"Broker-Dealers\", \"Investment Advisors\"],\n",
-    "    \"penalties\": {\n",
-    "        \"non_compliance\": \"Fines up to $1M per violation\",\n",
-    "        \"willful_violation\": \"Criminal penalties\"\n",
-    "    }\n",
-    "}\n",
-    "\n",
-    "with open(regulatory_file, 'w') as f:\n",
-    "    json.dump(regulatory_data, f, indent=2)\n",
-    "\n",
-    "file_objects = file_ingestor.ingest_file(regulatory_file, read_content=True)\n",
-    "parsed_data = structured_parser.parse_json(regulatory_file)\n",
-    "\n",
-    "# Ingest from regulatory sources\n",
-    "regulatory_web_list = []\n",
-    "for source_url in regulatory_sources[:1]:\n",
-    "    web_content = web_ingestor.ingest_url(source_url)\n",
-    "    if web_content:\n",
-    "        regulatory_web_list.append(web_content)\n",
-    "        print(f\"  Ingested regulatory source: {source_url}\")\n",
-    "\n",
-    "print(f\"\\n📊 Ingestion Summary:\")\n",
-    "print(f\"  Regulatory documents: {len([file_objects]) if file_objects else 0}\")\n",
-    "print(f\"  Regulatory web sources: {len(regulatory_web_list)}\")\n",
-    "print(f\"  Database sources: 1\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 2: Extract Compliance Rules\n",
-    "\n",
-    "Extract compliance rules and requirements from regulatory documents.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "ner_extractor = NERExtractor()\n",
-    "relation_extractor = RelationExtractor()\n",
-    "triplet_extractor = TripletExtractor()\n",
-    "semantic_analyzer = SemanticAnalyzer()\n",
-    "\n",
-    "compliance_entities = []\n",
-    "compliance_relationships = []\n",
-    "\n",
-    "# Extract from regulatory data\n",
-    "if parsed_data and parsed_data.data:\n",
-    "    regulation = parsed_data.data if isinstance(parsed_data.data, dict) else parsed_data.data[0] if isinstance(parsed_data.data, list) else {}\n",
-    "    \n",
-    "    if isinstance(regulation, dict):\n",
-    "        # Regulation entity\n",
-    "        compliance_entities.append({\n",
-    "            \"id\": regulation.get(\"regulation_id\", \"\"),\n",
-    "            \"type\": \"Regulation\",\n",
-    "            \"name\": regulation.get(\"title\", \"\"),\n",
-    "            \"properties\": {\n",
-    "                \"effective_date\": regulation.get(\"effective_date\", \"\"),\n",
-    "                \"regulation_id\": regulation.get(\"regulation_id\", \"\")\n",
-    "            }\n",
-    "        })\n",
-    "        \n",
-    "        # Compliance requirements\n",
-    "        for i, requirement in enumerate(regulation.get(\"compliance_requirements\", [])):\n",
-    "            compliance_entities.append({\n",
-    "                \"id\": f\"{regulation.get('regulation_id', '')}_req_{i}\",\n",
-    "                \"type\": \"Compliance_Requirement\",\n",
-    "                \"name\": requirement,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            compliance_relationships.append({\n",
-    "                \"source\": regulation.get(\"regulation_id\", \"\"),\n",
-    "                \"target\": f\"{regulation.get('regulation_id', '')}_req_{i}\",\n",
-    "                \"type\": \"has_requirement\",\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "        \n",
-    "        # Applicable entities\n",
-    "        for entity_type in regulation.get(\"applicable_entities\", []):\n",
-    "            compliance_entities.append({\n",
-    "                \"id\": entity_type,\n",
-    "                \"type\": \"Regulated_Entity\",\n",
-    "                \"name\": entity_type,\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "            compliance_relationships.append({\n",
-    "                \"source\": regulation.get(\"regulation_id\", \"\"),\n",
-    "                \"target\": entity_type,\n",
-    "                \"type\": \"applies_to\",\n",
-    "                \"properties\": {}\n",
-    "            })\n",
-    "\n",
-    "print(f\"Extracted {len(compliance_entities)} compliance entities\")\n",
-    "print(f\"Extracted {len(compliance_relationships)} compliance relationships\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 3: Build Compliance Ontology\n",
-    "\n",
-    "Build compliance ontology from extracted rules and requirements.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "builder = GraphBuilder()\n",
-    "ontology_generator = OntologyGenerator()\n",
-    "class_inferrer = ClassInferrer()\n",
-    "property_generator = PropertyGenerator()\n",
-    "ontology_validator = OntologyValidator()\n",
-    "\n",
-    "compliance_kg = builder.build(compliance_entities, compliance_relationships)\n",
-    "\n",
-    "compliance_ontology = ontology_generator.generate_ontology({\"entities\": compliance_entities, \"relationships\": compliance_relationships}, entities=compliance_entities, relationships=compliance_relationships)\n",
-    "\n",
-    "classes = class_inferrer.infer_classes(compliance_entities)\n",
-    "properties = property_generator.infer_properties(compliance_entities, compliance_relationships, classes)\n",
-    "\n",
-    "validation_result = ontology_validator.validate_ontology(compliance_ontology)\n",
-    "\n",
-    "print(f\"Built compliance knowledge graph\")\n",
-    "print(f\"  Entities: {len(compliance_kg.get('entities', []))}\")\n",
-    "print(f\"  Relationships: {len(compliance_kg.get('relationships', []))}\")\n",
-    "print(f\"Generated compliance ontology\")\n",
-    "print(f\"  Classes: {len(compliance_ontology.get('classes', []))}\")\n",
-    "print(f\"  Properties: {len(compliance_ontology.get('properties', []))}\")\n",
-    "print(f\"  Ontology valid: {validation_result.valid}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 4: Validate Compliance\n",
-    "\n",
-    "Validate data against compliance rules using inference engine.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "inference_engine = InferenceEngine()\n",
-    "rule_manager = RuleManager()\n",
-    "explanation_generator = ExplanationGenerator()\n",
-    "validation_engine = ValidationEngine()\n",
-    "graph_analyzer = GraphAnalyzer()\n",
-    "connectivity_analyzer = ConnectivityAnalyzer()\n",
-    "\n",
-    "# Define compliance validation rules\n",
-    "inference_engine.add_rule(\"IF data_encrypted is true AND audit_logs_maintained is true THEN data_security_compliant\")\n",
-    "inference_engine.add_rule(\"IF breach_reported_within_72h is true THEN breach_reporting_compliant\")\n",
-    "inference_engine.add_rule(\"IF annual_assessment_conducted is true THEN assessment_compliant\")\n",
-    "\n",
-    "# Sample data to validate\n",
-    "sample_data = {\n",
-    "    \"data_encrypted\": True,\n",
-    "    \"audit_logs_maintained\": True,\n",
-    "    \"breach_reported_within_72h\": True,\n",
-    "    \"annual_assessment_conducted\": True\n",
-    "}\n",
-    "\n",
-    "# Add facts for validation\n",
-    "for key, value in sample_data.items():\n",
-    "    inference_engine.add_fact({key: value})\n",
-    "\n",
-    "compliance_status = inference_engine.forward_chain()\n",
-    "\n",
-    "# Analyze compliance graph\n",
-    "metrics = graph_analyzer.compute_metrics(compliance_kg)\n",
-    "connectivity = connectivity_analyzer.analyze_connectivity(compliance_kg)\n",
-    "\n",
-    "print(f\"Compliance validation complete\")\n",
-    "print(f\"  Compliance status: {len(compliance_status)} rules satisfied\")\n",
-    "print(f\"  Graph metrics: density {metrics.get('density', 0):.3f}\")\n",
-    "print(f\"  Connected components: {len(connectivity.get('components', []))}\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 5: Generate Compliance Reports\n",
-    "\n",
-    "Generate comprehensive compliance reports.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "json_exporter = JSONExporter()\n",
-    "csv_exporter = CSVExporter()\n",
-    "rdf_exporter = RDFExporter()\n",
-    "owl_exporter = OWLExporter()\n",
-    "report_generator = ReportGenerator()\n",
-    "\n",
-    "json_exporter.export_knowledge_graph(compliance_kg, os.path.join(temp_dir, \"compliance_kg.json\"))\n",
-    "csv_exporter.export_entities(compliance_entities, os.path.join(temp_dir, \"compliance_entities.csv\"))\n",
-    "rdf_exporter.export_knowledge_graph(compliance_kg, os.path.join(temp_dir, \"compliance_kg.rdf\"))\n",
-    "owl_exporter.export(compliance_ontology, os.path.join(temp_dir, \"compliance_ontology.owl\"))\n",
-    "\n",
-    "report_data = {\n",
-    "    \"summary\": f\"Compliance validation identified {len(compliance_status)} satisfied rules\",\n",
-    "    \"regulations_analyzed\": len([e for e in compliance_entities if e.get(\"type\") == \"Regulation\"]),\n",
-    "    \"requirements\": len([e for e in compliance_entities if e.get(\"type\") == \"Compliance_Requirement\"]),\n",
-    "    \"compliance_status\": len(compliance_status)\n",
-    "}\n",
-    "\n",
-    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
-    "\n",
-    "print(f\"Report length: {len(report)} characters\")\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Step 6: Visualize Compliance\n",
-    "\n",
-    "Visualize compliance knowledge graph and ontology.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "kg_visualizer = KGVisualizer()\n",
-    "ontology_visualizer = OntologyVisualizer()\n",
-    "analytics_visualizer = AnalyticsVisualizer()\n",
-    "\n",
-    "kg_viz = kg_visualizer.visualize_network(compliance_kg, output=\"interactive\")\n",
-    "ontology_viz = ontology_visualizer.visualize_hierarchy(compliance_ontology, output=\"interactive\")\n",
-    "analytics_viz = analytics_visualizer.visualize_analytics(compliance_kg, output=\"interactive\")\n",
-    "\n",
-    "print(f\"Total modules used: 20+\")\n",
-    "print(f\"Pipeline complete: Regulatory Documents → Parse → Extract Rules → Build Ontology → Validate Compliance → Generate Reports → Visualize\")\n"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}