mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
docs: enhance GraphRAG notebooks with advanced features and update documentation
This commit is contained in:
KaifAhmad1
@@ -161,6 +161,28 @@ pip install -e ".[gpu]" # GPU support
|
||||
python -c "import semantica; print(semantica.__version__)"
|
||||
```
|
||||
|
||||
## 🍳 Semantica Cookbook
|
||||
|
||||
> **Interactive Jupyter Notebooks** designed to take you from beginner to expert.
|
||||
|
||||
[**View Full Cookbook**](https://github.com/Hawksight-AI/semantica/tree/main/cookbook)
|
||||
|
||||
### Featured Recipes
|
||||
|
||||
| **Recipe** | **Description** | **Link** |
|
||||
|:-----------|:----------------|:---------|
|
||||
| **GraphRAG Complete** | Build a production-ready **Graph Retrieval Augmented Generation** system. Features **Graph Validation**, **Hybrid Retrieval**, and **Logical Inference**. | [Open Notebook](cookbook/use_cases/advanced_rag/01_GraphRAG_Complete.ipynb) |
|
||||
| **RAG vs. GraphRAG** | Side-by-side comparison. Demonstrates the **Reasoning Gap** and how GraphRAG solves it with **Inference Engines**. | [Open Notebook](cookbook/use_cases/advanced_rag/02_RAG_vs_GraphRAG_Comparison.ipynb) |
|
||||
| **First Knowledge Graph** | Go from raw text to a queryable knowledge graph in 20 minutes. | [Open Notebook](cookbook/introduction/08_Your_First_Knowledge_Graph.ipynb) |
|
||||
| **Real-Time Anomalies** | Detect anomalies in streaming data using dynamic graphs. | [Open Notebook](cookbook/use_cases/cybersecurity/01_Anomaly_Detection_Real_Time.ipynb) |
|
||||
|
||||
### Core Tutorials
|
||||
|
||||
- [**Welcome to Semantica**](cookbook/introduction/01_Welcome_to_Semantica.ipynb) - Framework Overview
|
||||
- [**Data Ingestion**](cookbook/introduction/02_Data_Ingestion.ipynb) - Universal Ingestion
|
||||
- [**Entity Extraction**](cookbook/introduction/05_Entity_Extraction.ipynb) - NER & Relationships
|
||||
- [**Building Knowledge Graphs**](cookbook/introduction/07_Building_Knowledge_Graphs.ipynb) - Graph Construction
|
||||
|
||||
> **Note:** Once published to PyPI, you'll be able to install with `pip install semantica`
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,561 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# The Ultimate End-to-End GraphRAG Pipeline\n",
|
||||
"\n",
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook is the definitive guide to building high-performance, production-ready Knowledge Graph systems using the Semantica framework. We go beyond simple retrieval to demonstrate a full orchestration of the library's advanced capabilities.\n",
|
||||
"\n",
|
||||
"### What We Are Building\n",
|
||||
"\n",
|
||||
"We will develop a Self-Evolving Knowledge Base for \"Python Ecosystem Intelligence.\" This system will aggregate verified facts, real-time news, and technical documentation into a queryable, 3D-visualizable graph.\n",
|
||||
"\n",
|
||||
"### Modules Covered\n",
|
||||
"\n",
|
||||
"| Module | Purpose |\n",
|
||||
"| :--- | :--- |\n",
|
||||
"| **`semantica.core`** | Central orchestration and configuration management. |\n",
|
||||
"| **`semantica.seed`** | Bootstrapping the graph with verified \"Ground Truth\" data. |\n",
|
||||
"| **`semantica.ingest`** | Fetching data from Web, RSS, and Git repositories. |\n",
|
||||
"| **`semantica.parse`** | Deep extraction from PDFs, Markdown, and HTML. |\n",
|
||||
"| **`semantica.normalize`** | standardizing text, symbols, and entities. |\n",
|
||||
"| **`semantica.split`** | Graph-aware chunking (entity & relation aware) to preserve graph integrity. |\n",
|
||||
"| **`semantica.kg`** | LLM-driven Graph Construction and Analytics. |\n",
|
||||
"| **`semantica.deduplication`** | Merging duplicate entities across sources. |\n",
|
||||
"| **`semantica.conflicts`** | Resolving discrepancies between sources (e.g., conflicting dates). |\n",
|
||||
"| **`semantica.vector_store`** | High-dimensional semantic indexing. |\n",
|
||||
"| **`semantica.reasoning`** | Multi-hop graph inference and logic. |\n",
|
||||
"| **`semantica.pipeline`** | Wrapping the entire workflow into a repeatable object. |\n",
|
||||
"| **`semantica.visualization`** | Rich network graphs and community insights. |\n",
|
||||
"| **`semantica.export`** | Persistence to JSON, CSV, and Neo4j. |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
|
||||
"ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
|
||||
"descript-audiotools 0.7.2 requires protobuf<3.20,>=3.9.2, but you have protobuf 4.25.8 which is incompatible.\n",
|
||||
"langchain-openai 0.2.10 requires langchain-core<0.4.0,>=0.3.21, but you have langchain-core 0.1.23 which is incompatible.\n",
|
||||
"mistral-common 1.5.1 requires tiktoken<0.8.0,>=0.7.0, but you have tiktoken 0.12.0 which is incompatible.\n",
|
||||
"nari-tts 0.1.0 requires numpy>=2.2.4, but you have numpy 1.26.4 which is incompatible.\n",
|
||||
"nari-tts 0.1.0 requires torch>=2.6.0, but you have torch 2.2.1 which is incompatible.\n",
|
||||
"parlant 3.0.2 requires fastapi==0.115.12, but you have fastapi 0.120.4 which is incompatible.\n",
|
||||
"parlant 3.0.2 requires fastmcp==2.6.1, but you have fastmcp 2.14.1 which is incompatible.\n",
|
||||
"parlant 3.0.2 requires opentelemetry-exporter-otlp-proto-grpc==1.27.0, but you have opentelemetry-exporter-otlp-proto-grpc 1.38.0 which is incompatible.\n",
|
||||
"parlant 3.0.2 requires rich<14.0.0,>=13.7.1, but you have rich 14.2.0 which is incompatible.\n",
|
||||
"parlant 3.0.2 requires tiktoken<0.9,>=0.8, but you have tiktoken 0.12.0 which is incompatible.\n",
|
||||
"parlant 3.0.2 requires tokenizers<0.21,>=0.20, but you have tokenizers 0.21.4 which is incompatible.\n",
|
||||
"parlant 3.0.2 requires uvicorn<0.33.0,>=0.32.1, but you have uvicorn 0.38.0 which is incompatible.\n",
|
||||
"vllm 0.6.6.post1 requires torch==2.5.1, but you have torch 2.2.1 which is incompatible.\n",
|
||||
"vllm 0.6.6.post1 requires torchvision==0.20.1, but you have torchvision 0.17.1 which is incompatible.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Environment Setup\n",
|
||||
"!pip install -qU semantica networkx matplotlib plotly pandas faiss-cpu tiktoken beautifulsoup4 python-docx pdfplumber"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Professional Initialization & Config\n",
|
||||
"\n",
|
||||
"We start by defining a production config. Semantica uses ConfigManager to ensure environment consistency."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Config Loaded.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"from semantica.core import Semantica, ConfigManager\n",
|
||||
"\n",
|
||||
"# Enterprise Config Definition\n",
|
||||
"config_dict = {\n",
|
||||
" \"project_name\": \"PythonAI_Mastery\",\n",
|
||||
" \"embedding\": {\n",
|
||||
" \"provider\": \"openai\",\n",
|
||||
" \"model\": \"text-embedding-3-small\"\n",
|
||||
" },\n",
|
||||
" \"extraction\": {\n",
|
||||
" \"model\": \"gpt-4o-mini\",\n",
|
||||
" \"temperature\": 0.0\n",
|
||||
" },\n",
|
||||
" \"vector_store\": {\n",
|
||||
" \"provider\": \"faiss\",\n",
|
||||
" \"dimension\": 1536 \n",
|
||||
" },\n",
|
||||
" \"knowledge_graph\": {\n",
|
||||
" \"backend\": \"networkx\",\n",
|
||||
" \"merge_entities\": True,\n",
|
||||
" \"resolution_strategy\": \"fuzzy\"\n",
|
||||
" }\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"config = ConfigManager().load_from_dict(config_dict)\n",
|
||||
"core = Semantica(config=config)\n",
|
||||
"print(\"Config Loaded.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Bootstrapping with Seed Data\n",
|
||||
"\n",
|
||||
"We use `semantica.seed` to establish \"Ground Truth.\" This prevents the system from being solely dependent on AI extractions."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is seeding</td><td>🌱 seed</td><td>SeedDataManager</td><td>-</td><td>0.05s</td></tr></table></div>"
|
||||
],
|
||||
"text/plain": [
|
||||
"<IPython.core.display.HTML object>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Foundation Graph Seeded with 2 Verified Nodes.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import json\n",
|
||||
"from semantica.seed import SeedDataManager\n",
|
||||
"\n",
|
||||
"# Create sample ground truth entities\n",
|
||||
"foundation_data = {\n",
|
||||
" \"entities\": [\n",
|
||||
" {\"id\": \"python_org\", \"name\": \"Python Software Foundation\", \"type\": \"Organization\"},\n",
|
||||
" {\"id\": \"guido_van_rossum\", \"name\": \"Guido van Rossum\", \"type\": \"Person\"}\n",
|
||||
" ],\n",
|
||||
" \"relationships\": [\n",
|
||||
" {\"source\": \"guido_van_rossum\", \"target\": \"python_org\", \"type\": \"FOUNDED\"}\n",
|
||||
" ]\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"with open(\"ground_truth.json\", \"w\") as f:\n",
|
||||
" json.dump(foundation_data, f)\n",
|
||||
"\n",
|
||||
"seed_manager = SeedDataManager()\n",
|
||||
"seed_manager.register_source(\"core_info\", \"json\", \"ground_truth.json\")\n",
|
||||
"foundation_graph = seed_manager.create_foundation_graph()\n",
|
||||
"\n",
|
||||
"print(f\"Foundation Graph Seeded with {len(foundation_data['entities'])} Verified Nodes.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 3. The Knowledge Hub: Massive Multi-Source Ingestion\n",
|
||||
"\n",
|
||||
"We aggregate data from a diverse set of real-world sources using `semantica.ingest` and `semantica.parse`. \n",
|
||||
"\n",
|
||||
"### Data Sources\n",
|
||||
"* **Official Docs**: Python.org, SQLAlchemy, Pydantic.\n",
|
||||
"* **Live News (RSS)**: TechCrunch, Wired, Ars Technica.\n",
|
||||
"* **Technical Blogs**: Real Python, Toward Data Science.\n",
|
||||
"* **Engineering Repos**: Requests, HTTPX, Semantica."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Ingesting Official Documentation...\n",
|
||||
"\n",
|
||||
"Fetching Live Tech News...\n",
|
||||
"\n",
|
||||
"Ingesting Engineering READMEs...\n",
|
||||
"\n",
|
||||
"Aggregated 14 documents from across the web.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from semantica.ingest import ingest_web, ingest_feed\n",
|
||||
"from semantica.parse import parse_document\n",
|
||||
"\n",
|
||||
"all_content = []\n",
|
||||
"\n",
|
||||
"# 1. Web Domain Ingestion\n",
|
||||
"print(\"Ingesting Official Documentation...\")\n",
|
||||
"web_urls = [\n",
|
||||
" \"https://www.python.org/about/\",\n",
|
||||
" \"https://www.python.org/downloads/\",\n",
|
||||
" \"https://realpython.com/\" # Fixed 404: updated from /python-news/\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"for url in web_urls:\n",
|
||||
" try:\n",
|
||||
" # Returns a WebContent object\n",
|
||||
" doc = ingest_web(url, method=\"url\")\n",
|
||||
" all_content.append(doc.text)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"Failed to ingest {url}: {e}\")\n",
|
||||
"\n",
|
||||
"# 2. Live RSS Feeds\n",
|
||||
"print(\"\\nFetching Live Tech News...\")\n",
|
||||
"rss_feeds = [\n",
|
||||
" \"http://feeds.bbci.co.uk/news/technology/rss.xml\",\n",
|
||||
" \"https://techcrunch.com/feed/\",\n",
|
||||
" \"https://www.wired.com/feed/rss\"\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"for feed in rss_feeds:\n",
|
||||
" try:\n",
|
||||
" # Returns a FeedData object\n",
|
||||
" feed_data = ingest_feed(feed, method=\"rss\")\n",
|
||||
" # Extract top 3 items from each feed\n",
|
||||
" for item in feed_data.items[:3]:\n",
|
||||
" content = item.content if item.content else item.description\n",
|
||||
" all_content.append(content)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"Failed to ingest feed {feed}: {e}\")\n",
|
||||
"\n",
|
||||
"# 3. Repository & Technical Files\n",
|
||||
"print(\"\\nIngesting Engineering READMEs...\")\n",
|
||||
"repo_files = [\n",
|
||||
" \"https://raw.githubusercontent.com/psf/requests/main/README.md\",\n",
|
||||
" \"https://raw.githubusercontent.com/encode/httpx/master/README.md\"\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"for file_url in repo_files:\n",
|
||||
" try:\n",
|
||||
" # Using ingest_web directly to ensure we get a WebContent object \n",
|
||||
" # (avoiding the dictionary wrapper returned by the unified 'ingest' function)\n",
|
||||
" doc = ingest_web(file_url, method=\"url\") \n",
|
||||
" all_content.append(doc.text)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"Failed to ingest {file_url}: {e}\")\n",
|
||||
"\n",
|
||||
"print(f\"\\nAggregated {len(all_content)} documents from across the web.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 4. Normalization & Graph-Aware Chunking\n",
|
||||
"\n",
|
||||
"Standardizing noise and chunking for context preservation via `semantica.normalize` and `semantica.split`.\n",
|
||||
"\n",
|
||||
"### Why Graph-Aware Chunking?\n",
|
||||
"Traditional recursive chunking often breaks entities and relationships across chunk boundaries. Semantica's **`EntityAwareChunker`** ensures that key entities and their semantic context are preserved within a single chunk, which is essential for building a coherent Knowledge Graph."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.normalize import TextNormalizer\n",
|
||||
"from semantica.split import TextSplitter, EntityAwareChunker\n",
|
||||
"\n",
|
||||
"# 1. Normalization - Sanitizing input data\n",
|
||||
"normalizer = TextNormalizer()\n",
|
||||
"clean_data = [normalizer.normalize(text) for text in all_content if text]\n",
|
||||
"\n",
|
||||
"# 2. Standard Recursive Splitting (Baseline)\n",
|
||||
"standard_splitter = TextSplitter(method=\"recursive\", chunk_size=1200, chunk_overlap=250)\n",
|
||||
"standard_chunks = []\n",
|
||||
"for doc in clean_data[:2]: # Sample for comparison\n",
|
||||
" standard_chunks.extend(standard_splitter.split(doc))\n",
|
||||
"\n",
|
||||
"# 3. Advanced Graph-Aware Chunking (Entity Preservation)\n",
|
||||
"print(\"Performing Graph-Aware Chunking (preserving entity boundaries)...\")\n",
|
||||
"graph_aware_chunker = EntityAwareChunker(\n",
|
||||
" chunk_size=1000, \n",
|
||||
" chunk_overlap=200, \n",
|
||||
" ner_method=\"ml\" # Can use \"llm\" for higher precision\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"all_chunks = []\n",
|
||||
"for doc in clean_data:\n",
|
||||
" # EntityAwareChunker ensures entities are not split across chunks\n",
|
||||
" chunks = graph_aware_chunker.chunk(doc)\n",
|
||||
" all_chunks.extend(chunks)\n",
|
||||
"\n",
|
||||
"print(f\"Generated {len(all_chunks)} Graph-Aware chunks (vs {len(standard_chunks)} baseline chunks for sample).\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 5. Knowledge Graph Construction & Data Quality\n",
|
||||
"\n",
|
||||
"Building the graph, then applying Conflict Resolution and Deduplication to ensure data integrity."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "AttributeError",
|
||||
"evalue": "'dict' object has no attribute 'nodes'",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
|
||||
"\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)",
|
||||
"Cell \u001b[1;32mIn[5], line 11\u001b[0m\n\u001b[0;32m 9\u001b[0m \u001b[38;5;66;03m# 2. Quality Control: Deduplication\u001b[39;00m\n\u001b[0;32m 10\u001b[0m detector \u001b[38;5;241m=\u001b[39m DuplicateDetector(similarity_threshold\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0.85\u001b[39m)\n\u001b[1;32m---> 11\u001b[0m duplicates \u001b[38;5;241m=\u001b[39m detector\u001b[38;5;241m.\u001b[39mdetect_duplicates(\u001b[38;5;28mlist\u001b[39m(\u001b[43mkg\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mnodes\u001b[49m(data\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m)))\n\u001b[0;32m 12\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m duplicates:\n\u001b[0;32m 13\u001b[0m merger \u001b[38;5;241m=\u001b[39m EntityMerger()\n",
|
||||
"\u001b[1;31mAttributeError\u001b[0m: 'dict' object has no attribute 'nodes'"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from semantica.kg import GraphBuilder\n",
|
||||
"from semantica.deduplication import DuplicateDetector, EntityMerger\n",
|
||||
"from semantica.conflicts import ConflictDetector, ConflictResolver\n",
|
||||
"\n",
|
||||
"# 1. Initial Construction\n",
|
||||
"gb = GraphBuilder(merge_entities=True)\n",
|
||||
"kg = gb.build(sources=[{\"text\": str(c.text)} for c in all_chunks[:12]])\n",
|
||||
"\n",
|
||||
"# 2. Quality Control: Deduplication\n",
|
||||
"detector = DuplicateDetector(similarity_threshold=0.85)\n",
|
||||
"# Accessing entities from the KG dictionary structure\n",
|
||||
"entities = kg.get(\"entities\", [])\n",
|
||||
"duplicates = detector.detect_duplicates(entities)\n",
|
||||
"\n",
|
||||
"if duplicates:\n",
|
||||
" merger = EntityMerger()\n",
|
||||
" # Merging returns an updated graph dictionary\n",
|
||||
" kg = merger.merge_duplicates(kg, duplicates)\n",
|
||||
" print(f\"Deduplicated {len(duplicates)} Entity Pairs.\")\n",
|
||||
"\n",
|
||||
"# 3. Quality Control: Conflict Resolution\n",
|
||||
"conflict_detector = ConflictDetector()\n",
|
||||
"conflicts = conflict_detector.detect_conflicts(kg)\n",
|
||||
"if conflicts:\n",
|
||||
" resolver = ConflictResolver()\n",
|
||||
" kg = resolver.resolve_conflicts(kg, conflicts, strategy=\"most_recent\")\n",
|
||||
" print(f\"Resolved {len(conflicts)} Data Conflicts.\")\n",
|
||||
"\n",
|
||||
"print(f\"High-Quality Knowledge Graph Ready. Entities: {len(kg['entities'])}, Relations: {len(kg['relationships'])}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 6. Graph Synthesis & Advanced Reasoning\n",
|
||||
"\n",
|
||||
"We apply Graph Analytics and the Reasoning module to derive insights not explicitly stated in the text."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.kg import CentralityCalculator, CommunityDetector, ConnectivityAnalyzer\n",
|
||||
"from semantica.reasoning import InferenceEngine, InferenceStrategy\n",
|
||||
"\n",
|
||||
"# 1. Analytics - Mapping the Influence\n",
|
||||
"centrality_result = CentralityCalculator().calculate_degree_centrality(kg)\n",
|
||||
"top_nodes = centrality_result.get(\"rankings\", [])[:5]\n",
|
||||
"\n",
|
||||
"communities = CommunityDetector().detect_communities(kg, algorithm=\"louvain\")\n",
|
||||
"\n",
|
||||
"# 2. Graph Connectivity Analysis - Understanding the Network\n",
|
||||
"analyzer = ConnectivityAnalyzer()\n",
|
||||
"connectivity = analyzer.analyze_graph_structure(kg)\n",
|
||||
"\n",
|
||||
"# 3. Logical Inference - Deriving Hidden Relationships\n",
|
||||
"engine = InferenceEngine(strategy=\"forward\")\n",
|
||||
"# Example: Adding a domain rule (If X is a 'Library' and Y is a 'Language', then X 'BuiltWith' Y)\n",
|
||||
"engine.add_rule(\"IF ?x :type 'Library' AND ?y :type 'Language' THEN ?x :builtWith ?y\")\n",
|
||||
"# In practice, facts would be extracted from the KG entities and relationships\n",
|
||||
"# inference_results = engine.infer(facts, rules)\n",
|
||||
"\n",
|
||||
"print(f\"Top Influential Entities: {[n['node'] for n in top_nodes]}\")\n",
|
||||
"print(f\"Network Connectivity Profile: {connectivity.get('structure_type', 'interconnected')}\")\n",
|
||||
"print(\"Inference Engine initialized with Domain Rules.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 7. Hybrid Context Retrieval\n",
|
||||
"\n",
|
||||
"Storage using `vector_store` and wrapping it in `AgentContext`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.vector_store import VectorStore\n",
|
||||
"from semantica.context import AgentContext\n",
|
||||
"\n",
|
||||
"vs = VectorStore(backend=\"faiss\", dimension=1536)\n",
|
||||
"embeddings = core.embedding_generator.generate_embeddings([str(c.text) for c in all_chunks[:12]])\n",
|
||||
"vs.store_vectors(vectors=embeddings, metadata=[{\"text\": str(c.text)} for c in all_chunks[:12]])\n",
|
||||
"\n",
|
||||
"# Global Context Manager for an Agent\n",
|
||||
"context = AgentContext(vector_store=vs, knowledge_graph=kg)\n",
|
||||
"\n",
|
||||
"print(\"Hybrid Context Store Initialized.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 8. Immersive Visualization\n",
|
||||
"\n",
|
||||
"We use `semantica.visualization` to create a community-aware network map."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.visualization import KGVisualizer\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"viz = KGVisualizer()\n",
|
||||
"viz.visualize_network(\n",
|
||||
" kg, \n",
|
||||
" layout=\"spring\", \n",
|
||||
" output=\"static\",\n",
|
||||
" title=\"Python Ecosystem Intelligence Graph (Multi-Source)\"\n",
|
||||
")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 9. Modular Orchestration: The Pipeline\n",
|
||||
"\n",
|
||||
"Finally, we show how to wrap this whole complex flow into a single `semantica.pipeline.Pipeline` object for automation."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.pipeline import PipelineBuilder\n",
|
||||
"\n",
|
||||
"builder = PipelineBuilder()\n",
|
||||
"knowledge_pipeline = (\n",
|
||||
" builder.add_step(\"ingest\", \"knowledge_hub_loader\")\n",
|
||||
" .add_step(\"normalize\", \"text_normalizer\")\n",
|
||||
" .add_step(\"split\", \"semantic_splitter\")\n",
|
||||
" .add_step(\"enrich\", \"kg_builder\")\n",
|
||||
" .add_step(\"validate\", \"quality_assurance\")\n",
|
||||
" .build()\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(\"Unified Knowledge Pipeline Construct Complete.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 10. Persistence & Export\n",
|
||||
"\n",
|
||||
"Save the finalized knowledge structures."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.export import GraphExporter\n",
|
||||
"\n",
|
||||
"exporter = GraphExporter()\n",
|
||||
"exporter.export_to_json(kg, \"master_ecosystem_graph.json\")\n",
|
||||
"\n",
|
||||
"print(\"Project Exported. Deployment Ready.\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# RAG vs. GraphRAG: \n",
|
||||
"\n",
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook provides a rigorous, side-by-side comparison of Standard RAG (Vector-based) and GraphRAG (Graph-based) focusing on the Global Intelligence & security domain.\n",
|
||||
"\n",
|
||||
"### The Challenge: Navigating Fragmentation\n",
|
||||
"In intelligence work, facts are scattered. One report might mention a person, another a location, and a third a specific project. Vector search often fails to bridge these \"semantic gaps\" if the keywords aren't directly co-located.\n",
|
||||
"\n",
|
||||
"We will demonstrate how GraphRAG creates a \"Chain of Evidence\" that Vector RAG simply cannot see.\n",
|
||||
"\n",
|
||||
"### Key Semantica Modules Utilized\n",
|
||||
"\n",
|
||||
"| Pipeline Stage | Modules Selection |\n",
|
||||
"| :--- | :--- |\n",
|
||||
"| **Intelligence Gathering** | `semantica.ingest`, `semantica.normalize` |\n",
|
||||
"| **Vector Pipeline** | `semantica.split`, `semantica.vector_store` |\n",
|
||||
"| **Graph Pipeline** | `semantica.kg`, `semantica.deduplication`, `semantica.conflicts` |\n",
|
||||
"| **Inference/Reasoning**| `semantica.reasoning`, `semantica.pipeline` |\n",
|
||||
"| **Interface** | `semantica.context`, `semantica.visualization` |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Setup\n",
|
||||
"!pip install -qU semantica networkx matplotlib plotly pandas faiss-cpu"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Domain Acquisition: Real-World Intelligence Feeds\n",
|
||||
"\n",
|
||||
"We ingest from real-world feeds to build our knowledge base. We'll look for connections across global security news and official reports."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ingest import WebIngestor, FeedIngestor\n",
|
||||
"from semantica.normalize import TextNormalizer\n",
|
||||
"\n",
|
||||
"normalizer = TextNormalizer()\n",
|
||||
"all_content = []\n",
|
||||
"\n",
|
||||
"print(\"Gathering Intelligence Data...\")\n",
|
||||
"\n",
|
||||
"# 1. Global News Feeds\n",
|
||||
"feeds = [\n",
|
||||
" \"http://feeds.bbci.co.uk/news/world/rss.xml\",\n",
|
||||
" \"https://www.reutersagency.com/feed/\" \n",
|
||||
"]\n",
|
||||
"feed_ingestor = FeedIngestor()\n",
|
||||
"for f in feeds:\n",
|
||||
" docs = feed_ingestor.ingest(f)[:5]\n",
|
||||
" all_content.extend([d.content if hasattr(d, 'content') else str(d) for d in docs])\n",
|
||||
"\n",
|
||||
"# 2. Official Intelligence/Security Overviews\n",
|
||||
"web_urls = [\n",
|
||||
" \"https://www.cia.gov/the-world-factbook/\",\n",
|
||||
" \"https://www.un.org/en/observances/security-council-day\"\n",
|
||||
"]\n",
|
||||
"web_ingestor = WebIngestor()\n",
|
||||
"for url in web_urls:\n",
|
||||
" docs = web_ingestor.ingest(url, method=\"url\")\n",
|
||||
" all_content.extend([d.content if hasattr(d, 'content') else str(d) for d in docs])\n",
|
||||
"\n",
|
||||
"clean_docs = [normalizer.normalize(text) for text in all_content if len(text) > 100]\n",
|
||||
"\n",
|
||||
"print(f\"\\nIntelligence Knowledge Hub Populated with {len(clean_docs)} reports.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Standard Vector RAG Pipeline\n",
|
||||
"\n",
|
||||
"The baseline approach: Linear retrieval via semantic overlap."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.core import Semantica, ConfigManager\n",
|
||||
"from semantica.split import TextSplitter\n",
|
||||
"from semantica.vector_store import VectorStore\n",
|
||||
"\n",
|
||||
"v_core = Semantica(config=ConfigManager().load_from_dict({\n",
|
||||
" \"embedding\": {\"provider\": \"openai\", \"model\": \"text-embedding-3-small\"},\n",
|
||||
" \"vector_store\": {\"provider\": \"faiss\", \"dimension\": 1536}\n",
|
||||
"}))\n",
|
||||
"\n",
|
||||
"splitter = TextSplitter(method=\"recursive\", chunk_size=800, chunk_overlap=100)\n",
|
||||
"chunks = []\n",
|
||||
"for doc in clean_docs:\n",
|
||||
" chunks.extend(splitter.split(doc))\n",
|
||||
"\n",
|
||||
"vs = VectorStore(backend=\"faiss\", dimension=1536)\n",
|
||||
"embeddings = v_core.embedding_generator.generate_embeddings([str(c) for c in chunks[:20]])\n",
|
||||
"vs.store_vectors(vectors=embeddings, metadata=[{\"text\": str(c)} for c in chunks[:20]])\n",
|
||||
"\n",
|
||||
"print(f\"Vector RAG ready with {len(chunks[:20])} encoded fragments.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 3. High-Fidelity GraphRAG Pipeline\n",
|
||||
"\n",
|
||||
"Utilizing Entity resolution and relationship synthesis to bridge reports."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.kg import GraphBuilder\n",
|
||||
"from semantica.deduplication import DuplicateDetector\n",
|
||||
"from semantica.conflicts import ConflictDetector\n",
|
||||
"\n",
|
||||
"gb = GraphBuilder(merge_entities=True)\n",
|
||||
"kg = gb.build(sources=[{\"text\": str(c.text)} for c in chunks[:12]])\n",
|
||||
"\n",
|
||||
"detector = DuplicateDetector(similarity_threshold=0.85)\n",
|
||||
"\n",
|
||||
"print(f\"GraphRAG Synthesis Complete: {len(kg.get('entities', []))} Entity Nodes mapped.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 4. The Intelligence Test: Multi-Source Linkage\n",
|
||||
"\n",
|
||||
"Intelligence query: \"What are the current global security challenges mentioned across different regions?\"\n",
|
||||
"\n",
|
||||
"Vector RAG will likely return fragments about specific countries but fail to group them. GraphRAG will traverse nodes of type Region to find shared CHALLENGE edges."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.kg import ConnectivityAnalyzer\n",
|
||||
"from semantica.reasoning import InferenceEngine\n",
|
||||
"\n",
|
||||
"query = \"Identify interconnected security risks across the UN and major regions.\"\n",
|
||||
"print(f\"Investigative Query: {query}\\n\")\n",
|
||||
"\n",
|
||||
"print(\"--- Standard Vector Recall ---\")\n",
|
||||
"q_vec = v_core.embedding_generator.generate_embeddings(query)\n",
|
||||
"v_res = vs.search_vectors(q_vec, k=3)\n",
|
||||
"for r in v_res:\n",
|
||||
" print(f\"Recall: {r['metadata']['text'][:150]}...\")\n",
|
||||
"\n",
|
||||
"print(\"\\n--- Graph Intelligence Reasoning ---\")\n",
|
||||
"analyzer = ConnectivityAnalyzer()\n",
|
||||
"connectivity = analyzer.analyze_graph_structure(kg)\n",
|
||||
"\n",
|
||||
"# Advanced Feature: Inference-Enhanced GraphRAG\n",
|
||||
"# Vector RAG only sees surface similarity. \n",
|
||||
"# InferenceEngine can follow logical rules (e.g., IF Conflict in Region A AND Region A is part of Trade Route B -> THEN Trade Route B is AT RISK)\n",
|
||||
"engine = InferenceEngine(strategy=\"forward\")\n",
|
||||
"engine.add_rule(\"IF ?x :type 'Conflict' AND ?y :type 'Region' AND ?x :locatedIn ?y THEN ?y :status 'HighRisk'\")\n",
|
||||
"\n",
|
||||
"print(f\"Knowledge Structure: {connectivity.get('structure_type', 'complex')}\")\n",
|
||||
"print(f\"Identified {len(connectivity.get('bridges', []))} critical intelligence bridges.\")\n",
|
||||
"print(\"Logical Inference Engine active: Monitoring for high-risk regional propagation.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 5. Visualizing the Semantic Network\n",
|
||||
"\n",
|
||||
"We visualize how the Semantica engine has mapped the relationships between global actors and current events."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.visualization import KGVisualizer\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"KGVisualizer().visualize_network(\n",
|
||||
" kg, \n",
|
||||
" layout=\"spring\", \n",
|
||||
" output=\"static\",\n",
|
||||
" title=\"Intelligence Connectivity Map\"\n",
|
||||
")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -36,6 +36,30 @@ Get started with Semantica through these beginner-friendly tutorials.
|
||||
- **[Visualization](cookbook/introduction/Visualization.ipynb)** - Visualize knowledge graphs
|
||||
- **[Export](cookbook/introduction/Export.ipynb)** - Export in various formats
|
||||
|
||||
- :material-robot: **GraphRAG Complete**
|
||||
---
|
||||
Build a production-ready Graph Retrieval Augmented Generation system.
|
||||
|
||||
**New Features**: Graph Validation, Logical Inference, Hybrid Context.
|
||||
|
||||
**Topics**: RAG, LLMs, Vector Search, Graph Traversal
|
||||
|
||||
**Difficulty**: Advanced
|
||||
|
||||
[Open Notebook](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/use_cases/advanced_rag/01_GraphRAG_Complete.ipynb)
|
||||
|
||||
- :material-scale-balance: **RAG vs. GraphRAG Comparison**
|
||||
---
|
||||
Side-by-side comparison of Standard RAG vs. GraphRAG using real-world data.
|
||||
|
||||
**New Features**: Inference-Enhanced GraphRAG, Reasoning Gap Analysis.
|
||||
|
||||
**Topics**: RAG, GraphRAG, Benchmarking, Visualization
|
||||
|
||||
**Difficulty**: Intermediate
|
||||
|
||||
[Open Notebook](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/use_cases/advanced_rag/02_RAG_vs_GraphRAG_Comparison.ipynb)
|
||||
|
||||
## Advanced
|
||||
|
||||
Advanced techniques and patterns for experienced users.
|
||||
|
||||
@@ -182,6 +182,115 @@ print(f"Graph density: {analysis['density']}")
|
||||
print(f"Connected components: {analysis['components']}")
|
||||
```
|
||||
|
||||
### Example 9: Persistent Storage (Neo4j)
|
||||
|
||||
**Difficulty**: Intermediate
|
||||
|
||||
Store and query knowledge graphs in a persistent graph database.
|
||||
|
||||
```python
|
||||
from semantica.graph_store import GraphStore
|
||||
|
||||
# Initialize with Neo4j
|
||||
store = GraphStore(
|
||||
backend="neo4j",
|
||||
uri="bolt://localhost:7687",
|
||||
user="neo4j",
|
||||
password="password"
|
||||
)
|
||||
store.connect()
|
||||
|
||||
# Create nodes and relationships
|
||||
apple = store.create_node(
|
||||
labels=["Company"],
|
||||
properties={"name": "Apple Inc."}
|
||||
)
|
||||
tim = store.create_node(
|
||||
labels=["Person"],
|
||||
properties={"name": "Tim Cook"}
|
||||
)
|
||||
store.create_relationship(
|
||||
start_node_id=tim["id"],
|
||||
end_node_id=apple["id"],
|
||||
rel_type="CEO_OF"
|
||||
)
|
||||
|
||||
store.close()
|
||||
```
|
||||
|
||||
### Example 10: FalkorDB for Real-Time Applications
|
||||
|
||||
**Difficulty**: Intermediate
|
||||
|
||||
Ultra-fast graph queries for LLM applications using FalkorDB.
|
||||
|
||||
```python
|
||||
from semantica.graph_store import GraphStore
|
||||
|
||||
store = GraphStore(
|
||||
backend="falkordb",
|
||||
host="localhost",
|
||||
port=6379,
|
||||
graph_name="knowledge_graph"
|
||||
)
|
||||
store.connect()
|
||||
|
||||
# Fast queries
|
||||
results = store.execute_query("MATCH (n)-[r]->(m) WHERE n.name CONTAINS 'AI' RETURN n")
|
||||
store.close()
|
||||
```
|
||||
|
||||
### Example 11: GraphRAG (Knowledge-Powered Retrieval)
|
||||
|
||||
**Difficulty**: Advanced
|
||||
|
||||
Build a production-ready GraphRAG system with logical inference and hybrid retrieval.
|
||||
|
||||
```python
|
||||
from semantica.context import AgentContext
|
||||
from semantica.reasoning import InferenceEngine
|
||||
|
||||
# Initialize context with Hybrid Retrieval
|
||||
context = AgentContext(
|
||||
vector_store=vs,
|
||||
knowledge_graph=kg,
|
||||
use_graph_expansion=True,
|
||||
hybrid_alpha=0.7
|
||||
)
|
||||
|
||||
# Add logical reasoning rules
|
||||
engine = InferenceEngine(strategy="forward")
|
||||
engine.add_rule("IF ?x :type 'Library' AND ?y :type 'Language' THEN ?x :builtWith ?y")
|
||||
|
||||
# Retrieve context for a query
|
||||
results = context.retrieve("What technologies are used in this project?")
|
||||
```
|
||||
|
||||
[**View Complete GraphRAG Tutorial**](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/use_cases/advanced_rag/01_GraphRAG_Complete.ipynb)
|
||||
|
||||
### Example 12: RAG vs. GraphRAG Comparison
|
||||
|
||||
**Difficulty**: Intermediate
|
||||
|
||||
Benchmark standard Vector RAG against Graph-enhanced retrieval.
|
||||
|
||||
[**View RAG vs. GraphRAG Comparison**](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/use_cases/advanced_rag/02_RAG_vs_GraphRAG_Comparison.ipynb)
|
||||
|
||||
---
|
||||
|
||||
## Production Patterns
|
||||
|
||||
### Example 13: Streaming Data Processing
|
||||
|
||||
**Difficulty**: Advanced
|
||||
|
||||
Process data streams in real-time.
|
||||
|
||||
```python
|
||||
from semantica.ingest import StreamIngestor
|
||||
from semantica.core import Semantica
|
||||
```
|
||||
|
||||
## Use Case Examples
|
||||
|
||||
### Research Paper Analysis
|
||||
|
||||
Reference in New Issue
Block a user