Add RAG vs GraphRAG comparison notebook and update docs

cookbook/advanced/Advanced_Vector_Store_and_Search.ipynb

@@ -69,7 +69,7 @@
     "\n",
     "### Simple Rule\n",
     "- Less than 10,000 items? Use **Flat**\n",
-    "- Between 10,000 and 1 million? Use **HNSW** \u2705 (recommended)\n",
+    "- Between 10,000 and 1 million? Use **HNSW** ✅ (recommended)\n",
     "- More than 1 million? Use **IVF**"
    ]
   },
@@ -286,7 +286,7 @@
     "# Small dataset (< 10,000 items)\n",
     "index = adapter.create_index(index_type=\"flat\", metric=\"L2\")\n",
     "\n",
-    "# Medium dataset (10,000 - 1,000,000 items) \u2705 RECOMMENDED\n",
+    "# Medium dataset (10,000 - 1,000,000 items) ✅ RECOMMENDED\n",
     "index = adapter.create_index(index_type=\"hnsw\", metric=\"L2\", m=16)\n",
     "\n",
     "# Large dataset (> 1,000,000 items)\n",
@@ -340,10 +340,10 @@
     "\n",
     "You've learned:\n",
     "\n",
-    "1. \u2705 **Index Selection**: Use HNSW for most cases\n",
-    "2. \u2705 **Smart Filtering**: Combine vector search with metadata\n",
-    "3. \u2705 **Result Fusion**: Merge searches from different sources\n",
-    "4. \u2705 **Data Isolation**: Keep users' data separate\n",
+    "1. ✅ **Index Selection**: Use HNSW for most cases\n",
+    "2. ✅ **Smart Filtering**: Combine vector search with metadata\n",
+    "3. ✅ **Result Fusion**: Merge searches from different sources\n",
+    "4. ✅ **Data Isolation**: Keep users' data separate\n",
     "\n",
     "### Next Steps\n",
     "\n",
@@ -377,4 +377,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 2
-}
+}

cookbook/use_cases/advanced_rag/02_RAG_vs_GraphRAG_Comparison.ipynb

@@ -0,0 +1,277 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# RAG vs. GraphRAG: A Side-by-Side Comparison\n",
+    "\n",
+    "This notebook demonstrates the difference between **Standard RAG** (Vector-based) and **GraphRAG** (Graph-based) using the `semantica` framework. We will use a real-world text source to build both systems and compare their retrieval capabilities.\n",
+    "\n",
+    "## What is the difference?\n",
+    "- **Standard RAG**: Retrieves documents based on semantic similarity (vector distance). Good for direct matching but can miss connected concepts.\n",
+    "- **GraphRAG**: Retrieves information by traversing a Knowledge Graph (nodes and edges). Excellent for multi-hop reasoning and understanding relationships between entities.\n",
+    "\n",
+    "## Workflow\n",
+    "1. **Ingest Data**: Load a technical article.\n",
+    "2. **Build Standard RAG**: Chunk text -> Embeddings -> Vector Store -> Similarity Search.\n",
+    "3. **Build GraphRAG**: Extract Entities/Relations -> Knowledge Graph -> Graph Traversal.\n",
+    "4. **Compare Results**: Ask the same complex question to both systems."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install semantica if not already installed\n",
+    "!pip install semantica networkx matplotlib plotly sentence-transformers"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import networkx as nx\n",
+    "import matplotlib.pyplot as plt\n",
+    "import numpy as np\n",
+    "\n",
+    "# Import Semantica Core Modules\n",
+    "from semantica.core import Semantica\n",
+    "from semantica.ingest import WebIngestor\n",
+    "from semantica.split import TextSplitter  # Proper Semantica splitter\n",
+    "from semantica.vector_store import FAISSStore\n",
+    "from semantica.embeddings import EmbeddingGenerator  # Proper embedding generator\n",
+    "from semantica.semantic_extract import NamedEntityRecognizer, RelationExtractor\n",
+    "from semantica.kg import GraphBuilder\n",
+    "from semantica.visualization import KGVisualizer  # Proper visualization module\n",
+    "\n",
+    "# Initialize Semantica\n",
+    "semantica = Semantica()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Data Ingestion\n",
+    "We'll use a text about **\"The Impact of AI on Healthcare\"**. This topic has rich relationships (AI -> improves -> Diagnosis, Privacy -> challenges -> AI)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Sample \"Real World\" Data Source\n",
+    "text_content = \"\"\"\n",
+    "Artificial Intelligence (AI) is revolutionizing healthcare by enabling early diagnosis and personalized treatment plans.\n",
+    "Machine Learning algorithms analyze medical imaging to detect anomalies like tumors faster than human radiologists.\n",
+    "For instance, DeepMind's AlphaFold has solved the protein folding problem, accelerating drug discovery.\n",
+    "However, the integration of AI in healthcare faces significant challenges, primarily concerning patient data privacy and algorithmic bias.\n",
+    "Regulatory bodies like the FDA are establishing guidelines to ensure AI tools are safe and effective.\n",
+    "Unlike traditional software, AI systems can adapt and learn, which complicates validation processes.\n",
+    "Hospitals using predictive analytics have seen a 30% reduction in patient readmission rates.\n",
+    "Yet, cybersecurity threats remain a critical risk for connected medical devices.\n",
+    "\"\"\"\n",
+    "\n",
+    "print(f\"Loaded text with {len(text_content)} characters.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Standard RAG (Vector Search)\n",
+    "We will chunk the text using `TextSplitter`, create embeddings with `EmbeddingGenerator`, and store them in a `FAISSStore`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 1. Split Text using Semantica's TextSplitter\n",
+    "splitter = TextSplitter(method=\"recursive\", chunk_size=100, chunk_overlap=20)\n",
+    "chunks = splitter.split(text_content)\n",
+    "\n",
+    "# Extract text from chunk objects\n",
+    "chunk_texts = [chunk.text for chunk in chunks]\n",
+    "print(f\"Created {len(chunks)} chunks.\")\n",
+    "\n",
+    "# 2. Initialize Vector Store\n",
+    "vector_store = FAISSStore(dimension=384) # Standard dimension for MiniLM\n",
+    "vector_store.create_index(index_type=\"flat\", metric=\"L2\")\n",
+    "\n",
+    "# 3. Generate Embeddings using Semantica's EmbeddingGenerator\n",
+    "# This uses local Sentence Transformers by default\n",
+    "generator = EmbeddingGenerator(text={\"method\": \"sentence_transformers\", \"model_name\": \"all-MiniLM-L6-v2\"})\n",
+    "embeddings = generator.generate_embeddings(chunk_texts, data_type=\"text\")\n",
+    "\n",
+    "# 4. Store Vectors\n",
+    "vector_store.add_vectors(embeddings, ids=[f\"chunk_{i}\" for i in range(len(chunks))])\n",
+    "\n",
+    "# 5. Define a search function\n",
+    "def standard_rag_search(query):\n",
+    "    # Generate query embedding\n",
+    "    query_vec = generator.generate_embeddings(query, data_type=\"text\")\n",
+    "    \n",
+    "    # Search\n",
+    "    results = vector_store.search_similar(query_vec, k=2)\n",
+    "    \n",
+    "    # Retrieve actual text\n",
+    "    retrieved_texts = []\n",
+    "    for res in results:\n",
+    "        idx = int(res['id'].split('_')[1])\n",
+    "        retrieved_texts.append(chunk_texts[idx])\n",
+    "    return retrieved_texts\n",
+    "\n",
+    "print(\"Standard RAG Pipeline Built.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. GraphRAG (Knowledge Graph)\n",
+    "Now we extract entities and relationships to build a structured graph using `GraphBuilder` and visualize it with `KGVisualizer`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 1. Extract Entities and Relations\n",
+    "ner = NamedEntityRecognizer()\n",
+    "rel_extractor = RelationExtractor()\n",
+    "\n",
+    "entities = ner.extract_entities(text_content)\n",
+    "relations = rel_extractor.extract_relations(text_content, entities=entities)\n",
+    "\n",
+    "print(f\"Extracted {len(entities)} entities and {len(relations)} relationships.\")\n",
+    "\n",
+    "# 2. Build Graph using Semantica's GraphBuilder\n",
+    "builder = GraphBuilder()\n",
+    "kg = builder.build([{\"entities\": entities, \"relationships\": relations}])\n",
+    "\n",
+    "# 3. Visualize Graph using Semantica's KGVisualizer\n",
+    "viz = KGVisualizer()\n",
+    "fig = viz.visualize_network(kg, output=\"interactive\")\n",
+    "fig.show()\n",
+    "\n",
+    "# Fallback static visualization if interactive fails in some environments\n",
+    "# plt.figure(figsize=(10, 8))\n",
+    "# pos = nx.spring_layout(kg, k=0.5)\n",
+    "# nx.draw(kg, pos, with_labels=True, node_color='lightblue', node_size=2000, font_size=10)\n",
+    "# edge_labels = nx.get_edge_attributes(kg, 'relation')\n",
+    "# nx.draw_networkx_edge_labels(kg, pos, edge_labels=edge_labels)\n",
+    "# plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 4. Define Graph Search Function\n",
+    "def graph_rag_search(start_entity, hops=1):\n",
+    "    if start_entity not in kg:\n",
+    "        # Try simple fuzzy match\n",
+    "        for node in kg.nodes():\n",
+    "            if start_entity.lower() in node.lower():\n",
+    "                start_entity = node\n",
+    "                break\n",
+    "        else:\n",
+    "            return [f\"Entity '{start_entity}' not found in graph.\"]\n",
+    "    \n",
+    "    # Traverse graph\n",
+    "    subgraph_nodes = list(nx.bfs_tree(kg, source=start_entity, depth_limit=hops))\n",
+    "    subgraph = kg.subgraph(subgraph_nodes)\n",
+    "    \n",
+    "    # Convert paths to natural language\n",
+    "    facts = []\n",
+    "    for u, v, data in subgraph.edges(data=True):\n",
+    "        relation = data.get('relation', 'related to')\n",
+    "        facts.append(f\"{u} --[{relation}]--> {v}\")\n",
+    "    \n",
+    "    return facts"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Comparison Results\n",
+    "Let's compare what each system retrieves for the query: **\"What are the risks associated with AI?\"**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query = \"What are the risks associated with AI?\"\n",
+    "\n",
+    "print(f\"Query: {query}\\n\")\n",
+    "\n",
+    "print(\"--- Standard RAG Results (Vector Similarity) ---\")\n",
+    "rag_results = standard_rag_search(query)\n",
+    "for i, res in enumerate(rag_results):\n",
+    "    print(f\"{i+1}. {res}...\")\n",
+    "\n",
+    "print(\"\\n--- GraphRAG Results (Graph Traversal) ---\")\n",
+    "# In a full system, we would link the query \"AI\" to the node \"Artificial Intelligence\"\n",
+    "graph_results = graph_rag_search(\"Artificial Intelligence\", hops=2)\n",
+    "for i, res in enumerate(graph_results):\n",
+    "    print(f\"{i+1}. {res}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
+    "\n",
+    "| Feature | Standard RAG | GraphRAG |\n",
+    "|---------|--------------|----------|\n",
+    "| **Mechanism** | Semantic Similarity (Vector) | Graph Traversal (Structural) |\n",
+    "| **Strengths** | Fast, good for general queries | Captures explicit relationships, multi-hop reasoning |\n",
+    "| **Weaknesses**| May miss disconnected chunks | Requires entity extraction & graph construction overhead |\n",
+    "| **Best For** | Fact retrieval | Complex reasoning & relationship mapping |\n",
+    "\n",
+    "**Semantica** allows you to combine both into a **Hybrid RAG** system for optimal performance."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/cookbook.md

@@ -44,6 +44,16 @@ Hand-picked tutorials to show you the power of Semantica.
     
     [Open Notebook](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/use_cases/advanced_rag/01_GraphRAG_Complete.ipynb)
 
+-   :material-scale-balance: **RAG vs. GraphRAG Comparison**
+    ---
+    Side-by-side comparison of Standard RAG vs. GraphRAG using real-world data.
+    
+    **Topics**: RAG, GraphRAG, Benchmarking, Visualization
+    
+    **Difficulty**: Intermediate
+    
+    [Open Notebook](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/use_cases/advanced_rag/02_RAG_vs_GraphRAG_Comparison.ipynb)
+
 -   :material-graph: **Your First Knowledge Graph**
     ---
     Go from raw text to a queryable knowledge graph in 20 minutes.

docs/use-cases.md

@@ -252,6 +252,7 @@ ontology = ontology_gen.generate_from_graph(kg)
     **Goal**: Use the knowledge graph to retrieve precise context for RAG applications.
     
     [:material-arrow-right: View Cookbook](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/use_cases/advanced_rag/01_GraphRAG_Complete.ipynb)
+    [:material-scale-balance: RAG vs GraphRAG Benchmark](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/use_cases/advanced_rag/02_RAG_vs_GraphRAG_Comparison.ipynb)
 
 -   :material-domain: **Corporate Intelligence**
     ---