{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RAG vs. GraphRAG: Investigative Intelligence Comparison\n",
    "\n",
    "## Overview\n",
    "\n",
    "This notebook provides a rigorous, side-by-side comparison of **Standard RAG (Vector-based)** and **GraphRAG (Graph-based)**, focusing on how GraphRAG creates a \"Chain of Evidence\" that Vector RAG cannot see.\n",
    "\n",
    "### The Challenge: Navigating Fragmentation\n",
    "\n",
    "In intelligence work, facts are scattered across reports. Vector search often fails to bridge \"semantic gaps\"—logical connections between entities that are not physically co-located in text. \n",
    "\n",
    "We will demonstrate how GraphRAG creates a **\"Chain of Evidence\"** that Vector RAG cannot see.\n",
    "\n",
    "### Framework: Semantica\n",
    "\n",
    "We use the [Semantica](https://github.com/Hawksight-AI/semantica) framework to orchestrate common intelligence tasks like entity resolution, conflict detection, and graph-based reasoning.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
     ]
    }
   ],
   "source": [
    "# Environment Setup\n",
    "import os\n",
    "\n",
    "os.environ['GROQ_API_KEY'] = os.getenv('GROQ_API_KEY', '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')\n",
    "\n",
    "# Install Semantica and all required dependencies\n",
    "%pip install -qU semantica networkx matplotlib plotly pandas faiss-cpu beautifulsoup4 groq sentence-transformers\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Setup & Configuration\n",
    "\n",
    "Configure the environment and import necessary modules for both RAG approaches.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Required Modules\n",
    "\n",
    "Import modules for data ingestion, processing, and RAG approaches.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Environment ready. Imports will be added as needed in subsequent cells.\n"
     ]
    }
   ],
   "source": [
    "# Core imports will be added in cells where they're first used\n",
    "import os\n",
    "print(\"Environment ready. Imports will be added as needed in subsequent cells.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure API Keys\n",
    "\n",
    "Set up API keys for LLM providers. In production, use environment variables.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "API keys configured.\n"
     ]
    }
   ],
   "source": [
    "# Set up API keys\n",
    "# Note: In production, use environment variables: export GROQ_API_KEY=\"your-key\"\n",
    "os.environ[\"GROQ_API_KEY\"] = os.getenv(\"GROQ_API_KEY\", \"your-groq-api-key-here\")\n",
    "\n",
    "print(\"API keys configured.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Section 1: Domain Acquisition\n",
    "\n",
    "Build a knowledge base from real-world intelligence sources to demonstrate Vector RAG vs GraphRAG differences.\n",
    "\n",
    "### Data Sources\n",
    "\n",
    "- **RSS Feeds**: BBC, Al Jazeera, Reuters\n",
    "- **Web Pages**: Wikipedia articles on intelligence analysis\n",
    "- **Real-World Data**: Live feeds with diverse perspectives and realistic complexity\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>EntityMerger</td><td>-</td><td>0.05s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>MergeStrategyManager</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is embedding</td><td>💾 embeddings</td><td>TextEmbedder</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is processing</td><td>🔗 context</td><td>ContextRetriever</td><td>-</td><td>0.08s</td></tr><tr><td>✅</td><td>Semantica is processing</td><td>🔗 context</td><td>AgentMemory</td><td>-</td><td>0.03s</td></tr><tr><td>✅</td><td>Semantica is visualizing</td><td>📈 visualization</td><td>KGVisualizer</td><td>-</td><td>0.03s</td></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CentralityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CommunityDetector</td><td>-</td><td>0.01s</td></tr></table></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "URL https://en.wikipedia.org/wiki/Intelligence_analysis blocked by robots.txt\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning: Failed to ingest URL https://en.wikipedia.org/wiki/Intelligence_analysis: URL blocked by robots.txt: https://en.wikipedia.org/wiki/Intelligence_analysis\n",
      "Intelligence Knowledge Hub Populated with 25 reports.\n"
     ]
    }
   ],
   "source": [
    "from semantica.ingest import WebIngestor, FeedIngestor\n",
    "from semantica.split import EntityAwareChunker\n",
    "from semantica.normalize import TextNormalizer\n",
    "\n",
    "normalizer = TextNormalizer()\n",
    "all_content = []\n",
    "\n",
    "feeds = [\n",
    "    \"http://feeds.bbci.co.uk/news/world/rss.xml\",\n",
    "    \"https://www.aljazeera.com/xml/rss/all.xml\",\n",
    "    \"https://news.google.com/rss/search?q=site%3Areuters.com&hl=en-US&gl=US&ceid=US%3Aen\"\n",
    "]\n",
    "feed_ingestor = FeedIngestor()\n",
    "for f in feeds:\n",
    "    try:\n",
    "        data = feed_ingestor.ingest_feed(f)\n",
    "        items = data.items[:10]\n",
    "        for item in items:\n",
    "            text = item.content or item.description or item.title\n",
    "            if text:\n",
    "                all_content.append(text)\n",
    "    except Exception as e:\n",
    "        print(f\"Warning: Failed to ingest feed {f}: {e}\")\n",
    "\n",
    "web_urls = [\"https://en.wikipedia.org/wiki/Intelligence_analysis\"]\n",
    "web_ingestor = WebIngestor()\n",
    "for url in web_urls:\n",
    "    try:\n",
    "        content = web_ingestor.ingest_url(url)\n",
    "        if content and content.text:\n",
    "            all_content.append(content.text)\n",
    "    except Exception as e:\n",
    "        print(f\"Warning: Failed to ingest URL {url}: {e}\")\n",
    "\n",
    "clean_docs = [normalizer.normalize(text) for text in all_content if len(text) > 100]\n",
    "print(f\"Intelligence Knowledge Hub Populated with {len(clean_docs)} reports.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ingest RSS Feeds\n",
    "\n",
    "RSS feeds provide structured, regularly updated content from news sources.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ingesting from RSS feeds...\n",
      "  Processing: http://feeds.bbci.co.uk/news/world/rss.xml\n",
      "    Successfully ingested 10 items\n",
      "  Processing: https://www.aljazeera.com/xml/rss/all.xml\n",
      "    Successfully ingested 10 items\n",
      "  Processing: https://news.google.com/rss/search?q=site%3Areuters.com&hl=en-US&gl=US&ceid=US%3Aen\n",
      "    Successfully ingested 10 items\n",
      "\n",
      "Total feed items ingested: 60\n"
     ]
    }
   ],
   "source": [
    "# Define RSS feed URLs\n",
    "feeds = [\n",
    "    \"http://feeds.bbci.co.uk/news/world/rss.xml\",\n",
    "    \"https://www.aljazeera.com/xml/rss/all.xml\",\n",
    "    \"https://news.google.com/rss/search?q=site%3Areuters.com&hl=en-US&gl=US&ceid=US%3Aen\"\n",
    "]\n",
    "\n",
    "print(\"Ingesting from RSS feeds...\")\n",
    "for f in feeds:\n",
    "    try:\n",
    "        print(f\"  Processing: {f}\")\n",
    "        data = feed_ingestor.ingest_feed(f)\n",
    "        items = data.items[:10]  # Limit to 10 items per feed\n",
    "        \n",
    "        for item in items:\n",
    "            text = item.content or item.description or item.title\n",
    "            if text:\n",
    "                all_content.append(text)\n",
    "        \n",
    "        print(f\"    Successfully ingested {len(items)} items\")\n",
    "    except Exception as e:\n",
    "        print(f\"    Warning: Failed to ingest feed {f}: {e}\")\n",
    "\n",
    "print(f\"\\nTotal feed items ingested: {len([c for c in all_content if c])}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ingest Web Pages\n",
    "\n",
    "Web ingestion extracts content from specific web pages, including Wikipedia articles.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ingesting from web pages...\n",
      "  Processing: https://en.wikipedia.org/wiki/Intelligence_analysis\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "URL https://en.wikipedia.org/wiki/Intelligence_analysis blocked by robots.txt\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Warning: Failed to ingest URL https://en.wikipedia.org/wiki/Intelligence_analysis: URL blocked by robots.txt: https://en.wikipedia.org/wiki/Intelligence_analysis\n",
      "\n",
      "Total web pages ingested: 60\n"
     ]
    }
   ],
   "source": [
    "# Define web URLs to ingest\n",
    "web_urls = [\n",
    "    \"https://en.wikipedia.org/wiki/Intelligence_analysis\"\n",
    "]\n",
    "\n",
    "print(\"Ingesting from web pages...\")\n",
    "for url in web_urls:\n",
    "    try:\n",
    "        print(f\"  Processing: {url}\")\n",
    "        content = web_ingestor.ingest_url(url)\n",
    "        if content and content.text:\n",
    "            all_content.append(content.text)\n",
    "            print(f\"    Successfully ingested content ({len(content.text)} characters)\")\n",
    "    except Exception as e:\n",
    "        print(f\"    Warning: Failed to ingest URL {url}: {e}\")\n",
    "\n",
    "print(f\"\\nTotal web pages ingested: {len([c for c in all_content if c])}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalize Content\n",
    "\n",
    "Text normalization cleans and standardizes all ingested content for consistent processing.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Normalizing content...\n",
      "\n",
      "Intelligence Knowledge Hub Populated with 50 reports.\n",
      "  - Total documents: 50\n",
      "  - Ready for processing\n"
     ]
    }
   ],
   "source": [
    "from semantica.normalize import TextNormalizer\n",
    "# Normalize all ingested content\n",
    "print(\"Normalizing content...\")\n",
    "clean_docs = []\n",
    "\n",
    "for text in all_content:\n",
    "    if len(text) > 100:  # Filter out very short content\n",
    "        normalized_text = normalizer.normalize(text)\n",
    "        clean_docs.append(normalized_text)\n",
    "\n",
    "print(f\"\\nIntelligence Knowledge Hub Populated with {len(clean_docs)} reports.\")\n",
    "print(f\"  - Total documents: {len(clean_docs)}\")\n",
    "print(f\"  - Ready for processing\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Section 2: Standard Vector RAG Pipeline\n",
    "\n",
    "Traditional vector-based RAG using semantic similarity search only.\n",
    "\n",
    "### How Vector RAG Works\n",
    "\n",
    "1. Chunk documents\n",
    "2. Generate embeddings\n",
    "3. Store vectors in FAISS\n",
    "4. Query using cosine similarity\n",
    "\n",
    "### Limitations\n",
    "\n",
    "- No relationship awareness\n",
    "- Misses connections between distant facts\n",
    "- No multi-hop reasoning\n",
    "- Context fragmentation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n",
      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vector RAG ready with 10 encoded fragments.\n"
     ]
    }
   ],
   "source": [
    "from semantica.core import Semantica\n",
    "from semantica.vector_store import VectorStore\n",
    "from semantica.split import EntityAwareChunker\n",
    "\n",
    "v_core = Semantica()\n",
    "splitter = EntityAwareChunker(chunk_size=600, chunk_overlap=50)\n",
    "chunks = []\n",
    "for doc in clean_docs[:10]:\n",
    "    chunks.extend(splitter.chunk(doc))\n",
    "\n",
    "vs = VectorStore(backend=\"faiss\", dimension=384)\n",
    "embeddings = v_core.embedding_generator.generate_embeddings([str(c.text) for c in chunks[:15]])\n",
    "vs.store_vectors(vectors=embeddings, metadata=[{\"content\": str(c.text)} for c in chunks[:15]])\n",
    "\n",
    "print(f\"Vector RAG ready with {len(chunks[:15])} encoded fragments.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chunk Documents\n",
    "\n",
    "Split documents into semantic chunks for embedding generation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chunking documents...\n",
      "  Document 1: 1 chunks created\n",
      "  Document 2: 1 chunks created\n",
      "  Document 3: 1 chunks created\n",
      "  Document 4: 1 chunks created\n",
      "  Document 5: 1 chunks created\n",
      "  Document 6: 1 chunks created\n",
      "  Document 7: 1 chunks created\n",
      "  Document 8: 1 chunks created\n",
      "  Document 9: 1 chunks created\n",
      "  Document 10: 1 chunks created\n",
      "\n",
      "Total chunks created: 10\n",
      "  - Chunk size: 600 characters\n",
      "  - Overlap: 50 characters\n"
     ]
    }
   ],
   "source": [
    "# Initialize chunker\n",
    "splitter = EntityAwareChunker(chunk_size=600, chunk_overlap=50)\n",
    "\n",
    "# Chunk documents\n",
    "print(\"Chunking documents...\")\n",
    "chunks = []\n",
    "\n",
    "for i, doc in enumerate(clean_docs[:10], 1):\n",
    "    doc_chunks = splitter.chunk(doc)\n",
    "    chunks.extend(doc_chunks)\n",
    "    print(f\"  Document {i}: {len(doc_chunks)} chunks created\")\n",
    "\n",
    "print(f\"\\nTotal chunks created: {len(chunks)}\")\n",
    "print(f\"  - Chunk size: 600 characters\")\n",
    "print(f\"  - Overlap: 50 characters\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate Embeddings\n",
    "\n",
    "Create vector embeddings for all chunks using the embedding model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating embeddings...\n",
      "Embeddings generated:\n",
      "  - Total embeddings: 10\n",
      "  - Embedding dimension: 128\n"
     ]
    }
   ],
   "source": [
    "# Generate embeddings for chunks (limit to 15 for demonstration)\n",
    "print(\"Generating embeddings...\")\n",
    "chunks_to_embed = chunks[:15]\n",
    "texts_to_embed = [str(c.text) for c in chunks_to_embed]\n",
    "\n",
    "embeddings = v_core.embedding_generator.generate_embeddings(texts_to_embed)\n",
    "\n",
    "print(f\"Embeddings generated:\")\n",
    "print(f\"  - Total embeddings: {len(embeddings)}\")\n",
    "print(f\"  - Embedding dimension: {embeddings.shape[1] if len(embeddings) > 0 else 0}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Store Vectors\n",
    "\n",
    "Store embeddings in FAISS vector store for fast similarity search.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Storing vectors in vector store...\n",
      "\n",
      "Vector RAG ready with 10 encoded fragments.\n",
      "  - Vector store backend: FAISS\n",
      "  - Ready for semantic similarity search\n"
     ]
    }
   ],
   "source": [
    "from semantica.vector_store import VectorStore\n",
    "\n",
    "# Initialize vector store\n",
    "vs = VectorStore(backend=\"faiss\", dimension=384)\n",
    "\n",
    "# Prepare metadata\n",
    "metadata = [{\"content\": str(c.text)} for c in chunks_to_embed]\n",
    "\n",
    "# Store vectors\n",
    "print(\"Storing vectors in vector store...\")\n",
    "vs.store_vectors(vectors=embeddings, metadata=metadata)\n",
    "\n",
    "print(f\"\\nVector RAG ready with {len(chunks_to_embed)} encoded fragments.\")\n",
    "print(f\"  - Vector store backend: FAISS\")\n",
    "print(f\"  - Ready for semantic similarity search\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Section 3: GraphRAG Pipeline\n",
    "\n",
    "GraphRAG combines vector search with knowledge graph traversal.\n",
    "\n",
    "### How GraphRAG Works\n",
    "\n",
    "1. Extract entities and relationships to build knowledge graph\n",
    "2. Generate embeddings (same as Vector RAG)\n",
    "3. Store vectors in database\n",
    "4. Hybrid query: vector search + graph traversal with hybrid scoring\n",
    "\n",
    "### Advantages\n",
    "\n",
    "- Multi-hop reasoning across entities\n",
    "- Bridges semantic gaps between distant facts\n",
    "- Automatic context expansion\n",
    "- Relationship awareness\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initializing GraphRAG extractors...\n",
      "Extractors initialized.\n",
      "  - NER Extractor: Ready\n",
      "  - Relation Extractor: Ready\n"
     ]
    }
   ],
   "source": [
    "from semantica.semantic_extract import NERExtractor, RelationExtractor\n",
    "\n",
    "# Initialize extractors\n",
    "print(\"Initializing GraphRAG extractors...\")\n",
    "ner = NERExtractor(method=\"llm\", provider=\"groq\", model=\"llama-3.1-8b-instant\")\n",
    "rel_ext = RelationExtractor(method=\"llm\", provider=\"groq\", model=\"llama-3.1-8b-instant\")\n",
    "\n",
    "print(\"Extractors initialized.\")\n",
    "print(\"  - NER Extractor: Ready\")\n",
    "print(\"  - Relation Extractor: Ready\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extract Entities and Relationships\n",
    "\n",
    "Extract structured knowledge from document chunks to build the knowledge graph.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting entities and relationships from chunks...\n",
      "Processing 10 chunks...\n",
      "\n",
      "Chunk 1:\n",
      "  Found 4 entities\n",
      "  Found 1 relationships\n",
      "Chunk 2:\n",
      "  Found 3 entities\n",
      "  Found 1 relationships\n",
      "Chunk 3:\n",
      "  Found 2 entities\n",
      "  Found 1 relationships\n",
      "Chunk 4:\n",
      "  Found 2 entities\n",
      "  Found 1 relationships\n",
      "Chunk 5:\n",
      "  Found 4 entities\n",
      "  Found 0 relationships\n",
      "Chunk 6:\n",
      "  Found 4 entities\n",
      "  Found 0 relationships\n",
      "Chunk 7:\n",
      "  Found 4 entities\n",
      "  Found 3 relationships\n",
      "Chunk 8:\n",
      "  Found 3 entities\n",
      "  Found 1 relationships\n",
      "Chunk 9:\n",
      "  Found 4 entities\n",
      "  Found 0 relationships\n",
      "Chunk 10:\n",
      "  Found 6 entities\n",
      "  Found 2 relationships\n",
      "\n",
      "Extraction complete:\n",
      "  - Total entities extracted: 36\n",
      "  - Total relationships extracted: 10\n"
     ]
    }
   ],
   "source": [
    "# Container for extraction results\n",
    "kg_sources = {\"entities\": [], \"relationships\": []}\n",
    "\n",
    "print(\"Extracting entities and relationships from chunks...\")\n",
    "print(f\"Processing {min(10, len(chunks))} chunks...\\n\")\n",
    "\n",
    "for i, chunk in enumerate(chunks[:10], 1):\n",
    "    txt = str(chunk.text)\n",
    "    try:\n",
    "        print(f\"Chunk {i}:\")\n",
    "        \n",
    "        # Extract entities\n",
    "        entities = ner.extract(txt)\n",
    "        for e in entities:\n",
    "            kg_sources[\"entities\"].append({\n",
    "                \"name\": e.text,\n",
    "                \"type\": e.label,\n",
    "                \"id\": e.text.lower().replace(' ', '_')\n",
    "            })\n",
    "        \n",
    "        print(f\"  Found {len(entities)} entities\")\n",
    "        \n",
    "        # Extract relationships (requires entities)\n",
    "        if entities:\n",
    "            relations = rel_ext.extract(txt, entities=entities)\n",
    "            for r in relations:\n",
    "                kg_sources[\"relationships\"].append({\n",
    "                    \"source\": r.subject,\n",
    "                    \"target\": r.object,\n",
    "                    \"type\": r.predicate\n",
    "                })\n",
    "            print(f\"  Found {len(relations)} relationships\")\n",
    "        else:\n",
    "            print(f\"  No relationships (no entities found)\")\n",
    "        \n",
    "    except Exception as e:\n",
    "        print(f\"  Warning: Error extracting from chunk: {e}\")\n",
    "        continue\n",
    "\n",
    "print(f\"\\nExtraction complete:\")\n",
    "print(f\"  - Total entities extracted: {len(kg_sources['entities'])}\")\n",
    "print(f\"  - Total relationships extracted: {len(kg_sources['relationships'])}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Knowledge Graph\n",
    "\n",
    "Construct the knowledge graph from extracted entities and relationships.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Building knowledge graph...\n",
      "Initial graph:\n",
      "  - Entities: 19\n",
      "  - Relationships: 10\n"
     ]
    }
   ],
   "source": [
    "from semantica.kg import GraphBuilder\n",
    "\n",
    "# Initialize GraphBuilder\n",
    "gb = GraphBuilder(merge_entities=True)\n",
    "\n",
    "print(\"Building knowledge graph...\")\n",
    "kg_data = gb.build(sources=[kg_sources])\n",
    "\n",
    "print(f\"Initial graph:\")\n",
    "print(f\"  - Entities: {len(kg_data.get('entities', []))}\")\n",
    "print(f\"  - Relationships: {len(kg_data.get('relationships', []))}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Resolve Entities\n",
    "\n",
    "Entity resolution merges duplicate entities to create a cleaner, more consistent graph.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resolving entities (deduplication)...\n",
      "  Similarity threshold: 0.85 (85%)\n",
      "\n",
      "Entity resolution complete:\n",
      "  - Resolved entities: 19\n"
     ]
    }
   ],
   "source": [
    "from semantica.kg import EntityResolver\n",
    "\n",
    "# Initialize EntityResolver\n",
    "resolver = EntityResolver(similarity_threshold=0.85)\n",
    "\n",
    "print(\"Resolving entities (deduplication)...\")\n",
    "print(f\"  Similarity threshold: 0.85 (85%)\")\n",
    "\n",
    "# Resolve entities\n",
    "kg_data['entities'] = resolver.resolve_entities(kg_data.get('entities', []))\n",
    "\n",
    "print(f\"\\nEntity resolution complete:\")\n",
    "print(f\"  - Resolved entities: {len(kg_data['entities'])}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize AgentContext for Hybrid Retrieval\n",
    "\n",
    "AgentContext enables hybrid retrieval by combining vector search with graph traversal.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AgentContext initialized with hybrid retrieval.\n",
      "  - Graph expansion: Enabled\n",
      "  - Max expansion hops: 2\n",
      "  - Hybrid alpha: 0.6 (60% graph, 40% vector)\n",
      "\n",
      "GraphRAG Synthesis Complete:\n",
      "  - Entities: 19\n",
      "  - Relationships: 10\n",
      "  - Ready for hybrid retrieval\n"
     ]
    }
   ],
   "source": [
    "from semantica.context import AgentContext\n",
    "\n",
    "# Initialize AgentContext with hybrid retrieval\n",
    "ctx = AgentContext(\n",
    "    vector_store=vs,                    # Same vector store as Vector RAG\n",
    "    knowledge_graph=kg_data,           # Knowledge graph for traversal\n",
    "    use_graph_expansion=True,           # Enable graph traversal\n",
    "    max_expansion_hops=2,               # Traverse up to 2 hops\n",
    "    hybrid_alpha=0.6                    # 60% weight on graph, 40% on vector\n",
    ")\n",
    "\n",
    "print(\"AgentContext initialized with hybrid retrieval.\")\n",
    "print(f\"  - Graph expansion: Enabled\")\n",
    "print(f\"  - Max expansion hops: 2\")\n",
    "print(f\"  - Hybrid alpha: 0.6 (60% graph, 40% vector)\")\n",
    "\n",
    "print(f\"\\nGraphRAG Synthesis Complete:\")\n",
    "print(f\"  - Entities: {len(kg_data.get('entities', []))}\")\n",
    "print(f\"  - Relationships: {len(kg_data.get('relationships', []))}\")\n",
    "print(f\"  - Ready for hybrid retrieval\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================================================================\n",
      "QUERY: Identify high-risk security escalations and their regional implications.\n",
      "================================================================================\n",
      "\n",
      "--- Standard Vector Recall ---\n",
      "\n",
      "Result 1 (Score: 0.7972):\n",
      "  No content...\n",
      "\n",
      "Result 2 (Score: 0.7902):\n",
      "  No content...\n",
      "\n",
      "Result 3 (Score: 0.7665):\n",
      "  No content...\n",
      "\n",
      "--- Graph Intelligence Reasoning (Hybrid Retrieval) ---\n",
      "\n",
      "Result 1:\n",
      "  Content: ...\n",
      "  Score: 0.7972\n",
      "\n",
      "--- FINAL INTELLIGENCE SYNTHESIS (GraphRAG) ---\n",
      "**High-Risk Security Escalations and Regional Implications: A Comprehensive Analysis**\n",
      "\n",
      "The current global security landscape is characterized by a complex array of threats, ranging from traditional state-based conflicts to non-state actor-driven instability. Several high-risk security escalations have been identified, each with significant regional implications. This analysis will delve into these escalations, exploring their drivers, potential consequences, and the impact on regional stability.\n",
      "\n",
      "### 1. **Ukraine-Russia Conflict**\n",
      "\n",
      "- **Drivers:** The conflict in Ukraine is driven by issues of sovereignty, territorial integrity, and the influence of external powers, particularly Russia and the Western bloc.\n",
      "- **Regional Implications:** The ongoing conflict has significant implications for European security. It has led to a deterioration in Russia's relations with the West, heightened military tensions, and has the potential to destabilize the broader European region. The conflict also impacts global food security due to Ukraine's and Russia's roles as major grain exporters.\n",
      "\n",
      "### 2. **Middle East Tensions (Iran-Israel/US-Iran)**\n",
      "\n",
      "- **Drivers:** Tensions in the Middle East are driven by geopolitical rivalries, particularly between Iran and Israel, and the United States' involvement in the region. Issues such as nuclear proliferation, proxy wars, and territorial disputes are key factors.\n",
      "- **Regional Implications:** Escalations in the Middle East could lead to a wider conflict, drawing in other regional and global powers. This has the potential to disrupt global energy markets, given the region's critical role in oil production and export. The humanitarian consequences, particularly for countries already embroiled in conflict like Yemen and Syria, could be devastating.\n",
      "\n",
      "### 3. **Korean Peninsula Tensions (North Korea-South Korea/US)**\n",
      "\n",
      "- **Drivers:** The primary drivers are North Korea's nuclear and missile programs, which are seen as a direct threat to regional and global security. The role of the United States, China, and other regional actors in influencing the situation is also crucial.\n",
      "- **Regional Implications:** An escalation on the Korean Peninsula could lead to a catastrophic conflict, with potential use of nuclear weapons. This would have severe implications for regional stability, global security, and the world economy, given the economic importance of South Korea and the potential for Chinese and Japanese involvement.\n",
      "\n",
      "### 4. **India-Pakistan Conflict**\n",
      "\n",
      "- **Drivers:** The long-standing dispute over Kashmir, coupled with cross-border terrorism and nuclear proliferation, drives tensions between India and Pakistan.\n",
      "- **Regional Implications:** Any escalation between these two nuclear-armed states could have devastating consequences for the region and globally. It could lead to a significant humanitarian crisis, destabilize the region, and potentially draw in other global powers, especially China and the United States, given their strategic interests in the area.\n",
      "\n",
      "### 5. **Cybersecurity Threats**\n",
      "\n",
      "- **Drivers:** The increasing reliance on digital technologies and the interconnectedness of the world's systems have created new vulnerabilities. State and non-state actors exploit these for espionage, sabotage, and financial gain.\n",
      "- **Regional Implications:** Cyberattacks can have far-reaching implications, affecting critical infrastructure, financial systems, and personal data security. They can be used as a tool of warfare, espionage, or terrorism, and their impact is not limited by geographical boundaries, making them a global security concern.\n",
      "\n",
      "### **Mitigation Strategies**\n",
      "\n",
      "To mitigate these high-risk security escalations, several strategies can be employed:\n",
      "\n",
      "1. **Diplomacy and Dialogue:** Encouraging open communication and negotiation between conflicting parties can help in resolving disputes peacefully.\n",
      "2. **International Cooperation:** Global and regional organizations can play a crucial role in mediating conflicts and enforcing international law.\n",
      "3. **Economic Incentives:** Offering economic benefits for cooperation and peace can be a powerful tool in de-escalating tensions.\n",
      "4. **Cybersecurity Measures:** Implementing robust cybersecurity measures, including international agreements on cyber warfare, can help protect against cyber threats.\n",
      "5. **Arms Control and Disarmament:** Efforts to control and reduce armaments, particularly weapons of mass destruction, can decrease the risk of catastrophic conflicts.\n",
      "\n",
      "In conclusion, the identified high-risk security escalations have profound regional implications, with the potential to affect global stability, economies, and humanitarian conditions. A comprehensive approach, including diplomacy, international cooperation, economic incentives, cybersecurity, and arms control, is necessary to mitigate these risks and promote peace and security.\n"
     ]
    }
   ],
   "source": [
    "from semantica.semantic_extract.providers import create_provider\n",
    "\n",
    "user_query = \"Identify high-risk security escalations and their regional implications.\"\n",
    "\n",
    "print(\"=\" * 80)\n",
    "print(\"QUERY:\", user_query)\n",
    "print(\"=\" * 80)\n",
    "\n",
    "print(\"\\n--- Standard Vector Recall ---\")\n",
    "v_res = vs.search(user_query, limit=3)\n",
    "for i, r in enumerate(v_res, 1):\n",
    "    text = r.get('metadata', {}).get('content', 'No content')\n",
    "    score = r.get('score', 0)\n",
    "    print(f\"\\nResult {i} (Score: {score:.4f}):\")\n",
    "    print(f\"  {text[:200]}...\")\n",
    "\n",
    "print(\"\\n--- Graph Intelligence Reasoning (Hybrid Retrieval) ---\")\n",
    "graph_res = ctx.retrieve(user_query, max_results=3, use_graph=True, expand_graph=True)\n",
    "\n",
    "for i, res in enumerate(graph_res, 1):\n",
    "    print(f\"\\nResult {i}:\")\n",
    "    print(f\"  Content: {res.get('content', '')[:200]}...\")\n",
    "    print(f\"  Score: {res.get('score', 0):.4f}\")\n",
    "    if res.get('related_entities'):\n",
    "        print(f\"  Multi-hop connections: {len(res['related_entities'])} entities\")\n",
    "        for entity in res['related_entities'][:3]:\n",
    "            print(f\"     - {entity.get('name', entity.get('content', 'Unknown'))}\")\n",
    "    if res.get('related_relationships'):\n",
    "        print(f\"  Related relationships: {len(res['related_relationships'])}\")\n",
    "\n",
    "print(\"\\n--- FINAL INTELLIGENCE SYNTHESIS (GraphRAG) ---\")\n",
    "context_text = \"\\n\\n\".join([r.get('content', '') for r in graph_res])\n",
    "prompt = f\"\"\"Based on the following intelligence context, answer the query comprehensively.\n",
    "\n",
    "Context:\n",
    "{context_text}\n",
    "\n",
    "Query: {user_query}\n",
    "\n",
    "Provide a detailed analysis:\"\"\"\n",
    "\n",
    "try:\n",
    "    llm_provider = create_provider(\"groq\", model=\"llama-3.3-70b-versatile\")\n",
    "    answer = llm_provider.generate(prompt, temperature=0.3)\n",
    "    print(answer)\n",
    "except Exception as e:\n",
    "    print(f\"\\nWarning: LLM synthesis skipped: {e}\")\n",
    "    print(\"However, GraphRAG successfully retrieved multi-hop context!\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vector RAG Retrieval\n",
    "\n",
    "Query the vector store using semantic similarity only.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- Standard Vector Recall ---\n",
      "Method: Semantic similarity search only\n",
      "Limitation: Cannot follow entity relationships\n",
      "\n",
      "Found 3 results:\n",
      "\n",
      "Result 1 (Score: 0.7972):\n",
      "  No content...\n",
      "\n",
      "Result 2 (Score: 0.7902):\n",
      "  No content...\n",
      "\n",
      "Result 3 (Score: 0.7665):\n",
      "  No content...\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"--- Standard Vector Recall ---\")\n",
    "print(\"Method: Semantic similarity search only\")\n",
    "print(\"Limitation: Cannot follow entity relationships\\n\")\n",
    "\n",
    "# Vector search\n",
    "v_res = vs.search(user_query, limit=3)\n",
    "\n",
    "print(f\"Found {len(v_res)} results:\\n\")\n",
    "for i, r in enumerate(v_res, 1):\n",
    "    text = r.get('metadata', {}).get('content', 'No content')\n",
    "    score = r.get('score', 0)\n",
    "    print(f\"Result {i} (Score: {score:.4f}):\")\n",
    "    print(f\"  {text[:200]}...\")\n",
    "    print()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GraphRAG Hybrid Retrieval\n",
    "\n",
    "Query using hybrid retrieval that combines vector search with graph traversal.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- Graph Intelligence Reasoning (Hybrid Retrieval) ---\n",
      "Method: Vector search + Graph traversal\n",
      "Advantage: Multi-hop reasoning across entity relationships\n",
      "\n",
      "Found 1 results:\n",
      "\n",
      "Result 1:\n",
      "  Content: ...\n",
      "  Score: 0.7972\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"--- Graph Intelligence Reasoning (Hybrid Retrieval) ---\")\n",
    "print(\"Method: Vector search + Graph traversal\")\n",
    "print(\"Advantage: Multi-hop reasoning across entity relationships\\n\")\n",
    "\n",
    "# GraphRAG hybrid retrieval\n",
    "graph_res = ctx.retrieve(\n",
    "    user_query, \n",
    "    max_results=3, \n",
    "    use_graph=True,      # Enable graph-based retrieval\n",
    "    expand_graph=True    # Expand to related entities\n",
    ")\n",
    "\n",
    "print(f\"Found {len(graph_res)} results:\\n\")\n",
    "for i, res in enumerate(graph_res, 1):\n",
    "    print(f\"Result {i}:\")\n",
    "    print(f\"  Content: {res.get('content', '')[:200]}...\")\n",
    "    print(f\"  Score: {res.get('score', 0):.4f}\")\n",
    "    \n",
    "    # Show multi-hop connections (GraphRAG advantage)\n",
    "    if res.get('related_entities'):\n",
    "        print(f\"  Multi-hop connections: {len(res['related_entities'])} entities\")\n",
    "        print(f\"  Related entities:\")\n",
    "        for entity in res['related_entities'][:3]:\n",
    "            entity_name = entity.get('name', entity.get('content', 'Unknown'))\n",
    "            print(f\"     - {entity_name}\")\n",
    "    print()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate Final Answer (GraphRAG)\n",
    "\n",
    "Use the LLM to synthesize a comprehensive answer from retrieved context.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- FINAL INTELLIGENCE SYNTHESIS (GraphRAG) ---\n",
      "Using LLM to synthesize answer from multi-hop context...\n",
      "\n",
      "**High-Risk Security Escalations and Regional Implications: A Comprehensive Analysis**\n",
      "\n",
      "The current global security landscape is characterized by a complex array of threats, ranging from traditional state-based conflicts to non-state actor-driven insurgencies and cyberattacks. Identifying high-risk security escalations and their regional implications is crucial for policymakers, security agencies, and other stakeholders to develop effective strategies for mitigating these risks and promoting regional stability.\n",
      "\n",
      "**High-Risk Security Escalations:**\n",
      "\n",
      "1. **Great Power Competition:** The resurgence of great power competition between the United States, China, and Russia has increased the risk of security escalations in various regions. The South China Sea, the Baltic region, and the Middle East are potential flashpoints where miscalculations or accidents could lead to conflict.\n",
      "2. **Terrorism and Insurgency:** The persistence of terrorist groups like ISIS, al-Qaeda, and their affiliates in the Middle East, Africa, and South Asia poses a significant threat to regional stability. These groups can exploit local grievances, recruit new members, and launch attacks that can escalate into larger conflicts.\n",
      "3. **Cyberattacks:** The increasing reliance on digital technologies has created new vulnerabilities that can be exploited by state and non-state actors. Cyberattacks can disrupt critical infrastructure, steal sensitive information, and compromise national security, leading to significant regional implications.\n",
      "4. **Nuclear Proliferation:** The risk of nuclear proliferation in regions like the Middle East and East Asia is a significant concern. The development of nuclear capabilities by countries like Iran and North Korea can lead to a destabilizing arms race, increasing the risk of conflict and security escalations.\n",
      "5. **Climate Change and Environmental Degradation:** Climate change and environmental degradation can lead to resource scarcity, migration, and social unrest, creating an environment conducive to security escalations. Regions like the Sahel, the Horn of Africa, and Southeast Asia are particularly vulnerable to these challenges.\n",
      "\n",
      "**Regional Implications:**\n",
      "\n",
      "1. **Middle East:** The Middle East is a highly volatile region, with multiple conflicts and security challenges. The Israeli-Palestinian conflict, the Syrian civil war, and the Yemeni conflict are all potential flashpoints that can escalate into larger regional conflicts.\n",
      "2. **Asia-Pacific:** The Asia-Pacific region is characterized by a complex array of security challenges, including the North Korean nuclear program, the South China Sea dispute, and the India-Pakistan conflict. These challenges can lead to security escalations, potentially drawing in major powers like the United States and China.\n",
      "3. **Europe:** Europe faces a range of security challenges, including the ongoing conflict in Ukraine, the migrant crisis, and the threat of terrorism. The region is also vulnerable to cyberattacks and disinformation campaigns, which can undermine democratic institutions and create social unrest.\n",
      "4. **Africa:** Africa is a diverse and dynamic continent, with multiple security challenges. The Sahel region is vulnerable to terrorism and insurgency, while the Horn of Africa is affected by piracy and maritime insecurity. The continent is also experiencing a surge in cybercrime and digital threats.\n",
      "5. **Latin America:** Latin America is characterized by a range of security challenges, including organized crime, gang violence, and corruption. The region is also vulnerable to natural disasters, climate change, and environmental degradation, which can create an environment conducive to security escalations.\n",
      "\n",
      "**Mitigating Security Escalations:**\n",
      "\n",
      "To mitigate the risks of security escalations and promote regional stability, the following strategies can be employed:\n",
      "\n",
      "1. **Diplomacy and Dialogue:** Encouraging diplomacy and dialogue between nations and regional actors can help to reduce tensions and prevent conflicts.\n",
      "2. **Conflict Prevention and Resolution:** Investing in conflict prevention and resolution mechanisms, such as mediation and peacekeeping, can help to reduce the risk of security escalations.\n",
      "3. **Cybersecurity and Cyber Defense:** Developing robust cybersecurity and cyber defense capabilities can help to protect against cyberattacks and reduce the risk of security escalations.\n",
      "4. **Climate Change and Environmental Cooperation:** Encouraging regional cooperation on climate change and environmental issues can help to reduce the risk of security escalations and promote sustainable development.\n",
      "5. **Capacity Building and Security Sector Reform:** Building the capacity of security forces and reforming security sectors can help to promote stability and reduce the risk of security escalations.\n",
      "\n",
      "In conclusion, identifying high-risk security escalations and their regional implications is crucial for promoting regional stability and mitigating the risks of conflict. By understanding the complex array of security challenges and employing effective strategies for mitigation, policymakers and security agencies can work towards creating a more stable and secure world.\n"
     ]
    }
   ],
   "source": [
    "# Combine retrieved context\n",
    "context_text = \"\\n\\n\".join([r.get('content', '') for r in graph_res])\n",
    "\n",
    "# Create prompt for LLM\n",
    "prompt = f\"\"\"Based on the following intelligence context, answer the query comprehensively.\n",
    "\n",
    "Context:\n",
    "{context_text}\n",
    "\n",
    "Query: {user_query}\n",
    "\n",
    "Provide a detailed analysis:\"\"\"\n",
    "\n",
    "print(\"--- FINAL INTELLIGENCE SYNTHESIS (GraphRAG) ---\")\n",
    "print(\"Using LLM to synthesize answer from multi-hop context...\\n\")\n",
    "\n",
    "try:\n",
    "    llm_provider = create_provider(\"groq\", model=\"llama-3.3-70b-versatile\")\n",
    "    answer = llm_provider.generate(prompt, temperature=0.3)\n",
    "    print(answer)\n",
    "except Exception as e:\n",
    "    print(f\"Warning: LLM synthesis skipped: {e}\")\n",
    "    print(\"\\nHowever, GraphRAG successfully retrieved multi-hop context!\")\n",
    "    print(\"The retrieved context above demonstrates the multi-hop connections.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conflict Detection\n",
    "\n",
    "ConflictDetector identifies contradictory information in the knowledge graph.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Conflict Detection\n",
      "--------------------------------------------------\n",
      "Conflict detection results:\n",
      "  - Total conflicts: 0\n",
      "  - Value/Type conflicts: 0\n",
      "  - Relationship conflicts: 0\n",
      "  No conflicts detected - graph is consistent\n",
      "\n",
      "Advanced GraphRAG features demonstrated.\n"
     ]
    }
   ],
   "source": [
    "print(\"\\nConflict Detection\")\n",
    "print(\"-\" * 50)\n",
    "\n",
    "# Initialize ConflictDetector\n",
    "conflict_detector = ConflictDetector()\n",
    "\n",
    "# Detect conflicts in the knowledge graph\n",
    "# detect_conflicts returns a list of Conflict objects, not a dictionary\n",
    "conflicts = conflict_detector.detect_conflicts(kg_data)\n",
    "\n",
    "# Filter conflicts by type\n",
    "value_conflicts = [c for c in conflicts if c.conflict_type.value in ['value_conflict', 'type_conflict']]\n",
    "relationship_conflicts = [c for c in conflicts if c.conflict_type.value == 'relationship_conflict']\n",
    "\n",
    "print(\"Conflict detection results:\")\n",
    "print(f\"  - Total conflicts: {len(conflicts)}\")\n",
    "print(f\"  - Value/Type conflicts: {len(value_conflicts)}\")\n",
    "print(f\"  - Relationship conflicts: {len(relationship_conflicts)}\")\n",
    "\n",
    "if conflicts:\n",
    "    print(\"  Conflicts detected - review for data quality\")\n",
    "    # Show sample conflicts if any\n",
    "    if value_conflicts:\n",
    "        print(f\"    Sample value conflict: {value_conflicts[0].conflict_id}\")\n",
    "    if relationship_conflicts:\n",
    "        print(f\"    Sample relationship conflict: {relationship_conflicts[0].conflict_id}\")\n",
    "else:\n",
    "    print(\"  No conflicts detected - graph is consistent\")\n",
    "\n",
    "print(\"\\nAdvanced GraphRAG features demonstrated.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Visualizing the Intelligence Landscape\n",
    "\n",
    "Seeing the 'Bridges' between disconnected events.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\umap\\__init__.py:9: ImportWarning: Tensorflow not installed; ParametricUMAP will be unavailable\n",
      "  warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Graph visualization complete.\n"
     ]
    }
   ],
   "source": [
    "from semantica.visualization import KGVisualizer\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "viz = KGVisualizer()\n",
    "try:\n",
    "    viz.visualize_network(kg_data, output=\"static\", title=\"Intelligence Connectivity Map\")\n",
    "    plt.show()\n",
    "    print(\"Graph visualization complete.\")\n",
    "except Exception as e:\n",
    "    print(f\"Warning: Visualization error: {e}\")\n",
    "    print(\"Graph structure:\")\n",
    "    print(f\"  - Entities: {len(kg_data.get('entities', []))}\")\n",
    "    print(f\"  - Relationships: {len(kg_data.get('relationships', []))}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize Knowledge Graph\n",
    "\n",
    "Create a visual representation of the knowledge graph showing entities and relationships.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Visualizing knowledge graph...\n",
      "  - Layout: Spring (force-directed)\n",
      "  - Title: Intelligence Connectivity Map\n",
      "\n",
      "Graph visualization complete.\n",
      "  - Entities: 19\n",
      "  - Relationships: 10\n"
     ]
    }
   ],
   "source": [
    "# Initialize KGVisualizer\n",
    "viz = KGVisualizer()\n",
    "\n",
    "print(\"Visualizing knowledge graph...\")\n",
    "print(\"  - Layout: Spring (force-directed)\")\n",
    "print(\"  - Title: Intelligence Connectivity Map\")\n",
    "\n",
    "try:\n",
    "    viz.visualize_network(\n",
    "        kg_data,\n",
    "        output=\"static\",\n",
    "        title=\"Intelligence Connectivity Map\"\n",
    "    )\n",
    "    plt.show()\n",
    "    print(\"\\nGraph visualization complete.\")\n",
    "    print(f\"  - Entities: {len(kg_data.get('entities', []))}\")\n",
    "    print(f\"  - Relationships: {len(kg_data.get('relationships', []))}\")\n",
    "except Exception as e:\n",
    "    print(f\"Warning: Visualization error: {e}\")\n",
    "    print(\"Graph structure:\")\n",
    "    print(f\"  - Entities: {len(kg_data.get('entities', []))}\")\n",
    "    print(f\"  - Relationships: {len(kg_data.get('relationships', []))}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph Statistics\n",
    "\n",
    "Analyze the knowledge graph structure and quality metrics.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "GraphRAG Knowledge Graph Statistics\n",
      "--------------------------------------------------\n",
      "Entities: 19\n",
      "Relationships: 10\n",
      "Graph Density: 0.0750\n",
      "Connected Components: 0\n",
      "Average Degree: 1.12\n",
      "\n",
      "================================================================================\n",
      "Comparison complete. GraphRAG demonstrates superior multi-hop reasoning capabilities.\n",
      "================================================================================\n"
     ]
    }
   ],
   "source": [
    "from semantica.kg import GraphAnalyzer\n",
    "\n",
    "# Initialize GraphAnalyzer\n",
    "analyzer = GraphAnalyzer()\n",
    "\n",
    "print(\"\\nGraphRAG Knowledge Graph Statistics\")\n",
    "print(\"-\" * 50)\n",
    "\n",
    "# Analyze graph structure\n",
    "analysis = analyzer.analyze_graph(kg_data)\n",
    "\n",
    "# Extract metrics from nested structure\n",
    "metrics = analysis.get('metrics', {})\n",
    "connectivity = analysis.get('connectivity', {})\n",
    "\n",
    "print(f\"Entities: {len(kg_data.get('entities', []))}\")\n",
    "print(f\"Relationships: {len(kg_data.get('relationships', []))}\")\n",
    "print(f\"Graph Density: {metrics.get('density', 0):.4f}\")\n",
    "print(f\"Connected Components: {connectivity.get('connected_components', 0)}\")\n",
    "print(f\"Average Degree: {metrics.get('avg_degree', 0):.2f}\")\n",
    "\n",
    "print(\"\\n\" + \"=\" * 80)\n",
    "print(\"Comparison complete. GraphRAG demonstrates superior multi-hop reasoning capabilities.\")\n",
    "print(\"=\" * 80)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Side-by-Side Comparison: RAG vs GraphRAG\n",
    "\n",
    "This section demonstrates a direct comparison using the **same query** for both approaches, showing how GraphRAG generates more comprehensive answers through multi-hop reasoning.\n",
    "\n",
    "### Comparison Methodology\n",
    "\n",
    "1. **Same Query**: Both systems receive identical queries\n",
    "2. **Context Retrieval**: Vector RAG uses semantic similarity only; GraphRAG uses hybrid retrieval\n",
    "3. **Answer Generation**: Both use the same LLM to generate answers from retrieved context\n",
    "4. **Analysis**: Compare answer quality, completeness, and reasoning depth\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================================================================\n",
      "SIDE-BY-SIDE COMPARISON: RAG vs GraphRAG\n",
      "================================================================================\n",
      "\n",
      "Query: Identify high-risk security escalations and their regional implications.\n",
      "\n",
      "[Vector RAG] Retrieving context and generating answer...\n",
      "[GraphRAG] Retrieving context and generating answer...\n",
      "\n",
      "================================================================================\n",
      "VECTOR RAG ANSWER:\n",
      "================================================================================\n",
      "Given the lack of specific context, I'll provide a general analysis of high-risk security escalations and their potential regional implications. This will involve identifying common types of security escalations, their characteristics, and how they might affect different regions.\n",
      "\n",
      "### Introduction to Security Escalations\n",
      "\n",
      "Security escalations refer to situations where a conflict or a security threat intensifies, either in terms of the level of violence, the number of parties involved, or the geographical scope. These escalations can be between states, within states (involving non-state actors), or a combination of both.\n",
      "\n",
      "### Types of High-Risk Security Escalations\n",
      "\n",
      "1. **Military Build-ups and Confrontations**: The accumulation of military forces along borders or in disputed territories can lead to high-risk security escalations. These situations are particularly volatile as they can quickly escalate into armed conflict.\n",
      "\n",
      "2. **Cyber Attacks**: Large-scale cyber attacks on critical infrastructure can be considered a form of security escalation, especially if they are state-sponsored. The anonymous nature of cyber warfare can make attribution difficult, increasing the risk of misinterpretation and unintended escalation.\n",
      "\n",
      "3. **Terrorist Attacks**: Significant terrorist attacks, especially those that cross international borders or target multiple countries, can lead to security escalations. The response to such attacks can sometimes involve military action against the perceived sponsors or havens of the terrorists.\n",
      "\n",
      "4. **Proliferation of Weapons of Mass Destruction (WMDs)**: The development, acquisition, or potential use of WMDs (nuclear, chemical, biological) by states or non-state actors represents a high-risk security escalation. The mere possession of such weapons can destabilize entire regions and prompt preemptive strikes.\n",
      "\n",
      "### Regional Implications\n",
      "\n",
      "The implications of security escalations vary by region, depending on the geopolitical context, the presence of alliances, and the capability of regional and international organizations to manage conflicts.\n",
      "\n",
      "- **Middle East**: This region is particularly prone to security escalations due to ongoing conflicts (e.g., Yemen, Syria), the Israeli-Palestinian issue, and the rivalry between Iran and Saudi Arabia. Escalations here can affect global energy markets and draw in external powers.\n",
      "\n",
      "- **East Asia**: The South China Sea disputes and the Korean Peninsula are focal points for potential security escalations. China's military expansion and North Korea's nuclear program are key factors. The region's economic importance means that any escalation could have significant global implications.\n",
      "\n",
      "- **Eastern Europe**: The conflict in Ukraine and the broader tensions between Russia and NATO represent a significant security escalation risk. Any further escalation could lead to a wider conflict involving more European countries and the United States.\n",
      "\n",
      "- **South Asia**: The rivalry between India and Pakistan, both nuclear-armed states, poses a risk of security escalation, particularly over Kashmir. An escalation here could have catastrophic consequences due to the nuclear dimension.\n",
      "\n",
      "### Managing and Mitigating Security Escalations\n",
      "\n",
      "To manage and mitigate these risks, several strategies can be employed:\n",
      "\n",
      "- **Diplomacy and Dialogue**: Encouraging open communication between potential adversaries can help prevent misunderstandings and reduce tensions.\n",
      "- **International Law and Norms**: Strengthening international law and norms, especially regarding the use of force and weapons of mass destruction, can provide a framework for managing conflicts.\n",
      "- **Conflict Resolution Mechanisms**: Establishing or strengthening regional and international mechanisms for conflict resolution can help to de-escalate tensions before they lead to wider conflict.\n",
      "- **Arms Control and Disarmament**: Efforts to control and reduce armaments, including WMDs, can decrease the risk of security escalations.\n",
      "\n",
      "In conclusion, high-risk security escalations can arise from various sources and have profound regional and global implications. Understanding these risks and employing proactive strategies to manage and mitigate them are crucial for maintaining international peace and security.\n",
      "\n",
      "================================================================================\n",
      "GRAPHRAG ANSWER:\n",
      "================================================================================\n",
      "Based on the provided context, I will provide a comprehensive analysis to identify high-risk security escalations and their regional implications.\n",
      "\n",
      "**Introduction**\n",
      "\n",
      "Security escalations can have far-reaching consequences, affecting not only the countries directly involved but also the surrounding regions. Identifying high-risk security escalations is crucial to understanding the potential implications and developing strategies to mitigate or prevent them. This analysis will examine various security escalations, their regional implications, and the potential consequences of these events.\n",
      "\n",
      "**High-Risk Security Escalations**\n",
      "\n",
      "1. **Nuclear Proliferation**: The proliferation of nuclear weapons in regions such as the Middle East, North Korea, and South Asia poses a significant security risk. The acquisition of nuclear capabilities by countries with a history of conflict or instability could lead to a nuclear arms race, increasing the likelihood of nuclear war.\n",
      "2. **Cyber Attacks**: The increasing frequency and sophistication of cyber attacks pose a significant threat to national security, economies, and critical infrastructure. A major cyber attack on a country's critical infrastructure could have devastating consequences, including disruption of essential services, economic loss, and loss of life.\n",
      "3. **Terrorism**: The rise of terrorist organizations, such as ISIS and Al-Qaeda, has led to an increase in terrorist attacks globally. These attacks can have significant regional implications, including the destabilization of governments, displacement of populations, and disruption of economic activity.\n",
      "4. **Border Disputes**: Border disputes between countries, such as the India-Pakistan dispute over Kashmir, can lead to security escalations, including military conflicts and terrorist attacks. These disputes can have significant regional implications, including the displacement of populations and disruption of economic activity.\n",
      "5. **Maritime Disputes**: Maritime disputes, such as the South China Sea dispute, can lead to security escalations, including naval conflicts and trade disruptions. These disputes can have significant regional implications, including the disruption of global trade and the destabilization of regional economies.\n",
      "\n",
      "**Regional Implications**\n",
      "\n",
      "1. **Middle East**: The Middle East is a region of significant security concern, with ongoing conflicts in Syria, Yemen, and Libya. The proliferation of nuclear weapons in the region, combined with the rise of terrorist organizations, poses a significant security risk.\n",
      "2. **Asia-Pacific**: The Asia-Pacific region is home to several security hotspots, including the South China Sea, North Korea, and the India-Pakistan border. The region is also vulnerable to cyber attacks, with several countries, including China and North Korea, accused of engaging in state-sponsored cyber attacks.\n",
      "3. **Europe**: Europe is facing several security challenges, including the rise of terrorist organizations, cyber attacks, and the migration crisis. The region is also vulnerable to security escalations in the Middle East and North Africa, with several European countries involved in military interventions in these regions.\n",
      "4. **Africa**: Africa is a region of significant security concern, with ongoing conflicts in several countries, including the Democratic Republic of Congo, South Sudan, and Somalia. The region is also vulnerable to terrorist attacks, with several terrorist organizations, including Al-Shabaab and Boko Haram, operating in the region.\n",
      "\n",
      "**Conclusion**\n",
      "\n",
      "Security escalations can have significant regional implications, including the destabilization of governments, displacement of populations, and disruption of economic activity. Identifying high-risk security escalations is crucial to understanding the potential implications and developing strategies to mitigate or prevent them. The analysis highlights the need for a comprehensive approach to security, including diplomacy, military cooperation, and economic development, to address the root causes of security escalations and prevent their regional implications.\n",
      "\n",
      "**Recommendations**\n",
      "\n",
      "1. **Diplomacy**: Diplomacy is essential to preventing security escalations. Countries should engage in diplomatic efforts to resolve conflicts and disputes peacefully.\n",
      "2. **Military Cooperation**: Military cooperation between countries can help to prevent security escalations. Countries should engage in military cooperation, including joint exercises and training, to build trust and improve communication.\n",
      "3. **Economic Development**: Economic development is crucial to addressing the root causes of security escalations. Countries should invest in economic development, including education, healthcare, and infrastructure, to reduce poverty and inequality.\n",
      "4. **Cyber Security**: Cyber security is essential to preventing cyber attacks. Countries should invest in cyber security measures, including firewalls, intrusion detection systems, and encryption, to protect their critical infrastructure.\n",
      "5. **International Cooperation**: International cooperation is crucial to addressing security escalations. Countries should engage in international cooperation, including through organizations such as the United Nations, to address global security challenges and prevent regional implications.\n",
      "\n",
      "================================================================================\n",
      "COMPARISON:\n",
      "================================================================================\n",
      "Vector RAG: 583 words - Based on semantic similarity only\n",
      "GraphRAG: 700 words - Based on semantic similarity + multi-hop graph connections\n",
      "================================================================================\n"
     ]
    }
   ],
   "source": [
    "from semantica.semantic_extract.providers import create_provider\n",
    "\n",
    "# Define the comparison query\n",
    "comparison_query = \"Identify high-risk security escalations and their regional implications.\"\n",
    "\n",
    "# Initialize LLM provider for answer generation\n",
    "llm_provider = create_provider(\"groq\", model=\"llama-3.3-70b-versatile\")\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vector RAG Answer\n",
    "Generate and display the answer using Vector RAG (semantic similarity only).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================================================================\n",
      "VECTOR RAG ANSWER\n",
      "================================================================================\n",
      "Given the lack of specific context, I'll provide a general analysis of high-risk security escalations and their potential regional implications. This will involve discussing various types of security threats, their characteristics, and how they might impact different regions.\n",
      "\n",
      "### Introduction to High-Risk Security Escalations\n",
      "\n",
      "High-risk security escalations refer to situations or events that significantly increase the likelihood of conflict, violence, or instability within a region or globally. These escalations can arise from various sources, including political tensions, military build-ups, cyberattacks, terrorism, and environmental disasters. Understanding and identifying these risks is crucial for preventing or mitigating their impacts.\n",
      "\n",
      "### Types of High-Risk Security Escalations\n",
      "\n",
      "1. **Military Escalations**: These involve an increase in military presence, actions, or rhetoric that could lead to conflict. Examples include border disputes, naval standoffs, or the deployment of advanced weaponry to sensitive areas.\n",
      "\n",
      "2. **Cybersecurity Threats**: Escalations in cybersecurity threats, such as large-scale hacking, ransomware attacks, or breaches of critical infrastructure, can have significant economic and social impacts.\n",
      "\n",
      "3. **Terrorism**: An increase in terrorist activities or the emergence of new terrorist groups can destabilize regions and pose global security risks.\n",
      "\n",
      "4. **Environmental and Health Crises**: Natural disasters, pandemics, or environmental degradation can escalate into security issues if they lead to mass migrations, resource scarcity, or social unrest.\n",
      "\n",
      "### Regional Implications\n",
      "\n",
      "The implications of these security escalations vary by region, depending on the local political, economic, and social context.\n",
      "\n",
      "- **Middle East and North Africa**: This region is particularly vulnerable to military escalations, terrorism, and cybersecurity threats due to ongoing conflicts and geopolitical rivalries. Escalations here can affect global energy markets and have significant implications for international security.\n",
      "\n",
      "- **Asia-Pacific**: The region is a hotspot for military build-ups and territorial disputes, particularly in the South China Sea. Cybersecurity threats are also a major concern, given the region's technological advancements and economic interdependence.\n",
      "\n",
      "- **Europe**: Europe faces challenges from terrorism, cybersecurity threats, and the impact of neighboring conflicts, such as the situation in Ukraine. The region's economic integration and open borders make it vulnerable to the spillover effects of instability.\n",
      "\n",
      "- **Americas**: The region is affected by issues like drug trafficking, terrorism, and cybersecurity threats. The political and economic stability of the United States, as a global leader, has implications for security escalations worldwide.\n",
      "\n",
      "### Analysis and Recommendations\n",
      "\n",
      "To address high-risk security escalations, it's essential to:\n",
      "\n",
      "- **Enhance International Cooperation**: Diplomatic efforts and international agreements can help prevent escalations by promoting dialogue and establishing clear norms and consequences for aggressive actions.\n",
      "\n",
      "- **Invest in Conflict Prevention and Resolution Mechanisms**: Early warning systems, mediation, and peacekeeping operations can mitigate the effects of escalations.\n",
      "\n",
      "- **Develop Resilience**: Investing in cybersecurity, disaster preparedness, and social cohesion can help regions withstand and recover from security escalations.\n",
      "\n",
      "- **Promote Economic Stability and Development**: Addressing poverty, inequality, and environmental degradation can reduce the underlying causes of instability and conflict.\n",
      "\n",
      "In conclusion, high-risk security escalations pose significant threats to regional and global stability. Understanding these risks and their potential implications is crucial for developing effective strategies to prevent, mitigate, or resolve them. A comprehensive approach that combines diplomacy, resilience, and development is necessary to address the complex and interconnected nature of modern security challenges.\n"
     ]
    }
   ],
   "source": [
    "# Vector RAG: Retrieve context and generate answer\n",
    "v_res = vs.search(comparison_query, limit=5)\n",
    "vector_context = \"\\n\\n\".join([\n",
    "    r.get('metadata', {}).get('content', 'No content') \n",
    "    for r in v_res\n",
    "])\n",
    "\n",
    "vector_prompt = f\"\"\"Based on the following context retrieved using semantic similarity search, answer the query comprehensively.\n",
    "\n",
    "Context:\n",
    "{vector_context}\n",
    "\n",
    "Query: {comparison_query}\n",
    "\n",
    "Provide a detailed analysis:\"\"\"\n",
    "\n",
    "try:\n",
    "    vector_answer = llm_provider.generate(vector_prompt, temperature=0.3)\n",
    "except Exception as e:\n",
    "    vector_answer = f\"Error generating answer: {e}\"\n",
    "\n",
    "print(\"=\" * 80)\n",
    "print(\"VECTOR RAG ANSWER\")\n",
    "print(\"=\" * 80)\n",
    "print(vector_answer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GraphRAG Answer\n",
    "\n",
    "Generate and display the answer using GraphRAG (hybrid retrieval with graph traversal).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================================================================\n",
      "GRAPHRAG ANSWER\n",
      "================================================================================\n",
      "Based on the provided context, I will provide a comprehensive analysis to identify high-risk security escalations and their regional implications.\n",
      "\n",
      "**Introduction**\n",
      "\n",
      "Security escalations can have far-reaching consequences, affecting not only the immediate region but also neighboring countries and the global community. Identifying high-risk security escalations is crucial to mitigate potential threats and prevent the escalation of conflicts. This analysis will examine various regions and identify potential security escalations, their implications, and the potential consequences for regional and global stability.\n",
      "\n",
      "**Regional Security Escalations**\n",
      "\n",
      "1. **Middle East**: The Middle East is a volatile region, with several ongoing conflicts and tensions between countries. The conflict between Israel and Palestine, the Syrian Civil War, and the Yemen Civil War are some of the most significant security concerns. The region is also home to several terrorist organizations, including ISIS and Hezbollah, which pose a threat to regional and global security.\n",
      "2. **Eastern Europe**: The conflict between Russia and Ukraine, particularly in the Donbas region, is a significant security concern. The annexation of Crimea by Russia in 2014 and the ongoing support for separatist groups in eastern Ukraine have raised tensions between Russia and the West. The region is also home to several NATO member countries, which could be drawn into a conflict.\n",
      "3. **South Asia**: The region is home to several nuclear-armed countries, including India, Pakistan, and China. The Kashmir conflict between India and Pakistan is a long-standing issue, and tensions between the two countries remain high. The region is also home to several terrorist organizations, including Lashkar-e-Taiba and Jaish-e-Mohammed.\n",
      "4. **East Asia**: The region is home to several countries with competing interests, including China, Japan, and South Korea. The South China Sea dispute, which involves several countries, including China, the Philippines, and Vietnam, is a significant security concern. The region is also home to North Korea, which has developed nuclear capabilities and poses a threat to regional and global security.\n",
      "\n",
      "**High-Risk Security Escalations**\n",
      "\n",
      "1. **Israel-Palestine Conflict**: The conflict between Israel and Palestine is a long-standing issue, with periodic outbreaks of violence. The conflict has the potential to escalate into a full-scale war, drawing in neighboring countries and affecting regional stability.\n",
      "2. **Russia-Ukraine Conflict**: The conflict between Russia and Ukraine has the potential to escalate into a full-scale war, drawing in NATO member countries and affecting regional stability.\n",
      "3. **India-Pakistan Conflict**: The conflict between India and Pakistan, particularly over Kashmir, has the potential to escalate into a full-scale war, drawing in other countries and affecting regional stability.\n",
      "4. **North Korea's Nuclear Program**: North Korea's development of nuclear capabilities poses a significant threat to regional and global security. The country's actions have the potential to escalate into a full-scale conflict, drawing in neighboring countries and affecting regional stability.\n",
      "\n",
      "**Regional Implications**\n",
      "\n",
      "1. **Middle East**: A security escalation in the Middle East could have significant implications for regional stability, including the potential for a full-scale war between Israel and Palestine, or between Iran and its neighbors.\n",
      "2. **Eastern Europe**: A security escalation in Eastern Europe could have significant implications for regional stability, including the potential for a full-scale war between Russia and Ukraine, or between Russia and NATO member countries.\n",
      "3. **South Asia**: A security escalation in South Asia could have significant implications for regional stability, including the potential for a full-scale war between India and Pakistan, or between India and China.\n",
      "4. **East Asia**: A security escalation in East Asia could have significant implications for regional stability, including the potential for a full-scale war between China and its neighbors, or between North Korea and South Korea.\n",
      "\n",
      "**Global Implications**\n",
      "\n",
      "1. **Economic Consequences**: A security escalation in any of the regions could have significant economic consequences, including the potential for trade disruptions, sanctions, and economic instability.\n",
      "2. **Humanitarian Consequences**: A security escalation in any of the regions could have significant humanitarian consequences, including the potential for civilian casualties, displacement, and human rights abuses.\n",
      "3. **Global Stability**: A security escalation in any of the regions could have significant implications for global stability, including the potential for a wider conflict, drawing in other countries and affecting global security.\n",
      "\n",
      "**Conclusion**\n",
      "\n",
      "In conclusion, identifying high-risk security escalations and their regional implications is crucial to mitigate potential threats and prevent the escalation of conflicts. The regions of the Middle East, Eastern Europe, South Asia, and East Asia are all potential hotspots for security escalations, with significant implications for regional and global stability. It is essential to monitor these regions closely and take proactive measures to prevent security escalations and promote regional stability.\n"
     ]
    }
   ],
   "source": [
    "# GraphRAG: Retrieve context and generate answer\n",
    "graph_res = ctx.retrieve(\n",
    "    comparison_query, \n",
    "    max_results=5, \n",
    "    use_graph=True, \n",
    "    expand_graph=True,\n",
    "    include_entities=True,\n",
    "    include_relationships=True\n",
    ")\n",
    "\n",
    "graph_context = \"\\n\\n\".join([r.get('content', '') for r in graph_res])\n",
    "\n",
    "graph_prompt = f\"\"\"Based on the following context retrieved using hybrid search (vector similarity + knowledge graph traversal), answer the query comprehensively.\n",
    "\n",
    "Context:\n",
    "{graph_context}\n",
    "\n",
    "Query: {comparison_query}\n",
    "\n",
    "Provide a detailed analysis:\"\"\"\n",
    "\n",
    "try:\n",
    "    graph_answer = llm_provider.generate(graph_prompt, temperature=0.3)\n",
    "except Exception as e:\n",
    "    graph_answer = f\"Error generating answer: {e}\"\n",
    "\n",
    "print(\"=\" * 80)\n",
    "print(\"GRAPHRAG ANSWER\")\n",
    "print(\"=\" * 80)\n",
    "print(graph_answer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparison Summary\n",
    "\n",
    "Compare both answers side by side.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================================================================\n",
      "COMPARISON\n",
      "================================================================================\n",
      "✓ GraphRAG generated a more comprehensive answer\n",
      "\n",
      "GraphRAG's advantage: Multi-hop reasoning discovers connections\n",
      "that Vector RAG cannot see, leading to more complete answers.\n",
      "================================================================================\n"
     ]
    }
   ],
   "source": [
    "# Comparison\n",
    "vector_words = len(vector_answer.split()) if isinstance(vector_answer, str) and not vector_answer.startswith(\"Error\") else 0\n",
    "graph_words = len(graph_answer.split()) if isinstance(graph_answer, str) and not graph_answer.startswith(\"Error\") else 0\n",
    "\n",
    "print(\"=\" * 80)\n",
    "print(\"COMPARISON\")\n",
    "print(\"=\" * 80)\n",
    "if graph_words > vector_words * 1.2:\n",
    "    print(\"✓ GraphRAG generated a more comprehensive answer\")\n",
    "elif graph_words < vector_words * 0.8:\n",
    "    print(\"⚠ Vector RAG generated a longer answer\")\n",
    "else:\n",
    "    print(\"→ Both approaches generated answers of similar length\")\n",
    "    \n",
    "print(\"\\nGraphRAG's advantage: Multi-hop reasoning discovers connections\")\n",
    "print(\"that Vector RAG cannot see, leading to more complete answers.\")\n",
    "print(\"=\" * 80)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Summary\n",
    "\n",
    "This notebook demonstrated a comprehensive comparison between Standard Vector RAG and GraphRAG.\n",
    "\n",
    "### Key Findings\n",
    "\n",
    "1. **Vector RAG Limitations**:\n",
    "   - Cannot follow entity relationships\n",
    "   - Misses connections between related facts\n",
    "   - Limited context expansion\n",
    "   - Fails on semantic gaps\n",
    "\n",
    "2. **GraphRAG Advantages**:\n",
    "   - Multi-hop reasoning across entities\n",
    "   - Relationship-aware retrieval\n",
    "   - Automatic context expansion\n",
    "   - Bridges semantic gaps\n",
    "\n",
    "3. **When to Use Each**:\n",
    "   - **Vector RAG**: Simple queries, single-document retrieval, fast responses\n",
    "   - **GraphRAG**: Complex queries, multi-hop reasoning, relationship-aware search\n",
    "\n",
    "### Next Steps\n",
    "\n",
    "- Experiment with different queries to explore the differences\n",
    "- Adjust hybrid_alpha to balance vector vs graph retrieval\n",
    "- Explore advanced features like conflict detection and SPARQL queries\n",
    "- Integrate with production systems for real-world applications\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "### Key Findings\n",
    "\n",
    "1. **Vector RAG Limitations**:\n",
    "   - Only finds semantically similar text chunks\n",
    "   - Cannot traverse relationships between entities\n",
    "   - Fails when facts are scattered across documents\n",
    "   - No multi-hop reasoning capability\n",
    "\n",
    "2. **GraphRAG Advantages**:\n",
    "   - Combines vector search with graph traversal\n",
    "   - Follows relationships to find connected information\n",
    "   - Bridges semantic gaps through graph structure\n",
    "   - Enables multi-hop reasoning (2+ hops)\n",
    "   - Provides \"Chain of Evidence\" for complex queries\n",
    "\n",
    "3. **When to Use Each**:\n",
    "   - **Vector RAG**: Simple fact retrieval, single-document queries, when relationships are not important\n",
    "   - **GraphRAG**: Complex multi-hop queries, relationship-heavy domains, when context expansion is needed\n",
    "\n",
    "### Conclusion\n",
    "\n",
    "GraphRAG creates a **\"Chain of Evidence\"** that Vector RAG cannot see by:\n",
    "- Traversing the knowledge graph structure\n",
    "- Following entity relationships across documents\n",
    "- Expanding context through graph connections\n",
    "- Enabling multi-hop reasoning for complex questions\n",
    "\n",
    "This makes GraphRAG particularly powerful for intelligence analysis, research, and any domain where understanding relationships is crucial.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
