{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/01_Advanced_Extraction.ipynb)\n",
    "\n",
    "# Advanced Extraction\n",
    "\n",
    "## Overview\n",
    "\n",
    "This notebook demonstrates advanced semantic extraction using EventDetector, CoreferenceResolver, TripletExtractor, SemanticAnalyzer, SemanticNetworkExtractor, LLMEnhancer, and ExtractionValidator.\n",
    "\n",
    "\n",
    "**Documentation**: [API Reference](https://semantica.readthedocs.io/reference/semantic_extract/)\n",
    "\n",
    "### Learning Objectives\n",
    "\n",
    "- Use EventDetector to detect events\n",
    "- Use CoreferenceResolver to resolve coreferences\n",
    "- Use TripletExtractor to extract RDF triplets\n",
    "- Use SemanticAnalyzer for semantic analysis\n",
    "- Use SemanticNetworkExtractor to extract semantic networks\n",
    "- Use LLMEnhancer for LLM-based enhancement\n",
    "- Use ExtractionValidator to validate extractions\n",
    "\n",
    "## Installation\n",
    "\n",
    "Install Semantica from PyPI:\n",
    "\n",
    "```bash\n",
    "pip install semantica\n",
    "# Or with all optional dependencies:\n",
    "pip install semantica[all]\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Workflow: Event Detection → Coreference Resolution → Triplet Extraction → Semantic Analysis → Network Extraction → LLM Enhancement → Validation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
     ]
    }
   ],
   "source": [
    "!pip install -q semantica"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>EventDetector</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>CoreferenceResolver</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>TripletExtractor</td><td>-</td><td>1.32s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>NERExtractor</td><td>-</td><td>0.51s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>RelationExtractor</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>SemanticNetworkExtractor</td><td>-</td><td>1.21s</td></tr></table></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Detected 1 events\n",
      "  Event: founded - founded\n"
     ]
    }
   ],
   "source": [
    "from semantica.semantic_extract import (\n",
    "    EventDetector, CoreferenceResolver, TripletExtractor,\n",
    "    SemanticAnalyzer, SemanticNetworkExtractor, LLMEnhancer, ExtractionValidator\n",
    ")\n",
    "\n",
    "text = \"Apple Inc. was founded by Steve Jobs in 1976. The company is now led by Tim Cook.\"\n",
    "\n",
    "event_detector = EventDetector()\n",
    "events = event_detector.detect_events(text)\n",
    "\n",
    "print(f\"Detected {len(events)} events\")\n",
    "for event in events[:3]:\n",
    "    print(f\"  Event: {event.event_type} - {event.text[:50]}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Coreference Resolution\n",
    "\n",
    "Resolve coreferences in text.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resolved 0 coreference chains\n"
     ]
    }
   ],
   "source": [
    "coreference_resolver = CoreferenceResolver()\n",
    "\n",
    "coreferences = coreference_resolver.resolve(text)\n",
    "\n",
    "print(f\"Resolved {len(coreferences)} coreference chains\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Triplet Extraction\n",
    "\n",
    "Extract RDF triplets.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DEBUG: Entity map keys: ['apple inc.', 'steve jobs', '1976', 'tim cook']\n",
      "DEBUG: Match found! Subject='Apple Inc.', Object='Steve Jobs'\n",
      "DEBUG: Subject Entity found: True, Object Entity found: True\n",
      "DEBUG: Match found! Subject='Steve Jobs', Object='1976'\n",
      "DEBUG: Subject Entity found: True, Object Entity found: True\n",
      "Extracted 2 triplets\n",
      "  (Apple Inc., founded_by, Steve Jobs)\n",
      "  (Steve Jobs, located_in, 1976)\n"
     ]
    }
   ],
   "source": [
    "triplet_extractor = TripletExtractor()\n",
    "\n",
    "triplets = triplet_extractor.extract_triplets(text)\n",
    "\n",
    "print(f\"Extracted {len(triplets)} triplets\")\n",
    "for triplet in triplets[:3]:\n",
    "    print(f\"  ({triplet.get('subject', '')}, {triplet.get('predicate', '')}, {triplet.get('object', '')})\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Semantic Analysis\n",
    "\n",
    "Perform semantic analysis.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Analyzed semantic roles: 6\n"
     ]
    }
   ],
   "source": [
    "semantic_analyzer = SemanticAnalyzer()\n",
    "\n",
    "semantic_roles = semantic_analyzer.analyze_semantic_roles(text)\n",
    "\n",
    "print(f\"Analyzed semantic roles: {len(semantic_roles)}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Semantic Network Extraction\n",
    "\n",
    "Extract semantic networks.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DEBUG: Entity map keys: ['apple inc.', 'steve jobs', '1976', 'tim cook']\n",
      "DEBUG: Match found! Subject='Apple Inc.', Object='Steve Jobs'\n",
      "DEBUG: Subject Entity found: True, Object Entity found: True\n",
      "DEBUG: Match found! Subject='Steve Jobs', Object='1976'\n",
      "DEBUG: Subject Entity found: True, Object Entity found: True\n",
      "Extracted semantic network with 4 nodes\n",
      "Edges: 2\n"
     ]
    }
   ],
   "source": [
    "semantic_network_extractor = SemanticNetworkExtractor()\n",
    "\n",
    "semantic_network = semantic_network_extractor.extract_network(text)\n",
    "\n",
    "print(f\"Extracted semantic network with {len(semantic_network.get('nodes', []))} nodes\")\n",
    "print(f\"Edges: {len(semantic_network.get('edges', []))}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: LLM Enhancement\n",
    "\n",
    "Enhance extractions using LLM.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'LLMEnhancer' object has no attribute 'enhance_extractions'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[7], line 3\u001b[0m\n\u001b[0;32m      1\u001b[0m llm_enhancer \u001b[38;5;241m=\u001b[39m LLMEnhancer()\n\u001b[1;32m----> 3\u001b[0m enhanced_extractions \u001b[38;5;241m=\u001b[39m \u001b[43mllm_enhancer\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43menhance_extractions\u001b[49m(events, text)\n\u001b[0;32m      5\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEnhanced \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mlen\u001b[39m(enhanced_extractions)\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m extractions\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "\u001b[1;31mAttributeError\u001b[0m: 'LLMEnhancer' object has no attribute 'enhance_extractions'"
     ]
    }
   ],
   "source": [
    "llm_enhancer = LLMEnhancer()\n",
    "\n",
    "enhanced_extractions = llm_enhancer.enhance_extractions(events, text)\n",
    "\n",
    "print(f\"Enhanced {len(enhanced_extractions)} extractions\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 7: Extraction Validation\n",
    "\n",
    "Validate extractions.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "extraction_validator = ExtractionValidator()\n",
    "\n",
    "validation_result = extraction_validator.validate(events, text)\n",
    "\n",
    "print(f\"Extraction validation:\")\n",
    "print(f\"  Valid: {validation_result.valid}\")\n",
    "print(f\"  Confidence: {validation_result.confidence:.3f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "You've learned advanced extraction capabilities:\n",
    "\n",
    "- **EventDetector**: Event detection and classification\n",
    "- **CoreferenceResolver**: Coreference resolution\n",
    "- **TripletExtractor**: RDF triplet extraction\n",
    "- **SemanticAnalyzer**: Semantic analysis and role labeling\n",
    "- **SemanticNetworkExtractor**: Semantic network extraction\n",
    "- **LLMEnhancer**: LLM-based extraction enhancement\n",
    "- **ExtractionValidator**: Extraction validation\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
