{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c21e9c8d",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/19_Context_Module.ipynb)\n",
    "\n",
    "# Context Module — Practical Guide\n",
    "\n",
    "Semantica’s `context` module is the layer that makes an agent “stateful”. It combines:\n",
    "\n",
    "- **Memory** (short-term + long-term) via `AgentMemory`\n",
    "- **Graph context** via `ContextGraph`\n",
    "- **Hybrid retrieval** (vector + memory + graph) via `ContextRetriever`\n",
    "- **High-level UX** via `AgentContext` (recommended entry point)\n",
    "- **Entity linking** via `EntityLinker`\n",
    "- **Extensibility + config** via `registry` and `config`\n",
    "\n",
    "This notebook focuses on small, runnable examples and keeps imports scoped to each cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "257bad40",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
     ]
    }
   ],
   "source": [
    "!pip install -q semantica"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a48e0f10",
   "metadata": {},
   "source": [
    "## 1) Vector store (for long-term memory)\n",
    "\n",
    "The `VectorStore` can generate embeddings via its internal embedder. If no embedder is available in your environment, it falls back to random vectors so the API stays usable for demos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8c845a94",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\sentence_transformers\\cross_encoder\\CrossEncoder.py:13: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)\n",
      "  from tqdm.autonotebook import tqdm, trange\n",
      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n",
      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('inmemory', 384)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.vector_store import VectorStore\n",
    "\n",
    "vs = VectorStore(backend=\"inmemory\", dimension=384)\n",
    "\n",
    "if getattr(vs, \"embedder\", None) and hasattr(vs.embedder, \"set_text_model\"):\n",
    "    vs.embedder.set_text_model(method=\"fastembed\", model_name=\"BAAI/bge-small-en-v1.5\")\n",
    "\n",
    "vs.backend, vs.dimension"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1b1ba34",
   "metadata": {},
   "source": [
    "## 2) Quick start with `AgentContext` (recommended)\n",
    "\n",
    "`AgentContext` is the user-friendly interface that ties memory, vector store, and graph together. If you pass a `ContextGraph`, the system can do GraphRAG-style retrieval."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f4d788b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'retention_days': 30,\n",
       " 'max_memories': 10000,\n",
       " 'use_graph_expansion': True,\n",
       " 'max_expansion_hops': 2,\n",
       " 'hybrid_alpha': 0.5}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.context import AgentContext, ContextGraph\n",
    "\n",
    "kg = ContextGraph()\n",
    "context = AgentContext(vector_store=vs, knowledge_graph=kg)\n",
    "\n",
    "context.config"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "638bdbc8",
   "metadata": {},
   "source": [
    "## 3) Store and retrieve memory\n",
    "\n",
    "A single string is treated as a memory item. You can attach `conversation_id` and `user_id` through metadata-friendly parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5d65eb00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is processing</td><td>🔗 context</td><td>AgentMemory</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is embedding</td><td>💾 embeddings</td><td>TextEmbedder</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>VectorStore</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is processing</td><td>🔗 context</td><td>ContextRetriever</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is processing</td><td>🔗 context</td><td>EntityLinker</td><td>-</td><td>0.00s</td></tr></table></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "{'id': None,\n",
       " 'content': 'User prefers short answers about Python.',\n",
       " 'timestamp': '2025-12-18T20:46:17.265703',\n",
       " 'metadata': {'type': 'preference',\n",
       "  'conversation_id': 'conv_1',\n",
       "  'user_id': 'user_1'}}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memory_id = context.store(\n",
    "    \"User prefers short answers about Python.\",\n",
    "    conversation_id=\"conv_1\",\n",
    "    user_id=\"user_1\",\n",
    "    metadata={\"type\": \"preference\"},\n",
    ")\n",
    "\n",
    "context.get_memory(memory_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "3c1be718",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Mohd Kaif\\semantica\\semantica\\vector_store\\vector_store.py:480: RuntimeWarning: invalid value encountered in divide\n",
      "  similarities = np.dot(vectors, query_vector) / (vector_norms * query_norm)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'content': 'User prefers short answers about Python.',\n",
       "  'score': 1.0,\n",
       "  'source': 'short_term',\n",
       "  'metadata': {'type': 'preference',\n",
       "   'conversation_id': 'conv_1',\n",
       "   'user_id': 'user_1'},\n",
       "  'related_entities': []}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "context.store(\n",
    "    \"User is working on Semantica context module examples.\",\n",
    "    conversation_id=\"conv_1\",\n",
    "    user_id=\"user_1\",\n",
    "    metadata={\"type\": \"note\"},\n",
    ")\n",
    "\n",
    "context.retrieve(\"Python answers\", max_results=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "485acf33",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "context.conversation(\"conv_1\", max_items=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e43cddd",
   "metadata": {},
   "source": [
    "## 4) Export, save, load\n",
    "\n",
    "`AgentContext` includes simple persistence helpers. This example uses a temporary directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a264ef4d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'{\\n  \"exported_at\": \"2025-12-18T20:46:27.455602\",\\n  \"count\": 2,\\n  \"memories\": [\\n    {\\n      \"memory_id\": \"mem_5c7dba9a7373\",\\n      \"content\": \"User prefers short answers about Python.\",\\n      \"timestamp\": \"2025-12-18T20:46:17.265703\",\\n      \"metadata\": {\\n        \"type\": \"preference\",\\n        \"convers'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "export_json = context.export(conversation_id=\"conv_1\", format=\"json\")\n",
    "export_json[:300]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b62d1859",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'conversation_id': 'conv_1',\n",
       " 'message_count': 0,\n",
       " 'first_message': None,\n",
       " 'last_message': None}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tempfile\n",
    "\n",
    "with tempfile.TemporaryDirectory() as d:\n",
    "    context.save(d)\n",
    "    context.load(d)\n",
    "\n",
    "context.conversation_summary(\"conv_1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b8bf553",
   "metadata": {},
   "source": [
    "## 5) Store documents and build a context graph\n",
    "\n",
    "If you store a list, `AgentContext.store(...)` treats it as documents. To keep this notebook lightweight and deterministic, we pass pre-extracted entities and relationships per document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "72930ae7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'stored_count': 2,\n",
       " 'memory_ids': ['mem_b467da01b18a', 'mem_e3dbd310355d'],\n",
       " 'graph_nodes': 0,\n",
       " 'graph_edges': 0}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents = [\n",
    "    {\n",
    "        \"id\": \"doc_1\",\n",
    "        \"content\": \"Python is used for machine learning.\",\n",
    "        \"metadata\": {\"source\": \"docs\"},\n",
    "        \"entities\": [\n",
    "            {\"id\": \"e_python\", \"text\": \"Python\", \"type\": \"PROGRAMMING_LANGUAGE\"},\n",
    "            {\"id\": \"e_ml\", \"text\": \"Machine Learning\", \"type\": \"CONCEPT\"},\n",
    "        ],\n",
    "        \"relationships\": [\n",
    "            {\n",
    "                \"source_id\": \"e_python\",\n",
    "                \"target_id\": \"e_ml\",\n",
    "                \"type\": \"used_for\",\n",
    "                \"confidence\": 0.9,\n",
    "            }\n",
    "        ],\n",
    "    },\n",
    "    {\n",
    "        \"id\": \"doc_2\",\n",
    "        \"content\": \"PyTorch is a machine learning framework.\",\n",
    "        \"metadata\": {\"source\": \"docs\"},\n",
    "        \"entities\": [\n",
    "            {\"id\": \"e_pytorch\", \"text\": \"PyTorch\", \"type\": \"FRAMEWORK\"},\n",
    "            {\"id\": \"e_ml\", \"text\": \"Machine Learning\", \"type\": \"CONCEPT\"},\n",
    "        ],\n",
    "        \"relationships\": [\n",
    "            {\n",
    "                \"source_id\": \"e_pytorch\",\n",
    "                \"target_id\": \"e_ml\",\n",
    "                \"type\": \"implements\",\n",
    "                \"confidence\": 0.95,\n",
    "            }\n",
    "        ],\n",
    "    },\n",
    "]\n",
    "\n",
    "stats = context.store(\n",
    "    documents,\n",
    "    extract_entities=False,\n",
    "    extract_relationships=False,\n",
    "    link_entities=True,\n",
    ")\n",
    "\n",
    "stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24f8dba8",
   "metadata": {},
   "outputs": [],
   "source": [
    "kg.stats()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4671f2a",
   "metadata": {},
   "source": [
    "## 6) Explore the graph with `ContextGraph`\n",
    "\n",
    "The graph supports keyword querying and neighbor expansion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "df2e5fcd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'node': {'id': 'e_ml',\n",
       "   'type': 'CONCEPT',\n",
       "   'properties': {'content': 'Machine Learning',\n",
       "    'id': 'e_ml',\n",
       "    'text': 'Machine Learning',\n",
       "    'type': 'CONCEPT'}},\n",
       "  'score': 1.0,\n",
       "  'content': 'Machine Learning'}]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kg.query(\"machine learning\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0836feeb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'id': 'e_ml',\n",
       "  'type': 'CONCEPT',\n",
       "  'content': 'Machine Learning',\n",
       "  'relationship': 'used_for',\n",
       "  'weight': 0.9,\n",
       "  'hop': 1}]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kg.get_neighbors(\"e_python\", hops=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1be1adf1",
   "metadata": {},
   "source": [
    "## 7) Entity linking with `EntityLinker`\n",
    "\n",
    "`EntityLinker` assigns stable URIs and can link related or duplicate entities across sources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "24b011c0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('e1', 'https://semantica.dev/entity/python#programming_language', 1),\n",
       " ('e2', 'https://semantica.dev/entity/pytorch#framework', 0)]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.context import EntityLinker\n",
    "\n",
    "linker = EntityLinker(knowledge_graph={\"entities\": [{\"id\": \"e_py\", \"text\": \"Python\", \"type\": \"PROGRAMMING_LANGUAGE\"}]})\n",
    "\n",
    "entities = [\n",
    "    {\"id\": \"e1\", \"text\": \"Python\", \"type\": \"PROGRAMMING_LANGUAGE\"},\n",
    "    {\"id\": \"e2\", \"text\": \"PyTorch\", \"type\": \"FRAMEWORK\"},\n",
    "]\n",
    "\n",
    "linked = linker.link(\"Python and PyTorch\", entities=entities)\n",
    "[(e.entity_id, e.uri, len(e.linked_entities)) for e in linked]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a282de3a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[EntityLink(source_entity_id='e1', target_entity_id='e_py', link_type='same_as', confidence=1.0, source=None, metadata={'similarity': 1.0}),\n",
       " EntityLink(source_entity_id='e1', target_entity_id='e2', link_type='related_to', confidence=0.8, source=None, metadata={})]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "linker.link_entities(\"e1\", \"e2\", link_type=\"related_to\", confidence=0.8)\n",
    "linker.get_entity_links(\"e1\")[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d11f87f",
   "metadata": {},
   "outputs": [],
   "source": [
    "linker.build_entity_web()[\"statistics\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "072efafd",
   "metadata": {},
   "source": [
    "## 8) Low-level building blocks: `AgentMemory` and `ContextRetriever`\n",
    "\n",
    "If you want more control than `AgentContext`, you can wire the parts directly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3791d6c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Python powers Semantica.', 'short_term', 1.0),\n",
       " ('Python', 'graph:e_python', 0.5)]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.context import AgentMemory, ContextRetriever\n",
    "\n",
    "memory = AgentMemory(vector_store=vs, knowledge_graph=kg, retention_policy=\"unlimited\")\n",
    "memory.store(\"Python powers Semantica.\", metadata={\"type\": \"fact\", \"conversation_id\": \"conv_2\"})\n",
    "\n",
    "retriever = ContextRetriever(memory_store=memory, knowledge_graph=kg, vector_store=vs)\n",
    "results = retriever.retrieve(\"Python Semantica\", max_results=5)\n",
    "\n",
    "[(r.content, r.source, round(r.score, 3)) for r in results]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92060402",
   "metadata": {},
   "source": [
    "## 9) Methods, registry, and configuration\n",
    "\n",
    "The `methods` layer exposes convenience functions, while `registry` lets you plug in your own implementations. `config` provides runtime configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "896e7001",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'7_days'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.context.config import context_config\n",
    "\n",
    "context_config.set(\"retention_policy\", \"7_days\")\n",
    "context_config.get(\"retention_policy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "e925a2e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'graph': ['entities_relationships', 'conversations', 'hybrid', 'custom_demo']}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from semantica.context.methods import build_context_graph\n",
    "from semantica.context.registry import method_registry\n",
    "\n",
    "def custom_graph_method(entities, relationships, conversations=None, **kwargs):\n",
    "    return {\n",
    "        \"nodes\": [],\n",
    "        \"edges\": [],\n",
    "        \"statistics\": {\"node_count\": 0, \"edge_count\": 0},\n",
    "    }\n",
    "\n",
    "method_registry.register(\"graph\", \"custom_demo\", custom_graph_method)\n",
    "method_registry.list_all(\"graph\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "21fa6cb3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'nodes': [], 'edges': [], 'statistics': {'node_count': 0, 'edge_count': 0}}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "build_context_graph(\n",
    "    entities=[{\"id\": \"e1\", \"text\": \"Python\", \"type\": \"PROGRAMMING_LANGUAGE\"}],\n",
    "    relationships=[{\"source_id\": \"e1\", \"target_id\": \"e2\", \"type\": \"related_to\"}],\n",
    "    method=\"custom_demo\",\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
