semantica/cookbook/introduction/19_Context_Module.ipynb

Context Engineering Module

Overview

This notebook provides a comprehensive guide to Semantica's Context Engineering Module - a powerful system for building context graphs, managing agent memory, retrieving context, and linking entities. You'll learn how to use the new synchronous Architecture 2.0 features, including hierarchical memory with token management.

Documentation: API Reference

Learning Objectives

• Hierarchical Memory: Manage short-term (token-buffered) and long-term (vector-stored) memory
• Context Graph: Build and query dynamic knowledge graphs
• Hybrid Retrieval: Combine vector search, graph traversal, and keyword matching
• Entity Linking: Resolve entities across conversations
• Configuration: Customize behavior via YAML or environment variables

---

Installation

pip install semantica

In [ ]:

# Setup: Create a mock vector store for demonstration
from typing import List, Dict, Any, Optional
from semantica.context import VectorStore

class MockVectorStore(VectorStore):
    def __init__(self):
        self.items = {}
        self.counter = 0
        
    def add(self, texts: List[str], metadata: Optional[List[Dict[str, Any]]] = None, **kwargs) -> List[str]:
        ids = []
        for i, text in enumerate(texts):
            id_ = f"id_{self.counter}"
            self.items[id_] = {"text": text, "metadata": metadata[i] if metadata else {}}
            ids.append(id_)
            self.counter += 1
        return ids
        
    def search(self, query: str, limit: int = 5, **kwargs) -> List[Dict[str, Any]]:
        # Simple keyword match for mock
        results = []
        for id_, item in self.items.items():
            if any(w.lower() in item["text"].lower() for w in query.split()):
                results.append({
                    "id": id_,
                    "content": item["text"],
                    "score": 0.9,
                    "metadata": item["metadata"]
                })
        return results[:limit]
        
    def delete(self, ids: List[str], **kwargs) -> bool:
        for id_ in ids:
            self.items.pop(id_, None)
        return True

vs = MockVectorStore()

1. High-Level Interface: AgentContext

The AgentContext class is the easiest way to get started. It unifies vector storage, knowledge graphs, and memory management.

In [ ]:

from semantica.context import AgentContext, ContextGraph

# Initialize with vector store and a new in-memory knowledge graph
kg = ContextGraph()
context = AgentContext(
    vector_store=vs,
    knowledge_graph=kg,
    token_limit=2000,      # Max tokens in short-term memory
    short_term_limit=10    # Max items in short-term memory
)

# Store a memory (automatically goes to short-term and long-term)
context.store(
    "The user, Alice, is a data scientist interested in Python.",
    conversation_id="conv_1",
    user_id="alice_01"
)

# Retrieve context (automatically uses hybrid retrieval)
results = context.retrieve("What does Alice do?")

for res in results:
    print(f"Found: {res['content']} (Score: {res['score']})")

2. Hierarchical Memory Management

Semantica uses a two-tier memory system:

1. Short-Term Memory: A fast, in-memory buffer limited by tokens (to fit in LLM context windows) and item count.
2. Long-Term Memory: Persistent storage backed by the vector store.

Let's observe how the token limit works.

In [ ]:

from semantica.context import AgentMemory

# Initialize memory with strict limits for demonstration
memory = AgentMemory(
    vector_store=vs,
    token_limit=50,       # Very small token limit
    short_term_limit=5    # Max 5 items
)

# Add memories
for i in range(10):
    memory.store(f"Memory item {i}: This is a sentence with some tokens.")
    print(f"Added item {i}. Short-term size: {len(memory.short_term_memory)}")

print("\nFinal short-term memory content:")
for item in memory.short_term_memory:
    print(f"- {item.content}")
    
# Notice that older items are pruned to respect the token limit and item count.

3. Context Graph & GraphRAG

The ContextGraph allows you to structure information as nodes and edges, enabling "GraphRAG" - retrieving information based on relationships rather than just semantic similarity.

In [ ]:

from semantica.context import ContextGraph

graph = ContextGraph()

# Manually building a graph
graph.add_node("n1", "person", "Alice")
graph.add_node("n2", "language", "Python")
graph.add_node("n3", "library", "Semantica")

graph.add_edge("n1", "n2", "uses")
graph.add_edge("n2", "n3", "powers")

# Query the graph
neighbors = graph.get_neighbors("n2", hops=1)
print("Neighbors of Python:", neighbors)

# Using the graph in AgentContext
context = AgentContext(vector_store=vs, knowledge_graph=graph)

# Retrieve with graph expansion
results = context.retrieve(
    "Alice",
    use_graph=True,
    expand_graph=True  # Will pull in 'Python' because Alice uses it
)

print("\nGraph-enhanced Retrieval:")
for res in results:
    print(f"- {res['content']}")

4. Entity Linking

The EntityLinker helps ensure that "Alice", "Alice Smith", and "she" (in context) refer to the same entity ID.

In [ ]:

from semantica.context import EntityLinker

linker = EntityLinker()

# Generate a canonical URI
uri = linker.generate_uri("Python Programming Language")
print(f"Canonical URI: {uri}")

# Check similarity
score = linker._calculate_text_similarity("Python", "Python Lang")
print(f"Similarity Score: {score}")

5. Configuration

You can configure the context module using the config object or environment variables.

In [ ]:

from semantica.context import config

# Set global configuration
config.context_config.set("token_limit", 4096)
config.context_config.set("retention_days", 30)

print(f"Current Token Limit: {config.context_config.get('token_limit')}")