From 2d5d615f424a067a6eceb5561dd726b695563154 Mon Sep 17 00:00:00 2001 From: Sameer6305 Date: Tue, 23 Jun 2026 15:38:01 +0530 Subject: [PATCH 1/2] docs: improve context graph guide onboarding and concepts --- docs/guides/context-graphs.md | 74 ++++++++++++++++++++++++++++++++++- 1 file changed, 72 insertions(+), 2 deletions(-) diff --git a/docs/guides/context-graphs.md b/docs/guides/context-graphs.md index 15631df3..46d47a9c 100644 --- a/docs/guides/context-graphs.md +++ b/docs/guides/context-graphs.md @@ -4,14 +4,63 @@ description: "How Semantica stores, links, searches, and traverses knowledge as icon: "diagram-project" --- -`ContextGraph` is a thread-safe, in-memory property graph with temporal validity windows on every node and edge, built-in BFS traversal, a FAISS vector index for semantic search, and proximity-blended retrieval through `AgentContext`. Use it when multiple agents or threads write to a shared knowledge base while analysts query it in real time. +`ContextGraph` is a thread-safe, in-memory property graph with temporal validity windows on every node and edge, built-in Breadth-First Search (BFS) traversal, a FAISS vector index for semantic search, and proximity-blended retrieval through `AgentContext`. Use it when multiple agents or threads write to a shared knowledge base while analysts query it in real time. + +## What Is a Context Graph? + +A context graph is a property graph that stores entities as **nodes** and relationships as **edges**, enriched with metadata and temporal validity. + +**Nodes** represent entities in your domain — threat actors, vulnerabilities, companies, people, or any concept you want to track. Each node has an ID, a type, optional content text, and metadata properties. + +**Edges** represent relationships between entities — "APT29 uses SUNBURST", "Alice works_for Acme Corp", or "CVE-2024-3400 affects SolarWinds". Each edge has a type, weight, and optional metadata. + +**Metadata** stores additional properties on nodes and edges as key-value pairs — geographic origin, confidence scores, timestamps, or any domain-specific attributes. + +**Property graphs** like ContextGraph differ from simple networks by supporting rich metadata on both nodes and edges, making them suitable for complex real-world domains where relationships need context and attributes. + +## Why Use a Context Graph? + +**Relationship analysis.** Graph structure reveals how entities connect — who targets whom, what exploits what, which decisions led to which outcomes. Relationships that aren't obvious from individual documents become clear when connected. + +**Multi-hop reasoning.** Answer questions like "what can APT29 reach within 3 steps?" or "which vulnerabilities affect our critical systems?" by traversing the graph rather than keyword matching. + +**Context preservation.** Unlike vector search alone, graphs preserve the relationships between entities. When you find a relevant threat actor, you can immediately see their tools, targets, and infrastructure. + +**Temporal state tracking.** Track when relationships were valid, when infrastructure was active, or when decisions were made. Query historical states or filter to current information only. + +## When To Use / When Not To Use + +**Use context graphs for:** +- Complex domains with rich relationships between entities +- Multi-hop reasoning and traversal queries +- Temporal tracking of entity relationships and states +- Integration with structured data that has clear entity relationships +- Collaborative environments where multiple sources contribute connected data + +**Simple vector search may be sufficient for:** +- Pure document retrieval where relationships don't matter +- Single-hop similarity searches +- Exploratory research where structure isn't well-defined +- Read-only analysis of unstructured text without entity relationships + +**Graphs may be unnecessary for:** +- Simple keyword or semantic search tasks +- Static document collections without evolving relationships +- Single-user, short-term analysis projects +- Cases where setup complexity exceeds the relationship complexity - For analytical operations on top of a populated graph — centrality rankings, community detection, node embeddings, link prediction — see the [Graph Analytics guide](graph-analytics). For recording and querying decisions stored as nodes, see the [Decision Intelligence guide](decision-intelligence). + ContextGraph is an **in-memory data structure**. All nodes, edges, and metadata are stored in Python dictionaries and lists. Important graph state should be persisted using `save_to_file()` or `AgentContext.save()` to prevent data loss. For analytical operations on top of a populated graph — centrality rankings, community detection, node embeddings, link prediction — see the [Graph Analytics guide](graph-analytics). For recording and querying decisions stored as nodes, see the [Decision Intelligence guide](decision-intelligence). ## Constructing the Graph +You can construct a ContextGraph in two ways: + +**Manual construction** — Add nodes and edges programmatically using `add_node()` and `add_edge()`. This gives you complete control over the graph structure and is ideal when you have structured data or want to build specific relationship patterns. + +**Automated extraction workflows** — Use `AgentContext.store()` with entity and relationship extraction enabled. This analyzes text content, identifies entities, infers relationships, and builds the graph automatically. The extraction process creates nodes for detected entities and edges for discovered relationships. + The simplest possible graph needs no arguments: ```python @@ -411,6 +460,27 @@ context2 = AgentContext( context2.load("agent_state/") ``` +## Common Pitfalls + +**Duplicate entities.** Adding "APT-29", "APT29", and "Cozy Bear" as separate nodes fragments the graph when they should be one entity. Use consistent naming conventions or deduplication before adding nodes. + +**Inconsistent naming conventions.** Mixing "ThreatActor", "threat_actor", and "Threat-Actor" as node types breaks queries that filter by type. Pick one convention and enforce it across all data sources. + +**Over-connecting nodes.** Creating edges between every entity mentioned in the same document adds noise. Focus on meaningful relationships — direct causation, membership, or functional dependencies rather than co-occurrence. + +**Storing unnecessary information.** Adding every field from source data as metadata bloats memory usage. Include only properties needed for queries, filtering, or downstream analysis. + +**Failing to persist important graph state.** Since ContextGraph is in-memory, shutting down your application loses all nodes and edges unless you call `save_to_file()` or `AgentContext.save()`. Persist regularly during long-running ingestion processes. + +## Relationship Between Graph Structure and Vector Search + +ContextGraph structure and vector search serve complementary purposes: + +- **Graph structure** captures explicit relationships and enables traversal, reachability analysis, and multi-hop reasoning +- **Vector search** enables semantic similarity queries and fuzzy matching based on content + +When used together via `AgentContext`, you can blend both approaches — find semantically similar content while boosting results that are structurally close to your starting point in the graph. + ## Domain Examples From d74477bf94707d9a38a1f8a630dfd51486b7a59d Mon Sep 17 00:00:00 2001 From: KaifAhmad1 Date: Fri, 26 Jun 2026 21:32:55 +0530 Subject: [PATCH 2/2] fix(docs): correct API inaccuracies in context graph guide - Replace non-existent shortest_path() with get_neighbors() + path_to_anchor - Remove non-existent extract_subgraph() calls from all domain tab examples - Clarify automated extraction requires knowledge_graph= constructor arg and list input - Distinguish save_to_file() (graph only) from AgentContext.save() (graph + FAISS + memory) - Add resolve_links() step to serialization section for cross-graph link restoration - Link duplicate entities pitfall to the deduplication guide and its API --- docs/guides/context-graphs.md | 67 +++++++++++++++++------------------ 1 file changed, 32 insertions(+), 35 deletions(-) diff --git a/docs/guides/context-graphs.md b/docs/guides/context-graphs.md index 46d47a9c..8521db7e 100644 --- a/docs/guides/context-graphs.md +++ b/docs/guides/context-graphs.md @@ -50,7 +50,7 @@ A context graph is a property graph that stores entities as **nodes** and relati - Cases where setup complexity exceeds the relationship complexity - ContextGraph is an **in-memory data structure**. All nodes, edges, and metadata are stored in Python dictionaries and lists. Important graph state should be persisted using `save_to_file()` or `AgentContext.save()` to prevent data loss. For analytical operations on top of a populated graph — centrality rankings, community detection, node embeddings, link prediction — see the [Graph Analytics guide](graph-analytics). For recording and querying decisions stored as nodes, see the [Decision Intelligence guide](decision-intelligence). + ContextGraph is an **in-memory data structure**. All nodes, edges, and metadata are stored in Python dictionaries and lists. For standalone graphs, persist state with `save_to_file()`. When using `AgentContext`, call `AgentContext.save()` instead — it saves the graph, the FAISS vector index, and memory in one step. For analytical operations on top of a populated graph — centrality rankings, community detection, node embeddings, link prediction — see the [Graph Analytics guide](graph-analytics). For recording and querying decisions stored as nodes, see the [Decision Intelligence guide](decision-intelligence). ## Constructing the Graph @@ -59,7 +59,7 @@ You can construct a ContextGraph in two ways: **Manual construction** — Add nodes and edges programmatically using `add_node()` and `add_edge()`. This gives you complete control over the graph structure and is ideal when you have structured data or want to build specific relationship patterns. -**Automated extraction workflows** — Use `AgentContext.store()` with entity and relationship extraction enabled. This analyzes text content, identifies entities, infers relationships, and builds the graph automatically. The extraction process creates nodes for detected entities and edges for discovered relationships. +**Automated extraction workflows** — Pass a list of documents to `AgentContext.store()` with `extract_entities=True` or `extract_relationships=True`. This requires `knowledge_graph=` to be set in the `AgentContext` constructor; the extraction process then creates nodes for detected entities and edges for discovered relationships. Single strings passed to `store()` are stored as memory items only and do not trigger graph construction. The simplest possible graph needs no arguments: @@ -286,26 +286,15 @@ for n in neighbors: # SolarWinds reached via CVE: band=mid decay=0.700 ``` -For point-to-point routing, use the shortest path finder: +To trace the route from a starting node to a specific target, use `get_neighbors()` with `include_distance_metadata=True`. Each result includes a `path_to_anchor` list showing the exact sequence of node IDs from source to that neighbor: ```python -# shortest_path(source_id, target_id, edge_types=None) -> Optional[List[str]] -path = graph.shortest_path("APT29", "SolarWinds") -if path: - print(" → ".join(path)) -# APT29 → SUNBURST → SolarWinds -``` - -When you need to analyze a sub-cluster in isolation, `extract_subgraph()` gives you a new independent `ContextGraph` instance: - -```python -# extract_subgraph(node_ids, include_edges=True) -> ContextGraph -campaign_nodes = ["APT29", "SUNBURST", "CVE-2020-10148", "45.142.212.100", "SolarWinds"] -subgraph = graph.extract_subgraph(campaign_nodes) - -s = subgraph.stats() -print(f"Subgraph: {s['node_count']} nodes, {s['edge_count']} edges") -# Subgraph: 5 nodes, 6 edges +# get_neighbors with include_distance_metadata=True returns path_to_anchor +neighbors = graph.get_neighbors("APT29", hops=3, include_distance_metadata=True) +for n in neighbors: + if n["id"] == "SolarWinds": + print(" → ".join(n["path_to_anchor"])) + # APT29 → SUNBURST → SolarWinds ``` ## Handling Concurrent Writes @@ -447,6 +436,15 @@ d = graph.to_dict() # d["statistics"] → {"node_count": int, "edge_count": int} ``` +If the graph had cross-graph links created with `link_graph()`, call `resolve_links()` after loading to restore live navigation — object references cannot be serialized, so they must be reconnected manually: + +```python +g1b, g2b = ContextGraph(), ContextGraph() +g1b.load_from_file("actor_graph.json") +g2b.load_from_file("victim_graph.json") +g1b.resolve_links({g2b.graph_id: g2b}) +``` + For full session persistence (graph + FAISS vector index + memory), use `AgentContext.save()` / `AgentContext.load()`: ```python @@ -462,7 +460,7 @@ context2.load("agent_state/") ## Common Pitfalls -**Duplicate entities.** Adding "APT-29", "APT29", and "Cozy Bear" as separate nodes fragments the graph when they should be one entity. Use consistent naming conventions or deduplication before adding nodes. +**Duplicate entities.** Adding "APT-29", "APT29", and "Cozy Bear" as separate nodes fragments the graph when they should be one entity. Use consistent naming conventions upfront, or use `detect_duplicates()` and `EntityMerger` from the [Deduplication](deduplication) guide to merge them after ingestion. **Inconsistent naming conventions.** Mixing "ThreatActor", "threat_actor", and "Threat-Actor" as node types breaks queries that filter by type. Pick one convention and enforce it across all data sources. @@ -569,15 +567,14 @@ print("Reachable via lateral movement:") for n in reachable: print(f" hop={n['hop']} {n['id']}") -# Critical path to the backup server -path = graph.shortest_path("ws-finance-04", "srv-backup-01") -print(" → ".join(path)) -# ws-finance-04 → srv-dc-01 → srv-backup-01 - -# Isolate incident subgraph for reporting -incident_nodes = [n["id"] for n in reachable] + ["ws-finance-04"] -subgraph = graph.extract_subgraph(incident_nodes) -print(f"Incident scope: {subgraph.stats()['node_count']} hosts") +# Path from initial foothold to the backup server via path_to_anchor +all_paths = graph.get_neighbors("ws-finance-04", hops=3, + relationship_types=["lateral_move"], + include_distance_metadata=True) +for n in all_paths: + if n["id"] == "srv-backup-01": + print(" → ".join(n["path_to_anchor"])) + # ws-finance-04 → srv-dc-01 → srv-backup-01 ``` @@ -666,11 +663,11 @@ q4_exposures = graph.find_active_nodes( ) print(f"\nActive Q4 2024 exposures: {len(q4_exposures)}") -# Extract stress-test subgraph -stress_nodes = ["BankA", "SPV-EUR-01", "BankB", "CCP-LME"] -subgraph = graph.extract_subgraph(stress_nodes) -print(f"Stress-test scope: {subgraph.stats()['node_count']} entities, " - f"{subgraph.stats()['edge_count']} exposures") +# Reachability from BankA: identifies all entities in the stress-test scope +stress_reach = graph.get_neighbors("BankA", hops=2) +print(f"Stress-test reachable entities: {len(stress_reach)}") +for n in stress_reach: + print(f" hop={n['hop']} {n['id']}") ```