docs: apply full Mintlify component overhaul to all 27 reference pages and concepts.md

Replace plain markdown in every docs/reference/ file and docs/concepts.md with
rich Mintlify JSX components — CardGroup, Steps, Tabs, AccordionGroup, Tip,
Warning, Note, and CodeGroup — for a consistent, navigable, production-grade
developer experience.

docs/concepts.md

@@ -10,7 +10,21 @@ icon: "book-open"
 
 Semantica transforms unstructured data — documents, web pages, reports, databases — into **knowledge graphs**: structured representations that AI systems can query, reason about, and trace back to sources.
 
-At its core, Semantica adds a **context and intelligence layer** on top of your existing AI stack. It doesn't replace LangChain, LlamaIndex, or your LLM provider — it makes their outputs accountable.
+At its core, Semantica adds a **context and accountability layer** on top of your existing AI stack. It doesn't replace LangChain, LlamaIndex, or your LLM provider — it makes their outputs **grounded**, **traceable**, and **auditable**.
+
+<CardGroup cols={3}>
+  <Card title="Context Layer" icon="diagram-project">
+    Knowledge graphs, GraphRAG retrieval, semantic embeddings, and temporal intelligence ground every LLM response in structured, queryable facts.
+  </Card>
+  <Card title="Accountability Layer" icon="shield-check">
+    Provenance tracking, decision intelligence, conflict detection, and W3C PROV-O compliance make every claim in your AI stack auditable and explainable.
+  </Card>
+  <Card title="Extension Layer" icon="plug">
+    `PluginRegistry` and `MethodRegistry` let you replace or augment any component — ingestors, extractors, reasoning engines, backends — without changing framework code.
+  </Card>
+</CardGroup>
+
+---
 
 ## Knowledge Graphs
 
@@ -26,7 +40,7 @@ This structure makes knowledge **searchable**, **connectable**, **queryable**, a
 
 ## Entity Extraction (NER)
 
-Scanning text to find and classify real-world entities.
+Scanning text to find and classify real-world entities:
 
 ```python
 # Input: "Apple Inc. was founded by Steve Jobs in 1976 in Cupertino."
@@ -42,13 +56,15 @@ Scanning text to find and classify real-world entities.
 
 Each entity gets a type, confidence score, and a link to its source document. Three extraction methods are available:
 
-- **`"pattern"`** — fast, regex-based, no API key required
-- **`"ml"`** — local ML model, higher accuracy
-- **`"llm"`** — LLM-powered, highest accuracy, supports all 8 providers
+| Method | Speed | Accuracy | Requirements |
+| ------ | ----- | -------- | ------------ |
+| `"pattern"` | ⚡ Very fast | Moderate | No API key — regex-based |
+| `"ml"` | Fast | High | Local ML model |
+| `"llm"` | Medium | Highest | LLM provider — all 9 supported |
 
 ## Relationship Extraction
 
-Finding how entities connect to each other.
+Finding how entities connect to each other:
 
 ```python
 {
@@ -71,8 +87,9 @@ Semantica uses embeddings for:
 - **Entity resolution** — match the same entity across different sources
 - **Precedent search** — find similar past decisions
 - **GraphRAG retrieval** — hybrid vector + graph traversal
+- **Distance Intelligence** — N×N semantic distance matrices between any node set
 
-**Supported models:** Sentence-Transformers, FastEmbed, OpenAI, BGE
+**Supported models:** Sentence-Transformers, FastEmbed, OpenAI, BGE, Ollama local embeddings.
 
 ## GraphRAG
 
@@ -80,14 +97,24 @@ GraphRAG (Graph-Augmented Retrieval Augmented Generation) enhances LLM responses
 
 <img src="/assets/img/diagrams/graphrag-flow.svg" alt="GraphRAG flow: User Query → Vector Search + Graph Traversal → Context Builder → LLM → Grounded Answer" style={{ width: '100%', borderRadius: '12px', margin: '16px 0 20px' }} />
 
-**How it works:**
+<Steps>
+  <Step title="User submits a query">
+    The query is embedded and used to seed both vector search and graph traversal simultaneously.
+  </Step>
+  <Step title="Hybrid context retrieval">
+    Semantica retrieves relevant graph context — entities, typed relationships, and multi-hop reasoning paths — alongside vector-similar text chunks.
+  </Step>
+  <Step title="Context building">
+    Retrieved facts and reasoning paths are assembled into a structured prompt context, each fact tagged with its source node and confidence.
+  </Step>
+  <Step title="LLM generates a grounded response">
+    The LLM produces an answer where every claim links back to a source node in the graph — no floating assertions, no hallucinations from training data.
+  </Step>
+</Steps>
 
-1. User submits a query
-2. Semantica retrieves relevant graph context — entities, relationships, reasoning paths
-3. The LLM generates a response grounded in that context
-4. Every claim in the response links back to a source node in the graph
-
-This eliminates the hallucination and traceability problems of standard RAG.
+<Tip>
+  **GraphRAG eliminates the hallucination and traceability problems of standard RAG.** Standard RAG retrieves text chunks; GraphRAG retrieves structured facts with typed relationships. The LLM cannot confabulate structure that was never in the graph.
+</Tip>
 
 ## Ontology
 
@@ -104,7 +131,7 @@ ontology = {
 }
 ```
 
-Semantica can auto-generate ontologies from your knowledge graph or import existing OWL/RDF/Turtle ontologies. The **Ontology Hub** (v0.5.0) adds a visual editor, SHACL Studio, alignment authoring, and a live health dashboard.
+Semantica can auto-generate ontologies from your knowledge graph or import existing OWL/RDF/Turtle ontologies. The **Ontology Hub** (v0.5.0) adds a visual editor, SHACL Studio, alignment authoring, and a live health dashboard. See the [Ontology reference](reference/ontology) for the full 6-stage generation pipeline.
 
 ## Reasoning & Inference
 
@@ -116,16 +143,60 @@ Known:    Apple Inc. is headquartered in Cupertino
 Inferred: Steve Jobs has a connection to Cupertino
 ```
 
-| Engine | Description |
-| ------ | ----------- |
-| Forward chaining | Applies rules repeatedly until no new facts can be derived |
-| Rete network | Efficient pattern matching for large rule sets |
-| Deductive | Classical deductive reasoning |
-| Abductive | Infers the most likely explanation |
-| SPARQL | Query-based inference over RDF graphs |
-| Datalog | Recursive Horn clause rules with fixpoint semantics (v0.4.0) |
+<Tabs>
+  <Tab title="Forward Chaining">
+    Applies IF/THEN rules repeatedly until no new facts can be derived. Best for alert systems, compliance checks, and trigger-based workflows.
 
-All engines produce **explainable inference paths**, not black-box conclusions.
+    ```python
+    from semantica.reasoning import ReasoningEngine
+
+    engine = ReasoningEngine(llm_provider=llm)
+    result = engine.reason(facts=kg, rules=rule_set, method="forward_chaining")
+    ```
+  </Tab>
+  <Tab title="Rete Network">
+    Efficient pattern matching for large rule sets — the Rete algorithm avoids re-evaluating rules whose preconditions haven't changed. Best for thousands of rules over millions of facts.
+
+    ```python
+    engine = ReasoningEngine(llm_provider=llm)
+    result = engine.reason(facts=kg, rules=rule_set, method="rete")
+    ```
+  </Tab>
+  <Tab title="Deductive & Abductive">
+    **Deductive** — classical syllogistic reasoning from premises to guaranteed conclusions.
+
+    **Abductive** — infers the most likely explanation for observed evidence. Best for diagnostic and investigative use cases.
+
+    ```python
+    result = engine.reason(facts=kg, rules=rule_set, method="deductive")
+    result = engine.reason(facts=kg, rules=rule_set, method="abductive")
+    ```
+  </Tab>
+  <Tab title="Datalog (v0.4.0)">
+    Recursive Horn clause rules with fixpoint semantics — handles transitive closure and recursive relationships that forward chaining cannot express.
+
+    ```python
+    from semantica.reasoning import DatalogReasoner
+
+    reasoner = DatalogReasoner()
+    result   = reasoner.reason(facts=kg, rules=datalog_rules)
+    ```
+  </Tab>
+  <Tab title="Engine Comparison">
+
+    | Engine | Description | Best For |
+    | ------ | ----------- | -------- |
+    | Forward chaining | Applies rules until fixpoint | Alert systems, compliance checks |
+    | Rete network | Efficient pattern matching | Large rule sets, high fact throughput |
+    | Deductive | Classical syllogistic reasoning | Mathematical and logical inference |
+    | Abductive | Most likely explanation | Diagnostics, investigation |
+    | SPARQL | Query-based inference over RDF | Semantic web, ontology reasoning |
+    | Datalog (v0.4.0) | Recursive Horn clause rules | Transitive closure, graph reachability |
+
+  </Tab>
+</Tabs>
+
+All engines produce **explainable inference paths** — not black-box conclusions. Every derived fact includes the rules and premises that produced it.
 
 ## Temporal Intelligence
 
@@ -142,13 +213,13 @@ tkg.add_node("ceo_role", valid_from=datetime(2020, 1, 1), valid_until=datetime(2
 snapshot = tkg.at(datetime(2021, 6, 15))
 ```
 
-**Features:** Allen interval algebra (all 13 relations), OWL-Time export, `recorded_at` stamping, temporal provenance.
+**Supported features:** Allen interval algebra (all 13 temporal relations), OWL-Time export, `recorded_at` stamping, temporal provenance.
 
-**Common uses:** tracking company leadership changes, policy evolution, research timelines, financial instrument histories.
+**Common uses:** tracking company leadership changes, policy evolution, research timelines, financial instrument histories, regulatory compliance windows.
 
 ## Distance Intelligence
 
-Explore the semantic neighborhood of any entity in your graph. Useful for understanding what's conceptually close, detecting clusters, and visualizing knowledge topology.
+Explore the semantic neighborhood of any entity in your graph — useful for understanding what's conceptually close, detecting clusters, and visualizing knowledge topology.
 
 ```python
 from semantica.kg import DistanceCalculator
@@ -158,33 +229,68 @@ neighborhood = calc.semantic_neighborhood("Apple Inc.", radius=0.4)
 matrix       = calc.distance_matrix(["Apple Inc.", "Google", "Microsoft"])
 ```
 
-**Features:** N×N distance matrices, ego-mode visualization, distance band classification (`near` / `mid` / `far`), embedding cache optimization.
+**Features:** N×N semantic distance matrices, ego-mode visualization, distance band classification (`near` / `mid` / `far`), embedding cache optimization for large graphs.
+
+The [Visualization module](reference/visualization) renders distance matrices as interactive heatmaps and ego-mode neighborhood graphs. The [Explorer](reference/explorer) embeds distance intelligence directly in the browser dashboard.
 
 ## Deduplication & Entity Resolution
 
 Real-world data contains the same entity under many names — "Apple", "Apple Inc.", "Apple Computer Inc." Semantica's deduplication pipeline detects these, merges attributes, resolves conflicts, and preserves the original source provenance.
 
-**Strategies:**
+<Tabs>
+  <Tab title="Strategies">
 
-- **v1** — Jaro-Winkler similarity, suitable for small datasets
-- **`blocking_v2`** — candidate blocking for large corpora
-- **`hybrid_v2`** — combines blocking with semantic matching
-- **`semantic_v2`** — pure embedding-based resolution, up to 7x faster than v1
+    | Strategy | Algorithm | Best For |
+    | -------- | --------- | -------- |
+    | `v1` | Jaro-Winkler string similarity | Small datasets, fast baseline |
+    | `blocking_v2` | Candidate blocking + similarity | Large corpora — reduces O(n²) comparisons |
+    | `hybrid_v2` | Blocking + semantic embedding match | Mixed structured/unstructured entity names |
+    | `semantic_v2` | Pure embedding-based resolution | Up to 7× faster than v1; handles abbreviations and aliases |
+
+  </Tab>
+  <Tab title="Configuration">
+    ```python
+    from semantica.deduplication import EntityDeduplicator
+
+    deduplicator = EntityDeduplicator(
+        strategy="semantic_v2",
+        threshold=0.85,          # similarity threshold for merge decision
+        embedding_model="all-mpnet-base-v2",
+    )
+
+    deduplicated_entities = deduplicator.deduplicate(entities)
+    ```
+  </Tab>
+</Tabs>
 
 ## Provenance & Auditability
 
 Every fact in Semantica links back to:
 
-- The source document it came from
-- The extraction method used
-- The ontology rules applied
-- The reasoning steps that produced any inference
+- The **source document** it came from
+- The **extraction method** used (pattern / ML / LLM)
+- The **ontology rules** applied during graph construction
+- The **reasoning steps** that produced any inferred fact
 
-This is W3C PROV-O compliant lineage — suitable for regulated industries that require audit trails (HIPAA, SOX, GDPR, FDA 21 CFR Part 11).
+<Note>
+  This is W3C PROV-O compliant lineage — suitable for regulated industries that require audit trails (HIPAA, SOX, GDPR, FDA 21 CFR Part 11). Use `RDFExporter(include_provenance=True)` to embed provenance inline in any RDF export.
+</Note>
+
+```python
+from semantica.provenance import ProvenanceManager
+
+prov    = ProvenanceManager()
+lineage = prov.get_entity_lineage("apple_inc")
+
+print(f"Source:    {lineage.source_document}")
+print(f"Method:    {lineage.extraction_method}")
+print(f"Extracted: {lineage.timestamp}")
+print(f"Checksum:  {lineage.checksum}")
+```
 
 ## Decision Intelligence
 
-Every agent decision is a first-class object in Semantica — recorded, causally linked, and searchable by precedent.
+Every agent decision is a first-class object in Semantica — recorded, causally linked, and searchable by precedent. This is the **accountability layer** for AI pipelines: decisions are no longer ephemeral log messages, they are queryable knowledge graph nodes.
 
 ```python
 decision_id = context.record_decision(
@@ -195,11 +301,16 @@ decision_id = context.record_decision(
     confidence=0.91,
 )
 
+# Find similar past decisions before making a new one
 precedents = context.find_precedents("model selection reasoning", limit=5)
+
+# Trace downstream impact of a past decision
 influence  = context.analyze_decision_influence(decision_id)
 ```
 
-This prevents inconsistent decisions, enables audits, and lets agents learn from their own history.
+<Tip>
+  **Use `find_precedents()` before every high-stakes decision.** Hybrid similarity search over all recorded decisions surfaces past reasoning that may apply — reducing inconsistency across agent runs and enabling genuine organisational learning from AI decision history.
+</Tip>
 
 ## Conflict Detection
 
@@ -207,10 +318,78 @@ When multiple sources disagree on the same fact, Semantica flags and resolves th
 
 **Resolution strategies:**
 
-- Prefer the most recent source
-- Prefer the most reliable source
-- Majority vote across sources
-- Flag for manual review
+- **Recency** — prefer the most recent source
+- **Source credibility** — prefer the most reliable source (configurable credibility scores)
+- **Majority vote** — aggregate across all sources with ≥ 2 agreeing
+- **Manual review** — flag for human arbitration; continue pipeline without blocking
+
+See the [Conflicts reference](reference/conflicts) for `ConflictResolver`, `SourceTracker`, and `InvestigationGuideGenerator`.
+
+## Custom Plugin Development
+
+Semantica is designed for extension. Any component — ingestor, extractor, graph builder, reasoning engine — can be replaced or augmented with a custom implementation registered at runtime.
+
+<AccordionGroup>
+  <Accordion title="PluginRegistry — replace any component by name">
+
+    `PluginRegistry` provides dynamic plugin discovery, registration, and loading across all modules. Register your own class under a string key; Semantica will use it wherever that key is referenced in config or pipeline steps.
+
+    ```python
+    from semantica.core import PluginRegistry
+
+    registry = PluginRegistry()
+
+    # Register a custom ingestor
+    registry.register_plugin(
+        "my_sql_ingestor", MySQLIngestor,
+        version="1.0.0",
+        description="PostgreSQL ingestor for internal warehouse",
+        capabilities=["ingest"],
+    )
+
+    # Load and use
+    plugin = registry.load_plugin("my_sql_ingestor", connection_string="postgresql://...")
+    result = plugin.execute("SELECT * FROM documents")
+
+    # Reference by name in pipeline YAML — no code changes needed
+    ```
+
+    ```yaml
+    steps:
+      - name: ingest
+        plugin: my_sql_ingestor
+        config:
+          connection_string: "${DB_URL}"
+    ```
+
+    **Extension points available:** ingestors, parsers, normalizers, extractors, reasoning engines, export formats, vector store backends, graph store backends, visualization renderers.
+
+  </Accordion>
+  <Accordion title="MethodRegistry — add domain-specific graph operations">
+
+    `MethodRegistry` lets you register custom methods on knowledge graph objects by name — useful for adding domain-specific graph operations without subclassing.
+
+    ```python
+    from semantica.kg import MethodRegistry
+
+    registry = MethodRegistry()
+
+    def find_supply_chain_hops(graph, source_node, max_hops=3):
+        """Custom BFS traversal for supply chain graphs."""
+        ...
+
+    # Register under a string key
+    registry.register("supply_chain_hops", find_supply_chain_hops)
+
+    # Call by name on any graph object
+    result = registry.call("supply_chain_hops", kg, source_node="Supplier_A", max_hops=5)
+
+    # List all registered methods
+    print(registry.list_methods())   # ["supply_chain_hops", ...]
+    ```
+
+  </Accordion>
+</AccordionGroup>
 
 <CardGroup cols={2}>
   <Card title="Quickstart Tutorial" icon="play" href="quickstart">

docs/modules.md

@@ -439,41 +439,69 @@ python -m semantica.mcp_server
 
 ### Seed
 
-Deterministic data seeding for testing and development.
+Bootstrap knowledge graphs from verified structured sources — fixed-point reference data, controlled vocabularies, and domain anchors.
 
 ```python
 from semantica.seed import SeedManager
 
 seed = SeedManager()
 seed.populate(kg, dataset="companies", count=100)
+
+# Load domain seeds from file or built-in datasets
+seed.load_from_file("seed_data/industries.json")
+seed.inject(kg)   # merges seed nodes without duplicating existing entities
 ```
 
+**Use cases:** anchoring extraction with known entities, pre-populating ontology classes, deterministic test graph generation.
+
 ### Evals
 
-Evaluation harness for extraction and reasoning quality.
+Evaluation framework for measuring KG quality, extraction accuracy, and pipeline performance.
 
 ```python
-from semantica.evals import Evaluator
+from semantica.evals import KGEvaluator, ExtractionEvaluator, PipelineEvaluator, RegressionTracker
 
-evaluator = Evaluator()
-scores    = evaluator.evaluate(predicted_entities, ground_truth)
-# Returns: {"precision": 0.91, "recall": 0.87, "f1": 0.89}
+# KG quality
+report = KGEvaluator().evaluate(kg, ontology=ontology)
+print(f"Completeness: {report.completeness:.2%}  Consistency: {report.consistency:.2%}")
+
+# Extraction accuracy
+report = ExtractionEvaluator().evaluate_ner(predictions=extracted, gold_standard=annotated)
+print(f"Precision: {report.precision:.3f}  Recall: {report.recall:.3f}  F1: {report.f1:.3f}")
+
+# Pipeline throughput and latency
+metrics = PipelineEvaluator().benchmark(pipeline, data="data/", bench_runs=5)
+print(f"Throughput: {metrics.docs_per_second:.1f} docs/sec")
+
+# Regression tracking across runs
+tracker = RegressionTracker(db_path="eval_history.db")
+run_id  = tracker.record_run(pipeline_version="v1.2.0", metrics=metrics)
+diff    = tracker.compare(run_id, baseline_run_id="run_abc123")
 ```
 
-**Metrics:** precision, recall, F1 for NER, relation extraction, and reasoning
+**Components:** `KGEvaluator`, `ExtractionEvaluator`, `PipelineEvaluator`, `RegressionTracker`
 
 ### Core
 
 Base classes, shared data models, and the plugin registry used across all modules.
 
 ```python
-from semantica.core import Orchestrator, PluginRegistry
+from semantica.core import Semantica, PluginRegistry, ConfigManager
 
+# Top-level orchestrator
+sem = Semantica(config_path="config.yaml")
+sem.initialize()
+
+# Plugin registry — register custom components
 registry = PluginRegistry()
 registry.register("my_ingestor", MyCustomIngestor)
+
+# Config management
+config  = ConfigManager(config_path="config.yaml")
+batch   = config.get("processing.batch_size", default=32)
 ```
 
-**Components:** `Orchestrator`, `PluginRegistry`, `ConfigManager`, `Lifecycle`
+**Components:** `Semantica`, `PluginRegistry`, `ConfigManager`, `LifecycleManager`, `HealthMonitor`, `Config`
 
 ### Utils
 
@@ -504,8 +532,8 @@ from semantica.utils import helpers, validators, logging
 | [ingest](reference/ingest) | Data ingestion | `FileIngestor`, `WebIngestor`, `ParquetIngestor`, `XMLIngestor` |
 | [parse](reference/parse) | Document parsing | `DocumentParser`, `DoclingParser` |
 | [split](reference/split) | Text chunking | `TextSplitter` |
-| [normalize](reference/normalize) | Data cleaning | `DataNormalizer` |
-| [semantic_extract](reference/semantic_extract) | NER & relation extraction | `NERExtractor`, `RelationExtractor`, `TripletExtractor` |
+| [normalize](reference/normalize) | Data cleaning | `TextNormalizer`, `EntityNormalizer`, `LanguageDetector` |
+| [semantic_extract](reference/semantic_extract) | NER & relation extraction | `NERExtractor`, `RelationExtractor`, `TripletExtractor`, `SemanticAnalyzer`, `SemanticNetworkExtractor`, `ExtractionValidator` |
 | [kg](reference/kg) | Graph construction | `GraphBuilder`, `TemporalKnowledgeGraph`, `DistanceCalculator` |
 | [ontology](reference/ontology) | Schema management | `OntologyManager`, `SHACLGenerator` |
 | [reasoning](reference/reasoning) | Logical inference | `ReasoningEngine`, `DatalogEngine` |
@@ -513,7 +541,7 @@ from semantica.utils import helpers, validators, logging
 | [vector_store](reference/vector_store) | Vector database | `VectorStore` |
 | [graph_store](reference/graph_store) | Graph database | `GraphStore` |
 | [triplet_store](reference/triplet_store) | RDF triple store | `TripletStore` |
-| [deduplication](reference/deduplication) | Entity resolution | `EntityResolver`, `DuplicateDetector` |
+| [deduplication](reference/deduplication) | Entity resolution | `EntityResolver`, `DuplicateDetector`, `ClusterBuilder`, `MergeStrategyManager` |
 | [conflicts](reference/conflicts) | Conflict resolution | `ConflictDetector` |
 | [context](reference/context) | Agent context & decisions | `AgentContext`, `ContextGraph` |
 | [provenance](reference/provenance) | W3C PROV-O lineage | `ProvenanceManager` |
@@ -524,9 +552,9 @@ from semantica.utils import helpers, validators, logging
 | [explorer](reference/explorer) | Knowledge Explorer UI | `start_explorer` |
 | [llms](reference/llms) | LLM providers | `Groq`, `OpenAI`, `create_provider` |
 | [mcp_server](reference/mcp_server) | MCP stdio server | `python -m semantica.mcp_server` |
-| [seed](reference/seed) | Test data seeding | `SeedManager` |
-| [evals](reference/evals) | Quality evaluation | `Evaluator` |
-| [core](reference/core) | Base classes & registry | `Orchestrator`, `PluginRegistry` |
+| [seed](reference/seed) | KG bootstrapping from structured sources | `SeedManager` |
+| [evals](reference/evals) | Quality evaluation | `KGEvaluator`, `ExtractionEvaluator`, `PipelineEvaluator`, `RegressionTracker` |
+| [core](reference/core) | Base classes & registry | `Semantica`, `ConfigManager`, `PluginRegistry`, `LifecycleManager` |
 | [utils](reference/utils) | Shared utilities | `helpers`, `validators` |
 
 <CardGroup cols={2}>

docs/reference/change_management.md

@@ -4,36 +4,97 @@ description: "Version control, SHA-256 checksums, diff analysis, rollback, and a
 icon: "clock-rotate-left"
 ---
 
-`semantica.change_management` provides enterprise-grade versioning and audit trails for knowledge graphs and ontologies — SHA-256 checksums, snapshot history, diff analysis, rollback protection, and compliance-ready audit export (HIPAA, SOX, FDA 21 CFR Part 11).
+`semantica.change_management` provides enterprise-grade versioning and audit trails for knowledge graphs and ontologies. Every snapshot carries a SHA-256 checksum, every modification is logged, and every state can be diffed or rolled back — giving you a complete, tamper-evident record suitable for regulated industries.
+
+<Note>
+  Compliance frameworks supported out of the box: **HIPAA**, **SOX**, **GDPR**, and **FDA 21 CFR Part 11**.
+</Note>
 
 ## What You Get
 
-- **`TemporalVersionManager`** — snapshot, diff, rollback, and audit trail for knowledge graphs
-- **`OntologyVersionManager`** — version control for OWL ontologies with diff and migration support
-- **`VersionStorage`** — pluggable storage: `InMemoryVersionStorage` for tests, `SQLiteVersionStorage` for production
-- **`compute_checksum` / `verify_checksum`** — SHA-256 integrity verification
-- **`ChangeLogEntry`** — structured record of every change in a snapshot
+<CardGroup cols={2}>
+  <Card title="TemporalVersionManager" icon="code-branch">
+    Snapshot, diff, rollback, and per-entity audit trail for knowledge graphs.
+  </Card>
+  <Card title="OntologyVersionManager" icon="sitemap">
+    Version control for OWL ontologies with diff and schema migration support.
+  </Card>
+  <Card title="VersionStorage" icon="database">
+    Pluggable backends — `InMemoryVersionStorage` for tests, `SQLiteVersionStorage` for production.
+  </Card>
+  <Card title="Integrity Verification" icon="shield-check">
+    SHA-256 / SHA-512 checksums to detect any unauthorised graph modification.
+  </Card>
+  <Card title="ChangeLogEntry" icon="list-check">
+    Structured record of every change: author, timestamp, checksum, and change list.
+  </Card>
+  <Card title="Compliance Export" icon="file-shield">
+    Full audit trail as CSV or JSON for regulatory review and subject-access requests.
+  </Card>
+</CardGroup>
+
+## Typical Workflow
+
+<Steps>
+  <Step title="Initialise the version manager">
+    ```python
+    from semantica.change_management import TemporalVersionManager
+
+    manager = TemporalVersionManager(storage_path="versions.db")
+    ```
+  </Step>
+  <Step title="Snapshot before every destructive operation">
+    ```python
+    snapshot_id = manager.create_snapshot(
+        graph=kg,
+        version="v1.0",
+        author="user@example.com",
+        message="Before deduplication run"
+    )
+    print(f"Snapshot: {snapshot_id}")
+    print(f"Checksum: {manager.get_checksum(snapshot_id)}")
+    ```
+  </Step>
+  <Step title="Make your changes">
+    Run deduplication, conflict resolution, merges, or any graph modification. The version manager tracks nothing automatically — you control when snapshots are taken.
+  </Step>
+  <Step title="Snapshot the result">
+    ```python
+    snapshot_v2 = manager.create_snapshot(
+        graph=kg,
+        version="v2.0",
+        author="user@example.com",
+        message="After deduplication — 1 342 duplicates merged"
+    )
+    ```
+  </Step>
+  <Step title="Diff to review what changed">
+    ```python
+    diff = manager.diff("v1.0", "v2.0")
+    print(diff.summary)
+
+    for change in diff.changes:
+        print(f"  [{change.type}] {change.element}: {change.description}")
+    ```
+  </Step>
+  <Step title="Rollback if needed">
+    ```python
+    # Safe mode (default) — fails with a clear error rather than dropping nodes
+    manager.rollback(target_version="v1.0", allow_data_loss=False)
+    ```
+  </Step>
+</Steps>
 
 ## TemporalVersionManager
 
-Version control for knowledge graphs — snapshot, diff, and rollback:
+Version control for knowledge graphs — snapshot, diff, and rollback.
 
-```python
-from semantica.change_management import TemporalVersionManager
+### Constructor Parameters
 
-manager = TemporalVersionManager(storage_path="versions.db")
-
-# Create a snapshot
-snapshot_id = manager.create_snapshot(
-    graph=kg,
-    version="v1.0",
-    author="user@example.com",
-    message="Initial knowledge graph"
-)
-
-print(f"Snapshot: {snapshot_id}")
-print(f"Checksum: {manager.get_checksum(snapshot_id)}")
-```
+| Parameter | Type | Default | Description |
+| --------- | ---- | ------- | ----------- |
+| `storage_path` | `str` | `None` | Path to SQLite database; uses in-memory if omitted |
+| `storage` | `VersionStorage` | `None` | Explicit storage backend instance — overrides `storage_path` |
 
 ### List, Retrieve, and Rollback
 
@@ -46,32 +107,51 @@ for v in versions:
 # Retrieve a specific version
 kg_v1 = manager.get_version("v1.0")
 
-# Rollback to a previous version (safe by default — fails if data would be lost)
+# Rollback to a previous version
 manager.rollback(target_version="v1.0", allow_data_loss=False)
 ```
 
-### Constructor Parameters
-
-| Parameter | Type | Default | Description |
-| --------- | ---- | ------- | ----------- |
-| `storage_path` | `str` | `None` | Path to SQLite database; uses in-memory if omitted |
-| `storage` | `VersionStorage` | `None` | Explicit storage backend instance |
+<Warning>
+  `rollback(allow_data_loss=False)` is the safe default — it fails with a clear error if nodes were added after the target snapshot. Set `allow_data_loss=True` only when you explicitly intend to discard those changes.
+</Warning>
 
 ## Diff Analysis
 
-Compare any two snapshots to see exactly what changed:
+Compare any two snapshots to see exactly what changed — useful for code review, incident investigation, and regulatory audit:
 
 ```python
 diff = manager.diff("v1.0", "v2.0")
 
 print(f"Added nodes:    {len(diff.added_nodes)}")
 print(f"Removed nodes:  {len(diff.removed_nodes)}")
+print(f"Modified nodes: {len(diff.modified_nodes)}")
+print(f"Added edges:    {len(diff.added_edges)}")
+print(f"Removed edges:  {len(diff.removed_edges)}")
 print(f"Modified edges: {len(diff.modified_edges)}")
 
 for change in diff.changes:
     print(f"  [{change.type}] {change.element}: {change.description}")
 ```
 
+<Accordion title="DiffResult schema">
+
+```python
+@dataclass
+class DiffResult:
+    from_version:   str                # source snapshot ID
+    to_version:     str                # target snapshot ID
+    added_nodes:    List[str]          # IDs of newly added entities
+    removed_nodes:  List[str]          # IDs of deleted entities
+    modified_nodes: List[str]          # IDs of entities with changed properties
+    added_edges:    List[str]          # IDs of newly added relationships
+    removed_edges:  List[str]          # IDs of deleted relationships
+    modified_edges: List[str]          # IDs of relationships with changed properties
+    changes:        List[ChangeRecord] # ordered list of all individual changes
+    summary:        str                # human-readable summary line
+```
+
+</Accordion>
+
 ## OntologyVersionManager
 
 Version control for OWL ontologies — save, diff, and track schema migrations:
@@ -97,21 +177,38 @@ for change in diff.changes:
 
 ## VersionStorage Backends
 
-```python
-from semantica.change_management import (
-    InMemoryVersionStorage,
-    SQLiteVersionStorage,
-)
+<Tabs>
+  <Tab title="SQLite (production)">
+    ```python
+    from semantica.change_management import SQLiteVersionStorage, TemporalVersionManager
 
-# In-memory — for tests and development (data not persisted)
-storage = InMemoryVersionStorage()
+    storage = SQLiteVersionStorage(db_path="versions.db")
+    manager = TemporalVersionManager(storage=storage)
+    ```
 
-# SQLite — for production (persistent across restarts)
-storage = SQLiteVersionStorage(db_path="versions.db")
+    Persists all version history to disk. Survives process restarts. Recommended for any environment where you need to retain the audit trail.
 
-# Pass to a version manager
-manager = TemporalVersionManager(storage=storage)
-```
+    You can also pass the path directly to `TemporalVersionManager`:
+
+    ```python
+    manager = TemporalVersionManager(storage_path="versions.db")
+    ```
+  </Tab>
+  <Tab title="In-Memory (tests)">
+    ```python
+    from semantica.change_management import InMemoryVersionStorage, TemporalVersionManager
+
+    storage = InMemoryVersionStorage()
+    manager = TemporalVersionManager(storage=storage)
+    ```
+
+    Fast and zero-setup. Data is **not persisted** — all version history is lost when the process exits. Use this for unit tests and development only.
+  </Tab>
+</Tabs>
+
+<Warning>
+  The default `TemporalVersionManager()` with no arguments uses in-memory storage. Always pass `storage_path="versions.db"` or an explicit `SQLiteVersionStorage` in production — otherwise your entire version history disappears on restart.
+</Warning>
 
 ## Integrity Verification
 
@@ -130,24 +227,13 @@ if not is_valid:
     raise RuntimeError("Graph has been modified since the checksum was recorded")
 ```
 
-## Audit Trail
-
-Full per-entity audit trail with CSV and JSON export for compliance reporting:
-
-```python
-# Get all changes for a specific entity
-trail = manager.get_audit_trail(entity_id="apple_inc")
-for entry in trail:
-    print(f"{entry.timestamp} — {entry.author}: {entry.action} — {entry.description}")
-
-# Export audit trail for compliance
-manager.export_audit_trail("audit.csv",  format="csv")
-manager.export_audit_trail("audit.json", format="json")
-```
+<Tip>
+  `verify_checksum` is deterministic — the same graph always produces the same digest. Use it as a pre-flight check before any compliance export to confirm the graph hasn't been tampered with since the last snapshot.
+</Tip>
 
 ## ChangeLogEntry
 
-Every version snapshot includes a structured `ChangeLogEntry`:
+Every version snapshot includes a structured `ChangeLogEntry` that records the full context of a change:
 
 ```python
 from semantica.change_management import ChangeLogEntry
@@ -157,11 +243,124 @@ entry: ChangeLogEntry = manager.get_log_entry(snapshot_id)
 print(entry.version)      # "v1.0"
 print(entry.author)       # "user@example.com"
 print(entry.message)      # "Initial knowledge graph"
-print(entry.checksum)     # SHA-256 hex digest
-print(entry.created_at)   # datetime
-print(entry.changes)      # list of individual change records
+print(entry.checksum)     # SHA-256 hex digest of the full graph state
+print(entry.created_at)   # datetime of snapshot creation
+print(entry.node_count)   # total nodes at this snapshot
+print(entry.edge_count)   # total edges at this snapshot
+print(entry.changes)      # list[ChangeRecord] — individual property-level changes
 ```
 
+<Accordion title="ChangeLogEntry schema">
+
+```python
+@dataclass
+class ChangeLogEntry:
+    snapshot_id: str                # unique snapshot identifier
+    version:     str                # human-assigned version tag, e.g. "v1.0"
+    author:      str                # identity of the user or process that created it
+    message:     str                # commit-style description of what changed
+    checksum:    str                # SHA-256 hex digest — changes if graph is tampered
+    created_at:  datetime           # UTC timestamp of snapshot creation
+    node_count:  int                # total entity count at this point in time
+    edge_count:  int                # total relationship count at this point in time
+    changes:     List[ChangeRecord] # granular per-property change records
+    metadata:    Dict               # arbitrary key-value pairs for custom tagging
+```
+
+</Accordion>
+
+## Compliance and Audit Export
+
+All changes are preserved in a tamper-evident audit trail. Export for regulatory review:
+
+<CodeGroup>
+
+```python CSV export
+# Full audit trail
+manager.export_audit_trail("audit.csv", format="csv")
+
+# Scoped to a time range — SOX quarterly review
+from datetime import datetime
+
+trail = manager.get_audit_trail(
+    from_date=datetime(2026, 1, 1),
+    to_date=datetime(2026, 3, 31),
+)
+manager.export_audit_trail("q1_audit.csv", trail=trail, format="csv")
+```
+
+```python JSON export
+# Full audit trail
+manager.export_audit_trail("audit.json", format="json")
+
+# Scoped to a specific entity — HIPAA subject-access request
+trail = manager.get_audit_trail(entity_id="patient_001")
+manager.export_audit_trail("patient_001_audit.json", trail=trail, format="json")
+```
+
+</CodeGroup>
+
+You can also iterate the trail directly:
+
+```python
+trail = manager.get_audit_trail(entity_id="patient_001")
+for entry in trail:
+    print(f"{entry.timestamp.isoformat()} | {entry.author} | {entry.action} | {entry.description}")
+```
+
+### Audit Fields
+
+| Field | Description |
+| ----- | ----------- |
+| `timestamp` | UTC ISO 8601 datetime |
+| `entity_id` | ID of the affected entity or relationship |
+| `author` | User or process that made the change |
+| `action` | `CREATE` / `UPDATE` / `DELETE` / `MERGE` / `ROLLBACK` |
+| `property` | Property name that changed (UPDATE rows only) |
+| `old_value` | Previous value (UPDATE and DELETE rows) |
+| `new_value` | New value (CREATE and UPDATE rows) |
+| `snapshot_id` | ID of the containing snapshot |
+| `checksum` | SHA-256 of the entity state after the change |
+
+### Compliance Coverage
+
+<AccordionGroup>
+  <Accordion title="HIPAA — subject-access requests">
+    Use `get_audit_trail(entity_id="patient_001")` to retrieve every change ever made to a patient entity, then export to JSON for the access request response. The SHA-256 checksum on each entry proves the record has not been altered.
+  </Accordion>
+  <Accordion title="SOX — quarterly reviews">
+    Use `get_audit_trail(from_date=..., to_date=...)` to scope the export to the relevant quarter. Export to CSV for upload to your audit management system. The immutable snapshot chain provides the chain of custody required by SOX Section 404.
+  </Accordion>
+  <Accordion title="GDPR — right to erasure verification">
+    After deleting a data subject's entities, snapshot the graph and diff against the pre-deletion snapshot. `diff.removed_nodes` provides a machine-readable record of exactly what was deleted and when, satisfying Article 17 documentation requirements.
+  </Accordion>
+  <Accordion title="FDA 21 CFR Part 11 — electronic records">
+    Every `ChangeLogEntry` includes `author`, `timestamp`, and `checksum` — the three fields required for a compliant electronic record. `verify_checksum()` provides the tamper-evidence required by 21 CFR § 11.10(e).
+  </Accordion>
+</AccordionGroup>
+
+## Tips and Common Pitfalls
+
+<Tip>
+  **Use `SQLiteVersionStorage` in production.** The default in-memory storage loses all version history when the process exits. Pass `storage_path="versions.db"` to `TemporalVersionManager` or create `SQLiteVersionStorage(db_path="versions.db")` explicitly.
+</Tip>
+
+<Warning>
+  **Snapshot before every destructive operation.** Call `manager.create_snapshot()` before running deduplication, conflict resolution, or merge operations. `rollback()` is only possible if a snapshot exists before the change.
+</Warning>
+
+<Tip>
+  **Use `diff()` for code review and incident investigation.** `manager.diff("v1.0", "v2.0")` produces a human-readable change summary in seconds — faster than comparing raw graph exports. Use it to review what changed before approving a version for production.
+</Tip>
+
+<Tip>
+  **Export audit trails before compliance reviews.** `export_audit_trail("audit.csv", format="csv")` produces a complete tamper-evident record in one call. Schedule this export before quarterly reviews (SOX), regulatory inspections (FDA 21 CFR Part 11), or subject-access requests (GDPR).
+</Tip>
+
+<Tip>
+  **Use `get_audit_trail(from_date=..., to_date=...)` for scoped reviews.** Exporting the full audit trail for a multi-year graph can produce millions of rows. Scope to a time window or entity ID for faster, focused reports.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Provenance" icon="link" href="provenance">
     W3C PROV-O lineage tracking.

docs/reference/conflicts.md

@@ -6,74 +6,222 @@ icon: "triangle-exclamation"
 
 `semantica.conflicts` detects and resolves contradictions when multiple sources disagree on the same fact. It surfaces five conflict types, seven resolution strategies, and generates investigation guides for manual review — so conflicts never silently corrupt your knowledge graph.
 
+## Why Detect Conflicts?
+
+When you ingest data from multiple sources, contradictions are inevitable. One annual report says Apple's revenue was $391B; a financial newswire says $383B. Without conflict detection, both values land in your graph and queries silently return inconsistent answers.
+
+Semantica's conflict detection makes disagreements explicit and actionable:
+
+- **Value conflicts** — SEC says revenue is $391B; Reuters says $383B
+- **Type conflicts** — "Python" is a `ProgrammingLanguage` in one source, a `Snake` species in another
+- **Temporal conflicts** — a CEO had two different employers during overlapping date ranges
+- **Logical conflicts** — an entity simultaneously holds two mutually exclusive properties
+- **Relationship conflicts** — the same relationship has inconsistent cardinality or properties across sources
+
 ## What You Get
 
-- **`ConflictDetector`** — value, type, temporal, logical, and relationship conflict detection
-- **`ConflictResolver`** — 7 resolution strategies including voting, credibility-weighted, and temporal
-- **`SourceTracker`** — track which source each conflicting fact came from, with credibility scores
-- **`ConflictAnalyzer`** — pattern analysis, severity grouping, and trend identification
-- **`InvestigationGuideGenerator`** — auto-generate step-by-step investigation checklists for human review
+<CardGroup cols={2}>
+  <Card title="ConflictDetector" icon="magnifying-glass">
+    Value, type, temporal, logical, and relationship conflict detection across all entity pairs.
+  </Card>
+  <Card title="ConflictResolver" icon="check">
+    7 resolution strategies including voting, credibility-weighted, and temporal preference.
+  </Card>
+  <Card title="SourceTracker" icon="link">
+    Track which source each conflicting fact came from, with per-source credibility scores.
+  </Card>
+  <Card title="ConflictAnalyzer" icon="chart-line">
+    Pattern analysis, severity grouping, source-level statistics, and trend identification.
+  </Card>
+  <Card title="InvestigationGuideGenerator" icon="list-check">
+    Auto-generate step-by-step investigation checklists for human and expert review.
+  </Card>
+  <Card title="Convenience Functions" icon="bolt">
+    `detect_conflicts()` and `resolve_conflicts()` for one-call workflows.
+  </Card>
+</CardGroup>
+
+## Quick Start
+
+<Steps>
+  <Step title="Set credibility scores before ingestion">
+    ```python
+    from semantica.conflicts import SourceTracker
+
+    tracker = SourceTracker()
+    tracker.set_credibility("sec_filings",   0.95)
+    tracker.set_credibility("pubmed",        0.92)
+    tracker.set_credibility("wikipedia",     0.80)
+    tracker.set_credibility("news_articles", 0.65)
+    ```
+  </Step>
+  <Step title="Detect conflicts after building the graph">
+    ```python
+    from semantica.conflicts import ConflictDetector
+
+    detector  = ConflictDetector()
+    conflicts = detector.detect_conflicts(kg)
+    print(f"Found {len(conflicts)} conflicts")
+
+    for conflict in conflicts:
+        print(f"[{conflict.conflict_type}] entity='{conflict.entity_id}'  attr='{conflict.attribute}'")
+        print(f"  Values: {conflict.values}  Severity: {conflict.severity:.2f}")
+    ```
+  </Step>
+  <Step title="Triage by severity">
+    ```python
+    from semantica.conflicts import ConflictAnalyzer
+
+    analyzer    = ConflictAnalyzer()
+    by_severity = analyzer.group_by_severity(conflicts)
+    print(f"Critical: {len(by_severity['critical'])}")
+    print(f"High:     {len(by_severity['high'])}")
+    print(f"Low:      {len(by_severity['low'])}")
+    ```
+  </Step>
+  <Step title="Auto-resolve low-severity, escalate critical">
+    ```python
+    from semantica.conflicts import ConflictResolver, InvestigationGuideGenerator, ResolutionStrategy
+
+    resolver = ConflictResolver(source_tracker=tracker)
+
+    # Auto-resolve low-severity
+    auto_resolved = resolver.resolve_conflicts(
+        by_severity["low"],
+        strategy=ResolutionStrategy.CREDIBILITY_WEIGHTED,
+    )
+
+    # Generate investigation guides for critical conflicts
+    generator = InvestigationGuideGenerator()
+    for conflict in by_severity["critical"]:
+        guide = generator.generate(conflict)
+        print(f"\n{guide.title}")
+        for step in guide.steps:
+            print(f"  [{step.order}] ({step.priority.upper()}) {step.description}")
+    ```
+  </Step>
+</Steps>
 
 ## ConflictDetector
 
 ```python
 from semantica.conflicts import ConflictDetector
 
-detector = ConflictDetector()
+detector  = ConflictDetector()
 conflicts = detector.detect_conflicts(kg)
-
-for conflict in conflicts:
-    print(f"[{conflict.conflict_type}] '{conflict.entity}' — {conflict.attribute}")
-    print(f"  Sources:  {conflict.sources}")
-    print(f"  Severity: {conflict.severity:.2f}")
 ```
 
 ### Detection Types
 
-| Type | What It Detects |
-| ---- | --------------- |
-| `VALUE` | Same entity, same attribute, different values across sources |
-| `TYPE` | Same entity classified as different types in different sources |
-| `TEMPORAL` | Overlapping validity windows with contradictory facts |
-| `LOGICAL` | Facts that violate ontology axioms or SHACL constraints |
-| `RELATIONSHIP` | Inconsistent relationship properties across sources |
+| Type | What It Detects | Example |
+| ---- | --------------- | ------- |
+| `VALUE` | Same entity, same attribute, different values across sources | Revenue $391B vs $383B |
+| `TYPE` | Same entity classified as different types | "Python" as Language vs Snake |
+| `TEMPORAL` | Overlapping validity windows with contradictory facts | CEO at two companies simultaneously |
+| `LOGICAL` | Facts that violate ontology axioms or SHACL constraints | `is_alive=True` but `death_date` set |
+| `RELATIONSHIP` | Inconsistent relationship properties across sources | Edge weight 0.9 vs 0.3 from two sources |
 
 Run targeted detection by type:
 
 ```python
-# Detect all types (default)
+# Detect all types at once (default)
 conflicts = detector.detect_conflicts(kg)
 
-# Detect specific types only
-value_conflicts    = detector.detect_value_conflicts(entities, "name")
+# Detect specific types only — faster for targeted checks
+value_conflicts    = detector.detect_value_conflicts(entities, "revenue")
 type_conflicts     = detector.detect_type_conflicts(entities)
 relation_conflicts = detector.detect_relationship_conflicts(kg)
 ```
 
+**Key behaviours:**
+- Severity scores are computed from the magnitude of disagreement — a $8B revenue discrepancy scores higher than a $1M discrepancy
+- `LOGICAL` conflicts require an ontology or SHACL schema to be loaded; without one, they are not detected
+- Detection runs in O(n·sources) time — it groups by entity+attribute and checks disagreement within each group
+
 ## ConflictResolver
 
 ```python
 from semantica.conflicts import ConflictResolver, ResolutionStrategy
 
 resolver = ConflictResolver()
-results = resolver.resolve_conflicts(conflicts, strategy=ResolutionStrategy.VOTING)
+results  = resolver.resolve_conflicts(conflicts, strategy=ResolutionStrategy.VOTING)
 
 for result in results:
     print(f"Resolved '{result.attribute}' → {result.resolved_value}")
-    print(f"  Strategy: {result.strategy}")
+    print(f"  Strategy: {result.strategy}  Confidence: {result.confidence:.2f}")
 ```
 
-### Resolution Strategies
+### Choosing a Resolution Strategy
 
-| Strategy | Enum | Description |
-|----------|------|-------------|
-| Majority vote | `ResolutionStrategy.VOTING` | Most common value wins |
-| Credibility-weighted | `ResolutionStrategy.CREDIBILITY_WEIGHTED` | Weighted by source credibility score |
-| Most recent | `ResolutionStrategy.MOST_RECENT` | Prefer the most recently updated fact |
-| First seen | `ResolutionStrategy.FIRST_SEEN` | Prefer the first observed value |
-| Highest confidence | `ResolutionStrategy.HIGHEST_CONFIDENCE` | Prefer the fact with the highest confidence score |
-| Manual review | `ResolutionStrategy.MANUAL_REVIEW` | Flag for human review |
-| Expert review | `ResolutionStrategy.EXPERT_REVIEW` | Escalate to a domain expert |
+<Tabs>
+  <Tab title="CREDIBILITY_WEIGHTED (recommended)">
+    Weights each source's value by its assigned credibility score — favors authoritative sources automatically:
+
+    ```python
+    from semantica.conflicts import ConflictResolver, SourceTracker, ResolutionStrategy
+
+    tracker = SourceTracker()
+    tracker.set_credibility("sec_filings",   0.92)
+    tracker.set_credibility("wikipedia",     0.80)
+    tracker.set_credibility("news_articles", 0.65)
+
+    resolver = ConflictResolver(source_tracker=tracker)
+    results  = resolver.resolve_conflicts(
+        conflicts,
+        strategy=ResolutionStrategy.CREDIBILITY_WEIGHTED,
+    )
+    ```
+
+    Best for: sources with known reliability rankings (SEC > blog).
+  </Tab>
+  <Tab title="VOTING">
+    Majority vote — most common value across sources wins:
+
+    ```python
+    results = resolver.resolve_conflicts(conflicts, strategy=ResolutionStrategy.VOTING)
+    ```
+
+    Best for: 3+ sources with roughly equal credibility. When all sources have identical credibility scores, `CREDIBILITY_WEIGHTED` behaves identically to `VOTING`.
+  </Tab>
+  <Tab title="MOST_RECENT / FIRST_SEEN">
+    ```python
+    # Most recent source wins — for fast-changing facts
+    results = resolver.resolve_conflicts(conflicts, strategy=ResolutionStrategy.MOST_RECENT)
+
+    # First seen wins — for stable facts (founding date, original name)
+    results = resolver.resolve_conflicts(conflicts, strategy=ResolutionStrategy.FIRST_SEEN)
+    ```
+  </Tab>
+  <Tab title="MANUAL_REVIEW / EXPERT_REVIEW">
+    ```python
+    from semantica.conflicts import InvestigationGuideGenerator
+
+    # Flag for human review — use with InvestigationGuideGenerator
+    results   = resolver.resolve_conflicts(conflicts, strategy=ResolutionStrategy.MANUAL_REVIEW)
+    generator = InvestigationGuideGenerator()
+
+    for conflict in conflicts:
+        guide = generator.generate(conflict)
+        print(f"{guide.title}")
+        for step in guide.steps:
+            print(f"  [{step.order}] {step.description}")
+    ```
+
+    Best for: high-stakes decisions (severity > 0.8), regulated data (HIPAA/SOX), and domain-specific ambiguity.
+  </Tab>
+  <Tab title="Strategy Comparison">
+
+    | Strategy | Enum | When to Use |
+    | -------- | ---- | ----------- |
+    | Majority vote | `VOTING` | 3+ sources with roughly equal credibility |
+    | Credibility-weighted | `CREDIBILITY_WEIGHTED` | Sources have different authority levels |
+    | Most recent | `MOST_RECENT` | Fast-changing facts: stock price, headcount, status |
+    | First seen | `FIRST_SEEN` | Stable facts: founding date, original name |
+    | Highest confidence | `HIGHEST_CONFIDENCE` | Extraction pipeline outputs confidence scores |
+    | Manual review | `MANUAL_REVIEW` | High-stakes decisions, regulated data |
+    | Expert review | `EXPERT_REVIEW` | Domain-specific ambiguity — escalate to a specialist |
+  </Tab>
+</Tabs>
 
 Use the convenience aliases for shorter code:
 
@@ -83,92 +231,158 @@ from semantica.conflicts import voting, credibility_weighted, most_recent, highe
 results = resolver.resolve_conflicts(conflicts, strategy=voting)
 ```
 
-## Source Credibility Scoring
-
-Assign credibility weights per source so `CREDIBILITY_WEIGHTED` resolution favors authoritative sources:
+## SourceTracker
 
 ```python
 from semantica.conflicts import SourceTracker
+from datetime import datetime
 
 tracker = SourceTracker()
-tracker.set_credibility("pubmed",     0.95)
-tracker.set_credibility("wikipedia",  0.80)
-tracker.set_credibility("user_input", 0.60)
+tracker.set_credibility("sec_10k",   0.92)
+tracker.set_credibility("wikipedia", 0.80)
 
-resolver = ConflictResolver(source_tracker=tracker)
-results = resolver.resolve_conflicts(
-    conflicts,
-    strategy=ResolutionStrategy.CREDIBILITY_WEIGHTED
+tracker.track_property_source(
+    entity_id="apple_inc",
+    property_name="revenue",
+    value="$391B",
+    source="sec_10k_2023",
+    timestamp=datetime(2024, 1, 26),
 )
-```
 
-`SourceTracker` also builds full traceability chains:
-
-```python
-from semantica.conflicts import SourceTracker
-
-tracker = SourceTracker()
-tracker.track_entity_source("apple_inc", "crunchbase")
-tracker.track_property_source("apple_inc", "revenue", "annual_report_2023")
+sources = tracker.get_property_sources("apple_inc", "revenue")
+for s in sources:
+    print(f"{s.source}: {s.value}  (credibility: {s.credibility:.2f})")
 
 chain = tracker.get_traceability_chain("apple_inc")
 ```
 
-## ConflictAnalyzer
+**Key behaviours:**
+- Credibility scores default to 0.50 for any source not explicitly set
+- `SourceTracker` stores property-level provenance — so you can trace exactly which source contributed each value
 
-Identify patterns and trends across large conflict sets:
+## ConflictAnalyzer
 
 ```python
 from semantica.conflicts import ConflictAnalyzer
 
 analyzer = ConflictAnalyzer()
 
-# Detect recurring patterns
-patterns = analyzer.identify_patterns(conflicts)
-for pattern in patterns:
-    print(f"Pattern: {pattern.type} — {pattern.frequency} occurrences")
-
-# Group by severity
+patterns    = analyzer.identify_patterns(conflicts)
 by_severity = analyzer.group_by_severity(conflicts)
-print(f"Critical: {len(by_severity['critical'])}")
-print(f"High:     {len(by_severity['high'])}")
-print(f"Low:      {len(by_severity['low'])}")
+source_stats = analyzer.analyze_sources(conflicts)
+trends      = analyzer.analyze_trends(conflicts, time_window="30d")
 
-# Trend analysis over time
-trends = analyzer.analyze_trends(conflicts, time_window="30d")
+print(f"Trend direction: {trends['direction']}")   # "increasing" | "stable" | "decreasing"
+print(f"Change:          {trends['change_pct']:.1f}%")
 ```
 
+**Key behaviours:**
+- `identify_patterns()` groups conflicts by attribute name and type — use it to find systemic data quality issues
+- `analyze_sources()` flags sources with disproportionate conflict rates — a signal that a source's pipeline needs review
+- `analyze_trends()` compares conflict counts across time windows — a rising trend means a data source is degrading
+
 ## InvestigationGuideGenerator
 
-Auto-generate human-readable investigation guides for conflicts that can't be automatically resolved:
+Auto-generate human-readable investigation checklists for conflicts requiring manual or expert review:
 
 ```python
-from semantica.conflicts import InvestigationGuideGenerator, InvestigationGuide
+from semantica.conflicts import InvestigationGuideGenerator
 
 generator = InvestigationGuideGenerator()
-guide: InvestigationGuide = generator.generate(conflict)
+guide     = generator.generate(conflict)
+
+print(f"Title:   {guide.title}")
+print(f"Context: {guide.context}")
 
-print(guide.title)
-print(guide.context)
 for step in guide.steps:
-    print(f"  [{step.order}] {step.description}")
-    print(f"       Check: {step.check}")
+    print(f"  [{step.order}] ({step.priority.upper()}) {step.description}")
+    print(f"             → Verify: {step.check}")
 ```
 
-## Convenience Functions
+## Schemas
+
+<AccordionGroup>
+  <Accordion title="Conflict schema">
 
 ```python
-from semantica.conflicts import (
-    detect_conflicts, resolve_conflicts, analyze_conflicts,
-    track_sources, generate_investigation_guide
-)
-
-conflicts = detect_conflicts(entities, method="value")
-resolved  = resolve_conflicts(conflicts, strategy="voting")
-analysis  = analyze_conflicts(conflicts, method="pattern")
-guide     = generate_investigation_guide(conflicts[0])
+@dataclass
+class Conflict:
+    id:            str
+    entity_id:     str            # the entity involved
+    attribute:     str            # the conflicting property name
+    values:        List[str]      # conflicting values (one per source)
+    sources:       List[str]      # source IDs for each value
+    conflict_type: ConflictType   # VALUE | TYPE | TEMPORAL | LOGICAL | RELATIONSHIP
+    severity:      float          # 0.0 (minor) to 1.0 (critical)
+    confidence:    float          # detection confidence 0–1
+    detected_at:   datetime
+    metadata:      Dict[str, Any]
 ```
 
+  </Accordion>
+  <Accordion title="ConflictType enum">
+
+```python
+from semantica.conflicts import ConflictType
+
+ConflictType.VALUE_CONFLICT         # revenue is $391B in source A, $383B in source B
+ConflictType.TYPE_CONFLICT          # "Apple" is ORGANIZATION in one source, PRODUCT in another
+ConflictType.TEMPORAL_CONFLICT      # overlapping validity windows with contradictory states
+ConflictType.LOGICAL_CONFLICT       # fact violates an ontology axiom or SHACL constraint
+ConflictType.RELATIONSHIP_CONFLICT  # inconsistent relationship properties across sources
+```
+
+  </Accordion>
+  <Accordion title="InvestigationGuide and InvestigationStep schemas">
+
+```python
+@dataclass
+class InvestigationGuide:
+    title:    str                      # human-readable title for the conflict
+    context:  str                      # summary of the disagreement
+    steps:    List[InvestigationStep]  # ordered checklist for the reviewer
+
+@dataclass
+class InvestigationStep:
+    order:       int
+    description: str   # what to do
+    check:       str   # specific fact or document to verify
+    priority:    str   # "high" | "medium" | "low"
+```
+
+  </Accordion>
+</AccordionGroup>
+
+## Tips and Common Pitfalls
+
+<Warning>
+  **Detect before you merge, not after.** Run conflict detection on raw entity data before deduplication and graph construction. Detecting conflicts in a live graph that already contains merged entities is harder — you lose the original source attribution.
+</Warning>
+
+<Warning>
+  **Always set credibility scores.** The default credibility is 0.50 for all sources. Without explicit scores, `CREDIBILITY_WEIGHTED` behaves identically to `VOTING`. The power of this strategy is in the differentiation.
+</Warning>
+
+<Tip>
+  **Don't auto-resolve everything.** Use `MANUAL_REVIEW` for conflicts with severity > 0.8 — high severity means the disagreement is large and the stakes of getting it wrong are high.
+</Tip>
+
+<Warning>
+  **LOGICAL conflicts need a schema.** `detect_type_conflicts()` and `LOGICAL` detection only work if an OWL ontology or SHACL schema is loaded. Without one, `detect_conflicts()` will skip those types silently.
+</Warning>
+
+<Tip>
+  **Use `analyze_sources()` to identify bad data feeds.** A single source causing 80% of your conflicts is a data quality problem upstream, not a conflict to resolve record by record. Flag it and investigate the source pipeline.
+</Tip>
+
+<Tip>
+  **Severity is relative, not absolute.** A 0.5 severity score on a $1B revenue discrepancy and on a minor label difference both score 0.5 — the number reflects the disagreement structure, not the business impact. Domain context determines what to prioritize.
+</Tip>
+
+<Tip>
+  **Combine with provenance.** The `SourceTracker` feeds directly into the [Provenance](provenance) module's audit trail. If you need to explain how a resolved value was chosen, provenance records give you the full chain.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Deduplication" icon="copy" href="deduplication">
     Resolve duplicate entities before conflict detection.

docs/reference/context.md

@@ -8,81 +8,87 @@ icon: "brain"
 
 ## What You Get
 
-- **`AgentContext`** — unified interface for memory, decision tracking, and graph-backed retrieval
-- **`ContextGraph`** — persistent knowledge graph with centrality analysis, community detection, and decision management
-- **`AgentMemory`** — low-level embedding-backed memory with TTL, tagging, and importance scoring
-- **`DecisionRecorder`** — records decisions with causal chains, confidence scores, and outcome tracking
-- **`CausalAnalyzer`** — traces downstream impact of any decision
-- **`PolicyEngine`** — validates decisions against configurable rules before they're recorded
+<CardGroup cols={2}>
+  <Card title="AgentContext" icon="brain">
+    Unified interface for memory, decision tracking, and graph-backed retrieval.
+  </Card>
+  <Card title="ContextGraph" icon="diagram-project">
+    Persistent knowledge graph with centrality analysis, community detection, and decision management.
+  </Card>
+  <Card title="AgentMemory" icon="database">
+    Embedding-backed memory with TTL, tagging, and importance scoring.
+  </Card>
+  <Card title="DecisionRecorder" icon="list-check">
+    Records decisions with causal chains, confidence scores, and outcome tracking.
+  </Card>
+  <Card title="PolicyEngine" icon="shield-check">
+    Validates decisions against configurable rules before they're recorded.
+  </Card>
+  <Card title="EntityLinker" icon="link">
+    Maps entity mentions to canonical URIs — prevents "Apple", "Apple Inc.", and "AAPL" from becoming three separate nodes.
+  </Card>
+</CardGroup>
 
 <img src="/assets/img/diagrams/agent-context-flow.svg" alt="AgentContext hub: AI Agent calls store/retrieve against VectorStore and record_decision against ContextGraph" style={{ width: '100%', borderRadius: '12px', margin: '0 0 24px' }} />
 
+## Quick Start
+
+<Steps>
+  <Step title="Initialize the agent context">
+    ```python
+    from semantica.context import AgentContext, ContextGraph
+    from semantica.vector_store import VectorStore
+
+    context = AgentContext(
+        vector_store=VectorStore(backend="faiss", dimension=768, index_path="context.faiss"),
+        knowledge_graph=ContextGraph(advanced_analytics=True),
+        decision_tracking=True,
+    )
+    ```
+  </Step>
+  <Step title="Store facts and retrieve by semantic similarity">
+    ```python
+    memory_id = context.store(
+        "GPT-4 outperforms GPT-3.5 on reasoning benchmarks by 40%",
+        metadata={"source": "openai_blog", "date": "2024-01"}
+    )
+
+    results = context.retrieve("LLM benchmark comparisons", top_k=5)
+    for r in results:
+        print(f"{r['content']}  (score: {r['score']:.3f})")
+    ```
+  </Step>
+  <Step title="Record decisions with full provenance">
+    ```python
+    decision_id = context.record_decision(
+        category="model_selection",
+        scenario="Choose LLM for production reasoning pipeline",
+        reasoning="GPT-4 benchmark advantage justifies 3x cost increase",
+        outcome="selected_gpt4",
+        confidence=0.91,
+    )
+    ```
+  </Step>
+  <Step title="Find precedents before new decisions">
+    ```python
+    # Search past decisions — prevents contradictory choices across runs
+    precedents = context.find_precedents("model selection reasoning", limit=5)
+
+    for p in precedents:
+        print(f"[{p.category}] {p.outcome}  (similarity: {p.similarity:.2f})")
+        print(f"  Reasoning: {p.reasoning}")
+
+    # Analyze downstream impact of a past decision
+    influence = context.analyze_decision_influence(decision_id)
+    print(f"Decisions influenced: {len(influence.downstream_decisions)}")
+    ```
+  </Step>
+</Steps>
+
 ## AgentContext
 
 The main entry point. Wraps memory, graph, and decision tracking behind a single API.
 
-```python
-from semantica.context import AgentContext, ContextGraph
-from semantica.vector_store import VectorStore
-
-context = AgentContext(
-    vector_store=VectorStore(backend="faiss", dimension=768),
-    knowledge_graph=ContextGraph(advanced_analytics=True),
-    decision_tracking=True,
-)
-```
-
-### Store and Retrieve Memories
-
-```python
-# Store a fact — embedded and indexed automatically
-memory_id = context.store(
-    "GPT-4 outperforms GPT-3.5 on reasoning benchmarks by 40%",
-    metadata={"source": "openai_blog", "date": "2024-01"}
-)
-
-# Retrieve by semantic similarity
-results = context.retrieve("LLM benchmark comparisons", top_k=5)
-for r in results:
-    print(f"{r['content']}  (score: {r['score']:.3f})")
-```
-
-### Record and Search Decisions
-
-```python
-decision_id = context.record_decision(
-    category="model_selection",
-    scenario="Choose LLM for production reasoning pipeline",
-    reasoning="GPT-4 benchmark advantage justifies 3x cost increase",
-    outcome="selected_gpt4",
-    confidence=0.91,
-)
-
-# Find similar past decisions — prevents inconsistent choices
-precedents = context.find_precedents("model selection reasoning", limit=5)
-
-# Analyze downstream impact
-influence = context.analyze_decision_influence(decision_id)
-print(f"Decisions influenced: {len(influence.downstream_decisions)}")
-```
-
-### Multi-Hop GraphRAG
-
-```python
-from semantica.llms import Groq
-
-llm    = Groq(model="llama-3.3-70b-versatile")
-result = context.query_with_reasoning(
-    query="What technologies have we chosen and why?",
-    llm_provider=llm,
-    max_hops=2,
-)
-
-print(result["response"])
-for step in result["reasoning_path"]:
-    print(f"  {step}")
-```
-
 ### Constructor Parameters
 
 | Parameter | Type | Default | Description |
@@ -106,6 +112,23 @@ for step in result["reasoning_path"]:
 | `query_with_reasoning(query, llm_provider, max_hops)` | `Dict` | GraphRAG with multi-hop traversal |
 | `get_context_insights()` | `Dict` | Analytics summary |
 
+### Multi-Hop GraphRAG
+
+```python
+from semantica.llms import Groq
+
+llm    = Groq(model="llama-3.3-70b-versatile")
+result = context.query_with_reasoning(
+    query="What technologies have we chosen and why?",
+    llm_provider=llm,
+    max_hops=2,
+)
+
+print(result["response"])
+for step in result["reasoning_path"]:
+    print(f"  {step}")
+```
+
 ## ContextGraph
 
 The knowledge graph backing `AgentContext`. Can be used standalone for relationship modelling.
@@ -115,12 +138,10 @@ from semantica.context import ContextGraph
 
 graph = ContextGraph(advanced_analytics=True)
 
-# Add nodes and edges
 graph.add_node("Python",  "language",  properties={"paradigm": "multi-paradigm"})
 graph.add_node("FastAPI", "framework", properties={"language": "Python"})
 graph.add_edge("Python", "FastAPI", "enables")
 
-# Decision management
 decision_id = graph.add_decision_simple(
     category="technology_choice",
     scenario="Web API framework selection",
@@ -144,7 +165,134 @@ chain   = graph.trace_decision_chain(decision_id)
 | `community_detection` | `bool` | `False` | Louvain community clustering |
 | `node_embeddings` | `bool` | `False` | Node2Vec embeddings for structural similarity |
 
-## Decision Data Structure
+### ContextGraph — Full Method Reference
+
+| Method | Returns | Description |
+| ------ | ------- | ----------- |
+| `add_node(id, label, properties)` | `None` | Add a node to the context graph |
+| `add_edge(source, target, rel_type, properties)` | `None` | Add a directed edge |
+| `query_neighbors(node_id, depth)` | `List[ContextNode]` | BFS neighbors up to given depth |
+| `record_decision(...)` | `str` (decision_id) | Add decision node with causal edges |
+| `find_precedents(category, limit)` | `List[Decision]` | Recent decisions in this category |
+| `find_precedents_by_scenario(scenario, limit)` | `List[Decision]` | Semantically similar past scenarios |
+| `analyze_decision_impact(decision_id)` | `Dict` | Downstream nodes influenced |
+| `trace_decision_chain(decision_id)` | `CausalChain` | Full causality tree |
+| `get_decision_insights()` | `Dict` | Aggregate stats across all decisions |
+| `trace_decision_causality(decision_id)` | `CausalChain` | Alias for `trace_decision_chain` |
+
+## AgentMemory (Low-Level)
+
+For fine-grained control over memory storage, TTL, and importance scoring:
+
+```python
+from semantica.context import AgentMemory
+from semantica.vector_store import VectorStore
+
+memory = AgentMemory(
+    vector_store=VectorStore(backend="faiss", dimension=768),
+    capacity=10_000,       # max memories before oldest are evicted
+    ttl_days=90,           # memories older than this are auto-expired (None = never)
+)
+
+memory_id = memory.store(
+    "Critical compliance rule: all trades must be pre-approved",
+    importance=0.95,
+    tags=["compliance", "trading"],
+)
+
+results = memory.retrieve(
+    query="trade approval requirements",
+    top_k=5,
+    min_importance=0.5,
+    tags=["compliance"],
+)
+
+memory.update(memory_id, importance=1.0)
+memory.forget(memory_id)
+all_memories = memory.get_all()
+```
+
+| Parameter | Type | Default | Description |
+| --------- | ---- | ------- | ----------- |
+| `vector_store` | `VectorStore` | required | Embedding backend for semantic retrieval |
+| `capacity` | `int` | `1000` | Max items before LRU eviction |
+| `ttl_days` | `Optional[int]` | `None` | Days before automatic expiry; `None` = keep forever |
+
+## PolicyEngine
+
+Validate decisions against configurable rules before they're committed:
+
+```python
+from semantica.context import PolicyEngine
+
+policy = PolicyEngine()
+policy.add_rule("confidence_threshold", lambda d: d.confidence >= 0.7)
+policy.add_rule("requires_reasoning",   lambda d: len(d.reasoning) >= 20)
+
+is_valid, violations = policy.validate(decision_data)
+
+if is_valid:
+    context.record_decision(**decision_data)
+else:
+    # Create approval chain for manual review
+    chain = policy.create_approval_chain(
+        decision_data,
+        approvers=["manager@company.com", "compliance@company.com"],
+    )
+    print(f"Approval chain created: {chain.chain_id}")
+```
+
+## EntityLinker
+
+Maps extracted entity mentions to canonical URIs — essential for cross-document entity resolution:
+
+```python
+from semantica.context import EntityLinker
+
+linker = EntityLinker()
+
+entities = [
+    {"text": "Apple Inc.", "type": "ORGANIZATION"},
+    {"text": "Apple",      "type": "ORGANIZATION"},
+    {"text": "AAPL",       "type": "ORGANIZATION"},
+]
+linked = linker.link_entities(entities, sources=["reuters", "sec_filings"])
+
+for e in linked:
+    print(f"{e.text} → {e.canonical_form}  ({e.uri})")
+    print(f"  confidence: {e.confidence:.2f}, sources: {e.sources}")
+```
+
+## ContextRetriever
+
+Hybrid retrieval combining vector similarity, graph traversal, and memory — gives richer context than pure vector search:
+
+```python
+from semantica.context import ContextRetriever
+
+retriever = ContextRetriever(
+    vector_store=vector_store,
+    context_graph=context_graph,
+    agent_memory=memory,
+)
+
+results = retriever.retrieve(
+    query="What decisions were made about cloud infrastructure?",
+    top_k=10,
+    vector_weight=0.5,    # weight of vector similarity results
+    graph_weight=0.3,     # weight of graph-traversal results
+    memory_weight=0.2,    # weight of agent memory results
+    filters={"category": "infrastructure"},
+)
+
+for r in results:
+    print(f"[{r['source']}] score={r['score']:.3f}: {r['content'][:80]}")
+```
+
+## Data Structures
+
+<AccordionGroup>
+  <Accordion title="Decision schema">
 
 ```python
 @dataclass
@@ -162,84 +310,157 @@ class Decision:
     causal_chain:   List[str]   # IDs of related decisions
 ```
 
-## AgentMemory (Low-Level)
-
-For fine-grained control over memory storage, TTL, and importance scoring:
+  </Accordion>
+  <Accordion title="Precedent schema">
 
 ```python
-from semantica.context import AgentMemory
-
-memory = AgentMemory(
-    vector_store=VectorStore(backend="faiss", dimension=768),
-    max_memories=10_000,
-    ttl_days=90,
-)
-
-memory.store("Important fact", importance=0.9, tags=["compliance"])
-results = memory.retrieve("fact query", top_k=5, min_importance=0.5)
-memory.forget(memory_id)
+@dataclass
+class Precedent:
+    decision_id:    str
+    similarity:     float        # 0–1 match score to current scenario
+    category:       str
+    scenario:       str
+    outcome:        str
+    reasoning:      str
+    confidence:     float
+    timestamp:      datetime
 ```
 
-## PolicyEngine
-
-Validate decisions against configurable rules before they're committed:
+  </Accordion>
+  <Accordion title="PolicyException schema">
 
 ```python
-from semantica.context import PolicyEngine
-
-policy = PolicyEngine()
-policy.add_rule("confidence_threshold", lambda d: d.confidence >= 0.7)
-policy.add_rule("requires_reasoning",   lambda d: len(d.reasoning) >= 20)
-
-# Validate before recording
-is_valid, violations = policy.validate(decision_data)
-if is_valid:
-    context.record_decision(**decision_data)
+@dataclass
+class PolicyException:
+    exception_id:   str
+    policy_rule:    str          # name of the rule that was violated
+    decision_id:    str          # the decision that triggered the exception
+    justification:  str          # why the exception was granted
+    approved_by:    str          # approver identity
+    timestamp:      datetime
+    expiry:         Optional[datetime]
 ```
 
+  </Accordion>
+  <Accordion title="ApprovalChain schema">
+
+```python
+@dataclass
+class ApprovalChain:
+    chain_id:       str
+    decision_id:    str
+    steps:          List[ApprovalStep]
+    status:         str          # "pending" | "approved" | "rejected"
+    created_at:     datetime
+
+@dataclass
+class ApprovalStep:
+    step_id:        str
+    approver:       str
+    required:       bool
+    status:         str          # "pending" | "approved" | "rejected"
+    comment:        Optional[str]
+    timestamp:      Optional[datetime]
+```
+
+  </Accordion>
+  <Accordion title="LinkedEntity schema">
+
+```python
+@dataclass
+class LinkedEntity:
+    text:           str
+    canonical_form: str          # normalized primary name
+    uri:            str          # e.g. "http://dbpedia.org/resource/Apple_Inc."
+    confidence:     float
+    sources:        List[str]    # source documents that mention this entity
+    aliases:        List[str]    # all observed surface forms
+```
+
+  </Accordion>
+</AccordionGroup>
+
 ## Real-World Patterns
 
-### Healthcare — Treatment Decisions
+<Tabs>
+  <Tab title="Healthcare — Treatment Decisions">
+    ```python
+    from semantica.context import AgentContext
+    from semantica.vector_store import VectorStore
 
-```python
-health_agent = AgentContext(
-    vector_store=VectorStore(backend="faiss", dimension=768),
-    decision_tracking=True,
-)
+    health_agent = AgentContext(
+        vector_store=VectorStore(backend="faiss", dimension=768),
+        decision_tracking=True,
+    )
 
-health_agent.store("Patient has hypertension, type 2 diabetes")
-health_agent.store("Patient allergic to penicillin — verified 2024-01")
+    health_agent.store("Patient has hypertension, type 2 diabetes")
+    health_agent.store("Patient allergic to penicillin — verified 2024-01")
 
-decision_id = health_agent.record_decision(
-    category="treatment_plan",
-    scenario="Hypertension with comorbid diabetes",
-    reasoning="ACE inhibitors are renoprotective in diabetic patients — preferred over beta blockers",
-    outcome="prescribed_lisinopril",
-    confidence=0.91,
-)
+    decision_id = health_agent.record_decision(
+        category="treatment_plan",
+        scenario="Hypertension with comorbid diabetes",
+        reasoning="ACE inhibitors are renoprotective in diabetic patients — preferred over beta blockers",
+        outcome="prescribed_lisinopril",
+        confidence=0.91,
+    )
 
-# Check for similar cases
-precedents = health_agent.find_precedents("hypertension diabetes", limit=5)
-```
+    precedents = health_agent.find_precedents("hypertension diabetes", limit=5)
+    for p in precedents:
+        print(f"Past decision: {p.outcome}  (similarity: {p.similarity:.2f})")
+    ```
+  </Tab>
+  <Tab title="Finance — Loan Decisions">
+    ```python
+    from semantica.context import AgentContext
+    from semantica.vector_store import VectorStore
 
-### Finance — Loan Decisions
+    loan_agent = AgentContext(
+        vector_store=VectorStore(backend="faiss", dimension=768),
+        decision_tracking=True,
+    )
 
-```python
-loan_agent = AgentContext(
-    vector_store=VectorStore(backend="faiss", dimension=768),
-    decision_tracking=True,
-)
+    loan_agent.store("Applicant: credit score 750, DTI 28%, stable employment 4yr")
 
-loan_agent.store("Applicant: credit score 750, DTI 28%, stable employment 4yr")
+    decision_id = loan_agent.record_decision(
+        category="loan_approval",
+        scenario="First-time homebuyer — 30yr fixed, 20% down",
+        reasoning="Credit score above threshold, DTI within limits, stable income verified",
+        outcome="approved_300k",
+        confidence=0.94,
+    )
+    ```
+  </Tab>
+</Tabs>
 
-decision_id = loan_agent.record_decision(
-    category="loan_approval",
-    scenario="First-time homebuyer — 30yr fixed, 20% down",
-    reasoning="Credit score above threshold, DTI within limits, stable income verified",
-    outcome="approved_300k",
-    confidence=0.94,
-)
-```
+## Tips and Common Pitfalls
+
+<Warning>
+  **Persist your vector store between runs.** Use `VectorStore(backend="faiss", index_path="context.faiss")` — without a path, the FAISS index lives in memory and is lost on shutdown. An agent that forgets everything on restart isn't an agent.
+</Warning>
+
+<Warning>
+  **Enable `decision_tracking=True` from the start.** Adding it retroactively means historical decisions aren't linked to the causal chain — you lose the ability to trace how one decision influenced later ones. Enable it at agent initialization, even if you're not using it immediately.
+</Warning>
+
+<Tip>
+  **Use `find_precedents()` before every significant decision.** This is how the context module prevents agents from making contradictory choices across runs. If precedents exist, surface them to the LLM as context — "we chose X for similar reasons before."
+</Tip>
+
+<Tip>
+  **Set `ttl_days` to avoid memory bloat.** Without TTL, `AgentMemory` accumulates indefinitely. For operational agents, 30–90 day TTL keeps memory relevant to current context. Compliance-critical agents may need `ttl_days=None` (keep forever) with explicit archival.
+</Tip>
+
+<Warning>
+  **Use `PolicyEngine` before recording irreversible decisions.** Decisions recorded with `record_decision()` become part of the causal chain immediately. If you need a human approval gate, validate first with `policy.validate()` and create an `ApprovalChain` — don't record until approved.
+</Warning>
+
+<Tip>
+  **`ContextRetriever` is richer than direct vector search.** The three-channel fusion (vector + graph + memory) surfaces results that pure vector search misses — especially for decisions with complex causal relationships. Use it when you need comprehensive context assembly, not just semantic similarity.
+</Tip>
+
+<Tip>
+  **`EntityLinker` prevents entity proliferation.** Without it, "Apple", "Apple Inc.", and "AAPL" land as three separate nodes in `ContextGraph`. Run `EntityLinker` on mentions before storing them to maintain a clean, canonical graph.
+</Tip>
 
 <CardGroup cols={2}>
   <Card title="Vector Store" icon="database" href="vector_store">

docs/reference/core.md

@@ -8,16 +8,76 @@ icon: "gear"
 
 ## What You Get
 
-- **`Semantica`** — orchestration class for coordinating complex multi-module workflows
-- **`ConfigManager`** — unified config loading, merging, and validation with environment variable overrides
-- **`LifecycleManager`** — startup/shutdown hooks and component health monitoring
-- **`PluginRegistry`** — dynamic plugin discovery, registration, and loading
-- **`MethodRegistry`** — register and dispatch custom orchestration methods
+<CardGroup cols={2}>
+  <Card title="Semantica" icon="gear">
+    Orchestration class for coordinating complex multi-module workflows and full KG construction pipelines.
+  </Card>
+  <Card title="ConfigManager" icon="sliders">
+    Unified config loading, merging, and validation with environment variable overrides.
+  </Card>
+  <Card title="LifecycleManager" icon="rotate">
+    Startup/shutdown hooks with priority ordering and component health monitoring.
+  </Card>
+  <Card title="PluginRegistry" icon="plug">
+    Dynamic plugin discovery, registration, loading, and unloading.
+  </Card>
+  <Card title="MethodRegistry" icon="list">
+    Register and dispatch custom orchestration methods by name.
+  </Card>
+  <Card title="Config Class" icon="file-code">
+    Live configuration state — dot-notation access, update, validate, and serialize.
+  </Card>
+</CardGroup>
 
 <Tip>
   **Use individual modules directly** for the vast majority of use cases. Use the `Semantica` orchestration class only when you need application-level lifecycle management or a plugin system.
 </Tip>
 
+## Quick Start
+
+<Steps>
+  <Step title="Load configuration">
+    ```python
+    from semantica.core import ConfigManager
+
+    manager = ConfigManager()
+    config  = manager.load_from_file("config.yaml")
+
+    # Override one key at runtime
+    config.set("processing.batch_size", 64)
+    ```
+  </Step>
+  <Step title="Initialize the framework">
+    ```python
+    from semantica.core import Semantica
+
+    framework = Semantica(config=config)
+    framework.initialize()
+
+    status = framework.get_status()
+    print(f"State: {status['state']}")   # → "READY"
+    ```
+  </Step>
+  <Step title="Build a knowledge base">
+    ```python
+    result = framework.build_knowledge_base(
+        sources=["doc1.pdf", "doc2.docx"],
+        embeddings=True,
+        graph=True,
+    )
+    ```
+  </Step>
+  <Step title="Shut down gracefully">
+    ```python
+    # Always shut down in a finally block
+    try:
+        result = framework.build_knowledge_base(sources)
+    finally:
+        framework.shutdown(graceful=True)
+    ```
+  </Step>
+</Steps>
+
 ## Semantica (Orchestration)
 
 High-level entry point that coordinates the full KG construction pipeline:
@@ -26,7 +86,7 @@ High-level entry point that coordinates the full KG construction pipeline:
 from semantica.core import Semantica, ConfigManager
 
 config_manager = ConfigManager()
-config = config_manager.load_from_file("config.yaml")
+config         = config_manager.load_from_file("config.yaml")
 
 framework = Semantica(config=config)
 framework.initialize()
@@ -43,8 +103,6 @@ finally:
     framework.shutdown(graceful=True)
 ```
 
-### Core Methods
-
 | Method | Description |
 | ------ | ----------- |
 | `initialize()` | Initialize all framework components |
@@ -61,7 +119,7 @@ Centralized config loading with deep-merge and environment variable overrides:
 from semantica.core import ConfigManager
 
 manager = ConfigManager()
-config = manager.load_from_file("config.yaml")
+config  = manager.load_from_file("config.yaml")
 
 # Merge base config with environment-specific overrides
 merged = manager.merge_configs(
@@ -69,44 +127,45 @@ merged = manager.merge_configs(
     manager.load_from_file("prod.yaml"),
 )
 
-# Nested key access with dot notation
+# Nested dot-notation access
 batch_size = config.get("processing.batch_size", default=16)
 config.set("processing.batch_size", 64)
+config.update({"quality": {"min_confidence": 0.75}}, merge=True)
 config.validate()
+
+config_dict = config.to_dict()
 ```
 
-### YAML Configuration
+### Config Section Reference
 
-```yaml
-llm_provider:
-  name: openai
-  model: gpt-4o
-  api_key: ${OPENAI_API_KEY}
+| Section | Key Fields | Description |
+| ------- | ---------- | ----------- |
+| `llm_provider` | `name`, `model`, `api_key`, `base_url` | LLM used for extraction and reasoning |
+| `embedding_model` | `provider`, `model`, `dimension`, `device` | Embedding provider and model |
+| `vector_store` | `backend`, `dimension`, `index_type` | Vector storage backend |
+| `graph_db` | `backend`, `uri`, `user`, `password` | Graph database connection |
+| `processing` | `batch_size`, `max_workers`, `chunk_size` | Parallelism and batching |
+| `pipeline` | `retry_max`, `backoff`, `failure_strategy` | Pipeline retry and failure policy |
+| `logging` | `level`, `format`, `file` | Logging configuration |
+| `quality` | `min_confidence`, `dedup_threshold` | Quality thresholds |
+| `security` | `redact_pii`, `allowed_domains` | Security and compliance settings |
+| `custom` | any key | User-defined extension settings |
 
-processing:
-  batch_size: 32
-  max_workers: 4
+### Environment Variable Overrides
 
-quality:
-  min_confidence: 0.7
-
-logging:
-  level: INFO
-```
-
-Environment variable overrides (prefix `SEMANTICA_`):
+Any config key can be overridden with a `SEMANTICA_` prefix using double underscores for nesting:
 
 ```bash
-export SEMANTICA_PROCESSING_BATCH_SIZE=64
-export SEMANTICA_LOG_LEVEL=DEBUG
+export SEMANTICA_PROCESSING__BATCH_SIZE=64
+export SEMANTICA_LLM_PROVIDER__MODEL=gpt-4o
+export SEMANTICA_LOGGING__LEVEL=DEBUG
+export SEMANTICA_QUALITY__MIN_CONFIDENCE=0.8
 ```
 
 ## LifecycleManager
 
 Manages framework state with a defined state machine and ordered startup/shutdown hooks:
 
-**State machine:** `UNINITIALIZED` → `INITIALIZING` → `READY` → `RUNNING` → `STOPPING` → `STOPPED`
-
 ```python
 from semantica.core import LifecycleManager
 
@@ -139,7 +198,7 @@ manager.shutdown(graceful=True)
 
 ## PluginRegistry
 
-Register custom components that participate in the full pipeline — provenance tracking, retry policies, and parallel execution included:
+Register custom components that participate in the full pipeline:
 
 ```python
 from semantica.core import PluginRegistry
@@ -152,13 +211,23 @@ class MyPlugin:
         return {"processed": True}
 
 registry = PluginRegistry(plugin_paths=["./plugins"])
-registry.register_plugin("my_plugin", MyPlugin, version="1.0.0")
+registry.register_plugin(
+    "my_plugin", MyPlugin,
+    version="1.0.0",
+    description="Custom domain extractor",
+    author="team@example.com",
+    capabilities=["extract"],
+)
 
 plugin = registry.load_plugin("my_plugin", api_key="xxx")
 result = plugin.execute("sample data")
 
+# Inspect registered plugins
 for info in registry.list_plugins():
-    print(f"{info['name']}: {info['version']}")
+    print(f"{info['name']} v{info['version']} — {info['description']}")
+
+# Unload when done
+registry.unload_plugin("my_plugin")
 ```
 
 ## MethodRegistry
@@ -178,6 +247,177 @@ from semantica.core.methods import build_knowledge_base
 result = build_knowledge_base(sources=["doc.pdf"], method="fast")
 ```
 
+## Schemas
+
+<AccordionGroup>
+  <Accordion title="SystemState enum">
+
+```python
+from semantica.core import SystemState
+
+SystemState.UNINITIALIZED  # → startup() →
+SystemState.INITIALIZING   # → hooks complete →
+SystemState.READY          # → first operation →
+SystemState.RUNNING        # → shutdown() →
+SystemState.STOPPING       # → hooks complete →
+SystemState.STOPPED
+# Any unhandled exception during startup/shutdown →
+SystemState.ERROR
+```
+
+Check current state at any time:
+
+```python
+state = manager.get_state()
+if manager.is_ready():
+    result = framework.build_knowledge_base(sources)
+```
+
+  </Accordion>
+  <Accordion title="HealthStatus dataclass">
+
+```python
+@dataclass
+class HealthStatus:
+    component:  str             # component name
+    healthy:    bool            # True = operational
+    message:    str             # human-readable status
+    timestamp:  datetime        # time of last check
+    details:    Dict[str, Any]  # component-specific diagnostics
+```
+
+```python
+health = manager.health_check()
+for name, status in health.items():
+    icon = "✓" if status.healthy else "✗"
+    print(f"{icon} {name}: {status.message}")
+```
+
+  </Accordion>
+  <Accordion title="PluginInfo and LoadedPlugin dataclasses">
+
+```python
+@dataclass
+class PluginInfo:
+    name:          str
+    version:       str
+    plugin_class:  Type
+    description:   str
+    author:        str
+    dependencies:  List[str]    # pip package names required
+    capabilities:  List[str]    # e.g. ["ingest", "extract"]
+    metadata:      Dict[str, Any]
+
+@dataclass
+class LoadedPlugin:
+    info:       PluginInfo
+    instance:   Any             # the live plugin object
+    config:     Dict[str, Any]  # config passed at load time
+    loaded_at:  datetime
+```
+
+```python
+registry = PluginRegistry()
+registry.register_plugin("my_plugin", MyPlugin, version="1.0.0")
+
+if registry.is_plugin_loaded("my_plugin"):
+    plugin = registry.get_loaded_plugin("my_plugin")
+else:
+    plugin = registry.load_plugin("my_plugin")
+
+details = registry.get_plugin_info("my_plugin")
+```
+
+  </Accordion>
+</AccordionGroup>
+
+## Complete Configuration Example
+
+```yaml
+# config.yaml
+llm_provider:
+  name: groq
+  model: llama-3.3-70b-versatile
+  api_key: ${GROQ_API_KEY}
+
+embedding_model:
+  provider: sentence-transformers
+  model: all-mpnet-base-v2
+  dimension: 768
+  device: cpu               # "cpu" | "cuda" | "mps"
+
+vector_store:
+  backend: faiss
+  dimension: 768
+  index_type: hnsw          # "flat" | "ivf" | "hnsw" | "pq"
+
+graph_db:
+  backend: neo4j
+  uri: bolt://localhost:7687
+  user: neo4j
+  password: ${NEO4J_PASSWORD}
+
+processing:
+  batch_size: 32
+  max_workers: 4
+  chunk_size: 512
+
+pipeline:
+  retry_max: 3
+  backoff: exponential      # "fixed" | "linear" | "exponential"
+  failure_strategy: skip    # "skip" | "stop" | "retry"
+
+quality:
+  min_confidence: 0.7
+  dedup_threshold: 0.85
+
+logging:
+  level: INFO               # DEBUG | INFO | WARNING | ERROR
+  format: "%(asctime)s %(name)s %(levelname)s %(message)s"
+```
+
+Load and use:
+
+```python
+from semantica.core import ConfigManager, Semantica
+
+manager   = ConfigManager()
+config    = manager.load_from_file("config.yaml")
+
+config.set("processing.batch_size", 64)   # runtime override
+
+framework = Semantica(config=config)
+framework.initialize()
+result    = framework.build_knowledge_base(["doc1.pdf", "doc2.docx"])
+framework.shutdown()
+```
+
+## Tips and Common Pitfalls
+
+<Tip>
+  **Use individual modules directly unless you need application lifecycle management.** `Semantica` orchestrates the full pipeline, but for simple scripts and notebooks, using `FileIngestor`, `NERExtractor`, and `GraphBuilder` directly is clearer and more debuggable.
+</Tip>
+
+<Warning>
+  **Always call `framework.shutdown(graceful=True)` in a `finally` block.** Without graceful shutdown, in-flight pipeline steps may leave partial writes in your vector store or graph database. Wrapping in `try/finally` guarantees cleanup even on exceptions.
+</Warning>
+
+<Tip>
+  **Use `ConfigManager.merge_configs()` for environment-specific overrides.** Keep a `base.yaml` with default settings and a `prod.yaml` with overrides. Merge them at startup rather than maintaining separate copies — this prevents configuration drift between environments.
+</Tip>
+
+<Warning>
+  **Environment variable overrides use double underscores for nesting.** `SEMANTICA_PROCESSING__BATCH_SIZE=64` sets `processing.batch_size` — the double underscore (`__`) represents a nesting level. A single underscore is reserved for multi-word keys within the same level.
+</Warning>
+
+<Tip>
+  **Register startup hooks with explicit priorities.** `register_startup_hook(fn, priority=10)` — lower numbers run first. If your database hook (priority 10) must run before your cache hook (priority 20), those numbers guarantee the order. Without explicit priorities, execution order is undefined.
+</Tip>
+
+<Warning>
+  **Check `manager.is_ready()` before running pipelines.** If `Semantica.initialize()` failed partway through (e.g., a database connection refused), the state transitions to `ERROR` rather than `READY`. Always check before submitting work to avoid errors that are hard to trace.
+</Warning>
+
 <CardGroup cols={2}>
   <Card title="Pipeline" icon="arrows-turn-to-dots" href="pipeline">
     Pipeline execution and step orchestration.

docs/reference/deduplication.md

@@ -6,22 +6,127 @@ icon: "copy"
 
 `semantica.deduplication` detects and merges duplicate entities across sources to produce a clean, single-source-of-truth knowledge graph. **v2 strategies** (`blocking_v2`, `hybrid_v2`, `semantic_v2`) are up to **7x faster** than v1 with fine-grained result control.
 
+## Why Deduplicate?
+
+Real-world data sources disagree on names. "Apple Inc.", "Apple Computer", and "Apple International Ltd" can all refer to the same company — but if they land in your knowledge graph as separate nodes, every query that should return one entity returns three. Relationships, analytics, and retrieval all degrade.
+
+Deduplication solves this before it reaches the graph:
+
+- **Cross-source ingestion** — Wikipedia calls it "OpenAI", SEC filings call it "OpenAI, Inc.", your CRM calls it "OpenAI LLC"
+- **Data entry variation** — "Steve Jobs", "Steven P. Jobs", "S. Jobs" are the same person
+- **Transliteration differences** — Cyrillic, Chinese, or Arabic names romanized inconsistently across sources
+- **Abbreviation drift** — "US", "U.S.", "United States", "USA" all mean the same thing
+
 ## What You Get
 
-- **`DuplicateDetector`** — pairwise and batch duplicate detection with configurable strategies
-- **`EntityMerger`** — merge duplicate groups with configurable property-level merge policies
-- **`SimilarityCalculator`** — multi-factor similarity: Levenshtein, Jaro-Winkler, cosine, Jaccard, embedding
-- **`ClusterBuilder`** — Union-Find and hierarchical clustering for large-scale batch deduplication
-- **Convenience functions** — `detect_duplicates`, `merge_entities`, `calculate_similarity`
+<CardGroup cols={2}>
+  <Card title="DuplicateDetector" icon="copy">
+    Pairwise and batch duplicate detection with configurable strategies and result filtering.
+  </Card>
+  <Card title="EntityMerger" icon="code-merge">
+    Merge duplicate groups with configurable property-level merge policies.
+  </Card>
+  <Card title="SimilarityCalculator" icon="chart-line">
+    Multi-factor similarity: Levenshtein, Jaro-Winkler, cosine, Jaccard, and embedding.
+  </Card>
+  <Card title="ClusterBuilder" icon="diagram-project">
+    Union-Find and hierarchical clustering for large-scale batch deduplication.
+  </Card>
+  <Card title="MergeStrategyManager" icon="sliders">
+    Reusable per-property merge rule configurations — define once, apply across operations.
+  </Card>
+  <Card title="v2 Strategies" icon="bolt">
+    `blocking_v2`, `hybrid_v2`, `semantic_v2` — up to 7x faster than v1 equivalents.
+  </Card>
+</CardGroup>
+
+## Quick Start
+
+```python
+from semantica.deduplication import detect_duplicates, merge_entities
+
+# Detect
+duplicates = detect_duplicates(entities, method="hybrid_v2", similarity_threshold=0.85)
+
+# Merge
+merged = merge_entities(entities, duplicates, method="keep_most_complete")
+print(f"Reduced {len(entities)} → {len(merged)} entities")
+```
+
+## Choosing a Strategy
+
+<Tabs>
+  <Tab title="hybrid_v2 (recommended)">
+    Combines blocking + string similarity + semantic embedding — best default for production:
+
+    ```python
+    from semantica.deduplication import DuplicateDetector
+
+    detector   = DuplicateDetector(similarity_threshold=0.85)
+    duplicates = detector.detect_duplicates(entities, strategy="hybrid_v2")
+    ```
+
+    Best for: general production use — handles 95% of real-world cases without GPU. Catches string variants ("Apple Inc." / "Apple Inc") and semantic aliases ("Machine Learning" / "ML").
+  </Tab>
+  <Tab title="semantic_v2">
+    Pure embedding-based similarity — highest accuracy for cross-language and abbreviation matching:
+
+    ```python
+    detector   = DuplicateDetector(similarity_threshold=0.85)
+    duplicates = detector.detect_duplicates(entities, strategy="semantic_v2")
+    ```
+
+    Best for: entities that use different words for the same concept ("ML" vs "Machine Learning"), cross-language entity matching, and abbreviation expansion. Requires embedding model.
+  </Tab>
+  <Tab title="blocking_v2">
+    Blocking + Jaro-Winkler string similarity only — fastest option for CPU-only environments:
+
+    ```python
+    detector   = DuplicateDetector(similarity_threshold=0.85)
+    duplicates = detector.detect_duplicates(entities, strategy="blocking_v2")
+    ```
+
+    Best for: large datasets (500K+ entities) where speed is critical and entities are in the same language. No GPU required.
+  </Tab>
+  <Tab title="Strategy Comparison">
+
+    | Strategy | Algorithm | Speed | Accuracy | Best For |
+    | -------- | --------- | ----- | -------- | -------- |
+    | `jaro_winkler` | String similarity only (v1) | Fast | Medium | Small datasets, names, single-source |
+    | `blocking_v2` | Blocking + Jaro-Winkler | Very fast | Medium | Large datasets, speed-critical, CPU-only |
+    | `hybrid_v2` | Blocking + string + semantic | Fast | High | General production use — best default |
+    | `semantic_v2` | Embedding similarity | Medium | Highest | Semantic aliases, cross-language, abbreviations |
+
+    **Rules of thumb:**
+    - Start with `hybrid_v2` — it handles 95% of real-world cases without GPU
+    - Use `semantic_v2` when entities use different words for the same concept
+    - Use `blocking_v2` when processing >500k entities and speed matters most
+    - Never use v1 strategies on new projects — they exist for backwards compatibility only
+  </Tab>
+</Tabs>
+
+### Threshold Tuning
+
+| Domain | Recommended Threshold | Notes |
+| ------ | --------------------- | ----- |
+| Person names | 0.85–0.90 | Names vary a lot; too high misses "Steve" / "Steven" |
+| Organization names | 0.80–0.88 | Corporate suffixes create variation; lower threshold helps |
+| Product names | 0.88–0.95 | Product names are more stable |
+| Medical terms | 0.90–0.95 | High precision required; false merges are dangerous |
+| General entities | 0.85 | Safe default starting point |
+
+Start at 0.85, inspect false positives and false negatives, then adjust ±0.05.
 
 ## DuplicateDetector
 
-Find duplicate entity pairs with configurable strategies and result filtering:
+<Note>
+  **v0.5.0 fix:** `DuplicateDetector` no longer produces duplicate definition errors when the same entity appears in multiple sources with identical definitions.
+</Note>
 
 ```python
 from semantica.deduplication import DuplicateDetector
 
-detector = DuplicateDetector(similarity_threshold=0.85)
+detector   = DuplicateDetector(similarity_threshold=0.85)
 duplicates = detector.detect_duplicates(entities)
 
 for dup in duplicates:
@@ -33,30 +138,23 @@ Fine-grained control over strategy, thresholds, and result size:
 ```python
 duplicates = detector.detect_duplicates(
     entities,
-    strategy="semantic_v2",    # see strategies table below
-    min_similarity=0.85,        # minimum score to consider a match
-    top_k_per_entity=3,         # max candidates per entity
-    max_results=100,            # total result cap
-    sort_by="similarity",       # "similarity" | "entity_id" | "cluster_size"
+    strategy="hybrid_v2",
+    min_similarity=0.85,
+    top_k_per_entity=3,     # max candidates per entity — avoids false-positive floods
+    max_results=100,
+    sort_by="similarity",   # "similarity" | "entity_id" | "cluster_size"
 )
 ```
 
-### Detection Strategies
-
-| Strategy | Algorithm | Speed | Accuracy |
-| -------- | --------- | ----- | -------- |
-| `jaro_winkler` | String similarity (v1) | Fast | Medium |
-| `blocking_v2` | Blocking + Jaro-Winkler (v2) | Very fast | Medium |
-| `hybrid_v2` | Blocking + semantic + string (v2) | Fast | High |
-| `semantic_v2` | Embedding similarity (v2) | Medium | Highest |
-
-<Note>
-  **v0.5.0 fix:** `DuplicateDetector` no longer produces duplicate definition errors when the same entity appears in multiple sources with identical definitions.
-</Note>
+**Key behaviours:**
+- Defaults to `hybrid_v2` when no strategy is specified
+- `top_k_per_entity=3` prevents one entity from flooding results by being a near-match to everything
+- Pairs are returned once — never both `(A, B)` and `(B, A)`
+- `sort_by="similarity"` puts highest-confidence duplicates first for faster manual review
 
 ## EntityMerger
 
-Merges detected duplicate groups into canonical entities, preserving provenance:
+Merge detected duplicate groups into canonical entities, preserving provenance:
 
 ```python
 from semantica.deduplication import EntityMerger
@@ -64,22 +162,24 @@ from semantica.deduplication import EntityMerger
 merger = EntityMerger()
 merged_entities = merger.merge_duplicates(
     entities,
-    strategy="keep_most_complete",  # see strategies table below
-    preserve_provenance=True,        # keep source references after merge
+    strategy="keep_most_complete",
+    preserve_provenance=True,
 )
+
+print(f"Merged to: {len(merged_entities)} canonical entities")
 ```
 
 ### Merge Strategies
 
-| Strategy | Behavior |
-| -------- | -------- |
-| `keep_first` | Keep the first entity in each duplicate group |
-| `keep_last` | Keep the most recently seen entity |
-| `keep_most_complete` | Keep the entity with the most non-null properties |
-| `union` | Merge all properties — non-conflicting fields combined |
-| `voting` | Most common property value wins |
+| Strategy | Behavior | When to Use |
+| -------- | -------- | ----------- |
+| `keep_first` | Keep the first entity in each group | Source order is meaningful (most authoritative first) |
+| `keep_last` | Keep the most recently seen entity | Most recent source is most accurate |
+| `keep_most_complete` | Keep the entity with the most non-null properties | Default — maximizes data richness |
+| `union` | Merge all properties; combine non-conflicting fields | You want every known alias, tag, and label |
+| `voting` | Most common property value wins | Multiple semi-reliable sources |
 
-Fine-grained per-property merge rules:
+### Per-Property Merge Rules
 
 ```python
 from semantica.deduplication import EntityMerger, PropertyMergeRule
@@ -89,31 +189,41 @@ merger = EntityMerger(
         "name":        PropertyMergeRule.KEEP_FIRST,
         "aliases":     PropertyMergeRule.UNION,
         "description": PropertyMergeRule.KEEP_LONGEST,
+        "confidence":  PropertyMergeRule.MAX,
+        "created_at":  PropertyMergeRule.KEEP_FIRST,
+        "updated_at":  PropertyMergeRule.KEEP_LAST,
     }
 )
+
+merged_entities = merger.merge_duplicates(entities, preserve_provenance=True)
 ```
 
+| Rule | Behaviour |
+| ---- | --------- |
+| `KEEP_FIRST` | Value from the first entity in the group |
+| `KEEP_LAST` | Value from the last entity |
+| `KEEP_LONGEST` | Longest non-null string value |
+| `KEEP_MOST_COMPLETE` | Entity with the most non-null fields (default fallback) |
+| `UNION` | Combine all unique values into a list |
+| `MAX` | Numerically largest value |
+| `MIN` | Numerically smallest value |
+| `VOTING` | Most frequently occurring value |
+
 ## SimilarityCalculator
 
-Compute multi-factor similarity scores between entity pairs:
+Compute multi-factor similarity scores — useful for debugging why two entities were (or were not) detected as duplicates:
 
 ```python
 from semantica.deduplication import SimilarityCalculator
 
-calc = SimilarityCalculator()
-
+calc  = SimilarityCalculator()
 score = calc.calculate_similarity(entity_a, entity_b)
-# → SimilarityResult(score=0.91, components={...})
 
 print(score.score)                    # overall score 0.0–1.0
-print(score.components["label"])      # label similarity
-print(score.components["embedding"])  # semantic similarity
-print(score.components["property"])   # property overlap
-```
+print(score.components["label"])      # label similarity contribution
+print(score.components["embedding"])  # semantic similarity contribution
+print(score.components["property"])   # property overlap contribution
 
-Individual string and vector metrics:
-
-```python
 lev  = calc.levenshtein("Apple Inc.", "Apple Inc")
 jaro = calc.jaro_winkler("Steve Jobs", "Steven Jobs")
 cos  = calc.cosine_similarity(embedding_a, embedding_b)
@@ -127,57 +237,202 @@ Build entity clusters for large-scale batch deduplication:
 ```python
 from semantica.deduplication import ClusterBuilder
 
-builder = ClusterBuilder(algorithm="union_find")  # or "hierarchical"
-result = builder.build_clusters(entities, similarity_threshold=0.85)
+builder = ClusterBuilder(algorithm="union_find")   # or "hierarchical"
+result  = builder.build_clusters(entities, similarity_threshold=0.85)
+
+print(f"Original:   {len(entities)} entities")
+print(f"Clusters:   {len(result.clusters)} groups")
+print(f"Singletons: {result.singleton_count}")
 
-print(f"Clusters: {len(result.clusters)}")
 for cluster in result.clusters:
-    print(f"  [{cluster.id}] {cluster.members} — cohesion: {cluster.cohesion:.2f}")
+    print(f"  [{cluster.id}] members={cluster.members}  cohesion={cluster.cohesion:.2f}")
+```
+
+### Union-Find vs Hierarchical
+
+```python
+# Union-Find — O(n·α(n)), scales to millions; use for production
+builder = ClusterBuilder(algorithm="union_find")
+
+# Hierarchical — tighter clusters; use when quality matters more than speed
+builder = ClusterBuilder(algorithm="hierarchical", linkage="average")
+result  = builder.build_clusters(entities, similarity_threshold=0.85)
+```
+
+**Key behaviours:**
+- Union-Find groups entities transitively: if A≈B and B≈C, all three land in one cluster even if A and C are only 0.70 similar
+- Hierarchical with `linkage="average"` avoids chaining by requiring average similarity across all pairs to meet the threshold
+
+### Cluster Quality Metrics
+
+```python
+print(f"Silhouette score: {result.quality.silhouette_score:.3f}")
+# → -1.0 to 1.0; above 0.5 is good, above 0.7 is excellent
+
+print(f"Avg cohesion:     {result.quality.avg_cohesion:.3f}")
+print(f"Avg separation:   {result.quality.avg_separation:.3f}")
+```
+
+## MergeStrategyManager
+
+Define complex, reusable merge configurations once and apply them across multiple operations:
+
+```python
+from semantica.deduplication import MergeStrategyManager, PropertyMergeRule
+
+manager = MergeStrategyManager()
+manager.add_rule("name",        PropertyMergeRule.KEEP_FIRST)
+manager.add_rule("aliases",     PropertyMergeRule.UNION)
+manager.add_rule("description", PropertyMergeRule.KEEP_LONGEST)
+manager.add_rule("confidence",  PropertyMergeRule.MAX)
+manager.add_rule("sources",     PropertyMergeRule.UNION)
+manager.add_rule("created_at",  PropertyMergeRule.KEEP_FIRST)
+manager.add_rule("updated_at",  PropertyMergeRule.KEEP_LAST)
+manager.set_default_rule(PropertyMergeRule.KEEP_MOST_COMPLETE)
+
+merged_entity = manager.merge(duplicate_group)
 ```
 
 ## Blocking Strategies
 
-Blocking reduces the O(n²) pairwise comparison problem to a manageable candidate set:
-
 ```python
 detector = DuplicateDetector(
-    blocking_strategy="token",     # "token" | "phonetic" | "ngram"
+    blocking_strategy="token",       # "token" | "phonetic" | "ngram"
     blocking_threshold=0.6,
     similarity_threshold=0.85
 )
 ```
 
-## Custom Similarity Functions
+| Blocking Strategy | How It Works | Best For |
+| ----------------- | ------------ | -------- |
+| `token` | Shared token overlap (default) | General entity names |
+| `phonetic` | Soundex/Metaphone phonetic codes | Names with spelling variations |
+| `ngram` | Character n-gram overlap | Short strings, typos |
 
-Register domain-specific similarity logic:
+## Custom Similarity Functions
 
 ```python
 from semantica.deduplication import method_registry
 
-def drug_name_similarity(entity_a, entity_b):
-    # Match drug names by active compound
-    return score  # 0.0 to 1.0
+def drug_name_similarity(entity_a, entity_b) -> float:
+    compound_a = entity_a.properties.get("active_compound", "")
+    compound_b = entity_b.properties.get("active_compound", "")
+    return 1.0 if compound_a == compound_b else 0.0
 
 method_registry.register("similarity", "drug_name", drug_name_similarity)
 
-detector = DuplicateDetector(similarity_method="drug_name")
+detector = DuplicateDetector(similarity_method="drug_name", similarity_threshold=0.90)
 ```
 
-## Convenience Functions
+## Schemas
+
+<AccordionGroup>
+  <Accordion title="Cluster and ClusterResult schemas">
 
 ```python
-from semantica.deduplication import detect_duplicates, merge_entities, calculate_similarity
+@dataclass
+class Cluster:
+    id:        str
+    members:   List[str]   # entity IDs in this duplicate group
+    cohesion:  float       # mean pairwise similarity within the cluster (0–1)
+    centroid:  str         # member ID closest to the cluster centroid
 
-# Quick detection
-duplicates = detect_duplicates(entities, method="semantic_v2", similarity_threshold=0.85)
-
-# Quick merge
-merged = merge_entities(entities, duplicates, method="keep_most_complete")
-
-# Quick similarity
-score = calculate_similarity(entity_a, entity_b, method="hybrid_v2")
+@dataclass
+class ClusterResult:
+    clusters:         List[Cluster]
+    singleton_count:  int            # entities with no duplicates found
+    merge_candidates: int            # clusters with > 1 member
+    quality:          ClusterQuality
 ```
 
+  </Accordion>
+  <Accordion title="ClusterQuality schema">
+
+```python
+@dataclass
+class ClusterQuality:
+    silhouette_score:  float   # -1 to 1; above 0.5 is good
+    avg_cohesion:      float   # mean within-cluster similarity
+    avg_separation:    float   # mean between-cluster distance
+```
+
+  </Accordion>
+</AccordionGroup>
+
+## End-to-End Pipeline
+
+<Steps>
+  <Step title="Load entities from multiple sources">
+    ```python
+    entities = load_entities_from_sources(["crunchbase", "wikipedia", "internal_db"])
+    print(f"Loaded: {len(entities)} raw entities")
+    ```
+  </Step>
+  <Step title="Detect duplicate pairs">
+    ```python
+    from semantica.deduplication import DuplicateDetector
+
+    detector   = DuplicateDetector(similarity_threshold=0.85)
+    duplicates = detector.detect_duplicates(entities, strategy="hybrid_v2")
+    print(f"Found: {len(duplicates)} duplicate pairs")
+    ```
+  </Step>
+  <Step title="Configure property-level merge rules">
+    ```python
+    from semantica.deduplication import MergeStrategyManager, PropertyMergeRule
+
+    manager = MergeStrategyManager()
+    manager.add_rule("name",        PropertyMergeRule.KEEP_FIRST)
+    manager.add_rule("aliases",     PropertyMergeRule.UNION)
+    manager.add_rule("description", PropertyMergeRule.KEEP_LONGEST)
+    manager.set_default_rule(PropertyMergeRule.KEEP_MOST_COMPLETE)
+    ```
+  </Step>
+  <Step title="Merge and inspect results">
+    ```python
+    from semantica.deduplication import EntityMerger
+
+    merger = EntityMerger()
+    merged = merger.merge_duplicates(entities, preserve_provenance=True)
+    print(f"Result: {len(merged)} canonical entities")
+
+    for entity in merged:
+        if hasattr(entity, "source_entities"):
+            print(f"{entity.label} merged from: {entity.source_entities}")
+    ```
+  </Step>
+</Steps>
+
+## Tips and Common Pitfalls
+
+<Warning>
+  **Normalize before deduplicating.** Run `TextNormalizer` and `EntityNormalizer` first. "APPLE INC." and "apple inc." will score 0.50 on string similarity but 1.0 after case normalization. See the [Normalize](normalize) module.
+</Warning>
+
+<Tip>
+  **Too many false positives?** Raise `similarity_threshold` by 0.05 or switch from `blocking_v2` to `hybrid_v2` to add semantic precision on top of string matching.
+</Tip>
+
+<Tip>
+  **Too many missed duplicates?** Lower `similarity_threshold` by 0.05, or switch to `semantic_v2` to catch entities that use different words ("ML" vs "Machine Learning").
+</Tip>
+
+<Warning>
+  **Union-Find over-merges?** The transitive grouping means weak chains can connect unrelated entities. Switch to `hierarchical` clustering with `linkage="average"` to require that every pair within a cluster meets the threshold — not just a chain of nearby pairs.
+</Warning>
+
+<Tip>
+  **Preserve provenance on merge.** Set `preserve_provenance=True` so you can always trace which source contributed each property to the merged entity. Critical for audit and debugging.
+</Tip>
+
+<Tip>
+  **Inspect similarity components.** When a pair is flagged and you're not sure why, use `SimilarityCalculator.calculate_similarity()` to see the per-component breakdown (`label`, `embedding`, `property`) and identify which factor is driving the match.
+</Tip>
+
+<Warning>
+  **Don't deduplicate after building the graph.** Deduplicate entities *before* `GraphBuilder` ingests them. Merging inside a live graph is possible but requires tracking and rewriting all relationship endpoints.
+</Warning>
+
 <CardGroup cols={2}>
   <Card title="Conflicts" icon="triangle-exclamation" href="conflicts">
     Detect value conflicts between non-duplicate entities.
@@ -186,9 +441,9 @@ score = calculate_similarity(entity_a, entity_b, method="hybrid_v2")
     GraphBuilder uses deduplication during construction.
   </Card>
   <Card title="Normalize" icon="broom" href="normalize">
-    Normalize entity names before deduplication.
+    Normalize entity names before deduplication for better accuracy.
   </Card>
   <Card title="Provenance" icon="link" href="provenance">
-    Track merged entity lineage.
+    Track merged entity lineage and source attribution.
   </Card>
 </CardGroup>

docs/reference/embeddings.md

@@ -1,216 +1,400 @@
 ---
 title: "Embeddings Module"
-description: "Text and graph embedding generation — Sentence-Transformers, FastEmbed, OpenAI, BGE, LlamaStore, with pooling strategies and graph embedding managers."
+description: "Text and graph embedding generation — Sentence-Transformers, FastEmbed, OpenAI, BGE, Ollama — with pooling strategies, caching, and GPU acceleration."
 icon: "vector-square"
 ---
 
-`semantica.embeddings` converts text and graph structures into dense vectors for semantic search, entity resolution, and GraphRAG retrieval. A single provider-agnostic API abstracts Sentence-Transformers, FastEmbed, OpenAI, BGE, and Ollama.
+`semantica.embeddings` converts text and graph structures into dense vectors. These vectors power semantic search, entity resolution, GraphRAG retrieval, and Distance Intelligence across every Semantica module. A single provider-agnostic API abstracts Sentence-Transformers, FastEmbed, OpenAI, BGE, and Ollama behind one interface.
+
+## Why Embeddings Matter
+
+Raw text can't be compared mathematically. Embeddings translate meaning into geometry — two semantically similar sentences produce vectors that are close together in high-dimensional space, even when they share no words.
+
+Semantica uses embeddings for:
+
+- **Semantic search** — find knowledge graph nodes by meaning, not just keywords
+- **Entity resolution** — detect that "Apple Inc." and "Apple Computer" refer to the same entity
+- **Deduplication** — `semantic_v2` strategy measures entity similarity via embedding distance
+- **GraphRAG retrieval** — hybrid vector + graph traversal for grounded LLM answers
+- **Distance Intelligence** — N×N semantic distance matrices across entity sets
+- **Semantic chunking** — detect topic shift boundaries in `TextSplitter(method="semantic_transformer")`
 
 ## What You Get
 
-- **`EmbeddingGenerator`** — main entry point, provider-agnostic text embedding with batching
-- **`TextEmbedder`** — text-specific embedding with automatic batching and disk caching
-- **`GraphEmbeddingManager`** — node and subgraph embeddings for structural similarity
-- **`VectorEmbeddingManager`** — full embedding lifecycle for vector store integration
-- **Provider stores** — `OpenAIStore`, `BGEStore`, `FastEmbedStore`, `LlamaStore`, `ProviderStoreFactory`
-- **Pooling strategies** — Mean, Max, CLS, Attention, Hierarchical pooling
+<CardGroup cols={2}>
+  <Card title="EmbeddingGenerator" icon="vector-square">
+    Main entry point — provider-agnostic, handles batching automatically across all backends.
+  </Card>
+  <Card title="TextEmbedder" icon="text-size">
+    Text-specific with automatic batching, disk caching, and progress tracking.
+  </Card>
+  <Card title="GraphEmbeddingManager" icon="diagram-project">
+    Node and subgraph embeddings for structural similarity and GraphRAG context assembly.
+  </Card>
+  <Card title="VectorEmbeddingManager" icon="database">
+    Full lifecycle: embed → store → search in a single coordinated workflow.
+  </Card>
+  <Card title="Provider Stores" icon="plug">
+    `OpenAIStore`, `BGEStore`, `FastEmbedStore`, `LlamaStore`, and `ProviderStoreFactory`.
+  </Card>
+  <Card title="Pooling Strategies" icon="layer-group">
+    Mean, Max, CLS, Attention, and Hierarchical — control token-to-vector aggregation.
+  </Card>
+</CardGroup>
+
+## Installation
+
+| Provider | Install Command | API Key Required |
+| -------- | --------------- | ---------------- |
+| Sentence-Transformers (default) | `pip install semantica` | No |
+| FastEmbed | `pip install "semantica[fastembed]"` | No |
+| BGE | `pip install semantica` | No (uses sentence-transformers) |
+| OpenAI | `pip install "semantica[llm-openai]"` | Yes — `OPENAI_API_KEY` |
+| Ollama (LlamaStore) | `pip install "semantica[llm-ollama]"` | No — local server |
+| All providers | `pip install "semantica[all]"` | Varies |
+
+Check which providers are available in your environment:
+
+```python
+from semantica.embeddings import check_available_providers
+
+providers = check_available_providers()
+# → {"sentence_transformers": True, "fastembed": True, "openai": False, "ollama": True}
+```
+
+## Quick Start
+
+<Steps>
+  <Step title="Install and initialize a provider">
+    ```python
+    from semantica.embeddings import EmbeddingGenerator
+
+    # Default — Sentence-Transformers, free, runs locally
+    generator = EmbeddingGenerator(model="sentence-transformers")
+    ```
+  </Step>
+  <Step title="Generate embeddings">
+    ```python
+    embeddings = generator.generate(["Text about AI", "Machine learning concepts"])
+    ```
+  </Step>
+  <Step title="Compute similarity">
+    ```python
+    # Cosine similarity — 0.0 (unrelated) to 1.0 (identical meaning)
+    score = generator.similarity(embeddings[0], embeddings[1])
+    print(f"Similarity: {score:.3f}")
+    ```
+  </Step>
+  <Step title="Embed and store for search">
+    ```python
+    from semantica.embeddings import VectorEmbeddingManager, TextEmbedder
+    from semantica.vector_store import VectorStore
+
+    vector_store = VectorStore(backend="faiss", dimension=384)
+    manager = VectorEmbeddingManager(
+        embedder=TextEmbedder(model="sentence-transformers"),
+        vector_store=vector_store,
+    )
+    ids = manager.embed_and_store(documents, metadata=metadata_list)
+
+    results = manager.search("machine learning algorithms", top_k=10)
+    for result in results:
+        print(f"Score: {result.score:.3f}  —  {result.metadata['title']}")
+    ```
+  </Step>
+</Steps>
+
+## Supported Models
+
+| Provider | Model | Dimension | Speed | Best For |
+| -------- | ----- | --------- | ----- | -------- |
+| `sentence-transformers` | `all-MiniLM-L6-v2` | 384 | Fast | Default — good balance of speed and quality |
+| `sentence-transformers` | `all-mpnet-base-v2` | 768 | Medium | Higher retrieval quality |
+| `bge` | `BAAI/bge-large-en-v1.5` | 1024 | Medium | State-of-the-art retrieval accuracy |
+| `bge` | `BAAI/bge-small-en-v1.5` | 384 | Fast | Lightweight, competitive quality |
+| `fastembed` | `BAAI/bge-small-en-v1.5` | 384 | Very fast | CPU-optimised, low-latency production |
+| `openai` | `text-embedding-3-small` | 1536 | API | Cost-effective OpenAI embedding |
+| `openai` | `text-embedding-3-large` | 3072 | API | Highest quality via OpenAI API |
+| `llama` (Ollama) | Any Ollama model | Varies | Local | Fully local, no API key |
 
 ## EmbeddingGenerator
 
-Main entry point — handles provider selection and batching automatically:
+<Tabs>
+  <Tab title="Sentence-Transformers (default)">
+    ```python
+    from semantica.embeddings import EmbeddingGenerator
 
-```python
-from semantica.embeddings import EmbeddingGenerator
+    # Default model — all-MiniLM-L6-v2, dimension 384
+    generator = EmbeddingGenerator(model="sentence-transformers")
 
-# Sentence-Transformers (default, free, local)
-generator = EmbeddingGenerator(model="sentence-transformers")
-embeddings = generator.generate(["Text 1", "Text 2"])
+    # Specific HuggingFace model
+    generator = EmbeddingGenerator(model="BAAI/bge-large-en-v1.5")
 
-# Specific BGE model
-generator = EmbeddingGenerator(model="BAAI/bge-large-en-v1.5")
-embeddings = generator.generate(texts)
+    embeddings = generator.generate(texts)
+    similarity = generator.similarity(embeddings[0], embeddings[1])
+    ```
 
-# OpenAI
-import os
-generator = EmbeddingGenerator(
-    model="openai",
-    model_name="text-embedding-3-small",
-    api_key=os.getenv("OPENAI_API_KEY")
-)
+    Best for: default prototyping, no API key, good quality.
+  </Tab>
+  <Tab title="FastEmbed">
+    ```python
+    from semantica.embeddings import EmbeddingGenerator
 
-# FastEmbed (fast, CPU-optimized)
-generator = EmbeddingGenerator(model="fastembed")
-```
+    generator = EmbeddingGenerator(model="fastembed")
+    embeddings = generator.generate(texts)
+    ```
 
-### Supported Models
+    Best for: CPU-only production, lowest latency without GPU.
+  </Tab>
+  <Tab title="OpenAI">
+    ```python
+    from semantica.embeddings import EmbeddingGenerator
+    import os
 
-| Provider | Model | Dimension | Notes |
-| -------- | ----- | --------- | ----- |
-| `sentence-transformers` | `all-MiniLM-L6-v2` | 384 | Default, fast, free |
-| `sentence-transformers` | `all-mpnet-base-v2` | 768 | Higher quality |
-| `bge` | `BAAI/bge-large-en-v1.5` | 1024 | State-of-the-art retrieval |
-| `fastembed` | `BAAI/bge-small-en-v1.5` | 384 | Fast, CPU-optimized |
-| `openai` | `text-embedding-3-small` | 1536 | OpenAI API |
-| `openai` | `text-embedding-3-large` | 3072 | OpenAI API, highest quality |
-| `llama` | any Ollama model | varies | Fully local inference |
+    generator = EmbeddingGenerator(
+        model="openai",
+        model_name="text-embedding-3-small",
+        api_key=os.getenv("OPENAI_API_KEY"),
+    )
+    embeddings = generator.generate(texts)
+    ```
+
+    Best for: highest quality (3-large), or matching an OpenAI LLM pipeline.
+  </Tab>
+  <Tab title="Ollama (local)">
+    ```python
+    from semantica.embeddings import LlamaStore
+
+    store     = LlamaStore(model="llama3.2", base_url="http://localhost:11434")
+    embedding = store.embed("Hello world")
+    ```
+
+    Best for: air-gapped or privacy-sensitive environments — no data leaves your machine.
+  </Tab>
+  <Tab title="GPU acceleration">
+    ```python
+    from semantica.embeddings import EmbeddingGenerator
+
+    # NVIDIA GPU
+    generator = EmbeddingGenerator(model="sentence-transformers", device="cuda")
+
+    # Apple Silicon (M1/M2/M3)
+    generator = EmbeddingGenerator(model="sentence-transformers", device="mps")
+
+    # CPU (default)
+    generator = EmbeddingGenerator(model="sentence-transformers", device="cpu")
+    ```
+
+    GPU reduces embedding time by 5–20× depending on batch size and model.
+  </Tab>
+</Tabs>
+
+### Constructor Parameters
+
+| Parameter | Type | Default | Description |
+| --------- | ---- | ------- | ----------- |
+| `model` | `str` | `"sentence-transformers"` | Provider name or HuggingFace model ID |
+| `model_name` | `str` | Provider default | Specific model within a provider (OpenAI) |
+| `api_key` | `str` | `None` | API key for cloud providers; reads env var if omitted |
+| `device` | `str` | `"cpu"` | Compute device: `"cpu"` / `"cuda"` / `"mps"` |
+| `batch_size` | `int` | `32` | Texts per forward pass |
+| `normalize` | `bool` | `True` | L2-normalise output vectors (required for cosine similarity) |
+| `cache_dir` | `str` | `None` | Directory for disk caching of computed embeddings |
 
 ## TextEmbedder
 
-Specialized for text with automatic batching and optional disk cache:
+Specialised for text workloads — adds automatic batching, progress tracking, and disk caching:
 
 ```python
 from semantica.embeddings import TextEmbedder
 
-embedder = TextEmbedder(model="sentence-transformers", cache_dir=".emb_cache")
+embedder = TextEmbedder(
+    model="sentence-transformers",
+    cache_dir=".emb_cache",   # persist embeddings to disk
+    cache_ttl=86400,          # cache expiry in seconds (24h); None = never expires
+    batch_size=128,
+    show_progress=True,
+)
 
 # Single text
-embedding = embedder.embed("Hello world")
+embedding = embedder.embed("A knowledge graph connects entities with typed relationships.")
 
-# Batch — processes automatically in chunks
-embeddings = embedder.embed_batch(
-    ["Text 1", "Text 2", ..., "Text 10000"],
-    batch_size=128,
-    show_progress=True
-)
+# Batch — auto-splits into batch_size chunks, shows progress bar
+embeddings = embedder.embed_batch(texts, show_progress=True)
 ```
 
+**Key behaviours:**
+- Cache is keyed on text content + model name — identical texts return cached vectors instantly
+- Progress bar uses `tqdm` in terminal; switches to `tqdm.notebook` in Jupyter automatically
+- Large batches (> 10k texts) are chunked internally to avoid OOM on GPU
+
 ## Provider Stores
 
-Each provider implements the `ProviderStore` interface and can be used independently:
+Use provider stores directly when you need fine-grained control over a single backend:
 
 ```python
 from semantica.embeddings import (
     OpenAIStore, BGEStore, FastEmbedStore, LlamaStore,
-    ProviderStoreFactory
+    ProviderStoreFactory,
 )
+import os
 
 # OpenAI
-store = OpenAIStore(api_key=os.getenv("OPENAI_API_KEY"), model="text-embedding-3-small")
+store     = OpenAIStore(api_key=os.getenv("OPENAI_API_KEY"), model="text-embedding-3-small")
 embedding = store.embed("Hello world")
 
 # BGE (Sentence-Transformers wrapper)
-store = BGEStore(model="BAAI/bge-large-en-v1.5")
+store     = BGEStore(model="BAAI/bge-large-en-v1.5", device="cpu")
 embedding = store.embed("Hello world")
 
-# FastEmbed
-store = FastEmbedStore(model="BAAI/bge-small-en-v1.5")
+# FastEmbed — ONNX runtime, no CUDA required
+store     = FastEmbedStore(model="BAAI/bge-small-en-v1.5")
 embedding = store.embed("Hello world")
 
-# LlamaStore (Ollama — fully local)
-store = LlamaStore(model="llama3.2", base_url="http://localhost:11434")
+# Ollama — fully local
+store     = LlamaStore(model="llama3.2", base_url="http://localhost:11434")
 embedding = store.embed("Hello world")
 
-# Auto-select from config
-store = ProviderStoreFactory.create(provider="openai", model="text-embedding-3-small")
+# Auto-select from a name string — useful in config-driven pipelines
+store = ProviderStoreFactory.create(provider="bge", model="BAAI/bge-large-en-v1.5")
 ```
 
 ## Pooling Strategies
 
-Control how token-level embeddings are aggregated into a single vector:
+Transformer models produce one embedding per token. Pooling aggregates token embeddings into a single vector:
 
-```python
-from semantica.embeddings import (
-    MeanPooling, MaxPooling, CLSPooling,
-    AttentionPooling, HierarchicalPooling, PoolingStrategyFactory
-)
+<Tabs>
+  <Tab title="MeanPooling (default)">
+    ```python
+    from semantica.embeddings import MeanPooling
 
-# Mean pooling — default, best for most tasks
-pooler = MeanPooling()
-pooled = pooler.pool(token_embeddings)
+    pooler = MeanPooling()
+    pooled = pooler.pool(token_embeddings)   # shape: (hidden_dim,)
+    ```
 
-# Max pooling — captures strongest activated features
-pooler = MaxPooling()
+    Best for: retrieval, semantic search, and clustering — averages all token contributions.
+  </Tab>
+  <Tab title="MaxPooling">
+    ```python
+    from semantica.embeddings import MaxPooling
 
-# CLS token — good for classification tasks
-pooler = CLSPooling()
+    pooler = MaxPooling()
+    pooled = pooler.pool(token_embeddings)
+    ```
 
-# Attention-weighted pooling
-pooler = AttentionPooling()
+    Best for: capturing the presence of any feature — takes the max activation per dimension.
+  </Tab>
+  <Tab title="CLSPooling">
+    ```python
+    from semantica.embeddings import CLSPooling
 
-# Hierarchical: chunk-level → global mean (best for long documents)
-pooler = HierarchicalPooling(chunk_size=512)
+    pooler = CLSPooling()
+    pooled = pooler.pool(token_embeddings)
+    ```
 
-# Create from config string
-pooler = PoolingStrategyFactory.create(strategy="mean")
-```
+    Best for: classification-style tasks; models explicitly trained with CLS pooling (BERT).
+  </Tab>
+  <Tab title="HierarchicalPooling">
+    ```python
+    from semantica.embeddings import HierarchicalPooling
+
+    # Chunk text, mean-pool within chunks, then mean-pool chunks
+    pooler = HierarchicalPooling(chunk_size=512)
+    pooled = pooler.pool(token_embeddings)
+    ```
+
+    Best for: long documents exceeding the model's max sequence length — reports, papers, contracts.
+  </Tab>
+  <Tab title="Strategy Comparison">
+
+    | Strategy | When to Use |
+    | -------- | ----------- |
+    | `mean` | Default for retrieval, semantic search, and clustering |
+    | `max` | When you want to capture the presence of any feature, not average presence |
+    | `cls` | Classification-style tasks; models explicitly trained with CLS pooling (BERT) |
+    | `attention` | When token importance varies significantly; slower but more accurate |
+    | `hierarchical` | Long documents exceeding model context length; reports, papers, contracts |
+
+    ```python
+    from semantica.embeddings import PoolingStrategyFactory
+
+    pooler = PoolingStrategyFactory.create(strategy="mean")
+    ```
+
+  </Tab>
+</Tabs>
 
 ## GraphEmbeddingManager
 
-Embed graph nodes and subgraphs for structural similarity and GraphRAG context:
+Embed graph nodes and subgraphs for structural similarity and GraphRAG context assembly:
 
 ```python
-from semantica.embeddings import GraphEmbeddingManager
+from semantica.embeddings import GraphEmbeddingManager, TextEmbedder
 
 manager = GraphEmbeddingManager(
     text_embedder=TextEmbedder(model="sentence-transformers"),
-    graph_store=graph_store
+    graph_store=graph_store,   # optional — for persistence
 )
 
-# Embed all nodes in the graph
+# Embed all nodes — uses node label + property text
 node_embeddings = manager.embed_nodes(kg)
 
-# Embed a subgraph centered on a node (for GraphRAG context)
+# Embed a subgraph centred on a node (for GraphRAG context)
 subgraph_embedding = manager.embed_subgraph(
-    kg, center_node="Apple Inc.", hops=2
+    kg,
+    center_node="Apple Inc.",
+    hops=2,   # include neighbours up to 2 hops away
 )
 
-# Find semantically similar nodes
+# Find semantically similar nodes by ID
 similar = manager.find_similar_nodes("apple_inc", top_k=5)
+for node_id, score in similar:
+    print(f"{node_id}: {score:.3f}")
 ```
 
-## VectorEmbeddingManager
+**Key behaviours:**
+- Node embedding combines the label, type, and all property values into a single text string before embedding
+- `hops=2` captures the local neighbourhood — increase for richer context, decrease for speed
+- Results from `find_similar_nodes` are sorted by cosine similarity descending
 
-Manages the full embedding lifecycle — from raw text to stored, searchable vectors:
+## Embedding Cache
+
+The disk cache avoids recomputing embeddings for unchanged text — critical for large corpora and repeated pipeline runs:
 
 ```python
-from semantica.embeddings import VectorEmbeddingManager
-from semantica.vector_store import VectorStore
+from semantica.embeddings import TextEmbedder
 
-vector_store = VectorStore(backend="faiss", dimension=768)
-
-manager = VectorEmbeddingManager(
-    embedder=TextEmbedder(model="sentence-transformers"),
-    vector_store=vector_store
+embedder = TextEmbedder(
+    model="sentence-transformers",
+    cache_dir=".embeddings_cache",
+    cache_ttl=3600,   # seconds — None means cache never expires
 )
 
-# Embed documents and store in one step
-ids = manager.embed_and_store(documents, metadata=metadata_list)
+# First call: computes and caches
+embeddings = embedder.embed_batch(texts)
 
-# Search by semantic similarity
-results = manager.search("machine learning algorithms", top_k=10)
+# Second call (same texts): returns from cache instantly
+embeddings = embedder.embed_batch(texts)
 ```
 
+<Note>
+  The Distance Intelligence module (v0.5.0) uses the same cache to avoid recomputing embeddings during N×N matrix calculations across large entity sets.
+</Note>
+
 ## Similarity Computation
 
 ```python
 from semantica.embeddings import calculate_similarity
 
-# Cosine similarity (most common)
+# Cosine similarity — direction only, not magnitude; most common for text
 score = calculate_similarity(embedding_a, embedding_b, method="cosine")
-# → 0.0 to 1.0
+# → 0.0 (orthogonal / unrelated) to 1.0 (identical direction)
 
-# Euclidean distance (converted to similarity)
+# Euclidean distance converted to similarity
 score = calculate_similarity(embedding_a, embedding_b, method="euclidean")
-```
 
-## GPU Acceleration
-
-```python
-# Use CUDA GPU for faster embedding generation
-generator = EmbeddingGenerator(model="sentence-transformers", device="cuda")
-# device options: "cpu" | "cuda" | "mps"
-```
-
-## Embedding Cache
-
-The embedding cache is used by Distance Intelligence (v0.5.0) to avoid recomputing embeddings for large N×N distance matrix calculations:
-
-```python
-embedder = TextEmbedder(
-    model="sentence-transformers",
-    cache_dir=".embeddings_cache",
-    cache_ttl=3600    # seconds before cache entries expire
-)
+# Dot product — use when vectors are already normalised (equivalent to cosine)
+score = calculate_similarity(embedding_a, embedding_b, method="dot")
 ```
 
 ## Convenience Functions
@@ -218,10 +402,10 @@ embedder = TextEmbedder(
 ```python
 from semantica.embeddings import (
     embed_text, generate_embeddings, calculate_similarity,
-    pool_embeddings, check_available_providers
+    pool_embeddings, check_available_providers,
 )
 
-# Single text
+# Single text — fastest path
 emb = embed_text("Hello world", method="sentence_transformers")
 
 # Batch
@@ -232,6 +416,28 @@ providers = check_available_providers()
 # → {"sentence_transformers": True, "fastembed": True, "openai": False}
 ```
 
+## Tips and Common Pitfalls
+
+<Warning>
+  **Dimension mismatch.** The dimension you pass to `VectorStore(dimension=...)` must exactly match your embedding model's output. `all-MiniLM-L6-v2` → 384, `all-mpnet-base-v2` → 768, `bge-large-en-v1.5` → 1024. Check with `generator.dimension` before creating the store.
+</Warning>
+
+<Warning>
+  **Not normalising for cosine similarity.** If you compute cosine similarity directly (dot product), vectors must be L2-normalised first. `EmbeddingGenerator` normalises by default (`normalize=True`). If you disable it, use `calculate_similarity(..., method="cosine")` which normalises internally.
+</Warning>
+
+<Warning>
+  **Sequence length limits.** Most models have a 512-token limit. Text beyond that is silently truncated. Use `TextSplitter(method="hierarchical")` + `HierarchicalPooling` for long documents.
+</Warning>
+
+<Tip>
+  **Always use the same model for indexing and querying.** Vectors from different models are not comparable — they live in different vector spaces. Switching models requires re-embedding your entire corpus.
+</Tip>
+
+<Tip>
+  **Cache invalidation.** The cache key is the text + model name. Switching models requires clearing the cache or using a different `cache_dir` — otherwise you'll get stale vectors silently returned.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Vector Store" icon="database" href="vector_store">
     Store and search the generated embeddings.
@@ -240,9 +446,9 @@ providers = check_available_providers()
     Chunk text before embedding for better retrieval quality.
   </Card>
   <Card title="KG Module" icon="diagram-project" href="kg">
-    Distance Intelligence uses graph embeddings.
+    Distance Intelligence uses graph embeddings for semantic neighbourhoods.
   </Card>
   <Card title="Deduplication" icon="copy" href="deduplication">
-    Semantic deduplication uses embeddings for entity resolution.
+    Semantic deduplication uses embedding distance for entity resolution.
   </Card>
 </CardGroup>

docs/reference/evals.md

@@ -6,28 +6,221 @@ icon: "chart-line"
 
 `semantica.evals` provides a comprehensive evaluation framework for measuring extraction accuracy, graph quality, and pipeline performance. Use it to benchmark extractors, validate pipeline output, and track quality regressions across runs.
 
+## What You Get
+
+<CardGroup cols={2}>
+  <Card title="KGEvaluator" icon="diagram-project">
+    Completeness, consistency, schema compliance, coverage, and orphan node metrics.
+  </Card>
+  <Card title="ExtractionEvaluator" icon="magnifying-glass">
+    NER precision / recall / F1 and relation extraction metrics against gold-standard datasets.
+  </Card>
+  <Card title="PipelineEvaluator" icon="gear">
+    Throughput (docs/sec), per-step latency, peak memory, and error rate benchmarking.
+  </Card>
+  <Card title="RegressionTracker" icon="clock-rotate-left">
+    Record pipeline runs and compare metrics across commits or config changes.
+  </Card>
+  <Card title="Deduplication Accuracy" icon="copy">
+    Merge precision, false positive / false negative rates for deduplication strategies.
+  </Card>
+  <Card title="Reasoning Correctness" icon="microchip">
+    Inference accuracy, rule coverage, and derivation depth for reasoning engines.
+  </Card>
+</CardGroup>
+
+## Quick Start
+
+<Steps>
+  <Step title="Evaluate KG quality">
+    ```python
+    from semantica.evals import KGEvaluator
+
+    evaluator = KGEvaluator()
+    report    = evaluator.evaluate(kg, ontology=ontology)
+
+    print(f"Completeness: {report.completeness:.2%}")
+    print(f"Consistency:  {report.consistency:.2%}")
+    print(f"Coverage:     {report.coverage:.2%}")
+    print(f"Orphan nodes: {report.orphan_count}")
+    ```
+  </Step>
+  <Step title="Evaluate extraction accuracy against gold standard">
+    ```python
+    from semantica.evals import ExtractionEvaluator
+
+    evaluator = ExtractionEvaluator()
+    report    = evaluator.evaluate_ner(
+        predictions=extracted_entities,
+        gold_standard=annotated_entities,
+    )
+
+    print(f"Precision: {report.precision:.3f}")
+    print(f"Recall:    {report.recall:.3f}")
+    print(f"F1:        {report.f1:.3f}")
+    print(f"By type:   {report.per_type_metrics}")
+    ```
+  </Step>
+  <Step title="Benchmark pipeline throughput">
+    ```python
+    from semantica.evals import PipelineEvaluator
+
+    evaluator = PipelineEvaluator()
+    metrics   = evaluator.benchmark(pipeline, data="data/", warmup_runs=2, bench_runs=5)
+
+    print(f"Throughput:       {metrics.docs_per_second:.1f} docs/sec")
+    print(f"Total duration:   {metrics.total_seconds:.1f}s")
+    print(f"Per-step latency: {metrics.step_latencies}")
+    print(f"Peak memory (MB): {metrics.peak_memory_mb:.0f}")
+    print(f"Error rate:       {metrics.error_rate:.2%}")
+    ```
+  </Step>
+  <Step title="Track regressions across releases">
+    ```python
+    from semantica.evals import RegressionTracker
+
+    tracker = RegressionTracker(db_path="eval_history.db")
+
+    run_id = tracker.record_run(
+        pipeline_version="v1.2.0",
+        metrics=metrics,
+        config=config.to_dict(),
+    )
+
+    diff = tracker.compare(run_id, baseline_run_id="run_abc123")
+    for metric, change in diff.items():
+        direction = "↑" if change > 0 else "↓"
+        print(f"  {metric}: {direction} {abs(change):.2%}")
+    ```
+  </Step>
+</Steps>
+
+## Evaluation Areas
+
+<Tabs>
+  <Tab title="KG Quality">
+    Measure completeness, consistency, schema compliance, and structural health of a knowledge graph:
+
+    ```python
+    from semantica.evals import KGEvaluator
+
+    evaluator = KGEvaluator()
+    report    = evaluator.evaluate(kg, ontology=ontology)
+
+    print(f"Completeness:  {report.completeness:.2%}")   # % entities with all required fields
+    print(f"Consistency:   {report.consistency:.2%}")    # % entities without type conflicts
+    print(f"Coverage:      {report.coverage:.2%}")       # % entity types in ontology
+    print(f"Total nodes:   {report.node_count}")
+    print(f"Orphan nodes:  {report.orphan_count}")       # nodes with no edges
+    ```
+
+    **Key behaviours:**
+    - `consistency` requires an ontology — without one, it always returns 1.0
+    - `orphan_count` flags disconnected nodes that likely represent extraction or deduplication errors
+    - `completeness` checks required properties defined in the ontology schema
+  </Tab>
+  <Tab title="Extraction Accuracy">
+    Compare extracted entities and relations against annotated gold-standard data:
+
+    ```python
+    from semantica.evals import ExtractionEvaluator
+
+    evaluator = ExtractionEvaluator()
+
+    # NER evaluation
+    ner_report = evaluator.evaluate_ner(
+        predictions=extracted_entities,
+        gold_standard=annotated_entities,
+    )
+    print(f"Precision:    {ner_report.precision:.3f}")
+    print(f"Recall:       {ner_report.recall:.3f}")
+    print(f"F1:           {ner_report.f1:.3f}")
+    print(f"By type:      {ner_report.per_type_metrics}")
+
+    # Relation extraction evaluation
+    rel_report = evaluator.evaluate_relations(
+        predictions=extracted_relations,
+        gold_standard=annotated_relations,
+    )
+    print(f"Relation F1:  {rel_report.f1:.3f}")
+    ```
+  </Tab>
+  <Tab title="Pipeline Performance">
+    Benchmark throughput, latency, memory, and error rate across multiple runs:
+
+    ```python
+    from semantica.evals import PipelineEvaluator
+
+    evaluator = PipelineEvaluator()
+    metrics   = evaluator.benchmark(
+        pipeline,
+        data="data/",
+        warmup_runs=2,    # eliminate cold-start noise
+        bench_runs=5,     # average over 5 real runs
+    )
+
+    print(f"Throughput:       {metrics.docs_per_second:.1f} docs/sec")
+    print(f"Total duration:   {metrics.total_seconds:.1f}s")
+    print(f"Per-step latency: {metrics.step_latencies}")
+    print(f"Peak memory (MB): {metrics.peak_memory_mb:.0f}")
+    print(f"Error rate:       {metrics.error_rate:.2%}")
+    ```
+  </Tab>
+  <Tab title="Regression Tracking">
+    Store runs and compare metrics across pipeline versions:
+
+    ```python
+    from semantica.evals import RegressionTracker
+
+    tracker = RegressionTracker(db_path="eval_history.db")
+
+    # Record a run with version tag and full config snapshot
+    run_id = tracker.record_run(
+        pipeline_version="v1.2.0",
+        metrics=metrics,
+        config=config.to_dict(),
+    )
+
+    # Compare to a previous run
+    diff = tracker.compare(run_id, baseline_run_id="run_abc123")
+    for metric, change in diff.items():
+        direction = "↑" if change > 0 else "↓"
+        print(f"  {metric}: {direction} {abs(change):.2%}")
+    ```
+  </Tab>
+</Tabs>
+
+## When to Evaluate
+
+| Trigger | Evaluator to Use | What to Check |
+| ------- | ---------------- | ------------- |
+| New extraction model or method | `ExtractionEvaluator` | Precision, recall, F1 vs gold standard |
+| After changing LLM provider | `ExtractionEvaluator` | Per-type F1 — check if rare types regressed |
+| Before releasing new pipeline version | `PipelineEvaluator` | Throughput, latency, error rate |
+| After deduplication strategy change | `KGEvaluator` | Orphan count, consistency score |
+| Every production deployment | `RegressionTracker` | Compare vs previous baseline run |
+
+## Tips and Common Pitfalls
+
 <Warning>
-  **Coming Soon** — This module is currently in active development. Documentation will be expanded in the next release.
+  **Build a gold standard dataset early.** `ExtractionEvaluator` requires annotated ground truth. Without it, you're evaluating subjectively. Even 100 carefully annotated documents give you a meaningful baseline to track regressions against.
 </Warning>
 
-## Planned Capabilities
+<Tip>
+  **Evaluate per entity type, not just overall F1.** Aggregate F1 can hide regressions — if your model's PERSON F1 drops from 0.95 to 0.80 but ORGANIZATION improves, the average may look stable. Use `report.per_type_metrics` to catch type-specific regressions.
+</Tip>
 
-The Evals module will cover five evaluation areas:
+<Tip>
+  **Store every benchmark run with `RegressionTracker`.** Run ID + version tag + config snapshot gives you a reproducible audit trail. Without it, "did the last release make things better?" has no objective answer.
+</Tip>
 
-| Area | What It Measures |
-| ---- | ---------------- |
-| **KG Quality** | Completeness, consistency, schema compliance, coverage metrics |
-| **Extraction Accuracy** | NER precision / recall / F1, relation extraction metrics |
-| **Pipeline Performance** | Throughput (docs/sec), latency per step, error rates |
-| **Deduplication** | Merge accuracy, false positive / negative rates |
-| **Reasoning** | Inference correctness, rule coverage, derivation depth |
+<Tip>
+  **Run `PipelineEvaluator` with `warmup_runs=2`.** Cold starts are unrepresentative — model weights get cached, JIT compilation kicks in. Warmup runs eliminate this noise from your benchmark numbers.
+</Tip>
 
-## Scope
-
-- **Offline evaluation** — compare against gold-standard annotated datasets
-- **Regression tracking** — compare pipeline runs across commits or config changes
-- **Live monitoring** — record quality metrics during production pipeline runs
-- **Benchmark suites** — standard NER, RE, and KG construction benchmarks
+<Warning>
+  **`KGEvaluator` needs an ontology for consistency scoring.** Without an ontology, `consistency` always returns 1.0 — there's nothing to check against. Pass `ontology=ontology` to get meaningful consistency metrics.
+</Warning>
 
 <CardGroup cols={2}>
   <Card title="Semantic Extract" icon="magnifying-glass" href="semantic_extract">

docs/reference/explorer.md

@@ -8,11 +8,26 @@ icon: "map"
 
 ## What You Get
 
-- **Graph Explorer** — interactive node/edge search, filtering, path highlighting, and neighborhood views
-- **Ontology Hub** (v0.5.0) — browse class hierarchies, infer types, run SHACL validation, align ontologies
-- **Distance Intelligence** (v0.5.0) — semantic similarity search, ego-mode neighborhood views, distance heatmaps
-- **REST API** — 15+ endpoints for graph data, path finding, embeddings, and semantic search
-- **CLI launcher** — `semantica-explorer` command for quick local startup
+<CardGroup cols={2}>
+  <Card title="Graph Explorer" icon="diagram-project">
+    Interactive node/edge search, filtering, path highlighting, and neighborhood expansion. Indexed search at 0.004ms on 118k-node graphs.
+  </Card>
+  <Card title="Ontology Hub (v0.5.0)" icon="sitemap">
+    Visual ontology editor, SHACL Studio, alignment authoring, health dashboard, and version control — all in the browser.
+  </Card>
+  <Card title="Distance Intelligence (v0.5.0)" icon="circle-nodes">
+    Semantic similarity search, ego-mode neighborhood views, N×N distance heatmaps, and distance band classification.
+  </Card>
+  <Card title="REST API" icon="code">
+    15+ endpoints for graph data, path finding, embeddings, semantic search, analytics, and export — fully documented at `/docs`.
+  </Card>
+  <Card title="WebSocket Progress" icon="bolt">
+    Long-running exports and analyses stream progress events in real time — no polling required.
+  </Card>
+  <Card title="CLI Launcher" icon="terminal">
+    `semantica-explorer --graph my_graph.json` for instant local startup without writing any Python.
+  </Card>
+</CardGroup>
 
 ## Installation
 
@@ -24,103 +39,290 @@ Requires `uvicorn` and `fastapi`. Included automatically with `pip install seman
 
 ## Launch
 
-<CodeGroup>
+<Steps>
+  <Step title="Build your graph and launch Explorer">
+    ```python
+    from semantica.explorer import start_explorer
+    from semantica.kg import GraphBuilder
+    from semantica.ontology import OntologyManager
 
-```bash CLI
-# Start the explorer on a saved graph
-semantica-explorer --graph my_graph.json
+    kg       = GraphBuilder().build(entities=entities, relationships=relationships)
+    ontology = OntologyManager()
 
-# Custom host and port
-semantica-explorer --graph my_graph.json --host 0.0.0.0 --port 8080
+    start_explorer(
+        graph=kg,
+        ontology=ontology,     # optional — enables Ontology Hub tab
+        port=8080,
+        host="127.0.0.1",
+        open_browser=True,
+    )
+    # → Serving at http://127.0.0.1:8080
+    ```
+  </Step>
+  <Step title="Or launch from the CLI">
+    ```bash
+    # Start on a saved graph
+    semantica-explorer --graph my_graph.json
 
-# Skip auto-opening the browser
-semantica-explorer --graph my_graph.json --no-browser
-```
+    # Custom host and port
+    semantica-explorer --graph my_graph.json --host 0.0.0.0 --port 8080
 
-```python Python
-from semantica.context import ContextGraph
+    # Skip auto-opening the browser
+    semantica-explorer --graph my_graph.json --no-browser
+    ```
+  </Step>
+  <Step title="Enable authentication for shared environments">
+    ```python
+    start_explorer(
+        graph=kg,
+        port=8080,
+        enable_auth=True,
+        api_key="my-secret-key",
+        cors_origins=["https://app.example.com"],
+        session_timeout=1800,   # 30-minute inactivity timeout
+    )
+    ```
+  </Step>
+  <Step title="Switch graphs without restarting">
+    ```bash
+    curl -X POST http://localhost:8080/api/import \
+      -H "Content-Type: multipart/form-data" \
+      -F "file=@updated_graph.json"
+    # Browser dashboard reloads automatically
+    ```
+  </Step>
+</Steps>
 
-graph = ContextGraph(advanced_analytics=True)
-# ... build or load your graph ...
-graph.save_to_file("my_graph.json")
+## `start_explorer()` Parameters
 
-import subprocess
-subprocess.run(["semantica-explorer", "--graph", "my_graph.json", "--port", "8000"])
-```
-
-```python Module
-# Run directly as a Python module
-import subprocess
-subprocess.run(["python", "-m", "semantica.explorer", "--graph", "my_graph.json"])
-```
-
-</CodeGroup>
+| Parameter | Type | Default | Description |
+| --------- | ---- | ------- | ----------- |
+| `graph` | `KnowledgeGraph` or `ContextGraph` | *(required)* | The graph to load into Explorer |
+| `ontology` | `OntologyManager` | `None` | Ontology to load into the Ontology Hub tab |
+| `port` | `int` | `8000` | Port to bind the server |
+| `host` | `str` | `"127.0.0.1"` | Host to bind. Use `"0.0.0.0"` for network access |
+| `open_browser` | `bool` | `True` | Auto-open the dashboard in the default browser |
+| `session_timeout` | `int` | `3600` | Session inactivity timeout in seconds; `None` disables |
+| `enable_auth` | `bool` | `False` | Require `X-API-Key` header on all API requests |
+| `api_key` | `str` | `None` | API key value when `enable_auth=True` |
+| `cors_origins` | `list[str]` | `["*"]` | Allowed CORS origins. Restrict in production |
+| `log_level` | `str` | `"info"` | Uvicorn log level (`"debug"` / `"info"` / `"warning"`) |
 
 ## CLI Reference
 
 | Flag | Default | Description |
 | ---- | ------- | ----------- |
-| `--graph`, `-g` | *(required)* | Path to a ContextGraph JSON file |
+| `--graph`, `-g` | *(required)* | Path to a saved graph JSON file |
 | `--port`, `-p` | `8000` | Port to bind the server |
 | `--host` | `127.0.0.1` | Host to bind the server |
 | `--no-browser` | `false` | Skip auto-opening the browser |
+| `--enable-auth` | `false` | Require `X-API-Key` header on all requests |
+| `--api-key` | `None` | API key value when `--enable-auth` is set |
+| `--cors-origins` | `"*"` | Comma-separated list of allowed CORS origins |
+| `--log-level` | `"info"` | Uvicorn log level |
 
 ## Features
 
-### Graph Explorer
+<Tabs>
+  <Tab title="Graph Explorer">
+    Core dashboard for navigating knowledge graphs:
 
-Core dashboard for navigating knowledge graphs:
+    - **Indexed search** — find any node by label or type; 0.004ms on 118k-node graphs (v0.5.0)
+    - **Bidirectional path finding** — trace paths between any two nodes
+    - **Neighbor expansion** — click any node to expand its connections
+    - **Filter by entity type** — focus on Person, Organization, Event, or any custom type
+    - **Edge label display** — relationship types shown on all edges
+    - **Graph declutter** — workspace layout controls for dense graphs
+  </Tab>
+  <Tab title="Ontology Hub (v0.5.0)">
+    Full ontology lifecycle management in the browser:
 
-- **Indexed search** — find any node by label or type; 0.004ms on 118k-node graphs (v0.5.0)
-- **Bidirectional path finding** — trace paths between any two nodes
-- **Neighbor expansion** — click any node to expand its connections
-- **Filter by entity type** — focus on Person, Organization, Event, or any custom type
-- **Edge label display** — relationship types shown on all edges
-- **Graph declutter** — workspace layout controls for dense graphs
+    - **Visual ontology editor** — drag-and-drop class and property authoring
+    - **SHACL Studio** — create, validate, and test SHACL shapes with live feedback
+    - **Alignment authoring** — author ontology alignments across schemas
+    - **Health dashboard** — graph quality metrics, validation status, coverage reports
+    - **Version control** — snapshot, diff, and restore ontology versions
+  </Tab>
+  <Tab title="Distance Intelligence (v0.5.0)">
+    Semantic neighborhood analysis centered on any node:
 
-### Ontology Hub (v0.5.0)
+    - **N×N distance matrices** — pairwise semantic distances across a set of nodes
+    - **Ego-mode visualization** — focus on a single node's semantic neighborhood
+    - **Distance band classification** — nodes grouped as `near` / `mid` / `far`
+    - **Embedding cache** — optimized embedding reuse for large graphs
+  </Tab>
+  <Tab title="Session Management">
+    Thread-safe sessions with rollback protection:
 
-Full ontology lifecycle management in the browser:
+    ```python
+    start_explorer(
+        graph=kg,
+        session_timeout=1800,   # 30-minute inactivity timeout
+        enable_auth=True,
+        api_key="my-secret-key",
+    )
+    ```
 
-- **Visual ontology editor** — drag-and-drop class and property authoring
-- **SHACL Studio** — create, validate, and test SHACL shapes with live feedback
-- **Alignment authoring** — author ontology alignments across schemas
-- **Health dashboard** — graph quality metrics, validation status, coverage reports
-- **Version control** — snapshot, diff, and restore ontology versions
-
-### Distance Intelligence (v0.5.0)
-
-Semantic neighborhood analysis centered on any node:
-
-- **N×N distance matrices** — pairwise semantic distances across a set of nodes
-- **Ego-mode visualization** — focus on a single node's semantic neighborhood
-- **Distance band classification** — nodes grouped as `near` / `mid` / `far`
-- **Embedding cache** — optimized embedding reuse for large graphs
-
-### Knowledge Explorer API
-
-Full FastAPI backend accessible at `http://localhost:8000/docs`:
-
-- 12+ export formats (RDF, Parquet, AQL, JSON-LD, and more)
-- WebSocket progress streaming for long operations
-- Thread-safe sessions with rollback protection
-- Audit trail for all operations
+    - Sessions are per connected browser tab
+    - Write operations (annotate, import) roll back automatically on failure
+    - All writes appended to audit trail at `/api/provenance/audit`
+    - Session state held in memory — use `/api/export/json` to persist between restarts
+  </Tab>
+</Tabs>
 
 ## API Endpoints
 
-The FastAPI server exposes a REST API alongside the browser dashboard:
+Full interactive docs at `http://localhost:8000/docs`. All endpoints available via REST.
 
-| Endpoint | Method | Description |
-| -------- | ------ | ----------- |
-| `/api/graph/summary` | `GET` | Node count, edge count, entity types |
-| `/api/graph/search` | `GET` | Full-text and type-filtered node search |
-| `/api/graph/path` | `GET` | Bidirectional path between two nodes |
-| `/api/graph/neighbors` | `GET` | Neighbors of a node with optional depth |
-| `/api/ontology/validate` | `POST` | Run SHACL validation on the graph |
-| `/api/export/{format}` | `GET` | Export graph in specified format |
-| `/ws/progress` | `WS` | WebSocket stream for operation progress |
+<AccordionGroup>
+  <Accordion title="Graph endpoints">
 
-Full OpenAPI docs available at `http://localhost:8000/docs` when the server is running.
+    | Endpoint | Method | Description |
+    | -------- | ------ | ----------- |
+    | `/api/graph/summary` | `GET` | Node count, edge count, entity type distribution |
+    | `/api/graph/search` | `GET` | Indexed full-text and type-filtered node search |
+    | `/api/graph/node/{id}` | `GET` | Fetch a single node with all properties |
+    | `/api/graph/neighbors` | `GET` | Neighbors of a node — `?node_id=&depth=2` |
+    | `/api/graph/path` | `GET` | Bidirectional shortest path — `?source=&target=` |
+    | `/api/graph/subgraph` | `POST` | Extract a subgraph by node IDs or type filter |
+    | `/api/graph/annotate` | `POST` | Add a user annotation to a node or edge |
+    | `/api/graph/annotations` | `GET` | List all annotations on the graph |
+
+  </Accordion>
+  <Accordion title="Ontology endpoints">
+
+    | Endpoint | Method | Description |
+    | -------- | ------ | ----------- |
+    | `/api/ontology/classes` | `GET` | List all ontology classes and properties |
+    | `/api/ontology/validate` | `POST` | Run SHACL validation; returns violations |
+    | `/api/ontology/hierarchy` | `GET` | Class hierarchy as a tree structure |
+    | `/api/ontology/vocabulary` | `GET` | SKOS vocabulary terms and alt labels |
+    | `/api/ontology/align` | `POST` | Submit two ontologies for alignment |
+    | `/api/ontology/diff` | `POST` | Diff two ontology versions |
+
+  </Accordion>
+  <Accordion title="Provenance, Decisions & Analytics">
+
+    **Provenance:**
+
+    | Endpoint | Method | Description |
+    | -------- | ------ | ----------- |
+    | `/api/provenance/entity/{id}` | `GET` | Full provenance lineage for an entity |
+    | `/api/provenance/source/{id}` | `GET` | All entities sourced from a document |
+    | `/api/provenance/audit` | `GET` | Full audit trail of Explorer operations |
+
+    **Decisions:**
+
+    | Endpoint | Method | Description |
+    | -------- | ------ | ----------- |
+    | `/api/decisions/list` | `GET` | Paginated list of recorded decisions |
+    | `/api/decisions/{id}` | `GET` | Single decision with causal chain |
+    | `/api/decisions/search` | `GET` | Precedent search — `?query=&limit=5` |
+    | `/api/decisions/influence/{id}` | `GET` | Downstream influence of a decision |
+
+    **Analytics:**
+
+    | Endpoint | Method | Description |
+    | -------- | ------ | ----------- |
+    | `/api/analytics/centrality` | `GET` | Degree, betweenness, PageRank scores |
+    | `/api/analytics/communities` | `GET` | Community detection result |
+    | `/api/analytics/distance` | `POST` | N×N distance matrix for a node list |
+    | `/api/analytics/neighborhood` | `GET` | Semantic neighborhood — `?node=&radius=0.4` |
+
+  </Accordion>
+  <Accordion title="SPARQL, Temporal & Export">
+
+    **SPARQL & Temporal:**
+
+    | Endpoint | Method | Description |
+    | -------- | ------ | ----------- |
+    | `/api/sparql` | `POST` | Execute a SPARQL SELECT query |
+    | `/api/temporal/snapshot` | `GET` | Graph snapshot at a point in time — `?at=ISO8601` |
+    | `/api/temporal/range` | `GET` | Nodes/edges active in a time range |
+    | `/api/temporal/diff` | `POST` | Diff two temporal snapshots |
+
+    **Export & Import:**
+
+    | Endpoint | Method | Description |
+    | -------- | ------ | ----------- |
+    | `/api/export/{format}` | `GET` | Export in: `turtle`, `json-ld`, `ntriples`, `rdf-xml`, `parquet`, `aql`, `csv`, `owl`, `arrow`, `lpg`, `yaml`, `distance-matrix` |
+    | `/api/import` | `POST` | Import a graph from file (replaces current graph in session) |
+
+  </Accordion>
+</AccordionGroup>
+
+## WebSocket Progress
+
+Long-running operations stream progress events over WebSocket at `ws://localhost:8000/ws/progress`:
+
+```python
+import asyncio, websockets, json
+
+async def watch_progress():
+    async with websockets.connect("ws://localhost:8000/ws/progress") as ws:
+        async for message in ws:
+            event = json.loads(message)
+            print(f"[{event['operation']}] {event['step']} — {event['progress_pct']:.0f}%")
+            if event["status"] in ("completed", "failed"):
+                break
+
+asyncio.run(watch_progress())
+```
+
+WebSocket event schema:
+
+```json
+{
+  "operation":    "export",
+  "step":         "serializing nodes",
+  "current":      3500,
+  "total":        10000,
+  "progress_pct": 35.0,
+  "status":       "running",
+  "message":      "Serializing 10000 nodes to Turtle...",
+  "error":        null
+}
+```
+
+`status` values: `"running"` | `"completed"` | `"failed"` | `"cancelled"`
+
+## Performance
+
+| Scenario | Latency |
+| -------- | ------- |
+| Node search (118k nodes, indexed) | 0.004ms |
+| Neighbor expansion (depth 2) | < 5ms |
+| Bidirectional path (118k nodes) | < 50ms |
+| SPARQL SELECT (simple pattern) | < 20ms |
+| N×N distance matrix (100 nodes) | ~2s (with embedding cache) |
+
+The node search index is built on startup. For graphs > 500k nodes, pass `index_build_timeout=120` to `start_explorer()` to allow more time.
+
+## Tips and Common Pitfalls
+
+<Warning>
+  **Set `max_nodes` when loading large graphs.** `start_explorer(graph=kg, max_nodes=50000)` limits the rendered node count — Explorer's force-directed layout becomes unusable above ~10k nodes without limiting. Use `graph.filter(node_type="Organization")` first to focus on what matters.
+</Warning>
+
+<Warning>
+  **Use authentication in shared environments.** Pass `enable_auth=True, api_key="..."` whenever Explorer is accessible to more than one person. Without auth, anyone who can reach the port can write to the graph via the annotate and import endpoints.
+</Warning>
+
+<Warning>
+  **Export before the session ends.** Session state lives in memory and is lost on server restart. Call `/api/export/json` or `/api/export/turtle` to persist the current state before shutting down. Explorer does not auto-save.
+</Warning>
+
+<Tip>
+  **Use WebSocket progress for long operations.** Export, analysis, and large SPARQL queries stream progress to `ws://localhost:8000/ws/progress`. Polling the REST endpoints instead gives no progress signal — use the WebSocket client so users see incremental updates.
+</Tip>
+
+<Tip>
+  **Pass `session_timeout` for demos and shared notebooks.** The default session never expires. In Jupyter or shared environments, set `session_timeout=1800` (30 minutes) so stale sessions don't hold large graphs in memory.
+</Tip>
+
+<Tip>
+  **Use the REST API for automation, Explorer UI for exploration.** Explorer's REST endpoints are a stable programmatic API — pipe them into scripts to automate batch annotation, SPARQL querying, or exports. The browser UI is for interactive exploration and sharing; they use the same server.
+</Tip>
 
 <CardGroup cols={2}>
   <Card title="Context" icon="brain" href="context">
@@ -129,7 +331,7 @@ Full OpenAPI docs available at `http://localhost:8000/docs` when the server is r
   <Card title="Ontology" icon="sitemap" href="ontology">
     Programmatic ontology management and SHACL generation.
   </Card>
-  <Card title="Visualization" icon="chart-network" href="visualization">
+  <Card title="Visualization" icon="chart-bar" href="visualization">
     Programmatic graph rendering without the Explorer server.
   </Card>
   <Card title="Export" icon="file-export" href="export">

docs/reference/export.md

@@ -8,191 +8,270 @@ icon: "file-export"
 
 ## What You Get
 
-- **`RDFExporter`** — Turtle, JSON-LD, N-Triples, RDF/XML with namespace management
-- **`ParquetExporter`** — columnar storage for Spark, BigQuery, Databricks, Snowflake
-- **`LPGExporter`** — Cypher CREATE/MERGE statements for Neo4j and Memgraph
-- **`ArangoAQLExporter`** — AQL INSERT statements for ArangoDB multi-model graphs
-- **`GraphExporter`** — GraphML, GEXF, DOT for visualization tools like Gephi
-- **`OWLExporter`** — OWL 2.0 ontology export in Turtle, XML, and JSON-LD
-- **`CSVExporter`**, **`VectorExporter`**, **`ArrowExporter`**, **`DistanceExporter`**, **`ReportGenerator`**
+<CardGroup cols={2}>
+  <Card title="RDFExporter" icon="diagram-project">
+    Turtle, JSON-LD, N-Triples, RDF/XML with namespace management and optional PROV-O provenance embedding.
+  </Card>
+  <Card title="ParquetExporter" icon="layer-group">
+    Columnar storage for Spark, BigQuery, Databricks, and Snowflake with explicit PyArrow typing.
+  </Card>
+  <Card title="LPG & ArangoDB" icon="server">
+    Cypher CREATE/MERGE for Neo4j and Memgraph; AQL INSERT for ArangoDB vertex and edge collections.
+  </Card>
+  <Card title="Graph Formats" icon="chart-bar">
+    GraphML, GEXF, DOT for Gephi and Graphviz. OWL 2.0 ontology export in Turtle, XML, and JSON-LD.
+  </Card>
+  <Card title="Vector & Arrow" icon="vector-square">
+    JSON, NumPy `.npy`, and FAISS index export for embedding vectors. Apache Arrow IPC for zero-copy transfer.
+  </Card>
+  <Card title="Distance & Reports" icon="chart-line">
+    Distance matrix CSV/JSON from Distance Intelligence (v0.5.0). HTML, Markdown, and JSON analytics reports.
+  </Card>
+</CardGroup>
 
-## RDFExporter
+## Quick Start
+
+<Steps>
+  <Step title="Choose your format and instantiate an exporter">
+    ```python
+    from semantica.export import RDFExporter
+
+    exporter = RDFExporter()
+    ```
+  </Step>
+  <Step title="Export the graph">
+    ```python
+    # Interactive HTML — opens in browser, supports hover and click
+    exporter.export_to_file(graph, "output.ttl", format="turtle")
+    ```
+  </Step>
+  <Step title="Use convenience functions for one-liners">
+    ```python
+    from semantica.export import export_rdf, export_parquet, export_csv
+
+    export_rdf(graph,     "output.ttl",   format="turtle")
+    export_parquet(graph, "output/",      compression="snappy")
+    export_csv(graph,     "nodes.csv",    target="nodes")
+    ```
+  </Step>
+  <Step title="Stream large graphs to avoid OOM">
+    ```python
+    from semantica.export import ParquetExporter
+
+    exporter = ParquetExporter(compression="snappy")
+    exporter.export_stream(graph, output_dir="output/", batch_size=10_000)
+    ```
+  </Step>
+</Steps>
+
+## Exporters
+
+<Tabs>
+  <Tab title="RDF">
+    Export to W3C RDF formats — Turtle, JSON-LD, N-Triples, and RDF/XML:
+
+    ```python
+    from semantica.export import RDFExporter
+
+    exporter = RDFExporter()
+
+    # Turtle (most readable RDF format)
+    exporter.export_to_file(graph, "output.ttl",    format="turtle")
+
+    # JSON-LD (best for APIs and Linked Data)
+    exporter.export_to_file(graph, "output.jsonld", format="json-ld")
+
+    # N-Triples (streaming-friendly, one triple per line)
+    exporter.export_to_file(graph, "output.nt",     format="nt")
+
+    # RDF/XML (W3C standard, broadest compatibility)
+    exporter.export_to_file(graph, "output.xml",    format="xml")
+
+    # Export to string instead of file
+    rdf_str = exporter.export_to_rdf(graph, format="turtle")
+    ```
+
+    **Custom namespace management:**
+
+    ```python
+    from semantica.export import NamespaceManager, RDFExporter
+
+    ns_manager = NamespaceManager()
+    ns_manager.register("ex",     "http://example.org/")
+    ns_manager.register("schema", "https://schema.org/")
+
+    exporter = RDFExporter(namespace_manager=ns_manager)
+    ```
+
+    **Export with PROV-O provenance:**
+
+    ```python
+    from semantica.export import RDFExporter
+    from semantica.provenance import ProvenanceManager
+
+    provenance = ProvenanceManager()
+    # ... track entities during extraction ...
+
+    exporter = RDFExporter(include_provenance=True, provenance_manager=provenance)
+    exporter.export_to_file(graph, "output_with_prov.ttl", format="turtle")
+    # → Each entity's prov:wasGeneratedBy, prov:wasDerivedFrom, prov:hadPrimarySource
+    #   triples are included alongside the entity data triples
+    ```
+  </Tab>
+  <Tab title="Columnar & Analytics">
+    Columnar formats for analytics pipelines and human-readable export:
+
+    ```python
+    from semantica.export import ParquetExporter
+
+    exporter = ParquetExporter(compression="snappy")
+    # compression options: snappy | gzip | brotli | zstd | lz4
+
+    # Export nodes and edges as separate Parquet files
+    exporter.export_nodes(graph, "nodes.parquet")
+    exporter.export_edges(graph, "edges.parquet")
+
+    # Export full graph partitioned by node type
+    exporter.export(graph, output_dir="graph_parquet/", partition_by="node_type")
+    ```
+
+    Schema is explicitly typed with PyArrow for clean Spark/BigQuery ingestion.
+
+    ```python
+    from semantica.export import CSVExporter
+
+    exporter = CSVExporter(delimiter=",")
+    exporter.export_nodes(graph, "nodes.csv")
+    exporter.export_edges(graph, "edges.csv")
+    ```
+
+    ```python
+    from semantica.export import YAMLExporter
+
+    exporter = YAMLExporter()
+    exporter.export(graph, "graph.yaml")
+
+    yaml_str = exporter.to_string(graph)
+    ```
+  </Tab>
+  <Tab title="Graph DB Import">
+    Export Cypher or AQL statements for direct graph database import:
+
+    ```python
+    from semantica.export import LPGExporter
+
+    exporter = LPGExporter()
+
+    # CREATE statements
+    cypher = exporter.to_cypher(graph)
+    exporter.export(graph, "import.cypher", format="cypher")
+
+    # MERGE statements (idempotent — safe to re-run)
+    cypher_merge = exporter.to_cypher(graph, use_merge=True)
+    ```
+
+    ```python
+    from semantica.export import ArangoAQLExporter
+
+    exporter = ArangoAQLExporter(
+        vertex_collection="entities",
+        edge_collection="relationships"
+    )
+
+    aql = exporter.export(graph)                # returns AQL string
+    exporter.export_to_file(graph, "import.aql")
+    ```
+  </Tab>
+  <Tab title="Visualization">
+    Export for graph visualization tools and OWL ontology distribution:
+
+    ```python
+    from semantica.export import GraphExporter
+
+    exporter = GraphExporter()
+
+    exporter.export(graph, "graph.graphml", format="graphml")  # Gephi, yEd
+    exporter.export(graph, "graph.gexf",    format="gexf")     # Gephi streaming
+    exporter.export(graph, "graph.dot",     format="dot")      # Graphviz
+    ```
+
+    ```python
+    from semantica.export import OWLExporter
+
+    exporter = OWLExporter()
+    exporter.export(ontology, path="ontology.ttl",  format="turtle")
+    exporter.export(ontology, path="ontology.owl",  format="xml")
+    exporter.export(ontology, path="ontology.json", format="json-ld")
+    ```
+  </Tab>
+  <Tab title="Specialized">
+    Vector embeddings, Arrow IPC, distance matrices, and analytics reports:
+
+    ```python
+    from semantica.export import VectorExporter
+
+    exporter = VectorExporter()
+    exporter.export(embeddings, metadata, "vectors.json",  format="json")
+    exporter.export(embeddings, metadata, "vectors.npy",   format="numpy")
+    exporter.export(embeddings, metadata, "vectors.faiss", format="faiss")
+    ```
+
+    ```python
+    from semantica.export import ArrowExporter
+
+    exporter = ArrowExporter()
+    exporter.export(graph, "graph.arrow")   # requires pyarrow
+    ```
+
+    ```python
+    from semantica.export import DistanceExporter
+
+    exporter = DistanceExporter()
+    exporter.export_matrix(distance_matrix, node_labels, "distances.csv")
+    exporter.export_ego(ego_neighborhood, center_node="Apple Inc.", path="ego.json")
+    ```
+
+    ```python
+    from semantica.export import ReportGenerator
+
+    generator = ReportGenerator()
+    generator.generate(graph, analytics_result, "report.html",  format="html")
+    generator.generate(graph, analytics_result, "report.md",    format="markdown")
+    generator.generate(graph, analytics_result, "report.json",  format="json")
+    ```
+  </Tab>
+</Tabs>
+
+## Streaming Export
+
+For graphs too large to hold in memory, use streaming export — writes incrementally without buffering the full graph:
 
 ```python
-from semantica.export import RDFExporter
+from semantica.export import RDFExporter, ParquetExporter
 
+# Stream RDF — yields triples one at a time, no full-graph buffer
 exporter = RDFExporter()
+with exporter.stream(graph, format="turtle") as stream:
+    for triple_line in stream:
+        output_file.write(triple_line)
 
-# Turtle (most readable RDF format)
-exporter.export_to_file(graph, "output.ttl",    format="turtle")
-
-# JSON-LD (best for APIs and Linked Data)
-exporter.export_to_file(graph, "output.jsonld", format="json-ld")
-
-# N-Triples (streaming-friendly, one triple per line)
-exporter.export_to_file(graph, "output.nt",     format="nt")
-
-# RDF/XML (W3C standard, broadest compatibility)
-exporter.export_to_file(graph, "output.xml",    format="xml")
-
-# Export to string instead of file
-rdf_str = exporter.export_to_rdf(graph, format="turtle")
-```
-
-Custom namespace management:
-
-```python
-from semantica.export import NamespaceManager, RDFExporter
-
-ns_manager = NamespaceManager()
-ns_manager.register("ex",     "http://example.org/")
-ns_manager.register("schema", "https://schema.org/")
-
-exporter = RDFExporter(namespace_manager=ns_manager)
-```
-
-## ParquetExporter
-
-Columnar export for Spark, BigQuery, Databricks, and Snowflake analytics pipelines:
-
-```python
-from semantica.export import ParquetExporter
-
+# Stream Parquet — writes row groups incrementally
 exporter = ParquetExporter(compression="snappy")
-# compression options: snappy | gzip | brotli | zstd | lz4
-
-# Export nodes and edges as separate Parquet files
-exporter.export_nodes(graph, "nodes.parquet")
-exporter.export_edges(graph, "edges.parquet")
-
-# Export full graph partitioned by node type
-exporter.export(graph, output_dir="graph_parquet/", partition_by="node_type")
+exporter.export_stream(graph, output_dir="output/", batch_size=10_000)
 ```
 
-Schema is explicitly typed with PyArrow for clean Spark/BigQuery ingestion.
+Streaming is recommended for graphs with > 500k nodes.
 
-## LPGExporter
-
-Labeled Property Graph export — Cypher statements for Neo4j and Memgraph:
+## Selective Export
 
 ```python
-from semantica.export import LPGExporter
+# Export a subgraph
+subgraph = graph.subgraph(node_ids=["apple_inc", "steve_jobs"])
+export_rdf(subgraph, "subgraph.ttl", format="turtle")
 
-exporter = LPGExporter()
-
-# CREATE statements
-cypher = exporter.to_cypher(graph)
-exporter.export(graph, "import.cypher", format="cypher")
-
-# MERGE statements (idempotent — safe to re-run)
-cypher_merge = exporter.to_cypher(graph, use_merge=True)
-```
-
-## ArangoAQLExporter
-
-AQL INSERT statements for ArangoDB vertex and edge collections:
-
-```python
-from semantica.export import ArangoAQLExporter
-
-exporter = ArangoAQLExporter(
-    vertex_collection="entities",
-    edge_collection="relationships"
-)
-
-aql = exporter.export(graph)                # returns AQL string
-exporter.export_to_file(graph, "import.aql")
-```
-
-## GraphExporter
-
-Export for visualization tools — GraphML, GEXF, and Graphviz DOT:
-
-```python
-from semantica.export import GraphExporter
-
-exporter = GraphExporter()
-
-exporter.export(graph, "graph.graphml", format="graphml")  # Gephi, yEd
-exporter.export(graph, "graph.gexf",    format="gexf")     # Gephi streaming
-exporter.export(graph, "graph.dot",     format="dot")      # Graphviz
-```
-
-## OWLExporter
-
-OWL 2.0 ontology export in three serialization formats:
-
-```python
-from semantica.export import OWLExporter
-
-exporter = OWLExporter()
-exporter.export(ontology, path="ontology.ttl",  format="turtle")
-exporter.export(ontology, path="ontology.owl",  format="xml")
-exporter.export(ontology, path="ontology.json", format="json-ld")
-```
-
-## CSVExporter
-
-Flat CSV export for spreadsheets and simple data pipelines:
-
-```python
-from semantica.export import CSVExporter
-
-exporter = CSVExporter(delimiter=",")
-exporter.export_nodes(graph, "nodes.csv")
-exporter.export_edges(graph, "edges.csv")
-```
-
-## VectorExporter
-
-Export embedding vectors for use in external vector stores:
-
-```python
-from semantica.export import VectorExporter
-
-exporter = VectorExporter()
-exporter.export(embeddings, metadata, "vectors.json",  format="json")
-exporter.export(embeddings, metadata, "vectors.npy",   format="numpy")
-exporter.export(embeddings, metadata, "vectors.faiss", format="faiss")
-```
-
-## ArrowExporter
-
-Apache Arrow IPC format for zero-copy inter-process transfer:
-
-```python
-from semantica.export import ArrowExporter
-
-exporter = ArrowExporter()
-exporter.export(graph, "graph.arrow")
-```
-
-Requires `pyarrow`. Falls back gracefully if not installed.
-
-## DistanceExporter
-
-Export semantic distance matrices produced by Distance Intelligence (v0.5.0):
-
-```python
-from semantica.export import DistanceExporter
-
-exporter = DistanceExporter()
-exporter.export_matrix(distance_matrix, node_labels, "distances.csv")
-exporter.export_ego(ego_neighborhood, center_node="Apple Inc.", path="ego.json")
-```
-
-## ReportGenerator
-
-Generate human-readable analytics reports from graph metrics:
-
-```python
-from semantica.export import ReportGenerator
-
-generator = ReportGenerator()
-
-generator.generate(graph, analytics_result, "report.html",     format="html")
-generator.generate(graph, analytics_result, "report.md",       format="markdown")
-generator.generate(graph, analytics_result, "report.json",     format="json")
+# Export nodes by type
+org_nodes = graph.filter(node_type="Organization")
+export_parquet(org_nodes, "organizations.parquet")
 ```
 
 ## Convenience Functions
@@ -212,17 +291,55 @@ export_arango(graph,  "import.aql")
 export_graph(graph,   "graph.graphml", format="graphml")
 ```
 
-## Selective Export
+## Format Reference
 
-```python
-# Export a subgraph
-subgraph = graph.subgraph(node_ids=["apple_inc", "steve_jobs"])
-export_rdf(subgraph, "subgraph.ttl", format="turtle")
+| Format | Exporter | Output | Best For |
+| ------ | -------- | ------ | -------- |
+| `turtle` | `RDFExporter` | `.ttl` | Readable RDF, ontology sharing |
+| `json-ld` | `RDFExporter` | `.jsonld` | APIs, Linked Data, JSON pipelines |
+| `nt` | `RDFExporter` | `.nt` | Streaming RDF, line-by-line processing |
+| `xml` | `RDFExporter` | `.xml` | W3C RDF/XML, broadest compatibility |
+| `parquet` | `ParquetExporter` | `.parquet` | Spark, BigQuery, Databricks, Snowflake |
+| `cypher` | `LPGExporter` | `.cypher` | Neo4j, Memgraph import |
+| `aql` | `ArangoAQLExporter` | `.aql` | ArangoDB vertex + edge collections |
+| `graphml` | `GraphExporter` | `.graphml` | Gephi, yEd visualization |
+| `gexf` | `GraphExporter` | `.gexf` | Gephi streaming format |
+| `dot` | `GraphExporter` | `.dot` | Graphviz rendering |
+| `owl` | `OWLExporter` | `.owl` / `.ttl` | OWL 2.0 ontology distribution |
+| `csv` | `CSVExporter` | `.csv` | Spreadsheets, simple pipelines |
+| `yaml` | `YAMLExporter` | `.yaml` | Human-readable, config-driven use |
+| `arrow` | `ArrowExporter` | `.arrow` | Zero-copy inter-process transfer |
+| `numpy` | `VectorExporter` | `.npy` | NumPy arrays from embeddings |
+| `faiss` | `VectorExporter` | `.faiss` | Direct FAISS index files |
+| `distance-matrix` | `DistanceExporter` | `.csv` / `.json` | Distance Intelligence matrices |
+| `html` | `ReportGenerator` | `.html` | Human-readable analytics reports |
+| `markdown` | `ReportGenerator` | `.md` | Documentation, GitHub |
 
-# Export nodes by type
-org_nodes = graph.filter(node_type="Organization")
-export_parquet(org_nodes, "organizations.parquet")
-```
+## Tips and Common Pitfalls
+
+<Tip>
+  **Use `turtle` for human readability, `nt` for streaming.** Turtle is compact and readable for debugging and sharing ontologies. N-Triples (`.nt`) is line-oriented — one triple per line — making it safe to stream, concatenate, and process with standard Unix tools without loading the full file.
+</Tip>
+
+<Tip>
+  **Use `ParquetExporter` for downstream analytics.** Parquet preserves column types (int, float, datetime) that CSV loses and is natively supported by Spark, BigQuery, Databricks, and Snowflake. Use `compression="snappy"` for a good balance of speed and compression ratio.
+</Tip>
+
+<Warning>
+  **Stream large graphs with `export_stream()`.** For graphs with more than 500k nodes, use `exporter.export_stream(graph, ...)` instead of `exporter.export_to_file()`. Streaming writes incrementally without buffering the full graph in memory — without it, a million-node export will likely OOM.
+</Warning>
+
+<Tip>
+  **Include provenance for compliance exports.** For HIPAA, SOX, or FDA 21 CFR Part 11 exports, pass `include_provenance=True` to `RDFExporter`. This embeds W3C PROV-O lineage triples inline — auditors can verify every fact's source from a single file rather than cross-referencing separate systems.
+</Tip>
+
+<Tip>
+  **Use selective export to reduce file size.** `graph.subgraph(node_ids=[...])` and `graph.filter(node_type="Organization")` let you export only the relevant subset. Full graph exports for compliance reports include noise; scoped exports are faster to produce, review, and transfer.
+</Tip>
+
+<Warning>
+  **Match your export format to your consumer.** Neo4j → `cypher`; ArangoDB → `aql`; Gephi/yEd → `graphml` or `gexf`; semantic web tools → `turtle` or `json-ld`; analytics pipelines → `parquet`; zero-copy IPC → `arrow`. Using the wrong format forces the consumer to convert it, adding latency and potential data loss.
+</Warning>
 
 <CardGroup cols={2}>
   <Card title="Triplet Store" icon="table" href="triplet_store">

docs/reference/graph_store.md

@@ -8,119 +8,151 @@ icon: "server"
 
 ## What You Get
 
-- **`GraphStore`** — unified interface across all backends
-- **Backends** — Neo4j, FalkorDB, Apache AGE (PostgreSQL), Amazon Neptune, NetworkX (in-memory)
-- **Cypher queries** — full Cypher support for Neo4j and FalkorDB
-- **Bulk operations** — batched node and edge loading with configurable batch sizes
-- **Schema management** — create indexes and uniqueness constraints
-- **Path traversal** — find paths between nodes with hop limits and relationship type filters
+<CardGroup cols={2}>
+  <Card title="GraphStore" icon="server">
+    Unified interface across Neo4j, FalkorDB, Apache AGE, Amazon Neptune, and NetworkX.
+  </Card>
+  <Card title="QueryEngine" icon="magnifying-glass">
+    Parameterized Cypher construction, query optimization, and result caching.
+  </Card>
+  <Card title="GraphAnalytics" icon="chart-line">
+    Centrality, community detection, and path algorithms running directly against the backend.
+  </Card>
+  <Card title="Bulk Operations" icon="layer-group">
+    Batched node and edge loading with configurable batch sizes — 10–100× faster than individual writes.
+  </Card>
+  <Card title="Schema Management" icon="table">
+    Create indexes and uniqueness constraints to optimize query performance.
+  </Card>
+  <Card title="Path Traversal" icon="route">
+    Find paths between nodes with hop limits and relationship type filters.
+  </Card>
+</CardGroup>
 
-## Basic Usage
+## Quick Start
 
-```python
-from semantica.graph_store import GraphStore
+<Steps>
+  <Step title="Connect to a graph database">
+    ```python
+    from semantica.graph_store import GraphStore
 
-store = GraphStore(
-    backend="neo4j",
-    uri="bolt://localhost:7687",
-    user="neo4j",
-    password="password"
-)
-
-store.add_nodes(entities)
-store.add_edges(relationships)
-
-results = store.query("MATCH (n)-[r]->(m) RETURN n, r, m LIMIT 10")
-```
+    store = GraphStore(
+        backend="neo4j",
+        uri="bolt://localhost:7687",
+        user="neo4j",
+        password="password",
+    )
+    ```
+  </Step>
+  <Step title="Create indexes before loading data">
+    ```python
+    store.create_index(label="Person",       property="name")
+    store.create_index(label="Organization", property="name")
+    store.create_constraint(label="Organization", property="id", constraint_type="unique")
+    ```
+  </Step>
+  <Step title="Bulk-load nodes and edges">
+    ```python
+    store.add_nodes_bulk(entities,      batch_size=1000)
+    store.add_edges_bulk(relationships, batch_size=1000)
+    ```
+  </Step>
+  <Step title="Query the graph">
+    ```python
+    results = store.query(
+        "MATCH (p:Person)-[:WORKS_FOR]->(o:Organization) WHERE o.name = $org RETURN p",
+        parameters={"org": "Apple Inc."},
+    )
+    ```
+  </Step>
+</Steps>
 
 ## Backends
 
 <Tabs>
-  <Tab title="Neo4j">
+  <Tab title="Neo4j (recommended)">
+    ```python
+    from semantica.graph_store import GraphStore
 
-```python
-store = GraphStore(
-    backend="neo4j",
-    uri="bolt://localhost:7687",
-    user="neo4j",
-    password="password",
-    database="neo4j"    # optional — targets default database
-)
-```
-
-Best for: production workloads, complex Cypher queries, Bloom visualization.
+    store = GraphStore(
+        backend="neo4j",
+        uri="bolt://localhost:7687",
+        user="neo4j",
+        password="password",
+        database="neo4j",    # optional — targets default database
+    )
+    ```
 
+    Best for: production workloads, complex Cypher queries, Bloom visualization.
   </Tab>
   <Tab title="FalkorDB">
+    ```python
+    store = GraphStore(
+        backend="falkordb",
+        host="localhost",
+        port=6379,
+        graph_name="semantica",
+    )
+    ```
 
-```python
-store = GraphStore(
-    backend="falkordb",
-    host="localhost",
-    port=6379,
-    graph_name="semantica"
-)
-```
-
-Best for: ultra-low latency queries over Redis protocol, edge deployments.
-
+    Best for: ultra-low latency queries over Redis protocol, edge deployments.
   </Tab>
   <Tab title="Apache AGE">
+    ```python
+    store = GraphStore(
+        backend="apache_age",
+        connection_string="postgresql://user:pass@localhost/graphdb",
+        graph_name="semantica",
+    )
+    ```
 
-```python
-store = GraphStore(
-    backend="apache_age",
-    connection_string="postgresql://user:pass@localhost/graphdb",
-    graph_name="semantica"
-)
-```
-
-Best for: teams already running PostgreSQL who want graph queries without a separate service. See the [Apache AGE Guide](../graph_stores/apache_age) for setup.
-
+    Best for: teams already running PostgreSQL who want graph queries without a separate service. See the [Apache AGE Guide](../graph_stores/apache_age) for setup.
   </Tab>
   <Tab title="Amazon Neptune">
+    ```python
+    from semantica.graph_store import GraphStore
 
-```python
-store = GraphStore(
-    backend="neptune",
-    endpoint="your-cluster.cluster-xxxx.us-east-1.neptune.amazonaws.com",
-    port=8182,
-    region="us-east-1"
-)
-```
+    # IAM authentication (recommended for production)
+    store = GraphStore(
+        backend="neptune",
+        endpoint="your-cluster.cluster-xxxx.us-east-1.neptune.amazonaws.com",
+        port=8182,
+        region="us-east-1",
+        use_iam_auth=True,    # uses boto3 default credential chain
+    )
 
-Best for: managed AWS deployments needing both SPARQL and Gremlin support.
+    # Gremlin traversal
+    results = store.query("g.V().hasLabel('Person').limit(10)")
 
+    # openCypher query
+    results = store.query(
+        "MATCH (p:Person)-[:WORKS_FOR]->(o:Organization) RETURN p, o",
+        query_language="opencypher",
+    )
+    ```
+
+    Best for: managed AWS deployments needing both SPARQL and Gremlin support.
   </Tab>
-  <Tab title="In-Memory">
+  <Tab title="NetworkX (in-memory)">
+    ```python
+    store = GraphStore(backend="networkx")
+    ```
 
-```python
-store = GraphStore(backend="networkx")
-```
+    Best for: development, testing, and graphs that fit in RAM. Data is not persisted.
+  </Tab>
+  <Tab title="Backend Comparison">
 
-Best for: development, testing, and graphs that fit in RAM. Data is not persisted.
+    | Backend | Query Language | Deployment | IAM Auth | Best For |
+    | ------- | -------------- | ---------- | -------- | -------- |
+    | Neo4j | Cypher | Self-hosted / Aura | No | Production, complex traversals, Bloom UI |
+    | FalkorDB | Cypher | Redis-based | No | Ultra-low latency, edge deployments |
+    | Apache AGE | OpenCypher | PostgreSQL extension | No | Teams already on Postgres |
+    | Amazon Neptune | SPARQL / Gremlin / openCypher | AWS managed | Yes | Cloud-native, multi-model, compliance |
+    | NetworkX | Python API | In-memory | No | Development, unit testing |
 
   </Tab>
 </Tabs>
 
-## Querying
-
-```python
-# Cypher query with parameters (Neo4j, FalkorDB)
-results = store.query(
-    "MATCH (p:Person)-[:WORKS_FOR]->(o:Organization) WHERE o.name = $org RETURN p",
-    parameters={"org": "Apple Inc."}
-)
-
-# Path traversal between two nodes
-paths = store.find_paths(
-    start_node="steve_jobs",
-    end_node="apple_inc",
-    max_hops=3,
-    relationship_types=["FOUNDED", "WORKED_AT"]
-)
-```
-
 ## Graph Operations
 
 ```python
@@ -128,19 +160,19 @@ paths = store.find_paths(
 store.add_node(
     "apple_inc",
     node_type="Organization",
-    properties={"founded": 1976, "hq": "Cupertino"}
+    properties={"founded": 1976, "hq": "Cupertino"},
 )
 
 # Add a directed relationship
 store.add_edge(
     "steve_jobs", "apple_inc",
     "FOUNDED",
-    properties={"year": 1976}
+    properties={"year": 1976},
 )
 
 # Bulk operations — use for large datasets
-store.add_nodes_bulk(entities,       batch_size=1000)
-store.add_edges_bulk(relationships,  batch_size=1000)
+store.add_nodes_bulk(entities,      batch_size=1000)
+store.add_edges_bulk(relationships, batch_size=1000)
 
 # Delete
 store.delete_node("node_id")
@@ -150,13 +182,85 @@ store.delete_edge("edge_id")
 neighbors = store.get_neighbors(
     "apple_inc",
     relationship_type="HAS_EMPLOYEE",
-    direction="in"    # "in" | "out" | "both"
+    direction="in",    # "in" | "out" | "both"
+)
+
+# Path traversal between two nodes
+paths = store.find_paths(
+    start_node="steve_jobs",
+    end_node="apple_inc",
+    max_hops=3,
+    relationship_types=["FOUNDED", "WORKED_AT"],
 )
 ```
 
-## Schema Management
+## QueryEngine
 
-Create indexes and constraints to improve query performance:
+`QueryEngine` handles query construction, optimization, and caching:
+
+```python
+from semantica.graph_store import QueryEngine, GraphStore
+
+store  = GraphStore(backend="neo4j", uri="bolt://localhost:7687", user="neo4j", password="password")
+engine = QueryEngine(store, cache_ttl=300)   # cache results for 5 minutes
+
+# Build parameterized Cypher
+query, params = engine.build_query(
+    node_labels=["Person"],
+    filters={"department": "Engineering"},
+    return_fields=["name", "email"],
+    limit=50,
+)
+results = engine.execute(query, params)
+
+# Explain query plan (Neo4j)
+plan = engine.explain(query, params)
+print(plan["profile"])
+
+# Flush query cache
+engine.clear_cache()
+```
+
+## GraphAnalytics
+
+Built-in graph analytics that run directly against the stored backend — no data export required:
+
+```python
+from semantica.graph_store import GraphAnalytics, GraphStore
+
+store     = GraphStore(backend="neo4j", uri="bolt://localhost:7687", user="neo4j", password="password")
+analytics = GraphAnalytics(store)
+
+# Centrality
+centrality  = analytics.degree_centrality(node_label="Person", relationship_type="KNOWS")
+betweenness = analytics.betweenness_centrality(node_label="Person")
+
+# Community detection
+communities = analytics.detect_communities(
+    node_label="Person",
+    relationship_type="KNOWS",
+    algorithm="louvain",
+)
+print(f"Detected {len(communities)} communities")
+
+# Shortest path
+path = analytics.shortest_path("alice", "charlie", relationship_type="KNOWS")
+print(f"Hops: {len(path) - 1}, Path: {' → '.join(path)}")
+
+# All paths up to max_hops
+all_paths = analytics.all_paths("alice", "charlie", max_hops=4)
+```
+
+| Method | Description |
+| ------ | ----------- |
+| `degree_centrality(node_label, relationship_type)` | Degree-based node importance |
+| `betweenness_centrality(node_label)` | Bridge-based importance |
+| `pagerank(node_label, relationship_type, damping)` | PageRank scores |
+| `detect_communities(node_label, relationship_type, algorithm)` | Louvain / Label Propagation |
+| `shortest_path(source, target, relationship_type)` | Minimum-hop path |
+| `all_paths(source, target, max_hops)` | All paths up to max depth |
+
+## Schema Management
 
 ```python
 # Index for fast label lookups
@@ -166,7 +270,7 @@ store.create_index(label="Person", property="name")
 store.create_constraint(
     label="Organization",
     property="id",
-    constraint_type="unique"
+    constraint_type="unique",
 )
 
 # Inspect current schema
@@ -176,15 +280,31 @@ print(schema["indexes"])
 print(schema["constraints"])
 ```
 
-## Backend Comparison
+## Tips and Common Pitfalls
 
-| Backend | Query Language | Deployment | Best For |
-| ------- | -------------- | ---------- | -------- |
-| Neo4j | Cypher | Self-hosted / Aura | Production, complex traversals |
-| FalkorDB | Cypher | Redis-based | Ultra-low latency, edge |
-| Apache AGE | OpenCypher | PostgreSQL | Teams already on Postgres |
-| Amazon Neptune | SPARQL / Gremlin | AWS managed | Cloud-native AWS deployments |
-| NetworkX | Python API | In-memory | Development and testing |
+<Tip>
+  **Use `NetworkX` for development, Neo4j or FalkorDB for production.** `backend="networkx"` requires zero setup and runs in memory — ideal for local development and CI tests. Switch to a persistent backend before deploying — no code changes needed, just the backend parameter.
+</Tip>
+
+<Warning>
+  **Create indexes before bulk loading.** `store.create_index(label="Person", property="name")` makes `MATCH` queries on `name` orders of magnitude faster. Without indexes, every query does a full scan. Create indexes first, then load data.
+</Warning>
+
+<Tip>
+  **Use `add_nodes_bulk()` and `add_edges_bulk()` for large datasets.** Individual `add_node()` calls issue one network round-trip each. Bulk operations batch thousands of writes into a single transaction — 10–100× faster for initial loads.
+</Tip>
+
+<Warning>
+  **Use parameterized queries, never string interpolation.** `store.query("WHERE n.name = $name", parameters={"name": user_input})` prevents Cypher injection attacks. Never use `f"WHERE n.name = '{user_input}'"`.
+</Warning>
+
+<Tip>
+  **Enable `QueryEngine` caching for read-heavy workloads.** `QueryEngine(store, cache_ttl=300)` avoids repeated round-trips for identical queries within the cache window — useful for analytics dashboards that refresh frequently with the same aggregation queries.
+</Tip>
+
+<Warning>
+  **Apache AGE requires the PostgreSQL extension installed.** `backend="apache_age"` calls the AGE extension functions. If AGE is not installed in your PostgreSQL instance, you'll get a `ProgrammingError`. See the [Apache AGE Guide](../graph_stores/apache_age) for setup instructions.
+</Warning>
 
 <CardGroup cols={2}>
   <Card title="KG Module" icon="diagram-project" href="kg">

docs/reference/ingest.md

@@ -8,161 +8,409 @@ icon: "database"
 
 ## What You Get
 
-- **`FileIngestor`** — PDF, DOCX, HTML, JSON, CSV, Excel, PPTX, ZIP/TAR archives
-- **`ParquetIngestor`** — PyArrow-based Parquet with Hive-style partition support (v0.5.0)
-- **`XMLIngestor`** — XXE-safe lxml with XSD/DTD validation (v0.5.0)
-- **`WebIngestor`** — configurable web crawling with robots.txt support
-- **`FeedIngestor`** — RSS/Atom feeds with live monitoring
-- **`DBIngestor`** / **`SnowflakeIngestor`** — SQL databases and Snowflake
-- **`StreamIngestor`** — Kafka, RabbitMQ, Kinesis, Pulsar real-time streams
-- **`RepoIngestor`**, **`EmailIngestor`**, **`MCPIngestor`**, **`S3Ingestor`**, **`GCSIngestor`**
+<CardGroup cols={2}>
+  <Card title="FileIngestor" icon="file">
+    PDF, DOCX, HTML, JSON, CSV, Excel, PPTX, and ZIP/TAR archives — type auto-detected from extension.
+  </Card>
+  <Card title="ParquetIngestor" icon="table">
+    PyArrow-based Parquet with Hive-style partition support and column selection (v0.5.0).
+  </Card>
+  <Card title="XMLIngestor" icon="code">
+    XXE-safe lxml with XSD/DTD validation and directory scanning (v0.5.0).
+  </Card>
+  <Card title="StreamIngestor" icon="wave-square">
+    Real-time ingestion from Kafka, RabbitMQ, AWS Kinesis, and Apache Pulsar.
+  </Card>
+  <Card title="Cloud Storage" icon="cloud">
+    S3Ingestor, GCSIngestor, and GDriveIngestor with authentication options.
+  </Card>
+  <Card title="Database Ingestors" icon="database">
+    DBIngestor, SnowflakeIngestor, MongoIngestor, and DuckDBIngestor.
+  </Card>
+</CardGroup>
 
-## FileIngestor
+## Quick Start
+
+<Steps>
+  <Step title="Ingest local files">
+    ```python
+    from semantica.ingest import FileIngestor
+
+    ingestor = FileIngestor()
+
+    # Single file — type auto-detected from extension
+    sources = ingestor.ingest("data/report.pdf")
+
+    # Recursive directory scan
+    sources = ingestor.ingest_directory("data/", recursive=True)
+
+    # Glob pattern
+    sources = ingestor.ingest("data/**/*.docx")
+    ```
+  </Step>
+  <Step title="Connect to a remote source">
+    ```python
+    from semantica.ingest import DBIngestor
+
+    ingestor = DBIngestor(
+        connection_string="postgresql://user:pass@localhost/db",
+        query="SELECT id, content, created_at FROM documents WHERE status='active'"
+    )
+    sources = ingestor.ingest()
+    ```
+  </Step>
+  <Step title="Feed sources into the pipeline">
+    ```python
+    from semantica.pipeline import Pipeline
+    from semantica.parse import DocumentParser
+    from semantica.semantic_extract import NERExtractor
+    from semantica.llms import Groq
+
+    llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+
+    pipeline = Pipeline()
+    pipeline.add_step("ingest",   FileIngestor())
+    pipeline.add_step("parse",    DocumentParser())
+    pipeline.add_step("extract",  NERExtractor(method="llm", llm_provider=llm))
+    result = pipeline.run("data/")
+    ```
+  </Step>
+</Steps>
+
+## Ingestors
+
+<Tabs>
+  <Tab title="File-Based">
+    ### FileIngestor
+
+    ```python
+    from semantica.ingest import FileIngestor
+
+    ingestor = FileIngestor()
+    sources  = ingestor.ingest("data/report.pdf")
+    sources  = ingestor.ingest_directory("data/", recursive=True)
+    sources  = ingestor.ingest("data/**/*.docx")
+    ```
+
+    Supported formats: PDF, DOCX, TXT, HTML, JSON, CSV, Excel (XLSX/XLS), PPTX, ZIP/TAR archives.
+
+    ### ParquetIngestor (v0.5.0)
+
+    PyArrow-based ingestion for Apache Parquet files, including Hive-style partitioned datasets:
+
+    ```python
+    from semantica.ingest import ParquetIngestor
+
+    ingestor = ParquetIngestor()
+
+    # Single Parquet file
+    sources = ingestor.ingest("data/events.parquet")
+
+    # Partitioned directory (year=2024/month=01/...)
+    sources = ingestor.ingest("data/partitioned/")
+
+    # Load only specific columns
+    sources = ingestor.ingest("data/events.parquet", columns=["id", "text", "timestamp"])
+    ```
+
+    ### XMLIngestor (v0.5.0)
+
+    XXE-safe lxml-based ingestion with optional schema validation:
+
+    ```python
+    from semantica.ingest import XMLIngestor
+
+    ingestor = XMLIngestor()
+    sources  = ingestor.ingest("data/records.xml")
+
+    # With XSD validation
+    ingestor = XMLIngestor(validate_xsd="schema.xsd")
+    sources  = ingestor.ingest("data/records/")
+
+    # With DTD validation
+    ingestor = XMLIngestor(validate_dtd=True)
+    sources  = ingestor.ingest("data/feed.xml")
+    ```
+
+    <Note>
+      `XMLIngestor` uses lxml with `resolve_entities=False` to prevent XML External Entity (XXE) injection attacks.
+    </Note>
+  </Tab>
+  <Tab title="Web & Feed">
+    ### WebIngestor
+
+    ```python
+    from semantica.ingest import WebIngestor
+
+    ingestor = WebIngestor(
+        rate_limit=1.0,       # seconds between requests
+        respect_robots=True,  # honor robots.txt
+        max_depth=2           # crawl depth from seed URLs
+    )
+
+    sources = ingestor.ingest("https://example.com/about")
+    sources = ingestor.ingest_urls([
+        "https://example.com/page1",
+        "https://example.com/page2",
+    ])
+    ```
+
+    ### FeedIngestor (RSS/Atom)
+
+    ```python
+    from semantica.ingest import FeedIngestor
+
+    ingestor = FeedIngestor()
+    sources  = ingestor.ingest("https://feeds.example.com/rss")
+
+    # Live monitoring — callback fires on new items
+    ingestor.monitor(
+        "https://feeds.example.com/rss",
+        interval=300,
+        callback=process_new_items
+    )
+    ```
+
+    ### RepoIngestor
+
+    Ingest Git repositories — source code, commit history, and dependency graphs:
+
+    ```python
+    from semantica.ingest import RepoIngestor
+
+    ingestor = RepoIngestor(
+        branch="main",
+        file_types=[".py", ".md", ".yaml"],
+        include_commits=True,
+        commit_range="HEAD~100..HEAD",
+    )
+
+    sources = ingestor.ingest("https://github.com/org/repo")
+    sources = ingestor.ingest("/path/to/local/repo")
+    ```
+
+    ### EmailIngestor
+
+    Ingest emails via IMAP or POP3 with attachment extraction and thread analysis:
+
+    ```python
+    from semantica.ingest import EmailIngestor
+    import os
+
+    ingestor = EmailIngestor(
+        protocol="imap",
+        host="imap.gmail.com",
+        port=993,
+        use_ssl=True,
+        username=os.getenv("EMAIL_USER"),
+        password=os.getenv("EMAIL_PASS"),
+        folder="INBOX",
+        attachment_types=[".pdf", ".docx", ".txt"],
+        include_thread_analysis=True,
+        max_emails=500,
+    )
+    sources = ingestor.ingest()
+    ```
+  </Tab>
+  <Tab title="Cloud Storage">
+    ### S3Ingestor
+
+    Ingest files directly from AWS S3 buckets:
+
+    ```python
+    from semantica.ingest import S3Ingestor
+    import os
+
+    ingestor = S3Ingestor(
+        bucket="my-documents-bucket",
+        prefix="reports/2024/",
+        region="us-east-1",
+        aws_access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
+        aws_secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
+        # Or omit credentials to use IAM instance profile
+    )
+    sources = ingestor.ingest()
+    sources = ingestor.ingest(pattern="**/*.pdf")
+    ```
+
+    ### GCSIngestor
+
+    Ingest files from Google Cloud Storage:
+
+    ```python
+    from semantica.ingest import GCSIngestor
+
+    ingestor = GCSIngestor(
+        bucket="my-gcs-bucket",
+        prefix="data/",
+        credentials_file="gcp-credentials.json",  # or use ADC
+    )
+    sources = ingestor.ingest()
+    ```
+
+    ### GDriveIngestor
+
+    Ingest files from Google Drive folders via OAuth 2.0:
+
+    ```python
+    from semantica.ingest import GDriveIngestor
+
+    ingestor = GDriveIngestor(
+        credentials_file="oauth_credentials.json",
+        token_file="token.json",
+        folder_id="1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs9",
+        file_types=["pdf", "docx", "txt"],
+        recursive=True,
+    )
+    sources = ingestor.ingest()
+    ```
+  </Tab>
+  <Tab title="Database">
+    ### DBIngestor (SQL)
+
+    ```python
+    from semantica.ingest import DBIngestor
+
+    ingestor = DBIngestor(
+        connection_string="postgresql://user:pass@localhost/db",
+        query="SELECT id, content, created_at FROM documents WHERE status='active'"
+    )
+    sources = ingestor.ingest()
+    ```
+
+    ### SnowflakeIngestor
+
+    ```python
+    from semantica.ingest import SnowflakeIngestor
+    import os
+
+    ingestor = SnowflakeIngestor(
+        account=os.getenv("SNOWFLAKE_ACCOUNT"),
+        user=os.getenv("SNOWFLAKE_USER"),
+        password=os.getenv("SNOWFLAKE_PASSWORD"),
+        warehouse="COMPUTE_WH",
+        database="ANALYTICS",
+        schema="PUBLIC"
+    )
+    sources = ingestor.ingest(query="SELECT * FROM documents")
+    ```
+
+    ### MongoIngestor
+
+    Ingest documents from MongoDB collections:
+
+    ```python
+    from semantica.ingest import MongoIngestor
+    import os
+
+    ingestor = MongoIngestor(
+        connection_string=os.getenv("MONGO_URI"),
+        database="mydb",
+        collection="articles",
+        query={"status": "published", "year": {"$gte": 2022}},
+        projection={"title": 1, "body": 1, "author": 1},
+        content_field="body",
+        limit=10000,
+    )
+    sources = ingestor.ingest()
+    ```
+
+    ### DuckDBIngestor
+
+    Ingest data from DuckDB databases or directly from Parquet/CSV files via DuckDB SQL:
+
+    ```python
+    from semantica.ingest import DuckDBIngestor
+
+    # In-memory DuckDB — query a Parquet file directly
+    ingestor = DuckDBIngestor(
+        query="SELECT id, text, created_at FROM read_parquet('data/*.parquet') WHERE year >= 2023",
+    )
+    sources = ingestor.ingest()
+
+    # Persistent DuckDB database file
+    ingestor = DuckDBIngestor(
+        database_path="analytics.duckdb",
+        query="SELECT doc_id AS id, content, metadata FROM documents",
+        content_field="content",
+    )
+    sources = ingestor.ingest()
+    ```
+  </Tab>
+  <Tab title="Stream">
+    ### StreamIngestor
+
+    Real-time ingestion from message brokers:
+
+    ```python
+    from semantica.ingest import StreamIngestor
+
+    # Kafka
+    ingestor = StreamIngestor(
+        backend="kafka",
+        bootstrap_servers="localhost:9092",
+        topic="documents",
+        group_id="semantica-consumer",
+        auto_offset_reset="earliest",
+    )
+    sources = ingestor.ingest(max_messages=1000)
+
+    # RabbitMQ
+    ingestor = StreamIngestor(
+        backend="rabbitmq",
+        host="localhost",
+        queue="document_queue",
+        routing_key="docs.ingest",
+        prefetch_count=100,
+    )
+
+    # AWS Kinesis
+    ingestor = StreamIngestor(
+        backend="kinesis",
+        stream_name="documents-stream",
+        region="us-east-1",
+        shard_iterator_type="TRIM_HORIZON",
+    )
+
+    # Apache Pulsar
+    ingestor = StreamIngestor(
+        backend="pulsar",
+        service_url="pulsar://localhost:6650",
+        topic="persistent://public/default/documents",
+        subscription_name="semantica-sub",
+    )
+
+    # Live monitoring — callback fires on each new message
+    ingestor.monitor(callback=process_document, poll_interval=1.0)
+    ```
+
+    <Warning>
+      Without a `max_messages` limit, `StreamIngestor.ingest()` blocks indefinitely waiting for new messages. Use `max_messages=1000` for batch processing; use `.monitor(callback=...)` for continuous streaming.
+    </Warning>
+  </Tab>
+</Tabs>
+
+## OntologyIngestor
+
+Ingest existing OWL or RDF ontology files as structured knowledge sources:
 
 ```python
-from semantica.ingest import FileIngestor
+from semantica.ingest import OntologyIngestor
 
-ingestor = FileIngestor()
-
-# Single file — type auto-detected from extension
-sources = ingestor.ingest("data/report.pdf")
-
-# Recursive directory scan
-sources = ingestor.ingest_directory("data/", recursive=True)
-
-# Glob pattern
-sources = ingestor.ingest("data/**/*.docx")
-```
-
-Supported formats: PDF, DOCX, TXT, HTML, JSON, CSV, Excel (XLSX/XLS), PPTX, ZIP/TAR archives.
-
-## ParquetIngestor (v0.5.0)
-
-PyArrow-based ingestion for Apache Parquet files, including Hive-style partitioned datasets:
-
-```python
-from semantica.ingest import ParquetIngestor
-
-ingestor = ParquetIngestor()
-
-# Single Parquet file
-sources = ingestor.ingest("data/events.parquet")
-
-# Partitioned directory (year=2024/month=01/...)
-sources = ingestor.ingest("data/partitioned/")
-
-# Load only specific columns
-sources = ingestor.ingest("data/events.parquet", columns=["id", "text", "timestamp"])
-```
-
-## XMLIngestor (v0.5.0)
-
-XXE-safe lxml-based ingestion with optional schema validation:
-
-```python
-from semantica.ingest import XMLIngestor
-
-# Basic ingestion
-ingestor = XMLIngestor()
-sources = ingestor.ingest("data/records.xml")
-
-# With XSD validation
-ingestor = XMLIngestor(validate_xsd="schema.xsd")
-sources = ingestor.ingest("data/records/")
-
-# With DTD validation
-ingestor = XMLIngestor(validate_dtd=True)
-sources = ingestor.ingest("data/feed.xml")
-```
-
-<Note>
-  `XMLIngestor` uses lxml with `resolve_entities=False` to prevent XML External Entity (XXE) injection attacks.
-</Note>
-
-## WebIngestor
-
-```python
-from semantica.ingest import WebIngestor
-
-ingestor = WebIngestor(
-    rate_limit=1.0,       # seconds between requests
-    respect_robots=True,  # honor robots.txt
-    max_depth=2           # crawl depth from seed URLs
+ingestor = OntologyIngestor(
+    format="turtle",   # "turtle" | "xml" | "json-ld" | "nt" | "n3"
 )
 
-# Single URL
-sources = ingestor.ingest("https://example.com/about")
-
-# Multiple URLs
-sources = ingestor.ingest_urls([
-    "https://example.com/page1",
-    "https://example.com/page2",
-])
+sources = ingestor.ingest("domain_ontology.owl")
+sources = ingestor.ingest("ontologies/")
 ```
 
-## FeedIngestor (RSS/Atom)
-
-```python
-from semantica.ingest import FeedIngestor
-
-ingestor = FeedIngestor()
-sources = ingestor.ingest("https://feeds.example.com/rss")
-
-# Live monitoring — callback fires on new items
-ingestor.monitor(
-    "https://feeds.example.com/rss",
-    interval=300,
-    callback=process_new_items
-)
-```
-
-## DBIngestor (SQL)
-
-```python
-from semantica.ingest import DBIngestor
-
-ingestor = DBIngestor(
-    connection_string="postgresql://user:pass@localhost/db",
-    query="SELECT id, content, created_at FROM documents WHERE status='active'"
-)
-sources = ingestor.ingest()
-```
-
-## SnowflakeIngestor
-
-```python
-from semantica.ingest import SnowflakeIngestor
-import os
-
-ingestor = SnowflakeIngestor(
-    account=os.getenv("SNOWFLAKE_ACCOUNT"),
-    user=os.getenv("SNOWFLAKE_USER"),
-    password=os.getenv("SNOWFLAKE_PASSWORD"),
-    warehouse="COMPUTE_WH",
-    database="ANALYTICS",
-    schema="PUBLIC"
-)
-sources = ingestor.ingest(query="SELECT * FROM documents")
-```
-
-## Other Ingestors
-
-| Class | Source |
-| ----- | ------ |
-| `StreamIngestor` | Kafka, RabbitMQ, Kinesis, Pulsar |
-| `RepoIngestor` | Git repositories (GitHub, GitLab) |
-| `EmailIngestor` | IMAP/POP3 servers with attachment extraction |
-| `MCPIngestor` | Model Context Protocol servers |
-| `S3Ingestor` | AWS S3 buckets |
-| `GCSIngestor` | Google Cloud Storage |
-| `MongoIngestor` | MongoDB collections |
-| `DuckDBIngestor` | DuckDB databases |
-| `GDriveIngestor` | Google Drive |
-
 ## DataSource Object
 
 All ingestors return a list of `DataSource` objects with a consistent schema:
 
+<Accordion title="DataSource schema">
+
 ```python
 @dataclass
 class DataSource:
@@ -173,6 +421,8 @@ class DataSource:
     raw_bytes:   Optional[bytes] # original binary content if available
 ```
 
+</Accordion>
+
 ## Custom Ingestors
 
 Register a custom ingestor and it participates in the full pipeline:
@@ -181,12 +431,37 @@ Register a custom ingestor and it participates in the full pipeline:
 from semantica.ingest.registry import method_registry
 
 def my_ingestor(source, **kwargs):
-    # Return a list of DataSource-compatible dicts
     return [{"content": "...", "metadata": {}, "source_id": source}]
 
 method_registry.register("file", "my_format", my_ingestor)
 ```
 
+## Tips and Common Pitfalls
+
+<Tip>
+  **`FileIngestor` is always the fastest path for local files.** It auto-detects format from extension, handles ZIP/TAR archives automatically, and supports glob patterns. Only reach for `DoclingParser` when `DocumentParser` can't handle your layout.
+</Tip>
+
+<Tip>
+  **Use `ParquetIngestor` instead of `FileIngestor` for structured analytical data.** Parquet ingestion preserves column types (int, float, datetime) that CSV reading loses. Use `columns=["id", "text"]` to avoid loading unused columns — critical for wide tables with hundreds of columns.
+</Tip>
+
+<Warning>
+  **`XMLIngestor` is XXE-safe by default.** Do not use standard `xml.etree.ElementTree` to pre-parse XML before passing to Semantica — it doesn't block XXE attacks. `XMLIngestor` uses lxml with `resolve_entities=False` to safely parse untrusted XML.
+</Warning>
+
+<Warning>
+  **Stream ingestors need explicit `max_messages` for batch runs.** Without a limit, `StreamIngestor.ingest()` blocks indefinitely waiting for new messages. Use `max_messages=1000` for batch processing; use `.monitor(callback=...)` for continuous streaming.
+</Warning>
+
+<Tip>
+  **Rate-limit web crawling.** `WebIngestor(rate_limit=1.0, respect_robots=True)` is the responsible default. Without rate limiting, you risk getting blocked by the target server or violating its terms of service.
+</Tip>
+
+<Tip>
+  **All ingestors return the same `DataSource` schema.** This means you can mix sources in a single pipeline without any adapter code — `FileIngestor`, `MongoIngestor`, and `StreamIngestor` outputs are all directly composable with `DocumentParser` and `NERExtractor`.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Parse" icon="file-lines" href="parse">
     Parse raw sources into structured text and tables.

docs/reference/kg.md

@@ -8,14 +8,26 @@ icon: "diagram-project"
 
 ## What You Get
 
-- **`GraphBuilder`** — construct graphs from entities and relationships with automatic entity merging
-- **`TemporalKnowledgeGraph`** — time-aware edges (`valid_from`/`valid_until`) and point-in-time queries (v0.4.0)
-- **`DistanceCalculator`** — semantic neighborhoods, N×N distance matrices, and distance band classification (v0.5.0)
-- **`CentralityCalculator`** — PageRank, degree, betweenness, closeness, eigenvector centrality
-- **`CommunityDetector`** — Louvain, Leiden, Label Propagation, K-Clique community detection
-- **`PathFinder`** — Dijkstra, A\*, BFS, K-Shortest path algorithms
-- **`LinkPredictor`** — Preferential Attachment, Jaccard, Adamic-Adar link prediction
-- **`NodeEmbedder`** — Node2Vec, DeepWalk, Word2Vec structural embeddings
+<CardGroup cols={2}>
+  <Card title="GraphBuilder" icon="hammer">
+    Construct graphs from entities and relationships with automatic entity merging.
+  </Card>
+  <Card title="TemporalKnowledgeGraph" icon="clock">
+    Time-aware edges (`valid_from`/`valid_until`) and point-in-time queries (v0.4.0).
+  </Card>
+  <Card title="DistanceCalculator" icon="ruler">
+    Semantic neighborhoods, N×N distance matrices, and distance band classification (v0.5.0).
+  </Card>
+  <Card title="CentralityCalculator" icon="star">
+    PageRank, degree, betweenness, closeness, and eigenvector centrality.
+  </Card>
+  <Card title="CommunityDetector" icon="users">
+    Louvain, Leiden, Label Propagation, and K-Clique community detection.
+  </Card>
+  <Card title="PathFinder" icon="route">
+    Dijkstra, A\*, BFS, and K-Shortest path algorithms.
+  </Card>
+</CardGroup>
 
 <Tip>
   For conflict detection and advanced entity resolution, use `semantica.conflicts` and `semantica.deduplication` alongside this module.
@@ -23,6 +35,54 @@ icon: "diagram-project"
 
 <img src="/assets/img/diagrams/kg-structure.svg" alt="Knowledge graph entity and relation structure: Person, Organization, Location, Date nodes with typed labeled edges" style={{ width: '100%', borderRadius: '12px', margin: '0 0 24px' }} />
 
+## Quick Start
+
+<Steps>
+  <Step title="Build the graph from extracted entities and relationships">
+    ```python
+    from semantica.kg import GraphBuilder
+
+    builder = GraphBuilder(merge_entities=True)
+    kg      = builder.build(entities=entities, relationships=relationships)
+
+    print(f"Nodes: {kg.node_count}, Edges: {kg.edge_count}")
+    ```
+  </Step>
+  <Step title="Run centrality analysis to find key nodes">
+    ```python
+    from semantica.kg import CentralityCalculator
+
+    calc     = CentralityCalculator()
+    pagerank = calc.calculate_pagerank(kg, damping_factor=0.85)
+    top_10   = calc.get_top_nodes(pagerank, top_k=10)
+
+    for node_id, score in top_10:
+        print(f"  {node_id}: {score:.4f}")
+    ```
+  </Step>
+  <Step title="Detect thematic communities">
+    ```python
+    from semantica.kg import CommunityDetector
+
+    detector    = CommunityDetector()
+    communities = detector.detect_communities(kg, algorithm="louvain")
+    metrics     = detector.calculate_community_metrics(kg, communities)
+
+    print(f"Communities: {len(communities)}")
+    ```
+  </Step>
+  <Step title="Persist to a graph database">
+    ```python
+    from semantica.graph_store import GraphStore
+
+    store = GraphStore(backend="neo4j", uri="bolt://localhost:7687",
+                       user="neo4j", password="password")
+    store.add_nodes_bulk(kg.entities,      batch_size=1000)
+    store.add_edges_bulk(kg.relationships, batch_size=1000)
+    ```
+  </Step>
+</Steps>
+
 ## GraphBuilder
 
 Constructs knowledge graphs from extracted entities and relationships:
@@ -31,7 +91,7 @@ Constructs knowledge graphs from extracted entities and relationships:
 from semantica.kg import GraphBuilder
 
 builder = GraphBuilder(merge_entities=True)
-kg = builder.build(entities=entities, relationships=relationships)
+kg      = builder.build(entities=entities, relationships=relationships)
 ```
 
 | Method | Description |
@@ -40,6 +100,10 @@ kg = builder.build(entities=entities, relationships=relationships)
 | `build_single_source(data)` | Build graph from a single data source |
 | `merge_entities()` | Deduplicate and merge entities during construction |
 
+<Warning>
+  Always use `merge_entities=True` in production. Without it, "Steve Jobs" extracted from five different documents creates five separate person nodes. `GraphBuilder(merge_entities=True)` uses edit distance matching to consolidate them at build time.
+</Warning>
+
 ## Temporal Knowledge Graphs (v0.4.0)
 
 Attach `valid_from` / `valid_until` time windows to nodes and edges for point-in-time queries and historical analysis:
@@ -50,7 +114,6 @@ from datetime import datetime
 
 tkg = TemporalKnowledgeGraph()
 
-# Nodes and edges carry explicit validity windows
 tkg.add_node("ceo_role",  valid_from=datetime(2020, 1, 1), valid_until=datetime(2023, 6, 1))
 tkg.add_edge(
     "alice", "acme_corp", "ceo_of",
@@ -61,15 +124,17 @@ tkg.add_edge(
 # Point-in-time snapshot
 snapshot = tkg.at(datetime(2021, 6, 15))
 
-# Diff between two snapshots
-query = TemporalGraphQuery(tkg)
-snapshot_2020 = query.at_time("2020-01-01")
-snapshot_2023 = query.at_time("2023-01-01")
-diff = snapshot_2023.minus(snapshot_2020)
-print(f"New nodes since 2020: {len(diff.nodes)}")
+# Query and diff via TemporalGraphQuery
+query         = TemporalGraphQuery(tkg)
+snap_2020     = query.query_at_time(datetime(2020, 1, 1))
+snap_2023     = query.query_at_time(datetime(2023, 1, 1))
+added         = [r for r in snap_2023.relationships if r not in snap_2020.relationships]
+print(f"New edges since 2020: {len(added)}")
 ```
 
-Supports all 13 Allen interval algebra relations (before, after, meets, overlaps, during, starts, finishes, equals, and their inverses). OWL-Time export available.
+<Note>
+  Edges added without `valid_from`/`valid_until` are treated as **always-valid**. For historical data, always attach timestamps — otherwise point-in-time queries return misleading results.
+</Note>
 
 ## Distance Intelligence (v0.5.0)
 
@@ -90,102 +155,195 @@ matrix = calc.distance_matrix(["Apple Inc.", "Google", "Microsoft"])
 bands = calc.classify_bands(neighborhood)
 ```
 
+<Warning>
+  `DistanceCalculator` is expensive at large scale — the N×N matrix requires embedding all entities and computing pairwise cosine similarities. Cache the result between runs and only recompute for changed entities.
+</Warning>
+
 ## Graph Analytics
 
-### Centrality Analysis
+<Tabs>
+  <Tab title="Centrality">
+    Identify the most structurally important nodes in your graph:
 
-```python
-from semantica.kg import CentralityCalculator
+    ```python
+    from semantica.kg import CentralityCalculator
 
-calculator = CentralityCalculator()
+    calc = CentralityCalculator()
 
-centrality    = calculator.calculate_degree_centrality(graph)
-pagerank      = calculator.calculate_pagerank(graph, damping_factor=0.85)
-betweenness   = calculator.calculate_betweenness_centrality(graph)
-closeness     = calculator.calculate_closeness_centrality(graph)
-eigenvector   = calculator.calculate_eigenvector_centrality(graph)
-all_metrics   = calculator.calculate_all_centrality(graph)
+    pagerank    = calc.calculate_pagerank(kg, damping_factor=0.85)
+    degree      = calc.calculate_degree_centrality(kg)
+    betweenness = calc.calculate_betweenness_centrality(kg)
+    closeness   = calc.calculate_closeness_centrality(kg)
+    eigenvector = calc.calculate_eigenvector_centrality(kg)
+    all_metrics = calc.calculate_all_centrality(kg)
 
-top_nodes = calculator.get_top_nodes(centrality, top_k=10)
-```
+    top_nodes = calc.get_top_nodes(pagerank, top_k=10)
+    ```
 
-| Method | Algorithm |
-| ------ | --------- |
-| `calculate_degree_centrality()` | Degree-based importance |
-| `calculate_betweenness_centrality()` | Bridge-based importance (bottleneck nodes) |
-| `calculate_closeness_centrality()` | Distance-based importance |
-| `calculate_eigenvector_centrality()` | Influence-based importance |
-| `calculate_pagerank()` | Link-based importance (PageRank) |
-| `calculate_all_centrality()` | All measures at once |
+    | Measure | Best For |
+    | ------- | -------- |
+    | PageRank | Overall importance (link-based) |
+    | Degree | Most connected nodes |
+    | Betweenness | Bridge / bottleneck nodes |
+    | Closeness | Fastest to reach all others |
+    | Eigenvector | Connected to other important nodes |
+  </Tab>
+  <Tab title="Community Detection">
+    Partition the graph into thematically dense clusters:
 
-### Community Detection
+    ```python
+    from semantica.kg import CommunityDetector
 
-```python
-from semantica.kg import CommunityDetector
+    detector = CommunityDetector()
 
-detector = CommunityDetector()
+    # Louvain — fast, high quality (default)
+    communities = detector.detect_communities(kg, algorithm="louvain")
 
-# Louvain (default — fast, high quality)
-communities = detector.detect_communities(graph, algorithm="louvain")
+    # Leiden — higher quality, slower
+    leiden_communities = detector.detect_communities_leiden(kg, resolution=1.2)
 
-# Leiden (higher quality, slower)
-leiden_communities = detector.detect_communities_leiden(graph, resolution=1.2)
+    metrics = detector.calculate_community_metrics(kg, communities)
+    print(f"Communities: {len(communities)}")
+    ```
 
-metrics = detector.calculate_community_metrics(graph, communities)
-```
+    Algorithms available: **Louvain**, **Leiden**, **Label Propagation**, **K-Clique Communities**.
 
-Algorithms: Louvain, Leiden, Label Propagation, K-Clique Communities.
+    <Tip>
+      Community detection finds thematic clusters — often corresponding to real-world subject groups. Use cluster membership as context boundaries for GraphRAG retrieval.
+    </Tip>
+  </Tab>
+  <Tab title="Path Finding">
+    Find shortest and alternative paths between nodes:
 
-### Path Finding
+    ```python
+    from semantica.kg import PathFinder
 
-```python
-from semantica.kg import PathFinder
+    finder = PathFinder()
 
-finder = PathFinder()
+    path    = finder.dijkstra_shortest_path(kg, "node_a", "node_b")
+    paths   = finder.all_shortest_paths(kg, "source", "target")
+    k_paths = finder.find_k_shortest_paths(kg, "source", "target", k=3)
+    ```
 
-path   = finder.dijkstra_shortest_path(graph, "node_a", "node_b")
-paths  = finder.all_shortest_paths(graph, "source", "target")
-k_paths = finder.find_k_shortest_paths(graph, "source", "target", k=3)
-```
+    Algorithms: **Dijkstra**, **A\***, **BFS**, **All Shortest Paths**, **K-Shortest Paths**.
+  </Tab>
+  <Tab title="Connectivity">
+    Analyse graph structure — components, bridges, and density:
 
-Algorithms: Dijkstra, A\*, BFS, All Shortest Paths, K-Shortest Paths.
+    ```python
+    from semantica.kg import ConnectivityAnalyzer
 
-### Link Prediction
+    analyzer   = ConnectivityAnalyzer()
+    components = analyzer.find_connected_components(kg)
+    density    = analyzer.calculate_density(kg)
+    bridges    = analyzer.find_bridges(kg)
 
-```python
-from semantica.kg import LinkPredictor
+    print(f"Components: {len(components)}, Largest: {len(components[0])} nodes")
+    print(f"Density:    {density:.4f}")
+    print(f"Bridges:    {bridges}")
+    ```
 
-predictor = LinkPredictor(method="preferential_attachment")
-links = predictor.predict_links(graph, top_k=20)
-score = predictor.score_link(graph, "node_a", "node_b")
-```
+    | Method | Returns | Description |
+    | ------ | ------- | ----------- |
+    | `find_connected_components(kg)` | `List[List[str]]` | Groups of mutually reachable nodes |
+    | `calculate_density(kg)` | `float` | Edge density (actual / possible edges) |
+    | `find_bridges(kg)` | `List[str]` | Nodes whose removal disconnects the graph |
+  </Tab>
+  <Tab title="Link Prediction">
+    Predict which edges are likely missing from the graph:
 
-Algorithms: Preferential Attachment, Common Neighbors, Jaccard, Adamic-Adar, Resource Allocation.
+    ```python
+    from semantica.kg import LinkPredictor
 
-### Node Embeddings
+    predictor = LinkPredictor(method="preferential_attachment")
+    links     = predictor.predict_links(kg, top_k=20)
+    score     = predictor.score_link(kg, "node_a", "node_b")
+    ```
 
-```python
-from semantica.kg import NodeEmbedder
+    Algorithms: **Preferential Attachment**, **Common Neighbors**, **Jaccard**, **Adamic-Adar**, **Resource Allocation**.
+  </Tab>
+  <Tab title="Node Embeddings">
+    Compute structural embeddings for similarity search and downstream ML:
 
-embedder = NodeEmbedder(method="node2vec", embedding_dimension=128)
-embeddings   = embedder.compute_embeddings(graph_store, ["Entity"], ["RELATED_TO"])
-similar_nodes = embedder.find_similar_nodes(graph_store, "entity_123", top_k=10)
-```
+    ```python
+    from semantica.kg import NodeEmbedder
 
-Algorithms: Node2Vec, DeepWalk, Word2Vec.
+    embedder      = NodeEmbedder(method="node2vec", embedding_dimension=128)
+    embeddings    = embedder.compute_embeddings(graph_store, ["Entity"], ["RELATED_TO"])
+    similar_nodes = embedder.find_similar_nodes(graph_store, "entity_123", top_k=10)
+    ```
+
+    Algorithms: **Node2Vec**, **DeepWalk**, **Word2Vec**.
+  </Tab>
+</Tabs>
 
 ## Algorithm Summary
 
 | Category | Algorithms | Use Cases |
 | -------- | ---------- | --------- |
 | Node Embeddings | Node2Vec, DeepWalk, Word2Vec | Structural similarity, node representation |
-| Similarity | Cosine, Euclidean, Manhattan, Correlation | Node matching, recommendation |
 | Path Finding | Dijkstra, A\*, BFS, K-Shortest | Route planning, network analysis |
 | Link Prediction | Preferential Attachment, Jaccard, Adamic-Adar | Network completion |
 | Centrality | Degree, Betweenness, Closeness, PageRank | Influence analysis |
 | Community Detection | Louvain, Leiden, Label Propagation | Social clustering |
 | Connectivity | Components, Bridges, Density | Network robustness |
 
+## SeedManager
+
+Load and inject curated seed data into a knowledge graph:
+
+```python
+from semantica.kg import SeedManager
+
+manager    = SeedManager()
+seed_data  = manager.load_seed("seeds/domain_entities.json")
+normalized = manager.normalize(seed_data, source="manual_curation_v1")
+
+builder = GraphBuilder(merge_entities=True)
+kg      = builder.build(normalized + extracted_sources)
+```
+
+## MethodRegistry
+
+Register custom KG construction methods and dispatch by name:
+
+```python
+from semantica.kg import method_registry
+
+def my_kg_builder(entities, relationships, **kwargs):
+    filtered = [e for e in entities if e["confidence"] >= 0.9]
+    return {"entities": filtered, "relationships": relationships}
+
+method_registry.register("build", "high_confidence", my_kg_builder)
+
+from semantica.kg import build_knowledge_graph
+kg = build_knowledge_graph(sources, method="high_confidence")
+```
+
+## ProvenanceTracker
+
+Track entity and relationship lineage within a knowledge graph:
+
+```python
+from semantica.kg import ProvenanceTracker
+
+tracker = ProvenanceTracker()
+
+tracker.track_entity(
+    entity_id="apple_inc",
+    source="sec_filing_2024q1.pdf",
+    source_location="page 3, paragraph 2",
+    source_quote="Apple Inc. reported revenue of...",
+    confidence=0.98,
+)
+
+lineage = tracker.get_lineage("apple_inc")
+for entry in lineage.entries:
+    print(f"  Source: {entry.source}  ({entry.timestamp})")
+```
+
+For full W3C PROV-O compliance and provenance export, see the [Provenance module](provenance).
+
 ## Configuration
 
 ```yaml
@@ -199,6 +357,24 @@ kg:
     default_validity: infinite
 ```
 
+## Tips and Common Pitfalls
+
+<Warning>
+  **Deduplicate before `GraphBuilder`, not after.** It's far easier to merge entities before they become nodes than to update all relationship endpoints after the fact. Run `DuplicateDetector` on extracted entities before calling `builder.build()`.
+</Warning>
+
+<Tip>
+  **PageRank identifies your most connected, important nodes.** If you're not sure which entities in your graph are the most structurally significant, `CentralityCalculator.calculate_pagerank()` gives you a ranked list — useful for GraphRAG context anchoring.
+</Tip>
+
+<Tip>
+  **Community detection finds thematic clusters.** `CommunityDetector` with Louvain partitions your graph into clusters of densely-connected nodes — often corresponding to real-world thematic groups. Use these clusters for exploratory analysis and to scope GraphRAG retrieval.
+</Tip>
+
+<Tip>
+  **`ProvenanceTracker` links entities back to their source documents.** Use it during graph construction so you can always answer "where did this fact come from?" — critical for compliance and for debugging incorrect graph data.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Graph Store" icon="server" href="graph_store">
     Persist graphs in Neo4j, FalkorDB, or Apache AGE.

docs/reference/llms.md

@@ -1,19 +1,88 @@
 ---
 title: "LLMs Module"
-description: "Unified interface for Groq, OpenAI, Anthropic, Gemini, Ollama, DeepSeek, Novita AI, LiteLLM, and HuggingFace."
+description: "Unified interface for Groq, OpenAI, Anthropic, Gemini, Ollama, DeepSeek, Novita AI, LiteLLM, and HuggingFace — swap providers with a one-line change."
 icon: "microchip"
 ---
 
-`semantica.llms` provides a single consistent API across 8+ LLM providers. Every provider is a drop-in replacement for the `llm_provider=` parameter in extractors, reasoning engines, and agents.
+`semantica.llms` gives every Semantica module a single, consistent interface to 9+ LLM providers. Every extractor, reasoning engine, and context graph accepts any provider through the same `llm_provider=` parameter — swap Groq for Anthropic, or a cloud API for a local Ollama model, by changing one line.
 
 ## What You Get
 
-- **8+ provider integrations** — Groq, OpenAI, Anthropic, Gemini, Ollama, DeepSeek, Novita AI, LiteLLM, HuggingFace
-- **Unified `LLMProvider` interface** — swap providers with a one-line change, no application changes needed
-- **`ProviderFactory`** — instantiate any provider by name from a config dict
-- **Local models** — Ollama and HuggingFace run fully on-premise with no API key
-- **Streaming** — token-by-token output for low-latency UX
-- **Custom gateways** — point any OpenAI-compatible endpoint via `base_url`
+<CardGroup cols={2}>
+  <Card title="9+ Provider Integrations" icon="plug">
+    Groq, OpenAI, Anthropic, Gemini, Ollama, DeepSeek, Novita AI, LiteLLM, and HuggingFace — all behind one interface.
+  </Card>
+  <Card title="Unified LLMProvider Interface" icon="arrows-left-right">
+    `complete()`, `chat()`, and `stream()` work identically across all providers — swap with a one-line change.
+  </Card>
+  <Card title="create_provider() Factory" icon="gear">
+    Instantiate any provider from a name string — drive provider selection entirely from YAML config or environment variables.
+  </Card>
+  <Card title="Local Inference" icon="server">
+    Ollama and HuggingFace run fully on-premise — no API key, no data leaves your machine, air-gap compatible.
+  </Card>
+  <Card title="Streaming" icon="bolt">
+    Token-by-token output via `stream()` for responsive agent pipelines and live UI updates.
+  </Card>
+  <Card title="Retry & Error Handling" icon="rotate">
+    Configurable `max_retries` with exponential backoff. Typed exceptions: `LLMAuthenticationError`, `LLMRateLimitError`, `LLMContextLengthError`.
+  </Card>
+</CardGroup>
+
+## Installation
+
+The base install includes Groq and DeepSeek. Other providers require optional extras:
+
+| Provider | Install Command | API Key Required |
+| -------- | --------------- | ---------------- |
+| Groq | `pip install semantica` | Yes — `GROQ_API_KEY` |
+| DeepSeek | `pip install semantica` | Yes — `DEEPSEEK_API_KEY` |
+| Novita AI | `pip install semantica` | Yes — `NOVITA_API_KEY` |
+| OpenAI | `pip install "semantica[llm-openai]"` | Yes — `OPENAI_API_KEY` |
+| Anthropic | `pip install "semantica[llm-anthropic]"` | Yes — `ANTHROPIC_API_KEY` |
+| Gemini | `pip install "semantica[llm-gemini]"` | Yes — `GOOGLE_API_KEY` |
+| Ollama | `pip install "semantica[llm-ollama]"` | No — local server |
+| LiteLLM | `pip install "semantica[llm-litellm]"` | Varies by target |
+| HuggingFace | `pip install "semantica[llm-huggingface]"` | No — local weights |
+| All providers | `pip install "semantica[all]"` | Varies |
+
+## Quick Start
+
+<Steps>
+  <Step title="Pick a provider">
+    ```python
+    from semantica.llms import Groq
+    import os
+
+    llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+    ```
+  </Step>
+  <Step title="Pass it to any Semantica module">
+    ```python
+    from semantica.semantic_extract import NERExtractor
+
+    ner = NERExtractor(method="llm", llm_provider=llm)
+    ```
+  </Step>
+  <Step title="Extract — swap providers by changing only step 1">
+    ```python
+    entities = ner.extract("Apple Inc. was founded by Steve Jobs in 1976.")
+    ```
+  </Step>
+  <Step title="Use create_provider() for config-driven selection">
+    ```python
+    from semantica.llms import create_provider
+    from semantica.core import ConfigManager
+
+    config = ConfigManager("config.yaml")
+    llm    = create_provider(
+        config.get("llm_provider.name"),
+        model=config.get("llm_provider.model"),
+        api_key=config.get("llm_provider.api_key"),
+    )
+    ```
+  </Step>
+</Steps>
 
 ## Providers
 
@@ -24,62 +93,67 @@ from semantica.llms import Groq
 import os
 
 llm = Groq(
-    model="llama-3.3-70b-versatile",   # default
+    model="llama-3.3-70b-versatile",   # default and recommended
     api_key=os.getenv("GROQ_API_KEY"),
-    max_tokens=64000,
     temperature=0.0,
+    max_tokens=64000,
+    max_retries=3,
+    timeout=60,
 )
-# Best for: high-throughput extraction, fast inference
 ```
 
 ```python OpenAI
 from semantica.llms import OpenAI
 import os
 
-# pip install "semantica[llm-openai]"
 llm = OpenAI(
     model="gpt-4o",
     api_key=os.getenv("OPENAI_API_KEY"),
     temperature=0.0,
+    max_tokens=4096,
+    max_retries=3,
+    timeout=120,
+    organization=None,   # optional org ID
 )
-# Best for: general purpose, function calling
 ```
 
 ```python Anthropic
 from semantica.llms import Anthropic
 import os
 
-# pip install "semantica[llm-anthropic]"
 llm = Anthropic(
     model="claude-opus-4-7",
     api_key=os.getenv("ANTHROPIC_API_KEY"),
     max_tokens=8192,
+    temperature=0.0,
+    max_retries=3,
+    timeout=120,
 )
-# Best for: complex reasoning, long context, safety
 ```
 
 ```python Gemini
 from semantica.llms import Gemini
 import os
 
-# pip install "semantica[llm-gemini]"
 llm = Gemini(
     model="gemini-1.5-pro",
     api_key=os.getenv("GOOGLE_API_KEY"),
+    temperature=0.0,
+    max_tokens=8192,
+    timeout=120,
 )
-# Best for: long context (1M tokens), multimodal tasks
 ```
 
 ```python Ollama (Local)
 from semantica.llms import Ollama
 
-# pip install "semantica[llm-ollama]"
 llm = Ollama(
-    model="llama3.2:3b",
-    base_url="http://localhost:11434",
+    model="llama3.2",
+    base_url="http://localhost:11434",   # default Ollama address
+    temperature=0.0,
+    timeout=180,    # local models can be slower; increase for large models
 )
-# Best for: local inference, air-gapped environments
-# No API key required
+# No API key — model runs entirely on your machine
 ```
 
 ```python DeepSeek
@@ -89,64 +163,206 @@ import os
 llm = DeepSeek(
     model="deepseek-chat",
     api_key=os.getenv("DEEPSEEK_API_KEY"),
+    temperature=0.0,
+    max_tokens=4096,
+    max_retries=3,
 )
-# Best for: coding tasks and analysis at very low cost
+```
+
+```python Novita AI
+from semantica.llms import NovitaAI
+import os
+
+llm = NovitaAI(
+    model="deepseek/deepseek-v3.2",
+    api_key=os.getenv("NOVITA_API_KEY"),
+    temperature=0.0,
+    max_tokens=4096,
+)
+# OpenAI-compatible endpoint; also accepts deepseek/deepseek-r1, meta-llama models
 ```
 
 ```python LiteLLM (100+ models)
 from semantica.llms import LiteLLM
 import os
 
-# pip install "semantica[llm-litellm]"
 llm = LiteLLM(
-    model="gpt-4o",       # any LiteLLM-supported model string
+    model="gpt-4o",           # any LiteLLM model string
     api_key=os.getenv("OPENAI_API_KEY"),
+    temperature=0.0,
+    max_tokens=4096,
 )
-# Supports: OpenAI, Anthropic, Gemini, Cohere, Azure, Bedrock, and 90+ more
+# Supports: OpenAI, Anthropic, Gemini, Cohere, Azure, Bedrock, Together AI, and 90+ more
+# Use the LiteLLM model string format: "anthropic/claude-3-5-sonnet", "bedrock/anthropic.claude-v2"
 ```
 
-```python HuggingFace (BYOM)
+```python HuggingFace (Local)
 from semantica.llms import HuggingFace
 
 llm = HuggingFace(
     model="mistralai/Mistral-7B-Instruct-v0.3",
-    device="cuda",           # "cpu" | "cuda" | "mps"
+    device="cuda",           # "cpu" | "cuda" | "mps" (Apple Silicon)
     max_new_tokens=512,
     temperature=0.1,
+    load_in_4bit=True,       # enable 4-bit quantisation to reduce VRAM
 )
-# Bring your own model — full local control, no API key
+# No API key — weights downloaded from Hugging Face Hub (or loaded from local path)
 ```
 
 </CodeGroup>
 
-## Provider Factory
+## Constructor Parameters
 
-Instantiate any provider by name string — useful when provider is loaded from config:
+### Common Parameters
+
+| Parameter | Type | Default | Description |
+| --------- | ---- | ------- | ----------- |
+| `model` | `str` | Provider default | Model identifier string |
+| `api_key` | `str` | `None` | API key — reads from environment if omitted |
+| `temperature` | `float` | `0.0` | Sampling temperature: 0 = deterministic, 1 = creative |
+| `max_tokens` | `int` | Provider default | Maximum tokens in the response |
+| `max_retries` | `int` | `3` | Number of retry attempts on transient failures |
+| `timeout` | `int` | `60` | Request timeout in seconds |
+| `base_url` | `str` | Provider default | Override the API endpoint — useful for proxies and gateways |
+
+### Provider-Specific Parameters
+
+| Provider | Parameter | Description |
+| -------- | --------- | ----------- |
+| `OpenAI` | `organization` | OpenAI organisation ID |
+| `OpenAI` | `project` | OpenAI project ID |
+| `Ollama` | `base_url` | Ollama server address (default: `http://localhost:11434`) |
+| `HuggingFace` | `device` | Compute device: `"cpu"` / `"cuda"` / `"mps"` |
+| `HuggingFace` | `load_in_4bit` | Enable 4-bit quantisation (requires `bitsandbytes`) |
+| `HuggingFace` | `max_new_tokens` | Maximum new tokens to generate (replaces `max_tokens`) |
+| `LiteLLM` | `model` | Full LiteLLM model string, e.g. `"anthropic/claude-3-5-sonnet"` |
+
+## Direct API Usage
+
+Providers can be used directly — not just through Semantica modules:
 
 ```python
-from semantica.llms import create_provider
+from semantica.llms import Groq
+import os
 
-llm = create_provider("groq",      model="llama-3.3-70b-versatile")
-llm = create_provider("openai",    model="gpt-4o")
-llm = create_provider("anthropic", model="claude-opus-4-7")
-llm = create_provider("gemini",    model="gemini-1.5-pro")
-llm = create_provider("ollama",    model="llama3.2")
-llm = create_provider("deepseek",  model="deepseek-chat",   api_key=os.getenv("DEEPSEEK_API_KEY"))
-llm = create_provider("novita",    model="deepseek/deepseek-v3.2", api_key=os.getenv("NOVITA_API_KEY"))
-llm = create_provider("litellm",   model="gpt-4o")
+llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+
+# Single completion
+response = llm.complete("What is a knowledge graph?")
+print(response.text)          # answer string
+print(response.input_tokens)  # tokens consumed by the prompt
+print(response.output_tokens) # tokens in the response
+print(response.model)         # model that served the request
+
+# Multi-turn chat
+messages = [
+    {"role": "system", "content": "You are a knowledge graph expert."},
+    {"role": "user",   "content": "What is the difference between RDF and property graphs?"},
+]
+response = llm.chat(messages)
+print(response.text)
+
+# Streaming — token-by-token output
+for token in llm.stream("Explain knowledge graph reasoning in 3 sentences."):
+    print(token, end="", flush=True)
 ```
 
-## Custom / Enterprise Gateways
+## LLMResponse Object
 
-Any OpenAI-compatible endpoint — internal routing layers, Qwen proxies, or private LLaMA deployments:
+All three methods (`complete`, `chat`, `stream`) return a `LLMResponse` dataclass:
+
+<AccordionGroup>
+  <Accordion title="LLMResponse schema">
+
+```python
+@dataclass
+class LLMResponse:
+    text:          str    # the generated text
+    model:         str    # model identifier that served the request
+    input_tokens:  int    # tokens consumed by the prompt
+    output_tokens: int    # tokens in the generated response
+    total_tokens:  int    # input_tokens + output_tokens
+    latency_ms:    float  # wall-clock time for the API call in milliseconds
+    finish_reason: str    # "stop" | "length" | "content_filter" | "tool_calls"
+```
+
+  </Accordion>
+</AccordionGroup>
+
+## Error Handling
+
+<AccordionGroup>
+  <Accordion title="Exception hierarchy and when each is raised">
+
+```python
+from semantica.llms import Groq
+from semantica.utils import SemanticaError
+import os
+
+try:
+    llm      = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+    response = llm.complete("Summarise this document.")
+    print(response.text)
+
+except LLMAuthenticationError as e:
+    # Invalid or expired API key
+    print(f"Authentication failed: {e}")
+
+except LLMRateLimitError as e:
+    # Rate limit hit — Semantica retries automatically up to max_retries
+    print(f"Rate limited after retries: {e}")
+
+except LLMContextLengthError as e:
+    # Prompt exceeds the model's context window
+    print(f"Prompt too long ({e.token_count} tokens, limit {e.context_limit}): {e}")
+
+except LLMProviderError as e:
+    # General provider-side error (5xx, model unavailable, etc.)
+    print(f"Provider error: {e}")
+
+except SemanticaError as e:
+    # Catch-all for all Semantica framework errors
+    print(f"Framework error: {e}")
+```
+
+| Exception | When Raised |
+| --------- | ----------- |
+| `LLMAuthenticationError` | Invalid or missing API key |
+| `LLMRateLimitError` | Rate limit exceeded after all retries |
+| `LLMContextLengthError` | Prompt exceeds the model's context window |
+| `LLMProviderError` | Provider-side error (5xx, unavailability) |
+| `LLMTimeoutError` | Request exceeded the `timeout` parameter |
+
+  </Accordion>
+</AccordionGroup>
+
+## Custom and Enterprise Gateways
+
+Any provider that exposes an OpenAI-compatible REST API can be used by passing `base_url`:
 
 ```python
 from semantica.llms import OpenAI
+import os
 
+# Internal LLM routing gateway
 llm = OpenAI(
     model="qwen2.5-72b",
     api_key=os.getenv("GATEWAY_API_KEY"),
-    base_url="https://my-internal-gateway.company.com/v1",
+    base_url="https://llm-gateway.internal.company.com/v1",
+)
+
+# Azure OpenAI Service
+llm = OpenAI(
+    model="gpt-4o",
+    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
+    base_url="https://my-resource.openai.azure.com/openai/deployments/gpt-4o",
+)
+
+# Self-hosted vLLM server
+llm = OpenAI(
+    model="meta-llama/Llama-3.1-8B-Instruct",
+    api_key="not-needed",
+    base_url="http://localhost:8000/v1",
 )
 ```
 
@@ -154,49 +370,126 @@ llm = OpenAI(
   `base_url` is validated at construction time. Non-HTTP(S) schemes raise `ValueError` to prevent SSRF attacks (fixed in v0.5.0).
 </Note>
 
-## Using in Extractors
+## Using in Semantica Modules
 
-All extractors accept any provider as `llm_provider=`:
+Every module that uses an LLM accepts any provider through `llm_provider=`:
 
 ```python
+from semantica.llms import Groq, Anthropic
 from semantica.semantic_extract import NERExtractor, RelationExtractor, TripletExtractor
+from semantica.ontology import LLMOntologyGenerator
+from semantica.reasoning import ReasoningEngine
+from semantica.context import AgentContext, ContextGraph
+from semantica.vector_store import VectorStore
+import os
 
-llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+groq_llm   = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+claude_llm = Anthropic(model="claude-opus-4-7",     api_key=os.getenv("ANTHROPIC_API_KEY"))
 
-ner  = NERExtractor(method="llm",      llm_provider=llm, max_retries=3)
-rel  = RelationExtractor(method="llm", llm_provider=llm)
-trip = TripletExtractor(method="llm",  llm_provider=llm)
+# Extraction — use fast Groq for high-throughput NER
+ner  = NERExtractor(method="llm",      llm_provider=groq_llm)
+rel  = RelationExtractor(method="llm", llm_provider=groq_llm)
+trip = TripletExtractor(method="llm",  llm_provider=groq_llm)
+
+# Complex reasoning — use Claude for accuracy
+engine = ReasoningEngine(llm_provider=claude_llm)
+
+# Ontology generation from natural language
+gen = LLMOntologyGenerator(llm_provider=claude_llm)
 ```
 
 ## Provider Comparison
 
-| Provider | Speed | Cost | Local | Context | Best For |
-| -------- | ----- | ---- | ----- | ------- | -------- |
-| Groq | Very fast | Low | No | 128k | High-throughput extraction |
-| OpenAI | Fast | Medium | No | 128k | General purpose, function calling |
-| Anthropic | Fast | Medium | No | 200k | Complex reasoning, safety |
-| Gemini | Fast | Low | No | 1M | Long context, multimodal |
-| Ollama | Medium | Free | Yes | Varies | Privacy, no API key |
-| DeepSeek | Fast | Very low | No | 64k | Coding, analysis |
-| Novita AI | Fast | Low | No | Varies | DeepSeek-based tasks |
-| LiteLLM | Varies | Varies | Varies | Varies | Multi-provider routing |
-| HuggingFace | Slow | Free | Yes | Varies | Custom models, BYOM |
+| Provider | Speed | Cost | Local | Max Context | Best For |
+| -------- | ----- | ---- | ----- | ----------- | -------- |
+| **Groq** | ⚡ Very fast | 💲 Low | No | 128k | High-throughput extraction, fast pipelines |
+| **OpenAI** | Fast | 💲💲 Medium | No | 128k | General purpose, function calling, JSON mode |
+| **Anthropic** | Fast | 💲💲 Medium | No | 200k | Complex reasoning, long documents, safety |
+| **Gemini** | Fast | 💲 Low | No | 1M | Very long context, multimodal (text + image) |
+| **Ollama** | Medium | Free | ✅ Yes | Varies | Privacy, air-gapped, no API key |
+| **DeepSeek** | Fast | 💲 Very low | No | 64k | Coding tasks, structured analysis |
+| **Novita AI** | Fast | 💲 Low | No | Varies | DeepSeek and LLaMA models, cost-effective |
+| **LiteLLM** | Varies | Varies | Varies | Varies | Multi-provider routing, vendor abstraction |
+| **HuggingFace** | Slow | Free | ✅ Yes | Varies | Custom and fine-tuned models, full local control |
+
+## Environment Variables
+
+| Variable | Provider | Notes |
+| -------- | -------- | ----- |
+| `GROQ_API_KEY` | Groq | Required for Groq cloud |
+| `OPENAI_API_KEY` | OpenAI, LiteLLM | Also used for OpenAI-compatible gateways |
+| `ANTHROPIC_API_KEY` | Anthropic | Required for Claude |
+| `GOOGLE_API_KEY` | Gemini | Required for Gemini |
+| `DEEPSEEK_API_KEY` | DeepSeek | Required for DeepSeek cloud |
+| `NOVITA_API_KEY` | Novita AI | Required for Novita AI |
+| `HUGGINGFACE_HUB_TOKEN` | HuggingFace | Required for gated models (optional for public models) |
+
+## YAML Configuration
+
+```yaml
+# config.yaml
+llm_provider:
+  name: "groq"
+  model: "llama-3.3-70b-versatile"
+  api_key: "${GROQ_API_KEY}"   # reads from environment
+  temperature: 0.0
+  max_tokens: 64000
+  max_retries: 3
+  timeout: 60
+```
+
+Load it with `ConfigManager`:
+
+```python
+from semantica.core import ConfigManager
+from semantica.llms import create_provider
+
+config = ConfigManager("config.yaml")
+llm    = create_provider(
+    config.get("llm_provider.name"),
+    model=config.get("llm_provider.model"),
+    api_key=config.get("llm_provider.api_key"),
+    temperature=config.get("llm_provider.temperature", default=0.0),
+)
+```
+
+## Tips and Common Pitfalls
+
+<Warning>
+  **Always set `temperature=0.0` for extraction tasks.** NER, relation extraction, and triplet generation need deterministic output — any temperature above 0 introduces randomness that produces inconsistent entity types or hallucinated relationships. Reserve higher temperatures for creative or summarisation tasks.
+</Warning>
 
 <Tip>
-  For production extraction pipelines, Groq delivers the best throughput-to-cost ratio. For complex multi-hop reasoning, Claude Opus or GPT-4o provide the highest accuracy.
+  **Set `max_retries=3` in production.** Transient rate limits and 5xx errors are normal at scale. All providers retry automatically up to `max_retries` with exponential backoff. Without retries, a single rate-limit hit fails an entire pipeline step that would have succeeded on the second attempt.
+</Tip>
+
+<Tip>
+  **Use `create_provider()` for config-driven pipelines.** Hard-coding `Groq(...)` in Python means changing the provider requires a code change and redeploy. `create_provider(config.get("llm_provider.name"), ...)` lets you switch from Groq to Anthropic by editing `config.yaml` — no code changes.
+</Tip>
+
+<Tip>
+  **Use `base_url` for internal gateways and Azure.** Enterprise deployments often route LLM calls through an internal proxy or Azure OpenAI Service. Pass `base_url="https://llm-gateway.internal/v1"` to `OpenAI` — you get the same Semantica integration without changing any module code. Non-HTTP schemes raise `ValueError` (SSRF protection, v0.5.0+).
+</Tip>
+
+<Warning>
+  **Catch `LLMContextLengthError` explicitly.** If your chunking is misconfigured, a document chunk can exceed the model's context window. Catch `LLMContextLengthError` and log `e.token_count` — it tells you exactly how much to reduce your `chunk_size`. Don't let it surface as a generic failure.
+</Warning>
+
+<Tip>
+  **Use Ollama or HuggingFace for air-gapped environments.** When data cannot leave the network, Ollama (local inference) or HuggingFace (local weights) are the only viable options. Both support the same `llm_provider=` interface — no other code changes needed.
 </Tip>
 
 <CardGroup cols={2}>
   <Card title="Semantic Extract" icon="magnifying-glass" href="semantic_extract">
-    Use LLMs for NER and relation extraction.
-  </Card>
-  <Card title="Agno Integration" icon="robot" href="../integrations/agno">
-    LLM providers in Agno multi-agent teams.
+    NER, relation extraction, and triplet generation with LLMs.
   </Card>
   <Card title="Reasoning" icon="brain" href="reasoning">
-    LLM-backed deductive and abductive reasoning.
+    LLM-backed deductive, abductive, and Datalog reasoning.
+  </Card>
+  <Card title="Ontology" icon="sitemap" href="ontology">
+    Generate ontologies from natural language using LLMs.
   </Card>
   <Card title="Context" icon="diagram-project" href="context">
-    GraphRAG uses LLMs for reasoning over knowledge graphs.
+    GraphRAG and decision intelligence powered by LLMs.
   </Card>
 </CardGroup>

docs/reference/mcp_server.md

@@ -6,15 +6,32 @@ icon: "plug"
 
 `semantica.mcp_server` exposes Semantica's knowledge graph, decision intelligence, semantic extraction, and reasoning capabilities as an [MCP (Model Context Protocol)](https://modelcontextprotocol.io) server over stdio.
 
+Once configured, any connected AI assistant can extract entities, record decisions, query the graph, run reasoning, and export results — without writing a single line of Python.
+
 Compatible with **Claude Desktop**, **Windsurf**, **Cline**, **Continue**, **VS Code**, **Roo Code**, **Cursor**, and any MCP-aware client.
 
 ## What You Get
 
-- **12 MCP tools** — extract entities, build graphs, run SPARQL, find paths, get recommendations, embed, cluster, and more
-- **3 readable resources** — live graph JSON, entity list, and relationship list
-- **Zero infrastructure** — runs over stdio, no server or port needed
-- **Claude Desktop ready** — one config block to add to `claude_desktop_config.json`
-- **REST alternative** — the Explorer module offers a full HTTP API if you prefer
+<CardGroup cols={2}>
+  <Card title="12 MCP Tools" icon="wrench">
+    Extract entities, extract relations, record decisions, query decisions, find precedents, trace causal chains, add entities, add relationships, run analytics, summarise graph, run reasoning, export.
+  </Card>
+  <Card title="3 Readable Resources" icon="book-open">
+    Live graph JSON (`semantica://graph/summary`), decision list, and schema/version info — readable by any MCP client.
+  </Card>
+  <Card title="Zero Infrastructure" icon="bolt">
+    Runs over stdio — no server, no port, no Docker required. One config block to activate in any MCP client.
+  </Card>
+  <Card title="Persistent Graphs" icon="database">
+    Point `SEMANTICA_KG_PATH` at a saved graph file to reload it automatically on every server startup.
+  </Card>
+  <Card title="Decision Intelligence" icon="brain">
+    Record decisions, find precedents via hybrid similarity search, and trace causal chains across agent runs.
+  </Card>
+  <Card title="REST Alternative" icon="globe">
+    The [Explorer](explorer) module offers a full HTTP API and browser dashboard if you prefer programmatic access.
+  </Card>
+</CardGroup>
 
 ## Installation
 
@@ -26,46 +43,86 @@ The MCP server is included in the base install — no extras required.
 
 ## Configuration
 
-Add Semantica to your MCP client's settings file:
+<Steps>
+  <Step title="Find your MCP client's settings file">
 
-<CodeGroup>
+    | Client | Settings file |
+    | ------ | ------------- |
+    | Claude Desktop (macOS) | `~/Library/Application Support/Claude/claude_desktop_config.json` |
+    | Claude Desktop (Windows) | `%APPDATA%\Claude\claude_desktop_config.json` |
+    | Cursor | `.cursor/mcp.json` in your project, or `~/.cursor/mcp.json` globally |
+    | VS Code / Continue | `.vscode/mcp.json` or user settings |
+    | Windsurf / Cline / Roo Code | App-specific settings → MCP Servers |
 
-```json Claude Desktop / Windsurf / Cline
-{
-  "mcpServers": {
-    "semantica": {
-      "command": "semantica-mcp"
-    }
-  }
-}
-```
+  </Step>
+  <Step title="Add the Semantica MCP server config">
 
-```json VS Code / Continue / Roo Code
-{
-  "mcpServers": {
-    "semantica": {
-      "command": "python",
-      "args": ["-m", "semantica.mcp_server"]
-    }
-  }
-}
-```
+    <CodeGroup>
 
-```json With persistent graph
-{
-  "mcpServers": {
-    "semantica": {
-      "command": "semantica-mcp",
-      "env": {
-        "SEMANTICA_KG_PATH": "/path/to/my_graph.json",
-        "SEMANTICA_LOG_LEVEL": "INFO"
+    ```json Claude Desktop / Windsurf / Cline
+    {
+      "mcpServers": {
+        "semantica": {
+          "command": "semantica-mcp"
+        }
       }
     }
-  }
-}
-```
+    ```
 
-</CodeGroup>
+    ```json Cursor
+    {
+      "mcpServers": {
+        "semantica": {
+          "command": "semantica-mcp",
+          "env": {
+            "SEMANTICA_KG_PATH": "/path/to/my_graph.json"
+          }
+        }
+      }
+    }
+    ```
+
+    ```json VS Code / Continue / Roo Code
+    {
+      "mcpServers": {
+        "semantica": {
+          "command": "python",
+          "args": ["-m", "semantica.mcp_server"]
+        }
+      }
+    }
+    ```
+
+    ```json With persistent graph
+    {
+      "mcpServers": {
+        "semantica": {
+          "command": "semantica-mcp",
+          "env": {
+            "SEMANTICA_KG_PATH": "/path/to/my_graph.json",
+            "SEMANTICA_LOG_LEVEL": "INFO"
+          }
+        }
+      }
+    }
+    ```
+
+    </CodeGroup>
+
+  </Step>
+  <Step title="Test locally before configuring your client">
+    ```bash
+    # Run the server directly (reads from stdin, writes to stdout)
+    semantica-mcp
+
+    # Or via Python module
+    python -m semantica.mcp_server
+
+    # Send a JSON-RPC initialize message to confirm it's working
+    echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' | semantica-mcp
+    ```
+  </Step>
+</Steps>
 
 ## Environment Variables
 
@@ -78,6 +135,21 @@ Add Semantica to your MCP client's settings file:
 
 The MCP server exposes 12 tools that any connected AI assistant can call:
 
+| Tool | Category | Description |
+| ---- | -------- | ----------- |
+| `extract_entities` | Extraction | NER — find people, places, organisations, concepts |
+| `extract_relations` | Extraction | Typed relation and triplet extraction |
+| `record_decision` | Decision Intelligence | Save a decision with reasoning and outcome |
+| `query_decisions` | Decision Intelligence | Search recorded decisions by natural language |
+| `find_precedents` | Decision Intelligence | Hybrid similarity search over past decisions |
+| `get_causal_chain` | Decision Intelligence | Trace upstream / downstream causal chains |
+| `add_entity` | Graph Operations | Add a node to the live graph |
+| `add_relationship` | Graph Operations | Add a directed edge between two nodes |
+| `get_graph_analytics` | Graph Operations | PageRank + community detection |
+| `get_graph_summary` | Graph Operations | Node count, decision count, health status |
+| `run_reasoning` | Reasoning & Export | Forward-chain IF/THEN rules over facts |
+| `export_graph` | Reasoning & Export | Serialise the graph (Turtle, JSON-LD, JSON, etc.) |
+
 ### Knowledge Extraction
 
 <AccordionGroup>
@@ -87,11 +159,13 @@ The MCP server exposes 12 tools that any connected AI assistant can call:
 Extract named entities (people, places, organisations, concepts) from text using Semantica NER.
 
 **Input:**
+
 ```json
 { "text": "Apple Inc. was founded by Steve Jobs in Cupertino in 1976." }
 ```
 
 **Output:**
+
 ```json
 {
   "entities": [
@@ -110,11 +184,13 @@ Extract named entities (people, places, organisations, concepts) from text using
 Extract typed relations and `(subject, predicate, object)` triplets from text.
 
 **Input:**
+
 ```json
 { "text": "Steve Jobs founded Apple Inc. and led it until 2011." }
 ```
 
 **Output:**
+
 ```json
 {
   "relations": [
@@ -139,6 +215,7 @@ Extract typed relations and `(subject, predicate, object)` triplets from text.
 Record a decision with full context, reasoning, and metadata into the knowledge graph.
 
 **Input:**
+
 ```json
 {
   "category": "model_selection",
@@ -151,6 +228,7 @@ Record a decision with full context, reasoning, and metadata into the knowledge
 ```
 
 **Output:**
+
 ```json
 { "decision_id": "dec_a1b2c3", "status": "recorded" }
 ```
@@ -162,6 +240,7 @@ Record a decision with full context, reasoning, and metadata into the knowledge
 Query recorded decisions by natural language, category, or retrieve all recent decisions.
 
 **Input:**
+
 ```json
 { "query": "model selection", "limit": 5 }
 ```
@@ -173,6 +252,7 @@ Query recorded decisions by natural language, category, or retrieve all recent d
 Find past decisions similar to a given scenario using hybrid similarity search.
 
 **Input:**
+
 ```json
 { "scenario": "Choose cloud provider for HIPAA workload", "max_results": 3 }
 ```
@@ -184,6 +264,7 @@ Find past decisions similar to a given scenario using hybrid similarity search.
 Trace the causal chain upstream or downstream from a decision.
 
 **Input:**
+
 ```json
 { "decision_id": "dec_a1b2c3", "direction": "downstream", "max_depth": 5 }
 ```
@@ -201,6 +282,7 @@ Trace the causal chain upstream or downstream from a decision.
 Add a node/entity to the live knowledge graph.
 
 **Input:**
+
 ```json
 {
   "id": "apple_inc",
@@ -217,6 +299,7 @@ Add a node/entity to the live knowledge graph.
 Add a directed relationship (edge) between two existing entities.
 
 **Input:**
+
 ```json
 {
   "source": "steve_jobs",
@@ -251,6 +334,7 @@ Return node count, decision count, and graph health status.
 Run forward-chaining IF/THEN rules over a set of facts to derive new facts.
 
 **Input:**
+
 ```json
 {
   "facts": ["Employee(John)", "Manager(John)"],
@@ -259,6 +343,7 @@ Run forward-chaining IF/THEN rules over a set of facts to derive new facts.
 ```
 
 **Output:**
+
 ```json
 { "derived_facts": ["HasAuthority(John)"] }
 ```
@@ -270,6 +355,7 @@ Run forward-chaining IF/THEN rules over a set of facts to derive new facts.
 Export the current knowledge graph to a serialization format.
 
 **Input:**
+
 ```json
 { "format": "json-ld" }
 ```
@@ -282,7 +368,7 @@ Supported formats: `turtle`, `ttl`, `nt`, `xml`, `json-ld`, `json`.
 
 ## Resources
 
-The MCP server also exposes three readable resources:
+The MCP server exposes three readable resources:
 
 | URI | Description |
 | --- | ----------- |
@@ -290,21 +376,27 @@ The MCP server also exposes three readable resources:
 | `semantica://decisions/list` | All recorded decisions (up to 50) |
 | `semantica://schema/info` | Server version and available tools |
 
-## Test Locally
+## Tips and Common Pitfalls
 
-```bash
-# Run the server directly (reads from stdin, writes to stdout)
-semantica-mcp
+<Warning>
+  **Build the `ContextGraph` before starting the server.** The MCP server operates on a pre-built `ContextGraph` — it doesn't build the knowledge graph on demand. Construct and populate the graph first (ingest → extract → build KG → set `ContextGraph`), then pass it to `SemanticaMCPServer`. An empty or None graph results in empty query responses.
+</Warning>
 
-# Or via Python module
-python -m semantica.mcp_server
-```
+<Tip>
+  **Use `decision_tracking=True` for accountable agents.** Without decision tracking, `record_decision` and `query_decisions` calls succeed but nothing is stored. Enable it in the `ContextGraph` constructor when you want agents' decisions to be queryable for audit, compliance, or iterative reasoning.
+</Tip>
 
-Send a JSON-RPC `initialize` message to confirm it's working:
+<Tip>
+  **Use `find_precedents` before high-stakes decisions.** The tool performs hybrid similarity search across all recorded decisions. Call it at the start of any significant decision path — it surfaces past reasoning that may be directly applicable, reducing redundant work and improving consistency across agent runs.
+</Tip>
 
-```bash
-echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' | semantica-mcp
-```
+<Warning>
+  **Configure your MCP client's `command` field exactly.** The `command` field must point to the exact executable path (use `which semantica-mcp` on macOS/Linux to find it). A wrong path fails silently — the server just doesn't appear in the tools list. Test with the raw `echo | semantica-mcp` command first to confirm the binary works.
+</Warning>
+
+<Warning>
+  **The server communicates over stdio — don't add logging to stdout.** Any `print()` or logger output directed to stdout will corrupt the JSON-RPC message stream. Configure logging to write to a file or stderr only (`logging.basicConfig(filename="mcp.log")`). The MCP protocol assumes stdout carries only JSON-RPC frames.
+</Warning>
 
 <CardGroup cols={2}>
   <Card title="Context" icon="brain" href="context">

docs/reference/normalize.md

@@ -1,275 +1,404 @@
 ---
 title: "Normalize Module"
-description: "Text cleaning, entity canonicalization, date normalization, number/unit conversion, language detection, and encoding repair."
+description: "Text cleaning, entity canonicalization, date normalization, number conversion, language detection, and encoding repair — before extraction runs."
 icon: "broom"
 ---
 
-`semantica.normalize` standardizes raw data before extraction and graph construction — fixing encodings, canonicalizing entity names, normalizing dates, and detecting languages. All normalizers expose both convenience functions and stateful class instances.
+`semantica.normalize` standardizes raw data before extraction and graph construction. All normalizers expose both convenience functions (one-liners) and stateful class instances (full control over configuration and reuse).
+
+## Why Normalize Before Extraction
+
+Unstructured data is inconsistent by nature. Without normalization, the same real-world entity appears as dozens of variants in your graph:
+
+- `"Apple Inc."`, `"Apple Computer Inc."`, `"APPLE INC."`, `"Apple, Inc."` — four nodes, one company
+- `"Jan 1st, 2020"`, `"01/01/2020"`, `"2020-01-01"` — three formats, one date
+- `"$1.2B"`, `"1,200,000,000"`, `"1.2 billion USD"` — three strings, one number
+- `"Hello World"` vs `"Hello World"` — a non-breaking space that breaks string matching
+
+Normalization collapses these variants before any extractor, deduplicator, or graph builder sees the data — producing cleaner entities, fewer false duplicates, and more reliable downstream results.
 
 ## What You Get
 
-- **`TextNormalizer`** — Unicode, whitespace, HTML stripping, smart-quote/dash replacement
-- **`EntityNormalizer`** — alias resolution, disambiguation, name variant handling
-- **`DateNormalizer`** — ISO 8601 output, timezone conversion, relative date parsing
-- **`NumberNormalizer`** — currency, unit conversion, scientific notation, percentages
-- **`DataCleaner`** — duplicate detection, schema validation, missing value handling
-- **`LanguageDetector`** — 50+ language detection with confidence scoring
-- **`EncodingHandler`** — UTF-8 conversion, BOM removal, encoding detection
+<CardGroup cols={2}>
+  <Card title="TextNormalizer" icon="text-size">
+    Unicode forms, whitespace collapse, HTML stripping, smart-quote and dash replacement.
+  </Card>
+  <Card title="EntityNormalizer" icon="building">
+    Corporate suffix normalization, honorific removal, alias resolution, and disambiguation.
+  </Card>
+  <Card title="DateNormalizer" icon="calendar">
+    Any date format → ISO 8601; relative dates, timezones, and date ranges.
+  </Card>
+  <Card title="NumberNormalizer" icon="hashtag">
+    Currency, scientific notation, unit abbreviations, and percentages → float.
+  </Card>
+  <Card title="LanguageDetector" icon="globe">
+    50+ languages with confidence scoring and batch detection.
+  </Card>
+  <Card title="EncodingHandler" icon="code">
+    Encoding detection, UTF-8 conversion, BOM removal, and cp1252 repair.
+  </Card>
+</CardGroup>
 
 <Note>
-  **v0.5.0 fix:** Encoding repair now handles cp1252/latin-1 characters that previously caused crashes on Windows when processing documents with non-ASCII content.
+  **v0.5.0 fix:** Encoding repair now handles cp1252 and latin-1 characters that previously caused crashes on Windows when processing documents with non-ASCII content.
 </Note>
 
+## Recommended Processing Order
+
+<Steps>
+  <Step title="EncodingHandler — fix encoding first">
+    Broken bytes corrupt everything downstream. Always run this before anything else.
+
+    ```python
+    from semantica.normalize import EncodingHandler
+
+    handler   = EncodingHandler()
+    utf8_text = handler.to_utf8(raw_bytes)
+    ```
+  </Step>
+  <Step title="TextNormalizer — unicode, whitespace, HTML">
+    ```python
+    from semantica.normalize import TextNormalizer
+
+    normalizer  = TextNormalizer(strip_html=True, normalize_unicode=True)
+    clean_text  = normalizer.normalize_text(utf8_text)
+    ```
+  </Step>
+  <Step title="EntityNormalizer — canonicalize entity names">
+    ```python
+    from semantica.normalize import EntityNormalizer
+
+    normalizer = EntityNormalizer()
+    canonical  = normalizer.normalize_entity("Apple Computer Inc.", entity_type="Organization")
+    # → "Apple Inc."
+    ```
+  </Step>
+  <Step title="DateNormalizer and NumberNormalizer — parse structured values">
+    ```python
+    from semantica.normalize import DateNormalizer, NumberNormalizer
+
+    date_norm = DateNormalizer(target_timezone="UTC")
+    num_norm  = NumberNormalizer()
+
+    date = date_norm.normalize_date("Jan 1st, 2020")   # → "2020-01-01"
+    num  = num_norm.normalize_number("$1.2B")          # → 1200000000.0
+    ```
+  </Step>
+  <Step title="LanguageDetector — detect language on clean text">
+    ```python
+    from semantica.normalize import LanguageDetector
+
+    detector = LanguageDetector()
+    lang     = detector.detect("Bonjour le monde")
+    # → {"language": "fr", "confidence": 0.98}
+    ```
+  </Step>
+</Steps>
+
 ## Convenience Functions
 
-The fastest path — dispatch via function with a `method` parameter:
+The fastest path — one import, one call:
 
 ```python
 from semantica.normalize import (
     normalize_text, normalize_entity, normalize_date,
-    normalize_number, clean_data, detect_language, handle_encoding
+    normalize_number, clean_data, detect_language, handle_encoding,
 )
 
-clean  = normalize_text("  Hello,   World!!  \n\n")
-# → "Hello, World!!"
-
-entity = normalize_entity("Apple Computer Inc.", entity_type="Organization")
-# → "Apple Inc."
-
-date   = normalize_date("Jan 1st, 2020")
-# → "2020-01-01"
-
-num    = normalize_number("$1,234.56")
-# → 1234.56
-
-lang   = detect_language("Bonjour le monde")
-# → {"language": "fr", "confidence": 0.98}
+clean  = normalize_text("  Hello,   World!!  \n\n")         # → "Hello, World!!"
+entity = normalize_entity("Apple Computer Inc.", entity_type="Organization")  # → "Apple Inc."
+date   = normalize_date("Jan 1st, 2020")                    # → "2020-01-01"
+num    = normalize_number("$1.2B")                          # → 1200000000.0
+lang   = detect_language("Bonjour le monde")                # → {"language": "fr", "confidence": 0.98}
 ```
 
-## TextNormalizer
+## Normalizers
 
-```python
-from semantica.normalize import TextNormalizer
+<Tabs>
+  <Tab title="TextNormalizer">
+    Cleans raw text at the character and token level:
 
-normalizer = TextNormalizer()
+    ```python
+    from semantica.normalize import TextNormalizer
 
-normalized = normalizer.normalize_text(
-    raw_text,
-    lowercase=False,
-    remove_punctuation=False,
-    remove_extra_whitespace=True,
-    strip_html=True,             # remove HTML tags
-    normalize_unicode=True,      # NFC normalization
-)
-```
+    normalizer = TextNormalizer(
+        lowercase=False,
+        remove_punctuation=False,
+        remove_extra_whitespace=True,
+        strip_html=True,
+        normalize_unicode=True,
+        fix_encoding=True,
+        form="NFC",              # "NFC" | "NFD" | "NFKC" | "NFKD"
+    )
 
-Sub-normalizers for fine-grained control:
+    normalized = normalizer.normalize_text(raw_text)
+    ```
 
-```python
-from semantica.normalize import UnicodeNormalizer, WhitespaceNormalizer, SpecialCharacterProcessor
+    | Parameter | Type | Default | Description |
+    | --------- | ---- | ------- | ----------- |
+    | `lowercase` | `bool` | `False` | Convert to lowercase — use for bag-of-words matching, not NER |
+    | `remove_punctuation` | `bool` | `False` | Strip all punctuation — use for keyword extraction only |
+    | `remove_extra_whitespace` | `bool` | `True` | Collapse tabs, newlines, non-breaking spaces into single spaces |
+    | `strip_html` | `bool` | `False` | Remove HTML tags and decode `&amp;`, `&lt;`, etc. |
+    | `normalize_unicode` | `bool` | `True` | Apply Unicode normal form |
+    | `fix_encoding` | `bool` | `True` | Repair common encoding mojibake (cp1252 / latin-1 → UTF-8) |
+    | `form` | `str` | `"NFC"` | Unicode normalization form: `"NFC"` / `"NFD"` / `"NFKC"` / `"NFKD"` |
 
-# Unicode normalization forms: NFC | NFD | NFKC | NFKD
-unicode_norm = UnicodeNormalizer(form="NFC")
-text = unicode_norm.normalize("café")
+    **Unicode form guide:**
 
-# Collapse tabs, line breaks, and extra spaces
-ws_norm = WhitespaceNormalizer()
-text = ws_norm.normalize("Hello   World\t\n")  # → "Hello World"
+    | Form | Use When |
+    | ---- | -------- |
+    | `NFC` | Default — best for storage and display |
+    | `NFKC` | Search indexing — normalises ligatures, fullwidth chars, and fractions |
+    | `NFD` | Stripping diacritics — split é → e + combining accent, then strip accents |
+    | `NFKD` | Same as NFD but also decomposes compatibility characters |
 
-# Replace smart quotes, em-dashes, and ellipsis characters
-processor = SpecialCharacterProcessor()
-text = processor.process("‘Hello’")  # '' → ''
-```
+    **Sub-normalizers for fine-grained control:**
 
-## EntityNormalizer
+    ```python
+    from semantica.normalize import UnicodeNormalizer, WhitespaceNormalizer, SpecialCharacterProcessor
 
-Canonicalize entity names — handles corporate suffixes, punctuation, case, and honorifics:
+    unicode_norm = UnicodeNormalizer(form="NFC")
+    text = unicode_norm.normalize("café")
 
-```python
-from semantica.normalize import EntityNormalizer
+    ws_norm = WhitespaceNormalizer()
+    text    = ws_norm.normalize("Hello\t\t World\n\n")  # → "Hello World"
 
-normalizer = EntityNormalizer()
+    processor = SpecialCharacterProcessor()
+    text      = processor.process("'Hello'")  # '' → '', -- → -
+    ```
+  </Tab>
+  <Tab title="EntityNormalizer">
+    Canonicalises entity name variants — corporate suffixes, honorifics, case, and punctuation:
 
-# Company name normalization
-companies = ["Apple Computer, Inc.", "Apple Inc", "APPLE INC."]
-normalized = [normalizer.normalize_entity(c) for c in companies]
-# All → "Apple Inc."
+    ```python
+    from semantica.normalize import EntityNormalizer
 
-# Person name normalization
-name = normalizer.normalize_entity("JOBS, STEVE", entity_type="Person")
-# → "Steve Jobs"
-```
+    normalizer = EntityNormalizer()
 
-Sub-normalizers for entity-specific use cases:
+    # Corporate name normalization
+    normalizer.normalize_entity("Apple Computer, Inc.", entity_type="Organization")  # → "Apple Inc."
+    normalizer.normalize_entity("APPLE INC.",           entity_type="Organization")  # → "Apple Inc."
 
-```python
-from semantica.normalize import AliasResolver, EntityDisambiguator, NameVariantHandler
+    # Person name normalization
+    normalizer.normalize_entity("JOBS, STEVE", entity_type="Person")  # → "Steve Jobs"
+    ```
 
-# Dictionary-based alias expansion
-resolver = AliasResolver(aliases={
-    "ML": "Machine Learning",
-    "AI": "Artificial Intelligence",
-    "DL": "Deep Learning",
-})
-resolved = resolver.resolve("ML and DL are subsets of AI")
-# → "Machine Learning and Deep Learning are subsets of Artificial Intelligence"
+    **Key behaviours:**
+    - Corporate suffix normalization handles: `Inc`, `Inc.`, `Incorporated`, `Ltd`, `Limited`, `Corp`, `Corporation`, `LLC`, `GmbH`, `PLC`, and 30+ more
+    - `entity_type="Person"` activates last-name-first reversal, honorific removal, and suffix stripping
 
-# Context-aware disambiguation
-disambiguator = EntityDisambiguator()
-result = disambiguator.disambiguate(
-    "Apple", context="Steve Jobs founded Apple in Cupertino"
-)
-# → {"entity": "Apple Inc.", "type": "Organization", "confidence": 0.96}
+    **Sub-normalizers:**
 
-# Honorifics, titles, and cultural name variants
-handler = NameVariantHandler()
-canonical = handler.normalize("Dr. JOHN P. SMITH Jr.")
-# → "John P. Smith"
-```
+    ```python
+    from semantica.normalize import AliasResolver, EntityDisambiguator, NameVariantHandler
 
-## DateNormalizer
+    resolver = AliasResolver(aliases={
+        "ML":  "Machine Learning",
+        "NLP": "Natural Language Processing",
+    })
+    resolved = resolver.resolve("ML and NLP are subfields of AI")
 
-Parse and normalize dates from any format to ISO 8601:
+    disambiguator = EntityDisambiguator()
+    result = disambiguator.disambiguate(
+        "Apple",
+        context="Steve Jobs founded Apple in Cupertino in 1976",
+    )
+    # → {"entity": "Apple Inc.", "type": "Organization", "confidence": 0.96}
 
-```python
-from semantica.normalize import DateNormalizer
+    handler   = NameVariantHandler()
+    canonical = handler.normalize("Dr. JOHN P. SMITH Jr.")  # → "John P. Smith"
+    ```
+  </Tab>
+  <Tab title="DateNormalizer">
+    Parses any date format and outputs ISO 8601 strings. Handles relative expressions, timezones, and date ranges:
 
-normalizer = DateNormalizer()
+    ```python
+    from semantica.normalize import DateNormalizer
 
-dates = [
-    "January 1st, 2020",
-    "01/01/2020",
-    "2020-01-01T00:00:00Z",
-    "yesterday",
-    "3 weeks ago",
-]
-normalized = [normalizer.normalize_date(d) for d in dates]
-# All → ISO 8601 strings
+    normalizer = DateNormalizer(
+        target_timezone="UTC",
+        output_format="%Y-%m-%d",
+    )
 
-# With automatic UTC conversion
-normalizer_utc = DateNormalizer(target_timezone="UTC")
-utc_date = normalizer_utc.normalize_date("2024-01-01 09:00 EST")
-```
+    dates = [
+        "January 1st, 2020",
+        "01/01/2020",
+        "2020-01-01T00:00:00Z",
+        "yesterday",
+        "3 weeks ago",
+        "Q1 2024",
+    ]
+    normalized = [normalizer.normalize_date(d) for d in dates]
+    ```
 
-Sub-normalizers for advanced date handling:
+    **Sub-normalizers:**
 
-```python
-from semantica.normalize import (
-    TimeZoneNormalizer, RelativeDateProcessor, TemporalExpressionParser
-)
+    ```python
+    from semantica.normalize import TimeZoneNormalizer, RelativeDateProcessor, TemporalExpressionParser
+    from datetime import datetime
 
-# Timezone conversion
-tz_norm = TimeZoneNormalizer(target_tz="UTC")
-utc_dt = tz_norm.normalize("2024-01-01 09:00", source_tz="America/New_York")
+    tz_norm = TimeZoneNormalizer(target_tz="UTC")
+    utc_dt  = tz_norm.normalize("2024-01-01 09:00", source_tz="America/New_York")
+    # → datetime(2024, 1, 1, 14, 0, tzinfo=UTC)
 
-# Relative date resolution
-from datetime import datetime
-processor = RelativeDateProcessor(reference_date=datetime(2025, 1, 15))
-result = processor.process("3 days ago")
-# → datetime(2025, 1, 12)
+    processor = RelativeDateProcessor(reference_date=datetime(2025, 1, 15))
+    result    = processor.process("3 days ago")    # → datetime(2025, 1, 12)
+    result    = processor.process("next quarter")  # → {"start": "2025-04-01", "end": "2025-06-30"}
 
-# Date range parsing
-parser = TemporalExpressionParser()
-result = parser.parse("from January 2020 to March 2021")
-# → {"start": "2020-01-01", "end": "2021-03-31", "type": "range"}
-```
+    parser = TemporalExpressionParser()
+    result = parser.parse("from January 2020 to March 2021")
+    # → {"start": "2020-01-01", "end": "2021-03-31", "type": "range"}
 
-## NumberNormalizer
+    result = parser.parse("Q2 2023")
+    # → {"start": "2023-04-01", "end": "2023-06-30", "type": "quarter"}
+    ```
+  </Tab>
+  <Tab title="NumberNormalizer">
+    Converts number strings with units, currencies, and abbreviations to `float`:
 
-```python
-from semantica.normalize import NumberNormalizer
+    ```python
+    from semantica.normalize import NumberNormalizer
 
-normalizer = NumberNormalizer()
+    normalizer = NumberNormalizer()
 
-normalizer.normalize_number("$1,234.56")   # → 1234.56
-normalizer.normalize_number("€42K")         # → 42000.0
-normalizer.normalize_number("$1.2B")        # → 1200000000.0
-normalizer.normalize_number("3.14e-2")      # → 0.0314
-normalizer.normalize_number("42%")          # → 0.42
-```
+    normalizer.normalize_number("$1,234.56")   # → 1234.56
+    normalizer.normalize_number("€42K")         # → 42000.0
+    normalizer.normalize_number("$1.2B")        # → 1200000000.0
+    normalizer.normalize_number("3.14e-2")      # → 0.0314
+    normalizer.normalize_number("42%")          # → 0.42
+    normalizer.normalize_number("−7")           # → -7.0  (minus sign, not hyphen)
+    ```
 
-Unit and currency conversion:
+    **Unit and currency conversion:**
 
-```python
-from semantica.normalize import UnitConverter, CurrencyNormalizer
+    ```python
+    from semantica.normalize import UnitConverter, CurrencyNormalizer
 
-converter = UnitConverter()
-result = converter.convert(100, from_unit="km/h", to_unit="m/s")
-# → 27.78
+    converter = UnitConverter()
+    result    = converter.convert(100, from_unit="km/h", to_unit="m/s")
+    # → 27.78
 
-# Supported categories: length, weight, volume, temperature, speed, area
-categories = converter.list_categories()
+    categories = converter.list_categories()
+    # → ["length", "weight", "volume", "temperature", "speed", "area", "pressure", "energy"]
 
-currency_norm = CurrencyNormalizer()
-result = currency_norm.normalize("$42.50")
-# → {"amount": 42.50, "currency": "USD", "raw": "$42.50"}
-```
+    currency_norm = CurrencyNormalizer()
+    result = currency_norm.normalize("$42.50")
+    # → {"amount": 42.50, "currency": "USD", "raw": "$42.50"}
+    ```
+  </Tab>
+  <Tab title="Language & Encoding">
+    ### LanguageDetector
+
+    Identify the language of a text string. Used internally by the sentence splitter and chunker:
+
+    ```python
+    from semantica.normalize import LanguageDetector
+
+    detector = LanguageDetector()
+
+    lang  = detector.detect("Bonjour le monde")
+    # → {"language": "fr", "confidence": 0.98}
+
+    langs = detector.detect_top_n("This might be mixed", n=3)
+    # → [{"language": "en", "probability": 0.85}, ...]
+
+    results   = detector.detect_batch(["Hello", "Hola", "Bonjour", "Ciao"])
+    supported = detector.list_supported_languages()
+    ```
+
+    Supports 50+ languages: `en`, `de`, `fr`, `es`, `it`, `pt`, `nl`, `ru`, `zh`, `ja`, `ko`, `ar`, `hi`, `tr`, `pl`, `sv`, `da`, `no`, `fi`, and more.
+
+    ### EncodingHandler
+
+    Detect and repair character encoding issues:
+
+    ```python
+    from semantica.normalize import EncodingHandler
+
+    handler = EncodingHandler()
+
+    encoding  = handler.detect_encoding(raw_bytes)
+    # → {"encoding": "windows-1252", "confidence": 0.73}
+
+    utf8_text = handler.to_utf8(raw_bytes)
+    clean     = handler.remove_bom(text_with_bom)
+    repaired  = handler.repair_encoding(garbled_text, source_encoding="cp1252")
+    ```
+
+    **Key behaviours:**
+    - Encoding detection uses `chardet` internally — accuracy improves with longer input
+    - `to_utf8()` attempts cp1252 repair automatically when it detects mojibake patterns
+    - Always run `EncodingHandler` first — broken bytes cause cascading failures in every downstream normalizer
+  </Tab>
+</Tabs>
 
 ## DataCleaner
 
+Cleans structured record sets — useful before loading into a vector store or graph:
+
 ```python
 from semantica.normalize import DataCleaner, DataValidator
 
 cleaner = DataCleaner()
 
-# Remove near-duplicate records
 deduped = cleaner.remove_duplicates(records, similarity_threshold=0.9)
 
-# Fill missing values
-filled = cleaner.fill_missing(records, strategy="mean")
-# strategy options: "mean" | "median" | "mode" | "remove"
+filled = cleaner.fill_missing(
+    records,
+    strategy="mean",      # "mean" | "median" | "mode" | "remove" | "constant"
+    constant_value=None,
+)
 
-# Validate schema
 validator = DataValidator()
-result = validator.validate(records, schema={"name": str, "age": int})
-print(result.valid_count, result.errors)
-```
-
-## LanguageDetector
-
-```python
-from semantica.normalize import LanguageDetector
-
-detector = LanguageDetector()
-
-lang = detector.detect("Bonjour le monde")
-# → {"language": "fr", "confidence": 0.98}
-
-# Top N languages for mixed-language text
-langs = detector.detect_top_n("This might be mixed", n=3)
-# → [{"language": "en", "probability": 0.85}, ...]
-
-# Batch detection
-results = detector.detect_batch(["Hello", "Hola", "Bonjour"])
-```
-
-## EncodingHandler
-
-```python
-from semantica.normalize import EncodingHandler
-
-handler = EncodingHandler()
-
-encoding = handler.detect_encoding(raw_bytes)
-# → {"encoding": "windows-1252", "confidence": 0.73}
-
-utf8_text = handler.to_utf8(raw_bytes)
-clean     = handler.remove_bom(text_with_bom)
+result = validator.validate(records, schema={"name": str, "age": int, "active": bool})
+print(f"Valid:   {result.valid_count}")
+print(f"Invalid: {result.error_count}")
 ```
 
 ## Pipeline Integration
 
-For large datasets, use the pipeline instead of per-item normalization:
-
 ```python
 from semantica.pipeline import Pipeline
+from semantica.ingest import FileIngestor
 from semantica.normalize import TextNormalizer
+from semantica.semantic_extract import NERExtractor
+from semantica.llms import Groq
+import os
+
+llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
 
 pipeline = Pipeline()
-pipeline.add_step("normalize", TextNormalizer())
-result = pipeline.run(documents)
+pipeline.add_step("ingest",    FileIngestor())
+pipeline.add_step("normalize", TextNormalizer(strip_html=True, normalize_unicode=True))
+pipeline.add_step("extract",   NERExtractor(method="llm", llm_provider=llm))
+
+result = pipeline.run("data/documents/")
 ```
 
+## Tips and Common Pitfalls
+
+<Warning>
+  **Run encoding repair before anything else.** A single cp1252 character in a UTF-8 stream silently corrupts the surrounding text. Run `EncodingHandler` or set `fix_encoding=True` on `TextNormalizer` first.
+</Warning>
+
+<Warning>
+  **Don't lowercase before NER.** `normalize_text(lowercase=True)` before entity extraction destroys capitalization signals that NER relies on. Apply case normalization only after extraction if needed.
+</Warning>
+
+<Tip>
+  **AliasResolver is order-sensitive.** If you register overlapping aliases (`"ML"` and `"ML model"`), the longer match wins. Sort aliases by length descending for predictable behaviour.
+</Tip>
+
+<Warning>
+  **DateNormalizer and timezone.** Without `target_timezone`, dates without timezone information are returned as naive datetime strings. In regulated pipelines (HIPAA, SOX), always set `target_timezone="UTC"` for unambiguous timestamps.
+</Warning>
+
+<Tip>
+  **`DataCleaner.remove_duplicates()` is not the same as `DuplicateDetector`.** `DataCleaner` operates on flat records (dicts/rows) by field-level Jaccard similarity. `DuplicateDetector` in the Deduplication module operates on graph entities with embedding-based matching. Use the latter for entity resolution.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Parse" icon="file-lines" href="parse">
     Parse documents before normalization.
@@ -278,9 +407,9 @@ result = pipeline.run(documents)
     Chunk normalized text for embedding.
   </Card>
   <Card title="Deduplication" icon="copy" href="deduplication">
-    Resolve duplicate entities post-normalization.
+    Resolve duplicate entities after normalization.
   </Card>
   <Card title="Pipeline" icon="gear" href="pipeline">
-    Include normalization as a pipeline step.
+    Include normalization as a named pipeline step.
   </Card>
 </CardGroup>

docs/reference/ontology.md

@@ -8,36 +8,70 @@ icon: "sitemap"
 
 ## What You Get
 
-- **`OntologyManager`** — define classes, properties, relationships, and constraints
-- **`OntologyGenerator`** — auto-generate ontologies from existing knowledge graph data (6-stage pipeline)
-- **`SHACLGenerator`** / **`SHACLValidator`** — generate and validate SHACL shapes
-- **`SKOSVocabulary`** — controlled vocabulary and taxonomy management
-- **`OntologyAligner`** — align and merge ontologies across schemas
-- **`OntologyDiff`** / **`OntologyMigrator`** — diff and migrate ontology versions
-- **`OWLExporter`** — export to Turtle, RDF/XML, JSON-LD
-- **Ontology Hub** — visual browser UI for the full ontology lifecycle (v0.5.0)
+<CardGroup cols={2}>
+  <Card title="OntologyManager" icon="sitemap">
+    Define classes, properties, relationships, and constraints for your knowledge graph schema.
+  </Card>
+  <Card title="OntologyGenerator" icon="wand-magic-sparkles">
+    Auto-generate ontologies from existing graph data using a 6-stage pipeline.
+  </Card>
+  <Card title="SHACLGenerator / SHACLValidator" icon="shield-check">
+    Generate SHACL shapes from an ontology and validate graphs for constraint compliance.
+  </Card>
+  <Card title="SKOSVocabulary" icon="list-tree">
+    Controlled vocabulary and taxonomy management using the W3C SKOS standard.
+  </Card>
+  <Card title="OntologyAligner" icon="arrows-left-right">
+    Align and merge ontologies across schemas — maps concepts with confidence scores.
+  </Card>
+  <Card title="Ontology Hub (v0.5.0)" icon="display">
+    Visual browser UI for the full ontology lifecycle — editor, SHACL Studio, and health dashboard.
+  </Card>
+</CardGroup>
 
-## OntologyManager
+## Quick Start
 
-Define and validate a schema for your knowledge graph:
+<Steps>
+  <Step title="Define your schema">
+    ```python
+    from semantica.ontology import OntologyManager
 
-```python
-from semantica.ontology import OntologyManager
+    ontology = OntologyManager()
+    ontology.add_class("Person",       properties=["name", "birth_date"])
+    ontology.add_class("Organization", properties=["name", "founded_date"])
+    ontology.add_relationship("works_for", domain="Person", range="Organization")
+    ontology.add_constraint("Person", "must_have_name")
+    ```
+  </Step>
+  <Step title="Validate your graph against the schema">
+    ```python
+    is_valid = ontology.validate_graph(kg)
 
-ontology = OntologyManager()
-ontology.add_class("Person",       properties=["name", "birth_date"])
-ontology.add_class("Organization", properties=["name", "founded_date"])
-ontology.add_relationship("works_for", domain="Person", range="Organization")
-ontology.add_constraint("Person", "must_have_name")
+    # Or use SHACL for granular constraint reporting
+    from semantica.ontology import SHACLGenerator, SHACLValidator
 
-# Validate a graph against the ontology
-is_valid = ontology.validate_graph(kg)
+    shapes    = SHACLGenerator().generate(ontology)
+    validator = SHACLValidator()
+    report    = validator.validate(kg, shapes=shapes)
 
-# Export as OWL Turtle
-owl_ttl = ontology.export_owl(format="turtle")
-```
+    if not report.conforms:
+        for v in report.violations:
+            print(f"Violation: {v.message} on {v.node}  (path: {v.path})")
+    ```
+  </Step>
+  <Step title="Export to OWL">
+    ```python
+    from semantica.ontology import OWLExporter
 
-## Auto-Generation (6-Stage Pipeline)
+    exporter = OWLExporter()
+    exporter.export(ontology, path="ontology.ttl",  format="turtle")
+    exporter.export(ontology, path="ontology.owl",  format="xml")
+    exporter.export(ontology, path="ontology.json", format="json-ld")
+    ```
+  </Step>
+</Steps>
+
+## Auto-Generation — 6-Stage Pipeline
 
 Generate an ontology automatically from your knowledge graph data:
 
@@ -45,40 +79,74 @@ Generate an ontology automatically from your knowledge graph data:
 from semantica.ontology import OntologyGenerator
 
 generator = OntologyGenerator()
-ontology = generator.generate_from_graph(kg)
+ontology  = generator.generate_from_graph(kg)
 ```
 
 The pipeline runs through these stages in order:
 
-1. **Semantic Network Parsing** — extract concepts and patterns from entity/relationship data
-2. **YAML-to-Definition** — transform patterns into intermediate class definitions
-3. **Definition-to-Types** — map definitions to OWL types (`owl:Class`, `owl:ObjectProperty`)
-4. **Hierarchy Generation** — build taxonomy trees using transitive closure and cycle detection
-5. **TTL Generation** — serialize to Turtle format using `rdflib`
-6. **Quality Evaluation** — assess coverage, completeness, and granularity metrics
+<Steps>
+  <Step title="Stage 1 — Semantic Network Parsing">
+    Extracts concepts and patterns from entity/relationship data.
 
-## SHACL Validation
+    ```python
+    generator        = OntologyGenerator()
+    semantic_network = generator.parse_semantic_network(kg)
+    ```
+  </Step>
+  <Step title="Stage 2 — YAML-to-Definition (intermediate representation)">
+    Transforms extracted patterns into intermediate class definitions.
 
-Generate SHACL shapes from an ontology and validate any graph against them:
+    ```python
+    definitions = generator.build_definitions(semantic_network)
+    ```
+  </Step>
+  <Step title="Stage 3 — Definition-to-OWL Types">
+    Maps definitions to OWL types (`owl:Class`, `owl:ObjectProperty`).
 
-```python
-from semantica.ontology import SHACLGenerator, SHACLValidator
+    ```python
+    from semantica.ontology import ClassInferrer, PropertyGenerator
 
-# Generate shapes
-generator = SHACLGenerator()
-shapes    = generator.generate(ontology)
-shapes_ttl = shapes.serialize(format="turtle")
+    class_inferrer = ClassInferrer()
+    classes        = class_inferrer.infer_classes(kg.entities)
 
-# Validate a graph
-validator = SHACLValidator()
-report    = validator.validate(kg, shapes=shapes)
+    prop_generator = PropertyGenerator()
+    properties     = prop_generator.infer_properties(kg.entities, kg.relationships, classes)
+    ```
+  </Step>
+  <Step title="Stage 4 — Hierarchy Generation">
+    Builds taxonomy trees using transitive closure and cycle detection.
 
-if not report.conforms:
-    for violation in report.violations:
-        print(f"Violation: {violation.message} on {violation.node}")
-        print(f"  Path:     {violation.path}")
-        print(f"  Severity: {violation.severity}")
-```
+    ```python
+    hierarchy = generator.build_hierarchy(classes)
+    ```
+  </Step>
+  <Step title="Stage 5 — TTL Generation">
+    Serializes to Turtle format using `rdflib`.
+
+    ```python
+    from semantica.ontology import OWLGenerator
+
+    owl_gen = OWLGenerator()
+    ttl_str = owl_gen.generate_owl(
+        {"classes": classes, "properties": properties, "hierarchy": hierarchy},
+        format="turtle",   # "turtle" | "xml" | "json-ld" | "n3"
+    )
+    ```
+  </Step>
+  <Step title="Stage 6 — Quality Evaluation">
+    Assesses coverage, completeness, and granularity metrics.
+
+    ```python
+    from semantica.ontology import OntologyEvaluator
+
+    evaluator = OntologyEvaluator()
+    report    = evaluator.evaluate(ttl_str, kg)
+    print(f"Coverage:     {report.coverage:.2%}")
+    print(f"Completeness: {report.completeness:.2%}")
+    print(f"Granularity:  {report.granularity:.2%}")
+    ```
+  </Step>
+</Steps>
 
 ## SKOS Vocabularies
 
@@ -88,9 +156,9 @@ Build controlled vocabularies and taxonomies using the W3C SKOS standard:
 from semantica.ontology import SKOSVocabulary
 
 vocab = SKOSVocabulary()
-vocab.add_concept("Machine Learning",   broader="Artificial Intelligence")
-vocab.add_concept("Deep Learning",      broader="Machine Learning")
-vocab.add_concept("Computer Vision",    broader="Deep Learning")
+vocab.add_concept("Machine Learning",  broader="Artificial Intelligence")
+vocab.add_concept("Deep Learning",     broader="Machine Learning")
+vocab.add_concept("Computer Vision",   broader="Deep Learning")
 vocab.add_alt_label("ML", for_concept="Machine Learning")
 
 skos_ttl = vocab.export(format="turtle")
@@ -98,49 +166,187 @@ skos_ttl = vocab.export(format="turtle")
 
 ## Ontology Alignment
 
-Map concepts across two ontologies and merge them:
+<Tabs>
+  <Tab title="Align Two Ontologies">
+    Map concepts across two ontologies:
+
+    ```python
+    from semantica.ontology import OntologyAligner
+
+    aligner   = OntologyAligner()
+    alignment = aligner.align(source_ontology, target_ontology)
+
+    for mapping in alignment.mappings:
+        print(f"{mapping.source} → {mapping.target}  (confidence: {mapping.confidence:.2f})")
+
+    merged = aligner.merge(source_ontology, target_ontology, alignment)
+    ```
+  </Tab>
+  <Tab title="Diff and Migration">
+    Compare versions and generate migration scripts:
+
+    ```python
+    from semantica.ontology import OntologyDiff, OntologyMigrator
+
+    diff    = OntologyDiff()
+    changes = diff.compare(ontology_v1, ontology_v2)
+
+    for change in changes:
+        print(f"{change.type}: {change.element} — {change.description}")
+
+    migrator         = OntologyMigrator()
+    migration_script = migrator.generate_migration(changes)
+    migrator.apply(kg, migration_script)
+    ```
+  </Tab>
+</Tabs>
+
+## Advanced Generation Tools
+
+<Tabs>
+  <Tab title="RequirementsSpecManager">
+    Capture and manage ontology requirements before generation:
+
+    ```python
+    from semantica.ontology import RequirementsSpecManager
+
+    spec = RequirementsSpecManager()
+    spec.add_competency_question("What organizations are headquartered in California?")
+    spec.add_competency_question("Who founded each organization?")
+    spec.add_competency_question("What products does each organization sell?")
+
+    spec.set_scope(
+        domain="Technology industry",
+        excluded_types=["Event", "Date"],
+        min_confidence=0.7,
+    )
+
+    generator = OntologyGenerator()
+    ontology  = generator.generate_from_graph(kg, requirements=spec)
+    ```
+  </Tab>
+  <Tab title="LLMOntologyGenerator">
+    Generate ontologies directly from natural language:
+
+    ```python
+    from semantica.ontology import LLMOntologyGenerator
+    from semantica.llms import Groq
+    import os
+
+    llm       = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+    generator = LLMOntologyGenerator(llm_provider=llm)
+
+    ontology = generator.generate_from_description(
+        description="An ontology for tracking pharmaceutical clinical trials, including drugs, patients, dosages, outcomes, and adverse events.",
+        num_classes=20,
+    )
+
+    ontology = generator.generate_from_corpus(
+        documents=["clinical_trial_protocol.pdf"],
+        domain_hint="biomedical",
+    )
+
+    ontology = generator.refine(
+        ontology,
+        questions=["What dosage was given to each patient?", "What adverse events occurred?"],
+    )
+    ```
+  </Tab>
+  <Tab title="ReuseManager">
+    Integrate published ontologies instead of generating from scratch:
+
+    ```python
+    from semantica.ontology import ReuseManager
+
+    manager = ReuseManager()
+    manager.load_ontology("schema.org", source="https://schema.org/version/latest/schemaorg-current-https.ttl")
+    manager.load_ontology("foaf",       source="http://xmlns.com/foaf/spec/index.rdf")
+
+    candidates = manager.find_reusable_classes(
+        your_classes=["Person", "Organization", "Product"],
+        loaded_ontologies=["schema.org", "foaf"],
+    )
+    for candidate in candidates:
+        print(f"{candidate.local_class} → reuse {candidate.external_uri}  (similarity: {candidate.similarity:.2f})")
+
+    merged = manager.merge_reused(your_ontology, candidates)
+    ```
+  </Tab>
+  <Tab title="DomainOntologies">
+    Pre-built domain ontologies for common verticals:
+
+    ```python
+    from semantica.ontology import DomainOntologies
+
+    catalog = DomainOntologies()
+
+    for domain in catalog.list_domains():
+        print(f"{domain.name}: {domain.description}  ({domain.class_count} classes)")
+
+    biomedical   = catalog.load("biomedical")    # SNOMED CT-aligned
+    finance      = catalog.load("finance")       # FinancialInstrument, Company, Market
+    legal        = catalog.load("legal")         # Contract, Party, Jurisdiction
+    supply_chain = catalog.load("supply_chain")  # Supplier, Product, Shipment
+
+    biomedical.add_class("ClinicalTrial", parent="Study", properties=["phase", "participants"])
+    ```
+
+    Available domains: `biomedical`, `finance`, `legal`, `supply_chain`, `cybersecurity`, `e_commerce`, `hr`.
+  </Tab>
+</Tabs>
+
+## ModuleManager
+
+Build ontologies as composable modules — keep domain logic separated and reusable:
 
 ```python
-from semantica.ontology import OntologyAligner
+from semantica.ontology import ModuleManager
 
-aligner   = OntologyAligner()
-alignment = aligner.align(source_ontology, target_ontology)
+manager = ModuleManager()
 
-for mapping in alignment.mappings:
-    print(f"{mapping.source} → {mapping.target}  (confidence: {mapping.confidence:.2f})")
+core_module    = manager.create_module("core",    base_uri="http://example.org/core#")
+finance_module = manager.create_module("finance", base_uri="http://example.org/finance#")
 
-# Merge into a unified ontology
-merged = aligner.merge(source_ontology, target_ontology, alignment)
+core_module.add_class("Entity",     properties=["id", "name"])
+finance_module.add_class("Company", parent="Entity", properties=["ticker", "revenue"])
+
+finance_module.import_module(core_module)
+unified = manager.merge_modules([core_module, finance_module])
 ```
 
-## Diff and Migration
+## NamespaceManager
 
-Compare ontology versions and generate migration scripts for graph data:
+Manage IRI prefixes and generate consistent URIs for all ontology terms:
 
 ```python
-from semantica.ontology import OntologyDiff, OntologyMigrator
+from semantica.ontology import NamespaceManager
 
-diff    = OntologyDiff()
-changes = diff.compare(ontology_v1, ontology_v2)
+ns_manager = NamespaceManager(base_uri="http://example.org/")
+ns_manager.register("ex",     "http://example.org/")
+ns_manager.register("schema", "https://schema.org/")
 
-for change in changes:
-    print(f"{change.type}: {change.element} — {change.description}")
-
-# Generate and apply a migration script
-migrator         = OntologyMigrator()
-migration_script = migrator.generate_migration(changes)
-migrator.apply(kg, migration_script)
+class_iri = ns_manager.generate_iri("Person")          # → "http://example.org/Person"
+prop_iri  = ns_manager.generate_iri("worksFor", prefix="ex")  # → "http://example.org/worksFor"
+iri       = ns_manager.resolve("schema:Organization")   # → "https://schema.org/Organization"
 ```
 
-## OWL / RDF Export
+## OntologyEvaluator
+
+Measure quality across coverage, completeness, and competency questions:
 
 ```python
-from semantica.ontology import OWLExporter
+from semantica.ontology import OntologyEvaluator
 
-exporter = OWLExporter()
-exporter.export(ontology, path="ontology.ttl",  format="turtle")
-exporter.export(ontology, path="ontology.owl",  format="xml")
-exporter.export(ontology, path="ontology.json", format="json-ld")
+evaluator = OntologyEvaluator()
+report    = evaluator.evaluate(ontology, kg)
+print(f"Coverage:     {report.coverage:.2%}")
+print(f"Completeness: {report.completeness:.2%}")
+print(f"Granularity:  {report.granularity:.2%}")
+
+questions  = ["What organizations were founded in California?", "Who are the employees of Apple Inc.?"]
+cq_results = evaluator.validate_competency_questions(ontology, kg, questions)
+for q, result in zip(questions, cq_results):
+    print(f"Q: {q}  →  Answerable: {result.answerable}  ({result.reason})")
 ```
 
 ## Ontology Hub (v0.5.0)
@@ -158,13 +364,37 @@ start_explorer(graph=kg, port=8080)
 # Navigate to http://localhost:8080 → Ontology Hub tab
 ```
 
-Features:
+Features: visual editor, SHACL Studio, alignment authoring, health dashboard, and version control.
 
-- **Visual editor** — create and edit classes, properties, and relationships in the browser
-- **SHACL Studio** — author and validate SHACL shapes with live feedback
-- **Alignment authoring** — map concepts across ontologies with drag-and-drop
-- **Health dashboard** — coverage, completeness, and constraint violation metrics
-- **Version control** — snapshot, diff, and restore ontology versions
+## Tips and Common Pitfalls
+
+<Tip>
+  **Define competency questions before generating.** An ontology without competency questions has no measurable success criteria. Write 5–10 natural language questions your ontology must answer before calling `OntologyGenerator`. Then validate them with `OntologyEvaluator.validate_competency_questions()`.
+</Tip>
+
+<Tip>
+  **Reuse before generating.** `DomainOntologies` and `ReuseManager` give you schema.org, FOAF, and domain-specific ontologies that took years to develop. Reusing established classes (`schema:Organization`, `foaf:Person`) also improves interoperability with external data.
+</Tip>
+
+<Warning>
+  **`LLMOntologyGenerator` is great for prototyping, not production.** LLM-generated ontologies are a useful starting point but need expert review. Use `OntologyEvaluator` and manual SHACL authoring to harden the schema before relying on it for production graph validation.
+</Warning>
+
+<Warning>
+  **Always validate with SHACL after schema changes.** When you add new classes or properties, run `SHACLValidator.validate(kg, shapes)` immediately. SHACL violations often surface data quality issues that were silently passing before.
+</Warning>
+
+<Tip>
+  **Use `ModuleManager` for large ontologies.** A monolithic 500-class ontology becomes unmanageable quickly. Split by domain (`core`, `finance`, `legal`) and use `owl:imports` to compose them — changes in one module don't break others.
+</Tip>
+
+<Tip>
+  **Namespace your terms consistently.** All classes and properties need stable IRIs. Use `NamespaceManager` to generate them programmatically — never hardcode IRI strings in code, because base URIs change when projects move.
+</Tip>
+
+<Warning>
+  **Diff before migration.** Always run `OntologyDiff.compare()` before `OntologyMigrator.apply()`. The diff shows exactly which graph entities will be affected — some migrations (renaming a class) require updating thousands of existing nodes.
+</Warning>
 
 <CardGroup cols={2}>
   <Card title="Reasoning" icon="microchip" href="reasoning">

docs/reference/parse.md

@@ -8,75 +8,229 @@ icon: "file-lines"
 
 ## What You Get
 
-- **`DocumentParser`** — standard parser for PDF, DOCX, HTML, TXT, JSON, CSV, PPTX, XLSX
-- **`DoclingParser`** — advanced parser for complex layouts, merged-cell tables, multi-column PDFs, and OCR
-- **`ParsedDocument`** — structured output with `text`, `sections`, `tables`, and `metadata`
+<CardGroup cols={2}>
+  <Card title="DocumentParser" icon="file-lines">
+    Standard parser for PDF, DOCX, HTML, TXT, JSON, CSV, PPTX, XLSX — zero config, no extras.
+  </Card>
+  <Card title="DoclingParser" icon="file-pdf">
+    Advanced parser for complex layouts, merged-cell tables, multi-column PDFs, and OCR.
+  </Card>
+  <Card title="CodeParser" icon="code">
+    AST structure extraction — functions, classes, imports, dependencies — for 10+ languages.
+  </Card>
+  <Card title="ImageParser" icon="image">
+    EXIF metadata extraction and OCR via Tesseract for image files.
+  </Card>
+  <Card title="MediaParser" icon="photo-film">
+    Technical metadata from audio, video, and image files (duration, codec, resolution).
+  </Card>
+  <Card title="MCPParser" icon="plug">
+    Parse Model Context Protocol responses into structured `ParsedDocument` objects.
+  </Card>
+</CardGroup>
 
-## DocumentParser
+## Quick Start
 
-Standard parser for clean, machine-readable documents:
+<Steps>
+  <Step title="Parse a standard document">
+    ```python
+    from semantica.parse import DocumentParser
 
-```python
-from semantica.parse import DocumentParser
+    parser = DocumentParser()
+    parsed = parser.parse("data/report.pdf")
 
-parser = DocumentParser()
-parsed = parser.parse("data/report.pdf")
+    print(parsed.text)       # full clean text
+    print(parsed.metadata)   # title, author, date, page_count, language, etc.
+    print(parsed.sections)   # document structure as a list of Section objects
+    ```
+  </Step>
+  <Step title="Use DoclingParser for complex layouts">
+    ```bash
+    pip install "semantica[docling]"
+    ```
 
-print(parsed.text)       # full clean text
-print(parsed.metadata)   # title, author, date, page_count, language, etc.
-print(parsed.sections)   # document structure as a list of Section objects
-```
+    ```python
+    from semantica.parse import DoclingParser
 
-Supported formats: PDF, DOCX, HTML, TXT, JSON, CSV, PPTX, XLSX.
+    parser = DoclingParser(
+        extract_tables=True,       # structured table extraction with cell type detection
+        extract_images=True,       # extract image regions for downstream OCR
+        output_format="markdown",  # "markdown" | "html" | "json"
+    )
 
-## DoclingParser
+    parsed = parser.parse("data/annual_report.pdf")
+    print(parsed.tables)   # structured TableData objects with headers and rows
+    ```
+  </Step>
+  <Step title="Feed into the split and extract pipeline">
+    ```python
+    from semantica.split import TextSplitter
+    from semantica.semantic_extract import NERExtractor
+    from semantica.llms import Groq
+    import os
 
-Advanced parser using the Docling backend — handles layouts that `DocumentParser` cannot:
+    splitter = TextSplitter(method="structural")
+    chunks   = splitter.split_document(parsed)
 
-```bash
-pip install "semantica[docling]"
-```
+    llm       = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+    extractor = NERExtractor(method="llm", llm_provider=llm)
+    entities  = extractor.extract_batch([c.text for c in chunks])
+    ```
+  </Step>
+</Steps>
 
-```python
-from semantica.parse import DoclingParser
+## Parser Reference
 
-parser = DoclingParser(
-    extract_tables=True,       # structured table extraction with cell type detection
-    extract_images=True,       # extract image regions for downstream OCR
-    output_format="markdown",  # "markdown" | "html" | "json"
-)
+<Tabs>
+  <Tab title="DocumentParser">
+    Standard parser for clean, machine-readable documents — no extra dependencies required:
 
-parsed = parser.parse("data/annual_report.pdf")
+    ```python
+    from semantica.parse import DocumentParser
 
-print(parsed.text)     # full clean text
-print(parsed.tables)   # structured TableData objects with headers and rows
-print(parsed.sections) # document structure with heading hierarchy
-```
+    parser = DocumentParser()
+    parsed = parser.parse("data/report.pdf")
 
-Use `DoclingParser` for:
+    print(parsed.text)       # full clean text
+    print(parsed.metadata)   # title, author, date, page_count, language, etc.
+    print(parsed.sections)   # document structure as a list of Section objects
+    ```
 
-- Multi-column PDF layouts
-- Tables with merged cells or complex headers
-- PPTX slides with embedded charts
-- XLSX spreadsheets with formulas
-- Scanned documents with OCR
-- Academic papers and technical reports
+    **Supported formats:** PDF, DOCX, HTML, TXT, JSON, CSV, PPTX, XLSX.
+  </Tab>
+  <Tab title="DoclingParser">
+    Advanced parser using the Docling backend — handles layouts that `DocumentParser` cannot:
 
-## OCR Support
+    ```python
+    from semantica.parse import DoclingParser
 
-```python
-parser = DoclingParser(
-    ocr=True,
-    ocr_language=["en"],   # ISO 639-1 codes; list for multi-language documents
-    extract_tables=True,
-)
+    parser = DoclingParser(
+        extract_tables=True,       # structured table extraction with cell type detection
+        extract_images=True,       # extract image regions for downstream OCR
+        output_format="markdown",  # "markdown" | "html" | "json"
+    )
 
-parsed = parser.parse("data/scanned_contract.pdf")
-```
+    parsed = parser.parse("data/annual_report.pdf")
+    print(parsed.tables)    # structured TableData objects with headers and rows
+    print(parsed.sections)  # document structure with heading hierarchy
+    ```
 
-## Parsed Document Object
+    **Use `DoclingParser` for:**
+    - Multi-column PDF layouts
+    - Tables with merged cells or complex headers
+    - PPTX slides with embedded charts
+    - XLSX spreadsheets with formulas
+    - Scanned documents with OCR
+    - Academic papers and technical reports
 
-Both parsers return a `ParsedDocument` with the same structure:
+    **OCR support:**
+
+    ```python
+    parser = DoclingParser(
+        ocr=True,
+        ocr_language=["en"],   # ISO 639-1 codes; list for multi-language documents
+        extract_tables=True,
+    )
+    parsed = parser.parse("data/scanned_contract.pdf")
+    ```
+  </Tab>
+  <Tab title="CodeParser">
+    Parse source code files — extracts AST structure, functions, classes, imports, and comments:
+
+    ```python
+    from semantica.parse import CodeParser
+
+    parser = CodeParser(
+        extract_comments=True,       # include docstrings and inline comments
+        extract_dependencies=True,   # import/require statements
+        language="auto",             # "auto" | "python" | "javascript" | "java" | "go" | "rust" | "cpp"
+    )
+
+    parsed = parser.parse("src/main.py")
+
+    print(parsed.text)                      # raw source code as text
+    print(parsed.metadata["language"])      # detected language
+    print(parsed.metadata["functions"])     # list of function names
+    print(parsed.metadata["classes"])       # list of class names
+    print(parsed.metadata["imports"])       # list of import statements
+    print(parsed.metadata["comments"])      # docstrings and inline comments
+    ```
+
+    **Supported languages:** Python, JavaScript/TypeScript, Java, Go, Rust, C/C++, C#, Ruby, PHP, Swift.
+  </Tab>
+  <Tab title="ImageParser">
+    Extract EXIF metadata and optionally perform OCR on image files:
+
+    ```python
+    from semantica.parse import ImageParser
+
+    parser = ImageParser(
+        extract_exif=True,   # camera, GPS, timestamps, etc.
+        ocr=True,            # OCR via Tesseract (requires tesseract-ocr installed)
+        ocr_language="en",   # ISO 639-1 language code for OCR
+    )
+
+    parsed = parser.parse("photo.jpg")
+
+    print(parsed.text)                        # OCR-extracted text (if ocr=True)
+    print(parsed.metadata["width"])           # image dimensions
+    print(parsed.metadata["height"])
+    print(parsed.metadata["format"])          # "JPEG" | "PNG" | "TIFF" | ...
+    print(parsed.metadata["exif"]["GPS"])     # GPS coordinates if available
+    print(parsed.metadata["exif"]["DateTime"])
+    ```
+  </Tab>
+  <Tab title="MediaParser & MCPParser">
+    ### MediaParser
+
+    Extract technical metadata from audio, video, and image files:
+
+    ```python
+    from semantica.parse import MediaParser
+
+    parser = MediaParser()
+
+    # Video file
+    parsed = parser.parse("interview.mp4")
+    print(parsed.metadata["duration_seconds"])
+    print(parsed.metadata["codec"])
+    print(parsed.metadata["resolution"])
+    print(parsed.metadata["fps"])
+
+    # Audio file
+    parsed = parser.parse("podcast.mp3")
+    print(parsed.metadata["duration_seconds"])
+    print(parsed.metadata["bitrate"])
+    print(parsed.metadata["channels"])
+    ```
+
+    **Supported formats:** MP4, AVI, MOV, MKV, MP3, WAV, FLAC, OGG, JPEG, PNG, TIFF, WebP.
+
+    ### MCPParser
+
+    Parse Model Context Protocol (MCP) responses into structured `ParsedDocument` objects:
+
+    ```python
+    from semantica.parse import MCPParser
+
+    parser = MCPParser()
+
+    mcp_response = {
+        "content": [{"type": "text", "text": "Apple Inc. was founded in 1976..."}],
+        "metadata": {"tool": "web_search", "query": "Apple Inc history"}
+    }
+
+    parsed = parser.parse(mcp_response)
+    print(parsed.text)      # "Apple Inc. was founded in 1976..."
+    print(parsed.metadata)  # tool name, query, and other MCP metadata
+    ```
+  </Tab>
+</Tabs>
+
+## Parsed Document Schema
+
+<AccordionGroup>
+  <Accordion title="ParsedDocument dataclass">
 
 ```python
 @dataclass
@@ -86,7 +240,12 @@ class ParsedDocument:
     tables:    List[TableData]      # structured table data (DoclingParser only)
     metadata:  DocumentMetadata     # title, author, dates, page count
     source_id: str                  # links back to the original DataSource
+```
 
+  </Accordion>
+  <Accordion title="DocumentMetadata dataclass">
+
+```python
 @dataclass
 class DocumentMetadata:
     title:        Optional[str]
@@ -100,6 +259,21 @@ class DocumentMetadata:
     format:       str               # "pdf" | "docx" | "pptx" | ...
 ```
 
+  </Accordion>
+</AccordionGroup>
+
+## Choosing a Parser
+
+| Scenario | Parser |
+| -------- | ------ |
+| Clean PDFs, DOCX, HTML, TXT, CSV, Excel | `DocumentParser` — zero config, no extras |
+| Scanned PDFs, OCR required | `DoclingParser(ocr=True)` — requires `pip install "semantica[docling]"` |
+| Multi-column PDFs, merged-cell tables | `DoclingParser(extract_tables=True)` |
+| Source code files | `CodeParser(language="auto")` |
+| Images with embedded text | `ImageParser(ocr=True)` — requires Tesseract |
+| Audio/video metadata | `MediaParser()` |
+| MCP tool responses | `MCPParser()` |
+
 ## Integration with FileIngestor
 
 The most common pattern — ingest a directory then parse each source:
@@ -121,6 +295,28 @@ for source in sources:
   Docling is an optional dependency. If `docling` is not installed, `DoclingParser` raises an `ImportError` with installation instructions. `DocumentParser` is always available and requires no extras.
 </Note>
 
+## Tips and Common Pitfalls
+
+<Tip>
+  **Start with `DocumentParser` and only switch to `DoclingParser` when needed.** `DoclingParser` is significantly more powerful but slower and requires an additional dependency. For clean machine-readable PDFs and Office files, `DocumentParser` is fast and accurate enough.
+</Tip>
+
+<Warning>
+  **OCR requires Tesseract installed on the system.** `ImageParser(ocr=True)` and `DoclingParser(ocr=True)` both call Tesseract under the hood. Install it with `apt-get install tesseract-ocr` (Linux) or `brew install tesseract` (macOS) before enabling OCR.
+</Warning>
+
+<Tip>
+  **`extract_tables=True` is off by default for speed.** Table extraction in `DoclingParser` requires additional layout analysis passes. Only enable it when you actually need structured table data — for text-only extraction, leave it off.
+</Tip>
+
+<Tip>
+  **`CodeParser` outputs AST metadata, not just raw text.** The `parsed.metadata["functions"]` and `parsed.metadata["classes"]` lists are useful for building code-level knowledge graphs — function call graphs, class inheritance hierarchies, dependency graphs.
+</Tip>
+
+<Warning>
+  **Always pass the `ParsedDocument` to `TextSplitter` before extraction.** Raw `parsed.text` is a flat string. Use `TextSplitter` to chunk it into semantically meaningful pieces before running NER — this dramatically reduces context window overflow on large documents.
+</Warning>
+
 <CardGroup cols={2}>
   <Card title="Ingest" icon="database" href="ingest">
     Load files before parsing.

docs/reference/pipeline.md

@@ -6,34 +6,82 @@ icon: "gear"
 
 `semantica.pipeline` lets you chain Semantica components into reproducible, fault-tolerant workflows with parallel execution and configurable error handling. Pipelines are serializable — save them to YAML and reload in any environment.
 
-## What You Get
+## Why Use a Pipeline?
 
-- **`Pipeline`** — chain steps with parallel workers, retry policies, and failure handlers
-- **`PipelineBuilder`** — fluent DSL for building pipelines with a readable chain syntax
-- **`RetryPolicy`** — fixed, linear, and exponential backoff with configurable max retries
-- **`FailureHandler`** — skip, stop, or retry failed documents without halting the pipeline
-- **Progress tracking** — console (tqdm), WebSocket streaming, or file logging
+You could wire Semantica modules together with plain Python code. Pipelines add:
+
+<CardGroup cols={2}>
+  <Card title="Retry and failure handling" icon="arrow-rotate-right">
+    A single bad document doesn't crash a 10,000-document run.
+  </Card>
+  <Card title="Parallelism" icon="bolt">
+    Run extraction across multiple workers with one parameter.
+  </Card>
+  <Card title="Progress tracking" icon="chart-line">
+    tqdm console bar or WebSocket streaming to Explorer.
+  </Card>
+  <Card title="Reproducibility" icon="floppy-disk">
+    Save the exact pipeline configuration to YAML and replay on any machine.
+  </Card>
+  <Card title="Delta mode" icon="code-compare">
+    On re-runs, only process documents that changed since the last run.
+  </Card>
+  <Card title="Validation" icon="shield-check">
+    Catch misconfigured steps and dependency cycles before they fail mid-run.
+  </Card>
+</CardGroup>
+
+<Note>
+  Use plain module calls for quick scripts and notebooks. Use pipelines for anything you run repeatedly, at scale, or in production.
+</Note>
 
 <img src="/assets/img/diagrams/pipeline-flow.svg" alt="Pipeline step sequence: Ingest → Parse → Normalize → Extract → Build KG → QA → Store → Deliver" style={{ width: '100%', borderRadius: '10px', margin: '0 0 24px' }} />
 
-## Basic Pipeline
+## Quick Start
 
-```python
-from semantica.pipeline import Pipeline
-from semantica.ingest import FileIngestor
-from semantica.parse import DocumentParser
-from semantica.semantic_extract import NERExtractor
-from semantica.kg import GraphBuilder
+<Steps>
+  <Step title="Create a pipeline and add steps">
+    ```python
+    from semantica.pipeline import Pipeline
+    from semantica.ingest import FileIngestor
+    from semantica.parse import DocumentParser
+    from semantica.semantic_extract import NERExtractor
+    from semantica.kg import GraphBuilder
+    from semantica.llms import Groq
+    import os
 
-pipeline = Pipeline()
-pipeline.add_step("ingest",   FileIngestor())
-pipeline.add_step("parse",    DocumentParser())
-pipeline.add_step("extract",  NERExtractor(method="llm", llm_provider=llm))
-pipeline.add_step("build_kg", GraphBuilder(merge_entities=True))
+    llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
 
-result = pipeline.run("data/")
-kg = result.output
-```
+    pipeline = Pipeline()
+    pipeline.add_step("ingest",   FileIngestor())
+    pipeline.add_step("parse",    DocumentParser())
+    pipeline.add_step("extract",  NERExtractor(method="llm", llm_provider=llm))
+    pipeline.add_step("build_kg", GraphBuilder(merge_entities=True))
+    ```
+  </Step>
+  <Step title="Validate before running">
+    ```python
+    from semantica.pipeline import PipelineValidator
+
+    validator = PipelineValidator()
+    result = validator.validate_pipeline(pipeline)
+
+    if not result.is_valid:
+        for error in result.errors:
+            print(f"Error: {error.message}  (step: {error.step})")
+    ```
+  </Step>
+  <Step title="Run and inspect results">
+    ```python
+    result = pipeline.run("data/", show_progress=True)
+
+    kg = result.output
+    print(f"Processed: {result.processed_count}")
+    print(f"Failed:    {result.failed_count}")
+    print(f"Duration:  {result.duration_seconds:.1f}s")
+    ```
+  </Step>
+</Steps>
 
 ## Parallel Processing
 
@@ -52,43 +100,82 @@ result = pipeline.run("data/")
 
 ## Retry and Error Handling
 
-Configure retry behavior and failure strategy independently:
+<Tabs>
+  <Tab title="Exponential backoff (recommended)">
+    ```python
+    from semantica.pipeline import RetryPolicy, FailureHandler, Pipeline
 
-```python
-from semantica.pipeline import Pipeline, RetryPolicy, FailureHandler
+    retry = RetryPolicy(
+        max_retries=3,
+        backoff="exponential",
+        initial_delay=1.0    # 1s → 2s → 4s
+    )
 
-retry = RetryPolicy(
-    max_retries=3,
-    backoff="exponential",   # "fixed" | "linear" | "exponential"
-    initial_delay=1.0        # seconds before first retry
-)
+    handler = FailureHandler(strategy="skip", log_failures=True)
 
-handler = FailureHandler(
-    strategy="skip",         # "skip" | "stop" | "retry"
-    log_failures=True        # write failed documents to error log
-)
+    pipeline = Pipeline(retry_policy=retry, failure_handler=handler)
+    ```
 
-pipeline = Pipeline(retry_policy=retry, failure_handler=handler)
-```
+    Best for transient API errors and rate limits — waits longer with each retry, giving upstream services time to recover.
+  </Tab>
+  <Tab title="Linear backoff">
+    ```python
+    retry = RetryPolicy(
+        max_retries=3,
+        backoff="linear",
+        initial_delay=2.0    # 2s → 4s → 6s
+    )
+    ```
+
+    Use when the delay between retries should grow predictably — e.g., waiting for a database lock to release.
+  </Tab>
+  <Tab title="Fixed backoff">
+    ```python
+    retry = RetryPolicy(
+        max_retries=5,
+        backoff="fixed",
+        initial_delay=1.0    # 1s → 1s → 1s → 1s → 1s
+    )
+    ```
+
+    Use when retrying against a service with a fixed cooldown window.
+  </Tab>
+</Tabs>
+
+### Failure Strategies
+
+| Strategy | Behaviour | When to Use |
+| -------- | --------- | ----------- |
+| `"skip"` | Log failure, continue to next document | Production — one bad doc shouldn't stop 10k |
+| `"stop"` | Raise exception immediately | Development — surface errors fast |
+| `"retry"` | Retry via `RetryPolicy`, then skip | When failures are likely transient |
+
+<Warning>
+  Always use `strategy="skip"` in production. A single malformed document shouldn't stop a pipeline processing thousands of documents. Inspect `result.errors` after the run to find and reprocess failures.
+</Warning>
 
 ## Progress Tracking
 
-```python
-# Console progress bar (tqdm)
-result = pipeline.run("data/", show_progress=True)
+<Tabs>
+  <Tab title="Console (tqdm)">
+    ```python
+    result = pipeline.run("data/", show_progress=True)
+    ```
 
-# WebSocket progress — stream to Knowledge Explorer
-result = pipeline.run("data/", websocket_port=8080)
+    Displays a live tqdm progress bar in the terminal. Best for scripts and CLI tools.
+  </Tab>
+  <Tab title="WebSocket (Explorer)">
+    ```python
+    result = pipeline.run("data/", websocket_port=8080)
+    ```
 
-# Inspect results
-print(f"Processed: {result.processed_count}")
-print(f"Failed:    {result.failed_count}")
-print(f"Duration:  {result.duration_seconds:.1f}s")
-```
+    Streams progress events to Knowledge Explorer's dashboard. Best for long-running production jobs where you want a live web UI.
+  </Tab>
+</Tabs>
 
 ## Pipeline DSL
 
-The `PipelineBuilder` provides a fluent chain syntax that reads as a data flow:
+`PipelineBuilder` provides a fluent chain syntax that reads as a data flow:
 
 ```python
 from semantica.pipeline import PipelineBuilder
@@ -122,7 +209,206 @@ pipeline = Pipeline.load("pipeline_config.yaml")
 result = pipeline.run("data/")
 ```
 
-## Pipeline Result
+<Tip>
+  `pipeline.save()` preserves exact component configurations — LLM model names, retry policies, thresholds — everything. Without it, you can't guarantee that a re-run 3 months later uses the same settings.
+</Tip>
+
+## Pre-Built Templates
+
+`PipelineTemplateManager` wires common workflows with the correct step order — no manual wiring required:
+
+```python
+from semantica.pipeline import PipelineTemplateManager
+from semantica.llms import Groq
+import os
+
+llm     = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+manager = PipelineTemplateManager()
+```
+
+<CardGroup cols={2}>
+  <Card title="ingest-extract-build" icon="diagram-project">
+    **Ingest → Parse → Extract → Build KG**
+
+    Standard knowledge base construction from documents.
+
+    ```python
+    pipeline = manager.get_template(
+        "ingest-extract-build", llm_provider=llm
+    )
+    ```
+  </Card>
+  <Card title="graphrag" icon="magnifying-glass">
+    **Ingest → Parse → Embed → Index**
+
+    Retrieval-augmented generation — builds a vector-indexed knowledge graph.
+
+    ```python
+    pipeline = manager.get_template(
+        "graphrag", llm_provider=llm, vector_backend="faiss"
+    )
+    ```
+  </Card>
+  <Card title="analytics" icon="chart-bar">
+    **Build KG → Analytics → Export Report**
+
+    Graph analysis and reporting — centrality, community detection, HTML output.
+
+    ```python
+    pipeline = manager.get_template(
+        "analytics", export_format="html"
+    )
+    ```
+  </Card>
+  <Card title="full-qa" icon="shield-check">
+    **Ingest → Normalize → Extract → Dedup → Conflicts → Build**
+
+    Production-quality KG with full data quality pipeline.
+
+    ```python
+    pipeline = manager.get_template(
+        "full-qa", llm_provider=llm
+    )
+    ```
+  </Card>
+</CardGroup>
+
+## ExecutionEngine
+
+Fine-grained control over pipeline execution — pause, resume, cancel, and inspect live progress:
+
+```python
+from semantica.pipeline import ExecutionEngine
+
+engine = ExecutionEngine(config={"timeout_seconds": 300, "max_workers": 4})
+
+result = engine.execute_pipeline(pipeline, data="data/")
+
+# Pause after the current step finishes
+engine.pause_pipeline(result.pipeline_id)
+
+progress = engine.get_progress(result.pipeline_id)
+print(f"Completed: {progress['completed']}/{progress['total']}")
+print(f"Current step: {progress['current_step']}")
+
+engine.resume_pipeline(result.pipeline_id)
+engine.stop_pipeline(result.pipeline_id)
+```
+
+| Method | Returns | Description |
+| ------ | ------- | ----------- |
+| `execute_pipeline(pipeline, data)` | `ExecutionResult` | Execute pipeline from start to finish |
+| `get_status(pipeline_id)` | `PipelineStatus` | Current state (RUNNING, PAUSED, STOPPED) |
+| `get_progress(pipeline_id)` | `Dict` | Step completion counts and elapsed time |
+| `pause_pipeline(pipeline_id)` | `None` | Suspend after current step completes |
+| `resume_pipeline(pipeline_id)` | `None` | Resume from paused state |
+| `stop_pipeline(pipeline_id)` | `None` | Cancel and clean up immediately |
+
+## PipelineValidator
+
+Catches problems before they surface as mid-run failures:
+
+```python
+from semantica.pipeline import PipelineValidator
+
+validator = PipelineValidator()
+result    = validator.validate_pipeline(pipeline)
+
+if result.is_valid:
+    print("Pipeline is valid — safe to run")
+else:
+    for error in result.errors:
+        print(f"Error: {error.message}  (step: {error.step})")
+    for warning in result.warnings:
+        print(f"Warning: {warning}")
+```
+
+Checks performed:
+- **Dependency cycle detection** — A depends on B, B depends on A
+- **Step type validation** — each step type must be registered
+- **Connection integrity** — referenced step names must exist
+- **Configuration completeness** — required parameters must be present
+
+## ParallelismManager
+
+<Tabs>
+  <Tab title="Thread pool (I/O-bound)">
+    ```python
+    from semantica.pipeline import ParallelismManager
+
+    manager = ParallelismManager(max_workers=8, pool_type="thread")
+
+    tasks  = [{"fn": ner.extract, "args": [text]} for text in texts]
+    result = manager.execute_parallel(tasks, timeout=60)
+
+    print(f"Successful: {result.success_count}, Failed: {result.failure_count}")
+    ```
+
+    Use thread pools for **I/O-bound** steps: web fetching, database queries, API calls. Threads share memory and context-switch cheaply between waiting operations.
+  </Tab>
+  <Tab title="Process pool (CPU-bound)">
+    ```python
+    manager = ParallelismManager(max_workers=4, pool_type="process")
+
+    tasks  = [{"fn": embedder.embed, "args": [chunk]} for chunk in chunks]
+    result = manager.execute_parallel(tasks, timeout=120)
+    ```
+
+    Use process pools for **CPU-bound** steps: embedding computation, OCR, large NER batches. Processes bypass Python's GIL for true multi-core parallelism.
+  </Tab>
+</Tabs>
+
+## ResourceScheduler
+
+Prevents memory oversubscription on large runs:
+
+```python
+from semantica.pipeline import ResourceScheduler
+
+scheduler = ResourceScheduler()
+
+resources = scheduler.allocate_resources(
+    pipeline, max_memory_gb=8, max_workers=4
+)
+
+try:
+    result = pipeline.run("data/")
+finally:
+    scheduler.release_resources(resources)
+```
+
+## Delta Mode
+
+Re-process only data that has changed since the last run:
+
+```python
+pipeline = Pipeline()
+pipeline.add_step(
+    "ingest",  FileIngestor(),
+    delta_mode=True, base_version_id="v1", target_version_id="v2"
+)
+pipeline.add_step(
+    "extract", NERExtractor(),
+    delta_mode=True, base_version_id="v1", target_version_id="v2"
+)
+pipeline.add_step(
+    "build", GraphBuilder(),
+    delta_mode=False  # always rebuild the merged graph
+)
+
+result = pipeline.run("data/")
+print(f"Delta documents processed: {result.metadata.get('delta_count', 0)}")
+print(f"Skipped (unchanged):       {result.metadata.get('skipped_count', 0)}")
+```
+
+<Note>
+  Delta detection uses SHA-256 checksums on source content. Only sources whose checksum differs from `base_version_id` are passed to downstream steps. For pipelines that run hourly or daily against a growing corpus, delta mode eliminates redundant re-embedding and re-extraction.
+</Note>
+
+## Schemas
+
+<AccordionGroup>
+  <Accordion title="PipelineResult schema">
 
 ```python
 @dataclass
@@ -133,8 +419,66 @@ class PipelineResult:
     duration_seconds: float    # total wall-clock time
     step_metrics:     Dict     # per-step timing and counts
     errors:           List     # list of FailedDocument records
+    metadata:         Dict     # pipeline-level metadata (delta_count, etc.)
 ```
 
+  </Accordion>
+  <Accordion title="PipelineStep schema">
+
+```python
+@dataclass
+class PipelineStep:
+    name:              str
+    step_type:         str
+    config:            Dict[str, Any]
+    dependencies:      List[str]          # names of steps this step waits for
+    handler:           Optional[Callable]
+    status:            StepStatus
+    result:            Any
+    error:             Optional[Exception]
+    delta_mode:        bool               # True = process only changed data
+    base_version_id:   Optional[str]     # snapshot ID to diff against
+    target_version_id: Optional[str]     # snapshot ID being produced
+```
+
+  </Accordion>
+  <Accordion title="StepStatus enum">
+
+```python
+from semantica.pipeline import StepStatus
+
+StepStatus.PENDING    # Not yet started
+StepStatus.RUNNING    # Currently executing
+StepStatus.COMPLETED  # Finished successfully
+StepStatus.FAILED     # Error occurred — check step.error
+StepStatus.SKIPPED    # Skipped due to FailureHandler "skip" strategy
+```
+
+  </Accordion>
+</AccordionGroup>
+
+## Tips and Common Pitfalls
+
+<Tip>
+  **Use `PipelineValidator` before running in production.** It catches dependency cycles, missing step names, and misconfigured connections that would only surface as errors mid-run. Validation is instant; catching them after a 30-minute extraction job is not.
+</Tip>
+
+<Tip>
+  **Set `workers=` based on workload type.** Thread workers for I/O-bound steps (web fetching, DB queries), process workers for CPU-bound steps (embedding, OCR, large NER batches). Mixing pool types on the wrong step type wastes resources without speed gains.
+</Tip>
+
+<Warning>
+  **Use `failure_handler=FailureHandler(strategy="skip")` in production.** A single malformed document shouldn't stop a pipeline processing 10,000 documents. `skip` logs the failure and continues; inspect `result.errors` after the run to find and reprocess failed documents.
+</Warning>
+
+<Tip>
+  **Use templates from `PipelineTemplateManager` for common patterns.** `get_template("full-qa")` wires up normalization, deduplication, conflict detection, and graph construction in the right order — saving you from common mistakes like deduplicating before normalizing.
+</Tip>
+
+<Tip>
+  **Inspect `result.step_metrics` to find bottlenecks.** Each step reports its own duration and document count. If embedding is 10x slower than NER, that's where to optimize — increase `batch_size`, switch to a faster embedding model, or parallelize with GPU.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Ingest" icon="database" href="ingest">
     First step in most pipelines.

docs/reference/provenance.md

@@ -8,11 +8,75 @@ icon: "link"
 
 ## What You Get
 
-- **`ProvenanceManager`** — track entities, relationships, and activities with source attribution
-- **`ActivityTracker`** — record pipeline activities and which entities they produced or consumed
-- **Lineage graph** — full upstream lineage from any entity back to its source document
-- **W3C PROV-O export** — serialize lineage as Turtle RDF for compliance reporting
-- **`GraphBuilderWithProvenance`** — drop-in replacement that auto-tracks every node and edge
+<CardGroup cols={2}>
+  <Card title="ProvenanceManager" icon="link">
+    Track entities, relationships, and activities with full source attribution and confidence scores.
+  </Card>
+  <Card title="ActivityTracker" icon="clock">
+    Record pipeline activities and which entities they produced or consumed.
+  </Card>
+  <Card title="Lineage Graph" icon="diagram-project">
+    Full directed lineage from any entity back to its originating source document.
+  </Card>
+  <Card title="W3C PROV-O Export" icon="file-export">
+    Serialize lineage as Turtle RDF or JSON-LD for compliance reporting.
+  </Card>
+  <Card title="GraphBuilderWithProvenance" icon="hammer">
+    Drop-in replacement for GraphBuilder that auto-tracks every node and edge.
+  </Card>
+  <Card title="Integrity Verification" icon="shield-check">
+    SHA-256 checksums to detect tampering in HIPAA and FDA 21 CFR Part 11 environments.
+  </Card>
+</CardGroup>
+
+## Quick Start
+
+<Steps>
+  <Step title="Initialize ProvenanceManager with a storage backend">
+    ```python
+    from semantica.provenance import ProvenanceManager, SQLiteStorage
+
+    # SQLite — persistent across process restarts (recommended for production)
+    manager = ProvenanceManager(
+        storage=SQLiteStorage(db_path="provenance.db")
+    )
+    ```
+  </Step>
+  <Step title="Track entities and relationships at ingestion time">
+    ```python
+    manager.track_entity(
+        entity_id="apple_inc",
+        source="annual_report_2023.pdf",
+        entity_type="Organization",
+        extraction_method="llm",
+        confidence=0.98,
+    )
+
+    manager.track_relationship(
+        rel_id="steve_jobs_founded_apple",
+        source="annual_report_2023.pdf",
+        extraction_method="llm",
+        confidence=0.92,
+    )
+    ```
+  </Step>
+  <Step title="Query lineage for any entity">
+    ```python
+    lineage = manager.get_lineage("apple_inc")
+    print(f"Source:     {lineage.source}")
+    print(f"Extracted:  {lineage.extracted_at}")
+    print(f"Method:     {lineage.extraction_method}")
+    print(f"Confidence: {lineage.confidence}")
+    ```
+  </Step>
+  <Step title="Export PROV-O for compliance reporting">
+    ```python
+    # Full provenance graph for all tracked entities
+    manager.export_all(path="provenance.ttl",    format="turtle")
+    manager.export_all(path="provenance.jsonld", format="json-ld")
+    ```
+  </Step>
+</Steps>
 
 ## ProvenanceManager
 
@@ -27,7 +91,7 @@ manager.track_entity(
     source="annual_report_2023.pdf",
     entity_type="Organization",
     extraction_method="llm",
-    confidence=0.98
+    confidence=0.98,
 )
 
 # Track an extracted relationship
@@ -35,7 +99,7 @@ manager.track_relationship(
     rel_id="steve_jobs_founded_apple",
     source="annual_report_2023.pdf",
     extraction_method="llm",
-    confidence=0.92
+    confidence=0.92,
 )
 
 # Retrieve full lineage for any entity
@@ -55,7 +119,7 @@ Record pipeline activities — what was consumed and what was produced:
 activity_id = manager.start_activity(
     activity_type="ner_extraction",
     used=["annual_report_2023.pdf"],
-    generated=["apple_inc", "steve_jobs"]
+    generated=["apple_inc", "steve_jobs"],
 )
 
 manager.end_activity(activity_id)
@@ -82,24 +146,52 @@ for edge in lineage_graph.edges:
     print(f"{edge.source} → {edge.target} ({edge.relation})")
 ```
 
-## W3C PROV-O Export
+## Storage Backends
 
-Export lineage as W3C PROV-O Turtle for compliance reporting:
+<Tabs>
+  <Tab title="InMemoryStorage (development)">
+    Fast, no persistence — default backend. Data is lost on process exit.
 
-```python
-# Single entity lineage
-prov_ttl = manager.export_prov_o("apple_inc", format="turtle")
+    ```python
+    from semantica.provenance import ProvenanceManager, InMemoryStorage
 
-# Full provenance graph for all tracked entities
-manager.export_all(path="provenance.ttl", format="turtle")
+    manager = ProvenanceManager(storage=InMemoryStorage())
 
-# Compliance-ready JSON-LD export
-manager.export_all(path="provenance.jsonld", format="json-ld")
-```
+    manager.track_entity("apple_inc", source="report.pdf", confidence=0.98)
+    lineage = manager.get_lineage("apple_inc")
+    ```
+
+    Best for: development, unit tests, short-lived pipelines.
+  </Tab>
+  <Tab title="SQLiteStorage (production)">
+    Persistent file-based storage — survives process restarts. The API is identical to `InMemoryStorage`.
+
+    ```python
+    from semantica.provenance import ProvenanceManager, SQLiteStorage
+
+    manager = ProvenanceManager(
+        storage=SQLiteStorage(db_path="provenance.db")
+    )
+
+    manager.track_entity("apple_inc", source="report.pdf", confidence=0.98)
+    lineage = manager.get_lineage("apple_inc")
+    ```
+
+    Best for: production single-machine deployments, compliance environments.
+  </Tab>
+  <Tab title="Comparison">
+
+    | Storage | Persistence | Best For |
+    | ------- | ----------- | -------- |
+    | `InMemoryStorage` | No | Development, unit tests, short-lived pipelines |
+    | `SQLiteStorage` | Yes (file) | Production single-machine deployments |
+
+  </Tab>
+</Tabs>
 
 ## Integration with GraphBuilder
 
-`GraphBuilderWithProvenance` automatically records provenance for every node and edge constructed:
+`GraphBuilderWithProvenance` automatically records provenance for every node and edge constructed — no manual `track_entity()` calls needed:
 
 ```python
 from semantica.kg import GraphBuilderWithProvenance
@@ -107,15 +199,92 @@ from semantica.kg import GraphBuilderWithProvenance
 builder = GraphBuilderWithProvenance(provenance=True)
 result  = builder.build_single_source(graph_data)
 
-# Each node and edge has a source_id linking back to the originating document
+# Every node and edge has a source_id linking back to the originating document
 lineage = result.provenance_manager.get_lineage("apple_inc")
 print(f"Source document: {lineage.source}")
 print(f"Extracted by:    {lineage.extraction_method}")
 ```
 
-## Compliance Standards
+## Integrity Verification
 
-Provenance tracking in Semantica is designed to satisfy:
+Compute and verify checksums for provenance entries to detect tampering:
+
+```python
+from semantica.provenance import compute_checksum, verify_checksum
+
+# Compute a SHA-256 checksum over an entity's provenance record
+entry    = manager.get_provenance_entry("apple_inc")
+checksum = compute_checksum(entry, algorithm="sha256")
+print(f"Checksum: {checksum}")
+
+# Later — verify the record has not been modified
+is_valid = verify_checksum(entry, expected_checksum=checksum, algorithm="sha256")
+if not is_valid:
+    raise RuntimeError("Provenance record has been tampered with!")
+```
+
+Supported algorithms: `"sha256"` (default), `"sha512"`, `"md5"`.
+
+## W3C PROV-O Export
+
+```python
+# Single entity lineage
+prov_ttl = manager.export_prov_o("apple_inc", format="turtle")
+
+# Full provenance graph for all tracked entities
+manager.export_all(path="provenance.ttl",    format="turtle")
+manager.export_all(path="provenance.jsonld", format="json-ld")
+```
+
+## Schemas
+
+<AccordionGroup>
+  <Accordion title="ProvenanceEntry schema">
+
+```python
+@dataclass
+class ProvenanceEntry:
+    entity_id:         str
+    source:            str            # source document or system
+    source_location:   str            # e.g. "page 3, paragraph 2"
+    source_quote:      str            # verbatim text from source
+    extraction_method: str            # "llm" | "ml" | "pattern"
+    confidence:        float          # extraction confidence 0–1
+    timestamp:         datetime       # when this entry was recorded
+    entity_type:       Optional[str]
+```
+
+  </Accordion>
+  <Accordion title="SourceReference schema">
+
+```python
+@dataclass
+class SourceReference:
+    source_id:   str
+    doi:         Optional[str]      # academic DOI if available
+    page:        Optional[int]      # page number in document
+    paragraph:   Optional[int]
+    quote:       str                # verbatim text supporting the fact
+    url:         Optional[str]
+    accessed_at: Optional[datetime]
+```
+
+  </Accordion>
+</AccordionGroup>
+
+## W3C PROV-O Mapping
+
+| Semantica Concept | PROV-O Class / Property |
+| ----------------- | ----------------------- |
+| Entity (node/fact) | `prov:Entity` |
+| Extraction activity | `prov:Activity` |
+| Source document | `prov:Entity` |
+| `track_entity()` | `prov:wasDerivedFrom` |
+| `start_activity()` | `prov:wasGeneratedBy` |
+| Extraction method | `prov:wasAssociatedWith` |
+| Timestamp | `prov:startedAtTime`, `prov:endedAtTime` |
+
+## Compliance Standards
 
 | Standard | Requirement Met |
 | -------- | --------------- |
@@ -125,6 +294,32 @@ Provenance tracking in Semantica is designed to satisfy:
 | **GDPR** | Data lineage supporting right-to-erasure impact analysis |
 | **FDA 21 CFR Part 11** | Electronic records with origination timestamp and extraction method |
 
+## Tips and Common Pitfalls
+
+<Warning>
+  **Use `SQLiteStorage` in production, not `InMemoryStorage`.** The in-memory backend is the default for backwards compatibility, but provenance data is lost on process exit. Switch to `SQLiteStorage(db_path="provenance.db")` before going to production — migrating later means losing all historical lineage.
+</Warning>
+
+<Warning>
+  **Track provenance at ingestion time, not after.** The `source_location` and `source_quote` fields become unavailable once you've moved past the parsing stage. Capture them during ingestion and pass them to `track_entity()` immediately.
+</Warning>
+
+<Tip>
+  **Use `GraphBuilderWithProvenance` instead of plain `GraphBuilder`.** It auto-tracks every node and edge without manual `track_entity()` calls — ensuring nothing is accidentally omitted from the provenance record.
+</Tip>
+
+<Tip>
+  **Verify checksums on high-stakes data.** `compute_checksum()` + `verify_checksum()` detects any modification to a provenance entry since it was recorded — critical for HIPAA and FDA 21 CFR Part 11 environments where tampered records carry legal liability.
+</Tip>
+
+<Tip>
+  **Export PROV-O Turtle for external auditors.** Compliance teams and external auditors often need machine-readable lineage in a standard format. `manager.export_all("provenance.ttl", format="turtle")` produces W3C PROV-O Turtle that any RDF tool can parse.
+</Tip>
+
+<Tip>
+  **Use `get_audit_trail(entity_id=...)` for GDPR subject-access requests.** The GDPR right-of-access requires you to show what personal data you hold and where it came from. Scoped lineage per entity ID makes this a one-line export.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Change Management" icon="clock-rotate-left" href="change_management">
     Version control and snapshot audit trails.

docs/reference/reasoning.md

@@ -6,196 +6,274 @@ icon: "microchip"
 
 `semantica.reasoning` derives new knowledge from existing facts using logical rules. Every engine produces **explainable inference paths** — traceable chains of rules and facts, not black-box conclusions.
 
+## Why Reasoning?
+
+Knowledge graphs encode what you know explicitly. Reasoning lets you derive what must logically follow — without manually asserting every implication:
+
+- If A `located_in` B and B `located_in` C, then A `located_in` C — without storing that triple
+- If Alice `parent_of` Bob and Bob `parent_of` Charlie, then Alice `ancestor_of` Charlie
+- If a drug is contraindicated for a condition class, it's also contraindicated for all subclasses — inferred from the ontology hierarchy
+- If an employee's CEO tenure ended in 2020, they cannot have signed contracts as CEO in 2021 — caught by temporal reasoning
+
+Reasoning turns sparse explicit knowledge into a dense, coherent, contradiction-free knowledge base.
+
 ## What You Get
 
-- **`Reasoner`** — main facade for IF/THEN forward-chaining with variable substitution
-- **`GraphReasoner`** — inference over full knowledge graph structure (transitivity, symmetry, inverses)
-- **`ReteEngine`** — high-performance pattern matching via the Rete algorithm for large rule sets
-- **`SPARQLReasoner`** — query expansion and property chain inference over RDF graphs
-- **`DatalogReasoner`** — recursive Horn clause rules with guaranteed fixpoint termination (v0.4.0)
-- **`TemporalReasoningEngine`** — all 13 Allen interval algebra relations for time-aware inference
-- **`ExplanationGenerator`** — structured explanation paths for every derived conclusion
+<CardGroup cols={2}>
+  <Card title="Reasoner" icon="bolt">
+    Main facade — IF/THEN forward-chaining with variable substitution and rule templates.
+  </Card>
+  <Card title="GraphReasoner" icon="diagram-project">
+    Inference over full knowledge graph structure: transitivity, symmetry, inverses.
+  </Card>
+  <Card title="ReteEngine" icon="gauge-high">
+    High-performance pattern matching via the Rete algorithm for large rule sets.
+  </Card>
+  <Card title="SPARQLReasoner" icon="table">
+    Query expansion and property chain inference over RDF graphs.
+  </Card>
+  <Card title="DatalogReasoner" icon="rotate">
+    Recursive Horn clause rules with guaranteed fixpoint termination (v0.4.0).
+  </Card>
+  <Card title="TemporalReasoningEngine" icon="clock">
+    All 13 Allen interval algebra relations for time-aware inference.
+  </Card>
+</CardGroup>
 
 <img src="/assets/img/diagrams/reasoning-chain.svg" alt="Forward chaining inference: known facts + IF/THEN rules produce derived facts with a full traceable explanation path" style={{ width: '100%', borderRadius: '12px', margin: '0 0 24px' }} />
 
-## Reasoner (Main Facade)
+## Choosing a Reasoning Engine
 
-The unified entry point for rule-based forward-chaining inference:
+| Engine | When to Use |
+| ------ | ----------- |
+| `Reasoner` | Simple IF/THEN rules, transitivity/symmetry templates, one-shot inference |
+| `GraphReasoner` | Rules that operate on graph structure (paths, neighborhoods, multi-hop) |
+| `ReteEngine` | Large rule sets (100+), rules fire repeatedly, performance is critical |
+| `SPARQLReasoner` | Already using RDF/Turtle, need property chains, SPARQL ecosystem tools |
+| `DatalogReasoner` | Recursive rules (ancestry, reachability), guaranteed termination required |
+| `TemporalReasoningEngine` | Time-aware facts, interval relationships, historical validity |
 
-```python
-from semantica.reasoning import Reasoner, Rule, Fact, RuleType
+## Engines
 
-reasoner = Reasoner()
+<Tabs>
+  <Tab title="Reasoner">
+    The unified entry point for rule-based forward-chaining inference. Start here for most use cases.
 
-# Add base facts
-reasoner.add_fact(Fact(subject="John", predicate="is_a", obj="Manager"))
-reasoner.add_fact(Fact(subject="John", predicate="is_a", obj="Employee"))
+    ```python
+    from semantica.reasoning import Reasoner, Rule, Fact, RuleType
 
-# Add an IF/THEN rule
-reasoner.add_rule(Rule(
-    rule_type=RuleType.FORWARD_CHAIN,
-    conditions=[
-        {"subject": "?x", "predicate": "is_a", "object": "Manager"}
-    ],
-    conclusion={"subject": "?x", "predicate": "has_authority", "object": "true"}
-))
+    reasoner = Reasoner()
 
-# Run inference
-result = reasoner.infer()
-for inference in result.derived_facts:
-    print(f"{inference.subject} {inference.predicate} {inference.obj}")
-    print(f"  Derived via: {inference.explanation}")
-```
+    # Add base facts
+    reasoner.add_fact(Fact(subject="John", predicate="is_a", obj="Manager"))
+    reasoner.add_fact(Fact(subject="John", predicate="is_a", obj="Employee"))
 
-### Built-In Rule Templates
+    # Add an IF/THEN rule
+    reasoner.add_rule(Rule(
+        rule_type=RuleType.FORWARD_CHAIN,
+        conditions=[
+            {"subject": "?x", "predicate": "is_a", "object": "Manager"}
+        ],
+        conclusion={"subject": "?x", "predicate": "has_authority", "object": "true"}
+    ))
 
-```python
-engine = Reasoner()
+    # Run inference — always call explicitly after adding facts/rules
+    result = reasoner.infer()
+    for inference in result.derived_facts:
+        print(f"{inference.subject} {inference.predicate} {inference.obj}")
+        print(f"  Derived via: {inference.explanation}")
+    ```
 
-# Transitive closure: A→B, B→C ⟹ A→C
-engine.apply_transitivity("located_in")
+    ### Built-In Rule Templates
 
-# Symmetry: A knows B ⟹ B knows A
-engine.apply_symmetry("knows")
+    No manual rule authoring required for the three most common patterns:
 
-# Inverse: A parent_of B ⟹ B child_of A
-engine.apply_inverse("parent_of", "child_of")
-```
+    ```python
+    engine = Reasoner()
 
-## GraphReasoner
+    # Transitive closure: A→B, B→C ⟹ A→C
+    engine.apply_transitivity("located_in")
 
-Inference over the full knowledge graph structure:
+    # Symmetry: A knows B ⟹ B knows A
+    engine.apply_symmetry("knows")
 
-```python
-from semantica.reasoning import GraphReasoner
+    # Inverse: A parent_of B ⟹ B child_of A
+    engine.apply_inverse("parent_of", "child_of")
 
-graph_reasoner = GraphReasoner(kg)
+    result = engine.infer()
+    ```
 
-# Define a transitive ancestor rule
-graph_reasoner.add_rule({
-    "if": [
-        {"subject": "?a", "predicate": "parent_of", "object": "?b"},
-        {"subject": "?b", "predicate": "parent_of", "object": "?c"}
-    ],
-    "then": {"subject": "?a", "predicate": "ancestor_of", "object": "?c"}
-})
+    | Template | Parameters | Description |
+    | -------- | ---------- | ----------- |
+    | `apply_transitivity(predicate)` | `predicate: str` | Adds A→C rule for all A→B, B→C chains |
+    | `apply_symmetry(predicate)` | `predicate: str` | Adds B→A rule for every A→B fact |
+    | `apply_inverse(predicate, inverse)` | `predicate, inverse: str` | Adds inverse direction for every fact |
 
-inferences = graph_reasoner.infer(kg)
-for inf in inferences:
-    print(f"{inf['subject']} {inf['predicate']} {inf['object']}")
-```
+    <Warning>
+      Always call `reasoner.infer()` after adding facts and rules. Adding them updates internal state but does **not** trigger inference automatically.
+    </Warning>
+  </Tab>
 
-## ReteEngine
+  <Tab title="GraphReasoner">
+    Inference over the full knowledge graph structure — rules that operate on graph paths, neighborhoods, and multi-hop connections.
 
-High-performance pattern matching using the Rete algorithm — far faster than naive forward chaining for large rule sets because it caches partial matches across iterations:
+    ```python
+    from semantica.reasoning import GraphReasoner
 
-```python
-from semantica.reasoning import ReteEngine
+    graph_reasoner = GraphReasoner(kg)
 
-engine = ReteEngine()
-engine.load_rules("rules/domain_rules.json")
-results = engine.run(kg)
+    # Define a transitive ancestor rule
+    graph_reasoner.add_rule({
+        "if": [
+            {"subject": "?a", "predicate": "parent_of", "object": "?b"},
+            {"subject": "?b", "predicate": "parent_of", "object": "?c"}
+        ],
+        "then": {"subject": "?a", "predicate": "ancestor_of", "object": "?c"}
+    })
 
-# Inspect the Rete network
-root        = engine.get_root()
-alpha_nodes = engine.get_alpha_nodes()   # single-condition filters
-beta_nodes  = engine.get_beta_nodes()    # join nodes
-```
+    inferences = graph_reasoner.infer(kg)
+    for inf in inferences:
+        print(f"{inf['subject']} {inf['predicate']} {inf['object']}")
+    ```
+  </Tab>
 
-Rule format (JSON):
+  <Tab title="ReteEngine">
+    High-performance pattern matching using the Rete algorithm — far faster than naive forward chaining for large rule sets because it caches partial matches across iterations.
 
-```json
-{
-  "rules": [
+    ```python
+    from semantica.reasoning import ReteEngine
+
+    engine = ReteEngine()
+    engine.load_rules("rules/domain_rules.json")
+    results = engine.run(kg)
+
+    # Inspect the Rete network
+    root        = engine.get_root()
+    alpha_nodes = engine.get_alpha_nodes()   # single-condition filters
+    beta_nodes  = engine.get_beta_nodes()    # join nodes
+    ```
+
+    Rule file format (JSON):
+
+    ```json
     {
-      "name": "manager_authority",
-      "conditions": [
-        { "subject": "?x", "predicate": "role", "object": "Manager" }
-      ],
-      "action": { "subject": "?x", "predicate": "has_authority", "object": "true" }
+      "rules": [
+        {
+          "name": "manager_authority",
+          "conditions": [
+            { "subject": "?x", "predicate": "role",  "object": "Manager" },
+            { "subject": "?x", "predicate": "dept",  "object": "?dept"   }
+          ],
+          "action": {
+            "subject": "?x",
+            "predicate": "has_authority_over",
+            "object": "?dept"
+          },
+          "priority": 10
+        }
+      ]
     }
-  ]
-}
-```
+    ```
 
-## SPARQLReasoner
+    | Field | Type | Description |
+    | ----- | ---- | ----------- |
+    | `name` | `str` | Unique rule identifier — appears in `ExplanationGenerator` output |
+    | `conditions` | `List[Dict]` | Pattern to match — use `?variable` for wildcards |
+    | `action` | `Dict` | Fact to derive when all conditions match |
+    | `priority` | `int` | Higher priority rules fire first |
 
-Query-based inference over RDF graphs with property chain support:
+    <Tip>
+      Use `ReteEngine` when you have more than ~20 rules or when rules can fire repeatedly. `Reasoner` re-evaluates all rules from scratch each cycle; `ReteEngine` caches partial matches and is orders of magnitude faster.
+    </Tip>
+  </Tab>
 
-```python
-from semantica.reasoning import SPARQLReasoner
+  <Tab title="SPARQLReasoner">
+    Query-based inference over RDF graphs with property chain support. Use this when you're already in the RDF/Turtle ecosystem.
 
-reasoner = SPARQLReasoner(graph=rdf_graph)
+    ```python
+    from semantica.reasoning import SPARQLReasoner
 
-result = reasoner.query("""
-    PREFIX ex: <http://example.org/>
-    SELECT ?person ?company WHERE {
-        ?person ex:founded ?company .
-        ?company ex:located_in ex:SiliconValley .
-    }
-""")
+    reasoner = SPARQLReasoner(graph=rdf_graph)
 
-for row in result.bindings:
-    print(row["person"], row["company"])
+    result = reasoner.query("""
+        PREFIX ex: <http://example.org/>
+        SELECT ?person ?company WHERE {
+            ?person ex:founded ?company .
+            ?company ex:located_in ex:SiliconValley .
+        }
+    """)
 
-# Property chain inference: A knows B, B colleague_of C ⟹ A knows C
-reasoner.add_property_chain("knows", ["knows", "colleague_of"])
-inferences = reasoner.infer_property_chains()
-```
+    for row in result.bindings:
+        print(row["person"], row["company"])
 
-## DatalogReasoner (v0.4.0)
+    # Property chain inference: A knows B, B colleague_of C ⟹ A knows C
+    reasoner.add_property_chain("knows", ["knows", "colleague_of"])
+    inferences = reasoner.infer_property_chains()
+    ```
+  </Tab>
 
-Pure-Python bottom-up semi-naive fixpoint evaluation for recursive Horn clause rules. Termination is **guaranteed** — the engine detects fixpoint convergence and stops:
+  <Tab title="DatalogReasoner">
+    Pure-Python bottom-up semi-naive fixpoint evaluation for recursive Horn clause rules. Termination is **guaranteed** — the engine detects fixpoint convergence and stops.
 
-```python
-from semantica.reasoning import DatalogReasoner, DatalogFact, DatalogRule
+    <Note>
+      Added in **v0.4.0**. Use `DatalogReasoner` whenever your rules can create cycles — it's the only engine with a termination guarantee.
+    </Note>
 
-datalog = DatalogReasoner()
+    ```python
+    from semantica.reasoning import DatalogReasoner, DatalogFact, DatalogRule
 
-# Base facts
-datalog.add_fact(DatalogFact("parent", ("alice", "bob")))
-datalog.add_fact(DatalogFact("parent", ("bob",   "charlie")))
+    datalog = DatalogReasoner()
 
-# Recursive rules (Horn clauses)
-datalog.add_rule(DatalogRule("ancestor(?X, ?Y) :- parent(?X, ?Y)."))
-datalog.add_rule(DatalogRule("ancestor(?X, ?Z) :- parent(?X, ?Y), ancestor(?Y, ?Z)."))
+    # Base facts
+    datalog.add_fact(DatalogFact("parent", ("alice", "bob")))
+    datalog.add_fact(DatalogFact("parent", ("bob",   "charlie")))
 
-# Evaluate to fixpoint
-datalog.evaluate()
+    # Recursive rules (Horn clauses)
+    datalog.add_rule(DatalogRule("ancestor(?X, ?Y) :- parent(?X, ?Y)."))
+    datalog.add_rule(DatalogRule("ancestor(?X, ?Z) :- parent(?X, ?Y), ancestor(?Y, ?Z)."))
 
-# Query
-results = datalog.query("ancestor(alice, ?Z)")
-# → [{"Z": "bob"}, {"Z": "charlie"}]
-```
+    # Evaluate to fixpoint
+    datalog.evaluate()
 
-## TemporalReasoningEngine
+    # Query
+    results = datalog.query("ancestor(alice, ?Z)")
+    # → [{"Z": "bob"}, {"Z": "charlie"}]
+    ```
 
-Reason about time intervals using all 13 Allen interval algebra relations:
+    <Warning>
+      `Reasoner` has **no cycle detection** — rules that create cycles (A derives B, B derives C, C re-derives A) will loop infinitely. Use `DatalogReasoner` whenever recursive rules are involved.
+    </Warning>
+  </Tab>
 
-```python
-from semantica.reasoning import TemporalReasoningEngine, TemporalInterval, IntervalRelation
+  <Tab title="TemporalReasoningEngine">
+    Reason about time intervals using all 13 Allen interval algebra relations.
 
-engine = TemporalReasoningEngine()
+    ```python
+    from semantica.reasoning import TemporalReasoningEngine, TemporalInterval, IntervalRelation
 
-ceo_tenure  = TemporalInterval(start="1997-09-16", end="2011-08-24")
-board_member = TemporalInterval(start="2000-01-01", end="2012-06-01")
+    engine = TemporalReasoningEngine()
 
-relation = engine.get_relation(ceo_tenure, board_member)
-# → IntervalRelation.DURING  (ceo_tenure is fully inside board_member)
-```
+    ceo_tenure   = TemporalInterval(start="1997-09-16", end="2011-08-24")
+    board_member = TemporalInterval(start="2000-01-01", end="2012-06-01")
 
-All 13 Allen interval algebra relations are supported:
+    relation = engine.get_relation(ceo_tenure, board_member)
+    # → IntervalRelation.DURING  (ceo_tenure is fully inside board_member)
+    ```
 
-| Relation | Meaning |
-| -------- | ------- |
-| `BEFORE` | A ends before B starts |
-| `MEETS` | A ends exactly when B starts |
-| `OVERLAPS` | A starts before B, ends inside B |
-| `DURING` | A is fully inside B |
-| `STARTS` | A and B start together, A ends first |
-| `FINISHES` | A and B end together, A starts later |
-| `EQUALS` | Identical intervals |
-| + 6 inverses | `AFTER`, `MET_BY`, `OVERLAPPED_BY`, `CONTAINS`, `STARTED_BY`, `FINISHED_BY` |
+    All 13 Allen interval algebra relations:
+
+    | Relation | Meaning |
+    | -------- | ------- |
+    | `BEFORE` | A ends before B starts |
+    | `MEETS` | A ends exactly when B starts |
+    | `OVERLAPS` | A starts before B, ends inside B |
+    | `DURING` | A is fully inside B |
+    | `STARTS` | A and B start together, A ends first |
+    | `FINISHES` | A and B end together, A starts later |
+    | `EQUALS` | Identical intervals |
+    | + 6 inverses | `AFTER`, `MET_BY`, `OVERLAPPED_BY`, `CONTAINS`, `STARTED_BY`, `FINISHED_BY` |
+  </Tab>
+</Tabs>
 
 ## ExplanationGenerator
 
@@ -212,12 +290,139 @@ explanation = generator.explain(
 
 print(explanation.conclusion)
 print(f"Confidence: {explanation.confidence:.2f}")
+print(explanation.justification.summary)
 
 for step in explanation.reasoning_path.steps:
-    print(f"  Step {step.depth}: {step.fact}")
-    print(f"    via rule: '{step.rule_name}'")
+    indent = "  " * step.depth
+    print(f"{indent}Step {step.depth}: {step.fact}")
+    print(f"{indent}  via rule: '{step.rule_name}'")
+    print(f"{indent}  premises: {step.premises}")
 ```
 
+<Tip>
+  Name every rule with a descriptive string. `ExplanationGenerator` includes the rule name in each derivation step — unnamed rules produce useless explanations like "rule_0 fired." Use names like `"manager_authority"` or `"transitive_location"`.
+</Tip>
+
+<AccordionGroup>
+  <Accordion title="Explanation schema">
+
+```python
+@dataclass
+class Explanation:
+    conclusion:     Dict[str, str]   # the fact being explained
+    confidence:     float            # aggregated rule confidence
+    reasoning_path: ReasoningPath    # full derivation trace
+    justification:  Justification    # plain-language summary
+```
+
+  </Accordion>
+  <Accordion title="ReasoningPath and ReasoningStep schemas">
+
+```python
+@dataclass
+class ReasoningPath:
+    steps: List[ReasoningStep]       # ordered derivation steps
+
+@dataclass
+class ReasoningStep:
+    depth:       int                 # 0 = base fact, n = nth inference
+    fact:        Dict[str, str]      # the fact derived at this step
+    rule_name:   str                 # name of the rule that fired
+    premises:    List[Dict]          # facts that triggered this rule
+    confidence:  float               # confidence at this step
+```
+
+  </Accordion>
+  <Accordion title="Justification schema">
+
+```python
+@dataclass
+class Justification:
+    summary:   str         # one-sentence natural language explanation
+    evidence:  List[str]   # list of supporting source facts
+```
+
+  </Accordion>
+</AccordionGroup>
+
+## Combining Multiple Reasoning Engines
+
+Different engines cover different expressivity levels — compose them for richer inference:
+
+<Steps>
+  <Step title="Forward-chain structural rules with Reasoner">
+    ```python
+    from semantica.reasoning import Reasoner
+
+    engine = Reasoner()
+    engine.apply_transitivity("located_in")
+    engine.apply_symmetry("colleague_of")
+    structural_result = engine.infer()
+    ```
+  </Step>
+  <Step title="Pass derived facts to DatalogReasoner for recursive closure">
+    ```python
+    from semantica.reasoning import DatalogReasoner, DatalogFact, DatalogRule
+
+    datalog = DatalogReasoner()
+    for fact in structural_result.derived_facts:
+        datalog.add_fact(DatalogFact(fact.predicate, (fact.subject, fact.obj)))
+
+    datalog.add_rule(DatalogRule("reachable(?X, ?Z) :- located_in(?X, ?Y), reachable(?Y, ?Z)."))
+    datalog.evaluate()
+    ```
+  </Step>
+  <Step title="Filter results to a time window with TemporalReasoningEngine">
+    ```python
+    from semantica.reasoning import TemporalReasoningEngine
+    from datetime import datetime
+
+    temporal = TemporalReasoningEngine()
+    active_facts = [
+        f for f in datalog.query("reachable(?X, ?Z)")
+        if temporal.is_active(f, at=datetime(2024, 1, 1))
+    ]
+    ```
+  </Step>
+  <Step title="Explain any conclusion with ExplanationGenerator">
+    ```python
+    from semantica.reasoning import ExplanationGenerator
+
+    generator = ExplanationGenerator(engine)
+    explanation = generator.explain(
+        {"subject": "london_office", "predicate": "located_in", "object": "UK"}
+    )
+    print(explanation.summary)
+    ```
+  </Step>
+</Steps>
+
+## Tips and Common Pitfalls
+
+<Warning>
+  **Always call `reasoner.infer()` after adding facts and rules.** Adding facts and rules updates internal state but doesn't trigger inference automatically. Inference is a separate, explicit step.
+</Warning>
+
+<Tip>
+  **Use `ReteEngine` for large rule sets.** If you have more than ~20 rules and rules can fire repeatedly, `Reasoner` re-evaluates all rules from scratch on each cycle. `ReteEngine` caches partial matches and is orders of magnitude faster for complex rule sets.
+</Tip>
+
+<Warning>
+  **`DatalogReasoner` guarantees termination; `Reasoner` does not.** If your rules can create cycles (A derives B, B derives C, C re-derives A), `DatalogReasoner`'s semi-naive fixpoint evaluation will stop when no new facts are added. `Reasoner` has no cycle detection and may loop infinitely.
+</Warning>
+
+<Tip>
+  **Name every rule for readable explanations.** `ExplanationGenerator.explain()` includes the rule name in each derivation step. Unnamed or generic rule names produce useless explanations like "rule_0 fired." Use descriptive names: `"manager_authority"`, `"transitive_location"`.
+</Tip>
+
+<Tip>
+  **Use rule `priority` to control inference order.** When multiple rules could fire on the same facts, higher-priority rules fire first. This matters when a higher-priority rule produces a fact that gates a lower-priority rule's conditions.
+</Tip>
+
+<Tip>
+  **Combine engines for maximum expressivity.** Forward-chain structural rules with `Reasoner`, then pass derived facts to `DatalogReasoner` for recursive closure, then filter by time with `TemporalReasoningEngine`. Each engine covers a different expressivity class — they compose cleanly.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Knowledge Graph" icon="diagram-project" href="kg">
     The knowledge graph being reasoned over.

docs/reference/seed.md

@@ -1,119 +1,322 @@
 ---
 title: "Seed Module"
-description: "Seed data management for initializing Knowledge Graphs from trusted, verified sources."
+description: "Bootstrap Knowledge Graphs from verified, structured sources — taxonomies, reference tables, product catalogs, and domain anchors."
 icon: "database"
 ---
 
-`semantica.seed` provides a system for bootstrapping Knowledge Graphs with verified, structured data from trusted sources — taxonomies, reference tables, user lists, product catalogs — so you start with a reliable foundation rather than an empty graph.
+`semantica.seed` gives your knowledge graph a reliable starting point. Rather than building from an empty graph and hoping extraction produces consistent reference data, you load verified, structured sources first — ISO codes, employee rosters, product catalogs, domain taxonomies — then merge freshly extracted data on top.
 
 ## What You Get
 
-- **`SeedDataManager`** — register sources, build a foundation graph, and integrate with extracted data
-- **`SeedDataSource`** — define individual data sources with format, path, and config
-- **Merge strategies** — `seed_first`, `extracted_first`, `smart_merge` for combining seed and extracted data
-- **Validation** — check data quality and schema compliance before loading
-- **Versioning** — track and manage versions of seed data sources
+<CardGroup cols={2}>
+  <Card title="SeedDataManager" icon="database">
+    Register sources, build a foundation graph, validate quality, and merge with extracted data.
+  </Card>
+  <Card title="SeedDataSource" icon="file-code">
+    Typed source definition supporting CSV, JSON, SQL, API, and RDF with format-specific config.
+  </Card>
+  <Card title="populate() & inject()" icon="circle-plus">
+    Inject named built-in datasets (companies, countries, currencies) or custom seed files into an existing graph.
+  </Card>
+  <Card title="Merge Strategies" icon="arrows-merge">
+    `seed_first`, `extracted_first`, and `smart_merge` with property-level conflict detection.
+  </Card>
+  <Card title="Validation" icon="shield-check">
+    Required field checks, ID uniqueness, type consistency, reference integrity, and encoding validation before loading.
+  </Card>
+  <Card title="Versioning" icon="clock-rotate-left">
+    Track seed data versions across pipeline runs and diff changes between versions.
+  </Card>
+</CardGroup>
 
 <Tip>
-  **When to use the Seed Module:**
-
-- **Bootstrapping** — you have existing structured data (taxonomies, user lists, product catalogs) to build on
-- **Reference data** — load immutable reference information (countries, ISO codes, ontology terms)
-- **Testing** — load consistent, reproducible datasets for development and CI pipelines
+  **When to use the Seed Module:** Bootstrapping with structured reference data (taxonomies, user lists, product catalogs), loading immutable facts (ISO country codes, standard ontology terms) that extracted data should not override, ensuring test reproducibility with deterministic datasets, and anchoring entity disambiguation with canonical forms.
 </Tip>
 
-## SeedDataManager
+## Quick Start
 
-The primary interface for seed data:
+<Steps>
+  <Step title="Register your seed sources">
+    ```python
+    from semantica.seed import SeedDataManager
 
-```python
-from semantica.seed import SeedDataManager
+    manager = SeedDataManager()
 
-manager = SeedDataManager()
-manager.register_source("countries", "csv",  "data/countries.csv")
-manager.register_source("taxonomy",  "json", "data/taxonomy.json")
+    manager.register_source("countries",  "csv",  "data/countries.csv")
+    manager.register_source("taxonomy",   "json", "data/taxonomy.json")
+    manager.register_source("employees",  "csv",  "data/employees.csv")
+    ```
+  </Step>
+  <Step title="Build the foundation graph">
+    ```python
+    foundation_kg = manager.create_foundation_graph()
 
-# Build a foundation KG from all registered sources
-foundation_kg = manager.create_foundation_graph()
-```
+    print(f"Foundation nodes: {foundation_kg.node_count}")
+    print(f"Foundation edges: {foundation_kg.edge_count}")
+    ```
+  </Step>
+  <Step title="Validate before loading">
+    ```python
+    report = manager.validate_quality(manager.load_source("employees"))
 
-### Core Methods
+    if not report.is_valid:
+        for issue in report.issues:
+            print(f"[{issue.severity}] Row {issue.row}: {issue.message}")
+    else:
+        print(f"Validated {report.record_count} records — no issues found")
+    ```
+  </Step>
+  <Step title="Merge with extracted data">
+    ```python
+    final_kg = manager.integrate_with_extracted(
+        seed_graph=foundation_kg,
+        extracted_data=new_entities,
+        strategy="smart_merge",
+    )
+    print(f"Final graph: {final_kg.node_count} nodes, {final_kg.edge_count} edges")
+    ```
+  </Step>
+</Steps>
+
+## SeedDataSource Types
+
+<Tabs>
+  <Tab title="CSV">
+    ```python
+    from semantica.seed import SeedDataSource
+
+    csv_source = SeedDataSource(
+        name="employees",
+        type="csv",
+        path="data/employees.csv",
+        config={
+            "delimiter": ";",
+            "encoding":  "utf-8",
+            "id_column": "employee_id",
+            "type":      "Person",
+        }
+    )
+
+    manager.register_source_object(csv_source)
+    ```
+
+    Best for: employee rosters, product lists, reference tables.
+  </Tab>
+  <Tab title="JSON">
+    ```python
+    json_source = SeedDataSource(
+        name="taxonomy",
+        type="json",
+        path="data/taxonomy.json",
+        config={"encoding": "utf-8"}
+    )
+    ```
+
+    Expects an array of entity objects. Best for: taxonomies, ontology term lists, structured configs.
+  </Tab>
+  <Tab title="SQL">
+    ```python
+    sql_source = SeedDataSource(
+        name="products",
+        type="sql",
+        path="postgresql://user:pass@localhost/db",
+        config={"query": "SELECT id, name, category FROM products WHERE active = true"}
+    )
+    ```
+
+    Best for: live database tables — PostgreSQL, MySQL, SQLite.
+  </Tab>
+  <Tab title="API & RDF">
+    ```python
+    # API source — fetch from a REST endpoint
+    api_source = SeedDataSource(
+        name="geo_codes",
+        type="api",
+        path="https://restcountries.com/v3.1/all",
+        config={"fields": ["name", "cca2", "region"]}
+    )
+
+    # RDF source — OWL ontologies or Turtle files
+    rdf_source = SeedDataSource(
+        name="domain_ontology",
+        type="rdf",
+        path="data/ontology.ttl",
+        config={"format": "turtle"}
+    )
+    ```
+
+    API: external reference APIs (countries, currencies, geo). RDF: existing knowledge bases, OWL ontologies.
+  </Tab>
+  <Tab title="Type Reference">
+
+    | Type | Path Format | Use Case |
+    | ---- | ----------- | -------- |
+    | `csv` | File path | Employee rosters, product lists, reference tables |
+    | `json` | File path | Taxonomies, ontology term lists, structured configs |
+    | `sql` | Connection string | Live database tables — PostgreSQL, MySQL, SQLite |
+    | `api` | URL | External reference APIs (countries, currencies, geo) |
+    | `rdf` | File path | OWL ontologies, Turtle files, existing knowledge bases |
+
+  </Tab>
+</Tabs>
+
+## SeedDataManager Reference
 
 | Method | Description |
 | ------ | ----------- |
-| `register_source(name, format, location)` | Add a data source to the registry |
+| `register_source(name, format, path)` | Add a named data source to the registry |
 | `create_foundation_graph()` | Build a KG from all registered sources |
-| `validate_quality(seed_data)` | Check data quality and completeness |
-| `integrate_with_extracted(seed, extracted)` | Merge seed and extracted graphs |
-| `export_seed_data(path, format)` | Export seed data to RDF, JSON, or CSV |
-
-## SeedDataSource
-
-Define a source with format-specific configuration:
-
-```python
-from semantica.seed import SeedDataSource
-
-source = SeedDataSource(
-    name="taxonomy",
-    type="json",          # "csv" | "json" | "api" | "sql"
-    path="taxonomy.json",
-    config={"encoding": "utf-8"}
-)
-```
+| `validate_quality(seed_data)` | Check schema compliance, required fields, and duplicates |
+| `integrate_with_extracted(seed, extracted, strategy)` | Merge seed and extracted graphs |
+| `export_seed_data(path, format)` | Export seed graph to RDF (`turtle`, `json-ld`), JSON, or CSV |
+| `populate(kg, dataset, count)` | Inject a named built-in dataset into an existing graph |
+| `inject(kg)` | Merge all registered sources into `kg` without duplicating existing entities |
+| `load_from_file(path)` | Load seed nodes from JSON, CSV, or RDF file into the manager |
+| `list_sources()` | List all registered source names and their formats |
+| `get_version(name)` | Get the current version metadata for a named source |
 
 ## Merge Strategies
 
-Control how seed data and extracted data are combined:
+<Tabs>
+  <Tab title="seed_first">
+    Seed data wins on every conflicting property. Use when seed encodes authoritative reference facts that must not be overridden.
+
+    ```python
+    final_kg = manager.integrate_with_extracted(
+        seed_graph=foundation_kg,
+        extracted_data=new_entities,
+        strategy="seed_first",
+    )
+    ```
+
+    Best for: ISO codes, canonical entity names, official taxonomy IDs, employee records.
+  </Tab>
+  <Tab title="extracted_first">
+    Extracted data overrides seed on conflicting properties. Use when new documents contain more current information than your reference data.
+
+    ```python
+    final_kg = manager.integrate_with_extracted(
+        seed_graph=foundation_kg,
+        extracted_data=new_entities,
+        strategy="extracted_first",
+    )
+    ```
+
+    Best for: frequently changing attributes like addresses, titles, revenue figures.
+  </Tab>
+  <Tab title="smart_merge">
+    Property-level merge with conflict detection — irresolvable conflicts are logged for manual review rather than silently overwritten.
+
+    ```python
+    final_kg = manager.integrate_with_extracted(
+        seed_graph=foundation_kg,
+        extracted_data=new_entities,
+        strategy="smart_merge",
+    )
+    ```
+
+    Best for: general-purpose pipelines where surfacing conflicts is more valuable than silently losing data.
+  </Tab>
+</Tabs>
+
+## Built-in Datasets
+
+Inject canonical reference data without loading external files:
 
 ```python
-final_kg = manager.integrate_seed_extracted(
-    seed_graph=foundation_kg,
-    extracted_data=new_data,
-    strategy="seed_first"   # see options below
-)
+from semantica.seed import SeedDataManager
+from semantica.kg import GraphBuilder
+
+kg      = GraphBuilder().build(entities=entities, relationships=relationships)
+manager = SeedDataManager()
+
+# Inject a named built-in dataset
+manager.populate(kg, dataset="companies", count=100)
+manager.populate(kg, dataset="countries")
 ```
 
-| Strategy | Behavior |
-| -------- | -------- |
-| `seed_first` | Seed data wins on conflicts — use for authoritative reference data |
-| `extracted_first` | Extracted data overrides seed — use when new data is more current |
-| `smart_merge` | Property-level merging with conflict detection and resolution |
+| Dataset | Content |
+| ------- | ------- |
+| `companies` | Fortune 500 companies with type, sector, HQ |
+| `countries` | ISO 3166 country codes, regions, populations |
+| `currencies` | ISO 4217 codes, symbols, names |
+| `person_names` | Common first/last names for synthetic data |
 
-## Bootstrapping a KG
-
-Full example — load foundation data then merge with freshly ingested content:
+## Full Pipeline Example
 
 ```python
 from semantica.seed import SeedDataManager
 from semantica.ingest import FileIngestor
-from semantica.semantic_extract import NERExtractor
+from semantica.parse import DocumentParser
+from semantica.split import TextSplitter
+from semantica.semantic_extract import NERExtractor, RelationExtractor
+from semantica.kg import GraphBuilder
 from semantica.llms import Groq
 import os
 
-# Build foundation from verified reference data
-manager = SeedDataManager()
-manager.register_source("taxonomy",  "json", "taxonomy.json")
-manager.register_source("employees", "csv",  "employees.csv")
-foundation_kg = manager.create_foundation_graph()
-
-# Ingest and extract from new sources
 llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
-ingestor = FileIngestor()
-ner = NERExtractor(method="llm", llm_provider=llm)
 
-sources = ingestor.ingest("news_articles/")
-new_data = [ner.extract(s.content) for s in sources]
+# Step 1 — Build the foundation from verified reference data
+seed_manager = SeedDataManager()
+seed_manager.register_source("taxonomy",  "json", "data/taxonomy.json")
+seed_manager.register_source("employees", "csv",  "data/employees.csv")
+foundation_kg = seed_manager.create_foundation_graph()
 
-# Merge — seed data takes precedence for reference facts
-final_kg = manager.integrate_seed_extracted(
+# Step 2 — Ingest and extract from unstructured documents
+ingestor  = FileIngestor()
+parser    = DocumentParser()
+splitter  = TextSplitter(method="semantic_transformer", chunk_size=512)
+ner       = NERExtractor(method="llm", llm_provider=llm)
+rel_ext   = RelationExtractor(method="llm", llm_provider=llm)
+
+sources                 = ingestor.ingest("news_articles/")
+extracted_entities      = []
+extracted_relationships = []
+
+for source in sources:
+    parsed = parser.parse(source)
+    chunks = splitter.split_document(parsed)
+    for chunk in chunks:
+        entities      = ner.extract(chunk.text)
+        relationships = rel_ext.extract(chunk.text, entities=entities)
+        extracted_entities.extend(entities)
+        extracted_relationships.extend(relationships)
+
+# Step 3 — Merge seed and extracted data
+final_kg = seed_manager.integrate_with_extracted(
     seed_graph=foundation_kg,
-    extracted_data=new_data,
-    strategy="seed_first"
+    extracted_data=extracted_entities,
+    strategy="seed_first",
 )
+print(f"Final graph: {final_kg.node_count} nodes, {final_kg.edge_count} edges")
 ```
 
-## Configuration
+## Versioning
+
+Track seed data versions to detect when reference data changes between pipeline runs:
+
+```python
+manager = SeedDataManager()
+manager.register_source("taxonomy", "json", "data/taxonomy.json")
+
+version = manager.get_version("taxonomy")
+print(f"Version: {version.version_id}")
+print(f"Hash:    {version.checksum}")
+print(f"Records: {version.record_count}")
+print(f"Updated: {version.last_modified}")
+
+# Compare versions to detect changes
+old_version = manager.get_version("taxonomy", tag="previous")
+if version.checksum != old_version.checksum:
+    diff = manager.diff_versions("taxonomy", old_version.version_id, version.version_id)
+    print(f"Added:   {diff.added_count} records")
+    print(f"Removed: {diff.removed_count} records")
+    print(f"Changed: {diff.modified_count} records")
+```
+
+## YAML Configuration
+
+Define sources in YAML for production deployments — no code changes needed to switch environments:
 
 ```yaml
 seed:
@@ -121,22 +324,57 @@ seed:
     - name: "employees"
       type: "csv"
       path: "./data/employees.csv"
+      config:
+        id_column: "employee_id"
+        type: "Person"
     - name: "taxonomy"
       type: "json"
       path: "./data/taxonomy.json"
+    - name: "products"
+      type: "sql"
+      path: "${DATABASE_URL}"
+      config:
+        query: "SELECT id, name, category FROM products WHERE active = true"
   merge:
-    strategy: "seed_first"
+    strategy: "smart_merge"
   validation:
     strict: true
+    required_fields: ["id", "type"]
 ```
 
 Environment variable overrides:
 
 ```bash
-export SEED_DATA_DIR=./data/seed
-export SEED_MERGE_STRATEGY=seed_first
+export SEMANTICA_SEED_DATA_DIR=./data/seed
+export SEMANTICA_SEED_MERGE_STRATEGY=seed_first
 ```
 
+## Tips and Common Pitfalls
+
+<Warning>
+  **Load seed data before extracted data.** Seed data is your ground truth — normalised, curated, and already de-duplicated. Load it first with `create_foundation_graph()`, then merge extracted entities on top. Merging in the wrong order lets noisy extracted data overwrite trusted reference values.
+</Warning>
+
+<Tip>
+  **Use `seed_first` merge strategy for reference data.** When seed data encodes authoritative facts (official company names, canonical taxonomy IDs, employee records), `strategy="seed_first"` ensures those values win over extracted values. Use `smart_merge` only when extracted data may be more current than the seed.
+</Tip>
+
+<Warning>
+  **Validate before loading.** `manager.validate_quality(seed_data)` catches missing required fields, type inconsistencies, and duplicate IDs before they corrupt your graph. Running validation after loading means you'll need to roll back. Validation is fast — always run it first.
+</Warning>
+
+<Tip>
+  **Register all sources before calling `create_foundation_graph()`.** `create_foundation_graph()` processes all registered sources in one pass. Registering a source after calling it means that source is silently excluded. Register all sources at the start of your script, then call `create_foundation_graph()` once.
+</Tip>
+
+<Tip>
+  **Track seed versions to detect drift.** Use `manager.get_version()` and `manager.diff_versions()` to detect when reference data changes between pipeline runs. If a taxonomy file changes, downstream entity normalisation and deduplication thresholds may need re-tuning — don't treat seed data as static.
+</Tip>
+
+<Tip>
+  **Use YAML configuration for production deployments.** Hard-coding source paths in Python scripts makes environment-switching (dev → staging → prod) fragile. Declare sources in `config.yaml` under the `seed:` key and override paths with `SEMANTICA_SEED_DATA_DIR`. This way, the same code runs in every environment.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Ingest" icon="file-import" href="ingest">
     Load unstructured data alongside seed data.
@@ -148,6 +386,6 @@ export SEED_MERGE_STRATEGY=seed_first
     Handle duplicates during seed-extracted merge.
   </Card>
   <Card title="Pipeline" icon="gear" href="pipeline">
-    Incorporate seed loading as a pipeline step.
+    Incorporate seed loading as a named pipeline step.
   </Card>
 </CardGroup>

docs/reference/semantic_extract.md

@@ -6,34 +6,162 @@ icon: "magnifying-glass-chart"
 
 `semantica.semantic_extract` extracts structured information from unstructured text — the foundation of every knowledge graph in Semantica. All extractors support three modes: pattern-based (no API key), ML-based, and LLM-based.
 
+## Quick Start
+
+<Steps>
+  <Step title="Resolve coreferences (optional but recommended)">
+    ```python
+    from semantica.semantic_extract import CoreferenceResolver
+
+    resolver     = CoreferenceResolver()
+    resolved_text = resolver.resolve(
+        "Apple Inc. was founded in 1976. The company is headquartered in Cupertino."
+    )
+    # "Apple Inc." replaces "The company" — consistent downstream extraction
+    ```
+  </Step>
+  <Step title="Extract named entities">
+    ```python
+    from semantica.semantic_extract import NERExtractor
+    from semantica.llms import Groq
+    import os
+
+    llm      = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+    ner      = NERExtractor(method="llm", llm_provider=llm, max_retries=3)
+    entities = ner.extract(resolved_text)
+    # → [{"text": "Apple Inc.", "type": "ORGANIZATION", "confidence": 0.98, ...}]
+    ```
+  </Step>
+  <Step title="Extract relationships between entities">
+    ```python
+    from semantica.semantic_extract import RelationExtractor
+
+    rel           = RelationExtractor(method="llm", llm_provider=llm, max_retries=3)
+    relationships = rel.extract(resolved_text, entities=entities)
+    # → [{"subject": "Steve Jobs", "predicate": "founded", "object": "Apple Inc.", ...}]
+    ```
+  </Step>
+  <Step title="Validate and filter before building the graph">
+    ```python
+    from semantica.semantic_extract import ExtractionValidator
+
+    validator = ExtractionValidator(min_confidence=0.7)
+    valid_entities, _ = validator.validate_entities(entities)
+    valid_rels,     _ = validator.validate_relations(relationships)
+    ```
+  </Step>
+</Steps>
+
 ## What You Get
 
-- **`NERExtractor`** — named entity recognition: Person, Organization, Location, Date, and custom types
-- **`RelationExtractor`** — typed semantic relationships between entities (`founded_by`, `located_in`, etc.)
-- **`TripletExtractor`** — direct `(subject, predicate, object)` triplet generation for RDF-ready output
-- **`EventExtractor`** — event detection with participants, temporal context, and confidence scores
-- **`CoreferenceResolver`** — resolve "Apple" and "the company" to the same entity across a document
+<CardGroup cols={2}>
+  <Card title="NERExtractor" icon="tag">
+    Named entity recognition: Person, Organization, Location, Date, and custom types.
+  </Card>
+  <Card title="RelationExtractor" icon="arrow-right-arrow-left">
+    Typed semantic relationships between entities (`founded_by`, `located_in`, etc.).
+  </Card>
+  <Card title="TripletExtractor" icon="table">
+    Direct `(subject, predicate, object)` triplet generation for RDF-ready output.
+  </Card>
+  <Card title="EventExtractor" icon="calendar">
+    Event detection with participants, temporal context, and confidence scores.
+  </Card>
+  <Card title="CoreferenceResolver" icon="link">
+    Resolve "Apple" and "the company" to the same entity across a document.
+  </Card>
+  <Card title="SemanticAnalyzer" icon="chart-scatter">
+    Semantic role labeling, clustering, and entity similarity analysis.
+  </Card>
+</CardGroup>
 
 <img src="/assets/img/diagrams/extraction-pipeline.svg" alt="Semantic extraction pipeline: raw text fans into NER, Relation, and Coreference extractors, then merges into a Triplet Generator" style={{ width: '100%', borderRadius: '12px', margin: '0 0 24px' }} />
 
+## Extraction Methods
+
+<Tabs>
+  <Tab title="LLM (best accuracy)">
+    Uses a language model to extract entities and relationships. Handles complex schemas, novel entity types, and domain-specific language. Requires an API key.
+
+    ```python
+    from semantica.semantic_extract import NERExtractor
+    from semantica.llms import Groq
+    import os
+
+    llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+    ner = NERExtractor(method="llm", llm_provider=llm, max_retries=3)
+
+    entities = ner.extract("Apple Inc. was founded by Steve Jobs in Cupertino in 1976.")
+    ```
+
+    <Note>
+      **v0.5.0 fix:** `NERExtractor(method="llm")` no longer silently falls back to pattern extraction on custom gateways. The `response_format=json_object` parameter is now conditionally omitted for incompatible gateways, with a plain `generate()` + JSON parsing fallback applied automatically.
+    </Note>
+
+    Works with every Semantica LLM provider — swap `Groq` for `Anthropic`, `OpenAI`, `Gemini`, `Ollama`, `HuggingFace`, `DeepSeek`, or `Novita` with a one-line change:
+
+    ```python
+    from semantica.llms import Anthropic
+    llm = Anthropic(model="claude-opus-4-7", api_key=os.getenv("ANTHROPIC_API_KEY"))
+    ner = NERExtractor(method="llm", llm_provider=llm)
+    ```
+  </Tab>
+  <Tab title="ML (fast, free)">
+    Uses a pre-trained BERT-based NER model. High accuracy for standard entity types (Person, Organization, Location, Date) at zero API cost.
+
+    ```python
+    ner      = NERExtractor(method="ml", model="dslim/bert-large-NER")
+    entities = ner.extract(text)
+    ```
+
+    For relations, the ML backend uses the REBEL model:
+
+    ```python
+    rel           = RelationExtractor(method="ml")
+    relationships = rel.extract(text, entities=entities)
+    ```
+
+    Best for: high-throughput extraction where API cost matters and entity types are standard CoNLL/OntoNotes categories.
+  </Tab>
+  <Tab title="Pattern (zero cost)">
+    Dictionary and regex matching — extremely fast, zero API cost, zero model loading. Accuracy depends entirely on your dictionaries.
+
+    ```python
+    ner = NERExtractor(
+        method="pattern",
+        custom_entities={
+            "DRUG": ["aspirin", "ibuprofen", "metformin"],
+            "GENE": ["BRCA1", "TP53", "EGFR"]
+        }
+    )
+    entities = ner.extract(text)
+    ```
+
+    For relations, uses hand-crafted rules:
+
+    ```python
+    rel = RelationExtractor(method="rule")
+    ```
+
+    Best for: known entity sets (drug names, product codes, gene symbols), no-API-key environments, or as a first pass before LLM validation.
+
+    <Warning>
+      The pattern matcher is **case-sensitive and whitespace-sensitive**. Normalize text first with `TextNormalizer` so "BRCA1" and "brca1" both match. For fuzzy matching, use `method="ml"`.
+    </Warning>
+  </Tab>
+</Tabs>
+
+### Method Comparison
+
+| Method | Speed | Cost | Accuracy | Custom Types | Best For |
+| ------ | ----- | ---- | -------- | ------------ | -------- |
+| `pattern` | Very fast | Free | Medium | Yes (dictionary) | Known entity sets, no-API environments |
+| `ml` | Fast | Free | High | Limited | Standard types at scale, no API budget |
+| `llm` | Medium | API cost | Highest | Yes (schema) | Complex schemas, novel types, best accuracy |
+
 ## NERExtractor
 
 ```python
-from semantica.semantic_extract import NERExtractor
-from semantica.llms import Groq
-import os
-
-# Pattern-based — fast, no API key, good for standard entity types
-ner = NERExtractor(method="pattern")
-entities = ner.extract("Apple Inc. was founded by Steve Jobs in Cupertino.")
-
-# ML-based — higher accuracy, no API cost
-ner = NERExtractor(method="ml", model="dslim/bert-large-NER")
-entities = ner.extract(text)
-
-# LLM-based — best accuracy, handles complex schemas and custom types
-llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
-ner = NERExtractor(method="llm", llm_provider=llm, max_retries=3)
 entities = ner.extract(text)
 ```
 
@@ -47,28 +175,16 @@ Output format:
 ]
 ```
 
-### Custom Entity Types
+Batch processing for large corpora:
 
 ```python
-ner = NERExtractor(
-    method="pattern",
-    custom_entities={
-        "DRUG": ["aspirin", "ibuprofen", "metformin"],
-        "GENE": ["BRCA1", "TP53", "EGFR"]
-    }
-)
+texts         = ["Text 1...", "Text 2...", "Text 3..."]
+batch_results = ner.extract_batch(texts, batch_size=10)
 ```
 
-<Note>
-  **v0.5.0 fix:** `NERExtractor(method="llm")` no longer silently falls back to pattern extraction on custom gateways. The `response_format=json_object` parameter is now conditionally omitted for incompatible gateways, with a plain `generate()` + JSON parsing fallback applied automatically.
-</Note>
-
 ## RelationExtractor
 
 ```python
-from semantica.semantic_extract import RelationExtractor
-
-rel = RelationExtractor(method="llm", llm_provider=llm, max_retries=3)
 relationships = rel.extract(text, entities=entities)
 ```
 
@@ -81,7 +197,9 @@ Output format:
 ]
 ```
 
-Available methods: `"rule"` (pattern-based), `"ml"` (REBEL model), `"llm"`.
+<Tip>
+  Always pass `entities=entities` from your NER output. This anchors relationships to known entity spans — improving accuracy and eliminating hallucinated entity names.
+</Tip>
 
 ## TripletExtractor
 
@@ -90,12 +208,12 @@ Generate RDF-ready `(subject, predicate, object)` triplets directly from text:
 ```python
 from semantica.semantic_extract import TripletExtractor
 
-trip = TripletExtractor(method="llm", llm_provider=llm)
+trip     = TripletExtractor(method="llm", llm_provider=llm)
 triplets = trip.extract(text)
 # → [{"subject": "Steve Jobs", "predicate": "founded", "object": "Apple Inc.", ...}]
 ```
 
-Triplets are suitable for loading directly into a triplet store or knowledge graph.
+Triplets are suitable for loading directly into a triplet store or knowledge graph without a separate relation extraction step.
 
 ## EventExtractor
 
@@ -105,63 +223,127 @@ Detect events with participants and temporal context:
 from semantica.semantic_extract import EventExtractor
 
 extractor = EventExtractor(method="llm", llm_provider=llm)
-events = extractor.extract(text)
+events    = extractor.extract(text)
 ```
 
 Output includes: event type, participants (with roles), temporal information, location, and confidence score.
 
-## CoreferenceResolver
+## SemanticAnalyzer
 
-Resolve pronoun and alias references to canonical entities before extraction:
+Semantic role labeling, clustering, and similarity analysis on extracted content:
 
 ```python
-from semantica.semantic_extract import CoreferenceResolver
+from semantica.semantic_extract import SemanticAnalyzer
 
-resolver = CoreferenceResolver()
-resolved_text = resolver.resolve(
-    "Apple Inc. was founded in 1976. The company is headquartered in Cupertino."
+analyzer = SemanticAnalyzer()
+
+# Who did what to whom
+roles = analyzer.label_roles(text)
+# → [{"agent": "Apple", "action": "acquired", "patient": "Intel's modem unit"}]
+
+# Group similar entities
+clusters = analyzer.cluster_entities(entities, n_clusters=5)
+# → [{"cluster_id": 0, "entities": ["Apple Inc.", "Apple", "AAPL"]}, ...]
+
+# Pairwise semantic similarity
+score = analyzer.calculate_similarity(entity_a, entity_b)
+# → 0.87
+```
+
+| Method | Returns | Description |
+| ------ | ------- | ----------- |
+| `label_roles(text)` | `List[Dict]` | Semantic role labeling (agent, action, patient) |
+| `cluster_entities(entities, n_clusters)` | `List[Cluster]` | Group similar entities |
+| `calculate_similarity(a, b)` | `float` | Cosine similarity between entity embeddings |
+| `analyze_sentiment(text)` | `Dict` | Sentiment and subjectivity scores |
+
+## SemanticNetworkExtractor
+
+Extracts a full semantic network (nodes + typed edges) from text in one pass:
+
+```python
+from semantica.semantic_extract import SemanticNetworkExtractor
+
+extractor = SemanticNetworkExtractor(method="llm", llm_provider=llm)
+network   = extractor.extract_network(text)
+
+print(f"Nodes: {len(network.nodes)}")
+print(f"Edges: {len(network.edges)}")
+
+for edge in network.edges:
+    print(f"  {edge.source} --[{edge.relation}]--> {edge.target}  (conf: {edge.confidence:.2f})")
+```
+
+<Accordion title="SemanticNetwork schema">
+
+```python
+@dataclass
+class SemanticNetwork:
+    nodes: List[NetworkNode]
+    edges: List[NetworkEdge]
+
+@dataclass
+class NetworkNode:
+    id:         str
+    label:      str            # entity type (PERSON, ORG, etc.)
+    properties: Dict[str, Any]
+
+@dataclass
+class NetworkEdge:
+    source:     str
+    target:     str
+    relation:   str            # e.g. "founded_by", "located_in"
+    confidence: float
+    metadata:   Dict[str, Any]
+```
+
+</Accordion>
+
+## ExtractionValidator
+
+Validates extraction quality and filters low-confidence results:
+
+```python
+from semantica.semantic_extract import ExtractionValidator
+
+validator = ExtractionValidator(
+    min_confidence=0.7,       # drop entities below this score
+    require_entity_text=True, # entity text must be non-empty
+    max_entity_length=100,    # discard suspiciously long entities
 )
-# "Apple Inc." replaces "The company" for consistent downstream extraction
+
+valid_entities, rejected = validator.validate_entities(entities)
+valid_rels,     rejected = validator.validate_relations(relationships)
+
+report = validator.get_quality_report(entities, relationships)
+print(f"Precision estimate: {report['precision']:.2f}")
 ```
 
-## Batch Processing
+## Tips and Common Pitfalls
 
-All extractors support batch input for efficient large-scale processing:
+<Tip>
+  **Run `CoreferenceResolver` before extraction.** If a paragraph says "Apple Inc. was founded in 1976. The company launched..." without resolving "the company" → "Apple Inc.", your extractor may miss the second entity or create a phantom "The company" node.
+</Tip>
 
-```python
-texts = ["Text 1...", "Text 2...", "Text 3..."]
+<Tip>
+  **Always pass `entities=` to `RelationExtractor`.** Passing the entity list from NER output anchors relationships to known entity spans — improving accuracy and eliminating hallucinated entity names.
+</Tip>
 
-ner = NERExtractor(method="llm", llm_provider=llm)
-batch_results = ner.extract_batch(texts, batch_size=10)
-```
+<Warning>
+  **Validate before building the graph.** Use `ExtractionValidator(min_confidence=0.7)` to drop low-confidence entities before they reach `GraphBuilder`. Noisy extractions produce noisy graphs that corrupt analytics and search.
+</Warning>
 
-## Using All Extractors Together
+<Tip>
+  **Set `max_retries=3` for LLM extractors.** API calls fail transiently. Setting `max_retries=3` prevents pipeline crashes on flaky network conditions without slowing down the happy path.
+</Tip>
 
-The standard extraction pipeline — entities → relationships → triplets:
+<Warning>
+  **Don't mix extraction methods mid-pipeline.** If you extract entities with `pattern` and relations with `llm`, entity names in the relation output may not match the pattern-extracted entity IDs — causing alignment failures in `GraphBuilder`. Use the same method throughout, or normalize entity names before the relation step.
+</Warning>
 
-```python
-from semantica.semantic_extract import NERExtractor, RelationExtractor, TripletExtractor
-from semantica.llms import Groq
-import os
-
-llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
-
-ner  = NERExtractor(method="llm",      llm_provider=llm, max_retries=3)
-rel  = RelationExtractor(method="llm", llm_provider=llm, max_retries=3)
-trip = TripletExtractor(method="llm",  llm_provider=llm, max_retries=3)
-
-entities      = ner.extract(text)
-relationships = rel.extract(text, entities=entities)
-triplets      = trip.extract(text)
-```
-
-## Extraction Method Comparison
-
-| Method | Speed | Cost | Accuracy | Custom Types |
-| ------ | ----- | ---- | -------- | ------------ |
-| `pattern` | Very fast | Free | Medium | Yes (dictionary) |
-| `ml` | Fast | Free | High | Limited |
-| `llm` | Medium | API cost | Highest | Yes (schema) |
+<Tip>
+  **Batch large inputs.** Call `ner.extract_batch(texts, batch_size=10)` rather than looping over individual texts. Batch mode is significantly faster for both ML (GPU batching) and LLM (fewer API round-trips with prompt packing).
+</Tip>
 
 <CardGroup cols={2}>
   <Card title="LLM Providers" icon="microchip" href="llms">

docs/reference/split.md

@@ -1,145 +1,413 @@
 ---
 title: "Split Module"
-description: "15+ text chunking methods including recursive, semantic, entity-aware, and relation-aware splitting."
+description: "15+ text chunking methods including recursive, semantic, entity-aware, relation-aware, code, and structural splitting."
 icon: "scissors"
 ---
 
-`semantica.split` breaks documents into chunks while preserving semantic context — critical for embedding quality in RAG systems and accurate entity extraction in NER pipelines.
+`semantica.split` breaks documents into chunks that preserve semantic context. Chunking quality directly determines downstream accuracy — a poorly chunked document produces bad embeddings, missed entities, and broken relation triplets. Use the right strategy for your content type and pipeline goal.
+
+## Why Chunking Matters
+
+Most LLMs and embedding models have fixed context windows. Documents larger than that window must be split. But naive splitting (every 500 characters, regardless of structure) destroys semantic context:
+
+- An entity mention like "Apple Inc." split across two chunks loses its context in both
+- A relation triplet like "Steve Jobs founded Apple" split at "Steve Jobs" leaves a dangling subject
+- Embedding a chunk that mixes two unrelated topics produces a centroid vector that matches neither
+
+Semantica's chunking methods are designed to avoid these failure modes.
 
 ## What You Get
 
-- **`TextSplitter`** — unified interface for 9+ chunking strategies
-- **Entity-aware chunking** — entity mentions never split across chunk boundaries
-- **Relation-aware chunking** — subject–predicate–object triplets kept intact
-- **Semantic chunking** — split at topic shift boundaries using embedding similarity
-- **`Chunk`** — output object with text, token count, character offsets, and metadata
+<CardGroup cols={2}>
+  <Card title="TextSplitter" icon="scissors">
+    Unified interface for 11 chunking strategies — swap methods without changing downstream code.
+  </Card>
+  <Card title="Semantic Chunking" icon="brain">
+    Embedding-based topic shift detection — splits only when the topic actually changes.
+  </Card>
+  <Card title="Entity-Aware Chunking" icon="user">
+    Entity spans never cross chunk boundaries — guaranteed by boundary adjustment.
+  </Card>
+  <Card title="Relation-Aware Chunking" icon="arrows-left-right">
+    Subject–predicate–object triplets kept within a single chunk for KG pipelines.
+  </Card>
+  <Card title="Code Splitting" icon="code">
+    AST-level boundaries (function, class, method) for source code search and analysis.
+  </Card>
+  <Card title="Chunk Object" icon="box">
+    Output dataclass with text, token count, character offsets, entities, and full metadata.
+  </Card>
+</CardGroup>
 
-## TextSplitter
+## Quick Start
 
-```python
-from semantica.split import TextSplitter
+<Steps>
+  <Step title="Choose a splitting method">
+    ```python
+    from semantica.split import TextSplitter
 
-splitter = TextSplitter(
-    method="semantic_transformer",   # see methods table below
-    chunk_size=1000,                 # target tokens per chunk
-    chunk_overlap=200                # token overlap between adjacent chunks
-)
+    splitter = TextSplitter(
+        method="recursive",   # see Splitting Methods table
+        chunk_size=1000,
+        chunk_overlap=200,
+    )
+    ```
+  </Step>
+  <Step title="Split raw text">
+    ```python
+    chunks = splitter.split(text)
 
-chunks = splitter.split(text)
-for chunk in chunks:
-    print(f"Chunk {chunk.metadata['chunk_index']}: {chunk.text[:80]}...")
-    print(f"  Tokens: {chunk.token_count}")
-```
+    for chunk in chunks:
+        print(f"Chunk {chunk.metadata['chunk_index']} / {chunk.metadata['total_chunks']}")
+        print(f"  Tokens:  {chunk.token_count}")
+        print(f"  Preview: {chunk.text[:80]}...")
+    ```
+  </Step>
+  <Step title="Or split a ParsedDocument">
+    ```python
+    from semantica.parse import DocumentParser
+
+    parser = DocumentParser()
+    parsed = parser.parse("annual_report.pdf")
+
+    splitter = TextSplitter(method="structural")
+    chunks   = splitter.split_document(parsed)
+
+    for chunk in chunks:
+        print(f"[h{chunk.metadata['heading_level']}] {chunk.metadata['section_title']}")
+    ```
+  </Step>
+  <Step title="Batch-split a list of documents">
+    ```python
+    all_chunks = splitter.split_documents(parsed_docs)
+
+    from collections import defaultdict
+    by_source = defaultdict(list)
+    for chunk in all_chunks:
+        by_source[chunk.metadata['source_id']].append(chunk)
+    ```
+  </Step>
+</Steps>
 
 ## Splitting Methods
 
-| Method | Description | Best For |
-| ------ | ----------- | -------- |
-| `recursive` | Split by paragraph → sentence → word (cascading) | General purpose |
-| `semantic_transformer` | Split at semantic topic boundaries via sentence transformer | RAG retrieval |
-| `entity_aware` | Keep entity mentions intact across boundaries | NER pipelines |
-| `relation_aware` | Keep relation triplets intact | KG construction |
-| `sentence` | Split by sentence boundary | Short content |
-| `token` | Split by token count (tiktoken) | LLM context windows |
-| `fixed` | Fixed character count with overlap | Batch processing |
-| `markdown` | Split by Markdown heading hierarchy | Documentation |
-| `code` | Split by function/class/method boundaries | Code analysis |
+| Method | How It Splits | Best For |
+| ------ | ------------- | -------- |
+| `recursive` | Paragraph → sentence → word (cascading fallback) | General-purpose default |
+| `semantic_transformer` | Embeds sentences, splits at cosine similarity drops | RAG — topic coherence matters |
+| `entity_aware` | Adjusts boundaries so entity spans are never cut | NER pipelines |
+| `relation_aware` | Keeps subject–predicate–object triplets within one chunk | KG construction |
+| `sentence` | Language-aware sentence boundary detection (NLTK/spaCy) | Short documents, Q&A |
+| `token` | Exact token count via tiktoken; hard cutoff | LLM context window prep |
+| `fixed` | Fixed character count with overlap; fastest, no NLP | Simple batch jobs |
+| `sliding_window` | Fixed-step window — heavy overlap for dense retrieval | Bi-encoder retrieval (ColBERT, DPR) |
+| `markdown` | Splits at Markdown heading levels (configurable) | Documentation, wikis, MDX |
+| `structural` | Splits at `ParsedDocument.sections` boundaries | Structured PDFs and DOCX |
+| `code` | AST-level splits at function / class / method boundaries | Source code search and analysis |
 
-## Entity-Aware Chunking
+## Choosing a Strategy
 
-Entity mentions are never split across chunk boundaries, preserving context for downstream NER:
+Use this decision tree before picking a method:
 
-```python
-from semantica.split import TextSplitter
-from semantica.semantic_extract import NERExtractor
+- **Source code?** → `code`
+- **Markdown or structured doc with headings?** → `markdown` or `structural`
+- **Building a KG?** → `relation_aware` (keeps triplets intact), then `entity_aware` for pure NER
+- **RAG system where retrieval quality matters most?** → `semantic_transformer`
+- **Dense overlap for bi-encoder retrieval (ColBERT, DPR)?** → `sliding_window`
+- **Preparing prompts for a fixed-window LLM?** → `token`
+- **Fast splitting with no NLP overhead?** → `recursive` or `fixed`
 
-ner = NERExtractor()
-entities = ner.extract(text)
-
-splitter = TextSplitter(method="entity_aware")
-chunks = splitter.split(text, entities=entities)
-# → Each chunk contains only complete entity mentions
-```
-
-## Relation-Aware Chunking
-
-Subject–predicate–object triplets are kept within the same chunk:
+## TextSplitter Constructor
 
 ```python
 from semantica.split import TextSplitter
 
-splitter = TextSplitter(method="relation_aware")
-chunks = splitter.split(text, relationships=relationships)
-# → Triplets are never split across chunk boundaries
-```
-
-## Semantic Chunking
-
-Split at topic shift boundaries detected via embedding similarity:
-
-```python
-from semantica.split import TextSplitter
-from semantica.embeddings import EmbeddingGenerator
-
-embedder = EmbeddingGenerator(model="sentence-transformers")
 splitter = TextSplitter(
-    method="semantic_transformer",
-    embedder=embedder,
-    similarity_threshold=0.7   # split when consecutive sentence similarity drops below this
+    method="semantic_transformer",   # chunking strategy
+    chunk_size=1000,                 # target size in tokens
+    chunk_overlap=200,               # token overlap between adjacent chunks
+    tokenizer="cl100k_base",         # tiktoken encoding (GPT-4 default)
+    min_chunk_size=50,               # discard very short trailing chunks
+    include_metadata=True,           # attach source_id, page_number, section_title
+    language="en",                   # ISO 639-1 — used by sentence boundary detector
 )
-
-chunks = splitter.split(text)
 ```
 
-## Token-Based Chunking
+| Parameter | Type | Default | Description |
+| --------- | ---- | ------- | ----------- |
+| `method` | `str` | `"recursive"` | Chunking strategy — see table above |
+| `chunk_size` | `int` | `1000` | Target size in tokens (characters for `fixed`) |
+| `chunk_overlap` | `int` | `200` | Token overlap between adjacent chunks |
+| `tokenizer` | `str` | `"cl100k_base"` | tiktoken encoding: `"cl100k_base"` (GPT-4), `"p50k_base"` (GPT-3), `"r50k_base"` (Codex) |
+| `min_chunk_size` | `int` | `0` | Discard chunks shorter than this many tokens |
+| `similarity_threshold` | `float` | `0.7` | Cosine similarity cutoff for `semantic_transformer` |
+| `embedder` | `EmbeddingGenerator` | `None` | Custom embedder for `semantic_transformer` |
+| `include_metadata` | `bool` | `True` | Attach `source_id`, `page_number`, `section_title` to each chunk |
+| `language` | `str` | `"en"` | ISO 639-1 language code for sentence boundary detection |
+| `heading_levels` | `list[int]` | `[1, 2, 3]` | Heading levels to split on for `markdown` method |
+| `code_units` | `list[str]` | `["function", "class"]` | AST node types to split on for `code` method |
 
-Use tiktoken for precise token-count control when preparing LLM context windows:
+## Splitting Method Details
 
-```python
-splitter = TextSplitter(
-    method="token",
-    chunk_size=512,              # max tokens per chunk
-    chunk_overlap=50,            # overlap in tokens
-    tokenizer="cl100k_base"      # OpenAI tokenizer
-)
-chunks = splitter.split(text)
-```
+<Tabs>
+  <Tab title="Recursive (default)">
+    Tries paragraph breaks first, then sentence boundaries, then word boundaries — falling back only when the chunk exceeds `chunk_size`:
 
-## Chunk Object
+    ```python
+    splitter = TextSplitter(method="recursive", chunk_size=1000, chunk_overlap=200)
+    chunks   = splitter.split(text)
+    ```
+
+    **Key behaviours:**
+    - Preserves paragraph and sentence structure wherever possible
+    - Falls back gracefully — never produces chunks larger than `chunk_size`
+    - Overlap ensures context continuity across chunk boundaries
+    - Good starting point when you're unsure which method to use
+  </Tab>
+  <Tab title="Semantic">
+    Embeds each sentence, then splits whenever cosine similarity between consecutive sentences drops below `similarity_threshold`. Each chunk talks about one topic:
+
+    ```python
+    from semantica.split import TextSplitter
+    from semantica.embeddings import EmbeddingGenerator
+
+    embedder = EmbeddingGenerator(model="sentence-transformers")
+    splitter = TextSplitter(
+        method="semantic_transformer",
+        embedder=embedder,
+        similarity_threshold=0.7,   # 0.6 = more splits, 0.8 = fewer splits
+        chunk_size=800,
+        chunk_overlap=0,            # not needed — chunks are already coherent
+    )
+    chunks = splitter.split(text)
+    ```
+
+    **Key behaviours:**
+    - Produces variable-length chunks — some topics are short, others long
+    - Requires an embedder — defaults to `sentence-transformers/all-MiniLM-L6-v2` if not set
+    - Slower than `recursive` due to embedding computation; cache embeddings for repeated splits
+    - Best retrieval quality for semantic search — chunks map to single coherent topics
+  </Tab>
+  <Tab title="Entity-Aware">
+    Runs NER first, then adjusts chunk boundaries so no entity mention is split across two chunks:
+
+    ```python
+    from semantica.split import TextSplitter
+    from semantica.semantic_extract import NERExtractor
+    from semantica.llms import Groq
+    import os
+
+    llm      = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+    ner      = NERExtractor(method="llm", llm_provider=llm)
+    entities = ner.extract(text)
+
+    splitter = TextSplitter(method="entity_aware", chunk_size=512, chunk_overlap=50)
+    chunks   = splitter.split(text, entities=entities)
+
+    for chunk in chunks:
+        print(f"Chunk {chunk.metadata['chunk_index']}: {len(chunk.entities)} entities")
+    ```
+
+    **Key behaviours:**
+    - Entity spans in `chunk.entities` are guaranteed to fall entirely within `chunk.text`
+    - Chunk sizes vary slightly from `chunk_size` — boundary adjustments are ≤ one sentence
+    - Works with all entity types: PERSON, ORGANIZATION, LOCATION, DATE, custom types
+  </Tab>
+  <Tab title="Relation-Aware">
+    Keeps subject–predicate–object triplets within the same chunk — critical for KG pipelines:
+
+    ```python
+    from semantica.split import TextSplitter
+    from semantica.semantic_extract import RelationExtractor, NERExtractor
+    from semantica.llms import Groq
+    import os
+
+    llm           = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
+    ner           = NERExtractor(method="llm", llm_provider=llm)
+    rel_extractor = RelationExtractor(method="llm", llm_provider=llm)
+
+    entities      = ner.extract(text)
+    relationships = rel_extractor.extract(text, entities=entities)
+
+    splitter = TextSplitter(method="relation_aware", chunk_size=512)
+    chunks   = splitter.split(text, relationships=relationships)
+
+    for chunk in chunks:
+        print(f"Chunk {chunk.metadata['chunk_index']}: {len(chunk.relationships)} triplets")
+        for rel in chunk.relationships:
+            print(f"  {rel['subject']} —[{rel['predicate']}]→ {rel['object']}")
+    ```
+
+    **Key behaviours:**
+    - Relation triplets in `chunk.relationships` are always fully contained within the chunk
+    - Implies entity-aware behaviour — both entities in a triplet are kept whole too
+    - Best used as the split step in a `Parse → Split → Extract → Build KG` pipeline
+  </Tab>
+  <Tab title="Code">
+    Parses source files with `CodeParser` and splits at AST-level boundaries:
+
+    ```python
+    from semantica.parse import CodeParser
+    from semantica.split import TextSplitter
+
+    parser = CodeParser(extract_comments=True, extract_dependencies=True)
+    parsed = parser.parse("src/pipeline.py")
+
+    splitter = TextSplitter(
+        method="code",
+        code_units=["function", "class"],   # "function" | "class" | "method" | "block"
+        chunk_overlap=0,                     # code units are self-contained
+    )
+    chunks = splitter.split_document(parsed)
+
+    for chunk in chunks:
+        print(f"{chunk.metadata['unit_type']}: {chunk.metadata['unit_name']}")
+        print(f"  Lines {chunk.start_char}–{chunk.end_char}  ({chunk.token_count} tokens)")
+    ```
+
+    **Key behaviours:**
+    - Requires a `ParsedDocument` from `CodeParser` — use `split_document()`
+    - `chunk_overlap=0` recommended — functions and classes are logically self-contained
+    - If a class is too large, it is split at method boundaries automatically
+    - Supported languages: Python, JavaScript, TypeScript, Java, Go, Rust, C, C++, C#, Ruby, PHP, Swift
+  </Tab>
+  <Tab title="Structural & Markdown">
+    ### Structural
+
+    Uses `ParsedDocument.sections` as natural split points — each document section becomes one chunk:
+
+    ```python
+    from semantica.parse import DoclingParser
+    from semantica.split import TextSplitter
+
+    parser = DoclingParser(extract_tables=True)
+    parsed = parser.parse("annual_report.pdf")
+
+    splitter = TextSplitter(method="structural")
+    chunks   = splitter.split_document(parsed)
+
+    for chunk in chunks:
+        level = chunk.metadata['heading_level']
+        title = chunk.metadata['section_title']
+        print(f"{'  ' * (level - 1)}[h{level}] {title}  ({chunk.token_count} tokens)")
+    ```
+
+    ### Markdown
+
+    Splits at Markdown heading boundaries, configurable to specific heading levels:
+
+    ```python
+    splitter = TextSplitter(
+        method="markdown",
+        heading_levels=[1, 2],   # split at # and ## only; ### stays inline
+        chunk_size=800,
+    )
+    chunks = splitter.split(markdown_text)
+    ```
+
+  </Tab>
+</Tabs>
+
+## Chunk Schema
+
+<AccordionGroup>
+  <Accordion title="Chunk dataclass">
 
 ```python
 @dataclass
 class Chunk:
-    text:        str        # chunk text content
-    start_char:  int        # character offset in source document
-    end_char:    int        # character offset in source document
-    token_count: int        # number of tokens
-    metadata:    Dict       # source_id, chunk_index, section_title, page_number, etc.
-    entities:    List[Dict] # entities in chunk (entity_aware splitting only)
+    text:          str         # the chunk's text content
+    start_char:    int         # character offset of start in source document
+    end_char:      int         # character offset of end in source document
+    token_count:   int         # number of tokens (via configured tokenizer)
+    metadata:      Dict        # see metadata fields below
+    entities:      List[Dict]  # entity spans fully contained in this chunk
+    relationships: List[Dict]  # relation triplets fully contained in this chunk
 ```
 
+  </Accordion>
+  <Accordion title="Chunk metadata fields">
+
+| Field | Type | When Present | Description |
+| ----- | ---- | ------------ | ----------- |
+| `source_id` | `str` | Always | ID of the source `ParsedDocument` |
+| `chunk_index` | `int` | Always | Zero-based position within the document |
+| `total_chunks` | `int` | Always | Total chunks produced for this document |
+| `method` | `str` | Always | Splitting method that produced this chunk |
+| `section_title` | `str` | `structural`, `markdown` | Heading text of the containing section |
+| `heading_level` | `int` | `structural`, `markdown` | Depth: 1 = h1, 2 = h2, … |
+| `page_number` | `int` | `structural` (DoclingParser) | Source page number in PDF/DOCX |
+| `unit_type` | `str` | `code` | `"function"` / `"class"` / `"method"` |
+| `unit_name` | `str` | `code` | Name of the code unit, e.g. `"process_batch"` |
+| `language` | `str` | `sentence`, `recursive` | ISO 639-1 code for detected text language |
+| `similarity_score` | `float` | `semantic_transformer` | Cosine similarity to the adjacent chunk |
+
+  </Accordion>
+</AccordionGroup>
+
+## Tokenizer Options
+
+| Tokenizer | Models |
+| --------- | ------ |
+| `cl100k_base` | GPT-4, GPT-3.5-turbo, text-embedding-ada-002 |
+| `p50k_base` | GPT-3 (`text-davinci-003`), Codex |
+| `r50k_base` | GPT-3 (`davinci`) |
+
 ## Pipeline Integration
 
 ```python
 from semantica.pipeline import Pipeline
+from semantica.ingest import FileIngestor
+from semantica.parse import DocumentParser
 from semantica.split import TextSplitter
+from semantica.semantic_extract import NERExtractor
+from semantica.llms import Groq
+import os
+
+llm = Groq(model="llama-3.3-70b-versatile", api_key=os.getenv("GROQ_API_KEY"))
 
 pipeline = Pipeline()
-pipeline.add_step("split", TextSplitter(method="semantic_transformer", chunk_size=512))
-result = pipeline.run(documents)
+pipeline.add_step("ingest",   FileIngestor())
+pipeline.add_step("parse",    DocumentParser())
+pipeline.add_step("split",    TextSplitter(method="semantic_transformer", chunk_size=512))
+pipeline.add_step("extract",  NERExtractor(method="llm", llm_provider=llm))
+
+result = pipeline.run("data/reports/")
 ```
 
+## Tips and Common Pitfalls
+
+<Warning>
+  **`chunk_overlap` too small.** Without overlap, a fact that spans a chunk boundary is invisible in both chunks. A 10–20% overlap relative to `chunk_size` is a safe minimum — for `chunk_size=1000`, set `chunk_overlap=100` to `200`.
+</Warning>
+
+<Warning>
+  **Wrong tokenizer.** If you use `cl100k_base` (GPT-4) but send chunks to a model with a different vocabulary, your token counts will be wrong. Match the tokenizer to your target model.
+</Warning>
+
+<Tip>
+  **Semantic splitting needs enough sentences.** `semantic_transformer` needs several sentences to detect topic shifts. On documents shorter than ~300 words it behaves like `sentence` splitting — use `recursive` instead.
+</Tip>
+
+<Tip>
+  **Code units too coarse.** `code_units=["class"]` on a large codebase produces chunks too big to embed well. Use `["function", "method"]` for more granular, independently useful units.
+</Tip>
+
+<Tip>
+  **Set `min_chunk_size` to avoid fragment chunks.** `min_chunk_size=0` (default) can produce many tiny trailing chunks. Set to ~30–50 tokens to discard fragments that carry no retrieval value.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Parse" icon="file-lines" href="parse">
-    Parse documents before chunking.
+    Parse documents before chunking — produces sections and metadata.
   </Card>
   <Card title="Embeddings" icon="vector-square" href="embeddings">
     Embed chunks for vector search and semantic chunking.
   </Card>
   <Card title="Semantic Extract" icon="magnifying-glass" href="semantic_extract">
-    Extract entities from individual chunks.
+    Extract entities and relations from individual chunks.
   </Card>
   <Card title="Pipeline" icon="gear" href="pipeline">
-    Integrate splitting as a pipeline step.
+    Integrate splitting as a named pipeline step.
   </Card>
 </CardGroup>

docs/reference/triplet_store.md

@@ -8,31 +8,193 @@ icon: "table"
 
 ## What You Get
 
-- **`TripletStore`** — unified interface for all RDF backends
-- **Backends** — Blazegraph, Apache Jena (Fuseki), RDF4J
-- **SPARQL** — full SELECT, CONSTRUCT, ASK, and UPDATE query support
-- **Bulk loading** — efficient batch import for large triple sets
-- **Import / Export** — Turtle, JSON-LD, N-Triples, RDF/XML
+<CardGroup cols={2}>
+  <Card title="TripletStore" icon="server">
+    Unified interface across Blazegraph, Apache Jena (Fuseki), and RDF4J — swap backends with one parameter.
+  </Card>
+  <Card title="InMemoryTripletStore" icon="bolt">
+    Zero-setup in-memory store for unit tests and small datasets — no server, no Docker required.
+  </Card>
+  <Card title="SPARQL" icon="magnifying-glass">
+    Full SELECT, CONSTRUCT, ASK, and UPDATE query support with pagination for large result sets.
+  </Card>
+  <Card title="OWL Reasoning" icon="microchip">
+    Apache Jena supports OWL and RDFS inference natively — subclass and property chain queries automatically resolved.
+  </Card>
+  <Card title="Named Graphs" icon="diagram-project">
+    Isolate triples by source, dataset, or time period using named graph management.
+  </Card>
+  <Card title="Import / Export" icon="file-export">
+    Load and serialize to Turtle, JSON-LD, N-Triples, and RDF/XML with a single method call.
+  </Card>
+</CardGroup>
 
-## Basic Usage
+## Quick Start
+
+<Steps>
+  <Step title="Connect to a backend">
+    ```python
+    from semantica.triplet_store import TripletStore
+
+    store = TripletStore(
+        backend="blazegraph",
+        endpoint="http://localhost:9999/blazegraph/sparql"
+    )
+    ```
+  </Step>
+  <Step title="Add triplets">
+    ```python
+    # Add a single triplet
+    store.add_triplet(
+        subject="http://example.org/apple_inc",
+        predicate="http://example.org/founded_by",
+        obj="http://example.org/steve_jobs"
+    )
+
+    # Bulk load a list of triplets
+    store.add_triplets_bulk(triplets)
+    ```
+  </Step>
+  <Step title="Query with SPARQL">
+    ```python
+    results = store.sparql("""
+        PREFIX ex: <http://example.org/>
+        SELECT ?person ?company WHERE {
+            ?person ex:founded ?company .
+            ?company ex:located_in ex:SiliconValley .
+        }
+    """)
+
+    for row in results:
+        print(row["person"], row["company"])
+    ```
+  </Step>
+  <Step title="Export to file">
+    ```python
+    store.export("output.ttl", format="turtle")
+    store.export("output.nt",  format="nt")
+    store.export("output.xml", format="xml")
+    ```
+  </Step>
+</Steps>
+
+## Backends
+
+<Tabs>
+  <Tab title="Blazegraph">
+    ```python
+    from semantica.triplet_store import TripletStore
+
+    store = TripletStore(
+        backend="blazegraph",
+        endpoint="http://localhost:9999/blazegraph/sparql",
+        namespace="semantica"
+    )
+    ```
+
+    Best for: Wikidata-style workloads, high triple counts, SPARQL 1.1 full support.
+  </Tab>
+  <Tab title="Apache Jena">
+    ```python
+    store = TripletStore(
+        backend="jena",
+        endpoint="http://localhost:3030/dataset/sparql",
+        update_endpoint="http://localhost:3030/dataset/update"
+    )
+    ```
+
+    Best for: General RDF, standard SPARQL, production deployments needing OWL inference.
+
+    **Enable OWL reasoning:**
+
+    ```python
+    store = TripletStore(
+        backend="jena",
+        endpoint="http://localhost:3030/dataset/sparql",
+        update_endpoint="http://localhost:3030/dataset/update",
+        reasoner="OWL",        # "OWL" | "RDFS" | "OWL_MINI" | None
+    )
+
+    # Load an OWL ontology — subclass/property chain inferences are automatic
+    store.import_file("ontology.ttl", format="turtle")
+    store.add_triplets_bulk(data_triplets)
+
+    # Query using inferred relationships
+    results = store.sparql("""
+        SELECT ?person WHERE {
+            ?person a ex:Employee .       # inferred via subClassOf chain
+        }
+    """)
+    ```
+  </Tab>
+  <Tab title="RDF4J">
+    ```python
+    store = TripletStore(
+        backend="rdf4j",
+        server_url="http://localhost:8080/rdf4j-server",
+        repository_id="semantica"
+    )
+    ```
+
+    Best for: Enterprise Java ecosystems, Eclipse Foundation deployments, plugin-based reasoning.
+  </Tab>
+  <Tab title="InMemory">
+    ```python
+    from semantica.triplet_store import InMemoryTripletStore
+
+    store = InMemoryTripletStore()
+
+    store.add_triplet("ex:alice", "ex:knows", "ex:bob")
+    store.add_triplet("ex:bob",   "ex:works_for", "ex:acme")
+
+    results = store.sparql("""
+        SELECT ?person ?company WHERE {
+            ?person ex:works_for ?company .
+        }
+    """)
+
+    # Serialize to string for inspection
+    ttl = store.export_to_string(format="turtle")
+    print(ttl)
+    ```
+
+    `InMemoryTripletStore` shares the same interface as `TripletStore` — swap backends without changing query code.
+
+    Best for: unit tests, CI pipelines, small datasets, zero-infrastructure local exploration.
+  </Tab>
+  <Tab title="Backend Comparison">
+
+    | Backend | License | OWL Reasoning | Hosted Option | Best For |
+    | ------- | ------- | ------------- | ------------- | -------- |
+    | Blazegraph | Open source | No | Self-hosted | Wikidata-style workloads, high triple count |
+    | Apache Jena | Apache 2.0 | Yes (OWL/RDFS) | Self-hosted | General RDF, OWL reasoning, standard SPARQL |
+    | RDF4J | Eclipse 1.0 | Via plugin | Self-hosted or cloud | Enterprise Java ecosystems |
+    | InMemory | Built-in | No | N/A | Unit tests, small graphs, no server required |
+
+  </Tab>
+</Tabs>
+
+## Namespace Prefix Management
+
+Register custom prefixes to keep SPARQL queries readable:
 
 ```python
-from semantica.triplet_store import TripletStore
+from semantica.triplet_store import TripletStore, NamespacePrefixManager
 
-store = TripletStore(
-    backend="blazegraph",
-    endpoint="http://localhost:9999/blazegraph/sparql"
-)
+ns = NamespacePrefixManager()
+ns.register("ex",     "http://example.org/")
+ns.register("schema", "https://schema.org/")
+ns.register("owl",    "http://www.w3.org/2002/07/owl#")
 
-# Add a single triplet
-store.add_triplet(
-    subject="http://example.org/apple_inc",
-    predicate="http://example.org/founded_by",
-    obj="http://example.org/steve_jobs"
-)
+store = TripletStore(backend="jena", endpoint="...", namespace_manager=ns)
 
-# Bulk load a list of triplets
-store.add_triplets_bulk(triplets)
+# Registered prefixes are automatically prepended to every SPARQL query
+results = store.sparql("""
+    SELECT ?company WHERE {
+        ?person ex:works_for ?company ;
+                schema:name  "Alice" .
+    }
+""")
 ```
 
 ## SPARQL Queries
@@ -47,9 +209,6 @@ results = store.sparql("""
     }
 """)
 
-for row in results:
-    print(row["person"], row["company"])
-
 # CONSTRUCT — returns a graph of matched triples
 graph = store.sparql_construct("""
     PREFIX ex: <http://example.org/>
@@ -76,54 +235,29 @@ store.sparql_update("""
 """)
 ```
 
-## Backends
+## SPARQL Result Pagination
+
+For large result sets, paginate with LIMIT and OFFSET:
 
 ```python
-# Blazegraph — open source, SPARQL 1.1
-store = TripletStore(
-    backend="blazegraph",
-    endpoint="http://localhost:9999/blazegraph/sparql",
-    namespace="semantica"
-)
+page_size = 1000
+offset    = 0
 
-# Apache Jena Fuseki — open source, widely used
-store = TripletStore(
-    backend="jena",
-    endpoint="http://localhost:3030/dataset/sparql",
-    update_endpoint="http://localhost:3030/dataset/update"
-)
-
-# RDF4J — enterprise-grade, Eclipse Foundation
-store = TripletStore(
-    backend="rdf4j",
-    server_url="http://localhost:8080/rdf4j-server",
-    repository_id="semantica"
-)
+while True:
+    results = store.sparql(f"""
+        SELECT ?s ?p ?o WHERE {{
+            ?s ?p ?o .
+        }}
+        ORDER BY ?s
+        LIMIT {page_size} OFFSET {offset}
+    """)
+    if not results:
+        break
+    process_batch(results)
+    offset += page_size
 ```
 
-## Backend Comparison
-
-| Backend | License | Query Language | Best For |
-| ------- | ------- | -------------- | -------- |
-| Blazegraph | Open source | SPARQL 1.1 | Wikidata-style workloads |
-| Apache Jena | Apache 2.0 | SPARQL 1.1 | General RDF, OWL reasoning |
-| RDF4J | Eclipse 1.0 | SPARQL 1.1 | Enterprise, Java ecosystems |
-
-## Import and Export
-
-```python
-# Import from file
-store.import_file("ontology.ttl",  format="turtle")
-store.import_file("data.jsonld",   format="json-ld")
-store.import_file("triples.nt",    format="nt")
-
-# Export to file
-store.export("output.ttl", format="turtle")
-store.export("output.nt",  format="nt")
-store.export("output.xml", format="xml")
-```
-
-## Graph Management
+## Named Graph Management
 
 ```python
 # Named graphs — store triples in isolated contexts
@@ -168,6 +302,32 @@ store.import_file("output.ttl", format="turtle")
 results = store.sparql("SELECT * WHERE { ?s ?p ?o } LIMIT 10")
 ```
 
+## Tips and Common Pitfalls
+
+<Tip>
+  **Use `InMemoryTripletStore` for unit tests, Jena or Blazegraph for production.** The in-memory backend requires zero server setup and is safe for CI. It does not persist across process restarts — switch to a server-backed store before deploying. No code changes needed, just the `backend=` parameter.
+</Tip>
+
+<Warning>
+  **Paginate large SPARQL result sets.** A `SELECT * WHERE { ?s ?p ?o }` against a million-triple store can return gigabytes of data. Always include `LIMIT` and `OFFSET` in exploratory queries, and iterate with `page_size` when you need full coverage. Unbounded queries against large stores will OOM or timeout.
+</Warning>
+
+<Tip>
+  **Use named graphs to isolate sources.** `store.add_triplet(..., graph="http://example.org/source_A")` puts triples into a named graph. You can then query just that source, merge selectively, or clear it without touching other data — far safer than mixing all triples into the default graph.
+</Tip>
+
+<Tip>
+  **Register namespace prefixes before querying.** `NamespacePrefixManager` lets you write `?s ex:name ?o` instead of `?s <http://example.org/name> ?o`. Without prefixes, SPARQL queries against domain ontologies become unreadable and error-prone.
+</Tip>
+
+<Warning>
+  **Enable OWL reasoning only when you need it.** `reasoner="OWL"` significantly increases query planning overhead. For simple triple lookups or SPARQL SELECT queries, leave reasoning off (`reasoner=None`) and enable it only for queries that depend on class hierarchies or property chains.
+</Warning>
+
+<Tip>
+  **Export to Turtle before migrating backends.** If you need to move from Jena to Blazegraph (or any other store), `store.export("dump.ttl", format="turtle")` produces a portable file that any SPARQL store can import. Don't rely on backend-specific dump formats.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Export" icon="file-export" href="export">
     Export knowledge graphs to RDF formats.

docs/reference/utils.md

@@ -8,33 +8,61 @@ icon: "wrench"
 
 ## What You Get
 
-- **Logging** — structured logging with `@log_performance` decorator and quality metrics
-- **Validation** — `validate_entity`, `validate_config` with a typed `ValidationError`
-- **Progress tracking** — `track_progress` wraps any iterable with console, Jupyter, or file output
-- **Helper functions** — `clean_text`, `hash_data`, `safe_filename`
-- **Exception hierarchy** — `SemanticaError` → `ValidationError`, `ProcessingError`
+<CardGroup cols={2}>
+  <Card title="Logging" icon="scroll">
+    Structured logging with `@log_performance` decorator and quality metrics via environment variables.
+  </Card>
+  <Card title="Validation" icon="shield-check">
+    `validate_entity` and `validate_config` with a typed `ValidationError` carrying field and value context.
+  </Card>
+  <Card title="Progress Tracking" icon="bars-progress">
+    `track_progress` wraps any iterable — auto-detects console vs Jupyter for the right renderer.
+  </Card>
+  <Card title="Helper Functions" icon="wrench">
+    `clean_text`, `hash_data`, `safe_filename`, and nested dict utilities used throughout the framework.
+  </Card>
+  <Card title="Exception Hierarchy" icon="triangle-exclamation">
+    `SemanticaError` → `ValidationError`, `ProcessingError` — typed exceptions for targeted recovery.
+  </Card>
+  <Card title="File Utilities" icon="file">
+    `read_json_file` with `ProcessingError` on failure — no boilerplate try/except around JSON I/O.
+  </Card>
+</CardGroup>
 
 ## Logging
 
-```python
-from semantica.utils import setup_logging, get_logger, log_performance
+<Steps>
+  <Step title="Initialize logging at application startup">
+    ```python
+    from semantica.utils import setup_logging, get_logger
 
-setup_logging(level="INFO")   # "DEBUG" | "INFO" | "WARNING" | "ERROR"
-logger = get_logger(__name__)
+    setup_logging(level="INFO")   # "DEBUG" | "INFO" | "WARNING" | "ERROR"
+    logger = get_logger(__name__)
+    ```
+  </Step>
+  <Step title="Instrument expensive functions with the performance decorator">
+    ```python
+    from semantica.utils import log_performance, log_execution_time
 
-@log_performance
-def process_data(data):
-    logger.info(f"Processing {len(data)} items")
-    # Decorator automatically logs function name, duration, and any exception
-```
+    @log_performance
+    def process_data(data):
+        logger.info(f"Processing {len(data)} items")
+        # Logs function name, duration, and any exception automatically
 
-Configure via environment variables:
-
-```bash
-export SEMANTICA_LOG_LEVEL=DEBUG
-export SEMANTICA_LOG_FORMAT=json     # "json" | "text"
-export SEMANTICA_PROGRESS_BAR=true
-```
+    @log_execution_time
+    def expensive_step(data):
+        ...
+    # Logs: "expensive_step completed in 2.34s"
+    ```
+  </Step>
+  <Step title="Configure via environment variables">
+    ```bash
+    export SEMANTICA_LOG_LEVEL=DEBUG
+    export SEMANTICA_LOG_FORMAT=json     # "json" | "text"
+    export SEMANTICA_PROGRESS_BAR=true
+    ```
+  </Step>
+</Steps>
 
 ## Validation
 
@@ -72,9 +100,8 @@ for item in track_progress(items, desc="Processing documents"):
 ```
 
 Supports:
-
 - **Console** — tqdm progress bar with ETA
-- **Jupyter** — notebook-compatible widget
+- **Jupyter** — notebook-compatible widget (auto-detected)
 - **File** — write progress to a log file
 
 ## Helper Functions
@@ -83,17 +110,46 @@ Supports:
 from semantica.utils import clean_text, hash_data, safe_filename
 
 # Normalize whitespace and strip control characters
-clean  = clean_text("  Hello   World  ")   # → "Hello World"
+clean = clean_text("  Hello   World  ")     # → "Hello World"
 
-# Deterministic SHA-256 hash of any serializable object
-uid    = hash_data({"key": "value"})        # → hex digest string
+# Deterministic SHA-256 hash of any JSON-serializable object
+uid   = hash_data({"key": "value"})         # → hex digest string
 
 # Sanitize a string for use as a filename
-fname  = safe_filename("My File?.txt")      # → "My_File_.txt"
+fname = safe_filename("My File?.txt")       # → "My_File_.txt"
+```
+
+## Nested Dict Utilities
+
+Helper functions for deep configuration access — used extensively inside `Config` and `ConfigManager`:
+
+```python
+from semantica.utils import get_nested_value, set_nested_value, merge_dicts
+
+config = {
+    "processing": {"batch_size": 32, "max_workers": 4},
+    "llm":        {"provider": "groq", "model": "llama-3.3-70b-versatile"},
+}
+
+# Dot-notation read — returns default if key path is absent
+batch = get_nested_value(config, "processing.batch_size", default=16)
+# → 32
+
+# Dot-notation write
+set_nested_value(config, "processing.batch_size", 64)
+
+# Deep merge — nested keys are merged recursively
+base      = {"a": {"x": 1, "y": 2}, "b": 3}
+overrides = {"a": {"y": 99, "z": 4}, "c": 5}
+merged    = merge_dicts(base, overrides, deep=True)
+# → {"a": {"x": 1, "y": 99, "z": 4}, "b": 3, "c": 5}
 ```
 
 ## Exception Hierarchy
 
+<AccordionGroup>
+  <Accordion title="Exception types and when they're raised">
+
 ```python
 from semantica.utils import SemanticaError, ValidationError, ProcessingError
 
@@ -101,7 +157,7 @@ try:
     run_pipeline(data)
 except ValidationError as e:
     # Input data did not pass schema validation
-    logger.error(f"Validation failed: {e}")
+    logger.error(f"Validation failed at field '{e.field}': {e.message}")
 except ProcessingError as e:
     # Failure during extraction or graph construction
     logger.error(f"Processing failed at step {e.step}: {e}")
@@ -116,6 +172,40 @@ except SemanticaError as e:
 | `ValidationError` | Input data failed schema or type validation |
 | `ProcessingError` | Failure during extraction, graph build, or pipeline step |
 
+  </Accordion>
+</AccordionGroup>
+
+## File Utilities
+
+```python
+from semantica.utils import read_json_file
+
+# Read and parse a JSON file — raises ProcessingError on failure
+config = read_json_file("config.json")
+```
+
+## Tips and Common Pitfalls
+
+<Warning>
+  **Call `setup_logging(level="INFO")` once at application startup.** Without it, Semantica falls back to Python's root logger, which may be silent or misconfigured. Call it before importing other Semantica modules to capture initialization messages.
+</Warning>
+
+<Tip>
+  **Use `@log_performance` on expensive functions.** The decorator logs function name, duration, and any raised exception automatically — no manual `time.time()` bookkeeping needed. Essential for profiling multi-step pipelines where one step is a hidden bottleneck.
+</Tip>
+
+<Tip>
+  **`hash_data()` is deterministic across runs.** Given the same input dict (any JSON-serializable object), `hash_data()` always returns the same SHA-256 hex string — suitable as a cache key or idempotency token in pipeline steps.
+</Tip>
+
+<Tip>
+  **Catch `SemanticaError` as the broadest exception net.** All framework errors inherit from `SemanticaError`, so `except SemanticaError` catches validation failures, processing errors, and everything in between. Use specific subclasses for targeted recovery logic.
+</Tip>
+
+<Tip>
+  **`track_progress` auto-detects Jupyter.** In a terminal it renders a tqdm progress bar; in a Jupyter notebook it renders an interactive widget. You don't need to check the environment — the same call works in both.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Core" icon="gear" href="core">
     Framework orchestration that uses Utils internally.

docs/reference/vector_store.md

@@ -8,33 +8,70 @@ icon: "database"
 
 ## What You Get
 
-- **`VectorStore`** — unified interface across all backends
-- **Backends** — FAISS, Pinecone, Weaviate, Qdrant, Milvus, PgVector, in-memory
-- **Hybrid search** — combine dense vector similarity with sparse keyword/metadata filtering
-- **Metadata filtering** — rich filter expressions: `eq`, `ne`, `gt`, `lt`, `in`, `contains`, `$and`, `$or`
-- **Namespace isolation** — multi-tenant support via isolated namespaces
-- **Batch operations** — bulk add, delete, and metadata updates
+<CardGroup cols={2}>
+  <Card title="VectorStore" icon="database">
+    Unified interface across FAISS, Pinecone, Weaviate, Qdrant, Milvus, and PgVector.
+  </Card>
+  <Card title="HybridSearch" icon="magnifying-glass">
+    Combine dense vector similarity with sparse keyword/BM25 filtering and configurable fusion strategies.
+  </Card>
+  <Card title="MetadataStore" icon="table">
+    Rich metadata indexing and schema management — query by field values without a vector.
+  </Card>
+  <Card title="NamespaceManager" icon="folder-tree">
+    Multi-tenant namespace isolation — structural separation, not just metadata filters.
+  </Card>
+  <Card title="Batch Operations" icon="layer-group">
+    Bulk add, delete, and metadata updates — automatically chunked for memory efficiency.
+  </Card>
+  <Card title="FAISS Index Types" icon="chart-scatter">
+    Flat, IVF, HNSW, and PQ index types with full configuration control.
+  </Card>
+</CardGroup>
 
-## Basic Usage
+## Quick Start
 
-```python
-from semantica.vector_store import VectorStore
+<Steps>
+  <Step title="Create a vector store">
+    ```python
+    from semantica.vector_store import VectorStore
 
-# In-memory (development)
-store = VectorStore(backend="inmemory", dimension=768)
+    # In-memory (development)
+    store = VectorStore(backend="inmemory", dimension=768)
 
-# FAISS (local, production)
-store = VectorStore(backend="faiss", dimension=768, index_path="store.faiss")
-
-# Add vectors
-store.add_vectors(embeddings=embeddings, ids=["doc1", "doc2"], metadata=[{}, {}])
-
-# Semantic search
-results = store.search(query_vector, top_k=10)
-for r in results:
-    print(f"{r['id']} — score: {r['score']:.3f}")
-    print(f"  metadata: {r['metadata']}")
-```
+    # FAISS (local production — persists to disk)
+    store = VectorStore(backend="faiss", dimension=768, index_path="store.faiss")
+    ```
+  </Step>
+  <Step title="Add vectors">
+    ```python
+    store.add_vectors(
+        embeddings=embeddings,
+        ids=["doc1", "doc2"],
+        metadata=[{"title": "Document 1"}, {"title": "Document 2"}]
+    )
+    ```
+  </Step>
+  <Step title="Search by semantic similarity">
+    ```python
+    results = store.search(query_vector, top_k=10)
+    for r in results:
+        print(f"{r['id']} — score: {r['score']:.3f}")
+        print(f"  metadata: {r['metadata']}")
+    ```
+  </Step>
+  <Step title="Filter results by metadata">
+    ```python
+    # Equality, range, and set filters
+    results = store.search(query_vector, filters={
+        "$and": [
+            {"category": "research"},
+            {"year": {"$gte": 2022}}
+        ]
+    })
+    ```
+  </Step>
+</Steps>
 
 ## Backends
 
@@ -45,7 +82,7 @@ for r in results:
 store = VectorStore(
     backend="faiss",
     dimension=768,
-    index_type="IVF",       # "Flat" | "IVF" | "HNSW"
+    index_type="IVF",       # "Flat" | "IVF" | "HNSW" | "PQ"
     index_path="store.faiss"
 )
 ```
@@ -197,6 +234,150 @@ store.update_metadata("doc1", {"status": "archived", "reviewed": True})
 | PgVector | PostgreSQL | No | Limited | Postgres-native integration |
 | In-memory | Process | No | No | Development, testing |
 
+## HybridSearch
+
+`HybridSearch` is the low-level class behind `store.hybrid_search()` — use it directly when you need custom result fusion logic:
+
+```python
+from semantica.vector_store import HybridSearch, VectorStore
+
+store  = VectorStore(backend="faiss", dimension=768)
+hybrid = HybridSearch(vector_store=store)
+
+results = hybrid.search(
+    query_vector=query_embedding,
+    query_text="machine learning frameworks",
+    top_k=20,
+    vector_weight=0.7,       # weight for vector similarity leg
+    keyword_weight=0.3,      # weight for BM25/keyword leg
+    fusion="rrf",            # "rrf" (Reciprocal Rank Fusion) | "weighted_avg"
+    filters={"category": "research", "year": {"$gte": 2022}},
+    deduplicate=True,
+)
+
+for r in results:
+    print(f"{r['id']}  vector_score={r['vector_score']:.3f}  final_score={r['score']:.3f}")
+```
+
+| Fusion strategy | Description |
+| --------------- | ----------- |
+| `rrf` | Reciprocal Rank Fusion — rank-based combination, robust to score scale differences |
+| `weighted_avg` | Weighted average of normalised scores — requires `vector_weight` + `keyword_weight` = 1.0 |
+
+## MetadataStore
+
+`MetadataStore` manages structured metadata attached to vectors — query by field values without a vector:
+
+```python
+from semantica.vector_store import MetadataStore
+
+meta_store = MetadataStore()
+
+meta_store.register_schema({
+    "author":   "str",
+    "year":     "int",
+    "category": "str",
+    "score":    "float",
+})
+
+meta_store.add("doc1", {"author": "Alice", "year": 2024, "category": "research"})
+meta_store.add("doc2", {"author": "Bob",   "year": 2023, "category": "review"})
+
+results = meta_store.filter({"category": "research", "year": {"$gte": 2023}})
+meta    = meta_store.get("doc1")
+meta_store.update("doc1", {"score": 0.92})
+```
+
+## NamespaceManager
+
+Isolates vector collections per tenant, project, or model version:
+
+```python
+from semantica.vector_store import NamespaceManager, VectorStore
+
+base_store = VectorStore(backend="faiss", dimension=768)
+ns_manager  = NamespaceManager(vector_store=base_store)
+
+ns_manager.create_namespace("tenant_a", description="Customer A data")
+ns_manager.create_namespace("tenant_b", description="Customer B data")
+
+ns_manager.add_vectors("tenant_a", embeddings_a, ids_a, metadata_a)
+ns_manager.add_vectors("tenant_b", embeddings_b, ids_b, metadata_b)
+
+# Search is scoped — tenant_a never sees tenant_b's data
+results = ns_manager.search("tenant_a", query_vector, top_k=10)
+
+for ns in ns_manager.list_namespaces():
+    print(f"{ns['name']}: {ns['vector_count']} vectors")
+
+ns_manager.delete_namespace("tenant_a")
+```
+
+## FAISS Index Type Reference
+
+| Index | Memory | Speed | Accuracy | When to Use |
+| ----- | ------ | ----- | -------- | ----------- |
+| `Flat` | High | Slow | Exact (100%) | < 100K vectors, correctness critical |
+| `IVF` | Medium | Fast | ~95–98% | 100K–10M vectors, good balance |
+| `HNSW` | Medium-High | Very fast | ~97–99% | Low latency, production retrieval |
+| `PQ` | Low | Fast | ~90–95% | Millions of vectors, memory-constrained |
+
+```python
+# Flat — brute-force exact search
+store = VectorStore(backend="faiss", dimension=768, index_type="Flat")
+
+# IVF — inverted file index with nlist clusters
+store = VectorStore(backend="faiss", dimension=768, index_type="IVF", nlist=100)
+
+# HNSW — hierarchical navigable small world graph
+store = VectorStore(backend="faiss", dimension=768, index_type="HNSW", M=16, ef_construction=200)
+
+# PQ — product quantization for memory efficiency
+store = VectorStore(backend="faiss", dimension=768, index_type="PQ", m=8)
+```
+
+## Similarity Metrics
+
+| Metric | Constructor arg | Distance → Similarity | Best For |
+| ------ | --------------- | --------------------- | -------- |
+| Cosine | `metric="cosine"` | `1 - cosine_distance` | Text, embeddings |
+| L2 (Euclidean) | `metric="l2"` | `1 / (1 + distance)` | Image features |
+| Inner Product | `metric="ip"` | raw dot product | Recommendation systems |
+
+```python
+store = VectorStore(backend="faiss", dimension=768, metric="cosine")
+```
+
+## Tips and Common Pitfalls
+
+<Warning>
+  **Match vector dimension to your embedding model.** The `dimension` parameter must exactly match your embedding model's output size — `all-MiniLM-L6-v2` = 384, `all-mpnet-base-v2` = 768, `bge-large-en-v1.5` = 1024. A mismatch raises an error at insert time, not at store creation.
+</Warning>
+
+<Tip>
+  **Use `Flat` index only for small datasets.** Flat (brute-force) search has perfect recall but O(n) query time. At 500K+ vectors, switch to `IVF` or `HNSW` — they sacrifice less than 5% recall for 100–1000x speedup.
+</Tip>
+
+<Warning>
+  **Don't search without normalizing first.** If you disabled `normalize=True` in `EmbeddingGenerator`, compute cosine similarity with `metric="cosine"` (which normalizes internally). Raw dot product on un-normalized vectors produces incorrect similarity rankings.
+</Warning>
+
+<Tip>
+  **Use `hybrid_search` for precision-sensitive workloads.** Pure vector search finds semantically similar results but may miss keyword matches important to the user. Hybrid search (vector + BM25) combines both signals — especially valuable for domain-specific terminology.
+</Tip>
+
+<Tip>
+  **Use `NamespaceManager` for multi-tenant applications.** Storing all tenants' vectors in the same collection and filtering by metadata at query time is slow and leaks data if a filter is accidentally omitted. Namespace isolation is both faster (smaller search space) and safer (structural isolation).
+</Tip>
+
+<Warning>
+  **Persist FAISS indexes to disk.** `VectorStore(backend="faiss", index_path="store.faiss")` saves the index to disk on each write. Without a path, the index is in-memory only and is lost on process exit.
+</Warning>
+
+<Tip>
+  **Update metadata without re-embedding.** `store.update_metadata(id, {...})` changes attached fields (status, tags, review date) without re-running the embedding model. Use this for state changes that don't affect semantic content.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Embeddings" icon="vector-square" href="embeddings">
     Generate the vectors stored here.

docs/reference/visualization.md

@@ -8,130 +8,240 @@ icon: "chart-bar"
 
 ## What You Get
 
-- **`GraphVisualizer`** — interactive HTML (PyVis) and static image (Matplotlib) graph rendering
-- **`OntologyVisualizer`** — class hierarchy and property relationship visualization
-- **`EmbeddingVisualizer`** — UMAP, t-SNE, and PCA dimensionality reduction plots
-- **`TemporalVisualizer`** — timeline views and animated graph evolution
-- **`DistanceVisualizer`** — ego-mode neighborhood views and distance matrix heatmaps (v0.5.0)
+<CardGroup cols={2}>
+  <Card title="GraphVisualizer" icon="diagram-project">
+    Interactive HTML (PyVis) and static image (Matplotlib) graph rendering with layout options.
+  </Card>
+  <Card title="OntologyVisualizer" icon="sitemap">
+    Class hierarchy and property relationship visualization from any OntologyManager.
+  </Card>
+  <Card title="EmbeddingVisualizer" icon="vector-square">
+    UMAP, t-SNE, and PCA dimensionality reduction plots for embedding cluster analysis.
+  </Card>
+  <Card title="TemporalVisualizer" icon="clock">
+    Timeline views, animated evolution, snapshot comparison, and temporal pattern highlights.
+  </Card>
+  <Card title="DistanceVisualizer (v0.5.0)" icon="circle-nodes">
+    Ego-mode neighborhood views and N×N distance matrix heatmaps from Distance Intelligence.
+  </Card>
+  <Card title="AnalyticsVisualizer" icon="chart-bar">
+    Centrality rankings, community-colored graphs, and degree distribution histograms.
+  </Card>
+</CardGroup>
 
-## GraphVisualizer
+## Quick Start
+
+<Steps>
+  <Step title="Render a knowledge graph">
+    ```python
+    from semantica.visualization import GraphVisualizer
+
+    viz = GraphVisualizer()
+
+    # Interactive HTML — opens in browser, supports hover and click
+    viz.visualize(graph, output="graph.html")
+    ```
+  </Step>
+  <Step title="Apply layout and color options">
+    ```python
+    viz.visualize(
+        graph,
+        output="graph.html",
+        layout="force_directed",   # "force_directed" | "hierarchical" | "circular" | "spring"
+        node_color_by="type",      # color nodes by entity type attribute
+        edge_label="relation",     # show edge relationship labels
+        color_scheme="vibrant",    # color palette — see Color Schemes section
+        max_nodes=500,             # limit rendering for large graphs
+    )
+    ```
+  </Step>
+  <Step title="Export to static formats">
+    ```python
+    # Static PNG — for reports and embedding in documents
+    viz.visualize(graph, output="graph.png", dpi=150)
+
+    # Vector SVG — for publications and scalable diagrams
+    viz.visualize(graph, output="graph.svg")
+
+    # PDF — for print or compliance reports
+    viz.visualize(graph, output="graph.pdf")
+    ```
+  </Step>
+</Steps>
+
+## Visualizers
+
+<Tabs>
+  <Tab title="GraphVisualizer">
+    Interactive and static knowledge graph rendering:
+
+    ```python
+    from semantica.visualization import GraphVisualizer
+
+    viz = GraphVisualizer()
+
+    # Interactive HTML
+    viz.visualize(graph, output="graph.html")
+
+    # Static PNG with custom DPI
+    viz.visualize(graph, output="graph.png", backend="matplotlib", dpi=150)
+
+    # Display inline (Jupyter or default browser)
+    viz.show(graph)
+    ```
+
+    **Layout options:**
+
+    | Layout | Description | Best For |
+    | ------ | ----------- | -------- |
+    | `force_directed` | Physics simulation — clusters emerge naturally | General graphs |
+    | `hierarchical` | Top-down tree layout | Taxonomies, org charts |
+    | `circular` | Nodes on a circle, edges as chords | Small dense graphs |
+    | `spring` | Spring-force layout (Fruchterman-Reingold) | Medium graphs |
+  </Tab>
+  <Tab title="OntologyVisualizer">
+    Visualize class hierarchies and property relationships:
+
+    ```python
+    from semantica.visualization import OntologyVisualizer
+
+    viz = OntologyVisualizer()
+
+    # Full ontology graph — classes, properties, and constraints
+    viz.visualize(ontology, output="ontology.html")
+
+    # Class hierarchy only — cleaner for large ontologies
+    viz.visualize_hierarchy(ontology, output="hierarchy.html")
+    ```
+  </Tab>
+  <Tab title="EmbeddingVisualizer">
+    Project high-dimensional embeddings into 2D for cluster analysis:
+
+    ```python
+    from semantica.visualization import EmbeddingVisualizer
+
+    viz = EmbeddingVisualizer()
+
+    viz.visualize(
+        embeddings=embeddings,
+        labels=labels,
+        output="embeddings.html",
+        method="umap",    # "umap" | "tsne" | "pca"
+    )
+    ```
+
+    | Method | Speed | Preserves | Best For |
+    | ------ | ----- | --------- | -------- |
+    | `umap` | Fast | Global + local structure | Large datasets, cluster discovery |
+    | `tsne` | Medium | Local structure | Tight cluster separation |
+    | `pca` | Very fast | Variance | Quick overview, linear structure |
+  </Tab>
+  <Tab title="TemporalVisualizer">
+    Visualize how a knowledge graph changes over time:
+
+    ```python
+    from semantica.visualization import TemporalVisualizer
+    from datetime import datetime
+
+    viz = TemporalVisualizer()
+
+    # Static timeline of additions and removals
+    viz.visualize_timeline(temporal_kg, output="timeline.html")
+
+    # Animated evolution — one frame per time step
+    viz.animate(temporal_kg, output="evolution.html", fps=2)
+
+    # Side-by-side snapshot comparison
+    snap_a = temporal_kg.at(datetime(2020, 1, 1))
+    snap_b = temporal_kg.at(datetime(2023, 1, 1))
+    viz.compare_snapshots(snap_a, snap_b, output="snapshot_diff.html")
+
+    # Pattern visualization — highlight recurring temporal patterns
+    viz.visualize_patterns(temporal_kg, pattern_type="recurrence", output="patterns.html")
+    ```
+  </Tab>
+  <Tab title="DistanceVisualizer (v0.5.0)">
+    Semantic neighborhood and distance matrix visualization from Distance Intelligence:
+
+    ```python
+    from semantica.visualization import DistanceVisualizer
+
+    viz = DistanceVisualizer()
+
+    # Ego-mode: neighborhood of one node colored by distance band
+    viz.visualize_ego(
+        graph,
+        center_node="Apple Inc.",
+        output="ego.html",
+        radius=0.5,    # semantic distance radius
+    )
+
+    # N×N distance matrix heatmap
+    viz.visualize_distance_matrix(
+        matrix=distance_matrix,
+        labels=node_labels,
+        output="distance_heatmap.html",
+    )
+    ```
+  </Tab>
+  <Tab title="AnalyticsVisualizer">
+    Visualize graph analytics results — centrality, communities, and degree distribution:
+
+    ```python
+    from semantica.visualization import AnalyticsVisualizer
+    from semantica.kg import CentralityCalculator, CommunityDetector
+
+    calc       = CentralityCalculator()
+    centrality = calc.calculate_all_centrality(kg)
+
+    detector    = CommunityDetector()
+    communities = detector.detect_communities(kg, algorithm="louvain")
+
+    viz = AnalyticsVisualizer()
+
+    # Bar chart of top-N nodes by centrality measure
+    viz.visualize_centrality(centrality, metric="pagerank", top_k=20, output="centrality.html")
+
+    # Community-colored graph
+    viz.visualize_communities(kg, communities, output="communities.html")
+
+    # Degree distribution histogram
+    viz.visualize_degree_distribution(kg, output="degree_dist.html")
+
+    # Combined analytics dashboard
+    viz.visualize_analytics_dashboard(
+        kg, centrality=centrality, communities=communities,
+        output="analytics_dashboard.html",
+    )
+    ```
+  </Tab>
+</Tabs>
+
+## Color Schemes
+
+All visualizers accept a `color_scheme` parameter:
 
 ```python
-from semantica.visualization import GraphVisualizer
-
-viz = GraphVisualizer()
-
-# Interactive HTML — opens in browser, supports hover and click
-viz.visualize(graph, output="graph.html")
-
-# Static image — for reports and export
-viz.visualize(graph, output="graph.png", backend="matplotlib")
-
-# Display inline (Jupyter or default browser)
-viz.show(graph)
+viz.visualize(graph, output="graph.html", color_scheme="vibrant")
 ```
 
-### Layout and Styling Options
-
-```python
-viz.visualize(
-    graph,
-    output="graph.html",
-    layout="force_directed",   # "force_directed" | "hierarchical" | "circular" | "spring"
-    node_color_by="type",      # color nodes by entity type attribute
-    edge_label="relation",     # show edge relationship labels
-    max_nodes=500              # limit rendering for large graphs
-)
-```
-
-### Layout Options
-
-| Layout | Description | Best For |
+| Scheme | Description | Best For |
 | ------ | ----------- | -------- |
-| `force_directed` | Physics simulation — clusters emerge naturally | General graphs |
-| `hierarchical` | Top-down tree layout | Taxonomies, org charts |
-| `circular` | Nodes on a circle, edges as chords | Small dense graphs |
-| `spring` | Spring-force layout (Fruchterman-Reingold) | Medium graphs |
+| `default` | Blue-grey palette | General use |
+| `vibrant` | High-contrast, saturated colours | Presentations |
+| `pastel` | Soft, muted tones | Light backgrounds |
+| `dark` | Dark background with bright nodes | Dark-mode dashboards |
+| `light` | White background, thin edges | Publications, print |
+| `colorblind` | Okabe-Ito safe palette | Accessibility |
 
-## OntologyVisualizer
+## Export Formats
 
-Visualize class hierarchies and property relationships:
-
-```python
-from semantica.visualization import OntologyVisualizer
-
-viz = OntologyVisualizer()
-
-# Full ontology graph
-viz.visualize(ontology, output="ontology.html")
-
-# Class hierarchy only
-viz.visualize_hierarchy(ontology, output="hierarchy.html")
-```
-
-## EmbeddingVisualizer
-
-Project high-dimensional embeddings into 2D for cluster analysis:
-
-```python
-from semantica.visualization import EmbeddingVisualizer
-
-viz = EmbeddingVisualizer()
-
-viz.visualize(
-    embeddings=embeddings,
-    labels=labels,
-    output="embeddings.html",
-    method="umap"       # "umap" | "tsne" | "pca"
-)
-```
-
-| Method | Speed | Preserves | Best For |
-| ------ | ----- | --------- | -------- |
-| `umap` | Fast | Global + local structure | Large datasets, cluster discovery |
-| `tsne` | Medium | Local structure | Tight cluster separation |
-| `pca` | Very fast | Variance | Quick overview, linear structure |
-
-## TemporalVisualizer
-
-Visualize how a knowledge graph changes over time:
-
-```python
-from semantica.visualization import TemporalVisualizer
-
-viz = TemporalVisualizer()
-
-# Static timeline of additions and removals
-viz.visualize_timeline(temporal_kg, output="timeline.html")
-
-# Animated evolution — one frame per time step
-viz.animate(temporal_kg, output="evolution.html", fps=2)
-```
-
-## DistanceVisualizer (v0.5.0)
-
-Semantic neighborhood and distance matrix visualization from Distance Intelligence:
-
-```python
-from semantica.visualization import DistanceVisualizer
-
-viz = DistanceVisualizer()
-
-# Ego-mode: neighborhood of one node colored by distance band
-viz.visualize_ego(
-    graph,
-    center_node="Apple Inc.",
-    output="ego.html",
-    radius=0.5    # semantic distance radius
-)
-
-# N×N distance matrix heatmap
-viz.visualize_distance_matrix(
-    matrix=distance_matrix,
-    labels=node_labels,
-    output="distance_heatmap.html"
-)
-```
+| Format | Interactive | Scalable | Best For |
+| ------ | ----------- | -------- | -------- |
+| `.html` | Yes | N/A | Web dashboards, exploratory analysis |
+| `.png` | No | No | Reports, Jupyter notebooks |
+| `.svg` | No | Yes | Publications, slide decks |
+| `.pdf` | No | Yes | Print, compliance exports |
 
 ## Graph Explorer (Full Dashboard)
 
@@ -146,6 +256,32 @@ start_explorer(graph=kg, port=8080)
 
 See the [Explorer reference](explorer) for the full feature set and REST API.
 
+## Tips and Common Pitfalls
+
+<Warning>
+  **Use `max_nodes=500` for large graphs.** Force-directed layouts become unreadable and very slow above ~1,000 nodes. Limit with `max_nodes=500` or filter to a subgraph (e.g., top 100 nodes by PageRank) before visualizing.
+</Warning>
+
+<Tip>
+  **HTML output is always the best starting point.** Interactive HTML lets you zoom, pan, hover for details, and hide node types — giving you orders of magnitude more exploratory power than a static PNG. Only export to PNG/SVG/PDF when embedding in a report.
+</Tip>
+
+<Tip>
+  **Use `color_scheme="colorblind"` in publications and dashboards.** The Okabe-Ito palette is readable for everyone, including the ~8% of male readers who are red-green colorblind. Reserve `vibrant` for internal presentations only.
+</Tip>
+
+<Tip>
+  **UMAP is faster than t-SNE at scale.** For embedding spaces with >5,000 points, UMAP completes in seconds; t-SNE may take minutes. Both produce good cluster separation — use UMAP for exploratory speed, t-SNE for final publication-quality plots.
+</Tip>
+
+<Warning>
+  **`TemporalVisualizer.animate()` can produce large files.** Animated HTML files include all frames and can reach dozens of MB for long time series. Use `fps=1` or reduce the number of time steps for a manageable file size.
+</Warning>
+
+<Tip>
+  **For interactive dashboards, prefer Explorer.** `GraphVisualizer.visualize()` generates a self-contained HTML file. `start_explorer()` gives a full live web app with search, filtering, path-finding, and REST API. Use Explorer for team exploration, Visualizer for standalone report embeds.
+</Tip>
+
 <CardGroup cols={2}>
   <Card title="Knowledge Graph" icon="diagram-project" href="kg">
     The graph being visualized.