docs: premium README overhaul + ARCHITECTURE.md with Mermaid diagrams (#616)

- Rewrote README with verified code examples for all 18 modules - Added sections for semantica.split, semantica.conflicts, semantica.normalize - Added Recipes section (GraphRAG pipeline, audit trail, AML engine, ontology-to-KG) - Added REST API curl examples and MCP tools reference table - Added 9 contextual GitHub admonitions (NOTE/TIP/IMPORTANT/WARNING/CAUTION) - Fixed semantica.temporal (does not exist as standalone module — moved under semantica.kg) - Added ARCHITECTURE.md with two Mermaid flowcharts: · Full data pipeline (all sources → processing → storage → outputs) · Decision intelligence lifecycle (record → link → query → govern → audit) - Linked ARCHITECTURE.md from README nav and Architecture section
2026-08-29 04:26:20 +00:00 · 2026-06-12 22:42:46 +05:30
parent 946d826555
commit 9ba8b012bd
2 changed files with 693 additions and 111 deletions
@@ -0,0 +1,106 @@
+# Semantica — Architecture
+
+Complete data flow from every source type to every final output, and the decision intelligence lifecycle.
+
+---
+
+## Full Data Pipeline
+
+Every source, every processing step, every final artifact — in one diagram.
+
+```mermaid
+flowchart TD
+    %% ── SOURCES ──────────────────────────────────────────────────────
+    subgraph SRC["🗂️ Sources  (semantica.ingest)"]
+        direction LR
+        F["📄 Files\nPDF · DOCX · PPTX · HTML\nTXT · CSV · JSON · Excel · XML"]
+        W["🌐 Web\nPages · RSS/Atom Feeds\nPublic REST APIs"]
+        DB["🗃️ Databases\nPostgreSQL · MySQL · SQLite\nOracle · DuckDB · MongoDB"]
+        CL["☁️ Cloud\nSnowflake · Google Drive\nElasticsearch · HuggingFace"]
+        RT["⚡ Streams\nKafka · RabbitMQ\nAWS Kinesis · Pulsar"]
+        DV["🛠️ Dev\nGit Repos · Email IMAP/POP3\nMCP Resources · Parquet · Pandas"]
+    end
+
+    %% ── INGEST ───────────────────────────────────────────────────────
+    F  --> FI["FileIngestor"]
+    W  --> WI["WebIngestor"]
+    DB --> DI["DBIngestor"]
+    CL --> PI["ParquetIngestor\nSnowflakeIngestor"]
+    RT --> SI["StreamIngestor"]
+    DV --> RI["RepoIngestor\nEmailIngestor · MCPIngestor"]
+
+    FI & WI & DI & PI & SI & RI --> RAW[/"📦 Raw Documents"/]
+
+    %% ── PARSE ────────────────────────────────────────────────────────
+    RAW --> PRS["🔍 Parse  (semantica.parse)\nDocumentParser · StructuredDataParser\nCodeParser · WebParser · EmailParser"]
+
+    PRS --> NRM["🧹 Normalize  (semantica.normalize)\nTextNormalizer · EntityNormalizer\nDateNormalizer · NumberNormalizer · DataCleaner"]
+
+    NRM --> SPL["✂️ Split  (semantica.split)\nentity_aware · relation_aware\ngraph_based · ontology_aware · hierarchical"]
+
+    %% ── EXTRACT ──────────────────────────────────────────────────────
+    SPL --> EXT["🔬 Extract  (semantica.semantic_extract)\nNamedEntityRecognizer · RelationExtractor\nEventDetector · TripletExtractor · CoreferenceResolver"]
+
+    EXT --> CFT["⚠️ Conflict Detection  (semantica.conflicts)\nConflictDetector · ConflictResolver · SourceTracker"]
+
+    CFT --> DDP["🔁 Deduplication  (semantica.deduplication)\nDuplicateDetector · EntityMerger"]
+
+    DDP --> KGB["🕸️ KG Construction  (semantica.kg)\nGraphBuilder · EntityResolver\nBiTemporalFact · TemporalGraphQuery"]
+
+    KGB --> KG[/"🗺️ Knowledge Graph\nnodes · edges · temporal facts · provenance"/]
+
+    %% ── INTELLIGENCE LAYER ───────────────────────────────────────────
+    KG --> ONT["Ontology  (semantica.ontology)\nOntologyGenerator · OntologyValidator\nOWL · SHACL · SKOS"]
+    KG --> RSN["Reasoning  (semantica.reasoning)\nReteEngine · DatalogReasoner\nSPARQLReasoner · ExplanationGenerator"]
+    KG --> PRV["Provenance  (semantica.provenance)\nProvenanceManager · W3C PROV-O"]
+    KG --> CTX["Context & Decisions  (semantica.context)\nContextGraph · AgentContext\nDecisionRecorder · CausalChainAnalyzer · PolicyEngine"]
+
+    ONT & RSN & PRV & CTX --> EKG[/"🗃️ Enriched KG\n+ ontology · inferences · provenance · decisions"/]
+
+    %% ── STORAGE ──────────────────────────────────────────────────────
+    EKG --> VS["Vector Store  (semantica.vector_store)\nFAISS · Qdrant · Weaviate · Milvus · Pinecone · PgVector\nHybrid Search · RRF Fusion"]
+    EKG --> GS["Graph Store  (semantica.graph_store)\nNeo4j · FalkorDB · Apache AGE · Amazon Neptune"]
+
+    %% ── OUTPUTS ──────────────────────────────────────────────────────
+    VS & GS --> EXP["📦 Export  (semantica.export)\nRDF Turtle · JSON-LD · N-Triples · OWL · SHACL\nParquet · Cypher · ArangoDB AQL · GraphML · CSV · HTML"]
+    VS & GS --> VIZ["📊 Visualize  (semantica.visualization)\nKGVisualizer · OntologyVisualizer\nEmbeddingVisualizer · TemporalVisualizer"]
+    EKG --> SVC["🔌 Services\nREST API 109 ep · MCP Server 12 tools\nCLI 50+ cmds · Knowledge Explorer"]
+```
+
+---
+
+## Decision Intelligence Lifecycle
+
+```mermaid
+flowchart LR
+    subgraph RECORD["1️⃣  Record"]
+        R1["record_decision()\ncategory · scenario\nreasoning · outcome\nconfidence · metadata"]
+    end
+
+    subgraph LINK["2️⃣  Link"]
+        L1["add_causal_relationship()\ntriggers · enables\ncauses · precedes"]
+    end
+
+    subgraph QUERY["3️⃣  Query"]
+        Q1["find_similar_decisions()\nSemantic precedent search"]
+        Q2["trace_decision_chain()\nFull causal ancestry"]
+        Q3["analyze_decision_impact()\nDownstream influence map"]
+    end
+
+    subgraph GOVERN["4️⃣  Govern"]
+        G1["check_decision_rules()\nPolicy evaluation\nCompliance gate"]
+    end
+
+    subgraph AUDIT["5️⃣  Audit Export"]
+        A1["W3C PROV-O  ·  CSV  ·  JSON\nRegulator-ready audit trail"]
+    end
+
+    RECORD -->|decision_id| LINK
+    LINK   -->|causal graph| QUERY
+    QUERY  -->|results| GOVERN
+    GOVERN -->|signed-off decisions| AUDIT
+```
+
+---
+
+*→ [README](README.md) · [Docs](https://docs.getsemantica.ai/) · [Cookbook](https://github.com/semantica-agi/semantica/tree/main/cookbook)*