mengzhx/semantica
1
0
Fork 0
mirror of https://github.com/semantica-agi/semantica.git synced 2026-08-29 04:26:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into pr-1113-work

Browse Source 
# Conflicts:
#	semantica/export/rdf_exporter.py
This commit is contained in:
KaifAhmad1
2026-08-27 15:51:50 +05:30
parent 1c27a0ae7e 13b287b974
commit b2dc633796
210 changed files with 20678 additions and 2124 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/DISCUSSION_TEMPLATE/ideas.md
+1 -1
View File
@@ -69,5 +69,5 @@ If you have ideas on how this could be implemented, please share.
---
**Note**: For feature requests that are ready to be implemented, consider creating a [Feature Request issue](https://github.com/Hawksight-AI/semantica/issues/new?template=feature_request.md) instead.
**Note**: For feature requests that are ready to be implemented, consider creating a [Feature Request issue](https://github.com/semantica-agi/semantica/issues/new?template=feature_request.md) instead.
.github/DISCUSSION_TEMPLATE/qa.md
+2 -2
View File
@@ -46,8 +46,8 @@ If applicable, paste any error messages or describe unexpected behavior:
## Checklist
- [ ] I have searched existing [discussions](https://github.com/Hawksight-AI/semantica/discussions) and [issues](https://github.com/Hawksight-AI/semantica/issues)
- [ ] I have checked the [documentation](https://github.com/Hawksight-AI/semantica/tree/main/docs) and [FAQ](https://github.com/Hawksight-AI/semantica/blob/main/docs/faq.md)
- [ ] I have searched existing [discussions](https://github.com/semantica-agi/semantica/discussions) and [issues](https://github.com/semantica-agi/semantica/issues)
- [ ] I have checked the [documentation](https://github.com/semantica-agi/semantica/tree/main/docs) and [FAQ](https://github.com/semantica-agi/semantica/blob/main/docs/faq.md)
- [ ] I have provided a minimal code example (if applicable)
- [ ] I have included error messages (if applicable)
- [ ] I have provided environment details
.github/FUNDING.yml
+1 -1
View File
@@ -1,3 +1,3 @@
# Funding options for Semantica
github: Hawksight-AI
github: semantica-agi
.github/ISSUE_TEMPLATE/config.yml
+2 -2
View File
@@ -1,8 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: 📚 Documentation
url: https://github.com/Hawksight-AI/semantica/tree/main/docs
url: https://github.com/semantica-agi/semantica/tree/main/docs
about: Browse the documentation
- name: 💬 Discussions
url: https://github.com/Hawksight-AI/semantica/discussions
url: https://github.com/semantica-agi/semantica/discussions
about: Ask questions and discuss with the community
.github/SUPPORT.md
+9 -9
View File
@@ -3,31 +3,31 @@
## Getting Help
### 📚 Documentation
Check the [docs folder](https://github.com/Hawksight-AI/semantica/tree/main/docs) and [README](https://github.com/Hawksight-AI/semantica/blob/main/README.md) for guides and examples.
Check the [docs folder](https://github.com/semantica-agi/semantica/tree/main/docs) and [README](https://github.com/semantica-agi/semantica/blob/main/README.md) for guides and examples.
### 💬 Community Support
- **GitHub Discussions**: [Ask questions](https://github.com/Hawksight-AI/semantica/discussions)
- **GitHub Discussions**: [Ask questions](https://github.com/semantica-agi/semantica/discussions)
- **Discord**: Join our [Discord server](https://discord.gg/sV34vps5hH) for real-time chat
### 💭 Discussions
Join the conversation on [GitHub Discussions](https://github.com/Hawksight-AI/semantica/discussions):
Join the conversation on [GitHub Discussions](https://github.com/semantica-agi/semantica/discussions):
- **Q&A**: Ask questions and get help from the community
- **Ideas**: Share feature requests and suggestions
- **Show and Tell**: Showcase your projects and use cases
- **General**: General discussions about Semantica
### 🐛 Bug Reports
Found a bug? [Create an issue](https://github.com/Hawksight-AI/semantica/issues/new/choose)
Found a bug? [Create an issue](https://github.com/semantica-agi/semantica/issues/new/choose)
### 📖 Resources
- [Quick Start Guide](https://github.com/Hawksight-AI/semantica/blob/main/docs/quickstart.md)
- [FAQ](https://github.com/Hawksight-AI/semantica/blob/main/docs/faq.md)
- [Cookbook Examples](https://github.com/Hawksight-AI/semantica/tree/main/cookbook)
- [Quick Start Guide](https://github.com/semantica-agi/semantica/blob/main/docs/quickstart.md)
- [FAQ](https://github.com/semantica-agi/semantica/blob/main/docs/faq.md)
- [Cookbook Examples](https://github.com/semantica-agi/semantica/tree/main/cookbook)
## Commercial Support
For enterprise support, custom development, or consulting services:
- Contact us through [GitHub Issues](https://github.com/Hawksight-AI/semantica/issues)
- Contact us through [GitHub Issues](https://github.com/semantica-agi/semantica/issues)
- Include "Commercial Support" in the title
## Sponsorship
@@ -35,7 +35,7 @@ For enterprise support, custom development, or consulting services:
### Sponsor this project
Support Semantica development:
- [GitHub Sponsors](https://github.com/sponsors/Hawksight-AI)
- [GitHub Sponsors](https://github.com/sponsors/semantica-agi)
Your sponsorship helps us:
- Maintain and improve the framework
CHANGELOG.md
+129 -4
View File
@@ -11,6 +11,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **First-class LangChain integration** (closes #963; recreates #969)
- New `pip install semantica[langchain]` extra (`langchain-core>=0.3.0`), included in the `all` bundle
- `integrations/langchain/SemanticaRetriever` — LangChain `BaseRetriever` that seeds from `HybridSearch` then walks graph edges (`hops=2` default) for GraphRAG-style retrieval; falls back to `ContextGraph.query` when hybrid search is unavailable
- `integrations/langchain/SemanticaVectorStore` — LangChain `VectorStore` adapter over `HybridSearch` (`add_texts`, `similarity_search`, `similarity_search_with_score`, `from_texts`)
- `integrations/langchain/SemanticaKGTool` / `SemanticaDecisionTool` — `BaseTool` subclasses with Pydantic `args_schema` (`semantica_query_graph`, `semantica_query_decisions`); `build()` returns the tool, or `None` when langchain-core is absent
- Retriever and VectorStore read HybridSearch nested `metadata` (`content`, `node_id`, `node_type`) rather than top-level fields that HybridSearch does not set
- All adapters remain importable without langchain-core (`LANGCHAIN_AVAILABLE` flag)
- Docs: `docs/integrations/langchain.md`, README native-integration matrix, and `docs.json` nav entry
## [0.6.6] - 2026-08-20
### Added
- **Semantica RDF vocabulary, and deterministic entity/relationship IRIs** (#1109, closes #1107, closes #1101) by @fabio-rovai, reviewed by @KaifAhmad1
- Every RDF/JSON-LD export mints terms in `https://semantica.dev/ns#`, and until now nothing declared what those terms meant — the namespace 404s and no vocabulary shipped with the package, so a consumer receiving an export had no way to tell `sem:text` from a typo of it, and no closed-world checker could validate an export at all
- `semantica/ontology/vocabulary/semantica-ns.ttl` declares the terms the exporters actually emit — drawn from the emitting call sites in `export/rdf_exporter.py`, `export/json_exporter.py` and `provenance/manager.py`, not from what a vocabulary "ought" to contain. Ships inside the package (`from semantica.ontology.vocabulary import vocabulary_turtle`) so it loads without a network round trip, and is the same document intended to be served at the namespace IRI once hosting/content-negotiation is sorted
@@ -18,9 +31,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- The missing-id fallback minted entity/relationship IRIs from Python's builtin `hash()`, randomised per process (`PYTHONHASHSEED`), so the same entity got a different IRI on every run and exports couldn't be diffed, deduplicated, or joined to an earlier provenance record. It also wrote `<semantica:entity_N>`, an IRI in the scheme `semantica` rather than the expansion of the declared prefix, so those nodes never joined with anything written through it. Minting now uses SHA-256 and writes a full IRI in the declared namespace; the same fix applies to the default entity/relationship types in the Turtle path
- **Fixed during review** (Qodo): the temporal fallback minted from `source_id` only, while the main serializer accepts `source_id` or `source` — relationships using the second form hashed two empty strings, which the previous randomised `hash()` masked by making the IRI unstable anyway; once deterministic, unrelated relationships at the same list index collided on one IRI across exports. Endpoints are now resolved the same way `serialize_to_turtle` resolves them, before minting. `sem:confidence` also lost its declared `xsd:decimal` range: the N-Triples serializer types the same value `xsd:float`, and the two are disjoint, so declaring either contradicted one of the exporters (tracked in #1100) — a new `test_declared_ranges_do_not_contradict_what_the_exporters_emit` guards the whole class of that mistake
- **Fixed in follow-up**: `serialize_to_rdfxml`'s default entity type still wrote the bare string `"semantica:Entity"` into an `rdf:resource` attribute, which (unlike a Turtle angle-bracket or an XML element name) is not namespace-expanded — the exact #1101 failure mode, just on the untested RDF/XML path. `json_exporter.py`'s `semantica:format` and `@type: "semantica:KnowledgeGraph"` were emitted but absent from both the vocabulary and the test's `EMITTED_TERMS` guard set, so the "undeclared terms fail the build" claim didn't actually cover them — both are now declared and guarded. `MANIFEST.in` didn't mirror the `pyproject.toml` package-data addition, so a source-distribution install could omit the vocabulary file. The cross-process minting-stability test replaced the subprocess's entire environment with a POSIX-only `PATH`, breaking it on Windows; now overrides only `PYTHONHASHSEED` on top of the inherited environment
- 229 export and ontology tests pass
- **Also fixed, on the JSON-LD paths**: the first fix covered the Turtle, N-Triples and RDF/XML serializers, and left both JSON-LD writers interpolating the entity's own text into `f"semantica:entity/{text}"` and the endpoints into `f"semantica:rel/{source}_{target}"`. Three consequences, all live in 0.6.5: an entity whose text contained a space produced an invalid IRI, and a JSON-LD parser dropped that node in full rather than reporting it, so the entity disappeared from the export; every relationship carrying `source`/`target` rather than `source_id`/`target_id` minted the identical `semantica:rel/_`, collapsing all of them onto one node whose types and endpoints merged; and the JSON-LD `@id` disagreed with the Turtle IRI for the same entity, so the two serializations of one knowledge graph were two different graphs. Both JSON-LD writers now use `mint_entity_iri`/`mint_relationship_iri`, and `JSONExporter.export_entities`/`export_relationships` declare the `semantica` prefix their `@context` was already writing `semantica:entities` against — without it a processor reads that as an IRI in the scheme `semantica`, which is the original #1101 defect on a third path
- `tests/export/test_jsonld_iri_minting.py` parses each export with a real JSON-LD processor and asserts the entity survives, the relationships stay distinct, no term expands into the `semantica` scheme, and the JSON-LD `@id` equals the Turtle IRI
- 236 export and ontology tests pass
- **First-class CrewAI integration** (#962)
- **First-class CrewAI integration** (#988, closes #962) by @Shindevrp
- New `pip install semantica[crewai]` extra (`crewai>=0.80.0`) — crewai core provides `BaseTool`/`BaseKnowledgeSource`, so `crewai-tools` is intentionally not included, and the extra is intentionally **not** part of the `all` bundle: crewai hard-requires `chromadb~=1.1.0`, which is affected by the unpatched pre-auth code-injection CVE-2026-45829 (see `integrations/crewai/README.md`)
- `integrations/crewai/SemanticaKGTool` — a CrewAI `BaseTool` exposing 5 KG actions (`extract_entities`, `extract_relations`, `add_to_graph`, `query_graph`, `find_related`) backed by `NERExtractor` / `RelationExtractor` / `ContextGraph`; supports both sync `run()` and async `arun()`
- `integrations/crewai/SemanticaDecisionTool` — a CrewAI `BaseTool` wrapping `AgentContext` with 5 decision-intelligence actions (`record_decision`, `find_precedents`, `trace_causal_chain`, `analyze_impact`, `check_policy`)
@@ -50,6 +65,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- New `tests/export/test_distance_exporter_metric_errors.py`: 6 tests covering success, single/multiple failures, opt-out, the no-path-vs-error distinction, and default-schema stability; existing `tests/export/test_distance_exporter.py` updated for the new tuple return type
- Full `tests/export/` suite: 77 passed
- **`ContextGraph.to_kg_dict()`: an adapter converting a `ContextGraph`'s internal `nodes`/`edges`/`source` shape into the canonical `entities`/`relationships`/`source_id` shape `RDFExporter` and `TemporalGraphQuery` consume** (#1081) by @cxzg007
- Previously there was no supported way to feed a `ContextGraph` into those consumers without hand-rolling the field remapping; `to_kg_dict()` does it once, with an `entities_only` option that drops relationships left dangling by the filter
- **Fixed during review** (Qodo): null `properties`/`metadata` on a node loaded from JSON raised `TypeError` when copied — both are now guarded with `or {}`; entity ids are coerced to `str(node_id)` to match `ContextEdge`'s already-str-coerced endpoints, so valid relationships were no longer dropped by `entities_only` filtering
- `RDFExporter`'s validator and `TemporalGraphQuery` now also accept `source_id`/`target_id` endpoints, the shape `to_kg_dict()` emits
### Changed
- **`GraphBuilder`'s 6 public methods now have Google-style docstrings** (#878, closes #876) by @cakeni
@@ -59,7 +79,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Corrected during review**: `add_temporal_edge`/`create_temporal_snapshot` docstrings overclaimed numeric-timestamp support; `_parse_time()` only special-cases `str` and `datetime`, falling back to a bare `str()` cast for anything else (not true numeric parsing). Narrowed to "datetime or ISO-formatted string"
- **Fixed along the way**: `build()`'s `**options` documented a default only for `extract`; `extract_relations`, `extract_triplets`, `ner_method`, `relation_method`, and `triplet_method` all have concrete defaults in `_extract_from_text()` (`True`, `True`, `"llm"`, `"llm"`, `"llm"`) that were left unstated, inconsistent with CONTRIBUTING.md's own docstring example of noting defaults inline
- `python -m pytest tests/kg/test_kg.py tests/kg/test_graph_builder_external.py -q`: 45 passed
- **`GraphBuilder` raw-text extraction now defaults to local extractors instead of LLM extraction** (closes #930) by @dex0shubham
- **`GraphBuilder` raw-text extraction now defaults to local extractors instead of LLM extraction** (#941, closes #930) by @dex0shubham
- `GraphBuilder._extract_from_text()` defaulted `ner_method`, `relation_method`, and `triplet_method` to `"llm"`, and ran relation extraction unconditionally (`extract_relations` defaulted to `True`) — all four contradicting the defaults documented in the `build()` docstring at the time (`"ml"` / `"pattern"` / `False`), and diverging from the standalone extractors (`NERExtractor` defaults to `method="ml"`, `RelationExtractor` and `TripletExtractor` to `method="pattern"`). The practical effect was that any raw-text `build()` call silently required a configured provider, an API key, and network access
- Defaults are now `ner_method="ml"`, `relation_method="pattern"`, `triplet_method="pattern"`, and `extract_relations=False`, matching the docstring. LLM extraction remains fully available and is now opt-in
- **To restore the previous behaviour**, pass the methods explicitly:
@@ -80,8 +100,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- New regression coverage in `tests/kg/test_graph_builder_extraction_defaults.py` pinning all four defaults, verifying that no default resolves to `"llm"`, confirming explicit LLM opt-in still routes correctly, asserting extractors are constructed once across repeated texts, covering fallback method lists (e.g. `ner_method=["pattern", "ml"]`) for all three extractors, asserting relations are forwarded to triplet extraction (and that `None` is forwarded when relation extraction is disabled or fails), and running the real default path end to end with no provider mocked. Verified to fail against the pre-fix code
- Full `kg` suite: 473 passed
- **Explorer graph canvas now renders edge labels** (#1013, closes #1009) by @yzxcj797
- `GraphCanvas.tsx` had no edge-label rendering path at all; Sigma's edge-label renderer draws `data.label`, but the graph state stored the relationship type under `edgeType`, so simply enabling the renderer would have left every edge blank. `graphSceneState`'s edge reducer now maps `edgeType` onto `label` (suppressed for hidden edges)
- Rendering is gated behind a new `edgeLabelsEnabled` entry in the Effects panel (default on), wired through the existing `GraphEffectToggle`/`GraphEffectsState` plumbing, so dense graphs can still turn labels off
- **Fixed during review** (Qodo): two follow-up passes closed gaps the first cut left — label rendering wasn't wired through `explorationEffectsPluginPhaseC.tsx`'s Phase C variant, and toggling the effect off mid-session didn't clear already-rendered labels
- New coverage in `explorer/tests/graphSceneState.display.test.ts`
- **Removed `GraphWorkspaceShell.tsx`, `GraphRuntimeStage.tsx`, and `useGraphData.ts` — a second, unused implementation of the graph-loading/error-handling logic already fixed in `GraphWorkspace.tsx`** (#984, resolves the cleanup tracked in #981 by #980's review note) by @lakshayxi
- 1,564 lines removed; the surviving `GraphWorkspace` path is now the only implementation, so the "two copies that drifted apart" root cause #980 fixed can't recur in the copy nobody was maintaining
- **Explorer README and `docs/explorer-setup.md` corrected to describe the authentication 0.6.5 actually shipped**, plus a documented `/ws/graph-updates` auth note (#1040, fixes #1028) by @Kyou12138
- Both docs still claimed the Explorer API had no built-in authentication after v0.6.5 added mandatory `SEMANTICA_API_KEY` enforcement with a `503` fail-closed default; corrected to describe the actual behavior, including that only protected routes require the key (`/api/health`/`/api/info` stay open), the non-loopback-bind CLI warning only fires in anonymous mode or when the key is unset, and `SEMANTICA_API_KEY`/`SEMANTICA_ALLOW_ANONYMOUS` are documented in the environment-variable table
- **CI: pinned `github/codeql-action` to current v4** (#986) by @ZohaibHassan16, and **pinned Python dependencies in `requirements-ci.txt` for reproducible CI runs** (#945) by @yunaremaia, closing the gap where an unpinned CI dependency could silently change behavior between runs
- **README now states up front that Semantica's explainability is system-level, not foundation-model-internal** (#1033, #1034) by @KaifAhmad1
- Nothing in the README previously scoped what "explainable" meant, leaving readers to assume Semantica could expose or reconstruct an LLM's internal reasoning. A callout now states explicitly that Semantica explains and audits what the AI *system* did — context fed in, decisions produced, provenance, relationships, policies applied — not the model's private internal reasoning, and moved the note near the top of the README rather than leaving it implicit
### Fixed
- **The temporal-evolution `stability` metric was a hardcoded placeholder, not a duration**
- `TemporalGraphQuery.analyze_evolution()` documents `stability` as a "relationship duration/stability measure", but the implementation appended a constant `1` for every relationship with both `valid_from` and `valid_until` set (`durations.append(1) # Placeholder`). The reported stability was therefore always `1.0` when any bounded relationship existed and `0` otherwise — it never reflected how long relationships actually stayed valid, so it could not distinguish a graph of decade-long relationships from one of one-second relationships
- `stability` now computes the mean valid-time duration in seconds (`(valid_until - valid_from).total_seconds()`) across relationships that have both bounds set. Relationships with a missing or open `valid_from`/`valid_until` are skipped (their duration is unbounded), and non-positive intervals are clamped to `0`; an empty set still reports `0`
- New tests in `tests/kg/test_kg.py` assert the mean-duration result, the skipping of unbounded/half-open intervals, and the empty-graph zero case
- **Every timestamp an export or a provenance record wrote was timezone-naive** (closes #1114) by @fabio-rovai
- `semantica/export/` stamped with `datetime.now().isoformat()`, which reads the machine's **local** clock; `semantica/provenance/` stamped with `datetime.utcnow().isoformat()`, which reads **UTC**. Both produce a naive value and both serialize identically, so nothing downstream can tell which zone a given timestamp belongs to — the same string means two different instants depending on which module wrote it
- In RDF the consequence is silent rather than loud. Under XSD 1.1 a value with no timezone compared against one with a timezone is indeterminate whenever the two fall inside the ±14 hour window; SPARQL turns an indeterminate comparison into an error, and `FILTER` discards errors as non-matches. A timezone-qualified query over an Oxigraph store returns an answer with every Semantica-written record quietly absent from it, which is a poor property for `prov:generatedAtTime`, `prov:startedAtTime`, `prov:endedAtTime` and `prov:atTime` to have
- New `utc_now()`/`utc_now_iso()` in `semantica/utils/helpers.py`, exported from `semantica.utils`, and used at all 29 call sites in `export/` (`json_exporter`, `yaml_exporter`, `report_generator`, `export_provenance`) and `provenance/` (`manager`, `schemas`, `bridge_axiom`). Values now read `2026-08-19T14:19:04.229937+00:00`: one unambiguous instant, comparable against any correctly stamped value, and valid `xsd:dateTimeStamp`. `sem:exportedAt`'s range in `semantica/ontology/vocabulary/semantica-ns.ttl` is tightened from `xsd:dateTime` accordingly, and its comment no longer has to explain why the weaker range was necessary
- `datetime.utcnow()` is deprecated as of Python 3.12 and scheduled for removal; constructing a `ProvenanceEntry` under `-W error::DeprecationWarning` on 3.13 raised, and no longer does
- New `tests/export/test_timestamp_timezones.py` and `tests/provenance/test_timestamp_timezones.py`: offset presence on every export and provenance path, PROV-O literals valid as `xsd:dateTimeStamp`, comparison against a timezone-aware instant without `TypeError`, the Oxigraph filter that dropped the naive value (with a bound inside the indeterminate window, so the test cannot pass by accident), and the document `@id` remaining a valid IRI with `+00:00` in it. 11 of the 13 fail on the parent commit
- **Fixed during review** (Qodo): once new entries carry `+00:00` and stored ones do not, `ProvenanceManager.query_recorded_between` and `audit_log` compared ISO timestamps as raw strings, so they ordered by spelling rather than by instant — an inclusive naive bound naming a stored offset-bearing timestamp sorted *below* it and dropped the record, and a bound written in another offset landed wherever its digits fell (`19:45+05:30` is 14:15Z, but sorted after 14:19Z). Both now compare instants through a new `to_utc_datetime()` helper that reads a missing offset as UTC, which is what the values written before this change actually were; a bound that cannot be read as a timestamp keeps the historical string comparison rather than raising on a call that used to work
- The remaining 147 naive call sites are in `context/`, `vector_store/`, `seed/` and elsewhere, where timestamps are compared against values parsed from previously stored naive strings. Converting those without a read-side migration would raise `TypeError: can't compare offset-naive and offset-aware datetimes` on existing data, so they are deliberately left for a separate change
- **`SHACLGenerator` mangles `#`-terminated namespaces into `#/`, so generated shapes target nothing** (#1082) by @changshenhan
- `__init__` normalized `base_uri` with `rstrip("/") + "/"`, which turns `http://example.org/manufacturing#` into `...manufacturing#/` — the most common RDF namespace convention. Every generated URI (`sh:targetClass`, `sh:path`, shape URIs) then landed in a different namespace than the instance data, and SHACL validation silently passed because the shapes targeted nothing
- `__init__` now preserves a namespace already ending in `/` or `#`, matching the `#`-aware normalization `generate()` already applies; `shapes_uri` inherits the fix
- New `test_hash_namespace_base_uri_is_not_mangled` in `tests/ontology/test_ontology_advanced.py` fails on the pre-fix normalization and passes with it; full ontology suite (76 tests) green
- **`split`/chunking paths bypassed the centralized spaCy model cache, reloading the model on every call** (#1042, closes #998) by @Accute9, reviewed by @Sameer6305
- `semantica/split/methods.py`'s `split_by_sentences()` and `semantica/split/semantic_chunker.py`'s `SemanticChunker.__init__` each called `spacy.load()` directly instead of reusing the process-level cache added in #889/`semantic_extract/methods.py`'s `load_spacy_model()` — every call/construction re-paid the ~120ms model-load cost independently of `NERExtractor`, which already used the cache
- Both now route through `load_spacy_model()`, sharing one cached `Language` instance per model name across `split_by_sentences()`, `SemanticChunker`, and `NERExtractor`; a missing model still falls back to regex/paragraph chunking without poisoning the cache for a later successful load
@@ -183,6 +238,59 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- New regression coverage in `tests/export/test_distance_exporter.py`: warnings fire on exception for all four helpers, exported sentinel values/shape stay unchanged, and the legitimate "no KG backend" `None` path still logs nothing
- Full `tests/export/` suite: 71 passed
- **`explain_violations` rendered hardcoded placeholders (`min_count=1`, `max_count=1`) instead of the SHACL shape's real constraint values, and misused the violation message text as the datatype/class value** (#1094) by @cxzg007
- `_run_pyshacl` never read `sh:minCount`/`sh:maxCount`/`sh:datatype`/`sh:class` back from the violation's `sh:sourceShape`, so every plain-English explanation was wrong regardless of what the shape actually declared. `SHACLViolation` now carries those four fields (also exposed via `to_dict()`), populated by back-referencing `sh:sourceShape`; `explain_violations` renders the real values, falling back to `"?"` when a value is genuinely absent
- **Known limitation**: `sh:qualifiedMinCount`/`sh:qualifiedMaxCount` are not handled yet and still fall back to the `"?"` placeholder
- New regression tests cover both the rendering path and the `sh:sourceShape` back-reference (skipped when `pyshacl`/`rdflib` are absent)
- **Entity merging silently dropped `entity_id` aliases, and exact-match entity resolution had three correctness gaps** (#1086, #1026) by @T1mn
- `entity_merger.py`/`merge_strategy.py`/`entity_resolver.py` used inconsistent logic for extracting an entity's id across the merge path, so a merged entity could lose the `entity_id` aliases that let later lookups find it under its old identity. A new `semantica/utils/entity_ids.py` unifies id extraction across all three call sites
- `EntityResolver`'s exact-match path is now honored rather than silently falling through to fuzzy matching in some cases; entities with no identifier are preserved instead of being dropped, and blank exact-match names are ignored rather than matching every other blank name
- New/expanded coverage in `tests/kg/test_entity_pipeline.py` and `tests/kg/test_entity_resolver_exact.py`
- **`flatten_dict()` silently collided keys when a flattened path from one branch matched a literal key already present at the target depth** (#1062) by @shahzaib-ahmadcs
- Two differently-shaped inputs could flatten to the same output key, with the second write silently overwriting the first — no error, no warning, just a dropped value. Collisions are now detected and handled explicitly instead of overwriting
- **`ExcelParser.__init__` raised `NameError` on every instantiation — `get_progress_tracker()` was called but never imported** (#1016, closes #1014) by @pravit-amp
- Same defect as the one fixed for `SimilarityCalculator` in #530, this time in `semantica/parse/excel_parser.py`; the existing test imported the class but never constructed it, so nothing caught the missing import. Added construction coverage for every parser exported from `semantica.parse`, driven off `__all__` so future additions are covered automatically, living outside `test_parse_comprehensive.py` (whose `setUp` mocks `get_progress_tracker` into each module and would mock away the exact interaction under test)
- **Graph analytics (`centrality_calculator.py`, `community_detector.py`, `connectivity_analyzer.py`) dropped isolated nodes and diverged on how each computed its working view of the graph** (#1011) by @T1mn
- Each analyzer had its own ad hoc logic for building the node/edge set it operated over, and none of them included nodes with no edges — a node with zero connections simply vanished from centrality scores, community assignments, and connectivity reports instead of appearing with a zero/singleton value. A new shared `semantica/kg/_graph_view.py` centralizes graph-view construction (including node fallbacks and community payload shaping) for all three analyzers, which are now ~250 lines lighter combined
- New `tests/kg/test_analytics_node_scope.py` covering isolated-node presence across all three analyzers
- **Explorer fired temporal-bounds and snapshot requests before the graph itself had loaded, tripling failed requests when the backend was down and leaving the timeline scrubber with nothing to scrub** (#1003) by @lakshayxi
- Two new predicate functions gate the temporal effects on the graph having actually loaded (an empty graph still counts as loaded); confirmed against a downed backend that this cuts three failing requests per page load down to one
- **`SeedDataManager.load_from_database()` never actually reached the database, and connection failures were mislabeled as a missing optional dependency** (#995, closes #973) by @yzxcj797
- `DBIngestor.execute_query`/`export_table` need the connection string as their first positional argument; `load_from_database()` only passed it into the constructor's config dict, which those methods never read, so every call raised `TypeError` before connecting. Also split the combined `except (ImportError, OSError)` handling apart — a genuine connection failure was reported as `"module not available"`, sending debugging in the wrong direction; `OSError` now propagates as an actual failure, chained via `from e`
- **SPARQL `CONSTRUCT` detection matched inside a leading `#`-comment, misclassifying `SELECT`/`ASK` queries as `CONSTRUCT` across all four SPARQL backends** (#951) by @pravit-amp
- `CONSTRUCT_QUERY_RE` skipped comments with a bare `\#[^\n]*`, whose backtracking `*` let a `# CONSTRUCT ...` comment line "swallow" the real query-form keyword on the next line for a query like `# CONSTRUCT ...\nSELECT ...`. The mistaken `CONSTRUCT` classification sent `Accept: text/turtle` and tried to parse a SELECT/ASK response body as Turtle, failing with a misleading parse error. The regex now requires a comment to reach a line terminator (LF or CR, per the SPARQL grammar) before matching
- **`k_shortest_paths` mutated caller-visible graph state during traversal and ignored direction when excluding already-used edges** (#1000) by @T1mn
- `semantica/kg/path_finder.py`'s search left side effects behind after returning, and edge exclusion during Yen's-algorithm-style path removal didn't respect the traversal direction of directed graphs, letting a later search see edges that should have been available. Both fixed; new coverage in `tests/kg/test_path_finder.py`
- **`trace_decision_causality()` ignored explicitly recorded causal edges, inferring causes only from shared NER entities plus timestamp ordering** (#983) by @hsd2514
- A `CAUSED`/`INFLUENCED`/`PRECEDENT_FOR` edge added via `add_causal_relationship()` had no effect on the trace — when entity extraction found nothing in common between two decisions, `trace_decision_chain()` came back empty even with an explicit edge stored in the graph. Explicit causal edges are now traversed first as ground truth, with entity/timestamp inference kept as an additive fallback for pairs with no explicit link; edges whose source has no decision record (e.g. a graph restored via `from_dict`) are skipped so a stale edge can't abort the trace
- **`RepoIngestor`'s module-level DNS resolve cache had no lock, raising `RuntimeError: OrderedDict mutated during iteration` under concurrent `ingest_repository()` calls** (#979) by @manjunathbhaskar
- `_REPO_HOST_RESOLVE_CACHE` is a shared `OrderedDict` read, written, and pruned by every thread with no synchronization — reliably reproduced with 32 threads hammering resolution under a low TTL and small cache cap. Now guarded by a lock
- **`GraphBuilder` didn't remap relationship endpoints after entity resolution merged nodes, leaving relationships pointing at ids that no longer existed in the resolved graph** (#978) by @T1mn
- New coverage in `tests/kg/test_graph_builder_external.py`; a follow-up commit hardens the remapping against edge cases found during review
- **Explorer's dev server esbuild target didn't match the browser targets the production build declares**, occasionally producing dev-only syntax errors on older browsers (#966) by @le-czs
- `explorer/vite.config.ts` now sets the dev esbuild target explicitly to match
- **`normalize`'s number normalizer accepted currency symbols without validating them against the surrounding text, and an earlier fix's currency-code matching wasn't token-bounded** (#940) by @Mr-Neutr0n, reviewed by @ZohaibHassan16
- Symbol currencies are now validated before being accepted; currency codes are matched on token boundaries so a code embedded inside a longer token no longer false-positives
- **`ContextGraph.to_dict()` was the one reader on the class that didn't hold `self._lock`, raising `RuntimeError: dictionary changed size during iteration` under a concurrent writer and risking a torn snapshot otherwise** (#929) by @pravit-amp
- Every other reader (`stats()`, `density()`, `find_nodes()`, `find_edges()`, `get_neighbors()`, `get_nodes_by_label()`, `state_at()`, `save_to_file()`) already took the lock after it was introduced; `to_dict()` predated that change and was missed. `save_to_file()` was safe only incidentally, since it builds its payload inline under its own lock rather than delegating to `to_dict()`
- **`PipelineWithProvenance` had a broken import and no working `run()` method** (#862) by @Karunasagar12
- `from .pipeline import Pipeline` failed because `Pipeline` lives in `pipeline_builder.py`, not a nonexistent `pipeline.py` — fixed to `from .pipeline_builder import Pipeline`. The class also had no `run()`; it now delegates to `ExecutionEngine.execute_pipeline()`, the intended execution path for a built `Pipeline`. The constructor now accepts a built `Pipeline` instance directly
### Security
- **Tarball restore path traversal, latent SQL injection, DNS-rebinding TOCTOU in the shared SSRF guard, stored XSS in report generation, and unvalidated SPARQL object IRIs in AnzoStore** (#1079) by @KaifAhmad1
@@ -243,6 +351,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Caught by the new gate on its first run**: `python -m pip install -e ".[all]"` pulled in `setuptools==79.0.1`, vulnerable to CVE-2026-59890/GHSA-h35f-9h28-mq5c/PYSEC-2026-3447 (Unicode-normalization bypass of `MANIFEST.in` exclude/prune patterns on macOS APFS/HFS+, letting excluded files leak into a built sdist), fixed in `83.0.0`. `[build-system] requires` had the exact same too-permissive-floor pattern this whole entry is about (`setuptools>=61.0`), and `actions/setup-python`'s baked-in `setuptools` isn't governed by that pin at all since it's outside any isolated build. Bumped `[build-system] requires` to `setuptools>=83.0.0`, and the `Security` workflow now runs `pip install --upgrade pip setuptools` before auditing so the scanned environment can't have a stale ambient copy regardless of what governs it
- Full `explorer` suite: 241 passed
- **`SeedDataManager.load_from_api()` made unguarded HTTP requests, with no SSRF protection at all** (#942) by @ZohaibHassan16
- `load_from_api()` called `requests.get()` directly instead of going through `semantica/ingest/ssrf.py`'s `request_with_ssrf_guard()`, unlike every other ingestor in this module — a caller-supplied `api_url` could target internal/private network addresses with no validation. Now routes through the shared guard, gaining redirect validation and bounded DNS resolution for free
- **Follow-up** (#959, closes #943) by @yunaremaia: added an `allow_private_ips` opt-in (parsed via the shared `parse_bool` helper) for trusted internal deployments that legitimately need to load from a private-network API, while keeping the guard's block-by-default behavior for everyone else
## [0.6.5] - 2026-08-11
### Added
@@ -303,8 +415,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Entities and relationships round-trip as memory-local provenance only — Markdown import intentionally does not write into `ContextGraph`, matching the MVP scope agreed on in #765
- Documented the file contract and workflow in `docs/reference/context.md`; 43 new tests in `tests/context/test_agent_memory_markdown.py` cover round-trip losslessness, idempotency, validation errors, rollback on failure, and vector-store sync ordering
- **Markdown directory round trips for `ContextGraph`** (#852) by @SaurabhScripts
- `ContextGraph.save_to_file(..., format="markdown")` and `load_from_file(..., format="markdown")` persist a deterministic `graph.md` relationship manifest plus one human-editable Markdown file per node, preserving graph, node, edge, family, temporal, and cross-graph link identities
- Imports validate the complete directory before replacing graph state, rebuild indexes and analytics state atomically, create JSON-compatible stub nodes for dangling edge endpoints, and emit the same granular node/edge audit events as JSON loading
- Existing exports are replaced atomically only after their complete canonical layout is validated; untracked files, renamed node files, symlinks, Windows directory junctions, and other reparse points cause a fail-closed error instead of authorizing directory deletion
- Added 30 focused tests covering deterministic round trips, manual edits, validation rollback, managed-directory identity, publish rollback, audit-manager compatibility, stale-cache clearing, mocked and real Windows junctions, and missing-path behavior
### Fixed
- **Markdown import followed filesystem links even though Markdown export already refused to overwrite them** (#851, follow-up to #765, #786) by @SaurabhScripts
- `AgentMemory._read_markdown_path()` now rejects symlink files, broken symlinks, symlinked directories, Windows directory junctions, and other Windows reparse points supplied directly; linked entries discovered inside an otherwise valid directory are safely skipped, preserving the current directory-import contract
- `_read_markdown_file_content()` re-checks the file and parent directory immediately before and after opening, uses `O_NOFOLLOW` where available, and verifies the resulting descriptor is a regular file via `fstat`/`S_ISREG`, so link swaps are rejected rather than silently followed
- Junction detection uses `os.path.isjunction()` where available and falls back to the Windows reparse-point file attribute on older Python versions; export applies the same link check before replacing a Markdown file
- Documented the import restriction in `docs/reference/context.md`; added 11 tests to `tests/context/test_agent_memory_markdown.py` covering file/directory/broken-symlink rejection, simulated open races, mocked and real Windows junctions, and the reparse-point fallback
- Any additional review follow-up commits land in this same PR/entry rather than as a separate changelog item
- **`PipelineWithProvenance` raised `ModuleNotFoundError` on import and `AttributeError` on `.run()`** (#858, closes #858) by @Karunasagar12
- `from .pipeline import Pipeline` failed because `semantica/pipeline/pipeline.py` does not exist; corrected to `from .pipeline_builder import Pipeline`
- `.run()` called `self._pipeline.run()` on the `Pipeline` dataclass, which has no such method; replaced with `self._engine.execute_pipeline(self._pipeline, ...)` delegating to `ExecutionEngine`
@@ -1421,4 +1546,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
---
For detailed release notes, see [GitHub Releases](https://github.com/Hawksight-AI/semantica/releases).
For detailed release notes, see [GitHub Releases](https://github.com/semantica-agi/semantica/releases).
CODE_OF_CONDUCT.md
+1 -1
View File
@@ -58,7 +58,7 @@ representative at an online or offline event.
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement through
[GitHub Issues](https://github.com/Hawksight-AI/semantica/issues) with "[CoC]" prefix.
[GitHub Issues](https://github.com/semantica-agi/semantica/issues) with "[CoC]" prefix.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
CONTRIBUTORS.md
+3 -3
View File
@@ -44,7 +44,7 @@ We recognize all types of contributions:
All contributors are recognized in:
- This contributors list
- [GitHub contributors page](https://github.com/Hawksight-AI/semantica/graphs/contributors)
- [GitHub contributors page](https://github.com/semantica-agi/semantica/graphs/contributors)
- Release notes for significant contributions
- Community appreciation
@@ -54,7 +54,7 @@ All contributors are recognized in:
### Automatic Recognition
If you've made a commit, you'll automatically appear in [GitHub's contributors graph](https://github.com/Hawksight-AI/semantica/graphs/contributors).
If you've made a commit, you'll automatically appear in [GitHub's contributors graph](https://github.com/semantica-agi/semantica/graphs/contributors).
### Using All-Contributors Bot
@@ -111,4 +111,4 @@ Every contribution, no matter how small, helps make Semantica better. Thank you
**Want to contribute?**
⭐ Give us a Star • 🍴 [Fork us](https://github.com/Hawksight-AI/semantica/fork) • Check out our [Contributing Guide](CONTRIBUTING.md) to get started!
⭐ Give us a Star • 🍴 [Fork us](https://github.com/semantica-agi/semantica/fork) • Check out our [Contributing Guide](CONTRIBUTING.md) to get started!
Dockerfile
+1 -1
View File
@@ -9,7 +9,7 @@ RUN npm ci
COPY explorer/ ./
RUN mkdir -p /app/semantica && npm run build
FROM python:3.14-slim AS runtime
FROM python:3.13-slim AS runtime
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
LICENSE
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2026 Hawksight AI
Copyright (c) 2026 Semantica
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
README.md
+41 -26
View File
@@ -87,7 +87,7 @@ Semantica sits underneath your LLM, vector store, and agent framework as a deter
- **Graph Analytics:** Centrality, community detection, link prediction, and shortest-path queries over the graph you just built
- **Polyglot Graph Storage:** Native RDF (embedded Oxigraph, Blazegraph, Apache Jena, Eclipse RDF4J via SPARQL) and Labeled Property Graphs (Neo4j, FalkorDB, Apache AGE, AWS Neptune via Cypher), plus vector stores, all swappable without touching your code
- **Visualization:** Explore any graph, ontology, or timeline in an interactive browser workbench
- **Drop-in Integrations:** Native Agno and CrewAI support, a full-featured MCP server, a comprehensive CLI, a REST API, and plugins across major editors
- **Drop-in Integrations:** Native Agno, CrewAI, and LangChain support, a full-featured MCP server, a comprehensive CLI, a REST API, and plugins across major editors
---
@@ -142,11 +142,13 @@ compliant = graph.check_decision_rules({"category": "vendor_selection"}) # poli
```bash
semantica doctor
# Python 3.11.9 pass
# semantica 0.6.5 pass
# semantica 0.6.6 pass
# faiss vector store pass
# Config file pass ~/.semantica/config.yaml
```
**Running in a script or CI?** Progress bars are written only when stdout is an interactive terminal (or a Jupyter notebook), so piping and redirecting stay clean by default. Override with `SEMANTICA_DISABLE_PROGRESS=1` to silence progress everywhere, or `SEMANTICA_FORCE_PROGRESS=1` to keep it when stdout is redirected. `SEMANTICA_DISABLE_PROGRESS` takes precedence.
<div align="center">
If Semantica solves a real problem for you, a star helps others find it.
@@ -1186,7 +1188,7 @@ Start with `semantica`, verify with `doctor`, build a graph, and explore the com
## Integrations
Native plugin bundles for Claude Code, Cursor, Codex, Windsurf, Cline, Continue, VS Code, and OpenClaw; a full-featured MCP server for any MCP-compatible client; a comprehensive REST API; and first-class Agno and CrewAI support for agentic frameworks. Every major LLM provider is already supported via `semantica.llms` and LiteLLM: OpenAI, Anthropic, Gemini, Mistral, Llama, Groq, Cohere, Azure, Bedrock, Ollama, DeepSeek, HuggingFace, and more.
Native plugin bundles for Claude Code, Cursor, Codex, Windsurf, Cline, Continue, VS Code, and OpenClaw; a full-featured MCP server for any MCP-compatible client; a comprehensive REST API; and first-class Agno, CrewAI, and LangChain support for agentic frameworks. Every major LLM provider is already supported via `semantica.llms` and LiteLLM: OpenAI, Anthropic, Gemini, Mistral, Llama, Groq, Cohere, Azure, Bedrock, Ollama, DeepSeek, HuggingFace, and more.
MCP setup takes 30 seconds — see [MCP Server](#mcp-server) below.
@@ -1305,17 +1307,17 @@ MCP setup takes 30 seconds — see [MCP Server](#mcp-server) below.
<strong>CrewAI</strong><br/>
<sub>First-class · <code>pip install semantica[crewai]</code></sub>
</td>
<td align="center" width="12.5%">
<a href="https://github.com/langchain-ai/langchain"><img src="https://github.com/langchain-ai.png?size=120" alt="LangChain" width="48" height="48" /></a><br/>
<strong>LangChain</strong><br/>
<sub>First-class · <code>pip install semantica[langchain]</code></sub>
</td>
</tr>
<tr>
<th colspan="8" align="left">Already Supported via REST API &amp; MCP</th>
</tr>
<tr>
<td align="center" width="12.5%">
<a href="https://github.com/langchain-ai/langchain"><img src="https://github.com/langchain-ai.png?size=120" alt="LangChain" width="48" height="48" /></a><br/>
<strong>LangChain</strong><br/>
<sub>REST API · MCP</sub>
</td>
<td align="center" width="12.5%">
<a href="https://github.com/langchain-ai/langgraph"><img src="https://github.com/langchain-ai.png?size=120" alt="LangGraph" width="48" height="48" /></a><br/>
<strong>LangGraph</strong><br/>
<sub>REST API · MCP</sub>
@@ -1346,11 +1348,6 @@ MCP setup takes 30 seconds — see [MCP Server](#mcp-server) below.
</tr>
<tr>
<td align="center" width="12.5%">
<a href="https://github.com/langchain-ai/langchain"><img src="https://github.com/langchain-ai.png?size=120" alt="LangChain" width="48" height="48" /></a><br/>
<strong>LangChain</strong><br/>
<sub>Dedicated toolkit</sub>
</td>
<td align="center" width="12.5%">
<a href="https://github.com/run-llama/llama_index"><img src="https://github.com/run-llama.png?size=120" alt="LlamaIndex" width="48" height="48" /></a><br/>
<strong>LlamaIndex</strong><br/>
<sub>Dedicated toolkit</sub>
@@ -1466,18 +1463,18 @@ For contributor / dev-server setup: **[explorer/README.md: Local Setup Guide](ex
---
## What's New in v0.6.5
## What's New in v0.6.6
**Security release — upgrading is strongly recommended.** Fixes for 5 externally-reported vulnerabilities in the Explorer API and graph/triplet store backends, plus a CodeQL-flagged ReDoS:
**Security release — upgrading is strongly recommended.** Fixes for a privately disclosed batch of vulnerabilities spanning backup/restore, database export, outbound requests, and triplet-store backends, plus SSRF hardening across ingestion:
- **Missing authentication on all Explorer API routes** (GHSA-j4mq-hprp-987v, Critical): every route now requires `SEMANTICA_API_KEY`, fails closed (503) rather than open when unconfigured
- **SSRF via redirect bypass in ontology URL fetching** (GHSA-8c7v-62gr-hj6g, High): redirect targets are now re-validated at every hop and the connection is pinned to the validated address, closing a DNS check-then-use race
- **Cypher injection via unvalidated node labels and property keys** (GHSA-482h-hw99-h62p, Critical): Neptune, Neo4j, and FalkorDB now sanitize every label/relationship-type/property-key interpolation site
- **SPARQL injection via unvalidated triplet IRIs** (GHSA-8vgg-8mr4-r236, Critical): Blazegraph, RDF4J, and Jena now validate subject/predicate/object IRIs before interpolation
- **Missing Origin validation on the WebSocket handshake** (GHSA-4643-wpgq-w329, Moderate, anonymous-mode only): `/ws/graph-updates` now checks `Origin` against the same allowlist `CORSMiddleware` enforces for HTTP
- **Polynomial ReDoS in SPARQL query validation** (CodeQL `py/polynomial-redos`): fixed a backtracking regex in the Explorer's SPARQL route
- **Tarball restore path traversal**: `semantica backup restore` now validates every archive member for path containment and rejects symlink/hardlink escapes before extraction
- **Latent SQL injection in `DataExporter.export_table_data()`**: table/schema names are now identifier-allowlisted and `where`/`order_by` fragments are blocklist-checked
- **DNS-rebinding TOCTOU in the shared SSRF guard**: the resolved IP that passes validation is now the one the connection is pinned to, closing the check-then-use race (also closes the `100.64.0.0/10` CGNAT gap)
- **Stored XSS in HTML report generation** and **unvalidated SPARQL object IRIs in AnzoStore** (SPARQL injection): both now escape/validate before interpolation
- **`Authorization`/`Proxy-Authorization` credential leakage across redirects**, plus **SSRF gaps in `FeedIngestor`/`FeedMonitor`, `RepoIngestor`, and the MCP/public-API ingest paths**: all now route through the shared, redirect-safe SSRF guard
- **HTTP response header injection and an unbounded-memory DoS** in the Explorer API, and a **`fastapi`/`python-multipart` ReDoS** (PYSEC-2024-38): floors raised, inputs sanitized, candidate pools capped
Also includes: embedded Oxigraph backend for `TripletStore`, PROV-O trust/spec completeness for `ProvenanceManager`, and the Altair Anzo triplet store backend.
Also ships: **first-class CrewAI integration** (`semantica[crewai]`, extraction/decision tools + a knowledge source), **`ContextGraph` retraction and purge** (GDPR-style erasure without a full `clear()`), a declared **Semantica RDF vocabulary with deterministic entity/relationship IRIs** (stable, diffable exports), and **timezone-aware timestamps** across `export/` and `provenance/`.
→ [Full release notes](RELEASE_NOTES.md) · [Changelog](CHANGELOG.md)
@@ -1509,6 +1506,7 @@ pip install semantica[all] # everything
```bash
pip install semantica[agno] # Agno multi-agent integration
pip install semantica[crewai] # CrewAI integration
pip install semantica[langchain] # LangChain / LangGraph integration
pip install semantica[llm-litellm] # OpenAI, Anthropic, Gemini, Mistral, Llama, Groq, Cohere, Bedrock, Ollama, DeepSeek, and more
pip install semantica[graph-neo4j] # Neo4j graph store (LPG)
pip install semantica[graph-falkordb] # FalkorDB graph store (LPG)
@@ -1561,11 +1559,11 @@ On-premises deployment · Private cloud · Custom domain implementations · SLA-
## Star History
<a href="https://www.star-history.com/?repos=semantica-agi%2Fsemantica&type=date&legend=top-left">
<a href="https://star-history.dera.page/#semantica-agi/semantica&amp;type=date&amp;legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=semantica-agi/semantica&type=date&theme=dark&legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=semantica-agi/semantica&type=date&legend=top-left" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=semantica-agi/semantica&type=date&legend=top-left" />
<source media="(prefers-color-scheme: dark)" srcset="https://star-history.dera.page/svg?repos=semantica-agi/semantica&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://star-history.dera.page/svg?repos=semantica-agi/semantica&amp;type=date&amp;legend=top-left" />
<img alt="Star History Chart" src="https://star-history.dera.page/svg?repos=semantica-agi/semantica&amp;type=date&amp;legend=top-left" />
</picture>
</a>
@@ -1594,6 +1592,23 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines.
---
## Cite Us
If you use Semantica in your research or production systems, please cite it as:
```bibtex
@software{semantica2026,
title = {Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems},
author = {Semantica},
year = {2026},
url = {https://github.com/semantica-agi/semantica}
}
```
All citation formats (APA, MLA, Chicago, IEEE) live on the [Citation](https://docs.getsemantica.ai/citation) page — every format attributes authorship to **Semantica**, not individual contributors.
---
<div align="center">
MIT License · Built by [Semantica](https://github.com/semantica-agi)
cookbook/advanced/01_Advanced_Extraction.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/01_Advanced_Extraction.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/01_Advanced_Extraction.ipynb)\n",
"\n",
"# Advanced Extraction\n",
"\n",
cookbook/advanced/03_Complete_Visualization_Suite.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/03_Complete_Visualization_Suite.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/03_Complete_Visualization_Suite.ipynb)\n",
"\n",
"# Complete Visualization Suite\n",
"\n",
cookbook/advanced/05_Multi_Format_Export.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/05_Multi_Format_Export.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/05_Multi_Format_Export.ipynb)\n",
"\n",
"# Advanced Multi-Format Export\n",
"\n",
cookbook/advanced/08_Reasoning_and_Inference.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/08_Reasoning_and_Inference.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/08_Reasoning_and_Inference.ipynb)\n",
"\n",
"# Reasoning and Inference\n",
"\n",
cookbook/advanced/09_Semantic_Layer_Construction.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/09_Semantic_Layer_Construction.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/09_Semantic_Layer_Construction.ipynb)\n",
"\n",
"# Semantic Layer Construction\n",
"\n",
cookbook/advanced/10_Temporal_Knowledge_Graphs.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/10_Temporal_Knowledge_Graphs.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/10_Temporal_Knowledge_Graphs.ipynb)\n",
"\n",
"# Deep Dive: Temporal Knowledge Graphs\n",
"\n",
cookbook/advanced/12_Unstructured_to_Ontology.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/12_Unstructured_to_Ontology.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/12_Unstructured_to_Ontology.ipynb)\n",
"\n",
"# Unstructured Text to Ontology\n",
"\n",
cookbook/advanced/13_Manual_Ontology_Snowflake_Mapping.ipynb
+1 -1
View File
@@ -18,7 +18,7 @@
"id": "cell-0",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/13_Manual_Ontology_Snowflake_Mapping.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/13_Manual_Ontology_Snowflake_Mapping.ipynb)\n",
"\n",
"# Manual Ontology + Snowflake Mapping\n",
"\n",
cookbook/advanced/14_Datalog_Style_Reasoning.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/14_Datalog_Style_Reasoning.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/14_Datalog_Style_Reasoning.ipynb)\n",
"\n",
"# Datalog-Style Reasoning\n",
"\n",
cookbook/advanced/Advanced_Vector_Store_and_Search.ipynb
+2 -2
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/Advanced_Vector_Store_and_Search.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/advanced/Advanced_Vector_Store_and_Search.ipynb)\n",
"\n",
"# Advanced Vector Store - Made Easy\n",
"\n",
@@ -352,7 +352,7 @@
"- Build a multi-user application\n",
"- Explore the [introduction notebook](../introduction/13_Vector_Store.ipynb) for more basics\n",
"\n",
"**Need Help?** Check our [documentation](https://semantica.readthedocs.io) or ask on [GitHub](https://github.com/Hawksight-AI/semantica)."
"**Need Help?** Check our [documentation](https://semantica.readthedocs.io) or ask on [GitHub](https://github.com/semantica-agi/semantica)."
]
}
],
cookbook/introduction/01_Welcome_to_Semantica.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/01_Welcome_to_Semantica.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/01_Welcome_to_Semantica.ipynb)\n",
"\n",
"Semantica is a **semantic intelligence and knowledge engineering framework**. It helps you:\n",
"\n",
cookbook/introduction/02_Data_Ingestion.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/02_Data_Ingestion.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/02_Data_Ingestion.ipynb)\n",
"\n",
"# Data Ingestion - Comprehensive Guide\n",
"\n",
cookbook/introduction/03_Document_Parsing.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/04_Document_Parsing.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/03_Document_Parsing.ipynb)\n",
"\n",
"# Document Parsing\n",
"\n",
cookbook/introduction/04_Data_Normalization.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/05_Data_Normalization.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/04_Data_Normalization.ipynb)\n",
"\n",
"# Data Normalization\n",
"\n",
cookbook/introduction/05_Entity_Extraction.ipynb
+2 -2
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/05_Entity_Extraction.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/05_Entity_Extraction.ipynb)\n",
"\n",
"# Entity Extraction - Comprehensive Guide\n",
"\n",
@@ -622,7 +622,7 @@
"\n",
"---\n",
"\n",
"**Questions or Issues?** Check out our [GitHub repository](https://github.com/Hawksight-AI/semantica) or [documentation](https://semantica.readthedocs.io)."
"**Questions or Issues?** Check out our [GitHub repository](https://github.com/semantica-agi/semantica) or [documentation](https://semantica.readthedocs.io)."
]
}
],
cookbook/introduction/06_Relation_Extraction.ipynb
+2 -2
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/06_Relation_Extraction.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/06_Relation_Extraction.ipynb)\n",
"\n",
"# Relation Extraction - Comprehensive Guide\n",
"\n",
@@ -599,7 +599,7 @@
"\n",
"---\n",
"\n",
"**Questions or Issues?** Check out our [GitHub repository](https://github.com/Hawksight-AI/semantica) or [documentation](https://semantica.readthedocs.io)."
"**Questions or Issues?** Check out our [GitHub repository](https://github.com/semantica-agi/semantica) or [documentation](https://semantica.readthedocs.io)."
]
}
],
cookbook/introduction/07_Building_Knowledge_Graphs.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/08_Building_Knowledge_Graphs.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/07_Building_Knowledge_Graphs.ipynb)\n",
"\n",
"# Building Knowledge Graphs\n",
"\n",
cookbook/introduction/08_Your_First_Knowledge_Graph.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/09_Your_First_Knowledge_Graph.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/08_Your_First_Knowledge_Graph.ipynb)\n",
"\n",
"# 🚀 Your First Knowledge Graph\n",
"\n",
cookbook/introduction/10_Graph_Analytics.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/11_Graph_Analytics.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/10_Graph_Analytics.ipynb)\n",
"\n",
"# Graph Analytics\n",
"\n",
cookbook/introduction/11_Chunking_and_Splitting.ipynb
+2 -2
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/11_Chunking_and_Splitting.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/11_Chunking_and_Splitting.ipynb)\n",
"\n",
"# Chunking and Splitting - Comprehensive Guide\n",
"\n",
@@ -817,7 +817,7 @@
"\n",
"---\n",
"\n",
"**Questions or Issues?** Check out our [GitHub repository](https://github.com/Hawksight-AI/semantica) or [documentation](https://semantica.readthedocs.io)."
"**Questions or Issues?** Check out our [GitHub repository](https://github.com/semantica-agi/semantica) or [documentation](https://semantica.readthedocs.io)."
]
}
],
cookbook/introduction/12_Embedding_Generation.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/13_Embedding_Generation.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/12_Embedding_Generation.ipynb)\n",
"\n",
"# Embedding Generation\n",
"\n",
cookbook/introduction/13_Vector_Store.ipynb
+2 -2
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/13_Vector_Store.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/13_Vector_Store.ipynb)\n",
"\n",
"# Vector Store - Comprehensive Guide\n",
"\n",
@@ -492,7 +492,7 @@
"\n",
"---\n",
"\n",
"**Questions or Issues?** Check out our [GitHub repository](https://github.com/Hawksight-AI/semantica) or [documentation](https://semantica.readthedocs.io)."
"**Questions or Issues?** Check out our [GitHub repository](https://github.com/semantica-agi/semantica) or [documentation](https://semantica.readthedocs.io)."
]
}
],
cookbook/introduction/14_Ontology.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/14_Ontology.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/14_Ontology.ipynb)\n",
"\n",
"# Ontology Generation \n",
"\n",
cookbook/introduction/15_Export.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/15_Export.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/15_Export.ipynb)\n",
"\n",
"# Export Module - Comprehensive Guide\n",
"\n",
cookbook/introduction/16_Visualization.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/17_Visualization.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/16_Visualization.ipynb)\n",
"\n",
"# Visualization\n",
"\n",
cookbook/introduction/18_Deduplication.ipynb
+1 -1
View File
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/18_Deduplication.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/18_Deduplication.ipynb)\n",
"\n",
"# Deduplication in Semantica\n",
"\n",
cookbook/introduction/19_Context_Module.ipynb
+1 -1
View File
@@ -5,7 +5,7 @@
"id": "c21e9c8d",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/19_Context_Module.ipynb)\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/19_Context_Module.ipynb)\n",
"\n",
"# Context Module — Practical Guide\n",
"\n",
cookbook/introduction/22_Provenance_Tracking.ipynb
+253
View File
@@ -0,0 +1,253 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Provenance Tracking (W3C PROV-O)\n",
"\n",
"## Overview\n",
"\n",
"In high-stakes domains — healthcare, legal, finance, research — a Knowledge Graph is only as trustworthy as its ability to answer **\"where did this fact come from?\"**. Semantica's `provenance` module provides audit-grade, W3C PROV-O-aligned tracking for every entity, relationship and chunk that flows through your pipeline.\n",
"\n",
"In this cookbook you will learn how to:\n",
"\n",
"- Track entities and relationships with **source details** (DOI, page, verbatim quote, confidence)\n",
"- Walk the full **lineage** of a fact (document → chunk → entity → KG)\n",
"- Audit **revision history** and **all sources** behind an entity\n",
"- **Invalidate** a fact without deleting it (prov:Invalidation) — corrections stay provable\n",
"- Verify **tamper-evidence** with chained SHA-256 checksums\n",
"\n",
"**The Scenario:** a research team ingests findings from two scientific papers (with DOIs) into a Knowledge Graph. A regulator later asks: *\"Which paper, which figure, and which exact sentence supports the claim that fish biomass increased by 463%? And was that fact ever corrected?\"*"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install -q semantica"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"from semantica.provenance import (\n",
" ProvenanceManager,\n",
" compute_checksum,\n",
" verify_checksum,\n",
")\n",
"\n",
"# In-memory storage for this demo; pass storage_path=\"provenance.db\"\n",
"# (or a config with provenance.storage_path) for a persistent SQLite backend.\n",
"prov = ProvenanceManager()\n",
"print(\"ProvenanceManager ready (in-memory storage)\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1: Track Entities with Audit-Grade Source Details\n",
"\n",
"Every fact we ingest carries its evidence with it: the **source identifier** (a DOI here), the **location** inside the source (a figure), the **verbatim quote**, and the extractor's **confidence**."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Finding from paper #1\n",
"entry_biomass = prov.track_entity(\n",
" entity_id=\"claim_biomass_increase\",\n",
" source=\"DOI:10.1371/journal.pone.0023601\",\n",
" confidence=0.92,\n",
" source_location=\"Figure 2\",\n",
" source_quote=\"Total fish biomass increased by 463% ...\",\n",
")\n",
"\n",
"# Supporting entity from paper #2\n",
"entry_reserve = prov.track_entity(\n",
" entity_id=\"marine_reserve_1\",\n",
" source=\"DOI:10.1126/science.1088121\",\n",
" confidence=0.88,\n",
" source_location=\"Table 1\",\n",
" source_quote=\"... no-take marine reserve at Cabo Pulmo ...\",\n",
")\n",
"\n",
"print(\"Tracked:\", entry_biomass.entity_id, \"|\", entry_reserve.entity_id)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2: Track the Relationship Between Facts\n",
"\n",
"Facts rarely stand alone. The claim about biomass increase is *about* the marine reserve — that relationship is a first-class provenance-tracked object too.\n",
"\n",
"`track_relationship()` has no dedicated subject/object fields, so by convention we record which two entities it connects inside `metadata`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"rel = prov.track_relationship(\n",
" relationship_id=\"rel_biomass_about_reserve\",\n",
" source=\"DOI:10.1371/journal.pone.0023601\",\n",
" metadata={\n",
" \"type\": \"measured_at\",\n",
" # No dedicated endpoint fields on track_relationship() yet -- record\n",
" # which entities this relationship connects here by convention.\n",
" \"subject_entity_id\": \"claim_biomass_increase\",\n",
" \"object_entity_id\": \"marine_reserve_1\",\n",
" },\n",
")\n",
"\n",
"print(\"Relationship tracked:\", rel.entity_id, \"|\", rel.metadata[\"subject_entity_id\"], \"->\", rel.metadata[\"object_entity_id\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3: Walk the Lineage\n",
"\n",
"`get_lineage` reconstructs everything known about a fact; `trace_lineage` returns the ordered chain of `ProvenanceEntry` records — every version, every activity, every agent that touched it."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"lineage = prov.get_lineage(\"claim_biomass_increase\")\n",
"print(json.dumps(lineage, indent=2, default=str)[:800])\n",
"\n",
"print(\"\\n--- ordered chain ---\")\n",
"for e in prov.trace_lineage(\"claim_biomass_increase\"):\n",
" print(f\"{e.entity_id} | seq#{e.sequence_id} | {e.activity_id}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4: Audit Sources and Revision History\n",
"\n",
"When the regulator asks *\"has this fact ever been corrected?\"*, `revision_history` answers with the full version chain, and `get_all_sources` lists every source document that ever supported the entity."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"revisions = prov.revision_history(\"claim_biomass_increase\")\n",
"print(f\"{len(revisions)} revision(s) on record\")\n",
"\n",
"for s in prov.get_all_sources(\"claim_biomass_increase\"):\n",
" print(\"source:\", s)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5: Invalidate — Correct Without Deleting\n",
"\n",
"Suppose paper #1 is retracted in part. An audit trail must **not** silently delete the fact: `invalidate` archives the pre-invalidation state and appends a fresh `prov:Invalidation` entry naming **who** retracted it and **why**."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"invalidated = prov.invalidate(\n",
" entity_id=\"claim_biomass_increase\",\n",
" agent_id=\"reviewer_dr_chen\",\n",
" reason=\"Partial retraction: Figure 2 statistics corrected by publisher (see erratum).\",\n",
")\n",
"print(\"Invalidated:\", invalidated.entity_id, \"| invalidated flag:\", getattr(invalidated, \"invalidated\", True))\n",
"\n",
"stats = prov.get_statistics()\n",
"print(\"\\nStorage statistics:\", json.dumps(stats, indent=2, default=str))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 6: Verify Tamper-Evidence\n",
"\n",
"Each entry carries a deterministic SHA-256 checksum chained to the previous entry. Recompute and compare to detect any after-the-fact corruption of the provenance record."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# entry_biomass was returned by track_entity in Step 1\n",
"ok = verify_checksum(entry_biomass)\n",
"print(\"Checksum verified:\", ok)\n",
"\n",
"print(\"Computed:\", compute_checksum(entry_biomass)[:16], \"...\")\n",
"print(\"Stored: \", entry_biomass.checksum[:16] if getattr(entry_biomass, 'checksum', None) else \"(see entry fields)\")\n",
"chain = prov.verify_chain()\n",
"print(\"Chain verification:\", json.dumps(chain, default=str)[:200])\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"| Need | Call |\n",
"|---|---|\n",
"| Record a fact's evidence | `prov.track_entity(entity_id, source, confidence=..., source_location=..., source_quote=...)` |\n",
"| Record a relationship | `prov.track_relationship(relationship_id, source, metadata=...)` |\n",
"| Full lineage of a fact | `prov.get_lineage(entity_id)` / `prov.trace_lineage(entity_id)` |\n",
"| \"Was it ever corrected?\" | `prov.revision_history(entity_id)` |\n",
"| \"Which sources support it?\" | `prov.get_all_sources(entity_id)` |\n",
"| Retract without deleting | `prov.invalidate(entity_id, agent_id, reason=...)` |\n",
"| Tamper check | `verify_checksum(entry)` |\n",
"\n",
"### Where to go next\n",
"\n",
"- **Conflict Detection and Resolution** (notebook 17) — what happens when two sources disagree.\n",
"- **Your First Knowledge Graph** (notebook 08) — plug `provenance=True` into extractors so tracking happens automatically during ingestion.\n",
"- The module docstring (`help(semantica.provenance)`) documents opt-in integration with `kg`, `split` and `conflicts` trackers."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.11"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
cookbook/introduction/23_Reasoning.ipynb
+383
View File
@@ -0,0 +1,383 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "b76a5997",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/23_Reasoning.ipynb)\n",
"\n",
"# Reasoning Module — Practical Guide\n",
"\n",
"Semantica's `reasoning` module derives new knowledge from existing facts and knowledge graphs. It ships several strategies behind one facade:\n",
"\n",
"- **`Reasoner`** — unified facade with forward chaining, backward chaining, and one-shot `infer_facts`\n",
"- **`DatalogReasoner`** — semi-naive Datalog fixpoint evaluation with variable queries\n",
"- **`ExplanationGenerator`** — human-readable explanations and reasoning paths for inferred conclusions\n",
"- Plus lower-level engines: `ReteEngine`, `SPARQLReasoner`, `GraphReasoner`, temporal reasoning\n",
"\n",
"This notebook walks through the facade, the Datalog engine, and explanations. All APIs are verified against `semantica/reasoning/`."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "52073af7",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:45:55.427457Z",
"iopub.status.busy": "2026-08-26T18:45:55.427247Z",
"iopub.status.idle": "2026-08-26T18:45:57.266607Z",
"shell.execute_reply": "2026-08-26T18:45:57.264783Z"
}
},
"outputs": [],
"source": [
"!pip install -q semantica"
]
},
{
"cell_type": "markdown",
"id": "06deb916",
"metadata": {},
"source": [
"## 1) Forward chaining with the `Reasoner` facade\n",
"\n",
"Facts are simple `Predicate(args)` strings. Rules use `IF <conditions> THEN <conclusion>` with `?x`-style variables. `forward_chain()` derives everything possible and returns a list of `InferenceResult` objects."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "519ca92d",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:45:57.270791Z",
"iopub.status.busy": "2026-08-26T18:45:57.270352Z",
"iopub.status.idle": "2026-08-26T18:45:59.991941Z",
"shell.execute_reply": "2026-08-26T18:45:59.990678Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>Progress</th><th>ETA</th><th>Rate</th><th>Time</th><th>Extracted</th></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>100.0%</td><td>-</td><td>-</td><td>0.00s</td><td>-</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>DatalogReasoner</td><td>100.0%</td><td>-</td><td>-</td><td>0.00s</td><td>-</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>ExplanationGenerator</td><td>100.0%</td><td>-</td><td>-</td><td>0.00s</td><td>-</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"🔄 Semantica is reasoning: Performing forward chaining 🤔 reasoning Reasoner |░░░░░░░░░░░░░░░| 0.0% ETA: - Rate: - Time: 0.00s Extracted: -"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Inferred 2 new facts\n",
" Human(Jane) (rule: Rule 1, confidence: 1.0)\n",
" Human(John) (rule: Rule 1, confidence: 1.0)\n"
]
}
],
"source": [
"from semantica.reasoning import Reasoner\n",
"\n",
"reasoner = Reasoner()\n",
"\n",
"reasoner.add_fact(\"Person(John)\")\n",
"reasoner.add_fact(\"Person(Jane)\")\n",
"reasoner.add_rule(\"IF Person(?x) THEN Human(?x)\")\n",
"\n",
"results = reasoner.forward_chain()\n",
"print(f\"Inferred {len(results)} new facts\")\n",
"for res in results:\n",
" print(f\" {res.conclusion} (rule: {res.rule_used.name}, confidence: {res.confidence})\")"
]
},
{
"cell_type": "markdown",
"id": "c1131c45",
"metadata": {},
"source": [
"## 2) One-shot inference with `infer_facts`\n",
"\n",
"`infer_facts(facts, rules)` **adds** the given facts and rules to this `Reasoner` instance, runs forward chaining to fixpoint, and returns the derived facts as strings. It does not reset the instance's existing state — create a fresh `Reasoner()` first if you need isolation between runs."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "26249990",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:45:59.995447Z",
"iopub.status.busy": "2026-08-26T18:45:59.995069Z",
"iopub.status.idle": "2026-08-26T18:46:00.004107Z",
"shell.execute_reply": "2026-08-26T18:46:00.002873Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"['Employee(Jane, Acme)', 'Employee(John, Acme)']"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantica.reasoning import Reasoner\n",
"\n",
"derived = Reasoner().infer_facts(\n",
" facts=[\"WorksFor(John, Acme)\", \"WorksFor(Jane, Acme)\"],\n",
" rules=[\"IF WorksFor(?x, ?y) THEN Employee(?x, ?y)\"],\n",
")\n",
"derived"
]
},
{
"cell_type": "markdown",
"id": "d5504a38",
"metadata": {},
"source": [
"## 3) Backward chaining: proving a goal\n",
"\n",
"`backward_chain(goal)` works backwards from a conclusion through the rules. It returns the `InferenceResult` that proves the goal, or `None`."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "c4ef85dd",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:00.007740Z",
"iopub.status.busy": "2026-08-26T18:46:00.007346Z",
"iopub.status.idle": "2026-08-26T18:46:00.015561Z",
"shell.execute_reply": "2026-08-26T18:46:00.014145Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Human(John)\n",
"premises: ['Person(John)']\n"
]
}
],
"source": [
"from semantica.reasoning import Reasoner\n",
"\n",
"reasoner = Reasoner()\n",
"reasoner.add_fact(\"Person(John)\")\n",
"reasoner.add_rule(\"IF Person(?x) THEN Human(?x)\")\n",
"\n",
"proof = reasoner.backward_chain(\"Human(John)\")\n",
"print(proof.conclusion if proof else \"not provable\")\n",
"print(\"premises:\", proof.premises if proof else None)"
]
},
{
"cell_type": "markdown",
"id": "b245581d",
"metadata": {},
"source": [
"## 4) Re-run safety\n",
"\n",
"`add_rule` deduplicates rules with identical conditions and conclusion, so re-executing a setup cell (the common Jupyter re-run) does not duplicate rules — see issue #732."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "fb2aeb39",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:00.019091Z",
"iopub.status.busy": "2026-08-26T18:46:00.018881Z",
"iopub.status.idle": "2026-08-26T18:46:00.024042Z",
"shell.execute_reply": "2026-08-26T18:46:00.022836Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Skipping duplicate rule (same conditions/conclusion as 'rule_1'): IF Person(?x) THEN Human(?x)\n"
]
},
{
"data": {
"text/plain": [
"1"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantica.reasoning import Reasoner\n",
"\n",
"reasoner = Reasoner()\n",
"reasoner.add_fact(\"Person(John)\")\n",
"\n",
"# Simulate a Jupyter cell re-run: add the same rule twice\n",
"r1 = reasoner.add_rule(\"IF Person(?x) THEN Human(?x)\")\n",
"r2 = reasoner.add_rule(\"IF Person(?x) THEN Human(?x)\")\n",
"\n",
"len(reasoner.rules)"
]
},
{
"cell_type": "markdown",
"id": "ba2e5c4a",
"metadata": {},
"source": [
"## 5) Datalog reasoning\n",
"\n",
"`DatalogReasoner` uses classic Datalog syntax (`head :- body.`) and semi-naive fixpoint evaluation. Queries return variable bindings as a list of dicts — use uppercase variables to ask *which* facts hold."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "9ec5c0c4",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:00.026769Z",
"iopub.status.busy": "2026-08-26T18:46:00.026588Z",
"iopub.status.idle": "2026-08-26T18:46:00.034963Z",
"shell.execute_reply": "2026-08-26T18:46:00.032672Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"[{'X': 'tom', 'Z': 'ann'}]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantica.reasoning import DatalogReasoner\n",
"\n",
"datalog = DatalogReasoner()\n",
"datalog.add_fact(\"parent(tom, mary)\")\n",
"datalog.add_fact(\"parent(mary, ann)\")\n",
"datalog.add_rule(\"grandparent(X, Z) :- parent(X, Y), parent(Y, Z)\")\n",
"\n",
"datalog.derive_all()\n",
"datalog.query(\"grandparent(X, Z)\")"
]
},
{
"cell_type": "markdown",
"id": "d4f0689b",
"metadata": {},
"source": [
"## 6) Explanations for inferred conclusions\n",
"\n",
"`ExplanationGenerator` turns `InferenceResult` objects into structured `Explanation` and `ReasoningPath` records, so agents can show *why* they believe a derived fact."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "19dcd3a7",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:00.038649Z",
"iopub.status.busy": "2026-08-26T18:46:00.038396Z",
"iopub.status.idle": "2026-08-26T18:46:00.059188Z",
"shell.execute_reply": "2026-08-26T18:46:00.057805Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"('Explanation', 'ReasoningPath')"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantica.reasoning import Reasoner, ExplanationGenerator\n",
"\n",
"reasoner = Reasoner()\n",
"reasoner.add_fact(\"Person(John)\")\n",
"reasoner.add_rule(\"IF Person(?x) THEN Human(?x)\")\n",
"results = reasoner.forward_chain()\n",
"\n",
"gen = ExplanationGenerator()\n",
"explanation = gen.generate_explanation(results[0])\n",
"path = gen.show_reasoning_path(results[0])\n",
"\n",
"type(explanation).__name__, type(path).__name__"
]
},
{
"cell_type": "markdown",
"id": "fb882ee4",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"| Task | API |\n",
"|---|---|\n",
"| Derive all new facts | `Reasoner.forward_chain()` |\n",
"| One-shot inference | `Reasoner.infer_facts(facts, rules)` |\n",
"| Prove a goal | `Reasoner.backward_chain(goal)` |\n",
"| Datalog fixpoint | `DatalogReasoner.derive_all()` + `query(\"p(X, Y)\")` |\n",
"| Explain a conclusion | `ExplanationGenerator.generate_explanation(result)` |\n",
"\n",
"See also `semantica/reasoning/reasoning_usage.md` and the module docstrings for `ReteEngine`, `SPARQLReasoner`, and temporal reasoning."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
cookbook/introduction/24_Change_Management.ipynb
+299
View File
@@ -0,0 +1,299 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "8d7096ea",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/24_Change_Management.ipynb)\n",
"\n",
"# Change Management — Practical Guide\n",
"\n",
"Semantica's `change_management` module provides versioning, audit trails, and data-integrity checks for knowledge graphs and ontologies:\n",
"\n",
"- **`ChangeLogEntry`** — standardized change metadata (validated timestamp/author)\n",
"- **`InMemoryVersionStorage` / `SQLiteVersionStorage`** — version snapshot storage with named tags\n",
"- **`compute_checksum` / `verify_checksum`** — SHA-256 integrity verification\n",
"\n",
"This notebook runs a complete save → tag → verify → tamper-detect cycle. All outputs are real executed results verified against the repository's `semantica/change_management/` source at the time of writing (the `pip install` cell may fetch a newer release with slightly different behavior)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "7bdffec1",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:37.171333Z",
"iopub.status.busy": "2026-08-26T18:46:37.171183Z",
"iopub.status.idle": "2026-08-26T18:46:39.060860Z",
"shell.execute_reply": "2026-08-26T18:46:39.059594Z"
}
},
"outputs": [],
"source": [
"!pip install -q semantica"
]
},
{
"cell_type": "markdown",
"id": "169efee1",
"metadata": {},
"source": [
"## 1) A `ChangeLogEntry` records *who* changed *what*, *when*\n",
"\n",
"`author` must be a valid email — the dataclass validates on construction (`ValidationError` otherwise), which keeps audit trails clean."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "5b17acdb",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:39.064077Z",
"iopub.status.busy": "2026-08-26T18:46:39.063818Z",
"iopub.status.idle": "2026-08-26T18:46:39.321881Z",
"shell.execute_reply": "2026-08-26T18:46:39.321036Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"ChangeLogEntry(timestamp='2026-08-15T09:00:00Z', author='demo@example.com', description='initial version', change_id=None, related_changes=[])"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantica.change_management import ChangeLogEntry\n",
"\n",
"entry = ChangeLogEntry(\n",
" timestamp=\"2026-08-15T09:00:00Z\",\n",
" author=\"demo@example.com\",\n",
" description=\"initial version\",\n",
")\n",
"entry"
]
},
{
"cell_type": "markdown",
"id": "53d8df5c",
"metadata": {},
"source": [
"## 2) Save a versioned snapshot\n",
"\n",
"A snapshot is a dict with a required `label` plus your payload. Here we attach the KG data, the change log, and a SHA-256 `checksum` computed over everything except the checksum field itself."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "fec16f24",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:39.325528Z",
"iopub.status.busy": "2026-08-26T18:46:39.325140Z",
"iopub.status.idle": "2026-08-26T18:46:39.331480Z",
"shell.execute_reply": "2026-08-26T18:46:39.330586Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantica.change_management import InMemoryVersionStorage, compute_checksum\n",
"\n",
"storage = InMemoryVersionStorage()\n",
"\n",
"snapshot = {\n",
" \"label\": \"v1.0.0\",\n",
" \"data\": {\"entities\": {\"acme\": {\"type\": \"Company\"}}},\n",
" \"change_log\": {\n",
" \"timestamp\": entry.timestamp,\n",
" \"author\": entry.author,\n",
" \"description\": entry.description,\n",
" },\n",
"}\n",
"snapshot[\"checksum\"] = compute_checksum({k: v for k, v in snapshot.items() if k != \"checksum\"})\n",
"\n",
"storage.save(snapshot)\n",
"storage.exists(\"v1.0.0\")"
]
},
{
"cell_type": "markdown",
"id": "0f1c603b",
"metadata": {},
"source": [
"## 3) Named tags pin a version for releases\n",
"\n",
"`save_tag` / `get_tag` map stable names (e.g. `release`) to version labels, decoupling consumers from label churn."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "62f7643e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:39.335182Z",
"iopub.status.busy": "2026-08-26T18:46:39.334886Z",
"iopub.status.idle": "2026-08-26T18:46:39.339586Z",
"shell.execute_reply": "2026-08-26T18:46:39.338568Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"('v1.0.0', ['v1.0.0'])"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"storage.save_tag(\"release\", \"v1.0.0\")\n",
"\n",
"storage.get_tag(\"release\"), [s[\"label\"] for s in storage.list_all()]"
]
},
{
"cell_type": "markdown",
"id": "96df12da",
"metadata": {},
"source": [
"## 4) Verify integrity — and catch tampering\n",
"\n",
"`verify_checksum(snapshot)` recomputes the SHA-256 over the snapshot (minus its `checksum` field) and compares. A single mutated character in the data flips the result to `False`."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "26d0de85",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:39.342653Z",
"iopub.status.busy": "2026-08-26T18:46:39.342466Z",
"iopub.status.idle": "2026-08-26T18:46:39.346714Z",
"shell.execute_reply": "2026-08-26T18:46:39.345623Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"intact: True\n",
"tampered: False\n"
]
}
],
"source": [
"from semantica.change_management import verify_checksum\n",
"\n",
"stored = storage.get(\"v1.0.0\")\n",
"print(\"intact:\", verify_checksum(stored))\n",
"\n",
"tampered = storage.get(\"v1.0.0\")\n",
"tampered[\"data\"][\"entities\"][\"acme\"][\"note\"] = \"mutated after the fact\"\n",
"print(\"tampered:\", verify_checksum(tampered))"
]
},
{
"cell_type": "markdown",
"id": "bd14c3e4",
"metadata": {},
"source": [
"## 5) Retiring a version\n",
"\n",
"`delete(label)` removes a snapshot; tags pointing at it are your responsibility to update."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "de9fe3e5",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:46:39.349814Z",
"iopub.status.busy": "2026-08-26T18:46:39.349513Z",
"iopub.status.idle": "2026-08-26T18:46:39.354710Z",
"shell.execute_reply": "2026-08-26T18:46:39.353669Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"storage.delete(\"v1.0.0\")\n",
"storage.exists(\"v1.0.0\")"
]
},
{
"cell_type": "markdown",
"id": "ab667b32",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"| Task | API |\n",
"|---|---|\n",
"| Record audit metadata | `ChangeLogEntry(timestamp, author=email, description)` |\n",
"| Persist a version | `InMemoryVersionStorage().save({\"label\": ..., ...})` |\n",
"| Pin a release name | `save_tag(\"release\", \"v1.0.0\")` / `get_tag(\"release\")` |\n",
"| Integrity check | `compute_checksum(snap)` / `verify_checksum(snap)` |\n",
"| Persistent backend | `SQLiteVersionStorage(path)` — same interface |\n",
"\n",
"See also `semantica/change_management/change_management_usage.md` for the manager classes (`TemporalVersionManager`, `OntologyVersionManager`)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
cookbook/introduction/25_Seed_Data.ipynb
+314
View File
@@ -0,0 +1,314 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "6eb4dfba",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/25_Seed_Data.ipynb)\n",
"\n",
"# Seed Data — Practical Guide\n",
"\n",
"The `seed` module bootstraps a knowledge graph from **trusted, pre-known data** (CSV/JSON/database/API sources) before any extraction runs. This gives extraction a foundation to link against instead of starting from an empty graph.\n",
"\n",
"Key pieces:\n",
"\n",
"- **`SeedDataManager`** — registers data sources and builds foundation graphs\n",
"- **`create_foundation_graph()`** — turns registered sources into `entities` + `relationships` + `metadata`\n",
"- **`validate_quality()`** — checks a foundation graph before you commit it\n",
"\n",
"All examples below were executed against `semantica/seed/seed_manager.py`."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "32f80cc6",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:51:18.716466Z",
"iopub.status.busy": "2026-08-26T18:51:18.716264Z",
"iopub.status.idle": "2026-08-26T18:51:20.533828Z",
"shell.execute_reply": "2026-08-26T18:51:20.531402Z"
}
},
"outputs": [],
"source": [
"!pip install -q semantica"
]
},
{
"cell_type": "markdown",
"id": "75136e5f",
"metadata": {},
"source": [
"## 1) Prepare a seed CSV and register the source\n",
"\n",
"`register_source(name, format, location, entity_type=...)` records where trusted data lives. `verified=True` (the default) marks the source as pre-validated."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "a8089e1f",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:51:20.538772Z",
"iopub.status.busy": "2026-08-26T18:51:20.538323Z",
"iopub.status.idle": "2026-08-26T18:51:20.675403Z",
"shell.execute_reply": "2026-08-26T18:51:20.674060Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import csv\n",
"import tempfile\n",
"from pathlib import Path\n",
"from semantica.seed import SeedDataManager\n",
"\n",
"# Write the sample CSV into a session-scoped temp directory so we never\n",
"# clobber a companies.csv that might exist in the user's working directory.\n",
"seed_csv = Path(tempfile.mkdtemp(prefix=\"semantica-seed-\")) / \"companies.csv\"\n",
"with open(seed_csv, \"w\", newline=\"\") as f:\n",
" writer = csv.DictWriter(f, fieldnames=[\"id\", \"name\", \"type\", \"industry\"])\n",
" writer.writeheader()\n",
" writer.writerow({\"id\": \"c1\", \"name\": \"Acme\", \"type\": \"Company\", \"industry\": \"robotics\"})\n",
" writer.writerow({\"id\": \"c2\", \"name\": \"Globex\", \"type\": \"Company\", \"industry\": \"energy\"})\n",
"\n",
"manager = SeedDataManager()\n",
"manager.register_source(\"companies\", format=\"csv\", location=str(seed_csv), entity_type=\"Company\")\n"
]
},
{
"cell_type": "markdown",
"id": "e87221ba",
"metadata": {},
"source": [
"## 2) Load records from a registered source\n",
"\n",
"`load_source(name)` reads the source and enriches each record with `entity_type` and `source` provenance keys."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "f932e550",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:51:20.679424Z",
"iopub.status.busy": "2026-08-26T18:51:20.679156Z",
"iopub.status.idle": "2026-08-26T18:51:20.690659Z",
"shell.execute_reply": "2026-08-26T18:51:20.688812Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>Progress</th><th>ETA</th><th>Rate</th><th>Time</th><th>Extracted</th></tr><tr><td>✅</td><td>Semantica is seeding</td><td>🌱 seed</td><td>SeedDataManager</td><td>100.0%</td><td>-</td><td>-</td><td>0.00s</td><td>-</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"🔄 Semantica is seeding: Loading seed data from CSV: /var/folders/7s/bvvstgs10y963tz6_4bbnklr0000gn/T/semantica-seed-eu9__ep1/companies.csv 🌱 seed SeedDataManager |░░░░░░░░░░░░░░░| 0.0% ETA: - Rate: - Time: 0.00s Extracted: -"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"loaded 2 records\n"
]
},
{
"data": {
"text/plain": [
"{'id': 'c1',\n",
" 'name': 'Acme',\n",
" 'type': 'Company',\n",
" 'industry': 'robotics',\n",
" 'entity_type': 'Company',\n",
" 'source': 'companies'}"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"records = manager.load_source(\"companies\")\n",
"print(f\"loaded {len(records)} records\")\n",
"records[0]"
]
},
{
"cell_type": "markdown",
"id": "f2ebce64",
"metadata": {},
"source": [
"## 3) Build the foundation graph\n",
"\n",
"`create_foundation_graph()` converts every registered source into graph-ready entities and relationships. Entities carry `confidence: 1.0` — seed data is trusted by definition."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "09388c31",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:51:20.695259Z",
"iopub.status.busy": "2026-08-26T18:51:20.694928Z",
"iopub.status.idle": "2026-08-26T18:51:20.708595Z",
"shell.execute_reply": "2026-08-26T18:51:20.707072Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"['entities', 'metadata', 'relationships']"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"foundation = manager.create_foundation_graph()\n",
"sorted(foundation.keys())"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "4610a59f",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:51:20.713136Z",
"iopub.status.busy": "2026-08-26T18:51:20.712795Z",
"iopub.status.idle": "2026-08-26T18:51:20.718637Z",
"shell.execute_reply": "2026-08-26T18:51:20.716835Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"{'id': 'c1',\n",
" 'text': 'Acme',\n",
" 'type': 'Company',\n",
" 'confidence': 1.0,\n",
" 'metadata': {'industry': 'robotics', 'source': 'companies'}}"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"foundation[\"entities\"][0]"
]
},
{
"cell_type": "markdown",
"id": "f3a52dc7",
"metadata": {},
"source": [
"## 4) Validate quality before committing\n",
"\n",
"`validate_quality(foundation_graph)` returns `valid`, `errors`, `warnings`, and `metrics` so you can gate bad seed data before it pollutes the graph."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "4eb7e664",
"metadata": {
"execution": {
"iopub.execute_input": "2026-08-26T18:51:20.722674Z",
"iopub.status.busy": "2026-08-26T18:51:20.722118Z",
"iopub.status.idle": "2026-08-26T18:51:20.732003Z",
"shell.execute_reply": "2026-08-26T18:51:20.730170Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"quality = manager.validate_quality(foundation)\n",
"quality[\"valid\"]"
]
},
{
"cell_type": "markdown",
"id": "b534be89",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"| Task | API |\n",
"|---|---|\n",
"| Register a trusted source | `register_source(name, format, location, entity_type=...)` |\n",
"| Load records | `load_source(name)` — adds `entity_type` / `source` keys |\n",
"| Direct file load | `load_from_csv(path)` / `load_from_json(path)` |\n",
"| Build the graph | `create_foundation_graph()` → `entities` / `relationships` / `metadata` |\n",
"| Gate bad data | `validate_quality(graph)` → `valid` / `errors` / `warnings` / `metrics` |\n",
"\n",
"See also `semantica/seed/seed_usage.md` for `load_from_database`, `load_from_api`, and `integrate_with_extracted`."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/citation.md
+9 -10
View File
@@ -13,26 +13,25 @@ icon: "quote-left"
<Tab title="BibTeX">
```bibtex
@software{semantica2026,
title = {Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems},
author = {Semantica},
year = {2026},
url = {https://github.com/semantica-agi/semantica},
version = {0.6.5},
doi = {10.5281/zenodo.XXXXXXX}
title = {Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems},
author = {Semantica},
year = {2026},
url = {https://github.com/semantica-agi/semantica},
doi = {10.5281/zenodo.XXXXXXX}
}
```
</Tab>
<Tab title="APA">
Semantica. (2026). *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems* (Version 0.6.5) \[Computer software\]. https://github.com/semantica-agi/semantica
Semantica. (2026). *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems* \[Computer software\]. https://github.com/semantica-agi/semantica
</Tab>
<Tab title="MLA">
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. Version 0.6.5, GitHub, 2026, https://github.com/semantica-agi/semantica.
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. GitHub, 2026, https://github.com/semantica-agi/semantica.
</Tab>
<Tab title="Chicago">
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. Version 0.6.5. GitHub, 2026. https://github.com/semantica-agi/semantica.
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. GitHub, 2026. https://github.com/semantica-agi/semantica.
</Tab>
<Tab title="IEEE">
Semantica, "Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems," Version 0.6.5, GitHub, 2026. \[Online\]. Available: https://github.com/semantica-agi/semantica
Semantica, "Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems," GitHub, 2026. \[Online\]. Available: https://github.com/semantica-agi/semantica
</Tab>
</Tabs>
docs/cookbook.md
+5 -1
View File
@@ -35,6 +35,7 @@ Essential guides to master the Semantica framework.
- **[Vector Store](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/13_Vector_Store.ipynb)** — Setting up vector stores for similarity search and retrieval. *Intermediate*
- **[Graph Store](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/09_Graph_Store.ipynb)** — Persisting knowledge graphs in Neo4j or FalkorDB. Topics: Neo4j, Cypher, Persistence · *Intermediate*
- **[Ontology](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/14_Ontology.ipynb)** — Defining domain schemas and ontologies to structure your data. Topics: OWL, RDF, Schema Design · *Intermediate*
- **[Seed Data](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/25_Seed_Data.ipynb)** — Bootstrapping a knowledge graph from trusted CSV, JSON, database, and API sources before extraction runs. Topics: SeedDataManager, Foundation Graphs · *Intermediate*
## Advanced Concepts
@@ -50,6 +51,9 @@ Deep dive into advanced features, customization, and complex workflows.
- **[Multi-Source Integration](https://github.com/semantica-agi/semantica/blob/main/cookbook/advanced/06_Multi_Source_Data_Integration.ipynb)** — Merging data from disparate sources into a unified graph. Topics: Entity Resolution, Merging, Fusion · *Advanced*
- **[Reasoning and Inference](https://github.com/semantica-agi/semantica/blob/main/cookbook/advanced/08_Reasoning_and_Inference.ipynb)** — Using logical reasoning to infer new knowledge from existing facts. Topics: Logic Rules, Inference Engines · *Advanced*
- **[Temporal Knowledge Graphs](https://github.com/semantica-agi/semantica/blob/main/cookbook/advanced/10_Temporal_Knowledge_Graphs.ipynb)** — Modeling and querying data that changes over time. Topics: Time Series, Temporal Logic, Allen Algebra · *Advanced*
- **[Provenance Tracking](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/22_Provenance_Tracking.ipynb)** — Audit-grade, W3C PROV-O-aligned tracking of where every entity, relationship, and chunk came from. Topics: PROV-O, Lineage, Checksums, Invalidation · *Advanced*
- **[Reasoning Module](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/23_Reasoning.ipynb)** — Deriving new knowledge from existing facts with forward chaining, backward chaining, and Datalog strategies. Topics: Reasoner, Datalog, Explanations · *Advanced*
- **[Change Management](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/24_Change_Management.ipynb)** — Versioning, audit trails, and data-integrity checks for knowledge graphs and ontologies. Topics: ChangeLogEntry, Version Storage, Data Integrity · *Advanced*
## How to Run
@@ -80,6 +84,6 @@ Deep dive into advanced features, customization, and complex workflows.
You can also run the cookbook using Docker:
```bash
docker run -p 8888:8888 hawksight/semantica-cookbook
docker run -p 8888:8888 semantica/semantica-cookbook
```
</Tip>
docs/docs.json
+1
View File
@@ -103,6 +103,7 @@
"pages": [
"integrations/agno",
"integrations/crewai",
"integrations/langchain",
"integrations/docling",
"integrations/snowflake",
"integrations/databricks"
docs/faq.md
+1 -1
View File
@@ -17,7 +17,7 @@ icon: "circle-question"
| API key required? | Optional: pattern extraction works with no keys |
| Works with LangChain / LlamaIndex? | Yes: Semantica is a layer on top, not a replacement |
| Production-ready? | Yes: 1,000+ tests, v0.5.0 ships with 12 security fixes |
| Latest version? | **v0.6.5** (August 2026) |
| Latest version? | **v0.6.6** (August 2026) |
| Local LLMs? | Yes: Ollama via LiteLLM, HuggingFaceLLM for air-gapped |
docs/getting-started.md
+1 -1
View File
@@ -42,7 +42,7 @@ icon: "rocket"
Verify installation:
```python
import semantica
print(semantica.__version__) # 0.6.5
print(semantica.__version__) # 0.6.6
```
</Check>
</Step>
docs/governance.md
+2 -2
View File
@@ -4,12 +4,12 @@ description: "Project governance model: roles, decision process, release cadence
icon: "scale-balanced"
---
> Semantica is maintained by Hawksight AI with community contributions under an open governance model.
> Semantica is maintained by the Semantica team with community contributions under an open governance model.
## Roles
- **Maintainers** — Hawksight AI team: review and merge PRs, manage releases and code quality, set project direction and community standards.
- **Maintainers** — Semantica team: review and merge PRs, manage releases and code quality, set project direction and community standards.
- **Contributors** — Submit code, documentation, and bug reports. Help with issues and reviews. Recognized in [CONTRIBUTORS.md](https://github.com/semantica-agi/semantica/blob/main/CONTRIBUTORS.md).
- **Community Members** — Use Semantica, provide feedback, share use cases, and participate in GitHub Discussions and Discord.
docs/guides/context-graphs.md
+29
View File
@@ -436,6 +436,35 @@ d = graph.to_dict()
# d["statistics"] → {"node_count": int, "edge_count": int}
```
For a human-editable, version-control-friendly representation, save a Markdown
directory instead:
```python
graph.save_to_file("context_graph/", format="markdown")
restored = ContextGraph(advanced_analytics=True)
restored.load_from_file("context_graph/", format="markdown")
```
The directory contains a versioned `graph.md` manifest for graph identity,
relationships, and cross-graph link descriptors, plus one file per node under
`nodes/`. A node's content is its Markdown body; its ID, type, properties,
metadata, and temporal validity are YAML frontmatter. Node, edge, family, graph,
and cross-graph link IDs are preserved across round trips.
Markdown loading uses replacement semantics, like `from_dict()`: it parses and
validates the complete directory before replacing the current graph. Invalid YAML,
duplicate IDs, unsupported versions, and unsafe filesystem links fail without
partially mutating the graph. As with JSON loading, an edge endpoint without a node
file creates an `entity` stub node. Symlinks, Windows directory junctions, and other
Windows reparse points are rejected.
Re-exporting to an existing managed directory atomically replaces it, removing stale
node files. Before replacement, Semantica validates the complete canonical export
layout, not just the manifest header. Untracked files, assets, extra directories, or
renamed node files therefore cause the export to fail closed instead of being deleted.
Keep attachments and hand-written indexes outside the managed export directory.
If the graph had cross-graph links created with `link_graph()`, call `resolve_links()` after loading to restore live navigation — object references cannot be serialized, so they must be reconnected manually:
```python
docs/guides/pipeline.md
+4 -4
View File
@@ -222,10 +222,10 @@ builder.register_step_handler("ner_extract", run_ner)
builder.register_step_handler("triplet_extract", run_triplets)
builder.register_step_handler("kg_merge", merge_into_graph)
builder.add_step("ingest", "file_ingest", handler=ingest_stix_bundles, path="./stix_bundles/")
builder.add_step("ner", "ner_extract", handler=run_ner, confidence_threshold=0.75)
builder.add_step("triplets", "triplet_extract", handler=run_triplets, include_temporal=True)
builder.add_step("store", "kg_merge", handler=merge_into_graph, output_path="./cti_output/")
builder.add_step("ingest", "file_ingest", path="./stix_bundles/")
builder.add_step("ner", "ner_extract", confidence_threshold=0.75)
builder.add_step("triplets", "triplet_extract", include_temporal=True)
builder.add_step("store", "kg_merge", output_path="./cti_output/")
# ingest feeds both ner and triplets in parallel
builder.connect_steps("ingest", "ner")
docs/guides/reasoning.md
+19 -15
View File
@@ -150,6 +150,12 @@ HighRiskSupplier(DELTA-3) conf=100% rule=Rule 3
DELTA-3 is flagged even though no document described it that way — the system traced: DELTA-3 supplied GAMMA-7, and GAMMA-7 exploits critical CVEs. For rules that need priority ordering or graded confidence, use the `Rule` dataclass:
If a rule has side-effecting actions, one concrete activation runs those
actions at most once on a Reasoner instance. Re-running `forward_chain()` is
therefore safe: already-attempted actions are not repeated. Use
`reasoner.reset_action_history()` when you intentionally want to replay them;
`reasoner.clear()` and `reasoner.reset()` also clear the history.
```python
# Higher priority rules fire first; confidence propagates into InferenceResult.confidence
reasoner.add_rule(Rule(
@@ -269,7 +275,7 @@ print("Loaded {} facts from graph".format(count))
## Step 5 — SPARQL queries over enriched working memory
After forward chaining has derived new facts, `SPARQLReasoner` lets you query the enriched working memory using SPARQL triple-pattern matching with optional inference expansion:
After forward chaining has derived new facts, `SPARQLReasoner` prepares SPARQL queries over the enriched working memory with optional inference expansion:
```python
from semantica.reasoning import SPARQLReasoner
@@ -288,22 +294,13 @@ query = """
}
"""
# execute_query() runs: expansion → inference → deduplication
result = sparql.execute_query(query)
for binding in result.bindings:
print("Actor: {:15s} CVE: {}".format(
binding.get("actor", "?"),
binding.get("cve", "?"),
))
# metadata shows how many results came from inference vs ground facts
print("Original: {} Inferred: {}".format(
result.metadata.get("original_count", 0),
result.metadata.get("inferred_count", 0),
))
# expand_query() applies inference rules to the query text:
expanded = sparql.expand_query(query)
print(expanded)
```
`execute_query()` is not implemented yet: no triplet-store execution path exists, so it raises `NotImplementedError` rather than returning an empty result set that callers would misread as "no matches". Until execution lands, run the expanded query against your RDF store directly (for example with `rdflib`).
Inspect the expanded query before running it:
```python
@@ -369,6 +366,13 @@ engine.reset()
The rule network is compiled once by `build_network()`. Each subsequent `add_fact()` call propagates incrementally through only the nodes whose conditions it satisfies — not the full rule set — which keeps evaluation cost proportional to the number of new activations rather than the total rule count.
With a Reasoner bound, Rete action side effects are attempted once per rule,
bindings, and matched fact identity. Passing the same match to
`execute_matches()` again still returns the same conclusion, but does not repeat
its actions. Call `engine.reset_action_history()` to replay actions without
clearing working memory. `engine.reset()` and `engine.build_network()` also
clear the action history.
## Step 7 — Temporal interval reasoning
`TemporalReasoningEngine` computes Allen interval relations between time windows, letting you identify whether two events overlap, one contains the other, they meet at a boundary, and so on across your graph:
docs/guides/shacl-validation.md
+59 -21
View File
@@ -8,7 +8,7 @@ icon: "shield-check"
SHACL (Shapes Constraint Language) is a standard for validating graph-based data. While an ontology defines the conceptual *schema* (the "what" exists in your domain), SHACL defines the structural *rules and constraints* (the "how" it should be structured).
In Semantica, `SHACLGenerator` produces constraint rules (shapes) based on your ontology, and `_run_pyshacl` evaluates your actual data against these rules. If a node violates a rule (e.g., missing a required property or using the wrong datatype), a detailed violation report is generated.
In Semantica, `SHACLGenerator` produces constraint rules (shapes) based on your ontology, and the public `run_shacl_validation` function evaluates your actual data against these rules. If a node violates a rule (e.g., missing a required property or using the wrong datatype), a detailed violation report is generated. The historical `_run_pyshacl` name remains available as a compatibility alias.
## Why Use SHACL Validation?
@@ -55,7 +55,7 @@ Let's look at a simple, universally understood example: ensuring every `Employee
```python
from semantica.context import ContextGraph
from semantica.ontology import OntologyGenerator, SHACLGenerator, PropertyShape
from semantica.ontology.ontology_validator import _run_pyshacl
from semantica.ontology import run_shacl_validation
# 1. Prepare your data graph
graph = ContextGraph()
@@ -95,7 +95,7 @@ data_ttl = """
"""
# 5. Run Validation
report = _run_pyshacl(data_ttl, shacl_ttl)
report = run_shacl_validation(data_ttl, shacl_ttl)
# 6. Analyze the Report
print(f"Graph conforms: {report.conforms}")
@@ -265,10 +265,10 @@ cve_id_shape = NodeShape(
## Step 4 — Run validation and read the report
Serialize the graph to RDF, then run `_run_pyshacl` against the shapes.
Serialize the graph to RDF, then run `run_shacl_validation` against the shapes.
```python
from semantica.ontology.ontology_validator import _run_pyshacl
from semantica.ontology import run_shacl_validation
# Prepare your RDF data string (since export_rdf primarily exports structural metadata,
# you typically serialize your custom data graph to Turtle using rdflib or similar).
@@ -281,7 +281,7 @@ data_ttl = """
"""
# Run SHACL validation
report = _run_pyshacl(
report = run_shacl_validation(
data_ttl,
shacl_ttl,
data_graph_format="turtle",
@@ -366,8 +366,8 @@ print(f"Malware nodes missing 'family': {len(missing_family)}")
# e.g. graph.update_node(node_id, {"family": "UNKNOWN — requires triage"})
# After remediation, re-run validation to confirm the fix
# (re-export the patched graph to Turtle first, then call _run_pyshacl again)
report2 = _run_pyshacl(patched_data_ttl, shacl_ttl)
# (re-export the patched graph to Turtle first, then call run_shacl_validation again)
report2 = run_shacl_validation(patched_data_ttl, shacl_ttl)
print(f"Violations after remediation: {report2.violation_count}")
# Violations after remediation: 0
```
@@ -377,10 +377,49 @@ print(f"Violations after remediation: {report2.violation_count}")
## Common Pitfalls
- **Assuming the ontology automatically enforces data quality**: `SHACLGenerator` generates shapes based on what it observes in the data. If your data is missing a field, the generator won't know it was mandatory unless you explicitly inject the constraint (as shown in Step 3).
- **Passing `ContextGraph` directly to SHACL validators**: The `_run_pyshacl` function expects an RDF string (like Turtle format), not a raw Python dictionary or `ContextGraph` object.
- **Passing `ContextGraph` directly to SHACL validators**: The `run_shacl_validation` function expects an RDF string (like Turtle format), not a raw Python dictionary or `ContextGraph` object.
- **Forgetting RDF serialization**: You must serialize your graph (often via a temporary file using `export_rdf`) before validating it.
- **Treating validation as a one-time step**: Validation should be integrated as an automated step in your CI/CD pipeline or data ingestion flow, acting as a recurring gatekeeper rather than a one-off script.
- **Ignoring validation reports**: A graph that does not conform must be remediated. Failing to review the `violation_count` and address the issues negates the purpose of SHACL validation.
- **Validating `sh:class`/`sh:node` range checks on a property that declares `rdfs:range` with RDFS entailment on**: RDFS is an entailment rule, not a constraint. When pyshacl runs with `inference="rdfs"`, it infers the range class onto every object of the property, so class-based constraints on that property can never fail — the report says `conforms: True` on data that does not conform:
```python
from pyshacl import validate
from rdflib import Graph
data = Graph()
data.parse(
data="""
@prefix ex: <https://example.org/ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
ex:contains rdfs:domain ex:Container ; rdfs:range ex:Item .
ex:box a ex:Container ; ex:contains ex:notAnItem .
ex:notAnItem a ex:Fish .
""",
format="turtle",
)
shapes = Graph()
shapes.parse(
data="""
@prefix ex: <https://example.org/ns#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
ex:ContainerShape a sh:NodeShape ;
sh:targetClass ex:Container ;
sh:property [ sh:path ex:contains ; sh:class ex:Item ] .
""",
format="turtle",
)
for inference in ("none", "rdfs"):
conforms, _, _ = validate(data, shacl_graph=shapes, inference=inference)
print(inference, conforms)
# none False <- correct: notAnItem is a Fish, not an Item
# rdfs True <- the entailment manufactured the type
```
Mitigations: prefer not to declare `rdfs:range` on properties you intend to constrain with `sh:class`; when class membership is the thing under test, run validation without RDFS entailment (`inference="none"`); or express the check as a constraint the entailment cannot satisfy (for example a literal property constraint). Note the trade-off: with entailment off, `sh:targetClass` no longer reaches subclasses, so subclass hierarchies need explicit typing or inference-aware target selection. Semantica's own `run_shacl_validation` wrapper already calls pyshacl with `inference="none"`, so this pitfall only bites when calling `pyshacl.validate` directly with entailment enabled.
- **Trusting `conforms: True` without checking the inference mode**: an inference-enabled run can hide the exact violations the shapes were written to catch (see above). Record which inference mode validation ran under alongside the result, and re-run shape sets that contain `sh:class`/`sh:node` with entailment off before treating a pass as authoritative.
---
@@ -396,7 +435,7 @@ A DoD CTI team enforces STIX-compatible constraints on a threat graph before sha
from semantica.context import AgentContext, ContextGraph
from semantica.vector_store import VectorStore
from semantica.ontology import OntologyGenerator, SHACLGenerator, PropertyShape
from semantica.ontology.ontology_validator import _run_pyshacl
from semantica.ontology import run_shacl_validation
graph = ContextGraph()
ctx = AgentContext(
@@ -448,7 +487,7 @@ data_ttl = """
<http://example.org/hammertoss> a ex:Malware .
"""
report = _run_pyshacl(data_ttl, shacl_ttl)
report = run_shacl_validation(data_ttl, shacl_ttl)
print(f"CTI graph conforms : {report.conforms}")
print(f"Violations : {report.violation_count}")
print(f"Warnings : {report.warning_count}")
@@ -469,7 +508,7 @@ A SOC team validates zero-trust policy nodes before publishing them to the polic
```python
from semantica.context import ContextGraph
from semantica.ontology import OntologyGenerator, SHACLGenerator, PropertyShape
from semantica.ontology.ontology_validator import _run_pyshacl
from semantica.ontology import run_shacl_validation
graph = ContextGraph()
graph.add_node("policy-001", "Policy", "MFA Required for Tier-1 Resources",
@@ -516,7 +555,7 @@ data_ttl = """
<http://example.org/policy-002> a ex:Policy .
"""
report = _run_pyshacl(data_ttl, shacl_ttl)
report = run_shacl_validation(data_ttl, shacl_ttl)
print(f"Policy graph conforms: {report.conforms}")
# Policy graph conforms: False
@@ -534,7 +573,7 @@ A clinical informatics team validates trial ontology nodes before loading them i
```python
from semantica.ontology import LLMOntologyGenerator, SHACLGenerator, PropertyShape
from semantica.ontology.ontology_validator import _run_pyshacl
from semantica.ontology import run_shacl_validation
from semantica.export import export_rdf
import tempfile, os
@@ -586,7 +625,7 @@ with open(tmp.name) as f:
data_ttl = f.read()
os.unlink(tmp.name)
report = _run_pyshacl(data_ttl, shacl_ttl)
report = run_shacl_validation(data_ttl, shacl_ttl)
print(f"Trial data conforms: {report.conforms}")
print(f"Warnings : {report.warning_count}")
```
@@ -600,7 +639,7 @@ A credit risk team validates every `LoanApplication` node against Basel III CRE2
```python
from semantica.context import ContextGraph
from semantica.ontology import OntologyGenerator, SHACLGenerator, PropertyShape
from semantica.ontology.ontology_validator import _run_pyshacl
from semantica.ontology import run_shacl_validation
graph = ContextGraph()
graph.add_node("loan-001", "LoanApplication", "Prime mortgage APP-2025-88421",
@@ -645,7 +684,7 @@ data_ttl = """
ex:ltv "0.65" .
"""
report = _run_pyshacl(data_ttl, shacl_ttl)
report = run_shacl_validation(data_ttl, shacl_ttl)
print(f"Loan portfolio conforms: {report.conforms}")
# Loan portfolio conforms: False
@@ -675,14 +714,14 @@ Call this function as a pre-publish gate; exit code 1 blocks the pipeline.
```python
import sys
from semantica.ontology import OntologyGenerator, SHACLGenerator
from semantica.ontology.ontology_validator import _run_pyshacl
from semantica.ontology import run_shacl_validation
def validate_before_publish(data_graph_str: str, ontology: dict) -> None:
shacl_gen = SHACLGenerator(base_uri="https://example.org/shapes/")
shacl_graph = shacl_gen.generate(ontology)
shacl_ttl = shacl_gen.serialize(shacl_graph, format="turtle")
report = _run_pyshacl(data_graph_str, shacl_ttl)
report = run_shacl_validation(data_graph_str, shacl_ttl)
if not report.conforms:
print(f"Graph validation FAILED — {report.violation_count} violation(s)")
@@ -700,7 +739,6 @@ def validate_before_publish(data_graph_str: str, ontology: dict) -> None:
- [Ontology Management](ontology) — generate the OWL ontology that SHACL shapes are derived from
- [Reasoning & Rules](reasoning) — complement SHACL structural constraints with logical inference rules
- [Export & Serialization](export) — serialize graph data to Turtle/RDF/XML for `_run_pyshacl` input
- [Export & Serialization](export) — serialize graph data to Turtle/RDF/XML for `run_shacl_validation` input
- [Conflict Resolution](conflict-resolution) — detect and resolve data conflicts before SHACL validation
- [Change Management](change-management) — version-gate SHACL shapes alongside ontology versions
docs/integrations/langchain.md
+81
View File
@@ -0,0 +1,81 @@
---
title: "LangChain Integration"
description: "Drop Semantica into LangChain / LangGraph pipelines via a GraphRAG retriever, VectorStore adapter, and agent tools."
icon: "link"
---
> Three drop-in adapters that bring Semantica's context graph and hybrid search into LangChain chains and LangGraph agents.
## Installation
```bash
pip install "semantica[langchain]"
```
Requires `langchain-core >= 0.3`. If langchain-core is not installed, the integration still imports — every class carries the full Semantica API and degrades gracefully (`build()` returns `None`; branch on `LANGCHAIN_AVAILABLE`).
## Components at a Glance
- **SemanticaRetriever** — `BaseRetriever`: hybrid-search seeds retrieval, then graph edges are walked `hops` steps (default 2) for GraphRAG-style results.
- **SemanticaVectorStore** — `VectorStore`: `add_texts` / `similarity_search` / `similarity_search_with_score` / `from_texts` over `HybridSearch`.
- **SemanticaKGTool** / **SemanticaDecisionTool** — `BaseTool` subclasses: `semantica_query_graph` and `semantica_query_decisions` for LangGraph / tool-calling agents.
## Component Details
<Tabs>
<Tab title="SemanticaRetriever">
Hybrid search seeds retrieval; then graph edges are walked `hops` steps so results go beyond flat vector similarity. If hybrid search is omitted or fails, the retriever falls back to a `ContextGraph.query` keyword scan.
```python
from integrations.langchain import SemanticaRetriever
from semantica.context import ContextGraph
from semantica.vector_store import HybridSearch
graph = ContextGraph()
hybrid = HybridSearch()
retriever = SemanticaRetriever(graph=graph, hybrid=hybrid, hops=2, top_k=10)
from langchain.chains import RetrievalQA
qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)
```
</Tab>
<Tab title="SemanticaVectorStore">
Drop-in `VectorStore` for RetrievalQA / LCEL chains. `from_texts` requires a pre-configured `hybrid` instance.
```python
from integrations.langchain import SemanticaVectorStore
store = SemanticaVectorStore(hybrid=hybrid)
store.add_texts(
["document one", "document two"],
metadatas=[{"source": "a"}, {"source": "b"}],
)
docs = store.similarity_search("document", k=2)
docs, scores = store.similarity_search_with_score("document", k=2)
```
`add_texts` delegates to a Semantica vector store with `add_documents` (pass `vector_store=` to `HybridSearch` or to `SemanticaVectorStore`).
</Tab>
<Tab title="Agent tools">
Instances are LangChain `BaseTool`s and can be passed to an agent directly.
`.build()` returns the tool, or `None` when langchain-core is absent.
```python
from integrations.langchain import SemanticaKGTool, SemanticaDecisionTool
from langgraph.prebuilt import create_react_agent
tools = [
SemanticaKGTool(graph),
SemanticaDecisionTool(graph),
]
agent = create_react_agent(model, tools)
```
| Tool | Description |
| :------ | :------------- |
| `semantica_query_graph` | Keyword / NL query over the shared context graph |
| `semantica_query_decisions` | Search the recorded decision log |
</Tab>
</Tabs>
docs/project-license.md
+1 -1
View File
@@ -12,7 +12,7 @@ icon: "file-contract"
```
MIT License
Copyright (c) 2026 Hawksight AI
Copyright (c) 2026 Semantica
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
docs/reference/context.md
+6 -4
View File
@@ -435,8 +435,8 @@ print("Nodes: {}, Edges: {}".format(stats["node_count"], stats["edge_count"]))
| `query(query, skip, limit)` | `List[Dict]` | Full-text search over node content |
| `stats()` | `Dict` | Node/edge counts, type breakdowns, graph density |
| `density()` | `float` | Graph density score |
| `save_to_file(path)` | `None` | Persist graph to JSON |
| `load_from_file(path)` | `None` | Load graph from JSON |
| `save_to_file(path, format="json")` | `None` | Persist graph as JSON or a Markdown directory |
| `load_from_file(path, format="json")` | `None` | Replace graph state from JSON or a Markdown directory |
| `build_from_conversations(conversations, link_entities)` | `Dict` | Build graph from conversation data |
| `link_graph(other_graph, source_node_id, target_node_id, link_type)` | `str` | Create cross-graph navigation link; returns `link_id` |
| `navigate_to(link_id)` | `Tuple` | Follow a cross-graph link to `(target_graph, target_node_id)` |
@@ -625,8 +625,10 @@ malformed or duplicate fields before changing memory, and re-importing unchanged
files is idempotent. Memory-local `entities` and `relationships` are preserved as
provenance but are not applied to `ContextGraph` by Markdown import. Use a dedicated
export directory: matching files are overwritten, but unrelated or stale Markdown
files are not deleted automatically. Export refuses to overwrite symbolic links and
uses atomic file replacement. Timestamp offsets are preserved in Markdown and
files are not deleted automatically. Export refuses to overwrite filesystem links and
uses atomic file replacement; import also refuses symlinks, Windows directory
junctions, and other Windows reparse points.
Timestamp offsets are preserved in Markdown and
normalized to UTC only for comparisons, so aware and local-naive records can be
queried together safely. Vector-store writes are deferred until the in-memory import
commits; adapter synchronization remains best-effort and logs failures.
docs/reference/reasoning.md
+19 -2
View File
@@ -127,9 +127,19 @@ conclusions = reasoner.infer_facts(
| `forward_chain()` | `List[InferenceResult]` | Derive all possible conclusions iteratively until fixpoint |
| `backward_chain(goal, max_depth)` | `InferenceResult \| None` | Prove a specific goal string, returns `None` if unprovable |
| `infer_facts(facts, rules)` | `List[str]` | Load facts and rules then run `forward_chain()`, returns conclusion strings |
| `clear()` | `None` | Clear all facts and rules |
| `reset_action_history()` | `None` | Allow actions for previously fired activations to run again |
| `clear()` | `None` | Clear all facts, rules, and action activation history |
| `reset()` | `None` | Alias for `clear()` |
Rules with actions use at-most-once attempt semantics per concrete activation
(rule ID, bindings, and matched facts). Calling `forward_chain()` again on the
same instance does not repeat side effects for an activation that was already
attempted, even when an action raised an exception. Call
`reset_action_history()` to deliberately retry without clearing facts or rules;
`clear()` and `reset()` also clear this history. Replacing a rule's actions in
place does not invalidate an existing activation; reset the history explicitly
when the replacement should be replayed.
### Rule and Fact dataclass fields
```python
@@ -230,9 +240,16 @@ engine.reset()
| `add_fact(fact)` | `None` | Add a `Fact` to working memory and propagate through the network |
| `match_patterns(facts)` | `List[Match]` | Match all patterns; optionally add facts before matching |
| `execute_matches(matches)` | `List[Any]` | Execute matched rules and return their conclusion values |
| `reset()` | `None` | Clear facts and all node activation state |
| `reset_action_history()` | `None` | Allow actions for previously executed activations to run again |
| `reset()` | `None` | Clear facts, node activation state, and action activation history |
| `get_network_stats()` | `dict` | Return counts of alpha, beta, terminal nodes and facts |
When a Reasoner is bound, `execute_matches()` deduplicates action side effects
by rule ID, bindings, and matched fact identity. Re-executing a match still
returns its conclusion for compatibility, but its actions are skipped after the
first attempt. `reset_action_history()`, `reset()`, and `build_network()` allow
those actions to run again.
## SPARQLReasoner
docs/reference/triplet_store.md
+1 -1
View File
@@ -182,7 +182,7 @@ for row in result.bindings:
store = TripletStore(
backend="rdf4j",
endpoint="http://localhost:8080/rdf4j-server",
repository_id="semantica", # passed through **config
repository_id="semantica", # selects the remote repository
)
```
docs/reference/utils.md
+10
View File
@@ -77,7 +77,17 @@ Most users won't call utils directly: it's the **shared foundation** for all mod
export SEMANTICA_LOG_LEVEL=DEBUG
export SEMANTICA_LOG_FORMAT=json # "json" | "text"
export SEMANTICA_DISABLE_PROGRESS=true
export SEMANTICA_FORCE_PROGRESS=true
```
<Tip>
**Progress bars follow your terminal.** Console progress is written only when
stdout is an interactive terminal (or a Jupyter notebook), so piping or
redirecting output no longer fills logs with progress bars and escape
sequences. Set `SEMANTICA_DISABLE_PROGRESS` to silence progress even in a
terminal, or `SEMANTICA_FORCE_PROGRESS` to keep it when stdout is redirected.
`SEMANTICA_DISABLE_PROGRESS` wins if both are set.
</Tip>
</Step>
</Steps>
docs/storage-backends.md
+154
View File
@@ -0,0 +1,154 @@
# Graph storage backends and feature matrix
Semantica separates graph modeling from physical storage. LPG backends are accessed through `graph_store` adapters; RDF backends are accessed through `triplet_store` adapters.
This page is intentionally conservative: it distinguishes between an adapter existing, a feature being generally available with that model, and a backend needing user-supplied wiring.
## Status labels
- `built-in`: adapter implementation exists in Semantica core.
- `tested`: covered by automated integration fixtures or tests.
- `example-only`: usable example exists, but support is not asserted by integration tests.
- `interface/BYO`: interface or integration point exists; bring your own backend wiring.
## Adapter inventory
| Backend | Model | Adapter | Status | Reference |
| --- | --- | --- | --- | --- |
| Neo4j | LPG | `semantica.graph_store.Neo4jStore` | built-in | `cookbook/introduction/09_Graph_Store.ipynb` |
| FalkorDB | LPG | `semantica.graph_store.FalkorDBStore` | built-in | `docs/reference/graph_store.md` |
| Amazon Neptune | LPG | `semantica.graph_store.AmazonNeptuneStore` | built-in | `cookbook/introduction/21_Amazon_Neptune_Store.ipynb` |
| Apache AGE | LPG | `semantica.graph_store.ApacheAgeStore` | built-in | `docs/graph_stores/apache_age.md` |
| RDF4J | RDF | `semantica.triplet_store.RDF4JStore` | built-in | `cookbook/introduction/20_Triplet_Store.ipynb` |
| Apache Jena | RDF | `semantica.triplet_store.JenaStore` | built-in | `cookbook/introduction/20_Triplet_Store.ipynb` |
| Blazegraph | RDF | `semantica.triplet_store.BlazegraphStore` | built-in | `cookbook/introduction/20_Triplet_Store.ipynb` |
| Anzo | RDF | `semantica.triplet_store.AnzoStore` | built-in | `cookbook/introduction/20_Triplet_Store.ipynb` |
| Oxigraph | RDF | `semantica.triplet_store.OxigraphStore` | built-in | `docs/reference/triplet_store.md` |
## Feature matrix
`Yes` means the capability is expected to work with the adapter and graph model. `Partial` means the capability works with model-specific constraints. `BYO` means the user must supply or validate wiring for the backend.
| Backend | Model | Ingestion | Context graph construction | Reasoning/analytics | Provenance | Known limitations |
| --- | --- | --- | --- | --- | --- | --- |
| Neo4j | LPG | Yes | Yes | Yes | Partial | Provenance and context metadata are stored as node and edge properties; relationship properties and stable node identifiers are required. |
| FalkorDB | LPG | Yes | Yes | Partial | Partial | Redis-based; provenance depends on node/edge properties, and multi-graph isolation depends on the selected graph name. |
| Amazon Neptune | LPG | Yes | Yes | Partial | Partial | Use the property-graph endpoint; AWS auth, VPC, and endpoint configuration can affect local tests. Provenance depends on node/edge properties. |
| Apache AGE | LPG | Yes | Yes | Partial | Partial | Runs through PostgreSQL/AGE; Cypher compatibility and property handling can differ from standalone LPG engines. |
| RDF4J | RDF | Yes | Partial | Partial | Partial | Context separation relies on named graphs; triple-level provenance may require reification or graph-level metadata. |
| Apache Jena | RDF | Yes | Partial | Partial | Partial | Named graphs are needed for context separation; backend configuration and transaction behavior matter. |
| Blazegraph | RDF | Yes | Partial | Partial | Partial | Use quads/named graphs for context; IRI stability and graph naming matter for provenance. |
| Anzo | RDF | Yes | Partial | Partial | Partial | Anzo deployments are environment-specific; validate `dataset_uri`/graphmart naming, named-graph support, and provenance mapping. |
| Oxigraph | RDF | Yes | Partial | Partial | Partial | Embedded, single-process store (in-memory or on-disk); named graphs are supported, but there is no separate server process to scale independently. |
## RDF and LPG differences
- LPG backends store context and provenance as graph elements and properties. If a backend does not support relationship properties, some provenance patterns may be degraded.
- RDF backends rely on IRIs, named graphs, and optional reification. Context graphs and provenance are easiest to preserve when the store supports named graphs/quads.
- Ingestion works across both models, but the physical representation differs: LPG stores nodes/edges directly, while RDF stores subject-predicate-object statements.
- Reasoning and analytics should be validated against the adapter's query capabilities, especially for path traversal, property filters, and named-graph queries.
## Minimal connection examples
Prefer the referenced notebook cells for a working setup. The examples below show the intended adapter entrypoints, not a universal connection DSL.
### Neo4j
```python
import os
from semantica.graph_store import Neo4jStore
store = Neo4jStore(
uri='bolt://localhost:7687',
user='neo4j',
password=os.environ['NEO4J_PASSWORD']
)
```
### FalkorDB
```python
from semantica.graph_store import FalkorDBStore
store = FalkorDBStore(
host='localhost',
port=6379,
graph_name='semantica'
)
```
### Amazon Neptune
```python
from semantica.graph_store import AmazonNeptuneStore
store = AmazonNeptuneStore(
endpoint='your-neptune-cluster-endpoint',
port=8182,
region='us-east-1'
)
```
### Apache AGE
```python
from semantica.graph_store import ApacheAgeStore
store = ApacheAgeStore(
connection_string='host=localhost dbname=agedb user=postgres password=postgres',
graph_name='semantica'
)
```
### RDF4J
```python
from semantica.triplet_store import RDF4JStore
store = RDF4JStore(
endpoint='http://localhost:8080/rdf4j-server',
repository_id='semantica'
)
```
### Apache Jena
```python
from semantica.triplet_store import JenaStore
store = JenaStore(
endpoint='http://localhost:3030/ds'
)
```
### Blazegraph
```python
from semantica.triplet_store import BlazegraphStore
store = BlazegraphStore(
endpoint='http://localhost:9999/blazegraph/sparql'
)
```
### Anzo
```python
from semantica.triplet_store import AnzoStore
store = AnzoStore(
endpoint='http://anzo-host:8080',
dataset_uri='http://cambridgesemantics.com/Graphmart/your-graphmart-id'
)
```
### Oxigraph
```python
from semantica.triplet_store import OxigraphStore
# Omit `path` for an in-memory store; pass a directory for on-disk persistence.
store = OxigraphStore(path='./semantica-oxigraph-data')
```
Replace hostnames, ports, repositories, graphs, and credentials with values from your environment. For regulated or self-hosted deployments, keep credentials in environment variables or secret storage rather than source code.
explorer/package-lock.json
Generated
+1489 -14
View File
File diff suppressed because it is too large Load Diff
explorer/package.json
+3 -1
View File
@@ -9,7 +9,7 @@
"lint": "eslint .",
"preview": "vite preview",
"test:graph-store": "node --test tests/graphStore.multi-edge.test.mjs",
"test:graph-workspace": "node --import tsx --test tests/graphSceneState.display.test.ts tests/temporalLifecycle.test.ts",
"test:graph-workspace": "node --import tsx --test tests/markdownContentViewer.test.ts tests/graphSceneState.display.test.ts tests/temporalLifecycle.test.ts",
"test:plugin-registry": "node --import tsx --test tests/pluginRegistry.temporal.test.mjs"
},
"dependencies": {
@@ -29,6 +29,8 @@
"react-arborist": "^3.4.3",
"react-dom": "^19.2.4",
"react-dropzone": "^15.0.0",
"react-markdown": "^10.1.0",
"remark-gfm": "^4.0.1",
"sigma": "^3.0.2",
"vis-data": "^8.0.3",
"vis-timeline": "^8.5.0"
explorer/src/workspaces/GraphWorkspace/GraphInspectorPanel.tsx
+20
View File
@@ -3,6 +3,7 @@ import { Loader2 } from "lucide-react";
import { graph } from "../../store/graphStore";
import { GRAPH_THEME, withAlpha } from "./graphTheme";
import type { GraphSelectedNodeKind } from "./types";
import { MarkdownContentViewer } from "./MarkdownContentViewer";
export type LinkPrediction = {
target: string;
@@ -364,6 +365,11 @@ export function GraphInspectorPanel({
([key]) =>
!["x","y","valid_from","valid_until","content","source","source_url","pmid","pmids","evidence","provenance","confidence"].includes(key),
);
const nodeContent = (typeof attributes?.content === "string" && attributes.content)
? attributes.content
: (typeof properties.content === "string" && properties.content)
? properties.content
: "";
return (
<aside style={{ padding: 24, display: "flex", flexDirection: "column", gap: 18 }}>
@@ -408,6 +414,20 @@ export function GraphInspectorPanel({
</div>
) : null}
{/* Content Section — only rendered when the node carries actual content.
This matches the existing inspector convention: sections that have no
data for the current node are either hidden (temporal bounds) or closed
by default (Source Attribution, Properties). Always showing an open
empty panel would add noise for every relationship/predicate node. */}
{nodeContent && (
<details className="node-panel-collapse" open>
<summary className="node-panel-summary">Content</summary>
<div className="node-panel-body" style={{ marginTop: 8 }}>
<MarkdownContentViewer content={nodeContent} />
</div>
</details>
)}
{/* Actions */}
<section style={sectionStyle}>
<div style={sectionTitleStyle}>Actions</div>
explorer/src/workspaces/GraphWorkspace/MarkdownContentViewer.tsx
+404
View File
@@ -0,0 +1,404 @@
import { useState, useRef, useEffect, useMemo, type CSSProperties } from "react";
import ReactMarkdown, { type Components } from "react-markdown";
import remarkGfm from "remark-gfm";
import { Check, Copy, Code2, Eye, ExternalLink, Image as ImageIcon } from "lucide-react";
import { GRAPH_THEME } from "./graphTheme";
import { isSafeUrl } from "./markdownUrlSafety";
export interface MarkdownContentViewerProps {
content?: string | null;
className?: string;
defaultMode?: "preview" | "source";
}
export function MarkdownContentViewer({
content,
className,
defaultMode = "preview",
}: MarkdownContentViewerProps) {
const [activeMode, setActiveMode] = useState<"preview" | "source">(defaultMode);
const [copied, setCopied] = useState(false);
// Track the content value for which the copied indicator is valid.
// When content changes (i.e. the user selects a different node), reset the
// copied indicator inline during render rather than in a useEffect — this
// avoids a cascading-render lint error and is the React-recommended pattern
// for resetting derived visual state on prop changes.
const [copiedForContent, setCopiedForContent] = useState<string | null | undefined>(content);
if (copiedForContent !== content) {
setCopiedForContent(content);
if (copied) {
// Clear the stale indicator synchronously so the new node's copy button
// never shows "Copied" from the previous selection.
setCopied(false);
}
}
const copyTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
// Clean up any outstanding timeout on unmount.
useEffect(() => {
return () => {
if (copyTimeoutRef.current) {
clearTimeout(copyTimeoutRef.current);
}
};
}, []);
const rawContent = typeof content === "string" ? content : "";
const hasContent = rawContent.trim().length > 0;
// react-markdown runs the whole remark pipeline synchronously inside its own
// render, so without this memo every unrelated re-render of this component --
// clicking Copy, toggling Preview/Source -- re-parses the entire document.
// Measured at ~364ms per re-render for a 1000-row GFM table (issue #1118).
// Keyed on rawContent so a genuine node change still re-parses exactly once.
const renderedMarkdown = useMemo(
() => (
<ReactMarkdown remarkPlugins={REMARK_PLUGINS} components={MARKDOWN_COMPONENTS}>
{rawContent}
</ReactMarkdown>
),
[rawContent],
);
const handleCopy = async () => {
if (!hasContent) return;
try {
await navigator.clipboard.writeText(rawContent);
if (copyTimeoutRef.current) {
clearTimeout(copyTimeoutRef.current);
}
setCopied(true);
copyTimeoutRef.current = setTimeout(() => setCopied(false), 1500);
} catch {
// Clipboard write unavailable
}
};
return (
<div className={className} style={viewerContainerStyle}>
<div style={viewerHeaderStyle}>
<div style={{ display: "flex", gap: 4 }} role="tablist">
<button
type="button"
role="tab"
aria-selected={activeMode === "preview"}
onClick={() => setActiveMode("preview")}
style={{ ...tabBtnStyle, ...(activeMode === "preview" ? activeTabBtnStyle : {}) }}
>
<Eye size={12} style={{ marginRight: 5 }} />
Preview
</button>
<button
type="button"
role="tab"
aria-selected={activeMode === "source"}
onClick={() => setActiveMode("source")}
style={{ ...tabBtnStyle, ...(activeMode === "source" ? activeTabBtnStyle : {}) }}
>
<Code2 size={12} style={{ marginRight: 5 }} />
Source
</button>
</div>
{hasContent && (
<button type="button" onClick={() => void handleCopy()} style={copyBtnStyle} title="Copy raw content">
{copied ? (
<>
<Check size={12} color="#3fb950" style={{ marginRight: 4 }} />
<span style={{ color: "#3fb950", fontSize: 11 }}>Copied</span>
</>
) : (
<>
<Copy size={12} style={{ marginRight: 4 }} />
<span style={{ fontSize: 11 }}>Copy</span>
</>
)}
</button>
)}
</div>
<div style={viewerBodyStyle}>
{!hasContent ? (
<div style={emptyTextStyle}>No content available for this node.</div>
) : activeMode === "source" ? (
<pre style={sourcePreStyle}>
<code style={sourceCodeStyle}>{rawContent}</code>
</pre>
) : (
<div style={previewStyle}>{renderedMarkdown}</div>
)}
</div>
</div>
);
}
/* ─── Markdown rendering config ───────────────────────────────────── */
// Both props are hoisted to module scope so they keep a stable identity across
// renders. As inline literals they allocated a fresh plugin array and ~20 fresh
// arrow components on every render, which made React treat every mapped tag as a
// new element type and remount the entire rendered subtree instead of updating
// it (issue #1118). The arrow bodies only read the style constants below at call
// time, so declaring the map before them is safe.
const REMARK_PLUGINS = [remarkGfm];
const MARKDOWN_COMPONENTS: Components = {
// C-1: react-markdown passes a HAST `node` prop (the raw AST
// Element) to every custom component override via passNode:true.
// In React 19 any unknown prop spreads onto a native element are
// serialised as HTML attributes, producing node="[object Object]"
// on every rendered link. Fix: destructure `node` by name so it
// is explicitly discarded, then spread `...rest` to preserve all
// other legitimate HAST/remark-gfm attributes — e.g. the `id`,
// `aria-describedby`, `aria-label`, `data-footnote-ref`,
// `data-footnote-backref`, and `class` attrs that GFM footnotes
// require for correct in-page navigation and accessibility.
//
// C-2: fragment links (#anchor, GFM footnote backlinks) must
// navigate within the current document. External links continue
// to use target="_blank" with noopener noreferrer.
//
// eslint-disable-next-line @typescript-eslint/no-unused-vars
a: ({ href, children, title, node: _node, ...rest }) => {
if (!isSafeUrl(href)) {
return <span style={{ color: GRAPH_THEME.ui.text.muted, textDecoration: "line-through" }}>{children}</span>;
}
// isSafeUrl returning true guarantees href is a non-empty string.
const safeHref = href ?? "";
// Fragment links (#section, footnote backlinks like
// #user-content-fnref-1) are in-document anchors. Opening them
// in a new tab would break GFM footnote back-navigation.
const isFragment = safeHref.startsWith("#");
if (isFragment) {
return (
<a href={safeHref} title={title} style={linkStyle} {...rest}>
{children}
</a>
);
}
return (
<a href={safeHref} title={title} target="_blank" rel="noopener noreferrer" style={linkStyle} {...rest}>
{children}
<ExternalLink size={10} style={{ marginLeft: 3, verticalAlign: "middle", display: "inline" }} />
</a>
);
},
img: ({ src, alt }) => (
<span style={imageBadgeStyle} title={src || "Image"}>
<ImageIcon size={12} style={{ marginRight: 5 }} />
<span>Image: {alt || src || "unlabeled"}</span>
</span>
),
h1: ({ children }) => <h1 style={h1Style}>{children}</h1>,
h2: ({ children }) => <h2 style={h2Style}>{children}</h2>,
h3: ({ children }) => <h3 style={h3Style}>{children}</h3>,
h4: ({ children }) => <h4 style={h4Style}>{children}</h4>,
p: ({ children }) => <p style={{ margin: "0 0 8px 0" }}>{children}</p>,
ul: ({ children }) => <ul style={{ margin: "0 0 8px 0", paddingLeft: 18 }}>{children}</ul>,
ol: ({ children }) => <ol style={{ margin: "0 0 8px 0", paddingLeft: 18 }}>{children}</ol>,
li: ({ children }) => <li style={{ marginBottom: 3 }}>{children}</li>,
blockquote: ({ children }) => <blockquote style={blockquoteStyle}>{children}</blockquote>,
hr: () => <hr style={{ border: "none", borderTop: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}`, margin: "10px 0" }} />,
table: ({ children }) => (
<div style={{ width: "100%", overflowX: "auto", margin: "8px 0", borderRadius: 6, border: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}` }}>
<table style={{ width: "100%", borderCollapse: "collapse", fontSize: 12 }}>{children}</table>
</div>
),
thead: ({ children }) => <thead style={{ background: "rgba(255, 255, 255, 0.04)" }}>{children}</thead>,
tbody: ({ children }) => <tbody>{children}</tbody>,
tr: ({ children }) => <tr style={{ borderBottom: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}` }}>{children}</tr>,
th: ({ children }) => <th style={{ padding: "6px 8px", textAlign: "left", fontWeight: 700, color: GRAPH_THEME.ui.text.strong, borderRight: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}` }}>{children}</th>,
td: ({ children }) => <td style={{ padding: "6px 8px", color: GRAPH_THEME.ui.text.body, borderRight: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}` }}>{children}</td>,
pre: ({ children }) => <pre style={preBlockStyle}>{children}</pre>,
// C-1: discard `node` here too — code elements are custom components
// and would otherwise receive node="[object Object]" in the DOM.
code: ({ className: codeClass, children }) => {
const isInline = !codeClass && typeof children === "string" && !children.includes("\n");
return (
<code style={isInline ? inlineCodeStyle : blockCodeStyle}>
{children}
</code>
);
},
};
/* ─── Styles ──────────────────────────────────────────────────────── */
const viewerContainerStyle: CSSProperties = {
display: "flex",
flexDirection: "column",
background: "rgba(255, 255, 255, 0.025)",
border: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}`,
borderRadius: 12,
overflow: "hidden",
};
const viewerHeaderStyle: CSSProperties = {
display: "flex",
alignItems: "center",
justifyContent: "space-between",
padding: "6px 10px",
background: "rgba(0, 0, 0, 0.2)",
borderBottom: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}`,
};
const tabBtnStyle: CSSProperties = {
display: "inline-flex",
alignItems: "center",
padding: "4px 9px",
borderRadius: 6,
border: "1px solid transparent",
background: "transparent",
color: GRAPH_THEME.ui.text.muted,
fontSize: 12,
fontWeight: 600,
cursor: "pointer",
transition: "all 150ms ease",
};
const activeTabBtnStyle: CSSProperties = {
background: GRAPH_THEME.ui.timeline.playheadSoft,
border: `1px solid ${GRAPH_THEME.ui.control.activeBorder}`,
color: GRAPH_THEME.ui.timeline.playhead,
};
const copyBtnStyle: CSSProperties = {
display: "inline-flex",
alignItems: "center",
padding: "3px 8px",
borderRadius: 6,
border: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}`,
background: "rgba(255, 255, 255, 0.04)",
color: GRAPH_THEME.ui.text.subtle,
fontSize: 11,
cursor: "pointer",
};
const viewerBodyStyle: CSSProperties = {
padding: 12,
maxHeight: 380,
overflowY: "auto",
};
const emptyTextStyle: CSSProperties = {
color: GRAPH_THEME.ui.text.muted,
fontSize: 12,
lineHeight: 1.5,
fontStyle: "italic",
};
const sourcePreStyle: CSSProperties = {
margin: 0,
padding: 10,
borderRadius: 8,
background: "rgba(0, 0, 0, 0.3)",
border: "1px solid rgba(255, 255, 255, 0.05)",
overflowX: "auto",
};
const sourceCodeStyle: CSSProperties = {
fontFamily: "'JetBrains Mono', 'Fira Code', monospace",
fontSize: 12,
lineHeight: 1.6,
color: GRAPH_THEME.ui.text.strong,
whiteSpace: "pre-wrap",
wordBreak: "break-word",
userSelect: "text",
};
const previewStyle: CSSProperties = {
color: GRAPH_THEME.ui.text.body,
fontSize: 13,
lineHeight: 1.6,
wordBreak: "break-word",
};
const h1Style: CSSProperties = {
fontSize: 16,
fontWeight: 700,
color: GRAPH_THEME.ui.text.strong,
marginTop: 8,
marginBottom: 6,
paddingBottom: 3,
borderBottom: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}`,
};
const h2Style: CSSProperties = {
fontSize: 14,
fontWeight: 700,
color: GRAPH_THEME.ui.text.strong,
marginTop: 8,
marginBottom: 4,
};
const h3Style: CSSProperties = {
fontSize: 13,
fontWeight: 600,
color: GRAPH_THEME.ui.text.strong,
marginTop: 6,
marginBottom: 4,
};
const h4Style: CSSProperties = {
fontSize: 12,
fontWeight: 600,
color: GRAPH_THEME.ui.text.strong,
marginTop: 4,
marginBottom: 2,
};
const blockquoteStyle: CSSProperties = {
margin: "8px 0",
padding: "6px 12px",
borderLeft: `3px solid ${GRAPH_THEME.ui.timeline.playhead}`,
background: "rgba(98, 226, 205, 0.05)",
borderRadius: "0 6px 6px 0",
color: GRAPH_THEME.ui.text.body,
fontStyle: "italic",
};
const linkStyle: CSSProperties = {
color: "#79c0ff",
textDecoration: "underline",
textUnderlineOffset: "3px",
wordBreak: "break-all",
};
const imageBadgeStyle: CSSProperties = {
display: "inline-flex",
alignItems: "center",
padding: "3px 7px",
background: "rgba(255, 255, 255, 0.04)",
border: `1px solid ${GRAPH_THEME.ui.surface.panelBorder}`,
borderRadius: 6,
color: GRAPH_THEME.ui.text.muted,
fontSize: 11,
margin: "3px 0",
};
const inlineCodeStyle: CSSProperties = {
fontFamily: "'JetBrains Mono', monospace",
fontSize: 12,
padding: "2px 5px",
borderRadius: 4,
background: "rgba(255, 255, 255, 0.07)",
color: "#e6edf3",
border: "1px solid rgba(255, 255, 255, 0.08)",
};
const preBlockStyle: CSSProperties = {
margin: "8px 0",
padding: 10,
borderRadius: 8,
background: "rgba(0, 0, 0, 0.35)",
border: "1px solid rgba(255, 255, 255, 0.08)",
overflowX: "auto",
};
const blockCodeStyle: CSSProperties = {
fontFamily: "'JetBrains Mono', monospace",
fontSize: 12,
lineHeight: 1.5,
color: "#e6edf3",
};
explorer/src/workspaces/GraphWorkspace/markdownUrlSafety.ts
+29
View File
@@ -0,0 +1,29 @@
/**
* URL-safety predicate for the Markdown content viewer.
*
* Extracted into a pure module so the check can be unit-tested without
* importing the MarkdownContentViewer React component, and so the component
* module exports only components (react-refresh/only-export-components,
* issue #1119). The behaviour is unchanged from the original in-component
* implementation: only http, https, mailto, in-document fragments, and
* root-relative paths are permitted.
*/
export function isSafeUrl(url?: string): boolean {
if (!url) return false;
const trimmed = url.trim();
// Reject whitespace-only strings — new URL("", base) would resolve to the base
// protocol and produce a false positive. This guards direct callers of the exported
// function; markdown parsers normalise whitespace-only destinations to "" which
// already fails the !url check above.
if (!trimmed) return false;
if (trimmed.startsWith("//")) return false;
if (trimmed.startsWith("#")) return true;
if (trimmed.startsWith("/")) return true;
try {
const parsed = new URL(trimmed, "http://localhost");
return ["http:", "https:", "mailto:"].includes(parsed.protocol);
} catch {
return false;
}
}
explorer/tests/markdownContentViewer.test.ts
+265
View File
@@ -0,0 +1,265 @@
import test from "node:test";
import assert from "node:assert/strict";
import React from "react";
import { renderToString } from "react-dom/server";
(globalThis as any).React = React;
import { MarkdownContentViewer } from "../src/workspaces/GraphWorkspace/MarkdownContentViewer.tsx";
import { isSafeUrl } from "../src/workspaces/GraphWorkspace/markdownUrlSafety.ts";
test("isSafeUrl permits safe http, https, and mailto URLs and relative paths", () => {
assert.equal(isSafeUrl("https://example.com"), true);
assert.equal(isSafeUrl("http://localhost:8000"), true);
assert.equal(isSafeUrl("mailto:user@example.com"), true);
assert.equal(isSafeUrl("#section-1"), true);
assert.equal(isSafeUrl("/relative/path"), true);
});
test("isSafeUrl rejects protocol-relative URLs and dangerous schemes", () => {
// Protocol-relative URLs (must be blocked)
assert.equal(isSafeUrl("//evil.com"), false);
assert.equal(isSafeUrl("//localhost:8000"), false);
assert.equal(isSafeUrl("//"), false);
// Dangerous schemes
assert.equal(isSafeUrl("javascript:alert('xss')"), false);
assert.equal(isSafeUrl("JAVASCRIPT:alert(1)"), false);
assert.equal(isSafeUrl("data:text/html;base64,PHNjcmlwdD4="), false);
assert.equal(isSafeUrl("vbscript:MsgBox(1)"), false);
assert.equal(isSafeUrl(""), false);
assert.equal(isSafeUrl(undefined), false);
});
// ─── C URL contract: whitespace-only strings ────────────────────────────────
// The CommonMark parser normalises whitespace-only link destinations to "" so
// these values are unreachable through normal markdown rendering. However, the
// function is exported and its direct-call contract must be correct.
test("isSafeUrl rejects whitespace-only strings (contract correctness)", () => {
assert.equal(isSafeUrl(" "), false, "single space must be rejected");
assert.equal(isSafeUrl("\t"), false, "tab must be rejected");
assert.equal(isSafeUrl("\n"), false, "newline must be rejected");
assert.equal(isSafeUrl(" "), false, "multiple spaces must be rejected");
assert.equal(isSafeUrl(" \t\n "), false, "mixed whitespace must be rejected");
});
test("renders Preview mode with formatted Markdown elements and tabs", () => {
const markdown = `# Main Title\n\n**Bold Statement**\n\n* Item A\n* Item B`;
const html = renderToString(React.createElement(MarkdownContentViewer, { content: markdown, defaultMode: "preview" }));
// Tab buttons are present
assert.equal(html.includes("Preview"), true);
assert.equal(html.includes("Source"), true);
assert.equal(html.includes("Copy"), true);
// Formatted preview elements
assert.equal(html.includes("Main Title"), true);
assert.equal(html.includes("Bold Statement"), true);
assert.equal(html.includes("<strong>Bold Statement</strong>"), true);
assert.equal(html.includes("Item A"), true);
assert.equal(html.includes("Item B"), true);
});
test("renders Source mode with exact unmodified text inside pre/code", () => {
const markdown = `# Title 🚀\n\n * Indented item\n\n\`\`\`python\ndef test():\n return "α + β"\n\`\`\``;
const html = renderToString(React.createElement(MarkdownContentViewer, { content: markdown, defaultMode: "source" }));
assert.equal(html.includes("<pre"), true);
assert.equal(html.includes("<code"), true);
assert.equal(html.includes("# Title 🚀"), true);
assert.equal(html.includes(" * Indented item"), true);
assert.equal(html.includes('return &quot;α + β&quot;'), true);
});
test("renders raw HTML safely as escaped text without executing elements", () => {
const dangerousHtml = `<script>alert("XSS")</script><iframe src="https://evil.com"></iframe>`;
const html = renderToString(React.createElement(MarkdownContentViewer, { content: dangerousHtml, defaultMode: "preview" }));
// Script and iframe tags must NOT be rendered as active DOM tags
assert.equal(html.includes("<script>"), false);
assert.equal(html.includes("<iframe"), false);
// Content is escaped as text
assert.equal(html.includes("&lt;script&gt;"), true);
});
// ─── C-1: HAST node prop must not reach the DOM ─────────────────────────────
// react-markdown passes a HAST `node` (Element) object to custom component
// overrides. Before this fix, ...props spread caused React 19 to serialise it
// as node="[object Object]" on every <a> and <code> element.
test("rendered links do not expose the HAST node object as a DOM attribute", () => {
const content = `[Example](https://example.com)\n\nInline \`code\` here.`;
const html = renderToString(React.createElement(MarkdownContentViewer, { content, defaultMode: "preview" }));
// The rendered HTML must not contain the serialised HAST object
assert.equal(html.includes("node="), false, "node= attribute must not appear in rendered HTML");
assert.equal(html.includes("[object Object]"), false, "serialised HAST object must not appear in rendered HTML");
// The link must still render correctly with the right href
assert.equal(html.includes('href="https://example.com"'), true, "href must be present");
});
// ─── C-2: Fragment links must not open in a new tab ─────────────────────────
// Links to in-document anchors such as #section or GFM footnote backlinks like
// #user-content-fn-1 must stay in the current document. Only external links
// use target="_blank".
test("fragment links render in the current document without target blank", () => {
const content = `[Jump to section](#introduction)\n\n[External](https://example.com)`;
const html = renderToString(React.createElement(MarkdownContentViewer, { content, defaultMode: "preview" }));
// Fragment link must have the href
assert.equal(html.includes('href="#introduction"'), true, "fragment href must be present");
// Confirm no target=_blank attribute appears anywhere near the fragment link.
// We check that the output contains a fragment href WITHOUT target="_blank"
// by verifying the two strings are not both present (the external link has
// target blank; the fragment link must not).
const fragmentLinkIdx = html.indexOf('href="#introduction"');
assert.notEqual(fragmentLinkIdx, -1, "fragment link must be rendered");
// Inspect the 80 chars around the fragment href — should not contain target
const fragmentContext = html.slice(Math.max(0, fragmentLinkIdx - 10), fragmentLinkIdx + 90);
assert.equal(fragmentContext.includes('target="_blank"'), false, "fragment link must not have target=_blank");
// External link must still have target blank
assert.equal(html.includes('href="https://example.com"'), true, "external href must be present");
assert.equal(html.includes('target="_blank"'), true, "external link must have target=_blank");
assert.equal(html.includes('rel="noopener noreferrer"'), true, "external link must have rel");
});
test("GFM footnote backlinks render without target blank", () => {
// GFM footnote syntax: footnote ref in text + definition below
const content = `See the note[^1] for more.\n\n[^1]: This is the footnote text.`;
const html = renderToString(React.createElement(MarkdownContentViewer, { content, defaultMode: "preview" }));
// The footnote reference link (#user-content-fn-1) and backlink
// (#user-content-fnref-1) are fragment links and must not open in a new tab.
// We verify no fragment href is paired with target=_blank.
// Extract all href="#..." occurrences and confirm none is adjacent to target=_blank.
const anchorMatches = [...html.matchAll(/href="#[^"]*"/g)];
assert.ok(anchorMatches.length > 0, "GFM footnotes must produce fragment links");
for (const match of anchorMatches) {
const start = match.index ?? 0;
const context = html.slice(Math.max(0, start - 10), start + 120);
assert.equal(
context.includes('target="_blank"'),
false,
`fragment link ${match[0]} must not have target=_blank`,
);
}
});
// ─── C-1-R: GFM footnote attributes must be preserved (regression test) ─────
// The C-1 fix (removing the HAST `node` prop) must NOT silently drop other
// legitimate HAST attributes. remark-gfm generates the following on footnote
// links that are required for correct in-page navigation and accessibility:
//
// Footnote reference anchor:
// id="user-content-fnref-1" ← backlink target
// data-footnote-ref="true"
// aria-describedby="footnote-label"
//
// Footnote back-link anchor:
// data-footnote-backref=""
// aria-label="Back to reference 1" ← screen-reader label
// class="data-footnote-backref"
//
// If these are absent, clicking the ↩ back-link cannot scroll back to the
// in-text reference, and screen readers cannot announce the backlink purpose.
test("GFM footnote links preserve generated id, aria, and class attributes", () => {
const content = `See the note[^1] for more.\n\n[^1]: This is the footnote text.`;
const html = renderToString(React.createElement(MarkdownContentViewer, { content, defaultMode: "preview" }));
// The HAST `node` object must not appear serialised as a DOM attribute.
assert.equal(html.includes("node="), false, "node= attribute must not appear in HTML");
assert.equal(html.includes("[object Object]"), false, "serialised HAST object must not appear in HTML");
// Footnote reference anchor must retain its id so the backlink can navigate to it.
assert.equal(
html.includes('id="user-content-fnref-1"'),
true,
"footnote reference anchor must retain id for back-navigation",
);
// Footnote backlink must retain its aria-label for screen-reader accessibility.
assert.equal(
html.includes('aria-label="Back to reference 1"'),
true,
"footnote backlink must retain aria-label for accessibility",
);
// Footnote backlink must retain its class attribute.
assert.equal(
html.includes('class="data-footnote-backref"'),
true,
"footnote backlink must retain class attribute",
);
});
test("renders safe links as <a> with target blank and unclickable span for unsafe links", () => {
const content = `[Safe Link](https://getsemantica.ai)\n\n[Unsafe Scheme](javascript:alert(1))\n\n[Protocol Relative](//evil.com)`;
const html = renderToString(React.createElement(MarkdownContentViewer, { content, defaultMode: "preview" }));
// Safe link renders as <a> with security attributes
assert.equal(html.includes('href="https://getsemantica.ai"'), true);
assert.equal(html.includes('target="_blank"'), true);
assert.equal(html.includes('rel="noopener noreferrer"'), true);
// Unsafe links do NOT render as <a> tags
assert.equal(html.includes('href="javascript:alert(1)"'), false);
assert.equal(html.includes('href="//evil.com"'), false);
assert.equal(html.includes("Unsafe Scheme"), true);
assert.equal(html.includes("Protocol Relative"), true);
});
test("renders remote images as safe placeholder badges instead of <img> tags", () => {
const content = `![System Diagram](https://example.com/diagram.png)`;
const html = renderToString(React.createElement(MarkdownContentViewer, { content, defaultMode: "preview" }));
// No <img> tag rendered
assert.equal(html.includes("<img"), false);
// Image placeholder badge rendered
assert.equal(html.includes("Image:"), true);
assert.equal(html.includes("System Diagram"), true);
});
test("renders clear empty-state message when content is empty or null", () => {
const emptyHtml = renderToString(React.createElement(MarkdownContentViewer, { content: "" }));
assert.equal(emptyHtml.includes("No content available for this node."), true);
const nullHtml = renderToString(React.createElement(MarkdownContentViewer, { content: null }));
assert.equal(nullHtml.includes("No content available for this node."), true);
});
test("renders plain text cleanly without requiring Markdown formatting", () => {
const plainText = "Plain entity summary text without markdown formatting.";
const html = renderToString(React.createElement(MarkdownContentViewer, { content: plainText, defaultMode: "preview" }));
assert.equal(html.includes(plainText), true);
});
test("handles very large Markdown content without failure", () => {
const largeContent = `# Large Knowledge Node\n\n` + "Structured observation paragraph. ".repeat(400);
assert.equal(largeContent.length > 10000, true);
const html = renderToString(React.createElement(MarkdownContentViewer, { content: largeContent, defaultMode: "preview" }));
assert.equal(html.includes("Large Knowledge Node"), true);
});
// ─── H-2: Stale copied state lifecycle (SSR-compatible portion) ─────────────
// Full state-transition testing (Node A → copy → Node B) requires an interactive
// framework. The lifecycle correctness is guaranteed by the render-phase
// previous-prop synchronisation pattern: a `copiedForContent` state value tracks
// the content for which the copied indicator was set; when `content` changes, the
// mismatch is detected during render and `copied` is reset to false in the same
// React batch, before the new node's UI is painted. What we CAN verify in SSR
// is that the initial render for any content value shows the Copy button (not the
// Copied indicator), which confirms the initial state is always clean.
test("copy button always starts in un-copied state on initial render", () => {
const html = renderToString(React.createElement(MarkdownContentViewer, {
content: "# Some Node\n\nDescription text.",
defaultMode: "preview",
}));
// Initial render must show 'Copy', never 'Copied'
assert.equal(html.includes("Copy"), true, "Copy button must be present on initial render");
assert.equal(html.includes("Copied"), false, "Copied indicator must NOT be present on initial render");
});
integrations/__init__.py
+1 -1
View File
@@ -1,7 +1,7 @@
"""
Semantica Framework Integrations
Optional integration packages for agentic frameworks (Google ADK, Claude Agent SDK, Agno, etc.).
Optional integration packages for agentic frameworks (Google ADK, Claude Agent SDK, Agno, CrewAI, LangChain, etc.).
Each integration is self-contained, independently installable via extras_require, and maintains
zero impact on core Semantica - keeping the semantic layer lean while maximizing ecosystem reach.
"""
integrations/agno/knowledge_graph.py
+10 -13
View File
@@ -277,25 +277,22 @@ class AgnoKnowledgeGraph(_KnowledgeBase): # type: ignore[misc]
def load_urls(self, urls: List[str]) -> None:
"""Fetch each URL and ingest the response body.
Only ``http`` and ``https`` schemes are permitted to prevent SSRF.
Uses the shared SSRF guard so that ``http`` and ``https`` are the only
permitted schemes, private/loopback/link-local/cloud-metadata addresses
are blocked by default, DNS resolution is validated, and every redirect
hop is re-checked before being followed.
"""
import urllib.request
from urllib.parse import urlparse
from semantica.ingest.ssrf import request_with_ssrf_guard
from semantica.utils.exceptions import ValidationError
for url in urls:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
logger.warning(
"Skipping URL with disallowed scheme '%s': %s",
parsed.scheme,
url,
)
continue
try:
with urllib.request.urlopen(url, timeout=10) as resp: # noqa: S310
text = resp.read().decode("utf-8", errors="replace")
response = request_with_ssrf_guard("GET", url, timeout=10)
text = response.text
self._ingest_text(text, source=url)
logger.info("Loaded URL: %s", url)
except ValidationError as exc:
logger.warning("Skipping URL (SSRF check failed) %s: %s", url, exc)
except Exception as exc:
logger.warning("Failed to fetch %s: %s", url, exc)
integrations/langchain/README.md
+67
View File
@@ -0,0 +1,67 @@
# Semantica × LangChain
Drop Semantica into existing LangChain / LangGraph pipelines: GraphRAG-style
retrieval, a `VectorStore` adapter, and agent tools.
## Install
```bash
pip install semantica[langchain]
# or just the core adapter dependency:
pip install langchain-core
```
## Retriever (GraphRAG)
```python
from integrations.langchain import SemanticaRetriever
from semantica.context import ContextGraph
from semantica.vector_store import HybridSearch
graph = ContextGraph()
hybrid = HybridSearch()
retriever = SemanticaRetriever(graph=graph, hybrid=hybrid, hops=2, top_k=10)
# Use with any LangChain chain that accepts a retriever:
from langchain.chains import RetrievalQA
qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)
```
Hybrid search seeds retrieval; then graph edges are walked `hops` steps so
results go beyond flat vector similarity.
## VectorStore
```python
from integrations.langchain import SemanticaVectorStore
store = SemanticaVectorStore(hybrid=hybrid)
store.add_texts(["document one", "document two"], metadatas=[{"source": "a"}, {"source": "b"}])
docs = store.similarity_search("document", k=2)
docs, scores = store.similarity_search_with_score("document", k=2)
```
## Agent tools (LangGraph / tool-calling agents)
```python
from integrations.langchain import SemanticaKGTool, SemanticaDecisionTool
from langgraph.prebuilt import create_react_agent
tools = [
SemanticaKGTool(graph),
SemanticaDecisionTool(graph),
]
agent = create_react_agent(model, tools)
```
- `semantica_query_graph` — query the shared context graph (keyword / NL)
- `semantica_query_decisions` — search the recorded decision log
## Compatibility
- Requires `langchain-core >= 0.3`.
- All classes degrade gracefully when `langchain-core` is absent: they remain
importable (carrying the full Semantica API), and `build()` returns `None`,
so agents can branch on `LANGCHAIN_AVAILABLE`.
integrations/langchain/__init__.py
+48
View File
@@ -0,0 +1,48 @@
"""
Semantica × LangChain Integration
=================================
First-class integration between the Semantica semantic intelligence stack and
the `LangChain <https://github.com/langchain-ai/langchain>`_ / LangGraph
ecosystem.
Public surface
--------------
SemanticaRetriever — ``BaseRetriever`` with multi-hop GraphRAG (walks graph
edges from hybrid-search hits)
SemanticaVectorStore — ``VectorStore`` adapter over Semantica's hybrid search
(drop-in for RetrievalQA / LCEL chains)
SemanticaKGTool — ``BaseTool`` for querying the context graph
SemanticaDecisionTool — ``BaseTool`` exposing the recorded decision log
Quick start
-----------
pip install semantica[langchain]
>>> from integrations.langchain import (
... SemanticaRetriever,
... SemanticaVectorStore,
... SemanticaKGTool,
... SemanticaDecisionTool,
... )
Compatibility
-------------
Requires ``langchain-core >= 0.3``. All classes degrade gracefully when
``langchain-core`` is not installed — they are still importable and carry the
full Semantica API, but cannot be bound to LangChain chains/agents.
"""
from .retriever import LANGCHAIN_AVAILABLE, SemanticaRetriever
from .tools import SemanticaDecisionTool, SemanticaKGTool
from .vectorstore import SemanticaVectorStore
__all__ = [
"SemanticaRetriever",
"SemanticaVectorStore",
"SemanticaKGTool",
"SemanticaDecisionTool",
"LANGCHAIN_AVAILABLE",
]
__version__ = "0.1.0"
integrations/langchain/retriever.py
+216
View File
@@ -0,0 +1,216 @@
"""
SemanticaRetriever — LangChain ``BaseRetriever`` with multi-hop GraphRAG.
Hybrid search seeds the retrieval, then graph edges are walked for ``hops``
steps so results go beyond flat vector similarity.
"""
from __future__ import annotations
from typing import Any, Dict, List, Optional, Tuple
from semantica.utils.logging import get_logger
logger = get_logger(__name__)
# ---------------------------------------------------------------------------
# Optional: LangChain core
# ---------------------------------------------------------------------------
LANGCHAIN_AVAILABLE = False
LANGCHAIN_IMPORT_ERROR: Optional[str] = None
_BaseRetriever: Any = object
_Document: Any = None
def _get_document(**kwargs: Any) -> Any:
"""Instantiate a langchain Document lazily (keeps the import optional)."""
if _Document is None: # pragma: no cover - exercised only with langchain
raise RuntimeError(LANGCHAIN_IMPORT_ERROR or "langchain-core not installed")
return _Document(**kwargs)
try:
from langchain_core.documents import Document as _Document # type: ignore
from langchain_core.retrievers import (
BaseRetriever as _BaseRetriever, # type: ignore
)
LANGCHAIN_AVAILABLE = True
except ImportError: # pragma: no cover - exercised only without langchain
LANGCHAIN_IMPORT_ERROR = (
"langchain-core is not installed. Install with: pip install langchain-core"
)
logger.debug(LANGCHAIN_IMPORT_ERROR)
def _hit_layers(hit: Dict[str, Any]) -> Tuple[Dict[str, Any], Dict[str, Any]]:
"""Nested HybridSearch metadata and ContextGraph.query node, if present."""
metadata = hit.get("metadata") if isinstance(hit.get("metadata"), dict) else {}
node = hit.get("node") if isinstance(hit.get("node"), dict) else {}
return metadata, node
def _hit_id(hit: Dict[str, Any]) -> Optional[str]:
"""Graph node id, preferring metadata over a HybridSearch vector id."""
metadata, node = _hit_layers(hit)
return (
hit.get("node_id")
or metadata.get("node_id")
or node.get("id")
or node.get("node_id")
or hit.get("id")
)
def _hit_content(hit: Dict[str, Any], fallback: str = "") -> str:
metadata, node = _hit_layers(hit)
props = node.get("properties") if isinstance(node.get("properties"), dict) else {}
return (
hit.get("content")
or hit.get("text")
or metadata.get("content")
or metadata.get("text")
or props.get("content")
or fallback
)
def _hit_type(hit: Dict[str, Any]) -> str:
metadata, node = _hit_layers(hit)
return (
hit.get("node_type")
or hit.get("type")
or metadata.get("node_type")
or metadata.get("type")
or node.get("type")
or node.get("node_type")
or "node"
)
def _hit_score(hit: Dict[str, Any], default: float = 1.0) -> float:
return float(hit.get("score") if hit.get("score") is not None else hit.get("distance") or default)
class SemanticaRetriever(_BaseRetriever): # type: ignore[misc]
"""GraphRAG-style retriever over a Semantica ``ContextGraph``.
Args:
graph: A semantica.context.ContextGraph instance.
hybrid: A semantica.vector_store.HybridSearch instance used to seed
retrieval. If omitted, a best-effort keyword search on the graph
is used.
hops: Number of graph-edge expansion hops (default 2).
top_k: Number of seed hits (default 10).
"""
graph: Any
hybrid: Any = None
hops: int = 2
top_k: int = 10
def __init__(
self,
graph: Any,
hybrid: Any = None,
hops: int = 2,
top_k: int = 10,
**kwargs: Any,
) -> None:
"""Explicit init so the retriever works with and without langchain."""
if LANGCHAIN_AVAILABLE:
# BaseRetriever is a Pydantic model: pass the declared fields
# through so validation succeeds.
super().__init__(
graph=graph,
hybrid=hybrid,
hops=hops,
top_k=top_k,
**kwargs,
)
else:
# Without langchain-core, BaseRetriever is a plain object
super().__init__() # type: ignore[call-arg]
self.graph = graph
self.hybrid = hybrid
self.hops = hops
self.top_k = top_k
def _get_relevant_documents(self, query: str, **kwargs: Any) -> List[Any]:
"""LangChain BaseRetriever entry point."""
seed = self._seed_results(query)
if not seed:
return []
# Expand each seed node through the graph
expanded: Dict[str, Dict[str, Any]] = {}
for hit in seed:
node_id = _hit_id(hit)
if not node_id:
continue
metadata, _ = _hit_layers(hit)
expanded[node_id] = {
"content": _hit_content(hit, fallback=str(node_id)),
"node_type": _hit_type(hit),
"score": _hit_score(hit),
"metadata": metadata,
}
try:
neighbors = self.graph.get_neighbors(node_id, hops=self.hops)
for neighbor in neighbors:
nid = neighbor.get("node_id") or neighbor.get("id")
if nid and nid not in expanded:
expanded[nid] = {
"content": neighbor.get("content")
or neighbor.get("text")
or neighbor.get("name")
or str(nid),
"node_type": neighbor.get("node_type")
or neighbor.get("type")
or "node",
"score": float(neighbor.get("weight") or 0.5),
"metadata": {},
}
except Exception as exc: # graph expansion is best-effort
logger.debug("graph expansion failed for %s: %s", node_id, exc)
# Order: seed hits first (they have real scores), then neighbors.
# Keep a deterministic id->payload list (sets are unordered — see Qodo).
ordered_pairs: List[tuple] = []
seen_ids = set()
for hit in seed:
nid = _hit_id(hit)
if nid and nid in expanded and nid not in seen_ids:
ordered_pairs.append((nid, expanded[nid]))
seen_ids.add(nid)
for nid, item in expanded.items():
if nid not in seen_ids:
ordered_pairs.append((nid, item))
seen_ids.add(nid)
return [
_get_document(
page_content=item["content"],
metadata={
**item["metadata"],
"node_id": nid,
"node_type": item["node_type"],
"score": item["score"],
},
)
for nid, item in ordered_pairs
]
def _seed_results(self, query: str) -> List[Dict[str, Any]]:
"""Get seed results from hybrid search or a graph keyword scan."""
if self.hybrid is not None:
try:
return self.hybrid.search(query, k=self.top_k)
except Exception as exc:
logger.debug("hybrid search failed, falling back: %s", exc)
# Best-effort keyword scan over graph nodes (ContextGraph.query)
try:
return self.graph.query(query, limit=self.top_k)
except Exception:
return []
integrations/langchain/tools.py
+133
View File
@@ -0,0 +1,133 @@
"""
SemanticaKGTool / SemanticaDecisionTool — LangChain ``BaseTool`` adapters
for LangChain / LangGraph agents.
"""
from __future__ import annotations
import json
from typing import Any, Optional, Type
from pydantic import BaseModel, ConfigDict, Field
from semantica.utils.logging import get_logger
logger = get_logger(__name__)
# ---------------------------------------------------------------------------
# Optional: LangChain core
# ---------------------------------------------------------------------------
LANGCHAIN_AVAILABLE = False
LANGCHAIN_IMPORT_ERROR: Optional[str] = None
_BaseTool: Any = object
try:
from langchain_core.tools import BaseTool as _BaseTool # type: ignore
LANGCHAIN_AVAILABLE = True
except ImportError: # pragma: no cover
LANGCHAIN_IMPORT_ERROR = (
"langchain-core is not installed. Install with: pip install langchain-core"
)
logger.debug(LANGCHAIN_IMPORT_ERROR)
def _json(payload: Any) -> str:
return json.dumps(payload, default=str, ensure_ascii=False)
class QueryGraphInput(BaseModel):
query: str = Field(..., description="Natural-language or keyword graph query")
limit: int = Field(10, description="Maximum matching nodes to return")
class QueryDecisionsInput(BaseModel):
category: str = Field(
"",
description="Keyword to search recorded decisions; empty returns insights",
)
limit: int = Field(10, description="Maximum results when searching by keyword")
class SemanticaKGTool(_BaseTool): # type: ignore[misc]
"""LangChain tool for querying a Semantica ``ContextGraph``.
Args:
graph: A semantica.context.ContextGraph instance.
Example:
>>> tool = SemanticaKGTool(graph)
>>> agent = create_react_agent(model, tools=[tool])
"""
model_config = ConfigDict(arbitrary_types_allowed=True)
name: str = "semantica_query_graph"
description: str = (
"Query Semantica's shared context graph with a natural-language "
"keyword query. Returns matching entities and relationships."
)
args_schema: Type[BaseModel] = QueryGraphInput
graph: Any = None
def __init__(self, graph: Any = None, **kwargs: Any) -> None:
if LANGCHAIN_AVAILABLE:
super().__init__(graph=graph, **kwargs)
else:
super().__init__()
self.graph = graph
def build(self) -> Any:
"""Return this tool, or None if langchain-core is missing."""
return self if LANGCHAIN_AVAILABLE else None
def _run(self, query: str, limit: int = 10, **kwargs: Any) -> str:
try:
return _json(self.graph.query(query, limit=limit))
except Exception as exc:
return _json({"error": str(exc)})
async def _arun(self, query: str, limit: int = 10, **kwargs: Any) -> str:
return self._run(query, limit=limit)
class SemanticaDecisionTool(_BaseTool): # type: ignore[misc]
"""LangChain tool for searching Semantica's recorded decision log.
Args:
graph: A semantica.context.ContextGraph instance.
"""
model_config = ConfigDict(arbitrary_types_allowed=True)
name: str = "semantica_query_decisions"
description: str = (
"Search Semantica's recorded decision log with a keyword query. "
"Returns decisions, rationale, and context."
)
args_schema: Type[BaseModel] = QueryDecisionsInput
graph: Any = None
def __init__(self, graph: Any = None, **kwargs: Any) -> None:
if LANGCHAIN_AVAILABLE:
super().__init__(graph=graph, **kwargs)
else:
super().__init__()
self.graph = graph
def build(self) -> Any:
"""Return this tool, or None if langchain-core is missing."""
return self if LANGCHAIN_AVAILABLE else None
def _run(self, category: str = "", limit: int = 10, **kwargs: Any) -> str:
try:
if category:
return _json(self.graph.query(category, limit=limit))
return _json(self.graph.get_decision_insights())
except Exception as exc:
return _json({"error": str(exc)})
async def _arun(self, category: str = "", limit: int = 10, **kwargs: Any) -> str:
return self._run(category=category, limit=limit)
integrations/langchain/vectorstore.py
+143
View File
@@ -0,0 +1,143 @@
"""
SemanticaVectorStore — LangChain ``VectorStore`` adapter over Semantica's
hybrid search (``semantica.vector_store.HybridSearch``).
"""
from __future__ import annotations
from typing import Any, Dict, Iterable, List, Optional
from semantica.utils.logging import get_logger
from .retriever import _hit_content, _hit_id, _hit_score, _hit_type, _hit_layers
logger = get_logger(__name__)
# ---------------------------------------------------------------------------
# Optional: LangChain core
# ---------------------------------------------------------------------------
LANGCHAIN_AVAILABLE = False
LANGCHAIN_IMPORT_ERROR: Optional[str] = None
_VectorStoreBase: Any = object
_Document: Any = None
def _make_document(**kwargs: Any) -> Any:
if _Document is None: # pragma: no cover
raise RuntimeError(LANGCHAIN_IMPORT_ERROR or "langchain-core not installed")
return _Document(**kwargs)
try:
from langchain_core.documents import Document as _Document # type: ignore
from langchain_core.vectorstores import (
VectorStore as _VectorStoreBase, # type: ignore
)
LANGCHAIN_AVAILABLE = True
except ImportError: # pragma: no cover
LANGCHAIN_IMPORT_ERROR = (
"langchain-core is not installed. Install with: pip install langchain-core"
)
logger.debug(LANGCHAIN_IMPORT_ERROR)
def _document_from_hit(hit: Dict[str, Any], include_score: bool = True) -> Any:
metadata, _ = _hit_layers(hit)
node_id = _hit_id(hit)
doc_meta = {
**metadata,
"node_id": node_id,
"node_type": _hit_type(hit),
}
if include_score:
doc_meta["score"] = _hit_score(hit, default=0.0)
return _make_document(
page_content=_hit_content(hit),
metadata=doc_meta,
)
class SemanticaVectorStore(_VectorStoreBase): # type: ignore[misc]
"""Wrap Semantica hybrid search as a LangChain ``VectorStore``.
Args:
hybrid: A semantica.vector_store.HybridSearch instance.
vector_store: Optional Semantica vector store passed through to
``HybridSearch.add_texts``.
"""
hybrid: Any
vector_store: Any = None
def __init__(self, hybrid: Any, vector_store: Any = None, **kwargs: Any) -> None:
if LANGCHAIN_AVAILABLE:
super().__init__(**kwargs)
else:
super().__init__()
self.hybrid = hybrid
self.vector_store = vector_store
# -- required VectorStore API ------------------------------------------
def add_texts(
self,
texts: Iterable[str],
metadatas: Optional[List[Dict[str, Any]]] = None,
**kwargs: Any,
) -> List[str]:
"""Embed and store texts; return the generated IDs.
Delegates to the Semantica ``VectorStore.add_documents`` backing the
HybridSearch instance (or to ``hybrid.vector_store`` if provided).
"""
if self.vector_store is not None:
return self.vector_store.add_documents(
list(texts), metadata=metadatas, **kwargs
)
vs = getattr(self.hybrid, "vector_store", None)
if vs is not None and hasattr(vs, "add_documents"):
return vs.add_documents(list(texts), metadata=metadatas, **kwargs)
raise ValueError(
"SemanticaVectorStore requires a Semantica vector store with "
"add_documents (pass vector_store=... to the HybridSearch or to "
"SemanticaVectorStore)"
)
def similarity_search(self, query: str, k: int = 4, **kwargs: Any) -> List[Any]:
"""Return documents most similar to the query."""
return [_document_from_hit(hit) for hit in self.hybrid.search(query, k=k)]
def similarity_search_with_score(
self, query: str, k: int = 4, **kwargs: Any
) -> List[Any]:
"""Return (document, score) pairs."""
return [
(
_document_from_hit(hit, include_score=False),
_hit_score(hit, default=0.0),
)
for hit in self.hybrid.search(query, k=k)
]
@classmethod
def from_texts(
cls,
texts: List[str],
embedding: Any = None,
metadatas: Optional[List[Dict[str, Any]]] = None,
**kwargs: Any,
) -> "SemanticaVectorStore":
"""Build a store from a list of texts (LangChain convention).
Requires a pre-configured ``hybrid`` instance passed via kwargs.
"""
hybrid = kwargs.pop("hybrid", None)
if hybrid is None:
raise ValueError(
"SemanticaVectorStore.from_texts requires a 'hybrid' "
"HybridSearch instance as a keyword argument"
)
store = cls(hybrid=hybrid, **kwargs)
store.add_texts(texts, metadatas=metadatas)
return store
integrations/openclaw/mcp_tool.py
+35 -1
View File
@@ -116,7 +116,41 @@ class OpenClawKGTool:
)
def __init__(self, base_url: str = "http://localhost:8000", timeout: int = 30) -> None:
self.base_url = base_url.rstrip("/")
# Validate base_url at construction time so callers get an immediate,
# actionable error rather than a cryptic failure on the first request.
# allow_private_ips=True because the documented default (localhost:8000)
# is intentionally a local Semantica server; the scheme check and
# URL-structure check still apply unconditionally.
try:
from semantica.ingest.ssrf import validate_url_for_request
validate_url_for_request(base_url, allow_private_ips=True)
except ImportError:
# semantica.ingest not installed in minimal openclaw-only environments;
# mirror the structural checks that validate_url_for_request performs
# unconditionally (before allow_private_ips is consulted), so the
# guarantee in the comment above — "scheme check and URL-structure check
# still apply unconditionally" — holds in this path too.
from urllib.parse import urlparse as _urlparse
if not isinstance(base_url, str) or not base_url.strip():
raise ValueError("OpenClawKGTool base_url must be a non-empty string.")
_parsed = _urlparse(base_url.strip())
_scheme = (_parsed.scheme or "").lower()
if _scheme not in ("http", "https"):
raise ValueError(
f"OpenClawKGTool base_url scheme '{_parsed.scheme}' is not permitted. "
"Only http and https are allowed."
)
if not _parsed.netloc:
raise ValueError(
f"Invalid OpenClawKGTool base_url '{base_url}': "
"URL must include a netloc (domain or host)."
)
if not _parsed.hostname:
raise ValueError(
f"Invalid OpenClawKGTool base_url '{base_url}': "
"URL must include a hostname."
)
self.base_url = base_url.strip().rstrip("/")
self.timeout = timeout
self._session: Any = None
mcp/__init__.py
+11
View File
@@ -21,6 +21,17 @@ Configure in Claude Desktop, Windsurf, Cline, Continue, VS Code:
}
"""
import os
# MCP stdio framing IS stdout: any progress bar or console renderer that writes
# to stdout would interleave with the JSON-RPC stream and corrupt framing for
# every client. This package is always used as an MCP stdio server, so force
# progress tracking off for the entire process. Set before importing server /
# tools so the Semantica progress-tracker singleton is never created with
# output enabled (the singleton reads this variable at construction time and
# the enabled.setter re-checks it, so later re-enable attempts are also blocked).
os.environ["SEMANTICA_DISABLE_PROGRESS"] = "1"
# `semantica.__version__` is the authoritative package version — see
# semantica/mcp_server/__init__.py for why it is used directly rather than
# importlib.metadata.version("semantica").
mcp/tools/export.py
+6 -1
View File
@@ -80,7 +80,12 @@ def handle_export_graph(args: dict) -> dict:
if rdf_fmt:
try:
from semantica.export import RDFExporter
rdf_str = RDFExporter().export_to_rdf(graph, format=rdf_fmt)
# RDFExporter.export_to_rdf() expects the canonical kg dict
# {"entities": [...], "relationships": [...]}, not a ContextGraph
# object. Convert before handing off; passing the raw graph
# caused AttributeError: 'ContextGraph' object has no attribute
# 'get' on every RDF format.
rdf_str = RDFExporter().export_to_rdf(graph.to_kg_dict(), format=rdf_fmt)
return {"format": rdf_fmt, "data": rdf_str}
except Exception as exc:
return {"error": f"RDF export failed: {exc}"}
plugins/.claude-plugin/README.md
+1 -1
View File
@@ -53,7 +53,7 @@ plugins/
## Prerequisites
```bash
git clone https://github.com/Hawksight-AI/semantica.git
git clone https://github.com/semantica-agi/semantica.git
cd semantica
pip install semantica # Python 3.10+
```
plugins/.claude-plugin/marketplace.json
+2 -2
View File
@@ -1,8 +1,8 @@
{
"name": "semantica-local",
"owner": {
"name": "Hawksight AI",
"url": "https://github.com/Hawksight-AI/semantica"
"name": "Semantica",
"url": "https://github.com/semantica-agi/semantica"
},
"plugins": [
{
plugins/.claude-plugin/plugin.json
+2 -2
View File
@@ -5,8 +5,8 @@
"author": {
"name": "Semantica Contributors"
},
"homepage": "https://github.com/Hawksight-AI/semantica",
"repository": "https://github.com/Hawksight-AI/semantica",
"homepage": "https://github.com/semantica-agi/semantica",
"repository": "https://github.com/semantica-agi/semantica",
"license": "MIT",
"keywords": [
"semantica",
plugins/.cline-plugin/plugin.json
+2 -2
View File
@@ -6,8 +6,8 @@
"author": {
"name": "Semantica Contributors"
},
"homepage": "https://github.com/Hawksight-AI/semantica",
"repository": "https://github.com/Hawksight-AI/semantica",
"homepage": "https://github.com/semantica-agi/semantica",
"repository": "https://github.com/semantica-agi/semantica",
"license": "MIT",
"keywords": [
"semantica",
plugins/.codex-plugin/plugin.json
+2 -2
View File
@@ -5,8 +5,8 @@
"author": {
"name": "Semantica Contributors"
},
"homepage": "https://github.com/Hawksight-AI/semantica",
"repository": "https://github.com/Hawksight-AI/semantica",
"homepage": "https://github.com/semantica-agi/semantica",
"repository": "https://github.com/semantica-agi/semantica",
"license": "MIT",
"keywords": [
"semantica",
plugins/.continue-plugin/plugin.json
+2 -2
View File
@@ -6,8 +6,8 @@
"author": {
"name": "Semantica Contributors"
},
"homepage": "https://github.com/Hawksight-AI/semantica",
"repository": "https://github.com/Hawksight-AI/semantica",
"homepage": "https://github.com/semantica-agi/semantica",
"repository": "https://github.com/semantica-agi/semantica",
"license": "MIT",
"keywords": [
"semantica",
plugins/.cursor-plugin/plugin.json
+2 -2
View File
@@ -6,8 +6,8 @@
"author": {
"name": "Semantica Contributors"
},
"homepage": "https://github.com/Hawksight-AI/semantica",
"repository": "https://github.com/Hawksight-AI/semantica",
"homepage": "https://github.com/semantica-agi/semantica",
"repository": "https://github.com/semantica-agi/semantica",
"license": "MIT",
"keywords": [
"semantica",
plugins/.openclaw-plugin/plugin.json
+2 -2
View File
@@ -6,8 +6,8 @@
"author": {
"name": "Semantica Contributors"
},
"homepage": "https://github.com/Hawksight-AI/semantica",
"repository": "https://github.com/Hawksight-AI/semantica",
"homepage": "https://github.com/semantica-agi/semantica",
"repository": "https://github.com/semantica-agi/semantica",
"license": "MIT",
"keywords": [
"semantica",
plugins/.vscode-plugin/plugin.json
+2 -2
View File
@@ -6,8 +6,8 @@
"author": {
"name": "Semantica Contributors"
},
"homepage": "https://github.com/Hawksight-AI/semantica",
"repository": "https://github.com/Hawksight-AI/semantica",
"homepage": "https://github.com/semantica-agi/semantica",
"repository": "https://github.com/semantica-agi/semantica",
"license": "MIT",
"keywords": [
"semantica",
plugins/.windsurf-plugin/plugin.json
+2 -2
View File
@@ -6,8 +6,8 @@
"author": {
"name": "Semantica Contributors"
},
"homepage": "https://github.com/Hawksight-AI/semantica",
"repository": "https://github.com/Hawksight-AI/semantica",
"homepage": "https://github.com/semantica-agi/semantica",
"repository": "https://github.com/semantica-agi/semantica",
"license": "MIT",
"keywords": [
"semantica",
poc_runner.py
-9
View File
@@ -187,15 +187,6 @@ def poc_vuln3():
})
return nodes
# Simulate the CSV parser — mirrors export_import.py lines 131-133
def parse_import_csv_row(row: dict) -> dict:
"""Mirrors export_import.py CSV node ID extraction (no sanitization)."""
node_id = row.get("id") or row.get("node_id") or row.get(":ID") or row.get("_id")
return {
"id": str(node_id), # ← UNSANITIZED
"type": row.get("type", "entity"),
}
# Attack payloads
payloads = [
# Header injection payload (chained with VULN-1)
pyproject.toml
+7 -5
View File
@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
[project]
name = "semantica"
version = "0.6.5"
description = "Accountability and context layer for AI agents. Context graphs, decision intelligence, full provenance tracking, and explainable reasoning engines — every AI decision traceable, every output auditable."
version = "0.6.6"
description = "Graph-Native Infrastructure for Context and Accountable AI Systems: context graphs, decision intelligence, full provenance tracking, and explainable reasoning engines — every AI decision traceable, every output auditable."
readme = "README.md"
license = { text = "MIT" }
@@ -85,7 +85,8 @@ dependencies = [
"loguru>=0.7.3",
"structlog>=22.1.0",
"gensim>=4.4.0",
"httpx<0.29.0"
"httpx<0.29.0",
"pyarrow>=14.0.0"
]
[project.urls]
@@ -103,7 +104,7 @@ Discord = "https://discord.gg/sV34vps5hH"
llm-openai = ["openai>=1.0.0"]
llm-groq = ["groq>=0.4.0"]
llm-gemini = ["google-genai>=0.1.0"]
llm-anthropic = ["anthropic>=0.18.0"]
llm-anthropic = ["anthropic>=0.122.0"]
llm-ollama = ["ollama>=0.1.0"]
llm-deepseek = ["openai>=1.0.0"]
llm-litellm = ["litellm>=1.83.9"]
@@ -205,6 +206,7 @@ agno = ["agno>=1.0.0"]
# needed (it pulls vulnerable transitive deps like chromadb) and would only
# duplicate the prebuilt tooling users can install separately.
crewai = ["crewai>=0.80.0"]
langchain = ["langchain-core>=0.3.0"]
# ---- File Watching ----
watch = ["watchdog>=6.0.0"]
@@ -252,7 +254,7 @@ explorer-lite = [
# dependency-audit/security gates. Install it explicitly via ``semantica[crewai]``.
all = [
"semantica[dev,viz,infra,cloud,monitoring,watch,llm-all,models-huggingface,split-all,graph-all,tripletstore-oxigraph,vectorstore-all,parse-docling,ingest-parquet,ingest-arrow,shacl,explorer]",
"semantica[dev,viz,infra,cloud,monitoring,watch,llm-all,models-huggingface,split-all,graph-all,tripletstore-oxigraph,vectorstore-all,parse-docling,ingest-parquet,ingest-arrow,shacl,agno]"
"semantica[dev,viz,infra,cloud,monitoring,watch,llm-all,models-huggingface,split-all,graph-all,tripletstore-oxigraph,vectorstore-all,parse-docling,ingest-parquet,ingest-arrow,shacl,agno,langchain]"
]
# ---------------- ENTRYPOINTS ----------------
requirements-ci.txt
+800 -323
View File
File diff suppressed because it is too large Load Diff
semantica/__init__.py
+1 -1
View File
@@ -10,7 +10,7 @@ Main exports:
- Config: Configuration management
"""
__version__ = "0.6.5"
__version__ = "0.6.6"
__author__ = "Semantica Contributors"
__license__ = "MIT"
semantica/change_management/change_management_usage.md
+1 -1
View File
@@ -1039,6 +1039,6 @@ manager = TemporalVersionManager(storage_path="large_data.db")
## Support
For questions or issues:
- GitHub Issues: https://github.com/Hawksight-AI/semantica/issues
- GitHub Issues: https://github.com/semantica-agi/semantica/issues
- Documentation: https://semantica.readthedocs.io
- Community: https://discord.gg/sV34vps5hH
semantica/change_management/version_storage.py
+14
View File
@@ -31,6 +31,7 @@ import hashlib
import json
import sqlite3
import threading
import warnings
from abc import ABC, abstractmethod
from datetime import datetime
from pathlib import Path
@@ -62,6 +63,12 @@ def create_graph_snapshot_record(
"""
Creates a standardized snapshot metadata record for a named graph.
.. deprecated::
``create_graph_snapshot_record()`` is deprecated and will be removed in
a future major version. It has no callers inside Semantica; build the
record inline and checksum it with
:func:`semantica.change_management.compute_checksum` instead.
Args:
version_id: Unique identifier for this snapshot
graph_uri: The underlying named graph URI in the triplet store
@@ -69,6 +76,13 @@ def create_graph_snapshot_record(
description: Purpose or context of the snapshot
metadata: Additional tags or pipeline context
"""
warnings.warn(
"create_graph_snapshot_record() is deprecated and will be removed in a "
"future major version. Build the snapshot record inline and use "
"semantica.change_management.compute_checksum() instead.",
DeprecationWarning,
stacklevel=2,
)
record = {
"label": version_id,
semantica/cli.py
+193 -4
View File
@@ -773,10 +773,22 @@ def changelog(cli_ctx: CLIContext, local_json: bool) -> None:
_run_with_error_handling(_action)
class _DeepEmbeddingFailure(Exception):
"""A deep-probe failure from doctor's embedding checks.
Marks failures that happened AFTER the backend imported cleanly — model
load, probe, or runtime problems — so the check's hint can point at the
real remediation instead of `pip install`.
"""
@main.command()
@click.option("--json", "local_json", is_flag=True, default=False)
@click.option("--deep-embeddings", "deep_embeddings", is_flag=True, default=False,
help="Also instantiate the local embedding backends and embed a probe "
"text (catches backends that import cleanly but cannot load).")
@click.pass_obj
def doctor(cli_ctx: CLIContext, local_json: bool) -> None:
def doctor(cli_ctx: CLIContext, local_json: bool, deep_embeddings: bool) -> None:
"""Run a health check on all Semantica components and backends."""
import importlib.metadata
cli_ctx = _require_ctx(cli_ctx)
@@ -787,6 +799,16 @@ def doctor(cli_ctx: CLIContext, local_json: bool) -> None:
try:
note = fn()
return label, "ok", note, None
except _DeepEmbeddingFailure as exc:
# A deep-probe failure means the package IMPORTED fine: the pip
# hint would be the wrong remediation for what is actually a
# runtime/model-load problem (broken torch, failed model
# download, missing shared libs).
return label, "fail", str(exc), (
"runtime/model-load failure — reinstalling the package usually "
"does not help; check the warnings above (torch install, model "
"download, disk space)"
)
except Exception as exc:
return label, "fail", str(exc), hint
@@ -827,6 +849,50 @@ def doctor(cli_ctx: CLIContext, local_json: bool) -> None:
return f"{backend} importable"
checks.append(_check("Vector store", _vector, hint="pip install semantica[vectorstore-…]"))
# Embedding backends (#994): `doctor` used to report all green while
# every local embedding backend was non-functional — import success
# says nothing about model loading. Default checks stay cheap
# (import + version); --deep-embeddings (or
# SEMANTICA_DOCTOR_DEEP_EMBEDDINGS=1) instantiates the backend through
# TextEmbedder and embeds a probe, which is the only level that
# catches a backend that imports cleanly but cannot actually load.
deep = deep_embeddings or os.environ.get("SEMANTICA_DOCTOR_DEEP_EMBEDDINGS", "").strip().lower() in ("1", "true", "yes", "on")
def _embedding_backend(method: str) -> str:
if method == "sentence_transformers":
import sentence_transformers # noqa: F401
note = f"importable ({importlib.metadata.version('sentence-transformers')})"
else:
import fastembed # noqa: F401
note = f"importable ({importlib.metadata.version('fastembed')})"
if not deep:
return note
try:
from .embeddings import TextEmbedder
embedder = TextEmbedder(method=method)
if embedder.model is None and embedder.fastembed_model is None:
raise RuntimeError(
"model failed to load — the hash fallback is active "
"(see warnings above); embedding quality is degraded"
)
probe = embedder.embed_text("semantica doctor embedding probe")
except _DeepEmbeddingFailure:
raise
except Exception as exc:
raise _DeepEmbeddingFailure(str(exc)) from exc
return f"{note}; deep probe ok ({len(probe)}-dim)"
checks.append(_check(
"Embeddings (sentence-transformers)",
lambda: _embedding_backend("sentence_transformers"),
hint="pip install sentence-transformers",
))
checks.append(_check(
"Embeddings (fastembed)",
lambda: _embedding_backend("fastembed"),
hint="pip install fastembed",
))
# LLM provider keys
for provider, var in [("OpenAI", "OPENAI_API_KEY"), ("Anthropic", "ANTHROPIC_API_KEY"),
("Groq", "GROQ_API_KEY")]:
@@ -1666,6 +1732,93 @@ def embed(ctx: click.Context) -> None:
click.echo(ctx.get_help())
def _json_default(obj) -> object:
"""JSON serialiser that converts NumPy scalars/arrays to native Python types.
Falls back to ``str()`` for everything else so the writer never crashes on
unexpected types (e.g. ``datetime``, custom domain objects).
"""
try:
import numpy as np # local import — only needed when result contains numpy
if isinstance(obj, np.ndarray):
return obj.tolist()
if isinstance(obj, np.generic):
return obj.item()
except ImportError:
pass
return str(obj)
def _write_result_output(out_path: Path, result) -> None:
"""Serialize a structured CLI result (dict or list) for ``--output``.
Domain commands like ``deduplicate`` and ``ontology align`` produce dicts
and lists, not numeric matrices — routing them through the embeddings
writer rejected their shapes and extensions (.csv is documented for
deduplicate). JSON-family formats serialize anything; CSV serializes a
list of dicts (or a single dict as one row).
Accepted extensions: .json, .jsonl, .csv (no-extension and .txt are
rejected so the path reported to the caller always matches the file
actually created, consistent with every other --output in the CLI).
"""
import json as _json
suffix = out_path.suffix.lower()
# ── JSON ────────────────────────────────────────────────────────────────
if suffix == ".json":
with open(out_path, "w", encoding="utf-8") as fh:
_json.dump(result, fh, indent=2, default=_json_default)
return
# ── JSON Lines ──────────────────────────────────────────────────────────
# Every record must occupy exactly one line. Wrap a bare dict in a list
# so callers never need to know whether their result is singular or plural.
if suffix == ".jsonl":
items = result if isinstance(result, list) else [result]
with open(out_path, "w", encoding="utf-8") as fh:
for item in items:
fh.write(_json.dumps(item, default=_json_default) + "\n")
return
# ── CSV ─────────────────────────────────────────────────────────────────
if suffix == ".csv":
import pandas as pd
rows = result if isinstance(result, list) else [result]
if not rows:
raise click.ClickException(
"No results to write — output file not created."
)
# Normalise numpy scalars/arrays to Python natives so to_csv() does
# not fall back to repr() strings for array-valued cells.
def _normalise(row):
if not isinstance(row, dict):
return row
out = {}
for k, v in row.items():
try:
import numpy as np
if isinstance(v, np.ndarray):
v = v.tolist()
elif isinstance(v, np.generic):
v = v.item()
except ImportError:
pass
out[k] = v
return out
pd.DataFrame([_normalise(r) for r in rows]).to_csv(out_path, index=False)
return
# ── unsupported ─────────────────────────────────────────────────────────
display = suffix if suffix else "(no extension)"
raise click.ClickException(
f"Unsupported output format '{display}'. Use .json, .jsonl, or .csv"
)
@embed.command("generate")
@click.argument("input_path")
@click.option("--model",
@@ -1708,7 +1861,43 @@ def embed_generate(cli_ctx: CLIContext, input_path: str, model: str,
except ImportError as exc:
raise click.ClickException(f"Embeddings module not available: {exc}") from exc
if output:
Path(output).write_text(json.dumps(result, default=str), encoding="utf-8")
output_path = Path(output)
suffix = output_path.suffix.lower()
try:
import numpy as np
import pandas as pd
arr = np.asarray(result)
if arr.ndim == 1:
arr = arr[np.newaxis, :]
if arr.ndim != 2:
raise click.ClickException(
f"embed generate --output expects a 1-D or 2-D array, "
f"got {arr.ndim}-D (shape {arr.shape})"
)
rows = [list(row) for row in arr]
if suffix == ".parquet":
# Schema: single 'embedding' column (list[float] per row).
# embed index detects vector columns via
# isinstance(df[c].iloc[0], (list, np.ndarray)).
df = pd.DataFrame({"embedding": rows})
df.to_parquet(output_path, index=False)
elif suffix in (".json", ".jsonl"):
df = pd.DataFrame({"embedding": rows})
df.to_json(
output_path,
orient="records",
lines=(suffix == ".jsonl"),
)
else:
raise click.ClickException(
f"Unsupported output format '{suffix}'. "
"Use .parquet, .json, or .jsonl"
)
except ImportError as exc:
raise click.ClickException(
f"Missing dependency for --output: {exc}. "
"Install pyarrow with: pip install pyarrow"
) from exc
_ok(cli_ctx, f"Wrote {output}")
elif _is_json(cli_ctx, local_json):
_jecho(result if isinstance(result, dict) else {"status": "ok"})
@@ -1930,7 +2119,7 @@ def deduplicate(
except ImportError as exc:
raise click.ClickException(f"Deduplication module not available: {exc}") from exc
if output:
Path(output).write_text(json.dumps(result, default=str), encoding="utf-8")
_write_result_output(Path(output), result)
_ok(cli_ctx, f"Wrote {output}")
elif _is_json(cli_ctx, local_json):
_jecho(result if isinstance(result, (dict, list)) else {"result": str(result)})
@@ -3054,7 +3243,7 @@ def ontology_align(cli_ctx: CLIContext, source: str, target: str, strategy: str,
except ImportError as exc:
raise click.ClickException(f"Ontology module not available: {exc}") from exc
if output:
Path(output).write_text(json.dumps(result, default=str), encoding="utf-8")
_write_result_output(Path(output), result)
_ok(cli_ctx, f"Wrote {output}")
elif _is_json(cli_ctx, local_json):
_jecho(result if isinstance(result, dict) else {"alignments": str(result)})
semantica/context/_markdown_filesystem.py
+32
View File
@@ -0,0 +1,32 @@
"""Filesystem safety helpers for human-editable Markdown persistence."""
import os
import stat
from pathlib import Path
from typing import Optional
def is_filesystem_link(path: Path) -> bool:
"""Return whether *path* is a symlink, junction, or Windows reparse point."""
if path.is_symlink():
return True
isjunction = getattr(os.path, "isjunction", None)
if isjunction is not None and isjunction(path):
return True
try:
attributes = getattr(os.lstat(path), "st_file_attributes", 0)
except (FileNotFoundError, NotADirectoryError):
return False
reparse_point = getattr(stat, "FILE_ATTRIBUTE_REPARSE_POINT", 0x400)
return bool(attributes & reparse_point)
def find_filesystem_link(path: Path) -> Optional[Path]:
"""Return the first linked component in *path*, including its ancestors."""
for candidate in (path, *path.parents):
if is_filesystem_link(candidate):
return candidate
return None
semantica/context/agent_memory.py
+49 -37
View File
@@ -77,6 +77,7 @@ import yaml
from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker
from ..utils.types import EntityDict, RelationshipDict
from ._markdown_filesystem import find_filesystem_link
class _UniqueKeySafeLoader(yaml.SafeLoader):
@@ -1742,9 +1743,10 @@ class AgentMemory:
@staticmethod
def _write_markdown_file(file_path: Path, document: str) -> None:
"""Atomically replace a Markdown file without following output symlinks."""
if file_path.is_symlink():
if find_filesystem_link(file_path) is not None:
raise ValueError(
f"Refusing to overwrite Markdown symbolic link: {file_path}"
"Refusing to overwrite Markdown symbolic link or junction: "
f"{file_path}"
)
temporary_path = None
@@ -1867,7 +1869,8 @@ class AgentMemory:
if "\n" not in data and "\r" not in data:
candidate = Path(data)
try:
candidate_exists = candidate.exists()
candidate_is_link = find_filesystem_link(candidate) is not None
candidate_exists = candidate_is_link or candidate.exists()
except OSError as exc:
error_message = (
"Failed to inspect possible Markdown import "
@@ -1909,62 +1912,71 @@ class AgentMemory:
return memories
def _read_markdown_file_content(self, file_path: Path) -> str:
if file_path.is_symlink():
raise ValueError(f"Symlink Markdown import paths are rejected: {file_path}")
if find_filesystem_link(file_path) is not None:
raise ValueError(
"Symlink Markdown import paths are rejected; symbolic links and "
f"junctions are unsafe: {file_path}"
)
flags = os.O_RDONLY
if hasattr(os, "O_NOFOLLOW"):
# On POSIX, O_NOFOLLOW makes os.open() fail with ELOOP if the
# final path component is a symlink, atomically closing the TOCTOU
# window between the is_symlink() check above and the open call.
# On Windows, O_NOFOLLOW is not available; the is_symlink() pre-check
# above is the only symlink defense and remains vulnerable to a narrow
# race. The fstat()/S_ISREG guard below still rejects special files
# (FIFOs, devices) on both platforms.
flags |= os.O_NOFOLLOW
nofollow_flag = getattr(os, "O_NOFOLLOW", 0)
flags |= nofollow_flag
try:
fd = os.open(str(file_path), flags)
except OSError as exc:
if exc.errno == getattr(errno, "ELOOP", None):
if (
(nofollow_flag and exc.errno == errno.ELOOP)
or find_filesystem_link(file_path) is not None
):
raise ValueError(
f"Symlink Markdown import paths are rejected: {file_path}"
"Symlink Markdown import paths are rejected; symbolic links "
f"and junctions are unsafe: {file_path}"
) from exc
raise
try:
stat_res = os.fstat(fd)
if not stat.S_ISREG(stat_res.st_mode):
if find_filesystem_link(file_path) is not None:
raise ValueError(
"Symlink Markdown import paths are rejected; symbolic links "
f"and junctions are unsafe: {file_path}"
)
if not stat.S_ISREG(os.fstat(fd).st_mode):
raise ValueError(
f"Markdown import path is not a regular file: {file_path}"
)
with open(fd, "r", encoding="utf-8", closefd=True) as f:
return f.read()
except Exception:
try:
with os.fdopen(fd, mode="r", encoding="utf-8") as source:
fd = -1
return source.read()
finally:
if fd >= 0:
os.close(fd)
except OSError:
pass
raise
def _read_markdown_path(self, path: Path) -> List[Tuple[str, str]]:
if path.is_symlink():
raise ValueError(f"Symlink Markdown import paths are rejected: {path}")
if find_filesystem_link(path) is not None:
raise ValueError(
"Symlink Markdown import paths are rejected; symbolic links and "
f"junctions are unsafe: {path}"
)
if not path.exists():
raise FileNotFoundError(f"Markdown import path does not exist: {path}")
if path.is_dir():
file_paths = sorted(
(
file_path
for file_path in path.iterdir()
if file_path.is_file()
and not file_path.is_symlink()
and file_path.suffix.lower() in self._MARKDOWN_EXTENSIONS
),
key=lambda file_path: (file_path.name.casefold(), file_path.name),
)
file_paths = []
for file_path in path.iterdir():
if file_path.suffix.lower() not in self._MARKDOWN_EXTENSIONS:
continue
if find_filesystem_link(file_path) is not None:
continue
if file_path.is_file():
file_paths.append(file_path)
if find_filesystem_link(path) is not None:
raise ValueError(
"Symlink Markdown import paths are rejected; symbolic links "
f"and junctions are unsafe: {path}"
)
file_paths.sort(key=lambda item: (item.name.casefold(), item.name))
elif path.is_file():
file_paths = [path]
else:
semantica/context/context_graph.py
+1248 -42
View File
File diff suppressed because it is too large Load Diff
semantica/context/graph_schema.py
+15
View File
@@ -6,6 +6,7 @@ including node labels, relationship types, and indexes for graph databases.
"""
import json
import warnings
from typing import Dict, Any, List
from ..graph_store import GraphStore
@@ -460,11 +461,25 @@ def drop_decision_schema(graph_store: GraphStore) -> None:
"""
Drop decision tracking schema (for cleanup/testing).
.. deprecated::
``drop_decision_schema()`` is deprecated and will be removed in a future
major version. It has no callers inside Semantica; issue the DROP
CONSTRAINT / DROP INDEX / DETACH DELETE statements directly against your
:class:`~semantica.graph_store.GraphStore` instead.
Args:
graph_store: Graph database instance
"""
logger = get_logger(__name__)
warnings.warn(
"drop_decision_schema() is deprecated and will be removed in a future "
"major version. Issue the DROP CONSTRAINT / DROP INDEX / DETACH DELETE "
"statements directly against your GraphStore instead.",
DeprecationWarning,
stacklevel=2,
)
try:
# Drop constraints
constraints = [
semantica/deduplication/duplicate_detector.py
+36 -8
View File
@@ -278,8 +278,12 @@ class DuplicateDetector:
for i, (entity1, entity2, score) in enumerate(similarities):
candidate = self._create_duplicate_candidate(entity1, entity2, score)
# Filter by confidence threshold
if candidate.confidence >= self.confidence_threshold:
# Filter by confidence threshold; type mismatches are excluded
# structurally so no threshold value can admit them.
if (
candidate.confidence >= self.confidence_threshold
and "type_mismatch" not in candidate.reasons
):
candidates.append(candidate)
remaining = total_similarities - (i + 1)
@@ -624,8 +628,12 @@ class DuplicateDetector:
new_entity, existing_entity, similarity.score
)
# Filter by confidence threshold
if candidate.confidence >= self.confidence_threshold:
# Filter by confidence threshold; type mismatches are
# excluded structurally regardless of the threshold.
if (
candidate.confidence >= self.confidence_threshold
and "type_mismatch" not in candidate.reasons
):
candidates.append(candidate)
processed += 1
@@ -723,7 +731,9 @@ class DuplicateDetector:
if key == "name":
return getattr(entity, "text", default)
if key == "type":
return getattr(entity, "label", default)
# Entity objects store the type on .type; extraction entities
# may expose .label. Missing .label never means "no type".
return getattr(entity, "type", default) or getattr(entity, "label", default)
if key == "properties":
# Check metadata for properties
metadata = getattr(entity, "metadata", {})
@@ -757,6 +767,25 @@ class DuplicateDetector:
reasons = []
confidence = similarity_score
# Check entity type mismatch first: two entities with different
# explicit types are not duplicates, whatever their similarity.
entity_type1 = self._get_entity_value(entity1, "type")
entity_type2 = self._get_entity_value(entity2, "type")
if entity_type1 and entity_type2 and entity_type1 != entity_type2:
return DuplicateCandidate(
entity1=entity1,
entity2=entity2,
similarity_score=similarity_score,
confidence=0.0,
reasons=["type_mismatch"],
metadata={
"name_match": False,
"common_properties": 0,
"type_match": False,
"type_mismatch": True,
},
)
# Check for exact name match (strong indicator)
name1 = str(self._get_entity_value(entity1, "name", "")).lower().strip()
name2 = str(self._get_entity_value(entity2, "name", "")).lower().strip()
@@ -779,9 +808,8 @@ class DuplicateDetector:
# Boost confidence for each matching property
confidence += 0.05 * prop_matches
# Check entity type match
entity_type1 = self._get_entity_value(entity1, "type")
entity_type2 = self._get_entity_value(entity2, "type")
# Check entity type match (only boosts when types are equal; mismatch
# is handled above)
if entity_type1 and entity_type2 and entity_type1 == entity_type2:
reasons.append("same_type")
confidence += 0.05
semantica/deduplication/methods.py
+29 -31
View File
@@ -147,9 +147,12 @@ def calculate_similarity(
>>> result = calculate_similarity(entity1, entity2, method="levenshtein")
>>> print(f"Similarity: {result.score:.2f}")
"""
# Check for custom method in registry
# Check for custom method in registry, skip self-referential wrappers.
# _multi_factor_similarity is registered under "multi_factor" and calls back
# into calculate_similarity(method="multi_factor"), creating indirect
# infinite recursion. The identity guard short-circuits that loop.
custom_method = method_registry.get("similarity", method)
if custom_method:
if custom_method and custom_method is not calculate_similarity:
return custom_method(entity1, entity2, **kwargs)
# Use default SimilarityCalculator
@@ -235,9 +238,11 @@ def detect_duplicates(
>>> duplicates = detect_duplicates(entities, method="pairwise", similarity_threshold=0.8)
>>> print(f"Found {len(duplicates)} duplicate candidates")
"""
# Check for custom method in registry
# Check for custom method in registry, skip self-referential wrappers.
# _pairwise_detection is registered under "pairwise" and calls back into
# detect_duplicates(method="pairwise"), creating indirect infinite recursion.
custom_method = method_registry.get("detection", method)
if custom_method:
if custom_method and custom_method is not detect_duplicates:
return custom_method(
entities, similarity_threshold=similarity_threshold, **kwargs
)
@@ -282,9 +287,10 @@ def dedup_triplets(
List of duplicate relationship piars (rel1, rel2).
"""
# Check for custom method in registry (but not ourself)
# Check for custom method in registry (but not ourself — identity guard
# consistent with the other dispatch functions in this module).
custom_method = method_registry.get("detection", "triplets")
if custom_method and custom_method.__name__ != "dedup_triplets":
if custom_method and custom_method is not dedup_triplets:
return custom_method(relationships, mode=mode, threshold=threshold, **kwargs)
detector = DuplicateDetector(**kwargs)
@@ -328,9 +334,11 @@ def merge_entities(
>>> operations = merge_entities(duplicate_entities, method="keep_most_complete")
>>> print(f"Performed {len(operations)} merge operations")
"""
# Check for custom method in registry
# Check for custom method in registry, skip self-referential registration.
# merge_entities is now registered directly under its default method name;
# the identity guard prevents a direct recursion loop.
custom_method = method_registry.get("merging", method)
if custom_method:
if custom_method and custom_method is not merge_entities:
return custom_method(
entities, preserve_provenance=preserve_provenance, **kwargs
)
@@ -374,9 +382,12 @@ def build_clusters(
>>> result = build_clusters(entities, method="graph_based", similarity_threshold=0.8)
>>> print(f"Found {len(result.clusters)} clusters")
"""
# Check for custom method in registry
# Check for custom method in registry, skip self-referential wrappers.
# _graph_based_clustering is registered under "graph_based" and calls back
# into build_clusters(method="graph_based"), creating indirect infinite
# recursion.
custom_method = method_registry.get("clustering", method)
if custom_method:
if custom_method and custom_method is not build_clusters:
return custom_method(
entities, similarity_threshold=similarity_threshold, **kwargs
)
@@ -546,25 +557,12 @@ def list_available_methods(task: Optional[str] = None) -> Dict[str, List[str]]:
return result
# Register default methods with registry
def _multi_factor_similarity(e1, e2, **kw):
return calculate_similarity(e1, e2, method="multi_factor", **kw)
def _pairwise_detection(entities, **kw):
return detect_duplicates(entities, method="pairwise", **kw)
def _keep_most_complete_merging(entities, **kw):
return merge_entities(entities, method="keep_most_complete", **kw)
def _graph_based_clustering(entities, **kw):
return build_clusters(entities, method="graph_based", **kw)
method_registry.register("similarity", "multi_factor", _multi_factor_similarity)
method_registry.register("detection", "pairwise", _pairwise_detection)
method_registry.register("merging", "keep_most_complete", _keep_most_complete_merging)
method_registry.register("clustering", "graph_based", _graph_based_clustering)
# Register default methods with registry.
# The public dispatch functions are registered directly so the identity guard
# in each function short-circuits the self-reference rather than going through
# an intermediate wrapper that re-enters the same dispatch path.
method_registry.register("similarity", "multi_factor", calculate_similarity)
method_registry.register("detection", "pairwise", detect_duplicates)
method_registry.register("merging", "keep_most_complete", merge_entities)
method_registry.register("clustering", "graph_based", build_clusters)
method_registry.register("detection", "triplets", dedup_triplets)
semantica/embeddings/embeddings_provenance.py
+9
View File
@@ -69,6 +69,15 @@ class EmbeddingGeneratorWithProvenance:
return embeddings
def __getattr__(self, name):
# __getattr__ only runs when normal lookup fails. Accessing
# self._generator by attribute syntax HERE would re-enter
# __getattr__ for ever when _generator itself is missing — the shape
# pickle/copy protocol probes hit when __init__ never completed
# (#994's RecursionError family). Fail fast on private probes.
if name.startswith("_"):
raise AttributeError(
f"{type(self).__name__!r} object has no attribute {name!r}"
)
return getattr(self._generator, name)
semantica/embeddings/methods.py
+33 -32
View File
@@ -80,6 +80,7 @@ import numpy as np
from ..utils.exceptions import ConfigurationError, ProcessingError
from ..utils.logging import get_logger
from ..utils.custom_methods import CUSTOM_METHOD_FELL_BACK, call_custom_method
from .config import embeddings_config
from .embedding_generator import EmbeddingGenerator
from .pooling_strategies import PoolingStrategyFactory
@@ -116,15 +117,15 @@ def generate_embeddings(
>>> emb = generate_embeddings("Hello world", method="default")
>>> embs = generate_embeddings(["text1", "text2"], method="text")
"""
# Check for custom method in registry
# Check for custom method in registry, skip self-reference
custom_method = method_registry.get("generation", method)
if custom_method:
try:
return custom_method(data, data_type=data_type, **kwargs)
except Exception as e:
logger.warning(
f"Custom method {method} failed: {e}, falling back to default"
)
if custom_method and custom_method is not generate_embeddings:
fallback = kwargs.pop("fallback_on_custom_error", False)
result = call_custom_method(
logger, method, custom_method, data, data_type=data_type, fallback_on_custom_error=fallback, **kwargs
)
if result is not CUSTOM_METHOD_FELL_BACK:
return result
try:
if method == "default":
@@ -164,15 +165,15 @@ def embed_text(
>>> emb = embed_text("Hello world", method="sentence_transformers")
>>> embs = embed_text(["text1", "text2"], method="sentence_transformers")
"""
# Check for custom method in registry
# Check for custom method in registry, skip self-reference
custom_method = method_registry.get("text", method)
if custom_method:
try:
return custom_method(text, **kwargs)
except Exception as e:
logger.warning(
f"Custom method {method} failed: {e}, falling back to default"
)
if custom_method and custom_method is not embed_text:
fallback = kwargs.pop("fallback_on_custom_error", False)
result = call_custom_method(
logger, method, custom_method, text, fallback_on_custom_error=fallback, **kwargs
)
if result is not CUSTOM_METHOD_FELL_BACK:
return result
try:
# Get config
@@ -224,15 +225,15 @@ def calculate_similarity(
>>> similarity = calculate_similarity(emb1, emb2, method="cosine")
>>> print(f"Similarity: {similarity:.3f}")
"""
# Check for custom method in registry
# Check for custom method in registry, skip self-reference
custom_method = method_registry.get("similarity", method)
if custom_method:
try:
return custom_method(embedding1, embedding2, **kwargs)
except Exception as e:
logger.warning(
f"Custom method {method} failed: {e}, falling back to default"
)
if custom_method and custom_method is not calculate_similarity:
fallback = kwargs.pop("fallback_on_custom_error", False)
result = call_custom_method(
logger, method, custom_method, embedding1, embedding2, fallback_on_custom_error=fallback, **kwargs
)
if result is not CUSTOM_METHOD_FELL_BACK:
return result
try:
generator = EmbeddingGenerator(**kwargs)
@@ -271,15 +272,15 @@ def pool_embeddings(
>>> pooled = pool_embeddings(embeddings, method="mean")
>>> attention_pooled = pool_embeddings(embeddings, method="attention")
"""
# Check for custom method in registry
# Check for custom method in registry, skip self-reference
custom_method = method_registry.get("pooling", method)
if custom_method:
try:
return custom_method(embeddings, **kwargs)
except Exception as e:
logger.warning(
f"Custom method {method} failed: {e}, falling back to default"
)
if custom_method and custom_method is not pool_embeddings:
fallback = kwargs.pop("fallback_on_custom_error", False)
result = call_custom_method(
logger, method, custom_method, embeddings, fallback_on_custom_error=fallback, **kwargs
)
if result is not CUSTOM_METHOD_FELL_BACK:
return result
try:
strategy = PoolingStrategyFactory.create(method, **kwargs)

Some files were not shown because too many files have changed in this diff Show More