Compare commits

...
Author SHA1 Message Date
Nilay Mallik 7057387775 fix(context): forget(days_old=) deleted recent memories instead of stale ones (#1598)
* fix(context): forget(days_old=) deleted recent memories instead of stale ones

forget() filtered on start_date (a lower bound), so clear_memory purged
memories newer than the cutoff and kept older ones — the opposite of the
documented contract. Filter on end_date instead, with an aware-UTC cutoff
matching the store's aware stamps. clear() delegates to forget() and is
covered by the same fix.

* fix(context): reject negative days_old in forget()

A negative age built a future end_date cutoff, matching and deleting
every normally-timestamped memory (Qodo review on #1598).
2026-09-12 19:41:44 +05:30
Sameer Kadam 40478426fe fix: resolve 23 pre-existing test failures across pipeline, Snowflake, Salesforce, and CLI (#1593)
* fix: resolve 23 pre-existing test failures across pipeline, Snowflake, Salesforce, and CLI modules

Fixes 6 files, reduces the failing test count from 73 to 50.
The remaining 50 failures are all environment-bound (missing optional
dependencies: fastapi, pyoxigraph, defusedxml; real network calls;
FAISS SIMD hardware constraint; flaky concurrency test; stale top-level
test copies whose structured counterparts pass cleanly).

## Changes

### semantica/pipeline/pipeline_builder.py
- add_step() return type annotation corrected to PipelineStep (was already
  returning PipelineStep; annotation said PipelineBuilder by mistake).
- build() gains an optional validate=False parameter so test code can
  construct intentionally invalid pipelines to test the validator
  independently without triggering the built-in pre-build validation.
- build_pipeline() internal step-variable usage aligned with the correct
  PipelineStep return of add_step().

### tests/test_pipeline_orchestration.py
- All builder.add_step(...).build() fluent-chain calls separated into
  two statements because add_step() returns PipelineStep, not the builder.
- Validator tests (missing dependency, circular dependency) now pass
  validate=False to build() so they can construct invalid pipelines for
  external validation testing.
- test_execution_engine_failure: assertion changed from
  result.metrics['steps_failed'] to result.errors because the execution
  engine returns a metrics-less ExecutionResult on the exception path.
- test_pipeline_validator_missing_dependency: assertion relaxed to
  case-insensitive 'missing' substring match, aligning with the actual
  error message ('Missing dependency ...').

### tests/test_snowflake_ingestor.py
- test_ingest_table_with_limit: SnowflakeIngestor uses DB-API bind
  parameters (%s) for LIMIT, so the query string contains 'LIMIT %s'
  not 'LIMIT 100'. Assertion updated to check for the LIMIT keyword
  and verify 100 appears in the bound params tuple.

### tests/test_cli_commands.py
- test_migrate_refuses_unsupported_backend_pair: Rich wraps the error
  message across terminal lines, breaking a single 'faiss, pgvector,
  sqlite' substring match. Assertion updated to check each backend name
  individually, which is robust to line-wrapping.

### tests/test_issue_1513_slim_core.py
- test_salesforce_ingestor_package_import_missing_hint: the test patched
  SALESFORCE_AVAILABLE=False and removed the SalesforceIngestor cache
  entry but did not clean up, leaving _SalesforceAuthenticationFailed=None
  in the module for later tests. Added cleanup: restore the cached symbol
  to None so subsequent __getattr__ calls re-evaluate it fresh.

### tests/test_salesforce_ingestor.py
- _mock_simple_salesforce_if_needed autouse fixture: when simple-salesforce
  is not installed the module-level sentinels (_SalesforceAuthenticationFailed,
  _SalesforceError, SALESFORCE_AVAILABLE) are set to None/False at import
  time. The fixture now also patches those sentinels with real callable
  stub classes inside the sys.modules mock context, so tests that import
  _SalesforceAuthenticationFailed directly from the module get a usable
  class instead of None.

* fix(tests): resolve autouse fixture issues in Salesforce test suite

Two code-review findings addressed:

Finding 1 — fixture forced SALESFORCE_AVAILABLE=True for missing-dep tests
The _mock_simple_salesforce_if_needed autouse fixture patched
SALESFORCE_AVAILABLE=True for every test, including
TestImportBehaviourWithoutLib, which is specifically designed to verify
the production missing-dependency guard. Those tests were no longer
exercising the real guard — they were seeing the mocked available state.

Fix: the fixture now inspects the requesting test's class name via the
pytest 'request' fixture. If the test belongs to
TestImportBehaviourWithoutLib it skips all mocking entirely, letting the
genuinely unavailable module state stand. The import-behaviour tests now
exercise real production code.

Finding 2 — fixture did not clear package-level cached lazy exports
The lazy loader in semantica.ingest.__getattr__ permanently caches
resolved names into the package's globals() dict. Any test that accessed
SalesforceIngestor/SalesforceConnector/SalesforceData while the fixture's
stub was active would cache those mocked classes. Fixture teardown
restored SALESFORCE_AVAILABLE and the module sentinels via patch.object
but left the cached exports in the package globals — causing later
missing-dependency checks to find the stale mocked class instead of
re-running the real guard.

Fix: the fixture snapshots the three Salesforce export names in
semantica.ingest.__dict__ before yielding and removes any entries that
were added during the test at teardown, so subsequent availability checks
always re-run __getattr__ from a clean state.

Additionally, test_semantica_ingest_imports_cleanly_without_lib was
corrected: it previously asserted hasattr(pkg, 'SalesforceIngestor'),
but Python's hasattr() only catches AttributeError — the real guard
raises ImportError, which propagates and crashes the assertion. The test
is updated to verify __all__ membership (always present) and that
SalesforceData (which has no SDK guard) remains importable — which is
the true contract for graceful degradation.

* fix: apply reviewer-requested tweaks before merge

Three targeted changes requested by the code reviewer:

1. Wrap yield in try/finally so fixture teardown always runs
   tests/test_salesforce_ingestor.py: the dict pops that remove
   Salesforce lazy-export globals were after the yield, so they were
   skipped whenever a test raised. Moved into a try/finally block
   inside the innermost with-context so they fire on both normal exit
   and test failure.

   tests/test_issue_1513_slim_core.py: the final
   ingest_mod.__dict__.pop() had the same issue — it was after the
   with blocks so an assertion failure would skip it. Wrapped in
   try/finally.

2. Assert the missing-dep guard fires in test_semantica_ingest_imports_cleanly_without_lib
   Added pytest.raises(ImportError, match=r'semantica[db-salesforce]')
   to verify that accessing pkg.SalesforceIngestor when the library is
   absent actually fires the production ImportError with the correct
   install hint, rather than silently succeeding.

3. Move progress_tracker.update_tracking('Validating...') inside if validate:
   semantica/pipeline/pipeline_builder.py: the tracker message
   'Validating pipeline structure...' was emitted even when validate=False,
   which would log a misleading message for callers that explicitly opt
   out of validation. Moved inside the if validate: block.
2026-09-12 14:34:16 +05:00
05aec1975d feat: add Amazon Redshift ingestor (#1587)
* feat: add Redshift ingestor

* fix(redshift): resolve 5 code-review issues in RedshiftIngestor

Fix #1 — restore autocommit on caller-owned connections
All four methods (ingest_table, ingest_query, get_table_schema,
list_tables) now capture the connection's prior autocommit value before
enabling it for read-only ingestion and restore it in the finally block
when the connection was already open.  Transient connections are still
closed normally.

Fix #2 — batch fetching: process rows per-batch, not after accumulating
ingest_query's batch path now converts and extends data[] directly
inside the fetch loop so raw tuples from each batch are eligible for
GC before the next fetch.  The documentation is updated to accurately
state that batch_size controls driver round-trip size, not total memory.

Fix #3 — schema env-var now honoured
RedshiftIngestor.__init__ changes schema default from 'public' to None
so the existing os.getenv('REDSHIFT_SCHEMA', 'public') fallback in the
body is reached when no explicit argument is supplied.

Fix #4 — db-redshift included in db-all aggregate
pyproject.toml db-all now references db-redshift alongside the other
database extras.

Fix #5 — duplicate column labels no longer silently drop values
_disambiguate_columns() appends _1, _2, ... suffixes to repeated
cursor labels before dict(zip) so every value is retained.  A new
_make_row_dicts() helper encapsulates the zip+disambiguate pattern
used by both _fetch_all and ingest_query.

Tests: 22 new targeted tests added covering all five fixes.

---------

Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
Co-authored-by: Kaif <98801504+KaifAhmad1@users.noreply.github.com>
2026-09-11 18:43:14 +05:30
csy20andSameer Kadam bea9b2aa48 docs: rewrite remaining bare relative links that 404 on the live site (#1580)
#1407 skipped 89 Next Steps links whose targets exist in both guides/
and reference/. Rewrite them as /guides/<name> or /reference/<name>
from the source section so Mintlify+GitHub Pages trailing-slash
redirects no longer 404.

Also retarget three in-page heading anchors to Mintlify's rendered
slugs (data-&-features, reasoner-forward/backward-chaining,
litellm-100+-providers).

Closes #1566

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
2026-09-11 17:40:52 +05:30
Kaif 6148cb6f8c Merge pull request #1563 from gyroscope1110/fix/plugin-skills-stale-api-references
fix(plugins): repair broken API references in five skills
2026-09-11 17:06:18 +05:30
Kaif 202acc779e Merge branch 'main' into fix/plugin-skills-stale-api-references 2026-09-11 16:55:06 +05:30
1928f80104 fix(plugins): resolve Qodo-flagged bugs in rewritten skill examples
Addresses all 7 findings from the automated review on this PR:
- ontology: OntologyEngine() has no store configured, so
  list_concepts/list_vocabularies always raise ProcessingError; construct
  it with a TripletStore
- ontology: ValidationResult's field is `valid`, not `is_valid`
- change: state_at()["nodes"] is a list of dicts, so set(...) on it raises
  TypeError: unhashable type: 'dict' — diff by node id instead
- provenance/change: storage_path is passed to sqlite3.connect() unexpanded,
  so a literal "~/.semantica/prov.db" fails to open — expanduser + mkdir
- change/query: load_from_file() checks the literal path, so "~/..." never
  resolves and the graph loads empty — expanduser before calling
- query: QueryEngine.execute_query requires an object exposing
  execute_sparql(); the TripletStore wrapper doesn't expose that, only the
  raw backend (e.g. OxigraphStore) does
- query: the Cypher example constructed Neo4jStore but never called
  execute_query()

Co-Authored-By: gyro <zhuffwct@gmail.com>
Co-Authored-By: KaifAhmad1 <mohammadk78600@gmail.com>
2026-09-11 16:45:17 +05:30
dependabot[bot] 888c2c4e34 deps(deps): bump litellm from 1.99.0 to 1.100.0 (#1588)
Bumps [litellm](https://github.com/BerriAI/litellm) from 1.99.0 to 1.100.0.
- [Release notes](https://github.com/BerriAI/litellm/releases)
- [Commits](https://github.com/BerriAI/litellm/compare/v1.99.0...v1.100.0)

---
updated-dependencies:
- dependency-name: litellm
  dependency-version: 1.100.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-11 15:37:15 +05:30
Wu ShuwenandSameer Kadam f36a264571 fix(docs-check): preserve Mintlify output on Windows (#1579)
* fix(docs-check): preserve Mintlify output on Windows

* test(docs-check): add Windows regression coverage

* fix(docs-check): preserve Mintlify output on Windows

---------

Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
2026-09-11 10:09:40 +05:30
Sameer Kadam 76515280a4 docs: add multilingual README links (#1567)
Add links for multilingual README using zdoc service
2026-09-11 00:40:52 +05:00
Zheng Feng 9905e4321d fix(mcp): repair extraction handlers and pipeline serialization (#1535)
Fix runtime crashes and serialization defects across the packaged MCP extraction
handlers in semantica_mcp/mcp/tools/extraction.py.

* fix(mcp): use public named entity extraction API
  - Replace non-existent NamedEntityRecognizer.extract() with extract_entities()
  - Preserve original input text to prevent entity character offset coordinate drift
  - Fixes #1533

* fix(mcp): correct extraction pipeline serialization
  - Map Relation fields (subject, predicate, object) to MCP keys (source, type, target)
  - Retain coreference chains in result["coreferences"] instead of passing to text extractors
  - Fixes #1534

* fix(mcp): serialize event fields correctly
  - Map Event dataclass fields (event_type, text) to MCP keys (type, trigger)
  - Add regression tests covering entity offsets, relations, coreferences, and events
2026-09-10 20:43:13 +05:00
Mohd Kaif 161748f0b0 Merge pull request #1564 from Evanwang-3/codex/fix-faiss-pq-training
fix(vector-store): train PQ indexes in FAISSIndexBuilder
2026-09-10 16:22:53 +05:30
Mohd Kaif 731814d2b3 Merge branch 'main' into codex/fix-faiss-pq-training 2026-09-10 16:06:56 +05:30
6afe90e4fb fix(context): stamp AgentMemory in aware UTC, not mixed naive conventions (#1073)
* fix(context): stamp AgentMemory in aware UTC, not mixed naive conventions

MemoryItem timestamps had two producers with different naive conventions:
store() defaulted to datetime.now() (naive LOCAL) while from_dict fell back
to datetime.utcnow() (naive UTC). _timestamp_comparison_key interprets every
naive stamp as LOCAL time, so on any host off UTC the two producers disagree
by the host's offset — a UTC+8 host pushed freshly stored memories 8 hours
outside every start_date/end_date window, and cleanup_old_memories aged them
wrong by the same amount.

All three producers now stamp datetime.now(timezone.utc):

- store() default and the from_dict fallbacks stop introducing naive values;
- the cleanup cutoff compares against an aware instant instead of a naive
  local one;
- legacy naive stamps keep their documented local-time meaning through the
  comparison key, which is deliberately unchanged and now pinned by test.

Seven regression tests cover the producer contract, the naive-as-local
comparison semantics (so the key cannot silently flip to naive-as-UTC), the
isoformat round-trip, and end-to-end date-range filtering with UTC
boundaries on hosts in any timezone.

* test(context): rename add() references to store() in timestamp test

---------

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
2026-09-10 15:16:35 +05:30
xudong 33f8719c80 fix(vector-store): train PQ indexes in FAISSIndexBuilder 2026-09-10 16:45:58 +08:00
Mohd Kaif 3bb0d28112 docs(readme): note LLM use is optional and vendor-neutral (#1561)
Make explicit in the opening pitch that where Semantica does use an
LLM, it's optional and works with any major provider via
semantica.llms, not tied to one vendor.
2026-09-10 13:59:45 +05:30
1914dd508c fix: replace bare except with except Exception in 3 sites (#1069)
* fix: bare except -> except Exception in docling_parser

* fix: bare except -> except Exception in methods.py

* Address review: bound the spaCy model cache, serialize calls per model

Two findings from the Qodo review:

- _spacy_model_cache retained every successfully loaded model for the
  process lifetime — each spaCy Language costs hundreds of MB, so a service
  varying the model option grew without bound. Now an LRU capped at
  MAX_SPACY_MODELS_CACHED (4: lg/md/sm + one custom), evicting
  least-recently-used on insert.

- The cached Language was handed to every caller while batch extraction
  fans out over a ThreadPoolExecutor; spaCy pipelines are not safe to
  invoke from concurrent threads. Calls now go through per-model locks:
  spacy_pipeline_guard() yields the pipeline under its own lock (different
  models still run in parallel), and run_spacy_text() wraps the
  load/fallback/call chain the entity and relation extractors used to
  open-code. load_spacy_model() keeps its signature but documents that its
  result must only be called under the guard.

Four regression tests: the bound holds, LRU evicts oldest-not-recently-
used, concurrent calls on one model never overlap, and two models can be
inside calls at the same time (the lock is per-model, not global).

* fix: bare except, bounded LRU spaCy cache, per-model call lock (#1069)

Problems fixed
--------------
- Replace all 5 bare 'except:' clauses with 'except Exception:' in
  methods.py (get_nlp_model, extract_relations_similarity fallback) and
  docling_parser.py (3 sites, including 2 missed by the original PR).

- Replace the unbounded dict cache with an OrderedDict LRU bounded at
  MAX_SPACY_MODELS_CACHED=4. Each spaCy Language is 100-700 MB; the old
  design pinned every model name ever used for the lifetime of the process.

- Fix a TOCTOU race in _load_spacy_entry: the lockless fast-path could
  call move_to_end() on an entry that had just been evicted by another
  thread (KeyError). Rewrite as a single critical section: the cache
  lookup, LRU update, model load, insertion, and eviction all happen
  under one lock. spacy.load() inside the lock is acceptable because
  model loads are rare (at most MAX_SPACY_MODELS_CACHED per process).

- Add a per-model threading.Lock stored in each cache entry. All calls
  to nlp(text) go through spacy_pipeline_guard(), which acquires the
  entry's lock before yielding the Language. This serializes concurrent
  calls to the same model (spaCy pipelines are not thread-safe) while
  allowing different models to run in parallel.

- Add run_spacy_text() helper to DRY up load-with-fallback-and-logging
  used in extract_entities_ml, extract_relations_dependency, and
  extract_relations_similarity.

Behavior changes vs main
------------------------
- Pipeline errors during nlp(text) (non-OSError) now log a warning and
  fall back to pattern extraction, instead of propagating to the caller.
  This is strictly safer for a service context.
- extract_relations_similarity drops an unreachable bare-except path
  that tried en_core_web_sm when is_package() returned False for all
  three model names. The is_package() guard is unchanged.
- Log messages for model-not-found events now carry a function-scoped
  label (spaCy NER, spaCy dependency, spaCy similarity) for easier
  triage in production logs.

Tests
-----
- Keep and improve four existing cache/lock tests.
- Add test_bound_never_exceeded_under_concurrent_load: verifies the
  single-lock design never exceeds the bound under concurrent pressure
  (would have caught the old TOCTOU).
- Add test_spacy_not_installed_raises_import_error: verifies that
  spacy=None produces a clear ImportError, not an AttributeError.

---------

Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
2026-09-10 13:53:55 +05:30
gyroandClaude Opus 5 e883facedb fix(plugins): repair CP1252 bytes in the decision skill
Two em dashes were stored as the raw CP1252 byte 0x97 rather than UTF-8, so
the file is not valid UTF-8. Strict UTF-8 readers fail on it, and lenient ones
render the frontmatter description as "Semantica <?> record".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:19:54 +08:00
gyroandClaude Opus 5 b4a0bc4063 fix(plugins): correct the extraction cache import in extract and validate
Both skills instruct the agent to clear the result cache via
`from semantica.semantic_extract.cache import _result_cache`, but the module
exports the `ExtractionCache` singleton as `extraction_cache`. The private
name never existed, so the first step of both skills raises ImportError.

`extraction_cache.clear()` is the equivalent call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:19:54 +08:00
gyroandClaude Opus 5 cbd33fba67 fix(plugins): rewrite five skills against the real v0.7.0 API
The ontology, policy, provenance, change and query skills documented classes
and methods that do not exist in the package. Every import in them fails, so
following the skill produces ImportError or AttributeError immediately:

| Documented | Reality in 0.7.0 |
| --- | --- |
| `semantica.policy.PolicyEngine` | no `semantica.policy` module; `PolicyEngine` is in `semantica.context` |
| `semantica.query.QueryEngine` | no `semantica.query` module; `QueryEngine` is in `semantica.triplet_store` |
| `semantica.ontology.OntologyManager` | no such class; use `OntologyEngine` / `OntologyValidator` |
| `semantica.provenance.ProvenanceTracer` | no such class; use `ProvenanceManager` |
| `semantica.provenance.change_tracker.ChangeTracker` | no such module; ontology versioning lives in `semantica.change_management` |

The method names were wrong too, so a path-only fix was not possible:
`.check()`, `.list_rules()`, `.trace_node()`, `.get_audit_log()`,
`.compute_diff()`, `.get_node_history()`, `.query_sparql()`, `.query_cypher()`
and `.search()` do not exist on any of the real classes.

Each skill is rewritten against signatures verified by introspection on an
installed 0.7.0. Two notes on scope:

- policy now leads with `ContextGraph.check_decision_rules()` /
  `enforce_decision_policy()`, which need no graph store, and keeps
  `context.PolicyEngine` as the managed-policy path.
- change previously conflated graph-state-over-time with ontology versioning.
  These are separate mechanisms in 0.7.0, so the skill now documents
  `ContextGraph.state_at()` and `change_management.VersionManager` separately.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:19:54 +08:00
Mohd Kaif 375ffc8522 docs(readme): remove unshipped 0.7.0 references and duplicate content (#1559)
The v0.7.0 "What's New" section and the note pinning to <0.7.0 named a
version that hasn't been tagged or published to PyPI yet (latest release
remains v0.6.8); reword the installation note without asserting an
unshipped version boundary. Also drop the "Built for High-Stakes
Domains" section, which duplicated the top-of-file positioning, the
Regulated enterprises bullet, and the explainability [!NOTE] callout.
2026-09-10 13:46:25 +05:30
Mohd Kaif 365d473743 Merge pull request #1558 from semantica-agi/readme-positioning-refresh
docs(readme): tighten intro copy and enterprise data platform framing
2026-09-10 13:29:24 +05:30
KaifAhmad1andClaude Sonnet 5 e40ba9b254 docs(readme): tighten intro copy and clarify enterprise data platform support
Break the dense opening paragraph into scannable lines, lead with the
semantic/context layer framing, name Databricks/Snowflake/SAP together
under "enterprise data teams" instead of a two-platform subset, and
switch the explainability caveat to GitHub's native [!NOTE] alert.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-10 13:23:57 +05:30
Wei TaoandClaude Code 9d93a80840 fix(explorer): label the graph view control "Focus" (#1552)
* fix(explorer): label the graph view control "Focus"

The control read "Focused" before it was ever activated, which describes a
state the selection had already reached rather than the action available.
The view mode value, tooltip, enablement and active styling are unchanged.

The legend e2e drove this button by its accessible name, so the selector
moves with the label; it now also asserts the visible text, the
selection-dependent enablement and the active state.

Closes #1551

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(explorer): name the control in the grouped-selection hint, and trim the test

"Activate Focused mode" instructed the reader to press a control that no
longer carries that name. The surrounding strings describe the mode itself,
which is still called focused, so they stay.

Drop the label and active-state assertions from the colour-legend test: the
getByRole locator already fails when the accessible name is wrong, and the
rest belonged to the control's contract rather than to the legend's.

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-10 09:42:17 +05:30
a9f1b29292 fix(explorer): resolve ontology ownership on the backend for entity deep links (#1439)
* fix(explorer): resolve ontology ownership on the backend

Which registered ontology owns an entity was decided twice: the backend
applies nested-namespace boundaries, while the Ontology Editor did a bare
prefix match. The two had already drifted, so a deep link to an entity in an
unregistered nested namespace selected the parent ontology whose /graph
response excludes that entity, and the selection silently failed.

/api/ontology/entity now returns owning_ontology, resolved with the same rule
the graph endpoint filters by, and the editor prefers it. The frontend
namespace guess stays as the fallback for a missing verdict, documented as
non-authoritative.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(explorer): keep the backend's no-owner verdict authoritative

Review follow-up: loadOntologyEntityOwner collapsed the backend's
explicit owning_ontology: null into undefined, re-activating the
namespace prefix guess for exactly the unregistered-nested-namespace
case this PR exists to fix. The owner verdict is now three-state
(owner / authoritative none / unavailable) and resolveEditorOntology
in the model suppresses inference on an authoritative none; only an
unavailable verdict may fall back. Model tests pin all three states.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(explorer): trust the backend to send owning_ontology

The Explorer bundle ships in the same wheel as the route that emits this
field, so the legacy-response branch could never run. Dropping it lets
the type say what the wire actually carries, leaving undefined to mean
only what it should: the request failed.

Note why the endpoint derives its ontology-URI set inline rather than
calling _known_ontology_uris, so the next reader does not consolidate a
graph scan back in.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(explorer): stop the registry default overriding a no-owner verdict

resolveEditorOntology returned string | undefined, so the caller wrote
`resolved || entries[0]?.uri` and an authoritative "nothing owns this
entity" fell straight through to an arbitrary registry entry. When that
entry happened to be the parent, the deep link opened the parent whose
graph excludes the entity — the bug the verdict exists to prevent. Only
the ordering of the registry in the earlier test hid it.

It now returns a union: unowned and unresolved both mean "no ontology to
open" but the editor treats them oppositely, so collapsing them with ||
is a type error rather than a silent regression. An unowned entity is
reported on the canvas instead of quietly opening the wrong ontology.

Also:

- /entity resolves ownership through _known_ontology_uris, the same
  helper /graph uses, instead of deriving it from a get_nodes scan capped
  at 999,999. Past that cap the set was silently truncated and the two
  endpoints could disagree about who owns a node.
- _resolve_owning_ontology does one pass over the candidates rather than
  one pass per candidate, each rescanning the whole set: 516us -> 9us at
  50 ontologies, 125ms -> 138us at 800, same answers throughout. A test
  pins it against _node_belongs_to_ontology so the hand-rolled version
  cannot drift from the membership rule it has to mirror.
- An explicit scheme_uri is honoured even when the registry does not list
  it, on both sides. Discarding it and guessing by namespace answered a
  question nobody asked; an unregistered owner now surfaces as an
  explicit error from /graph instead.
- A missing owning_ontology field reads as "no verdict", not as the
  authoritative "nothing owns this". That claim now suppresses selection
  outright, so it must not be inferred from an absent field.

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
2026-09-09 23:39:39 +05:30
Wei TaoandClaude Fable 5 0338f90bca refactor(explorer): own Ontology Hub URL state in one module (#1440)
The deep-link protocol introduced in #1278 lived as bare "ontologyTab" /
"ontologyEntity" literals in five places across three files, each with its own
URLSearchParams plumbing and try/catch. Nothing tied the pieces together: in
particular the rule that a selection written under one ontology must be cleared
when the active ontology changes — otherwise a reload resolves the stale entity
and jumps back to the old ontology — was a comment at one call site with no
mechanism behind it.

ontologyUrlState.ts now owns the parameter names as private constants and
exposes the protocol as intent-named operations, with the write/clear pairing
documented where both halves live. Parsing and serialization are pure functions
over a search string, so they are covered by tests without a DOM; the window
and history.replaceState interaction stays in thin shells.

Absent parameters still read as undefined while blank ones read as empty
strings, which preserves the differing presence checks the workspace shell and
the tab selector each relied on.

One behavior change, inherited from all five original call sites: writing the
query string dropped any URL fragment, because replaceState with a bare "?..."
replaces the whole tail. updateSearch now carries window.location.hash across,
which fixes it for every writer at once — this module is the only place that
knows how the URL is written, so it is the only place the fix belongs.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-09 21:50:13 +05:30
Sameer KadamandSameer Kadam def18cd552 fix(explorer): remove stale 2030 temporal bound (#1549)
Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
2026-09-09 17:41:11 +05:30
Wei TaoandSameer Kadam 6b55cf1101 fix(explorer): end the temporal scrubber at now instead of inventing 2030 (#1541)
TimelinePanel fell back to a hardcoded 2030-01-01 whenever
/api/temporal/bounds reported max: null, then placed the playhead at the
midpoint of that fabricated window. session.get_temporal_bounds leaves max
open for any graph whose nodes carry valid_from instants and no
valid_until, so this was the normal response shape rather than bad data:
the header advertised a range no data supported and the workspace's first
/api/temporal/snapshot request asked about a time years ahead of the
present.

The fallback is now a `now` captured once per mount, and defaultTime is
that same `now` clamped into the range, so the initial snapshot describes
the current state. Three settings tuned for the fictional ~60-year window
follow from it: zoomMin drops from a year to a day, the timeAxis/format
pinning to 5-year ticks is removed so vis-timeline picks a granularity for
the real span, and the fixed 6-month play step becomes span/60 with a
one-day floor, which keeps a play-through at roughly 60 frames whether the
graph covers months or a decade.

The bound and step arithmetic moves into temporalScrubberBounds.ts so it
can be asserted directly, alongside the existing temporalLifecycle
predicate tests.

Closes #1536

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
2026-09-09 17:26:23 +05:30
bd41a5a24f fix(vector-store): prevent in-memory vector ID reuse (#1546)
* fix(vector-store): prevent in-memory vector id reuse

* fix(vector-store): synchronize in-memory mutations

* fix(vector-store): clarify no-silent-overwrite guarantee for in-memory ID generation

The while-loop in store_vectors() already prevents any generated candidate
from landing on a live key: it breaks only when the candidate is absent
from both pre_existing (the live store at lock-entry) and within_batch
(IDs chosen earlier in the same call).

This commit:
- Renames 'existing' to 'pre_existing' and introduces 'within_batch' to
  make the two-level de-dupe explicit and self-documenting.
- Tightens the break condition to 'not in pre_existing and not in
  within_batch' so within-batch duplicates are also guarded.
- Adds test_auto_generated_id_never_silently_overwrites_live_vector:
  stores 5, deletes 3, stores 2 more and asserts none of the new IDs
  land on a surviving vec_N.
- Adds test_collision_detection_no_silent_overwrite_even_with_corrupted_counter:
  rewinds _next_id to 0 with live vectors present and proves the loop
  finds a free slot without overwriting either existing vector or its
  metadata — the no-silent-overwrite invariant holds even under counter
  corruption.

The skip-over-occupied-key behaviour for caller-inserted vec_N IDs is
intentional and preserved (test_counter_skips_explicit_vec_n_ids), matching
FAISSStore's identical pattern.

Closes reviewer finding: 'Vector collisions do not fail writes'.

* fix(vector-store): protect concurrent in-memory access

---------

Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
Co-authored-by: Zohaib Hassnain <109234410+ZohaibHassan16@users.noreply.github.com>
2026-09-09 17:02:27 +05:30
dependabot[bot] fa8002e554 docker(deps): bump python from 3.13-slim to 3.14-slim (#1547)
Bumps python from 3.13-slim to 3.14-slim.

---
updated-dependencies:
- dependency-name: python
  dependency-version: 3.14-slim
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-09 15:29:53 +05:30
Mohd KaifandClaude Sonnet 5 d4127131ba fix(ci): match pip-audit ignore list against vuln aliases, not just id (#1543)
#1540 added GHSA-4j2p-28q2-5m79 to security-scan.yml's IGNORED_VULN_IDS to
suppress the open, unpatched accelerate<=1.14.0 path traversal advisory
(Semantica doesn't call load_checkpoint_in_model/load_checkpoint_and_dispatch).
That commit's own CI run still failed: pip-audit's OSV-backed report picked
CVE-2026-69112 as the vuln's canonical `id` and demoted the GHSA id to an
alias, but the shell/JS matching only ever compared against `.id`.

List both identifiers and match against `.id` plus `.aliases` (which
pip-audit includes by default for JSON output) in the audit gate, the
"Vulnerability details" printer, and the PR-comment script, so an ignored
advisory is excluded regardless of which alias the report surfaces as
canonical. Verified against the actual failing report from run 34296586683 -
the corrected filter now yields 0 actionable vulnerabilities.

Also add osv-scanner.toml (per GHSA-4j2p-28q2-5m79's Scorecard code-scanning
remediation) so the weekly Scorecard "Vulnerabilities" check stops flagging
the same accepted, unpatched advisory.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 12:46:02 +05:30
Zohaib Hassnain 42bbe51382 fix(ci): ignore upstream accelerate vulnerability in security scan (#1540)
Add GHSA-4j2p-28q2-5m79 to IGNORED_VULN_IDS in security-scan.yml.

accelerate<=1.14.0 has an open path traversal advisory (GHSA-4j2p-28q2-5m79) in load_checkpoint_in_model. 1.14.0 is currently the latest available release on PyPI, so no upstream patch exists yet. Semantica does not expose or call sharded checkpoint loading, making this non-actionable. Re-evaluate once an updated accelerate release is published.
2026-09-09 05:48:45 +05:00
Shubham Srivastava 4b11660f00 fix(cli): report the graph backend actually used, not a nonexistent memory one (#1539)
The status panel and doctor both defaulted to reporting a 'memory' graph
store, but GraphStore._initialize_store_backend only branches on neo4j,
falkordb, neptune and age - GraphStore(backend='memory') raises
ValidationError. doctor short-circuited on that name and returned a
passing check for a store no command could construct.

Both sites now report the backend _get_graph_store actually resolves to,
and doctor probes it instead of skipping. _get_graph_store's default is
unchanged, so no existing setup behaves differently.

Refs #1481
2026-09-09 02:29:25 +05:00
Besokus 44c0c7a76d feat(semantic_extract): schema-guided validation — SchemaValidator + ExtractionSchema (PR 1/3) (#1527)
Add a deterministic, ontology-based schema validator as a sibling to the
confidence-based ExtractionValidator. Both validators implement the same
validate_entities() and validate_relations() interfaces and return the
shared ValidationResult structure, enabling orthogonal composition across
extraction confidence and ontological conformance.

Key Additions & Behaviors:
- ExtractionSchema: Read-only view over domain ontologies (allowed concepts
  and predicates with optional domain/range constraints). Supports loading
  from dictionary representations (generate_ontology() or OntologyData from
  OntologyIngestor) via ExtractionSchema.from_ontology(), and from OWL/Turtle
  files/strings via ExtractionSchema.from_owl().
- SchemaValidator: Verifies entity labels against schema concepts and
  validates relation predicates against defined domain/range constraints.
  Provides filter_by_schema() and filter_relations_by_schema() to extract
  conforming subsets without mutating source data.
- Domain/Range Wildcarding: Treats owl:Thing (and unqualified Thing) as
  unconstrained wildcards so fallback types do not reject valid endpoints.
- Constructor Parity: Folds relation domain/range endpoint types into the
  concepts set across both from_ontology() and from_owl() to ensure logical
  consistency.
- Ingest Pipeline Interop: Automatically unwraps OntologyData-like objects
  via duck-typing on .data.
- Resilient Endpoint Resolution: Guards malformed relations missing subject
  or object endpoints without raising AttributeError, and prefers rdfs:label
  over URI suffixes while supporting both owl:Class and rdfs:Class.

Part of #1510.
2026-09-08 20:02:17 +05:00
Sameer Kadam 1c6296a5ab fix(mcp): repair GraphML and Parquet exports (#1367)
Repair broken GraphML and Parquet export handling in the MCP
`export_graph` tool.

Previously, both formats failed in the MCP handler: GraphML referenced a
non-existent `GraphMLExporter` class and passed unformatted graph objects,
while Parquet passed raw `ContextGraph` instances to `export()`, causing
runtime type errors and producing empty responses.

Key changes:
- Route GraphML export through `GraphExporter(format="graphml")` via
  `export_knowledge_graph()` with automatic `TemporaryDirectory` cleanup.
- Harden GraphML XML generation in `GraphExporter`: declare missing
  `label` and `confidence` keys with `for="all"` scope and use
  `xml.sax.saxutils` (`quoteattr`, `escape`) to properly escape XML
  attributes and node text.
- Route Parquet export through `ParquetExporter.export_knowledge_graph()`,
  returning generated `.parquet` files base64-encoded over the JSON MCP
  transport.
- Gracefully handle missing `pyarrow` dependencies with structured error
  responses.
- Preserve existing JSON, CSV, and RDF export behavior.
- Add comprehensive regression tests in
  `tests/test_mcp_package_export_graphml_parquet.py`.
2026-09-08 18:48:57 +05:00
Nilay Mallik 2383d228c7 fix(explorer): place static causal-distance route before dynamic decision_id route (#1532)
Move the static `GET /api/decisions/causal-distance` endpoint above the
dynamic `GET /api/decisions/{decision_id}` route in the Explorer router.

Starlette evaluates routes sequentially in declaration order. Because
`/{decision_id}` was previously defined first, requests targeting
`/api/decisions/causal-distance` were captured by the parameter route
with `decision_id="causal-distance"`, causing the endpoint to query the
graph for a non-existent decision and return 404.

Changes:
- Reorder `causal_distance` above `get_decision` in
  `semantica/explorer/routes/decisions.py` and add a comment guard
  against future route-ordering regressions.
- Add regression test suite in
  `tests/explorer/test_decisions_causal_distance_route.py` verifying
  proper distance reporting, multi-hop traversal, and that unmatched
  decision IDs still return 404.

Closes #1531
2026-09-08 17:12:05 +05:00
5a0d6ea431 Fix/parse and qdrant vector store (#1508)
* fix(parse): stop infinite recursion in default method dispatch

parse_document and its five sibling dispatchers (parse_web_content,
parse_structured_data, parse_email, parse_code, parse_media) were
registered in the method registry under their own task's "default"
name. Every dispatcher begins with method_registry.get(<task>, method),
so calling e.g. parse_document(file, method="default") found itself in
the registry and re-entered infinitely until RecursionError --
`semantica parse <any file>` crashed before any parsing ran.

Drop the six self-registrations. "default" remains the built-in code
path; users can still register their own "default" (or any other name)
to override it, and the existing "docling" registration is unaffected.

Verified: `semantica parse demo.pdf` now parses successfully (was
RecursionError). No repo code or tests consume
get_parse_method("document", "default"), so removing the entries
changes no behavior besides fixing the crash.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(deps): make resolution satisfiable on Python 3.9 and add parse-pdf extra

Dependency fixes so `uv lock` / `pip install semantica[all]` resolves
across the supported matrix (3.9-3.12):

- requires-python >=3.8 was unsatisfiable (numpy>=2.0.2 needs >=3.9) and
  3.9.0/3.9.1 can never resolve (every cryptography release excludes
  them) -> bump to >=3.9.2 and drop the 3.8 classifier.
- Split recently-raised floors that dropped 3.9 into marker pairs
  (3.9-capped / 3.10-unconstrained), following the pattern already used
  for scikit-learn/requests/etc.: pyarrow extras (>=24 needs 3.10),
  pre-commit 4.6, snowflake-connector 4.6, fastapi 0.129 + starlette 0.53
  (older fastapi caps starlette<0.53).
- Gate docling, litellm (its only 3.9 release pins
  python-dotenv==1.0.1, conflicting with the >=1.2.1 core floor), and
  crewai (no un-yanked 3.9 release) to >=3.10.

Also add a parse-pdf extra: the default PDFParser requires pdfplumber,
but no extra installed it, so `semantica parse file.pdf` failed on a
default install. Follows the parse-docling convention.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(vector-store): make the qdrant backend usable through VectorStore

The qdrant backend could not be used at all through the VectorStore
facade in 0.6.8 - every path raised:

1. store_vectors() only dispatched to backend add/add_vectors methods;
   QdrantStore exposes insert_vectors, so vs.store() raised
   NotImplementedError. Add an insert_vectors branch (uuid-generated
   ids, metadata -> payloads).
2. QdrantStore required an explicit create_collection() before any read
   or write, unlike FAISSStore's automatic index creation. Lazily attach
   the configured collection (config key "collection", default
   "semantica_default") on first insert/search, reusing an existing one.
3. search_points() called client.search(), removed from qdrant-client in
   favor of query_points() - use it when available, fall back otherwise.
4. store() silently dropped plain-string documents (it only extracted
   doc.metadata), so payloads lost the source text; keep them under
   payload "document".

Verified end-to-end against a Qdrant 5 server (docker): embed ->
store -> semantic search returns correctly ranked results whose payloads
carry the original documents.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(vector-store): address Qodo review findings on the qdrant write path

Four findings from the Qodo review of #1508:

1. store_vectors() returned QdrantStore.insert_vectors()' upsert status
   dict although the facade promises the stored vector IDs (decision
   storage indexes the result at position 0). Return the generated or
   caller-supplied IDs after a successful insert instead.
2. insert_vectors() pairs points with zip(vectors, ids), so a shorter
   non-empty id list silently dropped the unpaired vectors while the
   completion message still reported the full batch as inserted. Reject
   the mismatch with ValidationError before any write.
3. _ensure_default_collection() looked up the legacy "collection" config
   key, so the documented collection_name=... option was ignored and
   lazy init always fell back to semantica_default. Prefer
   collection_name, keep "collection" as an alias.
4. The new parse-pdf extra was missing from both aggregate "all"
   bundles, so semantica[all] still shipped without pdfplumber and the
   default PDFParser raised ProcessingError on first use.

Also removes the now-stale strict xfail for qdrant's write dispatch in
test_backend_facade_contract.py — that marker exists precisely to fail
once the wiring lands.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* test(vector-store): migrate qdrant mocks to the query_points API

The qdrant search path now prefers client.query_points() (qdrant-client
removed client.search), but these tests still mocked the legacy call.
A MagicMock exposes query_points too, so the code took the modern path,
read .points off an unconfigured mock, and all three tests failed on
the branch. Return the hits in response.points as the real client does.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(ci): regenerate requirements-ci.txt for the parse-pdf extra

The CI lockfile check re-resolves pyproject.toml --extra all and diffs
the pinned versions against requirements-ci.txt. The parse-pdf extra
added pdfplumber (+pdfminer-six) to the 'all' bundle without refreshing
the lockfile, so the diff failed and the build job exited 1.

Regenerated with the command from the file header. Only the two new
pins and their 'via' comments changed - every other version is
identical. Verified locally with the CI's own check command (diff of
pkg==ver lines exits clean).

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(tests): update stale default-method-registration assertion

test_parse_methods_dynamic_default_resolution asserted that "default"
resolves through the registry to parse_document, which was true only
because of the self-registration this PR removes (it's what caused the
infinite recursion in the first place). Update the assertion to match
the intended post-fix state: "default" is not registered at all, and
falls through to the built-in dispatch path unconditionally.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: yanyu <yanyu@polixir.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Mohd Kaif <98801504+KaifAhmad1@users.noreply.github.com>
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
2026-09-08 15:47:05 +05:30
Mohd Kaif 384c69293a Merge pull request #1530 from semantica-agi/fix/qdrant-client-search
fix: update Qdrant client compatibility
2026-09-08 14:01:07 +05:30
KaifAhmad1 5606767a29 Merge remote-tracking branch 'origin/main' into fix/qdrant-client-search
# Conflicts:
#	pyproject.toml
2026-09-08 13:46:53 +05:30
Mohd Kaif 7dbad0b167 Merge pull request #1528 from semantica-agi/slim-core-dependencies
feat(deps): slim core dependencies and move 22 heavy packages to opti…
2026-09-08 13:13:46 +05:30
KaifAhmad1andClaude Sonnet 5 b6538740e9 fix(tests): guard lxml-specific assertions in slim-core comment tests
test_xml_parser_handles_comments and test_public_api_ingestor_handles_xml_comments
called XMLParser(engine="lxml") / forced the lxml fallback path unconditionally,
but the core-only CI step installs from base-deps.txt, which no longer bundles
lxml or defusedxml under this PR's slim dependency layout. Skip the lxml-only
assertions when lxml/defusedxml aren't installed, while still exercising the
always-available etree path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 12:47:26 +05:30
Mohd Kaif 0b31d3a370 Merge branch 'main' into slim-core-dependencies 2026-09-08 11:56:58 +05:30
BingEdward c449209168 fix(cli): dispatch mcp call through semantica_mcp.mcp.server (#1368)
Route `semantica mcp call` and `semantica mcp list-tools` through the
canonical `semantica_mcp.mcp.server` packaged module instead of the
nonexistent `MCPSession` import.

Previously, `semantica mcp call` imported `MCPSession` from
`semantica_mcp.mcp.session`, which never defined it, causing every
call to fail with "MCP module not available". Meanwhile,
`semantica mcp list-tools` inspected `semantica_mcp.mcp.tools.__all__`,
which evaluated to `["TOOL_DEFINITIONS"]` and printed a single tool
named "TOOL_DEFINITIONS".

Key changes:
- Implement `semantica_mcp.mcp.server.call_tool(name, arguments)` as the
  shared in-process entry point used by both the CLI and the JSON-RPC
  `tools/call` handler, ensuring both surfaces expose identical tools.
- Add `UnknownToolError` to distinguish missing tools (JSON-RPC -32601)
  from handler-level `KeyError` exceptions (JSON-RPC -32603).
- Update `semantica mcp list-tools` to source tool names directly from
  `TOOL_DEFINITIONS` in `semantica_mcp.mcp.tools`.
- Validate `--args` payloads and reject non-object JSON inputs (arrays,
  primitives, null) with a clean `ClickException`.
- Update `tests/test_mcp_stdio_roundtrip.py` to spawn `semantica_mcp.mcp`
  instead of the pre-relocation `mcp` module, restoring clean stdio
  framing tests.

Fixes #1355
2026-09-08 02:40:53 +05:00
Sameer KadamandSameer Kadam 7be1582786 fix: correct vector store installation extras (#1529)
Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
2026-09-08 01:42:20 +05:00
Zohaib Hassnain 1ff890abdd fix(deps): address review feedback on import exceptions and probes (#1513) 2026-09-08 01:27:42 +05:00
Sameer Kadam 0646601219 fix: handle Qdrant vector count compatibility 2026-09-08 00:54:16 +05:30
Sameer Kadam ad0fbcf235 fix: update Qdrant client compatibility 2026-09-08 00:43:06 +05:30
Mohd Kaif a23f1edb9a Merge branch 'main' into slim-core-dependencies 2026-09-07 22:37:25 +05:30
Zohaib Hassnain a0de5c2cdd fix(deps): address review feedback on slim core dependencies (#1513)
- Remove thinc direct constraint from nlp-spacy in pyproject.toml and update README/CHANGELOG
- Raise ProcessingError with install hint when UMAP is requested but unavailable in EmbeddingVisualizer
- Guard PCA and TSNE dimensionality reducers against None with actionable error messages
- Prevent keyword collisions in EmbeddingVisualizer dimensionality reduction (_reduce_dimensions)
- Add core-only install & test step to .github/workflows/ci.yml using base-deps.txt
- Prevent AttributeError on module load in repo_ingestor.py and xml_ingestor.py when optional dependencies are absent
- Make TOML loading in tests/test_issue_1513_slim_core.py portable across Python versions via UTF-8 text decode and loads
- Expand slim-core test suite to 17 test cases covering 2D/3D projections, core imports, and options handling
2026-09-07 20:32:00 +05:00
Zohaib Hassnain 86ffa05dd4 feat(deps): slim core dependencies and move 22 heavy packages to optional extras (#1513)
- Reduce direct core dependencies in pyproject.toml from 44 to 22
- Move heavy and specialized packages into modular optional extras:
  * models-huggingface: torch, transformers
  * embeddings-local: sentence-transformers, fastembed, onnxruntime, tokenizers
  * nlp-spacy: spacy, thinc
  * viz: matplotlib, seaborn, plotly, ipywidgets, umap-learn (expanded)
  * media: librosa, opencv-python
  * vectorstore-faiss: faiss-cpu
  * documents: python-docx, openpyxl, lxml, beautifulsoup4
  * ingest-git: GitPython
  * graph-embeddings: gensim
- Update semantica[all] and semantica[vectorstore-all] to encompass all extras
- Ensure lazy parser construction (DOCXParser, ExcelParser, HTMLParser, XMLParser) without error on __init__(), failing only inside .parse() with clear hints
- Add stdlib xml.etree fallback in XMLParser when lxml is missing
- Guard unguarded matplotlib imports in EmbeddingVisualizer and OntologyVisualizer
- Standardize user-facing error messages to point to pip install 'semantica[extra]'
- Bump version to 0.7.0 in pyproject.toml, semantica/__init__.py, and CITATION.cff
- Add migration notes to README.md and CHANGELOG.md
- Recompile CI and Docker requirements lockfiles
- Add dedicated test suite tests/test_issue_1513_slim_core.py
2026-09-07 19:53:09 +05:00
162 changed files with 13514 additions and 9590 deletions
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+27 -18
View File
@@ -97,24 +97,10 @@ jobs:
- name: Build Explorer frontend - name: Build Explorer frontend
working-directory: explorer working-directory: explorer
run: npm run build run: npm run build
- name: Install Explorer backend test dependencies - name: Install core package and base dependencies
run: | run: |
# Run the deterministic backend path before the all-extras CI # Verify that core semantica installs cleanly with only its base dependencies
# environment is installed. The Explorer extra supplies the # (no optional extras) and that core imports and lazy missing-dependency hints work.
# production API dependencies without importing optional vector
# providers such as Pinecone during test collection.
#
# --no-deps + a separate hash-pinned install (rather than the old
# `pip install -e ".[explorer]" pytest==9.1.1`) so every fetched
# package is hash-verified (Scorecard Pinned-Dependencies); the
# local editable install itself has nothing to hash.
# .github/requirements/explorer-extra-py311.txt is
# `uv pip compile pyproject.toml --extra explorer --python-version 3.11 --constraint requirements-ci.txt --generate-hashes`
# - regenerate it the same way if pyproject.toml's base/explorer
# deps change. Resolved specifically for this job's python 3.11
# (see the Dockerfile's explorer-extra-py313.txt for why this
# can't be shared with python 3.13: audioread needs extra
# standard-aifc/standard-sunau hashes only on 3.13+).
# #
# --no-deps only skips *runtime* dependency resolution - `-e .` # --no-deps only skips *runtime* dependency resolution - `-e .`
# still does a PEP 517 build, which by default creates an isolated # still does a PEP 517 build, which by default creates an isolated
@@ -125,8 +111,31 @@ jobs:
# copies instead of fetching its own. # copies instead of fetching its own.
pip install -r .github/requirements/pep517-build.txt --require-hashes pip install -r .github/requirements/pep517-build.txt --require-hashes
pip install --no-deps --no-build-isolation -e . pip install --no-deps --no-build-isolation -e .
pip install -r .github/requirements/explorer-extra-py311.txt --require-hashes pip install -r .github/requirements/base-deps.txt --require-hashes
pip install -r .github/requirements/pytest-tool.txt --require-hashes pip install -r .github/requirements/pytest-tool.txt --require-hashes
- name: Verify core-only package importability and slim behavior
run: |
python -c "
import semantica
print('semantica', semantica.__version__, 'core installed and importable')
"
pytest -q tests/test_issue_1513_slim_core.py
pytest -q tests/test_docs_check.py
- name: Install Explorer backend test dependencies
run: |
# Run the deterministic backend path before the all-extras CI
# environment is installed. The Explorer extra supplies the
# production API dependencies without importing optional vector
# providers such as Pinecone during test collection.
#
# .github/requirements/explorer-extra-py311.txt is
# `uv pip compile pyproject.toml --extra explorer --python-version 3.11 --constraint requirements-ci.txt --generate-hashes`
# - regenerate it the same way if pyproject.toml's base/explorer
# deps change. Resolved specifically for this job's python 3.11
# (see the Dockerfile's explorer-extra-py313.txt for why this
# can't be shared with python 3.13: audioread needs extra
# standard-aifc/standard-sunau hashes only on 3.13+).
pip install -r .github/requirements/explorer-extra-py311.txt --require-hashes
- name: Test deterministic Explorer backend path - name: Test deterministic Explorer backend path
run: | run: |
pytest -q tests/explorer/test_explorer_deterministic_rendering_e2e.py pytest -q tests/explorer/test_explorer_deterministic_rendering_e2e.py
+6 -6
View File
@@ -34,7 +34,7 @@ jobs:
# meaningful state carried over from a failed attempt. # meaningful state carried over from a failed attempt.
- name: Initialize CodeQL (attempt 1) - name: Initialize CodeQL (attempt 1)
id: codeql-init-1 id: codeql-init-1
uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
continue-on-error: true continue-on-error: true
with: with:
languages: python languages: python
@@ -44,7 +44,7 @@ jobs:
- name: Initialize CodeQL (attempt 2) - name: Initialize CodeQL (attempt 2)
id: codeql-init-2 id: codeql-init-2
if: steps.codeql-init-1.outcome == 'failure' if: steps.codeql-init-1.outcome == 'failure'
uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
continue-on-error: true continue-on-error: true
with: with:
languages: python languages: python
@@ -54,17 +54,17 @@ jobs:
- name: Initialize CodeQL (attempt 3) - name: Initialize CodeQL (attempt 3)
id: codeql-init-3 id: codeql-init-3
if: steps.codeql-init-2.outcome == 'failure' if: steps.codeql-init-2.outcome == 'failure'
uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
with: with:
languages: python languages: python
queries: security-and-quality queries: security-and-quality
config-file: .github/codeql/codeql-config.yml config-file: .github/codeql/codeql-config.yml
- name: Autobuild - name: Autobuild
uses: github/codeql-action/autobuild@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/autobuild@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/analyze@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
with: with:
category: "/language:python" category: "/language:python"
upload: false upload: false
@@ -74,7 +74,7 @@ jobs:
# Uploads results only when Default Setup is not active. # Uploads results only when Default Setup is not active.
# If Default Setup is still enabled, this step skips gracefully # If Default Setup is still enabled, this step skips gracefully
# instead of failing the workflow with HTTP 409. # instead of failing the workflow with HTTP 409.
uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
with: with:
sarif_file: ${{ steps.codeql.outputs.sarif-output }} sarif_file: ${{ steps.codeql.outputs.sarif-output }}
category: "/language:python" category: "/language:python"
+1 -1
View File
@@ -61,7 +61,7 @@ jobs:
- name: Upload Trivy SARIF - name: Upload Trivy SARIF
if: always() if: always()
uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
with: with:
sarif_file: trivy-results.sarif sarif_file: trivy-results.sarif
category: trivy-container category: trivy-container
+2 -2
View File
@@ -59,7 +59,7 @@ jobs:
# avoiding the guardian.cmd/checkov exit-code bug in the MSDO wrapper. # avoiding the guardian.cmd/checkov exit-code bug in the MSDO wrapper.
tools: eslint,templateanalyzer,terrascan tools: eslint,templateanalyzer,terrascan
- name: Upload results to Security tab - name: Upload results to Security tab
uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
with: with:
sarif_file: ${{ steps.msdo.outputs.sarifFile }} sarif_file: ${{ steps.msdo.outputs.sarifFile }}
@@ -100,7 +100,7 @@ jobs:
run: python .github/scripts/filter_checkov_skipped.py reports/results_json.json reports/results_sarif.sarif reports/checkov.sarif run: python .github/scripts/filter_checkov_skipped.py reports/results_json.json reports/results_sarif.sarif reports/checkov.sarif
- name: Upload Checkov results to Security tab - name: Upload Checkov results to Security tab
uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
if: always() if: always()
with: with:
sarif_file: reports/checkov.sarif sarif_file: reports/checkov.sarif
+1 -1
View File
@@ -40,6 +40,6 @@ jobs:
retention-days: 5 retention-days: 5
- name: Upload to code-scanning - name: Upload to code-scanning
uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
with: with:
sarif_file: results.sarif sarif_file: results.sarif
+33 -10
View File
@@ -154,13 +154,21 @@ jobs:
fi fi
# Vulnerability IDs reviewed and accepted as non-actionable for this # Vulnerability IDs reviewed and accepted as non-actionable for this
# project. Empty for now: pip-audit's OSV-backed database doesn't # project:
# currently carry either of the findings Safety used to flag here # - GHSA-4j2p-28q2-5m79 (aka CVE-2026-69112): accelerate<=1.14.0
# (cuda-toolkit CVE-2025-33228, torchvision CVE-2026-65918), so # (transitive via docling-slim). Path traversal in sharded checkpoint
# there's nothing to exclude. Left in place so a future finding can # index loading (load_checkpoint_in_model). 1.14.0 is the latest
# be added the same way without restructuring this step - see git # available PyPI release; no upstream patch exists yet. Semantica does
# history on this file for the reasoning behind past entries. # not load arbitrary user checkpoints. Re-evaluate once accelerate
IGNORED_VULN_IDS="" # releases a fixed version.
# NOTE: pip-audit's OSV-backed report may surface either identifier as
# the primary `id` (with the other listed under `aliases`) depending on
# which alias the backing database picks as canonical, so both need to
# be listed here and the matching below checks aliases too - see
# https://github.com/semantica-agi/semantica/actions/runs/34296586683
# where this ignore list had only the GHSA id but the report's `id`
# was the CVE, so the gate still failed.
IGNORED_VULN_IDS="GHSA-4j2p-28q2-5m79,CVE-2026-69112"
# Exported so the "Comment PR with Security Results" step below can # Exported so the "Comment PR with Security Results" step below can
# apply the same exclusion list to the raw report - it reads # apply the same exclusion list to the raw report - it reads
@@ -176,9 +184,16 @@ jobs:
# no vulns field at all (see the skip_reason handling above) - # no vulns field at all (see the skip_reason handling above) -
# without the fallback, iterating `null[]` raises inside jq and # without the fallback, iterating `null[]` raises inside jq and
# this whole computation silently evaluates to empty. # this whole computation silently evaluates to empty.
#
# Matching checks `.id` AND `.aliases` (pip-audit includes aliases by
# default for JSON output): the OSV-backed report can surface either
# the GHSA or the CVE identifier as the canonical `id` for the same
# advisory, with the other one demoted to an alias, so matching on
# `.id` alone is not reliable.
VULNS=$(jq --arg ignored "$IGNORED_VULN_IDS" ' VULNS=$(jq --arg ignored "$IGNORED_VULN_IDS" '
($ignored | split(",") | map(select(length > 0))) as $ignore_list ($ignored | split(",") | map(select(length > 0))) as $ignore_list
| [.dependencies[] | (.vulns // [])[] | select(.id as $id | ($ignore_list | index($id)) | not)] | [.dependencies[] | (.vulns // [])[]
| select(([.id] + (.aliases // [])) as $ids | ($ids - $ignore_list | length) == ($ids | length))]
| length | length
' pip-audit-report.json 2>/dev/null) ' pip-audit-report.json 2>/dev/null)
@@ -199,7 +214,8 @@ jobs:
jq --arg ignored "$IGNORED_VULN_IDS" -r ' jq --arg ignored "$IGNORED_VULN_IDS" -r '
($ignored | split(",") | map(select(length > 0))) as $ignore_list ($ignored | split(",") | map(select(length > 0))) as $ignore_list
| .dependencies[] as $dependency | .dependencies[] as $dependency
| ($dependency.vulns // [])[] | select(.id as $id | ($ignore_list | index($id)) | not) | ($dependency.vulns // [])[]
| select(([.id] + (.aliases // [])) as $ids | ($ids - $ignore_list | length) == ($ids | length))
| "- \($dependency.name)==\($dependency.version): \(.id)" | "- \($dependency.name)==\($dependency.version): \(.id)"
' pip-audit-report.json || true ' pip-audit-report.json || true
exit 1 exit 1
@@ -345,9 +361,16 @@ jobs:
return null; return null;
} }
// A vuln's canonical `id` and its `aliases` (e.g. GHSA vs. CVE
// for the same advisory) are checked together - mirrors the
// shell gate above, which needs the same fallback because
// pip-audit's OSV-backed report doesn't consistently pick the
// same identifier as canonical across advisories.
return data.dependencies.flatMap((dependency) => return data.dependencies.flatMap((dependency) =>
(dependency.vulns || []) (dependency.vulns || [])
.filter((vulnerability) => !ignoredVulnIds.includes(vulnerability.id)) .filter((vulnerability) =>
![vulnerability.id, ...(vulnerability.aliases || [])].some((id) => ignoredVulnIds.includes(id))
)
.map( .map(
(vulnerability) => `- \`${dependency.name}==${dependency.version}\`: ${vulnerability.id}` + (vulnerability) => `- \`${dependency.name}==${dependency.version}\`: ${vulnerability.id}` +
(vulnerability.fix_versions?.length ? ` (fixed by ${vulnerability.fix_versions.join(', ')})` : '') (vulnerability.fix_versions?.length ? ` (fixed by ${vulnerability.fix_versions.join(', ')})` : '')
BIN
View File
Binary file not shown.
+35
View File
@@ -9,6 +9,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
### Added
- **Schema-guided extraction validation** (#1510) by @Besokus
- New `SchemaValidator` (`semantica.semantic_extract`, lazy export): a deterministic sibling of `ExtractionValidator` that checks extraction output for *conformance to a domain ontology* — an axis orthogonal to `ExtractionValidator`'s confidence checks. It mirrors the same interface (`validate_entities()` / `validate_relations()` returning `ValidationResult`, batch-aware), so the two compose back-to-back
- Entity labels must be concepts in the schema; relation predicates must be in the schema and satisfy their `domain` / `range`. Violations are reported in `ValidationResult.errors` with counts in `metrics` and `score` = conformance ratio; `filter_by_schema()` / `filter_relations_by_schema()` return the conforming subset (mirroring `filter_by_confidence`). No LLM required
- New `ExtractionSchema` (`semantica.semantic_extract`, lazy export): a lightweight, read-only view over a domain ontology (allowed concepts + predicates with optional `domain` / `range`). Reuses the project's existing OWL ontology representation rather than a parallel type — build one from a `generate_ontology`-style dict (`ExtractionSchema.from_ontology`) or an OWL/Turtle file/string (`ExtractionSchema.from_owl`, via the existing `rdflib` dependency). An empty `domain`/`range` means unconstrained, matching OWL
- Implements the deterministic core of ontology-based information extraction (OBIE; Wimalasuriya & Dou, 2010). No new runtime dependencies
- New `tests/semantic_extract/test_schema_validator.py`
## [0.7.0] - 2026-09-07
### Changed
- **Slim core dependencies: moved ~22 heavy packages to optional extras** (#1513)
- Core dependencies in `pyproject.toml` are now reduced to exactly 22 direct packages: `numpy`, `pandas`, `scipy`, `scikit-learn`, `rdflib`, `networkx`, `requests`, `chardet`, `protobuf`, `grpcio`, `pillow`, `pydantic`, `click`, `rich`, `tqdm`, `pyyaml`, `toml`, `python-dotenv`, `loguru`, `structlog`, `httpx`, and `pyarrow`.
- Heavy ML/NLP, visualization, document parsing, and ingestion packages moved into granular optional extras:
- `models-huggingface`: `torch`, `transformers`
- `embeddings-local`: `sentence-transformers`, `fastembed`, `onnxruntime`, `tokenizers`
- `nlp-spacy`: `spacy`
- `viz`: expanded to include `matplotlib`, `seaborn`, `plotly`, `ipywidgets`, `umap-learn`, alongside `pyvis`, `graphviz`, and `d3blocks`
- `media`: `librosa`, `opencv-python`
- `vectorstore-faiss`: `faiss-cpu` (also included in `vectorstore-all`)
- `documents`: `python-docx`, `openpyxl`, `lxml`, `beautifulsoup4`
- `ingest-git`: `GitPython`
- `graph-embeddings`: `gensim` (also included in `graph-all`)
- Full bundled behavior preserved via `pip install "semantica[all]"`, which includes all optional extras. Pinning `semantica<0.7.0` remains a permanent escape hatch for legacy workflows.
- Safe lazy construction across parsers and visualizers:
- `DOCXParser`, `ExcelParser`, `HTMLParser`, and `XMLParser` remain constructible without error on `__init__()`. They fail only upon calling `.parse()` with actionable error messages directing users to install `semantica[documents]`.
- `XMLParser` automatically falls back to standard library `xml.etree` (`_parse_with_etree`) when `lxml` is not installed, preserving XML parsing capabilities without extra dependencies.
- `EmbeddingVisualizer` and `OntologyVisualizer` safely guard `matplotlib` and optional reduction packages, advising `pip install 'semantica[viz]'`.
- `RepoIngestor` guards `GitPython` with a clear error pointing to `semantica[ingest-git]`.
- `PublicAPIIngestor` guards `lxml` and `_SAFE_XML_PARSER`.
- Updated user-facing installation hints across CLI doctor commands, node embeddings (`NodeEmbedder`), vector stores (`FAISSStore`), and model loaders.
- Recompiled CI lockfiles (`requirements-ci.txt`, `.github/requirements/explorer-extra-py311.txt`, `.github/requirements/explorer-extra-py313.txt`, and `.github/requirements/base-deps.txt`).
## [0.6.8] - 2026-09-05 ## [0.6.8] - 2026-09-05
### Added ### Added
+1 -1
View File
@@ -7,7 +7,7 @@ authors:
repository-code: "https://github.com/semantica-agi/semantica" repository-code: "https://github.com/semantica-agi/semantica"
url: "https://getsemantica.ai" url: "https://getsemantica.ai"
license: MIT license: MIT
version: 0.6.8 version: 0.7.0
date-released: 2026-09-05 date-released: 2026-09-05
keywords: keywords:
- knowledge-graph - knowledge-graph
+1 -1
View File
@@ -31,7 +31,7 @@ RUN mkdir -p /app/semantica && npm run build
# `pip index versions gensim` / the project's PyPI files page, not just # `pip index versions gensim` / the project's PyPI files page, not just
# whether `uv pip compile` resolves (resolution only reads sdist metadata, # whether `uv pip compile` resolves (resolution only reads sdist metadata,
# it doesn't attempt the build that fails here). # it doesn't attempt the build that fails here).
FROM python:3.13-slim@sha256:7ce4b6dfe35e55397b7cda544f8a13f191b7ae28dc5aad71fe664dbc9bc2623f AS runtime FROM python:3.14-slim@sha256:cad9a2c871761c413caa6fdd6441c783451e740a48aaeba60ae62a8b53525ef6 AS runtime
ENV PYTHONDONTWRITEBYTECODE=1 \ ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \ PYTHONUNBUFFERED=1 \
+49 -50
View File
@@ -42,6 +42,8 @@ pip install semantica
</div> </div>
[English](https://readme-i18n.com/semantica-agi/semantica?lang=en) · [Deutsch](https://readme-i18n.com/semantica-agi/semantica?lang=de) · [Français](https://readme-i18n.com/semantica-agi/semantica?lang=fr) · [Español](https://readme-i18n.com/semantica-agi/semantica?lang=es) · [Italiano](https://readme-i18n.com/semantica-agi/semantica?lang=it) · [Português](https://readme-i18n.com/semantica-agi/semantica?lang=pt) · [العربية](https://readme-i18n.com/semantica-agi/semantica?lang=ar) · [اردو](https://readme-i18n.com/semantica-agi/semantica?lang=ur) · [हिन्दी](https://readme-i18n.com/semantica-agi/semantica?lang=hi) · [中文](https://readme-i18n.com/semantica-agi/semantica?lang=zh) · [日本語](https://readme-i18n.com/semantica-agi/semantica?lang=ja) · [한국어](https://readme-i18n.com/semantica-agi/semantica?lang=ko)
--- ---
<div align="center"> <div align="center">
@@ -62,16 +64,21 @@ pip install semantica
--- ---
Most AI agents run on embeddings, not meaning: similarity scores with no structure, no relationships, and no way to explain why a result came back. Semantica is the semantic/context layer underneath your LLM, vector store, and agent framework: a deterministic infrastructure layer (no LLM required for graph construction, reasoning, or provenance) that turns fragmented enterprise data into a structured, queryable Context Graph and knowledge graph, governed by ontologies and controlled vocabularies (OWL, SHACL, SKOS) so the meaning of your data is explicit, not just its embedding. Decision provenance and audit trails fall out of that structure as a property, not the product itself; in domains a regulator can question, that same structure just happens to double as a straight answer to "why." Most AI agents run on embeddings, not meaning: similarity scores with no structure, no relationships, and no way to explain why a result came back.
> ⚠️ **System-level explainability, not foundation-model explainability.** Semantica does not expose or reconstruct what happens *inside* the LLM — its internal reasoning or chain-of-thought stays opaque, as it does for any external system. Semantica explains what's *outside* the model: the context and data fed in, the decision produced, its provenance, relevant relationships, applied policies, and the full execution trail. Semantica is the semantic/context layer underneath your LLM, vector store, and agent framework: deterministic infrastructure (no LLM required for graph construction, reasoning, or provenance; where an LLM is used, it's optional and vendor-neutral, every major provider supported, OpenAI, Anthropic, Gemini, and more, via `semantica.llms`) that turns fragmented enterprise data into a structured, queryable Context Graph and knowledge graph that carries the business context, not just the data structure. Ontologies and controlled vocabularies (OWL, SHACL, SKOS) make what an entity *means* to your business, its definitions, relationships, and rules, as explicit as the data itself, not just its embedding.
Decision provenance and audit trails aren't the product. They fall out of that structure for free, and in domains a regulator can question, the same structure that makes your agent smarter also gives you a straight answer to "why."
> [!NOTE]
> **System-level explainability, not foundation-model explainability.** Semantica doesn't expose or reconstruct what happens *inside* the LLM: its internal reasoning stays opaque, like it does for any external system. Semantica explains what's *outside* the model: the context fed in, the decision produced, its provenance, relevant relationships, applied policies, and the full execution trail.
**Who it's for:** **Who it's for:**
- **AI/ML platform teams** shipping agents that make consequential decisions and need structured, queryable context, not just a vector index - **AI/ML platform teams** shipping agents that make consequential decisions and need structured, queryable context, not just a vector index
- **Data platform teams on Databricks or Snowflake** turning tables already in Unity Catalog or a warehouse into a governed, lineage-tracked knowledge graph, without exporting to a third-party SaaS - **Enterprise data teams on Databricks, Snowflake, or SAP** turning tables already in the lakehouse or warehouse into a governed, lineage-tracked knowledge graph, without exporting to a third-party SaaS
- **Compliance, risk, and audit teams** who need a straight answer to "why did the AI do that?" in a format a regulator accepts - **Compliance, risk, and audit teams** who need a straight answer to "why did the AI do that?" in a format a regulator accepts
- **Regulated enterprises** (finance, healthcare, legal, government, defense) that can't ship a black box or send their data to someone else's SaaS to get one - **Regulated enterprises** (finance, healthcare, legal, government, defense) that can't ship a black box or hand their data to someone else's SaaS to get one
- **Platform and infra engineers** who want the KG, reasoning, and provenance stack self-hosted and swappable, not locked to one vendor's backend - **Platform and infra engineers** who want the KG, reasoning, and provenance stack self-hosted and swappable, not locked to one vendor's backend
- **Data and knowledge engineers** building a KG from messy, multi-source data, where conflicting facts get flagged and duplicates get merged, not silently overwritten - **Data and knowledge engineers** building a KG from messy, multi-source data, where conflicting facts get flagged and duplicates get merged, not silently overwritten
@@ -83,15 +90,15 @@ Most AI agents run on embeddings, not meaning: similarity scores with no structu
- **Context Graphs:** A structured, queryable graph of everything your agent knows, decides, and reasons about - **Context Graphs:** A structured, queryable graph of everything your agent knows, decides, and reasons about
- **Decision Intelligence:** Every decision is a first-class object: traceable, searchable by precedent, and causally linked - **Decision Intelligence:** Every decision is a first-class object: traceable, searchable by precedent, and causally linked
- **AI Governance & Ontology:** SHACL constraints, conflict detection, compliance rules, OWL generation, and SKOS vocabulary management with a visual editor - **AI Governance & Ontology:** SHACL constraints, conflict detection, compliance rules, OWL generation, and SKOS vocabularies, all with a visual editor
- **Full Auditability:** W3C PROV-O provenance on every fact, with audit trails exportable to JSON, CSV, or RDF - **Full Auditability:** W3C PROV-O provenance on every fact, exportable to JSON, CSV, or RDF
- **Deterministic Reasoning:** Forward chaining, Rete network, Datalog, and SPARQL with fully explainable paths, not black boxes - **Deterministic Reasoning:** Forward chaining, Rete network, Datalog, and SPARQL, with fully explainable paths, not black boxes
- **Knowledge Pipeline:** Multi-source ingestion, entity-aware chunking, NER/relation/event extraction, and knowledge graph construction, with semantic deduplication and provenance-preserving merges throughout - **Knowledge Pipeline:** Multi-source ingestion, entity-aware chunking, NER/relation/event extraction, and graph construction, with semantic dedup and provenance-preserving merges built in
- **Enterprise Data Platforms:** Native connectors for Databricks (Unity Catalog + Delta Lake, PAT/OAuth M2M auth, catalog/schema/table/lineage introspection), Snowflake (warehouse/database/schema, key-pair and OAuth auth), and SAP OData (Business Partners, Sales Orders, OAuth2/Basic auth), so data already living in your lakehouse or warehouse becomes graph nodes with provenance, not another export/import hop - **Enterprise Data Platforms:** Native connectors for Databricks (Unity Catalog + Delta Lake), Snowflake, and SAP OData, so data already in your lakehouse or warehouse becomes graph nodes with provenance, no export/import hop
- **Graph Analytics:** Centrality, community detection, link prediction, and shortest-path queries over the graph you just built - **Graph Analytics:** Centrality, community detection, link prediction, and shortest-path queries over the graph you just built
- **Polyglot Graph Storage:** Native RDF (embedded Oxigraph, Blazegraph, Apache Jena, Eclipse RDF4J via SPARQL) and Labeled Property Graphs (Neo4j, FalkorDB, Apache AGE, AWS Neptune via Cypher), plus vector stores, all swappable without touching your code - **Polyglot Graph Storage:** RDF (Oxigraph, Blazegraph, Jena, RDF4J) and Labeled Property Graphs (Neo4j, FalkorDB, AGE, Neptune), plus vector stores, all swappable without touching your code
- **Visualization:** Explore any graph, ontology, or timeline in an interactive browser workbench - **Visualization:** Explore any graph, ontology, or timeline in an interactive browser workbench
- **Drop-in Integrations:** Native Agno, CrewAI, and LangChain support, a full-featured MCP server, a comprehensive CLI, a REST API, and plugins across major editors - **Drop-in Integrations:** Agno, CrewAI, and LangChain support, a full MCP server, a CLI, a REST API, and plugins across major editors
--- ---
@@ -1480,8 +1487,6 @@ app = create_app(session=GraphSession(graph), agent_memory=memory)
The Memories workspace is shown only when `agent_memory` is provided. Apply The Memories workspace is shown only when `agent_memory` is provided. Apply
updates the supplied runtime object; it does not add disk persistence. updates the supplied runtime object; it does not add disk persistence.
---
## What's New in v0.6.8 ## What's New in v0.6.8
**Every release from here on is cryptographically signed** — the build now runs SLSA build-provenance attestation plus Sigstore signing, and `.sigstore.json` bundles ship alongside the wheel/sdist on every GitHub Release, closing the OpenSSF Scorecard Signed-Releases gap. Beyond that, this is a large fix-and-hardening release plus a batch of vector-store and LLM-provider additions: **Every release from here on is cryptographically signed** — the build now runs SLSA build-provenance attestation plus Sigstore signing, and `.sigstore.json` bundles ship alongside the wheel/sdist on every GitHub Release, closing the OpenSSF Scorecard Signed-Releases gap. Beyond that, this is a large fix-and-hardening release plus a batch of vector-store and LLM-provider additions:
@@ -1499,51 +1504,45 @@ Also fixes 35 correctness bugs (Python 3.9 install breakage, FAISS save/load met
--- ---
## Built for High-Stakes Domains
Semantica is designed for environments where AI outputs must be explainable, auditable, and defensible, and where the data itself can't leave your infrastructure. Self-hostable with zero vendor lock-in, it's built as much for organizations handling confidential or classified data as for regulated industries chasing an audit trail:
- **Finance:** Loan underwriting audit trails, fraud detection, AML compliance, regulatory risk knowledge graphs
- **Healthcare:** Clinical decision support, drug interaction graphs, and patient safety audit trails
- **Legal:** Evidence-backed research, contract analysis, case law reasoning, and privilege tracking
- **Government & Defense:** Policy decision records, classified information governance, and regulatory reporting, fully self-hosted with no data leaving your perimeter
- **Law Enforcement:** Case linkage, evidence provenance chains, and investigative knowledge graphs that hold up under legal scrutiny
- **Cybersecurity:** Threat attribution, incident response timelines, and IOC provenance tracking
- **Autonomous Systems:** Decision logs, safety validation, and explainable AI for certification
> ⚠️ **This is system-level explainability, not foundation-model explainability.** Semantica does not expose, reconstruct, or explain what happens *inside* the LLM/foundation model — its internal reasoning or chain-of-thought stays opaque, as it does for any external system. What Semantica explains is *outside* the model: the context and data fed in, the decision produced, its provenance, the relevant relationships, the policies applied, and the full execution trail. In short, Semantica explains and audits what the AI system did, not the LLM's private internal reasoning.
---
## Installation ## Installation
```bash ```bash
pip install semantica # core pip install semantica # lightweight core (22 essential dependencies)
pip install semantica[all] # everything pip install "semantica[all]" # full bundled behavior with all extras
``` ```
> **Note:** Heavy machine learning, NLP, visualization, and document dependencies live in optional extras to keep core installation lightweight and fast. If you want the previous bundled installation, install with `pip install "semantica[all]"`.
```bash ```bash
pip install semantica[agno] # Agno multi-agent integration # Granular Extras
pip install semantica[crewai] # CrewAI integration pip install "semantica[documents]" # Document parsing (docx, openpyxl, lxml, beautifulsoup4)
pip install semantica[langchain] # LangChain / LangGraph integration pip install "semantica[embeddings-local]" # Local embeddings (sentence-transformers, fastembed, onnxruntime)
pip install semantica[llm-litellm] # OpenAI, Anthropic, Gemini, Mistral, Llama, Groq, Cohere, Bedrock, Ollama, DeepSeek, and more pip install "semantica[models-huggingface]" # HuggingFace models (transformers, torch)
pip install semantica[graph-neo4j] # Neo4j graph store (LPG) pip install "semantica[nlp-spacy]" # spaCy NLP pipelines (spacy)
pip install semantica[graph-falkordb] # FalkorDB graph store (LPG) pip install "semantica[viz]" # Visualization (matplotlib, seaborn, plotly, pyvis, graphviz)
pip install semantica[graph-apache-age] # Apache AGE graph store (LPG) pip install "semantica[media]" # Audio & computer vision (librosa, opencv-python)
pip install semantica[graph-amazon-neptune] # AWS Neptune graph store (LPG) pip install "semantica[graph-embeddings]" # Knowledge graph embeddings (gensim / Node2Vec)
pip install semantica[tripletstore-oxigraph] # Embedded in-memory/on-disk RDF store pip install "semantica[ingest-git]" # Git repository ingestor (GitPython)
pip install "semantica[vectorstore-faiss]" # FAISS vector store
pip install "semantica[vectorstore-all]" # All vector stores (Qdrant, Pinecone, Weaviate, FAISS, PgVector, SQLite)
pip install "semantica[agno]" # Agno multi-agent integration
pip install "semantica[crewai]" # CrewAI integration
pip install "semantica[langchain]" # LangChain / LangGraph integration
pip install "semantica[llm-all]" # All LLM provider clients
pip install "semantica[graph-neo4j]" # Neo4j graph store (LPG)
pip install "semantica[graph-falkordb]" # FalkorDB graph store (LPG)
pip install "semantica[graph-apache-age]" # Apache AGE graph store (LPG)
pip install "semantica[graph-amazon-neptune]" # AWS Neptune graph store (LPG)
pip install "semantica[tripletstore-oxigraph]" # Embedded in-memory/on-disk RDF store
# RDF triple stores (Blazegraph, Apache Jena, Eclipse RDF4J) need no extra: # RDF triple stores (Blazegraph, Apache Jena, Eclipse RDF4J) need no extra:
# semantica.triplet_store talks SPARQL over HTTP using the core `requests` dependency # semantica.triplet_store talks SPARQL over HTTP using the core `requests` dependency
pip install semantica[vectorstore-qdrant] # Qdrant vector store pip install "semantica[db-snowflake]" # Snowflake
pip install semantica[vectorstore-pinecone] # Pinecone vector store pip install "semantica[db-databricks]" # Databricks (SDK + SQL connector)
pip install semantica[db-snowflake] # Snowflake pip install "semantica[ingest-sap]" # SAP OData
pip install semantica[db-databricks] # Databricks (SDK + SQL connector) pip install "semantica[ingest-parquet]" # Parquet / PyArrow
pip install semantica[ingest-sap] # SAP OData pip install "semantica[ingest-arrow]" # Apache Arrow, Feather, IPC
pip install semantica[ingest-parquet] # Parquet / PyArrow pip install "semantica[watch]" # Directory file watcher
pip install semantica[ingest-arrow] # Apache Arrow, Feather, IPC pip install "semantica[explorer]" # Knowledge Explorer dashboard
pip install semantica[viz] # HTML interactive visualization
pip install semantica[watch] # Directory file watcher
pip install semantica[explorer] # Knowledge Explorer dashboard
``` ```
For production deployments, use Docker or Kubernetes rather than a local `pip install`. Set `SEMANTICA_API_KEY`, configure a persistent LPG graph store (Neo4j / FalkorDB / Apache AGE / AWS Neptune) and/or RDF triple store (Blazegraph / Apache Jena / Eclipse RDF4J), and point the vector store at a hosted backend (Qdrant / Pinecone). See [ARCHITECTURE.md](ARCHITECTURE.md) for the full deployment topology. For production deployments, use Docker or Kubernetes rather than a local `pip install`. Set `SEMANTICA_API_KEY`, configure a persistent LPG graph store (Neo4j / FalkorDB / Apache AGE / AWS Neptune) and/or RDF triple store (Blazegraph / Apache Jena / Eclipse RDF4J), and point the vector store at a hosted backend (Qdrant / Pinecone). See [ARCHITECTURE.md](ARCHITECTURE.md) for the full deployment topology.
+2 -1
View File
@@ -107,7 +107,8 @@
"integrations/docling", "integrations/docling",
"integrations/snowflake", "integrations/snowflake",
"integrations/databricks", "integrations/databricks",
"integrations/salesforce" "integrations/salesforce",
"integrations/redshift"
] ]
}, },
{ {
+1 -1
View File
@@ -5,7 +5,7 @@ icon: "circle-question"
--- ---
<Info> <Info>
Use **Ctrl+F** / **Cmd+F** to search this page. Common jumps: [Installation](#installation) · [Data & Features](#data--features) · [Troubleshooting](#troubleshooting) Use **Ctrl+F** / **Cmd+F** to search this page. Common jumps: [Installation](#installation) · [Data & Features](#data-&-features) · [Troubleshooting](#troubleshooting)
</Info> </Info>
## Quick Answers ## Quick Answers
+2 -2
View File
@@ -661,7 +661,7 @@ print("Total memories: {}".format(s.get("total_items", 0)))
- [Decision Intelligence](/guides/decision-intelligence) — Recording decisions as graph nodes with causal chains and policy gating. - [Decision Intelligence](/guides/decision-intelligence) — Recording decisions as graph nodes with causal chains and policy gating.
- [Multi-Agent Systems](/guides/multi-agent) — Coordinating multiple agents through a shared `AgentContext` and save/load handoffs. - [Multi-Agent Systems](/guides/multi-agent) — Coordinating multiple agents through a shared `AgentContext` and save/load handoffs.
- [LLM Integrations](/guides/llm-integrations) — Configuring the LLM provider passed to `query_with_reasoning()`. - [LLM Integrations](/guides/llm-integrations) — Configuring the LLM provider passed to `query_with_reasoning()`.
- [Deduplication Guide](deduplication) — Full reference for `DuplicateDetector`, `EntityMerger`, similarity methods, and cluster strategies. - [Deduplication Guide](/guides/deduplication) — Full reference for `DuplicateDetector`, `EntityMerger`, similarity methods, and cluster strategies.
- [Ontology Management](ontology) — Generate and validate OWL ontologies from the knowledge graph; export to Turtle, OWL/XML, JSON-LD. - [Ontology Management](/guides/ontology) — Generate and validate OWL ontologies from the knowledge graph; export to Turtle, OWL/XML, JSON-LD.
- [Context Module Reference](../reference/context) — Full API: `AgentContext`, `AgentMemory`, `MemoryItem`, `ContextRetriever`. - [Context Module Reference](../reference/context) — Full API: `AgentContext`, `AgentMemory`, `MemoryItem`, `ContextRetriever`.
- [Vector Store Reference](../reference/vector_store) — FAISS, Qdrant, pgvector, Pinecone backends. - [Vector Store Reference](../reference/vector_store) — FAISS, Qdrant, pgvector, Pinecone backends.
+3 -3
View File
@@ -497,7 +497,7 @@ print("Model v1.1 verified and approved for production.")
## Related Guides ## Related Guides
- [Context Graphs](/guides/context-graphs) — `ContextGraph.to_dict()` feeds `create_snapshot()` - [Context Graphs](/guides/context-graphs) — `ContextGraph.to_dict()` feeds `create_snapshot()`
- [Ontology Management](ontology) — pair ontology versioning with graph versioning for a complete schema + data audit trail - [Ontology Management](/guides/ontology) — pair ontology versioning with graph versioning for a complete schema + data audit trail
- [SHACL Validation](/guides/shacl-validation) — validate graph data at each version gate before snapshotting - [SHACL Validation](/guides/shacl-validation) — validate graph data at each version gate before snapshotting
- [Provenance](provenance) — combine change management with W3C PROV-O lineage for a full audit trail - [Provenance](/guides/provenance) — combine change management with W3C PROV-O lineage for a full audit trail
- [Visualization](visualization) — `TemporalVisualizer.visualize_snapshot_comparison()` and `visualize_metrics_evolution()` render version diffs as interactive charts - [Visualization](/guides/visualization) — `TemporalVisualizer.visualize_snapshot_comparison()` and `visualize_metrics_evolution()` render version diffs as interactive charts
+4 -4
View File
@@ -65,7 +65,7 @@ flowchart TD
G --> H[SHACL Validation] G --> H[SHACL Validation]
``` ```
1. **Deduplication** — Merge duplicate nodes so each entity has exactly one canonical record. Conflict resolution operates on a single canonical entity; you must identify it before comparing what different sources say about it. See [Deduplication](deduplication). 1. **Deduplication** — Merge duplicate nodes so each entity has exactly one canonical record. Conflict resolution operates on a single canonical entity; you must identify it before comparing what different sources say about it. See [Deduplication](/guides/deduplication).
2. **Conflict Detection** — Call `detect_entity_conflicts()` to surface all property disagreements at once, or `detect_value_conflicts()` to target a specific property. 2. **Conflict Detection** — Call `detect_entity_conflicts()` to surface all property disagreements at once, or `detect_value_conflicts()` to target a specific property.
3. **Resolution** — For each conflict, apply a strategy (`CREDIBILITY_WEIGHTED`, `MOST_RECENT`, `VOTING`, etc.) or route it for expert review (`EXPERT_REVIEW`). 3. **Resolution** — For each conflict, apply a strategy (`CREDIBILITY_WEIGHTED`, `MOST_RECENT`, `VOTING`, etc.) or route it for expert review (`EXPERT_REVIEW`).
4. **Persist Canonical Values** — Write resolved values back to your canonical entities or graph store. See [Persisting resolved values](#persisting-resolved-values). 4. **Persist Canonical Values** — Write resolved values back to your canonical entities or graph store. See [Persisting resolved values](#persisting-resolved-values).
@@ -696,8 +696,8 @@ Calling `set_resolution_rule()` for every entity-property pair just to apply the
## Related Guides ## Related Guides
- [Deduplication](deduplication) — remove duplicate nodes before running conflict detection - [Deduplication](/guides/deduplication) — remove duplicate nodes before running conflict detection
- [Provenance](provenance) — track which source each resolved value came from, and verify the audit trail cryptographically - [Provenance](/guides/provenance) — track which source each resolved value came from, and verify the audit trail cryptographically
- [SHACL Validation](/guides/shacl-validation) — enforce structural constraints after conflicts are resolved - [SHACL Validation](/guides/shacl-validation) — enforce structural constraints after conflicts are resolved
- [Change Management](/guides/change-management) — snapshot the graph before and after conflict resolution runs - [Change Management](/guides/change-management) — snapshot the graph before and after conflict resolution runs
- [Ontology Management](ontology) — align entity types to a shared vocabulary to reduce type conflicts at the schema level - [Ontology Management](/guides/ontology) — align entity types to a shared vocabulary to reduce type conflicts at the schema level
+5 -5
View File
@@ -489,7 +489,7 @@ context2.load("agent_state/")
## Common Pitfalls ## Common Pitfalls
**Duplicate entities.** Adding "APT-29", "APT29", and "Cozy Bear" as separate nodes fragments the graph when they should be one entity. Use consistent naming conventions upfront, or use `detect_duplicates()` and `EntityMerger` from the [Deduplication](deduplication) guide to merge them after ingestion. **Duplicate entities.** Adding "APT-29", "APT29", and "Cozy Bear" as separate nodes fragments the graph when they should be one entity. Use consistent naming conventions upfront, or use `detect_duplicates()` and `EntityMerger` from the [Deduplication](/guides/deduplication) guide to merge them after ingestion.
**Inconsistent naming conventions.** Mixing "ThreatActor", "threat_actor", and "Threat-Actor" as node types breaks queries that filter by type. Pick one convention and enforce it across all data sources. **Inconsistent naming conventions.** Mixing "ThreatActor", "threat_actor", and "Threat-Actor" as node types breaks queries that filter by type. Pick one convention and enforce it across all data sources.
@@ -706,8 +706,8 @@ for n in stress_reach:
- [Graph Analytics](/guides/graph-analytics) — centrality rankings, community detection, node embeddings, and link prediction on a populated `ContextGraph` - [Graph Analytics](/guides/graph-analytics) — centrality rankings, community detection, node embeddings, and link prediction on a populated `ContextGraph`
- [Decision Intelligence](/guides/decision-intelligence) — recording decisions as typed nodes, causal chain analysis, precedent search, and policy enforcement - [Decision Intelligence](/guides/decision-intelligence) — recording decisions as typed nodes, causal chain analysis, precedent search, and policy enforcement
- [Ingest](ingest) — loading data from PDFs, APIs, databases, STIX bundles, and RSS feeds into the graph - [Ingest](/guides/ingest) — loading data from PDFs, APIs, databases, STIX bundles, and RSS feeds into the graph
- [Deduplication](deduplication) — detecting and merging near-duplicate nodes before insertion to prevent graph fragmentation - [Deduplication](/guides/deduplication) — detecting and merging near-duplicate nodes before insertion to prevent graph fragmentation
- [Reasoning](reasoning) — temporal interval algebra (Allen relations), forward/backward chaining, and SPARQL over the knowledge graph - [Reasoning](/guides/reasoning) — temporal interval algebra (Allen relations), forward/backward chaining, and SPARQL over the knowledge graph
- [Ontology Management](ontology) — deriving formal OWL ontologies from `graph.to_dict()` for downstream reasoning engines - [Ontology Management](/guides/ontology) — deriving formal OWL ontologies from `graph.to_dict()` for downstream reasoning engines
- [Context Module Reference](../reference/context) — full API for `AgentContext`, `ContextGraph`, `ContextNode`, `ContextEdge` - [Context Module Reference](../reference/context) — full API for `AgentContext`, `ContextGraph`, `ContextNode`, `ContextEdge`
+1 -1
View File
@@ -664,6 +664,6 @@ results = context.find_precedents("APT29 infrastructure attribution", limit=5)
- [Context Graphs](/guides/context-graphs) — how `ContextGraph` stores decision nodes and causal edges - [Context Graphs](/guides/context-graphs) — how `ContextGraph` stores decision nodes and causal edges
- [Distance Intelligence](/guides/distance-intelligence) — `trace_decision_causality()` annotates causal chains with confidence decay and distance bands - [Distance Intelligence](/guides/distance-intelligence) — `trace_decision_causality()` annotates causal chains with confidence decay and distance bands
- [Provenance](provenance) — W3C PROV-O audit trail that wraps decision records in standards-compliant provenance - [Provenance](/guides/provenance) — W3C PROV-O audit trail that wraps decision records in standards-compliant provenance
- [MCP Server](/guides/mcp-server) — expose decision recording and precedent search to LLM agents via the `record_decision` and `find_precedents` tools - [MCP Server](/guides/mcp-server) — expose decision recording and precedent search to LLM agents via the `record_decision` and `find_precedents` tools
- [Change Management](/guides/change-management) — checkpoint decision state with `flush_checkpoint()` for versioned snapshots - [Change Management](/guides/change-management) — checkpoint decision state with `flush_checkpoint()` for versioned snapshots
+3 -3
View File
@@ -611,8 +611,8 @@ The similarity threshold controls sensitivity. Start at 0.7 and examine false po
## Related Guides ## Related Guides
- [Ingest Anything](ingest) — multi-source ingestion creates the duplicates this module resolves - [Ingest Anything](/guides/ingest) — multi-source ingestion creates the duplicates this module resolves
- [Context Graphs](/guides/context-graphs) — store deduplicated entities directly in the knowledge graph - [Context Graphs](/guides/context-graphs) — store deduplicated entities directly in the knowledge graph
- [Conflict Resolution](/guides/conflict-resolution) — after merging, reconcile disagreeing property values on the canonical entity - [Conflict Resolution](/guides/conflict-resolution) — after merging, reconcile disagreeing property values on the canonical entity
- [Provenance](provenance) — track merge lineage so every canonical entity traces back to its original sources - [Provenance](/guides/provenance) — track merge lineage so every canonical entity traces back to its original sources
- [Pipeline](pipeline) — chain ingest, deduplicate, and store as a `PipelineBuilder` workflow - [Pipeline](/guides/pipeline) — chain ingest, deduplicate, and store as a `PipelineBuilder` workflow
+1 -1
View File
@@ -561,4 +561,4 @@ for chain in chains:
- [Graph Analytics](/guides/graph-analytics) — centrality, community detection, Node2Vec embeddings, link prediction - [Graph Analytics](/guides/graph-analytics) — centrality, community detection, Node2Vec embeddings, link prediction
- [Agent Memory](/guides/agent-memory) — proximity-blended retrieval (`proximity_weight`) integrates distance intelligence into memory search - [Agent Memory](/guides/agent-memory) — proximity-blended retrieval (`proximity_weight`) integrates distance intelligence into memory search
- [Decision Intelligence](/guides/decision-intelligence) — `trace_decision_causality()` for causal chains with distance annotations - [Decision Intelligence](/guides/decision-intelligence) — `trace_decision_causality()` for causal chains with distance annotations
- [Reasoning & Rules](reasoning) — `TemporalReasoningEngine` for Allen interval algebra over time-bounded graph nodes - [Reasoning & Rules](/guides/reasoning) — `TemporalReasoningEngine` for Allen interval algebra over time-bounded graph nodes
+3 -3
View File
@@ -444,7 +444,7 @@ For semantic reasoning and ontology work, OWL/XML is the format — it is the on
## Related Guides ## Related Guides
- [Context Graphs](/guides/context-graphs) — the `ContextGraph` object whose `to_dict()` feeds all exports - [Context Graphs](/guides/context-graphs) — the `ContextGraph` object whose `to_dict()` feeds all exports
- [Ontology Management](ontology) — export OWL ontologies generated from your graph - [Ontology Management](/guides/ontology) — export OWL ontologies generated from your graph
- [Reasoning & Rules](reasoning) — reasoning results can be exported as RDF triples - [Reasoning & Rules](/guides/reasoning) — reasoning results can be exported as RDF triples
- [Change Management](/guides/change-management) — snapshot a graph before exporting to prove the export was made from a verified state - [Change Management](/guides/change-management) — snapshot a graph before exporting to prove the export was made from a verified state
- [Pipeline](pipeline) — chain ingest, extract, and export in a single `PipelineBuilder` - [Pipeline](/guides/pipeline) — chain ingest, extract, and export in a single `PipelineBuilder`
+1 -1
View File
@@ -539,6 +539,6 @@ print(f"\n{len(result['communities'])} exposure clusters "
## Related Guides ## Related Guides
- [Context Graphs](/guides/context-graphs) — building and querying the underlying `ContextGraph` - [Context Graphs](/guides/context-graphs) — building and querying the underlying `ContextGraph`
- [Visualization](visualization) — render centrality rankings and community clusters as interactive dashboards - [Visualization](/guides/visualization) — render centrality rankings and community clusters as interactive dashboards
- [Decision Intelligence](/guides/decision-intelligence) — link prediction and structural similarity applied to decision nodes - [Decision Intelligence](/guides/decision-intelligence) — link prediction and structural similarity applied to decision nodes
- [GraphRAG](/guides/graphrag) — using analytics results to ground LLM generation in the most contextually relevant subgraph - [GraphRAG](/guides/graphrag) — using analytics results to ground LLM generation in the most contextually relevant subgraph
+2 -2
View File
@@ -950,9 +950,9 @@ print(f"Compliance graph: {graph.stats()['node_count']} nodes, "
## Related Guides ## Related Guides
- [Pipeline](pipeline) — chain ingest steps with `PipelineBuilder` for automated, retryable, parallelised workflows - [Pipeline](/guides/pipeline) — chain ingest steps with `PipelineBuilder` for automated, retryable, parallelised workflows
- [Context Graphs](/guides/context-graphs) — storing and querying the entities you ingest as a typed property graph - [Context Graphs](/guides/context-graphs) — storing and querying the entities you ingest as a typed property graph
- [Semantic Extraction](/guides/semantic-extraction) — NER, relation extraction, and triplet extraction from ingested text - [Semantic Extraction](/guides/semantic-extraction) — NER, relation extraction, and triplet extraction from ingested text
- [Provenance](provenance) — tracking the origin document, confidence score, and ingestion timestamp for every extracted entity - [Provenance](/guides/provenance) — tracking the origin document, confidence score, and ingestion timestamp for every extracted entity
- [Databricks Integration](../integrations/databricks) — Unity Catalog setup, PAT/OAuth M2M authentication, and lineage introspection - [Databricks Integration](../integrations/databricks) — Unity Catalog setup, PAT/OAuth M2M authentication, and lineage introspection
- [Snowflake Integration](../integrations/snowflake) — warehouse setup and password/key-pair/OAuth authentication - [Snowflake Integration](../integrations/snowflake) — warehouse setup and password/key-pair/OAuth authentication
+3 -3
View File
@@ -342,8 +342,8 @@ The result is a fully auditable credit decision trail with precedent links, read
## Related Guides ## Related Guides
- [Reasoning & Rules](reasoning) — the engine behind the `run_reasoning` tool - [Reasoning & Rules](/guides/reasoning) — the engine behind the `run_reasoning` tool
- [Decision Intelligence](/guides/decision-intelligence) — how decisions are stored as causal graph nodes - [Decision Intelligence](/guides/decision-intelligence) — how decisions are stored as causal graph nodes
- [Context Graphs](/guides/context-graphs) — the graph that `add_entity` and `add_relationship` write to - [Context Graphs](/guides/context-graphs) — the graph that `add_entity` and `add_relationship` write to
- [Export & Serialization](export) — all export formats available via `export_graph` - [Export & Serialization](/guides/export) — all export formats available via `export_graph`
- [Ontology Management](ontology) — generate OWL ontologies from the graph built via MCP - [Ontology Management](/guides/ontology) — generate OWL ontologies from the graph built via MCP
+2 -2
View File
@@ -504,7 +504,7 @@ else:
## Related Guides ## Related Guides
- [SHACL Validation](/guides/shacl-validation) — generate W3C SHACL constraint shapes from your ontology and validate live graph data against them - [SHACL Validation](/guides/shacl-validation) — generate W3C SHACL constraint shapes from your ontology and validate live graph data against them
- [Reasoning & Rules](reasoning) — apply forward/backward-chaining rules over your ontology to derive new facts - [Reasoning & Rules](/guides/reasoning) — apply forward/backward-chaining rules over your ontology to derive new facts
- [Export & Serialization](export) — export graphs to RDF, GraphML, CSV, and Neo4j Cypher - [Export & Serialization](/guides/export) — export graphs to RDF, GraphML, CSV, and Neo4j Cypher
- [Semantic Extraction](/guides/semantic-extraction) — extract entities and relationships that feed ontology generation - [Semantic Extraction](/guides/semantic-extraction) — extract entities and relationships that feed ontology generation
- [Context Graphs](/guides/context-graphs) — the knowledge graph that ontology generation reads from - [Context Graphs](/guides/context-graphs) — the knowledge graph that ontology generation reads from
+2 -2
View File
@@ -718,7 +718,7 @@ print(f"Compliance delta update: {result.output}")
## Related Guides ## Related Guides
- [Ingest](ingest) — all source types for the ingest step: PDFs, APIs, databases, RSS feeds, STIX directories, and streams - [Ingest](/guides/ingest) — all source types for the ingest step: PDFs, APIs, databases, RSS feeds, STIX directories, and streams
- [Semantic Extraction](/guides/semantic-extraction) — NER, relation extraction, triplet extraction, and event detection for the extract step - [Semantic Extraction](/guides/semantic-extraction) — NER, relation extraction, triplet extraction, and event detection for the extract step
- [Context Graphs](/guides/context-graphs) — building and querying the `ContextGraph` that the store step populates - [Context Graphs](/guides/context-graphs) — building and querying the `ContextGraph` that the store step populates
- [Provenance](provenance) — tracking the origin document, confidence score, and pipeline run ID for every extracted entity - [Provenance](/guides/provenance) — tracking the origin document, confidence score, and pipeline run ID for every extracted entity
+2 -2
View File
@@ -663,8 +663,8 @@ print("Policy updated to v2.4.0")
## Related Guides ## Related Guides
- [Decision Intelligence](/guides/decision-intelligence) — `record_decision()`, causal chains, and precedent search — the decisions that `check_compliance()` evaluates - [Decision Intelligence](/guides/decision-intelligence) — `record_decision()`, causal chains, and precedent search — the decisions that `check_compliance()` evaluates
- [Reasoning & Rules](reasoning) — complement policy rules with formal inference for logical conflict detection - [Reasoning & Rules](/guides/reasoning) — complement policy rules with formal inference for logical conflict detection
- [SHACL Validation](/guides/shacl-validation) — enforce structural constraints on policy nodes themselves - [SHACL Validation](/guides/shacl-validation) — enforce structural constraints on policy nodes themselves
- [Change Management](/guides/change-management) — version-snapshot the policy graph alongside the knowledge graph - [Change Management](/guides/change-management) — version-snapshot the policy graph alongside the knowledge graph
- [Provenance](provenance) — W3C PROV-O lineage for every policy decision and exception - [Provenance](/guides/provenance) — W3C PROV-O lineage for every policy decision and exception
- [MCP Server](/guides/mcp-server) — expose `record_decision` and `find_precedents` as MCP tools for AI agents - [MCP Server](/guides/mcp-server) — expose `record_decision` and `find_precedents` as MCP tools for AI agents
+1 -1
View File
@@ -661,5 +661,5 @@ Note: the banking example above passes `agent_id="credit_data_service_v2"` to `t
- [Semantic Extraction](/guides/semantic-extraction) — the NER and relation extraction pipeline that auto-generates provenance entries for every extracted entity - [Semantic Extraction](/guides/semantic-extraction) — the NER and relation extraction pipeline that auto-generates provenance entries for every extracted entity
- [Conflict Resolution](/guides/conflict-resolution) — provenance property sources feed directly into conflict detection; every resolved value is traceable to its source - [Conflict Resolution](/guides/conflict-resolution) — provenance property sources feed directly into conflict detection; every resolved value is traceable to its source
- [Deduplication](deduplication) — merge operations are recorded in merge history; pair with provenance for a complete lineage from source to canonical entity - [Deduplication](/guides/deduplication) — merge operations are recorded in merge history; pair with provenance for a complete lineage from source to canonical entity
- [Provenance Reference](../reference/provenance) — full storage backend API, `InMemoryStorage`, `SQLiteStorage`, and `ProvenanceEntry` schema - [Provenance Reference](../reference/provenance) — full storage backend API, `InMemoryStorage`, `SQLiteStorage`, and `ProvenanceEntry` schema
+1 -1
View File
@@ -840,7 +840,7 @@ if proof:
- [Semantic Extraction](/guides/semantic-extraction) — extract the entities and relationships that populate the graph facts you reason over - [Semantic Extraction](/guides/semantic-extraction) — extract the entities and relationships that populate the graph facts you reason over
- [GraphRAG](/guides/graphrag) — retrieve graph-grounded context for LLM responses - [GraphRAG](/guides/graphrag) — retrieve graph-grounded context for LLM responses
- [Ontology Management](ontology) — generate OWL ontologies to give your rules formal semantics - [Ontology Management](/guides/ontology) — generate OWL ontologies to give your rules formal semantics
- [Decision Intelligence](/guides/decision-intelligence) — record and trace inferred decisions through the full causal chain - [Decision Intelligence](/guides/decision-intelligence) — record and trace inferred decisions through the full causal chain
- [Context Graphs](/guides/context-graphs) — the knowledge graph that reasoning operates over - [Context Graphs](/guides/context-graphs) — the knowledge graph that reasoning operates over
- [MCP Server](/guides/mcp-server) — expose `run_reasoning` as a tool for Claude and other agents - [MCP Server](/guides/mcp-server) — expose `run_reasoning` as a tool for Claude and other agents
+3 -3
View File
@@ -71,7 +71,7 @@ This pipeline transforms documents like "APT29 deployed HAMMERTOSS malware targe
`semantica.semantic_extract` turns unstructured text into structured graph-ready output: it identifies named entities, extracts relationships between them, detects time-anchored events, resolves coreferences, and serialises everything as RDF triplets. Use it to populate a `ContextGraph` from raw documents — intelligence reports, clinical notes, regulatory filings, or any free-text corpus. `semantica.semantic_extract` turns unstructured text into structured graph-ready output: it identifies named entities, extracts relationships between them, detects time-anchored events, resolves coreferences, and serialises everything as RDF triplets. Use it to populate a `ContextGraph` from raw documents — intelligence reports, clinical notes, regulatory filings, or any free-text corpus.
<Info> <Info>
Extracted entities and relationships feed into `ContextGraph` via `AgentContext.store()`. For how they are attributed back to source documents, see the [Provenance Guide](provenance). For how the populated graph is queried and traversed, see [Context Graphs](/guides/context-graphs). Extracted entities and relationships feed into `ContextGraph` via `AgentContext.store()`. For how they are attributed back to source documents, see the [Provenance Guide](/guides/provenance). For how the populated graph is queried and traversed, see [Context Graphs](/guides/context-graphs).
</Info> </Info>
## Step 1 — Named Entity Recognition: who and what is in the text ## Step 1 — Named Entity Recognition: who and what is in the text
@@ -668,9 +668,9 @@ The fallback behaviour is automatic: if the primary method returns an empty list
## Related Guides ## Related Guides
- [Provenance Guide](provenance) — track every extracted entity and chunk back to its source document - [Provenance Guide](/guides/provenance) — track every extracted entity and chunk back to its source document
- [Agent Memory Guide](/guides/agent-memory) — store extracted knowledge as searchable agent memories with graph enrichment - [Agent Memory Guide](/guides/agent-memory) — store extracted knowledge as searchable agent memories with graph enrichment
- [Context Graphs Guide](/guides/context-graphs) — how extracted entities populate `ContextGraph` nodes and edges - [Context Graphs Guide](/guides/context-graphs) — how extracted entities populate `ContextGraph` nodes and edges
- [GraphRAG Guide](/guides/graphrag) — retrieve facts from the populated graph to ground LLM responses - [GraphRAG Guide](/guides/graphrag) — retrieve facts from the populated graph to ground LLM responses
- [Reasoning Guide](reasoning) — derive new facts, run SPARQL queries, and apply inference rules over the extracted graph - [Reasoning Guide](/guides/reasoning) — derive new facts, run SPARQL queries, and apply inference rules over the extracted graph
- [Semantic Extract Reference](../reference/semantic_extract) — full API for all extractor classes, providers, and validators - [Semantic Extract Reference](../reference/semantic_extract) — full API for all extractor classes, providers, and validators
+3 -3
View File
@@ -753,8 +753,8 @@ def validate_before_publish(data_graph_str: str, ontology: dict) -> None:
## Related Guides ## Related Guides
- [Ontology Management](ontology) — generate the OWL ontology that SHACL shapes are derived from - [Ontology Management](/guides/ontology) — generate the OWL ontology that SHACL shapes are derived from
- [Reasoning & Rules](reasoning) — complement SHACL structural constraints with logical inference rules - [Reasoning & Rules](/guides/reasoning) — complement SHACL structural constraints with logical inference rules
- [Export & Serialization](export) — serialize graph data to Turtle/RDF/XML for `run_shacl_validation` input - [Export & Serialization](/guides/export) — serialize graph data to Turtle/RDF/XML for `run_shacl_validation` input
- [Conflict Resolution](/guides/conflict-resolution) — detect and resolve data conflicts before SHACL validation - [Conflict Resolution](/guides/conflict-resolution) — detect and resolve data conflicts before SHACL validation
- [Change Management](/guides/change-management) — version-gate SHACL shapes alongside ontology versions - [Change Management](/guides/change-management) — version-gate SHACL shapes alongside ontology versions
+2 -2
View File
@@ -615,7 +615,7 @@ fig.write_html("out.html") # manual export
## Related Guides ## Related Guides
- [Context Graphs](/guides/context-graphs) — `graph.to_dict()` is the primary input for `KGVisualizer` - [Context Graphs](/guides/context-graphs) — `graph.to_dict()` is the primary input for `KGVisualizer`
- [Ontology Management](ontology) — `OntologyVisualizer` renders ontologies produced by `OntologyGenerator` - [Ontology Management](/guides/ontology) — `OntologyVisualizer` renders ontologies produced by `OntologyGenerator`
- [Change Management](/guides/change-management) — `TemporalVersionManager` snapshots feed `visualize_metrics_evolution()` and `visualize_snapshot_comparison()` - [Change Management](/guides/change-management) — `TemporalVersionManager` snapshots feed `visualize_metrics_evolution()` and `visualize_snapshot_comparison()`
- [Graph Analytics](/guides/graph-analytics) — centrality scores, community dicts, and connectivity results that feed the `AnalyticsVisualizer` - [Graph Analytics](/guides/graph-analytics) — centrality scores, community dicts, and connectivity results that feed the `AnalyticsVisualizer`
- [Export & Serialization](export) — export the same graph to GraphML, GEXF, or DOT for Gephi and Graphviz - [Export & Serialization](/guides/export) — export the same graph to GraphML, GEXF, or DOT for Gephi and Graphviz
+342
View File
@@ -0,0 +1,342 @@
---
title: "Amazon Redshift Integration"
description: "Ingest structured data from Amazon Redshift tables and queries into Semantica's KG pipeline."
icon: "database"
---
> Extract data from Amazon Redshift into Semantica with password/native or IAM-role authentication, using the PostgreSQL-compatible wire protocol.
## Installation
```bash
# Install with Redshift support
pip install "semantica[db-redshift]"
# Or install the connector separately
pip install redshift-connector>=2.0.0
```
`redshift-connector` is an optional dependency. A plain `pip install semantica` never pulls it in, and `import semantica.ingest` never loads it eagerly — the SDK is imported only when you first use `RedshiftConnector` or `RedshiftIngestor`.
<Note>
This connector uses the Redshift database wire protocol for read ingestion. COPY, UNLOAD, S3 integration, Spectrum external tables, and the Redshift Data API are outside the current scope of this integration.
</Note>
## Basic Usage
```python
from semantica.ingest import RedshiftIngestor
import os
ingestor = RedshiftIngestor(
host=os.getenv("REDSHIFT_HOST"),
database=os.getenv("REDSHIFT_DATABASE"),
user=os.getenv("REDSHIFT_USER"),
password=os.getenv("REDSHIFT_PASSWORD"),
)
data = ingestor.ingest_table("customers")
print(f"Retrieved {data.row_count} rows — columns: {data.columns}")
```
<Tip>
Use environment variables (or a `.env` file with `python-dotenv`) to keep credentials out of source code. `RedshiftIngestor()` with no arguments reads from `REDSHIFT_*` environment variables automatically.
</Tip>
## Authentication
<Tabs>
<Tab title="Password / Native">
The standard Redshift database username and password:
```python
import os
from semantica.ingest import RedshiftIngestor
ingestor = RedshiftIngestor(
host=os.getenv("REDSHIFT_HOST"), # e.g. cluster.abc.us-east-1.redshift.amazonaws.com
database=os.getenv("REDSHIFT_DATABASE"),
user=os.getenv("REDSHIFT_USER"),
password=os.getenv("REDSHIFT_PASSWORD"),
port=5439, # default; omit to use the default
ssl=True, # default
)
```
Required environment variables:
```bash
export REDSHIFT_HOST="cluster.abc.us-east-1.redshift.amazonaws.com"
export REDSHIFT_DATABASE="dev"
export REDSHIFT_USER="awsuser"
export REDSHIFT_PASSWORD="your-password"
```
</Tab>
<Tab title="IAM Role (Recommended for AWS)">
Use `iam=True` to obtain temporary credentials via
`GetClusterCredentials`. The connector delegates credential resolution
entirely to `redshift-connector` / boto3 — Semantica never calls AWS
APIs directly.
**Profile-based** (reads `~/.aws/credentials`):
```python
import os
from semantica.ingest import RedshiftIngestor
ingestor = RedshiftIngestor(
host=os.getenv("REDSHIFT_HOST"),
database=os.getenv("REDSHIFT_DATABASE"),
iam=True,
db_user=os.getenv("REDSHIFT_DB_USER"),
cluster_identifier=os.getenv("REDSHIFT_CLUSTER_IDENTIFIER"),
profile="default", # AWS credentials-file profile
)
```
**Explicit AWS credentials** (e.g. for CI/CD or IAM roles with short-lived keys):
```python
ingestor = RedshiftIngestor(
host=os.getenv("REDSHIFT_HOST"),
database=os.getenv("REDSHIFT_DATABASE"),
iam=True,
db_user=os.getenv("REDSHIFT_DB_USER"),
cluster_identifier=os.getenv("REDSHIFT_CLUSTER_IDENTIFIER"),
region=os.getenv("REDSHIFT_REGION"),
access_key_id=os.getenv("REDSHIFT_ACCESS_KEY_ID"),
secret_access_key=os.getenv("REDSHIFT_SECRET_ACCESS_KEY"),
session_token=os.getenv("REDSHIFT_SESSION_TOKEN"), # only for temporary creds
)
```
When neither `profile` nor explicit keys are supplied, `redshift-connector`
falls back to the standard AWS credential chain: `AWS_*` environment
variables, instance-profile metadata, etc.
Required for IAM mode: `host`, `database`, `db_user`, `cluster_identifier`.
The `region` parameter is optional when it can be inferred from the
credential chain or the cluster endpoint.
</Tab>
</Tabs>
## Environment Variables
All constructor parameters have `REDSHIFT_*` environment-variable fallbacks.
Explicit constructor values always take precedence.
| Variable | Parameter | Default |
|---|---|---|
| `REDSHIFT_HOST` | `host` | — |
| `REDSHIFT_DATABASE` | `database` | — |
| `REDSHIFT_USER` | `user` | — |
| `REDSHIFT_PASSWORD` | `password` | — |
| `REDSHIFT_PORT` | `port` | `5439` |
| `REDSHIFT_SCHEMA` | `schema` | `"public"` |
| `REDSHIFT_DB_USER` | `db_user` | — |
| `REDSHIFT_CLUSTER_IDENTIFIER` | `cluster_identifier` | — |
| `REDSHIFT_REGION` | `region` | — |
| `REDSHIFT_PROFILE` | `profile` | — |
| `REDSHIFT_ACCESS_KEY_ID` | `access_key_id` | — |
| `REDSHIFT_SECRET_ACCESS_KEY` | `secret_access_key` | — |
| `REDSHIFT_SESSION_TOKEN` | `session_token` | — |
## Querying
### Ingest a table
```python
data = ingestor.ingest_table("orders")
print(f"{data.row_count} rows, columns: {data.columns}")
```
### Schema, filters, and pagination
```python
data = ingestor.ingest_table(
"orders",
schema="sales", # defaults to the ingestor's schema attribute ("public")
database="analytics", # defaults to the connector's database
where="status = 'shipped' AND total > 100",
order_by="created_at DESC",
limit=5000,
offset=0,
)
```
<Warning>
`where` and `order_by` accept raw SQL fragments and must be trusted, operator-controlled input. Do not pass raw end-user strings here. They are validated against a blocklist that rejects statement separators, `UNION`, DML/DDL keywords, and time-based injection patterns, but this is not a full parser.
</Warning>
### Custom SQL
```python
data = ingestor.ingest_query("""
SELECT customer_id, SUM(total) AS lifetime_value
FROM sales.orders
WHERE status = 'completed'
GROUP BY customer_id
ORDER BY lifetime_value DESC
LIMIT 1000
""")
print(f"{data.row_count} rows")
```
### Parameterized queries
Use `%s` placeholders (DB-API 2.0 `format` paramstyle, which is the default for `redshift-connector`):
```python
data = ingestor.ingest_query(
"SELECT id, name FROM users WHERE region = %s AND active = %s",
params=("us-east-1", True),
)
```
### Batch fetching for large result sets
Use `batch_size` to control the driver fetch size — rows are fetched from
the server in chunks of that size rather than all at once, and each chunk is
converted immediately before the next is requested:
```python
data = ingestor.ingest_query(
"SELECT * FROM large_events_table",
batch_size=10000,
)
```
`batch_size` controls how many rows the driver reads from Redshift per
round-trip. The returned `RedshiftData.data` list still contains all matching
rows; use `LIMIT`/`OFFSET` in the query itself if you need to cap the total
result size.
## Schema Discovery
### List base tables in a schema
```python
tables = ingestor.list_tables(schema="public")
print(tables) # ["customers", "orders", "products", ...]
```
Views are excluded; only base tables are returned.
### Inspect column metadata
```python
schema = ingestor.get_table_schema("customers", schema="public")
for col in schema["columns"]:
print(f"{col['name']}: {col['type']} (nullable={col['nullable']})")
print("Primary keys:", schema["primary_keys"])
```
Each column dict contains:
| Key | Type | Description |
|---|---|---|
| `name` | `str` | Column name |
| `type` | `str` | Redshift data type (e.g. `"integer"`, `"character varying"`) |
| `nullable` | `bool` | Whether the column accepts `NULL` |
`primary_keys` is a list of column-name strings (empty list when no primary key is defined).
## Export as Semantica Documents
Convert ingested rows to the document format that `GraphBuilder` consumes:
```python
documents = ingestor.export_as_documents(
data,
id_field="customer_id", # column used as document ID; defaults to "id"
text_fields=["name", "notes"], # columns joined as document text; omit to auto-select
)
print(f"Created {len(documents)} documents")
# Each document:
# {
# "id": "12345",
# "text": "Alice Acme customer notes here",
# "metadata": {
# "source": "redshift",
# "table": "customers",
# "database": "analytics",
# "schema": "public",
# "row_data": { ... full cleaned row ... }
# }
# }
```
**ID resolution**: `str(row.get(id_field, row_index))` — the integer row index is used as a deterministic fallback when the `id_field` column is absent.
**Text when `text_fields` is provided**: each non-`None` field value is converted to a string and joined with a single space.
**Text when `text_fields=None`**: only columns whose values are already `str` type are joined. Integer, float, boolean, and `None` values are excluded, matching the Snowflake and Databricks connector behavior.
Pass the documents directly to `GraphBuilder`:
```python
from semantica.kg import GraphBuilder
builder = GraphBuilder()
graph = builder.build(documents)
print(f"Entities: {graph['metadata']['num_entities']}")
```
## Context Manager
Prefer the context manager for jobs that run multiple queries — it opens one connection on entry and closes it on exit, so every call inside the `with` block reuses the same authenticated session:
```python
with RedshiftIngestor(
host=os.getenv("REDSHIFT_HOST"),
database=os.getenv("REDSHIFT_DATABASE"),
user=os.getenv("REDSHIFT_USER"),
password=os.getenv("REDSHIFT_PASSWORD"),
) as ingestor:
customers = ingestor.ingest_table("customers", limit=50000)
orders = ingestor.ingest_table("orders", limit=50000)
schema = ingestor.get_table_schema("customers")
tables = ingestor.list_tables()
# Connection closed automatically on exit, even if an exception is raised.
```
Standalone calls (without `with`) open and close a transient connection per call.
## Connection Test
```python
from semantica.ingest import RedshiftConnector
connector = RedshiftConnector(
host="cluster.abc.us-east-1.redshift.amazonaws.com",
database="dev",
user="awsuser",
password="your-password",
)
if connector.test_connection():
print("Connection OK")
else:
print("Connection failed — check host, credentials, and network access")
```
## See Also
- [Ingest Module](../reference/ingest) — Full `RedshiftIngestor` reference and all other ingestors.
- [Snowflake Integration](/integrations/snowflake) — SQL data warehouse connector with similar table/query ingestion.
- [Databricks Integration](/integrations/databricks) — Delta Lake / Unity Catalog connector.
- [Pipeline](../reference/pipeline) — Use Redshift ingestion as a pipeline step.
- [Installation](../installation) — All optional dependency extras.
- [Knowledge Graph](../reference/kg) — Build a knowledge graph from ingested Redshift data.
+2 -2
View File
@@ -350,7 +350,7 @@ for record in history:
</Accordion> </Accordion>
</AccordionGroup> </AccordionGroup>
- [Provenance](provenance) — W3C PROV-O lineage tracking. - [Provenance](/reference/provenance) — W3C PROV-O lineage tracking.
- [Knowledge Graph](/reference/kg) — The graph being versioned. - [Knowledge Graph](/reference/kg) — The graph being versioned.
- [Export](export) — Export versioned snapshots. - [Export](/reference/export) — Export versioned snapshots.
- [Conflicts](/reference/conflicts) — Detect conflicts introduced between versions. - [Conflicts](/reference/conflicts) — Detect conflicts introduced between versions.
+4 -4
View File
@@ -317,7 +317,7 @@ chain = tracker.get_traceability_chain("apple_inc")
</Warning> </Warning>
<Tip> <Tip>
**Combine with provenance.** The `SourceTracker` feeds directly into the [Provenance](provenance) module's audit trail. If you need to explain how a resolved value was chosen, provenance records give you the full chain. **Combine with provenance.** The `SourceTracker` feeds directly into the [Provenance](/reference/provenance) module's audit trail. If you need to explain how a resolved value was chosen, provenance records give you the full chain.
</Tip> </Tip>
## ConflictAnalyzer ## ConflictAnalyzer
@@ -450,7 +450,7 @@ class InvestigationStep:
</Accordion> </Accordion>
</AccordionGroup> </AccordionGroup>
- [Deduplication](deduplication) — Resolve duplicate entities before conflict detection. - [Deduplication](/reference/deduplication) — Resolve duplicate entities before conflict detection.
- [Ontology](ontology) — Logical conflicts use SHACL shapes and ontology axioms. - [Ontology](/reference/ontology) — Logical conflicts use SHACL shapes and ontology axioms.
- [Provenance](provenance) — Track which source each conflicting fact came from. - [Provenance](/reference/provenance) — Track which source each conflicting fact came from.
- [Knowledge Graph](/reference/kg) — The graph being checked for conflicts. - [Knowledge Graph](/reference/kg) — The graph being checked for conflicts.
+1 -1
View File
@@ -226,7 +226,7 @@ result = build_knowledge_base(sources=["doc.pdf"], method="fast")
Use `Semantica` and `LifecycleManager` only when building a long-running application (e.g. a FastAPI service) that needs ordered startup, health checks, and graceful shutdown. For scripts and notebooks, use individual modules directly. Use `Semantica` and `LifecycleManager` only when building a long-running application (e.g. a FastAPI service) that needs ordered startup, health checks, and graceful shutdown. For scripts and notebooks, use individual modules directly.
</Tip> </Tip>
- [Pipeline](pipeline) — Pipeline execution and step orchestration. - [Pipeline](/reference/pipeline) — Pipeline execution and step orchestration.
- [Utils](/reference/utils) — Shared utilities used by Core internally. - [Utils](/reference/utils) — Shared utilities used by Core internally.
- [Getting Started](../getting-started) — Learn the basics before using Core. - [Getting Started](../getting-started) — Learn the basics before using Core.
- [LLMs](/reference/llms) — Configure LLM providers via ConfigManager. - [LLMs](/reference/llms) — Configure LLM providers via ConfigManager.
+1 -1
View File
@@ -440,4 +440,4 @@ result = calculate_similarity(entity_a, entity_b, method="drug_name")
- [Conflicts](/reference/conflicts) — Detect value conflicts between non-duplicate entities. - [Conflicts](/reference/conflicts) — Detect value conflicts between non-duplicate entities.
- [Knowledge Graph](/reference/kg) — GraphBuilder uses deduplication during construction. - [Knowledge Graph](/reference/kg) — GraphBuilder uses deduplication during construction.
- [Normalize](/reference/normalize) — Normalize entity names before deduplication. - [Normalize](/reference/normalize) — Normalize entity names before deduplication.
- [Provenance](provenance) — Track merged entity lineage. - [Provenance](/reference/provenance) — Track merged entity lineage.
+1 -1
View File
@@ -609,5 +609,5 @@ The Knowledge Explorer embeds Distance Intelligence directly in the browser dash
- [Context Module](/reference/context) — `ContextGraph.get_neighbors()` and proximity-blended retrieval. - [Context Module](/reference/context) — `ContextGraph.get_neighbors()` and proximity-blended retrieval.
- [Knowledge Graph Module](/reference/kg) — `NodeEmbedder`, `SimilarityCalculator`, and graph analytics. - [Knowledge Graph Module](/reference/kg) — `NodeEmbedder`, `SimilarityCalculator`, and graph analytics.
- [Visualization](visualization) — Programmatic distance heatmaps and ego-mode graph renders. - [Visualization](/reference/visualization) — Programmatic distance heatmaps and ego-mode graph renders.
- [Explorer](/reference/explorer) — Knowledge Explorer with built-in Distance Intelligence dashboard. - [Explorer](/reference/explorer) — Knowledge Explorer with built-in Distance Intelligence dashboard.
+3 -3
View File
@@ -404,6 +404,6 @@ Semantic neighborhood requires node embeddings stored in node properties (keys `
Session state is in-memory only. Use `POST /api/export` to save a JSON snapshot before shutting down. Session state is in-memory only. Use `POST /api/export` to save a JSON snapshot before shutting down.
- [Context](/reference/context) — Build and save the ContextGraph that Explorer loads. - [Context](/reference/context) — Build and save the ContextGraph that Explorer loads.
- [Ontology](ontology) — Programmatic ontology management and SHACL generation. - [Ontology](/reference/ontology) — Programmatic ontology management and SHACL generation.
- [Visualization](visualization) — Programmatic graph rendering without the Explorer server. - [Visualization](/reference/visualization) — Programmatic graph rendering without the Explorer server.
- [Export](export) — Export to RDF, Parquet, and other formats without launching a server. - [Export](/reference/export) — Export to RDF, Parquet, and other formats without launching a server.
+3 -3
View File
@@ -395,6 +395,6 @@ The `export_csv` convenience function delegates to `CSVExporter.export()`. For p
</Tip> </Tip>
- [Triplet Store](/reference/triplet_store) — Store RDF exports in a SPARQL-queryable backend. - [Triplet Store](/reference/triplet_store) — Store RDF exports in a SPARQL-queryable backend.
- [Ontology](ontology) — Export OWL ontologies. - [Ontology](/reference/ontology) — Export OWL ontologies.
- [Provenance](provenance) — Include provenance metadata in RDF exports. - [Provenance](/reference/provenance) — Include provenance metadata in RDF exports.
- [Pipeline](pipeline) — Add export as a final pipeline step. - [Pipeline](/reference/pipeline) — Add export as a final pipeline step.
+1 -1
View File
@@ -505,5 +505,5 @@ stats = store.get_stats()
- [KG Module](/reference/kg) — Build the graph before persisting it. - [KG Module](/reference/kg) — Build the graph before persisting it.
- [Triplet Store](/reference/triplet_store) — RDF triple store for semantic web and SPARQL queries. - [Triplet Store](/reference/triplet_store) — RDF triple store for semantic web and SPARQL queries.
- [Visualization](visualization) — Visualize graphs stored in any backend. - [Visualization](/reference/visualization) — Visualize graphs stored in any backend.
- [Context](/reference/context) — AgentContext uses GraphStore for memory retrieval. - [Context](/reference/context) — AgentContext uses GraphStore for memory retrieval.
+2 -2
View File
@@ -647,7 +647,7 @@ result = ingest_file("source_path", method="my_format")
``` ```
- [Parse](/reference/parse) — Parse raw sources into structured text and tables. - [Parse](/reference/parse) — Parse raw sources into structured text and tables.
- [Pipeline](pipeline) — Orchestrate ingest as the first pipeline step. - [Pipeline](/reference/pipeline) — Orchestrate ingest as the first pipeline step.
- [Snowflake Integration](../integrations/snowflake) — Snowflake-specific setup and authentication guide. - [Snowflake Integration](../integrations/snowflake) — Snowflake-specific setup and authentication guide.
- [Databricks Integration](../integrations/databricks) — Databricks Unity Catalog setup, authentication, and lineage guide. - [Databricks Integration](../integrations/databricks) — Databricks Unity Catalog setup, authentication, and lineage guide.
- [Provenance](provenance) — Track lineage from ingest through to inference. - [Provenance](/reference/provenance) — Track lineage from ingest through to inference.
+1 -1
View File
@@ -477,7 +477,7 @@ kg:
- [Graph Store](/reference/graph_store) — Persist graphs in Neo4j, FalkorDB, or Apache AGE. - [Graph Store](/reference/graph_store) — Persist graphs in Neo4j, FalkorDB, or Apache AGE.
- [Semantic Extract](/reference/semantic_extract) — Source of entities and relationships fed to GraphBuilder. - [Semantic Extract](/reference/semantic_extract) — Source of entities and relationships fed to GraphBuilder.
- [Visualization](visualization) — Visualize knowledge graphs interactively. - [Visualization](/reference/visualization) — Visualize knowledge graphs interactively.
- [Conflicts](/reference/conflicts) — Conflict detection and resolution. - [Conflicts](/reference/conflicts) — Conflict detection and resolution.
### Cookbooks ### Cookbooks
+2 -2
View File
@@ -27,7 +27,7 @@ from semantica.llms import Groq, OpenAI, LiteLLM, HuggingFaceLLM
| `HuggingFaceLLM` | Local HuggingFace Transformers | None (local) | | `HuggingFaceLLM` | Local HuggingFace Transformers | None (local) |
<Tip> <Tip>
**Anthropic, Gemini, Ollama, DeepSeek, Azure, Bedrock, Cohere, and 90+ others** are all available via `LiteLLM` using their model-string prefix. See the [LiteLLM section](#litellm-100-providers) below. **Anthropic, Gemini, Ollama, DeepSeek, Azure, Bedrock, Cohere, and 90+ others** are all available via `LiteLLM` using their model-string prefix. See the [LiteLLM section](#litellm-100+-providers) below.
</Tip> </Tip>
## What You Get ## What You Get
@@ -441,5 +441,5 @@ extractor = NERExtractor(
- [Semantic Extract](/reference/semantic_extract) — Use LLMs for NER and relation extraction. - [Semantic Extract](/reference/semantic_extract) — Use LLMs for NER and relation extraction.
- [Agno Integration](../integrations/agno) — LLM providers in Agno multi-agent teams. - [Agno Integration](../integrations/agno) — LLM providers in Agno multi-agent teams.
- [Reasoning](reasoning) — LLM-backed deductive and abductive reasoning. - [Reasoning](/reference/reasoning) — LLM-backed deductive and abductive reasoning.
- [Context](/reference/context) — GraphRAG uses LLMs for reasoning over knowledge graphs. - [Context](/reference/context) — GraphRAG uses LLMs for reasoning over knowledge graphs.
+1 -1
View File
@@ -495,5 +495,5 @@ The MCP server exposes three readable resources:
- [Context](/reference/context) — The ContextGraph that the MCP server operates on. - [Context](/reference/context) — The ContextGraph that the MCP server operates on.
- [Semantic Extract](/reference/semantic_extract) — NER and relation extraction powering the MCP tools. - [Semantic Extract](/reference/semantic_extract) — NER and relation extraction powering the MCP tools.
- [Reasoning](reasoning) — Forward-chaining engine behind run_reasoning. - [Reasoning](/reference/reasoning) — Forward-chaining engine behind run_reasoning.
- [Agno Integration](../integrations/agno) — Use Semantica inside Agno multi-agent teams. - [Agno Integration](../integrations/agno) — Use Semantica inside Agno multi-agent teams.
+2 -2
View File
@@ -586,5 +586,5 @@ normalized = normalize_text("Apple Inc.", method="expand_suffixes")
- [Parse](/reference/parse) — Parse documents before normalization. - [Parse](/reference/parse) — Parse documents before normalization.
- [Split](/reference/split) — Chunk normalized text for embedding. - [Split](/reference/split) — Chunk normalized text for embedding.
- [Deduplication](deduplication) — Resolve duplicate entities after normalization. - [Deduplication](/reference/deduplication) — Resolve duplicate entities after normalization.
- [Pipeline](pipeline) — Include normalization as a named pipeline step. - [Pipeline](/reference/pipeline) — Include normalization as a named pipeline step.
+2 -2
View File
@@ -316,7 +316,7 @@ ontology_data = ingest_ontology("schema.jsonld") # JSON-LD
Ontology versioning (`VersionManager`, `OntologyVersion`) has moved to `semantica.change_management`. Import from there: `from semantica.change_management import VersionManager`. Ontology versioning (`VersionManager`, `OntologyVersion`) has moved to `semantica.change_management`. Import from there: `from semantica.change_management import VersionManager`.
</Note> </Note>
- [Reasoning](reasoning) — Apply inference rules over ontology axioms. - [Reasoning](/reference/reasoning) — Apply inference rules over ontology axioms.
- [Knowledge Graph](/reference/kg) — The graph being modeled by the ontology. - [Knowledge Graph](/reference/kg) — The graph being modeled by the ontology.
- [Export](export) — Export ontologies as RDF, OWL, or JSON-LD. - [Export](/reference/export) — Export ontologies as RDF, OWL, or JSON-LD.
- [Conflicts](/reference/conflicts) — Detect ontology constraint violations. - [Conflicts](/reference/conflicts) — Detect ontology constraint violations.
+1 -1
View File
@@ -297,7 +297,7 @@ for source in sources:
Docling is an optional dependency. If `docling` is not installed, `DoclingParser` raises an `ImportError` with installation instructions: `pip install docling`. `DocumentParser` is always available and requires no extras. Docling is an optional dependency. If `docling` is not installed, `DoclingParser` raises an `ImportError` with installation instructions: `pip install docling`. `DocumentParser` is always available and requires no extras.
</Note> </Note>
- [Ingest](ingest) — Load files before parsing. - [Ingest](/reference/ingest) — Load files before parsing.
- [Split](/reference/split) — Chunk parsed text for embedding and extraction. - [Split](/reference/split) — Chunk parsed text for embedding and extraction.
- [Docling Integration](../integrations/docling) — Full Docling integration setup guide. - [Docling Integration](../integrations/docling) — Full Docling integration setup guide.
- [Semantic Extract](/reference/semantic_extract) — Extract entities and relations from parsed text. - [Semantic Extract](/reference/semantic_extract) — Extract entities and relations from parsed text.
+2 -2
View File
@@ -588,7 +588,7 @@ StepStatus.SKIPPED # Skipped due to FailureHandler "skip" strategy
</Accordion> </Accordion>
</AccordionGroup> </AccordionGroup>
- [Ingest](ingest) — First step in most pipelines. - [Ingest](/reference/ingest) — First step in most pipelines.
- [Semantic Extract](/reference/semantic_extract) — Core extraction step. - [Semantic Extract](/reference/semantic_extract) — Core extraction step.
- [Knowledge Graph](/reference/kg) — Graph construction step. - [Knowledge Graph](/reference/kg) — Graph construction step.
- [Export](export) — Final output step. - [Export](/reference/export) — Final output step.
+2 -2
View File
@@ -523,6 +523,6 @@ Provenance tracking in Semantica produces the following audit artifacts:
</Note> </Note>
- [Change Management](/reference/change_management) — Version control and snapshot audit trails. - [Change Management](/reference/change_management) — Version control and snapshot audit trails.
- [Ingest](ingest) — Provenance begins at the ingestion stage. - [Ingest](/reference/ingest) — Provenance begins at the ingestion stage.
- [Export](export) — Include provenance metadata in RDF exports. - [Export](/reference/export) — Include provenance metadata in RDF exports.
- [Context](/reference/context) — Decision provenance via AgentContext. - [Context](/reference/context) — Decision provenance via AgentContext.
+2 -2
View File
@@ -31,7 +31,7 @@ icon: "microchip"
## Which Engine Should I Use? ## Which Engine Should I Use?
- [Reasoner](#reasoner-forwardbackward-chaining) — IF/THEN rules, forward and backward chaining. **Start here**: covers 90% of use cases. No query language required. - [Reasoner](#reasoner-forward/backward-chaining) — IF/THEN rules, forward and backward chaining. **Start here**: covers 90% of use cases. No query language required.
- [GraphReasoner](#graphreasoner) — Natural language queries over a knowledge graph via LLM. No SPARQL or rules: just ask a question. - [GraphReasoner](#graphreasoner) — Natural language queries over a knowledge graph via LLM. No SPARQL or rules: just ask a question.
- [DatalogReasoner](#datalogreasoner) — Recursive Horn clause rules with guaranteed termination. Use for complex multi-hop transitive rules. - [DatalogReasoner](#datalogreasoner) — Recursive Horn clause rules with guaranteed termination. Use for complex multi-hop transitive rules.
- [ReteEngine](#reteengine) — Rete pattern matching for high-frequency inference. Use when you need to match many facts against many rules simultaneously. - [ReteEngine](#reteengine) — Rete pattern matching for high-frequency inference. Use when you need to match many facts against many rules simultaneously.
@@ -483,6 +483,6 @@ step.confidence # float
</Warning> </Warning>
- [Knowledge Graph](/reference/kg) — The knowledge graph being reasoned over. - [Knowledge Graph](/reference/kg) — The knowledge graph being reasoned over.
- [Ontology](ontology) — Ontology axioms and SHACL constraints for logical reasoning. - [Ontology](/reference/ontology) — Ontology axioms and SHACL constraints for logical reasoning.
- [Triplet Store](/reference/triplet_store) — RDF backend for SPARQL-based reasoning. - [Triplet Store](/reference/triplet_store) — RDF backend for SPARQL-based reasoning.
- [Context](/reference/context) — Reasoning integrated into agent decision intelligence. - [Context](/reference/context) — Reasoning integrated into agent decision intelligence.
+3 -3
View File
@@ -321,7 +321,7 @@ export SEMANTICA_SEED_MERGE_STRATEGY=seed_first
**Use YAML configuration for production deployments.** Hard-coding source paths in Python scripts makes environment-switching (dev → staging → prod) fragile. Declare sources in `config.yaml` under the `seed:` key and override paths with `SEMANTICA_SEED_DATA_DIR`. This way, the same code runs in every environment. **Use YAML configuration for production deployments.** Hard-coding source paths in Python scripts makes environment-switching (dev → staging → prod) fragile. Declare sources in `config.yaml` under the `seed:` key and override paths with `SEMANTICA_SEED_DATA_DIR`. This way, the same code runs in every environment.
</Tip> </Tip>
- [Ingest](ingest) — Load unstructured data alongside seed data. - [Ingest](/reference/ingest) — Load unstructured data alongside seed data.
- [Knowledge Graph](/reference/kg) — The target graph that seed data populates. - [Knowledge Graph](/reference/kg) — The target graph that seed data populates.
- [Deduplication](deduplication) — Handle duplicates during seed-extracted merge. - [Deduplication](/reference/deduplication) — Handle duplicates during seed-extracted merge.
- [Pipeline](pipeline) — Incorporate seed loading as a named pipeline step. - [Pipeline](/reference/pipeline) — Incorporate seed loading as a named pipeline step.
+1 -1
View File
@@ -448,4 +448,4 @@ triplets = trip.extract(text)
- [LLM Providers](/reference/llms) — Configure which LLM is used for extraction. - [LLM Providers](/reference/llms) — Configure which LLM is used for extraction.
- [Knowledge Graph](/reference/kg) — Build graphs from extracted entities and relationships. - [Knowledge Graph](/reference/kg) — Build graphs from extracted entities and relationships.
- [Parse Module](/reference/parse) — Parse documents before extraction. - [Parse Module](/reference/parse) — Parse documents before extraction.
- [Deduplication](deduplication) — Resolve duplicate entities after extraction. - [Deduplication](/reference/deduplication) — Resolve duplicate entities after extraction.
+2 -2
View File
@@ -371,9 +371,9 @@ for chunk in chunks:
print(f" {len(entities)} entities in chunk starting at {chunk.start_index}") print(f" {len(entities)} entities in chunk starting at {chunk.start_index}")
``` ```
For the full pipeline orchestration API, see the [Pipeline reference](pipeline). For the full pipeline orchestration API, see the [Pipeline reference](/reference/pipeline).
- [Parse](/reference/parse) — Parse documents before chunking: produces sections and metadata. - [Parse](/reference/parse) — Parse documents before chunking: produces sections and metadata.
- [Embeddings](/reference/embeddings) — Embed chunks for vector search and semantic chunking. - [Embeddings](/reference/embeddings) — Embed chunks for vector search and semantic chunking.
- [Semantic Extract](/reference/semantic_extract) — Extract entities and relations from individual chunks. - [Semantic Extract](/reference/semantic_extract) — Extract entities and relations from individual chunks.
- [Pipeline](pipeline) — Integrate splitting as a named pipeline step. - [Pipeline](/reference/pipeline) — Integrate splitting as a named pipeline step.
+2 -2
View File
@@ -876,8 +876,8 @@ kg:
- [Knowledge Graph Module](/reference/kg) — Core graph construction, `GraphBuilder`, analytics. - [Knowledge Graph Module](/reference/kg) — Core graph construction, `GraphBuilder`, analytics.
- [Context Module](/reference/context) — Decision temporal windows and `find_active_nodes()`. - [Context Module](/reference/context) — Decision temporal windows and `find_active_nodes()`.
- [Provenance](provenance) — W3C PROV-O lineage stamped alongside temporal metadata. - [Provenance](/reference/provenance) — W3C PROV-O lineage stamped alongside temporal metadata.
- [Export](export) — OWL, Turtle, JSON-LD, and Parquet export with temporal annotations. - [Export](/reference/export) — OWL, Turtle, JSON-LD, and Parquet export with temporal annotations.
- [Temporal Knowledge Graphs](https://github.com/semantica-agi/semantica/blob/main/cookbook/advanced/10_Temporal_Knowledge_Graphs.ipynb) — Temporal reasoning and Allen algebra · Advanced - [Temporal Knowledge Graphs](https://github.com/semantica-agi/semantica/blob/main/cookbook/advanced/10_Temporal_Knowledge_Graphs.ipynb) — Temporal reasoning and Allen algebra · Advanced
- [Context Module](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/19_Context_Module.ipynb) — Including temporal decision windows · Intermediate - [Context Module](https://github.com/semantica-agi/semantica/blob/main/cookbook/introduction/19_Context_Module.ipynb) — Including temporal decision windows · Intermediate
+3 -3
View File
@@ -561,7 +561,7 @@ for row in result.bindings:
print(row) print(row)
``` ```
- [Export](export) — Export knowledge graphs to RDF formats. - [Export](/reference/export) — Export knowledge graphs to RDF formats.
- [Ontology](ontology) — Load OWL ontologies and store as RDF triples. - [Ontology](/reference/ontology) — Load OWL ontologies and store as RDF triples.
- [Reasoning](reasoning) — SPARQL-based property chain inference. - [Reasoning](/reference/reasoning) — SPARQL-based property chain inference.
- [Graph Store](/reference/graph_store) — Property graph alternative for Cypher queries. - [Graph Store](/reference/graph_store) — Property graph alternative for Cypher queries.
+1 -1
View File
@@ -223,4 +223,4 @@ config = read_json_file("config.json")
``` ```
- [Core](/reference/core) — Framework orchestration that uses Utils internally. - [Core](/reference/core) — Framework orchestration that uses Utils internally.
- [Pipeline](pipeline) — Uses ProgressTracker for per-step tracking. - [Pipeline](/reference/pipeline) — Uses ProgressTracker for per-step tracking.
+5 -5
View File
@@ -160,7 +160,7 @@ No installation or API key required. FAISS requires `pip install faiss-cpu`.
<Tab title="Pinecone"> <Tab title="Pinecone">
```bash ```bash
pip install "semantica[pinecone]" pip install "semantica[vectorstore-pinecone]"
``` ```
```python ```python
@@ -178,7 +178,7 @@ store = VectorStore(
<Tab title="Weaviate"> <Tab title="Weaviate">
```bash ```bash
pip install "semantica[weaviate]" pip install "semantica[vectorstore-weaviate]"
``` ```
```python ```python
@@ -194,7 +194,7 @@ store = VectorStore(
<Tab title="Qdrant"> <Tab title="Qdrant">
```bash ```bash
pip install "semantica[qdrant]" pip install "semantica[vectorstore-qdrant]"
``` ```
```python ```python
@@ -210,7 +210,7 @@ store = VectorStore(
<Tab title="PgVector"> <Tab title="PgVector">
```bash ```bash
pip install "semantica[pgvector]" pip install "semantica[vectorstore-pgvector]"
``` ```
```python ```python
@@ -591,4 +591,4 @@ store.create_index(index_type="pq", metric="L2", m=8)
- [Embeddings](/reference/embeddings) — Generate the vectors stored here. - [Embeddings](/reference/embeddings) — Generate the vectors stored here.
- [Context](/reference/context) — AgentContext uses VectorStore for memory retrieval. - [Context](/reference/context) — AgentContext uses VectorStore for memory retrieval.
- [Split](/reference/split) — Chunk documents before embedding and storing. - [Split](/reference/split) — Chunk documents before embedding and storing.
- [Ingest](ingest) — Ingest documents before embedding and storing. - [Ingest](/reference/ingest) — Ingest documents before embedding and storing.
+1 -1
View File
@@ -291,6 +291,6 @@ semantica-explorer --graph my_graph.json
See the [Explorer reference](/reference/explorer) for the full feature set and REST API. See the [Explorer reference](/reference/explorer) for the full feature set and REST API.
- [Knowledge Graph](/reference/kg) — The graph being visualized. - [Knowledge Graph](/reference/kg) — The graph being visualized.
- [Ontology](ontology) — Visualize ontology class structure. - [Ontology](/reference/ontology) — Visualize ontology class structure.
- [Embeddings](/reference/embeddings) — Generate the embeddings visualized here. - [Embeddings](/reference/embeddings) — Generate the embeddings visualized here.
- [Explorer](/reference/explorer) — Full interactive Knowledge Explorer UI. - [Explorer](/reference/explorer) — Full interactive Knowledge Explorer UI.
+2
View File
@@ -236,6 +236,8 @@ def _() -> list[str]:
cwd=DOCS, cwd=DOCS,
capture_output=True, capture_output=True,
text=True, text=True,
encoding="utf-8",
errors="replace",
timeout=600, timeout=600,
) )
# Clean up zip regardless of outcome # Clean up zip regardless of outcome
+1 -1
View File
@@ -9,7 +9,7 @@
"lint": "eslint .", "lint": "eslint .",
"preview": "vite preview", "preview": "vite preview",
"test:graph-store": "node --test tests/graphStore.multi-edge.test.mjs", "test:graph-store": "node --test tests/graphStore.multi-edge.test.mjs",
"test:graph-workspace": "node --import tsx --test tests/markdownContentViewer.test.ts tests/markdownEditorInteraction.test.tsx tests/markdownEditorState.test.ts tests/nodeMarkdownSync.test.ts tests/graphSceneState.display.test.ts tests/temporalLifecycle.test.ts tests/deterministicExplorerRendering.test.ts tests/explorerCapabilities.test.tsx tests/smallGraphLayout.test.ts tests/realtimeGraphAttributes.test.ts tests/ontologyEditorModel.test.ts tests/graphColorLegend.test.ts", "test:graph-workspace": "node --import tsx --test tests/markdownContentViewer.test.ts tests/markdownEditorInteraction.test.tsx tests/markdownEditorState.test.ts tests/nodeMarkdownSync.test.ts tests/graphSceneState.display.test.ts tests/temporalLifecycle.test.ts tests/temporalScrubberBounds.test.ts tests/deterministicExplorerRendering.test.ts tests/explorerCapabilities.test.tsx tests/smallGraphLayout.test.ts tests/realtimeGraphAttributes.test.ts tests/ontologyEditorModel.test.ts tests/graphColorLegend.test.ts tests/ontologyUrlState.test.ts",
"test:deterministic-e2e": "node --import tsx --test tests/deterministicExplorerRendering.e2e.ts", "test:deterministic-e2e": "node --import tsx --test tests/deterministicExplorerRendering.e2e.ts",
"test:graph-legend-e2e": "node --import tsx --test tests/graphColorLegend.e2e.ts", "test:graph-legend-e2e": "node --import tsx --test tests/graphColorLegend.e2e.ts",
"test:plugin-registry": "node --import tsx --test tests/pluginRegistry.temporal.test.mjs" "test:plugin-registry": "node --import tsx --test tests/pluginRegistry.temporal.test.mjs"
+2 -9
View File
@@ -19,6 +19,7 @@ import {
import { ErrorBoundary } from './ErrorBoundary'; import { ErrorBoundary } from './ErrorBoundary';
import { ExploreWorkspaceTabs, type ExploreView } from './ExploreWorkspaceTabs'; import { ExploreWorkspaceTabs, type ExploreView } from './ExploreWorkspaceTabs';
import { fetchAgentMemoryAvailability } from './explorerCapabilities'; import { fetchAgentMemoryAvailability } from './explorerCapabilities';
import { hasOntologyUrlState } from './workspaces/OntologyWorkspace/ontologyUrlState';
const DecisionWorkspace = lazy(() => import('./workspaces/DecisionWorkspace/DecisionWorkspace').then((module) => ({ default: module.DecisionWorkspace }))); const DecisionWorkspace = lazy(() => import('./workspaces/DecisionWorkspace/DecisionWorkspace').then((module) => ({ default: module.DecisionWorkspace })));
const DiffMergeWorkspace = lazy(() => import('./workspaces/DiffMergeWorkspace/DiffMergeWorkspace').then((module) => ({ default: module.DiffMergeWorkspace }))); const DiffMergeWorkspace = lazy(() => import('./workspaces/DiffMergeWorkspace/DiffMergeWorkspace').then((module) => ({ default: module.DiffMergeWorkspace })));
@@ -96,15 +97,7 @@ const navItems: NavItem[] = [
]; ];
function readInitialWorkspace(): WorkspaceId { function readInitialWorkspace(): WorkspaceId {
try { return hasOntologyUrlState() ? 'ontology-hub' : 'welcome';
const params = new URLSearchParams(window.location.search);
if (params.has("ontologyTab") || params.has("ontologyEntity")) {
return "ontology-hub";
}
} catch {
// Default to the welcome screen when URL state is unavailable.
}
return "welcome";
} }
const shellStyles = ` const shellStyles = `
@@ -346,7 +346,7 @@ export function GraphInspectorPanel({
<div style={{ color: GRAPH_THEME.ui.text.strong, fontWeight: 600, marginBottom: 6 }}>Selected item is not directly inspectable in the current graph.</div> <div style={{ color: GRAPH_THEME.ui.text.strong, fontWeight: 600, marginBottom: 6 }}>Selected item is not directly inspectable in the current graph.</div>
<div style={{ color: GRAPH_THEME.ui.text.body, fontSize: 13, lineHeight: 1.6 }}> <div style={{ color: GRAPH_THEME.ui.text.body, fontSize: 13, lineHeight: 1.6 }}>
{canActivateFocused {canActivateFocused
? "Activate Focused mode to resolve this grouped selection to its canonical node." ? "Use Focus to resolve this grouped selection to its canonical node."
: (focusedUnavailableReason ?? "Focused mode is unavailable for the current selection.")} : (focusedUnavailableReason ?? "Focused mode is unavailable for the current selection.")}
</div> </div>
</div> </div>
@@ -2791,7 +2791,7 @@ export function GraphWorkspace({ externalFocusNodeId, externalFocusToken, onDirt
}, },
{ {
id: "view-focused", id: "view-focused",
label: "Focused", label: "Focus",
title: canActivateFocusedMode title: canActivateFocusedMode
? "Inspect the selected node in a focused local graph" ? "Inspect the selected node in a focused local graph"
: (focusedSelectionResolution.reason ?? "Focused mode is unavailable for the current selection"), : (focusedSelectionResolution.reason ?? "Focused mode is unavailable for the current selection"),
@@ -4,6 +4,7 @@ import { Timeline } from "vis-timeline";
import type { TimelineOptions } from "vis-timeline"; import type { TimelineOptions } from "vis-timeline";
import "vis-timeline/styles/vis-timeline-graph2d.css"; import "vis-timeline/styles/vis-timeline-graph2d.css";
import { GRAPH_THEME } from "./graphTheme"; import { GRAPH_THEME } from "./graphTheme";
import { DEFAULT_MIN_DATE, resolvePlayStepMs, resolveScrubberBounds } from "./temporalScrubberBounds";
export interface TimelinePanelProps { export interface TimelinePanelProps {
onTimeChange: (time: Date) => void; onTimeChange: (time: Date) => void;
@@ -11,11 +12,9 @@ export interface TimelinePanelProps {
maxDate?: string; maxDate?: string;
} }
const DEFAULT_MIN_DATE = new Date("1970-01-01T00:00:00Z");
const DEFAULT_MAX_DATE = new Date("2030-01-01T00:00:00Z");
const PLAYHEAD_ID = "playhead"; const PLAYHEAD_ID = "playhead";
const PLAY_INTERVAL_MS = 500; const PLAY_INTERVAL_MS = 500;
const PLAY_STEP_MONTHS = 6; const ONE_DAY_MS = 1000 * 60 * 60 * 24;
const VIS_OVERRIDE_CSS = ` const VIS_OVERRIDE_CSS = `
.sem-timeline-wrap .vis-timeline { border: none !important; background: transparent !important; overflow: visible !important; } .sem-timeline-wrap .vis-timeline { border: none !important; background: transparent !important; overflow: visible !important; }
@@ -54,12 +53,6 @@ const VIS_OVERRIDE_CSS = `
.sem-timeline-wrap .vis-panel.vis-left { display: none !important; } .sem-timeline-wrap .vis-panel.vis-left { display: none !important; }
`; `;
function safeDate(value: string | undefined, fallback: Date): Date {
if (!value) return fallback;
const parsed = new Date(value);
return Number.isNaN(parsed.getTime()) ? fallback : parsed;
}
function formatPlayheadLabel(value: Date): string { function formatPlayheadLabel(value: Date): string {
return `${value.getFullYear()}/${String(value.getMonth() + 1).padStart(2, "0")}`; return `${value.getFullYear()}/${String(value.getMonth() + 1).padStart(2, "0")}`;
} }
@@ -72,9 +65,13 @@ export function TimelinePanel({ onTimeChange, minDate, maxDate }: TimelinePanelP
const [isPlaying, setIsPlaying] = useState(false); const [isPlaying, setIsPlaying] = useState(false);
const [displayDate, setDisplayDate] = useState(formatPlayheadLabel(DEFAULT_MIN_DATE)); const [displayDate, setDisplayDate] = useState(formatPlayheadLabel(DEFAULT_MIN_DATE));
const minBound = useMemo(() => safeDate(minDate, DEFAULT_MIN_DATE), [minDate]); // Captured once per mount so re-renders keep the same reference and do not
const maxBound = useMemo(() => safeDate(maxDate, DEFAULT_MAX_DATE), [maxDate]); // retrigger the timeline effect below.
const defaultTime = useMemo(() => new Date(Math.round((minBound.getTime() + maxBound.getTime()) / 2)), [maxBound, minBound]); const now = useMemo(() => new Date(), []);
const { minBound, maxBound, defaultTime } = useMemo(
() => resolveScrubberBounds({ minDate, maxDate, now }),
[maxDate, minDate, now],
);
useEffect(() => { useEffect(() => {
if (!containerRef.current) return; if (!containerRef.current) return;
@@ -91,12 +88,10 @@ export function TimelinePanel({ onTimeChange, minDate, maxDate }: TimelinePanelP
showCurrentTime: false, showCurrentTime: false,
zoomable: true, zoomable: true,
moveable: true, moveable: true,
zoomMin: 1000 * 60 * 60 * 24 * 365, zoomMin: ONE_DAY_MS,
zoomMax: 1000 * 60 * 60 * 24 * 365 * 80, zoomMax: 1000 * 60 * 60 * 24 * 365 * 80,
showMajorLabels: true, showMajorLabels: true,
showMinorLabels: true, showMinorLabels: true,
timeAxis: { scale: "year", step: 5 },
format: { minorLabels: { year: "YYYY" }, majorLabels: { year: "YYYY" } },
orientation: { axis: "bottom" }, orientation: { axis: "bottom" },
margin: { item: 0, axis: 0 }, margin: { item: 0, axis: 0 },
selectable: false, selectable: false,
@@ -134,8 +129,7 @@ export function TimelinePanel({ onTimeChange, minDate, maxDate }: TimelinePanelP
playIntervalRef.current = setInterval(() => { playIntervalRef.current = setInterval(() => {
const timeline = timelineRef.current; const timeline = timelineRef.current;
if (!timeline) return; if (!timeline) return;
const next = new Date(playheadRef.current); const next = new Date(playheadRef.current.getTime() + resolvePlayStepMs(minBound, maxBound));
next.setMonth(next.getMonth() + PLAY_STEP_MONTHS);
if (next >= maxBound) { if (next >= maxBound) {
next.setTime(minBound.getTime()); next.setTime(minBound.getTime());
} }
@@ -91,7 +91,7 @@ export const temporalOverlayPlugin: GraphPlugin = {
<div style={detailRowStyle}> <div style={detailRowStyle}>
<span style={detailLabelStyle}>Bounds</span> <span style={detailLabelStyle}>Bounds</span>
<span style={detailValueStyle}> <span style={detailValueStyle}>
{(temporal?.minDate ?? "1970")} {(temporal?.maxDate ?? "2030")} {(temporal?.minDate ?? "1970")} {(temporal?.maxDate ?? "now")}
</span> </span>
</div> </div>
<div style={detailRowStyle}> <div style={detailRowStyle}>
@@ -0,0 +1,46 @@
export interface ScrubberBoundsInput {
minDate?: string;
maxDate?: string;
now: Date;
}
export interface ScrubberBounds {
minBound: Date;
maxBound: Date;
defaultTime: Date;
}
export const DEFAULT_MIN_DATE = new Date("1970-01-01T00:00:00Z");
const ONE_DAY_MS = 1000 * 60 * 60 * 24;
const PLAY_FRAMES = 60;
function parseBound(value: string | undefined, fallback: Date): Date {
if (!value) return fallback;
const parsed = new Date(value);
return Number.isNaN(parsed.getTime()) ? fallback : parsed;
}
function clamp(value: Date, minBound: Date, maxBound: Date): Date {
if (value < minBound) return minBound;
if (value > maxBound) return maxBound;
return value;
}
/**
* `/api/temporal/bounds` reports `max: null` for graphs whose nodes carry
* `valid_from` instants and no `valid_until`, which is the common case rather
* than malformed data. Such a graph is known up to the present and no further,
* so `now` is the honest upper bound and the honest starting playhead.
*/
export function resolveScrubberBounds({ minDate, maxDate, now }: ScrubberBoundsInput): ScrubberBounds {
const minBound = parseBound(minDate, DEFAULT_MIN_DATE);
const maxBound = parseBound(maxDate, now);
const orderedMax = maxBound > minBound ? maxBound : minBound;
return { minBound, maxBound: orderedMax, defaultTime: clamp(now, minBound, orderedMax) };
}
/** Keeps a play-through at ~PLAY_FRAMES steps whatever the span, with a one-day floor. */
export function resolvePlayStepMs(minBound: Date, maxBound: Date): number {
const span = maxBound.getTime() - minBound.getTime();
return Math.max(ONE_DAY_MS, Math.round(span / PLAY_FRAMES));
}
@@ -28,11 +28,12 @@ import { loadOntologyEntityOwner, loadOntologyGraph } from "./api";
import type { OntologyGraphEdge, OntologyGraphNode } from "./api"; import type { OntologyGraphEdge, OntologyGraphNode } from "./api";
import { import {
classifyNodeType, classifyNodeType,
inferOntologyUri,
isEditableEntityType, isEditableEntityType,
ONTOLOGY_MINIMAP_THEME, ONTOLOGY_MINIMAP_THEME,
resolveEditorOntology,
} from "./ontologyEditorModel"; } from "./ontologyEditorModel";
import type { EditorEntityType, RegistryEntry } from "./ontologyEditorModel"; import type { EditorEntityType, RegistryEntry } from "./ontologyEditorModel";
import { clearEntitySelection, readOntologyUrlState, writeEntitySelection } from "./ontologyUrlState";
type OntologyNodeData = { type OntologyNodeData = {
label?: string; label?: string;
@@ -137,11 +138,7 @@ interface DraftDiff {
} }
function requestedEntityUri(): string { function requestedEntityUri(): string {
try { return readOntologyUrlState().entityUri || "";
return new URLSearchParams(window.location.search).get("ontologyEntity") || "";
} catch {
return "";
}
} }
function nodeLabel(node: OntologyGraphNode): string { function nodeLabel(node: OntologyGraphNode): string {
@@ -225,6 +222,7 @@ export function OntologyEditor() {
const [flowInstance, setFlowInstance] = useState<ReactFlowInstance<OntologyNode, OntologyEdge> | null>(null); const [flowInstance, setFlowInstance] = useState<ReactFlowInstance<OntologyNode, OntologyEdge> | null>(null);
const [isLoadingGraph, setIsLoadingGraph] = useState(false); const [isLoadingGraph, setIsLoadingGraph] = useState(false);
const [graphError, setGraphError] = useState(""); const [graphError, setGraphError] = useState("");
const [unownedEntity, setUnownedEntity] = useState("");
const [draftDiff, setDraftDiff] = useState<DraftDiff>({ const [draftDiff, setDraftDiff] = useState<DraftDiff>({
added_classes: [], added_classes: [],
removed_classes: [], removed_classes: [],
@@ -250,11 +248,21 @@ export function OntologyEditor() {
? loadOntologyEntityOwner(requested).catch(() => undefined) ? loadOntologyEntityOwner(requested).catch(() => undefined)
: Promise.resolve(undefined), : Promise.resolve(undefined),
]) ])
.then(([entries, explicitOwner]: [RegistryEntry[], string | undefined]) => { .then(([entries, ownerVerdict]: [RegistryEntry[], string | null | undefined]) => {
if (cancelled) return; if (cancelled) return;
setRegistry(entries); setRegistry(entries);
const inferredOntology = inferOntologyUri(entries, requested, explicitOwner); const resolution = resolveEditorOntology(entries, requested, ownerVerdict);
setOntologyUri((current) => current || inferredOntology || entries[0]?.uri || ""); // The registry default is the right landing place for "no entity asked
// for", but not for "the backend says nothing owns the entity that was
// asked for" — that would open an arbitrary ontology whose graph
// excludes the entity, and report nothing about why.
if (resolution.status === "unowned") {
setUnownedEntity(resolution.entityUri);
return;
}
setUnownedEntity("");
const resolvedOntology = resolution.status === "resolved" ? resolution.uri : undefined;
setOntologyUri((current) => current || resolvedOntology || entries[0]?.uri || "");
}) })
.catch((error) => { .catch((error) => {
console.error("Failed to load ontology registry:", error); console.error("Failed to load ontology registry:", error);
@@ -377,14 +385,7 @@ export function OntologyEditor() {
const selectNode = useCallback((node: OntologyNode) => { const selectNode = useCallback((node: OntologyNode) => {
setSelectedElement(node); setSelectedElement(node);
try { writeEntitySelection(node.id);
const params = new URLSearchParams(window.location.search);
params.set("ontologyTab", "editor");
params.set("ontologyEntity", node.id);
window.history.replaceState(null, "", `?${params.toString()}`);
} catch {
// URL state is optional; the editor selection still works without it.
}
}, []); }, []);
const saveDraft = useCallback(async () => { const saveDraft = useCallback(async () => {
@@ -554,15 +555,8 @@ export function OntologyEditor() {
onChange={(event) => { onChange={(event) => {
setOntologyUri(event.target.value); setOntologyUri(event.target.value);
setSelectedElement(null); setSelectedElement(null);
try { setUnownedEntity("");
// Drop the previous ontology's entity from the URL, or a reload clearEntitySelection();
// would resolve the stale ID and jump back to that ontology.
const params = new URLSearchParams(window.location.search);
params.delete("ontologyEntity");
window.history.replaceState(null, "", `?${params.toString()}`);
} catch {
// URL state is optional; switching ontologies still works.
}
}} }}
style={selectStyle} style={selectStyle}
> >
@@ -633,7 +627,12 @@ export function OntologyEditor() {
{!isLoadingGraph && graphError && ( {!isLoadingGraph && graphError && (
<div style={{ ...canvasMessageStyle, color: "#ff9a8d" }}>{graphError}</div> <div style={{ ...canvasMessageStyle, color: "#ff9a8d" }}>{graphError}</div>
)} )}
{!isLoadingGraph && !graphError && ontologyUri && nodes.length === 0 && ( {!isLoadingGraph && !graphError && unownedEntity && (
<div style={{ ...canvasMessageStyle, color: "#f2b66d" }}>
No registered ontology owns {unownedEntity}. Pick an ontology above to start editing.
</div>
)}
{!isLoadingGraph && !graphError && !unownedEntity && ontologyUri && nodes.length === 0 && (
<div style={canvasMessageStyle}>This ontology has no editable classes or properties.</div> <div style={canvasMessageStyle}>This ontology has no editable classes or properties.</div>
)} )}
@@ -32,7 +32,11 @@ export type OntologyGraphResponse = {
}; };
export type OntologyEntityOwner = { export type OntologyEntityOwner = {
source_ontology?: string; // Optional on purpose, unlike OntologyGraphNode.entity_type. There, a missing
// field degrades to a read-only node — benign. Here it would be read as an
// authoritative "nothing owns this entity", which now suppresses selection
// outright, so presence has to be checked rather than assumed.
owning_ontology?: string | null;
}; };
async function parseResponse<T>(response: Response): Promise<T> { async function parseResponse<T>(response: Response): Promise<T> {
@@ -63,10 +67,21 @@ export async function loadOntologyGraph(uri: string, signal?: AbortSignal): Prom
); );
} }
export async function loadOntologyEntityOwner(uri: string): Promise<string | undefined> { // Three-state verdict: a string names the owner, null is the backend's
// authoritative "no known ontology owns this entity", and undefined means the
// request failed so there is no verdict to act on.
export type OntologyOwnerVerdict = string | null | undefined;
export async function loadOntologyEntityOwner(uri: string): Promise<OntologyOwnerVerdict> {
const response = await fetch(`/api/ontology/entity/${encodeURIComponent(uri)}`); const response = await fetch(`/api/ontology/entity/${encodeURIComponent(uri)}`);
if (!response.ok) return undefined; if (!response.ok) return undefined;
return (await response.json() as OntologyEntityOwner).source_ontology; const owner = await response.json() as OntologyEntityOwner | null;
// Only a field that is actually there carries the verdict. Coercing an absent
// field to null would assert the strongest available claim — "nothing owns
// this" — on the weakest possible evidence, and that claim now stops the
// editor selecting an ontology at all.
const verdict = owner?.owning_ontology;
return verdict === undefined ? undefined : verdict;
} }
export async function loadAlignments(uri?: string): Promise<OntologyAlignment[]> { export async function loadAlignments(uri?: string): Promise<OntologyAlignment[]> {
@@ -13,6 +13,7 @@ import { OntologyManager } from "./OntologyManager";
import { OntologyEditor } from "./OntologyEditor"; import { OntologyEditor } from "./OntologyEditor";
import { ShaclStudio } from "./ShaclStudio"; import { ShaclStudio } from "./ShaclStudio";
import { VersionsTab } from "./VersionsTab"; import { VersionsTab } from "./VersionsTab";
import { readOntologyUrlState, writeEntitySelection, writeTab } from "./ontologyUrlState";
export type OntologyHubTab = export type OntologyHubTab =
| "registry" | "registry"
@@ -22,8 +23,6 @@ export type OntologyHubTab =
| "health" | "health"
| "shacl"; | "shacl";
const TAB_PARAM = "ontologyTab";
const TABS: { id: OntologyHubTab; label: string; icon: typeof GitMerge }[] = [ const TABS: { id: OntologyHubTab; label: string; icon: typeof GitMerge }[] = [
{ id: "registry", label: "Registry", icon: BookMarked }, { id: "registry", label: "Registry", icon: BookMarked },
{ id: "editor", label: "Editor", icon: Sliders }, { id: "editor", label: "Editor", icon: Sliders },
@@ -33,37 +32,23 @@ const TABS: { id: OntologyHubTab; label: string; icon: typeof GitMerge }[] = [
{ id: "shacl", label: "SHACL", icon: Shield }, { id: "shacl", label: "SHACL", icon: Shield },
]; ];
function readTabParam(): OntologyHubTab { function readInitialTab(): OntologyHubTab {
try { const { tab, entityUri } = readOntologyUrlState();
const params = new URLSearchParams(window.location.search); const requested = TABS.find((candidate) => candidate.id === tab);
const raw = params.get(TAB_PARAM); if (requested) return requested.id;
if (raw && TABS.some((t) => t.id === raw)) return raw as OntologyHubTab; if (entityUri) return "editor";
if (params.get("ontologyEntity")) return "editor";
} catch {
// ignore
}
return "registry"; return "registry";
} }
function writeTabParam(tab: OntologyHubTab) {
try {
const params = new URLSearchParams(window.location.search);
params.set(TAB_PARAM, tab);
window.history.replaceState(null, "", `?${params.toString()}`);
} catch {
// ignore
}
}
interface OntologyWorkspaceProps { interface OntologyWorkspaceProps {
onJumpToGraphNode?: (nodeId: string) => void; onJumpToGraphNode?: (nodeId: string) => void;
} }
export function OntologyWorkspace({ onJumpToGraphNode }: OntologyWorkspaceProps) { export function OntologyWorkspace({ onJumpToGraphNode }: OntologyWorkspaceProps) {
const [activeTab, setActiveTab] = useState<OntologyHubTab>(readTabParam); const [activeTab, setActiveTab] = useState<OntologyHubTab>(readInitialTab);
useEffect(() => { useEffect(() => {
writeTabParam(activeTab); writeTab(activeTab);
}, [activeTab]); }, [activeTab]);
const handleTabChange = useCallback((tab: OntologyHubTab) => { const handleTabChange = useCallback((tab: OntologyHubTab) => {
@@ -71,10 +56,7 @@ export function OntologyWorkspace({ onJumpToGraphNode }: OntologyWorkspaceProps)
}, []); }, []);
const handleFixInEditor = useCallback((entityUri: string) => { const handleFixInEditor = useCallback((entityUri: string) => {
const params = new URLSearchParams(window.location.search); writeEntitySelection(entityUri);
params.set(TAB_PARAM, "editor");
params.set("ontologyEntity", entityUri);
window.history.replaceState(null, "", `?${params.toString()}`);
setActiveTab("editor"); setActiveTab("editor");
}, []); }, []);
@@ -45,6 +45,10 @@ export function classifyNodeType(rawType: string): EditorEntityType {
return "external"; return "external";
} }
// Last-resort guess, reached only when the backend gave no verdict: it has no
// notion of nested vocabularies, so it can name a parent that does not contain
// the entity. Authority is owning_ontology from /api/ontology/entity
// (_resolve_owning_ontology in semantica/explorer/routes/ontology.py).
function ownsByNamespace(entityUri: string, ontologyUri: string): boolean { function ownsByNamespace(entityUri: string, ontologyUri: string): boolean {
const stem = ontologyUri.replace(/[/#]+$/, ""); const stem = ontologyUri.replace(/[/#]+$/, "");
return entityUri === ontologyUri return entityUri === ontologyUri
@@ -52,12 +56,50 @@ function ownsByNamespace(entityUri: string, ontologyUri: string): boolean {
|| entityUri.startsWith(`${stem}/`); || entityUri.startsWith(`${stem}/`);
} }
/**
* Three outcomes, deliberately not collapsed into `string | undefined`.
*
* `unowned` and `unresolved` both yield "no ontology to open", but they must
* not be treated alike: a caller that writes `resolve(...) || entries[0]` turns
* the backend's authoritative "nothing owns this entity" into "open an
* arbitrary ontology", which reintroduces the parent-selection bug this whole
* verdict exists to prevent. The union makes that collapse a type error.
*/
export type EditorOntologyResolution =
| { status: "resolved"; uri: string }
| { status: "unowned"; entityUri: string }
| { status: "unresolved" };
// Picks the registered ontology to open for a deep-linked entity. A null
// verdict is the backend's authoritative "nothing owns this entity": the
// namespace guess must stay suppressed, or an unregistered nested namespace
// would select its registered parent again. Only an unavailable verdict
// (undefined) may fall back to inference.
export function resolveEditorOntology(
entries: RegistryEntry[],
entityUri: string,
ownerVerdict: string | null | undefined,
): EditorOntologyResolution {
if (ownerVerdict === null) {
return { status: "unowned", entityUri };
}
const uri = inferOntologyUri(entries, entityUri, ownerVerdict);
return uri === undefined ? { status: "unresolved" } : { status: "resolved", uri };
}
// Picks the registered ontology to open for an entity: the backend-resolved
// explicitOwner wins outright, the namespace guess is only the fallback.
export function inferOntologyUri( export function inferOntologyUri(
entries: RegistryEntry[], entries: RegistryEntry[],
entityUri: string, entityUri: string,
explicitOwner?: string, explicitOwner?: string,
): string | undefined { ): string | undefined {
if (explicitOwner && entries.some((entry) => entry.uri === explicitOwner)) { // Trusted even when the registry does not list it. Falling through to the
// namespace guess here would answer a question nobody asked — the backend
// named this entity's owner, and silently opening a *different* ontology is
// worse than opening one the registry has not been told about yet, which
// surfaces as an explicit error from /api/ontology/graph.
if (explicitOwner) {
return explicitOwner; return explicitOwner;
} }
return [...entries] return [...entries]
@@ -0,0 +1,94 @@
// Sole owner of the Ontology Hub deep-link query parameters: the names below must not be
// spelled out anywhere else, so that the protocol can change in one place.
const TAB_PARAM = "ontologyTab";
const ENTITY_PARAM = "ontologyEntity";
const EDITOR_TAB = "editor";
export interface OntologyUrlState {
/** Raw parameter value; the set of legal tab ids belongs to the workspace, not this module. */
tab?: string;
entityUri?: string;
}
/** `undefined` means the parameter is absent; an empty string means it is present but blank. */
export function parseOntologyUrlState(search: string): OntologyUrlState {
const params = new URLSearchParams(search);
return {
tab: params.get(TAB_PARAM) ?? undefined,
entityUri: params.get(ENTITY_PARAM) ?? undefined,
};
}
export function applyTab(search: string, tab: string): string {
const params = new URLSearchParams(search);
params.set(TAB_PARAM, tab);
return `?${params.toString()}`;
}
// A selected entity is only addressable from the editor, so the tab moves with it.
export function applyEntitySelection(search: string, entityUri: string): string {
const params = new URLSearchParams(search);
params.set(TAB_PARAM, EDITOR_TAB);
params.set(ENTITY_PARAM, entityUri);
return `?${params.toString()}`;
}
// Pairs with applyEntitySelection: an entity URI is resolved back to its owning ontology on
// load, so leaving a stale one behind when the active ontology changes reopens the old ontology.
export function removeEntitySelection(search: string): string {
const params = new URLSearchParams(search);
params.delete(ENTITY_PARAM);
return `?${params.toString()}`;
}
/**
* Deliberately dual-role, and the argument is what selects the role: given a
* `search` string this is pure and total, delegating straight to
* `parseOntologyUrlState`; called with no argument it reads live `window`
* state and yields empty state if the URL is unreadable. Callers in render or
* effect paths use the no-argument form; tests and any caller that already
* holds a search string pass it, which is the only form that is testable.
*/
export function readOntologyUrlState(search?: string): OntologyUrlState {
if (search !== undefined) {
return parseOntologyUrlState(search);
}
try {
return parseOntologyUrlState(window.location.search);
} catch {
return {};
}
}
/** True when the URL addresses the Ontology Hub at all, even with blank parameter values. */
export function hasOntologyUrlState(search?: string): boolean {
const { tab, entityUri } = readOntologyUrlState(search);
return tab !== undefined || entityUri !== undefined;
}
// The transform returns a query string only, so the fragment has to be carried
// across explicitly: replaceState with a bare "?..." drops it. This is the one
// place that knows how the URL is written, so it is the only place that can.
function updateSearch(transform: (search: string) => string): void {
try {
window.history.replaceState(
null,
"",
`${transform(window.location.search)}${window.location.hash}`,
);
} catch {
// Deep-link state is a convenience; every caller stays correct without it.
}
}
export function writeTab(tab: string): void {
updateSearch((search) => applyTab(search, tab));
}
export function writeEntitySelection(entityUri: string): void {
updateSearch((search) => applyEntitySelection(search, entityUri));
}
export function clearEntitySelection(): void {
updateSearch(removeEntitySelection);
}
+3 -1
View File
@@ -101,7 +101,9 @@ test("visible legend follows loaded data, reloads, focused views, and distance m
await heatmap.click(); await heatmap.click();
await legend.waitFor(); await legend.waitFor();
await assertLegendMatchesGraph(page); await assertLegendMatchesGraph(page);
await page.getByRole("button", { name: "Focused", exact: true }).click(); const focusButton = page.getByRole("button", { name: "Focus", exact: true });
assert.equal(await focusButton.isDisabled(), false, "Focus is enabled once a node is selected");
await focusButton.click();
await legend.getByText("Document", { exact: true }).waitFor({ state: "hidden" }); await legend.getByText("Document", { exact: true }).waitFor({ state: "hidden" });
await assertLegendMatchesGraph(page, ["alice", "acme", "research"]); await assertLegendMatchesGraph(page, ["alice", "acme", "research"]);
assert.equal(await legend.getByText("Researcher", { exact: true }).count(), 1); assert.equal(await legend.getByText("Researcher", { exact: true }).count(), 1);
@@ -6,6 +6,7 @@ import {
compactNodeType, compactNodeType,
inferOntologyUri, inferOntologyUri,
isEditableEntityType, isEditableEntityType,
resolveEditorOntology,
ONTOLOGY_MINIMAP_THEME, ONTOLOGY_MINIMAP_THEME,
} from "../src/workspaces/OntologyWorkspace/ontologyEditorModel"; } from "../src/workspaces/OntologyWorkspace/ontologyEditorModel";
@@ -29,6 +30,20 @@ test("explicit scheme ownership wins when an entity uses another namespace", ()
); );
}); });
test("an explicit owner missing from the registry is used, not quietly replaced", () => {
// The entity sits under a registered namespace, so the guess has an answer
// ready; the backend naming a different, unregistered owner must still win,
// or the editor opens an ontology nobody said owned this entity.
assert.equal(
inferOntologyUri(
registry,
"https://example.test/foo#Class",
"https://unregistered.test/vocab",
),
"https://unregistered.test/vocab",
);
});
test("only draft-supported class and property nodes are editable", () => { test("only draft-supported class and property nodes are editable", () => {
assert.equal(isEditableEntityType("class"), true); assert.equal(isEditableEntityType("class"), true);
assert.equal(isEditableEntityType("property"), true); assert.equal(isEditableEntityType("property"), true);
@@ -64,3 +79,36 @@ test("compactNodeType leaves unknown namespaces untouched", () => {
assert.equal(compactNodeType("https://example.org/custom#Thing"), "https://example.org/custom#Thing"); assert.equal(compactNodeType("https://example.org/custom#Thing"), "https://example.org/custom#Thing");
assert.equal(compactNodeType("owl:Class"), "owl:Class"); assert.equal(compactNodeType("owl:Class"), "owl:Class");
}); });
test("an authoritative no-owner verdict suppresses the namespace guess", () => {
// Without suppression the prefix guess would pick the registered parent
// for an unregistered nested entity — the deep link must not do that.
const nested = "https://example.test/foo/unregistered#Term";
assert.deepEqual(resolveEditorOntology(registry, nested, null), {
status: "unowned",
entityUri: nested,
});
// An unavailable verdict may still fall back to inference
assert.deepEqual(
resolveEditorOntology(registry, "https://example.test/foo#Class", undefined),
{ status: "resolved", uri: "https://example.test/foo" },
);
// A named owner wins outright
assert.deepEqual(
resolveEditorOntology(registry, nested, "https://example.test/foo/nested"),
{ status: "resolved", uri: "https://example.test/foo/nested" },
);
});
test("unowned is distinguishable from unresolved, so neither collapses to a default", () => {
// Both mean "no ontology to open", and the editor treats them oppositely:
// unresolved may land on the registry default, unowned must not. A caller
// writing `resolve(...) || entries[0]` reintroduced exactly the parent
// selection this verdict exists to prevent, so the difference is typed.
const unowned = resolveEditorOntology(registry, "https://example.test/foo/x#T", null);
const unresolved = resolveEditorOntology(registry, "https://elsewhere.test/T", undefined);
assert.equal(unowned.status, "unowned");
assert.equal(unresolved.status, "unresolved");
assert.notEqual(unowned.status, unresolved.status);
});
+102
View File
@@ -0,0 +1,102 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
applyEntitySelection,
applyTab,
clearEntitySelection,
hasOntologyUrlState,
parseOntologyUrlState,
readOntologyUrlState,
removeEntitySelection,
writeEntitySelection,
writeTab,
} from "../src/workspaces/OntologyWorkspace/ontologyUrlState";
function withStubbedLocation(search: string, hash: string, body: () => void): string[] {
const written: string[] = [];
const original = (globalThis as { window?: unknown }).window;
(globalThis as { window?: unknown }).window = {
location: { search, hash },
history: { replaceState: (_s: unknown, _t: string, url: string) => written.push(url) },
};
try {
body();
} finally {
(globalThis as { window?: unknown }).window = original;
}
return written;
}
test("selecting an entity round-trips and pins the editor tab", () => {
const search = applyEntitySelection("", "https://example.test/foo#Bar");
assert.deepEqual(parseOntologyUrlState(search), {
tab: "editor",
entityUri: "https://example.test/foo#Bar",
});
});
test("clearing the selection drops only the entity and keeps unrelated params", () => {
const search = applyEntitySelection("?view=graph&depth=2", "https://example.test/foo#Bar");
const cleared = parseOntologyUrlState(removeEntitySelection(search));
assert.equal(cleared.entityUri, undefined);
assert.equal(cleared.tab, "editor");
assert.equal(new URLSearchParams(removeEntitySelection(search)).get("depth"), "2");
});
test("writing a tab leaves an existing entity selection alone", () => {
const search = applyTab(applyEntitySelection("", "urn:x"), "health");
assert.deepEqual(parseOntologyUrlState(search), { tab: "health", entityUri: "urn:x" });
});
test("absent params read as undefined, blank params as empty strings", () => {
assert.deepEqual(parseOntologyUrlState(""), { tab: undefined, entityUri: undefined });
assert.deepEqual(parseOntologyUrlState("?other=1"), { tab: undefined, entityUri: undefined });
assert.deepEqual(parseOntologyUrlState("?ontologyTab=&ontologyEntity="), {
tab: "",
entityUri: "",
});
});
test("a present but blank param still counts as ontology deep-link state", () => {
assert.equal(hasOntologyUrlState("?ontologyEntity="), true);
assert.equal(hasOntologyUrlState("?ontologyTab="), true);
assert.equal(hasOntologyUrlState("?view=graph"), false);
assert.equal(hasOntologyUrlState(""), false);
});
test("malformed search strings degrade to plain values instead of throwing", () => {
assert.deepEqual(parseOntologyUrlState("???"), { tab: undefined, entityUri: undefined });
assert.deepEqual(parseOntologyUrlState("ontologyEntity=urn%3Ax&&=&"), {
tab: undefined,
entityUri: "urn:x",
});
});
test("entity URIs survive characters that need escaping", () => {
const entityUri = "https://example.test/vocab#Has Part/&?=";
const search = applyEntitySelection("?keep=1", entityUri);
assert.equal(parseOntologyUrlState(search).entityUri, entityUri);
});
test("every writer preserves the URL fragment", () => {
const written = withStubbedLocation("?view=graph", "#section-3", () => {
writeTab("health");
writeEntitySelection("urn:x");
clearEntitySelection();
});
assert.deepEqual(written, [
"?view=graph&ontologyTab=health#section-3",
"?view=graph&ontologyTab=editor&ontologyEntity=urn%3Ax#section-3",
"?view=graph#section-3",
]);
});
test("readOntologyUrlState with no argument reads live URL state", () => {
withStubbedLocation("?ontologyTab=editor&ontologyEntity=urn%3Ax", "", () => {
assert.deepEqual(readOntologyUrlState(), { tab: "editor", entityUri: "urn:x" });
assert.equal(hasOntologyUrlState(), true);
});
});
@@ -0,0 +1,113 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
DEFAULT_MIN_DATE,
resolvePlayStepMs,
resolveScrubberBounds,
} from "../src/workspaces/GraphWorkspace/temporalScrubberBounds.ts";
const NOW = new Date("2026-09-09T10:30:00Z");
// ── resolveScrubberBounds ────────────────────────────────────────────────────
test("scrubber bounds: open max ends the window at now, not at a future year", () => {
const { minBound, maxBound } = resolveScrubberBounds({ minDate: "2026-01-15T00:00:00Z", now: NOW });
assert.equal(minBound.toISOString(), "2026-01-15T00:00:00.000Z");
assert.equal(
maxBound.getTime(),
NOW.getTime(),
"a graph carrying only valid_from instants is known up to the present and no further",
);
});
test("scrubber bounds: playhead starts at now so the first snapshot describes the present", () => {
const { defaultTime } = resolveScrubberBounds({ minDate: "2026-01-15T00:00:00Z", now: NOW });
assert.equal(defaultTime.getTime(), NOW.getTime());
});
test("scrubber bounds: playhead is not the midpoint of the range", () => {
const { minBound, maxBound, defaultTime } = resolveScrubberBounds({
minDate: "2020-01-01T00:00:00Z",
maxDate: "2030-01-01T00:00:00Z",
now: NOW,
});
const midpoint = Math.round((minBound.getTime() + maxBound.getTime()) / 2);
assert.notEqual(defaultTime.getTime(), midpoint, "the midpoint was the source of the future start time");
assert.equal(defaultTime.getTime(), NOW.getTime());
});
test("scrubber bounds: reported max is honoured when the data supplies one", () => {
const { maxBound } = resolveScrubberBounds({
minDate: "2020-01-01T00:00:00Z",
maxDate: "2030-06-01T00:00:00Z",
now: NOW,
});
assert.equal(maxBound.toISOString(), "2030-06-01T00:00:00.000Z");
});
test("scrubber bounds: playhead clamps into a range that ends before now", () => {
const { maxBound, defaultTime } = resolveScrubberBounds({
minDate: "2019-01-01T00:00:00Z",
maxDate: "2020-01-01T00:00:00Z",
now: NOW,
});
assert.equal(defaultTime.getTime(), maxBound.getTime());
});
test("scrubber bounds: playhead clamps into a range that starts after now", () => {
const { minBound, defaultTime } = resolveScrubberBounds({
minDate: "2030-01-01T00:00:00Z",
maxDate: "2031-01-01T00:00:00Z",
now: NOW,
});
assert.equal(defaultTime.getTime(), minBound.getTime());
});
test("scrubber bounds: min ahead of an open max keeps the window ordered", () => {
const { minBound, maxBound } = resolveScrubberBounds({ minDate: "2031-01-01T00:00:00Z", now: NOW });
assert.ok(maxBound >= minBound, "vis-timeline requires min <= max");
});
test("scrubber bounds: malformed and missing dates fall back without producing Invalid Date", () => {
const { minBound, maxBound } = resolveScrubberBounds({ minDate: "not-a-date", maxDate: "also-bad", now: NOW });
assert.equal(minBound.getTime(), DEFAULT_MIN_DATE.getTime());
assert.equal(maxBound.getTime(), NOW.getTime());
});
// ── resolvePlayStepMs ────────────────────────────────────────────────────────
test("play step: a one-year span advances in ~60 frames, not 2", () => {
const minBound = new Date("2026-01-01T00:00:00Z");
const maxBound = new Date("2027-01-01T00:00:00Z");
const span = maxBound.getTime() - minBound.getTime();
const frames = span / resolvePlayStepMs(minBound, maxBound);
assert.ok(frames > 50 && frames < 70, `expected ~60 frames, got ${frames}`);
});
test("play step: a decade-long span also advances in ~60 frames", () => {
const minBound = new Date("2016-01-01T00:00:00Z");
const maxBound = new Date("2026-01-01T00:00:00Z");
const span = maxBound.getTime() - minBound.getTime();
const frames = span / resolvePlayStepMs(minBound, maxBound);
assert.ok(frames > 50 && frames < 70, `expected ~60 frames, got ${frames}`);
});
test("play step: a span of hours still advances by at least a day", () => {
const minBound = new Date("2026-09-09T00:00:00Z");
const maxBound = new Date("2026-09-09T06:00:00Z");
assert.equal(resolvePlayStepMs(minBound, maxBound), 1000 * 60 * 60 * 24);
});
+89 -83
View File
@@ -21,6 +21,7 @@ from typing import Any, List, Optional
try: try:
from google.adk.events import Event from google.adk.events import Event
from google.adk.sessions import BaseSessionService, Session from google.adk.sessions import BaseSessionService, Session
try: try:
from google.adk.sessions import ListSessionsResponse from google.adk.sessions import ListSessionsResponse
except ImportError: except ImportError:
@@ -33,7 +34,7 @@ try:
except ImportError: except ImportError:
from google.adk.sessions.base_session_service import GetSessionConfig from google.adk.sessions.base_session_service import GetSessionConfig
ADK_AVAILABLE = True ADK_AVAILABLE = True
except (ImportError, ModuleNotFoundError): except (ImportError, OSError):
ADK_AVAILABLE = False ADK_AVAILABLE = False
BaseSessionService = object BaseSessionService = object
Session = Any Session = Any
@@ -162,10 +163,10 @@ class SemanticaSessionService(BaseSessionService):
return {} return {}
def _find_session_node( def _find_session_node(
self, self,
app_name: str, app_name: str,
user_id: str, user_id: str,
session_id: str, session_id: str,
) -> Optional[Any]: ) -> Optional[Any]:
"""Find a session node by its logical ADK session ID.""" """Find a session node by its logical ADK session ID."""
expected_node_id = self._node_id(app_name, user_id, session_id) expected_node_id = self._node_id(app_name, user_id, session_id)
@@ -182,18 +183,18 @@ class SemanticaSessionService(BaseSessionService):
metadata = node.get("metadata") metadata = node.get("metadata")
if ( if (
isinstance(metadata, dict) isinstance(metadata, dict)
and str(metadata.get("session_id")) == str(session_id) and str(metadata.get("session_id")) == str(session_id)
and str(metadata.get("app_name")) == str(app_name) and str(metadata.get("app_name")) == str(app_name)
and str(metadata.get("user_id")) == str(user_id) and str(metadata.get("user_id")) == str(user_id)
): ):
return node return node
return None return None
def _find_node_by_id( def _find_node_by_id(
self, self,
node_id: str, node_id: str,
) -> Optional[Any]: ) -> Optional[Any]:
"""Find a ContextGraph node by graph node ID.""" """Find a ContextGraph node by graph node ID."""
for node in self.graph.find_nodes() or []: for node in self.graph.find_nodes() or []:
@@ -212,13 +213,13 @@ class SemanticaSessionService(BaseSessionService):
data = SemanticaSessionService._safe_dict(event) data = SemanticaSessionService._safe_dict(event)
for field in ( for field in (
"id", "id",
"invocation_id", "invocation_id",
"author", "author",
"timestamp", "timestamp",
"partial", "partial",
"turn_complete", "turn_complete",
"branch", "branch",
): ):
if field not in data and hasattr(event, field): if field not in data and hasattr(event, field):
value = getattr(event, field) value = getattr(event, field)
@@ -231,10 +232,10 @@ class SemanticaSessionService(BaseSessionService):
return data return data
def _event_nodes( def _event_nodes(
self, self,
app_name: str, app_name: str,
user_id: str, user_id: str,
session_id: str, session_id: str,
) -> List[Any]: ) -> List[Any]:
"""Return all event nodes connected to a session.""" """Return all event nodes connected to a session."""
session_node_id = self._node_id(app_name, user_id, session_id) session_node_id = self._node_id(app_name, user_id, session_id)
@@ -275,8 +276,8 @@ class SemanticaSessionService(BaseSessionService):
return str(timestamp) return str(timestamp)
def _event_from_node( def _event_from_node(
self, self,
node: Any, node: Any,
) -> Any: ) -> Any:
""" """
Reconstruct an ADK Event from its stored metadata. Reconstruct an ADK Event from its stored metadata.
@@ -289,7 +290,7 @@ class SemanticaSessionService(BaseSessionService):
if not event_id and graph_node_id: if not event_id and graph_node_id:
graph_node_id = str(graph_node_id) graph_node_id = str(graph_node_id)
if graph_node_id.startswith("adk-event:"): if graph_node_id.startswith("adk-event:"):
event_id = graph_node_id[len("adk-event:"):] event_id = graph_node_id[len("adk-event:") :]
if event_id: if event_id:
properties["id"] = event_id properties["id"] = event_id
@@ -310,11 +311,11 @@ class SemanticaSessionService(BaseSessionService):
@staticmethod @staticmethod
def _session_kwargs( def _session_kwargs(
app_name: str, app_name: str,
user_id: str, user_id: str,
session_id: str, session_id: str,
state: Optional[dict], state: Optional[dict],
events: Optional[List[Any]], events: Optional[List[Any]],
) -> dict: ) -> dict:
"""Build kwargs for the ADK Session model.""" """Build kwargs for the ADK Session model."""
return { return {
@@ -326,8 +327,8 @@ class SemanticaSessionService(BaseSessionService):
} }
def _session_from_node( def _session_from_node(
self, self,
node: Any, node: Any,
) -> Session: ) -> Session:
"""Reconstruct an ADK Session from a ContextGraph node.""" """Reconstruct an ADK Session from a ContextGraph node."""
properties = self._node_properties(node) properties = self._node_properties(node)
@@ -347,14 +348,14 @@ class SemanticaSessionService(BaseSessionService):
# splitting on ':' after the prefix always yields # splitting on ':' after the prefix always yields
# exactly 3 parts regardless of what characters the # exactly 3 parts regardless of what characters the
# original app_name/user_id/session_id contained. # original app_name/user_id/session_id contained.
parts = graph_node_id[len("adk-session:"):].split(":") parts = graph_node_id[len("adk-session:") :].split(":")
if len(parts) == 3: if len(parts) == 3:
decoded = [urllib.parse.unquote(part) for part in parts] decoded = [urllib.parse.unquote(part) for part in parts]
app_name = app_name or decoded[0] app_name = app_name or decoded[0]
user_id = user_id or decoded[1] user_id = user_id or decoded[1]
session_id = decoded[2] session_id = decoded[2]
else: else:
session_id = graph_node_id[len("adk-session:"):] session_id = graph_node_id[len("adk-session:") :]
else: else:
session_id = graph_node_id session_id = graph_node_id
@@ -383,12 +384,12 @@ class SemanticaSessionService(BaseSessionService):
# ------------------------------------------------------------------ # ------------------------------------------------------------------
async def create_session( async def create_session(
self, self,
*, *,
app_name: str, app_name: str,
user_id: str, user_id: str,
state: Optional[dict[str, Any]] = None, state: Optional[dict[str, Any]] = None,
session_id: Optional[str] = None, session_id: Optional[str] = None,
) -> Session: ) -> Session:
"""Create and persist an ADK session.""" """Create and persist an ADK session."""
return await asyncio.to_thread( return await asyncio.to_thread(
@@ -396,11 +397,11 @@ class SemanticaSessionService(BaseSessionService):
) )
def _create_session_sync( def _create_session_sync(
self, self,
app_name: str, app_name: str,
user_id: str, user_id: str,
state: Optional[dict[str, Any]], state: Optional[dict[str, Any]],
session_id: Optional[str], session_id: Optional[str],
) -> Session: ) -> Session:
with self._lock: with self._lock:
session_id = session_id or str(uuid.uuid4()) session_id = session_id or str(uuid.uuid4())
@@ -430,12 +431,12 @@ class SemanticaSessionService(BaseSessionService):
) )
async def get_session( async def get_session(
self, self,
*, *,
app_name: str, app_name: str,
user_id: str, user_id: str,
session_id: str, session_id: str,
config: Optional[GetSessionConfig] = None, config: Optional[GetSessionConfig] = None,
) -> Optional[Session]: ) -> Optional[Session]:
"""Retrieve an ADK session from ContextGraph.""" """Retrieve an ADK session from ContextGraph."""
return await asyncio.to_thread( return await asyncio.to_thread(
@@ -443,11 +444,11 @@ class SemanticaSessionService(BaseSessionService):
) )
def _get_session_sync( def _get_session_sync(
self, self,
app_name: str, app_name: str,
user_id: str, user_id: str,
session_id: str, session_id: str,
config: Optional[GetSessionConfig], config: Optional[GetSessionConfig],
) -> Optional[Session]: ) -> Optional[Session]:
with self._lock: with self._lock:
node = self._find_session_node(app_name, user_id, session_id) node = self._find_session_node(app_name, user_id, session_id)
@@ -468,7 +469,7 @@ class SemanticaSessionService(BaseSessionService):
# trims the already-built Session object. # trims the already-built Session object.
if config: if config:
if config.num_recent_events: if config.num_recent_events:
session.events = session.events[-config.num_recent_events:] session.events = session.events[-config.num_recent_events :]
if config.after_timestamp: if config.after_timestamp:
i = len(session.events) - 1 i = len(session.events) - 1
while i >= 0: while i >= 0:
@@ -476,14 +477,14 @@ class SemanticaSessionService(BaseSessionService):
break break
i -= 1 i -= 1
if i >= 0: if i >= 0:
session.events = session.events[i + 1:] session.events = session.events[i + 1 :]
return session return session
async def append_event( async def append_event(
self, self,
session: Session, session: Session,
event: Event, event: Event,
) -> Event: ) -> Event:
"""Persist an ADK event and associate it with a session.""" """Persist an ADK event and associate it with a session."""
# ADK's own base implementation is a no-op for partial/streaming # ADK's own base implementation is a no-op for partial/streaming
@@ -510,8 +511,13 @@ class SemanticaSessionService(BaseSessionService):
# Verify cross-tenant security # Verify cross-tenant security
properties = self._node_properties(session_node) properties = self._node_properties(session_node)
if properties.get("app_name") != app_name or properties.get("user_id") != user_id: if (
raise ValueError("Cross-tenant session write denied: app_name or user_id mismatch.") properties.get("app_name") != app_name
or properties.get("user_id") != user_id
):
raise ValueError(
"Cross-tenant session write denied: app_name or user_id mismatch."
)
# Apply ADK in-memory event and state delta semantics. This # Apply ADK in-memory event and state delta semantics. This
# runs inside asyncio.to_thread's worker thread, which has no # runs inside asyncio.to_thread's worker thread, which has no
@@ -553,11 +559,11 @@ class SemanticaSessionService(BaseSessionService):
) )
async def delete_session( async def delete_session(
self, self,
*, *,
app_name: str, app_name: str,
user_id: str, user_id: str,
session_id: str, session_id: str,
) -> None: ) -> None:
"""Delete a session and all of its graph-backed events.""" """Delete a session and all of its graph-backed events."""
await asyncio.to_thread( await asyncio.to_thread(
@@ -565,10 +571,10 @@ class SemanticaSessionService(BaseSessionService):
) )
def _delete_session_sync( def _delete_session_sync(
self, self,
app_name: str, app_name: str,
user_id: str, user_id: str,
session_id: str, session_id: str,
) -> None: ) -> None:
with self._lock: with self._lock:
session_node = self._find_session_node(app_name, user_id, session_id) session_node = self._find_session_node(app_name, user_id, session_id)
@@ -590,9 +596,9 @@ class SemanticaSessionService(BaseSessionService):
continue continue
if ( if (
edge.get("source") == session_node_id edge.get("source") == session_node_id
and edge.get("type") == "HAS_EVENT" and edge.get("type") == "HAS_EVENT"
and edge.get("target") and edge.get("target")
): ):
event_node_ids.append(str(edge["target"])) event_node_ids.append(str(edge["target"]))
@@ -602,18 +608,18 @@ class SemanticaSessionService(BaseSessionService):
self.graph.purge_node(session_node_id) self.graph.purge_node(session_node_id)
async def list_sessions( async def list_sessions(
self, self,
*, *,
app_name: str, app_name: str,
user_id: Optional[str] = None, user_id: Optional[str] = None,
) -> ListSessionsResponse: ) -> ListSessionsResponse:
"""List sessions for an app, optionally scoped to one user.""" """List sessions for an app, optionally scoped to one user."""
return await asyncio.to_thread(self._list_sessions_sync, app_name, user_id) return await asyncio.to_thread(self._list_sessions_sync, app_name, user_id)
def _list_sessions_sync( def _list_sessions_sync(
self, self,
app_name: str, app_name: str,
user_id: Optional[str], user_id: Optional[str],
) -> ListSessionsResponse: ) -> ListSessionsResponse:
with self._lock: with self._lock:
sessions: List[Session] = [] sessions: List[Session] = []
@@ -643,4 +649,4 @@ class SemanticaSessionService(BaseSessionService):
__all__ = [ __all__ = [
"ADK_AVAILABLE", "ADK_AVAILABLE",
"SemanticaSessionService", "SemanticaSessionService",
] ]
+20
View File
@@ -0,0 +1,20 @@
# OSV-Scanner ignore config (also consumed by OpenSSF Scorecard's
# "Vulnerabilities" check, which reports advisories found in this repo's
# dependency manifests via https://osv.dev).
#
# See https://github.com/google/osv-scanner#ignore-vulnerabilities-by-id for
# the file format.
[[IgnoredVulns]]
id = "GHSA-4j2p-28q2-5m79"
reason = """
accelerate<=1.14.0 (transitive dependency via docling-slim, pinned in
requirements-ci.txt) has an open path traversal / DoS advisory (also tracked
as CVE-2026-69112) in load_checkpoint_in_model / load_checkpoint_and_dispatch,
which fail to sanitize weight_map entries from sharded checkpoint indexes.
1.14.0 is the latest release on PyPI; no patched version exists yet.
Semantica does not call either function or load arbitrary/untrusted sharded
checkpoints, so the vulnerable code path is not reachable. Re-evaluate once
accelerate ships a fix - see .github/workflows/security-scan.yml for the
matching pip-audit exclusion.
"""
+56 -14
View File
@@ -1,38 +1,80 @@
--- ---
name: change name: change
description: Track and inspect graph changes, diffs, temporal updates, and the impact of new data on Semantica knowledge graphs. description: Inspect graph changes over time and ontology version diffs in Semantica. Uses ContextGraph.state_at for point-in-time graph state and change_management.VersionManager for ontology versioning.
--- ---
# /semantica:change # /semantica:change
Inspect changes over time and evaluate updates. Usage: `/semantica:change <task> [args]` Track what changed. Usage: `/semantica:change <task> [args]`
`$ARGUMENTS` = task + optional node, time window, or filter. > Two distinct mechanisms cover this, and they are **not** interchangeable:
>
> | Question | Tool |
> | --- | --- |
> | "What did the *graph* look like on date X?" | `ContextGraph.state_at()` |
> | "What changed between *ontology* versions?" | `change_management.VersionManager` |
--- ---
## `diff [--from <ts>] [--to <ts>] [--node <id>]` ## `graph-at <timestamp>` — point-in-time graph state
Compute graph diffs between two snapshots.
```python ```python
from semantica.provenance.change_tracker import ChangeTracker import os
from semantica.context import ContextGraph from semantica.context import ContextGraph
tracker = ChangeTracker() graph = ContextGraph()
diff = tracker.compute_diff(from_ts=from_ts, to_ts=to_ts, node_id=node_id) graph.load_from_file(os.path.expanduser("~/.semantica/kg.json")) # load_from_file does not expand ~
snapshot = graph.state_at("2026-06-01") # str | int | float | datetime
``` ```
Output: added/removed nodes and edges, attribute changes, and impact summary. Diff two moments by comparing node IDs — `state_at()["nodes"]` is a list of
dicts (unhashable), so compare the `id` fields, not the dicts themselves:
```python
before = graph.state_at("2026-06-01")
after = graph.state_at("2026-09-01")
before_ids = {n["id"] for n in before["nodes"]}
after_ids = {n["id"] for n in after["nodes"]}
added = after_ids - before_ids
```
For richer temporal work (scrubbing, evolution, temporal patterns) use
`/semantica:temporal`, which wraps the same layer.
--- ---
## `history <node_id> [--limit N]` ## `node-history <node_id>` — who touched this node
Show the change history for a node or relationship. Node-level history is provenance, not change management:
```python ```python
history = tracker.get_node_history(node_id=node_id, limit=limit) import os
from semantica.provenance import ProvenanceManager
db_path = os.path.expanduser("~/.semantica/prov.db") # storage_path is passed to
os.makedirs(os.path.dirname(db_path), exist_ok=True) # sqlite3.connect() unexpanded
pm = ProvenanceManager(storage_path=db_path)
history = pm.revision_history(node_id)
log = pm.audit_log(since="2026-01-01")
``` ```
Return: revisions, timestamps, authors, and summary comments. ---
## `versions` / `diff <v1> <v2>` — ontology versioning
```python
from semantica.change_management import VersionManager
vm = VersionManager()
vm.create_version("1.1.0", ontology)
vm.list_versions()
vm.get_latest_version()
delta = vm.compare_versions("1.0.0", "1.1.0")
delta = vm.diff_ontologies(base_ontology, target_ontology)
migrated = vm.migrate_ontology("1.0.0", "1.1.0", ontology)
```
`TemporalVersionManager` and `OntologyVersionManager` are also exported for
time-scoped and ontology-specific variants.
+2 -2
View File
@@ -1,6 +1,6 @@
--- ---
name: decision name: decision
description: Full decision lifecycle in Semantica — record, query, find precedents (hybrid/advanced), analyze influence, explain, insights dashboard, list, and record exceptions. Uses AgentContext, ContextGraph, DecisionQuery, CausalChainAnalyzer, DecisionRecorder. description: Full decision lifecycle in Semantica — record, query, find precedents (hybrid/advanced), analyze influence, explain, insights dashboard, list, and record exceptions. Uses AgentContext, ContextGraph, DecisionQuery, CausalChainAnalyzer, DecisionRecorder.
--- ---
# /semantica:decision # /semantica:decision
@@ -114,7 +114,7 @@ Output: Influence score + influenced decisions table + predicted new relationshi
## `explain <decision_id>` ## `explain <decision_id>`
Full explainability trace — reasoning steps, causal antecedents, policy compliance. Full explainability trace — reasoning steps, causal antecedents, policy compliance.
```python ```python
from semantica.context import AgentContext, ContextGraph from semantica.context import AgentContext, ContextGraph
+2 -2
View File
@@ -21,8 +21,8 @@ Run the full extraction pipeline. Usage: `/semantica:extract [file_path | "inlin
**2. Clear the result cache** to prevent cross-invocation pollution: **2. Clear the result cache** to prevent cross-invocation pollution:
```python ```python
from semantica.semantic_extract.cache import _result_cache from semantica.semantic_extract.cache import extraction_cache
_result_cache.clear() extraction_cache.clear()
``` ```
**3. Run the full pipeline:** **3. Run the full pipeline:**
+65 -13
View File
@@ -1,37 +1,89 @@
--- ---
name: ontology name: ontology
description: Manage ontology schemas, concepts, relationships, and alignments for Semantica knowledge graphs. description: Manage ontology schemas, concepts, alignments, and SHACL/OWL validation for Semantica knowledge graphs. Uses OntologyEngine and OntologyValidator.
--- ---
# /semantica:ontology # /semantica:ontology
Manage ontology definitions and validation. Usage: `/semantica:ontology <task> [args]` Manage ontology definitions and validation. Usage: `/semantica:ontology <task> [args]`
`$ARGUMENTS` = task + optional ontology item or schema file. > Entry points: `OntologyEngine` (authoring, export, alignments) and
> `OntologyValidator` (consistency checking).
--- ---
## `describe <concept>` ## `concepts <scheme_uri>`
Show ontology concept details. List SKOS concepts in a vocabulary scheme.
```python ```python
from semantica.ontology import OntologyManager from semantica.ontology import OntologyEngine
from semantica.triplet_store import TripletStore
manager = OntologyManager() store = TripletStore(backend="oxigraph") # needs semantica[tripletstore-oxigraph]
concept = manager.get_concept(concept_name) engine = OntologyEngine(store=store) # list_concepts/list_vocabularies need a
# configured store — raises ProcessingError without one
concepts = engine.list_concepts(scheme_uri)
vocabs = engine.list_vocabularies()
``` ```
Output: properties, relationships, inherited types, and examples.
--- ---
## `validate [--schema <file>]` ## `validate <ontology>`
Validate the graph or schema against the ontology. Check an ontology for consistency and satisfiability.
```python ```python
result = manager.validate_graph(graph=graph, schema_file=schema_file) from semantica.ontology import OntologyValidator
validator = OntologyValidator(check_consistency=True, check_satisfiability=True)
result = validator.validate(ontology) # dict or path to an ontology file
# result.valid, result.errors, result.warnings
``` ```
Return: validation status, errors, and correction suggestions. For SHACL shape validation of instance data use `SHACLGenerator` / `SHACLValidationReport`:
```python
from semantica.ontology import SHACLGenerator
```
---
## `build <text|data>`
Generate an ontology from unstructured text or structured records.
```python
engine = OntologyEngine()
onto = engine.from_text(text) # LLM-assisted (needs an llm-* extra + API key)
onto = engine.from_data(records) # deterministic, from structured data
```
---
## `export <ontology> <path> [--format turtle]`
```python
engine.export_owl(onto, path, format="turtle")
engine.export_shacl(onto, path, format="turtle")
```
---
## `align <source_uri> <target_uri> <predicate>`
```python
engine.create_alignment(source_uri, target_uri, predicate)
engine.get_alignments(entity_uri)
engine.list_alignments()
```
---
## `evaluate <ontology>`
Quality-gate an ontology (`OntologyEvaluator` / `OntologyQualityReport` under the hood).
```python
report = engine.evaluate(onto)
```
+42 -13
View File
@@ -1,37 +1,66 @@
--- ---
name: policy name: policy
description: Define and enforce policies, access controls, and compliance rules over Semantica knowledge graphs. description: Define and enforce decision policies, compliance rules, and exceptions over Semantica graphs. Uses ContextGraph.check_decision_rules/enforce_decision_policy and context.PolicyEngine.
--- ---
# /semantica:policy # /semantica:policy
Apply policy rules and checks. Usage: `/semantica:policy <task> [args]` Policy governance over recorded decisions. Usage: `/semantica:policy <task> [args]`
`$ARGUMENTS` = task + optional policy name, rule set, or target entity. > `PolicyEngine` lives in `semantica.context`. For most cases the two policy
> methods on `ContextGraph` itself are enough.
--- ---
## `check [--rule <name>] [--target <id>]` ## `check <decision>` — the simple path
Run policy checks against the graph. No policy store needed; rules default to a built-in policy set.
```python ```python
from semantica.policy import PolicyEngine from semantica.context import ContextGraph
engine = PolicyEngine() graph = ContextGraph()
result = engine.check(rule_name=rule_name, target=target) result = graph.check_decision_rules({
"category": "vendor_selection",
"outcome": "approved",
"confidence": 0.93,
"decision_maker": "gyro",
})
# {'compliant': bool, 'violations': [...], 'warnings': [...], 'policy_rules': {...}}
``` ```
Output: compliance status, failing rules, and remediation guidance. Default rules: `min_confidence=0.7`, `required_outcomes=['approved','rejected','flagged']`,
`required_metadata=['decision_maker']`, `max_reasoning_length=1000`. Override by
passing your own `rules=` dict.
## `enforce <decision> [--rules <dict>]`
```python
verdict = graph.enforce_decision_policy(decision_data, policy_rules=None)
```
--- ---
## `list` ## Managed policies — the full path
List available policy rules and categories. `PolicyEngine` requires a graph store and versioned `Policy` objects.
```python ```python
rules = engine.list_rules() from semantica.context import PolicyEngine
from semantica.context.decision_models import Policy
engine = PolicyEngine(graph_store)
policy_id = engine.add_policy(Policy(...))
policies = engine.get_applicable_policies(category="vendor_selection", entities=[...])
ok = engine.check_compliance(decision, policy_id)
history = engine.get_policy_history(policy_id)
engine.update_policy(policy_id, rules={...}, change_reason="tightened threshold")
engine.record_exception(decision_id, policy_id, reason="...", approver="...")
impact = engine.analyze_policy_impact(policy_id, proposed_rules={...})
affected = engine.get_affected_decisions(policy_id, from_version, to_version)
``` ```
Return: rule name, description, severity, and category. Note `check_compliance` takes a `Decision` object, not a dict — fetch it from the
graph rather than constructing one by hand.
+50 -17
View File
@@ -1,37 +1,70 @@
--- ---
name: provenance name: provenance
description: Trace data lineage, source attribution, audit trails, and provenance assertions in Semantica graphs. description: Trace data lineage, source attribution, audit trails, and W3C PROV-O export in Semantica graphs. Uses ProvenanceManager.
--- ---
# /semantica:provenance # /semantica:provenance
Inspect provenance metadata. Usage: `/semantica:provenance <task> [args]` Lineage and audit trails. Usage: `/semantica:provenance <task> [args]`
`$ARGUMENTS` = task + optional node, edge, or time range.
--- ---
## `trace <node_id> [--depth N]` ## `lineage <entity_id> [--depth N]`
Trace the provenance of a node or fact.
```python ```python
from semantica.provenance import ProvenanceTracer import os
from semantica.provenance import ProvenanceManager
tracer = ProvenanceTracer() db_path = os.path.expanduser("~/.semantica/prov.db") # storage_path is passed to
trace = tracer.trace_node(node_id=node_id, depth=depth) os.makedirs(os.path.dirname(db_path), exist_ok=True) # sqlite3.connect() unexpanded
pm = ProvenanceManager(storage_path=db_path) # SQLite, or omit for in-memory
chain = pm.lineage(entity_id, depth=3)
full = pm.get_lineage(entity_id) # complete ancestry
down = pm.get_descendants(entity_id) # what this entity influenced
``` ```
Output: source chain, authors, timestamps, and validation status.
--- ---
## `audit [--since <ts>] [--actor <id>]` ## `sources <entity_id>`
View audit logs for graph changes.
```python ```python
audit_log = tracer.get_audit_log(since=since, actor=actor) srcs = pm.get_all_sources(entity_id) # every source that contributed
prov = pm.get_provenance(entity_id) # the raw PROV entry
hist = pm.revision_history(entity_id)
``` ```
Return: change events, actor, affected objects, and action details. ---
## `audit [--since <iso-date>] [--format table|json]`
```python
log = pm.audit_log(since="2026-01-01", format="table")
between = pm.query_recorded_between(start, end)
stats = pm.get_statistics()
```
---
## `export [--format turtle|json-ld|xml]`
W3C PROV-O export — this is the regulator-facing artifact.
```python
rdf = pm.export_prov(format="turtle", base_uri="https://example.org/prov/")
```
---
## `invalidate <entity_id> <agent_id> [--reason ...]`
Mark an entity superseded without deleting history.
```python
pm.invalidate(entity_id, agent_id, reason="source retracted")
```
## `check [--strict]`
```python
report = pm.check(strict=False) # integrity check over the provenance store
```
+54 -19
View File
@@ -1,49 +1,84 @@
--- ---
name: query name: query
description: Query the Semantica knowledge graph using SPARQL, Cypher, keyword search, and structured graph query patterns. description: Query Semantica knowledge graphs — in-memory ContextGraph search, SPARQL over RDF triple stores, and Cypher over LPG backends.
--- ---
# /semantica:query # /semantica:query
Run graph queries and search. Usage: `/semantica:query <mode> [args]` Query the graph. Usage: `/semantica:query <task> [args]`
`$ARGUMENTS` = query mode + query string or filter. > Which API you want depends on where the graph lives.
--- ---
## `sparql <query>` ## `search "<keywords>"` — the in-memory ContextGraph
Execute a SPARQL query against the graph. This is the one that works with no external server.
```python ```python
from semantica.query import QueryEngine import os
from semantica.context import ContextGraph
engine = QueryEngine() graph = ContextGraph()
results = engine.query_sparql(query) graph.load_from_file(os.path.expanduser("~/.semantica/kg.json")) # load_from_file does not expand ~
results = graph.query("vendor selection", skip=0, limit=20)
``` ```
Return: query bindings as a Markdown table. Related lookups on the same object:
```python
graph.find_nodes(...) graph.find_node(...)
graph.find_related_nodes(...) graph.get_neighbors(node_id)
graph.find_similar_nodes(...) graph.get_nodes_by_label(label)
```
Decision-specific queries belong to `/semantica:decision`.
--- ---
## `cypher <query>` ## `sparql "<query>"` — RDF triple stores
Execute a Cypher-like query.
```python ```python
results = engine.query_cypher(query) from semantica.triplet_store import TripletStore
store = TripletStore(backend="oxigraph") # embedded; needs semantica[tripletstore-oxigraph]
# or backend="blazegraph" | "jena" | "rdf4j" with endpoint="http://..."
result = store.execute_query(sparql)
``` ```
Output: node/relationship results and path summaries. For query planning, optimisation, and caching over a backend:
```python
from semantica.triplet_store import QueryEngine, OxigraphStore
# QueryEngine needs an object exposing execute_sparql() — the raw backend,
# not the TripletStore wrapper above (which only exposes execute_query()).
backend = OxigraphStore()
qe = QueryEngine()
plan = qe.plan_query(sparql)
tuned = qe.optimize_query(sparql)
result = qe.execute_query(sparql, store_backend=backend)
stats = qe.get_query_statistics()
```
Blazegraph / Jena / RDF4J need **no** extra — `semantica.triplet_store` speaks
SPARQL over HTTP using the core `requests` dependency.
--- ---
## `search <keywords> [--filter <type>]` ## `cypher "<query>"` — labeled property graphs
Search graph entities by keyword.
```python ```python
results = engine.search(keywords=keywords, filter_type=filter_type) from semantica.graph_store import Neo4jStore # needs semantica[graph-neo4j]
store = Neo4jStore(uri=..., user=..., password=...)
result = store.execute_query(query, parameters={...})
``` ```
Return: ranked matches with entity types and relevance scores. Also available: `FalkorDBStore`, `ApacheAgeStore`, `AmazonNeptuneStore`,
and `GraphManager` / `GraphStore` for backend-agnostic access.
**Not installed in this environment** — add the backend extra first, e.g.
`pip install "semantica[graph-neo4j]"`.
+2 -2
View File
@@ -119,9 +119,9 @@ from semantica.semantic_extract import (
NamedEntityRecognizer, NamedEntityRecognizer,
RelationExtractor, RelationExtractor,
) )
from semantica.semantic_extract.cache import _result_cache from semantica.semantic_extract.cache import extraction_cache
_result_cache.clear() # prevent cross-invocation cache pollution extraction_cache.clear() # prevent cross-invocation cache pollution
text = open(file_path).read() text = open(file_path).read()
+90 -53
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "semantica" name = "semantica"
version = "0.6.8" version = "0.7.0"
description = "Graph-Native Infrastructure for Context and Accountable AI Systems: context graphs, decision intelligence, full provenance tracking, and explainable reasoning engines — every AI decision traceable, every output auditable." description = "Graph-Native Infrastructure for Context and Accountable AI Systems: context graphs, decision intelligence, full provenance tracking, and explainable reasoning engines — every AI decision traceable, every output auditable."
readme = "README.md" readme = "README.md"
license = { text = "MIT" } license = { text = "MIT" }
@@ -12,7 +12,11 @@ license = { text = "MIT" }
authors = [{ name = "Semantica", email = "kaif@getsemantica.ai" }] authors = [{ name = "Semantica", email = "kaif@getsemantica.ai" }]
maintainers = [{ name = "Semantica", email = "kaif@getsemantica.ai" }] maintainers = [{ name = "Semantica", email = "kaif@getsemantica.ai" }]
requires-python = ">=3.8" # 3.8 is already unsatisfiable in practice (numpy>=2.0.2 requires >=3.9) and is
# not exercised by the Install Matrix (3.9-3.12). The floor is 3.9.2 rather
# than 3.9.0 because cryptography (db-snowflake) excludes 3.9.0/3.9.1 from
# every release's requires-python, so those patch levels can never resolve.
requires-python = ">=3.9.2"
classifiers = [ classifiers = [
"Development Status :: 5 - Production/Stable", "Development Status :: 5 - Production/Stable",
@@ -22,7 +26,6 @@ classifiers = [
"License :: OSI Approved :: MIT License", "License :: OSI Approved :: MIT License",
"Operating System :: OS Independent", "Operating System :: OS Independent",
"Programming Language :: Python :: 3", "Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.11",
@@ -52,30 +55,13 @@ dependencies = [
# last 3.9-compatible release line; 3.10+ is left unconstrained. # last 3.9-compatible release line; 3.10+ is left unconstrained.
"scikit-learn>=1.6.1,<1.7.0; python_version < '3.10'", "scikit-learn>=1.6.1,<1.7.0; python_version < '3.10'",
"scikit-learn>=1.7.2; python_version >= '3.10'", "scikit-learn>=1.7.2; python_version >= '3.10'",
"umap-learn>=0.5.12",
# thinc (spacy's core dep) dropped Python 3.9 wheels at 8.3.10, and later
# spacy patch releases (3.8.8+) require thinc>=8.3.9-only-on-3.10+ ranges,
# which forces a source build that fails outright on 3.9 (see Install
# Matrix run history). Capping both keeps 3.9 on the last wheel-compatible
# pair; 3.10+ is left unconstrained to always get the latest spacy/thinc.
"spacy>=3.4.0,<3.8.8; python_version < '3.10'",
"spacy>=3.4.0; python_version >= '3.10'",
"thinc<8.3.5; python_version < '3.10'",
"transformers>=4.20.0",
"torch>=1.13.1",
"sentence-transformers>=2.2.0",
"rdflib>=6.2.0", "rdflib>=6.2.0",
"networkx>=2.8.0", "networkx>=2.8.0",
"matplotlib>=3.9.4",
"seaborn>=0.13.2",
"plotly>=6.8.0",
"ipywidgets>=8.0.0",
# requests dropped Python 3.9 support at 2.33.0 (requires_python >=3.10), # requests dropped Python 3.9 support at 2.33.0 (requires_python >=3.10),
# so an unqualified >=2.34.2 floor is unsatisfiable on 3.9. Cap 3.9 to the # so an unqualified >=2.34.2 floor is unsatisfiable on 3.9. Cap 3.9 to the
# last 3.9-compatible release; 3.10+ is left unconstrained. # last 3.9-compatible release; 3.10+ is left unconstrained.
"requests>=2.32.5,<2.33.0; python_version < '3.10'", "requests>=2.32.5,<2.33.0; python_version < '3.10'",
"requests>=2.34.2; python_version >= '3.10'", "requests>=2.34.2; python_version >= '3.10'",
"GitPython>=3.1.58",
# chardet dropped Python 3.9 support at 6.0.0 (requires_python >=3.10), so # chardet dropped Python 3.9 support at 6.0.0 (requires_python >=3.10), so
# an unqualified >=7.4.3 floor is unsatisfiable on 3.9. Cap 3.9 to the last # an unqualified >=7.4.3 floor is unsatisfiable on 3.9. Cap 3.9 to the last
# 3.9-compatible release; 3.10+ is left unconstrained. # 3.9-compatible release; 3.10+ is left unconstrained.
@@ -87,26 +73,11 @@ dependencies = [
# 3.9-compatible release; 3.10+ is left unconstrained. # 3.9-compatible release; 3.10+ is left unconstrained.
"grpcio>=1.80.0,<1.81.0; python_version < '3.10'", "grpcio>=1.80.0,<1.81.0; python_version < '3.10'",
"grpcio>=1.81.1; python_version >= '3.10'", "grpcio>=1.81.1; python_version >= '3.10'",
"beautifulsoup4>=4.15.0",
"lxml>=6.1.1",
"python-docx>=1.2.0",
"openpyxl>=3.1.5",
# pillow dropped Python 3.9 support at 12.0.0 (requires_python >=3.10), so # pillow dropped Python 3.9 support at 12.0.0 (requires_python >=3.10), so
# an unqualified >=12.2.0 floor is unsatisfiable on 3.9. Cap 3.9 to the last # an unqualified >=12.2.0 floor is unsatisfiable on 3.9. Cap 3.9 to the last
# 3.9-compatible release; 3.10+ is left unconstrained. # 3.9-compatible release; 3.10+ is left unconstrained.
"pillow>=11.3.0,<12.0.0; python_version < '3.10'", "pillow>=11.3.0,<12.0.0; python_version < '3.10'",
"pillow>=12.2.0; python_version >= '3.10'", "pillow>=12.2.0; python_version >= '3.10'",
"librosa>=0.9.0",
"opencv-python>=4.13.0.92",
"faiss-cpu>=1.7.0",
"fastembed>=0.2.0",
# onnxruntime stopped shipping cp39 wheels at 1.20.0 (its PyPI metadata
# still claims requires_python >=3.9, but no matching wheel exists), so an
# unqualified >=1.20.1 floor is unsatisfiable on 3.9. Cap 3.9 to the last
# release with a cp39 wheel; 3.10+ is left unconstrained.
"onnxruntime>=1.19.2,<1.20.0; python_version < '3.10'",
"onnxruntime>=1.20.1; python_version >= '3.10'",
"tokenizers>=0.15.0",
"pydantic>=2.13.4", "pydantic>=2.13.4",
# click dropped Python 3.9 support at 8.2.0 (requires_python >=3.10), so an # click dropped Python 3.9 support at 8.2.0 (requires_python >=3.10), so an
# unqualified >=8.4.2 floor is unsatisfiable on 3.9. Cap 3.9 to the last # unqualified >=8.4.2 floor is unsatisfiable on 3.9. Cap 3.9 to the last
@@ -120,7 +91,6 @@ dependencies = [
"python-dotenv>=1.2.1", "python-dotenv>=1.2.1",
"loguru>=0.7.3", "loguru>=0.7.3",
"structlog>=22.1.0", "structlog>=22.1.0",
"gensim>=4.4.0",
"httpx<0.29.0", "httpx<0.29.0",
"pyarrow>=14.0.0" "pyarrow>=14.0.0"
] ]
@@ -144,7 +114,10 @@ llm-anthropic = ["anthropic>=0.122.0"]
llm-ollama = ["ollama>=0.1.0"] llm-ollama = ["ollama>=0.1.0"]
llm-deepseek = ["openai>=1.0.0"] llm-deepseek = ["openai>=1.0.0"]
llm-novita = ["openai>=1.0.0"] llm-novita = ["openai>=1.0.0"]
llm-litellm = ["litellm>=1.83.9"] # litellm>=1.83.10 requires Python>=3.10, and the last 3.9-compatible release
# (1.83.9) pins python-dotenv==1.0.1, which conflicts with our >=1.2.1 core
# floor — so no litellm satisfies 3.9 at all. Gate it to 3.10+.
llm-litellm = ["litellm>=1.83.9; python_version >= '3.10'"]
llm-instructor = ["instructor>=1.15.3"] llm-instructor = ["instructor>=1.15.3"]
llm-all = [ llm-all = [
@@ -152,22 +125,53 @@ llm-all = [
] ]
# ---- Document Parsing ---- # ---- Document Parsing ----
parse-docling = ["docling>=2.107.0"] documents = [
"python-docx>=1.2.0",
"openpyxl>=3.1.5",
"lxml>=6.1.1",
"beautifulsoup4>=4.15.0"
]
# every docling release requires Python>=3.10 (no 3.9-compatible version
# exists to cap to), so gate it like google-adk below rather than split it.
parse-docling = ["docling>=2.107.0; python_version >= '3.10'"]
# pdfplumber powers the default PDFParser; not pulled in by any other extra.
parse-pdf = ["pdfplumber>=0.10.0"]
# ---- SHACL Validation ---- # ---- SHACL Validation ----
shacl = ["pyshacl>=0.25.0"] shacl = ["pyshacl>=0.25.0"]
# ---- Database Connectors ---- # ---- Database Connectors ----
db-snowflake = ["snowflake-connector-python>=4.6.0", "cryptography>=49.0.0"] # snowflake-connector-python dropped Python 3.9 support at 4.6.0
# (requires_python >=3.10), so an unqualified >=4.6.0 floor is unsatisfiable
# on 3.9. Cap 3.9 below it; 3.10+ keeps the newer floor.
db-snowflake = [
"snowflake-connector-python>=4.6.0; python_version >= '3.10'",
"snowflake-connector-python>=3.13.0,<4.6.0; python_version < '3.10'",
"cryptography>=49.0.0"
]
db-databricks = ["databricks-sdk>=0.60.0", "databricks-sql-connector>=4.0.0"] db-databricks = ["databricks-sdk>=0.60.0", "databricks-sql-connector>=4.0.0"]
db-arrow = ["pyarrow>=24.0.0"] # pyarrow dropped Python 3.9 support at 24.0.0 (requires_python >=3.10), so an
# unqualified >=24.0.0 floor is unsatisfiable on 3.9. Cap 3.9 below the last
# 3.9-compatible release line; 3.10+ is left unconstrained.
db-arrow = [
"pyarrow>=24.0.0; python_version >= '3.10'",
"pyarrow>=14.0.0,<24.0.0; python_version < '3.10'"
]
db-salesforce = ["simple-salesforce>=1.12.0"] db-salesforce = ["simple-salesforce>=1.12.0"]
ingest-parquet = ["pyarrow>=24.0.0"] db-redshift = ["redshift-connector>=2.0.0"]
ingest-arrow = ["pyarrow>=24.0.0"] ingest-parquet = [
"pyarrow>=24.0.0; python_version >= '3.10'",
"pyarrow>=14.0.0,<24.0.0; python_version < '3.10'"
]
ingest-arrow = [
"pyarrow>=24.0.0; python_version >= '3.10'",
"pyarrow>=14.0.0,<24.0.0; python_version < '3.10'"
]
ingest-sap = ["requests>=2.28.0"] ingest-sap = ["requests>=2.28.0"]
ingest-git = ["GitPython>=3.1.58"]
db-all = [ db-all = [
"semantica[db-snowflake,db-databricks,db-salesforce,db-arrow]" "semantica[db-snowflake,db-databricks,db-salesforce,db-redshift,db-arrow]"
] ]
# ---- Embedding / Models ---- # ---- Embedding / Models ----
@@ -175,22 +179,35 @@ models-huggingface = [
"transformers>=4.20.0", "transformers>=4.20.0",
"torch>=1.13.1" "torch>=1.13.1"
] ]
embeddings-local = [
"sentence-transformers>=2.2.0",
"fastembed>=0.2.0",
"onnxruntime>=1.19.2,<1.20.0; python_version < '3.10'",
"onnxruntime>=1.20.1; python_version >= '3.10'",
"tokenizers>=0.15.0"
]
nlp-spacy = [
"spacy>=3.4.0,<3.8.8; python_version < '3.10'",
"spacy>=3.4.0; python_version >= '3.10'"
]
# ---- Graph Backends ---- # ---- Graph Backends ----
graph-neo4j = ["neo4j>=5.0.0"] graph-neo4j = ["neo4j>=5.0.0"]
graph-falkordb = ["falkordb>=1.0.0", "redis>=4.3.0"] graph-falkordb = ["falkordb>=1.0.0", "redis>=4.3.0"]
graph-amazon-neptune = ["boto3>=1.24.0", "neo4j>=5.0.0"] graph-amazon-neptune = ["boto3>=1.24.0", "neo4j>=5.0.0"]
graph-apache-age = ["psycopg2-binary>=2.9.0"] graph-apache-age = ["psycopg2-binary>=2.9.0"]
graph-embeddings = ["gensim>=4.4.0"]
graph-all = [ graph-all = [
"semantica[graph-neo4j,graph-falkordb,graph-amazon-neptune,graph-apache-age]" "semantica[graph-neo4j,graph-falkordb,graph-amazon-neptune,graph-apache-age,graph-embeddings]"
] ]
# ---- Triplet Store Backends ---- # ---- Triplet Store Backends ----
tripletstore-oxigraph = ["pyoxigraph>=0.5.0"] tripletstore-oxigraph = ["pyoxigraph>=0.5.0"]
# ---- Vector Store Backends ---- # ---- Vector Store Backends ----
vectorstore-qdrant = ["qdrant-client>=1.0.0"] vectorstore-faiss = ["faiss-cpu>=1.7.0"]
vectorstore-qdrant = ["qdrant-client>=1.10.0"]
vectorstore-weaviate = ["weaviate-client>=4.0.0"] vectorstore-weaviate = ["weaviate-client>=4.0.0"]
vectorstore-pinecone = ["pinecone>=3.0.0"] vectorstore-pinecone = ["pinecone>=3.0.0"]
vectorstore-milvus = ["pymilvus>=2.0.0"] vectorstore-milvus = ["pymilvus>=2.0.0"]
@@ -198,7 +215,7 @@ vectorstore-pgvector = ["psycopg[binary,pool]>=3.0.0", "pgvector>=0.2.0"]
vectorstore-sqlite = ["sqlite-vec>=0.1.1"] vectorstore-sqlite = ["sqlite-vec>=0.1.1"]
vectorstore-all = [ vectorstore-all = [
"semantica[vectorstore-qdrant,vectorstore-weaviate,vectorstore-pinecone,vectorstore-milvus,vectorstore-pgvector,vectorstore-sqlite]" "semantica[vectorstore-qdrant,vectorstore-weaviate,vectorstore-pinecone,vectorstore-milvus,vectorstore-pgvector,vectorstore-sqlite,vectorstore-faiss]"
] ]
# ---- Infra / Queues / Workers ---- # ---- Infra / Queues / Workers ----
@@ -230,7 +247,18 @@ monitoring = [
viz = [ viz = [
"pyvis>=0.3.0", "pyvis>=0.3.0",
"graphviz>=0.21", "graphviz>=0.21",
"d3blocks>=1.0.0" "d3blocks>=1.0.0",
"matplotlib>=3.9.4",
"seaborn>=0.13.2",
"plotly>=6.8.0",
"ipywidgets>=8.0.0",
"umap-learn>=0.5.12"
]
# ---- Media ----
media = [
"librosa>=0.9.0",
"opencv-python>=4.13.0.92"
] ]
# ---- GPU ---- # ---- GPU ----
@@ -244,7 +272,9 @@ agno = ["agno>=1.0.0"]
# crewai core provides BaseTool and BaseKnowledgeSource; crewai-tools is not # crewai core provides BaseTool and BaseKnowledgeSource; crewai-tools is not
# needed (it pulls vulnerable transitive deps like chromadb) and would only # needed (it pulls vulnerable transitive deps like chromadb) and would only
# duplicate the prebuilt tooling users can install separately. # duplicate the prebuilt tooling users can install separately.
crewai = ["crewai>=0.80.0"] # No un-yanked crewai release supports Python 3.9 (all require >=3.10), so
# gate the extra to 3.10+.
crewai = ["crewai>=0.80.0; python_version >= '3.10'"]
langchain = ["langchain-core>=0.3.0"] langchain = ["langchain-core>=0.3.0"]
google-adk = ["google-adk>=1.27.0; python_version >= '3.10'"] google-adk = ["google-adk>=1.27.0; python_version >= '3.10'"]
@@ -269,15 +299,23 @@ dev = [
"isort>=6.1.0", "isort>=6.1.0",
"flake8>=4.0.0", "flake8>=4.0.0",
"mypy>=0.971", "mypy>=0.971",
"pre-commit>=4.6.0", # pre-commit dropped Python 3.9 support at 4.6.0 (requires_python >=3.10).
# Cap 3.9 below it; 3.10+ keeps the >=4.6.0 floor.
"pre-commit>=4.0.0,<4.6.0; python_version < '3.10'",
"pre-commit>=4.6.0; python_version >= '3.10'",
"jupyter>=1.0.0", "jupyter>=1.0.0",
"ipykernel>=6.15.0" "ipykernel>=6.15.0"
] ]
# Explorer Dashboard # Explorer Dashboard
# fastapi dropped Python 3.9 support at 0.129.0 (requires_python >=3.10), and
# every fastapi below that caps starlette<0.53.0 — so the 3.10+ starlette
# floor is unsatisfiable on 3.9. Cap both on 3.9; 3.10+ keeps the newer floors.
explorer = [ explorer = [
"fastapi>=0.109.2", "fastapi>=0.109.2,<0.129.0; python_version < '3.10'",
"starlette>=0.53.0", "fastapi>=0.109.2; python_version >= '3.10'",
"starlette>=0.36.3,<0.53.0; python_version < '3.10'",
"starlette>=0.53.0; python_version >= '3.10'",
"uvicorn[standard]>=0.22.0", "uvicorn[standard]>=0.22.0",
"websockets>=15.0.1", "websockets>=15.0.1",
"python-multipart>=0.0.7", "python-multipart>=0.0.7",
@@ -294,8 +332,7 @@ explorer-lite = [
# (CVE-2026-45829) with no fixed release — including it here would fail the CI # (CVE-2026-45829) with no fixed release — including it here would fail the CI
# dependency-audit/security gates. Install it explicitly via ``semantica[crewai]``. # dependency-audit/security gates. Install it explicitly via ``semantica[crewai]``.
all = [ all = [
"semantica[dev,viz,infra,cloud,monitoring,watch,llm-all,models-huggingface,split-all,graph-all,tripletstore-oxigraph,vectorstore-all,parse-docling,ingest-parquet,ingest-arrow,shacl,explorer]", "semantica[dev,viz,media,infra,cloud,monitoring,watch,llm-all,models-huggingface,embeddings-local,nlp-spacy,documents,ingest-git,graph-embeddings,split-all,graph-all,tripletstore-oxigraph,vectorstore-all,parse-docling,parse-pdf,ingest-parquet,ingest-arrow,shacl,explorer,agno,langchain,google-adk]"
"semantica[dev,viz,infra,cloud,monitoring,watch,llm-all,models-huggingface,split-all,graph-all,tripletstore-oxigraph,vectorstore-all,parse-docling,ingest-parquet,ingest-arrow,shacl,agno,langchain,google-adk]"
] ]
# ---------------- ENTRYPOINTS ---------------- # ---------------- ENTRYPOINTS ----------------
+26 -11
View File
@@ -181,6 +181,7 @@ anyio==4.14.2 \
# jupyter-server # jupyter-server
# langsmith # langsmith
# openai # openai
# pinecone
# starlette # starlette
# watchfiles # watchfiles
argon2-cffi==25.1.0 \ argon2-cffi==25.1.0 \
@@ -786,7 +787,9 @@ charset-normalizer==3.5.1 \
--hash=sha256:fd0350afdc3aabd5576f60ea109228bd5538139713c7b094c5cd27c73a98bc6f \ --hash=sha256:fd0350afdc3aabd5576f60ea109228bd5538139713c7b094c5cd27c73a98bc6f \
--hash=sha256:fd0a274c0e5f9a21565cd9d3dd749b61f96b7aa1e20a93aa1ba4029518f2e5c0 \ --hash=sha256:fd0a274c0e5f9a21565cd9d3dd749b61f96b7aa1e20a93aa1ba4029518f2e5c0 \
--hash=sha256:fdb8a068947befafba9952162645dc2fecaeb400e64584829ed5e9b2fbe21a7f --hash=sha256:fdb8a068947befafba9952162645dc2fecaeb400e64584829ed5e9b2fbe21a7f
# via requests # via
# pdfminer-six
# requests
click==8.5.0 \ click==8.5.0 \
--hash=sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360 \ --hash=sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360 \
--hash=sha256:ba0d2089de75ea0310e2dde03160e6ca10009947fb95a182f9b54021bb272e34 --hash=sha256:ba0d2089de75ea0310e2dde03160e6ca10009947fb95a182f9b54021bb272e34
@@ -1097,6 +1100,7 @@ cryptography==50.0.1 \
# azure-storage-blob # azure-storage-blob
# google-auth # google-auth
# joserfc # joserfc
# pdfminer-six
cuda-bindings==13.3.1 \ cuda-bindings==13.3.1 \
--hash=sha256:04436a9364059c84b8f9636f359eccda1cf814341f5b670c71d80d2f79dbc708 \ --hash=sha256:04436a9364059c84b8f9636f359eccda1cf814341f5b670c71d80d2f79dbc708 \
--hash=sha256:120fcc53d57903df529c3486962c56528cba5b7d6c57c99537320ed9922c8b86 \ --hash=sha256:120fcc53d57903df529c3486962c56528cba5b7d6c57c99537320ed9922c8b86 \
@@ -2733,15 +2737,15 @@ librt==0.15.0 \
--hash=sha256:fc1ed11c4ad0b91af24def2050f2840ea4567828e3dd058fbe608d982f6e5465 \ --hash=sha256:fc1ed11c4ad0b91af24def2050f2840ea4567828e3dd058fbe608d982f6e5465 \
--hash=sha256:febb1ce6cac545a54e6b769982824e955a700fdd9fbf3a08a3d82c990968b57d --hash=sha256:febb1ce6cac545a54e6b769982824e955a700fdd9fbf3a08a3d82c990968b57d
# via mypy # via mypy
litellm==1.99.0 \ litellm==1.100.0 \
--hash=sha256:1c45097e426fed2ae7fbd38b5404c3addeb203d0e1148c0a59848aabd5fe83c6 \ --hash=sha256:098a413e398e2220734cf9c8dd75fb34a38b65b64090619c2869af1fdeaf4ae5 \
--hash=sha256:5617804e838499bce8fecb41ad9bc984b7977361e557666fed0fef0c4623ce62 \ --hash=sha256:0b7ec93013e18535481cd811b776ee95c6b957b3a9fb44dc53f7c459e7e60e38 \
--hash=sha256:594bf4b6ff6b79c6aa3c3b78c0e939d4afd12687e076ba3fb608d38a5aa7f9c6 \ --hash=sha256:0f87fae695edbca27e5cf970bea52fa405fa9910fdf7c29c963d6413db767299 \
--hash=sha256:71109c323164b4b6776ff259876523e6e883a465aa1413dd51a1bda8e92efc5f \ --hash=sha256:8224c8eed9cab3319a88e6665d1275ad8faf21d353b1b22223a6d6115a302ea2 \
--hash=sha256:a43e8716da8beed04480e91b4233ff2f1ab1fedd84cad332dbc526b54a9229ca \ --hash=sha256:a07370d116905485e9ac99679bac991a0a6c81e13c99ca3f007128fbdf2b0082 \
--hash=sha256:e2b383070656fdbec4bc44602edaaed2a21e99ceee4ea0a4650c8cb381e67b59 \ --hash=sha256:c6f2f56808d05d8d2a7766129d958101eafb1a47e30ba5ddcfa900cdbc50af67 \
--hash=sha256:e42f94731665b68e263481efd79f7629e9b97eed7e10c57dc37a890eca058227 \ --hash=sha256:e3787fb7ad1f20aebdde7686a85061880bba6550c9d62bfa1cf8df089fe7899b \
--hash=sha256:e461b7ce53af990e5287cf7ae30d82c956b56dcce886f63ef39a76e678f82a3b --hash=sha256:ece94e817a453a5b3a9517c03547c428d501cea719edb728c1b260e53f78ea35
# via semantica (pyproject.toml) # via semantica (pyproject.toml)
llvmlite==0.49.0 \ llvmlite==0.49.0 \
--hash=sha256:00f16db782f4a13c78c5804aedc434e46794a77e89999a168f9401106270e50a \ --hash=sha256:00f16db782f4a13c78c5804aedc434e46794a77e89999a168f9401106270e50a \
@@ -4298,6 +4302,14 @@ patsy==1.0.3 \
--hash=sha256:79ebf4c93ff4d296e58a9d5be2b2ee31bd49d737cf11d70ffbd8a44b2de42e65 \ --hash=sha256:79ebf4c93ff4d296e58a9d5be2b2ee31bd49d737cf11d70ffbd8a44b2de42e65 \
--hash=sha256:d3dbebe8fd5f46e29912d030b63c6268647b59bf788a99e2af28a30234cf357c --hash=sha256:d3dbebe8fd5f46e29912d030b63c6268647b59bf788a99e2af28a30234cf357c
# via statsmodels # via statsmodels
pdfminer-six==20260107 \
--hash=sha256:366585ba97e80dffa8f00cebe303d2f381884d8637af4ce422f1df3ef38111a9 \
--hash=sha256:96bfd431e3577a55a0efd25676968ca4ce8fd5b53f14565f85716ff363889602
# via pdfplumber
pdfplumber==0.11.10 \
--hash=sha256:7741ea81bf165b474b153e6789d10d18e06b6ddcf3ec84289c3ef2fed6802580 \
--hash=sha256:b95b2d28c66efb0a794a83b88c6c6aea5987532a445d20a1cbcfa657022e6e57
# via semantica (pyproject.toml)
pexpect==4.9.0 \ pexpect==4.9.0 \
--hash=sha256:7236d1e080e4936be2dc3e326cec0af72acf9212a7e1d060210e70a47e253523 \ --hash=sha256:7236d1e080e4936be2dc3e326cec0af72acf9212a7e1d060210e70a47e253523 \
--hash=sha256:ee7d41123f3c9911050ea2c2dac107568dc43b2d3b0c7557a33212c398ead30f --hash=sha256:ee7d41123f3c9911050ea2c2dac107568dc43b2d3b0c7557a33212c398ead30f
@@ -4406,6 +4418,7 @@ pillow==12.3.0 \
# docling-slim # docling-slim
# fastembed # fastembed
# matplotlib # matplotlib
# pdfplumber
# python-pptx # python-pptx
# rapidocr # rapidocr
# torchvision # torchvision
@@ -5293,7 +5306,9 @@ pypdfium2==5.13.0 \
--hash=sha256:d96929bde3bd64c771ab3558ca1ffd7704cc4d872ab92cd9f8f8b8a20f7f36b8 \ --hash=sha256:d96929bde3bd64c771ab3558ca1ffd7704cc4d872ab92cd9f8f8b8a20f7f36b8 \
--hash=sha256:d96beb7f379e6c76d874ca93fcd182ac3168dd499056407070f9927fb1061b8e \ --hash=sha256:d96beb7f379e6c76d874ca93fcd182ac3168dd499056407070f9927fb1061b8e \
--hash=sha256:da5c7b74eebf40b5c1fbe1de01aa1edc8827a79fb1efd999616bc20dcaf77ba4 --hash=sha256:da5c7b74eebf40b5c1fbe1de01aa1edc8827a79fb1efd999616bc20dcaf77ba4
# via docling-slim # via
# docling-slim
# pdfplumber
pypickle==2.0.2 \ pypickle==2.0.2 \
--hash=sha256:d3307127314465fe3dc8f0162e11777d5e8284f3a29dc48b0f770d364a85d998 \ --hash=sha256:d3307127314465fe3dc8f0162e11777d5e8284f3a29dc48b0f770d364a85d998 \
--hash=sha256:d577e39cf501c7c80b1387f6d7dc885cf4efeba65f213df41226d1f24881b1e8 --hash=sha256:d577e39cf501c7c80b1387f6d7dc885cf4efeba65f213df41226d1f24881b1e8
+1 -1
View File
@@ -10,7 +10,7 @@ Main exports:
- Config: Configuration management - Config: Configuration management
""" """
__version__ = "0.6.8" __version__ = "0.7.0"
__author__ = "Semantica Contributors" __author__ = "Semantica Contributors"
__license__ = "MIT" __license__ = "MIT"
+26 -22
View File
@@ -465,7 +465,7 @@ def _show_startup(cli_ctx: CLIContext) -> None:
return return
cfg = cli_ctx.config.to_dict() cfg = cli_ctx.config.to_dict()
graph_store = ( graph_store = (
cli_ctx.store_backend or cfg.get("graph_db", {}).get("backend", "memory") cli_ctx.store_backend or cfg.get("graph_db", {}).get("backend", "neo4j")
) )
vector_store = ( vector_store = (
cli_ctx.vector_store_backend cli_ctx.vector_store_backend
@@ -851,9 +851,7 @@ def doctor(cli_ctx: CLIContext, local_json: bool, deep_embeddings: bool) -> None
# Graph store reachability # Graph store reachability
def _graph() -> str: def _graph() -> str:
cfg = cli_ctx.config.to_dict() cfg = cli_ctx.config.to_dict()
backend = cli_ctx.store_backend or cfg.get("graph_db", {}).get("backend", "memory") backend = cli_ctx.store_backend or cfg.get("graph_db", {}).get("backend", "neo4j")
if backend == "memory":
return "memory (always available)"
gs = _get_graph_store(cli_ctx) gs = _get_graph_store(cli_ctx)
gs.ping() if hasattr(gs, "ping") else gs.connect() gs.ping() if hasattr(gs, "ping") else gs.connect()
return f"{backend} reachable" return f"{backend} reachable"
@@ -880,10 +878,18 @@ def doctor(cli_ctx: CLIContext, local_json: bool, deep_embeddings: bool) -> None
def _embedding_backend(method: str) -> str: def _embedding_backend(method: str) -> str:
if method == "sentence_transformers": if method == "sentence_transformers":
import sentence_transformers # noqa: F401 import sentence_transformers # noqa: F401
note = f"importable ({importlib.metadata.version('sentence-transformers')})" try:
ver = importlib.metadata.version("sentence-transformers")
except Exception:
ver = getattr(sentence_transformers, "__version__", "installed")
note = f"importable ({ver})"
else: else:
import fastembed # noqa: F401 import fastembed # noqa: F401
note = f"importable ({importlib.metadata.version('fastembed')})" try:
ver = importlib.metadata.version("fastembed")
except Exception:
ver = getattr(fastembed, "__version__", "installed")
note = f"importable ({ver})"
if not deep: if not deep:
return note return note
try: try:
@@ -904,12 +910,12 @@ def doctor(cli_ctx: CLIContext, local_json: bool, deep_embeddings: bool) -> None
checks.append(_check( checks.append(_check(
"Embeddings (sentence-transformers)", "Embeddings (sentence-transformers)",
lambda: _embedding_backend("sentence_transformers"), lambda: _embedding_backend("sentence_transformers"),
hint="pip install sentence-transformers", hint="pip install 'semantica[embeddings-local]'",
)) ))
checks.append(_check( checks.append(_check(
"Embeddings (fastembed)", "Embeddings (fastembed)",
lambda: _embedding_backend("fastembed"), lambda: _embedding_backend("fastembed"),
hint="pip install fastembed", hint="pip install 'semantica[embeddings-local]'",
)) ))
# LLM provider keys # LLM provider keys
@@ -4764,15 +4770,10 @@ def mcp_list_tools(cli_ctx: CLIContext, local_json: bool) -> None:
cli_ctx = _require_ctx(cli_ctx) cli_ctx = _require_ctx(cli_ctx)
def _action() -> None: def _action() -> None:
try: # Same catalog the server exposes via tools/list, so `list-tools`
from semantica_mcp.mcp.tools import __all__ as tools # and `mcp start` can't drift (issue #1355).
except ImportError: from semantica_mcp.mcp.tools import TOOL_DEFINITIONS
tools = [ tools = [t["name"] for t in TOOL_DEFINITIONS]
"extract_entities", "extract_relations", "build_graph",
"query_graph", "get_graph_analytics", "run_reasoning",
"record_decision", "get_decisions", "export_graph",
"validate_shacl", "get_provenance", "embed_and_search",
]
if _is_json(cli_ctx, local_json): if _is_json(cli_ctx, local_json):
_jecho({"tools": list(tools)}) _jecho({"tools": list(tools)})
else: else:
@@ -4805,12 +4806,15 @@ def mcp_call(cli_ctx: CLIContext, tool_name: str, args: str, local_json: bool) -
tool_args = json.loads(args) tool_args = json.loads(args)
except json.JSONDecodeError as exc: except json.JSONDecodeError as exc:
raise click.ClickException(f"Invalid JSON in --args: {exc}") from exc raise click.ClickException(f"Invalid JSON in --args: {exc}") from exc
if not isinstance(tool_args, dict):
raise click.ClickException("--args must be a JSON object")
# Dispatch through the same server `mcp start` spawns; its session
# module never defined MCPSession (issue #1355).
from semantica_mcp.mcp.server import UnknownToolError, call_tool
try: try:
from semantica_mcp.mcp.session import MCPSession result = call_tool(tool_name, tool_args)
session = MCPSession(config=cli_ctx.config.to_dict()) except UnknownToolError as exc:
result = session.call_tool(tool_name, **tool_args) raise click.ClickException(str(exc)) from exc
except ImportError as exc:
raise click.ClickException(f"MCP module not available: {exc}") from exc
if _is_json(cli_ctx, local_json): if _is_json(cli_ctx, local_json):
_jecho(result if isinstance(result, (dict, list)) else {"result": str(result)}) _jecho(result if isinstance(result, (dict, list)) else {"result": str(result)})
else: else:
+5 -3
View File
@@ -688,10 +688,12 @@ class AgentContext:
if user_id: if user_id:
filter_dict["user_id"] = user_id filter_dict["user_id"] = user_id
if days_old: if days_old:
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
filter_dict["start_date"] = ( if days_old < 0:
datetime.now() - timedelta(days=days_old) raise ValueError("days_old must be non-negative")
filter_dict["end_date"] = (
datetime.now(timezone.utc) - timedelta(days=days_old)
).isoformat() ).isoformat()
filter_dict.update(filters) filter_dict.update(filters)
+11 -4
View File
@@ -152,9 +152,9 @@ class MemoryItem:
"""Reconstruct a MemoryItem from a serialised dict.""" """Reconstruct a MemoryItem from a serialised dict."""
raw_ts = data.get("timestamp") raw_ts = data.get("timestamp")
try: try:
ts = datetime.fromisoformat(raw_ts) if raw_ts else datetime.utcnow() ts = datetime.fromisoformat(raw_ts) if raw_ts else datetime.now(timezone.utc)
except (ValueError, TypeError): except (ValueError, TypeError):
ts = datetime.utcnow() ts = datetime.now(timezone.utc)
return cls( return cls(
content=data.get("content", ""), content=data.get("content", ""),
timestamp=ts, timestamp=ts,
@@ -355,7 +355,14 @@ class AgentMemory:
try: try:
memory_id = options.get("memory_id") or self._generate_memory_id() memory_id = options.get("memory_id") or self._generate_memory_id()
timestamp = options.get("timestamp") or datetime.now() # Aware UTC, deliberately: _timestamp_comparison_key interprets a
# naive stamp as LOCAL time, so a naive local "now" here and a
# naive utcnow() elsewhere differ by the host's UTC offset — on a
# UTC+8 host that pushed freshly added memories 8 hours outside
# every start_date/end_date window. One aware producer removes
# the ambiguity; legacy naive stamps keep their local-time
# meaning through the comparison key.
timestamp = options.get("timestamp") or datetime.now(timezone.utc)
if memory_id in self.memory_items: if memory_id in self.memory_items:
replacement_options = dict(options) replacement_options = dict(options)
@@ -1073,7 +1080,7 @@ class AgentMemory:
else: else:
days = 30 days = 30
cutoff_date = datetime.now() - timedelta(days=days) cutoff_date = datetime.now(timezone.utc) - timedelta(days=days)
# Delete old items # Delete old items
memory_ids_to_delete = [] memory_ids_to_delete = []
+1 -1
View File
@@ -212,7 +212,7 @@ class FastEmbedStore(ProviderStore):
self.logger.info(f"Loaded FastEmbed model: {self.model_name}") self.logger.info(f"Loaded FastEmbed model: {self.model_name}")
except (ImportError, OSError): except (ImportError, OSError):
self.logger.warning( self.logger.warning(
"fastembed not available. Install with: pip install fastembed" "fastembed not available. Install with: pip install 'semantica[embeddings-local]'"
) )
except Exception as e: except Exception as e:
self.logger.warning(f"Failed to load FastEmbed model: {e}") self.logger.warning(f"Failed to load FastEmbed model: {e}")
+4 -2
View File
@@ -35,6 +35,7 @@ try:
SENTENCE_TRANSFORMERS_AVAILABLE = True SENTENCE_TRANSFORMERS_AVAILABLE = True
except (ImportError, OSError): except (ImportError, OSError):
SentenceTransformer = None
SENTENCE_TRANSFORMERS_AVAILABLE = False SENTENCE_TRANSFORMERS_AVAILABLE = False
try: try:
@@ -42,6 +43,7 @@ try:
FASTEMBED_AVAILABLE = True FASTEMBED_AVAILABLE = True
except (ImportError, OSError): except (ImportError, OSError):
TextEmbedding = None
FASTEMBED_AVAILABLE = False FASTEMBED_AVAILABLE = False
@@ -156,7 +158,7 @@ class TextEmbedder:
else: else:
self.logger.warning( self.logger.warning(
"fastembed not available. " "fastembed not available. "
"Install with: pip install fastembed. " "Install with: pip install 'semantica[embeddings-local]'. "
"Using fallback embedding method." "Using fallback embedding method."
) )
else: else:
@@ -178,7 +180,7 @@ class TextEmbedder:
else: else:
self.logger.warning( self.logger.warning(
"sentence-transformers not available. " "sentence-transformers not available. "
"Install with: pip install sentence-transformers. " "Install with: pip install 'semantica[embeddings-local]'. "
"Using fallback embedding method." "Using fallback embedding method."
) )
+17 -14
View File
@@ -52,6 +52,23 @@ async def list_decisions(
return [_node_to_decision(node) for node in nodes[skip : skip + limit]] return [_node_to_decision(node) for node in nodes[skip : skip + limit]]
@router.get("/causal-distance", response_model=CausalDistanceReport)
async def causal_distance(
source: str = Query(..., description="Source node/decision ID"),
target: str = Query(..., description="Target node/decision ID"),
session: GraphSession = Depends(get_session),
):
"""FR-8 — Compute causal distance between two decisions via causal-edge-only traversal."""
from ...context.causal_analyzer import CausalChainAnalyzer
analyzer = CausalChainAnalyzer(session.graph)
report = await asyncio.to_thread(analyzer.interpret_causal_distance, source, target)
return CausalDistanceReport(**report)
# NOTE: static routes (e.g. /causal-distance above) must stay above this
# dynamic route — Starlette matches in definition order, otherwise the static
# path is captured as decision_id (see issue #1531).
@router.get("/{decision_id}", response_model=DecisionResponse) @router.get("/{decision_id}", response_model=DecisionResponse)
async def get_decision( async def get_decision(
decision_id: str, decision_id: str,
@@ -125,20 +142,6 @@ async def get_precedents(
return [_node_to_decision(decision) for _, decision in scored[:limit]] return [_node_to_decision(decision) for _, decision in scored[:limit]]
@router.get("/causal-distance", response_model=CausalDistanceReport)
async def causal_distance(
source: str = Query(..., description="Source node/decision ID"),
target: str = Query(..., description="Target node/decision ID"),
session: GraphSession = Depends(get_session),
):
"""FR-8 — Compute causal distance between two decisions via causal-edge-only traversal."""
from ...context.causal_analyzer import CausalChainAnalyzer
analyzer = CausalChainAnalyzer(session.graph)
report = await asyncio.to_thread(analyzer.interpret_causal_distance, source, target)
return CausalDistanceReport(**report)
@router.get("/{decision_id}/compliance", response_model=ComplianceResponse) @router.get("/{decision_id}/compliance", response_model=ComplianceResponse)
async def check_compliance( async def check_compliance(
decision_id: str, decision_id: str,
+60
View File
@@ -244,6 +244,7 @@ class EntityDetailResponse(BaseModel):
entity_type: str entity_type: str
definition: Optional[str] = None definition: Optional[str] = None
source_ontology: Optional[str] = None source_ontology: Optional[str] = None
owning_ontology: Optional[str] = None
superclasses: List[str] = Field(default_factory=list) superclasses: List[str] = Field(default_factory=list)
subclasses: List[str] = Field(default_factory=list) subclasses: List[str] = Field(default_factory=list)
domain: List[str] = Field(default_factory=list) domain: List[str] = Field(default_factory=list)
@@ -814,6 +815,55 @@ def _node_belongs_to_ontology(
return "#" not in local_name and "/" not in local_name return "#" not in local_name and "/" not in local_name
def _resolve_owning_ontology(
node: Dict[str, Any],
known_ontology_uris: set[str],
) -> Optional[str]:
"""Return the one known ontology that owns this node, or None if none does.
The most specific (longest) match wins, so a nested vocabulary claims its
own terms instead of the parent absorbing them.
Kept agreeing with _node_belongs_to_ontology by construction the same
three rules in the same order but in one pass over the candidates rather
than one pass per candidate, each of which rescanned the whole set to find
the longest namespace. That made resolution quadratic in the number of
registered ontologies.
"""
nid = str(node.get("id", ""))
if not nid:
return None
# An ontology node owns itself, ahead of any scheme_uri it may carry.
if nid in known_ontology_uris:
return nid
# An explicit owner is authoritative even when it is not registered:
# naming a different ontology by namespace guess would be worse than
# reporting the one the node itself points at.
explicit_owner = _node_source_ontology(node)
if explicit_owner:
return explicit_owner
longest_namespace: Optional[str] = None
for candidate in known_ontology_uris:
stem = candidate.rstrip("#/")
if not nid.startswith((stem + "#", stem + "/")):
continue
if longest_namespace is None or len(candidate) > len(longest_namespace):
longest_namespace = candidate
if longest_namespace is None:
return None
# Prefix ownership only extends to names minted directly in the namespace.
# A further delimiter marks a nested vocabulary, which stays unowned until
# it is registered or carries an explicit owner.
local_name = nid[len(longest_namespace.rstrip("#/")) + 1 :]
if "#" in local_name or "/" in local_name:
return None
return longest_namespace
def _is_ontology_entity(node: Dict[str, Any]) -> bool: def _is_ontology_entity(node: Dict[str, Any]) -> bool:
return _classify_node_type(node.get("type", "")) in {"class", "property", "concept", "scheme"} return _classify_node_type(node.get("type", "")) in {"class", "property", "concept", "scheme"}
@@ -1952,6 +2002,7 @@ async def get_ontology_graph(
@router.get("/entity/{entity_uri:path}", response_model=EntityDetailResponse) @router.get("/entity/{entity_uri:path}", response_model=EntityDetailResponse)
async def get_entity_detail( async def get_entity_detail(
entity_uri: str, entity_uri: str,
request: Request,
session: GraphSession = Depends(get_session), session: GraphSession = Depends(get_session),
): ):
node = await asyncio.to_thread(session.get_node, entity_uri) node = await asyncio.to_thread(session.get_node, entity_uri)
@@ -1973,12 +2024,21 @@ async def get_entity_detail(
all_nodes, _ = await asyncio.to_thread(session.get_nodes, skip=0, limit=999_999) all_nodes, _ = await asyncio.to_thread(session.get_nodes, skip=0, limit=999_999)
instance_count = sum(1 for n in all_nodes if n.get("type") == entity_uri) instance_count = sum(1 for n in all_nodes if n.get("type") == entity_uri)
# Ownership must use the same candidate set as /graph, so it goes through the
# same helper rather than being derived from all_nodes above: that scan is
# capped at 999,999, and on a larger graph a truncated set would silently
# drop ontologies and make the two endpoints disagree about who owns a node.
# The helper iterates only the ontology node types, so it is not a full scan.
known_ontology_uris = await asyncio.to_thread(
_known_ontology_uris, session, _get_registry(request)
)
return EntityDetailResponse( return EntityDetailResponse(
uri=entity_uri, label=label, uri=entity_uri, label=label,
type=ntype, entity_type=_classify_node_type(ntype), type=ntype, entity_type=_classify_node_type(ntype),
definition=definition, definition=definition,
source_ontology=props.get("scheme_uri"), source_ontology=props.get("scheme_uri"),
owning_ontology=_resolve_owning_ontology(node, known_ontology_uris),
superclasses=superclasses, subclasses=subclasses, superclasses=superclasses, subclasses=subclasses,
domain=domain, range=range_, domain=domain, range=range_,
instance_count=instance_count, properties=props, instance_count=instance_count, properties=props,
+38 -15
View File
@@ -329,12 +329,27 @@ class GraphExporter:
lines.append(' http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">') lines.append(' http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">')
lines.append("") lines.append("")
# Define attribute keys # Define attribute keys.
# GraphML requires every key id referenced by a <data> element to be
# declared here with a matching <key> element.
#
# for="all" — key is valid on both nodes and edges
# for="node" — key is valid on nodes only
# for="edge" — key is valid on edges only
#
# label : used on <node> (human-readable label) and <edge> (type).
# Declared for="all" so both uses are schema-valid.
# type : node entity type; only written on nodes.
# confidence: written on both nodes and edges when include_attributes
# is True; declared for="all" so edge confidence is valid.
lines.append(
' <key id="label" for="all" attr.name="label" attr.type="string"/>'
)
lines.append( lines.append(
' <key id="type" for="node" attr.name="type" attr.type="string"/>' ' <key id="type" for="node" attr.name="type" attr.type="string"/>'
) )
lines.append( lines.append(
' <key id="confidence" for="node" attr.name="confidence" attr.type="double"/>' ' <key id="confidence" for="all" attr.name="confidence" attr.type="double"/>'
) )
lines.append("") lines.append("")
@@ -342,23 +357,31 @@ class GraphExporter:
lines.append(' <graph id="G" edgedefault="directed">') lines.append(' <graph id="G" edgedefault="directed">')
lines.append("") lines.append("")
# xml.sax.saxutils helpers:
# escape(v) escapes & < > in text node content
# quoteattr(v) escapes & < > " ' and wraps in the
# appropriate quote character for use as an
# XML attribute value (including the quotes)
from xml.sax.saxutils import escape, quoteattr
# Export nodes # Export nodes
nodes = graph_data.get("nodes", []) nodes = graph_data.get("nodes", [])
for node in nodes: for node in nodes:
node_id = node.get("id", "") node_id = str(node.get("id") or "")
label = node.get("label", "") label = str(node.get("label") or "")
node_type = node.get("type", "") node_type = str(node.get("type") or "")
lines.append(f' <node id="{node_id}">') # quoteattr produces the surrounding quotes; do NOT add extra "…"
lines.append(f' <data key="label">{label}</data>') lines.append(f" <node id={quoteattr(node_id)}>")
lines.append(f' <data key="type">{node_type}</data>') lines.append(f" <data key=\"label\">{escape(label)}</data>")
lines.append(f" <data key=\"type\">{escape(node_type)}</data>")
# Add attributes if requested # Add attributes if requested
if self.include_attributes and "attributes" in node: if self.include_attributes and "attributes" in node:
attrs = node["attributes"] attrs = node["attributes"]
if "confidence" in attrs: if "confidence" in attrs:
lines.append( lines.append(
f' <data key="confidence">{attrs["confidence"]}</data>' f" <data key=\"confidence\">{escape(str(attrs['confidence']))}</data>"
) )
lines.append(" </node>") lines.append(" </node>")
@@ -368,19 +391,19 @@ class GraphExporter:
# Export edges # Export edges
edges = graph_data.get("edges", []) edges = graph_data.get("edges", [])
for edge in edges: for edge in edges:
source = edge.get("source", "") source = str(edge.get("source") or "")
target = edge.get("target", "") target = str(edge.get("target") or "")
edge_type = edge.get("type", "") edge_type = str(edge.get("type") or "")
lines.append(f' <edge source="{source}" target="{target}">') lines.append(f" <edge source={quoteattr(source)} target={quoteattr(target)}>")
lines.append(f' <data key="label">{edge_type}</data>') lines.append(f" <data key=\"label\">{escape(edge_type)}</data>")
# Add attributes if requested # Add attributes if requested
if self.include_attributes and "attributes" in edge: if self.include_attributes and "attributes" in edge:
attrs = edge["attributes"] attrs = edge["attributes"]
if "confidence" in attrs: if "confidence" in attrs:
lines.append( lines.append(
f' <data key="confidence">{attrs["confidence"]}</data>' f" <data key=\"confidence\">{escape(str(attrs['confidence']))}</data>"
) )
lines.append(" </edge>") lines.append(" </edge>")

Some files were not shown because too many files have changed in this diff Show More