mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
Merge branch 'main' into feat/explorer-markdown-content-view
This commit is contained in:
Mohd Kaif
+103
-2
@@ -9,9 +9,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.6.6] - 2026-08-20
|
||||
|
||||
### Added
|
||||
|
||||
- **First-class CrewAI integration** (#962)
|
||||
- **Semantica RDF vocabulary, and deterministic entity/relationship IRIs** (#1109, closes #1107, closes #1101) by @fabio-rovai, reviewed by @KaifAhmad1
|
||||
- Every RDF/JSON-LD export mints terms in `https://semantica.dev/ns#`, and until now nothing declared what those terms meant — the namespace 404s and no vocabulary shipped with the package, so a consumer receiving an export had no way to tell `sem:text` from a typo of it, and no closed-world checker could validate an export at all
|
||||
- `semantica/ontology/vocabulary/semantica-ns.ttl` declares the terms the exporters actually emit — drawn from the emitting call sites in `export/rdf_exporter.py`, `export/json_exporter.py` and `provenance/manager.py`, not from what a vocabulary "ought" to contain. Ships inside the package (`from semantica.ontology.vocabulary import vocabulary_turtle`) so it loads without a network round trip, and is the same document intended to be served at the namespace IRI once hosting/content-negotiation is sorted
|
||||
- `tests/ontology/test_vocabulary.py` ties the document to the code: every term a serializer can write must be declared, so adding a term to an exporter without declaring it fails the build
|
||||
- The missing-id fallback minted entity/relationship IRIs from Python's builtin `hash()`, randomised per process (`PYTHONHASHSEED`), so the same entity got a different IRI on every run and exports couldn't be diffed, deduplicated, or joined to an earlier provenance record. It also wrote `<semantica:entity_N>`, an IRI in the scheme `semantica` rather than the expansion of the declared prefix, so those nodes never joined with anything written through it. Minting now uses SHA-256 and writes a full IRI in the declared namespace; the same fix applies to the default entity/relationship types in the Turtle path
|
||||
- **Fixed during review** (Qodo): the temporal fallback minted from `source_id` only, while the main serializer accepts `source_id` or `source` — relationships using the second form hashed two empty strings, which the previous randomised `hash()` masked by making the IRI unstable anyway; once deterministic, unrelated relationships at the same list index collided on one IRI across exports. Endpoints are now resolved the same way `serialize_to_turtle` resolves them, before minting. `sem:confidence` also lost its declared `xsd:decimal` range: the N-Triples serializer types the same value `xsd:float`, and the two are disjoint, so declaring either contradicted one of the exporters (tracked in #1100) — a new `test_declared_ranges_do_not_contradict_what_the_exporters_emit` guards the whole class of that mistake
|
||||
- **Fixed in follow-up**: `serialize_to_rdfxml`'s default entity type still wrote the bare string `"semantica:Entity"` into an `rdf:resource` attribute, which (unlike a Turtle angle-bracket or an XML element name) is not namespace-expanded — the exact #1101 failure mode, just on the untested RDF/XML path. `json_exporter.py`'s `semantica:format` and `@type: "semantica:KnowledgeGraph"` were emitted but absent from both the vocabulary and the test's `EMITTED_TERMS` guard set, so the "undeclared terms fail the build" claim didn't actually cover them — both are now declared and guarded. `MANIFEST.in` didn't mirror the `pyproject.toml` package-data addition, so a source-distribution install could omit the vocabulary file. The cross-process minting-stability test replaced the subprocess's entire environment with a POSIX-only `PATH`, breaking it on Windows; now overrides only `PYTHONHASHSEED` on top of the inherited environment
|
||||
- **Also fixed, on the JSON-LD paths**: the first fix covered the Turtle, N-Triples and RDF/XML serializers, and left both JSON-LD writers interpolating the entity's own text into `f"semantica:entity/{text}"` and the endpoints into `f"semantica:rel/{source}_{target}"`. Three consequences, all live in 0.6.5: an entity whose text contained a space produced an invalid IRI, and a JSON-LD parser dropped that node in full rather than reporting it, so the entity disappeared from the export; every relationship carrying `source`/`target` rather than `source_id`/`target_id` minted the identical `semantica:rel/_`, collapsing all of them onto one node whose types and endpoints merged; and the JSON-LD `@id` disagreed with the Turtle IRI for the same entity, so the two serializations of one knowledge graph were two different graphs. Both JSON-LD writers now use `mint_entity_iri`/`mint_relationship_iri`, and `JSONExporter.export_entities`/`export_relationships` declare the `semantica` prefix their `@context` was already writing `semantica:entities` against — without it a processor reads that as an IRI in the scheme `semantica`, which is the original #1101 defect on a third path
|
||||
- `tests/export/test_jsonld_iri_minting.py` parses each export with a real JSON-LD processor and asserts the entity survives, the relationships stay distinct, no term expands into the `semantica` scheme, and the JSON-LD `@id` equals the Turtle IRI
|
||||
- 236 export and ontology tests pass
|
||||
|
||||
- **First-class CrewAI integration** (#988, closes #962) by @Shindevrp
|
||||
- New `pip install semantica[crewai]` extra (`crewai>=0.80.0`) — crewai core provides `BaseTool`/`BaseKnowledgeSource`, so `crewai-tools` is intentionally not included, and the extra is intentionally **not** part of the `all` bundle: crewai hard-requires `chromadb~=1.1.0`, which is affected by the unpatched pre-auth code-injection CVE-2026-45829 (see `integrations/crewai/README.md`)
|
||||
- `integrations/crewai/SemanticaKGTool` — a CrewAI `BaseTool` exposing 5 KG actions (`extract_entities`, `extract_relations`, `add_to_graph`, `query_graph`, `find_related`) backed by `NERExtractor` / `RelationExtractor` / `ContextGraph`; supports both sync `run()` and async `arun()`
|
||||
- `integrations/crewai/SemanticaDecisionTool` — a CrewAI `BaseTool` wrapping `AgentContext` with 5 decision-intelligence actions (`record_decision`, `find_precedents`, `trace_causal_chain`, `analyze_impact`, `check_policy`)
|
||||
@@ -41,6 +54,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- New `tests/export/test_distance_exporter_metric_errors.py`: 6 tests covering success, single/multiple failures, opt-out, the no-path-vs-error distinction, and default-schema stability; existing `tests/export/test_distance_exporter.py` updated for the new tuple return type
|
||||
- Full `tests/export/` suite: 77 passed
|
||||
|
||||
- **`ContextGraph.to_kg_dict()`: an adapter converting a `ContextGraph`'s internal `nodes`/`edges`/`source` shape into the canonical `entities`/`relationships`/`source_id` shape `RDFExporter` and `TemporalGraphQuery` consume** (#1081) by @cxzg007
|
||||
- Previously there was no supported way to feed a `ContextGraph` into those consumers without hand-rolling the field remapping; `to_kg_dict()` does it once, with an `entities_only` option that drops relationships left dangling by the filter
|
||||
- **Fixed during review** (Qodo): null `properties`/`metadata` on a node loaded from JSON raised `TypeError` when copied — both are now guarded with `or {}`; entity ids are coerced to `str(node_id)` to match `ContextEdge`'s already-str-coerced endpoints, so valid relationships were no longer dropped by `entities_only` filtering
|
||||
- `RDFExporter`'s validator and `TemporalGraphQuery` now also accept `source_id`/`target_id` endpoints, the shape `to_kg_dict()` emits
|
||||
|
||||
### Changed
|
||||
|
||||
- **`GraphBuilder`'s 6 public methods now have Google-style docstrings** (#878, closes #876) by @cakeni
|
||||
@@ -50,7 +68,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- **Corrected during review**: `add_temporal_edge`/`create_temporal_snapshot` docstrings overclaimed numeric-timestamp support; `_parse_time()` only special-cases `str` and `datetime`, falling back to a bare `str()` cast for anything else (not true numeric parsing). Narrowed to "datetime or ISO-formatted string"
|
||||
- **Fixed along the way**: `build()`'s `**options` documented a default only for `extract`; `extract_relations`, `extract_triplets`, `ner_method`, `relation_method`, and `triplet_method` all have concrete defaults in `_extract_from_text()` (`True`, `True`, `"llm"`, `"llm"`, `"llm"`) that were left unstated, inconsistent with CONTRIBUTING.md's own docstring example of noting defaults inline
|
||||
- `python -m pytest tests/kg/test_kg.py tests/kg/test_graph_builder_external.py -q`: 45 passed
|
||||
- **`GraphBuilder` raw-text extraction now defaults to local extractors instead of LLM extraction** (closes #930) by @dex0shubham
|
||||
- **`GraphBuilder` raw-text extraction now defaults to local extractors instead of LLM extraction** (#941, closes #930) by @dex0shubham
|
||||
- `GraphBuilder._extract_from_text()` defaulted `ner_method`, `relation_method`, and `triplet_method` to `"llm"`, and ran relation extraction unconditionally (`extract_relations` defaulted to `True`) — all four contradicting the defaults documented in the `build()` docstring at the time (`"ml"` / `"pattern"` / `False`), and diverging from the standalone extractors (`NERExtractor` defaults to `method="ml"`, `RelationExtractor` and `TripletExtractor` to `method="pattern"`). The practical effect was that any raw-text `build()` call silently required a configured provider, an API key, and network access
|
||||
- Defaults are now `ner_method="ml"`, `relation_method="pattern"`, `triplet_method="pattern"`, and `extract_relations=False`, matching the docstring. LLM extraction remains fully available and is now opt-in
|
||||
- **To restore the previous behaviour**, pass the methods explicitly:
|
||||
@@ -71,8 +89,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- New regression coverage in `tests/kg/test_graph_builder_extraction_defaults.py` pinning all four defaults, verifying that no default resolves to `"llm"`, confirming explicit LLM opt-in still routes correctly, asserting extractors are constructed once across repeated texts, covering fallback method lists (e.g. `ner_method=["pattern", "ml"]`) for all three extractors, asserting relations are forwarded to triplet extraction (and that `None` is forwarded when relation extraction is disabled or fails), and running the real default path end to end with no provider mocked. Verified to fail against the pre-fix code
|
||||
- Full `kg` suite: 473 passed
|
||||
|
||||
- **Explorer graph canvas now renders edge labels** (#1013, closes #1009) by @yzxcj797
|
||||
- `GraphCanvas.tsx` had no edge-label rendering path at all; Sigma's edge-label renderer draws `data.label`, but the graph state stored the relationship type under `edgeType`, so simply enabling the renderer would have left every edge blank. `graphSceneState`'s edge reducer now maps `edgeType` onto `label` (suppressed for hidden edges)
|
||||
- Rendering is gated behind a new `edgeLabelsEnabled` entry in the Effects panel (default on), wired through the existing `GraphEffectToggle`/`GraphEffectsState` plumbing, so dense graphs can still turn labels off
|
||||
- **Fixed during review** (Qodo): two follow-up passes closed gaps the first cut left — label rendering wasn't wired through `explorationEffectsPluginPhaseC.tsx`'s Phase C variant, and toggling the effect off mid-session didn't clear already-rendered labels
|
||||
- New coverage in `explorer/tests/graphSceneState.display.test.ts`
|
||||
|
||||
- **Removed `GraphWorkspaceShell.tsx`, `GraphRuntimeStage.tsx`, and `useGraphData.ts` — a second, unused implementation of the graph-loading/error-handling logic already fixed in `GraphWorkspace.tsx`** (#984, resolves the cleanup tracked in #981 by #980's review note) by @lakshayxi
|
||||
- 1,564 lines removed; the surviving `GraphWorkspace` path is now the only implementation, so the "two copies that drifted apart" root cause #980 fixed can't recur in the copy nobody was maintaining
|
||||
|
||||
- **Explorer README and `docs/explorer-setup.md` corrected to describe the authentication 0.6.5 actually shipped**, plus a documented `/ws/graph-updates` auth note (#1040, fixes #1028) by @Kyou12138
|
||||
- Both docs still claimed the Explorer API had no built-in authentication after v0.6.5 added mandatory `SEMANTICA_API_KEY` enforcement with a `503` fail-closed default; corrected to describe the actual behavior, including that only protected routes require the key (`/api/health`/`/api/info` stay open), the non-loopback-bind CLI warning only fires in anonymous mode or when the key is unset, and `SEMANTICA_API_KEY`/`SEMANTICA_ALLOW_ANONYMOUS` are documented in the environment-variable table
|
||||
|
||||
- **CI: pinned `github/codeql-action` to current v4** (#986) by @ZohaibHassan16, and **pinned Python dependencies in `requirements-ci.txt` for reproducible CI runs** (#945) by @yunaremaia, closing the gap where an unpinned CI dependency could silently change behavior between runs
|
||||
|
||||
- **README now states up front that Semantica's explainability is system-level, not foundation-model-internal** (#1033, #1034) by @KaifAhmad1
|
||||
- Nothing in the README previously scoped what "explainable" meant, leaving readers to assume Semantica could expose or reconstruct an LLM's internal reasoning. A callout now states explicitly that Semantica explains and audits what the AI *system* did — context fed in, decisions produced, provenance, relationships, policies applied — not the model's private internal reasoning, and moved the note near the top of the README rather than leaving it implicit
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Every timestamp an export or a provenance record wrote was timezone-naive** (closes #1114) by @fabio-rovai
|
||||
- `semantica/export/` stamped with `datetime.now().isoformat()`, which reads the machine's **local** clock; `semantica/provenance/` stamped with `datetime.utcnow().isoformat()`, which reads **UTC**. Both produce a naive value and both serialize identically, so nothing downstream can tell which zone a given timestamp belongs to — the same string means two different instants depending on which module wrote it
|
||||
- In RDF the consequence is silent rather than loud. Under XSD 1.1 a value with no timezone compared against one with a timezone is indeterminate whenever the two fall inside the ±14 hour window; SPARQL turns an indeterminate comparison into an error, and `FILTER` discards errors as non-matches. A timezone-qualified query over an Oxigraph store returns an answer with every Semantica-written record quietly absent from it, which is a poor property for `prov:generatedAtTime`, `prov:startedAtTime`, `prov:endedAtTime` and `prov:atTime` to have
|
||||
- New `utc_now()`/`utc_now_iso()` in `semantica/utils/helpers.py`, exported from `semantica.utils`, and used at all 29 call sites in `export/` (`json_exporter`, `yaml_exporter`, `report_generator`, `export_provenance`) and `provenance/` (`manager`, `schemas`, `bridge_axiom`). Values now read `2026-08-19T14:19:04.229937+00:00`: one unambiguous instant, comparable against any correctly stamped value, and valid `xsd:dateTimeStamp`. `sem:exportedAt`'s range in `semantica/ontology/vocabulary/semantica-ns.ttl` is tightened from `xsd:dateTime` accordingly, and its comment no longer has to explain why the weaker range was necessary
|
||||
- `datetime.utcnow()` is deprecated as of Python 3.12 and scheduled for removal; constructing a `ProvenanceEntry` under `-W error::DeprecationWarning` on 3.13 raised, and no longer does
|
||||
- New `tests/export/test_timestamp_timezones.py` and `tests/provenance/test_timestamp_timezones.py`: offset presence on every export and provenance path, PROV-O literals valid as `xsd:dateTimeStamp`, comparison against a timezone-aware instant without `TypeError`, the Oxigraph filter that dropped the naive value (with a bound inside the indeterminate window, so the test cannot pass by accident), and the document `@id` remaining a valid IRI with `+00:00` in it. 11 of the 13 fail on the parent commit
|
||||
- **Fixed during review** (Qodo): once new entries carry `+00:00` and stored ones do not, `ProvenanceManager.query_recorded_between` and `audit_log` compared ISO timestamps as raw strings, so they ordered by spelling rather than by instant — an inclusive naive bound naming a stored offset-bearing timestamp sorted *below* it and dropped the record, and a bound written in another offset landed wherever its digits fell (`19:45+05:30` is 14:15Z, but sorted after 14:19Z). Both now compare instants through a new `to_utc_datetime()` helper that reads a missing offset as UTC, which is what the values written before this change actually were; a bound that cannot be read as a timestamp keeps the historical string comparison rather than raising on a call that used to work
|
||||
- The remaining 147 naive call sites are in `context/`, `vector_store/`, `seed/` and elsewhere, where timestamps are compared against values parsed from previously stored naive strings. Converting those without a read-side migration would raise `TypeError: can't compare offset-naive and offset-aware datetimes` on existing data, so they are deliberately left for a separate change
|
||||
|
||||
- **`split`/chunking paths bypassed the centralized spaCy model cache, reloading the model on every call** (#1042, closes #998) by @Accute9, reviewed by @Sameer6305
|
||||
- `semantica/split/methods.py`'s `split_by_sentences()` and `semantica/split/semantic_chunker.py`'s `SemanticChunker.__init__` each called `spacy.load()` directly instead of reusing the process-level cache added in #889/`semantic_extract/methods.py`'s `load_spacy_model()` — every call/construction re-paid the ~120ms model-load cost independently of `NERExtractor`, which already used the cache
|
||||
- Both now route through `load_spacy_model()`, sharing one cached `Language` instance per model name across `split_by_sentences()`, `SemanticChunker`, and `NERExtractor`; a missing model still falls back to regex/paragraph chunking without poisoning the cache for a later successful load
|
||||
@@ -174,6 +218,59 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- New regression coverage in `tests/export/test_distance_exporter.py`: warnings fire on exception for all four helpers, exported sentinel values/shape stay unchanged, and the legitimate "no KG backend" `None` path still logs nothing
|
||||
- Full `tests/export/` suite: 71 passed
|
||||
|
||||
- **`explain_violations` rendered hardcoded placeholders (`min_count=1`, `max_count=1`) instead of the SHACL shape's real constraint values, and misused the violation message text as the datatype/class value** (#1094) by @cxzg007
|
||||
- `_run_pyshacl` never read `sh:minCount`/`sh:maxCount`/`sh:datatype`/`sh:class` back from the violation's `sh:sourceShape`, so every plain-English explanation was wrong regardless of what the shape actually declared. `SHACLViolation` now carries those four fields (also exposed via `to_dict()`), populated by back-referencing `sh:sourceShape`; `explain_violations` renders the real values, falling back to `"?"` when a value is genuinely absent
|
||||
- **Known limitation**: `sh:qualifiedMinCount`/`sh:qualifiedMaxCount` are not handled yet and still fall back to the `"?"` placeholder
|
||||
- New regression tests cover both the rendering path and the `sh:sourceShape` back-reference (skipped when `pyshacl`/`rdflib` are absent)
|
||||
|
||||
- **Entity merging silently dropped `entity_id` aliases, and exact-match entity resolution had three correctness gaps** (#1086, #1026) by @T1mn
|
||||
- `entity_merger.py`/`merge_strategy.py`/`entity_resolver.py` used inconsistent logic for extracting an entity's id across the merge path, so a merged entity could lose the `entity_id` aliases that let later lookups find it under its old identity. A new `semantica/utils/entity_ids.py` unifies id extraction across all three call sites
|
||||
- `EntityResolver`'s exact-match path is now honored rather than silently falling through to fuzzy matching in some cases; entities with no identifier are preserved instead of being dropped, and blank exact-match names are ignored rather than matching every other blank name
|
||||
- New/expanded coverage in `tests/kg/test_entity_pipeline.py` and `tests/kg/test_entity_resolver_exact.py`
|
||||
|
||||
- **`flatten_dict()` silently collided keys when a flattened path from one branch matched a literal key already present at the target depth** (#1062) by @shahzaib-ahmadcs
|
||||
- Two differently-shaped inputs could flatten to the same output key, with the second write silently overwriting the first — no error, no warning, just a dropped value. Collisions are now detected and handled explicitly instead of overwriting
|
||||
|
||||
- **`ExcelParser.__init__` raised `NameError` on every instantiation — `get_progress_tracker()` was called but never imported** (#1016, closes #1014) by @pravit-amp
|
||||
- Same defect as the one fixed for `SimilarityCalculator` in #530, this time in `semantica/parse/excel_parser.py`; the existing test imported the class but never constructed it, so nothing caught the missing import. Added construction coverage for every parser exported from `semantica.parse`, driven off `__all__` so future additions are covered automatically, living outside `test_parse_comprehensive.py` (whose `setUp` mocks `get_progress_tracker` into each module and would mock away the exact interaction under test)
|
||||
|
||||
- **Graph analytics (`centrality_calculator.py`, `community_detector.py`, `connectivity_analyzer.py`) dropped isolated nodes and diverged on how each computed its working view of the graph** (#1011) by @T1mn
|
||||
- Each analyzer had its own ad hoc logic for building the node/edge set it operated over, and none of them included nodes with no edges — a node with zero connections simply vanished from centrality scores, community assignments, and connectivity reports instead of appearing with a zero/singleton value. A new shared `semantica/kg/_graph_view.py` centralizes graph-view construction (including node fallbacks and community payload shaping) for all three analyzers, which are now ~250 lines lighter combined
|
||||
- New `tests/kg/test_analytics_node_scope.py` covering isolated-node presence across all three analyzers
|
||||
|
||||
- **Explorer fired temporal-bounds and snapshot requests before the graph itself had loaded, tripling failed requests when the backend was down and leaving the timeline scrubber with nothing to scrub** (#1003) by @lakshayxi
|
||||
- Two new predicate functions gate the temporal effects on the graph having actually loaded (an empty graph still counts as loaded); confirmed against a downed backend that this cuts three failing requests per page load down to one
|
||||
|
||||
- **`SeedDataManager.load_from_database()` never actually reached the database, and connection failures were mislabeled as a missing optional dependency** (#995, closes #973) by @yzxcj797
|
||||
- `DBIngestor.execute_query`/`export_table` need the connection string as their first positional argument; `load_from_database()` only passed it into the constructor's config dict, which those methods never read, so every call raised `TypeError` before connecting. Also split the combined `except (ImportError, OSError)` handling apart — a genuine connection failure was reported as `"module not available"`, sending debugging in the wrong direction; `OSError` now propagates as an actual failure, chained via `from e`
|
||||
|
||||
- **SPARQL `CONSTRUCT` detection matched inside a leading `#`-comment, misclassifying `SELECT`/`ASK` queries as `CONSTRUCT` across all four SPARQL backends** (#951) by @pravit-amp
|
||||
- `CONSTRUCT_QUERY_RE` skipped comments with a bare `\#[^\n]*`, whose backtracking `*` let a `# CONSTRUCT ...` comment line "swallow" the real query-form keyword on the next line for a query like `# CONSTRUCT ...\nSELECT ...`. The mistaken `CONSTRUCT` classification sent `Accept: text/turtle` and tried to parse a SELECT/ASK response body as Turtle, failing with a misleading parse error. The regex now requires a comment to reach a line terminator (LF or CR, per the SPARQL grammar) before matching
|
||||
|
||||
- **`k_shortest_paths` mutated caller-visible graph state during traversal and ignored direction when excluding already-used edges** (#1000) by @T1mn
|
||||
- `semantica/kg/path_finder.py`'s search left side effects behind after returning, and edge exclusion during Yen's-algorithm-style path removal didn't respect the traversal direction of directed graphs, letting a later search see edges that should have been available. Both fixed; new coverage in `tests/kg/test_path_finder.py`
|
||||
|
||||
- **`trace_decision_causality()` ignored explicitly recorded causal edges, inferring causes only from shared NER entities plus timestamp ordering** (#983) by @hsd2514
|
||||
- A `CAUSED`/`INFLUENCED`/`PRECEDENT_FOR` edge added via `add_causal_relationship()` had no effect on the trace — when entity extraction found nothing in common between two decisions, `trace_decision_chain()` came back empty even with an explicit edge stored in the graph. Explicit causal edges are now traversed first as ground truth, with entity/timestamp inference kept as an additive fallback for pairs with no explicit link; edges whose source has no decision record (e.g. a graph restored via `from_dict`) are skipped so a stale edge can't abort the trace
|
||||
|
||||
- **`RepoIngestor`'s module-level DNS resolve cache had no lock, raising `RuntimeError: OrderedDict mutated during iteration` under concurrent `ingest_repository()` calls** (#979) by @manjunathbhaskar
|
||||
- `_REPO_HOST_RESOLVE_CACHE` is a shared `OrderedDict` read, written, and pruned by every thread with no synchronization — reliably reproduced with 32 threads hammering resolution under a low TTL and small cache cap. Now guarded by a lock
|
||||
|
||||
- **`GraphBuilder` didn't remap relationship endpoints after entity resolution merged nodes, leaving relationships pointing at ids that no longer existed in the resolved graph** (#978) by @T1mn
|
||||
- New coverage in `tests/kg/test_graph_builder_external.py`; a follow-up commit hardens the remapping against edge cases found during review
|
||||
|
||||
- **Explorer's dev server esbuild target didn't match the browser targets the production build declares**, occasionally producing dev-only syntax errors on older browsers (#966) by @le-czs
|
||||
- `explorer/vite.config.ts` now sets the dev esbuild target explicitly to match
|
||||
|
||||
- **`normalize`'s number normalizer accepted currency symbols without validating them against the surrounding text, and an earlier fix's currency-code matching wasn't token-bounded** (#940) by @Mr-Neutr0n, reviewed by @ZohaibHassan16
|
||||
- Symbol currencies are now validated before being accepted; currency codes are matched on token boundaries so a code embedded inside a longer token no longer false-positives
|
||||
|
||||
- **`ContextGraph.to_dict()` was the one reader on the class that didn't hold `self._lock`, raising `RuntimeError: dictionary changed size during iteration` under a concurrent writer and risking a torn snapshot otherwise** (#929) by @pravit-amp
|
||||
- Every other reader (`stats()`, `density()`, `find_nodes()`, `find_edges()`, `get_neighbors()`, `get_nodes_by_label()`, `state_at()`, `save_to_file()`) already took the lock after it was introduced; `to_dict()` predated that change and was missed. `save_to_file()` was safe only incidentally, since it builds its payload inline under its own lock rather than delegating to `to_dict()`
|
||||
|
||||
- **`PipelineWithProvenance` had a broken import and no working `run()` method** (#862) by @Karunasagar12
|
||||
- `from .pipeline import Pipeline` failed because `Pipeline` lives in `pipeline_builder.py`, not a nonexistent `pipeline.py` — fixed to `from .pipeline_builder import Pipeline`. The class also had no `run()`; it now delegates to `ExecutionEngine.execute_pipeline()`, the intended execution path for a built `Pipeline`. The constructor now accepts a built `Pipeline` instance directly
|
||||
|
||||
### Security
|
||||
|
||||
- **Tarball restore path traversal, latent SQL injection, DNS-rebinding TOCTOU in the shared SSRF guard, stored XSS in report generation, and unvalidated SPARQL object IRIs in AnzoStore** (#1079) by @KaifAhmad1
|
||||
@@ -234,6 +331,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- **Caught by the new gate on its first run**: `python -m pip install -e ".[all]"` pulled in `setuptools==79.0.1`, vulnerable to CVE-2026-59890/GHSA-h35f-9h28-mq5c/PYSEC-2026-3447 (Unicode-normalization bypass of `MANIFEST.in` exclude/prune patterns on macOS APFS/HFS+, letting excluded files leak into a built sdist), fixed in `83.0.0`. `[build-system] requires` had the exact same too-permissive-floor pattern this whole entry is about (`setuptools>=61.0`), and `actions/setup-python`'s baked-in `setuptools` isn't governed by that pin at all since it's outside any isolated build. Bumped `[build-system] requires` to `setuptools>=83.0.0`, and the `Security` workflow now runs `pip install --upgrade pip setuptools` before auditing so the scanned environment can't have a stale ambient copy regardless of what governs it
|
||||
- Full `explorer` suite: 241 passed
|
||||
|
||||
- **`SeedDataManager.load_from_api()` made unguarded HTTP requests, with no SSRF protection at all** (#942) by @ZohaibHassan16
|
||||
- `load_from_api()` called `requests.get()` directly instead of going through `semantica/ingest/ssrf.py`'s `request_with_ssrf_guard()`, unlike every other ingestor in this module — a caller-supplied `api_url` could target internal/private network addresses with no validation. Now routes through the shared guard, gaining redirect validation and bounded DNS resolution for free
|
||||
- **Follow-up** (#959, closes #943) by @yunaremaia: added an `allow_private_ips` opt-in (parsed via the shared `parse_bool` helper) for trusted internal deployments that legitimately need to load from a private-network API, while keeping the guard's block-by-default behavior for everyone else
|
||||
|
||||
## [0.6.5] - 2026-08-11
|
||||
|
||||
### Added
|
||||
|
||||
@@ -1 +1,2 @@
|
||||
recursive-include semantica/static *
|
||||
recursive-include semantica/ontology/vocabulary *.ttl
|
||||
|
||||
@@ -142,7 +142,7 @@ compliant = graph.check_decision_rules({"category": "vendor_selection"}) # poli
|
||||
```bash
|
||||
semantica doctor
|
||||
# Python 3.11.9 pass
|
||||
# semantica 0.6.5 pass
|
||||
# semantica 0.6.6 pass
|
||||
# faiss vector store pass
|
||||
# Config file pass ~/.semantica/config.yaml
|
||||
```
|
||||
@@ -1466,18 +1466,18 @@ For contributor / dev-server setup: **[explorer/README.md: Local Setup Guide](ex
|
||||
|
||||
---
|
||||
|
||||
## What's New in v0.6.5
|
||||
## What's New in v0.6.6
|
||||
|
||||
**Security release — upgrading is strongly recommended.** Fixes for 5 externally-reported vulnerabilities in the Explorer API and graph/triplet store backends, plus a CodeQL-flagged ReDoS:
|
||||
**Security release — upgrading is strongly recommended.** Fixes for a privately disclosed batch of vulnerabilities spanning backup/restore, database export, outbound requests, and triplet-store backends, plus SSRF hardening across ingestion:
|
||||
|
||||
- **Missing authentication on all Explorer API routes** (GHSA-j4mq-hprp-987v, Critical): every route now requires `SEMANTICA_API_KEY`, fails closed (503) rather than open when unconfigured
|
||||
- **SSRF via redirect bypass in ontology URL fetching** (GHSA-8c7v-62gr-hj6g, High): redirect targets are now re-validated at every hop and the connection is pinned to the validated address, closing a DNS check-then-use race
|
||||
- **Cypher injection via unvalidated node labels and property keys** (GHSA-482h-hw99-h62p, Critical): Neptune, Neo4j, and FalkorDB now sanitize every label/relationship-type/property-key interpolation site
|
||||
- **SPARQL injection via unvalidated triplet IRIs** (GHSA-8vgg-8mr4-r236, Critical): Blazegraph, RDF4J, and Jena now validate subject/predicate/object IRIs before interpolation
|
||||
- **Missing Origin validation on the WebSocket handshake** (GHSA-4643-wpgq-w329, Moderate, anonymous-mode only): `/ws/graph-updates` now checks `Origin` against the same allowlist `CORSMiddleware` enforces for HTTP
|
||||
- **Polynomial ReDoS in SPARQL query validation** (CodeQL `py/polynomial-redos`): fixed a backtracking regex in the Explorer's SPARQL route
|
||||
- **Tarball restore path traversal**: `semantica backup restore` now validates every archive member for path containment and rejects symlink/hardlink escapes before extraction
|
||||
- **Latent SQL injection in `DataExporter.export_table_data()`**: table/schema names are now identifier-allowlisted and `where`/`order_by` fragments are blocklist-checked
|
||||
- **DNS-rebinding TOCTOU in the shared SSRF guard**: the resolved IP that passes validation is now the one the connection is pinned to, closing the check-then-use race (also closes the `100.64.0.0/10` CGNAT gap)
|
||||
- **Stored XSS in HTML report generation** and **unvalidated SPARQL object IRIs in AnzoStore** (SPARQL injection): both now escape/validate before interpolation
|
||||
- **`Authorization`/`Proxy-Authorization` credential leakage across redirects**, plus **SSRF gaps in `FeedIngestor`/`FeedMonitor`, `RepoIngestor`, and the MCP/public-API ingest paths**: all now route through the shared, redirect-safe SSRF guard
|
||||
- **HTTP response header injection and an unbounded-memory DoS** in the Explorer API, and a **`fastapi`/`python-multipart` ReDoS** (PYSEC-2024-38): floors raised, inputs sanitized, candidate pools capped
|
||||
|
||||
Also includes: embedded Oxigraph backend for `TripletStore`, PROV-O trust/spec completeness for `ProvenanceManager`, and the Altair Anzo triplet store backend.
|
||||
Also ships: **first-class CrewAI integration** (`semantica[crewai]`, extraction/decision tools + a knowledge source), **`ContextGraph` retraction and purge** (GDPR-style erasure without a full `clear()`), a declared **Semantica RDF vocabulary with deterministic entity/relationship IRIs** (stable, diffable exports), and **timezone-aware timestamps** across `export/` and `provenance/`.
|
||||
|
||||
→ [Full release notes](RELEASE_NOTES.md) · [Changelog](CHANGELOG.md)
|
||||
|
||||
|
||||
+5
-5
@@ -17,22 +17,22 @@ icon: "quote-left"
|
||||
author = {Semantica},
|
||||
year = {2026},
|
||||
url = {https://github.com/semantica-agi/semantica},
|
||||
version = {0.6.5},
|
||||
version = {0.6.6},
|
||||
doi = {10.5281/zenodo.XXXXXXX}
|
||||
}
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="APA">
|
||||
Semantica. (2026). *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems* (Version 0.6.5) \[Computer software\]. https://github.com/semantica-agi/semantica
|
||||
Semantica. (2026). *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems* (Version 0.6.6) \[Computer software\]. https://github.com/semantica-agi/semantica
|
||||
</Tab>
|
||||
<Tab title="MLA">
|
||||
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. Version 0.6.5, GitHub, 2026, https://github.com/semantica-agi/semantica.
|
||||
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. Version 0.6.6, GitHub, 2026, https://github.com/semantica-agi/semantica.
|
||||
</Tab>
|
||||
<Tab title="Chicago">
|
||||
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. Version 0.6.5. GitHub, 2026. https://github.com/semantica-agi/semantica.
|
||||
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. Version 0.6.6. GitHub, 2026. https://github.com/semantica-agi/semantica.
|
||||
</Tab>
|
||||
<Tab title="IEEE">
|
||||
Semantica, "Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems," Version 0.6.5, GitHub, 2026. \[Online\]. Available: https://github.com/semantica-agi/semantica
|
||||
Semantica, "Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems," Version 0.6.6, GitHub, 2026. \[Online\]. Available: https://github.com/semantica-agi/semantica
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
|
||||
+1
-1
@@ -17,7 +17,7 @@ icon: "circle-question"
|
||||
| API key required? | Optional: pattern extraction works with no keys |
|
||||
| Works with LangChain / LlamaIndex? | Yes: Semantica is a layer on top, not a replacement |
|
||||
| Production-ready? | Yes: 1,000+ tests, v0.5.0 ships with 12 security fixes |
|
||||
| Latest version? | **v0.6.5** (August 2026) |
|
||||
| Latest version? | **v0.6.6** (August 2026) |
|
||||
| Local LLMs? | Yes: Ollama via LiteLLM, HuggingFaceLLM for air-gapped |
|
||||
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ icon: "rocket"
|
||||
Verify installation:
|
||||
```python
|
||||
import semantica
|
||||
print(semantica.__version__) # 0.6.5
|
||||
print(semantica.__version__) # 0.6.6
|
||||
```
|
||||
</Check>
|
||||
</Step>
|
||||
|
||||
+4
-4
@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "semantica"
|
||||
version = "0.6.5"
|
||||
description = "Accountability and context layer for AI agents. Context graphs, decision intelligence, full provenance tracking, and explainable reasoning engines — every AI decision traceable, every output auditable."
|
||||
version = "0.6.6"
|
||||
description = "Graph-Native Infrastructure for Context and Accountable AI Systems: context graphs, decision intelligence, full provenance tracking, and explainable reasoning engines — every AI decision traceable, every output auditable."
|
||||
readme = "README.md"
|
||||
license = { text = "MIT" }
|
||||
|
||||
@@ -103,7 +103,7 @@ Discord = "https://discord.gg/sV34vps5hH"
|
||||
llm-openai = ["openai>=1.0.0"]
|
||||
llm-groq = ["groq>=0.4.0"]
|
||||
llm-gemini = ["google-genai>=0.1.0"]
|
||||
llm-anthropic = ["anthropic>=0.18.0"]
|
||||
llm-anthropic = ["anthropic>=0.122.0"]
|
||||
llm-ollama = ["ollama>=0.1.0"]
|
||||
llm-deepseek = ["openai>=1.0.0"]
|
||||
llm-litellm = ["litellm>=1.83.9"]
|
||||
@@ -271,7 +271,7 @@ include = ["semantica*", "integrations*"]
|
||||
[tool.setuptools.package-data]
|
||||
# Explicit patterns are more reliable than **/* across setuptools versions.
|
||||
# static/* covers index.html / favicon; static/assets/* covers all JS/CSS chunks.
|
||||
"semantica" = ["static/*", "static/assets/*"]
|
||||
"semantica" = ["static/*", "static/assets/*", "ontology/vocabulary/*.ttl"]
|
||||
|
||||
[tool.black]
|
||||
line-length = 88
|
||||
|
||||
+3
-3
@@ -159,9 +159,9 @@ annotated-types==0.8.0 \
|
||||
--hash=sha256:13b2beaad985e05e2d6407ee4c4f35590b11f8d693a258a561055cac8f64cab7 \
|
||||
--hash=sha256:f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0
|
||||
# via pydantic
|
||||
anthropic==0.121.0 \
|
||||
--hash=sha256:6048713fa441e59e1cba8363171cd2a86273b25bd213e9c7ac70a523af88b011 \
|
||||
--hash=sha256:e79d6e08ab3376602fc9a70d4d5ea3540817c76cf7e16658bed790834e1833d6
|
||||
anthropic==0.122.0 \
|
||||
--hash=sha256:45ec906452ffae6b5f7f0c53d01f50bfb7e4ce878d7ae8e4309d13171e557e67 \
|
||||
--hash=sha256:ffec56ae96657c8d19fa575ec96f140f380c353a07ab7d61b92eb18ee6536601
|
||||
# via semantica (pyproject.toml)
|
||||
antlr4-python3-runtime==4.9.3 \
|
||||
--hash=sha256:f224469b4168294902bb1efa80a8bf7855f24c99aef99cbefc1bcd3cce77881b
|
||||
|
||||
@@ -10,7 +10,7 @@ Main exports:
|
||||
- Config: Configuration management
|
||||
"""
|
||||
|
||||
__version__ = "0.6.5"
|
||||
__version__ = "0.6.6"
|
||||
__author__ = "Semantica Contributors"
|
||||
__license__ = "MIT"
|
||||
|
||||
|
||||
@@ -45,6 +45,7 @@ License: MIT
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
from ..utils.entity_ids import get_entity_id
|
||||
from ..utils.exceptions import ProcessingError, ValidationError
|
||||
from ..utils.logging import get_logger
|
||||
from ..utils.progress_tracker import get_progress_tracker
|
||||
@@ -504,7 +505,7 @@ class EntityMerger:
|
||||
# Record source entities
|
||||
provenance["merged_from"] = [
|
||||
{
|
||||
"id": self._get_entity_value(e, "id"),
|
||||
"id": get_entity_id(e),
|
||||
"name": self._get_entity_value(e, "name"),
|
||||
"source": self._get_entity_value(e, "metadata", {}).get("source") if hasattr(e, "metadata") or isinstance(e, dict) else None,
|
||||
}
|
||||
|
||||
@@ -45,6 +45,7 @@ from dataclasses import dataclass, field
|
||||
from enum import Enum
|
||||
from typing import Any, Callable, Dict, List, Optional, Tuple, Union
|
||||
|
||||
from ..utils.entity_ids import get_entity_id
|
||||
from ..utils.exceptions import ProcessingError, ValidationError
|
||||
from ..utils.logging import get_logger
|
||||
from ..utils.progress_tracker import get_progress_tracker
|
||||
@@ -317,14 +318,20 @@ class MergeStrategyManager:
|
||||
message=f"Building merged entity... ({current_step}/{total_steps}, remaining: {remaining_steps} steps)"
|
||||
)
|
||||
# Build merged entity
|
||||
merged_from = []
|
||||
for entity in entities:
|
||||
entity_id = get_entity_id(entity)
|
||||
if entity_id is not None:
|
||||
merged_from.append(entity_id)
|
||||
|
||||
merged_entity = {
|
||||
"id": base_entity.get("id"),
|
||||
"id": get_entity_id(base_entity),
|
||||
"name": self._merge_top_level_field("name", entities, base_entity),
|
||||
"type": self._merge_top_level_field("type", entities, base_entity),
|
||||
"properties": merged_properties,
|
||||
"relationships": merged_relationships,
|
||||
"metadata": self._merge_metadata(entities, base_entity),
|
||||
"merged_from": [e.get("id") for e in entities if e.get("id")],
|
||||
"merged_from": merged_from,
|
||||
"merge_strategy": strategy.value,
|
||||
}
|
||||
|
||||
|
||||
@@ -14,9 +14,10 @@ License: MIT
|
||||
"""
|
||||
|
||||
from typing import Any, Optional
|
||||
from datetime import datetime
|
||||
import uuid
|
||||
|
||||
from ..utils.helpers import utc_now_iso
|
||||
|
||||
|
||||
class ExporterWithProvenance:
|
||||
"""Base exporter with provenance tracking."""
|
||||
@@ -45,9 +46,9 @@ class ExporterWithProvenance:
|
||||
|
||||
def export(self, data: Any, destination: str, **kwargs):
|
||||
"""Export data with provenance tracking."""
|
||||
activity_started_at_time = datetime.utcnow().isoformat()
|
||||
activity_started_at_time = utc_now_iso()
|
||||
result = self._exporter.export(data, destination, **kwargs)
|
||||
activity_ended_at_time = datetime.utcnow().isoformat()
|
||||
activity_ended_at_time = utc_now_iso()
|
||||
|
||||
if self.provenance and self._prov_manager:
|
||||
self._prov_manager.track_entity(
|
||||
|
||||
@@ -24,14 +24,14 @@ License: MIT
|
||||
"""
|
||||
|
||||
import json
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
from ..utils.exceptions import ProcessingError, ValidationError
|
||||
from ..utils.helpers import ensure_directory, write_json_file
|
||||
from ..utils.helpers import ensure_directory, utc_now_iso, write_json_file
|
||||
from ..utils.logging import get_logger
|
||||
from ..utils.progress_tracker import get_progress_tracker
|
||||
from .rdf_exporter import SEMANTICA_NS, mint_entity_iri, mint_relationship_iri
|
||||
|
||||
|
||||
class JSONExporter:
|
||||
@@ -265,11 +265,12 @@ class JSONExporter:
|
||||
json_data = {
|
||||
"@context": {
|
||||
"@vocab": "https://semantica.dev/vocab/",
|
||||
"semantica": SEMANTICA_NS,
|
||||
"entities": {"@id": "semantica:entities", "@container": "@list"},
|
||||
},
|
||||
"entities": entities,
|
||||
"metadata": {
|
||||
"exported_at": datetime.now().isoformat(),
|
||||
"exported_at": utc_now_iso(),
|
||||
"entity_count": len(entities),
|
||||
**options.get("metadata", {}),
|
||||
},
|
||||
@@ -294,6 +295,7 @@ class JSONExporter:
|
||||
json_data = {
|
||||
"@context": {
|
||||
"@vocab": "https://semantica.dev/vocab/",
|
||||
"semantica": SEMANTICA_NS,
|
||||
"relationships": {
|
||||
"@id": "semantica:relationships",
|
||||
"@container": "@list",
|
||||
@@ -301,7 +303,7 @@ class JSONExporter:
|
||||
},
|
||||
"relationships": relationships,
|
||||
"metadata": {
|
||||
"exported_at": datetime.now().isoformat(),
|
||||
"exported_at": utc_now_iso(),
|
||||
"relationship_count": len(relationships),
|
||||
**options.get("metadata", {}),
|
||||
},
|
||||
@@ -339,7 +341,7 @@ class JSONExporter:
|
||||
if include_metadata:
|
||||
if "metadata" not in result:
|
||||
result["metadata"] = {}
|
||||
result["metadata"]["exported_at"] = datetime.now().isoformat()
|
||||
result["metadata"]["exported_at"] = utc_now_iso()
|
||||
if include_provenance:
|
||||
result["metadata"]["format"] = "json"
|
||||
|
||||
@@ -349,7 +351,7 @@ class JSONExporter:
|
||||
"data": data,
|
||||
"count": len(data),
|
||||
"metadata": {
|
||||
"exported_at": datetime.now().isoformat(),
|
||||
"exported_at": utc_now_iso(),
|
||||
"format": "json" if include_provenance else None,
|
||||
**options.get("metadata", {}),
|
||||
},
|
||||
@@ -358,7 +360,7 @@ class JSONExporter:
|
||||
# Single value
|
||||
return {
|
||||
"value": data,
|
||||
"metadata": {"exported_at": datetime.now().isoformat()}
|
||||
"metadata": {"exported_at": utc_now_iso()}
|
||||
if include_metadata
|
||||
else {},
|
||||
}
|
||||
@@ -410,9 +412,9 @@ class JSONExporter:
|
||||
|
||||
# Add metadata and provenance if requested
|
||||
if include_metadata:
|
||||
jsonld["@id"] = f"https://semantica.dev/data/{datetime.now().isoformat()}"
|
||||
jsonld["@id"] = f"https://semantica.dev/data/{utc_now_iso()}"
|
||||
if include_provenance:
|
||||
jsonld["semantica:exportedAt"] = datetime.now().isoformat()
|
||||
jsonld["semantica:exportedAt"] = utc_now_iso()
|
||||
jsonld["semantica:format"] = "json-ld"
|
||||
|
||||
return jsonld
|
||||
@@ -444,7 +446,7 @@ class JSONExporter:
|
||||
"nodes": kg.get("nodes", []),
|
||||
"edges": kg.get("edges", []),
|
||||
"metadata": {
|
||||
"exported_at": datetime.now().isoformat(),
|
||||
"exported_at": utc_now_iso(),
|
||||
**kg.get("metadata", {}),
|
||||
**options.get("metadata", {}),
|
||||
},
|
||||
@@ -481,7 +483,7 @@ class JSONExporter:
|
||||
"rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
|
||||
"rdfs": "http://www.w3.org/2000/01/rdf-schema#",
|
||||
},
|
||||
"@id": f"https://semantica.dev/graph/{datetime.now().isoformat()}",
|
||||
"@id": f"https://semantica.dev/graph/{utc_now_iso()}",
|
||||
"@type": "semantica:KnowledgeGraph",
|
||||
}
|
||||
|
||||
@@ -495,14 +497,15 @@ class JSONExporter:
|
||||
relationships = kg.get("relationships", [])
|
||||
if relationships:
|
||||
jsonld["semantica:relationships"] = [
|
||||
self._relationship_to_jsonld(r) for r in relationships
|
||||
self._relationship_to_jsonld(r, index)
|
||||
for index, r in enumerate(relationships)
|
||||
]
|
||||
self.logger.debug(
|
||||
f"Converted {len(relationships)} relationship(s) to JSON-LD"
|
||||
)
|
||||
|
||||
# Add metadata
|
||||
jsonld["semantica:exportedAt"] = datetime.now().isoformat()
|
||||
jsonld["semantica:exportedAt"] = utc_now_iso()
|
||||
if "metadata" in kg:
|
||||
jsonld["semantica:metadata"] = kg["metadata"]
|
||||
|
||||
@@ -526,11 +529,13 @@ class JSONExporter:
|
||||
Returns:
|
||||
Dictionary in JSON-LD format representing the entity
|
||||
"""
|
||||
# Generate @id if not provided
|
||||
entity_id = entity.get("id")
|
||||
if not entity_id:
|
||||
entity_text = entity.get("text") or entity.get("label", "unknown")
|
||||
entity_id = f"semantica:entity/{entity_text}"
|
||||
# Generate @id if not provided. Minted exactly as the RDF serializers
|
||||
# mint it (#1101), so the JSON-LD and Turtle exports of one knowledge
|
||||
# graph name the same entity with the same IRI. Interpolating the raw
|
||||
# text into f"semantica:entity/{text}" produced an invalid IRI for any
|
||||
# text containing a space, and a JSON-LD parser dropped the whole node.
|
||||
entity_text = entity.get("text") or entity.get("label", "unknown")
|
||||
entity_id = entity.get("id") or mint_entity_iri(entity_text)
|
||||
|
||||
jsonld = {
|
||||
"@id": entity_id,
|
||||
@@ -545,7 +550,9 @@ class JSONExporter:
|
||||
|
||||
return jsonld
|
||||
|
||||
def _relationship_to_jsonld(self, rel: Dict[str, Any]) -> Dict[str, Any]:
|
||||
def _relationship_to_jsonld(
|
||||
self, rel: Dict[str, Any], index: int = 0
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Convert relationship to JSON-LD format.
|
||||
|
||||
@@ -560,16 +567,18 @@ class JSONExporter:
|
||||
- type: Relationship type (optional)
|
||||
- confidence: Confidence score (optional)
|
||||
- metadata: Metadata dictionary (optional)
|
||||
index: Position of the relationship in the exported list, used when
|
||||
minting an IRI for a relationship that arrived without an id
|
||||
|
||||
Returns:
|
||||
Dictionary in JSON-LD format representing the relationship
|
||||
"""
|
||||
# Generate @id if not provided
|
||||
rel_id = rel.get("id")
|
||||
if not rel_id:
|
||||
source_id = rel.get("source_id") or rel.get("source", "")
|
||||
target_id = rel.get("target_id") or rel.get("target", "")
|
||||
rel_id = f"semantica:rel/{source_id}_{target_id}"
|
||||
# Generate @id if not provided, from the same mint the RDF serializers
|
||||
# use, including the list index that separates two relationships
|
||||
# sharing a pair of endpoints (#1101).
|
||||
source_id = rel.get("source_id") or rel.get("source", "")
|
||||
target_id = rel.get("target_id") or rel.get("target", "")
|
||||
rel_id = rel.get("id") or mint_relationship_iri(index, source_id, target_id)
|
||||
|
||||
jsonld = {
|
||||
"@id": rel_id,
|
||||
|
||||
@@ -33,11 +33,42 @@ from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Set, Union
|
||||
|
||||
from ..utils.exceptions import ProcessingError, ValidationError
|
||||
from ..utils.helpers import ensure_directory
|
||||
from ..utils.helpers import ensure_directory, hash_data
|
||||
from ..utils.logging import get_logger
|
||||
from ..utils.progress_tracker import get_progress_tracker
|
||||
|
||||
|
||||
SEMANTICA_NS = "https://semantica.dev/ns#"
|
||||
|
||||
#: Written when an entity carries no type of its own. A full IRI rather than the
|
||||
#: prefixed form, because the Turtle serializer writes it inside angle brackets,
|
||||
#: where `semantica:Entity` would be read as an IRI in the scheme `semantica`
|
||||
#: rather than as the prefix expansion (issue #1101).
|
||||
DEFAULT_ENTITY_TYPE = f"{SEMANTICA_NS}Entity"
|
||||
|
||||
#: Written when a relationship carries no type of its own. Same reasoning.
|
||||
DEFAULT_RELATION_TYPE = f"{SEMANTICA_NS}related_to"
|
||||
|
||||
|
||||
def mint_entity_iri(text: str) -> str:
|
||||
"""Mint a stable IRI for an entity that arrived without an id.
|
||||
|
||||
Python's builtin ``hash()`` is randomised per process (PYTHONHASHSEED), so
|
||||
minting from it gave the same entity a different IRI on every run: exports
|
||||
could not be diffed, deduplicated against an earlier load, or joined to a
|
||||
provenance record written by an earlier process. SHA-256 is stable across
|
||||
runs and machines, which is what an identifier has to be.
|
||||
"""
|
||||
digest = hash_data(str(text))[:16]
|
||||
return f"{SEMANTICA_NS}entity_{digest}"
|
||||
|
||||
|
||||
def mint_relationship_iri(index: int, source: Any, target: Any) -> str:
|
||||
"""Mint a stable IRI for a relationship that arrived without an id."""
|
||||
digest = hash_data(f"{source}\x00{target}")[:16]
|
||||
return f"{SEMANTICA_NS}rel_{index}_{digest}"
|
||||
|
||||
|
||||
class NamespaceManager:
|
||||
"""
|
||||
RDF namespace management engine.
|
||||
@@ -360,9 +391,9 @@ class RDFSerializer:
|
||||
entity_id = entity.get("id")
|
||||
if not entity_id:
|
||||
entity_text = entity.get("text", "")
|
||||
entity_id = f"semantica:entity_{hash(entity_text)}"
|
||||
entity_id = mint_entity_iri(entity_text)
|
||||
|
||||
entity_type = entity.get("type", "semantica:Entity")
|
||||
entity_type = entity.get("type", DEFAULT_ENTITY_TYPE)
|
||||
text = entity.get("text") or entity.get("label", "")
|
||||
confidence = entity.get("confidence", 1.0)
|
||||
|
||||
@@ -376,7 +407,7 @@ class RDFSerializer:
|
||||
for idx, rel in enumerate(relationships):
|
||||
source_id = rel.get("source_id") or rel.get("source")
|
||||
target_id = rel.get("target_id") or rel.get("target")
|
||||
rel_type = rel.get("type", "semantica:related_to")
|
||||
rel_type = rel.get("type", DEFAULT_RELATION_TYPE)
|
||||
|
||||
lines.append(f"<{source_id}> <{rel_type}> <{target_id}> .")
|
||||
|
||||
@@ -413,10 +444,14 @@ class RDFSerializer:
|
||||
if time_axis in ("transaction", "both"):
|
||||
axes.append(("tx", rel.get("recorded_at"), rel.get("superseded_at")))
|
||||
|
||||
rel_base_id = (
|
||||
rel.get("id")
|
||||
or f"semantica:rel_{idx}_{hash(str(rel.get('source_id', '')) + str(rel.get('target_id', '')))}"
|
||||
)
|
||||
# Resolve endpoints the same way serialize_to_turtle does: both
|
||||
# representations are accepted upstream, and minting from source_id
|
||||
# alone hashes empty strings for every relationship that uses source,
|
||||
# so unrelated relationships at the same index would collide on a
|
||||
# deterministic IRI.
|
||||
source_id = rel.get("source_id") or rel.get("source") or ""
|
||||
target_id = rel.get("target_id") or rel.get("target") or ""
|
||||
rel_base_id = rel.get("id") or mint_relationship_iri(idx, source_id, target_id)
|
||||
|
||||
lines = [""] # blank separator
|
||||
for axis_name, from_val, until_val in axes:
|
||||
@@ -488,9 +523,9 @@ class RDFSerializer:
|
||||
entity_id = entity.get("id")
|
||||
if not entity_id:
|
||||
entity_text = entity.get("text", "")
|
||||
entity_id = f"semantica:entity_{hash(entity_text)}"
|
||||
entity_id = mint_entity_iri(entity_text)
|
||||
|
||||
entity_type = entity.get("type", "semantica:Entity")
|
||||
entity_type = entity.get("type", DEFAULT_ENTITY_TYPE)
|
||||
text = entity.get("text") or entity.get("label", "")
|
||||
confidence = entity.get("confidence", 1.0)
|
||||
|
||||
@@ -565,11 +600,13 @@ class RDFSerializer:
|
||||
# Convert entities to JSON-LD
|
||||
entities = rdf_data.get("entities", [])
|
||||
for entity in entities:
|
||||
# Generate @id if not provided
|
||||
entity_id = entity.get("id")
|
||||
if not entity_id:
|
||||
entity_text = entity.get("text", "")
|
||||
entity_id = f"semantica:entity/{entity_text}"
|
||||
# Generate @id if not provided. Minted the same way the Turtle and
|
||||
# N-Triples paths mint it (#1101), so one knowledge graph carries
|
||||
# the same node identity whichever serializer wrote it. The former
|
||||
# f"semantica:entity/{text}" interpolated the raw text into an IRI:
|
||||
# any entity whose text contained a space produced an invalid IRI
|
||||
# and was dropped in full by a JSON-LD parser, silently.
|
||||
entity_id = entity.get("id") or mint_entity_iri(entity.get("text", ""))
|
||||
|
||||
jsonld["@graph"].append(
|
||||
{
|
||||
@@ -582,24 +619,22 @@ class RDFSerializer:
|
||||
|
||||
# Convert relationships to JSON-LD
|
||||
relationships = rdf_data.get("relationships", [])
|
||||
for rel in relationships:
|
||||
# Generate @id if not provided
|
||||
rel_id = rel.get("id")
|
||||
if not rel_id:
|
||||
source_id = rel.get("source_id", "")
|
||||
target_id = rel.get("target_id", "")
|
||||
rel_id = f"semantica:rel/{source_id}_{target_id}"
|
||||
for index, rel in enumerate(relationships):
|
||||
# Endpoints are resolved both ways, as serialize_to_turtle resolves
|
||||
# them: a relationship carrying source/target rather than
|
||||
# source_id/target_id used to hash into f"semantica:rel/_", so every
|
||||
# such relationship in an export collapsed onto one node and their
|
||||
# types and endpoints merged.
|
||||
source = rel.get("source_id") or rel.get("source", "")
|
||||
target = rel.get("target_id") or rel.get("target", "")
|
||||
rel_id = rel.get("id") or mint_relationship_iri(index, source, target)
|
||||
|
||||
jsonld["@graph"].append(
|
||||
{
|
||||
"@id": rel_id,
|
||||
"@type": "semantica:Relationship",
|
||||
"semantica:source": {
|
||||
"@id": rel.get("source_id") or rel.get("source")
|
||||
},
|
||||
"semantica:target": {
|
||||
"@id": rel.get("target_id") or rel.get("target")
|
||||
},
|
||||
"semantica:source": {"@id": source},
|
||||
"semantica:target": {"@id": target},
|
||||
"semantica:type": rel.get("type", "related_to"),
|
||||
}
|
||||
)
|
||||
@@ -644,7 +679,7 @@ class RDFSerializer:
|
||||
entity_id = entity.get("id")
|
||||
if not entity_id:
|
||||
entity_text = entity.get("text", "")
|
||||
entity_id = f"semantica:entity_{hash(entity_text)}"
|
||||
entity_id = mint_entity_iri(entity_text)
|
||||
|
||||
subject = expand_uri(entity_id)
|
||||
|
||||
|
||||
@@ -25,12 +25,11 @@ License: MIT
|
||||
|
||||
import html
|
||||
import json
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
from ..utils.exceptions import ProcessingError, ValidationError
|
||||
from ..utils.helpers import ensure_directory
|
||||
from ..utils.helpers import ensure_directory, utc_now_iso
|
||||
from ..utils.logging import get_logger
|
||||
from ..utils.progress_tracker import get_progress_tracker
|
||||
|
||||
@@ -252,7 +251,7 @@ class ReportGenerator:
|
||||
# Build report data with summary
|
||||
report_data = {
|
||||
"title": "Quality Assurance Report",
|
||||
"generated_at": datetime.now().isoformat(),
|
||||
"generated_at": utc_now_iso(),
|
||||
"metrics": quality_metrics,
|
||||
"summary": self._generate_quality_summary(quality_metrics),
|
||||
}
|
||||
@@ -278,7 +277,7 @@ class ReportGenerator:
|
||||
"""
|
||||
report_data = {
|
||||
"title": "Analysis Report",
|
||||
"generated_at": datetime.now().isoformat(),
|
||||
"generated_at": utc_now_iso(),
|
||||
"analysis": analysis_results,
|
||||
"summary": self._generate_analysis_summary(analysis_results),
|
||||
}
|
||||
@@ -304,7 +303,7 @@ class ReportGenerator:
|
||||
"""
|
||||
report_data = {
|
||||
"title": "Framework Metrics Report",
|
||||
"generated_at": datetime.now().isoformat(),
|
||||
"generated_at": utc_now_iso(),
|
||||
"metrics": metrics,
|
||||
"summary": self._generate_metrics_summary(metrics),
|
||||
}
|
||||
|
||||
@@ -22,7 +22,6 @@ License: MIT
|
||||
"""
|
||||
|
||||
from collections.abc import Mapping
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
@@ -33,6 +32,7 @@ from ..utils.helpers import (
|
||||
_require_recognized_keys,
|
||||
ensure_directory,
|
||||
normalize_graph_payload,
|
||||
utc_now_iso,
|
||||
)
|
||||
from ..utils.logging import get_logger
|
||||
from ..utils.progress_tracker import get_progress_tracker
|
||||
@@ -216,7 +216,7 @@ class SemanticNetworkYAMLExporter:
|
||||
records = normalize_graph_payload(semantic_network)
|
||||
yaml_data = {
|
||||
"metadata": {
|
||||
"exported_at": datetime.now().isoformat(),
|
||||
"exported_at": utc_now_iso(),
|
||||
"version": "1.0",
|
||||
**semantic_network.get("metadata", {}),
|
||||
},
|
||||
@@ -309,7 +309,7 @@ class SemanticNetworkYAMLExporter:
|
||||
|
||||
if include_metadata:
|
||||
yaml_data["metadata"] = {
|
||||
"exported_at": datetime.now().isoformat(),
|
||||
"exported_at": utc_now_iso(),
|
||||
"entity_count": len(entities),
|
||||
}
|
||||
|
||||
@@ -333,7 +333,7 @@ class SemanticNetworkYAMLExporter:
|
||||
|
||||
if include_properties:
|
||||
yaml_data["metadata"] = {
|
||||
"exported_at": datetime.now().isoformat(),
|
||||
"exported_at": utc_now_iso(),
|
||||
"relationship_count": len(relationships),
|
||||
}
|
||||
|
||||
@@ -370,7 +370,7 @@ class SemanticNetworkYAMLExporter:
|
||||
}
|
||||
|
||||
yaml_data["metadata"] = {
|
||||
"exported_at": datetime.now().isoformat(),
|
||||
"exported_at": utc_now_iso(),
|
||||
"triplet_count": len(triplets),
|
||||
}
|
||||
|
||||
@@ -410,7 +410,7 @@ class SemanticNetworkYAMLExporter:
|
||||
yaml_data = {
|
||||
"pipeline_stage": pipeline_stage,
|
||||
"metadata": {
|
||||
"extracted_at": datetime.now().isoformat(),
|
||||
"extracted_at": utc_now_iso(),
|
||||
**extracted_data.get("metadata", {}),
|
||||
},
|
||||
"semantic_network": semantic_network,
|
||||
|
||||
@@ -24,6 +24,7 @@ from typing import Any, Dict, List, Optional
|
||||
|
||||
from ..deduplication.duplicate_detector import DuplicateDetector, DuplicateGroup
|
||||
from ..deduplication.entity_merger import EntityMerger
|
||||
from ..utils.entity_ids import get_entity_id
|
||||
from ..utils.logging import get_logger
|
||||
from ..utils.progress_tracker import get_progress_tracker
|
||||
|
||||
@@ -240,9 +241,7 @@ class EntityResolver:
|
||||
@staticmethod
|
||||
def _get_entity_id(entity: Any) -> Any:
|
||||
"""Return an entity ID while supporting dictionary and object inputs."""
|
||||
if isinstance(entity, dict):
|
||||
return entity.get("id") or entity.get("entity_id")
|
||||
return getattr(entity, "id", None) or getattr(entity, "entity_id", None)
|
||||
return get_entity_id(entity)
|
||||
|
||||
@staticmethod
|
||||
def _get_entity_name(entity: Any) -> Optional[str]:
|
||||
@@ -279,13 +278,13 @@ class EntityResolver:
|
||||
processed_ids = set()
|
||||
for op in merge_operations:
|
||||
for source_entity in op.source_entities:
|
||||
entity_id = source_entity.get("id") or source_entity.get("entity_id")
|
||||
if entity_id:
|
||||
entity_id = get_entity_id(source_entity)
|
||||
if entity_id is not None:
|
||||
processed_ids.add(entity_id)
|
||||
|
||||
for entity in entities:
|
||||
entity_id = entity.get("id") or entity.get("entity_id")
|
||||
if entity_id and entity_id not in processed_ids:
|
||||
entity_id = get_entity_id(entity)
|
||||
if entity_id is not None and entity_id not in processed_ids:
|
||||
merged_entities.append(entity)
|
||||
|
||||
self.logger.info(f"Merged to {len(merged_entities)} entities")
|
||||
|
||||
@@ -33,6 +33,12 @@ class SHACLViolation:
|
||||
value: Optional[str] = None
|
||||
shape: Optional[str] = None
|
||||
explanation: Optional[str] = None
|
||||
# Real constraint parameters extracted from the source shape (sh:sourceShape),
|
||||
# used to render accurate plain-English explanations.
|
||||
min_count: Optional[int] = None
|
||||
max_count: Optional[int] = None
|
||||
datatype: Optional[str] = None
|
||||
class_: Optional[str] = None
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
@@ -44,6 +50,10 @@ class SHACLViolation:
|
||||
"value": self.value,
|
||||
"shape": self.shape,
|
||||
"explanation": self.explanation,
|
||||
"min_count": self.min_count,
|
||||
"max_count": self.max_count,
|
||||
"datatype": self.datatype,
|
||||
"class_": self.class_,
|
||||
}
|
||||
|
||||
|
||||
@@ -118,10 +128,10 @@ class SHACLValidationReport:
|
||||
focus_node=v.focus_node,
|
||||
path=v.result_path or "",
|
||||
value=v.value or "",
|
||||
min_count=1,
|
||||
max_count=1,
|
||||
datatype=v.message or "",
|
||||
class_=v.message or "",
|
||||
min_count=v.min_count if v.min_count is not None else "?",
|
||||
max_count=v.max_count if v.max_count is not None else "?",
|
||||
datatype=v.datatype or "the expected datatype",
|
||||
class_=v.class_ or "the required class",
|
||||
)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
@@ -208,6 +218,32 @@ def _run_pyshacl(
|
||||
shape_node = results_graph.value(result, SH.sourceShape)
|
||||
shape = str(shape_node) if shape_node is not None else None
|
||||
|
||||
# Look up the real constraint parameters from the source shape so that
|
||||
# explain_violations can render accurate values instead of placeholders.
|
||||
# Note: sh:qualifiedMinCount / sh:qualifiedMaxCount are not handled here;
|
||||
# such violations fall back to the "?" placeholder in explain_violations.
|
||||
min_count: Optional[int] = None
|
||||
max_count: Optional[int] = None
|
||||
datatype: Optional[str] = None
|
||||
class_: Optional[str] = None
|
||||
if shape_node is not None:
|
||||
min_node = shacl_g.value(shape_node, SH.minCount)
|
||||
if min_node is not None:
|
||||
try:
|
||||
min_count = int(str(min_node))
|
||||
except (TypeError, ValueError):
|
||||
min_count = None
|
||||
max_node = shacl_g.value(shape_node, SH.maxCount)
|
||||
if max_node is not None:
|
||||
try:
|
||||
max_count = int(str(max_node))
|
||||
except (TypeError, ValueError):
|
||||
max_count = None
|
||||
dt_node = shacl_g.value(shape_node, SH.datatype)
|
||||
datatype = str(dt_node) if dt_node is not None else None
|
||||
cls_node = shacl_g.value(shape_node, SH["class"])
|
||||
class_ = str(cls_node) if cls_node is not None else None
|
||||
|
||||
v = SHACLViolation(
|
||||
focus_node=focus,
|
||||
result_path=path,
|
||||
@@ -216,6 +252,10 @@ def _run_pyshacl(
|
||||
message=msg,
|
||||
value=val,
|
||||
shape=shape,
|
||||
min_count=min_count,
|
||||
max_count=max_count,
|
||||
datatype=datatype,
|
||||
class_=class_,
|
||||
)
|
||||
if sev_str == "Violation":
|
||||
violations.append(v)
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
"""The vocabulary Semantica's exporters emit terms from.
|
||||
|
||||
Every RDF export mints terms in ``https://semantica.dev/ns#``: ``sem:text``,
|
||||
``sem:confidence``, the default ``sem:Entity`` type, and the rest. Until this
|
||||
file existed, nothing declared what those terms meant, so a consumer receiving
|
||||
an export could not tell ``sem:text`` from a typo of it, and no closed-world
|
||||
check could be run against them at all (issue #1107).
|
||||
|
||||
The document ships inside the package so it can be loaded without a network
|
||||
round trip, and is the same file intended to be served at the namespace IRI.
|
||||
|
||||
>>> from semantica.ontology.vocabulary import vocabulary_turtle
|
||||
>>> ttl = vocabulary_turtle()
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
VOCABULARY_FILENAME = "semantica-ns.ttl"
|
||||
|
||||
#: The namespace the vocabulary declares terms in.
|
||||
NAMESPACE = "https://semantica.dev/ns#"
|
||||
|
||||
__all__ = ["NAMESPACE", "VOCABULARY_FILENAME", "vocabulary_path", "vocabulary_turtle"]
|
||||
|
||||
|
||||
def vocabulary_path() -> Path:
|
||||
"""Filesystem path to the vocabulary document."""
|
||||
return Path(__file__).parent / VOCABULARY_FILENAME
|
||||
|
||||
|
||||
def vocabulary_turtle() -> str:
|
||||
"""The vocabulary document as Turtle."""
|
||||
return vocabulary_path().read_text(encoding="utf-8")
|
||||
@@ -0,0 +1,157 @@
|
||||
@prefix owl: <http://www.w3.org/2002/07/owl#> .
|
||||
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
|
||||
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
|
||||
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
|
||||
@prefix dct: <http://purl.org/dc/terms/> .
|
||||
@prefix prov: <http://www.w3.org/ns/prov#> .
|
||||
@prefix time: <http://www.w3.org/2006/time#> .
|
||||
@prefix sem: <https://semantica.dev/ns#> .
|
||||
|
||||
<https://semantica.dev/ns> a owl:Ontology ;
|
||||
rdfs:label "Semantica vocabulary" ;
|
||||
rdfs:comment """Declares the terms the Semantica exporters emit in
|
||||
https://semantica.dev/ns#. Drafted from the emitting call sites in
|
||||
semantica 0.6.5: export/rdf_exporter.py, export/json_exporter.py and
|
||||
provenance/manager.py. Every term below appears in output the package
|
||||
produces today; no term has been invented for completeness.""" ;
|
||||
owl:versionInfo "0.1.0-draft" ;
|
||||
dct:created "2026-08-19"^^xsd:date .
|
||||
|
||||
# ── Classes ──────────────────────────────────────────────────────────────────
|
||||
|
||||
sem:Entity a owl:Class ;
|
||||
rdfs:label "Entity" ;
|
||||
rdfs:comment """The default type given to an extracted entity when the
|
||||
source carries no type of its own. Emitted by serialize_to_turtle as the
|
||||
fallback for entity.get("type").""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:Relationship a owl:Class ;
|
||||
rdfs:label "Relationship" ;
|
||||
rdfs:comment """A reified relationship, as emitted in the JSON-LD export
|
||||
where a relationship carries sem:type, sem:source and sem:target rather than
|
||||
being written as a single triple.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:KnowledgeGraph a owl:Class ;
|
||||
rdfs:label "Knowledge Graph" ;
|
||||
rdfs:comment """The document-level type of a JSON-LD export: the @type of
|
||||
the top-level node carrying sem:entities, sem:relationships and
|
||||
sem:exportedAt. Emitted by _convert_kg_to_jsonld in export/json_exporter.py.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Properties on an entity ──────────────────────────────────────────────────
|
||||
|
||||
sem:text a owl:DatatypeProperty ;
|
||||
rdfs:label "text" ;
|
||||
rdfs:comment """The surface text of an extracted entity. Carries the same
|
||||
intent as rdfs:label; declared separately because the exporters emit it under
|
||||
this IRI.""" ;
|
||||
rdfs:domain sem:Entity ;
|
||||
rdfs:range xsd:string ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:confidence a owl:DatatypeProperty ;
|
||||
rdfs:label "confidence" ;
|
||||
rdfs:comment """Extractor confidence in the assertion, on the unit interval.
|
||||
Emitted for both entities and relationships, so the domain is left open rather
|
||||
than tied to sem:Entity.
|
||||
|
||||
No rdfs:range is declared, deliberately. The Turtle serializer writes the value
|
||||
bare, which the Turtle grammar reads as xsd:decimal, while the N-Triples
|
||||
serializer types it xsd:float explicitly, and those two datatypes are disjoint.
|
||||
Declaring either one would make the vocabulary contradict one of the exporters.
|
||||
Issue #1100 tracks the disagreement; a range belongs here once the serializers
|
||||
agree on one.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:metadata a owl:AnnotationProperty ;
|
||||
rdfs:label "metadata" ;
|
||||
rdfs:comment """Free-form metadata carried through from extraction. An
|
||||
annotation property because its value is an arbitrary structure rather than a
|
||||
modelled one.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Relationship terms (JSON-LD export) ──────────────────────────────────────
|
||||
|
||||
sem:related_to a owl:ObjectProperty ;
|
||||
rdfs:label "related to" ;
|
||||
rdfs:comment """The default predicate for a relationship whose type the
|
||||
extractor did not determine. Deliberately unspecific: it asserts that two
|
||||
entities are connected and nothing about how.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:source a owl:ObjectProperty ;
|
||||
rdfs:label "source" ;
|
||||
rdfs:comment "The subject entity of a reified relationship." ;
|
||||
rdfs:domain sem:Relationship ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:target a owl:ObjectProperty ;
|
||||
rdfs:label "target" ;
|
||||
rdfs:comment "The object entity of a reified relationship." ;
|
||||
rdfs:domain sem:Relationship ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:type a owl:DatatypeProperty ;
|
||||
rdfs:label "type" ;
|
||||
rdfs:comment """The relationship type as a label, as emitted in the JSON-LD
|
||||
export. Distinct from rdf:type, which relates a node to a class rather than to
|
||||
a string.""" ;
|
||||
rdfs:domain sem:Relationship ;
|
||||
rdfs:range xsd:string ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Document-level terms (JSON-LD export) ────────────────────────────────────
|
||||
|
||||
sem:entities a owl:ObjectProperty ;
|
||||
rdfs:label "entities" ;
|
||||
rdfs:comment "Ordered list of entities in an exported graph document." ;
|
||||
rdfs:range sem:Entity ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:relationships a owl:ObjectProperty ;
|
||||
rdfs:label "relationships" ;
|
||||
rdfs:comment "Ordered list of relationships in an exported graph document." ;
|
||||
rdfs:range sem:Relationship ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:exportedAt a owl:DatatypeProperty ;
|
||||
rdfs:label "exported at" ;
|
||||
rdfs:comment """When the export was written, as an ISO 8601 timestamp with
|
||||
an explicit UTC offset. The range was xsd:dateTime while the exporters stamped
|
||||
with a naive datetime.now(); with the offset present (#1114) the value is a
|
||||
determinate instant, comparable against a timestamp written anywhere else, so
|
||||
the range is the stricter xsd:dateTimeStamp, which requires the offset.""" ;
|
||||
rdfs:range xsd:dateTimeStamp ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:format a owl:DatatypeProperty ;
|
||||
rdfs:label "format" ;
|
||||
rdfs:comment """The serialization format label written on a JSON-LD
|
||||
document (currently always the literal "json-ld"). Emitted by
|
||||
JSONExporter.export_to_jsonld in export/json_exporter.py.""" ;
|
||||
rdfs:range xsd:string ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Temporal term (OWL-Time export) ──────────────────────────────────────────
|
||||
|
||||
sem:openEndedInterval a owl:DatatypeProperty ;
|
||||
rdfs:label "open ended interval" ;
|
||||
rdfs:comment """True when an interval has no known end. OWL-Time has no
|
||||
standard predicate for this, which is the reason the exporter mints one: an
|
||||
interval with no time:hasEnd is ambiguous between "ongoing" and "end not
|
||||
recorded", and this term resolves that in favour of the first.""" ;
|
||||
rdfs:domain time:Interval ;
|
||||
rdfs:range xsd:boolean ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Provenance roles ─────────────────────────────────────────────────────────
|
||||
|
||||
sem:role_generator a prov:Role ;
|
||||
rdfs:label "generator" ;
|
||||
rdfs:comment """The default role in a prov:qualifiedAssociation, used when
|
||||
an agent generated an entity rather than approving or reviewing it. Typed as
|
||||
prov:Role so that prov:hadRole has a declared value rather than an undeclared
|
||||
IRI.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
@@ -61,9 +61,10 @@ License: MIT
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional, Dict, Any, List
|
||||
from datetime import datetime
|
||||
import uuid
|
||||
|
||||
from ..utils.helpers import utc_now_iso
|
||||
|
||||
|
||||
@dataclass
|
||||
class BridgeAxiom:
|
||||
@@ -280,7 +281,7 @@ class TranslationChain:
|
||||
"type": layer_type,
|
||||
"value": value,
|
||||
"source": source,
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"timestamp": utc_now_iso(),
|
||||
**kwargs
|
||||
}
|
||||
self.layers.append(layer)
|
||||
|
||||
@@ -26,7 +26,7 @@ License: MIT
|
||||
|
||||
from typing import Optional, List, Dict, Any, Union
|
||||
from collections.abc import Mapping
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from contextlib import contextmanager
|
||||
import copy
|
||||
import inspect
|
||||
@@ -36,8 +36,13 @@ import threading
|
||||
from .schemas import ProvenanceEntry, SourceReference, AgentRecord, ActivityRecord
|
||||
from .storage import ProvenanceStorage, InMemoryStorage, SQLiteStorage
|
||||
from .integrity import compute_checksum, verify_checksum
|
||||
from ..utils.helpers import to_utc_datetime, utc_now_iso
|
||||
from ..utils.logging import get_logger
|
||||
|
||||
#: Sort key for an entry whose timestamp cannot be read as one, so an
|
||||
#: unreadable value orders first instead of raising during a sort.
|
||||
_EPOCH = datetime(1, 1, 1, tzinfo=timezone.utc)
|
||||
|
||||
# Issue #825, Part B Tier 3 — configurable base URI for export_prov(), shared
|
||||
# with RDFExporter's NamespaceManager "semantica" entry (semantica/export/
|
||||
# rdf_exporter.py) so KG-exported and PROV-exported URIs for the same
|
||||
@@ -363,8 +368,8 @@ class ProvenanceManager:
|
||||
source_quote=kwargs.get("source_quote"),
|
||||
confidence=kwargs.get("confidence", 1.0),
|
||||
metadata=metadata or {},
|
||||
first_seen=existing.first_seen if existing else datetime.utcnow().isoformat(),
|
||||
last_updated=datetime.utcnow().isoformat(),
|
||||
first_seen=existing.first_seen if existing else utc_now_iso(),
|
||||
last_updated=utc_now_iso(),
|
||||
parent_entity_id=parent_id,
|
||||
used_entities=list(kwargs.get("used_entities", [])),
|
||||
activity_started_at_time=activity_info["activity_started_at_time"],
|
||||
@@ -455,8 +460,8 @@ class ProvenanceManager:
|
||||
source_location=kwargs.get("source_location"),
|
||||
confidence=kwargs.get("confidence", 1.0),
|
||||
metadata=metadata or {},
|
||||
first_seen=datetime.utcnow().isoformat(),
|
||||
last_updated=datetime.utcnow().isoformat(),
|
||||
first_seen=utc_now_iso(),
|
||||
last_updated=utc_now_iso(),
|
||||
activity_started_at_time=activity_info["activity_started_at_time"],
|
||||
activity_ended_at_time=activity_info["activity_ended_at_time"],
|
||||
acted_on_behalf_of=kwargs.get("acted_on_behalf_of"),
|
||||
@@ -534,7 +539,7 @@ class ProvenanceManager:
|
||||
# split (issue #825, Part A item 4).
|
||||
derived_from_id=parent_chunk_id,
|
||||
metadata=metadata,
|
||||
timestamp=datetime.utcnow().isoformat(),
|
||||
timestamp=utc_now_iso(),
|
||||
activity_started_at_time=activity_info["activity_started_at_time"],
|
||||
activity_ended_at_time=activity_info["activity_ended_at_time"],
|
||||
)
|
||||
@@ -604,7 +609,7 @@ class ProvenanceManager:
|
||||
**metadata,
|
||||
**source.metadata
|
||||
},
|
||||
timestamp=datetime.utcnow().isoformat(),
|
||||
timestamp=utc_now_iso(),
|
||||
activity_started_at_time=activity_info["activity_started_at_time"],
|
||||
activity_ended_at_time=activity_info["activity_ended_at_time"],
|
||||
)
|
||||
@@ -959,11 +964,27 @@ class ProvenanceManager:
|
||||
Returns:
|
||||
List of matching entries as dicts, sorted by timestamp ascending.
|
||||
"""
|
||||
matches = [
|
||||
e for e in self.storage.retrieve_all()
|
||||
if e.timestamp and start <= e.timestamp <= end
|
||||
]
|
||||
matches.sort(key=lambda e: e.timestamp)
|
||||
# Compare instants, not spellings. Since #1114 new entries carry a
|
||||
# +00:00 offset while entries written earlier do not, and a raw string
|
||||
# comparison orders those two by length: an inclusive naive bound equal
|
||||
# to a stored offset-bearing timestamp would sort below it and drop the
|
||||
# record. A bound in another offset was mis-ordered the same way.
|
||||
start_at = to_utc_datetime(start)
|
||||
end_at = to_utc_datetime(end)
|
||||
entries = [e for e in self.storage.retrieve_all() if e.timestamp]
|
||||
|
||||
if start_at is None or end_at is None:
|
||||
# A bound this module cannot read as a timestamp keeps the historical
|
||||
# string comparison rather than raising on a call that used to work.
|
||||
matches = [e for e in entries if start <= e.timestamp <= end]
|
||||
else:
|
||||
matches = [
|
||||
e for e in entries
|
||||
if (at := to_utc_datetime(e.timestamp)) is not None
|
||||
and start_at <= at <= end_at
|
||||
]
|
||||
|
||||
matches.sort(key=lambda e: (to_utc_datetime(e.timestamp) or _EPOCH, e.timestamp))
|
||||
return [e.to_dict() for e in matches]
|
||||
|
||||
def get_all_sources(self, entity_id: str) -> List[Dict[str, Any]]:
|
||||
@@ -1071,7 +1092,7 @@ class ProvenanceManager:
|
||||
|
||||
entry = copy.deepcopy(existing)
|
||||
entry.invalidated = True
|
||||
entry.invalidated_at_time = datetime.utcnow().isoformat()
|
||||
entry.invalidated_at_time = utc_now_iso()
|
||||
entry.invalidated_by = agent_id
|
||||
entry.invalidation_reason = reason
|
||||
entry.previous_version_id = history_id
|
||||
@@ -1163,8 +1184,22 @@ class ProvenanceManager:
|
||||
"""
|
||||
entries = self.storage.retrieve_all()
|
||||
if since:
|
||||
entries = [e for e in entries if getattr(e, "timestamp", "") >= since]
|
||||
entries.sort(key=lambda e: getattr(e, "timestamp", ""))
|
||||
since_at = to_utc_datetime(since)
|
||||
if since_at is None:
|
||||
entries = [e for e in entries
|
||||
if getattr(e, "timestamp", "") >= since]
|
||||
else:
|
||||
entries = [
|
||||
e for e in entries
|
||||
if (at := to_utc_datetime(getattr(e, "timestamp", None)))
|
||||
is not None and at >= since_at
|
||||
]
|
||||
entries.sort(
|
||||
key=lambda e: (
|
||||
to_utc_datetime(getattr(e, "timestamp", None)) or _EPOCH,
|
||||
getattr(e, "timestamp", ""),
|
||||
)
|
||||
)
|
||||
|
||||
if format == "json":
|
||||
return [
|
||||
|
||||
@@ -30,6 +30,8 @@ from dataclasses import dataclass, field
|
||||
from typing import Optional, List, Dict, Any
|
||||
from datetime import datetime
|
||||
|
||||
from ..utils.helpers import utc_now_iso
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProvenanceEntry:
|
||||
@@ -91,7 +93,7 @@ class ProvenanceEntry:
|
||||
source_quote: Optional[str] = None
|
||||
|
||||
# Temporal tracking (from kg.ProvenanceTracker)
|
||||
timestamp: str = field(default_factory=lambda: datetime.utcnow().isoformat())
|
||||
timestamp: str = field(default_factory=lambda: utc_now_iso())
|
||||
first_seen: Optional[str] = None
|
||||
last_updated: Optional[str] = None
|
||||
|
||||
|
||||
@@ -38,6 +38,7 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
from ..ingest.ssrf import request_with_ssrf_guard
|
||||
from ..utils.exceptions import ProcessingError, ValidationError
|
||||
from ..utils.helpers import read_json_file, write_json_file
|
||||
from ..utils.logging import get_logger
|
||||
@@ -456,8 +457,8 @@ class SeedDataManager:
|
||||
"""
|
||||
Load seed data from API.
|
||||
|
||||
Makes an HTTP GET request to an API endpoint and parses the JSON
|
||||
response. Handles various response structures (list, dict with
|
||||
Makes an SSRF-protected HTTP GET request to an API endpoint and parses
|
||||
the JSON response. Handles various response structures (list, dict with
|
||||
'entities', 'data', 'results', 'items' keys). Automatically adds
|
||||
entity_type, relationship_type, and source metadata if provided.
|
||||
|
||||
@@ -493,8 +494,6 @@ class SeedDataManager:
|
||||
... )
|
||||
"""
|
||||
try:
|
||||
import requests
|
||||
|
||||
# Build full URL
|
||||
if endpoint:
|
||||
full_url = f"{api_url.rstrip('/')}/{endpoint.lstrip('/')}"
|
||||
|
||||
@@ -82,6 +82,9 @@ from .helpers import (
|
||||
normalize_entities,
|
||||
normalize_graph_payload,
|
||||
parse_timestamp,
|
||||
to_utc_datetime,
|
||||
utc_now,
|
||||
utc_now_iso,
|
||||
read_json_file,
|
||||
retry_on_error,
|
||||
safe_filename,
|
||||
@@ -193,6 +196,9 @@ __all__ = [
|
||||
"get_file_size",
|
||||
"format_timestamp",
|
||||
"parse_timestamp",
|
||||
"to_utc_datetime",
|
||||
"utc_now",
|
||||
"utc_now_iso",
|
||||
"merge_dicts",
|
||||
"chunk_list",
|
||||
"flatten_dict",
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Helpers for reading entity identifiers consistently across the KG pipeline."""
|
||||
|
||||
from typing import Any
|
||||
|
||||
|
||||
def get_entity_id(entity: Any) -> Any:
|
||||
"""Return a truthy identifier from either supported entity ID field.
|
||||
|
||||
The KG pipeline treats empty and otherwise falsy identifiers as missing.
|
||||
Prefer the canonical ``id`` field when it is populated, then fall back to
|
||||
the compatible ``entity_id`` alias.
|
||||
"""
|
||||
if isinstance(entity, dict):
|
||||
return entity.get("id") or entity.get("entity_id") or None
|
||||
|
||||
return getattr(entity, "id", None) or getattr(entity, "entity_id", None) or None
|
||||
@@ -320,6 +320,65 @@ def format_timestamp(
|
||||
return dt.strftime(format_str)
|
||||
|
||||
|
||||
def utc_now() -> datetime:
|
||||
"""
|
||||
Current instant as a timezone-aware UTC datetime.
|
||||
|
||||
``datetime.now()`` reads the local clock and ``datetime.utcnow()`` reads UTC,
|
||||
but both return a naive datetime, and the two are indistinguishable once
|
||||
serialized: a consumer cannot tell which zone the value belongs to, and an
|
||||
RDF timestamp without an offset is not comparable against one that has an
|
||||
offset (a SPARQL FILTER drops it rather than reporting an error). Use this
|
||||
for any timestamp that leaves the process.
|
||||
|
||||
Returns:
|
||||
Current UTC time, timezone-aware
|
||||
"""
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def utc_now_iso() -> str:
|
||||
"""
|
||||
Current instant as an ISO 8601 string carrying an explicit UTC offset.
|
||||
|
||||
Returns:
|
||||
Timestamp string such as ``2026-08-19T14:19:04.229937+00:00``, which is
|
||||
a valid ``xsd:dateTimeStamp`` and orders correctly against timestamps
|
||||
written in any other timezone
|
||||
"""
|
||||
return utc_now().isoformat()
|
||||
|
||||
|
||||
def to_utc_datetime(value: Union[str, datetime, None]) -> Optional[datetime]:
|
||||
"""
|
||||
Read an ISO 8601 timestamp as a timezone-aware UTC instant.
|
||||
|
||||
Timestamps written before #1114 carry no offset. They were produced by
|
||||
``datetime.utcnow()``, so a missing offset is read as UTC: that keeps a
|
||||
stored naive value and the same instant written with an offset comparing
|
||||
equal, instead of ordering by how the timestamp happens to be spelled.
|
||||
|
||||
Args:
|
||||
value: ISO 8601 string or datetime. ``Z`` is accepted as the offset.
|
||||
|
||||
Returns:
|
||||
Timezone-aware UTC datetime, or None if the value cannot be read as a
|
||||
timestamp, so callers can fall back rather than raise on stored data
|
||||
"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, datetime):
|
||||
parsed = value
|
||||
else:
|
||||
try:
|
||||
parsed = datetime.fromisoformat(str(value).replace("Z", "+00:00"))
|
||||
except ValueError:
|
||||
return None
|
||||
if parsed.tzinfo is None:
|
||||
parsed = parsed.replace(tzinfo=timezone.utc)
|
||||
return parsed.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def parse_timestamp(timestamp_str: str, format_str: Optional[str] = None) -> datetime:
|
||||
"""
|
||||
Parse timestamp string to datetime.
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
"""The JSON-LD paths must mint the same IRIs as the RDF paths (issue #1101).
|
||||
|
||||
#1101 was fixed for the Turtle, N-Triples and RDF/XML serializers: an entity
|
||||
arriving without an id gets a deterministic IRI in the declared namespace. The
|
||||
JSON-LD paths were left interpolating the entity's own text into
|
||||
``f"semantica:entity/{text}"`` and the relationship's endpoints into
|
||||
``f"semantica:rel/{source}_{target}"``, which fails three ways:
|
||||
|
||||
* a text containing a space produces an invalid IRI, and a JSON-LD parser drops
|
||||
the whole node rather than complaining, so the entity vanishes from the export;
|
||||
* relationships carrying ``source``/``target`` rather than ``source_id``/
|
||||
``target_id`` all minted ``semantica:rel/_``, so every one of them collapsed
|
||||
onto a single node whose types and endpoints merged;
|
||||
* the JSON-LD @id and the Turtle IRI for one entity disagreed, so the two
|
||||
serializations of one knowledge graph were two different graphs.
|
||||
|
||||
``JSONExporter.export_entities`` and ``export_relationships`` also wrote
|
||||
``semantica:entities`` into a context that never declared the ``semantica``
|
||||
prefix, which a JSON-LD processor reads as an IRI in the scheme ``semantica`` —
|
||||
the original #1101 failure mode, on a path the first fix did not cover.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from semantica.export.json_exporter import JSONExporter
|
||||
from semantica.export.rdf_exporter import (
|
||||
RDFExporter,
|
||||
SEMANTICA_NS,
|
||||
mint_entity_iri,
|
||||
mint_relationship_iri,
|
||||
)
|
||||
|
||||
KG = {
|
||||
"entities": [
|
||||
{"text": "Acme Corp", "type": "https://example.org/Org"},
|
||||
{"id": "https://example.org/e2", "text": "Bob"},
|
||||
],
|
||||
"relationships": [
|
||||
{"source": "https://example.org/a", "target": "https://example.org/b",
|
||||
"type": "https://example.org/employs"},
|
||||
{"source": "https://example.org/b", "target": "https://example.org/a",
|
||||
"type": "https://example.org/works_for"},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _graph(document: str):
|
||||
"""Parse a JSON-LD document the way a consumer would."""
|
||||
rdflib = pytest.importorskip("rdflib")
|
||||
graph = rdflib.Graph()
|
||||
graph.parse(data=document, format="json-ld")
|
||||
return graph
|
||||
|
||||
|
||||
def test_jsonld_entity_id_is_the_minted_iri_not_the_interpolated_text():
|
||||
graph = json.loads(RDFExporter().export_to_rdf(KG, format="jsonld"))["@graph"]
|
||||
|
||||
assert graph[0]["@id"] == mint_entity_iri("Acme Corp")
|
||||
assert graph[0]["@id"].startswith(SEMANTICA_NS)
|
||||
assert "semantica:entity/" not in json.dumps(graph)
|
||||
|
||||
|
||||
def test_json_exporter_mints_the_same_entity_iri_as_the_rdf_exporter():
|
||||
"""One knowledge graph, two exporters, one node identity."""
|
||||
from_rdf = json.loads(RDFExporter().export_to_rdf(KG, format="jsonld"))["@graph"]
|
||||
from_json = JSONExporter()._convert_kg_to_jsonld(KG)
|
||||
|
||||
assert from_json["semantica:entities"][0]["@id"] == from_rdf[0]["@id"]
|
||||
assert from_json["semantica:relationships"][0]["@id"] == from_rdf[2]["@id"]
|
||||
|
||||
|
||||
def test_minted_jsonld_iri_agrees_with_the_turtle_serialization():
|
||||
"""The two serializations of one graph must name the same entity alike."""
|
||||
turtle = RDFExporter().export_to_rdf(KG, format="turtle")
|
||||
jsonld = json.loads(RDFExporter().export_to_rdf(KG, format="jsonld"))
|
||||
|
||||
minted = mint_entity_iri("Acme Corp")
|
||||
assert f"<{minted}>" in turtle
|
||||
assert jsonld["@graph"][0]["@id"] == minted
|
||||
|
||||
|
||||
def test_entity_whose_text_contains_a_space_survives_a_jsonld_parse():
|
||||
"""The regression that lost data: an invalid IRI is dropped, not reported."""
|
||||
kg = {"entities": [{"text": "Acme Corp"}], "relationships": []}
|
||||
graph = _graph(json.dumps(JSONExporter()._convert_kg_to_jsonld(kg)))
|
||||
|
||||
subjects = {str(s) for s in graph.subjects()}
|
||||
assert mint_entity_iri("Acme Corp") in subjects
|
||||
|
||||
|
||||
def test_relationships_carrying_source_and_target_do_not_collide():
|
||||
"""Two relationships, two nodes: ``semantica:rel/_`` merged them into one."""
|
||||
jsonld = json.loads(RDFExporter().export_to_rdf(KG, format="jsonld"))
|
||||
relationships = [n for n in jsonld["@graph"]
|
||||
if n["@type"] == "semantica:Relationship"]
|
||||
|
||||
ids = {node["@id"] for node in relationships}
|
||||
assert len(ids) == len(relationships) == 2
|
||||
assert ids == {
|
||||
mint_relationship_iri(0, "https://example.org/a", "https://example.org/b"),
|
||||
mint_relationship_iri(1, "https://example.org/b", "https://example.org/a"),
|
||||
}
|
||||
|
||||
graph = _graph(json.dumps(jsonld))
|
||||
assert len({str(s) for s in graph.subjects()} & ids) == 2
|
||||
|
||||
|
||||
def test_no_export_path_writes_an_iri_in_the_semantica_scheme(tmp_path):
|
||||
"""Nothing may expand to the scheme ``semantica`` rather than the namespace."""
|
||||
documents = [
|
||||
RDFExporter().export_to_rdf(KG, format="jsonld"),
|
||||
json.dumps(JSONExporter()._convert_kg_to_jsonld(KG)),
|
||||
]
|
||||
exporter = JSONExporter()
|
||||
exporter.export_entities(KG["entities"], tmp_path / "entities.json")
|
||||
exporter.export_relationships(KG["relationships"], tmp_path / "relationships.json")
|
||||
documents.append((tmp_path / "entities.json").read_text())
|
||||
documents.append((tmp_path / "relationships.json").read_text())
|
||||
|
||||
for document in documents:
|
||||
for term in _graph(document).all_nodes():
|
||||
assert not str(term).startswith("semantica:"), document
|
||||
for predicate in _graph(document).predicates():
|
||||
assert not str(predicate).startswith("semantica:"), document
|
||||
|
||||
|
||||
def test_entity_and_relationship_lists_expand_into_the_declared_namespace(tmp_path):
|
||||
exporter = JSONExporter()
|
||||
exporter.export_entities(KG["entities"], tmp_path / "entities.json")
|
||||
exporter.export_relationships(KG["relationships"], tmp_path / "relationships.json")
|
||||
|
||||
predicates = set()
|
||||
for name in ("entities.json", "relationships.json"):
|
||||
predicates |= {str(p) for p in _graph((tmp_path / name).read_text()).predicates()}
|
||||
|
||||
assert f"{SEMANTICA_NS}entities" in predicates
|
||||
assert f"{SEMANTICA_NS}relationships" in predicates
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Minted IRIs must be stable and must sit in the declared namespace (issue #1101).
|
||||
|
||||
An entity that arrives without an id gets one minted for it. That identifier was
|
||||
built from Python's builtin ``hash()``, which is randomised per process, so the
|
||||
same entity received a different IRI on every run and exports could not be
|
||||
diffed, deduplicated against an earlier load, or joined to a provenance record
|
||||
written by an earlier process.
|
||||
|
||||
It was also written as ``semantica:entity_N`` inside angle brackets, which is an
|
||||
IRI in the scheme ``semantica`` rather than the expansion of the declared
|
||||
``semantica:`` prefix, so it never joined with anything written through it.
|
||||
"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from semantica.export.rdf_exporter import (
|
||||
DEFAULT_ENTITY_TYPE,
|
||||
DEFAULT_RELATION_TYPE,
|
||||
RDFExporter,
|
||||
SEMANTICA_NS,
|
||||
mint_entity_iri,
|
||||
mint_relationship_iri,
|
||||
)
|
||||
|
||||
UNIDENTIFIED = {
|
||||
"entities": [{"text": "Acme Corp", "type": "https://example.org/Org"}],
|
||||
"relationships": [],
|
||||
}
|
||||
|
||||
|
||||
def test_minted_entity_iri_is_stable_within_a_process():
|
||||
assert mint_entity_iri("Acme Corp") == mint_entity_iri("Acme Corp")
|
||||
|
||||
|
||||
def test_minted_entity_iri_is_stable_across_processes():
|
||||
"""The regression that matters: identity must survive a restart."""
|
||||
script = (
|
||||
"from semantica.export.rdf_exporter import mint_entity_iri;"
|
||||
"print(mint_entity_iri('Acme Corp'))"
|
||||
)
|
||||
runs = {
|
||||
subprocess.run(
|
||||
[sys.executable, "-c", script],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
env={**os.environ, "PYTHONHASHSEED": seed},
|
||||
).stdout.strip()
|
||||
for seed in ("0", "1", "random")
|
||||
}
|
||||
assert len(runs) == 1, f"minted IRI differs between processes: {runs}"
|
||||
|
||||
|
||||
def test_minted_iris_are_in_the_declared_namespace():
|
||||
assert mint_entity_iri("Acme Corp").startswith(SEMANTICA_NS)
|
||||
assert mint_relationship_iri(0, "a", "b").startswith(SEMANTICA_NS)
|
||||
|
||||
|
||||
def test_distinct_entities_get_distinct_iris():
|
||||
assert mint_entity_iri("Acme Corp") != mint_entity_iri("Acme Corporation")
|
||||
|
||||
|
||||
def test_turtle_export_writes_a_resolvable_minted_iri():
|
||||
turtle = RDFExporter().export_to_rdf(UNIDENTIFIED, format="turtle")
|
||||
|
||||
assert f"<{SEMANTICA_NS}entity_" in turtle
|
||||
assert "<semantica:entity_" not in turtle, "scheme 'semantica' is not the prefix"
|
||||
|
||||
|
||||
def test_ntriples_export_agrees_with_turtle_on_the_minted_iri():
|
||||
exporter = RDFExporter()
|
||||
minted = mint_entity_iri("Acme Corp")
|
||||
|
||||
assert minted in exporter.export_to_rdf(UNIDENTIFIED, format="turtle")
|
||||
assert minted in exporter.export_to_rdf(UNIDENTIFIED, format="ntriples")
|
||||
|
||||
|
||||
def test_default_types_are_written_as_full_iris_in_turtle():
|
||||
untyped = {"entities": [{"id": "https://example.org/e1", "text": "A"}],
|
||||
"relationships": [{"source": "https://example.org/e1",
|
||||
"target": "https://example.org/e2"}]}
|
||||
turtle = RDFExporter().export_to_rdf(untyped, format="turtle")
|
||||
|
||||
assert f"<{DEFAULT_ENTITY_TYPE}>" in turtle
|
||||
assert f"<{DEFAULT_RELATION_TYPE}>" in turtle
|
||||
assert "<semantica:Entity>" not in turtle
|
||||
assert "<semantica:related_to>" not in turtle
|
||||
|
||||
|
||||
def test_default_entity_type_is_a_full_iri_in_rdfxml():
|
||||
"""RDF/XML's rdf:resource is an attribute value, not a QName context, so a
|
||||
|
||||
prefixed default there (``semantica:Entity``) resolves to the scheme
|
||||
``semantica`` rather than the declared namespace — the same failure mode
|
||||
fixed for Turtle in #1101, missed here because the original tests only
|
||||
checked Turtle output.
|
||||
"""
|
||||
untyped = {"entities": [{"id": "https://example.org/e1", "text": "A"}],
|
||||
"relationships": []}
|
||||
rdfxml = RDFExporter().export_to_rdf(untyped, format="rdfxml")
|
||||
|
||||
assert f'rdf:resource="{DEFAULT_ENTITY_TYPE}"' in rdfxml
|
||||
assert 'rdf:resource="semantica:Entity"' not in rdfxml
|
||||
|
||||
|
||||
def test_temporal_minting_uses_either_endpoint_representation():
|
||||
"""Relationships may carry source/target or source_id/target_id (#1109 review).
|
||||
|
||||
Minting from source_id alone hashed empty strings for every relationship
|
||||
that used the other representation, so once the IRI became deterministic,
|
||||
unrelated relationships at the same index collided on it and their temporal
|
||||
data aliased when the exports were loaded together.
|
||||
"""
|
||||
def temporal(rel):
|
||||
return RDFExporter().export_to_rdf(
|
||||
{"entities": [], "relationships": [rel]},
|
||||
format="turtle",
|
||||
include_temporal=True,
|
||||
)
|
||||
|
||||
a = temporal({"source": "https://example.org/a", "target": "https://example.org/b",
|
||||
"type": "https://example.org/worksFor", "valid_from": "2020-01-01T00:00:00Z"})
|
||||
b = temporal({"source": "https://example.org/c", "target": "https://example.org/d",
|
||||
"type": "https://example.org/worksFor", "valid_from": "2020-01-01T00:00:00Z"})
|
||||
|
||||
assert f"<{SEMANTICA_NS}rel_" in a
|
||||
assert a != b, "different endpoints must not mint the same temporal IRI"
|
||||
|
||||
|
||||
def test_temporal_minting_agrees_across_the_two_representations():
|
||||
"""The same relationship written either way is the same relationship."""
|
||||
def mint(rel):
|
||||
return mint_relationship_iri(
|
||||
0,
|
||||
rel.get("source_id") or rel.get("source") or "",
|
||||
rel.get("target_id") or rel.get("target") or "",
|
||||
)
|
||||
|
||||
assert mint({"source": "a", "target": "b"}) == mint({"source_id": "a", "target_id": "b"})
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Timestamps that leave the process must carry a timezone (issue #1114).
|
||||
|
||||
Every timestamp an exporter wrote was naive: ``datetime.now().isoformat()``
|
||||
reads the local clock, ``datetime.utcnow().isoformat()`` reads UTC, and the two
|
||||
serialize identically, so nothing downstream can tell which zone a value belongs
|
||||
to. In RDF the consequence is not a parse error but a silent one: under XSD 1.1
|
||||
a value with no timezone compared against one with a timezone is indeterminate
|
||||
whenever they fall inside the +/-14 hour window, SPARQL turns that into an error,
|
||||
and FILTER discards errors as non-matches. A timezone-qualified query therefore
|
||||
returns an answer with every Semantica-written record quietly missing from it.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from semantica.export.json_exporter import JSONExporter
|
||||
from semantica.export.report_generator import ReportGenerator
|
||||
from semantica.export.yaml_exporter import SemanticNetworkYAMLExporter
|
||||
from semantica.utils.helpers import utc_now, utc_now_iso
|
||||
|
||||
KG = {
|
||||
"entities": [{"id": "https://example.org/e1", "text": "Bob"}],
|
||||
"relationships": [],
|
||||
}
|
||||
|
||||
|
||||
def assert_offset_aware(value):
|
||||
"""An ISO 8601 string is only an instant if it says which zone it is in."""
|
||||
assert isinstance(value, str), value
|
||||
parsed = datetime.fromisoformat(value)
|
||||
assert parsed.tzinfo is not None, f"timezone-naive timestamp: {value!r}"
|
||||
assert parsed.utcoffset() is not None
|
||||
|
||||
|
||||
def test_utc_now_iso_is_offset_aware():
|
||||
assert_offset_aware(utc_now_iso())
|
||||
assert utc_now().tzinfo is not None
|
||||
|
||||
|
||||
def test_jsonld_export_timestamp_is_offset_aware():
|
||||
document = JSONExporter()._convert_kg_to_jsonld(KG)
|
||||
assert_offset_aware(document["semantica:exportedAt"])
|
||||
|
||||
|
||||
def test_json_export_metadata_timestamp_is_offset_aware(tmp_path):
|
||||
import json
|
||||
|
||||
exporter = JSONExporter()
|
||||
exporter.export_entities(KG["entities"], tmp_path / "entities.json")
|
||||
exporter.export_relationships([], tmp_path / "relationships.json")
|
||||
|
||||
for name in ("entities.json", "relationships.json"):
|
||||
payload = json.loads((tmp_path / name).read_text())
|
||||
assert_offset_aware(payload["metadata"]["exported_at"])
|
||||
|
||||
|
||||
def test_yaml_export_timestamp_is_offset_aware():
|
||||
yaml = pytest.importorskip("yaml")
|
||||
|
||||
document = SemanticNetworkYAMLExporter().export_entities(KG["entities"])
|
||||
payload = yaml.safe_load(document)
|
||||
assert_offset_aware(payload["metadata"]["exported_at"])
|
||||
|
||||
|
||||
def test_report_timestamp_is_offset_aware():
|
||||
import json
|
||||
|
||||
report = json.loads(
|
||||
ReportGenerator().generate_quality_report({"score": 0.9}, format="json")
|
||||
)
|
||||
assert_offset_aware(report["generated_at"])
|
||||
|
||||
|
||||
def test_exported_timestamp_compares_against_a_timezone_aware_instant():
|
||||
"""The naive form raised TypeError here, or compared as if it were UTC."""
|
||||
exported = datetime.fromisoformat(
|
||||
JSONExporter()._convert_kg_to_jsonld(KG)["semantica:exportedAt"]
|
||||
)
|
||||
assert exported <= utc_now()
|
||||
assert exported > datetime(2020, 1, 1, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def test_exported_timestamp_survives_a_timezone_qualified_sparql_filter():
|
||||
"""The regression in #1114: a strict engine dropped the naive value."""
|
||||
pyoxigraph = pytest.importorskip("pyoxigraph")
|
||||
|
||||
exported = JSONExporter()._convert_kg_to_jsonld(KG)["semantica:exportedAt"]
|
||||
store = pyoxigraph.Store()
|
||||
store.load(
|
||||
(
|
||||
'<https://example.org/export> '
|
||||
'<https://semantica.dev/ns#exportedAt> '
|
||||
f'"{exported}"^^<http://www.w3.org/2001/XMLSchema#dateTime> .'
|
||||
).encode(),
|
||||
format=pyoxigraph.RdfFormat.N_TRIPLES,
|
||||
)
|
||||
# The bound has to sit inside the +/-14 hour window that makes an
|
||||
# untimezoned comparison indeterminate. A bound years away is determinate
|
||||
# even for a naive value, and the test would pass without the fix.
|
||||
bound = (utc_now() + timedelta(hours=1)).isoformat().replace("+00:00", "Z")
|
||||
rows = list(store.query(
|
||||
"PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> "
|
||||
"SELECT ?e WHERE { ?e <https://semantica.dev/ns#exportedAt> ?t . "
|
||||
f'FILTER (?t < "{bound}"^^xsd:dateTime) }}'
|
||||
))
|
||||
assert len(rows) == 1, "the export was dropped by a timezone-qualified filter"
|
||||
|
||||
|
||||
def test_document_iri_carrying_an_offset_is_a_valid_iri():
|
||||
"""The offset puts '+' and ':' in the @id; both are legal in a path."""
|
||||
rdflib = pytest.importorskip("rdflib")
|
||||
|
||||
document_iri = JSONExporter()._convert_kg_to_jsonld(KG)["@id"]
|
||||
assert "+00:00" in document_iri
|
||||
assert rdflib.term._is_valid_uri(document_iri)
|
||||
|
||||
graph = rdflib.Graph()
|
||||
graph.add((
|
||||
rdflib.URIRef(document_iri),
|
||||
rdflib.RDF.type,
|
||||
rdflib.URIRef("https://semantica.dev/ns#KnowledgeGraph"),
|
||||
))
|
||||
reparsed = rdflib.Graph().parse(data=graph.serialize(format="nt"), format="nt")
|
||||
assert document_iri in {str(s) for s in reparsed.subjects()}
|
||||
|
||||
|
||||
def test_vocabulary_range_matches_what_the_exporter_writes():
|
||||
"""The declared range says the offset is required; the export must carry it."""
|
||||
rdflib = pytest.importorskip("rdflib")
|
||||
from rdflib.namespace import RDFS, XSD
|
||||
|
||||
from semantica.ontology.vocabulary import NAMESPACE, vocabulary_turtle
|
||||
|
||||
graph = rdflib.Graph()
|
||||
graph.parse(data=vocabulary_turtle(), format="turtle")
|
||||
declared = graph.value(rdflib.URIRef(f"{NAMESPACE}exportedAt"), RDFS.range)
|
||||
assert declared == XSD.dateTimeStamp
|
||||
|
||||
exported = JSONExporter()._convert_kg_to_jsonld(KG)["semantica:exportedAt"]
|
||||
assert datetime.fromisoformat(exported).utcoffset() is not None
|
||||
@@ -5,6 +5,7 @@ from unittest.mock import MagicMock, patch
|
||||
|
||||
from semantica.ingest import FileIngestor, WebIngestor, DBIngestor, StreamIngestor, FeedIngestor
|
||||
from semantica.kg import GraphBuilder, EntityResolver, ProvenanceTracker
|
||||
from semantica.provenance import InMemoryStorage, ProvenanceManager
|
||||
from semantica.conflicts import ConflictDetector
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
@@ -75,13 +76,30 @@ class TestNotebook06MultiSourceIntegration:
|
||||
for entity in all_entities:
|
||||
provenance_tracker.track_entity(entity.get("id"), entity.get("source"), entity)
|
||||
|
||||
# Endpoints stay on "source"/"target", which is what GraphBuilder's
|
||||
# dict normalization expects; the originating document moves to
|
||||
# "document". The literal previously set "source" twice, so the
|
||||
# endpoint id was silently overwritten by the document name.
|
||||
relationships = [
|
||||
{"source": "e2", "target": "e1", "type": "CEO_of", "source": "file1"}
|
||||
{"id": "r1", "source": "e2", "target": "e1",
|
||||
"type": "CEO_of", "document": "file1"}
|
||||
]
|
||||
|
||||
with patch.object(provenance_tracker, 'track_relationship'):
|
||||
for rel in relationships:
|
||||
provenance_tracker.track_relationship(rel.get("source"), rel.get("target"), rel.get("source"), rel)
|
||||
|
||||
# kg.ProvenanceTracker has no track_relationship and never did; that
|
||||
# lives on ProvenanceManager, which is where ProvenanceTracker's own
|
||||
# DeprecationWarning points callers. Called for real rather than
|
||||
# patched, so this step actually exercises something.
|
||||
#
|
||||
# Storage is pinned to in-memory: with no argument, ProvenanceManager
|
||||
# falls back to the mutable class-level _default_storage_path, so an
|
||||
# earlier test setting it would make this write SQLite to disk and
|
||||
# turn the result order-dependent.
|
||||
provenance_manager = ProvenanceManager(storage=InMemoryStorage())
|
||||
for rel in relationships:
|
||||
entry = provenance_manager.track_relationship(
|
||||
rel["id"], rel["document"], metadata=rel
|
||||
)
|
||||
assert entry is not None
|
||||
|
||||
# --- Step 5: Build Unified KG ---
|
||||
builder = GraphBuilder()
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
|
||||
import pytest
|
||||
from semantica.utils.types import Entity
|
||||
from semantica.kg.graph_builder import GraphBuilder
|
||||
from semantica.kg.entity_resolver import EntityResolver
|
||||
from semantica.kg.graph_analyzer import GraphAnalyzer
|
||||
from semantica.utils.entity_ids import get_entity_id
|
||||
from semantica.utils.types import Entity
|
||||
|
||||
def test_full_entity_pipeline():
|
||||
"""
|
||||
@@ -107,6 +108,60 @@ def test_direct_entity_objects_in_analyzer():
|
||||
|
||||
print("Direct Entity objects test passed successfully!")
|
||||
|
||||
|
||||
def test_entity_id_only_merge_remaps_relationship_endpoints():
|
||||
"""Entity aliases must survive merging and relationship remapping."""
|
||||
builder = GraphBuilder(
|
||||
merge_entities=True,
|
||||
entity_resolution_strategy="exact",
|
||||
resolve_conflicts=False,
|
||||
)
|
||||
|
||||
graph = builder.build(
|
||||
{
|
||||
"entities": [
|
||||
{"entity_id": "alice:1", "name": "Alice", "type": "Person"},
|
||||
{"entity_id": "alice:2", "name": " Alice ", "type": "Person"},
|
||||
{"entity_id": "org:1", "name": "Acme", "type": "Organization"},
|
||||
],
|
||||
"relationships": [
|
||||
{
|
||||
"source_id": "alice:2",
|
||||
"target_id": "org:1",
|
||||
"type": "WORKS_FOR",
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
merged_alice = next(
|
||||
entity for entity in graph["entities"] if entity["name"] == "Alice"
|
||||
)
|
||||
relationship = graph["relationships"][0]
|
||||
entity_ids = {
|
||||
entity.get("id") or entity.get("entity_id") for entity in graph["entities"]
|
||||
}
|
||||
|
||||
assert merged_alice["id"] == "alice:1"
|
||||
assert set(merged_alice["merged_from"]) == {"alice:1", "alice:2"}
|
||||
assert {
|
||||
item["id"] for item in merged_alice["metadata"]["provenance"]["merged_from"]
|
||||
} == {"alice:1", "alice:2"}
|
||||
assert relationship["source"] == "alice:1"
|
||||
assert relationship["target"] == "org:1"
|
||||
assert {relationship["source"], relationship["target"]} <= entity_ids
|
||||
|
||||
|
||||
def test_entity_id_helper_ignores_falsy_identifiers():
|
||||
"""ID extraction must match the KG pipeline's falsy-ID contract."""
|
||||
assert get_entity_id({"id": "", "entity_id": "alias:1"}) == "alias:1"
|
||||
assert get_entity_id({"id": 0, "entity_id": "alias:2"}) == "alias:2"
|
||||
assert (
|
||||
get_entity_id({"id": "primary:1", "entity_id": "alias:3"})
|
||||
== "primary:1"
|
||||
)
|
||||
assert get_entity_id({"id": "", "entity_id": 0}) is None
|
||||
|
||||
if __name__ == "__main__":
|
||||
test_full_entity_pipeline()
|
||||
test_direct_entity_objects_in_analyzer()
|
||||
|
||||
@@ -414,6 +414,103 @@ class TestSHACLHierarchicalAndValidation(unittest.TestCase):
|
||||
self.assertIsNotNone(v.explanation)
|
||||
self.assertIn("https://example.com/john", v.explanation)
|
||||
|
||||
# 32b
|
||||
def test_explain_violations_uses_real_constraint_values(self):
|
||||
"""explain_violations must render the real min/max/datatype/class values,
|
||||
not hardcoded placeholders (regression for PR #318)."""
|
||||
from semantica.ontology.ontology_validator import (
|
||||
SHACLValidationReport,
|
||||
SHACLViolation,
|
||||
)
|
||||
|
||||
max_v = SHACLViolation(
|
||||
focus_node="https://example.com/john",
|
||||
result_path="ex:age",
|
||||
constraint="MaxCountConstraintComponent",
|
||||
max_count=3,
|
||||
)
|
||||
dt_v = SHACLViolation(
|
||||
focus_node="https://example.com/john",
|
||||
result_path="ex:age",
|
||||
constraint="DatatypeConstraintComponent",
|
||||
value="abc",
|
||||
datatype="http://www.w3.org/2001/XMLSchema#integer",
|
||||
)
|
||||
cls_v = SHACLViolation(
|
||||
focus_node="https://example.com/john",
|
||||
result_path="ex:knows",
|
||||
constraint="ClassConstraintComponent",
|
||||
value="https://example.com/thing",
|
||||
class_="https://example.com/Person",
|
||||
)
|
||||
report = SHACLValidationReport(
|
||||
conforms=False, violations=[max_v, dt_v, cls_v]
|
||||
)
|
||||
report.explain_violations()
|
||||
# MaxCount must show the real limit (3), not the hardcoded 1.
|
||||
self.assertIn("3", max_v.explanation)
|
||||
self.assertNotIn("At most 1 value", max_v.explanation)
|
||||
# Datatype must show the real datatype IRI, not the message.
|
||||
self.assertIn(
|
||||
"http://www.w3.org/2001/XMLSchema#integer", dt_v.explanation
|
||||
)
|
||||
# Class must show the real class IRI.
|
||||
self.assertIn("https://example.com/Person", cls_v.explanation)
|
||||
|
||||
# 32c
|
||||
def test_run_pyshacl_extracts_constraint_values_from_shape(self):
|
||||
"""_run_pyshacl must back-reference sh:sourceShape to populate the real
|
||||
constraint parameters on each violation."""
|
||||
try:
|
||||
import pyshacl # noqa: F401
|
||||
import rdflib # noqa: F401
|
||||
except ImportError:
|
||||
self.skipTest("pyshacl/rdflib not installed")
|
||||
from semantica.ontology.ontology_validator import _run_pyshacl
|
||||
|
||||
shacl = """
|
||||
@prefix sh: <http://www.w3.org/ns/shacl#> .
|
||||
@prefix ex: <http://example.com/> .
|
||||
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
|
||||
|
||||
ex:PersonShape a sh:NodeShape ;
|
||||
sh:targetClass ex:Person ;
|
||||
sh:property [
|
||||
sh:path ex:age ;
|
||||
sh:datatype xsd:integer ;
|
||||
sh:maxCount 2 ;
|
||||
] .
|
||||
"""
|
||||
data = """
|
||||
@prefix ex: <http://example.com/> .
|
||||
ex:john a ex:Person ;
|
||||
ex:age "not-a-number" ;
|
||||
ex:age 1 ;
|
||||
ex:age 2 ;
|
||||
ex:age 3 .
|
||||
"""
|
||||
report = _run_pyshacl(data, shacl)
|
||||
self.assertFalse(report.conforms)
|
||||
# Datatype violation should carry the real xsd:integer datatype.
|
||||
dt = [
|
||||
v
|
||||
for v in report.violations
|
||||
if v.constraint == "DatatypeConstraintComponent"
|
||||
]
|
||||
self.assertTrue(dt)
|
||||
self.assertTrue(
|
||||
dt[0].datatype.endswith("integer"),
|
||||
f"expected integer datatype, got {dt[0].datatype}",
|
||||
)
|
||||
# MaxCount violation should carry the real max_count == 2.
|
||||
mc = [
|
||||
v
|
||||
for v in report.violations
|
||||
if v.constraint == "MaxCountConstraintComponent"
|
||||
]
|
||||
if mc:
|
||||
self.assertEqual(mc[0].max_count, 2)
|
||||
|
||||
# 33
|
||||
def test_shacl_violation_to_dict(self):
|
||||
from semantica.ontology.ontology_validator import SHACLViolation
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
"""The vocabulary must stay true to what the exporters emit (issue #1107).
|
||||
|
||||
A vocabulary document that drifts from the code is worse than none, because it
|
||||
states that terms mean something while the exporters emit different ones. These
|
||||
tests tie the two together: every term the serializers can write must be
|
||||
declared here, so adding a term to an exporter without declaring it fails the
|
||||
build rather than shipping an undeclared IRI.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
rdflib = pytest.importorskip("rdflib")
|
||||
|
||||
from semantica.export.rdf_exporter import ( # noqa: E402
|
||||
DEFAULT_ENTITY_TYPE,
|
||||
DEFAULT_RELATION_TYPE,
|
||||
SEMANTICA_NS,
|
||||
)
|
||||
from semantica.ontology.vocabulary import ( # noqa: E402
|
||||
NAMESPACE,
|
||||
vocabulary_path,
|
||||
vocabulary_turtle,
|
||||
)
|
||||
|
||||
#: Every term the exporters emit in the Semantica namespace, by local name.
|
||||
#: RDF and OWL-Time paths in export/rdf_exporter.py, document and relationship
|
||||
#: terms in export/json_exporter.py, roles in provenance/manager.py.
|
||||
EMITTED_TERMS = {
|
||||
"Entity",
|
||||
"Relationship",
|
||||
"KnowledgeGraph",
|
||||
"text",
|
||||
"confidence",
|
||||
"metadata",
|
||||
"related_to",
|
||||
"source",
|
||||
"target",
|
||||
"type",
|
||||
"entities",
|
||||
"relationships",
|
||||
"exportedAt",
|
||||
"format",
|
||||
"openEndedInterval",
|
||||
"role_generator",
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def graph():
|
||||
g = rdflib.Graph()
|
||||
g.parse(data=vocabulary_turtle(), format="turtle")
|
||||
return g
|
||||
|
||||
|
||||
def test_vocabulary_ships_with_the_package():
|
||||
assert vocabulary_path().is_file()
|
||||
|
||||
|
||||
def test_vocabulary_parses(graph):
|
||||
assert len(graph) > 0
|
||||
|
||||
|
||||
def test_namespace_matches_the_one_the_exporters_use():
|
||||
assert NAMESPACE == SEMANTICA_NS
|
||||
|
||||
|
||||
def test_every_emitted_term_is_declared(graph):
|
||||
declared = {
|
||||
str(s)[len(NAMESPACE) :]
|
||||
for s in set(graph.subjects())
|
||||
if isinstance(s, rdflib.URIRef) and str(s).startswith(NAMESPACE)
|
||||
}
|
||||
missing = EMITTED_TERMS - declared
|
||||
assert not missing, f"emitted but not declared in the vocabulary: {sorted(missing)}"
|
||||
|
||||
|
||||
def test_the_defaults_the_exporters_fall_back_to_are_declared(graph):
|
||||
for iri in (DEFAULT_ENTITY_TYPE, DEFAULT_RELATION_TYPE):
|
||||
assert (rdflib.URIRef(iri), None, None) in graph, f"{iri} is not declared"
|
||||
|
||||
|
||||
def test_every_declared_term_carries_a_label_and_a_comment(graph):
|
||||
for subject in set(graph.subjects()):
|
||||
if not (isinstance(subject, rdflib.URIRef) and str(subject).startswith(NAMESPACE)):
|
||||
continue
|
||||
assert graph.value(subject, rdflib.RDFS.label), f"{subject} has no rdfs:label"
|
||||
assert graph.value(subject, rdflib.RDFS.comment), f"{subject} has no rdfs:comment"
|
||||
|
||||
|
||||
def test_declared_ranges_do_not_contradict_what_the_exporters_emit(graph):
|
||||
"""A declared range must match the datatype the serializers actually write.
|
||||
|
||||
Caught by review on #1109: sem:confidence was declared xsd:decimal while the
|
||||
N-Triples serializer types the same value xsd:float. A vocabulary that
|
||||
contradicts the code is worse than no vocabulary, so any range declared here
|
||||
has to be one the exporters really emit.
|
||||
"""
|
||||
import re
|
||||
|
||||
from semantica.export.rdf_exporter import RDFExporter
|
||||
|
||||
sample = {
|
||||
"entities": [{"id": "https://example.org/e1", "text": "A",
|
||||
"type": "https://example.org/T", "confidence": 0.5}],
|
||||
"relationships": [],
|
||||
}
|
||||
emitted = RDFExporter().export_to_rdf(sample, format="ntriples")
|
||||
|
||||
for subject, _, range_ in graph.triples((None, rdflib.RDFS.range, None)):
|
||||
if not str(subject).startswith(NAMESPACE):
|
||||
continue
|
||||
local = str(subject)[len(NAMESPACE):]
|
||||
for match in re.finditer(rf'<{NAMESPACE}{local}> "[^"]*"\^\^<([^>]+)>', emitted):
|
||||
assert match.group(1) == str(range_), (
|
||||
f"{local}: vocabulary declares {range_}, N-Triples emits {match.group(1)}"
|
||||
)
|
||||
@@ -0,0 +1,153 @@
|
||||
"""Provenance timestamps must carry a timezone (issue #1114).
|
||||
|
||||
The provenance package stamped every record with ``datetime.utcnow()``, which
|
||||
returns a naive datetime that happens to hold UTC. The exporters stamped theirs
|
||||
with ``datetime.now()``, which returns a naive datetime holding local time. Both
|
||||
serialize identically, so a graph mixing the two cannot be ordered, and the
|
||||
values reach RDF as ``prov:generatedAtTime``/``startedAtTime``/``endedAtTime``
|
||||
typed ``xsd:dateTime``, where a timezone-qualified SPARQL comparison discards
|
||||
them. ``datetime.utcnow()`` is also deprecated as of Python 3.12.
|
||||
"""
|
||||
|
||||
import warnings
|
||||
from datetime import datetime
|
||||
|
||||
import pytest
|
||||
|
||||
from semantica.provenance.manager import ProvenanceManager
|
||||
from semantica.provenance.schemas import ProvenanceEntry
|
||||
from semantica.utils.helpers import utc_now
|
||||
|
||||
|
||||
def assert_offset_aware(value):
|
||||
parsed = datetime.fromisoformat(value)
|
||||
assert parsed.tzinfo is not None, f"timezone-naive timestamp: {value!r}"
|
||||
|
||||
|
||||
def test_provenance_entry_default_timestamp_is_offset_aware():
|
||||
entry = ProvenanceEntry(entity_id="e1", entity_type="Doc", activity_id="act1")
|
||||
assert_offset_aware(entry.timestamp)
|
||||
assert datetime.fromisoformat(entry.timestamp) <= utc_now()
|
||||
|
||||
|
||||
def test_creating_an_entry_raises_no_deprecation_warning():
|
||||
"""datetime.utcnow() is deprecated and scheduled for removal."""
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("error", DeprecationWarning)
|
||||
ProvenanceEntry(entity_id="e1", entity_type="Doc", activity_id="act1")
|
||||
|
||||
|
||||
def test_tracked_entity_timestamps_are_offset_aware():
|
||||
manager = ProvenanceManager()
|
||||
manager.track_entity("e1", source="doc.pdf")
|
||||
|
||||
entry = manager.storage.retrieve_all()[0]
|
||||
assert_offset_aware(entry.timestamp)
|
||||
for field in ("first_seen", "last_updated"):
|
||||
value = getattr(entry, field, None)
|
||||
if value:
|
||||
assert_offset_aware(value)
|
||||
|
||||
|
||||
def test_prov_o_export_timestamps_are_offset_aware():
|
||||
"""The values land in RDF typed xsd:dateTime, so the offset is the contract."""
|
||||
rdflib = pytest.importorskip("rdflib")
|
||||
from rdflib.namespace import XSD
|
||||
|
||||
manager = ProvenanceManager()
|
||||
manager.track_entity("e_parent", source="doc.pdf")
|
||||
manager.track_entity(
|
||||
"e_child", source="doc.pdf", parent_entity_id="e_parent",
|
||||
used_entities=["e_parent"], activity_id="act_transform",
|
||||
)
|
||||
|
||||
graph = rdflib.Graph()
|
||||
graph.parse(data=manager.export_prov(format="turtle"), format="turtle")
|
||||
|
||||
stamps = [o for o in graph.objects()
|
||||
if isinstance(o, rdflib.Literal) and o.datatype == XSD.dateTime]
|
||||
assert stamps, "no xsd:dateTime literals in the PROV-O export"
|
||||
for stamp in stamps:
|
||||
assert_offset_aware(str(stamp))
|
||||
|
||||
|
||||
def test_prov_o_timestamps_are_valid_datetimestamp():
|
||||
"""xsd:dateTimeStamp requires an explicit timezone; these now qualify."""
|
||||
rdflib = pytest.importorskip("rdflib")
|
||||
from rdflib.namespace import XSD
|
||||
|
||||
manager = ProvenanceManager()
|
||||
manager.track_entity("e1", source="doc.pdf")
|
||||
graph = rdflib.Graph()
|
||||
graph.parse(data=manager.export_prov(format="turtle"), format="turtle")
|
||||
|
||||
for stamp in [o for o in graph.objects()
|
||||
if isinstance(o, rdflib.Literal) and o.datatype == XSD.dateTime]:
|
||||
assert rdflib.Literal(str(stamp), datatype=XSD.dateTime).ill_typed is False
|
||||
assert datetime.fromisoformat(str(stamp)).utcoffset() is not None
|
||||
|
||||
|
||||
class TestRangeQueriesCompareInstants:
|
||||
"""Range APIs compared ISO strings, so they ordered by spelling (#1121 review).
|
||||
|
||||
Once new entries carry ``+00:00`` and stored ones do not, a raw string
|
||||
comparison puts an inclusive naive bound *below* the offset-bearing
|
||||
timestamp it names, dropping the record, and a bound written in another
|
||||
offset lands wherever its digits fall rather than at its instant.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _manager_with(timestamps):
|
||||
manager = ProvenanceManager()
|
||||
for index, stamp in enumerate(timestamps):
|
||||
manager.storage.store(ProvenanceEntry(
|
||||
entity_id=f"e{index}", entity_type="Doc",
|
||||
activity_id="act", timestamp=stamp,
|
||||
))
|
||||
return manager
|
||||
|
||||
def test_inclusive_bound_written_without_an_offset_still_matches(self):
|
||||
manager = self._manager_with(["2026-08-19T14:19:04.229937+00:00"])
|
||||
|
||||
found = manager.query_recorded_between(
|
||||
"2026-08-19T00:00:00", "2026-08-19T14:19:04.229937"
|
||||
)
|
||||
assert [e["entity_id"] for e in found] == ["e0"]
|
||||
|
||||
def test_bound_in_another_offset_selects_by_instant(self):
|
||||
"""19:45+05:30 is 14:15Z: before the entry, though its digits are after."""
|
||||
manager = self._manager_with(["2026-08-19T14:19:04+00:00"])
|
||||
|
||||
assert manager.query_recorded_between(
|
||||
"2026-08-19T00:00:00Z", "2026-08-19T19:45:00+05:30"
|
||||
) == []
|
||||
assert len(manager.query_recorded_between(
|
||||
"2026-08-19T00:00:00Z", "2026-08-19T19:50:00+05:30"
|
||||
)) == 1
|
||||
|
||||
def test_legacy_and_offset_bearing_entries_are_both_found_and_ordered(self):
|
||||
manager = self._manager_with([
|
||||
"2026-08-19T14:19:05+00:00", # written after #1114
|
||||
"2026-08-19T14:19:04", # written before it, meaning UTC
|
||||
])
|
||||
|
||||
found = manager.query_recorded_between(
|
||||
"2026-08-19T14:00:00Z", "2026-08-19T15:00:00Z"
|
||||
)
|
||||
assert [e["entity_id"] for e in found] == ["e1", "e0"]
|
||||
|
||||
def test_audit_log_since_reads_a_naive_bound_as_utc(self):
|
||||
manager = self._manager_with([
|
||||
"2026-08-19T14:19:05+00:00",
|
||||
"2026-08-19T09:00:00",
|
||||
])
|
||||
|
||||
recent = manager.audit_log(since="2026-08-19T14:19:05", format="json")
|
||||
assert [e["entity_id"] for e in recent] == ["e0"]
|
||||
|
||||
def test_an_unreadable_bound_falls_back_to_the_previous_behaviour(self):
|
||||
"""A call that used to work with a non-timestamp bound must not raise."""
|
||||
manager = self._manager_with(["2026-08-19T14:19:04+00:00"])
|
||||
|
||||
assert manager.query_recorded_between("not-a-date", "also-not") == []
|
||||
assert manager.audit_log(since="not-a-date", format="json") == []
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Shared helper for visualization tests.
|
||||
|
||||
The visualization modules treat Plotly as optional: they bind ``px``, ``go`` and
|
||||
``make_subplots`` to ``None`` when the import fails, and raise ``ProcessingError``
|
||||
from ``_check_dependencies()``. Tests that exercise a Plotly-backed path need
|
||||
those names to be usable, otherwise ``patch("...go.Figure")`` fails on ``None``
|
||||
and the visualizers refuse to run.
|
||||
|
||||
``plotly_doubles`` fills in a double for each alias that is ``None``, so the
|
||||
tests describe their own requirements instead of depending on whether Plotly
|
||||
happens to be installed. When Plotly is installed the aliases are left alone and
|
||||
the patches keep asserting against the real attribute names.
|
||||
"""
|
||||
|
||||
from contextlib import ExitStack, contextmanager
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
PLOTLY_ALIASES = ("px", "go", "make_subplots")
|
||||
|
||||
|
||||
@contextmanager
|
||||
def plotly_doubles(*modules):
|
||||
"""Stand in for the module-level Plotly aliases that are unavailable."""
|
||||
with ExitStack() as stack:
|
||||
for module in modules:
|
||||
for alias in PLOTLY_ALIASES:
|
||||
if getattr(module, alias, "unused") is None:
|
||||
stack.enter_context(patch.object(module, alias, MagicMock()))
|
||||
yield
|
||||
@@ -9,27 +9,16 @@ and must raise a clear ProcessingError for anything else.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import sys
|
||||
import unittest
|
||||
from dataclasses import dataclass, field
|
||||
from typing import List
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stub out heavy optional deps before importing the module under test
|
||||
# ---------------------------------------------------------------------------
|
||||
sys.modules.setdefault("matplotlib", MagicMock())
|
||||
sys.modules.setdefault("matplotlib.pyplot", MagicMock())
|
||||
sys.modules.setdefault("matplotlib.patches", MagicMock())
|
||||
sys.modules.setdefault("plotly", MagicMock())
|
||||
sys.modules.setdefault("plotly.express", MagicMock())
|
||||
sys.modules.setdefault("plotly.graph_objects", MagicMock())
|
||||
sys.modules.setdefault("plotly.subplots", MagicMock())
|
||||
sys.modules.setdefault("graphviz", MagicMock())
|
||||
sys.modules.setdefault("seaborn", MagicMock())
|
||||
|
||||
from semantica.utils.exceptions import ProcessingError # noqa: E402
|
||||
from semantica.visualization import kg_visualizer # noqa: E402
|
||||
from semantica.visualization.kg_visualizer import KGVisualizer # noqa: E402
|
||||
from tests.visualization._plotly_doubles import plotly_doubles # noqa: E402
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Minimal fixtures
|
||||
@@ -191,10 +180,6 @@ class TestVisualizeNetworkAcceptsKGObject(unittest.TestCase):
|
||||
def _run_visualize_network(self, graph_arg):
|
||||
"""Run visualize_network with all Plotly internals mocked."""
|
||||
mock_fig = MagicMock()
|
||||
mock_go = sys.modules["plotly.graph_objects"]
|
||||
mock_go.Figure.return_value = mock_fig
|
||||
mock_go.Scatter.return_value = MagicMock()
|
||||
mock_go.Layout.return_value = MagicMock()
|
||||
|
||||
viz = _make_viz()
|
||||
|
||||
@@ -207,6 +192,10 @@ class TestVisualizeNetworkAcceptsKGObject(unittest.TestCase):
|
||||
|
||||
# ColorPalette helpers
|
||||
with (
|
||||
plotly_doubles(kg_visualizer),
|
||||
patch("semantica.visualization.kg_visualizer.go.Figure", return_value=mock_fig),
|
||||
patch("semantica.visualization.kg_visualizer.go.Scatter"),
|
||||
patch("semantica.visualization.kg_visualizer.go.Layout"),
|
||||
patch(
|
||||
"semantica.visualization.kg_visualizer.ColorPalette.get_entity_type_colors",
|
||||
return_value={"Person": "#ff0000"},
|
||||
@@ -254,9 +243,12 @@ class TestAllVisualizeMethodsAcceptKGObject(unittest.TestCase):
|
||||
self.viz._normalize_graph = MagicMock(return_value=GRAPH_DICT)
|
||||
self.viz._visualize_network_plotly = MagicMock(return_value=MagicMock())
|
||||
communities = {"node_assignments": {"e1": 0, "e2": 1}, "num_communities": 2}
|
||||
with patch(
|
||||
"semantica.visualization.kg_visualizer.ColorPalette.get_community_colors",
|
||||
return_value=["#ff0000", "#00ff00"],
|
||||
with (
|
||||
plotly_doubles(kg_visualizer),
|
||||
patch(
|
||||
"semantica.visualization.kg_visualizer.ColorPalette.get_community_colors",
|
||||
return_value=["#ff0000", "#00ff00"],
|
||||
),
|
||||
):
|
||||
self.viz.visualize_communities(self.kg, communities=communities)
|
||||
self.viz._normalize_graph.assert_called_once_with(self.kg)
|
||||
@@ -264,22 +256,24 @@ class TestAllVisualizeMethodsAcceptKGObject(unittest.TestCase):
|
||||
def test_visualize_centrality_accepts_kg_object(self):
|
||||
self.viz._normalize_graph = MagicMock(return_value=GRAPH_DICT)
|
||||
self.viz._visualize_network_plotly = MagicMock(return_value=MagicMock())
|
||||
self.viz.visualize_centrality(self.kg, centrality={"centrality": {}})
|
||||
with plotly_doubles(kg_visualizer):
|
||||
self.viz.visualize_centrality(self.kg, centrality={"centrality": {}})
|
||||
self.viz._normalize_graph.assert_called_once_with(self.kg)
|
||||
|
||||
def test_visualize_entity_types_accepts_kg_object(self):
|
||||
self.viz._normalize_graph = MagicMock(return_value=GRAPH_DICT)
|
||||
mock_px = sys.modules["plotly.express"]
|
||||
mock_px.bar.return_value = MagicMock()
|
||||
self.viz.visualize_entity_types(self.kg)
|
||||
with plotly_doubles(kg_visualizer), patch("semantica.visualization.kg_visualizer.px.bar"):
|
||||
self.viz.visualize_entity_types(self.kg)
|
||||
self.viz._normalize_graph.assert_called_once_with(self.kg)
|
||||
|
||||
def test_visualize_relationship_matrix_accepts_kg_object(self):
|
||||
self.viz._normalize_graph = MagicMock(return_value=GRAPH_DICT)
|
||||
mock_go = sys.modules["plotly.graph_objects"]
|
||||
mock_go.Figure.return_value = MagicMock()
|
||||
mock_go.Heatmap.return_value = MagicMock()
|
||||
self.viz.visualize_relationship_matrix(self.kg)
|
||||
with (
|
||||
plotly_doubles(kg_visualizer),
|
||||
patch("semantica.visualization.kg_visualizer.go.Figure"),
|
||||
patch("semantica.visualization.kg_visualizer.go.Heatmap"),
|
||||
):
|
||||
self.viz.visualize_relationship_matrix(self.kg)
|
||||
self.viz._normalize_graph.assert_called_once_with(self.kg)
|
||||
|
||||
|
||||
@@ -361,10 +355,6 @@ class TestFormalKnowledgeGraphType(unittest.TestCase):
|
||||
|
||||
def _run_visualize_network(self, graph_arg):
|
||||
mock_fig = MagicMock()
|
||||
mock_go = sys.modules["plotly.graph_objects"]
|
||||
mock_go.Figure.return_value = mock_fig
|
||||
mock_go.Scatter.return_value = MagicMock()
|
||||
mock_go.Layout.return_value = MagicMock()
|
||||
viz = _make_viz()
|
||||
fake_pos = {"e1": (0.0, 0.0), "e2": (1.0, 1.0)}
|
||||
viz.force_layout = MagicMock()
|
||||
@@ -372,6 +362,10 @@ class TestFormalKnowledgeGraphType(unittest.TestCase):
|
||||
viz.hierarchical_layout = MagicMock()
|
||||
viz.circular_layout = MagicMock()
|
||||
with (
|
||||
plotly_doubles(kg_visualizer),
|
||||
patch("semantica.visualization.kg_visualizer.go.Figure", return_value=mock_fig),
|
||||
patch("semantica.visualization.kg_visualizer.go.Scatter"),
|
||||
patch("semantica.visualization.kg_visualizer.go.Layout"),
|
||||
patch(
|
||||
"semantica.visualization.kg_visualizer.ColorPalette.get_entity_type_colors",
|
||||
return_value={"Person": "#ff0000"},
|
||||
@@ -392,9 +386,12 @@ class TestFormalKnowledgeGraphType(unittest.TestCase):
|
||||
viz._normalize_graph = MagicMock(return_value=GRAPH_DICT)
|
||||
viz._visualize_network_plotly = MagicMock(return_value=MagicMock())
|
||||
communities = {"node_assignments": {"e1": 0, "e2": 1}, "num_communities": 2}
|
||||
with patch(
|
||||
"semantica.visualization.kg_visualizer.ColorPalette.get_community_colors",
|
||||
return_value=["#ff0000", "#00ff00"],
|
||||
with (
|
||||
plotly_doubles(kg_visualizer),
|
||||
patch(
|
||||
"semantica.visualization.kg_visualizer.ColorPalette.get_community_colors",
|
||||
return_value=["#ff0000", "#00ff00"],
|
||||
),
|
||||
):
|
||||
viz.visualize_communities(kg, communities=communities)
|
||||
viz._normalize_graph.assert_called_once_with(kg)
|
||||
@@ -404,24 +401,28 @@ class TestFormalKnowledgeGraphType(unittest.TestCase):
|
||||
viz = _make_viz()
|
||||
viz._normalize_graph = MagicMock(return_value=GRAPH_DICT)
|
||||
viz._visualize_network_plotly = MagicMock(return_value=MagicMock())
|
||||
viz.visualize_centrality(kg, centrality={"centrality": {}})
|
||||
with plotly_doubles(kg_visualizer):
|
||||
viz.visualize_centrality(kg, centrality={"centrality": {}})
|
||||
viz._normalize_graph.assert_called_once_with(kg)
|
||||
|
||||
def test_visualize_entity_types_accepts_knowledge_graph(self):
|
||||
kg = self._make_kg()
|
||||
viz = _make_viz()
|
||||
viz._normalize_graph = MagicMock(return_value=GRAPH_DICT)
|
||||
sys.modules["plotly.express"].bar.return_value = MagicMock()
|
||||
viz.visualize_entity_types(kg)
|
||||
with plotly_doubles(kg_visualizer), patch("semantica.visualization.kg_visualizer.px.bar"):
|
||||
viz.visualize_entity_types(kg)
|
||||
viz._normalize_graph.assert_called_once_with(kg)
|
||||
|
||||
def test_visualize_relationship_matrix_accepts_knowledge_graph(self):
|
||||
kg = self._make_kg()
|
||||
viz = _make_viz()
|
||||
viz._normalize_graph = MagicMock(return_value=GRAPH_DICT)
|
||||
sys.modules["plotly.graph_objects"].Figure.return_value = MagicMock()
|
||||
sys.modules["plotly.graph_objects"].Heatmap.return_value = MagicMock()
|
||||
viz.visualize_relationship_matrix(kg)
|
||||
with (
|
||||
plotly_doubles(kg_visualizer),
|
||||
patch("semantica.visualization.kg_visualizer.go.Figure"),
|
||||
patch("semantica.visualization.kg_visualizer.go.Heatmap"),
|
||||
):
|
||||
viz.visualize_relationship_matrix(kg)
|
||||
viz._normalize_graph.assert_called_once_with(kg)
|
||||
|
||||
def test_knowledge_graph_importable_from_kg_module(self):
|
||||
|
||||
@@ -1,149 +1,115 @@
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from contextlib import contextmanager
|
||||
from unittest.mock import patch
|
||||
|
||||
import numpy as np
|
||||
|
||||
# Helper to mock modules
|
||||
def mock_module(name):
|
||||
m = MagicMock()
|
||||
sys.modules[name] = m
|
||||
return m
|
||||
from tests.visualization._plotly_doubles import plotly_doubles
|
||||
|
||||
|
||||
@contextmanager
|
||||
def import_without(module_name, *dependencies):
|
||||
"""Import a module with selected optional dependencies unavailable."""
|
||||
package_name, attribute = module_name.rsplit(".", 1)
|
||||
package = importlib.import_module(package_name)
|
||||
missing = object()
|
||||
original_module = sys.modules.pop(module_name, missing)
|
||||
original_attribute = getattr(package, attribute, missing)
|
||||
|
||||
try:
|
||||
with patch.dict(sys.modules, {name: None for name in dependencies}):
|
||||
yield importlib.import_module(module_name)
|
||||
finally:
|
||||
sys.modules.pop(module_name, None)
|
||||
if original_module is not missing:
|
||||
sys.modules[module_name] = original_module
|
||||
if original_attribute is missing:
|
||||
package.__dict__.pop(attribute, None)
|
||||
else:
|
||||
setattr(package, attribute, original_attribute)
|
||||
|
||||
|
||||
class TestOptionalDependencies(unittest.TestCase):
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
# Mock heavy/problematic dependencies globally to prevent environment crashes
|
||||
# We use a dict to save original modules if they exist, but for this test file
|
||||
# we generally want to run in a controlled "clean" environment.
|
||||
cls.modules_to_patch = [
|
||||
'sklearn', 'sklearn.decomposition', 'sklearn.manifold',
|
||||
'scipy', 'scipy.optimize',
|
||||
'matplotlib', 'matplotlib.pyplot', 'matplotlib.patches',
|
||||
'plotly', 'plotly.express', 'plotly.graph_objects', 'plotly.subplots',
|
||||
'networkx', 'seaborn'
|
||||
]
|
||||
|
||||
cls.original_modules = {}
|
||||
for mod in cls.modules_to_patch:
|
||||
if mod in sys.modules:
|
||||
cls.original_modules[mod] = sys.modules[mod]
|
||||
sys.modules[mod] = MagicMock()
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
# Restore original modules
|
||||
for mod in cls.modules_to_patch:
|
||||
if mod in cls.original_modules:
|
||||
sys.modules[mod] = cls.original_modules[mod]
|
||||
else:
|
||||
del sys.modules[mod]
|
||||
|
||||
def setUp(self):
|
||||
# Clear cached visualization modules to ensure fresh imports
|
||||
self.viz_modules = [
|
||||
'semantica.visualization.embedding_visualizer',
|
||||
'semantica.visualization.ontology_visualizer',
|
||||
'semantica.visualization.kg_visualizer',
|
||||
'semantica.visualization.utils.export_formats'
|
||||
]
|
||||
for mod in self.viz_modules:
|
||||
if mod in sys.modules:
|
||||
del sys.modules[mod]
|
||||
|
||||
def test_embedding_visualizer_without_umap(self):
|
||||
"""Test EmbeddingVisualizer behavior when umap is missing."""
|
||||
# Ensure umap is missing
|
||||
with patch.dict(sys.modules, {'umap': None}):
|
||||
from semantica.visualization.embedding_visualizer import EmbeddingVisualizer
|
||||
|
||||
# Setup PCA mock to verify fallback
|
||||
mock_pca_class = sys.modules['sklearn.decomposition'].PCA
|
||||
mock_pca_instance = mock_pca_class.return_value
|
||||
# Configure fit_transform to return correct shape (n_samples, 2)
|
||||
mock_pca_instance.fit_transform.return_value = np.zeros((4, 2))
|
||||
|
||||
viz = EmbeddingVisualizer()
|
||||
# Use numpy array!
|
||||
embeddings = np.array([[0, 1, 2], [1, 0, 3], [0, 0, 0], [1, 1, 1]])
|
||||
|
||||
# Should fallback to PCA when method="umap" is used but umap is None
|
||||
# The code logs a warning and uses PCA
|
||||
viz.visualize_2d_projection(embeddings, method="umap")
|
||||
|
||||
# Verify PCA was called
|
||||
with import_without(
|
||||
"semantica.visualization.embedding_visualizer", "umap"
|
||||
) as module:
|
||||
with plotly_doubles(module), patch.object(module, "PCA") as mock_pca_class:
|
||||
mock_pca_class.return_value.fit_transform.return_value = np.zeros((4, 2))
|
||||
|
||||
viz = module.EmbeddingVisualizer()
|
||||
embeddings = np.array([[0, 1, 2], [1, 0, 3], [0, 0, 0], [1, 1, 1]])
|
||||
viz.visualize_2d_projection(embeddings, method="umap")
|
||||
|
||||
mock_pca_class.assert_called()
|
||||
|
||||
def test_ontology_visualizer_without_graphviz(self):
|
||||
"""Test OntologyVisualizer behavior when graphviz is missing."""
|
||||
# Ensure graphviz is missing
|
||||
with patch.dict(sys.modules, {'graphviz': None}):
|
||||
from semantica.visualization.ontology_visualizer import OntologyVisualizer, ProcessingError
|
||||
|
||||
viz = OntologyVisualizer()
|
||||
with import_without(
|
||||
"semantica.visualization.ontology_visualizer", "graphviz"
|
||||
) as module:
|
||||
viz = module.OntologyVisualizer()
|
||||
ontology = {
|
||||
"classes": [
|
||||
{"name": "A", "label": "A"},
|
||||
{"name": "B", "label": "B", "parent": "A"}
|
||||
{"name": "B", "label": "B", "parent": "A"},
|
||||
]
|
||||
}
|
||||
|
||||
with self.assertRaises(ProcessingError) as cm:
|
||||
|
||||
with self.assertRaises(module.ProcessingError) as cm:
|
||||
viz.visualize_hierarchy(ontology, output="dot", file_path="test.dot")
|
||||
|
||||
|
||||
self.assertIn("Graphviz is required for DOT export", str(cm.exception))
|
||||
|
||||
def test_analytics_visualizer_without_plotly(self):
|
||||
"""Test AnalyticsVisualizer behavior when plotly is missing."""
|
||||
with patch.dict(sys.modules, {'plotly': None, 'plotly.express': None, 'plotly.graph_objects': None}):
|
||||
from semantica.visualization.analytics_visualizer import AnalyticsVisualizer, ProcessingError
|
||||
|
||||
# Need to ensure numpy is available for init (it's imported at top level)
|
||||
# But we are testing plotly missing.
|
||||
|
||||
viz = AnalyticsVisualizer()
|
||||
|
||||
with self.assertRaises(ProcessingError) as cm:
|
||||
viz.visualize_centrality_rankings({"node1": 1.0})
|
||||
|
||||
self.assertIn("Plotly is required", str(cm.exception))
|
||||
with import_without(
|
||||
"semantica.visualization.analytics_visualizer",
|
||||
"plotly",
|
||||
"plotly.express",
|
||||
"plotly.graph_objects",
|
||||
) as module:
|
||||
viz = module.AnalyticsVisualizer()
|
||||
|
||||
def test_analytics_visualizer_without_plotly(self):
|
||||
"""Test AnalyticsVisualizer behavior when plotly is missing."""
|
||||
with patch.dict(sys.modules, {'plotly': None, 'plotly.express': None, 'plotly.graph_objects': None}):
|
||||
from semantica.visualization.analytics_visualizer import AnalyticsVisualizer, ProcessingError
|
||||
|
||||
viz = AnalyticsVisualizer()
|
||||
|
||||
with self.assertRaises(ProcessingError) as cm:
|
||||
with self.assertRaises(module.ProcessingError) as cm:
|
||||
viz.visualize_centrality_rankings({})
|
||||
|
||||
|
||||
self.assertIn("Plotly is required", str(cm.exception))
|
||||
|
||||
def test_semantic_network_visualizer_without_plotly(self):
|
||||
"""Test SemanticNetworkVisualizer behavior when plotly is missing."""
|
||||
with patch.dict(sys.modules, {'plotly': None, 'plotly.express': None, 'plotly.graph_objects': None}):
|
||||
from semantica.visualization.semantic_network_visualizer import SemanticNetworkVisualizer, ProcessingError
|
||||
|
||||
viz = SemanticNetworkVisualizer()
|
||||
|
||||
with self.assertRaises(ProcessingError) as cm:
|
||||
with import_without(
|
||||
"semantica.visualization.semantic_network_visualizer",
|
||||
"plotly",
|
||||
"plotly.express",
|
||||
"plotly.graph_objects",
|
||||
) as module:
|
||||
viz = module.SemanticNetworkVisualizer()
|
||||
|
||||
with self.assertRaises(module.ProcessingError) as cm:
|
||||
viz.visualize_network({})
|
||||
|
||||
|
||||
self.assertIn("Plotly is required", str(cm.exception))
|
||||
|
||||
def test_temporal_visualizer_without_plotly(self):
|
||||
"""Test TemporalVisualizer behavior when plotly is missing."""
|
||||
with patch.dict(sys.modules, {'plotly': None, 'plotly.express': None, 'plotly.graph_objects': None}):
|
||||
from semantica.visualization.temporal_visualizer import TemporalVisualizer, ProcessingError
|
||||
|
||||
viz = TemporalVisualizer()
|
||||
|
||||
with self.assertRaises(ProcessingError) as cm:
|
||||
with import_without(
|
||||
"semantica.visualization.temporal_visualizer",
|
||||
"plotly",
|
||||
"plotly.express",
|
||||
"plotly.graph_objects",
|
||||
) as module:
|
||||
viz = module.TemporalVisualizer()
|
||||
|
||||
with self.assertRaises(module.ProcessingError) as cm:
|
||||
viz.visualize_timeline({"events": []})
|
||||
|
||||
|
||||
self.assertIn("Plotly is required", str(cm.exception))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -1,27 +1,5 @@
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch, ANY
|
||||
import sys
|
||||
import types
|
||||
|
||||
# Helper to create a mock package
|
||||
def mock_package(name):
|
||||
m = MagicMock()
|
||||
m.__path__ = []
|
||||
sys.modules[name] = m
|
||||
return m
|
||||
|
||||
# Mock libraries before importing module under test
|
||||
# We need to ensure matplotlib behaves like a package for seaborn
|
||||
sys.modules['matplotlib'] = MagicMock()
|
||||
sys.modules['matplotlib.colors'] = MagicMock()
|
||||
sys.modules['matplotlib.pyplot'] = MagicMock()
|
||||
sys.modules['matplotlib.patches'] = MagicMock()
|
||||
sys.modules['plotly'] = MagicMock()
|
||||
sys.modules['plotly.express'] = MagicMock()
|
||||
sys.modules['plotly.graph_objects'] = MagicMock()
|
||||
sys.modules['plotly.subplots'] = MagicMock()
|
||||
sys.modules['graphviz'] = MagicMock()
|
||||
sys.modules['seaborn'] = MagicMock()
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from semantica.visualization.kg_visualizer import KGVisualizer
|
||||
from semantica.visualization.ontology_visualizer import OntologyVisualizer
|
||||
|
||||
@@ -1,34 +1,25 @@
|
||||
|
||||
import unittest
|
||||
from contextlib import ExitStack
|
||||
from unittest.mock import MagicMock, patch
|
||||
import sys
|
||||
import numpy as np
|
||||
|
||||
# Mock heavy libraries before importing visualization modules
|
||||
sys.modules['matplotlib'] = MagicMock()
|
||||
sys.modules['matplotlib.pyplot'] = MagicMock()
|
||||
sys.modules['matplotlib.colors'] = MagicMock()
|
||||
sys.modules['matplotlib.patches'] = MagicMock()
|
||||
sys.modules['plotly'] = MagicMock()
|
||||
sys.modules['plotly.express'] = MagicMock()
|
||||
sys.modules['plotly.graph_objects'] = MagicMock()
|
||||
sys.modules['plotly.subplots'] = MagicMock()
|
||||
sys.modules['seaborn'] = MagicMock()
|
||||
sys.modules['umap'] = MagicMock()
|
||||
sys.modules['sklearn'] = MagicMock()
|
||||
sys.modules['sklearn.decomposition'] = MagicMock()
|
||||
sys.modules['sklearn.manifold'] = MagicMock()
|
||||
|
||||
from semantica.visualization import analytics_visualizer, embedding_visualizer
|
||||
from semantica.visualization.analytics_visualizer import AnalyticsVisualizer
|
||||
from semantica.visualization.embedding_visualizer import EmbeddingVisualizer
|
||||
from semantica.visualization.utils.color_schemes import ColorScheme
|
||||
from tests.visualization._plotly_doubles import plotly_doubles
|
||||
|
||||
class TestVisualizationAdvanced(unittest.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
self.mock_logger = MagicMock()
|
||||
self.mock_tracker = MagicMock()
|
||||
|
||||
|
||||
stack = ExitStack()
|
||||
self.addCleanup(stack.close)
|
||||
stack.enter_context(plotly_doubles(analytics_visualizer, embedding_visualizer))
|
||||
|
||||
self.patchers = [
|
||||
patch('semantica.visualization.analytics_visualizer.get_logger', return_value=self.mock_logger),
|
||||
patch('semantica.visualization.analytics_visualizer.get_progress_tracker', return_value=self.mock_tracker),
|
||||
@@ -53,22 +44,17 @@ class TestVisualizationAdvanced(unittest.TestCase):
|
||||
viz = AnalyticsVisualizer()
|
||||
centrality = {"n1": 0.5, "n2": 0.3}
|
||||
|
||||
# Access the mock that was injected
|
||||
import plotly.graph_objects as go
|
||||
# Reset mock to ensure clean state
|
||||
go.Bar.reset_mock()
|
||||
|
||||
viz.visualize_centrality_rankings(centrality, output="interactive")
|
||||
go.Bar.assert_called()
|
||||
with (
|
||||
patch('semantica.visualization.analytics_visualizer.go.Bar') as mock_bar,
|
||||
patch('semantica.visualization.analytics_visualizer.go.Figure'),
|
||||
):
|
||||
viz.visualize_centrality_rankings(centrality, output="interactive")
|
||||
mock_bar.assert_called()
|
||||
|
||||
def test_visualize_community_structure(self):
|
||||
viz = AnalyticsVisualizer()
|
||||
|
||||
if hasattr(viz, 'visualize_community_structure'):
|
||||
import plotly.graph_objects as go
|
||||
# Reset mocks
|
||||
go.Figure.reset_mock()
|
||||
|
||||
graph = MagicMock()
|
||||
communities = {"c1": ["n1", "n2"]}
|
||||
|
||||
@@ -89,8 +75,6 @@ class TestVisualizationAdvanced(unittest.TestCase):
|
||||
viz = EmbeddingVisualizer()
|
||||
embeddings = np.random.rand(10, 128)
|
||||
|
||||
import plotly.graph_objects as go
|
||||
|
||||
# Mock UMAP/TSNE/PCA
|
||||
with patch('semantica.visualization.embedding_visualizer.umap') as mock_umap, \
|
||||
patch('semantica.visualization.embedding_visualizer.TSNE') as mock_tsne, \
|
||||
@@ -116,12 +100,13 @@ class TestVisualizationAdvanced(unittest.TestCase):
|
||||
viz = EmbeddingVisualizer()
|
||||
embeddings = np.random.rand(5, 5)
|
||||
|
||||
import plotly.graph_objects as go
|
||||
go.Heatmap.reset_mock()
|
||||
|
||||
if hasattr(viz, 'visualize_similarity_heatmap'):
|
||||
viz.visualize_similarity_heatmap(embeddings)
|
||||
go.Heatmap.assert_called()
|
||||
with (
|
||||
patch('semantica.visualization.embedding_visualizer.go.Heatmap') as mock_heatmap,
|
||||
patch('semantica.visualization.embedding_visualizer.go.Figure'),
|
||||
):
|
||||
viz.visualize_similarity_heatmap(embeddings)
|
||||
mock_heatmap.assert_called()
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
||||
@@ -1,26 +1,9 @@
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch
|
||||
import sys
|
||||
import numpy as np
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
# Mock heavy libraries before importing visualization modules
|
||||
sys.modules['matplotlib'] = MagicMock()
|
||||
sys.modules['matplotlib.pyplot'] = MagicMock()
|
||||
sys.modules['matplotlib.colors'] = MagicMock()
|
||||
sys.modules['matplotlib.patches'] = MagicMock()
|
||||
sys.modules['plotly'] = MagicMock()
|
||||
sys.modules['plotly.express'] = MagicMock()
|
||||
sys.modules['plotly.graph_objects'] = MagicMock()
|
||||
sys.modules['plotly.subplots'] = MagicMock()
|
||||
sys.modules['seaborn'] = MagicMock()
|
||||
sys.modules['umap'] = MagicMock()
|
||||
sys.modules['sklearn'] = MagicMock()
|
||||
sys.modules['sklearn.decomposition'] = MagicMock()
|
||||
sys.modules['sklearn.manifold'] = MagicMock()
|
||||
sys.modules['networkx'] = MagicMock()
|
||||
sys.modules['graphviz'] = MagicMock()
|
||||
|
||||
# Import visualizers
|
||||
from semantica.visualization.kg_visualizer import KGVisualizer
|
||||
@@ -52,24 +35,24 @@ class TestVisualizationComprehensive(unittest.TestCase):
|
||||
patch('semantica.visualization.analytics_visualizer.get_progress_tracker', return_value=self.mock_tracker),
|
||||
patch('semantica.visualization.temporal_visualizer.get_logger', return_value=self.mock_logger),
|
||||
patch('semantica.visualization.temporal_visualizer.get_progress_tracker', return_value=self.mock_tracker),
|
||||
# Mock Layouts
|
||||
patch('semantica.visualization.kg_visualizer.ForceDirectedLayout', MagicMock()),
|
||||
patch('semantica.visualization.kg_visualizer.HierarchicalLayout', MagicMock()),
|
||||
patch('semantica.visualization.kg_visualizer.CircularLayout', MagicMock()),
|
||||
patch('semantica.visualization.ontology_visualizer.HierarchicalLayout', MagicMock()),
|
||||
patch('semantica.visualization.semantic_network_visualizer.ForceDirectedLayout', MagicMock()),
|
||||
patch('semantica.visualization.kg_visualizer.go', MagicMock()),
|
||||
patch('semantica.visualization.kg_visualizer.px', MagicMock()),
|
||||
patch('semantica.visualization.ontology_visualizer.go', MagicMock()),
|
||||
patch('semantica.visualization.ontology_visualizer.make_subplots', MagicMock()),
|
||||
patch('semantica.visualization.embedding_visualizer.go', MagicMock()),
|
||||
patch('semantica.visualization.embedding_visualizer.px', MagicMock()),
|
||||
patch('semantica.visualization.semantic_network_visualizer.go', MagicMock()),
|
||||
patch('semantica.visualization.semantic_network_visualizer.px', MagicMock()),
|
||||
patch('semantica.visualization.analytics_visualizer.go', MagicMock()),
|
||||
patch('semantica.visualization.analytics_visualizer.px', MagicMock()),
|
||||
patch('semantica.visualization.analytics_visualizer.make_subplots', MagicMock()),
|
||||
patch('semantica.visualization.temporal_visualizer.go', MagicMock()),
|
||||
patch('semantica.visualization.temporal_visualizer.px', MagicMock()),
|
||||
]
|
||||
|
||||
for p in self.patchers:
|
||||
p.start()
|
||||
|
||||
# Reset plotly mocks
|
||||
import plotly.graph_objects as go
|
||||
import plotly.express as px
|
||||
go.Figure.reset_mock()
|
||||
px.bar.reset_mock()
|
||||
px.scatter.reset_mock()
|
||||
|
||||
def tearDown(self):
|
||||
for p in self.patchers:
|
||||
p.stop()
|
||||
@@ -210,8 +193,8 @@ class TestVisualizationComprehensive(unittest.TestCase):
|
||||
embeddings = np.random.rand(10, 10)
|
||||
|
||||
# Test visualize_2d_projection (mock UMAP/PCA)
|
||||
with patch('semantica.visualization.embedding_visualizer.umap.UMAP') as MockUMAP:
|
||||
MockUMAP.return_value.fit_transform.return_value = np.random.rand(10, 2)
|
||||
with patch('semantica.visualization.embedding_visualizer.umap', MagicMock()) as mock_umap:
|
||||
mock_umap.UMAP.return_value.fit_transform.return_value = np.random.rand(10, 2)
|
||||
viz.visualize_2d_projection(embeddings)
|
||||
|
||||
# Test visualize_similarity_heatmap
|
||||
@@ -219,8 +202,8 @@ class TestVisualizationComprehensive(unittest.TestCase):
|
||||
|
||||
# Test visualize_clustering
|
||||
clusters = [0, 1, 0, 1, 0, 1, 0, 1, 0, 1]
|
||||
with patch('semantica.visualization.embedding_visualizer.umap.UMAP') as MockUMAP:
|
||||
MockUMAP.return_value.fit_transform.return_value = np.random.rand(10, 2)
|
||||
with patch('semantica.visualization.embedding_visualizer.umap', MagicMock()) as mock_umap:
|
||||
mock_umap.UMAP.return_value.fit_transform.return_value = np.random.rand(10, 2)
|
||||
viz.visualize_clustering(embeddings, clusters)
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
||||
Reference in New Issue
Block a user