From 2d75952476f2e489838e3dc4a5fa88b690ee5c43 Mon Sep 17 00:00:00 2001 From: KaifAhmad1 Date: Wed, 19 Aug 2026 19:09:02 +0530 Subject: [PATCH] fix: close remaining review gaps in vocabulary/deterministic-IRI PR serialize_to_rdfxml still defaulted entity_type to the bare string "semantica:Entity" written into an rdf:resource attribute, which isn't namespace-expanded the way a Turtle angle-bracket or XML element name is - the same #1101 failure mode, just on the path the original tests didn't cover. Now uses the full-IRI DEFAULT_ENTITY_TYPE like the Turtle path. json_exporter.py emits semantica:format and @type: "semantica:KnowledgeGraph", neither of which was declared in the vocabulary or included in EMITTED_TERMS, so the "undeclared terms fail the build" guarantee didn't actually cover them. Both are now declared with rdfs:label/comment and added to the guard set. MANIFEST.in didn't mirror the pyproject.toml package-data addition, so a source-distribution install could ship without the vocabulary file. The cross-process minting-stability test replaced the subprocess's entire environment with a POSIX-only PATH, breaking it on Windows and any host needing other inherited env vars; now overrides only PYTHONHASHSEED on top of the inherited environment. Also folds mint_entity_iri/mint_relationship_iri's hand-rolled hashlib.sha256(...).hexdigest() into the existing hash_data() helper this file already imports alongside. 229 export and ontology tests pass, including a new regression test for the RDF/XML default-type fix. Co-Authored-By: fabio-rovai --- CHANGELOG.md | 9 +++++++++ MANIFEST.in | 1 + semantica/export/rdf_exporter.py | 9 ++++----- .../ontology/vocabulary/semantica-ns.ttl | 15 +++++++++++++++ tests/export/test_rdf_exporter_iri_minting.py | 19 ++++++++++++++++++- tests/ontology/test_vocabulary.py | 2 ++ 6 files changed, 49 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bbd7cfb2..2c81bc4a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Semantica RDF vocabulary, and deterministic entity/relationship IRIs** (#1109, closes #1107, closes #1101) by @fabio-rovai, reviewed by @KaifAhmad1 + - Every RDF/JSON-LD export mints terms in `https://semantica.dev/ns#`, and until now nothing declared what those terms meant — the namespace 404s and no vocabulary shipped with the package, so a consumer receiving an export had no way to tell `sem:text` from a typo of it, and no closed-world checker could validate an export at all + - `semantica/ontology/vocabulary/semantica-ns.ttl` declares the terms the exporters actually emit — drawn from the emitting call sites in `export/rdf_exporter.py`, `export/json_exporter.py` and `provenance/manager.py`, not from what a vocabulary "ought" to contain. Ships inside the package (`from semantica.ontology.vocabulary import vocabulary_turtle`) so it loads without a network round trip, and is the same document intended to be served at the namespace IRI once hosting/content-negotiation is sorted + - `tests/ontology/test_vocabulary.py` ties the document to the code: every term a serializer can write must be declared, so adding a term to an exporter without declaring it fails the build + - The missing-id fallback minted entity/relationship IRIs from Python's builtin `hash()`, randomised per process (`PYTHONHASHSEED`), so the same entity got a different IRI on every run and exports couldn't be diffed, deduplicated, or joined to an earlier provenance record. It also wrote ``, an IRI in the scheme `semantica` rather than the expansion of the declared prefix, so those nodes never joined with anything written through it. Minting now uses SHA-256 and writes a full IRI in the declared namespace; the same fix applies to the default entity/relationship types in the Turtle path + - **Fixed during review** (Qodo): the temporal fallback minted from `source_id` only, while the main serializer accepts `source_id` or `source` — relationships using the second form hashed two empty strings, which the previous randomised `hash()` masked by making the IRI unstable anyway; once deterministic, unrelated relationships at the same list index collided on one IRI across exports. Endpoints are now resolved the same way `serialize_to_turtle` resolves them, before minting. `sem:confidence` also lost its declared `xsd:decimal` range: the N-Triples serializer types the same value `xsd:float`, and the two are disjoint, so declaring either contradicted one of the exporters (tracked in #1100) — a new `test_declared_ranges_do_not_contradict_what_the_exporters_emit` guards the whole class of that mistake + - **Fixed in follow-up**: `serialize_to_rdfxml`'s default entity type still wrote the bare string `"semantica:Entity"` into an `rdf:resource` attribute, which (unlike a Turtle angle-bracket or an XML element name) is not namespace-expanded — the exact #1101 failure mode, just on the untested RDF/XML path. `json_exporter.py`'s `semantica:format` and `@type: "semantica:KnowledgeGraph"` were emitted but absent from both the vocabulary and the test's `EMITTED_TERMS` guard set, so the "undeclared terms fail the build" claim didn't actually cover them — both are now declared and guarded. `MANIFEST.in` didn't mirror the `pyproject.toml` package-data addition, so a source-distribution install could omit the vocabulary file. The cross-process minting-stability test replaced the subprocess's entire environment with a POSIX-only `PATH`, breaking it on Windows; now overrides only `PYTHONHASHSEED` on top of the inherited environment + - 229 export and ontology tests pass + - **First-class CrewAI integration** (#962) - New `pip install semantica[crewai]` extra (`crewai>=0.80.0`) — crewai core provides `BaseTool`/`BaseKnowledgeSource`, so `crewai-tools` is intentionally not included, and the extra is intentionally **not** part of the `all` bundle: crewai hard-requires `chromadb~=1.1.0`, which is affected by the unpatched pre-auth code-injection CVE-2026-45829 (see `integrations/crewai/README.md`) - `integrations/crewai/SemanticaKGTool` — a CrewAI `BaseTool` exposing 5 KG actions (`extract_entities`, `extract_relations`, `add_to_graph`, `query_graph`, `find_related`) backed by `NERExtractor` / `RelationExtractor` / `ContextGraph`; supports both sync `run()` and async `arun()` diff --git a/MANIFEST.in b/MANIFEST.in index aa726d6b..7d60a31a 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -1 +1,2 @@ recursive-include semantica/static * +recursive-include semantica/ontology/vocabulary *.ttl diff --git a/semantica/export/rdf_exporter.py b/semantica/export/rdf_exporter.py index 0d8dcdee..63eac4fe 100644 --- a/semantica/export/rdf_exporter.py +++ b/semantica/export/rdf_exporter.py @@ -30,11 +30,10 @@ License: MIT """ from pathlib import Path -import hashlib from typing import Any, Dict, List, Optional, Set, Union from ..utils.exceptions import ProcessingError, ValidationError -from ..utils.helpers import ensure_directory +from ..utils.helpers import ensure_directory, hash_data from ..utils.logging import get_logger from ..utils.progress_tracker import get_progress_tracker @@ -60,13 +59,13 @@ def mint_entity_iri(text: str) -> str: provenance record written by an earlier process. SHA-256 is stable across runs and machines, which is what an identifier has to be. """ - digest = hashlib.sha256(str(text).encode("utf-8")).hexdigest()[:16] + digest = hash_data(str(text))[:16] return f"{SEMANTICA_NS}entity_{digest}" def mint_relationship_iri(index: int, source: Any, target: Any) -> str: """Mint a stable IRI for a relationship that arrived without an id.""" - digest = hashlib.sha256(f"{source}\x00{target}".encode("utf-8")).hexdigest()[:16] + digest = hash_data(f"{source}\x00{target}")[:16] return f"{SEMANTICA_NS}rel_{index}_{digest}" @@ -526,7 +525,7 @@ class RDFSerializer: entity_text = entity.get("text", "") entity_id = mint_entity_iri(entity_text) - entity_type = entity.get("type", "semantica:Entity") + entity_type = entity.get("type", DEFAULT_ENTITY_TYPE) text = entity.get("text") or entity.get("label", "") confidence = entity.get("confidence", 1.0) diff --git a/semantica/ontology/vocabulary/semantica-ns.ttl b/semantica/ontology/vocabulary/semantica-ns.ttl index 19943eac..c6843a32 100644 --- a/semantica/ontology/vocabulary/semantica-ns.ttl +++ b/semantica/ontology/vocabulary/semantica-ns.ttl @@ -33,6 +33,13 @@ where a relationship carries sem:type, sem:source and sem:target rather than being written as a single triple.""" ; rdfs:isDefinedBy . +sem:KnowledgeGraph a owl:Class ; + rdfs:label "Knowledge Graph" ; + rdfs:comment """The document-level type of a JSON-LD export: the @type of +the top-level node carrying sem:entities, sem:relationships and +sem:exportedAt. Emitted by _convert_kg_to_jsonld in export/json_exporter.py.""" ; + rdfs:isDefinedBy . + # ── Properties on an entity ────────────────────────────────────────────────── sem:text a owl:DatatypeProperty ; @@ -117,6 +124,14 @@ values carry no timezone offset.""" ; rdfs:range xsd:dateTime ; rdfs:isDefinedBy . +sem:format a owl:DatatypeProperty ; + rdfs:label "format" ; + rdfs:comment """The serialization format label written on a JSON-LD +document (currently always the literal "json-ld"). Emitted by +JSONExporter.export_to_jsonld in export/json_exporter.py.""" ; + rdfs:range xsd:string ; + rdfs:isDefinedBy . + # ── Temporal term (OWL-Time export) ────────────────────────────────────────── sem:openEndedInterval a owl:DatatypeProperty ; diff --git a/tests/export/test_rdf_exporter_iri_minting.py b/tests/export/test_rdf_exporter_iri_minting.py index fc56b74e..3650dec2 100644 --- a/tests/export/test_rdf_exporter_iri_minting.py +++ b/tests/export/test_rdf_exporter_iri_minting.py @@ -11,6 +11,7 @@ IRI in the scheme ``semantica`` rather than the expansion of the declared ``semantica:`` prefix, so it never joined with anything written through it. """ +import os import subprocess import sys @@ -45,7 +46,7 @@ def test_minted_entity_iri_is_stable_across_processes(): capture_output=True, text=True, check=True, - env={"PYTHONHASHSEED": seed, "PATH": "/usr/bin:/bin"}, + env={**os.environ, "PYTHONHASHSEED": seed}, ).stdout.strip() for seed in ("0", "1", "random") } @@ -88,6 +89,22 @@ def test_default_types_are_written_as_full_iris_in_turtle(): assert "" not in turtle +def test_default_entity_type_is_a_full_iri_in_rdfxml(): + """RDF/XML's rdf:resource is an attribute value, not a QName context, so a + + prefixed default there (``semantica:Entity``) resolves to the scheme + ``semantica`` rather than the declared namespace — the same failure mode + fixed for Turtle in #1101, missed here because the original tests only + checked Turtle output. + """ + untyped = {"entities": [{"id": "https://example.org/e1", "text": "A"}], + "relationships": []} + rdfxml = RDFExporter().export_to_rdf(untyped, format="rdfxml") + + assert f'rdf:resource="{DEFAULT_ENTITY_TYPE}"' in rdfxml + assert 'rdf:resource="semantica:Entity"' not in rdfxml + + def test_temporal_minting_uses_either_endpoint_representation(): """Relationships may carry source/target or source_id/target_id (#1109 review). diff --git a/tests/ontology/test_vocabulary.py b/tests/ontology/test_vocabulary.py index e59d0d77..34eb4e5d 100644 --- a/tests/ontology/test_vocabulary.py +++ b/tests/ontology/test_vocabulary.py @@ -28,6 +28,7 @@ from semantica.ontology.vocabulary import ( # noqa: E402 EMITTED_TERMS = { "Entity", "Relationship", + "KnowledgeGraph", "text", "confidence", "metadata", @@ -38,6 +39,7 @@ EMITTED_TERMS = { "entities", "relationships", "exportedAt", + "format", "openEndedInterval", "role_generator", }