fix: close remaining review gaps in vocabulary/deterministic-IRI PR

serialize_to_rdfxml still defaulted entity_type to the bare string
"semantica:Entity" written into an rdf:resource attribute, which isn't
namespace-expanded the way a Turtle angle-bracket or XML element name is -
the same #1101 failure mode, just on the path the original tests didn't
cover. Now uses the full-IRI DEFAULT_ENTITY_TYPE like the Turtle path.

json_exporter.py emits semantica:format and @type: "semantica:KnowledgeGraph",
neither of which was declared in the vocabulary or included in
EMITTED_TERMS, so the "undeclared terms fail the build" guarantee didn't
actually cover them. Both are now declared with rdfs:label/comment and
added to the guard set.

MANIFEST.in didn't mirror the pyproject.toml package-data addition, so a
source-distribution install could ship without the vocabulary file.

The cross-process minting-stability test replaced the subprocess's entire
environment with a POSIX-only PATH, breaking it on Windows and any host
needing other inherited env vars; now overrides only PYTHONHASHSEED on top
of the inherited environment.

Also folds mint_entity_iri/mint_relationship_iri's hand-rolled
hashlib.sha256(...).hexdigest() into the existing hash_data() helper this
file already imports alongside.

229 export and ontology tests pass, including a new regression test for
the RDF/XML default-type fix.

Co-Authored-By: fabio-rovai <fabio@thetesseractacademy.com>
This commit is contained in:
KaifAhmad1
2026-08-19 19:09:02 +05:30
co-authored by fabio-rovai
parent e55c03bd39
commit 2d75952476
6 changed files with 49 additions and 6 deletions
+9
View File
@@ -11,6 +11,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added ### Added
- **Semantica RDF vocabulary, and deterministic entity/relationship IRIs** (#1109, closes #1107, closes #1101) by @fabio-rovai, reviewed by @KaifAhmad1
- Every RDF/JSON-LD export mints terms in `https://semantica.dev/ns#`, and until now nothing declared what those terms meant — the namespace 404s and no vocabulary shipped with the package, so a consumer receiving an export had no way to tell `sem:text` from a typo of it, and no closed-world checker could validate an export at all
- `semantica/ontology/vocabulary/semantica-ns.ttl` declares the terms the exporters actually emit — drawn from the emitting call sites in `export/rdf_exporter.py`, `export/json_exporter.py` and `provenance/manager.py`, not from what a vocabulary "ought" to contain. Ships inside the package (`from semantica.ontology.vocabulary import vocabulary_turtle`) so it loads without a network round trip, and is the same document intended to be served at the namespace IRI once hosting/content-negotiation is sorted
- `tests/ontology/test_vocabulary.py` ties the document to the code: every term a serializer can write must be declared, so adding a term to an exporter without declaring it fails the build
- The missing-id fallback minted entity/relationship IRIs from Python's builtin `hash()`, randomised per process (`PYTHONHASHSEED`), so the same entity got a different IRI on every run and exports couldn't be diffed, deduplicated, or joined to an earlier provenance record. It also wrote `<semantica:entity_N>`, an IRI in the scheme `semantica` rather than the expansion of the declared prefix, so those nodes never joined with anything written through it. Minting now uses SHA-256 and writes a full IRI in the declared namespace; the same fix applies to the default entity/relationship types in the Turtle path
- **Fixed during review** (Qodo): the temporal fallback minted from `source_id` only, while the main serializer accepts `source_id` or `source` — relationships using the second form hashed two empty strings, which the previous randomised `hash()` masked by making the IRI unstable anyway; once deterministic, unrelated relationships at the same list index collided on one IRI across exports. Endpoints are now resolved the same way `serialize_to_turtle` resolves them, before minting. `sem:confidence` also lost its declared `xsd:decimal` range: the N-Triples serializer types the same value `xsd:float`, and the two are disjoint, so declaring either contradicted one of the exporters (tracked in #1100) — a new `test_declared_ranges_do_not_contradict_what_the_exporters_emit` guards the whole class of that mistake
- **Fixed in follow-up**: `serialize_to_rdfxml`'s default entity type still wrote the bare string `"semantica:Entity"` into an `rdf:resource` attribute, which (unlike a Turtle angle-bracket or an XML element name) is not namespace-expanded — the exact #1101 failure mode, just on the untested RDF/XML path. `json_exporter.py`'s `semantica:format` and `@type: "semantica:KnowledgeGraph"` were emitted but absent from both the vocabulary and the test's `EMITTED_TERMS` guard set, so the "undeclared terms fail the build" claim didn't actually cover them — both are now declared and guarded. `MANIFEST.in` didn't mirror the `pyproject.toml` package-data addition, so a source-distribution install could omit the vocabulary file. The cross-process minting-stability test replaced the subprocess's entire environment with a POSIX-only `PATH`, breaking it on Windows; now overrides only `PYTHONHASHSEED` on top of the inherited environment
- 229 export and ontology tests pass
- **First-class CrewAI integration** (#962) - **First-class CrewAI integration** (#962)
- New `pip install semantica[crewai]` extra (`crewai>=0.80.0`) — crewai core provides `BaseTool`/`BaseKnowledgeSource`, so `crewai-tools` is intentionally not included, and the extra is intentionally **not** part of the `all` bundle: crewai hard-requires `chromadb~=1.1.0`, which is affected by the unpatched pre-auth code-injection CVE-2026-45829 (see `integrations/crewai/README.md`) - New `pip install semantica[crewai]` extra (`crewai>=0.80.0`) — crewai core provides `BaseTool`/`BaseKnowledgeSource`, so `crewai-tools` is intentionally not included, and the extra is intentionally **not** part of the `all` bundle: crewai hard-requires `chromadb~=1.1.0`, which is affected by the unpatched pre-auth code-injection CVE-2026-45829 (see `integrations/crewai/README.md`)
- `integrations/crewai/SemanticaKGTool` — a CrewAI `BaseTool` exposing 5 KG actions (`extract_entities`, `extract_relations`, `add_to_graph`, `query_graph`, `find_related`) backed by `NERExtractor` / `RelationExtractor` / `ContextGraph`; supports both sync `run()` and async `arun()` - `integrations/crewai/SemanticaKGTool` — a CrewAI `BaseTool` exposing 5 KG actions (`extract_entities`, `extract_relations`, `add_to_graph`, `query_graph`, `find_related`) backed by `NERExtractor` / `RelationExtractor` / `ContextGraph`; supports both sync `run()` and async `arun()`
+1
View File
@@ -1 +1,2 @@
recursive-include semantica/static * recursive-include semantica/static *
recursive-include semantica/ontology/vocabulary *.ttl
+4 -5
View File
@@ -30,11 +30,10 @@ License: MIT
""" """
from pathlib import Path from pathlib import Path
import hashlib
from typing import Any, Dict, List, Optional, Set, Union from typing import Any, Dict, List, Optional, Set, Union
from ..utils.exceptions import ProcessingError, ValidationError from ..utils.exceptions import ProcessingError, ValidationError
from ..utils.helpers import ensure_directory from ..utils.helpers import ensure_directory, hash_data
from ..utils.logging import get_logger from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker from ..utils.progress_tracker import get_progress_tracker
@@ -60,13 +59,13 @@ def mint_entity_iri(text: str) -> str:
provenance record written by an earlier process. SHA-256 is stable across provenance record written by an earlier process. SHA-256 is stable across
runs and machines, which is what an identifier has to be. runs and machines, which is what an identifier has to be.
""" """
digest = hashlib.sha256(str(text).encode("utf-8")).hexdigest()[:16] digest = hash_data(str(text))[:16]
return f"{SEMANTICA_NS}entity_{digest}" return f"{SEMANTICA_NS}entity_{digest}"
def mint_relationship_iri(index: int, source: Any, target: Any) -> str: def mint_relationship_iri(index: int, source: Any, target: Any) -> str:
"""Mint a stable IRI for a relationship that arrived without an id.""" """Mint a stable IRI for a relationship that arrived without an id."""
digest = hashlib.sha256(f"{source}\x00{target}".encode("utf-8")).hexdigest()[:16] digest = hash_data(f"{source}\x00{target}")[:16]
return f"{SEMANTICA_NS}rel_{index}_{digest}" return f"{SEMANTICA_NS}rel_{index}_{digest}"
@@ -526,7 +525,7 @@ class RDFSerializer:
entity_text = entity.get("text", "") entity_text = entity.get("text", "")
entity_id = mint_entity_iri(entity_text) entity_id = mint_entity_iri(entity_text)
entity_type = entity.get("type", "semantica:Entity") entity_type = entity.get("type", DEFAULT_ENTITY_TYPE)
text = entity.get("text") or entity.get("label", "") text = entity.get("text") or entity.get("label", "")
confidence = entity.get("confidence", 1.0) confidence = entity.get("confidence", 1.0)
@@ -33,6 +33,13 @@ where a relationship carries sem:type, sem:source and sem:target rather than
being written as a single triple.""" ; being written as a single triple.""" ;
rdfs:isDefinedBy <https://semantica.dev/ns> . rdfs:isDefinedBy <https://semantica.dev/ns> .
sem:KnowledgeGraph a owl:Class ;
rdfs:label "Knowledge Graph" ;
rdfs:comment """The document-level type of a JSON-LD export: the @type of
the top-level node carrying sem:entities, sem:relationships and
sem:exportedAt. Emitted by _convert_kg_to_jsonld in export/json_exporter.py.""" ;
rdfs:isDefinedBy <https://semantica.dev/ns> .
# ── Properties on an entity ────────────────────────────────────────────────── # ── Properties on an entity ──────────────────────────────────────────────────
sem:text a owl:DatatypeProperty ; sem:text a owl:DatatypeProperty ;
@@ -117,6 +124,14 @@ values carry no timezone offset.""" ;
rdfs:range xsd:dateTime ; rdfs:range xsd:dateTime ;
rdfs:isDefinedBy <https://semantica.dev/ns> . rdfs:isDefinedBy <https://semantica.dev/ns> .
sem:format a owl:DatatypeProperty ;
rdfs:label "format" ;
rdfs:comment """The serialization format label written on a JSON-LD
document (currently always the literal "json-ld"). Emitted by
JSONExporter.export_to_jsonld in export/json_exporter.py.""" ;
rdfs:range xsd:string ;
rdfs:isDefinedBy <https://semantica.dev/ns> .
# ── Temporal term (OWL-Time export) ────────────────────────────────────────── # ── Temporal term (OWL-Time export) ──────────────────────────────────────────
sem:openEndedInterval a owl:DatatypeProperty ; sem:openEndedInterval a owl:DatatypeProperty ;
+18 -1
View File
@@ -11,6 +11,7 @@ IRI in the scheme ``semantica`` rather than the expansion of the declared
``semantica:`` prefix, so it never joined with anything written through it. ``semantica:`` prefix, so it never joined with anything written through it.
""" """
import os
import subprocess import subprocess
import sys import sys
@@ -45,7 +46,7 @@ def test_minted_entity_iri_is_stable_across_processes():
capture_output=True, capture_output=True,
text=True, text=True,
check=True, check=True,
env={"PYTHONHASHSEED": seed, "PATH": "/usr/bin:/bin"}, env={**os.environ, "PYTHONHASHSEED": seed},
).stdout.strip() ).stdout.strip()
for seed in ("0", "1", "random") for seed in ("0", "1", "random")
} }
@@ -88,6 +89,22 @@ def test_default_types_are_written_as_full_iris_in_turtle():
assert "<semantica:related_to>" not in turtle assert "<semantica:related_to>" not in turtle
def test_default_entity_type_is_a_full_iri_in_rdfxml():
"""RDF/XML's rdf:resource is an attribute value, not a QName context, so a
prefixed default there (``semantica:Entity``) resolves to the scheme
``semantica`` rather than the declared namespace — the same failure mode
fixed for Turtle in #1101, missed here because the original tests only
checked Turtle output.
"""
untyped = {"entities": [{"id": "https://example.org/e1", "text": "A"}],
"relationships": []}
rdfxml = RDFExporter().export_to_rdf(untyped, format="rdfxml")
assert f'rdf:resource="{DEFAULT_ENTITY_TYPE}"' in rdfxml
assert 'rdf:resource="semantica:Entity"' not in rdfxml
def test_temporal_minting_uses_either_endpoint_representation(): def test_temporal_minting_uses_either_endpoint_representation():
"""Relationships may carry source/target or source_id/target_id (#1109 review). """Relationships may carry source/target or source_id/target_id (#1109 review).
+2
View File
@@ -28,6 +28,7 @@ from semantica.ontology.vocabulary import ( # noqa: E402
EMITTED_TERMS = { EMITTED_TERMS = {
"Entity", "Entity",
"Relationship", "Relationship",
"KnowledgeGraph",
"text", "text",
"confidence", "confidence",
"metadata", "metadata",
@@ -38,6 +39,7 @@ EMITTED_TERMS = {
"entities", "entities",
"relationships", "relationships",
"exportedAt", "exportedAt",
"format",
"openEndedInterval", "openEndedInterval",
"role_generator", "role_generator",
} }