fix: close remaining review gaps in vocabulary/deterministic-IRI PR

serialize_to_rdfxml still defaulted entity_type to the bare string
"semantica:Entity" written into an rdf:resource attribute, which isn't
namespace-expanded the way a Turtle angle-bracket or XML element name is -
the same #1101 failure mode, just on the path the original tests didn't
cover. Now uses the full-IRI DEFAULT_ENTITY_TYPE like the Turtle path.

json_exporter.py emits semantica:format and @type: "semantica:KnowledgeGraph",
neither of which was declared in the vocabulary or included in
EMITTED_TERMS, so the "undeclared terms fail the build" guarantee didn't
actually cover them. Both are now declared with rdfs:label/comment and
added to the guard set.

MANIFEST.in didn't mirror the pyproject.toml package-data addition, so a
source-distribution install could ship without the vocabulary file.

The cross-process minting-stability test replaced the subprocess's entire
environment with a POSIX-only PATH, breaking it on Windows and any host
needing other inherited env vars; now overrides only PYTHONHASHSEED on top
of the inherited environment.

Also folds mint_entity_iri/mint_relationship_iri's hand-rolled
hashlib.sha256(...).hexdigest() into the existing hash_data() helper this
file already imports alongside.

229 export and ontology tests pass, including a new regression test for
the RDF/XML default-type fix.

Co-Authored-By: fabio-rovai <fabio@thetesseractacademy.com>
This commit is contained in:
KaifAhmad1
2026-08-19 19:09:02 +05:30
co-authored by fabio-rovai
parent e55c03bd39
commit 2d75952476
6 changed files with 49 additions and 6 deletions
+9
View File
@@ -11,6 +11,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **Semantica RDF vocabulary, and deterministic entity/relationship IRIs** (#1109, closes #1107, closes #1101) by @fabio-rovai, reviewed by @KaifAhmad1
- Every RDF/JSON-LD export mints terms in `https://semantica.dev/ns#`, and until now nothing declared what those terms meant — the namespace 404s and no vocabulary shipped with the package, so a consumer receiving an export had no way to tell `sem:text` from a typo of it, and no closed-world checker could validate an export at all
- `semantica/ontology/vocabulary/semantica-ns.ttl` declares the terms the exporters actually emit — drawn from the emitting call sites in `export/rdf_exporter.py`, `export/json_exporter.py` and `provenance/manager.py`, not from what a vocabulary "ought" to contain. Ships inside the package (`from semantica.ontology.vocabulary import vocabulary_turtle`) so it loads without a network round trip, and is the same document intended to be served at the namespace IRI once hosting/content-negotiation is sorted
- `tests/ontology/test_vocabulary.py` ties the document to the code: every term a serializer can write must be declared, so adding a term to an exporter without declaring it fails the build
- The missing-id fallback minted entity/relationship IRIs from Python's builtin `hash()`, randomised per process (`PYTHONHASHSEED`), so the same entity got a different IRI on every run and exports couldn't be diffed, deduplicated, or joined to an earlier provenance record. It also wrote `<semantica:entity_N>`, an IRI in the scheme `semantica` rather than the expansion of the declared prefix, so those nodes never joined with anything written through it. Minting now uses SHA-256 and writes a full IRI in the declared namespace; the same fix applies to the default entity/relationship types in the Turtle path
- **Fixed during review** (Qodo): the temporal fallback minted from `source_id` only, while the main serializer accepts `source_id` or `source` — relationships using the second form hashed two empty strings, which the previous randomised `hash()` masked by making the IRI unstable anyway; once deterministic, unrelated relationships at the same list index collided on one IRI across exports. Endpoints are now resolved the same way `serialize_to_turtle` resolves them, before minting. `sem:confidence` also lost its declared `xsd:decimal` range: the N-Triples serializer types the same value `xsd:float`, and the two are disjoint, so declaring either contradicted one of the exporters (tracked in #1100) — a new `test_declared_ranges_do_not_contradict_what_the_exporters_emit` guards the whole class of that mistake
- **Fixed in follow-up**: `serialize_to_rdfxml`'s default entity type still wrote the bare string `"semantica:Entity"` into an `rdf:resource` attribute, which (unlike a Turtle angle-bracket or an XML element name) is not namespace-expanded — the exact #1101 failure mode, just on the untested RDF/XML path. `json_exporter.py`'s `semantica:format` and `@type: "semantica:KnowledgeGraph"` were emitted but absent from both the vocabulary and the test's `EMITTED_TERMS` guard set, so the "undeclared terms fail the build" claim didn't actually cover them — both are now declared and guarded. `MANIFEST.in` didn't mirror the `pyproject.toml` package-data addition, so a source-distribution install could omit the vocabulary file. The cross-process minting-stability test replaced the subprocess's entire environment with a POSIX-only `PATH`, breaking it on Windows; now overrides only `PYTHONHASHSEED` on top of the inherited environment
- 229 export and ontology tests pass
- **First-class CrewAI integration** (#962)
- New `pip install semantica[crewai]` extra (`crewai>=0.80.0`) — crewai core provides `BaseTool`/`BaseKnowledgeSource`, so `crewai-tools` is intentionally not included, and the extra is intentionally **not** part of the `all` bundle: crewai hard-requires `chromadb~=1.1.0`, which is affected by the unpatched pre-auth code-injection CVE-2026-45829 (see `integrations/crewai/README.md`)
- `integrations/crewai/SemanticaKGTool` — a CrewAI `BaseTool` exposing 5 KG actions (`extract_entities`, `extract_relations`, `add_to_graph`, `query_graph`, `find_related`) backed by `NERExtractor` / `RelationExtractor` / `ContextGraph`; supports both sync `run()` and async `arun()`
+1
View File
@@ -1 +1,2 @@
recursive-include semantica/static *
recursive-include semantica/ontology/vocabulary *.ttl
+4 -5
View File
@@ -30,11 +30,10 @@ License: MIT
"""
from pathlib import Path
import hashlib
from typing import Any, Dict, List, Optional, Set, Union
from ..utils.exceptions import ProcessingError, ValidationError
from ..utils.helpers import ensure_directory
from ..utils.helpers import ensure_directory, hash_data
from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker
@@ -60,13 +59,13 @@ def mint_entity_iri(text: str) -> str:
provenance record written by an earlier process. SHA-256 is stable across
runs and machines, which is what an identifier has to be.
"""
digest = hashlib.sha256(str(text).encode("utf-8")).hexdigest()[:16]
digest = hash_data(str(text))[:16]
return f"{SEMANTICA_NS}entity_{digest}"
def mint_relationship_iri(index: int, source: Any, target: Any) -> str:
"""Mint a stable IRI for a relationship that arrived without an id."""
digest = hashlib.sha256(f"{source}\x00{target}".encode("utf-8")).hexdigest()[:16]
digest = hash_data(f"{source}\x00{target}")[:16]
return f"{SEMANTICA_NS}rel_{index}_{digest}"
@@ -526,7 +525,7 @@ class RDFSerializer:
entity_text = entity.get("text", "")
entity_id = mint_entity_iri(entity_text)
entity_type = entity.get("type", "semantica:Entity")
entity_type = entity.get("type", DEFAULT_ENTITY_TYPE)
text = entity.get("text") or entity.get("label", "")
confidence = entity.get("confidence", 1.0)
@@ -33,6 +33,13 @@ where a relationship carries sem:type, sem:source and sem:target rather than
being written as a single triple.""" ;
rdfs:isDefinedBy <https://semantica.dev/ns> .
sem:KnowledgeGraph a owl:Class ;
rdfs:label "Knowledge Graph" ;
rdfs:comment """The document-level type of a JSON-LD export: the @type of
the top-level node carrying sem:entities, sem:relationships and
sem:exportedAt. Emitted by _convert_kg_to_jsonld in export/json_exporter.py.""" ;
rdfs:isDefinedBy <https://semantica.dev/ns> .
# ── Properties on an entity ──────────────────────────────────────────────────
sem:text a owl:DatatypeProperty ;
@@ -117,6 +124,14 @@ values carry no timezone offset.""" ;
rdfs:range xsd:dateTime ;
rdfs:isDefinedBy <https://semantica.dev/ns> .
sem:format a owl:DatatypeProperty ;
rdfs:label "format" ;
rdfs:comment """The serialization format label written on a JSON-LD
document (currently always the literal "json-ld"). Emitted by
JSONExporter.export_to_jsonld in export/json_exporter.py.""" ;
rdfs:range xsd:string ;
rdfs:isDefinedBy <https://semantica.dev/ns> .
# ── Temporal term (OWL-Time export) ──────────────────────────────────────────
sem:openEndedInterval a owl:DatatypeProperty ;
+18 -1
View File
@@ -11,6 +11,7 @@ IRI in the scheme ``semantica`` rather than the expansion of the declared
``semantica:`` prefix, so it never joined with anything written through it.
"""
import os
import subprocess
import sys
@@ -45,7 +46,7 @@ def test_minted_entity_iri_is_stable_across_processes():
capture_output=True,
text=True,
check=True,
env={"PYTHONHASHSEED": seed, "PATH": "/usr/bin:/bin"},
env={**os.environ, "PYTHONHASHSEED": seed},
).stdout.strip()
for seed in ("0", "1", "random")
}
@@ -88,6 +89,22 @@ def test_default_types_are_written_as_full_iris_in_turtle():
assert "<semantica:related_to>" not in turtle
def test_default_entity_type_is_a_full_iri_in_rdfxml():
"""RDF/XML's rdf:resource is an attribute value, not a QName context, so a
prefixed default there (``semantica:Entity``) resolves to the scheme
``semantica`` rather than the declared namespace the same failure mode
fixed for Turtle in #1101, missed here because the original tests only
checked Turtle output.
"""
untyped = {"entities": [{"id": "https://example.org/e1", "text": "A"}],
"relationships": []}
rdfxml = RDFExporter().export_to_rdf(untyped, format="rdfxml")
assert f'rdf:resource="{DEFAULT_ENTITY_TYPE}"' in rdfxml
assert 'rdf:resource="semantica:Entity"' not in rdfxml
def test_temporal_minting_uses_either_endpoint_representation():
"""Relationships may carry source/target or source_id/target_id (#1109 review).
+2
View File
@@ -28,6 +28,7 @@ from semantica.ontology.vocabulary import ( # noqa: E402
EMITTED_TERMS = {
"Entity",
"Relationship",
"KnowledgeGraph",
"text",
"confidence",
"metadata",
@@ -38,6 +39,7 @@ EMITTED_TERMS = {
"entities",
"relationships",
"exportedAt",
"format",
"openEndedInterval",
"role_generator",
}