mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
Merge branch 'main' into fix/kg-validator-entity-id
This commit is contained in:
@@ -11,6 +11,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Added
|
||||
|
||||
- **Semantica RDF vocabulary, and deterministic entity/relationship IRIs** (#1109, closes #1107, closes #1101) by @fabio-rovai, reviewed by @KaifAhmad1
|
||||
- Every RDF/JSON-LD export mints terms in `https://semantica.dev/ns#`, and until now nothing declared what those terms meant — the namespace 404s and no vocabulary shipped with the package, so a consumer receiving an export had no way to tell `sem:text` from a typo of it, and no closed-world checker could validate an export at all
|
||||
- `semantica/ontology/vocabulary/semantica-ns.ttl` declares the terms the exporters actually emit — drawn from the emitting call sites in `export/rdf_exporter.py`, `export/json_exporter.py` and `provenance/manager.py`, not from what a vocabulary "ought" to contain. Ships inside the package (`from semantica.ontology.vocabulary import vocabulary_turtle`) so it loads without a network round trip, and is the same document intended to be served at the namespace IRI once hosting/content-negotiation is sorted
|
||||
- `tests/ontology/test_vocabulary.py` ties the document to the code: every term a serializer can write must be declared, so adding a term to an exporter without declaring it fails the build
|
||||
- The missing-id fallback minted entity/relationship IRIs from Python's builtin `hash()`, randomised per process (`PYTHONHASHSEED`), so the same entity got a different IRI on every run and exports couldn't be diffed, deduplicated, or joined to an earlier provenance record. It also wrote `<semantica:entity_N>`, an IRI in the scheme `semantica` rather than the expansion of the declared prefix, so those nodes never joined with anything written through it. Minting now uses SHA-256 and writes a full IRI in the declared namespace; the same fix applies to the default entity/relationship types in the Turtle path
|
||||
- **Fixed during review** (Qodo): the temporal fallback minted from `source_id` only, while the main serializer accepts `source_id` or `source` — relationships using the second form hashed two empty strings, which the previous randomised `hash()` masked by making the IRI unstable anyway; once deterministic, unrelated relationships at the same list index collided on one IRI across exports. Endpoints are now resolved the same way `serialize_to_turtle` resolves them, before minting. `sem:confidence` also lost its declared `xsd:decimal` range: the N-Triples serializer types the same value `xsd:float`, and the two are disjoint, so declaring either contradicted one of the exporters (tracked in #1100) — a new `test_declared_ranges_do_not_contradict_what_the_exporters_emit` guards the whole class of that mistake
|
||||
- **Fixed in follow-up**: `serialize_to_rdfxml`'s default entity type still wrote the bare string `"semantica:Entity"` into an `rdf:resource` attribute, which (unlike a Turtle angle-bracket or an XML element name) is not namespace-expanded — the exact #1101 failure mode, just on the untested RDF/XML path. `json_exporter.py`'s `semantica:format` and `@type: "semantica:KnowledgeGraph"` were emitted but absent from both the vocabulary and the test's `EMITTED_TERMS` guard set, so the "undeclared terms fail the build" claim didn't actually cover them — both are now declared and guarded. `MANIFEST.in` didn't mirror the `pyproject.toml` package-data addition, so a source-distribution install could omit the vocabulary file. The cross-process minting-stability test replaced the subprocess's entire environment with a POSIX-only `PATH`, breaking it on Windows; now overrides only `PYTHONHASHSEED` on top of the inherited environment
|
||||
- 229 export and ontology tests pass
|
||||
|
||||
- **First-class CrewAI integration** (#962)
|
||||
- New `pip install semantica[crewai]` extra (`crewai>=0.80.0`) — crewai core provides `BaseTool`/`BaseKnowledgeSource`, so `crewai-tools` is intentionally not included, and the extra is intentionally **not** part of the `all` bundle: crewai hard-requires `chromadb~=1.1.0`, which is affected by the unpatched pre-auth code-injection CVE-2026-45829 (see `integrations/crewai/README.md`)
|
||||
- `integrations/crewai/SemanticaKGTool` — a CrewAI `BaseTool` exposing 5 KG actions (`extract_entities`, `extract_relations`, `add_to_graph`, `query_graph`, `find_related`) backed by `NERExtractor` / `RelationExtractor` / `ContextGraph`; supports both sync `run()` and async `arun()`
|
||||
|
||||
@@ -1 +1,2 @@
|
||||
recursive-include semantica/static *
|
||||
recursive-include semantica/ontology/vocabulary *.ttl
|
||||
|
||||
+1
-1
@@ -271,7 +271,7 @@ include = ["semantica*", "integrations*"]
|
||||
[tool.setuptools.package-data]
|
||||
# Explicit patterns are more reliable than **/* across setuptools versions.
|
||||
# static/* covers index.html / favicon; static/assets/* covers all JS/CSS chunks.
|
||||
"semantica" = ["static/*", "static/assets/*"]
|
||||
"semantica" = ["static/*", "static/assets/*", "ontology/vocabulary/*.ttl"]
|
||||
|
||||
[tool.black]
|
||||
line-length = 88
|
||||
|
||||
@@ -33,11 +33,42 @@ from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Set, Union
|
||||
|
||||
from ..utils.exceptions import ProcessingError, ValidationError
|
||||
from ..utils.helpers import ensure_directory
|
||||
from ..utils.helpers import ensure_directory, hash_data
|
||||
from ..utils.logging import get_logger
|
||||
from ..utils.progress_tracker import get_progress_tracker
|
||||
|
||||
|
||||
SEMANTICA_NS = "https://semantica.dev/ns#"
|
||||
|
||||
#: Written when an entity carries no type of its own. A full IRI rather than the
|
||||
#: prefixed form, because the Turtle serializer writes it inside angle brackets,
|
||||
#: where `semantica:Entity` would be read as an IRI in the scheme `semantica`
|
||||
#: rather than as the prefix expansion (issue #1101).
|
||||
DEFAULT_ENTITY_TYPE = f"{SEMANTICA_NS}Entity"
|
||||
|
||||
#: Written when a relationship carries no type of its own. Same reasoning.
|
||||
DEFAULT_RELATION_TYPE = f"{SEMANTICA_NS}related_to"
|
||||
|
||||
|
||||
def mint_entity_iri(text: str) -> str:
|
||||
"""Mint a stable IRI for an entity that arrived without an id.
|
||||
|
||||
Python's builtin ``hash()`` is randomised per process (PYTHONHASHSEED), so
|
||||
minting from it gave the same entity a different IRI on every run: exports
|
||||
could not be diffed, deduplicated against an earlier load, or joined to a
|
||||
provenance record written by an earlier process. SHA-256 is stable across
|
||||
runs and machines, which is what an identifier has to be.
|
||||
"""
|
||||
digest = hash_data(str(text))[:16]
|
||||
return f"{SEMANTICA_NS}entity_{digest}"
|
||||
|
||||
|
||||
def mint_relationship_iri(index: int, source: Any, target: Any) -> str:
|
||||
"""Mint a stable IRI for a relationship that arrived without an id."""
|
||||
digest = hash_data(f"{source}\x00{target}")[:16]
|
||||
return f"{SEMANTICA_NS}rel_{index}_{digest}"
|
||||
|
||||
|
||||
class NamespaceManager:
|
||||
"""
|
||||
RDF namespace management engine.
|
||||
@@ -360,9 +391,9 @@ class RDFSerializer:
|
||||
entity_id = entity.get("id")
|
||||
if not entity_id:
|
||||
entity_text = entity.get("text", "")
|
||||
entity_id = f"semantica:entity_{hash(entity_text)}"
|
||||
entity_id = mint_entity_iri(entity_text)
|
||||
|
||||
entity_type = entity.get("type", "semantica:Entity")
|
||||
entity_type = entity.get("type", DEFAULT_ENTITY_TYPE)
|
||||
text = entity.get("text") or entity.get("label", "")
|
||||
confidence = entity.get("confidence", 1.0)
|
||||
|
||||
@@ -376,7 +407,7 @@ class RDFSerializer:
|
||||
for idx, rel in enumerate(relationships):
|
||||
source_id = rel.get("source_id") or rel.get("source")
|
||||
target_id = rel.get("target_id") or rel.get("target")
|
||||
rel_type = rel.get("type", "semantica:related_to")
|
||||
rel_type = rel.get("type", DEFAULT_RELATION_TYPE)
|
||||
|
||||
lines.append(f"<{source_id}> <{rel_type}> <{target_id}> .")
|
||||
|
||||
@@ -413,10 +444,14 @@ class RDFSerializer:
|
||||
if time_axis in ("transaction", "both"):
|
||||
axes.append(("tx", rel.get("recorded_at"), rel.get("superseded_at")))
|
||||
|
||||
rel_base_id = (
|
||||
rel.get("id")
|
||||
or f"semantica:rel_{idx}_{hash(str(rel.get('source_id', '')) + str(rel.get('target_id', '')))}"
|
||||
)
|
||||
# Resolve endpoints the same way serialize_to_turtle does: both
|
||||
# representations are accepted upstream, and minting from source_id
|
||||
# alone hashes empty strings for every relationship that uses source,
|
||||
# so unrelated relationships at the same index would collide on a
|
||||
# deterministic IRI.
|
||||
source_id = rel.get("source_id") or rel.get("source") or ""
|
||||
target_id = rel.get("target_id") or rel.get("target") or ""
|
||||
rel_base_id = rel.get("id") or mint_relationship_iri(idx, source_id, target_id)
|
||||
|
||||
lines = [""] # blank separator
|
||||
for axis_name, from_val, until_val in axes:
|
||||
@@ -488,9 +523,9 @@ class RDFSerializer:
|
||||
entity_id = entity.get("id")
|
||||
if not entity_id:
|
||||
entity_text = entity.get("text", "")
|
||||
entity_id = f"semantica:entity_{hash(entity_text)}"
|
||||
entity_id = mint_entity_iri(entity_text)
|
||||
|
||||
entity_type = entity.get("type", "semantica:Entity")
|
||||
entity_type = entity.get("type", DEFAULT_ENTITY_TYPE)
|
||||
text = entity.get("text") or entity.get("label", "")
|
||||
confidence = entity.get("confidence", 1.0)
|
||||
|
||||
@@ -644,7 +679,7 @@ class RDFSerializer:
|
||||
entity_id = entity.get("id")
|
||||
if not entity_id:
|
||||
entity_text = entity.get("text", "")
|
||||
entity_id = f"semantica:entity_{hash(entity_text)}"
|
||||
entity_id = mint_entity_iri(entity_text)
|
||||
|
||||
subject = expand_uri(entity_id)
|
||||
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
"""The vocabulary Semantica's exporters emit terms from.
|
||||
|
||||
Every RDF export mints terms in ``https://semantica.dev/ns#``: ``sem:text``,
|
||||
``sem:confidence``, the default ``sem:Entity`` type, and the rest. Until this
|
||||
file existed, nothing declared what those terms meant, so a consumer receiving
|
||||
an export could not tell ``sem:text`` from a typo of it, and no closed-world
|
||||
check could be run against them at all (issue #1107).
|
||||
|
||||
The document ships inside the package so it can be loaded without a network
|
||||
round trip, and is the same file intended to be served at the namespace IRI.
|
||||
|
||||
>>> from semantica.ontology.vocabulary import vocabulary_turtle
|
||||
>>> ttl = vocabulary_turtle()
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
VOCABULARY_FILENAME = "semantica-ns.ttl"
|
||||
|
||||
#: The namespace the vocabulary declares terms in.
|
||||
NAMESPACE = "https://semantica.dev/ns#"
|
||||
|
||||
__all__ = ["NAMESPACE", "VOCABULARY_FILENAME", "vocabulary_path", "vocabulary_turtle"]
|
||||
|
||||
|
||||
def vocabulary_path() -> Path:
|
||||
"""Filesystem path to the vocabulary document."""
|
||||
return Path(__file__).parent / VOCABULARY_FILENAME
|
||||
|
||||
|
||||
def vocabulary_turtle() -> str:
|
||||
"""The vocabulary document as Turtle."""
|
||||
return vocabulary_path().read_text(encoding="utf-8")
|
||||
@@ -0,0 +1,155 @@
|
||||
@prefix owl: <http://www.w3.org/2002/07/owl#> .
|
||||
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
|
||||
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
|
||||
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
|
||||
@prefix dct: <http://purl.org/dc/terms/> .
|
||||
@prefix prov: <http://www.w3.org/ns/prov#> .
|
||||
@prefix time: <http://www.w3.org/2006/time#> .
|
||||
@prefix sem: <https://semantica.dev/ns#> .
|
||||
|
||||
<https://semantica.dev/ns> a owl:Ontology ;
|
||||
rdfs:label "Semantica vocabulary" ;
|
||||
rdfs:comment """Declares the terms the Semantica exporters emit in
|
||||
https://semantica.dev/ns#. Drafted from the emitting call sites in
|
||||
semantica 0.6.5: export/rdf_exporter.py, export/json_exporter.py and
|
||||
provenance/manager.py. Every term below appears in output the package
|
||||
produces today; no term has been invented for completeness.""" ;
|
||||
owl:versionInfo "0.1.0-draft" ;
|
||||
dct:created "2026-08-19"^^xsd:date .
|
||||
|
||||
# ── Classes ──────────────────────────────────────────────────────────────────
|
||||
|
||||
sem:Entity a owl:Class ;
|
||||
rdfs:label "Entity" ;
|
||||
rdfs:comment """The default type given to an extracted entity when the
|
||||
source carries no type of its own. Emitted by serialize_to_turtle as the
|
||||
fallback for entity.get("type").""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:Relationship a owl:Class ;
|
||||
rdfs:label "Relationship" ;
|
||||
rdfs:comment """A reified relationship, as emitted in the JSON-LD export
|
||||
where a relationship carries sem:type, sem:source and sem:target rather than
|
||||
being written as a single triple.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:KnowledgeGraph a owl:Class ;
|
||||
rdfs:label "Knowledge Graph" ;
|
||||
rdfs:comment """The document-level type of a JSON-LD export: the @type of
|
||||
the top-level node carrying sem:entities, sem:relationships and
|
||||
sem:exportedAt. Emitted by _convert_kg_to_jsonld in export/json_exporter.py.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Properties on an entity ──────────────────────────────────────────────────
|
||||
|
||||
sem:text a owl:DatatypeProperty ;
|
||||
rdfs:label "text" ;
|
||||
rdfs:comment """The surface text of an extracted entity. Carries the same
|
||||
intent as rdfs:label; declared separately because the exporters emit it under
|
||||
this IRI.""" ;
|
||||
rdfs:domain sem:Entity ;
|
||||
rdfs:range xsd:string ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:confidence a owl:DatatypeProperty ;
|
||||
rdfs:label "confidence" ;
|
||||
rdfs:comment """Extractor confidence in the assertion, on the unit interval.
|
||||
Emitted for both entities and relationships, so the domain is left open rather
|
||||
than tied to sem:Entity.
|
||||
|
||||
No rdfs:range is declared, deliberately. The Turtle serializer writes the value
|
||||
bare, which the Turtle grammar reads as xsd:decimal, while the N-Triples
|
||||
serializer types it xsd:float explicitly, and those two datatypes are disjoint.
|
||||
Declaring either one would make the vocabulary contradict one of the exporters.
|
||||
Issue #1100 tracks the disagreement; a range belongs here once the serializers
|
||||
agree on one.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:metadata a owl:AnnotationProperty ;
|
||||
rdfs:label "metadata" ;
|
||||
rdfs:comment """Free-form metadata carried through from extraction. An
|
||||
annotation property because its value is an arbitrary structure rather than a
|
||||
modelled one.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Relationship terms (JSON-LD export) ──────────────────────────────────────
|
||||
|
||||
sem:related_to a owl:ObjectProperty ;
|
||||
rdfs:label "related to" ;
|
||||
rdfs:comment """The default predicate for a relationship whose type the
|
||||
extractor did not determine. Deliberately unspecific: it asserts that two
|
||||
entities are connected and nothing about how.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:source a owl:ObjectProperty ;
|
||||
rdfs:label "source" ;
|
||||
rdfs:comment "The subject entity of a reified relationship." ;
|
||||
rdfs:domain sem:Relationship ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:target a owl:ObjectProperty ;
|
||||
rdfs:label "target" ;
|
||||
rdfs:comment "The object entity of a reified relationship." ;
|
||||
rdfs:domain sem:Relationship ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:type a owl:DatatypeProperty ;
|
||||
rdfs:label "type" ;
|
||||
rdfs:comment """The relationship type as a label, as emitted in the JSON-LD
|
||||
export. Distinct from rdf:type, which relates a node to a class rather than to
|
||||
a string.""" ;
|
||||
rdfs:domain sem:Relationship ;
|
||||
rdfs:range xsd:string ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Document-level terms (JSON-LD export) ────────────────────────────────────
|
||||
|
||||
sem:entities a owl:ObjectProperty ;
|
||||
rdfs:label "entities" ;
|
||||
rdfs:comment "Ordered list of entities in an exported graph document." ;
|
||||
rdfs:range sem:Entity ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:relationships a owl:ObjectProperty ;
|
||||
rdfs:label "relationships" ;
|
||||
rdfs:comment "Ordered list of relationships in an exported graph document." ;
|
||||
rdfs:range sem:Relationship ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:exportedAt a owl:DatatypeProperty ;
|
||||
rdfs:label "exported at" ;
|
||||
rdfs:comment """When the export was written. Emitted as an ISO 8601 local
|
||||
timestamp, so the range is xsd:dateTime rather than xsd:dateTimeStamp: the
|
||||
values carry no timezone offset.""" ;
|
||||
rdfs:range xsd:dateTime ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
sem:format a owl:DatatypeProperty ;
|
||||
rdfs:label "format" ;
|
||||
rdfs:comment """The serialization format label written on a JSON-LD
|
||||
document (currently always the literal "json-ld"). Emitted by
|
||||
JSONExporter.export_to_jsonld in export/json_exporter.py.""" ;
|
||||
rdfs:range xsd:string ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Temporal term (OWL-Time export) ──────────────────────────────────────────
|
||||
|
||||
sem:openEndedInterval a owl:DatatypeProperty ;
|
||||
rdfs:label "open ended interval" ;
|
||||
rdfs:comment """True when an interval has no known end. OWL-Time has no
|
||||
standard predicate for this, which is the reason the exporter mints one: an
|
||||
interval with no time:hasEnd is ambiguous between "ongoing" and "end not
|
||||
recorded", and this term resolves that in favour of the first.""" ;
|
||||
rdfs:domain time:Interval ;
|
||||
rdfs:range xsd:boolean ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
|
||||
# ── Provenance roles ─────────────────────────────────────────────────────────
|
||||
|
||||
sem:role_generator a prov:Role ;
|
||||
rdfs:label "generator" ;
|
||||
rdfs:comment """The default role in a prov:qualifiedAssociation, used when
|
||||
an agent generated an entity rather than approving or reviewing it. Typed as
|
||||
prov:Role so that prov:hadRole has a declared value rather than an undeclared
|
||||
IRI.""" ;
|
||||
rdfs:isDefinedBy <https://semantica.dev/ns> .
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Minted IRIs must be stable and must sit in the declared namespace (issue #1101).
|
||||
|
||||
An entity that arrives without an id gets one minted for it. That identifier was
|
||||
built from Python's builtin ``hash()``, which is randomised per process, so the
|
||||
same entity received a different IRI on every run and exports could not be
|
||||
diffed, deduplicated against an earlier load, or joined to a provenance record
|
||||
written by an earlier process.
|
||||
|
||||
It was also written as ``semantica:entity_N`` inside angle brackets, which is an
|
||||
IRI in the scheme ``semantica`` rather than the expansion of the declared
|
||||
``semantica:`` prefix, so it never joined with anything written through it.
|
||||
"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from semantica.export.rdf_exporter import (
|
||||
DEFAULT_ENTITY_TYPE,
|
||||
DEFAULT_RELATION_TYPE,
|
||||
RDFExporter,
|
||||
SEMANTICA_NS,
|
||||
mint_entity_iri,
|
||||
mint_relationship_iri,
|
||||
)
|
||||
|
||||
UNIDENTIFIED = {
|
||||
"entities": [{"text": "Acme Corp", "type": "https://example.org/Org"}],
|
||||
"relationships": [],
|
||||
}
|
||||
|
||||
|
||||
def test_minted_entity_iri_is_stable_within_a_process():
|
||||
assert mint_entity_iri("Acme Corp") == mint_entity_iri("Acme Corp")
|
||||
|
||||
|
||||
def test_minted_entity_iri_is_stable_across_processes():
|
||||
"""The regression that matters: identity must survive a restart."""
|
||||
script = (
|
||||
"from semantica.export.rdf_exporter import mint_entity_iri;"
|
||||
"print(mint_entity_iri('Acme Corp'))"
|
||||
)
|
||||
runs = {
|
||||
subprocess.run(
|
||||
[sys.executable, "-c", script],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
env={**os.environ, "PYTHONHASHSEED": seed},
|
||||
).stdout.strip()
|
||||
for seed in ("0", "1", "random")
|
||||
}
|
||||
assert len(runs) == 1, f"minted IRI differs between processes: {runs}"
|
||||
|
||||
|
||||
def test_minted_iris_are_in_the_declared_namespace():
|
||||
assert mint_entity_iri("Acme Corp").startswith(SEMANTICA_NS)
|
||||
assert mint_relationship_iri(0, "a", "b").startswith(SEMANTICA_NS)
|
||||
|
||||
|
||||
def test_distinct_entities_get_distinct_iris():
|
||||
assert mint_entity_iri("Acme Corp") != mint_entity_iri("Acme Corporation")
|
||||
|
||||
|
||||
def test_turtle_export_writes_a_resolvable_minted_iri():
|
||||
turtle = RDFExporter().export_to_rdf(UNIDENTIFIED, format="turtle")
|
||||
|
||||
assert f"<{SEMANTICA_NS}entity_" in turtle
|
||||
assert "<semantica:entity_" not in turtle, "scheme 'semantica' is not the prefix"
|
||||
|
||||
|
||||
def test_ntriples_export_agrees_with_turtle_on_the_minted_iri():
|
||||
exporter = RDFExporter()
|
||||
minted = mint_entity_iri("Acme Corp")
|
||||
|
||||
assert minted in exporter.export_to_rdf(UNIDENTIFIED, format="turtle")
|
||||
assert minted in exporter.export_to_rdf(UNIDENTIFIED, format="ntriples")
|
||||
|
||||
|
||||
def test_default_types_are_written_as_full_iris_in_turtle():
|
||||
untyped = {"entities": [{"id": "https://example.org/e1", "text": "A"}],
|
||||
"relationships": [{"source": "https://example.org/e1",
|
||||
"target": "https://example.org/e2"}]}
|
||||
turtle = RDFExporter().export_to_rdf(untyped, format="turtle")
|
||||
|
||||
assert f"<{DEFAULT_ENTITY_TYPE}>" in turtle
|
||||
assert f"<{DEFAULT_RELATION_TYPE}>" in turtle
|
||||
assert "<semantica:Entity>" not in turtle
|
||||
assert "<semantica:related_to>" not in turtle
|
||||
|
||||
|
||||
def test_default_entity_type_is_a_full_iri_in_rdfxml():
|
||||
"""RDF/XML's rdf:resource is an attribute value, not a QName context, so a
|
||||
|
||||
prefixed default there (``semantica:Entity``) resolves to the scheme
|
||||
``semantica`` rather than the declared namespace — the same failure mode
|
||||
fixed for Turtle in #1101, missed here because the original tests only
|
||||
checked Turtle output.
|
||||
"""
|
||||
untyped = {"entities": [{"id": "https://example.org/e1", "text": "A"}],
|
||||
"relationships": []}
|
||||
rdfxml = RDFExporter().export_to_rdf(untyped, format="rdfxml")
|
||||
|
||||
assert f'rdf:resource="{DEFAULT_ENTITY_TYPE}"' in rdfxml
|
||||
assert 'rdf:resource="semantica:Entity"' not in rdfxml
|
||||
|
||||
|
||||
def test_temporal_minting_uses_either_endpoint_representation():
|
||||
"""Relationships may carry source/target or source_id/target_id (#1109 review).
|
||||
|
||||
Minting from source_id alone hashed empty strings for every relationship
|
||||
that used the other representation, so once the IRI became deterministic,
|
||||
unrelated relationships at the same index collided on it and their temporal
|
||||
data aliased when the exports were loaded together.
|
||||
"""
|
||||
def temporal(rel):
|
||||
return RDFExporter().export_to_rdf(
|
||||
{"entities": [], "relationships": [rel]},
|
||||
format="turtle",
|
||||
include_temporal=True,
|
||||
)
|
||||
|
||||
a = temporal({"source": "https://example.org/a", "target": "https://example.org/b",
|
||||
"type": "https://example.org/worksFor", "valid_from": "2020-01-01T00:00:00Z"})
|
||||
b = temporal({"source": "https://example.org/c", "target": "https://example.org/d",
|
||||
"type": "https://example.org/worksFor", "valid_from": "2020-01-01T00:00:00Z"})
|
||||
|
||||
assert f"<{SEMANTICA_NS}rel_" in a
|
||||
assert a != b, "different endpoints must not mint the same temporal IRI"
|
||||
|
||||
|
||||
def test_temporal_minting_agrees_across_the_two_representations():
|
||||
"""The same relationship written either way is the same relationship."""
|
||||
def mint(rel):
|
||||
return mint_relationship_iri(
|
||||
0,
|
||||
rel.get("source_id") or rel.get("source") or "",
|
||||
rel.get("target_id") or rel.get("target") or "",
|
||||
)
|
||||
|
||||
assert mint({"source": "a", "target": "b"}) == mint({"source_id": "a", "target_id": "b"})
|
||||
@@ -0,0 +1,116 @@
|
||||
"""The vocabulary must stay true to what the exporters emit (issue #1107).
|
||||
|
||||
A vocabulary document that drifts from the code is worse than none, because it
|
||||
states that terms mean something while the exporters emit different ones. These
|
||||
tests tie the two together: every term the serializers can write must be
|
||||
declared here, so adding a term to an exporter without declaring it fails the
|
||||
build rather than shipping an undeclared IRI.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
rdflib = pytest.importorskip("rdflib")
|
||||
|
||||
from semantica.export.rdf_exporter import ( # noqa: E402
|
||||
DEFAULT_ENTITY_TYPE,
|
||||
DEFAULT_RELATION_TYPE,
|
||||
SEMANTICA_NS,
|
||||
)
|
||||
from semantica.ontology.vocabulary import ( # noqa: E402
|
||||
NAMESPACE,
|
||||
vocabulary_path,
|
||||
vocabulary_turtle,
|
||||
)
|
||||
|
||||
#: Every term the exporters emit in the Semantica namespace, by local name.
|
||||
#: RDF and OWL-Time paths in export/rdf_exporter.py, document and relationship
|
||||
#: terms in export/json_exporter.py, roles in provenance/manager.py.
|
||||
EMITTED_TERMS = {
|
||||
"Entity",
|
||||
"Relationship",
|
||||
"KnowledgeGraph",
|
||||
"text",
|
||||
"confidence",
|
||||
"metadata",
|
||||
"related_to",
|
||||
"source",
|
||||
"target",
|
||||
"type",
|
||||
"entities",
|
||||
"relationships",
|
||||
"exportedAt",
|
||||
"format",
|
||||
"openEndedInterval",
|
||||
"role_generator",
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def graph():
|
||||
g = rdflib.Graph()
|
||||
g.parse(data=vocabulary_turtle(), format="turtle")
|
||||
return g
|
||||
|
||||
|
||||
def test_vocabulary_ships_with_the_package():
|
||||
assert vocabulary_path().is_file()
|
||||
|
||||
|
||||
def test_vocabulary_parses(graph):
|
||||
assert len(graph) > 0
|
||||
|
||||
|
||||
def test_namespace_matches_the_one_the_exporters_use():
|
||||
assert NAMESPACE == SEMANTICA_NS
|
||||
|
||||
|
||||
def test_every_emitted_term_is_declared(graph):
|
||||
declared = {
|
||||
str(s)[len(NAMESPACE) :]
|
||||
for s in set(graph.subjects())
|
||||
if isinstance(s, rdflib.URIRef) and str(s).startswith(NAMESPACE)
|
||||
}
|
||||
missing = EMITTED_TERMS - declared
|
||||
assert not missing, f"emitted but not declared in the vocabulary: {sorted(missing)}"
|
||||
|
||||
|
||||
def test_the_defaults_the_exporters_fall_back_to_are_declared(graph):
|
||||
for iri in (DEFAULT_ENTITY_TYPE, DEFAULT_RELATION_TYPE):
|
||||
assert (rdflib.URIRef(iri), None, None) in graph, f"{iri} is not declared"
|
||||
|
||||
|
||||
def test_every_declared_term_carries_a_label_and_a_comment(graph):
|
||||
for subject in set(graph.subjects()):
|
||||
if not (isinstance(subject, rdflib.URIRef) and str(subject).startswith(NAMESPACE)):
|
||||
continue
|
||||
assert graph.value(subject, rdflib.RDFS.label), f"{subject} has no rdfs:label"
|
||||
assert graph.value(subject, rdflib.RDFS.comment), f"{subject} has no rdfs:comment"
|
||||
|
||||
|
||||
def test_declared_ranges_do_not_contradict_what_the_exporters_emit(graph):
|
||||
"""A declared range must match the datatype the serializers actually write.
|
||||
|
||||
Caught by review on #1109: sem:confidence was declared xsd:decimal while the
|
||||
N-Triples serializer types the same value xsd:float. A vocabulary that
|
||||
contradicts the code is worse than no vocabulary, so any range declared here
|
||||
has to be one the exporters really emit.
|
||||
"""
|
||||
import re
|
||||
|
||||
from semantica.export.rdf_exporter import RDFExporter
|
||||
|
||||
sample = {
|
||||
"entities": [{"id": "https://example.org/e1", "text": "A",
|
||||
"type": "https://example.org/T", "confidence": 0.5}],
|
||||
"relationships": [],
|
||||
}
|
||||
emitted = RDFExporter().export_to_rdf(sample, format="ntriples")
|
||||
|
||||
for subject, _, range_ in graph.triples((None, rdflib.RDFS.range, None)):
|
||||
if not str(subject).startswith(NAMESPACE):
|
||||
continue
|
||||
local = str(subject)[len(NAMESPACE):]
|
||||
for match in re.finditer(rf'<{NAMESPACE}{local}> "[^"]*"\^\^<([^>]+)>', emitted):
|
||||
assert match.group(1) == str(range_), (
|
||||
f"{local}: vocabulary declares {range_}, N-Triples emits {match.group(1)}"
|
||||
)
|
||||
Reference in New Issue
Block a user