Review finding on #1121, and correct: with new entries carrying +00:00
and entries written earlier carrying nothing, query_recorded_between()
and audit_log() compared ISO strings directly, which orders by how a
timestamp is spelled rather than when it happened.
Two consequences, both introduced by the offset this PR adds:
- An inclusive naive bound naming a stored offset-bearing timestamp
sorts below it, because the stored value is the longer string, so the
record it names is excluded from its own range.
- A bound in another offset lands wherever its digits fall.
"2026-08-19T19:45:00+05:30" is 14:15Z, before an entry at 14:19Z, but
string comparison puts it after.
Both paths now compare instants, through a new to_utc_datetime() helper
that reads a missing offset as UTC. That is what the naive values
actually were: provenance stamped with datetime.utcnow(), so reading
them as UTC keeps a stored naive value and the same instant written with
an offset comparing equal instead of ordering by representation. It is
also the read side the remaining 147 call sites will need whenever the
rest of the package is converted.
A bound that cannot be read as a timestamp keeps the historical string
comparison rather than raising on a call that used to work.
Five new tests cover the inclusive naive bound, the other-offset bound,
legacy and offset-bearing entries ordered together, audit_log's since
filter, and the unreadable-bound fallback. The first two fail with
manager.py reverted; the rest are guards.
569 provenance, export and ontology tests pass, and the full-suite
failure set is unchanged at 329, all from optional dependencies missing
locally.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
semantica/export/ stamped every value with datetime.now().isoformat(),
which reads the machine's local clock. semantica/provenance/ stamped its
own with datetime.utcnow().isoformat(), which reads UTC. Both return a
naive datetime and both serialize identically, so once the value is out
of the process nothing distinguishes them: the same string means two
different instants depending on which module wrote it.
In RDF the consequence is silent rather than loud. Under XSD 1.1 a value
with no timezone compared against one with a timezone is indeterminate
whenever the two fall inside the 14-hour window, SPARQL turns an
indeterminate comparison into an error, and FILTER discards errors as
non-matches. Loading a Semantica-stamped export into Oxigraph next to two
correctly stamped ones and asking which were written before a given
instant returns the other two and drops ours, with no error anywhere.
prov:generatedAtTime, prov:startedAtTime, prov:endedAtTime and
prov:atTime all carry values written this way, so an audit trail cannot
be ordered against timestamps from any other system.
Adds utc_now()/utc_now_iso() to semantica/utils/helpers.py, exported from
semantica.utils, and uses them at all 29 call sites in export/
(json_exporter, yaml_exporter, report_generator, export_provenance) and
provenance/ (manager, schemas, bridge_axiom). Values now read
2026-08-19T14:19:04.229937+00:00: one unambiguous instant, comparable
against any correctly stamped value, and valid xsd:dateTimeStamp.
sem:exportedAt's range in the vocabulary that landed with #1109 is
tightened from xsd:dateTime to xsd:dateTimeStamp accordingly. Its comment
had to explain why the weaker range was necessary; that reason is gone.
datetime.utcnow() is also deprecated as of Python 3.12 and scheduled for
removal. Constructing a ProvenanceEntry under -W error::DeprecationWarning
on 3.13 raised; it no longer does.
Two new test modules cover offset presence on every export and provenance
path, PROV-O literals valid as xsd:dateTimeStamp, comparison against a
timezone-aware instant without TypeError, the Oxigraph filter that
dropped the naive value, the declared range matching what the exporter
writes, and the document @id remaining a valid IRI with +00:00 in it. The
filter test picks a bound inside the indeterminate window on purpose: a
bound years away is determinate even for a naive value, and the test
would pass without the fix. 13 of the 14 fail with this commit's
semantica/export, semantica/provenance and vocabulary reverted.
The remaining 147 naive call sites, in context/, vector_store/, seed/ and
elsewhere, are deliberately untouched: those timestamps are compared
against values parsed back from previously stored naive strings, so
converting the write side alone would raise TypeError on existing data.
That sweep needs a read-side migration and belongs in its own change.
No new failures across the suite: 329 pre-existing failures before and
after, all from optional dependencies missing in the local environment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The #1101 fix covered serialize_to_turtle, serialize_to_ntriples and
serialize_to_rdfxml. Both JSON-LD writers were left interpolating the
entity's own text into f"semantica:entity/{text}" and the endpoints into
f"semantica:rel/{source}_{target}". Three consequences, all reproducible
on 0.6.5 through the public API:
- An entity whose text contains a space, which is most organisation and
person names an extractor produces, mints an invalid IRI. A JSON-LD
parser drops that node in full and says nothing, so the entity is
simply missing from the export: rdflib reads 6 triples for
{"text": "AcmeCorp"} and 2 for {"text": "Acme Corp"}.
- serialize_to_jsonld resolved endpoints from source_id/target_id only,
while the rest of the module accepts source/target too. Every
relationship carrying the second form minted the identical
"semantica:rel/_", so all of them collapsed onto one node and their
types and endpoints merged into a graph nobody wrote.
- The JSON-LD @id and the Turtle IRI for one entity disagreed
(ns#entity/Acme Corp vs ns#entity_a73cb4563ee2e72c), so the two
serializations of one knowledge graph were two different graphs.
Both writers now use mint_entity_iri/mint_relationship_iri, resolving
endpoints both ways and passing the list index the RDF paths pass, so
one knowledge graph carries one node identity whichever serializer
wrote it.
JSONExporter.export_entities and export_relationships also declare the
semantica prefix their @context was already writing "semantica:entities"
against. Without the declaration a processor reads that as an IRI in the
scheme semantica rather than the namespace expansion, which is the
original #1101 defect on a third path: rdflib returns the predicate
literally as semantica:entities.
tests/export/test_jsonld_iri_minting.py parses each export with a real
JSON-LD processor rather than asserting on the JSON text, and covers all
seven claims above. Each test fails on the parent commit.
236 export and ontology tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
serialize_to_rdfxml still defaulted entity_type to the bare string
"semantica:Entity" written into an rdf:resource attribute, which isn't
namespace-expanded the way a Turtle angle-bracket or XML element name is -
the same #1101 failure mode, just on the path the original tests didn't
cover. Now uses the full-IRI DEFAULT_ENTITY_TYPE like the Turtle path.
json_exporter.py emits semantica:format and @type: "semantica:KnowledgeGraph",
neither of which was declared in the vocabulary or included in
EMITTED_TERMS, so the "undeclared terms fail the build" guarantee didn't
actually cover them. Both are now declared with rdfs:label/comment and
added to the guard set.
MANIFEST.in didn't mirror the pyproject.toml package-data addition, so a
source-distribution install could ship without the vocabulary file.
The cross-process minting-stability test replaced the subprocess's entire
environment with a POSIX-only PATH, breaking it on Windows and any host
needing other inherited env vars; now overrides only PYTHONHASHSEED on top
of the inherited environment.
Also folds mint_entity_iri/mint_relationship_iri's hand-rolled
hashlib.sha256(...).hexdigest() into the existing hash_data() helper this
file already imports alongside.
229 export and ontology tests pass, including a new regression test for
the RDF/XML default-type fix.
Co-Authored-By: fabio-rovai <fabio@thetesseractacademy.com>
Both from review on #1109.
The temporal fallback minted from source_id only, while the main serializer
accepts source_id or source. Relationships using the second form therefore
hashed two empty strings, and once the IRI became deterministic that turned a
latent problem into an active one: unrelated relationships at the same list
index collided on the same IRI across exports, so their temporal data aliased
when loaded together. Endpoints are now resolved the way serialize_to_turtle
resolves them, before minting.
The vocabulary declared sem:confidence with range xsd:decimal, which the
N-Triples serializer contradicts by typing the same value xsd:float. Neither is
safe to declare while the two serializers disagree, since the Turtle path writes
the value bare and the Turtle grammar reads that as xsd:decimal. The range is
dropped with the reasoning recorded on the term and a pointer to #1100, which
tracks the disagreement itself.
Extends the drift guard rather than only fixing the instance: a new test asserts
that any range this vocabulary declares matches the datatype the serializers
actually emit, so the class of contradiction that review caught fails the build
next time.
228 export and ontology tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes#1107, closes#1101.
Every RDF export mints terms in https://semantica.dev/ns#, and nothing declared
what those terms meant. The namespace returns 404 and no vocabulary shipped with
the package, so a consumer receiving an export could not tell semantica:text
from a typo of it: in the open world an undeclared IRI is unknown rather than
wrong, and every RDF tool treats the two alike. Closed-world checking is what
separates them, and it needs a document to check against.
semantica/ontology/vocabulary/semantica-ns.ttl declares the fourteen terms the
exporters actually emit, drawn from the emitting call sites rather than from
what a vocabulary ought to contain. It ships inside the package so it loads
without a network round trip, and is the same document intended to be served at
the namespace IRI once hosting and content negotiation are sorted.
tests/ontology/test_vocabulary.py ties the document to the code: every term the
serializers can write must be declared, so adding a term to an exporter without
declaring it fails the build rather than shipping an undeclared IRI.
The vocabulary alone would not have made those IRIs resolve, because the
fallback path minted them from Python's builtin hash(). That is randomised per
process, so the same entity received a different IRI on every run and exports
could not be diffed, deduplicated against an earlier load, or joined to a
provenance record written by an earlier process. Minting now uses SHA-256 and
writes a full IRI in the declared namespace rather than semantica:entity_N,
which inside angle brackets is an IRI in the scheme "semantica" rather than the
prefix expansion, and so never joined with anything written through the prefix.
The same applies to the default entity and relationship types in the Turtle
path.
134 export tests and 91 ontology tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
explain_violations previously rendered hardcoded placeholders (min_count=1,
max_count=1) and misused the violation message as the datatype/class value,
so plain-English explanations were inaccurate. The root cause is that
_run_pyshacl never read the real constraint parameters from sh:sourceShape
when building each SHACLViolation.
Changes:
- SHACLViolation: add min_count/max_count/datatype/class_ fields and include
them in to_dict()
- _run_pyshacl: back-reference sh:sourceShape to extract the real
sh:minCount/sh:maxCount/sh:datatype/sh:class values
- explain_violations: render the real values, falling back to "?" or
descriptive text when unknown
Note: sh:qualifiedMinCount/qualifiedMaxCount are not handled and fall back to
the "?" placeholder.
Adds regression tests covering both the formatting path and the sh:sourceShape
back-reference (skips when pyshacl/rdflib are absent).
* feat(context): add to_kg_dict() adapter for canonical KG shape
Convert ContextGraph internal nodes/edges/source representation into the canonical entities/relationships/source_id shape consumed by RDFExporter and TemporalGraphQuery. Add entities_only filtering that drops dangling relationships, plus README examples and unit tests.
* fix(context): harden to_kg_dict against null props and non-str node ids
- Guard properties/metadata with 'or {}' so nodes loaded from JSON null
no longer raise TypeError when copied (Qodo bug 1)
- Coerce entity id to str(n.node_id) so it matches ContextEdge's
str-coerced endpoints, preventing valid relationships from being
dropped during entities_only filtering (Qodo bug 3)
* fix(kg): accept source_id/target_id endpoints in validator and temporal query
to_kg_dict() emits canonical source_id/target_id keys, but GraphValidator
and TemporalGraphQuery only read the legacy source/target keys, so its
output failed validation and lost relationships (Qodo bug 2).
- GraphValidator: resolve endpoints from either key variant and treat a
resolvable source/target (plus type) as satisfying required fields
- TemporalGraphQuery.analyze_evolution/find_paths: read either variant
- tests: add regression coverage for null props/metadata (bug 1),
non-string node ids (bug 3), and KG-utility consumability (bug 2)
---------
Co-authored-by: 江俊杰 <jiangjunjie.37@jd.com>
- export_table_data() re-raises ValidationError instead of masking it
as ProcessingError via the blanket except Exception.
- _apply_connection_pin() restores the session's original Host header
state on an unpinned hop instead of unconditionally clearing it,
which was dropping a caller-supplied session's own Host override.
- SQL fragment blocklist now masks quoted string/identifier literal
contents before matching, so legitimate data containing a blocked
keyword (e.g. status = 'union') no longer false-positives; a
malformed/unterminated quote stays unmasked and still scrutinized.
Documents the tarball path traversal, latent SQLi, DNS-rebinding TOCTOU,
stored XSS, and SPARQL injection fixes, plus the follow-up hardening
found in review, under [Unreleased] > Security.
Fixes a set of runtime trust-boundary issues from a private security
disclosure (checkout 7c3372c0): tarball restore path traversal, latent
SQL injection in the DB exporter, a DNS-rebinding TOCTOU gap in the
shared SSRF guard, unescaped HTML in report generation, and unvalidated
SPARQL object IRIs in AnzoStore, plus several lower-severity hardening
items found in the same review.
* docs(context): fix unrunnable ContextGraph docstring example
The module docstring's Example Usage block called add_node/add_edge with
keyword arguments they do not accept. add_node(node_id, node_type, ...) takes
node_type positionally and has no properties parameter, so the documented call
raised TypeError; add_edge's parameter is edge_type, so type= fell through to
**properties and polluted edge metadata while appearing to work.
Two of the three broken forms failed silently rather than raising, storing a
nested properties dict or a stray type key instead of erroring.
Add regression tests that execute the documented calls and assert the docstring
itself does not reintroduce the invalid kwargs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(context): close two blind spots in the docstring regression guards
The guards added in the previous commit could pass while checking nothing.
_example_block() terminated the capture at the first "\n\n". The Example
Usage block already contains ">>> " spacer lines, so any reformatting that
turned one into a bare blank line would truncate the capture -- potentially
to empty -- and the guards would then scan a block that no longer held the
add_node/add_edge calls they exist to police.
Both guards also iterated over re.findall() without asserting a match. Zero
matches meant zero assertions and a green test, so the two failure modes
compounded: a truncated block produced no matches, and no matches produced
a pass.
Terminate the block at the next top-level section header (^\S) or end of
docstring instead, so blank lines inside the example are harmless, and
assert the captured block, the parsed statement list, and each guard's
match list are all non-empty.
Extract statements with doctest.DocTestParser rather than a line regex.
This also catches a call reformatted across "..." continuation lines, which
the ">>> graph.add_node(.*" pattern silently skipped, and lets
test_documented_calls_execute exec the docstring's own statements instead
of a retyped copy that could drift from it. Full doctest.testmod isn't
usable here: add_node/add_edge return True and the docs carry no
expected-output lines, so it reports 4 spurious failures.
Narrow the kwarg check to (?<![\w])type\s*= so a legitimate node_type=
or edge_type= in the docs no longer trips a guard aimed at bare type=.
Verified by mutating the module docstring and re-running the guards: extra
blank lines with a valid example still pass; regressed add_node/add_edge,
a type= on a continuation line, deleted calls, and a deleted section all
fail; a legitimate node_type= passes. 6 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(context): correct precedent lookup in docstring example
---------
Co-authored-by: Pravit Ampapathini <pravitampapathini@Pravits-MacBook-Air-3.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
Co-authored-by: Mohd Kaif <98801504+KaifAhmad1@users.noreply.github.com>
Address review feedback:
- State that only protected routes require the API key and note that
/api/health and /api/info are intentionally unauthenticated.
- Note the CLI warning on non-loopback binds only fires in anonymous
mode or when SEMANTICA_API_KEY is unset.
- Add SEMANTICA_API_KEY and SEMANTICA_ALLOW_ANONYMOUS to the Environment
variables table.
* test(export): guard Parquet tests on pyarrow itself, not the exporter import
Closes#1054
* test(export): guard on PARQUET_AVAILABLE so the skip matches the runtime check
find_spec only proves pyarrow is discoverable, not importable. Addresses
review feedback on #1056.
---------
Two findings from the Qodo review:
- Sigma's edge label renderer draws data.label, but the graph stores the
relationship type in edgeType — enabling renderEdgeLabels alone left
edges blank. The edgeReducer now maps edgeType onto label (suppressed for
hidden edges).
- renderEdgeLabels was hardcoded on with no way to disable it. It now
follows a new edgeLabelsEnabled entry in the Effects panel (default on),
wired through the existing GraphEffectToggle/GraphEffectsState plumbing,
so dense graphs get their label-free edges back.
* fix(security): prevent auth header leakage across redirects
* fix(security): harden redirect credential handling
Address Copilot and Qodo review findings for #947.
- Remove unused variables, imports, and unnecessary pass statements from tests.
- Harden cross-origin redirect handling for per-request auth credentials.
- Strip session-level auth handlers before cross-origin redirect hops.
- Prevent session.auth from regenerating Authorization headers.
- Disable trust_env during cross-origin hops to prevent .netrc credential injection.
- Restore session auth and trust_env state reliably with try/finally.
- Add regression coverage for auth=, session.auth, trust_env, and multi-hop redirects.
- Preserve existing security behavior and same-origin authentication semantics.
Validated with 189/189 security and affected tests passing.
* fix(security): scope allow_private_ips to same-host redirects, fix error handling gaps
Follow-up to review findings on #1067:
- MCPClient hardcoded allow_private_ips=True for every redirect hop, not
just its operator-configured host, so a compromised/malicious MCP server
could 302 into private address space (e.g. cloud metadata) unchecked.
request_with_ssrf_guard() gains allow_private_ips_on_redirect: a redirect
target inherits the original host's private-IP trust only when it matches
that host; MCPClient now pins it to False.
- detect_public_api() only caught requests.exceptions.RequestException, but
the SSRF guard raises ValidationError for blocked hosts/redirects, unlike
its sibling ingest_public_api(). Now catches and re-raises it the same way.
- detect_public_api()/ingest_public_api() forwarded session/allow_private_ips
through **options into request_with_ssrf_guard(), which already passes
both explicitly -- a caller supplying either would hit a duplicate-kwarg
TypeError. Both are now popped from request_options first.
New regression coverage for all three in tests/ingest/, plus a CHANGELOG
entry under Unreleased/Security.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
The star history chart was broken due to GitHub stargazer API restrictions, so it could no longer be rendered. Update the README to point to a working alternative that uses a different data source requiring no API token.
SemanticChunker.__init__ only caught OSError around load_spacy_model(),
while NERExtractor's identical call (fixed earlier in this PR) also
catches generic Exception for a model that is installed but fails at
runtime. Bring SemanticChunker in line so a broken spaCy config
degrades to fallback chunking instead of crashing __init__.
Adds a regression test mirroring the existing NERExtractor case, and a
CHANGELOG entry for #998/#1042.
T1mn