1. An absurd magnitude expanded instead of being rejected. xsd:decimal
has no exponent notation, so the value has to be written out in full,
and "1e100000000" is eleven characters that expand to a hundred
million digits. "1e100000" already produced a 100,001 character string
here. The export path continues past validation errors, so one
malformed field could exhaust memory. Values beyond
MAX_CONFIDENCE_EXPONENT are now omitted like any other unusable value.
1e-9 still round-trips.
2. Decimal keeps the sign of zero, so 0.0 and -0.0 serialised as "0" and
"-0", which are two distinct RDF terms. That is exactly the duplicate
this PR exists to remove, so zero is normalised.
4 further tests.
#1100 — the four serializers rendered the same confidence four different
ways. Turtle wrote it bare, which the Turtle grammar reads as
xsd:decimal. N-Triples typed it xsd:float. RDF/XML wrote a plain literal
with no datatype. JSON-LD wrote a native JSON number, which expands to
xsd:double. For confidence 0.9 that is four distinct RDF terms, so a
FILTER matches at most one of them, and merging two exports of one graph
gives an entity two different confidence values.
N-Triples also omitted the triple entirely when confidence was absent,
while the other three wrote the 1.0 default, so the two serializations
differed in the number of triples as well as in their datatype.
`normalize_confidence` now produces one canonical lexical form and every
path writes it with CONFIDENCE_DATATYPE. xsd:decimal is the choice
because it is what the Turtle path already produced, so the most used
output is unchanged, and because it is exact: xsd:float is 32 bit binary
and cannot represent 0.9 at all. Values that arrive in exponent notation
are reformatted, since 1e-05 is not a valid xsd:decimal.
#1102 — the Turtle path interpolated the value with no type check, so a
confidence of "high" produced `semantica:confidence high .` and made the
entire document unparseable. One bad field cost the whole export. A value
that cannot be a decimal is now omitted with a warning naming the entity,
rather than written as something the vocabulary contradicts. Numeric
strings are still accepted. Booleans are not, since bool subclasses int
and True would otherwise become a confidence of 1.
sem:confidence in the shipped vocabulary declared no rdfs:range,
deliberately, because declaring one would have contradicted three of the
four exporters. It now declares xsd:decimal, and a drift guard asserts
the vocabulary and the serializers agree.
20 tests in tests/export/test_confidence_literal_typing.py, comparing the
parsed graphs of all four formats rather than their text. Export and
ontology suites pass at 240 tests.
serialize_to_rdfxml still defaulted entity_type to the bare string
"semantica:Entity" written into an rdf:resource attribute, which isn't
namespace-expanded the way a Turtle angle-bracket or XML element name is -
the same #1101 failure mode, just on the path the original tests didn't
cover. Now uses the full-IRI DEFAULT_ENTITY_TYPE like the Turtle path.
json_exporter.py emits semantica:format and @type: "semantica:KnowledgeGraph",
neither of which was declared in the vocabulary or included in
EMITTED_TERMS, so the "undeclared terms fail the build" guarantee didn't
actually cover them. Both are now declared with rdfs:label/comment and
added to the guard set.
MANIFEST.in didn't mirror the pyproject.toml package-data addition, so a
source-distribution install could ship without the vocabulary file.
The cross-process minting-stability test replaced the subprocess's entire
environment with a POSIX-only PATH, breaking it on Windows and any host
needing other inherited env vars; now overrides only PYTHONHASHSEED on top
of the inherited environment.
Also folds mint_entity_iri/mint_relationship_iri's hand-rolled
hashlib.sha256(...).hexdigest() into the existing hash_data() helper this
file already imports alongside.
229 export and ontology tests pass, including a new regression test for
the RDF/XML default-type fix.
Co-Authored-By: fabio-rovai <fabio@thetesseractacademy.com>
Both from review on #1109.
The temporal fallback minted from source_id only, while the main serializer
accepts source_id or source. Relationships using the second form therefore
hashed two empty strings, and once the IRI became deterministic that turned a
latent problem into an active one: unrelated relationships at the same list
index collided on the same IRI across exports, so their temporal data aliased
when loaded together. Endpoints are now resolved the way serialize_to_turtle
resolves them, before minting.
The vocabulary declared sem:confidence with range xsd:decimal, which the
N-Triples serializer contradicts by typing the same value xsd:float. Neither is
safe to declare while the two serializers disagree, since the Turtle path writes
the value bare and the Turtle grammar reads that as xsd:decimal. The range is
dropped with the reasoning recorded on the term and a pointer to #1100, which
tracks the disagreement itself.
Extends the drift guard rather than only fixing the instance: a new test asserts
that any range this vocabulary declares matches the datatype the serializers
actually emit, so the class of contradiction that review caught fails the build
next time.
228 export and ontology tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes#1107, closes#1101.
Every RDF export mints terms in https://semantica.dev/ns#, and nothing declared
what those terms meant. The namespace returns 404 and no vocabulary shipped with
the package, so a consumer receiving an export could not tell semantica:text
from a typo of it: in the open world an undeclared IRI is unknown rather than
wrong, and every RDF tool treats the two alike. Closed-world checking is what
separates them, and it needs a document to check against.
semantica/ontology/vocabulary/semantica-ns.ttl declares the fourteen terms the
exporters actually emit, drawn from the emitting call sites rather than from
what a vocabulary ought to contain. It ships inside the package so it loads
without a network round trip, and is the same document intended to be served at
the namespace IRI once hosting and content negotiation are sorted.
tests/ontology/test_vocabulary.py ties the document to the code: every term the
serializers can write must be declared, so adding a term to an exporter without
declaring it fails the build rather than shipping an undeclared IRI.
The vocabulary alone would not have made those IRIs resolve, because the
fallback path minted them from Python's builtin hash(). That is randomised per
process, so the same entity received a different IRI on every run and exports
could not be diffed, deduplicated against an earlier load, or joined to a
provenance record written by an earlier process. Minting now uses SHA-256 and
writes a full IRI in the declared namespace rather than semantica:entity_N,
which inside angle brackets is an IRI in the scheme "semantica" rather than the
prefix expansion, and so never joined with anything written through the prefix.
The same applies to the default entity and relationship types in the Turtle
path.
134 export tests and 91 ontology tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
explain_violations previously rendered hardcoded placeholders (min_count=1,
max_count=1) and misused the violation message as the datatype/class value,
so plain-English explanations were inaccurate. The root cause is that
_run_pyshacl never read the real constraint parameters from sh:sourceShape
when building each SHACLViolation.
Changes:
- SHACLViolation: add min_count/max_count/datatype/class_ fields and include
them in to_dict()
- _run_pyshacl: back-reference sh:sourceShape to extract the real
sh:minCount/sh:maxCount/sh:datatype/sh:class values
- explain_violations: render the real values, falling back to "?" or
descriptive text when unknown
Note: sh:qualifiedMinCount/qualifiedMaxCount are not handled and fall back to
the "?" placeholder.
Adds regression tests covering both the formatting path and the sh:sourceShape
back-reference (skips when pyshacl/rdflib are absent).
* feat(context): add to_kg_dict() adapter for canonical KG shape
Convert ContextGraph internal nodes/edges/source representation into the canonical entities/relationships/source_id shape consumed by RDFExporter and TemporalGraphQuery. Add entities_only filtering that drops dangling relationships, plus README examples and unit tests.
* fix(context): harden to_kg_dict against null props and non-str node ids
- Guard properties/metadata with 'or {}' so nodes loaded from JSON null
no longer raise TypeError when copied (Qodo bug 1)
- Coerce entity id to str(n.node_id) so it matches ContextEdge's
str-coerced endpoints, preventing valid relationships from being
dropped during entities_only filtering (Qodo bug 3)
* fix(kg): accept source_id/target_id endpoints in validator and temporal query
to_kg_dict() emits canonical source_id/target_id keys, but GraphValidator
and TemporalGraphQuery only read the legacy source/target keys, so its
output failed validation and lost relationships (Qodo bug 2).
- GraphValidator: resolve endpoints from either key variant and treat a
resolvable source/target (plus type) as satisfying required fields
- TemporalGraphQuery.analyze_evolution/find_paths: read either variant
- tests: add regression coverage for null props/metadata (bug 1),
non-string node ids (bug 3), and KG-utility consumability (bug 2)
---------
Co-authored-by: 江俊杰 <jiangjunjie.37@jd.com>
- export_table_data() re-raises ValidationError instead of masking it
as ProcessingError via the blanket except Exception.
- _apply_connection_pin() restores the session's original Host header
state on an unpinned hop instead of unconditionally clearing it,
which was dropping a caller-supplied session's own Host override.
- SQL fragment blocklist now masks quoted string/identifier literal
contents before matching, so legitimate data containing a blocked
keyword (e.g. status = 'union') no longer false-positives; a
malformed/unterminated quote stays unmasked and still scrutinized.
Documents the tarball path traversal, latent SQLi, DNS-rebinding TOCTOU,
stored XSS, and SPARQL injection fixes, plus the follow-up hardening
found in review, under [Unreleased] > Security.
Fixes a set of runtime trust-boundary issues from a private security
disclosure (checkout 7c3372c0): tarball restore path traversal, latent
SQL injection in the DB exporter, a DNS-rebinding TOCTOU gap in the
shared SSRF guard, unescaped HTML in report generation, and unvalidated
SPARQL object IRIs in AnzoStore, plus several lower-severity hardening
items found in the same review.
* docs(context): fix unrunnable ContextGraph docstring example
The module docstring's Example Usage block called add_node/add_edge with
keyword arguments they do not accept. add_node(node_id, node_type, ...) takes
node_type positionally and has no properties parameter, so the documented call
raised TypeError; add_edge's parameter is edge_type, so type= fell through to
**properties and polluted edge metadata while appearing to work.
Two of the three broken forms failed silently rather than raising, storing a
nested properties dict or a stray type key instead of erroring.
Add regression tests that execute the documented calls and assert the docstring
itself does not reintroduce the invalid kwargs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(context): close two blind spots in the docstring regression guards
The guards added in the previous commit could pass while checking nothing.
_example_block() terminated the capture at the first "\n\n". The Example
Usage block already contains ">>> " spacer lines, so any reformatting that
turned one into a bare blank line would truncate the capture -- potentially
to empty -- and the guards would then scan a block that no longer held the
add_node/add_edge calls they exist to police.
Both guards also iterated over re.findall() without asserting a match. Zero
matches meant zero assertions and a green test, so the two failure modes
compounded: a truncated block produced no matches, and no matches produced
a pass.
Terminate the block at the next top-level section header (^\S) or end of
docstring instead, so blank lines inside the example are harmless, and
assert the captured block, the parsed statement list, and each guard's
match list are all non-empty.
Extract statements with doctest.DocTestParser rather than a line regex.
This also catches a call reformatted across "..." continuation lines, which
the ">>> graph.add_node(.*" pattern silently skipped, and lets
test_documented_calls_execute exec the docstring's own statements instead
of a retyped copy that could drift from it. Full doctest.testmod isn't
usable here: add_node/add_edge return True and the docs carry no
expected-output lines, so it reports 4 spurious failures.
Narrow the kwarg check to (?<![\w])type\s*= so a legitimate node_type=
or edge_type= in the docs no longer trips a guard aimed at bare type=.
Verified by mutating the module docstring and re-running the guards: extra
blank lines with a valid example still pass; regressed add_node/add_edge,
a type= on a continuation line, deleted calls, and a deleted section all
fail; a legitimate node_type= passes. 6 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(context): correct precedent lookup in docstring example
---------
Co-authored-by: Pravit Ampapathini <pravitampapathini@Pravits-MacBook-Air-3.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
Co-authored-by: Mohd Kaif <98801504+KaifAhmad1@users.noreply.github.com>
Address review feedback:
- State that only protected routes require the API key and note that
/api/health and /api/info are intentionally unauthenticated.
- Note the CLI warning on non-loopback binds only fires in anonymous
mode or when SEMANTICA_API_KEY is unset.
- Add SEMANTICA_API_KEY and SEMANTICA_ALLOW_ANONYMOUS to the Environment
variables table.
* test(export): guard Parquet tests on pyarrow itself, not the exporter import
Closes#1054
* test(export): guard on PARQUET_AVAILABLE so the skip matches the runtime check
find_spec only proves pyarrow is discoverable, not importable. Addresses
review feedback on #1056.
---------
Two findings from the Qodo review:
- Sigma's edge label renderer draws data.label, but the graph stores the
relationship type in edgeType — enabling renderEdgeLabels alone left
edges blank. The edgeReducer now maps edgeType onto label (suppressed for
hidden edges).
- renderEdgeLabels was hardcoded on with no way to disable it. It now
follows a new edgeLabelsEnabled entry in the Effects panel (default on),
wired through the existing GraphEffectToggle/GraphEffectsState plumbing,
so dense graphs get their label-free edges back.
* fix(security): prevent auth header leakage across redirects
* fix(security): harden redirect credential handling
Address Copilot and Qodo review findings for #947.
- Remove unused variables, imports, and unnecessary pass statements from tests.
- Harden cross-origin redirect handling for per-request auth credentials.
- Strip session-level auth handlers before cross-origin redirect hops.
- Prevent session.auth from regenerating Authorization headers.
- Disable trust_env during cross-origin hops to prevent .netrc credential injection.
- Restore session auth and trust_env state reliably with try/finally.
- Add regression coverage for auth=, session.auth, trust_env, and multi-hop redirects.
- Preserve existing security behavior and same-origin authentication semantics.
Validated with 189/189 security and affected tests passing.
* fix(security): scope allow_private_ips to same-host redirects, fix error handling gaps
Follow-up to review findings on #1067:
- MCPClient hardcoded allow_private_ips=True for every redirect hop, not
just its operator-configured host, so a compromised/malicious MCP server
could 302 into private address space (e.g. cloud metadata) unchecked.
request_with_ssrf_guard() gains allow_private_ips_on_redirect: a redirect
target inherits the original host's private-IP trust only when it matches
that host; MCPClient now pins it to False.
- detect_public_api() only caught requests.exceptions.RequestException, but
the SSRF guard raises ValidationError for blocked hosts/redirects, unlike
its sibling ingest_public_api(). Now catches and re-raises it the same way.
- detect_public_api()/ingest_public_api() forwarded session/allow_private_ips
through **options into request_with_ssrf_guard(), which already passes
both explicitly -- a caller supplying either would hit a duplicate-kwarg
TypeError. Both are now popped from request_options first.
New regression coverage for all three in tests/ingest/, plus a CHANGELOG
entry under Unreleased/Security.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
SemanticChunker.__init__ only caught OSError around load_spacy_model(),
while NERExtractor's identical call (fixed earlier in this PR) also
catches generic Exception for a model that is installed but fails at
runtime. Bring SemanticChunker in line so a broken spaCy config
degrades to fallback chunking instead of crashing __init__.
Adds a regression test mirroring the existing NERExtractor case, and a
CHANGELOG entry for #998/#1042.
The Explorer API has required SEMANTICA_API_KEY (X-API-Key header) since
v0.6.5, failing closed with 503 when unconfigured. Both the explorer
README security note and docs/explorer-setup.md still claimed there was
no built-in authentication.
Update both to describe the actual behavior: API-key enforcement,
the 503 fail-closed mode, and the explicit SEMANTICA_ALLOW_ANONYMOUS=true
opt-in for local development.
Fixes#1028
Moves a concise version of the system-level vs. foundation-model
explainability clarification up next to the opening pitch, so it's
visible before readers scroll to the high-stakes-domains section.
Adds a consistent scope note to README and docs (concepts, FAQ, index)
stating Semantica does not expose or reconstruct an LLM's internal
reasoning/chain-of-thought. It explains and audits the AI system
around the model: context, provenance, policies, decisions, and
execution history.
* docs(contributing): formalize issue assignment and duplicate-PR triage workflow
Comments are no longer required before an issue can be assigned - maintainers
may assign directly based on recent activity. Also documents the duplicate-PR
priority order for triage (contributor PR, claimed issue, activity tiebreak,
late duplicates, overlapping scope).
* docs(contributing): clarify assignment precedence and define activity tiebreak
Addresses Qodo review feedback on PR #1030: the duplicate-PR priority list
now states these rules apply on top of the assignment workflow (opening a PR
pre-assignment doesn't grant priority), and the "most active" tiebreak now
specifies a concrete 60-day window and signals instead of being subjective.
ExcelParser.__init__ called get_progress_tracker() without importing it,
so every instantiation raised NameError and the class was unusable. The
existing test imported ExcelParser but never constructed it, so nothing
caught it. Same defect as #530 in SimilarityCalculator, which was fixed
without sweeping the rest of the codebase.
Add construction coverage for every parser exported from semantica.parse,
driven off __all__ so later additions are covered automatically. These
live outside test_parse_comprehensive.py, whose setUp patches
get_progress_tracker into each parse module and would mock away the
interaction under test.
Closes#1014
Co-authored-by: Pravit Ampapathini <pravitampapathini@Pravits-MacBook-Air-3.local>
FABIOTESS