Compare commits

..
Author SHA1 Message Date
Sameer Kadam 8ef7c9f760 Merge branch 'main' into fix/mcp-server-version 2026-08-12 12:49:31 +05:30
918830a821 fix(pipeline): resolve broken import and missing run() in PipelineWithProvenance (#862)
* fix(pipeline): resolve broken import and missing run() in PipelineWithProvenance

Fix two bugs in pipeline_provenance.py:

1. Wrong import path: `from .pipeline import Pipeline` fails because
   `semantica/pipeline/pipeline.py` does not exist. Pipeline lives in
   `pipeline_builder.py`. Fixed to `from .pipeline_builder import Pipeline`.

2. Pipeline dataclass has no run() method. PipelineWithProvenance.run()
   now delegates to ExecutionEngine.execute_pipeline(), which is the
   intended execution path for built pipelines.

Additional changes:
- Constructor now accepts a built Pipeline instance (breaking the previous
  unusable API that tried to instantiate a dataclass with **config).
- Replace deprecated datetime.utcnow() with datetime.now(timezone.utc).
- Add test suite covering import, instantiation, execution, attribute
  delegation, and provenance graceful degradation.

Fixes #858

* test: address Qodo review findings

- Remove redundant test_import_succeeds (module-level import already
  guards against import regression at collection time).
- Fix test_provenance_disabled_when_import_fails to deterministically
  simulate ImportError via sys.modules patch and assert provenance is
  actually toggled off (runner.provenance is False).

* fix(pipeline): update provenance callers for Pipeline API

---------

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
Co-authored-by: Russell Jurney <russell.jurney@gmail.com>
2026-08-11 16:49:54 -07:00
Mohd Kaif 5b319560fb chore: bump version to 0.6.5 (#918)
Security release bundling fixes for GHSA-j4mq (missing auth), GHSA-8c7v
(SSRF via redirect bypass), GHSA-482h (Cypher injection), GHSA-8vgg
(SPARQL injection), GHSA-4643 (WebSocket Origin validation), and a
CodeQL-flagged ReDoS in the SPARQL route validator.
2026-08-11 22:41:19 +05:30
Mohd KaifandSameer Kadam f29c4310a1 security: validate WebSocket Origin against the CORS allowlist (GHSA-4643) (#917)
CORSMiddleware doesn't cover WebSocket handshakes at all (Starlette's CORS
support only wraps HTTP), so under SEMANTICA_ALLOW_ANONYMOUS=true --
the mode docker-compose.dev.yml ships -- is_valid_api_key's anonymous
bypass accepted a /ws/graph-updates connection from any origin. Loopback
binding isn't a boundary against a browser: any page the operator has
open can still reach ws://localhost:8000/ws/graph-updates directly, and
ConnectionManager.broadcast sends every graph_mutation to every
connected socket with no per-connection scoping. Combined with
/api/import accepting multipart/form-data (a CORS-safelisted content
type that skips preflight), a hostile page could write to the graph
over REST and read the result back over the unauthenticated WebSocket
-- demonstrated end-to-end in the report with a real client.

Not affected: any deployment with SEMANTICA_API_KEY configured -- the
handshake already rejects without a valid key in that mode. This is an
anonymous-mode-only, development-configuration exposure.

Fix: check the handshake's Origin header against
app.state.explorer_settings['allowed_origins'], the same list
CORSMiddleware already enforces for HTTP, before the key check. A
missing Origin (native/CLI clients, which never set the header --
only browsers do) is still allowed through, since the browser is the
only threat this closes.

4 new tests in test_explorer_auth.py: hostile Origin rejected under
anonymous mode; hostile Origin rejected even with a correct key
(Origin is checked before the key, so a leaked key alone can't
hijack the socket); an allowlisted Origin still connects under
anonymous mode; a missing Origin still connects under anonymous mode
(native clients keep working). Full explorer suite: 226 passed.

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
2026-08-11 22:11:17 +05:30
Mohd Kaif a2886a4e41 Merge pull request #916 from semantica-agi/security/ssrf-dns-pinning-and-object-iri
security: DNS check-then-use pinning for SSRF fetcher, close object-IRI gap
2026-08-11 21:36:42 +05:30
Sameer Kadam a0aa415fc4 Merge branch 'main' into security/ssrf-dns-pinning-and-object-iri 2026-08-11 20:49:18 +05:30
Mohd Kaif ae4f1d4030 Merge pull request #915 from Sameer6305/fix/redos-prefix-decl-regex
fix: resolve ReDoS in _PREFIX_DECL regex (CodeQL py/polynomial-redos #1897)
2026-08-11 19:56:57 +05:30
KaifAhmad1 ea3416ed32 fix: enforce a definitive no-proxy policy for the pinned SSRF fetcher
Qodo's re-review confirmed the multi-IP fallback fix but kept the proxy
finding open: logging-and-falling-back when a proxy applies still let
the DNS-pinning protection be silently skipped under proxy
configuration, rather than enforcing a clear policy either way.

Implemented Qodo's preferred option: proxies are now disabled outright
for this SSRF-sensitive fetcher via session.trust_env = False, so
HTTP_PROXY/HTTPS_PROXY/NO_PROXY env vars are never consulted in the
first place (a configured proxy would perform its own DNS resolution
of the target host outside this process's control, reopening the
DNS check-then-use race pinning exists to close). The adapter also
keeps a fail-closed backstop: if a proxy is somehow still configured
despite trust_env=False (e.g. set explicitly by future code), it now
raises a clear 502 instead of silently connecting through the proxy
unpinned.

_validate_fetch_url's destination classification (blocking private/
internal targets) is unaffected either way — it runs before any of
this and doesn't depend on proxy configuration.

4 new tests: trust_env is disabled on every pinned session; an
HTTP_PROXY env var pointed at an address that would fail if contacted
is confirmed genuinely unused (real local-server fetch still succeeds
directly); and the fail-closed backstop actually raises when a proxy
is forced onto the session. Full explorer + triplet_store suite: 572
passed.
2026-08-11 19:16:29 +05:30
KaifAhmad1 154a7347cd fix: address CI/review findings on DNS pinning (multi-IP fallback, TLS min version)
Four findings from PR #916's automated review, all addressed:

- CodeQL (HIGH): the test HTTPS server's SSLContext allowed TLSv1/TLSv1.1
  by not setting a minimum version. Added
  ssl_ctx.minimum_version = ssl.TLSVersion.TLSv1_2.
- github-code-quality: unused `cryptography` local in
  _make_self_signed_cert — importorskip's return value was never used.
- Qodo (reliability): _validate_fetch_url() only returned the first
  validated IP, and _make_pinned_session() pinned to just that one
  address, so a fetch would fail outright if the first-returned A/AAAA
  record happened to be unreachable even though a later one would work.
  _validate_fetch_url() now returns every validated IP (deduplicated,
  in resolution order); _make_pinned_session() takes the full list and
  falls back through each one via a custom Connection._new_conn
  override, matching the fallback behavior a normal DNS-resolving
  connection would already get for free. Verified with a real test:
  pin to an unreachable loopback address followed by a real one, confirm
  the fetch still succeeds by falling back; and a real test confirming
  it still raises (rather than silently re-resolving the hostname) when
  every pinned address is unreachable.
- Qodo (security): when an HTTP(S) proxy applies, the adapter falls back
  to the unpinned path rather than pinning. This is a real, but
  architecturally unavoidable, limitation from the client side: for a
  forward proxy, the *proxy* performs its own DNS resolution of the
  target host on the application's behalf, a resolution this process
  has no visibility into or control over — there's no client-side pin
  that closes that race. _validate_fetch_url's destination
  classification still fully applies either way; only the secondary
  DNS-pinning hardening doesn't extend through a proxy. Added an info
  log when this fallback path is taken so it's observable rather than
  silent, and expanded the code comment to make the reasoning explicit
  for the next reader/reviewer rather than looking like an oversight.

Tests: 3 new tests in test_ontology_dns_pinning.py (multi-IP fallback
success, all-unreachable failure, deduplicated multi-record resolution).
Full explorer + triplet_store suite: 569 passed.
2026-08-11 19:10:01 +05:30
KaifAhmad1 f2f1d6787d docs(changelog): add PR #916 (DNS pinning + object-IRI gap) entry 2026-08-11 18:57:07 +05:30
KaifAhmad1 646c70ce63 security: DNS check-then-use pinning for SSRF fetcher, close object-IRI gap
Two follow-up hardening items flagged as secondary/deferred during
GHSA-8c7v-62gr-hj6g and GHSA-8vgg-8mr4-r236's fixes:

1. DNS check-then-use (TOCTOU) window in the ontology URL fetcher.
   _validate_fetch_url() resolved and validated a hostname once, but
   _fetch_url_sync() then let requests resolve the same hostname again
   independently at connect time — a low-TTL or rebinding DNS answer
   could differ between the two lookups, reopening the SSRF window the
   validation exists to close.

   _validate_fetch_url() now returns the validated IP, and a new
   _make_pinned_session() builds a per-hop requests.Session whose
   connection pool is pinned directly to that IP (bypassing DNS
   resolution for the connection entirely), while explicitly restoring
   the real hostname as the outgoing HTTP Host header and, for HTTPS,
   the TLS SNI server_hostname/assert_hostname — so the connection
   reaches the validated IP but still presents (and is verified
   against) the real hostname's identity, keeping virtual hosting and
   certificate validation correct.

   Note: an earlier version of this fix set `_dns_host` post-construction
   assuming it was decoupled from `host`, matching some other urllib3
   releases; in the installed version (2.7.0), `host` is a property
   that reads/writes `_dns_host` directly, so that approach silently
   changed the Host header too. Verified with a real (non-mocked) local
   HTTP server, a real local HTTPS server with a self-signed cert
   (proving SNI/cert-hostname verification checks the real hostname,
   not the pinned IP), and a negative control confirming a hostname/cert
   mismatch is still correctly rejected — not silently bypassed.

2. Pre-wrapped object IRIs skipped full validation in
   _format_object_for_sparql/_format_object_for_ntriples (Blazegraph,
   RDF4J). A triplet object already wrapped in `<...>` only had its
   inner content checked for a literal space or `>`, not run through
   sparql_escaping.validate_uri() like the unwrapped-object branch —
   flagged by automated review during GHSA-8vgg-8mr4-r236's fix. Both
   branches now validate identically.

Tests: tests/explorer/test_ontology_dns_pinning.py (6 tests, including
2 real local-server end-to-end checks and 2 real-TLS checks with a
generated self-signed cert, gracefully skipped if `cryptography` isn't
installed); updated tests/explorer/test_ontology_ssrf.py for the new
per-hop session construction; 4 new tests in
tests/triplet_store/test_sparql_injection.py for the object-IRI fix.
Full explorer + triplet_store suite: 566 passed.
2026-08-11 18:52:26 +05:30
Sameer6305 c5981aa306 fix: address qodo review findings on _PREFIX_DECL and query-length guard
Two follow-up fixes to the initial ReDoS patch (CodeQL py/polynomial-redos
#1897), raised during code review:

--- Fix 1: _PREFIX_DECL regression — inline prologues and CRLF (#review-1) ---

The first ReDoS fix replaced the ambiguous trailing \s* with [ \t]*(?:\n|$),
but that introduced a behavioral regression:

  * Inline prologues — PREFIX ex: <...> SELECT ... on a single line were no
    longer stripped because the mandatory (?:\n|$) anchor never matched when
    non-whitespace content followed the IRI on the same line.
  * CRLF line endings — PREFIX ex: <...>\r\n failed because \r is not in
    [ \t]* and the anchor expected a bare \n.

Root cause: the end-of-line anchor was unnecessary; the only thing needed
to eliminate backtracking ambiguity is ensuring the IRI body character class
and the trailing whitespace quantifier are disjoint.

Fix: change the IRI body from <[^>]*> to <[^>\r\n]*>, which:
  - excludes CR and LF from the IRI match (semantically correct — SPARQL
    IRIs cannot span line boundaries)
  - makes [^>\r\n]* and the trailing [ \t]* have zero character overlap,
    eliminating all backtracking ambiguity without any end-of-line anchor

No anchor is used, so both inline prologues and CRLF/LF endings work
naturally. ReDoS payloads (base< + !< x 10,000) still complete in <1 ms.

--- Fix 2: oversized-query length guard obscured error (#review-2) ---

The initial patch placed the _SPARQL_MAX_QUERY_LEN guard inside
_is_read_only_query(), which caused execute_sparql() to return the same
generic 'Only SELECT' error for both genuinely disallowed query types and
oversized inputs. Clients could not distinguish the two rejection reasons.

Fix: move the length check out of _is_read_only_query() and into
execute_sparql() as an explicit early gate, alongside the other resource
limits (_SPARQL_MAX_ROWS, _SPARQL_MAX_GRAPH_NODES). Oversized queries now
return a specific message naming the limit, the received length, and the
remediation step. _is_read_only_query() is documented to be length-agnostic.
_SPARQL_MAX_QUERY_LEN is relocated to the resource-limits block with the
other constants.

--- Tests added ---

tests/test_security_regression.py:
  - test_inline_prefix_before_select_allowed   (Fix 1 regression)
  - test_crlf_line_endings_with_prefix         (Fix 1 regression)
  - test_crlf_multiple_prefixes_then_select    (Fix 1 regression)
  - test_inline_prefix_before_insert_still_blocked (Fix 1 security check)
  - test_long_valid_query_not_rejected_by_is_read_only (Fix 2 separation)

tests/explorer/test_sparql_route.py:
  - test_oversized_query_returns_distinct_length_error (Fix 2 error message)
  - test_oversized_query_never_touches_the_graph       (Fix 2 short-circuit)
  - test_query_exactly_at_length_limit_is_accepted     (Fix 2 boundary)

All 82 tests pass.
2026-08-11 18:37:50 +05:30
Sameer6305 d507fda1b0 fix: resolve ReDoS in _PREFIX_DECL regex (CodeQL #1897)
The _PREFIX_DECL pattern used \s* as a trailing quantifier after
<[^>]*>. On inputs that start with ase< but contain no closing >
(e.g. ase<!<<!<<!<...), the regex engine explores exponentially many
ways to split the match between [^>]* and \s*, causing polynomial
backtracking against user-controlled SPARQL query input.

Fix:
- Replace ^\s* / \s+ / \s* with ^[ \t]* / [ \t]+ / [ \t]*
  so the leading/internal whitespace quantifiers only match horizontal
  whitespace (no overlap with the <[^>]*> IRI part).
- Replace the ambiguous trailing \s* with [ \t]*(?:\n|$), which
  matches only horizontal whitespace followed by a hard line boundary.
  [^>]* and [ \t]* have disjoint character sets, eliminating the
  backtracking ambiguity entirely.
- Add _SPARQL_MAX_QUERY_LEN = 10_000 guard at the top of
  _is_read_only_query as defence-in-depth: rejects oversized input
  before any regex work, bounding worst-case cost even if a future
  pattern change reintroduces ambiguity.

Verified: ReDoS payload ase< + !< x 5000 completes in <1 ms.
Normal PREFIX/BASE stripping and read-only query detection unchanged.

Fixes: CodeQL py/polynomial-redos alert #1897
CWE: CWE-1333, CWE-730, CWE-400
2026-08-11 18:05:29 +05:30
Mohd Kaif 7bf7474ac1 Merge pull request #911 from semantica-agi/security/sparql-injection
security: validate triplet IRIs before SPARQL interpolation (GHSA-8vgg)
2026-08-11 16:41:16 +05:30
KaifAhmad1 546e27cec5 Merge remote-tracking branch 'origin/security/sparql-injection' into security/sparql-injection 2026-08-11 16:30:33 +05:30
KaifAhmad1 a8330874d3 Merge remote-tracking branch 'origin/main' into security/sparql-injection
# Conflicts:
#	CHANGELOG.md
2026-08-11 16:29:39 +05:30
Sameer6305 1c3ac66fd9 fix(rdf4j): preserve literal objects in delete_triplet 2026-08-11 16:27:52 +05:30
Mohd KaifandSameer6305 b846ff88d4 security: sanitize Cypher labels/relationship types/property keys (GHSA-482h) (#910)
* security: sanitize Cypher labels/relationship types/property keys (GHSA-482h-hw99-h62p)

Node labels and property keys passed to create_node/create_relationship
were interpolated directly into Cypher strings in the Neptune, Neo4j, and
FalkorDB graph stores. Property values are parameterized, but labels and
keys can't be bound as parameters, and nothing validated them, so a
document-derived entity type or property name could close the current
Cypher token early and append arbitrary statements (e.g. DETACH DELETE),
running with the application's database credentials.

- New shared semantica/graph_store/query_sanitize.py: sanitize_identifier()
  generalizes age_store.py's existing _sanitize_label/_sanitize_rel_type
  (the only backend that already validated this) into a helper the other
  backends can import without an import cycle with graph_store.py/methods.py.
- Applied at every label/relationship-type/property-key interpolation site
  in amazon_neptune.py, neo4j_store.py, falkordb_store.py, graph_store.py
  (degree_centrality's own query builder), and methods.py
  (update_relationship's own query builder) — create_node, create_nodes,
  create_relationship, get_nodes, get_relationships, get_neighbors,
  shortest_path, update_node, create_index, and all relationship-type
  filters.
- depth/max_depth path-length parameters are also cast to int before
  interpolation as defense-in-depth (they're already typed int, but
  Python doesn't enforce that at runtime).

Added tests/graph_store/test_cypher_injection.py (12 tests covering the
sanitizer directly and reproducing the advisory's injection payload
against Neptune/Neo4j/FalkorDB create_node/create_relationship — asserts
the malicious query is never built or sent), plus regression tests for
graph_store.py's degree_centrality and methods.py's update_relationship.
Full graph_store test suite (224 tests) passes with no regressions.

* fix(graph-store): prevent depth-based Cypher injection

* test(graph-store): tighten injection regression assertions

* docs(changelog): add PR #910 (GHSA-482h Cypher injection) entry

---------

Co-authored-by: Sameer6305 <sskadam6305@gmail.com>
2026-08-11 16:24:28 +05:30
KaifAhmad1 69b79e3d67 docs(changelog): add PR #911 (GHSA-8vgg SPARQL injection) entry 2026-08-11 16:19:06 +05:30
KaifAhmad1 9012492c97 Merge remote-tracking branch 'origin/main' into security/sparql-injection 2026-08-11 16:18:20 +05:30
Mohd Kaif 6ec546b551 Merge pull request #898 from Sunil56224972/security/fix-critical-vulnerabilities
security: fix 4 critical vulnerabilities (RCE, SSRF, XXE, DoS)
2026-08-11 15:50:06 +05:30
KaifAhmad1 6002965c55 docs(changelog): document PR #898's full scope, including the maintainer follow-up fixes 2026-08-11 15:39:11 +05:30
KaifAhmad1 abc10bc8e0 fix(security): restore GHSA-j4mq auth enforcement, fix SPARQL comment-regex bug
Two issues in the last round of commits:

1. explorer/auth.py added a new, opt-in APIKeyAuthMiddleware
   (EXPLORER_API_KEY) and wired it into create_app(), but in doing so
   removed the Depends(require_auth) dependency from every router and
   deleted the /ws/graph-updates handshake check entirely. The new
   middleware also fails OPEN (allows all requests) when its key is
   unset, the opposite of require_auth's fail-closed design. Since
   GHSA-j4mq-hprp-987v (the unauthenticated-Explorer-API advisory) is
   already merged into main via require_auth, this would have reverted
   a merged Critical fix the moment this branch merges. Removed
   explorer/auth.py, restored the per-router dependencies and the
   WebSocket auth check. Kept auth.py's one genuine improvement (adding
   X-API-Key to the CORS allow_headers list) by folding it into the
   existing CORS middleware config.

2. sparql.py's new _is_read_only_query() hardening (comment/PREFIX
   stripping + forbidden-keyword scan) used `#[^\n]*` to strip SPARQL
   comments, but a bare '#' also appears inside standard RDF namespace
   IRIs (e.g. ".../1999/02/22-rdf-syntax-ns#") — the regex struck
   everything after that '#' as a "comment", corrupting the query and
   rejecting any legitimate SELECT using rdf:/rdfs:-style PREFIX
   declarations. Confirmed by the fact the new hardening's own inlined
   test copy failed against two of its own cases. Fixed by only
   treating '#' as a comment-start at line-start or after whitespace,
   which distinguishes ".../ns#" (preceded by a word character) from an
   actual comment (preceded by whitespace/newline in every realistic
   case, including the attacker's own comment-hiding PoC). Also fixed
   the companion PREFIX/BASE regex, which required a prefix-name token
   between the keyword and the IRI even for bare `BASE <...>`
   declarations (which have none).

tests/test_security_regression.py's SPARQL section now imports the real
_is_read_only_query instead of maintaining a parallel inlined copy that
had silently drifted from — and shared the same bug as — the real
implementation; removed its TestAPIKeyAuth class (tested the now-deleted
auth.py) since equivalent, more thorough coverage already exists in
tests/explorer/test_explorer_auth.py. Updated tests/explorer/test_sparql_route.py's
multi-statement-injection test to reflect that the keyword scan now
catches "SELECT ... ; DROP ALL" itself rather than relying on rdflib's
parser, and added a new test confirming the parser still catches
multi-statement syntax that doesn't contain any forbidden keyword.

Full explorer/vector_store/security-regression/age_store suite: 543
passed (the only failures are 6 pre-existing, unrelated Pinecone-client
mocking issues).
2026-08-11 15:36:49 +05:30
Zohaib Hassnain 1f053e005c fix object injection and test flakiness 2026-08-11 14:40:49 +05:00
KaifAhmad1 9ecae47a8a security: validate triplet IRIs before SPARQL interpolation (GHSA-8vgg-8mr4-r236)
Triplet.subject and Triplet.predicate (and, in some builders, .object)
were interpolated directly into SPARQL update/query strings in the
Blazegraph and RDF4J stores, and into a SELECT filter in the Jena store.
A subject containing '>' closes the '<...>' IRI token early, so the rest
of the value is parsed as more SPARQL. Entity names are document text in
the normal ingest pipeline, so anyone whose content gets processed could
append operations like CLEAR ALL, running with the application's store
credentials.

Applied the existing sparql_escaping.validate_uri (already used by
anzo_store.py, the one backend that was already hardened) at every
subject/predicate/object interpolation site:

- blazegraph_store.py: _build_insert_data, _triplets_to_rdf (unreachable
  dead code today but same fix applied for consistency/future-proofing),
  bulk_load's graph option, get_triplets's filter, delete_triplet.
- rdf4j_store.py: _triplets_to_ntriples, get_triplets's filter,
  delete_triplet. (add_triplets's graph option was already validated.)
- jena_store.py: get_triplets's filter — the only vulnerable site;
  add_triplets/delete_triplet already use rdflib's native Python API
  (Graph.add/.remove with URIRef) rather than building query strings, so
  they were never exploitable this way.

Added tests/triplet_store/test_sparql_injection.py (12 tests) reproducing
the advisory's own injection payload against all three backends' write
and read paths, asserting the malicious query is never built or sent.
Full triplet_store suite (330 tests) passes with no regressions.

Note: while adding read-path test coverage, found that jena_store.py's
get_triplets() WHERE-clause filter syntax is malformed SPARQL (missing a
FILTER()/separator before the equality conditions) — a pre-existing
correctness bug unrelated to this fix, worth a separate follow-up.
2026-08-11 14:58:40 +05:30
Sameer Kadam bde6e2d68e Merge branch 'main' into fix/mcp-server-version 2026-08-10 21:38:16 +05:30
Sameer Kadam 01bd908f86 Merge branch 'main' into fix/mcp-server-version 2026-08-10 20:50:49 +05:30
Joey@macstudio 00f4e79d3e fix(mcp): report package version 2026-08-10 20:29:19 +08:00
37 changed files with 2046 additions and 397 deletions
+59
View File
@@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.6.5] - 2026-08-11
### Added
- **Embedded Oxigraph backend for `TripletStore`** (#838, closes #834) by @Linxiushen
@@ -69,6 +71,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- **`PipelineWithProvenance` raised `ModuleNotFoundError` on import and `AttributeError` on `.run()`** (#858, closes #858) by @Karunasagar12
- `from .pipeline import Pipeline` failed because `semantica/pipeline/pipeline.py` does not exist; corrected to `from .pipeline_builder import Pipeline`
- `.run()` called `self._pipeline.run()` on the `Pipeline` dataclass, which has no such method; replaced with `self._engine.execute_pipeline(self._pipeline, ...)` delegating to `ExecutionEngine`
- Constructor now accepts a built `Pipeline` instance (from `PipelineBuilder.build()`) instead of `**config`; the old `Pipeline(**config)` internal construction was invalid and never functional
- Replaced deprecated `datetime.utcnow()` with `datetime.now(timezone.utc)` in `run()`
- **`VectorStore.search_vectors()` returned inconsistent result shapes across backend implementations** (#853, closes #845) by @Sameer6305, reviewed by @KaifAhmad1
- Every built-in backend (FAISS, Milvus, pgvector, Pinecone, Qdrant, SQLite-vec, Weaviate, in-memory) now returns the same canonical `SearchResult` shape (`id`, `score`, `metadata`, `vector`, `distance`), instead of some backends omitting `vector`/`metadata`/`distance` or, for Weaviate, returning a backend-specific `properties` key instead of `metadata`
- Added a `SearchResult` `TypedDict` (`semantica/vector_store/vector_store.py`, exported from `semantica.vector_store`) documenting the contract; `metadata` now always defaults to `{}` rather than being absent, and `id` accepts `Union[str, int]` to accommodate Milvus/Qdrant's native integer IDs without casting
@@ -244,6 +252,57 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Security
- **DNS check-then-use hardening for the ontology URL fetcher, and a remaining object-IRI validation gap** (#916, follow-up to GHSA-8c7v-62gr-hj6g and GHSA-8vgg-8mr4-r236) by @KaifAhmad1
- **DNS check-then-use (TOCTOU) window**: GHSA-8c7v-62gr-hj6g's own fix description flagged this as a secondary gap — `_validate_fetch_url()` resolved and validated a hostname once, but `_fetch_url_sync()` then let `requests` resolve the same hostname again independently at connect time. A low-TTL or rebinding DNS answer could differ between the two lookups, reopening the SSRF window the validation exists to close
- `_validate_fetch_url()` now returns the validated IP, and a new `_make_pinned_session()` builds a per-hop `requests.Session` whose connection pool is pinned directly to that IP — bypassing DNS resolution for the connection entirely — while explicitly restoring the real hostname as the outgoing HTTP `Host` header and, for HTTPS, the TLS SNI `server_hostname`/`assert_hostname`, so the connection reaches the validated IP but still presents (and is verified against) the real hostname's identity, keeping virtual hosting and certificate validation correct
- Caught during implementation: an earlier draft set urllib3's `_dns_host` post-construction, assuming (as in some urllib3 releases) that it was decoupled from `host`. In the version this project installs (2.7.0), `host` is a property that reads/writes `_dns_host` directly, so that approach would have silently changed the Host header too — caught by an end-to-end test against a real local server before landing, rather than shipping. Verified with real (non-mocked) local HTTP and HTTPS servers, the latter using a generated self-signed certificate to prove SNI/cert-hostname verification checks the real hostname rather than the pinned IP, plus a negative control confirming a hostname/cert mismatch is still correctly rejected, not silently bypassed
- **Object-IRI validation gap** (GHSA-8vgg-8mr4-r236 follow-up, distinct from the object-branch fix already shipped in #911): a triplet object already wrapped in `<...>` skipped `sparql_escaping.validate_uri()` in both `blazegraph_store.py` and `rdf4j_store.py`'s `_format_object_for_sparql`/`_format_object_for_ntriples`, only checking the inner content for a literal space or `>` — the pre-wrapped and unwrapped branches now validate identically
- **Fixed along the way** (caught in automated review across two follow-up rounds): `_validate_fetch_url()` originally pinned to only the first resolved IP, so a hostname with multiple A/AAAA records would fail outright if that specific address was unreachable — it now returns every validated IP and `_make_pinned_session()` falls back through all of them, verified by pinning to a genuinely unreachable address followed by a working one and confirming the fetch still succeeds; the test HTTPS server allowed TLSv1/TLSv1.1 by not setting a minimum version, now pinned to TLSv1.2; and when an HTTP(S) proxy applied, pinning was silently skipped in favor of the unpinned path — proxies are now disabled outright for this fetcher (`session.trust_env = False`, so `HTTP_PROXY`/`HTTPS_PROXY` env vars are never consulted) with a fail-closed 502 backstop if a proxy is ever forced onto the session some other way, verified by pointing `HTTP_PROXY` at an address that would fail if actually used and confirming the fetch still succeeds directly
- New `tests/explorer/test_ontology_dns_pinning.py` (12 tests: real local HTTP/HTTPS servers including 2 real-TLS checks, multi-IP fallback success/failure, and no-proxy-trust verification — gracefully skipped without the optional `cryptography` package where applicable); updated `tests/explorer/test_ontology_ssrf.py` for the new per-hop session construction; 4 new tests in `tests/triplet_store/test_sparql_injection.py` for the object-IRI fix. Full `explorer` + `triplet_store` suite: 572 passed
- **Missing Origin validation on the `/ws/graph-updates` WebSocket handshake** (#917, GHSA-4643-wpgq-w329) by @KaifAhmad1
- `CORSMiddleware` doesn't cover WebSocket handshakes at all (Starlette's CORS support only wraps HTTP), so under `SEMANTICA_ALLOW_ANONYMOUS=true` — the mode `docker-compose.dev.yml` ships — the anonymous-mode key bypass accepted a `/ws/graph-updates` connection from any origin. Loopback binding isn't a boundary against a browser: any page the operator has open can still reach `ws://localhost:8000/ws/graph-updates` directly, and `ConnectionManager.broadcast` sends every `graph_mutation` to every connected socket with no per-connection scoping. Combined with `/api/import` accepting `multipart/form-data` (a CORS-safelisted content type that skips preflight), a hostile page could write to the graph over REST and read the result back over the unauthenticated WebSocket
- Not affected: any deployment with `SEMANTICA_API_KEY` configured — the handshake already rejects without a valid key in that mode. This was an anonymous-mode-only, development-configuration exposure
- Fix: check the handshake's `Origin` header against `app.state.explorer_settings['allowed_origins']` — the same list `CORSMiddleware` already enforces for HTTP — before the key check. A missing `Origin` (native/CLI clients, which never set the header) is still allowed through, since the browser is the only threat this closes
- 4 new tests in `tests/explorer/test_explorer_auth.py`: hostile Origin rejected under anonymous mode; hostile Origin rejected even with a correct key (Origin is checked first, so a leaked key alone can't hijack the socket); an allowlisted Origin still connects; a missing Origin still connects. Full `explorer` suite: 226 passed
- **Polynomial-time ReDoS in the SPARQL route's `_PREFIX_DECL` regex** (#915, CodeQL `py/polynomial-redos`) by @Sameer6305
- The prior pattern's trailing `\s*` overlapped with the preceding `<[^>]*>` IRI-body match on inputs containing no closing `>` (e.g. `base<` followed by thousands of `!<` repetitions), forcing the regex engine to explore every possible split between the two quantifiers — O(n²) backtracking reachable from `req.query` via `_is_read_only_query()`
- Fixed by making the two quantifiers character-disjoint: horizontal whitespace only (`[ \t]`, never overlapping the IRI body) instead of `\s*`, and excluding CR/LF from the IRI body (`[^>\r\n]*`) so it can never span a line boundary. Independently verified: the exact pathological payload (`base<` + `!<` × 5,000/20,000) scales linearly (0.238ms → 0.841ms for 4x input, not the ~16x a surviving quadratic blowup would show)
- Added `_SPARQL_MAX_QUERY_LEN = 10_000` as defense-in-depth, checked in `execute_sparql()` before any regex work so a future pattern regression stays bounded regardless
- Two correctness regressions raised in review were checked and did not reproduce: comment-then-prefix stripping order means an inline comment after a `PREFIX` line (`PREFIX ex: <...> # comment`) is already gone by the time `_PREFIX_DECL` runs, verified directly against the pipeline; and the allowlist's `.sub()`-based cleaning only ever affects the yes/no decision, never the query actually sent to `graph.query()` — so even the narrow case of a multi-line string literal that happens to start a line with the literal text `PREFIX` or `BASE` can only cause a legitimate query to be wrongly rejected, never let something malicious through, since rdflib's parser still gates whatever actually executes
- 20 new/updated tests in `tests/explorer/test_sparql_route.py` and `tests/test_security_regression.py` (inline prologues, CRLF line endings, multi-line CRLF prefix chains, oversized-query rejection). 225 `explorer` + 82 SPARQL-specific tests passing
- **SPARQL injection via unvalidated triplet IRIs** (#911, GHSA-8vgg-8mr4-r236) by @KaifAhmad1
- `Triplet.subject`/`.predicate` (and, in some builders, `.object`) were interpolated directly into SPARQL update/query strings in the Blazegraph and RDF4J stores, and into a SELECT filter in the Jena store. A subject containing `>` closes the `<...>` IRI token early, so the rest of the value is parsed as more SPARQL. Entity names are document text in the normal ingest pipeline, so anyone whose content gets processed could append operations like `CLEAR ALL`, running with the application's store credentials
- Applied the existing `sparql_escaping.validate_uri` (already used by `anzo_store.py`, the one backend that was already hardened — this generalizes its approach rather than inventing a new one) at every subject/predicate/object interpolation site: `blazegraph_store.py`'s `_build_insert_data`, `_triplets_to_rdf`, `bulk_load`'s `graph` option, `get_triplets`'s filter, and `delete_triplet`; `rdf4j_store.py`'s `_triplets_to_ntriples`, `get_triplets`'s filter, and `delete_triplet`; `jena_store.py`'s `get_triplets`'s filter (the only vulnerable site there — `add_triplets`/`delete_triplet` already use rdflib's native `Graph.add`/`.remove` with `URIRef` rather than building query strings)
- **Fixed along the way** (caught in review, by @ZohaibHassan16): `_format_object_for_sparql`'s URI branch — used when a triplet's *object* is itself a URI rather than a literal — only checked for spaces and `>` inline instead of running the same `validate_uri` check applied to subject/predicate, leaving the object position as a narrower but real gap in both Blazegraph and RDF4J. Also fixed test flakiness in `RDF4JStore`'s test fixtures, which weren't mocking `_connect()` and so were making real network calls
- New `tests/triplet_store/test_sparql_injection.py` (12+ tests) reproducing the advisory's own injection payload (`http://example.com/a> ... ; CLEAR ALL ; INSERT DATA { ...`) against all three backends' write and read paths, asserting the malicious query is never built or sent. Full triplet_store suite: 330+ tests passing
- Side note, not part of this fix: found that `jena_store.py`'s `get_triplets()` builds syntactically invalid SPARQL for its WHERE-clause filters (missing a `FILTER()`/separator before the equality conditions) — a pre-existing correctness bug, unrelated to the injection fix, left alone here and worth a separate follow-up
- **Cypher injection via unvalidated node labels, relationship types, and property keys** (#910, GHSA-482h-hw99-h62p) by @KaifAhmad1
- Node labels and property keys passed to `create_node`/`create_relationship` were interpolated directly into Cypher strings in the Neptune, Neo4j, and FalkorDB graph stores. Property *values* are parameterized, but labels and keys can't be bound as query parameters, and nothing validated them — so a document-derived entity type or property name (the normal ingest path) could close the current Cypher token early and append arbitrary statements (e.g. `DETACH DELETE`), running with the application's database credentials
- New shared `semantica/graph_store/query_sanitize.py`: `sanitize_identifier()` generalizes `age_store.py`'s existing `_sanitize_label`/`_sanitize_rel_type` (the only backend that already validated this) into a helper the other backends import without an import cycle with `graph_store.py`/`methods.py`
- Applied at every label/relationship-type/property-key interpolation site in `amazon_neptune.py`, `neo4j_store.py`, `falkordb_store.py`, `graph_store.py` (`degree_centrality`'s own query builder), and `methods.py` (`update_relationship`'s own query builder) — covers `create_node`, `create_nodes`, `create_relationship`, `get_nodes`, `get_relationships`, `get_neighbors`, `shortest_path`, `update_node`, `create_index`, and all relationship-type filters across the three backends
- **Fixed along the way** (caught in review, by @Sameer6305): `depth`/`max_depth` path-length parameters are meant to be integers, but `Neo4jStore.get_neighbors()`/`shortest_path()` interpolated them into the Cypher variable-length-path syntax (`*1..{depth}`) without coercion — unlike the Neptune/FalkorDB equivalents, which already cast to `int()`. A string `depth` (e.g. `"1]->(x) DETACH DELETE x //"`) reached the query verbatim. Added the same `int()` coercion Neptune/FalkorDB already had, plus `GraphStore.get_neighbors()`'s `hops`/`depth` alias resolution
- New `tests/graph_store/test_cypher_injection.py` (unit tests on `sanitize_identifier` plus the labels/keys/rel-types injection payload run against Neptune/Neo4j/FalkorDB `create_node`/`create_relationship`, asserting the malicious query is never built or sent) and the depth-coercion regression above; plus additions to `tests/test_graph_store.py` (`degree_centrality`) and `tests/test_graph_store_methods.py` (`update_relationship`). Full graph_store suite: 224+ tests passing
- **4 critical/high vulnerabilities in the Explorer API and vector store: RCE, SSRF, XXE, and DoS, plus Cypher/SPARQL injection hardening found along the way** (#898) by @Sunil56224972
- **[CWE-502] Arbitrary code execution via `pickle.load()`**: `VectorStore.save()`/`load()` used `pickle` for the on-disk `store_data.pkl`; a crafted `.pkl` file placed in the store directory (file upload, shared filesystem, or supply-chain compromise) could execute arbitrary code on deserialization. Replaced with JSON — vectors and metadata are fully JSON-serializable, so nothing is lost — and `load()` now refuses any legacy `.pkl` file it finds with a migration error rather than deserializing it
- **[CWE-918] SSRF via redirect bypass in `ontology.py`'s URL fetcher**: `_validate_fetch_url()` correctly blocked private/loopback/reserved addresses on the caller-supplied URL, but `_fetch_url_sync()` fetched with `allow_redirects=True`, so a validated *public* first hop could 302 to `http://169.254.169.254/...` (cloud instance metadata) or an internal service, and `requests` followed it with no re-check. Redirects are now followed manually, capped at 5 hops, with `_validate_fetch_url()` re-run against every hop's target — including relative `Location` headers, resolved via `urljoin()` before validation — and every response (redirect or final) is explicitly closed to avoid leaking connections back to the pool
- **[CWE-611] XXE injection in the RDF/XML parser**: `_safe_parse_rdf()` depended on `defusedxml` for XXE protection, but `defusedxml` wasn't declared in `pyproject.toml`'s `explorer` extra, so it was silently absent in normal installs and the code fell back to a bare warning plus unsafe parsing — a crafted RDF/XML ontology with an external entity could read arbitrary server files. Added `defusedxml>=0.7.1` to the extra, and `_safe_parse_rdf()` now fails closed: it raises rather than parsing untrusted RDF/XML if `defusedxml` isn't importable, replacing an earlier regex-based DOCTYPE-stripping fallback that was reviewed and rejected as bypassable
- **[CWE-770] DoS via unbounded SPARQL graph materialization**: `_build_rdflib_graph()` loaded up to 999,999 nodes and 999,999 edges into memory per query, and with up to 4 concurrent SPARQL requests permitted, an attacker could exhaust server memory. Added a 50,000 node/edge cap (`_SPARQL_MAX_GRAPH_NODES`); oversized graphs now return a clean error instead of attempting materialization
- **Cypher injection via Apache AGE's `graph_name` and `$$`-delimiter breakout**: `graph_name` was interpolated unvalidated into `cypher('{graph_name}', $$ ... $$)`, and raw Cypher query text containing `$$` could close AGE's dollar-quoted string delimiter early and append arbitrary SQL. `graph_name` is now validated against the same identifier allowlist `age_store.py` already used for labels/relationship types, and any query containing `$$` is rejected outright
- **SPARQL Explorer route (`/api/sparql`) hardened against comment/PREFIX-hiding bypass**: `_is_read_only_query()` now strips comments and PREFIX/BASE declarations before checking the leading keyword, and additionally scans the full query body for SPARQL Update keywords (INSERT/DELETE/DROP/LOAD/CLEAR/CREATE/COPY/MOVE/ADD) — so `SELECT ... ; DROP ALL` is now rejected by the keyword scan itself rather than relying solely on rdflib's parser
- **Fixed along the way** (maintainer follow-up, addressing automated review findings and a regression introduced across several rounds of iteration on the original fix):
- `VectorStore.save()`'s numpy handling used `list(v)` for the JSON fallback path, which produces `numpy.float32` elements that `json.dump()` can't serialize — changed to `v.tolist()`
- the SPARQL graph-size `ValueError` was raised outside `execute_sparql()`'s exception handling and surfaced as an unhandled 500 instead of a clean API error — moved inside
- every streamed `requests` response in the ontology redirect loop, including the one actually read and returned, is now closed in a `finally` block — a connection-pool leak that a rework of the redirect logic had briefly reintroduced after an earlier fix
- a later commit meant to add opt-in API-key auth (`explorer/auth.py`, gated on `EXPLORER_API_KEY`) instead **replaced and silently disabled** the `Depends(require_auth)` enforcement already merged into `main` for GHSA-j4mq-hprp-987v (Critical — unauthenticated Explorer API), removed the `/ws/graph-updates` handshake check, and — unlike `require_auth` — failed *open* (allowed all requests) whenever its key was unset. Merging that version would have silently reverted an already-fixed Critical CVE the moment this branch landed. Removed `explorer/auth.py`; restored the per-router `Depends(require_auth)` wiring and the WebSocket auth check; kept the one genuine improvement in that commit (adding `X-API-Key` to the CORS `allow_headers` list) by folding it into the existing CORS config
- the new SPARQL keyword-scan's comment-stripping regex (`#[^\n]*`) also matched the `#` inside standard RDF namespace IRIs (e.g. `.../1999/02/22-rdf-syntax-ns#`), corrupting any query with a normal `rdf:`/`rdfs:`-style `PREFIX` declaration — caught because the hardening's own bundled tests failed against two of its own cases. Fixed by only treating `#` as a comment-start at line-start or after whitespace; the companion `PREFIX`/`BASE` regex was also fixed to accept bare `BASE <...>` declarations, which have no prefix-name token between the keyword and the IRI
- New/updated regression tests: `tests/explorer/test_ontology_ssrf.py` (redirect re-validation, relative-redirect resolution, response closing, redirect-cap enforcement), `tests/test_security_regression.py` (Cypher/SPARQL injection, XXE, numpy serialization, SSRF redirect handling), plus additions to `tests/explorer/test_sparql_route.py`, `tests/vector_store/test_vector_store.py`, and `tests/explorer/test_explorer_auth.py`
- Note: the Cypher-injection hardening here is scoped to `age_store.py`'s `graph_name`/`$$` breakout, found while reviewing this PR. The broader label/property-key/relationship-type injection across the Neptune, Neo4j, and FalkorDB backends (GHSA-482h-hw99-h62p, #910) and the triplet-store SPARQL injection across Blazegraph/RDF4J/Jena (GHSA-8vgg-8mr4-r236, #911) are covered by separate, still-open PRs, as is the unauthenticated-Explorer-API fix referenced above (GHSA-j4mq-hprp-987v, #909, already merged)
- **CI/CD supply-chain hardening against mutable-tag Action compromise (LiteLLM/Trivy-class attack)** (#824) by @KaifAhmad1
- Every third-party GitHub Action across all 8 workflows is now pinned to a full commit SHA instead of a mutable tag (`@v7``@3d3c42e... # v7`), closing the exact vector used against LiteLLM in March 2026 (a compromised Trivy Action tag stole a long-lived publishing token)
- Added `verify-action-pins.yml` + `.github/scripts/verify-action-pins.sh`: a CI check that fails closed on any `uses:` reference that isn't a full SHA (catching a newly introduced mutable tag, not just auditing existing pins) and re-verifies every pin against the GitHub API on each workflow change, on push to `main`, and weekly; an unresolvable API lookup is treated as a failure rather than a silent skip
+12 -6
View File
@@ -132,7 +132,7 @@ compliant = graph.check_decision_rules({"category": "vendor_selection"}) # poli
```bash
semantica doctor
# Python 3.11.9 pass
# semantica 0.6.0 pass
# semantica 0.6.5 pass
# faiss vector store pass
# Config file pass ~/.semantica/config.yaml
```
@@ -1474,12 +1474,18 @@ For contributor / dev-server setup: **[explorer/README.md: Local Setup Guide](ex
---
## What's New in v0.6.0
## What's New in v0.6.5
- **Named-Graph Support for `JenaStore`:** Migrated onto `rdflib.Dataset(default_union=False)`, completing cross-backend named-graph parity across Blazegraph, RDF4J, and Jena; `add_triplets()` gains a `graph=` option
- **SPARQL CONSTRUCT Query Templates:** Parameterized, injection-safe `CONSTRUCT` templates extended from Blazegraph-only to RDF4J and Jena, plus pipeline integration via the `construct_template` step type
- **Databricks Connector:** `DatabricksIngestor` for Unity Catalog + Delta Lake ingestion, with PAT/OAuth M2M auth, table/query ingestion, and catalog/schema/table/lineage introspection. Install with `pip install "semantica[db-databricks]"`
- **SQLite Vector Store Backend:** `SQLiteVecStore`, a disk-backed local vector store on `sqlite-vec`'s `vec0` virtual tables, with Cosine/L2 metrics, metadata filtering, and WAL mode. Install with `pip install semantica[vectorstore-sqlite]`
**Security release — upgrading is strongly recommended.** Fixes for 5 externally-reported vulnerabilities in the Explorer API and graph/triplet store backends, plus a CodeQL-flagged ReDoS:
- **Missing authentication on all Explorer API routes** (GHSA-j4mq-hprp-987v, Critical): every route now requires `SEMANTICA_API_KEY`, fails closed (503) rather than open when unconfigured
- **SSRF via redirect bypass in ontology URL fetching** (GHSA-8c7v-62gr-hj6g, High): redirect targets are now re-validated at every hop and the connection is pinned to the validated address, closing a DNS check-then-use race
- **Cypher injection via unvalidated node labels and property keys** (GHSA-482h-hw99-h62p, Critical): Neptune, Neo4j, and FalkorDB now sanitize every label/relationship-type/property-key interpolation site
- **SPARQL injection via unvalidated triplet IRIs** (GHSA-8vgg-8mr4-r236, Critical): Blazegraph, RDF4J, and Jena now validate subject/predicate/object IRIs before interpolation
- **Missing Origin validation on the WebSocket handshake** (GHSA-4643-wpgq-w329, Moderate, anonymous-mode only): `/ws/graph-updates` now checks `Origin` against the same allowlist `CORSMiddleware` enforces for HTTP
- **Polynomial ReDoS in SPARQL query validation** (CodeQL `py/polynomial-redos`): fixed a backtracking regex in the Explorer's SPARQL route
Also includes: embedded Oxigraph backend for `TripletStore`, PROV-O trust/spec completeness for `ProvenanceManager`, and the Altair Anzo triplet store backend.
→ [Full release notes](RELEASE_NOTES.md) · [Changelog](CHANGELOG.md)
+8 -8
View File
@@ -13,33 +13,33 @@ icon: "quote-left"
<Tab title="BibTeX">
```bibtex
@software{semantica2026,
title = {Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering},
author = {Hawksight AI},
title = {Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems},
author = {Semantica},
year = {2026},
url = {https://github.com/semantica-agi/semantica},
version = {0.6.0},
version = {0.6.5},
doi = {10.5281/zenodo.XXXXXXX}
}
```
</Tab>
<Tab title="APA">
Hawksight AI. (2026). *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering* (Version 0.6.0) \[Computer software\]. https://github.com/semantica-agi/semantica
Semantica. (2026). *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems* (Version 0.6.5) \[Computer software\]. https://github.com/semantica-agi/semantica
</Tab>
<Tab title="MLA">
Hawksight AI. *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering*. Version 0.6.0, GitHub, 2026, https://github.com/semantica-agi/semantica.
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. Version 0.6.5, GitHub, 2026, https://github.com/semantica-agi/semantica.
</Tab>
<Tab title="Chicago">
Hawksight AI. *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering*. Version 0.6.0. GitHub, 2026. https://github.com/semantica-agi/semantica.
Semantica. *Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems*. Version 0.6.5. GitHub, 2026. https://github.com/semantica-agi/semantica.
</Tab>
<Tab title="IEEE">
Hawksight AI, "Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering," Version 0.6.0, GitHub, 2026. \[Online\]. Available: https://github.com/semantica-agi/semantica
Semantica, "Semantica: Graph-Native Infrastructure for Context and Accountable AI Systems," Version 0.6.5, GitHub, 2026. \[Online\]. Available: https://github.com/semantica-agi/semantica
</Tab>
</Tabs>
## Acknowledgment Text
> "This work uses Semantica (Hawksight AI, 2026), an open-source framework for semantic layer construction and knowledge engineering."
> "This work uses Semantica (2026), an open-source graph-native infrastructure framework for context and accountable AI systems, providing Context Graphs, knowledge graphs, and full decision provenance."
## Share Your Research
+1 -1
View File
@@ -17,7 +17,7 @@ icon: "circle-question"
| API key required? | Optional: pattern extraction works with no keys |
| Works with LangChain / LlamaIndex? | Yes: Semantica is a layer on top, not a replacement |
| Production-ready? | Yes: 1,000+ tests, v0.5.0 ships with 12 security fixes |
| Latest version? | **v0.6.0** (July 2026) |
| Latest version? | **v0.6.5** (August 2026) |
| Local LLMs? | Yes: Ollama via LiteLLM, HuggingFaceLLM for air-gapped |
+1 -1
View File
@@ -42,7 +42,7 @@ icon: "rocket"
Verify installation:
```python
import semantica
print(semantica.__version__) # 0.6.0
print(semantica.__version__) # 0.6.5
```
</Check>
</Step>
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "semantica"
version = "0.6.0"
version = "0.6.5"
description = "Accountability and context layer for AI agents. Context graphs, decision intelligence, full provenance tracking, and explainable reasoning engines — every AI decision traceable, every output auditable."
readme = "README.md"
license = { text = "MIT" }
+1 -1
View File
@@ -10,7 +10,7 @@ Main exports:
- Config: Configuration management
"""
__version__ = "0.6.0"
__version__ = "0.6.5"
__author__ = "Semantica Contributors"
__license__ = "MIT"
+59 -21
View File
@@ -8,16 +8,16 @@ from contextlib import asynccontextmanager
from pathlib import Path
from typing import Optional
from fastapi import FastAPI, HTTPException, Request, WebSocket, WebSocketDisconnect
from fastapi import Depends, FastAPI, HTTPException, Request, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse
from fastapi.staticfiles import StaticFiles
from .. import __version__
from ..context.context_graph import ContextGraph
from .dependencies import anonymous_access_allowed, get_expected_api_key, is_valid_api_key, require_auth
from .session import GraphSession
from .ws import ConnectionManager
from .auth import APIKeyAuthMiddleware, warn_if_unauthenticated
def _read_int_env(name: str, default: int) -> int:
@@ -98,6 +98,22 @@ def create_app(
@asynccontextmanager
async def lifespan(app: FastAPI):
import logging as _lifespan_logging
_lifespan_logger = _lifespan_logging.getLogger(__name__)
if anonymous_access_allowed():
_lifespan_logger.warning(
"Explorer is running with SEMANTICA_ALLOW_ANONYMOUS=true — "
"all API routes are unauthenticated. Do not expose this "
"process beyond localhost."
)
elif get_expected_api_key():
_lifespan_logger.info("Explorer API authentication: enabled (SEMANTICA_API_KEY set).")
else:
_lifespan_logger.warning(
"Explorer API authentication: NOT CONFIGURED. All protected "
"routes will return 503 until SEMANTICA_API_KEY is set."
)
app.state.event_loop = asyncio.get_running_loop()
app.state.ws_manager = ConnectionManager()
app.state.session = active_session
@@ -114,10 +130,10 @@ def create_app(
app.state.explorer_settings = settings
# allow_credentials lets browsers send cookies/auth headers cross-origin.
# The Explorer has no authentication, so credentials serve no purpose and
# enabling them when origins are broadened creates cross-site request risk.
# Set EXPLORER_CORS_CREDENTIALS=true explicitly to opt in (e.g. for a
# reverse-proxy setup that injects its own auth layer).
# Credentials aren't needed for the X-API-Key auth scheme below, and
# enabling them when origins are broadened creates cross-site request
# risk. Set EXPLORER_CORS_CREDENTIALS=true explicitly to opt in (e.g.
# for a reverse-proxy setup that injects its own cookie-based auth).
_allow_credentials = os.environ.get("EXPLORER_CORS_CREDENTIALS", "false").lower() == "true"
app.add_middleware(
CORSMiddleware,
@@ -128,10 +144,6 @@ def create_app(
max_age=600,
)
# API key authentication (opt-in via EXPLORER_API_KEY env var)
app.add_middleware(APIKeyAuthMiddleware)
warn_if_unauthenticated()
import logging as _logging
_logger = _logging.getLogger(__name__)
@@ -164,22 +176,48 @@ def create_app(
from .routes.temporal import router as temporal_router
from .routes.vocabulary import router as vocabulary_router
app.include_router(graph_router)
app.include_router(analytics_router)
app.include_router(decisions_router)
app.include_router(temporal_router)
app.include_router(enrich_router)
app.include_router(export_import_router)
app.include_router(annotations_router)
app.include_router(sparql_router)
app.include_router(provenance_router)
app.include_router(vocabulary_router)
app.include_router(ontology_router)
_auth = [Depends(require_auth)]
app.include_router(graph_router, dependencies=_auth)
app.include_router(analytics_router, dependencies=_auth)
app.include_router(decisions_router, dependencies=_auth)
app.include_router(temporal_router, dependencies=_auth)
app.include_router(enrich_router, dependencies=_auth)
app.include_router(export_import_router, dependencies=_auth)
app.include_router(annotations_router, dependencies=_auth)
app.include_router(sparql_router, dependencies=_auth)
app.include_router(provenance_router, dependencies=_auth)
app.include_router(vocabulary_router, dependencies=_auth)
app.include_router(ontology_router, dependencies=_auth)
_WS_MAX_MESSAGE_BYTES = 64 * 1024 # 64 KB — control messages only
@app.websocket("/ws/graph-updates")
async def websocket_endpoint(websocket: WebSocket):
# CORSMiddleware doesn't cover WebSocket handshakes (Starlette's
# CORS support only wraps HTTP), so under SEMANTICA_ALLOW_ANONYMOUS
# the key check below accepts any origin — loopback binding isn't a
# boundary against a browser, since any page the operator has open
# can still reach ws://localhost:.../ws/graph-updates directly.
# Reject a foreign Origin explicitly here, against the same
# allowlist CORSMiddleware already enforces for HTTP
# (GHSA-4643-wpgq-w329). Browsers always send Origin on a
# cross-origin WebSocket handshake; native/CLI clients omit it
# entirely, so a missing Origin is allowed through — the browser is
# the only threat this check is closing.
origin = websocket.headers.get("origin")
allowed_origins = app.state.explorer_settings["allowed_origins"]
if origin is not None and origin not in allowed_origins:
await websocket.close(code=4403) # forbidden
return
# Browsers can't set custom headers on a WebSocket handshake, so
# accept the key via header (non-browser clients) or query param
# (browser clients), same SEMANTICA_API_KEY the REST routes check.
candidate = websocket.headers.get("x-api-key") or websocket.query_params.get("api_key")
if not is_valid_api_key(candidate):
await websocket.close(code=4401) # unauthorized
return
manager: ConnectionManager = app.state.ws_manager
await manager.connect(websocket)
await manager.send_personal(websocket, "connection_ack", {"connected": True})
-102
View File
@@ -1,102 +0,0 @@
"""
Semantica Explorer : Authentication Middleware
Provides opt-in API key authentication for all Explorer API routes.
Enable by setting the ``EXPLORER_API_KEY`` environment variable. When set,
every request to ``/api/*`` must include either:
- An ``Authorization: Bearer <key>`` header, or
- An ``X-API-Key: <key>`` header.
When ``EXPLORER_API_KEY`` is not set, authentication is disabled and the
Explorer operates in open/development mode (with a startup warning).
"""
import hmac
import logging
import os
from typing import Optional
from fastapi import HTTPException, Request, status
from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
from starlette.responses import Response
_logger = logging.getLogger(__name__)
def _get_api_key() -> Optional[str]:
"""Read the configured API key from the environment."""
return os.environ.get("EXPLORER_API_KEY")
def _extract_token(request: Request) -> Optional[str]:
"""Extract the API key from the request headers."""
# Check Authorization: Bearer <key>
auth_header = request.headers.get("Authorization", "")
if auth_header.startswith("Bearer "):
return auth_header[7:].strip()
# Check X-API-Key: <key>
api_key_header = request.headers.get("X-API-Key", "")
if api_key_header:
return api_key_header.strip()
return None
class APIKeyAuthMiddleware(BaseHTTPMiddleware):
"""
Middleware that enforces API key authentication on ``/api/*`` routes.
Skips authentication for:
- Non-API routes (static files, health checks, WebSocket, docs)
- OPTIONS requests (CORS preflight)
- When ``EXPLORER_API_KEY`` is not configured (open mode)
"""
async def dispatch(
self, request: Request, call_next: RequestResponseEndpoint
) -> Response:
api_key = _get_api_key()
# If no API key is configured, allow all requests (open mode)
if not api_key:
return await call_next(request)
# Skip authentication for non-API paths
path = request.url.path
if not path.startswith("/api/"):
return await call_next(request)
# Skip CORS preflight
if request.method == "OPTIONS":
return await call_next(request)
# Validate the token
token = _extract_token(request)
if not token:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Missing API key. Provide via 'Authorization: Bearer <key>' or 'X-API-Key: <key>' header.",
headers={"WWW-Authenticate": "Bearer"},
)
# Constant-time comparison to prevent timing attacks
if not hmac.compare_digest(token, api_key):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Invalid API key.",
)
return await call_next(request)
def warn_if_unauthenticated() -> None:
"""Log a warning at startup if no API key is configured."""
if not _get_api_key():
_logger.warning(
"EXPLORER_API_KEY is not set. The Explorer API is running WITHOUT "
"authentication. Set EXPLORER_API_KEY to enable API key protection "
"for all /api/* endpoints."
)
+151 -57
View File
@@ -978,28 +978,18 @@ def _normalize_format(fmt: Optional[str]) -> str:
return _FORMAT_ALIASES.get(lower, lower)
import threading
import urllib3.util.connection
def _validate_fetch_url(url: str) -> List[str]:
"""Reject non-HTTP(S) schemes and private/loopback/link-local targets.
if not hasattr(urllib3.util.connection, "_orig_create_connection"):
urllib3.util.connection._orig_create_connection = urllib3.util.connection.create_connection
_dns_pin_tls = threading.local()
def _patched_create_connection(address, *args, **kwargs):
host, port = address
pinned_host = getattr(_dns_pin_tls, 'pinned_host', None)
pinned_ip = getattr(_dns_pin_tls, 'pinned_ip', None)
if pinned_host and pinned_ip and host == pinned_host:
return urllib3.util.connection._orig_create_connection((pinned_ip, port), *args, **kwargs)
return urllib3.util.connection._orig_create_connection(address, *args, **kwargs)
urllib3.util.connection.create_connection = _patched_create_connection
def _validate_fetch_url(url: str) -> str:
"""Reject non-HTTP(S) schemes and private/loopback/link-local targets."""
Returns every resolved, validated IP address (deduplicated, in
resolution order) so the caller can pin the actual connection to them
(see _make_pinned_session) with fallback across all of them not just
the first since a hostname can have multiple A/AAAA records and the
first one isn't guaranteed reachable. Resolving the hostname again at
connect time would open a DNS check-then-use window (a low-TTL or
rebinding DNS answer could differ between this check and the client's
own lookup), which is what pinning to these specific addresses avoids.
"""
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
raise HTTPException(status_code=422, detail="Only http and https URLs are allowed.")
@@ -1010,7 +1000,7 @@ def _validate_fetch_url(url: str) -> str:
addrinfos = socket.getaddrinfo(hostname, None)
except socket.gaierror as exc:
raise HTTPException(status_code=422, detail=f"Cannot resolve hostname '{hostname}': {exc}") from exc
safe_ips = []
validated_ips: List[str] = []
for _family, _type, _proto, _canonname, sockaddr in addrinfos:
try:
ip = ipaddress.ip_address(sockaddr[0])
@@ -1021,56 +1011,160 @@ def _validate_fetch_url(url: str) -> str:
status_code=422,
detail="Fetching from private, loopback, or reserved network addresses is not allowed.",
)
safe_ips.append(str(ip))
if not safe_ips:
raise HTTPException(status_code=422, detail="Could not resolve to a valid IP address.")
return safe_ips[0]
if sockaddr[0] not in validated_ips:
validated_ips.append(sockaddr[0])
if not validated_ips:
raise HTTPException(status_code=422, detail=f"Cannot resolve hostname '{hostname}' to a usable address.")
return validated_ips
def _make_pinned_session(pinned_ips: List[str], url: str):
"""Build a requests.Session whose connection is pinned to pinned_ips
(tried in order, falling back on connection failure), regardless of
what url's hostname resolves to at connect time.
_validate_fetch_url() resolves and validates the hostname once; letting
the HTTP client resolve it again independently at connect time reopens
the exact gap that validation exists to close a low-TTL or rebinding
DNS answer can differ between the two lookups. This pins the pool's
connect target to the already-validated addresses directly (bypassing
DNS resolution for the connection entirely), while keeping the original
hostname as the outgoing HTTP Host header and, for HTTPS, the TLS SNI
server_hostname / assert_hostname otherwise the connection would
reach the right IP but present the wrong identity, breaking name-based
virtual hosting and (for HTTPS) certificate hostname verification.
Falls back across every validated address (not just the first) so a
hostname with multiple A/AAAA records doesn't fail outright just
because the first-returned address happens to be unreachable.
Note: urllib3's Connection.host is a property that reads/writes the
same underlying value as `_dns_host` in this version it is NOT the
separate "presented identity" field it is in some older releases, so
overriding just `_dns_host` post-construction (as an earlier version of
this fix did) actually changes the Host header too. Pinning the pool's
`host` directly and restoring the real hostname via an explicit Host
header (+ SNI params for HTTPS) is the correct mechanism here.
"""
import requests as _req
import urllib3.util.connection as _u3_connection
from urllib3.exceptions import NewConnectionError
parsed = urlparse(url)
hostname = parsed.hostname
port = parsed.port
default_port = 443 if parsed.scheme == "https" else 80
host_header = hostname if port in (None, default_port) else f"{hostname}:{port}"
class _MultiIPConnectionMixin:
"""Overrides _new_conn to fall back across every pinned IP in
order, instead of urllib3's default single-host connect."""
def _new_conn(self):
last_exc: Optional[BaseException] = None
for ip in pinned_ips:
try:
return _u3_connection.create_connection(
(ip, self.port),
self.timeout,
source_address=self.source_address,
socket_options=self.socket_options,
)
except OSError as exc:
last_exc = exc
continue
raise NewConnectionError(
self, f"Failed to establish a connection to any of {pinned_ips}: {last_exc}"
)
class _PinnedIPHTTPAdapter(_req.adapters.HTTPAdapter):
def get_connection_with_tls_context(self, request, verify, proxies=None, cert=None):
# A proxy would perform its own DNS resolution of the target
# host on this process's behalf — a resolution outside this
# process's visibility or control, so there is no client-side
# pin that closes that race. Proxies are disabled outright for
# this SSRF-sensitive fetcher (session.trust_env=False below),
# so this should be unreachable via environment proxies; fail
# closed rather than silently skip pinning if a proxy is
# somehow still configured (e.g. passed explicitly in the
# future). _validate_fetch_url's destination classification is
# a separate, always-enforced check — this only guards the
# secondary DNS-pinning hardening.
if _req.utils.select_proxy(request.url, proxies):
raise HTTPException(
status_code=502,
detail="Proxied requests are not supported for ontology URL fetching.",
)
host_params, pool_kwargs = self.build_connection_pool_key_attributes(request, verify, cert)
if host_params.get("scheme") == "https":
pool_kwargs.setdefault("assert_hostname", hostname)
pool_kwargs.setdefault("server_hostname", hostname)
host_params["host"] = pinned_ips[0]
pool = self.poolmanager.connection_from_host(**host_params, pool_kwargs=pool_kwargs)
base_connection_cls = pool.ConnectionCls
if not issubclass(base_connection_cls, _MultiIPConnectionMixin):
pool.ConnectionCls = type(
"_PinnedConnection", (_MultiIPConnectionMixin, base_connection_cls), {}
)
return pool
session = _req.Session()
# Never honor HTTP_PROXY/HTTPS_PROXY/NO_PROXY env vars for this
# SSRF-sensitive fetcher: a configured proxy would perform its own DNS
# resolution of the target host outside this process's control,
# silently reopening the DNS check-then-use race pinning exists to
# close. See _PinnedIPHTTPAdapter.get_connection_with_tls_context for
# the fail-closed backstop if a proxy is somehow still configured.
session.trust_env = False
session.headers["Host"] = host_header
adapter = _PinnedIPHTTPAdapter()
session.mount("http://", adapter)
session.mount("https://", adapter)
return session
def _fetch_url_sync(url: str) -> bytes:
import requests as _req
pinned_ips = _validate_fetch_url(url)
_MAX_REDIRECTS = 5
current_url = url
try:
for _ in range(_MAX_REDIRECTS + 1):
safe_ip = _validate_fetch_url(current_url)
_dns_pin_tls.pinned_host = urlparse(current_url).hostname
_dns_pin_tls.pinned_ip = safe_ip
session = _make_pinned_session(pinned_ips, current_url)
try:
resp = _req.get(
resp = session.get(
current_url,
headers={"Accept": "text/turtle, application/rdf+xml, application/ld+json, */*;q=0.1"},
timeout=30,
stream=True,
allow_redirects=False, # SECURITY: follow redirects manually
)
if resp.is_redirect or resp.is_permanent_redirect:
redirect_url = resp.headers.get("Location")
resp.close() # Release the streamed connection before following the redirect
if not redirect_url:
raise HTTPException(status_code=502, detail="Redirect without Location header.")
# Resolve relative redirects (e.g. /ontology.ttl) against the current URL
redirect_url = urljoin(current_url, redirect_url)
# Re-validate the redirect target to prevent SSRF via
# open-redirect to internal/cloud-metadata endpoints, and
# get fresh pins for the new host.
pinned_ips = _validate_fetch_url(redirect_url)
current_url = redirect_url
continue
try:
resp.raise_for_status()
chunks: List[bytes] = []
total = 0
for chunk in resp.iter_content(65536):
total += len(chunk)
if total > _MAX_FETCH_BYTES:
raise HTTPException(status_code=413, detail="Remote resource exceeds 20 MB limit.")
chunks.append(chunk)
return b"".join(chunks)
finally:
resp.close() # Release the streamed connection once fully read (or on error)
finally:
_dns_pin_tls.pinned_host = None
_dns_pin_tls.pinned_ip = None
if resp.is_redirect or resp.is_permanent_redirect:
redirect_url = resp.headers.get("Location")
resp.close() # Release the streamed connection before following the redirect
if not redirect_url:
raise HTTPException(status_code=502, detail="Redirect without Location header.")
# Resolve relative redirects (e.g. /ontology.ttl) against the current URL
redirect_url = urljoin(current_url, redirect_url)
# Re-validate the redirect target to prevent SSRF via
# open-redirect to internal/cloud-metadata endpoints.
_validate_fetch_url(redirect_url)
current_url = redirect_url
continue
try:
resp.raise_for_status()
chunks: List[bytes] = []
total = 0
for chunk in resp.iter_content(65536):
total += len(chunk)
if total > _MAX_FETCH_BYTES:
raise HTTPException(status_code=413, detail="Remote resource exceeds 20 MB limit.")
chunks.append(chunk)
return b"".join(chunks)
finally:
resp.close() # Release the streamed connection once fully read (or on error)
session.close()
raise HTTPException(status_code=502, detail=f"Too many redirects (max {_MAX_REDIRECTS}).")
except HTTPException:
raise
+69 -12
View File
@@ -3,11 +3,16 @@ SPARQL routes backed by an in-memory rdflib projection of the current graph.
Security contract
-----------------
* Only SELECT, ASK, CONSTRUCT, and DESCRIBE are accepted (allowlist enforced
before graph construction so rejected queries never touch the session).
* Multi-statement injections that start with an allowed keyword (e.g.
``SELECT ... ; DROP ALL``) pass the prefix check and reach rdflib, which
rejects non-SELECT/ASK/CONSTRUCT/DESCRIBE update syntax in the parser.
* Only SELECT, ASK, CONSTRUCT, and DESCRIBE are accepted, and the query
body is scanned for SPARQL Update keywords (INSERT/DELETE/DROP/LOAD/
CLEAR/CREATE/COPY/MOVE/ADD) after stripping comments and PREFIX/BASE
declarations both enforced before graph construction, so rejected
queries never touch the session. A multi-statement injection appended
after an allowed keyword (e.g. ``SELECT ... ; DROP ALL``) is caught by
the keyword scan itself, not left to rdflib's parser.
* rdflib's parser remains a second line of defense for malformed multi-
statement syntax that doesn't contain any forbidden keyword (e.g.
``SELECT ... ; ASK ...``), which SPARQL 1.1 Query doesn't permit.
* The in-memory rdflib graph is a read-only projection the live
``GraphSession`` is never mutated by this route.
"""
@@ -38,9 +43,34 @@ _FORBIDDEN_KEYWORDS = re.compile(
re.IGNORECASE,
)
# Matches SPARQL single-line comments (# ...) and PREFIX declarations
_COMMENT_LINE = re.compile(r"#[^\n]*", re.MULTILINE)
_PREFIX_DECL = re.compile(r"^\s*(?:PREFIX|BASE)\s+(?:\S+\s+)?<[^>]*>\s*", re.IGNORECASE | re.MULTILINE)
# Matches SPARQL single-line comments (# ...) and PREFIX/BASE declarations.
# The comment regex only treats '#' as a comment-starter at line-start or
# after whitespace — not mid-token — since RDF namespace IRIs commonly
# contain a literal '#' (e.g. ".../1999/02/22-rdf-syntax-ns#"), and a naive
# `#[^\n]*` would truncate every such PREFIX declaration's IRI, corrupting
# the query. BASE declarations have no prefix name between the keyword and
# the IRI (`BASE <...>`, vs. `PREFIX ex: <...>`), so the prefix-name token
# is optional.
#
# ReDoS fix (CodeQL py/polynomial-redos, issue #1897):
#
# The original pattern `<[^>]*>\s*` was vulnerable because `\s*` (which
# matches newlines) could overlap with `[^>]*` on inputs that contain no
# closing `>` (e.g. `base<!!<!<...`), forcing the engine to explore every
# possible split between the two quantifiers — O(n²) backtracking.
#
# The fix uses `<[^>\r\n]*>` for the IRI body: excluding CR and LF from
# the character class means the IRI match can never span a line boundary,
# and the disjoint trailing `[ \t]*` (horizontal whitespace only) has zero
# character-class overlap with `[^>\r\n]*`, so the engine has exactly one
# way to match. No end-of-line anchor is needed or used, which correctly
# handles both inline prologues (`PREFIX ex: <...> SELECT ...` on one line)
# and CRLF line endings (`\r\n`) without any special casing.
_COMMENT_LINE = re.compile(r"(?:^|(?<=\s))#[^\n]*", re.MULTILINE)
_PREFIX_DECL = re.compile(
r"^[ \t]*(?:PREFIX[ \t]+\S+|BASE)[ \t]*<[^>\r\n]*>[ \t]*",
re.IGNORECASE | re.MULTILINE,
)
def _is_read_only_query(query: str) -> bool:
@@ -50,11 +80,15 @@ def _is_read_only_query(query: str) -> bool:
checking the first keyword. Also rejects queries containing SPARQL Update
keywords anywhere in the body, preventing injection via embedded strings
or multi-statement tricks.
Note: callers are responsible for enforcing any input-length limit *before*
calling this function so that an oversized-query rejection can be surfaced
as a distinct, actionable error rather than the generic read-only message.
"""
# 1. Remove PREFIX/BASE declarations (do this first so # in URIs aren't mangled by comment stripping)
cleaned = _PREFIX_DECL.sub("", query)
# 2. Remove single-line comments that could hide the real query type
cleaned = _COMMENT_LINE.sub("", cleaned)
# 1. Remove single-line comments that could hide the real query type
cleaned = _COMMENT_LINE.sub("", query)
# 2. Remove PREFIX/BASE declarations
cleaned = _PREFIX_DECL.sub("", cleaned)
# 3. Strip remaining whitespace
cleaned = cleaned.strip()
@@ -144,6 +178,11 @@ _SPARQL_MAX_ROWS = 5_000 # hard cap on returned rows
_SPARQL_TIMEOUT_S = 30 # seconds before abandoning the await
_SPARQL_MAX_CONCURRENT = 4 # semaphore: max simultaneous executions
_SPARQL_MAX_GRAPH_NODES = 50_000 # cap on graph nodes/edges to prevent OOM
# Defense-in-depth against ReDoS: reject inputs longer than this before any
# regex work so that even a future regex regression is bounded. Checked in
# execute_sparql() (not inside _is_read_only_query) so the route can return
# a distinct, actionable error message rather than the generic read-only one.
_SPARQL_MAX_QUERY_LEN = 10_000 # chars
# Semaphore caps how many graph.query calls run concurrently so that
# timed-out threads (which keep running in the pool) cannot crowd out
@@ -172,6 +211,24 @@ async def execute_sparql(
req: SparqlRequest,
session: GraphSession = Depends(get_session),
):
# Resource-limit check: reject oversized queries before any regex work.
# This is intentionally a separate, earlier check from _is_read_only_query
# so clients receive a specific, actionable message rather than the generic
# read-only rejection, and operators can tune _SPARQL_MAX_QUERY_LEN without
# touching query-semantics code.
if len(req.query) > _SPARQL_MAX_QUERY_LEN:
return SparqlResponse(
columns=[],
rows=[],
total=0,
error=(
f"Query exceeds the maximum allowed length of "
f"{_SPARQL_MAX_QUERY_LEN:,} characters "
f"({len(req.query):,} received). "
f"Please shorten your query."
),
)
if not _is_read_only_query(req.query):
return SparqlResponse(
columns=[],
+16 -15
View File
@@ -47,6 +47,7 @@ from typing import Any, Dict, List, Optional, Union
from ..utils.exceptions import ProcessingError, ValidationError
from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker
from .query_sanitize import sanitize_identifier
# Optional boto3 for AWS credentials and SigV4 signing
try:
@@ -981,7 +982,8 @@ class AmazonNeptuneStore:
node_id = props_copy.pop("id", None) or self._generate_id()
use_merge = options.get("merge", True)
label_str = ":".join(labels) if labels else "Node"
label_str = ":".join(sanitize_identifier(l, "label") for l in labels) if labels else "Node"
safe_keys = [sanitize_identifier(k, "property key") for k in props_copy.keys()]
# Build parameters
params = {"node_id": str(node_id)}
@@ -990,9 +992,7 @@ class AmazonNeptuneStore:
if use_merge:
# MERGE: Return existing node if ID matches, or create new
set_parts = []
for key in props_copy.keys():
set_parts.append(f"n.{key} = ${key}")
set_parts = [f"n.{key} = ${key}" for key in safe_keys]
if set_parts:
set_clause = ", ".join(set_parts)
@@ -1005,9 +1005,7 @@ class AmazonNeptuneStore:
query = f"MERGE (n:{label_str} {{`~id`: $node_id}}) RETURN n"
else:
# CREATE: Will fail if node with same ID exists
prop_parts = ["`~id`: $node_id"]
for key in props_copy.keys():
prop_parts.append(f"{key}: ${key}")
prop_parts = ["`~id`: $node_id"] + [f"{key}: ${key}" for key in safe_keys]
prop_assignments = ", ".join(prop_parts)
query = f"CREATE (n:{label_str} {{{prop_assignments}}}) RETURN n"
@@ -1162,7 +1160,7 @@ class AmazonNeptuneStore:
# Build query
if labels:
label_str = ":".join(labels)
label_str = ":".join(sanitize_identifier(l, "label") for l in labels)
query = f"MATCH (n:{label_str})"
else:
query = "MATCH (n)"
@@ -1172,8 +1170,9 @@ class AmazonNeptuneStore:
if properties:
conditions = []
for key, value in properties.items():
param_key = f"prop_{key}"
conditions.append(f"n.{key} = ${param_key}")
safe_key = sanitize_identifier(key, "property key")
param_key = f"prop_{safe_key}"
conditions.append(f"n.{safe_key} = ${param_key}")
params[param_key] = value
query += " WHERE " + " AND ".join(conditions)
@@ -1341,15 +1340,16 @@ class AmazonNeptuneStore:
}
# Build property assignments including ~id
safe_rel_type = sanitize_identifier(rel_type, "relationship type")
prop_parts = ["`~id`: $rel_id"]
for key, value in props_copy.items():
prop_parts.append(f"{key}: ${key}")
prop_parts.append(f"{sanitize_identifier(key, 'property key')}: ${key}")
params[key] = value
prop_assignments = ", ".join(prop_parts)
query = (
f"MATCH (a), (b) WHERE id(a) = $start_id AND id(b) = $end_id "
f"CREATE (a)-[r:{rel_type} {{{prop_assignments}}}]->(b) RETURN r"
f"CREATE (a)-[r:{safe_rel_type} {{{prop_assignments}}}]->(b) RETURN r"
)
records = self._run_query(query, params)
@@ -1405,7 +1405,7 @@ class AmazonNeptuneStore:
try:
self._ensure_connected()
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
params = {}
if node_id is not None:
@@ -1564,7 +1564,8 @@ class AmazonNeptuneStore:
try:
self._ensure_connected()
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
depth = int(depth)
if direction == "out":
pattern = f"-[r{type_filter}*1..{depth}]->"
@@ -1634,7 +1635,7 @@ class AmazonNeptuneStore:
try:
self._ensure_connected()
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
# Neptune doesn't support named path patterns in shortestPath
# Use iterative depth search instead
+25 -14
View File
@@ -41,6 +41,7 @@ from typing import Any, Dict, List, Optional, Union
from ..utils.exceptions import ProcessingError, ValidationError
from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker
from .query_sanitize import sanitize_identifier
# Optional FalkorDB import
try:
@@ -330,11 +331,11 @@ class FalkorDBStore:
try:
graph = self._ensure_graph()
label_str = ":".join(labels)
label_str = ":".join(sanitize_identifier(l, "label") for l in labels)
# Build property string for Cypher
props_str = ", ".join(
f"{k}: ${k}" for k in properties.keys()
f"{sanitize_identifier(k, 'property key')}: ${k}" for k in properties.keys()
)
query = f"CREATE (n:{label_str} {{{props_str}}}) RETURN id(n) as id, n"
@@ -392,9 +393,9 @@ class FalkorDBStore:
labels = node.get("labels", [])
properties = node.get("properties", {})
label_str = ":".join(labels) if labels else "Node"
label_str = ":".join(sanitize_identifier(l, "label") for l in labels) if labels else "Node"
props_str = ", ".join(
f"{k}: ${k}" for k in properties.keys()
f"{sanitize_identifier(k, 'property key')}: ${k}" for k in properties.keys()
)
query = f"CREATE (n:{label_str} {{{props_str}}}) RETURN id(n) as id"
@@ -447,7 +448,7 @@ class FalkorDBStore:
# Build query
if labels:
label_str = ":".join(labels)
label_str = ":".join(sanitize_identifier(l, "label") for l in labels)
query = f"MATCH (n:{label_str})"
else:
query = "MATCH (n)"
@@ -456,7 +457,8 @@ class FalkorDBStore:
if properties:
conditions = []
for key in properties.keys():
conditions.append(f"n.{key} = ${key}")
safe_key = sanitize_identifier(key, "property key")
conditions.append(f"n.{safe_key} = ${safe_key}")
query += " WHERE " + " AND ".join(conditions)
query += f" RETURN id(n) as id, n, labels(n) as labels LIMIT {limit}"
@@ -504,7 +506,8 @@ class FalkorDBStore:
# Build SET clause
set_parts = []
for key in properties.keys():
set_parts.append(f"n.{key} = ${key}")
safe_key = sanitize_identifier(key, "property key")
set_parts.append(f"n.{safe_key} = ${safe_key}")
if merge:
query = f"MATCH (n) WHERE id(n) = $node_id SET {', '.join(set_parts)} RETURN id(n) as id, n, labels(n) as labels"
@@ -592,9 +595,13 @@ class FalkorDBStore:
graph = self._ensure_graph()
properties = properties or {}
safe_rel_type = sanitize_identifier(rel_type, "relationship type")
# Build property string
if properties:
props_str = ", ".join(f"{k}: ${k}" for k in properties.keys())
props_str = ", ".join(
f"{sanitize_identifier(k, 'property key')}: ${k}" for k in properties.keys()
)
props_str = f" {{{props_str}}}"
else:
props_str = ""
@@ -602,7 +609,7 @@ class FalkorDBStore:
query = f"""
MATCH (a), (b)
WHERE id(a) = $start_id AND id(b) = $end_id
CREATE (a)-[r:{rel_type}{props_str}]->(b)
CREATE (a)-[r:{safe_rel_type}{props_str}]->(b)
RETURN id(r) as id, type(r) as type
"""
@@ -656,7 +663,7 @@ class FalkorDBStore:
"""
try:
graph = self._ensure_graph()
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
if node_id is not None:
if direction == "out":
@@ -806,7 +813,8 @@ class FalkorDBStore:
"""
try:
graph = self._ensure_graph()
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
depth = int(depth)
if direction == "out":
pattern = f"-[r{type_filter}*1..{depth}]->"
@@ -860,7 +868,8 @@ class FalkorDBStore:
"""
try:
graph = self._ensure_graph()
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
max_depth = int(max_depth)
query = f"""
MATCH path = shortestPath((start)-[r{type_filter}*..{max_depth}]-(end))
@@ -929,11 +938,13 @@ class FalkorDBStore:
"""
try:
graph = self._ensure_graph()
safe_label = sanitize_identifier(label, "label")
safe_property = sanitize_identifier(property_name, "property key")
if index_type == "fulltext":
query = f"CALL db.idx.fulltext.createNodeIndex('{label}', '{property_name}')"
query = f"CALL db.idx.fulltext.createNodeIndex('{safe_label}', '{safe_property}')"
else:
query = f"CREATE INDEX FOR (n:{label}) ON (n.{property_name})"
query = f"CREATE INDEX FOR (n:{safe_label}) ON (n.{safe_property})"
graph.query(query)
self.logger.info(f"Created {index_type} index on {label}.{property_name}")
+4 -3
View File
@@ -38,6 +38,7 @@ from ..utils.exceptions import ValidationError
from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker
from .config import graph_store_config
from .query_sanitize import sanitize_identifier
class NodeManager:
@@ -393,12 +394,12 @@ class GraphAnalytics:
"""
# Build query based on direction
if labels:
label_str = ":".join(labels)
label_str = ":".join(sanitize_identifier(l, "label") for l in labels)
match = f"MATCH (n:{label_str})"
else:
match = "MATCH (n)"
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
if direction == "out":
query = f"""
@@ -756,7 +757,7 @@ class GraphStore:
**options: Additional options
"""
# Support 'hops' as alias for 'depth' for ContextRetriever compatibility
actual_depth = options.get("hops", depth)
actual_depth = int(options.get("hops", depth))
return self._manager.analytics.get_neighbors(
node_id, rel_type, direction, actual_depth, **options
)
+3 -1
View File
@@ -62,6 +62,7 @@ from typing import Any, Dict, List, Optional, Union
from .config import graph_store_config
from .graph_store import GraphAnalytics, GraphStore, NodeManager, QueryEngine, RelationshipManager
from .query_sanitize import sanitize_identifier
from .registry import method_registry
# Global store instance
@@ -357,7 +358,8 @@ def update_relationship(
# Default implementation - execute update query
store = _get_store()
set_parts = ", ".join(f"r.{k} = ${k}" for k in properties.keys())
safe_keys = [sanitize_identifier(k, "property key") for k in properties.keys()]
set_parts = ", ".join(f"r.{k} = ${k}" for k in safe_keys)
query = f"MATCH ()-[r]->() WHERE id(r) = $rel_id SET {set_parts} RETURN id(r) as id, type(r) as type, r"
params = {"rel_id": rel_id, **properties}
result = store.execute_query(query, params)
+21 -11
View File
@@ -38,6 +38,7 @@ from typing import Any, Dict, List, Optional, Union
from ..utils.exceptions import ProcessingError, ValidationError
from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker
from .query_sanitize import sanitize_identifier
# Optional Neo4j import
try:
@@ -366,7 +367,7 @@ class Neo4jStore:
)
try:
label_str = ":".join(labels)
label_str = ":".join(sanitize_identifier(l, "label") for l in labels)
query = f"CREATE (n:{label_str} $props) RETURN id(n) as id, n"
with self.get_session() as session:
@@ -424,7 +425,7 @@ class Neo4jStore:
labels = node.get("labels", [])
properties = node.get("properties", {})
label_str = ":".join(labels) if labels else "Node"
label_str = ":".join(sanitize_identifier(l, "label") for l in labels) if labels else "Node"
query = f"CREATE (n:{label_str} $props) RETURN id(n) as id, n"
result = session.run(query, {"props": properties})
@@ -505,7 +506,7 @@ class Neo4jStore:
try:
# Build query
if labels:
label_str = ":".join(labels)
label_str = ":".join(sanitize_identifier(l, "label") for l in labels)
query = f"MATCH (n:{label_str})"
else:
query = "MATCH (n)"
@@ -514,7 +515,8 @@ class Neo4jStore:
if properties:
conditions = []
for key, value in properties.items():
conditions.append(f"n.{key} = ${key}")
safe_key = sanitize_identifier(key, "property key")
conditions.append(f"n.{safe_key} = ${safe_key}")
query += " WHERE " + " AND ".join(conditions)
query += f" RETURN id(n) as id, n, labels(n) as labels LIMIT {limit}"
@@ -635,10 +637,11 @@ class Neo4jStore:
try:
properties = properties or {}
safe_rel_type = sanitize_identifier(rel_type, "relationship type")
query = f"""
MATCH (a), (b)
WHERE id(a) = $start_id AND id(b) = $end_id
CREATE (a)-[r:{rel_type} $props]->(b)
CREATE (a)-[r:{safe_rel_type} $props]->(b)
RETURN id(r) as id, type(r) as type, r
"""
@@ -696,7 +699,7 @@ class Neo4jStore:
List of matching relationships
"""
try:
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
if node_id is not None:
if direction == "out":
@@ -857,7 +860,8 @@ class Neo4jStore:
List of neighboring nodes with path information
"""
try:
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
depth = int(depth)
if direction == "out":
pattern = f"-[r{type_filter}*1..{depth}]->"
@@ -910,7 +914,8 @@ class Neo4jStore:
Shortest path information or None if not found
"""
try:
type_filter = f":{rel_type}" if rel_type else ""
type_filter = f":{sanitize_identifier(rel_type, 'relationship type')}" if rel_type else ""
max_depth = int(max_depth)
query = f"""
MATCH path = shortestPath((start)-[r{type_filter}*..{max_depth}]-(end))
@@ -975,17 +980,22 @@ class Neo4jStore:
True if index created successfully
"""
try:
index_name = options.get("index_name", f"idx_{label}_{property_name}")
safe_label = sanitize_identifier(label, "label")
safe_property = sanitize_identifier(property_name, "property key")
index_name = sanitize_identifier(
options.get("index_name", f"idx_{safe_label}_{safe_property}"),
"index name",
)
if index_type == "fulltext":
query = f"""
CREATE FULLTEXT INDEX {index_name} IF NOT EXISTS
FOR (n:{label}) ON EACH [n.{property_name}]
FOR (n:{safe_label}) ON EACH [n.{safe_property}]
"""
else:
query = f"""
CREATE INDEX {index_name} IF NOT EXISTS
FOR (n:{label}) ON (n.{property_name})
FOR (n:{safe_label}) ON (n.{safe_property})
"""
with self.get_session() as session:
+32
View File
@@ -0,0 +1,32 @@
"""
Shared identifier validation for Cypher/SPARQL query builders.
Node labels, relationship types, and property keys can't be bound as query
parameters the way values can, so any such identifier that reaches a query
string unvalidated is a direct injection point (GHSA-482h-hw99-h62p).
`age_store.py` already validates its labels/relationship types this way;
this module generalizes that pattern for reuse across the other graph
store backends without introducing an import cycle with `graph_store.py`
or `methods.py`.
"""
import re
from ..utils.exceptions import ValidationError
_IDENTIFIER_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
def sanitize_identifier(name: str, kind: str = "identifier") -> str:
"""Validate a Cypher/SPARQL label, relationship type, or property key.
Only alphanumeric/underscore identifiers starting with a letter or
underscore are allowed.
"""
if not isinstance(name, str) or not _IDENTIFIER_RE.match(name):
raise ValidationError(
f"Invalid {kind}: {name!r}. Must start with a letter or "
"underscore and contain only alphanumeric characters and "
"underscores."
)
return name
+9 -2
View File
@@ -45,8 +45,15 @@ import json
import logging
import os
import sys
from importlib.metadata import PackageNotFoundError, version
from typing import Any
try:
_SEMANTICA_VERSION = version("semantica")
except PackageNotFoundError:
# Preserve direct source-tree execution when distribution metadata is absent.
from semantica import __version__ as _SEMANTICA_VERSION
# ── logging ────────────────────────────────────────────────────────────────
_log_level = getattr(logging, os.environ.get("SEMANTICA_LOG_LEVEL", "WARNING").upper(), logging.WARNING)
logging.basicConfig(stream=sys.stderr, level=_log_level,
@@ -477,7 +484,7 @@ def _read_resource(uri: str) -> dict:
if uri == "semantica://schema/info":
return {
"name": "Semantica",
"version": "0.4.0",
"version": _SEMANTICA_VERSION,
"tools": [t["name"] for t in TOOLS],
"resources": [r["uri"] for r in RESOURCES],
}
@@ -490,7 +497,7 @@ def _read_resource(uri: str) -> dict:
SERVER_INFO = {
"name": "semantica",
"version": "0.4.0",
"version": _SEMANTICA_VERSION,
}
CAPABILITIES = {
+43 -18
View File
@@ -6,9 +6,14 @@ capturing all steps, inputs, outputs, and transformations.
Usage:
from semantica.pipeline.pipeline_provenance import PipelineWithProvenance
pipeline = PipelineWithProvenance(provenance=True)
result = pipeline.run(data)
from semantica.pipeline import PipelineBuilder
builder = PipelineBuilder()
builder.add_step("ingest", "file_ingest")
pipeline = builder.build("my_pipeline")
runner = PipelineWithProvenance(pipeline, provenance=True)
result = runner.run(data)
# Tracks all pipeline steps with complete lineage
Author: Semantica Contributors
@@ -16,26 +21,37 @@ License: MIT
"""
from typing import Optional, Any, Dict, List
from datetime import datetime
from datetime import datetime, timezone
import uuid
import time
from .pipeline_builder import Pipeline
from .execution_engine import ExecutionEngine
class PipelineWithProvenance:
"""Pipeline executor with complete provenance tracking."""
def __init__(
self,
pipeline: Pipeline,
provenance: bool = False,
agent_id: Optional[str] = None,
is_automated: bool = True,
**config,
**engine_config,
):
"""Initialize pipeline with optional provenance."""
from .pipeline import Pipeline
"""Initialize provenance-tracked pipeline runner.
Args:
pipeline: A built Pipeline instance (from PipelineBuilder.build()).
provenance: Whether to record provenance metadata.
agent_id: Identifier for the agent running the pipeline.
is_automated: Whether the execution is automated (vs. human-triggered).
**engine_config: Extra keyword arguments forwarded to ExecutionEngine.
"""
self._pipeline = pipeline
self._engine = ExecutionEngine(**engine_config)
self.provenance = provenance
self._pipeline = Pipeline(**config)
self._prov_manager = None
self._agent_id = agent_id or self.__class__.__name__
self._is_automated = is_automated
@@ -47,15 +63,24 @@ class PipelineWithProvenance:
except ImportError:
self.provenance = False
def run(self, data: Any, source: Optional[str] = None, **kwargs):
"""Run pipeline with provenance tracking."""
def run(self, data: Any = None, source: Optional[str] = None, **kwargs):
"""Run pipeline with provenance tracking.
Args:
data: Input data to feed into the pipeline.
source: Provenance source label (defaults to "pipeline_execution").
**kwargs: Extra options forwarded to ExecutionEngine.execute_pipeline().
Returns:
ExecutionResult from the engine.
"""
pipeline_id = f"pipeline_{uuid.uuid4().hex[:8]}"
start_time = time.time()
activity_started_at_time = datetime.utcnow().isoformat()
activity_started_at_time = datetime.now(timezone.utc).isoformat()
result = self._pipeline.run(data, **kwargs)
result = self._engine.execute_pipeline(self._pipeline, data=data, **kwargs)
elapsed = time.time() - start_time
activity_ended_at_time = datetime.utcnow().isoformat()
activity_ended_at_time = datetime.now(timezone.utc).isoformat()
if self.provenance and self._prov_manager:
self._prov_manager.track_entity(
@@ -69,14 +94,14 @@ class PipelineWithProvenance:
activity_started_at_time=activity_started_at_time,
activity_ended_at_time=activity_ended_at_time,
metadata={
"steps": len(self._pipeline.steps) if hasattr(self._pipeline, 'steps') else 0,
"steps": len(self._pipeline.steps),
"duration_seconds": elapsed,
"status": "completed"
"status": "completed" if result.success else "failed",
}
)
return result
def __getattr__(self, name):
return getattr(self._pipeline, name)
+7 -3
View File
@@ -399,12 +399,16 @@ response = llm.generate("What is artificial intelligence?")
```python
from semantica.pipeline.pipeline_provenance import PipelineWithProvenance
from semantica.pipeline import PipelineBuilder
# Create pipeline with provenance
pipeline = PipelineWithProvenance(provenance=True)
builder = PipelineBuilder()
builder.add_step("ingest", "file_ingest")
pipeline = builder.build("my_pipeline")
runner = PipelineWithProvenance(pipeline, provenance=True)
# Run pipeline - all steps tracked
result = pipeline.run(
result = runner.run(
data=input_data,
source="input_file.json"
)
+28 -12
View File
@@ -312,7 +312,7 @@ class BlazegraphStore:
try:
# Use SPARQL INSERT for bulk loading
graph = options.get("graph", "")
graph_clause = f"GRAPH <{graph}>" if graph else ""
graph_clause = f"GRAPH <{sparql_escaping.validate_uri(graph)}>" if graph else ""
# Build INSERT query
insert_data = self._build_insert_data(triplets)
@@ -344,8 +344,10 @@ class BlazegraphStore:
if format == "turtle":
lines = []
for triplet in triplets:
subject = sparql_escaping.validate_uri(triplet.subject)
predicate = sparql_escaping.validate_uri(triplet.predicate)
lines.append(
f"<{triplet.subject}> <{triplet.predicate}> {self._format_object_for_sparql(triplet)} ."
f"<{subject}> <{predicate}> {self._format_object_for_sparql(triplet)} ."
)
return "\n".join(lines)
else:
@@ -353,11 +355,20 @@ class BlazegraphStore:
return self._triplets_to_rdf(triplets, "turtle")
def _build_insert_data(self, triplets: List[Triplet]) -> str:
"""Build SPARQL INSERT DATA clause."""
"""Build SPARQL INSERT DATA clause.
Validates subject/predicate as safe IRIs via
sparql_escaping.validate_uri before interpolation unlike the
object (handled by _format_object_for_sparql), subject/predicate
can't be parameterized in a raw HTTP SPARQL Update POST, so an
unvalidated value is a direct injection point (GHSA-8vgg-8mr4-r236).
"""
lines = []
for triplet in triplets:
subject = sparql_escaping.validate_uri(triplet.subject)
predicate = sparql_escaping.validate_uri(triplet.predicate)
lines.append(
f"<{triplet.subject}> <{triplet.predicate}> {self._format_object_for_sparql(triplet)} ."
f"<{subject}> <{predicate}> {self._format_object_for_sparql(triplet)} ."
)
return " ".join(lines)
@@ -380,11 +391,14 @@ class BlazegraphStore:
if self._is_uri_value(obj):
if obj.startswith("<") and obj.endswith(">"):
inner = obj[1:-1]
if " " in inner or ">" in inner:
raise ValueError(f"IRI contains invalid characters: {obj!r}")
return obj
return f"<{obj}>"
# Validate the inner IRI with the same disallowed-character
# set as the unwrapped branch below — a narrower ad-hoc
# check here previously let a pre-wrapped object bypass
# validate_uri() entirely (GHSA-8vgg-8mr4-r236 follow-up).
inner = sparql_escaping.validate_uri(obj[1:-1])
return f"<{inner}>"
validated_obj = sparql_escaping.validate_uri(obj)
return f"<{validated_obj}>"
escaped = self._escape_literal(obj)
datatype = metadata.get("datatype") or metadata.get("literal_datatype")
@@ -460,9 +474,9 @@ class BlazegraphStore:
# Build SPARQL query
where_clauses = []
if subject:
where_clauses.append(f"?s = <{subject}>")
where_clauses.append(f"?s = <{sparql_escaping.validate_uri(subject)}>")
if predicate:
where_clauses.append(f"?p = <{predicate}>")
where_clauses.append(f"?p = <{sparql_escaping.validate_uri(predicate)}>")
if object:
where_clauses.append(
f"?o = {self._format_object_for_sparql(Triplet(subject='', predicate='', object=object))}"
@@ -494,8 +508,10 @@ class BlazegraphStore:
update_endpoint = self._get_update_endpoint()
subject = sparql_escaping.validate_uri(triplet.subject)
predicate = sparql_escaping.validate_uri(triplet.predicate)
query = (
f"DELETE DATA {{ <{triplet.subject}> <{triplet.predicate}> "
f"DELETE DATA {{ <{subject}> <{predicate}> "
f"{self._format_object_for_sparql(triplet)} }}"
)
+3 -3
View File
@@ -319,11 +319,11 @@ class JenaStore:
# Build SPARQL query
query_parts = []
if subject:
query_parts.append(f"?s = <{subject}>")
query_parts.append(f"?s = <{sparql_escaping.validate_uri(subject)}>")
if predicate:
query_parts.append(f"?p = <{predicate}>")
query_parts.append(f"?p = <{sparql_escaping.validate_uri(predicate)}>")
if object:
query_parts.append(f"?o = <{object}>")
query_parts.append(f"?o = <{sparql_escaping.validate_uri(object)}>")
where_clause = " ".join(query_parts) if query_parts else ""
query = f"SELECT ?s ?p ?o WHERE {{ ?s ?p ?o {where_clause} }}"
+33 -12
View File
@@ -452,11 +452,11 @@ class RDF4JStore:
# Build SPARQL query
where_clauses = []
if subject:
where_clauses.append(f"?s = <{subject}>")
where_clauses.append(f"?s = <{sparql_escaping.validate_uri(subject)}>")
if predicate:
where_clauses.append(f"?p = <{predicate}>")
where_clauses.append(f"?p = <{sparql_escaping.validate_uri(predicate)}>")
if object:
where_clauses.append(f"?o = <{object}>")
where_clauses.append(f"?o = <{sparql_escaping.validate_uri(object)}>")
where_clause = " ".join(where_clauses) if where_clauses else ""
query = f"SELECT ?s ?p ?o WHERE {{ ?s ?p ?o {where_clause} }}"
@@ -484,8 +484,18 @@ class RDF4JStore:
update_endpoint = self._get_update_endpoint()
# Use SPARQL DELETE
query = f"DELETE DATA {{ <{triplet.subject}> <{triplet.predicate}> <{triplet.object}> }}"
# Use SPARQL DELETE.
# subject/predicate must be IRIs — validate_uri enforces that and
# blocks injection through '>' or other SPARQL metacharacters.
# object can be an IRI *or* a literal, so it is routed through
# _format_object_for_ntriples (which internally calls validate_uri
# for URI-shaped values and escape_literal for strings), matching
# the same object-handling semantics used by the add path and by
# BlazegraphStore.delete_triplet (GHSA-8vgg-8mr4-r236 regression fix).
subject = sparql_escaping.validate_uri(triplet.subject)
predicate = sparql_escaping.validate_uri(triplet.predicate)
obj_str = self._format_object_for_ntriples(triplet)
query = f"DELETE DATA {{ <{subject}> <{predicate}> {obj_str} }}"
try:
response = requests.post(
@@ -529,11 +539,14 @@ class RDF4JStore:
if self._is_uri_value(obj):
if obj.startswith("<") and obj.endswith(">"):
inner = obj[1:-1]
if " " in inner or ">" in inner:
raise ValueError(f"IRI contains invalid characters: {obj!r}")
return obj
return f"<{obj}>"
# Validate the inner IRI with the same disallowed-character
# set as the unwrapped branch below — a narrower ad-hoc
# check here previously let a pre-wrapped object bypass
# validate_uri() entirely (GHSA-8vgg-8mr4-r236 follow-up).
inner = sparql_escaping.validate_uri(obj[1:-1])
return f"<{inner}>"
validated_obj = sparql_escaping.validate_uri(obj)
return f"<{validated_obj}>"
escaped = sparql_escaping.escape_literal(obj)
datatype = metadata.get("datatype") or metadata.get("literal_datatype")
@@ -550,9 +563,17 @@ class RDF4JStore:
return f'"{escaped}"'
def _triplets_to_ntriples(self, triplets: List[Triplet]) -> str:
"""Convert triplets to N-Triples format."""
"""Convert triplets to N-Triples format.
Validates subject/predicate via sparql_escaping.validate_uri: an
unvalidated value containing '>' or a newline could terminate the
current triple line early and splice extra triples into the
upload stream (GHSA-8vgg-8mr4-r236).
"""
lines = []
for triplet in triplets:
subject = sparql_escaping.validate_uri(triplet.subject)
predicate = sparql_escaping.validate_uri(triplet.predicate)
obj_str = self._format_object_for_ntriples(triplet)
lines.append(f"<{triplet.subject}> <{triplet.predicate}> {obj_str} .")
lines.append(f"<{subject}> <{predicate}> {obj_str} .")
return "\n".join(lines)
+59
View File
@@ -150,3 +150,62 @@ def test_websocket_accepts_connection_with_header_key(client, monkeypatch):
) as websocket:
ack = websocket.receive_json()
assert ack["event"] == "connection_ack"
# ---------------------------------------------------------------------------
# WebSocket Origin validation (GHSA-4643-wpgq-w329): CORSMiddleware doesn't
# cover WebSocket handshakes at all, so under SEMANTICA_ALLOW_ANONYMOUS the
# key check alone accepted a handshake from any origin — loopback binding is
# not a boundary against a browser, since any page the operator has open can
# still reach ws://localhost:.../ws/graph-updates. These pin the fix: a
# hostile Origin is refused even in anonymous mode (and even with a correct
# key), a same-origin/allowlisted Origin still works, and a missing Origin
# (native/CLI clients, which never set the header) is still allowed through.
# ---------------------------------------------------------------------------
def test_websocket_rejects_hostile_origin_under_anonymous_mode(client, monkeypatch):
monkeypatch.setenv("SEMANTICA_ALLOW_ANONYMOUS", "true")
monkeypatch.delenv("SEMANTICA_API_KEY", raising=False)
with pytest.raises(Exception):
with client.websocket_connect(
"/ws/graph-updates", headers={"Origin": "https://evil.example"}
):
pass
def test_websocket_rejects_hostile_origin_even_with_correct_key(client, monkeypatch):
"""Defense in depth: Origin is checked before the API key, so a hostile
page that somehow obtained a valid key still can't hijack the socket."""
monkeypatch.delenv("SEMANTICA_ALLOW_ANONYMOUS", raising=False)
monkeypatch.setenv("SEMANTICA_API_KEY", "correct-key")
with pytest.raises(Exception):
with client.websocket_connect(
"/ws/graph-updates",
headers={"Origin": "https://evil.example", "X-API-Key": "correct-key"},
):
pass
def test_websocket_accepts_allowlisted_origin_under_anonymous_mode(client, monkeypatch):
monkeypatch.setenv("SEMANTICA_ALLOW_ANONYMOUS", "true")
monkeypatch.delenv("SEMANTICA_API_KEY", raising=False)
with client.websocket_connect(
"/ws/graph-updates", headers={"Origin": "http://localhost:5173"}
) as websocket:
ack = websocket.receive_json()
assert ack["event"] == "connection_ack"
def test_websocket_accepts_missing_origin_under_anonymous_mode(client, monkeypatch):
"""Native/CLI clients never send an Origin header — only browsers do —
so a missing Origin must still be allowed through; the browser is the
only threat this check closes."""
monkeypatch.setenv("SEMANTICA_ALLOW_ANONYMOUS", "true")
monkeypatch.delenv("SEMANTICA_API_KEY", raising=False)
with client.websocket_connect("/ws/graph-updates") as websocket:
ack = websocket.receive_json()
assert ack["event"] == "connection_ack"
+392
View File
@@ -0,0 +1,392 @@
"""Regression tests for DNS check-then-use (TOCTOU) hardening in the
ontology URL fetcher (GHSA-8c7v-62gr-hj6g's secondary "smaller" gap).
`_validate_fetch_url` resolves and validates a hostname once; if the actual
HTTP client resolved it again independently at connect time, a low-TTL or
rebinding DNS answer could differ between the two lookups, reopening the
SSRF window the validation exists to close. `_make_pinned_session` closes
this by pinning the connection pool's `host` directly to the already-
validated IP (bypassing DNS resolution for the connection entirely), while
explicitly restoring the real hostname as the outgoing HTTP `Host` header
and, for HTTPS, the TLS SNI `server_hostname` / `assert_hostname` so the
connection reaches the pinned IP but still presents (and verifies against)
the original hostname's identity.
test_ontology_ssrf.py covers the redirect-handling logic around this with
mocks; this file proves the pinning mechanism itself works end-to-end
against real local servers, with no DNS mocking at all the test hostname
is never resolved, which is exactly the property being verified. It also
includes a negative control (mismatched cert hostname) proving TLS
verification is genuinely enforced against the real hostname, not silently
bypassed or checked against the pinned IP instead.
"""
import http.server
import socket
import threading
import pytest
from semantica.explorer.routes import ontology as ontology_mod
def _start_local_server():
captured = {}
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
captured["host_header"] = self.headers.get("Host")
body = b"pinned response"
self.send_response(200)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *_args):
pass
server = http.server.HTTPServer(("127.0.0.1", 0), Handler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
return server, thread, captured
def test_pinned_session_connects_to_pinned_ip_without_resolving_hostname():
"""A session built by _make_pinned_session must reach the pinned IP
directly. The request URL uses a hostname that cannot be resolved via
real DNS ('.invalid' is reserved by RFC 2606) if pinning weren't
working, this request would fail with a name-resolution error instead
of reaching the local server, since nothing else could route it there.
"""
server, thread, captured = _start_local_server()
port = server.server_address[1]
url = f"http://pinned-test.invalid:{port}/resource"
try:
session = ontology_mod._make_pinned_session(["127.0.0.1"], url)
try:
resp = session.get(url, timeout=5)
assert resp.status_code == 200
assert resp.content == b"pinned response"
finally:
session.close()
finally:
server.shutdown()
thread.join(timeout=2)
# Host header must still be the original hostname, not the pinned IP —
# proving connection target and presented identity are decoupled
# correctly (this is what keeps virtual hosting / TLS SNI correct).
assert captured["host_header"] == f"pinned-test.invalid:{port}"
def test_pinned_session_ignores_a_different_real_resolution():
"""Even if the hostname *does* resolve to something else via real DNS,
the pinned session must still go to the pinned IP this is the actual
TOCTOU property: the connection uses what was validated, not whatever
a fresh lookup returns. 'localhost' reliably resolves to a loopback
address, which is deliberately NOT where our test server listens on
(127.0.0.1 specifically) but since Windows/most stacks map
'localhost' to 127.0.0.1 too, use a distinct high loopback address
(127.0.0.2) for the server so a real 'localhost' resolution (127.0.0.1)
provably would NOT reach it, isolating the assertion to pinning alone.
"""
captured = {}
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
captured["host_header"] = self.headers.get("Host")
body = b"pinned via explicit ip"
self.send_response(200)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *_args):
pass
try:
server = http.server.HTTPServer(("127.0.0.2", 0), Handler)
except OSError:
# 127.0.0.2 isn't bindable in this environment (uncommon, but
# possible in some sandboxes) — skip rather than false-fail.
import pytest
pytest.skip("127.0.0.2 is not bindable in this environment")
port = server.server_address[1]
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
url = f"http://localhost:{port}/resource"
try:
session = ontology_mod._make_pinned_session(["127.0.0.2"], url)
try:
resp = session.get(url, timeout=5)
assert resp.status_code == 200
assert resp.content == b"pinned via explicit ip"
finally:
session.close()
finally:
server.shutdown()
thread.join(timeout=2)
assert captured["host_header"] == f"localhost:{port}"
def test_pinned_session_falls_back_across_multiple_pinned_ips():
"""A hostname can have multiple A/AAAA records; pinning to only the
first-returned address means a fetch fails outright if that specific
address happens to be unreachable even though a later one would work.
_make_pinned_session must fall back through every pinned IP in order.
"""
server, thread, captured = _start_local_server()
port = server.server_address[1]
url = f"http://pinned-test.invalid:{port}/resource"
# 127.0.0.3 has nothing listening on this port — connection refused,
# forcing a fallback to the second (real) address.
unreachable_ip = "127.0.0.3"
try:
session = ontology_mod._make_pinned_session([unreachable_ip, "127.0.0.1"], url)
try:
resp = session.get(url, timeout=5)
assert resp.status_code == 200
assert resp.content == b"pinned response"
finally:
session.close()
finally:
server.shutdown()
thread.join(timeout=2)
def test_pinned_session_raises_when_every_pinned_ip_is_unreachable():
"""If none of the pinned IPs are reachable, the session must raise
rather than silently falling back to resolving the hostname itself
(which would reopen the exact TOCTOU window pinning exists to close)."""
import requests
url = "http://pinned-test.invalid:9/resource" # port 9 (discard) — nothing listens
session = ontology_mod._make_pinned_session(["127.0.0.3", "127.0.0.4"], url)
try:
with pytest.raises(requests.exceptions.ConnectionError):
session.get(url, timeout=5)
finally:
session.close()
def test_pinned_session_disables_environment_proxy_trust():
"""A pinned session must never honor HTTP_PROXY/HTTPS_PROXY env vars —
a proxy would perform its own DNS resolution of the target host outside
this process's control, reopening the exact TOCTOU window pinning
exists to close."""
session = ontology_mod._make_pinned_session(["127.0.0.1"], "http://example.org/")
try:
assert session.trust_env is False
finally:
session.close()
def test_pinned_session_ignores_env_proxy_and_connects_directly(monkeypatch):
"""End-to-end: even with HTTP_PROXY pointed at an address that would
fail if contacted, a pinned session must reach the real local server
directly proving the env var is genuinely not consulted, not just
that the trust_env flag is set."""
monkeypatch.setenv("HTTP_PROXY", "http://127.0.0.5:1/") # would fail if ever used
server, thread, _captured = _start_local_server()
port = server.server_address[1]
url = f"http://pinned-test.invalid:{port}/resource"
try:
session = ontology_mod._make_pinned_session(["127.0.0.1"], url)
try:
resp = session.get(url, timeout=5)
assert resp.status_code == 200
assert resp.content == b"pinned response"
finally:
session.close()
finally:
server.shutdown()
thread.join(timeout=2)
def test_pinned_session_fails_closed_if_a_proxy_is_explicitly_forced():
"""Backstop: if a proxy is somehow still configured on the session
despite trust_env=False (e.g. set explicitly, as a future code path
might), the adapter must fail closed with a clear error rather than
silently connecting through the proxy unpinned."""
from fastapi import HTTPException
session = ontology_mod._make_pinned_session(["127.0.0.1"], "http://example.org/")
session.proxies = {"http": "http://127.0.0.5:1"}
try:
with pytest.raises(HTTPException) as exc_info:
session.get("http://example.org/", timeout=5)
assert exc_info.value.status_code == 502
finally:
session.close()
def test_validate_fetch_url_returns_the_resolved_ip():
"""_validate_fetch_url must return every IP it validated, so callers can
pin the connection to them (with fallback across all of them)."""
def fake_getaddrinfo(host, *_a, **_k):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("93.184.216.34", 0))]
import unittest.mock as mock
with mock.patch.object(ontology_mod.socket, "getaddrinfo", side_effect=fake_getaddrinfo):
resolved_ips = ontology_mod._validate_fetch_url("http://example.org/ontology.ttl")
assert resolved_ips == ["93.184.216.34"]
def test_validate_fetch_url_returns_all_validated_ips_deduplicated():
"""A hostname with multiple A/AAAA records must return every distinct
validated address, in resolution order, so the caller can fall back
across all of them rather than failing if only the first is
unreachable."""
def fake_getaddrinfo(host, *_a, **_k):
return [
(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("93.184.216.34", 0)),
(socket.AF_INET, socket.SOCK_DGRAM, 17, "", ("93.184.216.34", 0)), # duplicate, different socktype
(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("93.184.216.35", 0)),
]
import unittest.mock as mock
with mock.patch.object(ontology_mod.socket, "getaddrinfo", side_effect=fake_getaddrinfo):
resolved_ips = ontology_mod._validate_fetch_url("http://example.org/ontology.ttl")
assert resolved_ips == ["93.184.216.34", "93.184.216.35"]
def test_validate_fetch_url_still_rejects_private_ip():
"""Confirm the pinning refactor didn't loosen the original address
classification a hostname resolving to a private/internal address
must still be rejected before any IP is returned."""
import ipaddress
import unittest.mock as mock
import pytest
from fastapi import HTTPException
def fake_getaddrinfo(host, *_a, **_k):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("169.254.169.254", 0))]
with mock.patch.object(ontology_mod.socket, "getaddrinfo", side_effect=fake_getaddrinfo):
with pytest.raises(HTTPException) as exc_info:
ontology_mod._validate_fetch_url("http://attacker.example/ontology.ttl")
assert exc_info.value.status_code == 422
# ---------------------------------------------------------------------------
# HTTPS: SNI + certificate hostname verification must use the real hostname,
# not the pinned IP — this is the highest-risk part of pinning to get wrong,
# since a mistake here could silently weaken TLS verification rather than
# just breaking connectivity. Requires the optional `cryptography` package
# to mint a throwaway self-signed cert; skipped gracefully without it.
# ---------------------------------------------------------------------------
def _make_self_signed_cert(hostname: str, tmp_path):
import datetime
pytest.importorskip("cryptography")
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.x509.oid import NameOID
key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, hostname)])
now = datetime.datetime.now(datetime.timezone.utc)
cert = (
x509.CertificateBuilder()
.subject_name(name)
.issuer_name(name)
.public_key(key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now - datetime.timedelta(days=1))
.not_valid_after(now + datetime.timedelta(days=1))
.add_extension(x509.SubjectAlternativeName([x509.DNSName(hostname)]), critical=False)
.sign(key, hashes.SHA256())
)
cert_path = tmp_path / "cert.pem"
key_path = tmp_path / "key.pem"
cert_path.write_bytes(cert.public_bytes(serialization.Encoding.PEM))
key_path.write_bytes(
key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.TraditionalOpenSSL,
encryption_algorithm=serialization.NoEncryption(),
)
)
return str(cert_path), str(key_path)
def _start_local_https_server(cert_path, key_path):
import ssl
captured = {}
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
captured["host_header"] = self.headers.get("Host")
body = b"tls pinned response"
self.send_response(200)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *_args):
pass
server = http.server.HTTPServer(("127.0.0.1", 0), Handler)
ssl_ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ssl_ctx.minimum_version = ssl.TLSVersion.TLSv1_2
ssl_ctx.load_cert_chain(cert_path, key_path)
server.socket = ssl_ctx.wrap_socket(server.socket, server_side=True)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
return server, thread, captured
def test_pinned_https_session_verifies_against_real_hostname_not_pinned_ip(tmp_path):
"""A pinned HTTPS connection must present + verify SNI/cert against the
real hostname, even though the socket connects to the pinned IP. The
cert's SAN is the hostname, never '127.0.0.1' — if pinning verified
against the IP instead (or against nothing), this would either fail
for the wrong reason or silently succeed with no real verification."""
cert_path, key_path = _make_self_signed_cert("pinned-tls-test.invalid", tmp_path)
server, thread, captured = _start_local_https_server(cert_path, key_path)
port = server.server_address[1]
url = f"https://pinned-tls-test.invalid:{port}/resource"
try:
session = ontology_mod._make_pinned_session(["127.0.0.1"], url)
try:
resp = session.get(url, timeout=5, verify=cert_path)
finally:
session.close()
finally:
server.shutdown()
thread.join(timeout=2)
assert resp.status_code == 200
assert resp.content == b"tls pinned response"
assert captured["host_header"] == f"pinned-tls-test.invalid:{port}"
def test_pinned_https_session_rejects_hostname_mismatch(tmp_path):
"""Negative control: requesting a hostname that does NOT match the
cert's SAN must still fail verification — proving pinning doesn't
silently bypass or misdirect certificate hostname checking."""
cert_path, key_path = _make_self_signed_cert("pinned-tls-test.invalid", tmp_path)
server, thread, _captured = _start_local_https_server(cert_path, key_path)
port = server.server_address[1]
url = f"https://wrong-name.invalid:{port}/resource"
try:
session = ontology_mod._make_pinned_session(["127.0.0.1"], url)
try:
import requests
with pytest.raises(requests.exceptions.SSLError):
session.get(url, timeout=5, verify=cert_path)
finally:
session.close()
finally:
server.shutdown()
thread.join(timeout=2)
+26 -6
View File
@@ -3,12 +3,17 @@
`_fetch_url_sync` disables `requests`' automatic redirect following and
re-validates every hop with `_validate_fetch_url` (see GHSA-8c7v-62gr-hj6g:
unvalidated redirect targets previously let a public first hop 302 the
server into fetching cloud metadata / loopback services).
server into fetching cloud metadata / loopback services). It also pins each
hop's connection to the IP `_validate_fetch_url` already resolved and
validated, via `_make_pinned_session`, closing the DNS check-then-use gap
between that validation and the client's own (potentially different) lookup.
These tests cover the redirect-handling logic itself: relative `Location`
headers must resolve correctly instead of being rejected outright, redirect
targets that resolve to private/loopback addresses must still be blocked,
and every response must be closed (no leaked connections across hops).
`test_ontology_dns_pinning.py` covers the pinning mechanism
(`_make_pinned_session`, `_validate_fetch_url`'s returned IP) directly.
"""
import socket
@@ -37,6 +42,17 @@ def _make_response(is_redirect=False, is_permanent=False, location=None, body=b"
return resp
def _patch_session(responses):
"""Patch _make_pinned_session so _fetch_url_sync's session.get(...)
calls return the given responses in order, without touching the real
requests.Session/pinning machinery (that's covered by test_pinning.py).
"""
fake_session = MagicMock()
fake_session.get = MagicMock(side_effect=responses)
fake_session.close = MagicMock()
return patch.object(ontology_mod, "_make_pinned_session", return_value=fake_session), fake_session
@patch.object(ontology_mod.socket, "getaddrinfo", side_effect=_fake_getaddrinfo)
def test_relative_redirect_location_is_resolved(mock_getaddrinfo):
"""A relative Location header (e.g. '/ontology.ttl') must resolve against
@@ -44,11 +60,12 @@ def test_relative_redirect_location_is_resolved(mock_getaddrinfo):
redirect_resp = _make_response(is_redirect=True, location="/ontology.ttl")
final_resp = _make_response(body=b"final content")
with patch("requests.get", side_effect=[redirect_resp, final_resp]) as mock_get:
patcher, fake_session = _patch_session([redirect_resp, final_resp])
with patcher:
result = ontology_mod._fetch_url_sync("http://example.org/start")
assert result == b"final content"
second_call_url = mock_get.call_args_list[1].args[0]
second_call_url = fake_session.get.call_args_list[1].args[0]
assert second_call_url == "http://example.org/ontology.ttl"
redirect_resp.close.assert_called_once()
final_resp.close.assert_called_once()
@@ -67,7 +84,8 @@ def test_redirect_to_private_ip_is_rejected(mock_getaddrinfo):
mock_getaddrinfo.side_effect = getaddrinfo_side_effect
redirect_resp = _make_response(is_redirect=True, location="http://internal.example/latest/meta-data/")
with patch("requests.get", side_effect=[redirect_resp]):
patcher, fake_session = _patch_session([redirect_resp])
with patcher:
with pytest.raises(ontology_mod.HTTPException) as exc_info:
ontology_mod._fetch_url_sync("http://example.org/start")
@@ -78,7 +96,8 @@ def test_redirect_to_private_ip_is_rejected(mock_getaddrinfo):
@patch.object(ontology_mod.socket, "getaddrinfo", side_effect=_fake_getaddrinfo)
def test_final_response_is_closed(mock_getaddrinfo):
final_resp = _make_response(body=b"content")
with patch("requests.get", side_effect=[final_resp]):
patcher, _fake_session = _patch_session([final_resp])
with patcher:
ontology_mod._fetch_url_sync("http://example.org/start")
final_resp.close.assert_called_once()
@@ -86,7 +105,8 @@ def test_final_response_is_closed(mock_getaddrinfo):
@patch.object(ontology_mod.socket, "getaddrinfo", side_effect=_fake_getaddrinfo)
def test_redirect_chain_exceeding_cap_is_rejected(mock_getaddrinfo):
responses = [_make_response(is_redirect=True, location=f"/hop{i}") for i in range(10)]
with patch("requests.get", side_effect=responses):
patcher, _fake_session = _patch_session(responses)
with patcher:
with pytest.raises(ontology_mod.HTTPException) as exc_info:
ontology_mod._fetch_url_sync("http://example.org/start")
assert exc_info.value.status_code == 502
+59 -5
View File
@@ -199,14 +199,27 @@ def test_allowlist_rejected_query_never_touches_the_graph(client, query):
mock_build.assert_not_called()
def test_multi_statement_injection_reaches_graph_but_fails_in_parser(client):
"""Confirms the distinction between allowlist rejection and parser rejection:
a string starting with SELECT passes _is_read_only_query and builds a graph,
but rdflib.Graph.query() rejects the trailing '; DROP ALL' syntax."""
def test_multi_statement_injection_is_rejected_by_forbidden_keyword_check(client):
"""A string starting with an allowed keyword (SELECT) but containing a
forbidden Update keyword later in the body ('; DROP ALL') is now
rejected by _is_read_only_query's keyword scan itself, before a graph
is ever built a stronger, earlier rejection than relying solely on
rdflib's parser to reject the syntax."""
with patch.object(sparql_mod, "_build_rdflib_graph") as mock_build:
resp = _post(client, "SELECT ?s WHERE { ?s ?p ?o } ; DROP ALL")
assert resp.status_code == 200
assert resp.json()["error"] is not None
mock_build.assert_not_called()
def test_malformed_syntax_without_forbidden_keywords_still_fails_in_parser(client):
"""The parser remains a real second line of defense for malformed
queries that don't contain any forbidden keyword — these pass
_is_read_only_query and reach rdflib, which rejects the syntax."""
with patch.object(
sparql_mod, "_build_rdflib_graph", wraps=sparql_mod._build_rdflib_graph
) as spy_build:
resp = _post(client, "SELECT ?s WHERE { ?s ?p ?o } ; DROP ALL")
resp = _post(client, "SELECT ?s WHERE { ?s ?p ?o } ; ASK { ?x ?y ?z }")
assert resp.status_code == 200
assert resp.json()["error"] is not None
spy_build.assert_called_once()
@@ -304,6 +317,47 @@ def test_oversized_graph_returns_clean_error_not_a_crash(client):
assert payload["rows"] == []
def test_oversized_query_returns_distinct_length_error(client):
"""A query exceeding _SPARQL_MAX_QUERY_LEN must be rejected with a
specific, actionable error message not the generic read-only message.
Clients need to distinguish a size-limit rejection from an actual
non-read-only query rejection to react correctly (e.g. split the query
vs. rewrite it)."""
with patch.object(sparql_mod, "_SPARQL_MAX_QUERY_LEN", 10):
resp = _post(client, "SELECT ?s WHERE { ?s ?p ?o }") # 30 chars > 10
assert resp.status_code == 200
payload = resp.json()
assert payload["error"] is not None
# Must mention the limit, not the generic read-only message
assert "length" in payload["error"].lower() or "characters" in payload["error"].lower()
assert "Only SELECT" not in payload["error"]
assert payload["rows"] == []
assert payload["columns"] == []
assert payload["total"] == 0
def test_oversized_query_never_touches_the_graph(client):
"""An oversized query must be rejected before _build_rdflib_graph is
called the length guard must short-circuit the entire pipeline."""
with patch.object(sparql_mod, "_SPARQL_MAX_QUERY_LEN", 10):
with patch.object(sparql_mod, "_build_rdflib_graph") as mock_build:
resp = _post(client, "SELECT ?s WHERE { ?s ?p ?o }")
assert resp.status_code == 200
assert resp.json()["error"] is not None
mock_build.assert_not_called()
def test_query_exactly_at_length_limit_is_accepted(client):
"""A query whose length equals the limit exactly must not be rejected —
the guard is strictly greater-than, not greater-than-or-equal."""
short_query = "ASK {}"
with patch.object(sparql_mod, "_SPARQL_MAX_QUERY_LEN", len(short_query)):
resp = _post(client, short_query)
assert resp.status_code == 200
payload = resp.json()
assert payload["error"] is None
# ---------------------------------------------------------------------------
# Data-mapping fidelity: does the graph->RDF projection reflect session state?
# ---------------------------------------------------------------------------
+396
View File
@@ -0,0 +1,396 @@
"""Regression tests for GHSA-482h-hw99-h62p: unvalidated node labels and
property keys allowed arbitrary Cypher injection in the Neptune, Neo4j, and
FalkorDB graph stores (labels/keys can't be bound as query parameters, so
an unvalidated value reaching the query string is a direct injection
point).
Mirrors the advisory's own PoC shape: a label/key crafted to close the
current Cypher token early and append a destructive statement
(`DETACH DELETE victim`). Before the fix, these reached `_run_query` /
`session.run` / `graph.query` verbatim. After the fix, `sanitize_identifier`
(graph_store/query_sanitize.py) rejects them with ValidationError before
any query is built, matching the existing age_store.py `_sanitize_label`
behavior used as the reference implementation.
"""
import pytest
import unittest
from unittest.mock import MagicMock
from semantica.graph_store.query_sanitize import sanitize_identifier
from semantica.utils.exceptions import ProcessingError, ValidationError
# AmazonNeptuneStore/Neo4jStore/FalkorDBStore.create_node() wrap their whole
# body in `except Exception: raise ProcessingError(...)` (pre-existing,
# unrelated to this fix), so the ValidationError sanitize_identifier raises
# surfaces to callers as ProcessingError. Either way the malicious query is
# never built or sent — these tests assert exactly that via the query-capture
# stubs, and check the wrapped message to confirm it's the sanitizer firing.
EVIL_LABEL = "N}) MATCH (victim) DETACH DELETE victim //"
EVIL_KEY = "k1`: 1}) MATCH (victim) DETACH DELETE victim //"
def _wire(store):
store.logger = MagicMock()
store.progress_tracker = MagicMock()
store.progress_tracker.start_tracking.return_value = "tid"
store.config = {}
return store
class TestSanitizeIdentifier(unittest.TestCase):
def test_valid_identifiers_pass_through_unchanged(self):
self.assertEqual(sanitize_identifier("Person"), "Person")
self.assertEqual(sanitize_identifier("_hidden"), "_hidden")
self.assertEqual(sanitize_identifier("Rel_Type2"), "Rel_Type2")
def test_injection_payload_is_rejected(self):
with self.assertRaises(ValidationError):
sanitize_identifier(EVIL_LABEL)
def test_property_key_injection_payload_is_rejected(self):
with self.assertRaises(ValidationError):
sanitize_identifier(EVIL_KEY)
def test_rejects_non_string(self):
with self.assertRaises(ValidationError):
sanitize_identifier(123) # type: ignore[arg-type]
def test_rejects_spaces_and_dashes(self):
with self.assertRaises(ValidationError):
sanitize_identifier("no spaces")
with self.assertRaises(ValidationError):
sanitize_identifier("no-dashes")
class TestAmazonNeptuneCypherInjection(unittest.TestCase):
def _make_store(self):
from semantica.graph_store.amazon_neptune import AmazonNeptuneStore
store = _wire(AmazonNeptuneStore.__new__(AmazonNeptuneStore))
store._connected = True
store._ensure_connected = lambda: None
store._generate_id = lambda: "generated-id"
store._run_query = MagicMock(return_value=[])
store._parse_results = lambda r: []
return store
def test_create_node_rejects_malicious_label_before_querying(self):
store = self._make_store()
with self.assertRaises(ProcessingError) as ctx:
store.create_node(labels=[EVIL_LABEL], properties={"name": "x"})
self.assertIn("Invalid label", str(ctx.exception))
store._run_query.assert_not_called()
def test_create_node_rejects_malicious_property_key_before_querying(self):
store = self._make_store()
with self.assertRaises(ProcessingError) as ctx:
store.create_node(labels=["Person"], properties={"name": "x", EVIL_KEY: 1})
self.assertIn("Invalid property key", str(ctx.exception))
store._run_query.assert_not_called()
def test_create_node_with_legitimate_labels_still_works(self):
store = self._make_store()
store.create_node(labels=["Person", "Employee"], properties={"name": "Alice"})
query = store._run_query.call_args[0][0]
self.assertIn("Person:Employee", query)
self.assertNotIn("DETACH DELETE", query)
class TestNeo4jCypherInjection(unittest.TestCase):
def _make_store(self):
from semantica.graph_store import neo4j_store as m
store = _wire(m.Neo4jStore.__new__(m.Neo4jStore))
captured = {}
class Session:
def run(self, q, params=None):
captured["query"] = q
rec = {"n": {"name": "x"}, "id": 1}
return type("R", (), {"single": lambda self: rec})()
def __enter__(self):
return self
def __exit__(self, *a):
return False
store.get_session = lambda: Session()
store._captured = captured
return store
def test_create_node_rejects_malicious_label_before_querying(self):
store = self._make_store()
with self.assertRaises(ProcessingError) as ctx:
store.create_node(labels=["Person", EVIL_LABEL], properties={"name": "x"})
self.assertIn("Invalid label", str(ctx.exception))
self.assertNotIn("query", store._captured)
def test_create_relationship_rejects_malicious_rel_type(self):
store = self._make_store()
with self.assertRaises(ProcessingError) as ctx:
store.create_relationship(start_node_id=1, end_node_id=2, rel_type=EVIL_LABEL)
self.assertIn("Invalid relationship type", str(ctx.exception))
self.assertNotIn("query", store._captured)
class TestFalkorDBCypherInjection(unittest.TestCase):
def _make_store(self):
from semantica.graph_store import falkordb_store as m
store = _wire(m.FalkorDBStore.__new__(m.FalkorDBStore))
captured = {}
class Graph:
def query(self, q, params=None):
captured["query"] = q
return type("R", (), {"result_set": []})()
store._ensure_graph = lambda: Graph()
store._captured = captured
return store
def test_create_node_rejects_malicious_label_and_key(self):
store = self._make_store()
with self.assertRaises(ProcessingError) as ctx:
store.create_node(labels=[EVIL_LABEL], properties={"name": "x", EVIL_KEY: 1})
self.assertIn("Invalid label", str(ctx.exception))
self.assertNotIn("query", store._captured)
def test_create_node_with_legitimate_input_still_works(self):
store = self._make_store()
store.create_node(labels=["Person"], properties={"name": "Alice"})
query = store._captured["query"]
self.assertIn("Person", query)
self.assertNotIn("DETACH DELETE", query)
# ---------------------------------------------------------------------------
# BLOCKER 1 regression: Neo4jStore.get_neighbors / shortest_path depth coercion
# ---------------------------------------------------------------------------
# Payload representative of the confirmed injection (review BLOCKER 1):
# supplying a string as `depth` / `max_depth` previously reached the Cypher
# f-string verbatim because Neo4jStore did not call int() like Neptune/FalkorDB.
#
# After the fix `depth = int(depth)` / `max_depth = int(max_depth)` are added
# at the top of each method's try-block. A malicious string raises ValueError
# (wrapped in ProcessingError), and the session.run / _run_query mock must
# never be called.
EVIL_DEPTH = "1]->(x) DETACH DELETE x //"
class TestNeo4jDepthInjection(unittest.TestCase):
"""Regression tests for BLOCKER 1: Neo4jStore depth/max_depth coercion."""
def _make_store(self):
"""Build a Neo4jStore with a session that records every query string sent to it."""
from semantica.graph_store import neo4j_store as m
store = _wire(m.Neo4jStore.__new__(m.Neo4jStore))
captured = {}
class IterSession:
"""Returns an empty iterator so get_neighbors' for-loop completes cleanly."""
def run(self, q, params=None):
captured["query"] = q
return iter([])
def __enter__(self):
return self
def __exit__(self, *a):
return False
class SingleSession:
"""Returns a FakeResult whose .single() yields None (shortest_path)."""
def run(self, q, params=None):
captured["query"] = q
return type("R", (), {"single": lambda self: None})()
def __enter__(self):
return self
def __exit__(self, *a):
return False
store._captured = captured
store._iter_session = IterSession
store._single_session = SingleSession
return store
# -- get_neighbors --------------------------------------------------------
def test_get_neighbors_malicious_depth_raises_before_query(self):
"""Malicious string depth must never reach session.run."""
store = self._make_store()
store.get_session = lambda: store._iter_session()
with self.assertRaises(ProcessingError):
store.get_neighbors(node_id=1, depth=EVIL_DEPTH)
self.assertNotIn("query", store._captured,
"session.run was called — injected query reached the database layer")
def test_get_neighbors_malicious_depth_does_not_contain_payload(self):
"""Double-check: if somehow a query were built, it must not contain the payload."""
store = self._make_store()
store.get_session = lambda: store._iter_session()
with pytest.raises(ProcessingError):
store.get_neighbors(node_id=1, depth=EVIL_DEPTH)
query = store._captured.get("query", "")
self.assertNotIn("DETACH DELETE", query,
f"Injection payload found in query: {query!r}")
def test_get_neighbors_legitimate_depth_works(self):
"""Valid integer depth must still produce a correct traversal pattern."""
store = self._make_store()
store.get_session = lambda: store._iter_session()
result = store.get_neighbors(node_id=1, depth=2)
self.assertIsInstance(result, list)
self.assertIn("query", store._captured)
self.assertIn("*1..2", store._captured["query"])
self.assertNotIn("DETACH DELETE", store._captured["query"])
def test_get_neighbors_depth_string_int_is_coerced(self):
"""A string representation of a valid integer must be coerced and work."""
store = self._make_store()
store.get_session = lambda: store._iter_session()
result = store.get_neighbors(node_id=1, depth="3")
self.assertIsInstance(result, list)
self.assertIn("*1..3", store._captured["query"])
# -- shortest_path --------------------------------------------------------
def test_shortest_path_malicious_max_depth_raises_before_query(self):
"""Malicious string max_depth must never reach session.run."""
store = self._make_store()
store.get_session = lambda: store._single_session()
with self.assertRaises(ProcessingError):
store.shortest_path(start_node_id=1, end_node_id=2, max_depth=EVIL_DEPTH)
self.assertNotIn("query", store._captured,
"session.run was called — injected query reached the database layer")
def test_shortest_path_malicious_max_depth_does_not_contain_payload(self):
store = self._make_store()
store.get_session = lambda: store._single_session()
with pytest.raises(ProcessingError):
store.shortest_path(start_node_id=1, end_node_id=2, max_depth=EVIL_DEPTH)
query = store._captured.get("query", "")
self.assertNotIn("DETACH DELETE", query,
f"Injection payload found in query: {query!r}")
def test_shortest_path_legitimate_max_depth_works(self):
"""Valid integer max_depth must produce a correct shortestPath pattern."""
store = self._make_store()
store.get_session = lambda: store._single_session()
result = store.shortest_path(start_node_id=1, end_node_id=2, max_depth=5)
self.assertIsNone(result) # single() returns None → correct
self.assertIn("query", store._captured)
self.assertIn("*..5", store._captured["query"])
self.assertNotIn("DETACH DELETE", store._captured["query"])
def test_shortest_path_max_depth_string_int_is_coerced(self):
store = self._make_store()
store.get_session = lambda: store._single_session()
store.shortest_path(start_node_id=1, end_node_id=2, max_depth="7")
self.assertIn("*..7", store._captured["query"])
# ---------------------------------------------------------------------------
# BLOCKER 2 regression: GraphStore.get_neighbors hops forwarding
# ---------------------------------------------------------------------------
# Before the fix, GraphStore.get_neighbors() set:
# actual_depth = options.get("hops", depth)
# and forwarded the raw value to Neo4jStore.get_neighbors(depth=actual_depth).
# Because Neo4jStore did not coerce depth, an attacker-controlled hops string
# reached Cypher verbatim.
#
# The fix adds int() at the GraphStore facade:
# actual_depth = int(options.get("hops", depth))
# This closes the path regardless of which backend is wired up.
class TestGraphStoreHopsForwarding(unittest.TestCase):
"""Regression tests for BLOCKER 2: GraphStore hops→depth forwarding."""
def _make_graph_store_with_neo4j(self):
"""
Wire a GraphStore whose backend is a Neo4j store stub that records every
query passed to session.run. Returns (graph_store, captured_dict).
"""
from semantica.graph_store import neo4j_store as m
from semantica.graph_store.graph_store import (
GraphAnalytics,
GraphManager,
GraphStore,
)
captured = {}
class IterSession:
def run(self, q, params=None):
captured["query"] = q
return iter([])
def __enter__(self):
return self
def __exit__(self, *a):
return False
neo4j_stub = _wire(m.Neo4jStore.__new__(m.Neo4jStore))
neo4j_stub.get_session = lambda: IterSession()
gs = GraphStore.__new__(GraphStore)
gs.logger = MagicMock()
gs.progress_tracker = MagicMock()
gs._store_backend = neo4j_stub
gs._manager = GraphManager(neo4j_stub)
return gs, captured
# -- hops injection -------------------------------------------------------
def test_hops_malicious_string_raises_before_query(self):
"""Malicious hops string must be rejected before session.run is reached."""
gs, captured = self._make_graph_store_with_neo4j()
with self.assertRaises(Exception):
gs.get_neighbors(node_id=1, hops=EVIL_DEPTH)
self.assertNotIn("query", captured,
"session.run was called — injected hops reached the database layer")
def test_hops_malicious_string_payload_not_in_any_query(self):
"""Belt-and-suspenders: payload text must not appear in any built query."""
gs, captured = self._make_graph_store_with_neo4j()
with pytest.raises(ValueError):
gs.get_neighbors(node_id=1, hops=EVIL_DEPTH)
query = captured.get("query", "")
self.assertNotIn("DETACH DELETE", query,
f"Injection payload found in forwarded query: {query!r}")
def test_hops_legitimate_integer_works(self):
"""Valid integer hops value must produce a correct query."""
gs, captured = self._make_graph_store_with_neo4j()
result = gs.get_neighbors(node_id=1, hops=2)
self.assertIsInstance(result, list)
self.assertIn("query", captured)
self.assertIn("*1..2", captured["query"])
self.assertNotIn("DETACH DELETE", captured["query"])
def test_hops_string_int_is_coerced_and_works(self):
"""String '3' forwarded as hops must be coerced to int and produce *1..3."""
gs, captured = self._make_graph_store_with_neo4j()
result = gs.get_neighbors(node_id=1, hops="3")
self.assertIsInstance(result, list)
self.assertIn("*1..3", captured["query"])
def test_depth_param_still_works_without_hops(self):
"""depth positional arg (no hops kwarg) must still be coerced and forwarded."""
gs, captured = self._make_graph_store_with_neo4j()
result = gs.get_neighbors(node_id=1, depth=4)
self.assertIsInstance(result, list)
self.assertIn("*1..4", captured["query"])
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,64 @@
"""Tests for PipelineWithProvenance.
Verifies that:
1. The import path is correct (no ModuleNotFoundError).
2. PipelineWithProvenance accepts a built Pipeline and runs it via ExecutionEngine.
3. Provenance tracking gracefully degrades when the provenance package is absent.
"""
import sys
from unittest.mock import patch
import pytest
from semantica.pipeline import PipelineBuilder
from semantica.pipeline.pipeline_provenance import PipelineWithProvenance
from semantica.pipeline.execution_engine import ExecutionResult
class TestPipelineWithProvenance:
"""Tests for PipelineWithProvenance."""
@pytest.fixture
def simple_pipeline(self):
"""Build a minimal two-step pipeline for testing."""
builder = PipelineBuilder()
builder.add_step("ingest", "file_ingest")
builder.add_step("parse", "document_parse")
return builder.build("test_provenance_pipeline")
def test_instantiation_with_pipeline(self, simple_pipeline):
"""Should accept a built Pipeline instance."""
runner = PipelineWithProvenance(simple_pipeline, provenance=False)
assert runner._pipeline is simple_pipeline
def test_run_returns_execution_result(self, simple_pipeline):
"""run() should delegate to ExecutionEngine and return an ExecutionResult."""
runner = PipelineWithProvenance(simple_pipeline, provenance=False)
result = runner.run()
assert isinstance(result, ExecutionResult)
assert result.success is True
def test_getattr_delegates_to_pipeline(self, simple_pipeline):
"""Attribute access should fall through to the wrapped Pipeline."""
runner = PipelineWithProvenance(simple_pipeline, provenance=False)
assert runner.name == "test_provenance_pipeline"
assert len(runner.steps) == 2
def test_provenance_disabled_when_import_fails(self, simple_pipeline):
"""When semantica.provenance is unavailable, provenance should be disabled."""
# Force the provenance import to raise ImportError
with patch.dict(sys.modules, {"semantica.provenance": None}):
runner = PipelineWithProvenance(simple_pipeline, provenance=True)
assert runner.provenance is False
assert runner._prov_manager is None
# Should still execute successfully without provenance
result = runner.run()
assert isinstance(result, ExecutionResult)
assert result.success is True
def test_run_with_data(self, simple_pipeline):
"""run() should accept data and kwargs without error."""
runner = PipelineWithProvenance(simple_pipeline, provenance=False)
result = runner.run(data={"key": "value"})
assert isinstance(result, ExecutionResult)
@@ -167,7 +167,6 @@ class TestProvenanceEnabledDisabled:
"""Test all modules work with provenance=False."""
modules_to_test = [
('semantica.context.context_provenance', 'ContextManagerWithProvenance'),
('semantica.pipeline.pipeline_provenance', 'PipelineWithProvenance'),
]
for module_path, class_name in modules_to_test:
@@ -183,7 +182,6 @@ class TestProvenanceEnabledDisabled:
"""Test all modules work with provenance=True."""
modules_to_test = [
('semantica.context.context_provenance', 'ContextManagerWithProvenance'),
('semantica.pipeline.pipeline_provenance', 'PipelineWithProvenance'),
]
for module_path, class_name in modules_to_test:
@@ -195,6 +193,23 @@ class TestProvenanceEnabledDisabled:
except ImportError:
pytest.skip(f"{module_path} not available")
def test_pipeline_with_provenance_supports_provenance_flag(self):
"""PipelineWithProvenance accepts provenance=True/False.
PipelineWithProvenance requires a built Pipeline instance (unlike
other *WithProvenance wrappers that own their internal state), so it
cannot participate in the generic no-argument constructor loop above.
"""
try:
from semantica.pipeline.pipeline_builder import Pipeline
from semantica.pipeline.pipeline_provenance import PipelineWithProvenance
pipeline = Pipeline(name="compat_test")
assert PipelineWithProvenance(pipeline, provenance=False).provenance is False
assert PipelineWithProvenance(pipeline, provenance=True).provenance is True
except ImportError:
pytest.skip("pipeline_provenance not available")
class TestAllModulesEdgeCases:
"""Test edge cases across all provenance modules."""
@@ -219,10 +234,14 @@ class TestAllModulesEdgeCases:
"""Test each module has independent provenance manager."""
try:
from semantica.context.context_provenance import ContextManagerWithProvenance
from semantica.pipeline.pipeline_builder import Pipeline
from semantica.pipeline.pipeline_provenance import PipelineWithProvenance
ctx = ContextManagerWithProvenance(provenance=True)
pipe = PipelineWithProvenance(provenance=True)
pipe = PipelineWithProvenance(
Pipeline(name="independence_test"),
provenance=True,
)
# Each should have its own manager
assert ctx._prov_manager is not None
@@ -154,10 +154,13 @@ class TestModuleSpecificEdgeCases:
def test_pipeline_with_empty_data(self):
"""Test pipeline with empty data."""
try:
from semantica.pipeline.pipeline_builder import Pipeline
from semantica.pipeline.pipeline_provenance import PipelineWithProvenance
pipeline = PipelineWithProvenance(provenance=True)
pipeline = Pipeline(name="edge_case_test")
runner = PipelineWithProvenance(pipeline, provenance=True)
# Should handle empty data
assert pipeline is not None
assert runner is not None
except ImportError:
pytest.skip("Pipeline not available")
@@ -29,14 +29,17 @@ class TestRealModuleIntegration:
def test_pipeline_real_execution_tracking(self):
"""Test pipeline tracks execution with provenance."""
try:
from semantica.pipeline.pipeline_builder import Pipeline
from semantica.pipeline.pipeline_provenance import PipelineWithProvenance
pipeline = PipelineWithProvenance(provenance=True)
pipeline = Pipeline(name="provenance_tracking_test")
runner = PipelineWithProvenance(pipeline, provenance=True)
# Verify provenance setup
assert pipeline.provenance is True
assert pipeline._prov_manager is not None
assert runner.provenance is True
assert runner._prov_manager is not None
assert isinstance(runner._prov_manager, ProvenanceManager)
except ImportError:
pytest.skip("Pipeline not available")
+20
View File
@@ -215,6 +215,26 @@ class TestGraphStore(unittest.TestCase):
result = self.store.execute_query("MATCH (n) RETURN n")
self.assertEqual(result["summary"], "Mock query executed")
def test_degree_centrality_rejects_malicious_label(self):
"""Regression test for GHSA-482h-hw99-h62p: degree_centrality()
interpolates labels/rel_type directly into a Cypher MATCH clause
(graph_store.py's own query builder, not delegated to the backend),
so an unvalidated label was a direct injection point."""
from semantica.utils.exceptions import ValidationError
evil_label = "N}) MATCH (victim) DETACH DELETE victim //"
with self.assertRaises(ValidationError):
self.store._manager.analytics.degree_centrality(labels=[evil_label])
def test_degree_centrality_rejects_malicious_rel_type(self):
from semantica.utils.exceptions import ValidationError
evil_rel_type = "R]-() DETACH DELETE n //"
with self.assertRaises(ValidationError):
self.store._manager.analytics.degree_centrality(rel_type=evil_rel_type)
def test_degree_centrality_with_legitimate_input_still_works(self):
result = self.store._manager.analytics.degree_centrality(labels=["Person"])
self.assertEqual(result, []) # MockGraphStore.execute_query returns no records
class TestGraphStoreInitialization(unittest.TestCase):
def test_falkordb_initialization(self):
with patch('semantica.graph_store.falkordb_store.FalkorDBStore', side_effect=MockGraphStore) as mock_falkor:
+21
View File
@@ -57,5 +57,26 @@ class TestGraphStoreMethods(unittest.TestCase):
# Verify
self.mock_store.execute_query.assert_called_once_with(query, None)
def test_update_relationship_rejects_malicious_property_key(self):
"""Regression test for GHSA-482h-hw99-h62p: update_relationship()
interpolates property keys directly into a Cypher SET clause
(methods.py's own query builder, not delegated to the backend
store), so an unvalidated key was a direct injection point."""
from semantica.utils.exceptions import ValidationError
evil_key = "x} MATCH (victim) DETACH DELETE victim //"
with self.assertRaises(ValidationError):
methods.update_relationship(1, {evil_key: "value"})
self.mock_store.execute_query.assert_not_called()
def test_update_relationship_with_legitimate_keys_still_works(self):
self.mock_store.execute_query.return_value = {
"records": [{"id": 1, "type": "KNOWS"}]
}
result = methods.update_relationship(1, {"weight": 0.5})
query = self.mock_store.execute_query.call_args[0][0]
self.assertIn("r.weight = $weight", query)
self.assertNotIn("DETACH DELETE", query)
if __name__ == '__main__':
unittest.main()
+36
View File
@@ -0,0 +1,36 @@
"""Regression tests for MCP server version reporting."""
import unittest
from importlib.metadata import PackageNotFoundError, version
import semantica
from semantica import mcp_server
class TestMCPServerVersion(unittest.TestCase):
def test_server_info_uses_distribution_version(self):
try:
expected = version("semantica")
except PackageNotFoundError:
expected = semantica.__version__
self.assertEqual(mcp_server.SERVER_INFO["version"], expected)
def test_initialize_reports_package_version(self):
response = mcp_server._handle(
{"jsonrpc": "2.0", "id": 1, "method": "initialize"}
)
self.assertIsNotNone(response)
self.assertEqual(
response["result"]["serverInfo"]["version"], semantica.__version__
)
def test_schema_info_resource_reports_package_version(self):
resource = mcp_server._read_resource("semantica://schema/info")
self.assertEqual(resource["version"], semantica.__version__)
if __name__ == "__main__":
unittest.main()
+76 -71
View File
@@ -2,48 +2,29 @@
Regression tests for security fixes in PR #898.
Covers:
1. API Key Authentication (Explorer)
2. Cypher Injection Prevention (AGE Store)
3. SPARQL Injection Prevention (read-only query validation)
4. XXE Protection (rdf_parser fail-closed)
5. Vector save numpy serialization
6. SPARQL graph cap error handling
7. SSRF redirect handling (relative URLs, resp.close)
1. Cypher Injection Prevention (AGE Store)
2. SPARQL Injection Prevention (read-only query validation)
3. XXE Protection (rdf_parser fail-closed)
4. Vector save numpy serialization
5. SPARQL graph cap error handling
6. SSRF redirect handling (relative URLs, resp.close)
Explorer API-key authentication (GHSA-j4mq-hprp-987v) has its own, more
thorough test suite at tests/explorer/test_explorer_auth.py it isn't
duplicated here.
"""
import re
import pytest
# ===================================================================
# 1. SPARQL read-only query validation (injection prevention)
# ===================================================================
# Inline the validation logic so tests don't require full app context
_ALLOWED_QUERY_TYPES = re.compile(
r"^(SELECT|ASK|CONSTRUCT|DESCRIBE)\b",
re.IGNORECASE,
)
_FORBIDDEN_KEYWORDS = re.compile(
r"\b(INSERT|DELETE|DROP|LOAD|CLEAR|CREATE|COPY|MOVE|ADD)\b",
re.IGNORECASE,
)
_COMMENT_LINE = re.compile(r"#[^\n]*", re.MULTILINE)
_PREFIX_DECL = re.compile(
r"^\s*(?:PREFIX|BASE)\s+(?:\S+\s+)?<[^>]*>\s*",
re.IGNORECASE | re.MULTILINE,
)
def _is_read_only_query(query: str) -> bool:
cleaned = _PREFIX_DECL.sub("", query)
cleaned = _COMMENT_LINE.sub("", cleaned)
cleaned = cleaned.strip()
if not _ALLOWED_QUERY_TYPES.match(cleaned):
return False
if _FORBIDDEN_KEYWORDS.search(cleaned):
return False
return True
# Import the real implementation rather than re-declaring the regexes here:
# an earlier version of this file inlined a copy that silently drifted from
# semantica/explorer/routes/sparql.py's actual behavior (the inlined
# _COMMENT_LINE regex stripped '#' mid-token, corrupting any PREFIX
# declaration using a namespace IRI with a literal '#', e.g. the standard
# rdf:/rdfs: namespaces) and neither the code nor this test caught it,
# since both had the same bug. Importing the real function makes that class
# of drift impossible.
from semantica.explorer.routes.sparql import _is_read_only_query
class TestSparqlReadOnlyValidation:
@@ -122,6 +103,63 @@ class TestSparqlReadOnlyValidation:
query = "BASE <http://example.org/>\nSELECT ?s WHERE { ?s ?p ?o }"
assert _is_read_only_query(query)
def test_namespace_iri_with_hash_fragment_not_treated_as_comment(self):
"""A '#' inside a PREFIX declaration's IRI (standard for RDF/RDFS/OWL
namespaces) must not be mistaken for a comment-start a naive
`#[^\\n]*` strip corrupts the IRI and truncates the rest of the
query with it."""
query = (
"PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\n"
"SELECT ?s WHERE { ?s rdf:type ?o }"
)
assert _is_read_only_query(query)
def test_real_comment_after_namespace_iri_still_stripped(self):
"""A genuine trailing comment must still be recognized even on a
line that also contains a '#'-bearing IRI earlier in the query."""
query = (
"PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\n"
"SELECT ?s WHERE { ?s rdf:type ?o } # trailing comment INSERT DATA"
)
assert _is_read_only_query(query)
def test_inline_prefix_before_select_allowed(self):
"""PREFIX declaration on the same line as the query keyword (inline
prologue) must be stripped correctly so SELECT is seen first.
Regression for the [ \\t]*(?:\\n|$) anchor that rejected this form."""
query = "PREFIX ex: <http://example.org/> SELECT ?s WHERE { ?s ex:p ?o }"
assert _is_read_only_query(query)
def test_crlf_line_endings_with_prefix(self):
"""Windows CRLF line endings (\\r\\n) between PREFIX and SELECT must
be handled correctly. The previous (?:\\n|$) anchor did not allow
the \\r before \\n, causing stripping to fail."""
query = "PREFIX ex: <http://example.org/>\r\nSELECT ?s WHERE { ?s ?p ?o }"
assert _is_read_only_query(query)
def test_crlf_multiple_prefixes_then_select(self):
"""Multiple PREFIX lines with CRLF endings should all be stripped."""
query = (
"PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\r\n"
"PREFIX ex: <http://example.org/>\r\n"
"SELECT ?s WHERE { ?s rdf:type ex:Thing }"
)
assert _is_read_only_query(query)
def test_inline_prefix_before_insert_still_blocked(self):
"""An inline PREFIX followed by INSERT must not be allowed — the
inline stripping fix must not open a bypass for write operations."""
query = "PREFIX ex: <http://example.org/> INSERT DATA { ex:s ex:p ex:o }"
assert not _is_read_only_query(query)
def test_long_valid_query_not_rejected_by_is_read_only(self):
"""_is_read_only_query must not enforce the length limit itself —
that responsibility belongs to execute_sparql() so the route can
return a distinct, actionable error. A syntactically valid but long
SELECT query must still return True from this function."""
long_query = "SELECT ?s WHERE { ?s ?p ?o } # " + ("x" * 20_000)
assert _is_read_only_query(long_query)
# ===================================================================
# 2. Cypher injection prevention
@@ -270,38 +308,5 @@ class TestSSRFRedirectHandling:
assert result == "https://example.com/api/data.ttl"
# ===================================================================
# 6. API Key Auth
# ===================================================================
class TestAPIKeyAuth:
"""Regression tests for API key authentication."""
def test_auth_module_importable(self):
from semantica.explorer.auth import APIKeyAuthMiddleware
assert APIKeyAuthMiddleware is not None
def test_extract_bearer_token(self):
from semantica.explorer.auth import _extract_token
from unittest.mock import MagicMock
req = MagicMock()
req.headers = {"Authorization": "Bearer test-key-123"}
assert _extract_token(req) == "test-key-123"
def test_extract_api_key_header(self):
from semantica.explorer.auth import _extract_token
from unittest.mock import MagicMock
req = MagicMock()
req.headers = {"X-API-Key": "my-secret-key", "Authorization": ""}
assert _extract_token(req) == "my-secret-key"
def test_extract_no_token(self):
from semantica.explorer.auth import _extract_token
from unittest.mock import MagicMock
req = MagicMock()
req.headers = {}
assert _extract_token(req) is None
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,275 @@
"""Regression tests for GHSA-8vgg-8mr4-r236: unvalidated triplet IRIs
allowed arbitrary SPARQL update injection in the Blazegraph and RDF4J
stores, and query-filter injection on the Jena read path.
Triplet.subject/predicate (and, in some builders, .object) are document
text in the normal ingest pipeline entity names extracted from ingested
content. A subject containing '>' closes the '<...>' IRI token early, so
the rest of the value is parsed as more SPARQL, letting an attacker
append operations like CLEAR ALL that run with the application's store
credentials.
Mirrors the advisory's own PoC payload: a subject/predicate crafted to
close the current triple pattern and append a destructive `; CLEAR ALL ;`
statement. After the fix, sparql_escaping.validate_uri (already used by
anzo_store.py, the one backend that was already hardened) rejects it
before any query/update text is built.
"""
import unittest
from unittest.mock import patch
from semantica.semantic_extract.triplet_extractor import Triplet
from semantica.triplet_store.blazegraph_store import BlazegraphStore
from semantica.triplet_store.rdf4j_store import RDF4JStore
from semantica.triplet_store.jena_store import JenaStore
from semantica.utils.exceptions import ProcessingError, ValidationError
# The advisory's own injection payload: closes the <...> token, then the
# triple pattern, then appends a store-wide wipe and a rogue insert.
EVIL_SUBJECT = (
"http://example.com/a> <http://example.com/b> <http://example.com/c> . } "
"; CLEAR ALL ; INSERT DATA { <http://evil.example/owned"
)
class TestBlazegraphSparqlInjection(unittest.TestCase):
@patch.object(BlazegraphStore, "_connect", autospec=True)
def _make_store(self, _mock_connect):
return BlazegraphStore(endpoint="http://localhost:9999/blazegraph")
def test_build_insert_data_rejects_malicious_subject(self):
store = self._make_store()
triplet = Triplet(subject=EVIL_SUBJECT, predicate="http://p", object="x")
with self.assertRaises(ValidationError):
store._build_insert_data([triplet])
def test_triplets_to_rdf_rejects_malicious_subject(self):
store = self._make_store()
triplet = Triplet(subject=EVIL_SUBJECT, predicate="http://p", object="x")
with self.assertRaises(ValidationError):
store._triplets_to_rdf([triplet])
def test_delete_triplet_rejects_malicious_subject(self):
store = self._make_store()
store.connected = True
triplet = Triplet(subject=EVIL_SUBJECT, predicate="http://p", object="x")
with self.assertRaises(ValidationError):
store.delete_triplet(triplet)
def test_get_triplets_rejects_malicious_subject_filter(self):
store = self._make_store()
with self.assertRaises(ValidationError):
store.get_triplets(subject=EVIL_SUBJECT)
def test_bulk_load_rejects_malicious_graph_option(self):
# bulk_load() wraps its whole body in except Exception: raise
# ProcessingError(...) (pre-existing, unrelated to this fix), so the
# ValidationError sanitize_uri raises surfaces as ProcessingError.
# The security property that matters — the malicious query is never
# sent — holds either way.
store = self._make_store()
store.connected = True
triplet = Triplet(subject="http://s", predicate="http://p", object="x")
with self.assertRaises(ProcessingError) as ctx:
store.bulk_load([triplet], graph=EVIL_SUBJECT)
self.assertIn("Invalid URI", str(ctx.exception))
def test_legitimate_triplet_still_builds_correct_query(self):
store = self._make_store()
triplet = Triplet(subject="http://s", predicate="http://p", object="x")
insert_data = store._build_insert_data([triplet])
self.assertIn("<http://s> <http://p>", insert_data)
self.assertNotIn("CLEAR ALL", insert_data)
def test_format_object_rejects_malicious_pre_wrapped_iri(self):
"""A caller-supplied object already wrapped in '<...>' must still be
fully validated, not just checked for a literal space/'>' a
narrower ad-hoc check here previously let this branch bypass
validate_uri() entirely (Codex-flagged follow-up to GHSA-8vgg)."""
store = self._make_store()
evil_object = f"<{EVIL_SUBJECT}>"
triplet = Triplet(subject="http://s", predicate="http://p", object=evil_object)
with self.assertRaises(ValidationError):
store._format_object_for_sparql(triplet)
def test_format_object_accepts_legitimate_pre_wrapped_iri(self):
store = self._make_store()
triplet = Triplet(subject="http://s", predicate="http://p", object="<http://o>")
self.assertEqual(store._format_object_for_sparql(triplet), "<http://o>")
class TestRDF4JSparqlInjection(unittest.TestCase):
@patch.object(RDF4JStore, "_connect", autospec=True)
def _make_store(self, _mock_connect):
return RDF4JStore(endpoint="http://localhost:9999/rdf4j", repository_id="mem")
def test_triplets_to_ntriples_rejects_malicious_subject(self):
store = self._make_store()
triplet = Triplet(subject=EVIL_SUBJECT, predicate="http://p", object="x")
with self.assertRaises(ValidationError):
store._triplets_to_ntriples([triplet])
def test_delete_triplet_rejects_malicious_subject(self):
store = self._make_store()
store.connected = True
triplet = Triplet(subject=EVIL_SUBJECT, predicate="http://p", object="http://o")
with self.assertRaises(ValidationError):
store.delete_triplet(triplet)
# ------------------------------------------------------------------
# Regression tests for the literal-object bug fixed after the
# adversarial review of PR #911: delete_triplet() previously called
# validate_uri(triplet.object) unconditionally, which rejected every
# non-URI object with ValidationError even though literal objects are
# perfectly legal in RDF. The fix routes the object through
# _format_object_for_ntriples so URI-valued objects are still validated
# while literal objects go through escape_literal unchanged.
# ------------------------------------------------------------------
def _make_connected_store_with_captured_query(self):
"""Return (store, captured_dict) where captured['update'] is the
SPARQL update string passed to requests.post once delete_triplet
succeeds."""
import requests as req_mod
from unittest.mock import MagicMock
store = self._make_store()
store.connected = True
captured = {}
mock_resp = MagicMock()
mock_resp.raise_for_status = MagicMock()
def fake_post(url, **kwargs):
captured["update"] = kwargs.get("data", {}).get("update", "")
return mock_resp
store._post = fake_post # not used directly; patch requests.post below
store._captured = captured
return store, captured
def test_delete_triplet_literal_object_succeeds(self):
"""A triplet with a plain-string literal object must delete without
raising ValidationError the regression that prompted this fix."""
store, captured = self._make_connected_store_with_captured_query()
with patch("requests.post") as mock_post:
mock_post.return_value.__enter__ = lambda s: s
mock_post.return_value.raise_for_status = lambda: None
result = store.delete_triplet(
Triplet(subject="http://s", predicate="http://p", object="Paris")
)
self.assertEqual(result, {"success": True})
# Confirm query shape: object must be a quoted literal, not <Paris>
query_sent = mock_post.call_args[1]["data"]["update"]
self.assertIn("<http://s> <http://p>", query_sent)
self.assertIn('"Paris"', query_sent)
self.assertNotIn("<Paris>", query_sent)
self.assertNotIn("CLEAR ALL", query_sent)
def test_delete_triplet_uri_object_still_works(self):
"""A triplet whose object is a URI must still delete correctly."""
store, _ = self._make_connected_store_with_captured_query()
with patch("requests.post") as mock_post:
mock_post.return_value.raise_for_status = lambda: None
result = store.delete_triplet(
Triplet(subject="http://s", predicate="http://p", object="http://o")
)
self.assertEqual(result, {"success": True})
query_sent = mock_post.call_args[1]["data"]["update"]
self.assertIn("<http://s> <http://p> <http://o>", query_sent)
self.assertNotIn("CLEAR ALL", query_sent)
def test_delete_triplet_malicious_uri_object_rejected_before_post(self):
"""A URI-shaped object containing '>' must be rejected by
_format_object_for_ntriples validate_uri before requests.post
is ever called."""
evil_obj = "http://evil.com/a>;CLEARALL"
store, _ = self._make_connected_store_with_captured_query()
with patch("requests.post") as mock_post:
with self.assertRaises(ValidationError):
store.delete_triplet(
Triplet(subject="http://s", predicate="http://p", object=evil_obj)
)
mock_post.assert_not_called()
def test_delete_triplet_malicious_subject_still_rejected(self):
"""Subject injection protection must remain intact after the fix."""
store, _ = self._make_connected_store_with_captured_query()
with patch("requests.post") as mock_post:
with self.assertRaises(ValidationError):
store.delete_triplet(
Triplet(subject=EVIL_SUBJECT, predicate="http://p", object="http://o")
)
mock_post.assert_not_called()
def test_get_triplets_rejects_malicious_subject_filter(self):
store = self._make_store()
with self.assertRaises(ValidationError):
store.get_triplets(subject=EVIL_SUBJECT)
def test_legitimate_triplet_still_builds_correct_ntriples(self):
store = self._make_store()
triplet = Triplet(subject="http://s", predicate="http://p", object="x")
ntriples = store._triplets_to_ntriples([triplet])
self.assertIn("<http://s> <http://p>", ntriples)
self.assertNotIn("CLEAR ALL", ntriples)
def test_format_object_rejects_malicious_pre_wrapped_iri(self):
"""Same pre-wrapped-object bypass as Blazegraph, fixed in
_format_object_for_ntriples."""
store = self._make_store()
evil_object = f"<{EVIL_SUBJECT}>"
triplet = Triplet(subject="http://s", predicate="http://p", object=evil_object)
with self.assertRaises(ValidationError):
store._format_object_for_ntriples(triplet)
def test_format_object_accepts_legitimate_pre_wrapped_iri(self):
store = self._make_store()
triplet = Triplet(subject="http://s", predicate="http://p", object="<http://o>")
self.assertEqual(store._format_object_for_ntriples(triplet), "<http://o>")
class TestJenaSparqlInjection(unittest.TestCase):
def setUp(self):
from rdflib import Graph
self.store = JenaStore()
self.store.graph = Graph()
def test_get_triplets_never_queries_with_malicious_subject_filter(self):
"""get_triplets() catches all exceptions and returns [] (pre-existing,
broad error-handling behavior unrelated to this fix), so the
observable contract is: the malicious filter must never reach
graph.query() at all."""
with patch.object(self.store.graph, "query", wraps=self.store.graph.query) as spy:
result = self.store.get_triplets(subject=EVIL_SUBJECT)
self.assertEqual(result, [])
spy.assert_not_called()
def test_get_triplets_with_legitimate_filter_still_reaches_query(self):
"""A validated identifier must not be rejected by the sanitizer —
it should reach graph.query(). (Whether the WHERE-clause filter
syntax jena_store.py builds is itself correct SPARQL is a separate,
pre-existing question this test doesn't assert on: the query here
is `{ ?s ?p ?o ?s = <http://s> }`, missing a FILTER()/separator,
and unrelated to sanitize_uri.)"""
self.store.graph.parse(
data='<http://s> <http://p> "x" .', format="ntriples"
)
with patch.object(self.store.graph, "query", wraps=self.store.graph.query) as spy:
self.store.get_triplets(subject="http://s")
spy.assert_called_once()
self.assertIn("http://s", spy.call_args[0][0])
if __name__ == "__main__":
unittest.main()