fix(export): keep caller data out of the shipped ns# namespace
Every JSON-LD context set @vocab to https://semantica.dev/vocab/,
which 404s, so every bare term in caller data (extracted entity/
relationship types, arbitrary metadata keys) minted under a namespace
the package never ships. The obvious fix, pointing @vocab at
SEMANTICA_NS instead, turned out to be worse than the dead link: since
that namespace is real and populated, every bare term a caller happens
to use now expands into something that looks like official Semantica
vocabulary. An extracted type "ORG" became ns#ORG, a class the
vocabulary never defines. A metadata key "source" attached a plain
string value to sem:source, an owl:ObjectProperty that already exists
in semantica-ns.ttl with a resource-valued range, silently corrupting
its semantics.
@vocab is now removed from all five contexts (four in
json_exporter.py, one in rdf_exporter.py) rather than repointed.
Every document already used explicit semantica: prefixes for its own
terms, so nothing else in the output changes; an unscoped bare term
now simply fails to expand, which is standard JSON-LD behavior for a
context that doesn't know it, instead of being silently claimed by
our namespace.
Two call sites needed to stop handing caller data to @type/bare terms
in the first place:
- Entity nodes are always typed semantica:Entity now, with the
caller's label carried as a semantica:type string instead of
minted into @type. This matches how relationship nodes already
carried their type. sem:type's domain in semantica-ns.ttl opens up
to cover entities as well as relationships, following the
sem:confidence precedent, since the property is now legitimately
emitted for both.
- semantica:metadata gets an explicit @json term definition, so a
caller's metadata dict travels as one rdf:JSON literal instead of
having its keys expand as separate predicates. A metadata key can
no longer collide with a real ontology term no matter what the
caller names it.
Both JSONExporter and RDFExporter.serialize_to_jsonld got the same
treatment, since they build separate JSON-LD structures for the same
underlying data.
The regression tests assert the negative space this bug lived in: no
context declares @vocab, no caller type label appears as an rdf:type
under ns#, and no caller metadata key appears as a predicate under
ns# at all, only as content inside the single JSON literal.
Closes#1146
fix(ontology): coalesce normalized property collisions
Different raw property spellings can normalize to the same ontology
name and IRI. works_for and worksFor, for example, both normalize to
worksFor, but property inference emitted a separate definition for
each spelling, so the generated ontology declared two distinct
properties under what would become the same IRI once minted. The same
collapse could also happen across kinds: a relationship type and an
entity attribute that normalize to the same name would previously
produce a data property and an object property sharing one name, with
no signal that anything was wrong.
infer_properties() now runs a coalescing pass after object and data
properties are both inferred. Properties are grouped by (kind, name).
Object properties that collide are merged in occurrence order:
domains and ranges are unioned rather than overwritten, so a property
seen across several source classes keeps every domain instead of
losing all but the first, and occurrence_count is summed across the
merged spellings so downstream confidence/frequency signals stay
correct. Data properties merge domains the same way and reconcile
differing ranges through the existing _get_more_general_type()
widening logic already used elsewhere in this file, rather than a new
implementation.
A name that resolves to both an object property and a data property
is not silently coalesced into either one, since the two kinds mean
different things in the emitted ontology. That case raises a
ValidationError up front, naming every colliding name and which kinds
collided, so the conflict surfaces before an ambiguous ontology is
written rather than after.
Verified beyond the two cases in the new test file: a data property
colliding across two different domain classes correctly unions the
domain instead of keeping only the first class, and three distinct
spellings of the same relationship type collapse into one property
with the occurrence count correctly summed across all three.
Follow-up to #1170 (relationship endpoint types) and #1171 (retained
data properties for normalized class names).
Adds `SAPODataConnector`, `SAPODataEntity`, and `SAPIngestor` for ingesting master and transactional data from SAP OData services, mainly things like Business Partners and Sales Orders.
Tested around S/4HANA Cloud, SuccessFactors, and on-prem NetWeaver Gateway style OData endpoints.
Main pieces included:
* OAuth2 client credentials and Basic auth support. Both go through the existing `ssrf.py` checks, including the OAuth token request.
* Small EDMX parser used by `discover_service()` so we don't need to pull in `pyodata`.
* Server-side pagination support for both OData versions:
* v2: `__next`, including plain string and `__deferred` formats
* v4: `@odata.nextLink`
* Keeps the service path in the base URL correctly whether the URL has a trailing slash or not. This is normalized in `SAPIngestor.__init__`.
* Adds an `ingest-sap` extra with just `requests`, so there is no SAP/proprietary SDK dependency.
This is meant to be a fairly small first version of the connector without adding a lot of SAP-specific dependencies.
Closes#1228
PipelineBuilder.set_parallelism() validated and stored a level in
pipeline config, but ExecutionEngine._execute_steps() had no parallel
code path and no code ever read it back, so steps always ran strictly
sequentially regardless of the configured value. parallelism was also
lost across a serialize/deserialize round trip, since the nested
config key was never promoted to the top-level dict build_pipeline()
reads.
Steps are now grouped into dependency layers (declaration order
preserved within each layer). A layer runs concurrently, bounded by
ThreadPoolExecutor(max_workers=min(configured parallelism, engine
max_workers)), only when every one of the following holds: more than
one step in the layer, the shared input is a dict, every step is
opted in via the new PipelineStep.parallel_safe flag, and no step is
in delta_mode. Any layer that doesn't meet all four falls back to the
existing sequential path unchanged.
Each step's input is deep-copied before any handler in the layer
starts, so concurrent steps never share mutable state. Layer results
are merged back in declaration order, not completion order; keys
whose value is unchanged from the shared input are treated as an
echo rather than a write, so two handlers both returning {**data, ...}
don't spuriously conflict on keys neither of them actually touched.
Genuinely conflicting values for the same key raise ProcessingError
naming both the key and the two steps involved. Retry policy, step
status, result/error tracking, and progress reporting are shared
between the sequential and parallel paths so behavior stays identical
either way. On step failure, not yet started futures in the same
layer are cancelled and the error propagates, so no downstream layer
ever runs.
parallel_safe is opt-in per step because handlers that share mutable
state or depend on strict ordering are not safe to run concurrently.
ParallelismManager.execute_pipeline_steps_parallel() is deliberately
not reused here; the engine implements its own bounded layer
scheduler so retry/status semantics stay identical between the
sequential and parallel code paths instead of diverging.
fix(pipeline): address qodo review findings on PR #1226
- detect circular/unknown dependencies in parallel grouping
(ValidationError instead of RecursionError/KeyError)
- skip unchanged echoed keys in parallel result merging to
avoid false conflicts
- fail before COMPLETED status when a parallel step returns a
non-dict; never retry such contract violations
- require strict bool parallel_safe in builder and engine gate
- make per-step progress tracking IDs unique across same-type
parallel steps
---------
Co-authored-by: 江俊杰 <jiangjunjie.37@jd.com>
* test(ner): fix NER configuration tests for the typed LLM extraction API
Two of the three failing tests tracked in #1059 were still red after
#1070 was closed because the mocks targeted the pre-typed provider API:
- test_ner_llm_config mocked generate_structured, but the LLM path now
goes through generate_typed with a Pydantic schema. Mock the typed
response (namespace items with .text/.label/.start/.end/.confidence)
and expect extraction_method 'llm_typed'.
- test_ner_pattern_config asserted 'Apple Inc' without the trailing
dot, but the ORG pattern captures it via (?:\.|\b). Assert 'Apple
Inc.' to match current production behavior.
Verified locally: 8/8 pass in test_ner_configurations.py; the
performance-test failures in tests/semantic_extract/ reproduce on a
clean main checkout and are unrelated.
Fixes#1059
Signed-off-by: Yunare Maia <yunare@gmail.com>
* refactor(ner): remove dead _extract_with_spacy method and unused self.nlp
_extract_with_spacy() had no callers: the ML dispatch path goes through
get_entity_method('ml') -> extract_entities_ml(), which loads the spaCy
model lazily via the process-level cache in methods.py. The instance
attribute self.nlp was only read by that dead method, so __init__ now
just validates the runtime (keeping the _ml_runtime_usable gate) instead
of eagerly loading a model that was never used.
Fixes#1058
Signed-off-by: Yunare Maia <yunare@gmail.com>
* test(split): rewrite NERExtractor cache tests to not rely on removed .nlp attribute
NERExtractor.nlp was removed in this PR as part of dead-code cleanup
(the attribute was only used by the equally-dead _extract_with_spacy()).
The three affected tests in TestNERExtractorSpacyModelCache previously
verified cache behavior through .nlp identity comparisons; rewrite them
to use load-call counts and direct se_methods.load_spacy_model() cache
queries instead:
- test_ner_extractor_reuses_cached_model_across_instances: drop the
e1.nlp is e2.nlp is e3.nlp assertion; len(calls)==1 already proves
reuse; add a cache query to confirm the cached object is non-None.
- test_ner_extractor_distinct_model_names_load_separately: store each
mock nlp in a dict keyed by name, then query the cache to assert
sm_cached is loaded['en_core_web_sm'] and sm_cached is not lg_cached.
- test_ner_extractor_failed_load_not_cached_and_retried: replace
extractor.nlp is None/not None with is-not-None construction checks
and a final cache query that verifies the recovered model is the
exact object returned by working_load.
All three tests still exercise the original behavioral contract (no
crash on missing model, failures not cached / retried, successful load
shared across instances); they just no longer rely on a private
instance attribute that no longer exists.
---------
Signed-off-by: Yunare Maia <yunare@gmail.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
fix(config): honor boolean env overrides in Config.get() (#1038)
Config.get() checked int before bool. Since bool subclasses int, boolean
environment values could be ignored or returned as integers.
Check bool first and strip whitespace before parsing boolean environment
values. This applies to the config modules for conflicts, deduplication,
split, embeddings, export, ingest, kg, normalize, ontology, and parse.
Also make _load_env_vars() use the same whitespace handling for mapped and
generic environment variables.
Fixes#1035
* fix(parse): import email.message and repair pdfplumber test mock
- email_parser.py uses email.message.Message at class-definition time but
only did 'import email', so 'import semantica.parse' fails in a fresh
Python process unless something else imported email.message first
- test_pdf_parser patched semantica.parse.pdf_parser.pdfplumber, which
never exists as a module attribute (pdfplumber is imported inside
PDFParser.parse); inject a fake module via sys.modules instead
* fix(parse): warn when PDF parse yields no text layer (scanned PDFs)
Scanned (image-only) PDFs parsed via the default pdfplumber route
returned an empty full_text with progress status 'completed' - no error,
no warning - so the failure only surfaced far downstream. Warn in
PDFParser.parse() when every parsed page yields no text (and extract_text
is enabled), pointing users to method='docling' with enable_ocr=True.
* fix(parse): improve scanned PDF detection
---------
Co-authored-by: shanyu910 <208111055+shanyu910@users.noreply.github.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
* fix(provenance): use timezone-aware UTC and assert stored records (#946)
Replace datetime.utcnow() in ProvenanceManager, ProvenanceEntry,
BridgeAxiom, and GraphBuilderWithProvenance with
datetime.now(timezone.utc), matching PipelineWithProvenance.
KG workflow and integration tests now read provenance back through
get_provenance() and assert algorithm metadata instead of generated
IDs, and call tracker methods that actually persist records.
* fix(provenance): compare provenance timestamps as instants, not strings
query_recorded_between() and audit_log() filtered and sorted on raw ISO
strings. With the timezone-aware change, a store can hold both pre-existing
naive stamps and offset-bearing ones, and the two are not string-comparable:
"...500000+00:00" sorts above "...500000", so a record at the identical
instant as a naive bound falls outside the range that should contain it.
Both now parse through _parse_timestamp() before comparing, reading naive
values as UTC. This mirrors ProvenanceTracker._parse_dt() in kg/, the class
ProvenanceManager replaces, so both sides of the migration answer a range
query the same way. Unparseable stored timestamps are skipped and logged
rather than silently dropped; unparseable bounds raise ValueError.
---------
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
* feat(reasoning): rule-driven actions with provenance
Add a structured Action layer so matched rules can trigger side effects
instead of only deriving new facts, turning the reasoner into a
production-rule system.
L1 - Action type system:
- Action base class with execute(bindings, reasoner) + ?var substitution
- AssertAction (optional write-back to KnowledgeGraph), RetractAction,
CallAction (structured replacement for the unused Rule.handler),
EmitEventAction (delivers to a registered event sink)
- Rule.actions field; wired into Reasoner.forward_chain() and
ReteEngine.execute_matches() (via optional bind_reasoner)
L2 - Provenance-aware actions:
- Reasoner records fired actions (rule, bindings, confidence) to
action_log when provenance is enabled
- Fix dangling import in reasoning_provenance.py (ReasoningEngine ->
Reasoner, infer -> infer_facts)
Backward compatible: rules using the legacy handler still fire (wrapped
as a CallAction); rules without actions behave exactly as before.
Adds tests/reasoning/test_rule_actions.py (9 tests).
Closes#1095
* fix(reasoning): address qodo review findings on rule actions
- Token-aware variable substitution to avoid ?x/?xy prefix collision
- KnowledgeGraph write-back protocol (explicit API -> canonical translation -> ValueError)
- Structured action_log entries with timestamp
- Decouple action firing from conclusion dedup via per-activation tracking
(fires known conclusions once; retract-self no longer loops to max_iterations)
- Add Reasoner.infer_with_results preserving confidence; infer_facts delegates
- Forward provenance flag in ReasoningProvenance; drop **kwargs; propagate confidence
- Populate Rete Match.bindings from rule conditions
- Add regression tests for each fix
* fix(reasoning): persist fired action activations
* fix(reasoning): deduplicate Rete action execution
* fix(reasoning): canonicalize action activation identity
* docs(reasoning): explain action replay controls
---------
Co-authored-by: 江俊杰 <jiangjunjie.37@jd.com>
* fix(ingest): import sqlalchemy text where DBIngestor and DataExporter use it
sqlalchemy.text was imported function-locally in DatabaseConnector.connect
and test_connection, but called in DataExporter.export_table_data and
DBIngestor.execute_query, which never imported it. Both raised NameError,
re-wrapped by their except handlers into a ProcessingError reading
'Failed to execute query: name text is not defined' -- a message that
looks like a database fault rather than a missing import.
No test exercised either method, so this also repairs a pre-existing
failure in tests/ingest/test_notebook_02.py::test_08_database_ingestion.
Add SQLite-backed coverage for all three call sites, including the
SELECT COUNT(*) branch that only runs when no limit is passed and would
otherwise stay untested.
Closes#1015
* test(ingest): register setUp cleanups with addCleanup
TemporaryDirectory and the SQLAlchemy engine were released only in tearDown, which unittest skips when setUp raises partway through. Register each cleanup as soon as its resource exists so a failed setUp still disposes the engine and removes the temp directory. LIFO ordering keeps dispose before cleanup, as tearDown had it.
---------
Co-authored-by: Pravit Ampapathini <pravitampapathini@Pravits-MacBook-Air-3.local>
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
* fix(ontology): preserve #-terminated namespaces in SHACLGenerator base_uri (#1082)
SHACLGenerator.__init__ normalized base_uri with rstrip('/') + '/', turning a #-terminated RDF namespace (e.g. http://example.org/manufacturing#) into ...#/. Every generated URI then landed in a different namespace than the instance data, so SHACL validation silently passed because the shapes targeted nothing.
__init__ now preserves a base_uri already ending in '/' or '#', matching the #-aware normalization generate() already applies. shapes_uri inherits the fix.
Adds test_hash_namespace_base_uri_is_not_mangled (fails on the old normalization), plus a CHANGELOG entry. Full ontology suite green.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(ontology): collapse slash runs, only preserve #-terminated base_uri
Qodo review caught that preserving any endswith('/') base left redundant
trailing slashes (e.g. .../ns////) intact, leaking a different namespace
into emitted IRIs. Now only '#'-terminated bases are kept verbatim; slash
runs are collapsed to a single '/', matching generate() normalization.
Adds test_slash_run_normalization_regression.
---------
Co-authored-by: changshenhan <217217832+changshenhan@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
* fix(cli): write embed generate output in the format embed index reads
* Address review: structured results get their own --output writer
deduplicate --output and ontology align --output were routed through
_write_embeddings_output, a helper for numeric matrices: it rejects the
dict/list shapes these commands produce and the .csv extension deduplicate
documents. New _write_result_output serializes structured results — JSON,
JSON-lines for lists, CSV for rows — and both commands use it. embed
generate keeps the embeddings writer, whose strictness is what #994 fixed.
On the pyarrow gap: the parquet writer already fails with an actionable
message (install pyarrow or use .json). Silently writing JSON bytes to a
.parquet path would recreate #994's magic-bytes failure, so the error stays
an error and the default suggestion stays .json.
* fix(cli): improve structured output serialization
---------
Co-authored-by: Sameer6305 <sskadam6305@gmail.com>
* refactor(export): consolidate duplicate Turtle/N-Triples literal escapers
_escape_literal (module-level) and RDFSerializer._escape_turtle_literal did
identical work in the same order (backslash, double-quote, newline, CR, tab).
Drop the newer static helper added in #1148 and route all call sites through
_escape_literal instead. Behaviour no-op.
Closes#1218.
* fix(export): handle datetime/None temporal bounds safely in OWL-Time
_escape_literal is str-only, so routing datetime or None temporal bounds
through it raised AttributeError during Turtle export. Stringify non-str
bounds (plain f-string semantics) before escaping, and render None as an
empty bound. Add regression tests for datetime bounds and end-only
intervals. Addresses Qodo high-priority finding #2 on #1221.
* fix(export): use isoformat for datetime temporal bounds
str() on a datetime drops the ISO-8601 T separator, producing a lexically invalid xsd:dateTimeStamp. Use isoformat() when available; strengthen the test to assert the exact T-separated form.
---------
_calculate_decision_content_similarity's character-bigram fallback was
unconditional, so ordinary multi-word English queries could pick up
incidental bigram overlap with unrelated decisions via max(word_sim,
bigram_sim). Gate it to only activate for CJK-like scripts or queries
with at most one whitespace token, matching its documented purpose.
Separately, _add_decision_to_graph never persisted recorded_at as a
node property, so _rebuild_decision_indexes/_sync_decision_from_node
(which already read it back) always recovered "" after any reload.
* fix(explorer): coerce decision timestamp to str to prevent 422 on /api/decisions
ContextGraph stores decision timestamps as POSIX floats (e.g. 1786513069.69),
but DecisionResponse.timestamp is typed Optional[str]. Pydantic strict
validation rejects the float and the whole /api/decisions endpoint returns
HTTP 422 "Invalid input", which breaks the Decisions workspace in the
Knowledge Explorer entirely (no decision can be listed).
Coerce the value to str (preserving None) in _node_to_decision so the
response validates. Verified: /api/decisions now returns 200 and the 3
sample decisions render in the Decisions workspace.
* test(explorer): cover decision timestamp coercion in _node_to_decision
Regression tests for the 422 fix in _node_to_decision. Covers the cases
that produced HTTP 422 (float / int timestamps from ContextGraph) and
the ones that must keep working (None, already-string, missing key).
Verified the suite catches the regression: with the fix reverted, the
float / int / nan / inf cases fail with the same ValidationError that
caused the 422; with the fix applied all 6 pass.
* fix(explorer): preserve decision timestamp normalization
The route-level str() cast introduced in the initial fix bypasses
DecisionResponse._normalize_timestamp, the field validator on main that
converts POSIX float epochs to ISO-8601 strings via
datetime.fromtimestamp(value, tz=UTC).isoformat().
With the cast in place the API emits raw numeric strings such as
'1786513069.69' instead of '2026-08-12T05:37:49+00:00', breaking
datetime.fromisoformat() for every caller and failing
TestRecordedDecisions::test_list_decisions_serializes_float_timestamp.
It also silently accepts nan/inf/out-of-range epochs that the validator
is designed to reject.
Restore _node_to_decision() to pass the raw stored value through
unchanged so DecisionResponse._normalize_timestamp remains the single
normalization boundary for all three affected endpoints:
GET /api/decisions
GET /api/decisions/{id}
GET /api/decisions/{id}/precedents
Rewrite test_decision_route_timestamp.py so every assertion uses
datetime.fromisoformat() to verify ISO-8601 output and explicitly
asserts ValidationError for nan, inf, -inf and out-of-range epochs.
Add three TestClient integration tests covering the full production
path: record_decision() -> float stored in graph -> HTTP GET -> JSON.
---------
Co-authored-by: administrator <administrator@administratordeMac-mini.local>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
RelationExtractor.extract_relations(text, entities, ...) requires
entities, but the tool called it with only text, raising TypeError
on every invocation. Run NER first and pass the resulting entities
through, matching how the rest of the pipeline extracts relations.
* Add tests for max_tokens propagation in LLM methods
This test verifies that the max_tokens parameter is correctly propagated to the generate_typed method for different extraction functions.
* fix(tests): make issue-176 regression tests discoverable by pytest
The contributor's PR added tests/optimize reproduce_issue_176.py — a file
with a space in its name that never matched pytest's test_*.py discovery
pattern, so the regression would have been silently skipped in CI/local runs.
The repository already contained a richer canonical regression file at
tests/reproduce_issue_176.py (11 tests across three classes) which had
the same naming problem: it was also never auto-discovered.
The contributor's file added only TestMaxTokensPropagation (3 tests), which
is a strict subset of what the canonical file already covers. No unique
coverage is lost by removing it.
Changes:
- Rename tests/reproduce_issue_176.py -> tests/test_reproduce_issue_176.py
so all 11 regression tests are collected by 'pytest tests/'
- Remove tests/optimize reproduce_issue_176.py (redundant strict subset)
No production code changes. All 11 regression tests pass.
---------
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
* fix(export): escape Turtle/N-Triples string literals (fixes#1098)
Add RDFSerializer._escape_turtle_literal and apply it to the semantica:text
literal in serialize_to_turtle and the N-Triples text triple. Backslash,
double quote, newline, CR, and tab are escaped per the RDF 1.1 Turtle
STRING_LITERAL_QUOTE grammar, so entity text containing quotes or control
characters no longer emits invalid Turtle/N-Triples.
N-Triples previously escaped only quotes and newlines; now it also handles
backslashes and tabs via the shared escaper.
* fix(export): escape OWL-Time timestamp literals in Turtle output
Addresses Qodo finding on #1148: the OWL-Time branch of
serialize_to_turtle interpolated from_val/until_val directly into quoted
literals. Apply _escape_turtle_literal there too so timestamps containing
quotes, backslashes, or control characters cannot produce invalid Turtle.
* chore: remove stray local files (AGENTS.md, evals superpowers docs) from PR branch
---------
_tool_export_graph fell through to json.dumps(kg) for any format outside
the RDF set, including values never declared in the tool's own inputSchema
enum. Nothing in this server validates tool-call args against inputSchema
before dispatch, so a typo'd or unsupported format (e.g. "yaml") silently
returned JSON data labeled with the wrong format and no error.
Validate against the declared format list up front and reuse the same
constant for the inputSchema enum so the two can't drift apart again.
* fix(triplet_store): OxigraphStore silently ignores storage_path and skips flush
Two persistence bugs in OxigraphStore:
1. `storage_path=...` was silently swallowed by **config. The __init__
parameter is named `path`, so passing the project-conventional
`storage_path` (used by ProvenanceManager and other stores) left
self.path = None and the store silently degraded to in-memory —
no error, no warning, data gone on exit. Accept `storage_path` as
an alias for `path`.
2. add_triplets never called flush(). pyoxigraph auto-flushes via
background threads but, per its docs, "might lag a little bit" —
that lag is a race where reopening or crashing immediately after a
write observes fewer triples. Call flush() explicitly for on-disk
stores to close the window.
Both verified: with the fix, `OxigraphStore(storage_path=...)` persists
across reopen; without it, data is lost.
* fix(triplet_store): improve oxigraph persistence
* test(triplet_store): clarify oxigraph persistence test
---------
Co-authored-by: administrator <administrator@administratordeMac-mini.local>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
* fix: preserve generation kwargs in relation extraction
* fix: include generation params in extraction cache keys
* fix: cover provider-specific generation params in extraction cache key
_GENERATION_CACHE_KEYS only covered the common OpenAI-shaped generation
params, so calls that differed only in Anthropic's system/stop_sequences,
Gemini's candidate_count, or Ollama's repeat_penalty/num_ctx/context_window
could still return a stale cached result generated under different settings.
Add these provider-specific keys to the cache key and add regression tests
covering system prompt, stop_sequences, and repeat_penalty.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
``requests.exceptions.RequestException`` subclasses ``OSError``, so the
``except (ImportError, OSError)`` handler in ``load_from_api`` swallowed
genuine network failures (connection errors, timeouts, HTTP errors) and
reported them as "requests library not available", hiding the real cause.
Remove the obsolete handler so those failures fall through to the generic
handler, which reports "Failed to load from API: ..." and chains the real
exception as ``__cause__``. Update the docstring's ``Raises`` section to
match the actual behavior.
Fixes#949
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
* fix: guard integration HTTP requests against SSRF
* fix(openclaw): complete fallback validation and base URL handling
Address the remaining review findings in the OpenClaw integration.
- Strengthen fallback base_url validation to require a non-empty string, valid HTTP(S) scheme, netloc, and hostname.
- Strip leading and trailing whitespace from base_url before storing it.
- Replace the flaky endpoint-construction test that made a real network connection with mocked session assertions.
- Add coverage for _get and _post endpoint construction and timeout forwarding.
- Add regression tests for whitespace-padded base URLs and the fallback validation path.
These changes complete the Qodo review fixes and harden OpenClaw URL handling without changing the intended localhost/private deployment behavior.
Add a Cite Us section to the README with BibTeX citation info, and
align it with docs/citation.md (author/organization: Semantica, 2026).
Update LICENSE and docs/project-license.md copyright holder to
Semantica, and replace the stale Hawksight-AI GitHub org slug with
semantica-agi across READMEs, plugin manifests, cookbook notebooks,
and GitHub templates.
_turtle_object() wrote an IRI-valued metadata value (currently only
sem:sourceUri, from the "uri" metadata key) straight into `<{value}>`
with no escaping. Turtle/N-Triples IRIREFs exclude control
characters, space, and <>"{}|^`\ unescaped, so a value shaped like
`<goodIRI> . <injected> <p> <o>` closed the reference early and let
the rest of the string be parsed as an attacker-chosen extra triple:
metadata={"uri": "https://x> . <https://injected> <https://p> <https://o"}
produced a well-formed Turtle/N-Triples document containing a triple
the caller never asked for.
RDF/XML was already safe (_rdfxml_metadata_lines runs the value
through _escape_xml before putting it in an rdf:resource attribute),
and JSON-LD is safe by construction (json.dumps makes structural
injection impossible) — only the Turtle/N-Triples "iri" literal path
in _turtle_object was unguarded.
Adds _safe_iri_ref(), a narrow percent-encoder for exactly the
characters an IRIREF may not contain unescaped. It's deliberately not
_as_turtle_iri: that also resolves registered prefixes, which a
metadata value never needs, so a dedicated guard stays simpler than
threading namespaces into a module-level helper that has no `self`.
Two regression tests, parametrised over turtle/ntriples: the `>`
delimiter-breaking payload from the report, and a control-character
(newline/tab) variant covering the other half of the excluded set.
Resolves the conflict in semantica/export/rdf_exporter.py between this
branch's metadata clauses (entity/graph metadata statements) and
main's IRI-normalization and XML-escaping hardening
(_as_turtle_iri / xml_escape, landed after this branch's last sync).
Kept both: entity/relationship/graph subjects and objects now go
through _as_turtle_iri (Turtle) or _as_turtle_iri + xml_escape
(RDF/XML), same as every other identifier in these serializers,
while the metadata-clause list building and graph_uri handling from
this branch are preserved unchanged. graph_uri is now normalized the
same way for consistency with the rest of the file.
Verified: tests/export + tests/ontology (411 tests) and the existing
Turtle-IRI regression suite (test_rdf_exporter_turtle_iris.py, 9
tests) all pass against the merged code.
ProgressTracker attached ConsoleProgressDisplay unconditionally, so any
script or CI job that piped or redirected stdout had one progress bar per
stage written into its output, escape sequences included. A plain
`python demo.py > out.txt` captured 173 bytes of progress-bar noise around
10 bytes of the program's own output.
Console progress is now attached only when stdout is an interactive
terminal, when running under Jupyter, or when SEMANTICA_FORCE_PROGRESS is
set. FileProgressDisplay is untouched, so progress logging still works in
pipelines, and SEMANTICA_DISABLE_PROGRESS keeps its existing meaning and
still takes precedence.
Both progress environment variables are now documented in the README and
the utils reference; SEMANTICA_DISABLE_PROGRESS previously existed only in
the reference page.
Deviations from the issue: the issue suggested disabling the tracker on
non-TTY stdout. This gates the display instead, because disabling the
tracker would short-circuit before FileProgressDisplay and take file
progress logging down with it, and the ~20 modules that set
`progress_tracker.enabled = True` in __init__ would need the property
setter taught about TTY state to avoid undoing it. Gating the display
leaves both alone.
Design note: the claim comment on the issue proposed an
`enabled: Optional[bool] = None` constructor opt-in; during implementation
the opt-in became SEMANTICA_FORCE_PROGRESS, which needs no signature change
and follows the NO_COLOR/FORCE_COLOR convention. Known limitation: TTY
detection runs once at tracker construction (the tracker is a process-wide
singleton), so a process that redirects stdout after first use needs the
env vars to change behaviour.
Fixes#1185
Kevin Zhang