* fix(cli): write embed generate output in the format embed index reads
* Address review: structured results get their own --output writer
deduplicate --output and ontology align --output were routed through
_write_embeddings_output, a helper for numeric matrices: it rejects the
dict/list shapes these commands produce and the .csv extension deduplicate
documents. New _write_result_output serializes structured results — JSON,
JSON-lines for lists, CSV for rows — and both commands use it. embed
generate keeps the embeddings writer, whose strictness is what #994 fixed.
On the pyarrow gap: the parquet writer already fails with an actionable
message (install pyarrow or use .json). Silently writing JSON bytes to a
.parquet path would recreate #994's magic-bytes failure, so the error stays
an error and the default suggestion stays .json.
* fix(cli): improve structured output serialization
---------
Co-authored-by: Sameer6305 <sskadam6305@gmail.com>
* refactor(export): consolidate duplicate Turtle/N-Triples literal escapers
_escape_literal (module-level) and RDFSerializer._escape_turtle_literal did
identical work in the same order (backslash, double-quote, newline, CR, tab).
Drop the newer static helper added in #1148 and route all call sites through
_escape_literal instead. Behaviour no-op.
Closes#1218.
* fix(export): handle datetime/None temporal bounds safely in OWL-Time
_escape_literal is str-only, so routing datetime or None temporal bounds
through it raised AttributeError during Turtle export. Stringify non-str
bounds (plain f-string semantics) before escaping, and render None as an
empty bound. Add regression tests for datetime bounds and end-only
intervals. Addresses Qodo high-priority finding #2 on #1221.
* fix(export): use isoformat for datetime temporal bounds
str() on a datetime drops the ISO-8601 T separator, producing a lexically invalid xsd:dateTimeStamp. Use isoformat() when available; strengthen the test to assert the exact T-separated form.
---------
_calculate_decision_content_similarity's character-bigram fallback was
unconditional, so ordinary multi-word English queries could pick up
incidental bigram overlap with unrelated decisions via max(word_sim,
bigram_sim). Gate it to only activate for CJK-like scripts or queries
with at most one whitespace token, matching its documented purpose.
Separately, _add_decision_to_graph never persisted recorded_at as a
node property, so _rebuild_decision_indexes/_sync_decision_from_node
(which already read it back) always recovered "" after any reload.
* fix(explorer): coerce decision timestamp to str to prevent 422 on /api/decisions
ContextGraph stores decision timestamps as POSIX floats (e.g. 1786513069.69),
but DecisionResponse.timestamp is typed Optional[str]. Pydantic strict
validation rejects the float and the whole /api/decisions endpoint returns
HTTP 422 "Invalid input", which breaks the Decisions workspace in the
Knowledge Explorer entirely (no decision can be listed).
Coerce the value to str (preserving None) in _node_to_decision so the
response validates. Verified: /api/decisions now returns 200 and the 3
sample decisions render in the Decisions workspace.
* test(explorer): cover decision timestamp coercion in _node_to_decision
Regression tests for the 422 fix in _node_to_decision. Covers the cases
that produced HTTP 422 (float / int timestamps from ContextGraph) and
the ones that must keep working (None, already-string, missing key).
Verified the suite catches the regression: with the fix reverted, the
float / int / nan / inf cases fail with the same ValidationError that
caused the 422; with the fix applied all 6 pass.
* fix(explorer): preserve decision timestamp normalization
The route-level str() cast introduced in the initial fix bypasses
DecisionResponse._normalize_timestamp, the field validator on main that
converts POSIX float epochs to ISO-8601 strings via
datetime.fromtimestamp(value, tz=UTC).isoformat().
With the cast in place the API emits raw numeric strings such as
'1786513069.69' instead of '2026-08-12T05:37:49+00:00', breaking
datetime.fromisoformat() for every caller and failing
TestRecordedDecisions::test_list_decisions_serializes_float_timestamp.
It also silently accepts nan/inf/out-of-range epochs that the validator
is designed to reject.
Restore _node_to_decision() to pass the raw stored value through
unchanged so DecisionResponse._normalize_timestamp remains the single
normalization boundary for all three affected endpoints:
GET /api/decisions
GET /api/decisions/{id}
GET /api/decisions/{id}/precedents
Rewrite test_decision_route_timestamp.py so every assertion uses
datetime.fromisoformat() to verify ISO-8601 output and explicitly
asserts ValidationError for nan, inf, -inf and out-of-range epochs.
Add three TestClient integration tests covering the full production
path: record_decision() -> float stored in graph -> HTTP GET -> JSON.
---------
Co-authored-by: administrator <administrator@administratordeMac-mini.local>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
RelationExtractor.extract_relations(text, entities, ...) requires
entities, but the tool called it with only text, raising TypeError
on every invocation. Run NER first and pass the resulting entities
through, matching how the rest of the pipeline extracts relations.
* Add tests for max_tokens propagation in LLM methods
This test verifies that the max_tokens parameter is correctly propagated to the generate_typed method for different extraction functions.
* fix(tests): make issue-176 regression tests discoverable by pytest
The contributor's PR added tests/optimize reproduce_issue_176.py — a file
with a space in its name that never matched pytest's test_*.py discovery
pattern, so the regression would have been silently skipped in CI/local runs.
The repository already contained a richer canonical regression file at
tests/reproduce_issue_176.py (11 tests across three classes) which had
the same naming problem: it was also never auto-discovered.
The contributor's file added only TestMaxTokensPropagation (3 tests), which
is a strict subset of what the canonical file already covers. No unique
coverage is lost by removing it.
Changes:
- Rename tests/reproduce_issue_176.py -> tests/test_reproduce_issue_176.py
so all 11 regression tests are collected by 'pytest tests/'
- Remove tests/optimize reproduce_issue_176.py (redundant strict subset)
No production code changes. All 11 regression tests pass.
---------
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
* fix(export): escape Turtle/N-Triples string literals (fixes#1098)
Add RDFSerializer._escape_turtle_literal and apply it to the semantica:text
literal in serialize_to_turtle and the N-Triples text triple. Backslash,
double quote, newline, CR, and tab are escaped per the RDF 1.1 Turtle
STRING_LITERAL_QUOTE grammar, so entity text containing quotes or control
characters no longer emits invalid Turtle/N-Triples.
N-Triples previously escaped only quotes and newlines; now it also handles
backslashes and tabs via the shared escaper.
* fix(export): escape OWL-Time timestamp literals in Turtle output
Addresses Qodo finding on #1148: the OWL-Time branch of
serialize_to_turtle interpolated from_val/until_val directly into quoted
literals. Apply _escape_turtle_literal there too so timestamps containing
quotes, backslashes, or control characters cannot produce invalid Turtle.
* chore: remove stray local files (AGENTS.md, evals superpowers docs) from PR branch
---------
Profiling the viewer in headless Chromium (real DOM, production React)
separated remark parse time, React commit time and DOM node count across
large-prose, large-code-block, deep-nested-list and GFM-table fixtures.
Two findings, one of which is fixed here.
1. Every re-render re-parsed the whole document and remounted the whole
subtree. remarkPlugins and the ~20-entry components map were inline
literals, so each render allocated fresh arrow components; React saw a new
element type per mapped tag and replaced the DOM rather than updating it. A
DOM-identity probe confirmed the remount on every fixture. Because
react-markdown runs the remark pipeline inside its own render, an unrelated
state change -- clicking Copy, toggling Preview/Source -- re-paid the full
parse. Measured 364ms for a 1000-row GFM table and 1121ms for 2000 rows.
Hoisting both props to module scope and memoising the rendered element on
rawContent drops re-render cost to ~0.1ms across every fixture and removes
the remount (DOM identity now survives). Initial mount and node switching
are unchanged, since those are genuine parses.
2. Initial parse of large GFM tables is quadratic and lives upstream in
remark-gfm: the same table text parses in 12.5ms without the plugin and
1156ms with it at 2000 rows. Not addressed here -- any mitigation is a
product decision and is tracked on the issue.
Note that document size is the wrong threshold for this: 562KB of prose parses
in 85ms while a 27KB GFM table takes 102ms. Row count, not bytes, predicts cost.
Rendered output is unchanged; the components map is moved verbatim. All 66
Explorer graph-workspace tests pass.
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
MarkdownContentViewer.tsx exported the isSafeUrl helper alongside the
component so it could be unit tested, which tripped
react-refresh/only-export-components.
Move the helper into a sibling pure module, markdownUrlSafety.ts,
following the existing GraphWorkspace convention for testable non-component
logic (graphAnalytics.ts, pluginRegistryPredicates.ts,
temporalLifecyclePredicates.ts). The function body is moved verbatim — the
scheme allowlist, protocol-relative rejection, whitespace-only guard and
malformed-URL handling are unchanged — so the existing URL-safety tests pass
untouched apart from the import path.
The component module now exports only its component and prop type, clearing
the lint error without any change to the lint configuration.
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
Dataset(default_union=True) presents triples from every named graph as a
single merged view and is itself an rdflib.Graph subclass, so it satisfies
_convert_to_dict()'s Graph-typed contract directly. Drops the O(n) manual
quad-copy loop while keeping the same named-graph fix and behavior.
_tool_export_graph fell through to json.dumps(kg) for any format outside
the RDF set, including values never declared in the tool's own inputSchema
enum. Nothing in this server validates tool-call args against inputSchema
before dispatch, so a typo'd or unsupported format (e.g. "yaml") silently
returned JSON data labeled with the wrong format and no error.
Validate against the declared format list up front and reuse the same
constant for the inputSchema enum so the two can't drift apart again.
* fix(triplet_store): OxigraphStore silently ignores storage_path and skips flush
Two persistence bugs in OxigraphStore:
1. `storage_path=...` was silently swallowed by **config. The __init__
parameter is named `path`, so passing the project-conventional
`storage_path` (used by ProvenanceManager and other stores) left
self.path = None and the store silently degraded to in-memory —
no error, no warning, data gone on exit. Accept `storage_path` as
an alias for `path`.
2. add_triplets never called flush(). pyoxigraph auto-flushes via
background threads but, per its docs, "might lag a little bit" —
that lag is a race where reopening or crashing immediately after a
write observes fewer triples. Call flush() explicitly for on-disk
stores to close the window.
Both verified: with the fix, `OxigraphStore(storage_path=...)` persists
across reopen; without it, data is lost.
* fix(triplet_store): improve oxigraph persistence
* test(triplet_store): clarify oxigraph persistence test
---------
Co-authored-by: administrator <administrator@administratordeMac-mini.local>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
* fix: preserve generation kwargs in relation extraction
* fix: include generation params in extraction cache keys
* fix: cover provider-specific generation params in extraction cache key
_GENERATION_CACHE_KEYS only covered the common OpenAI-shaped generation
params, so calls that differed only in Anthropic's system/stop_sequences,
Gemini's candidate_count, or Ollama's repeat_penalty/num_ctx/context_window
could still return a stale cached result generated under different settings.
Add these provider-specific keys to the cache key and add regression tests
covering system prompt, stop_sequences, and repeat_penalty.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
``requests.exceptions.RequestException`` subclasses ``OSError``, so the
``except (ImportError, OSError)`` handler in ``load_from_api`` swallowed
genuine network failures (connection errors, timeouts, HTTP errors) and
reported them as "requests library not available", hiding the real cause.
Remove the obsolete handler so those failures fall through to the generic
handler, which reports "Failed to load from API: ..." and chains the real
exception as ``__cause__``. Update the docstring's ``Raises`` section to
match the actual behavior.
Fixes#949
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
* Remove unreachable dead code
Delete symbols with no callers anywhere in the codebase, tests, or docs,
confirmed by a repo-wide search. These are internal/private or app-layer
(explorer) symbols, not part of the importable library's public API
(no __all__ / package re-export), so there is no user-facing change.
Removed:
- poc_runner.py: parse_import_csv_row (unused nested helper)
- change_management/version_storage.py: create_graph_snapshot_record
- context/graph_schema.py: drop_decision_schema
- explorer/dependencies.py: get_ws_manager (+ now-unused ConnectionManager import)
- explorer/routes/graph.py: _extract_node_embeddings (+ stale cross-ref comment)
- explorer/routes/ontology.py: ProposalState
- explorer/schemas.py: ErrorResponse, TemporalSnapshotResponse, ExportResponse,
StandardMessageResponse
- semantic_extract/methods.py: _parse_entity_result, _parse_triplet_result
- triplet_store/methods.py: _get_query_engine (+ now-unused _global_query_engine)
Co-Authored-By: Vinv-AI <309466812+Vinv-AI@users.noreply.github.com>
* Address review: drop now-orphaned helper and fix stale docstring
- Remove _coerce_embedding_vector from explorer/routes/graph.py: its only
non-recursive caller was _extract_node_embeddings (removed in this PR), so
it is now dead. The live coercion logic lives in
GraphSession._coerce_embedding_vector.
- Update explorer/dependencies.py module docstring: it no longer injects
ConnectionManager (get_ws_manager was removed); note that websocket manager
access is via app.state.ws_manager.
Co-Authored-By: Vinv-AI <309466812+Vinv-AI@users.noreply.github.com>
* Keep public helpers with a DeprecationWarning instead of removing them
create_graph_snapshot_record() and drop_decision_schema() are not
underscore-prefixed, so downstream users can import them directly from
their modules even though they are not re-exported from the package
__init__.py. A repo search only proves there are no in-tree callers.
Restore both unchanged and emit a DeprecationWarning on call, with a
matching ".. deprecated::" note in each docstring pointing at the
replacement. This keeps the PR non-breaking; the actual removal can
happen in a future major version.
The underscore-prefixed helper removals are unaffected.
---------
Co-authored-by: noQbot <noQbot@users.noreply.github.com>
Co-authored-by: Vinv-AI <309466812+Vinv-AI@users.noreply.github.com>
Co-authored-by: noQbot <anshul@vinv.ai>
* fix: guard integration HTTP requests against SSRF
* fix(openclaw): complete fallback validation and base URL handling
Address the remaining review findings in the OpenClaw integration.
- Strengthen fallback base_url validation to require a non-empty string, valid HTTP(S) scheme, netloc, and hostname.
- Strip leading and trailing whitespace from base_url before storing it.
- Replace the flaky endpoint-construction test that made a real network connection with mocked session assertions.
- Add coverage for _get and _post endpoint construction and timeout forwarding.
- Add regression tests for whitespace-padded base URLs and the fallback validation path.
These changes complete the Qodo review fixes and harden OpenClaw URL handling without changing the intended localhost/private deployment behavior.
The previous commit's fix to 03_Document_Parsing.ipynb collapsed the
cell's source array into a single string and dropped the trailing
newline. Restore the original array-of-lines formatting so the diff
is limited to the corrected badge URL.
Seven introduction notebooks linked to a different notebook's filename
in their Colab badge (off-by-one numbering), sending readers to the
wrong notebook or a 404. Point each badge back at its own file.
Add a Cite Us section to the README with BibTeX citation info, and
align it with docs/citation.md (author/organization: Semantica, 2026).
Update LICENSE and docs/project-license.md copyright holder to
Semantica, and replace the stale Hawksight-AI GitHub org slug with
semantica-agi across READMEs, plugin manifests, cookbook notebooks,
and GitHub templates.
_turtle_object() wrote an IRI-valued metadata value (currently only
sem:sourceUri, from the "uri" metadata key) straight into `<{value}>`
with no escaping. Turtle/N-Triples IRIREFs exclude control
characters, space, and <>"{}|^`\ unescaped, so a value shaped like
`<goodIRI> . <injected> <p> <o>` closed the reference early and let
the rest of the string be parsed as an attacker-chosen extra triple:
metadata={"uri": "https://x> . <https://injected> <https://p> <https://o"}
produced a well-formed Turtle/N-Triples document containing a triple
the caller never asked for.
RDF/XML was already safe (_rdfxml_metadata_lines runs the value
through _escape_xml before putting it in an rdf:resource attribute),
and JSON-LD is safe by construction (json.dumps makes structural
injection impossible) — only the Turtle/N-Triples "iri" literal path
in _turtle_object was unguarded.
Adds _safe_iri_ref(), a narrow percent-encoder for exactly the
characters an IRIREF may not contain unescaped. It's deliberately not
_as_turtle_iri: that also resolves registered prefixes, which a
metadata value never needs, so a dedicated guard stays simpler than
threading namespaces into a module-level helper that has no `self`.
Two regression tests, parametrised over turtle/ntriples: the `>`
delimiter-breaking payload from the report, and a control-character
(newline/tab) variant covering the other half of the excluded set.
Resolves the conflict in semantica/export/rdf_exporter.py between this
branch's metadata clauses (entity/graph metadata statements) and
main's IRI-normalization and XML-escaping hardening
(_as_turtle_iri / xml_escape, landed after this branch's last sync).
Kept both: entity/relationship/graph subjects and objects now go
through _as_turtle_iri (Turtle) or _as_turtle_iri + xml_escape
(RDF/XML), same as every other identifier in these serializers,
while the metadata-clause list building and graph_uri handling from
this branch are preserved unchanged. graph_uri is now normalized the
same way for consistency with the rest of the file.
Verified: tests/export + tests/ontology (411 tests) and the existing
Turtle-IRI regression suite (test_rdf_exporter_turtle_iris.py, 9
tests) all pass against the merged code.
Matches the README's existing convention for query params inside
HTML attribute URLs (e.g. the Trendshift badge), per review feedback
from Zohaib Hassan and Qodo on this PR.
Co-authored-by: OctoBored <212877535+OctoBored@users.noreply.github.com>
