Profiling the viewer in headless Chromium (real DOM, production React)
separated remark parse time, React commit time and DOM node count across
large-prose, large-code-block, deep-nested-list and GFM-table fixtures.
Two findings, one of which is fixed here.
1. Every re-render re-parsed the whole document and remounted the whole
subtree. remarkPlugins and the ~20-entry components map were inline
literals, so each render allocated fresh arrow components; React saw a new
element type per mapped tag and replaced the DOM rather than updating it. A
DOM-identity probe confirmed the remount on every fixture. Because
react-markdown runs the remark pipeline inside its own render, an unrelated
state change -- clicking Copy, toggling Preview/Source -- re-paid the full
parse. Measured 364ms for a 1000-row GFM table and 1121ms for 2000 rows.
Hoisting both props to module scope and memoising the rendered element on
rawContent drops re-render cost to ~0.1ms across every fixture and removes
the remount (DOM identity now survives). Initial mount and node switching
are unchanged, since those are genuine parses.
2. Initial parse of large GFM tables is quadratic and lives upstream in
remark-gfm: the same table text parses in 12.5ms without the plugin and
1156ms with it at 2000 rows. Not addressed here -- any mitigation is a
product decision and is tracked on the issue.
Note that document size is the wrong threshold for this: 562KB of prose parses
in 85ms while a 27KB GFM table takes 102ms. Row count, not bytes, predicts cost.
Rendered output is unchanged; the components map is moved verbatim. All 66
Explorer graph-workspace tests pass.
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
MarkdownContentViewer.tsx exported the isSafeUrl helper alongside the
component so it could be unit tested, which tripped
react-refresh/only-export-components.
Move the helper into a sibling pure module, markdownUrlSafety.ts,
following the existing GraphWorkspace convention for testable non-component
logic (graphAnalytics.ts, pluginRegistryPredicates.ts,
temporalLifecyclePredicates.ts). The function body is moved verbatim — the
scheme allowlist, protocol-relative rejection, whitespace-only guard and
malformed-URL handling are unchanged — so the existing URL-safety tests pass
untouched apart from the import path.
The component module now exports only its component and prop type, clearing
the lint error without any change to the lint configuration.
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
Dataset(default_union=True) presents triples from every named graph as a
single merged view and is itself an rdflib.Graph subclass, so it satisfies
_convert_to_dict()'s Graph-typed contract directly. Drops the O(n) manual
quad-copy loop while keeping the same named-graph fix and behavior.
_tool_export_graph fell through to json.dumps(kg) for any format outside
the RDF set, including values never declared in the tool's own inputSchema
enum. Nothing in this server validates tool-call args against inputSchema
before dispatch, so a typo'd or unsupported format (e.g. "yaml") silently
returned JSON data labeled with the wrong format and no error.
Validate against the declared format list up front and reuse the same
constant for the inputSchema enum so the two can't drift apart again.
* fix(triplet_store): OxigraphStore silently ignores storage_path and skips flush
Two persistence bugs in OxigraphStore:
1. `storage_path=...` was silently swallowed by **config. The __init__
parameter is named `path`, so passing the project-conventional
`storage_path` (used by ProvenanceManager and other stores) left
self.path = None and the store silently degraded to in-memory —
no error, no warning, data gone on exit. Accept `storage_path` as
an alias for `path`.
2. add_triplets never called flush(). pyoxigraph auto-flushes via
background threads but, per its docs, "might lag a little bit" —
that lag is a race where reopening or crashing immediately after a
write observes fewer triples. Call flush() explicitly for on-disk
stores to close the window.
Both verified: with the fix, `OxigraphStore(storage_path=...)` persists
across reopen; without it, data is lost.
* fix(triplet_store): improve oxigraph persistence
* test(triplet_store): clarify oxigraph persistence test
---------
Co-authored-by: administrator <administrator@administratordeMac-mini.local>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
* fix: preserve generation kwargs in relation extraction
* fix: include generation params in extraction cache keys
* fix: cover provider-specific generation params in extraction cache key
_GENERATION_CACHE_KEYS only covered the common OpenAI-shaped generation
params, so calls that differed only in Anthropic's system/stop_sequences,
Gemini's candidate_count, or Ollama's repeat_penalty/num_ctx/context_window
could still return a stale cached result generated under different settings.
Add these provider-specific keys to the cache key and add regression tests
covering system prompt, stop_sequences, and repeat_penalty.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
``requests.exceptions.RequestException`` subclasses ``OSError``, so the
``except (ImportError, OSError)`` handler in ``load_from_api`` swallowed
genuine network failures (connection errors, timeouts, HTTP errors) and
reported them as "requests library not available", hiding the real cause.
Remove the obsolete handler so those failures fall through to the generic
handler, which reports "Failed to load from API: ..." and chains the real
exception as ``__cause__``. Update the docstring's ``Raises`` section to
match the actual behavior.
Fixes#949
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
* Remove unreachable dead code
Delete symbols with no callers anywhere in the codebase, tests, or docs,
confirmed by a repo-wide search. These are internal/private or app-layer
(explorer) symbols, not part of the importable library's public API
(no __all__ / package re-export), so there is no user-facing change.
Removed:
- poc_runner.py: parse_import_csv_row (unused nested helper)
- change_management/version_storage.py: create_graph_snapshot_record
- context/graph_schema.py: drop_decision_schema
- explorer/dependencies.py: get_ws_manager (+ now-unused ConnectionManager import)
- explorer/routes/graph.py: _extract_node_embeddings (+ stale cross-ref comment)
- explorer/routes/ontology.py: ProposalState
- explorer/schemas.py: ErrorResponse, TemporalSnapshotResponse, ExportResponse,
StandardMessageResponse
- semantic_extract/methods.py: _parse_entity_result, _parse_triplet_result
- triplet_store/methods.py: _get_query_engine (+ now-unused _global_query_engine)
Co-Authored-By: Vinv-AI <309466812+Vinv-AI@users.noreply.github.com>
* Address review: drop now-orphaned helper and fix stale docstring
- Remove _coerce_embedding_vector from explorer/routes/graph.py: its only
non-recursive caller was _extract_node_embeddings (removed in this PR), so
it is now dead. The live coercion logic lives in
GraphSession._coerce_embedding_vector.
- Update explorer/dependencies.py module docstring: it no longer injects
ConnectionManager (get_ws_manager was removed); note that websocket manager
access is via app.state.ws_manager.
Co-Authored-By: Vinv-AI <309466812+Vinv-AI@users.noreply.github.com>
* Keep public helpers with a DeprecationWarning instead of removing them
create_graph_snapshot_record() and drop_decision_schema() are not
underscore-prefixed, so downstream users can import them directly from
their modules even though they are not re-exported from the package
__init__.py. A repo search only proves there are no in-tree callers.
Restore both unchanged and emit a DeprecationWarning on call, with a
matching ".. deprecated::" note in each docstring pointing at the
replacement. This keeps the PR non-breaking; the actual removal can
happen in a future major version.
The underscore-prefixed helper removals are unaffected.
---------
Co-authored-by: noQbot <noQbot@users.noreply.github.com>
Co-authored-by: Vinv-AI <309466812+Vinv-AI@users.noreply.github.com>
Co-authored-by: noQbot <anshul@vinv.ai>
* fix: guard integration HTTP requests against SSRF
* fix(openclaw): complete fallback validation and base URL handling
Address the remaining review findings in the OpenClaw integration.
- Strengthen fallback base_url validation to require a non-empty string, valid HTTP(S) scheme, netloc, and hostname.
- Strip leading and trailing whitespace from base_url before storing it.
- Replace the flaky endpoint-construction test that made a real network connection with mocked session assertions.
- Add coverage for _get and _post endpoint construction and timeout forwarding.
- Add regression tests for whitespace-padded base URLs and the fallback validation path.
These changes complete the Qodo review fixes and harden OpenClaw URL handling without changing the intended localhost/private deployment behavior.
The previous commit's fix to 03_Document_Parsing.ipynb collapsed the
cell's source array into a single string and dropped the trailing
newline. Restore the original array-of-lines formatting so the diff
is limited to the corrected badge URL.
Seven introduction notebooks linked to a different notebook's filename
in their Colab badge (off-by-one numbering), sending readers to the
wrong notebook or a 404. Point each badge back at its own file.
Add a Cite Us section to the README with BibTeX citation info, and
align it with docs/citation.md (author/organization: Semantica, 2026).
Update LICENSE and docs/project-license.md copyright holder to
Semantica, and replace the stale Hawksight-AI GitHub org slug with
semantica-agi across READMEs, plugin manifests, cookbook notebooks,
and GitHub templates.
_turtle_object() wrote an IRI-valued metadata value (currently only
sem:sourceUri, from the "uri" metadata key) straight into `<{value}>`
with no escaping. Turtle/N-Triples IRIREFs exclude control
characters, space, and <>"{}|^`\ unescaped, so a value shaped like
`<goodIRI> . <injected> <p> <o>` closed the reference early and let
the rest of the string be parsed as an attacker-chosen extra triple:
metadata={"uri": "https://x> . <https://injected> <https://p> <https://o"}
produced a well-formed Turtle/N-Triples document containing a triple
the caller never asked for.
RDF/XML was already safe (_rdfxml_metadata_lines runs the value
through _escape_xml before putting it in an rdf:resource attribute),
and JSON-LD is safe by construction (json.dumps makes structural
injection impossible) — only the Turtle/N-Triples "iri" literal path
in _turtle_object was unguarded.
Adds _safe_iri_ref(), a narrow percent-encoder for exactly the
characters an IRIREF may not contain unescaped. It's deliberately not
_as_turtle_iri: that also resolves registered prefixes, which a
metadata value never needs, so a dedicated guard stays simpler than
threading namespaces into a module-level helper that has no `self`.
Two regression tests, parametrised over turtle/ntriples: the `>`
delimiter-breaking payload from the report, and a control-character
(newline/tab) variant covering the other half of the excluded set.
Resolves the conflict in semantica/export/rdf_exporter.py between this
branch's metadata clauses (entity/graph metadata statements) and
main's IRI-normalization and XML-escaping hardening
(_as_turtle_iri / xml_escape, landed after this branch's last sync).
Kept both: entity/relationship/graph subjects and objects now go
through _as_turtle_iri (Turtle) or _as_turtle_iri + xml_escape
(RDF/XML), same as every other identifier in these serializers,
while the metadata-clause list building and graph_uri handling from
this branch are preserved unchanged. graph_uri is now normalized the
same way for consistency with the rest of the file.
Verified: tests/export + tests/ontology (411 tests) and the existing
Turtle-IRI regression suite (test_rdf_exporter_turtle_iris.py, 9
tests) all pass against the merged code.
Matches the README's existing convention for query params inside
HTML attribute URLs (e.g. the Trendshift badge), per review feedback
from Zohaib Hassan and Qodo on this PR.
Co-authored-by: OctoBored <212877535+OctoBored@users.noreply.github.com>
ProgressTracker attached ConsoleProgressDisplay unconditionally, so any
script or CI job that piped or redirected stdout had one progress bar per
stage written into its output, escape sequences included. A plain
`python demo.py > out.txt` captured 173 bytes of progress-bar noise around
10 bytes of the program's own output.
Console progress is now attached only when stdout is an interactive
terminal, when running under Jupyter, or when SEMANTICA_FORCE_PROGRESS is
set. FileProgressDisplay is untouched, so progress logging still works in
pipelines, and SEMANTICA_DISABLE_PROGRESS keeps its existing meaning and
still takes precedence.
Both progress environment variables are now documented in the README and
the utils reference; SEMANTICA_DISABLE_PROGRESS previously existed only in
the reference page.
Deviations from the issue: the issue suggested disabling the tracker on
non-TTY stdout. This gates the display instead, because disabling the
tracker would short-circuit before FileProgressDisplay and take file
progress logging down with it, and the ~20 modules that set
`progress_tracker.enabled = True` in __init__ would need the property
setter taught about TTY state to avoid undoing it. Gating the display
leaves both alone.
Design note: the claim comment on the issue proposed an
`enabled: Optional[bool] = None` constructor opt-in; during implementation
the opt-in became SEMANTICA_FORCE_PROGRESS, which needs no signature change
and follows the NO_COLOR/FORCE_COLOR convention. Known limitation: TTY
detection runs once at tracker construction (the tracker is a process-wide
singleton), so a process that redirects stdout after first use needs the
env vars to change behaviour.
Fixes#1185
_as_turtle_iri() re-encoded absolute IRIs wholesale, turning already-valid
percent-escapes like %20 into %2520. Only spans outside existing valid
%XX escapes are quoted now, so malformed escapes (%zz) still get repaired
while valid ones pass through unchanged.
serialize_to_ntriples()/serialize_to_rdfxml() also passed only the
@context-derived namespaces into _as_turtle_iri(), which shadowed the
built-in semantica:/rdf:/rdfs:/owl: prefixes entirely whenever any
@context was present. _as_turtle_iri() now always merges the built-ins
with whatever namespaces the caller passes.
The adapter inventory and connection examples referenced classes that
don't exist in semantica.graph_store (Neo4jGraphStore, NeptuneGraphStore,
AgeGraphStore) and used constructor kwargs that don't match the actual
adapters (username vs user, host vs endpoint, url vs endpoint, etc.),
verified against each adapter's real __init__ signature and by
constructing every example against the live classes.
- Correct class names: Neo4jStore, AmazonNeptuneStore, ApacheAgeStore
- Fix kwargs for all seven examples to match actual constructors
- Fix ApacheAgeStore's connection_string to libpq keyword=value format
instead of a postgresql:// DSN, which the adapter doesn't accept
- Reclassify Anzo from interface/BYO to built-in — AnzoStore is a real,
exported, tested adapter
- Add the two adapters missing from the inventory: FalkorDBStore and
OxigraphStore
- Replace the literal password='password' example with an env var
- Note a real RDF4JStore bug found while verifying the RDF4J example:
repository_id is a named constructor parameter but the implementation
reads it from **config instead, so it's silently ignored and the
store always connects to the "default" repository
