Compare commits

..
Author SHA1 Message Date
1928f80104 fix(plugins): resolve Qodo-flagged bugs in rewritten skill examples
Addresses all 7 findings from the automated review on this PR:
- ontology: OntologyEngine() has no store configured, so
  list_concepts/list_vocabularies always raise ProcessingError; construct
  it with a TripletStore
- ontology: ValidationResult's field is `valid`, not `is_valid`
- change: state_at()["nodes"] is a list of dicts, so set(...) on it raises
  TypeError: unhashable type: 'dict' — diff by node id instead
- provenance/change: storage_path is passed to sqlite3.connect() unexpanded,
  so a literal "~/.semantica/prov.db" fails to open — expanduser + mkdir
- change/query: load_from_file() checks the literal path, so "~/..." never
  resolves and the graph loads empty — expanduser before calling
- query: QueryEngine.execute_query requires an object exposing
  execute_sparql(); the TripletStore wrapper doesn't expose that, only the
  raw backend (e.g. OxigraphStore) does
- query: the Cypher example constructed Neo4jStore but never called
  execute_query()

Co-Authored-By: gyro <zhuffwct@gmail.com>
Co-Authored-By: KaifAhmad1 <mohammadk78600@gmail.com>
2026-09-11 16:45:17 +05:30
gyroandClaude Opus 5 e883facedb fix(plugins): repair CP1252 bytes in the decision skill
Two em dashes were stored as the raw CP1252 byte 0x97 rather than UTF-8, so
the file is not valid UTF-8. Strict UTF-8 readers fail on it, and lenient ones
render the frontmatter description as "Semantica <?> record".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:19:54 +08:00
gyroandClaude Opus 5 b4a0bc4063 fix(plugins): correct the extraction cache import in extract and validate
Both skills instruct the agent to clear the result cache via
`from semantica.semantic_extract.cache import _result_cache`, but the module
exports the `ExtractionCache` singleton as `extraction_cache`. The private
name never existed, so the first step of both skills raises ImportError.

`extraction_cache.clear()` is the equivalent call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:19:54 +08:00
gyroandClaude Opus 5 cbd33fba67 fix(plugins): rewrite five skills against the real v0.7.0 API
The ontology, policy, provenance, change and query skills documented classes
and methods that do not exist in the package. Every import in them fails, so
following the skill produces ImportError or AttributeError immediately:

| Documented | Reality in 0.7.0 |
| --- | --- |
| `semantica.policy.PolicyEngine` | no `semantica.policy` module; `PolicyEngine` is in `semantica.context` |
| `semantica.query.QueryEngine` | no `semantica.query` module; `QueryEngine` is in `semantica.triplet_store` |
| `semantica.ontology.OntologyManager` | no such class; use `OntologyEngine` / `OntologyValidator` |
| `semantica.provenance.ProvenanceTracer` | no such class; use `ProvenanceManager` |
| `semantica.provenance.change_tracker.ChangeTracker` | no such module; ontology versioning lives in `semantica.change_management` |

The method names were wrong too, so a path-only fix was not possible:
`.check()`, `.list_rules()`, `.trace_node()`, `.get_audit_log()`,
`.compute_diff()`, `.get_node_history()`, `.query_sparql()`, `.query_cypher()`
and `.search()` do not exist on any of the real classes.

Each skill is rewritten against signatures verified by introspection on an
installed 0.7.0. Two notes on scope:

- policy now leads with `ContextGraph.check_decision_rules()` /
  `enforce_decision_policy()`, which need no graph store, and keeps
  `context.PolicyEngine` as the managed-policy path.
- change previously conflated graph-state-over-time with ontology versioning.
  These are separate mechanisms in 0.7.0, so the skill now documents
  `ContextGraph.state_at()` and `change_management.VersionManager` separately.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:19:54 +08:00
Wei TaoandClaude Code 9d93a80840 fix(explorer): label the graph view control "Focus" (#1552)
* fix(explorer): label the graph view control "Focus"

The control read "Focused" before it was ever activated, which describes a
state the selection had already reached rather than the action available.
The view mode value, tooltip, enablement and active styling are unchanged.

The legend e2e drove this button by its accessible name, so the selector
moves with the label; it now also asserts the visible text, the
selection-dependent enablement and the active state.

Closes #1551

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(explorer): name the control in the grouped-selection hint, and trim the test

"Activate Focused mode" instructed the reader to press a control that no
longer carries that name. The surrounding strings describe the mode itself,
which is still called focused, so they stay.

Drop the label and active-state assertions from the colour-legend test: the
getByRole locator already fails when the accessible name is wrong, and the
rest belonged to the control's contract rather than to the legend's.

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-10 09:42:17 +05:30
a9f1b29292 fix(explorer): resolve ontology ownership on the backend for entity deep links (#1439)
* fix(explorer): resolve ontology ownership on the backend

Which registered ontology owns an entity was decided twice: the backend
applies nested-namespace boundaries, while the Ontology Editor did a bare
prefix match. The two had already drifted, so a deep link to an entity in an
unregistered nested namespace selected the parent ontology whose /graph
response excludes that entity, and the selection silently failed.

/api/ontology/entity now returns owning_ontology, resolved with the same rule
the graph endpoint filters by, and the editor prefers it. The frontend
namespace guess stays as the fallback for a missing verdict, documented as
non-authoritative.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(explorer): keep the backend's no-owner verdict authoritative

Review follow-up: loadOntologyEntityOwner collapsed the backend's
explicit owning_ontology: null into undefined, re-activating the
namespace prefix guess for exactly the unregistered-nested-namespace
case this PR exists to fix. The owner verdict is now three-state
(owner / authoritative none / unavailable) and resolveEditorOntology
in the model suppresses inference on an authoritative none; only an
unavailable verdict may fall back. Model tests pin all three states.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(explorer): trust the backend to send owning_ontology

The Explorer bundle ships in the same wheel as the route that emits this
field, so the legacy-response branch could never run. Dropping it lets
the type say what the wire actually carries, leaving undefined to mean
only what it should: the request failed.

Note why the endpoint derives its ontology-URI set inline rather than
calling _known_ontology_uris, so the next reader does not consolidate a
graph scan back in.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(explorer): stop the registry default overriding a no-owner verdict

resolveEditorOntology returned string | undefined, so the caller wrote
`resolved || entries[0]?.uri` and an authoritative "nothing owns this
entity" fell straight through to an arbitrary registry entry. When that
entry happened to be the parent, the deep link opened the parent whose
graph excludes the entity — the bug the verdict exists to prevent. Only
the ordering of the registry in the earlier test hid it.

It now returns a union: unowned and unresolved both mean "no ontology to
open" but the editor treats them oppositely, so collapsing them with ||
is a type error rather than a silent regression. An unowned entity is
reported on the canvas instead of quietly opening the wrong ontology.

Also:

- /entity resolves ownership through _known_ontology_uris, the same
  helper /graph uses, instead of deriving it from a get_nodes scan capped
  at 999,999. Past that cap the set was silently truncated and the two
  endpoints could disagree about who owns a node.
- _resolve_owning_ontology does one pass over the candidates rather than
  one pass per candidate, each rescanning the whole set: 516us -> 9us at
  50 ontologies, 125ms -> 138us at 800, same answers throughout. A test
  pins it against _node_belongs_to_ontology so the hand-rolled version
  cannot drift from the membership rule it has to mirror.
- An explicit scheme_uri is honoured even when the registry does not list
  it, on both sides. Discarding it and guessing by namespace answered a
  question nobody asked; an unregistered owner now surfaces as an
  explicit error from /graph instead.
- A missing owning_ontology field reads as "no verdict", not as the
  authoritative "nothing owns this". That claim now suppresses selection
  outright, so it must not be inferred from an absent field.

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
2026-09-09 23:39:39 +05:30
Wei TaoandClaude Fable 5 0338f90bca refactor(explorer): own Ontology Hub URL state in one module (#1440)
The deep-link protocol introduced in #1278 lived as bare "ontologyTab" /
"ontologyEntity" literals in five places across three files, each with its own
URLSearchParams plumbing and try/catch. Nothing tied the pieces together: in
particular the rule that a selection written under one ontology must be cleared
when the active ontology changes — otherwise a reload resolves the stale entity
and jumps back to the old ontology — was a comment at one call site with no
mechanism behind it.

ontologyUrlState.ts now owns the parameter names as private constants and
exposes the protocol as intent-named operations, with the write/clear pairing
documented where both halves live. Parsing and serialization are pure functions
over a search string, so they are covered by tests without a DOM; the window
and history.replaceState interaction stays in thin shells.

Absent parameters still read as undefined while blank ones read as empty
strings, which preserves the differing presence checks the workspace shell and
the tab selector each relied on.

One behavior change, inherited from all five original call sites: writing the
query string dropped any URL fragment, because replaceState with a bare "?..."
replaces the whole tail. updateSearch now carries window.location.hash across,
which fixes it for every writer at once — this module is the only place that
knows how the URL is written, so it is the only place the fix belongs.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-09 21:50:13 +05:30
Sameer KadamandSameer Kadam def18cd552 fix(explorer): remove stale 2030 temporal bound (#1549)
Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
2026-09-09 17:41:11 +05:30
Wei TaoandSameer Kadam 6b55cf1101 fix(explorer): end the temporal scrubber at now instead of inventing 2030 (#1541)
TimelinePanel fell back to a hardcoded 2030-01-01 whenever
/api/temporal/bounds reported max: null, then placed the playhead at the
midpoint of that fabricated window. session.get_temporal_bounds leaves max
open for any graph whose nodes carry valid_from instants and no
valid_until, so this was the normal response shape rather than bad data:
the header advertised a range no data supported and the workspace's first
/api/temporal/snapshot request asked about a time years ahead of the
present.

The fallback is now a `now` captured once per mount, and defaultTime is
that same `now` clamped into the range, so the initial snapshot describes
the current state. Three settings tuned for the fictional ~60-year window
follow from it: zoomMin drops from a year to a day, the timeAxis/format
pinning to 5-year ticks is removed so vis-timeline picks a granularity for
the real span, and the fixed 6-month play step becomes span/60 with a
one-day floor, which keeps a play-through at roughly 60 frames whether the
graph covers months or a decade.

The bound and step arithmetic moves into temporalScrubberBounds.ts so it
can be asserted directly, alongside the existing temporalLifecycle
predicate tests.

Closes #1536

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
2026-09-09 17:26:23 +05:30
bd41a5a24f fix(vector-store): prevent in-memory vector ID reuse (#1546)
* fix(vector-store): prevent in-memory vector id reuse

* fix(vector-store): synchronize in-memory mutations

* fix(vector-store): clarify no-silent-overwrite guarantee for in-memory ID generation

The while-loop in store_vectors() already prevents any generated candidate
from landing on a live key: it breaks only when the candidate is absent
from both pre_existing (the live store at lock-entry) and within_batch
(IDs chosen earlier in the same call).

This commit:
- Renames 'existing' to 'pre_existing' and introduces 'within_batch' to
  make the two-level de-dupe explicit and self-documenting.
- Tightens the break condition to 'not in pre_existing and not in
  within_batch' so within-batch duplicates are also guarded.
- Adds test_auto_generated_id_never_silently_overwrites_live_vector:
  stores 5, deletes 3, stores 2 more and asserts none of the new IDs
  land on a surviving vec_N.
- Adds test_collision_detection_no_silent_overwrite_even_with_corrupted_counter:
  rewinds _next_id to 0 with live vectors present and proves the loop
  finds a free slot without overwriting either existing vector or its
  metadata — the no-silent-overwrite invariant holds even under counter
  corruption.

The skip-over-occupied-key behaviour for caller-inserted vec_N IDs is
intentional and preserved (test_counter_skips_explicit_vec_n_ids), matching
FAISSStore's identical pattern.

Closes reviewer finding: 'Vector collisions do not fail writes'.

* fix(vector-store): protect concurrent in-memory access

---------

Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan>
Co-authored-by: Zohaib Hassnain <109234410+ZohaibHassan16@users.noreply.github.com>
2026-09-09 17:02:27 +05:30
dependabot[bot] fa8002e554 docker(deps): bump python from 3.13-slim to 3.14-slim (#1547)
Bumps python from 3.13-slim to 3.14-slim.

---
updated-dependencies:
- dependency-name: python
  dependency-version: 3.14-slim
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-09 15:29:53 +05:30
Mohd KaifandClaude Sonnet 5 d4127131ba fix(ci): match pip-audit ignore list against vuln aliases, not just id (#1543)
#1540 added GHSA-4j2p-28q2-5m79 to security-scan.yml's IGNORED_VULN_IDS to
suppress the open, unpatched accelerate<=1.14.0 path traversal advisory
(Semantica doesn't call load_checkpoint_in_model/load_checkpoint_and_dispatch).
That commit's own CI run still failed: pip-audit's OSV-backed report picked
CVE-2026-69112 as the vuln's canonical `id` and demoted the GHSA id to an
alias, but the shell/JS matching only ever compared against `.id`.

List both identifiers and match against `.id` plus `.aliases` (which
pip-audit includes by default for JSON output) in the audit gate, the
"Vulnerability details" printer, and the PR-comment script, so an ignored
advisory is excluded regardless of which alias the report surfaces as
canonical. Verified against the actual failing report from run 34296586683 -
the corrected filter now yields 0 actionable vulnerabilities.

Also add osv-scanner.toml (per GHSA-4j2p-28q2-5m79's Scorecard code-scanning
remediation) so the weekly Scorecard "Vulnerabilities" check stops flagging
the same accepted, unpatched advisory.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 12:46:02 +05:30
Zohaib Hassnain 42bbe51382 fix(ci): ignore upstream accelerate vulnerability in security scan (#1540)
Add GHSA-4j2p-28q2-5m79 to IGNORED_VULN_IDS in security-scan.yml.

accelerate<=1.14.0 has an open path traversal advisory (GHSA-4j2p-28q2-5m79) in load_checkpoint_in_model. 1.14.0 is currently the latest available release on PyPI, so no upstream patch exists yet. Semantica does not expose or call sharded checkpoint loading, making this non-actionable. Re-evaluate once an updated accelerate release is published.
2026-09-09 05:48:45 +05:00
31 changed files with 1714 additions and 216 deletions
+32 -9
View File
@@ -155,12 +155,20 @@ jobs:
# Vulnerability IDs reviewed and accepted as non-actionable for this
# project:
# - GHSA-4j2p-28q2-5m79: accelerate<=1.14.0 (transitive via docling-slim).
# Path traversal in sharded checkpoint index loading (load_checkpoint_in_model).
# 1.14.0 is the latest available PyPI release; no upstream patch exists yet.
# Semantica does not load arbitrary user checkpoints. Re-evaluate once
# accelerate releases a fixed version.
IGNORED_VULN_IDS="GHSA-4j2p-28q2-5m79"
# - GHSA-4j2p-28q2-5m79 (aka CVE-2026-69112): accelerate<=1.14.0
# (transitive via docling-slim). Path traversal in sharded checkpoint
# index loading (load_checkpoint_in_model). 1.14.0 is the latest
# available PyPI release; no upstream patch exists yet. Semantica does
# not load arbitrary user checkpoints. Re-evaluate once accelerate
# releases a fixed version.
# NOTE: pip-audit's OSV-backed report may surface either identifier as
# the primary `id` (with the other listed under `aliases`) depending on
# which alias the backing database picks as canonical, so both need to
# be listed here and the matching below checks aliases too - see
# https://github.com/semantica-agi/semantica/actions/runs/34296586683
# where this ignore list had only the GHSA id but the report's `id`
# was the CVE, so the gate still failed.
IGNORED_VULN_IDS="GHSA-4j2p-28q2-5m79,CVE-2026-69112"
# Exported so the "Comment PR with Security Results" step below can
# apply the same exclusion list to the raw report - it reads
@@ -176,9 +184,16 @@ jobs:
# no vulns field at all (see the skip_reason handling above) -
# without the fallback, iterating `null[]` raises inside jq and
# this whole computation silently evaluates to empty.
#
# Matching checks `.id` AND `.aliases` (pip-audit includes aliases by
# default for JSON output): the OSV-backed report can surface either
# the GHSA or the CVE identifier as the canonical `id` for the same
# advisory, with the other one demoted to an alias, so matching on
# `.id` alone is not reliable.
VULNS=$(jq --arg ignored "$IGNORED_VULN_IDS" '
($ignored | split(",") | map(select(length > 0))) as $ignore_list
| [.dependencies[] | (.vulns // [])[] | select(.id as $id | ($ignore_list | index($id)) | not)]
| [.dependencies[] | (.vulns // [])[]
| select(([.id] + (.aliases // [])) as $ids | ($ids - $ignore_list | length) == ($ids | length))]
| length
' pip-audit-report.json 2>/dev/null)
@@ -199,7 +214,8 @@ jobs:
jq --arg ignored "$IGNORED_VULN_IDS" -r '
($ignored | split(",") | map(select(length > 0))) as $ignore_list
| .dependencies[] as $dependency
| ($dependency.vulns // [])[] | select(.id as $id | ($ignore_list | index($id)) | not)
| ($dependency.vulns // [])[]
| select(([.id] + (.aliases // [])) as $ids | ($ids - $ignore_list | length) == ($ids | length))
| "- \($dependency.name)==\($dependency.version): \(.id)"
' pip-audit-report.json || true
exit 1
@@ -345,9 +361,16 @@ jobs:
return null;
}
// A vuln's canonical `id` and its `aliases` (e.g. GHSA vs. CVE
// for the same advisory) are checked together - mirrors the
// shell gate above, which needs the same fallback because
// pip-audit's OSV-backed report doesn't consistently pick the
// same identifier as canonical across advisories.
return data.dependencies.flatMap((dependency) =>
(dependency.vulns || [])
.filter((vulnerability) => !ignoredVulnIds.includes(vulnerability.id))
.filter((vulnerability) =>
![vulnerability.id, ...(vulnerability.aliases || [])].some((id) => ignoredVulnIds.includes(id))
)
.map(
(vulnerability) => `- \`${dependency.name}==${dependency.version}\`: ${vulnerability.id}` +
(vulnerability.fix_versions?.length ? ` (fixed by ${vulnerability.fix_versions.join(', ')})` : '')
+1 -1
View File
@@ -31,7 +31,7 @@ RUN mkdir -p /app/semantica && npm run build
# `pip index versions gensim` / the project's PyPI files page, not just
# whether `uv pip compile` resolves (resolution only reads sdist metadata,
# it doesn't attempt the build that fails here).
FROM python:3.13-slim@sha256:7ce4b6dfe35e55397b7cda544f8a13f191b7ae28dc5aad71fe664dbc9bc2623f AS runtime
FROM python:3.14-slim@sha256:cad9a2c871761c413caa6fdd6441c783451e740a48aaeba60ae62a8b53525ef6 AS runtime
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
+1 -1
View File
@@ -9,7 +9,7 @@
"lint": "eslint .",
"preview": "vite preview",
"test:graph-store": "node --test tests/graphStore.multi-edge.test.mjs",
"test:graph-workspace": "node --import tsx --test tests/markdownContentViewer.test.ts tests/markdownEditorInteraction.test.tsx tests/markdownEditorState.test.ts tests/nodeMarkdownSync.test.ts tests/graphSceneState.display.test.ts tests/temporalLifecycle.test.ts tests/deterministicExplorerRendering.test.ts tests/explorerCapabilities.test.tsx tests/smallGraphLayout.test.ts tests/realtimeGraphAttributes.test.ts tests/ontologyEditorModel.test.ts tests/graphColorLegend.test.ts",
"test:graph-workspace": "node --import tsx --test tests/markdownContentViewer.test.ts tests/markdownEditorInteraction.test.tsx tests/markdownEditorState.test.ts tests/nodeMarkdownSync.test.ts tests/graphSceneState.display.test.ts tests/temporalLifecycle.test.ts tests/temporalScrubberBounds.test.ts tests/deterministicExplorerRendering.test.ts tests/explorerCapabilities.test.tsx tests/smallGraphLayout.test.ts tests/realtimeGraphAttributes.test.ts tests/ontologyEditorModel.test.ts tests/graphColorLegend.test.ts tests/ontologyUrlState.test.ts",
"test:deterministic-e2e": "node --import tsx --test tests/deterministicExplorerRendering.e2e.ts",
"test:graph-legend-e2e": "node --import tsx --test tests/graphColorLegend.e2e.ts",
"test:plugin-registry": "node --import tsx --test tests/pluginRegistry.temporal.test.mjs"
+2 -9
View File
@@ -19,6 +19,7 @@ import {
import { ErrorBoundary } from './ErrorBoundary';
import { ExploreWorkspaceTabs, type ExploreView } from './ExploreWorkspaceTabs';
import { fetchAgentMemoryAvailability } from './explorerCapabilities';
import { hasOntologyUrlState } from './workspaces/OntologyWorkspace/ontologyUrlState';
const DecisionWorkspace = lazy(() => import('./workspaces/DecisionWorkspace/DecisionWorkspace').then((module) => ({ default: module.DecisionWorkspace })));
const DiffMergeWorkspace = lazy(() => import('./workspaces/DiffMergeWorkspace/DiffMergeWorkspace').then((module) => ({ default: module.DiffMergeWorkspace })));
@@ -96,15 +97,7 @@ const navItems: NavItem[] = [
];
function readInitialWorkspace(): WorkspaceId {
try {
const params = new URLSearchParams(window.location.search);
if (params.has("ontologyTab") || params.has("ontologyEntity")) {
return "ontology-hub";
}
} catch {
// Default to the welcome screen when URL state is unavailable.
}
return "welcome";
return hasOntologyUrlState() ? 'ontology-hub' : 'welcome';
}
const shellStyles = `
@@ -346,7 +346,7 @@ export function GraphInspectorPanel({
<div style={{ color: GRAPH_THEME.ui.text.strong, fontWeight: 600, marginBottom: 6 }}>Selected item is not directly inspectable in the current graph.</div>
<div style={{ color: GRAPH_THEME.ui.text.body, fontSize: 13, lineHeight: 1.6 }}>
{canActivateFocused
? "Activate Focused mode to resolve this grouped selection to its canonical node."
? "Use Focus to resolve this grouped selection to its canonical node."
: (focusedUnavailableReason ?? "Focused mode is unavailable for the current selection.")}
</div>
</div>
@@ -2791,7 +2791,7 @@ export function GraphWorkspace({ externalFocusNodeId, externalFocusToken, onDirt
},
{
id: "view-focused",
label: "Focused",
label: "Focus",
title: canActivateFocusedMode
? "Inspect the selected node in a focused local graph"
: (focusedSelectionResolution.reason ?? "Focused mode is unavailable for the current selection"),
@@ -4,6 +4,7 @@ import { Timeline } from "vis-timeline";
import type { TimelineOptions } from "vis-timeline";
import "vis-timeline/styles/vis-timeline-graph2d.css";
import { GRAPH_THEME } from "./graphTheme";
import { DEFAULT_MIN_DATE, resolvePlayStepMs, resolveScrubberBounds } from "./temporalScrubberBounds";
export interface TimelinePanelProps {
onTimeChange: (time: Date) => void;
@@ -11,11 +12,9 @@ export interface TimelinePanelProps {
maxDate?: string;
}
const DEFAULT_MIN_DATE = new Date("1970-01-01T00:00:00Z");
const DEFAULT_MAX_DATE = new Date("2030-01-01T00:00:00Z");
const PLAYHEAD_ID = "playhead";
const PLAY_INTERVAL_MS = 500;
const PLAY_STEP_MONTHS = 6;
const ONE_DAY_MS = 1000 * 60 * 60 * 24;
const VIS_OVERRIDE_CSS = `
.sem-timeline-wrap .vis-timeline { border: none !important; background: transparent !important; overflow: visible !important; }
@@ -54,12 +53,6 @@ const VIS_OVERRIDE_CSS = `
.sem-timeline-wrap .vis-panel.vis-left { display: none !important; }
`;
function safeDate(value: string | undefined, fallback: Date): Date {
if (!value) return fallback;
const parsed = new Date(value);
return Number.isNaN(parsed.getTime()) ? fallback : parsed;
}
function formatPlayheadLabel(value: Date): string {
return `${value.getFullYear()}/${String(value.getMonth() + 1).padStart(2, "0")}`;
}
@@ -72,9 +65,13 @@ export function TimelinePanel({ onTimeChange, minDate, maxDate }: TimelinePanelP
const [isPlaying, setIsPlaying] = useState(false);
const [displayDate, setDisplayDate] = useState(formatPlayheadLabel(DEFAULT_MIN_DATE));
const minBound = useMemo(() => safeDate(minDate, DEFAULT_MIN_DATE), [minDate]);
const maxBound = useMemo(() => safeDate(maxDate, DEFAULT_MAX_DATE), [maxDate]);
const defaultTime = useMemo(() => new Date(Math.round((minBound.getTime() + maxBound.getTime()) / 2)), [maxBound, minBound]);
// Captured once per mount so re-renders keep the same reference and do not
// retrigger the timeline effect below.
const now = useMemo(() => new Date(), []);
const { minBound, maxBound, defaultTime } = useMemo(
() => resolveScrubberBounds({ minDate, maxDate, now }),
[maxDate, minDate, now],
);
useEffect(() => {
if (!containerRef.current) return;
@@ -91,12 +88,10 @@ export function TimelinePanel({ onTimeChange, minDate, maxDate }: TimelinePanelP
showCurrentTime: false,
zoomable: true,
moveable: true,
zoomMin: 1000 * 60 * 60 * 24 * 365,
zoomMin: ONE_DAY_MS,
zoomMax: 1000 * 60 * 60 * 24 * 365 * 80,
showMajorLabels: true,
showMinorLabels: true,
timeAxis: { scale: "year", step: 5 },
format: { minorLabels: { year: "YYYY" }, majorLabels: { year: "YYYY" } },
orientation: { axis: "bottom" },
margin: { item: 0, axis: 0 },
selectable: false,
@@ -134,8 +129,7 @@ export function TimelinePanel({ onTimeChange, minDate, maxDate }: TimelinePanelP
playIntervalRef.current = setInterval(() => {
const timeline = timelineRef.current;
if (!timeline) return;
const next = new Date(playheadRef.current);
next.setMonth(next.getMonth() + PLAY_STEP_MONTHS);
const next = new Date(playheadRef.current.getTime() + resolvePlayStepMs(minBound, maxBound));
if (next >= maxBound) {
next.setTime(minBound.getTime());
}
@@ -91,7 +91,7 @@ export const temporalOverlayPlugin: GraphPlugin = {
<div style={detailRowStyle}>
<span style={detailLabelStyle}>Bounds</span>
<span style={detailValueStyle}>
{(temporal?.minDate ?? "1970")} {(temporal?.maxDate ?? "2030")}
{(temporal?.minDate ?? "1970")} {(temporal?.maxDate ?? "now")}
</span>
</div>
<div style={detailRowStyle}>
@@ -0,0 +1,46 @@
export interface ScrubberBoundsInput {
minDate?: string;
maxDate?: string;
now: Date;
}
export interface ScrubberBounds {
minBound: Date;
maxBound: Date;
defaultTime: Date;
}
export const DEFAULT_MIN_DATE = new Date("1970-01-01T00:00:00Z");
const ONE_DAY_MS = 1000 * 60 * 60 * 24;
const PLAY_FRAMES = 60;
function parseBound(value: string | undefined, fallback: Date): Date {
if (!value) return fallback;
const parsed = new Date(value);
return Number.isNaN(parsed.getTime()) ? fallback : parsed;
}
function clamp(value: Date, minBound: Date, maxBound: Date): Date {
if (value < minBound) return minBound;
if (value > maxBound) return maxBound;
return value;
}
/**
* `/api/temporal/bounds` reports `max: null` for graphs whose nodes carry
* `valid_from` instants and no `valid_until`, which is the common case rather
* than malformed data. Such a graph is known up to the present and no further,
* so `now` is the honest upper bound and the honest starting playhead.
*/
export function resolveScrubberBounds({ minDate, maxDate, now }: ScrubberBoundsInput): ScrubberBounds {
const minBound = parseBound(minDate, DEFAULT_MIN_DATE);
const maxBound = parseBound(maxDate, now);
const orderedMax = maxBound > minBound ? maxBound : minBound;
return { minBound, maxBound: orderedMax, defaultTime: clamp(now, minBound, orderedMax) };
}
/** Keeps a play-through at ~PLAY_FRAMES steps whatever the span, with a one-day floor. */
export function resolvePlayStepMs(minBound: Date, maxBound: Date): number {
const span = maxBound.getTime() - minBound.getTime();
return Math.max(ONE_DAY_MS, Math.round(span / PLAY_FRAMES));
}
@@ -28,11 +28,12 @@ import { loadOntologyEntityOwner, loadOntologyGraph } from "./api";
import type { OntologyGraphEdge, OntologyGraphNode } from "./api";
import {
classifyNodeType,
inferOntologyUri,
isEditableEntityType,
ONTOLOGY_MINIMAP_THEME,
resolveEditorOntology,
} from "./ontologyEditorModel";
import type { EditorEntityType, RegistryEntry } from "./ontologyEditorModel";
import { clearEntitySelection, readOntologyUrlState, writeEntitySelection } from "./ontologyUrlState";
type OntologyNodeData = {
label?: string;
@@ -137,11 +138,7 @@ interface DraftDiff {
}
function requestedEntityUri(): string {
try {
return new URLSearchParams(window.location.search).get("ontologyEntity") || "";
} catch {
return "";
}
return readOntologyUrlState().entityUri || "";
}
function nodeLabel(node: OntologyGraphNode): string {
@@ -225,6 +222,7 @@ export function OntologyEditor() {
const [flowInstance, setFlowInstance] = useState<ReactFlowInstance<OntologyNode, OntologyEdge> | null>(null);
const [isLoadingGraph, setIsLoadingGraph] = useState(false);
const [graphError, setGraphError] = useState("");
const [unownedEntity, setUnownedEntity] = useState("");
const [draftDiff, setDraftDiff] = useState<DraftDiff>({
added_classes: [],
removed_classes: [],
@@ -250,11 +248,21 @@ export function OntologyEditor() {
? loadOntologyEntityOwner(requested).catch(() => undefined)
: Promise.resolve(undefined),
])
.then(([entries, explicitOwner]: [RegistryEntry[], string | undefined]) => {
.then(([entries, ownerVerdict]: [RegistryEntry[], string | null | undefined]) => {
if (cancelled) return;
setRegistry(entries);
const inferredOntology = inferOntologyUri(entries, requested, explicitOwner);
setOntologyUri((current) => current || inferredOntology || entries[0]?.uri || "");
const resolution = resolveEditorOntology(entries, requested, ownerVerdict);
// The registry default is the right landing place for "no entity asked
// for", but not for "the backend says nothing owns the entity that was
// asked for" — that would open an arbitrary ontology whose graph
// excludes the entity, and report nothing about why.
if (resolution.status === "unowned") {
setUnownedEntity(resolution.entityUri);
return;
}
setUnownedEntity("");
const resolvedOntology = resolution.status === "resolved" ? resolution.uri : undefined;
setOntologyUri((current) => current || resolvedOntology || entries[0]?.uri || "");
})
.catch((error) => {
console.error("Failed to load ontology registry:", error);
@@ -377,14 +385,7 @@ export function OntologyEditor() {
const selectNode = useCallback((node: OntologyNode) => {
setSelectedElement(node);
try {
const params = new URLSearchParams(window.location.search);
params.set("ontologyTab", "editor");
params.set("ontologyEntity", node.id);
window.history.replaceState(null, "", `?${params.toString()}`);
} catch {
// URL state is optional; the editor selection still works without it.
}
writeEntitySelection(node.id);
}, []);
const saveDraft = useCallback(async () => {
@@ -554,15 +555,8 @@ export function OntologyEditor() {
onChange={(event) => {
setOntologyUri(event.target.value);
setSelectedElement(null);
try {
// Drop the previous ontology's entity from the URL, or a reload
// would resolve the stale ID and jump back to that ontology.
const params = new URLSearchParams(window.location.search);
params.delete("ontologyEntity");
window.history.replaceState(null, "", `?${params.toString()}`);
} catch {
// URL state is optional; switching ontologies still works.
}
setUnownedEntity("");
clearEntitySelection();
}}
style={selectStyle}
>
@@ -633,7 +627,12 @@ export function OntologyEditor() {
{!isLoadingGraph && graphError && (
<div style={{ ...canvasMessageStyle, color: "#ff9a8d" }}>{graphError}</div>
)}
{!isLoadingGraph && !graphError && ontologyUri && nodes.length === 0 && (
{!isLoadingGraph && !graphError && unownedEntity && (
<div style={{ ...canvasMessageStyle, color: "#f2b66d" }}>
No registered ontology owns {unownedEntity}. Pick an ontology above to start editing.
</div>
)}
{!isLoadingGraph && !graphError && !unownedEntity && ontologyUri && nodes.length === 0 && (
<div style={canvasMessageStyle}>This ontology has no editable classes or properties.</div>
)}
@@ -32,7 +32,11 @@ export type OntologyGraphResponse = {
};
export type OntologyEntityOwner = {
source_ontology?: string;
// Optional on purpose, unlike OntologyGraphNode.entity_type. There, a missing
// field degrades to a read-only node — benign. Here it would be read as an
// authoritative "nothing owns this entity", which now suppresses selection
// outright, so presence has to be checked rather than assumed.
owning_ontology?: string | null;
};
async function parseResponse<T>(response: Response): Promise<T> {
@@ -63,10 +67,21 @@ export async function loadOntologyGraph(uri: string, signal?: AbortSignal): Prom
);
}
export async function loadOntologyEntityOwner(uri: string): Promise<string | undefined> {
// Three-state verdict: a string names the owner, null is the backend's
// authoritative "no known ontology owns this entity", and undefined means the
// request failed so there is no verdict to act on.
export type OntologyOwnerVerdict = string | null | undefined;
export async function loadOntologyEntityOwner(uri: string): Promise<OntologyOwnerVerdict> {
const response = await fetch(`/api/ontology/entity/${encodeURIComponent(uri)}`);
if (!response.ok) return undefined;
return (await response.json() as OntologyEntityOwner).source_ontology;
const owner = await response.json() as OntologyEntityOwner | null;
// Only a field that is actually there carries the verdict. Coercing an absent
// field to null would assert the strongest available claim — "nothing owns
// this" — on the weakest possible evidence, and that claim now stops the
// editor selecting an ontology at all.
const verdict = owner?.owning_ontology;
return verdict === undefined ? undefined : verdict;
}
export async function loadAlignments(uri?: string): Promise<OntologyAlignment[]> {
@@ -13,6 +13,7 @@ import { OntologyManager } from "./OntologyManager";
import { OntologyEditor } from "./OntologyEditor";
import { ShaclStudio } from "./ShaclStudio";
import { VersionsTab } from "./VersionsTab";
import { readOntologyUrlState, writeEntitySelection, writeTab } from "./ontologyUrlState";
export type OntologyHubTab =
| "registry"
@@ -22,8 +23,6 @@ export type OntologyHubTab =
| "health"
| "shacl";
const TAB_PARAM = "ontologyTab";
const TABS: { id: OntologyHubTab; label: string; icon: typeof GitMerge }[] = [
{ id: "registry", label: "Registry", icon: BookMarked },
{ id: "editor", label: "Editor", icon: Sliders },
@@ -33,37 +32,23 @@ const TABS: { id: OntologyHubTab; label: string; icon: typeof GitMerge }[] = [
{ id: "shacl", label: "SHACL", icon: Shield },
];
function readTabParam(): OntologyHubTab {
try {
const params = new URLSearchParams(window.location.search);
const raw = params.get(TAB_PARAM);
if (raw && TABS.some((t) => t.id === raw)) return raw as OntologyHubTab;
if (params.get("ontologyEntity")) return "editor";
} catch {
// ignore
}
function readInitialTab(): OntologyHubTab {
const { tab, entityUri } = readOntologyUrlState();
const requested = TABS.find((candidate) => candidate.id === tab);
if (requested) return requested.id;
if (entityUri) return "editor";
return "registry";
}
function writeTabParam(tab: OntologyHubTab) {
try {
const params = new URLSearchParams(window.location.search);
params.set(TAB_PARAM, tab);
window.history.replaceState(null, "", `?${params.toString()}`);
} catch {
// ignore
}
}
interface OntologyWorkspaceProps {
onJumpToGraphNode?: (nodeId: string) => void;
}
export function OntologyWorkspace({ onJumpToGraphNode }: OntologyWorkspaceProps) {
const [activeTab, setActiveTab] = useState<OntologyHubTab>(readTabParam);
const [activeTab, setActiveTab] = useState<OntologyHubTab>(readInitialTab);
useEffect(() => {
writeTabParam(activeTab);
writeTab(activeTab);
}, [activeTab]);
const handleTabChange = useCallback((tab: OntologyHubTab) => {
@@ -71,10 +56,7 @@ export function OntologyWorkspace({ onJumpToGraphNode }: OntologyWorkspaceProps)
}, []);
const handleFixInEditor = useCallback((entityUri: string) => {
const params = new URLSearchParams(window.location.search);
params.set(TAB_PARAM, "editor");
params.set("ontologyEntity", entityUri);
window.history.replaceState(null, "", `?${params.toString()}`);
writeEntitySelection(entityUri);
setActiveTab("editor");
}, []);
@@ -45,6 +45,10 @@ export function classifyNodeType(rawType: string): EditorEntityType {
return "external";
}
// Last-resort guess, reached only when the backend gave no verdict: it has no
// notion of nested vocabularies, so it can name a parent that does not contain
// the entity. Authority is owning_ontology from /api/ontology/entity
// (_resolve_owning_ontology in semantica/explorer/routes/ontology.py).
function ownsByNamespace(entityUri: string, ontologyUri: string): boolean {
const stem = ontologyUri.replace(/[/#]+$/, "");
return entityUri === ontologyUri
@@ -52,12 +56,50 @@ function ownsByNamespace(entityUri: string, ontologyUri: string): boolean {
|| entityUri.startsWith(`${stem}/`);
}
/**
* Three outcomes, deliberately not collapsed into `string | undefined`.
*
* `unowned` and `unresolved` both yield "no ontology to open", but they must
* not be treated alike: a caller that writes `resolve(...) || entries[0]` turns
* the backend's authoritative "nothing owns this entity" into "open an
* arbitrary ontology", which reintroduces the parent-selection bug this whole
* verdict exists to prevent. The union makes that collapse a type error.
*/
export type EditorOntologyResolution =
| { status: "resolved"; uri: string }
| { status: "unowned"; entityUri: string }
| { status: "unresolved" };
// Picks the registered ontology to open for a deep-linked entity. A null
// verdict is the backend's authoritative "nothing owns this entity": the
// namespace guess must stay suppressed, or an unregistered nested namespace
// would select its registered parent again. Only an unavailable verdict
// (undefined) may fall back to inference.
export function resolveEditorOntology(
entries: RegistryEntry[],
entityUri: string,
ownerVerdict: string | null | undefined,
): EditorOntologyResolution {
if (ownerVerdict === null) {
return { status: "unowned", entityUri };
}
const uri = inferOntologyUri(entries, entityUri, ownerVerdict);
return uri === undefined ? { status: "unresolved" } : { status: "resolved", uri };
}
// Picks the registered ontology to open for an entity: the backend-resolved
// explicitOwner wins outright, the namespace guess is only the fallback.
export function inferOntologyUri(
entries: RegistryEntry[],
entityUri: string,
explicitOwner?: string,
): string | undefined {
if (explicitOwner && entries.some((entry) => entry.uri === explicitOwner)) {
// Trusted even when the registry does not list it. Falling through to the
// namespace guess here would answer a question nobody asked — the backend
// named this entity's owner, and silently opening a *different* ontology is
// worse than opening one the registry has not been told about yet, which
// surfaces as an explicit error from /api/ontology/graph.
if (explicitOwner) {
return explicitOwner;
}
return [...entries]
@@ -0,0 +1,94 @@
// Sole owner of the Ontology Hub deep-link query parameters: the names below must not be
// spelled out anywhere else, so that the protocol can change in one place.
const TAB_PARAM = "ontologyTab";
const ENTITY_PARAM = "ontologyEntity";
const EDITOR_TAB = "editor";
export interface OntologyUrlState {
/** Raw parameter value; the set of legal tab ids belongs to the workspace, not this module. */
tab?: string;
entityUri?: string;
}
/** `undefined` means the parameter is absent; an empty string means it is present but blank. */
export function parseOntologyUrlState(search: string): OntologyUrlState {
const params = new URLSearchParams(search);
return {
tab: params.get(TAB_PARAM) ?? undefined,
entityUri: params.get(ENTITY_PARAM) ?? undefined,
};
}
export function applyTab(search: string, tab: string): string {
const params = new URLSearchParams(search);
params.set(TAB_PARAM, tab);
return `?${params.toString()}`;
}
// A selected entity is only addressable from the editor, so the tab moves with it.
export function applyEntitySelection(search: string, entityUri: string): string {
const params = new URLSearchParams(search);
params.set(TAB_PARAM, EDITOR_TAB);
params.set(ENTITY_PARAM, entityUri);
return `?${params.toString()}`;
}
// Pairs with applyEntitySelection: an entity URI is resolved back to its owning ontology on
// load, so leaving a stale one behind when the active ontology changes reopens the old ontology.
export function removeEntitySelection(search: string): string {
const params = new URLSearchParams(search);
params.delete(ENTITY_PARAM);
return `?${params.toString()}`;
}
/**
* Deliberately dual-role, and the argument is what selects the role: given a
* `search` string this is pure and total, delegating straight to
* `parseOntologyUrlState`; called with no argument it reads live `window`
* state and yields empty state if the URL is unreadable. Callers in render or
* effect paths use the no-argument form; tests and any caller that already
* holds a search string pass it, which is the only form that is testable.
*/
export function readOntologyUrlState(search?: string): OntologyUrlState {
if (search !== undefined) {
return parseOntologyUrlState(search);
}
try {
return parseOntologyUrlState(window.location.search);
} catch {
return {};
}
}
/** True when the URL addresses the Ontology Hub at all, even with blank parameter values. */
export function hasOntologyUrlState(search?: string): boolean {
const { tab, entityUri } = readOntologyUrlState(search);
return tab !== undefined || entityUri !== undefined;
}
// The transform returns a query string only, so the fragment has to be carried
// across explicitly: replaceState with a bare "?..." drops it. This is the one
// place that knows how the URL is written, so it is the only place that can.
function updateSearch(transform: (search: string) => string): void {
try {
window.history.replaceState(
null,
"",
`${transform(window.location.search)}${window.location.hash}`,
);
} catch {
// Deep-link state is a convenience; every caller stays correct without it.
}
}
export function writeTab(tab: string): void {
updateSearch((search) => applyTab(search, tab));
}
export function writeEntitySelection(entityUri: string): void {
updateSearch((search) => applyEntitySelection(search, entityUri));
}
export function clearEntitySelection(): void {
updateSearch(removeEntitySelection);
}
+3 -1
View File
@@ -101,7 +101,9 @@ test("visible legend follows loaded data, reloads, focused views, and distance m
await heatmap.click();
await legend.waitFor();
await assertLegendMatchesGraph(page);
await page.getByRole("button", { name: "Focused", exact: true }).click();
const focusButton = page.getByRole("button", { name: "Focus", exact: true });
assert.equal(await focusButton.isDisabled(), false, "Focus is enabled once a node is selected");
await focusButton.click();
await legend.getByText("Document", { exact: true }).waitFor({ state: "hidden" });
await assertLegendMatchesGraph(page, ["alice", "acme", "research"]);
assert.equal(await legend.getByText("Researcher", { exact: true }).count(), 1);
@@ -6,6 +6,7 @@ import {
compactNodeType,
inferOntologyUri,
isEditableEntityType,
resolveEditorOntology,
ONTOLOGY_MINIMAP_THEME,
} from "../src/workspaces/OntologyWorkspace/ontologyEditorModel";
@@ -29,6 +30,20 @@ test("explicit scheme ownership wins when an entity uses another namespace", ()
);
});
test("an explicit owner missing from the registry is used, not quietly replaced", () => {
// The entity sits under a registered namespace, so the guess has an answer
// ready; the backend naming a different, unregistered owner must still win,
// or the editor opens an ontology nobody said owned this entity.
assert.equal(
inferOntologyUri(
registry,
"https://example.test/foo#Class",
"https://unregistered.test/vocab",
),
"https://unregistered.test/vocab",
);
});
test("only draft-supported class and property nodes are editable", () => {
assert.equal(isEditableEntityType("class"), true);
assert.equal(isEditableEntityType("property"), true);
@@ -64,3 +79,36 @@ test("compactNodeType leaves unknown namespaces untouched", () => {
assert.equal(compactNodeType("https://example.org/custom#Thing"), "https://example.org/custom#Thing");
assert.equal(compactNodeType("owl:Class"), "owl:Class");
});
test("an authoritative no-owner verdict suppresses the namespace guess", () => {
// Without suppression the prefix guess would pick the registered parent
// for an unregistered nested entity — the deep link must not do that.
const nested = "https://example.test/foo/unregistered#Term";
assert.deepEqual(resolveEditorOntology(registry, nested, null), {
status: "unowned",
entityUri: nested,
});
// An unavailable verdict may still fall back to inference
assert.deepEqual(
resolveEditorOntology(registry, "https://example.test/foo#Class", undefined),
{ status: "resolved", uri: "https://example.test/foo" },
);
// A named owner wins outright
assert.deepEqual(
resolveEditorOntology(registry, nested, "https://example.test/foo/nested"),
{ status: "resolved", uri: "https://example.test/foo/nested" },
);
});
test("unowned is distinguishable from unresolved, so neither collapses to a default", () => {
// Both mean "no ontology to open", and the editor treats them oppositely:
// unresolved may land on the registry default, unowned must not. A caller
// writing `resolve(...) || entries[0]` reintroduced exactly the parent
// selection this verdict exists to prevent, so the difference is typed.
const unowned = resolveEditorOntology(registry, "https://example.test/foo/x#T", null);
const unresolved = resolveEditorOntology(registry, "https://elsewhere.test/T", undefined);
assert.equal(unowned.status, "unowned");
assert.equal(unresolved.status, "unresolved");
assert.notEqual(unowned.status, unresolved.status);
});
+102
View File
@@ -0,0 +1,102 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
applyEntitySelection,
applyTab,
clearEntitySelection,
hasOntologyUrlState,
parseOntologyUrlState,
readOntologyUrlState,
removeEntitySelection,
writeEntitySelection,
writeTab,
} from "../src/workspaces/OntologyWorkspace/ontologyUrlState";
function withStubbedLocation(search: string, hash: string, body: () => void): string[] {
const written: string[] = [];
const original = (globalThis as { window?: unknown }).window;
(globalThis as { window?: unknown }).window = {
location: { search, hash },
history: { replaceState: (_s: unknown, _t: string, url: string) => written.push(url) },
};
try {
body();
} finally {
(globalThis as { window?: unknown }).window = original;
}
return written;
}
test("selecting an entity round-trips and pins the editor tab", () => {
const search = applyEntitySelection("", "https://example.test/foo#Bar");
assert.deepEqual(parseOntologyUrlState(search), {
tab: "editor",
entityUri: "https://example.test/foo#Bar",
});
});
test("clearing the selection drops only the entity and keeps unrelated params", () => {
const search = applyEntitySelection("?view=graph&depth=2", "https://example.test/foo#Bar");
const cleared = parseOntologyUrlState(removeEntitySelection(search));
assert.equal(cleared.entityUri, undefined);
assert.equal(cleared.tab, "editor");
assert.equal(new URLSearchParams(removeEntitySelection(search)).get("depth"), "2");
});
test("writing a tab leaves an existing entity selection alone", () => {
const search = applyTab(applyEntitySelection("", "urn:x"), "health");
assert.deepEqual(parseOntologyUrlState(search), { tab: "health", entityUri: "urn:x" });
});
test("absent params read as undefined, blank params as empty strings", () => {
assert.deepEqual(parseOntologyUrlState(""), { tab: undefined, entityUri: undefined });
assert.deepEqual(parseOntologyUrlState("?other=1"), { tab: undefined, entityUri: undefined });
assert.deepEqual(parseOntologyUrlState("?ontologyTab=&ontologyEntity="), {
tab: "",
entityUri: "",
});
});
test("a present but blank param still counts as ontology deep-link state", () => {
assert.equal(hasOntologyUrlState("?ontologyEntity="), true);
assert.equal(hasOntologyUrlState("?ontologyTab="), true);
assert.equal(hasOntologyUrlState("?view=graph"), false);
assert.equal(hasOntologyUrlState(""), false);
});
test("malformed search strings degrade to plain values instead of throwing", () => {
assert.deepEqual(parseOntologyUrlState("???"), { tab: undefined, entityUri: undefined });
assert.deepEqual(parseOntologyUrlState("ontologyEntity=urn%3Ax&&=&"), {
tab: undefined,
entityUri: "urn:x",
});
});
test("entity URIs survive characters that need escaping", () => {
const entityUri = "https://example.test/vocab#Has Part/&?=";
const search = applyEntitySelection("?keep=1", entityUri);
assert.equal(parseOntologyUrlState(search).entityUri, entityUri);
});
test("every writer preserves the URL fragment", () => {
const written = withStubbedLocation("?view=graph", "#section-3", () => {
writeTab("health");
writeEntitySelection("urn:x");
clearEntitySelection();
});
assert.deepEqual(written, [
"?view=graph&ontologyTab=health#section-3",
"?view=graph&ontologyTab=editor&ontologyEntity=urn%3Ax#section-3",
"?view=graph#section-3",
]);
});
test("readOntologyUrlState with no argument reads live URL state", () => {
withStubbedLocation("?ontologyTab=editor&ontologyEntity=urn%3Ax", "", () => {
assert.deepEqual(readOntologyUrlState(), { tab: "editor", entityUri: "urn:x" });
assert.equal(hasOntologyUrlState(), true);
});
});
@@ -0,0 +1,113 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
DEFAULT_MIN_DATE,
resolvePlayStepMs,
resolveScrubberBounds,
} from "../src/workspaces/GraphWorkspace/temporalScrubberBounds.ts";
const NOW = new Date("2026-09-09T10:30:00Z");
// ── resolveScrubberBounds ────────────────────────────────────────────────────
test("scrubber bounds: open max ends the window at now, not at a future year", () => {
const { minBound, maxBound } = resolveScrubberBounds({ minDate: "2026-01-15T00:00:00Z", now: NOW });
assert.equal(minBound.toISOString(), "2026-01-15T00:00:00.000Z");
assert.equal(
maxBound.getTime(),
NOW.getTime(),
"a graph carrying only valid_from instants is known up to the present and no further",
);
});
test("scrubber bounds: playhead starts at now so the first snapshot describes the present", () => {
const { defaultTime } = resolveScrubberBounds({ minDate: "2026-01-15T00:00:00Z", now: NOW });
assert.equal(defaultTime.getTime(), NOW.getTime());
});
test("scrubber bounds: playhead is not the midpoint of the range", () => {
const { minBound, maxBound, defaultTime } = resolveScrubberBounds({
minDate: "2020-01-01T00:00:00Z",
maxDate: "2030-01-01T00:00:00Z",
now: NOW,
});
const midpoint = Math.round((minBound.getTime() + maxBound.getTime()) / 2);
assert.notEqual(defaultTime.getTime(), midpoint, "the midpoint was the source of the future start time");
assert.equal(defaultTime.getTime(), NOW.getTime());
});
test("scrubber bounds: reported max is honoured when the data supplies one", () => {
const { maxBound } = resolveScrubberBounds({
minDate: "2020-01-01T00:00:00Z",
maxDate: "2030-06-01T00:00:00Z",
now: NOW,
});
assert.equal(maxBound.toISOString(), "2030-06-01T00:00:00.000Z");
});
test("scrubber bounds: playhead clamps into a range that ends before now", () => {
const { maxBound, defaultTime } = resolveScrubberBounds({
minDate: "2019-01-01T00:00:00Z",
maxDate: "2020-01-01T00:00:00Z",
now: NOW,
});
assert.equal(defaultTime.getTime(), maxBound.getTime());
});
test("scrubber bounds: playhead clamps into a range that starts after now", () => {
const { minBound, defaultTime } = resolveScrubberBounds({
minDate: "2030-01-01T00:00:00Z",
maxDate: "2031-01-01T00:00:00Z",
now: NOW,
});
assert.equal(defaultTime.getTime(), minBound.getTime());
});
test("scrubber bounds: min ahead of an open max keeps the window ordered", () => {
const { minBound, maxBound } = resolveScrubberBounds({ minDate: "2031-01-01T00:00:00Z", now: NOW });
assert.ok(maxBound >= minBound, "vis-timeline requires min <= max");
});
test("scrubber bounds: malformed and missing dates fall back without producing Invalid Date", () => {
const { minBound, maxBound } = resolveScrubberBounds({ minDate: "not-a-date", maxDate: "also-bad", now: NOW });
assert.equal(minBound.getTime(), DEFAULT_MIN_DATE.getTime());
assert.equal(maxBound.getTime(), NOW.getTime());
});
// ── resolvePlayStepMs ────────────────────────────────────────────────────────
test("play step: a one-year span advances in ~60 frames, not 2", () => {
const minBound = new Date("2026-01-01T00:00:00Z");
const maxBound = new Date("2027-01-01T00:00:00Z");
const span = maxBound.getTime() - minBound.getTime();
const frames = span / resolvePlayStepMs(minBound, maxBound);
assert.ok(frames > 50 && frames < 70, `expected ~60 frames, got ${frames}`);
});
test("play step: a decade-long span also advances in ~60 frames", () => {
const minBound = new Date("2016-01-01T00:00:00Z");
const maxBound = new Date("2026-01-01T00:00:00Z");
const span = maxBound.getTime() - minBound.getTime();
const frames = span / resolvePlayStepMs(minBound, maxBound);
assert.ok(frames > 50 && frames < 70, `expected ~60 frames, got ${frames}`);
});
test("play step: a span of hours still advances by at least a day", () => {
const minBound = new Date("2026-09-09T00:00:00Z");
const maxBound = new Date("2026-09-09T06:00:00Z");
assert.equal(resolvePlayStepMs(minBound, maxBound), 1000 * 60 * 60 * 24);
});
+20
View File
@@ -0,0 +1,20 @@
# OSV-Scanner ignore config (also consumed by OpenSSF Scorecard's
# "Vulnerabilities" check, which reports advisories found in this repo's
# dependency manifests via https://osv.dev).
#
# See https://github.com/google/osv-scanner#ignore-vulnerabilities-by-id for
# the file format.
[[IgnoredVulns]]
id = "GHSA-4j2p-28q2-5m79"
reason = """
accelerate<=1.14.0 (transitive dependency via docling-slim, pinned in
requirements-ci.txt) has an open path traversal / DoS advisory (also tracked
as CVE-2026-69112) in load_checkpoint_in_model / load_checkpoint_and_dispatch,
which fail to sanitize weight_map entries from sharded checkpoint indexes.
1.14.0 is the latest release on PyPI; no patched version exists yet.
Semantica does not call either function or load arbitrary/untrusted sharded
checkpoints, so the vulnerable code path is not reachable. Re-evaluate once
accelerate ships a fix - see .github/workflows/security-scan.yml for the
matching pip-audit exclusion.
"""
+56 -14
View File
@@ -1,38 +1,80 @@
---
name: change
description: Track and inspect graph changes, diffs, temporal updates, and the impact of new data on Semantica knowledge graphs.
description: Inspect graph changes over time and ontology version diffs in Semantica. Uses ContextGraph.state_at for point-in-time graph state and change_management.VersionManager for ontology versioning.
---
# /semantica:change
Inspect changes over time and evaluate updates. Usage: `/semantica:change <task> [args]`
Track what changed. Usage: `/semantica:change <task> [args]`
`$ARGUMENTS` = task + optional node, time window, or filter.
> Two distinct mechanisms cover this, and they are **not** interchangeable:
>
> | Question | Tool |
> | --- | --- |
> | "What did the *graph* look like on date X?" | `ContextGraph.state_at()` |
> | "What changed between *ontology* versions?" | `change_management.VersionManager` |
---
## `diff [--from <ts>] [--to <ts>] [--node <id>]`
Compute graph diffs between two snapshots.
## `graph-at <timestamp>` — point-in-time graph state
```python
from semantica.provenance.change_tracker import ChangeTracker
import os
from semantica.context import ContextGraph
tracker = ChangeTracker()
diff = tracker.compute_diff(from_ts=from_ts, to_ts=to_ts, node_id=node_id)
graph = ContextGraph()
graph.load_from_file(os.path.expanduser("~/.semantica/kg.json")) # load_from_file does not expand ~
snapshot = graph.state_at("2026-06-01") # str | int | float | datetime
```
Output: added/removed nodes and edges, attribute changes, and impact summary.
Diff two moments by comparing node IDs — `state_at()["nodes"]` is a list of
dicts (unhashable), so compare the `id` fields, not the dicts themselves:
```python
before = graph.state_at("2026-06-01")
after = graph.state_at("2026-09-01")
before_ids = {n["id"] for n in before["nodes"]}
after_ids = {n["id"] for n in after["nodes"]}
added = after_ids - before_ids
```
For richer temporal work (scrubbing, evolution, temporal patterns) use
`/semantica:temporal`, which wraps the same layer.
---
## `history <node_id> [--limit N]`
## `node-history <node_id>` — who touched this node
Show the change history for a node or relationship.
Node-level history is provenance, not change management:
```python
history = tracker.get_node_history(node_id=node_id, limit=limit)
import os
from semantica.provenance import ProvenanceManager
db_path = os.path.expanduser("~/.semantica/prov.db") # storage_path is passed to
os.makedirs(os.path.dirname(db_path), exist_ok=True) # sqlite3.connect() unexpanded
pm = ProvenanceManager(storage_path=db_path)
history = pm.revision_history(node_id)
log = pm.audit_log(since="2026-01-01")
```
Return: revisions, timestamps, authors, and summary comments.
---
## `versions` / `diff <v1> <v2>` — ontology versioning
```python
from semantica.change_management import VersionManager
vm = VersionManager()
vm.create_version("1.1.0", ontology)
vm.list_versions()
vm.get_latest_version()
delta = vm.compare_versions("1.0.0", "1.1.0")
delta = vm.diff_ontologies(base_ontology, target_ontology)
migrated = vm.migrate_ontology("1.0.0", "1.1.0", ontology)
```
`TemporalVersionManager` and `OntologyVersionManager` are also exported for
time-scoped and ontology-specific variants.
+2 -2
View File
@@ -1,6 +1,6 @@
---
name: decision
description: Full decision lifecycle in Semantica — record, query, find precedents (hybrid/advanced), analyze influence, explain, insights dashboard, list, and record exceptions. Uses AgentContext, ContextGraph, DecisionQuery, CausalChainAnalyzer, DecisionRecorder.
description: Full decision lifecycle in Semantica — record, query, find precedents (hybrid/advanced), analyze influence, explain, insights dashboard, list, and record exceptions. Uses AgentContext, ContextGraph, DecisionQuery, CausalChainAnalyzer, DecisionRecorder.
---
# /semantica:decision
@@ -114,7 +114,7 @@ Output: Influence score + influenced decisions table + predicted new relationshi
## `explain <decision_id>`
Full explainability trace — reasoning steps, causal antecedents, policy compliance.
Full explainability trace — reasoning steps, causal antecedents, policy compliance.
```python
from semantica.context import AgentContext, ContextGraph
+2 -2
View File
@@ -21,8 +21,8 @@ Run the full extraction pipeline. Usage: `/semantica:extract [file_path | "inlin
**2. Clear the result cache** to prevent cross-invocation pollution:
```python
from semantica.semantic_extract.cache import _result_cache
_result_cache.clear()
from semantica.semantic_extract.cache import extraction_cache
extraction_cache.clear()
```
**3. Run the full pipeline:**
+65 -13
View File
@@ -1,37 +1,89 @@
---
name: ontology
description: Manage ontology schemas, concepts, relationships, and alignments for Semantica knowledge graphs.
description: Manage ontology schemas, concepts, alignments, and SHACL/OWL validation for Semantica knowledge graphs. Uses OntologyEngine and OntologyValidator.
---
# /semantica:ontology
Manage ontology definitions and validation. Usage: `/semantica:ontology <task> [args]`
`$ARGUMENTS` = task + optional ontology item or schema file.
> Entry points: `OntologyEngine` (authoring, export, alignments) and
> `OntologyValidator` (consistency checking).
---
## `describe <concept>`
## `concepts <scheme_uri>`
Show ontology concept details.
List SKOS concepts in a vocabulary scheme.
```python
from semantica.ontology import OntologyManager
from semantica.ontology import OntologyEngine
from semantica.triplet_store import TripletStore
manager = OntologyManager()
concept = manager.get_concept(concept_name)
store = TripletStore(backend="oxigraph") # needs semantica[tripletstore-oxigraph]
engine = OntologyEngine(store=store) # list_concepts/list_vocabularies need a
# configured store — raises ProcessingError without one
concepts = engine.list_concepts(scheme_uri)
vocabs = engine.list_vocabularies()
```
Output: properties, relationships, inherited types, and examples.
---
## `validate [--schema <file>]`
## `validate <ontology>`
Validate the graph or schema against the ontology.
Check an ontology for consistency and satisfiability.
```python
result = manager.validate_graph(graph=graph, schema_file=schema_file)
from semantica.ontology import OntologyValidator
validator = OntologyValidator(check_consistency=True, check_satisfiability=True)
result = validator.validate(ontology) # dict or path to an ontology file
# result.valid, result.errors, result.warnings
```
Return: validation status, errors, and correction suggestions.
For SHACL shape validation of instance data use `SHACLGenerator` / `SHACLValidationReport`:
```python
from semantica.ontology import SHACLGenerator
```
---
## `build <text|data>`
Generate an ontology from unstructured text or structured records.
```python
engine = OntologyEngine()
onto = engine.from_text(text) # LLM-assisted (needs an llm-* extra + API key)
onto = engine.from_data(records) # deterministic, from structured data
```
---
## `export <ontology> <path> [--format turtle]`
```python
engine.export_owl(onto, path, format="turtle")
engine.export_shacl(onto, path, format="turtle")
```
---
## `align <source_uri> <target_uri> <predicate>`
```python
engine.create_alignment(source_uri, target_uri, predicate)
engine.get_alignments(entity_uri)
engine.list_alignments()
```
---
## `evaluate <ontology>`
Quality-gate an ontology (`OntologyEvaluator` / `OntologyQualityReport` under the hood).
```python
report = engine.evaluate(onto)
```
+42 -13
View File
@@ -1,37 +1,66 @@
---
name: policy
description: Define and enforce policies, access controls, and compliance rules over Semantica knowledge graphs.
description: Define and enforce decision policies, compliance rules, and exceptions over Semantica graphs. Uses ContextGraph.check_decision_rules/enforce_decision_policy and context.PolicyEngine.
---
# /semantica:policy
Apply policy rules and checks. Usage: `/semantica:policy <task> [args]`
Policy governance over recorded decisions. Usage: `/semantica:policy <task> [args]`
`$ARGUMENTS` = task + optional policy name, rule set, or target entity.
> `PolicyEngine` lives in `semantica.context`. For most cases the two policy
> methods on `ContextGraph` itself are enough.
---
## `check [--rule <name>] [--target <id>]`
## `check <decision>` — the simple path
Run policy checks against the graph.
No policy store needed; rules default to a built-in policy set.
```python
from semantica.policy import PolicyEngine
from semantica.context import ContextGraph
engine = PolicyEngine()
result = engine.check(rule_name=rule_name, target=target)
graph = ContextGraph()
result = graph.check_decision_rules({
"category": "vendor_selection",
"outcome": "approved",
"confidence": 0.93,
"decision_maker": "gyro",
})
# {'compliant': bool, 'violations': [...], 'warnings': [...], 'policy_rules': {...}}
```
Output: compliance status, failing rules, and remediation guidance.
Default rules: `min_confidence=0.7`, `required_outcomes=['approved','rejected','flagged']`,
`required_metadata=['decision_maker']`, `max_reasoning_length=1000`. Override by
passing your own `rules=` dict.
## `enforce <decision> [--rules <dict>]`
```python
verdict = graph.enforce_decision_policy(decision_data, policy_rules=None)
```
---
## `list`
## Managed policies — the full path
List available policy rules and categories.
`PolicyEngine` requires a graph store and versioned `Policy` objects.
```python
rules = engine.list_rules()
from semantica.context import PolicyEngine
from semantica.context.decision_models import Policy
engine = PolicyEngine(graph_store)
policy_id = engine.add_policy(Policy(...))
policies = engine.get_applicable_policies(category="vendor_selection", entities=[...])
ok = engine.check_compliance(decision, policy_id)
history = engine.get_policy_history(policy_id)
engine.update_policy(policy_id, rules={...}, change_reason="tightened threshold")
engine.record_exception(decision_id, policy_id, reason="...", approver="...")
impact = engine.analyze_policy_impact(policy_id, proposed_rules={...})
affected = engine.get_affected_decisions(policy_id, from_version, to_version)
```
Return: rule name, description, severity, and category.
Note `check_compliance` takes a `Decision` object, not a dict — fetch it from the
graph rather than constructing one by hand.
+50 -17
View File
@@ -1,37 +1,70 @@
---
name: provenance
description: Trace data lineage, source attribution, audit trails, and provenance assertions in Semantica graphs.
description: Trace data lineage, source attribution, audit trails, and W3C PROV-O export in Semantica graphs. Uses ProvenanceManager.
---
# /semantica:provenance
Inspect provenance metadata. Usage: `/semantica:provenance <task> [args]`
`$ARGUMENTS` = task + optional node, edge, or time range.
Lineage and audit trails. Usage: `/semantica:provenance <task> [args]`
---
## `trace <node_id> [--depth N]`
Trace the provenance of a node or fact.
## `lineage <entity_id> [--depth N]`
```python
from semantica.provenance import ProvenanceTracer
import os
from semantica.provenance import ProvenanceManager
tracer = ProvenanceTracer()
trace = tracer.trace_node(node_id=node_id, depth=depth)
db_path = os.path.expanduser("~/.semantica/prov.db") # storage_path is passed to
os.makedirs(os.path.dirname(db_path), exist_ok=True) # sqlite3.connect() unexpanded
pm = ProvenanceManager(storage_path=db_path) # SQLite, or omit for in-memory
chain = pm.lineage(entity_id, depth=3)
full = pm.get_lineage(entity_id) # complete ancestry
down = pm.get_descendants(entity_id) # what this entity influenced
```
Output: source chain, authors, timestamps, and validation status.
---
## `audit [--since <ts>] [--actor <id>]`
View audit logs for graph changes.
## `sources <entity_id>`
```python
audit_log = tracer.get_audit_log(since=since, actor=actor)
srcs = pm.get_all_sources(entity_id) # every source that contributed
prov = pm.get_provenance(entity_id) # the raw PROV entry
hist = pm.revision_history(entity_id)
```
Return: change events, actor, affected objects, and action details.
---
## `audit [--since <iso-date>] [--format table|json]`
```python
log = pm.audit_log(since="2026-01-01", format="table")
between = pm.query_recorded_between(start, end)
stats = pm.get_statistics()
```
---
## `export [--format turtle|json-ld|xml]`
W3C PROV-O export — this is the regulator-facing artifact.
```python
rdf = pm.export_prov(format="turtle", base_uri="https://example.org/prov/")
```
---
## `invalidate <entity_id> <agent_id> [--reason ...]`
Mark an entity superseded without deleting history.
```python
pm.invalidate(entity_id, agent_id, reason="source retracted")
```
## `check [--strict]`
```python
report = pm.check(strict=False) # integrity check over the provenance store
```
+54 -19
View File
@@ -1,49 +1,84 @@
---
name: query
description: Query the Semantica knowledge graph using SPARQL, Cypher, keyword search, and structured graph query patterns.
description: Query Semantica knowledge graphs — in-memory ContextGraph search, SPARQL over RDF triple stores, and Cypher over LPG backends.
---
# /semantica:query
Run graph queries and search. Usage: `/semantica:query <mode> [args]`
Query the graph. Usage: `/semantica:query <task> [args]`
`$ARGUMENTS` = query mode + query string or filter.
> Which API you want depends on where the graph lives.
---
## `sparql <query>`
## `search "<keywords>"` — the in-memory ContextGraph
Execute a SPARQL query against the graph.
This is the one that works with no external server.
```python
from semantica.query import QueryEngine
import os
from semantica.context import ContextGraph
engine = QueryEngine()
results = engine.query_sparql(query)
graph = ContextGraph()
graph.load_from_file(os.path.expanduser("~/.semantica/kg.json")) # load_from_file does not expand ~
results = graph.query("vendor selection", skip=0, limit=20)
```
Return: query bindings as a Markdown table.
Related lookups on the same object:
```python
graph.find_nodes(...) graph.find_node(...)
graph.find_related_nodes(...) graph.get_neighbors(node_id)
graph.find_similar_nodes(...) graph.get_nodes_by_label(label)
```
Decision-specific queries belong to `/semantica:decision`.
---
## `cypher <query>`
Execute a Cypher-like query.
## `sparql "<query>"` — RDF triple stores
```python
results = engine.query_cypher(query)
from semantica.triplet_store import TripletStore
store = TripletStore(backend="oxigraph") # embedded; needs semantica[tripletstore-oxigraph]
# or backend="blazegraph" | "jena" | "rdf4j" with endpoint="http://..."
result = store.execute_query(sparql)
```
Output: node/relationship results and path summaries.
For query planning, optimisation, and caching over a backend:
```python
from semantica.triplet_store import QueryEngine, OxigraphStore
# QueryEngine needs an object exposing execute_sparql() — the raw backend,
# not the TripletStore wrapper above (which only exposes execute_query()).
backend = OxigraphStore()
qe = QueryEngine()
plan = qe.plan_query(sparql)
tuned = qe.optimize_query(sparql)
result = qe.execute_query(sparql, store_backend=backend)
stats = qe.get_query_statistics()
```
Blazegraph / Jena / RDF4J need **no** extra — `semantica.triplet_store` speaks
SPARQL over HTTP using the core `requests` dependency.
---
## `search <keywords> [--filter <type>]`
Search graph entities by keyword.
## `cypher "<query>"` — labeled property graphs
```python
results = engine.search(keywords=keywords, filter_type=filter_type)
from semantica.graph_store import Neo4jStore # needs semantica[graph-neo4j]
store = Neo4jStore(uri=..., user=..., password=...)
result = store.execute_query(query, parameters={...})
```
Return: ranked matches with entity types and relevance scores.
Also available: `FalkorDBStore`, `ApacheAgeStore`, `AmazonNeptuneStore`,
and `GraphManager` / `GraphStore` for backend-agnostic access.
**Not installed in this environment** — add the backend extra first, e.g.
`pip install "semantica[graph-neo4j]"`.
+2 -2
View File
@@ -119,9 +119,9 @@ from semantica.semantic_extract import (
NamedEntityRecognizer,
RelationExtractor,
)
from semantica.semantic_extract.cache import _result_cache
from semantica.semantic_extract.cache import extraction_cache
_result_cache.clear() # prevent cross-invocation cache pollution
extraction_cache.clear() # prevent cross-invocation cache pollution
text = open(file_path).read()
+60
View File
@@ -244,6 +244,7 @@ class EntityDetailResponse(BaseModel):
entity_type: str
definition: Optional[str] = None
source_ontology: Optional[str] = None
owning_ontology: Optional[str] = None
superclasses: List[str] = Field(default_factory=list)
subclasses: List[str] = Field(default_factory=list)
domain: List[str] = Field(default_factory=list)
@@ -814,6 +815,55 @@ def _node_belongs_to_ontology(
return "#" not in local_name and "/" not in local_name
def _resolve_owning_ontology(
node: Dict[str, Any],
known_ontology_uris: set[str],
) -> Optional[str]:
"""Return the one known ontology that owns this node, or None if none does.
The most specific (longest) match wins, so a nested vocabulary claims its
own terms instead of the parent absorbing them.
Kept agreeing with _node_belongs_to_ontology by construction — the same
three rules in the same order — but in one pass over the candidates rather
than one pass per candidate, each of which rescanned the whole set to find
the longest namespace. That made resolution quadratic in the number of
registered ontologies.
"""
nid = str(node.get("id", ""))
if not nid:
return None
# An ontology node owns itself, ahead of any scheme_uri it may carry.
if nid in known_ontology_uris:
return nid
# An explicit owner is authoritative even when it is not registered:
# naming a different ontology by namespace guess would be worse than
# reporting the one the node itself points at.
explicit_owner = _node_source_ontology(node)
if explicit_owner:
return explicit_owner
longest_namespace: Optional[str] = None
for candidate in known_ontology_uris:
stem = candidate.rstrip("#/")
if not nid.startswith((stem + "#", stem + "/")):
continue
if longest_namespace is None or len(candidate) > len(longest_namespace):
longest_namespace = candidate
if longest_namespace is None:
return None
# Prefix ownership only extends to names minted directly in the namespace.
# A further delimiter marks a nested vocabulary, which stays unowned until
# it is registered or carries an explicit owner.
local_name = nid[len(longest_namespace.rstrip("#/")) + 1 :]
if "#" in local_name or "/" in local_name:
return None
return longest_namespace
def _is_ontology_entity(node: Dict[str, Any]) -> bool:
return _classify_node_type(node.get("type", "")) in {"class", "property", "concept", "scheme"}
@@ -1952,6 +2002,7 @@ async def get_ontology_graph(
@router.get("/entity/{entity_uri:path}", response_model=EntityDetailResponse)
async def get_entity_detail(
entity_uri: str,
request: Request,
session: GraphSession = Depends(get_session),
):
node = await asyncio.to_thread(session.get_node, entity_uri)
@@ -1973,12 +2024,21 @@ async def get_entity_detail(
all_nodes, _ = await asyncio.to_thread(session.get_nodes, skip=0, limit=999_999)
instance_count = sum(1 for n in all_nodes if n.get("type") == entity_uri)
# Ownership must use the same candidate set as /graph, so it goes through the
# same helper rather than being derived from all_nodes above: that scan is
# capped at 999,999, and on a larger graph a truncated set would silently
# drop ontologies and make the two endpoints disagree about who owns a node.
# The helper iterates only the ontology node types, so it is not a full scan.
known_ontology_uris = await asyncio.to_thread(
_known_ontology_uris, session, _get_registry(request)
)
return EntityDetailResponse(
uri=entity_uri, label=label,
type=ntype, entity_type=_classify_node_type(ntype),
definition=definition,
source_ontology=props.get("scheme_uri"),
owning_ontology=_resolve_owning_ontology(node, known_ontology_uris),
superclasses=superclasses, subclasses=subclasses,
domain=domain, range=range_,
instance_count=instance_count, properties=props,
+111 -35
View File
@@ -68,6 +68,7 @@ License: MIT
from typing import Any, Dict, List, Optional, Tuple, TypedDict, Union, cast
import concurrent.futures
import inspect
import threading
import uuid
import numpy as np
@@ -142,6 +143,14 @@ class VectorStore:
if self.backend == "inmemory":
self.vectors: Dict[str, np.ndarray] = {}
self.metadata: Dict[str, Dict[str, Any]] = {}
# Monotonic counter for default ID generation. Never decremented
# on deletion, so IDs generated by consecutive store_vectors calls
# can never collide with surviving IDs (fixes #1029).
self._next_id: int = 0
# Reentrant lock protecting all in-memory state mutations:
# _next_id, vectors, metadata, and index rebuilds. Matches the
# threading model used by SQLiteVecStore and AgentMemory.
self._inmemory_lock = threading.RLock()
# Initialize backend-specific indexer
# Avoid duplicate dimension argument
@@ -557,18 +566,47 @@ class VectorStore:
self.progress_tracker.update_tracking(
tracking_id, message="Storing vectors..."
)
start_idx = len(self.vectors)
for i, (vector, meta) in enumerate(zip(vectors, metadata)):
vector_id = f"vec_{start_idx + i}"
self.vectors[vector_id] = vector
self.metadata[vector_id] = meta
vector_ids.append(vector_id)
# Hold the lock for the entire ID-allocation → dict-write →
# index-rebuild sequence so concurrent callers cannot observe
# half-written state or generate the same candidate ID.
with self._inmemory_lock:
# Snapshot the live key-set at lock-entry so the generator
# and the within-batch de-dupe use a consistent view.
pre_existing = set(self.vectors)
# within_batch tracks IDs chosen during *this* call so the
# same candidate is never returned twice in one batch.
within_batch: set = set()
# Update index
self.progress_tracker.update_tracking(
tracking_id, message="Updating vector index..."
)
self.indexer.create_index(list(self.vectors.values()), vector_ids)
for vector, meta in zip(vectors, metadata):
# Advance the monotonic counter until we find a candidate
# that is free both in the live store and in this batch.
#
# The counter is never decremented on deletion, so under
# normal operation every candidate it produces is genuinely
# fresh. The only reason a candidate can be occupied is
# that a caller pre-inserted a ``vec_N`` key ahead of the
# counter (e.g. manually writing to self.vectors). Skipping
# over such keys is intentional and matches FAISSStore's
# identical behaviour. Nothing is overwritten: the loop
# breaks only on a candidate that is absent from both
# pre_existing and within_batch.
while True:
candidate = f"vec_{self._next_id}"
self._next_id += 1
if candidate not in pre_existing and candidate not in within_batch:
break
within_batch.add(candidate)
self.vectors[candidate] = vector
self.metadata[candidate] = meta
vector_ids.append(candidate)
# Update index inside the lock so readers always see a
# consistent (vectors, index) pair.
self.progress_tracker.update_tracking(
tracking_id, message="Updating vector index..."
)
self.indexer.create_index(list(self.vectors.values()), list(self.vectors.keys()))
self.progress_tracker.stop_tracking(
tracking_id,
@@ -610,7 +648,11 @@ class VectorStore:
"metadata": getattr(self, "metadata", {}),
"config": self.config,
"backend": self.backend,
"dimension": self.dimension
"dimension": self.dimension,
# Persist the monotonic counter so that load() can restore it
# rather than re-deriving it from len(vectors), which would be
# too small after a deletion and cause ID collisions (issue #1029).
"next_id": getattr(self, "_next_id", None),
}
with open(os.path.join(path, "store_data.json"), "w", encoding="utf-8") as f:
@@ -657,6 +699,25 @@ class VectorStore:
self.config = data.get("config", {})
self.backend = data.get("backend", "faiss")
self.dimension = data.get("dimension", 768)
# Restore the monotonic ID counter. Always clamp to at least
# max(vec_N suffix)+1 so a stale or missing persisted value (e.g.
# written before this field was added, or written before a deletion
# that lowered the count) cannot produce IDs that collide with
# existing vectors (issue #1029).
if self.backend == "inmemory":
_vec_nums = [
int(v[4:]) + 1
for v in self.vectors
if v.startswith("vec_") and v[4:].isdigit()
]
_inferred = max(_vec_nums) if _vec_nums else 0
persisted_next_id = data.get("next_id")
if persisted_next_id is not None:
self._next_id = max(int(persisted_next_id), _inferred)
else:
# Older store files lack this field; use the safe inferred value.
self._next_id = _inferred
# Restore backend-specific index
indexer = getattr(self, "indexer", None)
@@ -737,14 +798,25 @@ class VectorStore:
)
return []
# Snapshot vectors and metadata together under the lock so a
# concurrent delete_vectors / store_vectors cannot cause
# "RuntimeError: dictionary changed size during iteration" and
# cannot produce an inconsistent (values, keys) pair where one
# list is shorter than the other. The lock is released before
# the (potentially slow) similarity computation.
with self._inmemory_lock:
snapshot_vectors = list(self.vectors.values())
snapshot_keys = list(self.vectors.keys())
snapshot_metadata = dict(self.metadata)
# Use retriever for similarity search
self.progress_tracker.update_tracking(
tracking_id, message="Performing similarity search..."
)
results = self.retriever.search_similar(
query_vector,
list(self.vectors.values()),
list(self.vectors.keys()),
snapshot_vectors,
snapshot_keys,
k=k,
**options,
)
@@ -752,8 +824,8 @@ class VectorStore:
# Add metadata to results; guarantee the key always exists.
for result in results:
vector_id = result.get("id")
if vector_id and vector_id in self.metadata:
result["metadata"] = self.metadata[vector_id]
if vector_id and vector_id in snapshot_metadata:
result["metadata"] = snapshot_metadata[vector_id]
elif "metadata" not in result:
result["metadata"] = {}
@@ -782,19 +854,21 @@ class VectorStore:
else:
raise NotImplementedError(f"Backend store {type(self._backend_store).__name__} does not have update or update_vectors method")
for vec_id, new_vec in zip(vector_ids, new_vectors):
if vec_id in self.vectors:
self.vectors[vec_id] = new_vec
with self._inmemory_lock:
for vec_id, new_vec in zip(vector_ids, new_vectors):
if vec_id in self.vectors:
self.vectors[vec_id] = new_vec
if metadata:
for vec_id, meta in zip(vector_ids, metadata):
if vec_id in self.metadata:
self.metadata[vec_id] = meta
if metadata:
for vec_id, meta in zip(vector_ids, metadata):
if vec_id in self.metadata:
self.metadata[vec_id] = meta
# Rebuild index
self.indexer.create_index(
list(self.vectors.values()), list(self.vectors.keys())
)
# Rebuild index under the lock so readers see a consistent
# (vectors, index) pair, matching store_vectors and delete_vectors.
self.indexer.create_index(
list(self.vectors.values()), list(self.vectors.keys())
)
return True
@@ -809,15 +883,17 @@ class VectorStore:
else:
raise NotImplementedError(f"Backend store {type(self._backend_store).__name__} does not have delete or delete_vectors method")
for vec_id in vector_ids:
self.vectors.pop(vec_id, None)
self.metadata.pop(vec_id, None)
with self._inmemory_lock:
for vec_id in vector_ids:
self.vectors.pop(vec_id, None)
self.metadata.pop(vec_id, None)
# Rebuild index
if self.vectors:
self.indexer.create_index(
list(self.vectors.values()), list(self.vectors.keys())
)
# Rebuild index under the lock so a concurrent search cannot
# see vectors without a corresponding index entry.
if self.vectors:
self.indexer.create_index(
list(self.vectors.values()), list(self.vectors.keys())
)
return True
+84
View File
@@ -17,6 +17,7 @@ from semantica.explorer.routes.ontology import ( # noqa: E402
OntologyEntry,
_convert_ontology_to_graph,
_node_belongs_to_ontology,
_resolve_owning_ontology,
)
from semantica.explorer.session import GraphSession # noqa: E402
@@ -259,6 +260,89 @@ def test_node_belongs_to_ontology_nested_namespace_matrix():
assert _node_belongs_to_ontology(node(f"{child}/Term"), child, {parent, child})
def test_entity_detail_reports_explicit_owner(client):
response = client.get(
f"/api/ontology/entity/{quote('http://example.org/onto-a#Person', safe='')}"
)
assert response.status_code == 200
payload = response.json()
assert payload["source_ontology"] == "http://example.org/onto-a"
assert payload["owning_ontology"] == "http://example.org/onto-a"
def test_entity_detail_reports_namespace_owner_without_explicit_scheme(client):
graph = client.app.state.session.graph
minted_directly = "http://example.org/onto-a#Address"
graph.add_node(minted_directly, node_type="owl:Class", content="Address")
response = client.get(f"/api/ontology/entity/{quote(minted_directly, safe='')}")
assert response.status_code == 200
assert response.json()["owning_ontology"] == "http://example.org/onto-a"
def test_entity_detail_reports_no_owner_for_unregistered_nested_namespace(client):
graph = client.app.state.session.graph
nested_term = "http://example.org/onto-a/nested#Term"
graph.add_node(nested_term, node_type="owl:Class", content="Nested Term")
response = client.get(f"/api/ontology/entity/{quote(nested_term, safe='')}")
assert response.status_code == 200
# onto-a must not claim a nested vocabulary its own /graph response
# excludes, or a deep link selects onto-a and then finds nothing to select.
assert response.json()["owning_ontology"] is None
def test_entity_detail_reports_an_explicit_owner_outside_the_registry(client):
graph = client.app.state.session.graph
borrowed = "http://example.org/onto-a#Borrowed"
unregistered_owner = "http://unregistered.example/vocab"
# Sits directly in onto-a's namespace, so the namespace rule has an answer
# ready — the node's own scheme_uri still has to win, or /entity reports an
# owner that contradicts the node and the editor opens the wrong ontology.
graph.add_node(
borrowed,
node_type="owl:Class",
content="Borrowed",
scheme_uri=unregistered_owner,
)
response = client.get(f"/api/ontology/entity/{quote(borrowed, safe='')}")
assert response.status_code == 200
assert response.json()["owning_ontology"] == unregistered_owner
def test_owner_resolution_agrees_with_graph_membership(client):
"""_resolve_owning_ontology and _node_belongs_to_ontology must not diverge.
The two answer the same question from opposite directions, and /entity and
/graph each use one of them. If they disagree, a deep link opens an ontology
whose graph then excludes the entity it was opened for.
"""
parent = "http://example.org/onto-a"
nested = "http://example.org/onto-a/nested"
known = {parent, nested}
cases = [
{"id": parent},
{"id": f"{parent}#Direct"},
{"id": f"{parent}/Direct"},
{"id": f"{nested}#Term"},
{"id": f"{parent}/unregistered#Term"},
{"id": "http://elsewhere.example/Thing"},
{"id": f"{parent}#Explicit", "properties": {"scheme_uri": nested}},
]
for node in cases:
owner = _resolve_owning_ontology(node, known)
for candidate in known:
assert _node_belongs_to_ontology(node, candidate, known) == (
owner == candidate
), f"{node['id']} vs {candidate}: owner={owner}"
def test_load_fallback_import_without_declaration_is_editable(client):
turtle = """
@prefix ex: <http://data.example.org/people#> .
@@ -0,0 +1,614 @@
"""Regression tests for issue #1029: VectorStore inmemory backend must not
reissue live vector IDs after a deletion.
Before the fix, `store_vectors()` derived new IDs as
``f"vec_{len(self.vectors) + i}"``. After any deletion `len` decreases,
so the next insertion generates an ID that already belongs to a surviving
vector, silently overwriting its embedding and metadata.
Three test groups:
1. ``TestInmemoryIdNoReuseAfterDelete`` direct VectorStore path
2. ``TestAgentMemoryIdNoReuseAfterDelete`` AgentMemory path
3. ``TestInmemoryIdPersistence`` save / load / delete / store cycle
"""
from __future__ import annotations
import json
import shutil
import tempfile
import unittest
from unittest.mock import MagicMock, patch
import numpy as np
# ---------------------------------------------------------------------------
# Helper: build a lightweight VectorStore(backend="inmemory") without
# triggering the real EmbeddingGenerator or VectorIndexer.
# ---------------------------------------------------------------------------
def _make_store(dim: int = 4) -> "VectorStore": # noqa: F821
"""Return an inmemory VectorStore with heavy components mocked out."""
from semantica.vector_store.vector_store import VectorStore
with patch("semantica.vector_store.vector_store.get_logger",
return_value=MagicMock()), \
patch("semantica.vector_store.vector_store.get_progress_tracker",
return_value=MagicMock()), \
patch("semantica.vector_store.vector_store.VectorIndexer"), \
patch("semantica.vector_store.vector_store.VectorRetriever"), \
patch("semantica.vector_store.vector_store.EmbeddingGenerator"):
store = VectorStore(backend="inmemory", config={"dimension": dim})
store.embedder = None
return store
_RNG = np.random.default_rng(seed=1029)
def _vec(dim: int = 4) -> np.ndarray:
return _RNG.random(dim).astype(np.float32)
# ---------------------------------------------------------------------------
# 1. Direct VectorStore path
# ---------------------------------------------------------------------------
class TestInmemoryIdNoReuseAfterDelete(unittest.TestCase):
"""Store A+B, delete A, store C — B and C must have distinct IDs and both
survive with correct metadata."""
def _store(self) -> "VectorStore": # noqa: F821
return _make_store()
def test_b_and_c_have_different_ids(self):
"""Core regression: id_c must not equal id_b."""
store = self._store()
id_a, id_b = store.store_vectors(
[_vec(), _vec()], [{"label": "A"}, {"label": "B"}]
)
store.delete_vectors([id_a])
(id_c,) = store.store_vectors([_vec()], [{"label": "C"}])
self.assertNotEqual(
id_b, id_c,
f"ID collision: both B and C got id={id_b!r}",
)
def test_both_vectors_remain_after_delete_reinsert(self):
"""B's entry must survive the deletion of A and the insertion of C."""
store = self._store()
id_a, id_b = store.store_vectors(
[_vec(), _vec()], [{"label": "A"}, {"label": "B"}]
)
store.delete_vectors([id_a])
(id_c,) = store.store_vectors([_vec()], [{"label": "C"}])
self.assertIn(id_b, store.vectors, "B's vector was lost")
self.assertIn(id_c, store.vectors, "C's vector was not stored")
self.assertIn(id_b, store.metadata, "B's metadata was lost")
self.assertIn(id_c, store.metadata, "C's metadata was not stored")
def test_count_is_two_after_store_delete_store(self):
"""After storing 2, deleting 1, storing 1 the count must be 2."""
store = self._store()
id_a, _ = store.store_vectors(
[_vec(), _vec()], [{}, {}]
)
store.delete_vectors([id_a])
store.store_vectors([_vec()], [{}])
self.assertEqual(len(store.vectors), 2)
def test_b_metadata_not_overwritten_by_c(self):
"""B's metadata must be unchanged after C is stored."""
store = self._store()
id_a, id_b = store.store_vectors(
[_vec(), _vec()],
[{"label": "A"}, {"label": "B", "sentinel": True}],
)
store.delete_vectors([id_a])
store.store_vectors([_vec()], [{"label": "C"}])
self.assertEqual(
store.metadata[id_b],
{"label": "B", "sentinel": True},
"B's metadata was silently overwritten",
)
def test_id_uniqueness_across_multiple_delete_reinsert_cycles(self):
"""Each cycle of delete-then-store must produce a fresh, unique ID."""
store = self._store()
seen_ids: set = set()
# Initial batch
batch = store.store_vectors([_vec() for _ in range(3)], [{} for _ in range(3)])
seen_ids.update(batch)
# Three delete-then-store cycles
for current_id in list(batch):
store.delete_vectors([current_id])
(new_id,) = store.store_vectors([_vec()], [{}])
self.assertNotIn(
new_id, seen_ids,
f"Generated ID {new_id!r} collides with a previously used ID",
)
seen_ids.add(new_id)
def test_counter_skips_explicit_vec_n_ids(self):
"""The monotonic counter must skip over any explicit ``vec_N`` already
present so it never collides with a manually supplied ID."""
store = self._store()
# Manually insert vec_1 so the counter must skip it
store.vectors["vec_1"] = _vec()
store.metadata["vec_1"] = {"explicit": True}
# Ask for two auto-generated IDs; one would be vec_1 if the counter
# did not skip it.
ids = store.store_vectors([_vec(), _vec()], [{}, {}])
self.assertNotIn("vec_1", ids, "Monotonic counter re-generated an explicit ID")
# vec_1's explicit entry must be intact
self.assertEqual(store.metadata["vec_1"], {"explicit": True})
# Both new vectors must actually be in the store
for new_id in ids:
self.assertIn(new_id, store.vectors)
def test_auto_generated_id_never_silently_overwrites_live_vector(self):
"""An automatically generated ID must never land on top of a live
auto-generated vector, regardless of deletion history.
This is the core of 'Never overwrite an existing live vector id
silently' from issue #1029: after any sequence of stores and deletes
every auto-generated ID must map to exactly one vector.
"""
store = self._store()
# Store 5 vectors — auto-ids vec_0..vec_4
first_batch = store.store_vectors([_vec() for _ in range(5)], [{} for _ in range(5)])
# Delete vec_0, vec_1, vec_2 — _next_id stays at 5, so the next
# auto-id should be vec_5, vec_6 … NOT vec_2/vec_3/vec_4.
store.delete_vectors(first_batch[:3])
surviving = set(store.vectors.keys()) # {vec_3, vec_4}
second_batch = store.store_vectors([_vec(), _vec()], [{}, {}])
# None of the new IDs must collide with surviving ones
for new_id in second_batch:
self.assertNotIn(
new_id, surviving,
f"Auto-generated ID {new_id!r} silently landed on a live vector",
)
# Both new vectors must be independently present
for new_id in second_batch:
self.assertIn(new_id, store.vectors)
self.assertIn(new_id, store.metadata)
# Total count: 2 surviving + 2 new
self.assertEqual(len(store.vectors), 4)
def test_collision_detection_no_silent_overwrite_even_with_corrupted_counter(self):
"""Even if _next_id is externally wound back (simulating a corrupt
load), the generator must never silently overwrite a live vector.
The while-loop guarantees this by skipping every occupied candidate
until it finds a free slot. The existing vector and its metadata
must be completely unchanged after the call.
"""
store = self._store()
# Store vec_0 and vec_1 (_next_id advances to 2)
ids = store.store_vectors([_vec(), _vec()], [{"orig": 0}, {"orig": 1}])
id_b = ids[1] # vec_1
vec_b_before = store.vectors[id_b].copy()
meta_b_before = dict(store.metadata[id_b])
# Corrupt the counter: reset to 0 so candidates start at vec_0/vec_1
store._next_id = 0
# store_vectors must succeed without raising and without overwriting
new_ids = store.store_vectors([_vec()], [{"new": True}])
# The new ID must be some other slot — not vec_0 or vec_1
self.assertNotIn(
new_ids[0], {ids[0], ids[1]},
f"New vector landed on a live ID {new_ids[0]!r} "
"(no-silent-overwrite invariant violated)",
)
# vec_1 must be completely unchanged
np.testing.assert_array_equal(
store.vectors[id_b], vec_b_before,
err_msg="Live vector vec_1 was overwritten by the post-corruption store call",
)
self.assertEqual(
store.metadata[id_b], meta_b_before,
"Live metadata for vec_1 was overwritten by the post-corruption store call",
)
# New vector must actually be in the store
self.assertIn(new_ids[0], store.vectors)
self.assertEqual(store.metadata[new_ids[0]], {"new": True})
# ---------------------------------------------------------------------------
class TestAgentMemoryIdNoReuseAfterDelete(unittest.TestCase):
"""Exercise the ID-collision fix through AgentMemory.store() /
delete_memory() rather than VectorStore directly."""
def setUp(self):
"""Build a real VectorStore(inmemory) and bind it to AgentMemory.
EmbeddingGenerator is mocked so the test doesn't need a model.
"""
from semantica.context.agent_memory import AgentMemory
from semantica.vector_store.vector_store import VectorStore
self._embedding_patch = patch(
"semantica.context.agent_memory.AgentMemory._generate_embedding",
side_effect=lambda text: np.ones(8, dtype=np.float32),
)
self._embedding_patch.start()
self.store = VectorStore(backend="inmemory", config={"dimension": 8})
# Suppress real EmbeddingGenerator on the store itself
self.store.embedder = None
self.memory = AgentMemory(vector_store=self.store)
def tearDown(self):
self._embedding_patch.stop()
def test_b_and_c_have_different_vector_ids(self):
"""After storing A+B, deleting A, storing C, B and C must have
distinct vector IDs."""
self.memory.store("content A", memory_id="mem_a", skip_graph=True)
self.memory.store("content B", memory_id="mem_b", skip_graph=True)
vid_b_before = self.memory.vector_ids_for("mem_b")
self.memory.delete_memory("mem_a")
self.memory.store("content C", memory_id="mem_c", skip_graph=True)
vid_b = self.memory.vector_ids_for("mem_b")
vid_c = self.memory.vector_ids_for("mem_c")
# B's vector IDs must be unchanged — it was never touched.
self.assertEqual(vid_b, vid_b_before, "mem_b's vector IDs changed unexpectedly")
self.assertTrue(vid_b, "mem_b has no tracked vector IDs")
self.assertTrue(vid_c, "mem_c has no tracked vector IDs")
self.assertTrue(
set(vid_b).isdisjoint(set(vid_c)),
f"B and C share vector IDs: {set(vid_b) & set(vid_c)}",
)
def test_both_memories_remain_independently_retrievable(self):
"""mem_b and mem_c must both survive and report correct vector-store
embeddings after the delete-reinsert cycle."""
self.memory.store("content B", memory_id="mem_b", skip_graph=True)
self.memory.store("content A", memory_id="mem_a", skip_graph=True)
self.memory.delete_memory("mem_a")
self.memory.store("content C", memory_id="mem_c", skip_graph=True)
self.assertIn("mem_b", self.memory.memory_items)
self.assertIn("mem_c", self.memory.memory_items)
self.assertNotIn("mem_a", self.memory.memory_items)
# Each surviving memory must have its own live vector in the store
for mid in ("mem_b", "mem_c"):
vids = self.memory.vector_ids_for(mid)
for vid in vids:
self.assertIn(
vid, self.store.vectors,
f"{mid!r} vector id {vid!r} is missing from the store",
)
def test_vector_store_count_is_correct(self):
"""After storing 2, deleting 1, storing 1 the vector count must be 2."""
self.memory.store("content A", memory_id="mem_a", skip_graph=True)
self.memory.store("content B", memory_id="mem_b", skip_graph=True)
self.memory.delete_memory("mem_a")
self.memory.store("content C", memory_id="mem_c", skip_graph=True)
self.assertEqual(self.store.count(), 2)
def test_b_embedding_not_overwritten(self):
"""mem_b's vector must be the original embedding, not C's."""
# Give B a distinct embedding so we can detect overwriting
call_order: list = []
def _side_effect(text: str) -> np.ndarray:
call_order.append(text)
# Unique per-call vector based on call order length
v = np.zeros(8, dtype=np.float32)
v[len(call_order) % 8] = float(len(call_order))
return v
with patch(
"semantica.context.agent_memory.AgentMemory._generate_embedding",
side_effect=_side_effect,
):
mem2 = __import__(
"semantica.context.agent_memory", fromlist=["AgentMemory"]
).AgentMemory(vector_store=self.store)
mem2.store("content A", memory_id="mem_a2", skip_graph=True)
mem2.store("content B", memory_id="mem_b2", skip_graph=True)
b_embedding = mem2.memory_items["mem_b2"].embedding
mem2.delete_memory("mem_a2")
mem2.store("content C", memory_id="mem_c2", skip_graph=True)
vid_b = mem2.vector_ids_for("mem_b2")
self.assertTrue(vid_b, "mem_b2 has no tracked vector ID")
stored_b_vec = self.store.vectors.get(vid_b[0])
self.assertIsNotNone(stored_b_vec)
np.testing.assert_array_equal(
stored_b_vec,
b_embedding if hasattr(b_embedding, "__len__") else np.array(b_embedding),
err_msg="B's stored vector was silently overwritten by C's embedding",
)
# ---------------------------------------------------------------------------
# 3. Persistence: save → load → delete → store must not collide
# ---------------------------------------------------------------------------
class TestInmemoryIdPersistence(unittest.TestCase):
"""The _next_id counter must survive save/load so that inserting after a
delete-and-reload cycle cannot generate an ID already held by a surviving
vector."""
def setUp(self):
self.tmpdir = tempfile.mkdtemp()
def tearDown(self):
shutil.rmtree(self.tmpdir, ignore_errors=True)
def _fresh_store(self, dim: int = 4):
return _make_store(dim=dim)
def test_next_id_is_persisted_in_json(self):
"""save() must write ``next_id`` into store_data.json."""
store = self._fresh_store()
store.store_vectors([_vec(), _vec(), _vec()], [{}, {}, {}])
store.save(self.tmpdir)
with open(f"{self.tmpdir}/store_data.json", encoding="utf-8") as fh:
data = json.load(fh)
self.assertIn("next_id", data, "save() did not write 'next_id' to JSON")
self.assertGreaterEqual(data["next_id"], 3)
def test_load_restores_next_id(self):
"""load() must restore _next_id so the counter does not restart at 0."""
store = self._fresh_store()
store.store_vectors([_vec(), _vec(), _vec()], [{}, {}, {}])
store.save(self.tmpdir)
loaded = self._fresh_store()
loaded.load(self.tmpdir)
self.assertGreaterEqual(
loaded._next_id, 3,
f"load() set _next_id={loaded._next_id}, expected >= 3",
)
def test_no_collision_after_save_load_delete_store(self):
"""Full lifecycle: save → load → delete one → store one new vector.
The new ID must not collide with any surviving vector."""
store = self._fresh_store()
# Store vec_0, vec_1, vec_2
ids = store.store_vectors([_vec() for _ in range(3)], [{} for _ in range(3)])
store.save(self.tmpdir)
# Load fresh instance
loaded = self._fresh_store()
loaded.load(self.tmpdir)
# Delete vec_0 (ntotal drops to 2; without the fix, next id = vec_2)
loaded.delete_vectors([ids[0]])
surviving = set(loaded.vectors.keys())
# Store a new vector — must not reuse any surviving ID
(new_id,) = loaded.store_vectors([_vec()], [{"new": True}])
self.assertNotIn(
new_id, surviving,
f"Generated ID {new_id!r} collides with a surviving ID "
f"(surviving={sorted(surviving)})",
)
self.assertEqual(len(loaded.vectors), 3)
def test_stale_next_id_in_json_is_clamped(self):
"""load() must clamp a stale persisted next_id to at least
max(vec_N suffix)+1, guarding against corrupted saves."""
store = self._fresh_store()
# Stores vec_0, vec_1, vec_2
store.store_vectors([_vec() for _ in range(3)], [{}, {}, {}])
store.save(self.tmpdir)
# Corrupt the JSON: set next_id to 1 (below vec_2's suffix+1 = 3)
json_path = f"{self.tmpdir}/store_data.json"
with open(json_path, encoding="utf-8") as fh:
data = json.load(fh)
data["next_id"] = 1
with open(json_path, "w", encoding="utf-8") as fh:
json.dump(data, fh)
loaded = self._fresh_store()
loaded.load(self.tmpdir)
self.assertGreaterEqual(
loaded._next_id, 3,
f"Stale next_id=1 was not clamped; got {loaded._next_id} (expected >= 3)",
)
def test_missing_next_id_in_old_json_inferred_from_vec_suffixes(self):
"""Older store files without 'next_id' must have the counter inferred
from the highest ``vec_N`` suffix so that loading them is safe."""
store = self._fresh_store()
store.store_vectors([_vec() for _ in range(4)], [{} for _ in range(4)])
store.save(self.tmpdir)
# Remove next_id to simulate an older save file
json_path = f"{self.tmpdir}/store_data.json"
with open(json_path, encoding="utf-8") as fh:
data = json.load(fh)
data.pop("next_id", None)
with open(json_path, "w", encoding="utf-8") as fh:
json.dump(data, fh)
loaded = self._fresh_store()
loaded.load(self.tmpdir)
# _next_id must be at least 4 (vec_0..vec_3 → max suffix+1 = 4)
self.assertGreaterEqual(
loaded._next_id, 4,
f"Missing next_id not inferred correctly; got {loaded._next_id}",
)
# And a subsequent insert must not collide
surviving = set(loaded.vectors.keys())
(new_id,) = loaded.store_vectors([_vec()], [{}])
self.assertNotIn(
new_id, surviving,
f"Post-load insert collided: {new_id!r} already in {sorted(surviving)}",
)
if __name__ == "__main__":
unittest.main()
# ---------------------------------------------------------------------------
# 4. Concurrency: concurrent store_vectors calls must not produce duplicate IDs
# ---------------------------------------------------------------------------
class TestInmemoryIdConcurrency(unittest.TestCase):
"""Concurrent store_vectors calls on the same VectorStore must each get
unique IDs and all vectors must survive (no silent overwrites)."""
def test_concurrent_store_vectors_produce_unique_ids(self):
"""Two threads storing vectors simultaneously must not collide."""
import threading as _threading
store = _make_store(dim=4)
results: list = []
errors: list = []
def _store_batch(n: int) -> None:
try:
ids = store.store_vectors(
[_vec() for _ in range(n)],
[{"batch": n, "idx": i} for i in range(n)],
)
results.extend(ids)
except Exception as exc:
errors.append(exc)
threads = [_threading.Thread(target=_store_batch, args=(5,)) for _ in range(6)]
for t in threads:
t.start()
for t in threads:
t.join()
self.assertFalse(errors, f"Threads raised: {errors}")
total = 6 * 5
self.assertEqual(len(results), total, "Some store calls lost vectors")
# All returned IDs must be unique — no two threads got the same ID
self.assertEqual(
len(set(results)), total,
f"Duplicate IDs produced under concurrency: "
f"{[x for x in results if results.count(x) > 1]}",
)
# Every returned ID must be present in the store
for vid in results:
self.assertIn(vid, store.vectors, f"ID {vid!r} not in store after concurrent insert")
def test_concurrent_delete_and_store_no_phantom_ids(self):
"""A thread deleting while another is storing must not leave the store
with stale index entries or inconsistent counts."""
import threading as _threading
store = _make_store(dim=4)
initial = store.store_vectors([_vec() for _ in range(4)], [{} for _ in range(4)])
errors: list = []
def _deleter():
try:
store.delete_vectors(initial[:2])
except Exception as exc:
errors.append(exc)
def _storer():
try:
store.store_vectors([_vec(), _vec()], [{}, {}])
except Exception as exc:
errors.append(exc)
t1 = _threading.Thread(target=_deleter)
t2 = _threading.Thread(target=_storer)
t1.start()
t2.start()
t1.join()
t2.join()
self.assertFalse(errors, f"Threads raised: {errors}")
# After the dust settles the vectors dict and metadata must agree
self.assertEqual(
set(store.vectors.keys()), set(store.metadata.keys()),
"vectors and metadata dicts are out of sync after concurrent delete+store",
)
def test_concurrent_search_and_delete_no_runtime_error(self):
"""search_vectors() must not raise RuntimeError when a concurrent
delete_vectors() modifies the store during iteration.
Uses threading.Barrier to make the race deterministic: the search
thread announces it is ready just before calling search_similar, and
the delete thread fires only after that signal has been received.
Without the lock-protected snapshot in search_vectors(), the delete
would mutate self.vectors while list() is iterating it, reliably
causing 'RuntimeError: dictionary changed size during iteration'.
"""
import threading as _threading
store = _make_store(dim=4)
vecs = store.store_vectors([_vec() for _ in range(8)], [{} for _ in range(8)])
query = _vec()
errors: list = []
# Barrier with 2 parties: searcher + deleter.
barrier = _threading.Barrier(2)
original_search_similar = store.retriever.search_similar
def _patched_search_similar(q, vectors, keys, k, **kw):
# Signal the deleter that iteration is about to begin, then wait
# for it to be ready too. Both threads proceed together.
barrier.wait(timeout=5)
return original_search_similar(q, vectors, keys, k, **kw)
store.retriever.search_similar = _patched_search_similar
def _searcher():
try:
store.search_vectors(query, k=4)
except Exception as exc:
errors.append(exc)
def _deleter():
barrier.wait(timeout=5) # wait until searcher is mid-search
try:
store.delete_vectors(vecs[:4])
except Exception as exc:
errors.append(exc)
t1 = _threading.Thread(target=_searcher)
t2 = _threading.Thread(target=_deleter)
t1.start()
t2.start()
t1.join(timeout=10)
t2.join(timeout=10)
self.assertFalse(
errors,
f"Concurrent search+delete raised: {errors}",
)