Compare commits

..
4 Commits
Author SHA1 Message Date
30fc6447fd fix(cli): prevent doctor Note/Hint columns from wrapping into unreadable fragments (#1428) (#1475)
* fix(cli): prevent doctor Note/Hint columns from wrapping into unreadable fragments (#1428)

* fix(cli): keep doctor check labels readable

---------

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
Co-authored-by: Sameer Kadam <sameerkadam@Sameers-MacBook-Air.local>
2026-09-05 21:55:12 +05:30
Wei Tao c25b88c07e perf(explorer): bound the ontology graph fetch and hydrate external nodes concurrently (#1441)
`GET /api/ontology/graph` used to fetch every node and edge for each relevant type with `limit=2**63-1`, normalize all of it, and only then check the result against `_MAX_ANALYSIS_NODES`. So the cap limited the response size, but not the amount of work or memory needed to build it.

Using the existing `paginate_nodes`/`paginate_edges` APIs wouldn't help much either. Each call still loads and normalizes the full matching set before slicing out a page, so paging through the results would repeat that cost for every page.

This adds `GraphSession.iter_nodes` and `iter_edges`, which yield matches one at a time instead of materializing everything up front. The graph endpoint now applies the ownership filter while scanning, so nodes from other ontologies are dropped before they're normalized or kept in memory. It also stops as soon as the requested ontology's own nodes or selected edges exceed the cap.

The ownership check intentionally happens before the cap check. That way, a graph containing a large number of nodes from unrelated ontologies can't make the requested ontology appear too large to open.

The old `candidates_by_id` map is gone as well. We now only retain nodes actually owned by the requested ontology instead of building payloads for every schema-typed node in the graph.

I also split the endpoint into `_known_ontology_uris`, `_collect_core_nodes`, and `_select_structure_edges` so the main flow is a little easier to follow.

External edge endpoints were previously hydrated one at a time with an `await` for each node. With a few thousand selected edges, that meant a few thousand sequential thread dispatches. They're now hydrated concurrently with `asyncio.gather` over `to_thread` calls, using the default thread pool. The final node and edge lists are sorted before building the response, so completion order doesn't affect the output.

This isn't fully lazy all the way down. `iter_edges` still gets the full result for an edge type from `ContextGraph.find_edges` before it starts yielding, and `iter_nodes` still snapshots and sorts all node IDs for a type first. A very large type such as `rdf:type` can therefore still do work proportional to its graph-wide size before the cap gets a chance to stop the scan.

What this change avoids is the more expensive part: normalizing every schema node in the graph and keeping the full candidate payload map in memory. Making `ContextGraph.find_edges` lazy would address the remaining issue, but that's a lower-level API used elsewhere and needs its own locking design, so that's better handled separately.

The old final cap check is removed too. The streaming collectors now raise as soon as their running count exceeds `_MAX_ANALYSIS_NODES`, so by the time collection finishes, `core_node_ids` and `selected_edges` are already guaranteed to be within the limit. There's a comment at that boundary documenting the invariant instead of keeping a redundant check around.
2026-09-05 20:05:24 +05:00
14d25cabcd fix(cli): dispatch reason run and store connect to real APIs (#1372)
* fix(cli): dispatch reason run and store connect to real APIs

semantica reason run called Reasoner.run(), which does not exist --
the facade's API is infer_facts(facts, rules). The command now reads
nodes/relationships from the configured graph store as fact strings
(the same conventions Reasoner.add_fact applies to KG-style dicts),
loads --rules as a YAML list/mapping or plain-text lines, and reports
the inferred facts.

semantica store connect called get_graph_store_method(backend), which
is the (task, method_name) method registry, not a backend factory, so
it raised a TypeError before any connection attempt and failed
identically with or without valid credentials. It now builds the store
via GraphStore(backend=...) and probes connect(), so real
connectivity/auth errors surface.

Fixes #1354

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cli): use backend relationship keys, all labels, and honest engine dispatch in reason run

Review follow-ups on the reason run fact conversion:

- Relationships from every graph store backend carry start_node_id /
  end_node_id; _graph_store_facts() read start_id / end_id, so each
  relationship became TYPE(None, None) and relationship rules never
  matched. Read the real keys and cover it with a rule that matches the
  relationship fact.
- Emit a fact for every node label, not just the first, so rules on
  secondary labels can match.
- reason run only executes the Reasoner facade's forward-chaining
  inference; --engine values with different input paradigms (datalog,
  sparql, abductive, deductive, graph) now fail with a clear message
  (pointing SPARQL/Datalog at reason query) instead of reporting an
  engine that never ran.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cli): harden reason run rule loading

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
Co-authored-by: Sameer Kadam <sameerkadam@Sameers-MacBook-Air.local>
2026-09-05 20:29:46 +05:30
KaifAhmad1 c2bde11f3c test: verify SSH commit signing 2026-09-05 19:48:19 +05:30
5 changed files with 665 additions and 88 deletions
+92 -12
View File
@@ -917,11 +917,11 @@ def doctor(cli_ctx: CLIContext, local_json: bool, deep_embeddings: bool) -> None
for lbl, st, note, hint in checks])
return
tbl = Table(box=_TABLE_BOX, show_edge=False, padding=(0, 2))
tbl.add_column("Check", style=_KEY, no_wrap=True, min_width=16)
tbl.add_column("Status", no_wrap=True, min_width=6)
tbl.add_column("Note", style=_DIM)
tbl.add_column("Hint", style=_DIM)
tbl = Table(box=_TABLE_BOX, show_edge=False, padding=(0, 2), expand=True)
tbl.add_column("Check", style=_KEY, no_wrap=True, min_width=34)
tbl.add_column("Status", no_wrap=True, min_width=4)
tbl.add_column("Note", style=_DIM, min_width=15, ratio=2, overflow="fold")
tbl.add_column("Hint", style=_DIM, min_width=20, ratio=3, overflow="fold")
icons = {"ok": f"[{_SUCCESS}] ✓[/{_SUCCESS}]",
"warn": f"[{_WARN_STY}] ⚠[/{_WARN_STY}]",
@@ -1155,6 +1155,57 @@ def _get_graph_store(cli_ctx: CLIContext) -> Any:
return GraphStore(backend=backend, **graph_db)
def _load_rule_definitions(path: str) -> List[str]:
"""Load reasoning rule definitions from a YAML or plain-text rules file.
YAML files may hold a list of rule strings or a mapping with a ``rules``
list; anything else (e.g. Datalog) is read as one rule per non-comment
line. The strings are handed to ``Reasoner.add_rule()`` untouched.
"""
text = Path(path).read_text(encoding="utf-8")
try:
data = yaml.safe_load(text)
except yaml.YAMLError:
data = None
if isinstance(data, dict):
rules_value = data.get("rules")
if rules_value is None and "rules" not in data:
raise click.ClickException(
f"Rules file '{path}' is a YAML mapping but has no 'rules' key. "
"Expected either a YAML list or a mapping with a 'rules' list."
)
data = rules_value
if isinstance(data, list):
return [str(item) for item in data]
return [line.strip() for line in text.splitlines()
if line.strip() and not line.lstrip().startswith("#")]
def _graph_store_facts(cli_ctx: CLIContext) -> List[str]:
"""Read the configured graph store into Reasoner fact strings.
Follows the same conventions ``Reasoner.add_fact()`` applies to
KG-style dicts: nodes become ``Label(name)`` and relationships become
``TYPE(source, target)``, with internal node ids resolved to names.
"""
gs = _get_graph_store(cli_ctx)
nodes = gs.get_nodes(limit=sys.maxsize)
relationships = gs.get_relationships(limit=sys.maxsize)
names: Dict[Any, Any] = {}
facts: List[str] = []
for node in nodes:
props = node.get("properties") or {}
name = props.get("name") or props.get("id") or node.get("id")
names[node.get("id")] = name
for label in node.get("labels") or ["Entity"]:
facts.append(f"{label}({name})")
for rel in relationships:
source = names.get(rel.get("start_node_id"), rel.get("start_node_id"))
target = names.get(rel.get("end_node_id"), rel.get("end_node_id"))
facts.append(f"{rel.get('type', 'RELATED_TO')}({source}, {target})")
return facts
# ─── Output helpers ──────────────────────────────────────────────────────────
@@ -2212,17 +2263,43 @@ def reason_run(cli_ctx: CLIContext, engine: str, rules: Optional[str],
cli_ctx = _require_ctx(cli_ctx)
def _action() -> None:
# Only the forward-chaining production-rule engines run through
# Reasoner.infer_facts(); the other engines take different inputs
# (SPARQL/Datalog queries, observations, premises) and are not wired
# to this command yet. Fail honestly instead of silently
# forward-chaining under another engine's name.
if engine not in ("rete", "forward-chain"):
hint = (" Use 'semantica reason query' for SPARQL/Datalog queries."
if engine in ("sparql", "datalog") else "")
raise click.ClickException(
f"Engine '{engine}' is not wired to 'reason run' yet; "
f"supported engines: rete, forward-chain.{hint}")
try:
from .reasoning import Reasoner
# Reasoner has no run() method (#1354); dispatch to its real
# API: facts from the configured graph store + rules from the
# optional --rules file into infer_facts().
r = Reasoner(engine=engine, config=cli_ctx.config.to_dict())
rule_defs = _load_rule_definitions(rules) if rules else None
facts = _graph_store_facts(cli_ctx)
def _infer() -> Dict[str, Any]:
inferred = r.infer_facts(facts, rule_defs)
return {
"engine": engine,
"facts": len(facts),
"inferred_count": len(inferred),
"inferred_facts": inferred,
}
if cli_ctx.quiet or cli_ctx.json_output:
result = r.run(rules_file=rules)
result = _infer()
else:
with console.status(
f"[{_DIM}]Running {engine} reasoning engine…[/{_DIM}]",
spinner="dots",
):
result = r.run(rules_file=rules)
result = _infer()
except ImportError as exc:
raise click.ClickException(f"Reasoning module not available: {exc}") from exc
if _is_json(cli_ctx, local_json):
@@ -3713,14 +3790,17 @@ def store_connect(cli_ctx: CLIContext, backend: str, uri: Optional[str], local_j
def _action() -> None:
try:
from .graph_store import get_graph_store_method
store_cls = get_graph_store_method(backend)
# get_graph_store_method(task, method_name) is the method
# registry, not a backend factory (#1354); build the store
# through GraphStore, which resolves the backend by name.
from .graph_store import GraphStore
cfg = dict(cli_ctx.config.to_dict().get("graph_db", {}))
cfg.pop("backend", None)
if uri:
cfg["uri"] = uri
# Attempt instantiation as the minimal connectivity probe; backends
# that require a live connection will fail here if unreachable.
store_instance = store_cls(config=cfg)
# Instantiation only wires the backend; the probe below performs
# the live connectivity check and raises if unreachable.
store_instance = GraphStore(backend=backend, **cfg)
for probe in ("health_check", "ping", "connect"):
fn = getattr(store_instance, probe, None)
if callable(fn):
+102 -75
View File
@@ -35,6 +35,10 @@ router = APIRouter(prefix="/api/ontology", tags=["ontology"])
_MAX_FETCH_BYTES = 20 * 1024 * 1024 # 20 MB
_MAX_ANALYSIS_NODES = 5_000 # cap for health/suggest/shacl node scans to avoid OOM
_MAX_ENTITIES_PER_SIDE = 500 # per-ontology cap for the O(n²) pairwise suggestion loop
_GRAPH_TOO_LARGE_DETAIL = (
"Ontology editor graph exceeds the maximum size "
f"({_MAX_ANALYSIS_NODES} nodes or edges)."
)
class GraphTruncationError(Exception):
@@ -72,6 +76,20 @@ _ONTOLOGY_TYPES = frozenset({
}) | _SCHEME_TYPES
_SEARCHABLE_TYPES = _CLASS_TYPES | _PROPERTY_TYPES | _INDIVIDUAL_TYPES | _CONCEPT_TYPES | _SCHEME_TYPES
_SCHEMA_NODE_TYPES = _CLASS_TYPES | _PROPERTY_TYPES | _CONCEPT_TYPES | _ONTOLOGY_TYPES
_STRUCTURE_EDGE_TYPES = frozenset({
"rdf:type",
"rdfs:subClassOf",
"rdfs:domain",
"rdfs:range",
"owl:disjointWith",
"owl:equivalentClass",
"owl:equivalentProperty",
"owl:inverseOf",
"skos:broader",
"skos:narrower",
"skos:related",
})
_URI_PREFIX_MAP = {
"http://www.w3.org/2002/07/owl#": "owl:",
@@ -1821,6 +1839,64 @@ async def search_entities(
return results
def _known_ontology_uris(
session: GraphSession, registry: Dict[str, OntologyEntry]
) -> set[str]:
known = set(registry)
for node_type in _ONTOLOGY_TYPES:
for node in session.iter_nodes(node_type=node_type):
node_id = str(node.get("id", ""))
if node_id:
known.add(node_id)
return known
def _collect_core_nodes(
session: GraphSession, uri: str, known_ontology_uris: set[str]
) -> Dict[str, Dict[str, Any]]:
"""Stream schema nodes, keeping only the ones this ontology owns.
Filtering as each node arrives makes _MAX_ANALYSIS_NODES bound the work and
not merely the response: foreign nodes are discarded instead of materialized,
and the scan stops once the owned ones pass the cap. The ownership filter has
to stay ahead of that check — thousands of *other* ontologies' nodes must
never make this one too large to open. Requesting pages instead would bound
nothing: paginate_nodes normalizes the whole matching set on every call.
"""
core_nodes_by_id: Dict[str, Dict[str, Any]] = {}
for node_type in _SCHEMA_NODE_TYPES:
for node in session.iter_nodes(node_type=node_type):
node_id = str(node.get("id", ""))
if not node_id or not _node_belongs_to_ontology(
node, uri, known_ontology_uris
):
continue
core_nodes_by_id[node_id] = node
if len(core_nodes_by_id) > _MAX_ANALYSIS_NODES:
raise GraphTruncationError(_GRAPH_TOO_LARGE_DETAIL)
return core_nodes_by_id
def _select_structure_edges(
session: GraphSession, core_node_ids: set[str]
) -> List[Dict[str, Any]]:
"""Stream structural edges, keeping only those leaving a core node.
The requested ontology may reference outward (e.g. rdfs:range to an external
vocabulary), but an unrelated ontology's property pointing at a core class
must not leak inward.
"""
selected_edges: List[Dict[str, Any]] = []
for edge_type in _STRUCTURE_EDGE_TYPES:
for edge in session.iter_edges(edge_type=edge_type):
if str(edge.get("source", "")) not in core_node_ids:
continue
selected_edges.append(edge)
if len(selected_edges) > _MAX_ANALYSIS_NODES:
raise GraphTruncationError(_GRAPH_TOO_LARGE_DETAIL)
return selected_edges
@router.get("/graph", response_model=OntologyGraphResponse)
async def get_ontology_graph(
request: Request,
@@ -1828,88 +1904,39 @@ async def get_ontology_graph(
session: GraphSession = Depends(get_session),
):
"""Return the editable schema subgraph for one registered ontology."""
registry = _get_registry(request)
ontology_nodes: List[Dict[str, Any]] = []
for node_type in _ONTOLOGY_TYPES:
nodes, _ = await asyncio.to_thread(
session.get_nodes, node_type=node_type, skip=0, limit=2**63 - 1
)
ontology_nodes.extend(nodes)
known_ontology_uris = set(registry) | {
str(node.get("id", "")) for node in ontology_nodes if node.get("id")
}
known_ontology_uris = await asyncio.to_thread(
_known_ontology_uris, session, _get_registry(request)
)
if uri not in known_ontology_uris:
raise HTTPException(status_code=404, detail="Ontology not found in registry.")
schema_types = _CLASS_TYPES | _PROPERTY_TYPES | _CONCEPT_TYPES | _ONTOLOGY_TYPES
candidates_by_id: Dict[str, Dict[str, Any]] = {}
for node_type in schema_types:
nodes, _ = await asyncio.to_thread(
session.get_nodes, node_type=node_type, skip=0, limit=2**63 - 1
try:
core_nodes_by_id = await asyncio.to_thread(
_collect_core_nodes, session, uri, known_ontology_uris
)
candidates_by_id.update(
(str(node.get("id", "")), node) for node in nodes if node.get("id")
if not core_nodes_by_id:
raise HTTPException(status_code=404, detail="Ontology graph not found.")
core_node_ids = set(core_nodes_by_id)
selected_edges = await asyncio.to_thread(
_select_structure_edges, session, core_node_ids
)
except GraphTruncationError as exc:
raise HTTPException(status_code=413, detail=str(exc)) from exc
core_node_ids = {
str(node.get("id", ""))
for node in candidates_by_id.values()
if _node_belongs_to_ontology(node, uri, known_ontology_uris)
}
if not core_node_ids:
raise HTTPException(status_code=404, detail="Ontology graph not found.")
# Invariant: the helpers raise the moment their accumulation passes
# _MAX_ANALYSIS_NODES, so core_nodes_by_id and selected_edges are both
# within the cap here; a post-filter re-check would be unreachable.
external_node_ids = {
node_id
for edge in selected_edges
for node_id in (str(edge.get("source", "")), str(edge.get("target", "")))
} - core_node_ids
external_nodes = await asyncio.gather(
*(asyncio.to_thread(session.get_node, node_id) for node_id in external_node_ids)
)
structure_edge_types = {
"rdf:type",
"rdfs:subClassOf",
"rdfs:domain",
"rdfs:range",
"owl:disjointWith",
"owl:equivalentClass",
"owl:equivalentProperty",
"owl:inverseOf",
"skos:broader",
"skos:narrower",
"skos:related",
}
selected_edges: List[Dict[str, Any]] = []
for edge_type in structure_edge_types:
edges, _ = await asyncio.to_thread(
session.get_edges,
edge_type=edge_type,
skip=0,
limit=2**63 - 1,
)
# Keep only edges whose source is a core node: the requested ontology
# may reference outward (e.g. rdfs:range to an external vocabulary),
# but an unrelated ontology's property pointing at a core class must
# not leak inward.
selected_edges.extend(
edge for edge in edges
if str(edge.get("source", "")) in core_node_ids
)
if (
len(core_node_ids) > _MAX_ANALYSIS_NODES
or len(selected_edges) > _MAX_ANALYSIS_NODES
):
raise HTTPException(
status_code=413,
detail=(
"Ontology editor graph exceeds the maximum size "
f"({_MAX_ANALYSIS_NODES} nodes or edges)."
),
)
selected_node_ids = set(core_node_ids)
for edge in selected_edges:
selected_node_ids.add(str(edge.get("source", "")))
selected_node_ids.add(str(edge.get("target", "")))
selected_nodes = [candidates_by_id[node_id] for node_id in core_node_ids]
for node_id in selected_node_ids - core_node_ids:
external = await asyncio.to_thread(session.get_node, node_id)
if external is not None:
selected_nodes.append(external)
selected_nodes = list(core_nodes_by_id.values())
selected_nodes.extend(node for node in external_nodes if node is not None)
selected_nodes.sort(key=lambda node: str(node.get("id", "")))
selected_edges.sort(
key=lambda edge: (
+47 -1
View File
@@ -9,7 +9,7 @@ import threading
import time
import uuid
from datetime import UTC, datetime
from typing import Any, Dict, Iterable, List, Optional
from typing import Any, Dict, Iterable, Iterator, List, Optional
from ..context.context_graph import ContextGraph, _resolve_edge_identity
from .search_index import GraphSearchIndex
@@ -375,6 +375,52 @@ class GraphSession:
)
return page, total
def iter_nodes(self, node_type: Optional[str] = None) -> Iterator[Dict[str, Any]]:
"""Yield matching nodes one at a time, in the same order as ``paginate_nodes``.
``paginate_nodes`` normalizes and holds the entire matching set before it
slices out a page, so a caller that filters the result down itself cannot
bound its cost by asking for smaller pages it would re-pay that full
cost per page. Streaming lets such a caller retain only what it selects
and stop scanning as soon as it has enough.
Only the id list is snapshotted under the lock; nodes are read one at a
time, so a concurrent mutation can be observed mid-iteration and ids that
disappear are skipped. ``paginate_nodes`` is the atomic alternative.
"""
with self._lock:
source_ids = (
self.graph.node_type_index.get(node_type, set())
if node_type
else self.graph.nodes.keys()
)
node_ids = sorted(
(node_id for node_id in source_ids if node_id is not None),
key=lambda value: str(value),
)
for node_id in node_ids:
with self._lock:
raw = self.graph.find_node(node_id)
if raw is None:
continue
yield self.normalize_node(raw)
def iter_edges(self, edge_type: Optional[str] = None) -> Iterator[Dict[str, Any]]:
"""Yield matching edges one at a time, in raw graph order.
Same rationale as ``iter_nodes``. Edge normalization derives an identity
hash per edge, which ``paginate_edges`` pays for every matching edge (and
then sorts) before paging; a filtering caller only needs it for the edges
it keeps. Callers that need a stable order sort the subset they select.
"""
with self._lock:
raw_edges = self.graph.find_edges(edge_type=edge_type)
for edge in raw_edges:
normalized = self.normalize_edge(edge)
if not normalized["source"] or not normalized["target"]:
continue
yield normalized
def get_raw_counts(self) -> tuple[int, int]:
"""O(1) node/edge counts from the raw collections, with no per-item
normalization.
+61
View File
@@ -13,6 +13,7 @@ pytest.importorskip("fastapi")
from semantica.explorer.app import create_app # noqa: E402
from semantica.explorer.routes.ontology import ( # noqa: E402
_MAX_ANALYSIS_NODES,
OntologyEntry,
_convert_ontology_to_graph,
_node_belongs_to_ontology,
@@ -310,6 +311,66 @@ def test_ontology_graph_ignores_unrelated_data_when_enforcing_size_limit(client)
}
def test_ontology_graph_rejects_oversized_core_and_stops_scanning(client, monkeypatch):
graph = client.app.state.session.graph
for index in range(5_001):
graph.add_node(
f"http://example.org/onto-a#Bulk{index:05d}",
node_type="owl:Class",
content="Bulk",
scheme_uri="http://example.org/onto-a",
)
for index in range(3_000):
graph.add_node(
f"urn:unrelated:{index}",
node_type="owl:Class",
content="Unrelated",
scheme_uri="http://example.org/onto-b",
)
streamed = 0
original_iter_nodes = GraphSession.iter_nodes
def counting_iter_nodes(self, node_type=None):
nonlocal streamed
for node in original_iter_nodes(self, node_type=node_type):
streamed += 1
yield node
monkeypatch.setattr(GraphSession, "iter_nodes", counting_iter_nodes)
response = client.get(
"/api/ontology/graph",
params={"uri": "http://example.org/onto-a"},
)
assert response.status_code == 413
assert str(_MAX_ANALYSIS_NODES) in response.json()["detail"]
# The graph holds 8,001 owl:Class nodes and onto-a's own sort first, so a
# scan that abandons at the cap sees far fewer than the whole type.
assert streamed < 6_000
def test_ontology_graph_hydrates_external_edge_targets_in_sorted_order(client):
graph = client.app.state.session.graph
external = "http://external.example/Thing"
also_external = "http://external.example/Aardvark"
graph.add_node(external, node_type="owl:Class", content="External Thing")
graph.add_node(also_external, node_type="owl:Class", content="External Aardvark")
graph.add_edge("http://example.org/onto-a#name", external, edge_type="rdfs:range")
graph.add_edge("http://example.org/onto-a#name", also_external, edge_type="rdfs:range")
response = client.get(
"/api/ontology/graph",
params={"uri": "http://example.org/onto-a"},
)
assert response.status_code == 200
node_ids = [node["id"] for node in response.json()["nodes"]]
assert {external, also_external} <= set(node_ids)
assert node_ids == sorted(node_ids)
def test_shacl_generate_and_shapes(client):
response = client.post(
"/api/ontology/shacl/generate",
+363
View File
@@ -857,6 +857,156 @@ class TestReason:
assert result.exit_code != 0
assert "Traceback" not in result.output
def test_run_infers_from_graph_store_facts(self, runner, monkeypatch, tmp_path):
# reason run used to call Reasoner.run(), which does not exist
# (#1354); it must feed graph store facts + --rules into
# Reasoner.infer_facts().
pytest.importorskip("numpy", reason="semantica.reasoning needs numpy")
rules_file = tmp_path / "rules.yaml"
rules_file.write_text(
'- IF Person(?x) THEN Human(?x)\n'
'- IF MANAGES(?x, ?y) THEN Manager(?x)\n'
'- IF Employee(?x) THEN Staff(?x)\n',
encoding="utf-8")
class _FakeStore:
# Same dict schema as the real backends: nodes carry
# labels/properties, relationships carry start_node_id/end_node_id.
def get_nodes(self, limit=None):
return [{"id": 1, "labels": ["Person"],
"properties": {"name": "Alice"}},
{"id": 2, "labels": ["Person", "Employee"],
"properties": {"name": "Bob"}}]
def get_relationships(self, limit=None):
return [{"id": 9, "type": "MANAGES",
"start_node_id": 1, "end_node_id": 2}]
monkeypatch.setattr(cli_module, "_get_graph_store", lambda ctx: _FakeStore())
result = runner.invoke(
cli_module.main,
["--json", "reason", "run", "--rules", str(rules_file)],
)
_ok(result)
data = json.loads(result.output.strip())
# Person(Alice), Person(Bob), Employee(Bob), MANAGES(Alice, Bob)
assert data["facts"] == 4
assert "Human(Alice)" in data["inferred_facts"]
# Relationship endpoints resolve node ids to names.
assert "Manager(Alice)" in data["inferred_facts"]
# Secondary labels also become facts.
assert "Staff(Bob)" in data["inferred_facts"]
assert data["inferred_count"] == len(data["inferred_facts"])
def test_run_rejects_unwired_engine(self, runner):
result = runner.invoke(cli_module.main,
["reason", "run", "--engine", "sparql"])
assert result.exit_code != 0
assert "not wired" in result.output
assert "reason query" in result.output
assert "Traceback" not in result.output
def test_load_rule_definitions_formats(self, tmp_path):
yaml_list = tmp_path / "list.yaml"
yaml_list.write_text('- IF A(?x) THEN B(?x)\n- IF B(?x) THEN C(?x)\n',
encoding="utf-8")
assert cli_module._load_rule_definitions(str(yaml_list)) == [
"IF A(?x) THEN B(?x)", "IF B(?x) THEN C(?x)"]
yaml_map = tmp_path / "map.yaml"
yaml_map.write_text('rules:\n - IF A(?x) THEN B(?x)\n', encoding="utf-8")
assert cli_module._load_rule_definitions(str(yaml_map)) == [
"IF A(?x) THEN B(?x)"]
plain = tmp_path / "rules.dl"
plain.write_text('# comment\nIF A(?x) THEN B(?x)\n\n', encoding="utf-8")
assert cli_module._load_rule_definitions(str(plain)) == [
"IF A(?x) THEN B(?x)"]
def test_run_empty_graph_returns_zero_facts(self, runner, monkeypatch):
"""reason run with an empty graph store should not crash and report 0 facts."""
pytest.importorskip("numpy", reason="semantica.reasoning needs numpy")
class _EmptyStore:
def get_nodes(self, limit=None): return []
def get_relationships(self, limit=None): return []
monkeypatch.setattr(cli_module, "_get_graph_store", lambda ctx: _EmptyStore())
result = runner.invoke(cli_module.main, ["--json", "reason", "run"])
_ok(result)
data = json.loads(result.output.strip())
assert data["facts"] == 0
assert data["inferred_count"] == 0
assert data["inferred_facts"] == []
def test_run_graph_store_error_surfaces_cleanly(self, runner, monkeypatch):
"""A graph-store connectivity error must surface as a clean error, not a Traceback."""
def _bad_store(ctx):
raise RuntimeError("connection refused")
monkeypatch.setattr(cli_module, "_get_graph_store", _bad_store)
result = runner.invoke(cli_module.main, ["reason", "run"])
assert result.exit_code != 0
assert "Traceback" not in result.output
assert "connection refused" in result.output
def test_run_no_rules_uses_empty_ruleset(self, runner, monkeypatch):
"""reason run without --rules should still succeed (zero rules -> zero inferences)."""
pytest.importorskip("numpy", reason="semantica.reasoning needs numpy")
class _FakeStore:
def get_nodes(self, limit=None):
return [{"id": 1, "labels": ["Person"], "properties": {"name": "Alice"}}]
def get_relationships(self, limit=None):
return []
monkeypatch.setattr(cli_module, "_get_graph_store", lambda ctx: _FakeStore())
result = runner.invoke(cli_module.main, ["--json", "reason", "run"])
_ok(result)
data = json.loads(result.output.strip())
assert data["facts"] == 1
assert data["inferred_count"] == 0
def test_load_rule_definitions_yaml_mapping_without_rules_key_raises(self, tmp_path):
"""A YAML mapping with no 'rules' key must raise ClickException, not silently
pass the raw YAML lines as rules."""
bad = tmp_path / "bad.yaml"
bad.write_text("some_key: some_value\nother_key: other_value\n", encoding="utf-8")
import click as _click
with pytest.raises(_click.ClickException, match="no 'rules' key"):
cli_module._load_rule_definitions(str(bad))
def test_load_rule_definitions_empty_file_returns_empty_list(self, tmp_path):
empty = tmp_path / "empty.yaml"
empty.write_text("", encoding="utf-8")
assert cli_module._load_rule_definitions(str(empty)) == []
def test_load_rule_definitions_yaml_rules_null_falls_to_plaintext(self, tmp_path):
"""rules: null is valid YAML with the key present; the null value is
not a list, so the function falls through to plain-text parsing and
returns the literal line (one no-op rule). This documents the edge
case rather than asserting a specific useful behaviour."""
f = tmp_path / "null_rules.yaml"
f.write_text("rules: null\n", encoding="utf-8")
result = cli_module._load_rule_definitions(str(f))
# Plain-text fallback: the non-comment, non-blank line becomes a rule.
assert result == ["rules: null"]
def test_run_rejects_deductive_engine(self, runner, monkeypatch):
"""Engines other than rete/forward-chain must be rejected with a helpful message."""
class _EmptyStore:
def get_nodes(self, limit=None): return []
def get_relationships(self, limit=None): return []
monkeypatch.setattr(cli_module, "_get_graph_store", lambda ctx: _EmptyStore())
result = runner.invoke(cli_module.main, ["reason", "run", "--engine", "deductive"])
assert result.exit_code != 0
assert "not wired" in result.output
assert "Traceback" not in result.output
def test_explain_requires_conclusion(self, runner):
result = runner.invoke(cli_module.main, ["reason", "explain"])
assert result.exit_code != 0
@@ -1436,6 +1586,65 @@ class TestStore:
result = runner.invoke(cli_module.main, ["store", "connect", "--backend", "neo4j"])
_ok(result)
def test_connect_dispatches_through_graph_store(self, runner, monkeypatch):
# store connect used to call get_graph_store_method(backend) — the
# method registry, which needs (task, method_name) — so it raised a
# TypeError before any connection attempt (#1354).
calls = {}
class _FakeGraphStore:
def __init__(self, backend=None, **cfg):
calls["backend"] = backend
calls["cfg"] = cfg
def connect(self):
calls["connected"] = True
return True
import semantica.graph_store as gs_mod
monkeypatch.setattr(gs_mod, "GraphStore", _FakeGraphStore)
result = runner.invoke(cli_module.main, [
"store", "connect", "--backend", "neo4j",
"--uri", "bolt://example:7687", "--json"])
_ok(result)
data = _json_output(result)
assert data == {"backend": "neo4j", "connected": True}
assert calls["backend"] == "neo4j"
assert calls["cfg"].get("uri") == "bolt://example:7687"
assert calls.get("connected") is True
def test_connect_invalid_backend_reports_error_not_dispatch_error(self, runner):
"""An unknown backend name must produce a meaningful backend error, not a
Python TypeError from the old get_graph_store_method() dispatch (#1354)."""
result = runner.invoke(cli_module.main,
["store", "connect", "--backend", "does-not-exist"])
# Exit 0 because store_connect always catches and reports errors gracefully.
_ok(result)
# The output must mention the backend, not a Python internal error.
assert "does-not-exist" in result.output
assert "TypeError" not in result.output
assert "Traceback" not in result.output
def test_connect_backend_error_surfaces_in_json(self, runner, monkeypatch):
"""A connect() failure must appear in JSON output as connected=False with an error field."""
class _FailingStore:
def __init__(self, backend=None, **cfg):
pass
def connect(self):
raise RuntimeError("auth failed")
import semantica.graph_store as gs_mod
monkeypatch.setattr(gs_mod, "GraphStore", _FailingStore)
result = runner.invoke(cli_module.main, [
"store", "connect", "--backend", "neo4j", "--json"])
_ok(result)
data = _json_output(result)
assert data["connected"] is False
assert "auth failed" in data.get("error", "")
assert data["backend"] == "neo4j"
def test_migrate_dry_run(self, runner):
result = runner.invoke(cli_module.main, ["store", "migrate",
"--from", "faiss", "--to", "qdrant", "--dry-run"])
@@ -2157,6 +2366,160 @@ class TestDoctorEmbeddingHintsAndEnv:
assert "hash fallback" in st["note"], "padded/caps env value must enable deep mode"
class TestDoctorTableLayout:
"""#1428 + Qodo review: doctor table must keep Check labels and Hint text
readable at a normal 80-column terminal.
These tests render the *human-readable* (non-JSON) doctor table into a
captured 80-column Rich console so they cover the actual column-width
arithmetic, not just the JSON data.
Two regressions are protected:
A. #1428 — Hint (and Note) columns must not collapse into unreadable
single-character fragments or be silently truncated with a layout ''.
overflow="fold" on both columns ensures content wraps across lines while
remaining fully present.
B. Qodo Long Check labels such as "Embeddings (sentence-transformers)"
must not be truncated/ellipsized. Assigning ratio=1 to the Check column
(as the original PR did) caused Rich to squeeze it below its min_width
at narrow terminals, so the fix removes ratio from the fixed-size columns.
"""
def _render_doctor_at_80(self, runner, monkeypatch):
"""Return the plain-text (ANSI-stripped) doctor table rendered at 80 cols."""
import io
import re
from rich.console import Console
# Unset LLM-provider env vars so the warn rows (with hints) are always present.
for var in ("OPENAI_API_KEY", "ANTHROPIC_API_KEY", "GROQ_API_KEY"):
monkeypatch.delenv(var, raising=False)
buf = io.StringIO()
narrow_console = Console(
file=buf, width=80, highlight=False, force_terminal=True, no_color=True
)
monkeypatch.setattr(cli_module, "console", narrow_console)
result = runner.invoke(cli_module.main, ["doctor"])
assert result.exit_code == 0, f"doctor exited non-zero: {result.output!r}"
return re.sub(r"\x1b\[[0-9;]*m", "", buf.getvalue())
def _hint_column_parts(self, output: str) -> "list[str]":
"""Extract non-blank Hint-column segments from each rendered line.
Locates the Hint column start from the header row and slices that
suffix from every subsequent line, so the test is insensitive to the
exact widths of the other columns.
"""
lines = output.splitlines()
# Line 0 is blank (console.print() blank line before table).
hdr = next((l for l in lines if "Hint" in l and "Check" in l), None)
assert hdr is not None, "Could not find table header in doctor output"
hint_start = hdr.index("Hint")
parts = []
for line in lines:
if len(line) > hint_start:
seg = line[hint_start:].rstrip()
if seg and seg != "Hint" and not set(seg).issubset({"", " "}):
parts.append(seg)
return parts
# ── B: Qodo regression ────────────────────────────────────────────────────
def test_long_check_label_not_truncated_at_80_cols(self, runner, monkeypatch):
"""'Embeddings (sentence-transformers)' must appear verbatim at 80 cols.
Before the fix, ratio=1 on the Check column let Rich squeeze it below
its min_width, turning the label into 'Embedd…' or similar.
"""
output = self._render_doctor_at_80(runner, monkeypatch)
assert "Embeddings (sentence-transformers)" in output, (
"Check label 'Embeddings (sentence-transformers)' was truncated in "
"the 80-column doctor table — the ratio= constraint on the Check "
"column must be removed so min_width=34 is always honoured."
)
def test_all_check_labels_not_truncated_at_80_cols(self, runner, monkeypatch):
"""Every standard Check label must appear verbatim at 80 cols."""
output = self._render_doctor_at_80(runner, monkeypatch)
for label in (
"Python",
"semantica",
"rich",
"Graph store",
"Vector store",
"Embeddings (sentence-transformers)",
"Embeddings (fastembed)",
"OpenAI",
"Anthropic",
"Groq",
"Config file",
"Log directory",
):
assert label in output, (
f"Check label {label!r} was truncated or missing in the "
"80-column doctor table."
)
# ── A: #1428 regression ───────────────────────────────────────────────────
def test_hint_content_fully_present_at_80_cols(self, runner, monkeypatch):
"""The LLM-provider hints must be fully present (folded, not ellipsized).
With overflow='fold' the full hint text wraps across lines; no
characters are discarded. Joining the Hint-column segments (stripping
whitespace) must reconstruct each complete hint string.
"""
import re
output = self._render_doctor_at_80(runner, monkeypatch)
parts = self._hint_column_parts(output)
hint_joined = re.sub(r"\s+", "", "".join(parts))
# Each LLM-provider hint must be fully recoverable from the folded lines.
for expected in (
"exportOPENAI_API_KEY=\u2026", # export OPENAI_API_KEY=…
"exportANTHROPIC_API_KEY=\u2026", # export ANTHROPIC_API_KEY=…
"exportGROQ_API_KEY=\u2026", # export GROQ_API_KEY=…
):
assert expected in hint_joined, (
f"Hint content {expected!r} is missing from the 80-column "
"doctor table — overflow='fold' must be set on the Hint column "
"so no content is silently discarded."
)
def test_hint_column_has_no_single_char_fragments_at_80_cols(
self, runner, monkeypatch
):
"""No Hint-column line must be a single alphabetic character.
The original #1428 bug produced outputs like:
export
O
P
E
N
A
I
...
because Rich allocated the Hint column only 12 characters of content
width. overflow='fold' on a properly-wide column eliminates this.
"""
output = self._render_doctor_at_80(runner, monkeypatch)
parts = self._hint_column_parts(output)
single_char_alpha = [p for p in parts if len(p.strip()) == 1 and p.strip().isalpha()]
assert not single_char_alpha, (
f"Hint column contains single-character lines {single_char_alpha!r} "
"at 80 columns — the Hint column is too narrow; check min_width and "
"ratio settings."
)
class TestEmbedGenerateOutput:
"""#994: `embed generate --output` must write files `embed index` can read."""