Commit Graph
45 Commits
Author SHA1 Message Date
Mohd KaifandZohaib Hassnain 3496d62335 security: require API-key auth on all Explorer API routes (GHSA-j4mq) (#909)
* security: require API-key auth on all Explorer API routes (GHSA-j4mq-hprp-987v)

Every Explorer route (bulk import/export, delete, LLM-backed ontology
generation, SPARQL, etc.) was mounted with no authentication, and both
server entrypoints bind 0.0.0.0 by default. Anyone reaching the port got
full read/write/delete on the graph.

- Add require_auth dependency (explorer/dependencies.py): checks
  X-API-Key against SEMANTICA_API_KEY, fails closed with 503 if
  unconfigured (not silently anonymous), 401 on wrong/missing key.
  SEMANTICA_ALLOW_ANONYMOUS=true opts out explicitly for local dev.
- Wire dependencies=[Depends(require_auth)] into all 11 API routers in
  both explorer/app.py and server.py. /health, /api/info, static assets,
  and the SPA catch-all stay public.
- /ws/graph-updates handshake now checks the same key via header or
  ?api_key= query param (browsers can't set custom WS headers) before
  accepting the connection.
- Default bind changed from 0.0.0.0 to 127.0.0.1 in server.py's main()
  and cli.py's `server start`; the CLI warns if a non-loopback host is
  passed explicitly without a key configured.
- Startup logging reports the resolved auth mode in both app factories.
- Document/generate SEMANTICA_API_KEY in the deploy recipes that expose
  a public endpoint by default: docker-compose, Railway, Fly, Render.

Added tests/explorer/test_explorer_auth.py covering fail-closed default,
wrong/missing/correct key, anonymous opt-in, public-route exemptions, and
the WS handshake. Added tests/explorer/conftest.py defaulting the
pre-existing ~200 explorer tests to SEMANTICA_ALLOW_ANONYMOUS=true so
they keep exercising route logic without needing a key.

* fix CORS

---------

Co-authored-by: Zohaib Hassnain <109234410+ZohaibHassan16@users.noreply.github.com>
2026-08-11 14:05:00 +05:00
KaifAhmad1 0a8330cbb0 fix(provenance): address code review findings on PR #827
- SQLiteStorage now migrates an existing (pre-#825) provenance.db in place
  via ALTER TABLE ADD COLUMN for any columns introduced since, instead of
  only ever running CREATE TABLE IF NOT EXISTS. Without this, opening an
  older database with the new code would break on the first insert/select
  since the row width and _row_to_entry's fixed indices grew past the old
  schema. Added test_migrates_pre_existing_old_schema_database.

- verify_chain() now also checks that sequence_id is exactly the
  predecessor's plus one (no gap, no duplicate), in addition to the existing
  previous_checksum comparison. Hardens against the narrow case where
  compute_checksum()'s deliberate exclusion of entity_id could let two
  distinct rows coincidentally share a checksum, which alone would let a
  checksum-only comparison miss a gap. Added
  test_verify_chain_detects_tampered_sequence_gap.

- Explorer provenance route: edge ids now include direction
  (f"{src}-{eid}-{direction}") to match the seen_edges dedupe key, which
  already included it. The same (src, target) pair can legitimately appear
  in both the upstream and downstream chains (cycles/overlap), and without
  this the two edges collided on the same id. Added
  test_add_chain_edges_ids_distinguish_direction.

- Removed an unused `Any` import in parse_provenance.py.
2026-08-03 22:52:34 +05:30
Sameer6305 6f6c825f3d fixed qodo reviews
- Removed unused `validate_skos_hierarchy` import from
  test_ontology_subissue3.py (flake8 F401); the test uses a `wraps=` spy
  on the real add_nodes_and_edges instead of calling the helper directly.
- refresh_ontology tests now percent-encode the ontology URI with
  urllib.parse.quote before interpolating it into the {ontology_uri:path}
  request path, matching the already-encoded unknown-uri refresh test in
  the same file instead of embedding a raw http://... URI with slashes.
- Reworded the cyclic-SKOS refresh test's comment and section header:
  GraphSession.add_nodes_and_edges() documents pre-write validation and
  lock-based mutual exclusion, not transactional rollback, so "atomic"
  was replaced with "single combined add_nodes_and_edges() call" to avoid
  implying rollback guarantees that don't exist.

Verified: tests/explorer/test_ontology_subissue3.py (34 passed) and
tests/explorer/ (204 passed), no regressions.
2026-08-01 19:36:16 +05:30
Sameer6305 d293ca6009 fix(explorer): complete atomic ontology refresh writes (#775) 2026-08-01 19:12:00 +05:30
KaifAhmad1andmikemikimike bc75768afe Fix two review findings in SKOS cycle validation
- validate_skos_hierarchy() re-walked every existing hierarchy edge in
  the graph on each write, so one pre-existing cycle anywhere would
  block all unrelated future SKOS writes. It now only traverses
  concepts touched by the edges actually being written, while still
  checking those against existing edges for cross-boundary cycles.
- In /api/ontology/load, `except HTTPException: raise` sat after a
  broader `except Exception` clause that already matched HTTPException,
  so a 422 raised after a successful OntologyIngestor parse was
  silently swallowed and retried via the fallback RDF parser instead of
  reaching the caller. Reordered the except clauses.

Co-authored-by: mikemikimike <13286568797@163.com>
2026-08-01 11:46:13 +05:30
mikemikimike f992504227 Make SKOS hierarchy imports atomic 2026-07-31 23:23:10 +05:30
mikemikimike d41530930d Centralize SKOS cycle validation 2026-07-31 23:07:58 +05:30
mikemikimike 692260cc76 Reject cyclic SKOS hierarchies 2026-07-31 23:07:58 +05:30
Sameer6305 c2079d8e92 fix(explorer): preserve audit evidence fields in provenance nodes and reports (#792)
- Extend ProvenanceNode in semantica/explorer/schemas.py with audit evidence fields: source_document, source_location, source_quote, confidence, and checksum.

- Update _transform_audit_lineage in semantica/explorer/routes/provenance.py to populate these evidence fields for each lineage node from ProvenanceEntry records, while keeping default None values for orphan nodes.

- Include source_document, confidence, and checksum in markdown report rendering (_render_markdown) so exported markdown reports surface attribution and integrity evidence.

- Add unit test test_provenance_audit_evidence_fields_preserved in test_provenance_manager_wiring.py verifying that evidence fields are present across /api/provenance JSON responses and exported JSON/markdown reports.
2026-07-28 17:15:49 +05:30
Sameer6305 cc864362aa fix(provenance): verify checksum integrity of lineage entries before labeling source as audit (#792)
- Update verify_checksum and compute_checksum in semantica/provenance/integrity.py to support both ProvenanceEntry objects and serialized dictionary entries.

- Add integrity_verified flag computed via verify_checksum to the dictionary returned by ProvenanceManager.get_lineage().

- Update _build_provenance in semantica/explorer/routes/provenance.py to verify every returned lineage entry before labeling the result as source=audit. If verification fails due to missing checksums or corrupted records, log a warning and fall back cleanly to graph traversal.

- Add unit test test_provenance_manager_wiring_checksum_failure_falls_back in test_provenance_manager_wiring.py verifying that tampered lineage entries trigger fallback to source=graph_traversal.
2026-07-28 17:08:12 +05:30
Sameer6305 d30aea79a7 fix(explorer): classify multi-hop audit lineage as upstream and enforce provenance storage configuration (#792)
- Fix _transform_audit_lineage to classify all non-downstream ancestor derivation edges as 'upstream' instead of 'lateral', correcting multi-hop lineage direction in JSON and markdown reports.

- Add GraphSession.set_provenance_storage_path() to explicitly reject conflicting preconfigured storage paths or path mutations after provenance_manager initialization.

- Update create_app() to call active_session.set_provenance_storage_path(prov_path), preventing silent retention of conflicting paths or un-redirectable cached managers.

- Remove unused logging import in app.py.

- Add comprehensive unit tests in test_provenance_manager_wiring.py for upstream edge classification, markdown report grouping, conflicting path rejection, and manager initialization lockouts.
2026-07-28 16:57:38 +05:30
Sameer6305 2ee4da0b47 fix(explorer): wire ProvenanceManager into provenance routes (closes #792)
- Explorer's /api/provenance now queries the audit-grade ProvenanceManager
  (SQLite-backed, checksummed) first, falling back to the naive 2-hop
  graph traversal when no audit records exist for a node.
- Fixed a process-global mutable-state risk in the initial approach:
  provenance storage path is threaded per-session via GraphSession,
  not via ProvenanceManager's global set_default_storage_path classmethod.
- Added source: 'audit' | 'graph_traversal' to the response so callers
  can distinguish which path served the data.
- Documented a known limitation: ProvenanceManager currently only
  traces upstream/ancestor lineage, not descendants — the naive
  fallback remains the only source for downstream relationships until
  ProvenanceManager gains a reverse lookup (tracked separately).
- Warns (rather than silently no-ops) if a provided session's
  provenance_manager was already constructed before create_app()
  applied a provenance_storage_path.
- Never lets a provenance-manager failure crash the route; degrades
  to the naive path with a logged warning instead.

Tests: 5 new tests in test_provenance_manager_wiring.py covering the
audit path, empty-record fallback, storage-failure degradation, app
startup wiring, and cross-session storage isolation. Full
tests/explorer/ + tests/provenance/ suite passing, order-invariant.
2026-07-28 16:23:02 +05:30
Mohd Kaif 80bce453c3 Merge pull request #805 from Sameer6305/fix/773-sparql-test-coverage
test(explorer): add coverage for SPARQL route (#773)
2026-07-27 19:33:07 +05:30
KaifAhmad1 d102584af6 fix(explorer): dedupe SPARQL row-cap logic and cover CONSTRUCT/DESCRIBE truncation
Extracts the row-cap-and-truncate loop (duplicated between the
CONSTRUCT/DESCRIBE and SELECT branches) into a shared _cap_rows()
helper, and adds a test for the previously-uncovered CONSTRUCT/DESCRIBE
truncation path. Addresses review nits on PR #805.
2026-07-27 19:27:35 +05:30
KaifAhmad1 28fe304f76 fix(ontology): address review follow-ups on live SHACL validation (#804)
- Revert create_ontology silently falling back to a near-empty ontology on
  generation failure; restores the HTTPException(500) behavior from #770/#787
  that this PR had accidentally undone (and re-enables TestOntologyCreateFailures)
- Fold sh:Warning/sh:Info severity pySHACL results into the /shacl/validate
  response's violations array instead of silently dropping them, so a
  non-conforming report is never returned with an empty violations list
- Share a single nodes/edges fetch between _generated_shacl_for_uri and
  _data_graph_turtle_for_uri via new _fetch_analysis_graph(), so /health
  no longer re-queries and re-truncation-checks the same ontology twice
2026-07-27 18:28:11 +05:30
Sameer6305 db95cedf34 fixed qodo reviews and hardened implementation 2026-07-27 15:42:49 +05:30
Sameer6305 dd7b090aec test(explorer): add coverage for SPARQL route (#773)
sparql.py handles direct SPARQL query execution against the live graph with no test coverage anywhere in the repo. Adds coverage for the read-only allowlist (the actual security boundary here), row/timeout limits, error handling, and RDF projection fidelity.
2026-07-27 15:00:47 +05:30
Sameer6305 8430d4a56e fix(ontology): address qodo review findings for SHACL validation
- DoS guardrails: enforce byte size, triple count, concurrency, and timeout limits on /shacl/validate

- Slash namespace resolution: preserve trailing slash in _resolve_uri so local terms match SHACL shapes

- Truncation safety: raise GraphTruncationError and report unavailable/413/critical when graphs exceed analysis limits

- JSON-LD dict lists: unwrap uri/id/@id in _as_uri_list and _data_graph_turtle_for_uri property loops

- Observability & efficiency: add warning logs on truncation and avoid duplicate UTF-8 encoding in size check
2026-07-27 14:01:45 +05:30
Sameer6305 3a1ab1a8a6 fix(ontology): wire live SHACL validation into /shacl/validate and /health
Closes #772
2026-07-27 13:02:04 +05:30
KaifAhmad1 b1deed5857 Address review: harden analytics status codes, add failure-path tests
207 alone is indistinguishable from 200 to callers that only check
response.ok, so /api/analytics now raises 500 when every requested
metric fails and reserves 207 for genuine partial failure. Adds
regression tests for the temporal, analytics, and ontology-create
failure paths introduced in this PR, and logs the fix in the
changelog's Unreleased section.
2026-07-23 18:13:26 +05:30
KaifAhmad1 b9e069301f fix(deploy): address security and correctness blockers from PR review
- gcp/cloudrun-service.yaml: add comment + README sed one-liner so PROJECT_ID
  is substituted before gcloud run services replace (was a literal placeholder
  that caused image-pull failure on the declarative deploy path)
- azure/main.parameters.json: replace wildcard allowedOrigins "*" with a
  REPLACE_ME placeholder; add README note to set the real URL after first deploy
- kubernetes/networkpolicy.yaml + helm networkpolicy template: add from: selector
  (ingress-nginx namespace + same-namespace pods) so ingress is no longer
  allow-all; restrict egress to FalkorDB port 6379 and DNS port 53 instead of
  the allow-all egress: - {} wildcard
- helm/values.yaml: expose networkPolicy.ingressNamespace and falkordbPort values
- kubernetes/deployment.yaml: add secretRef for knowledge-explorer-secrets so
  FALKORDB_PASSWORD is actually injected into the container
- app.py: add _mutation_bridge_installed guard to prevent closure stacking when
  the same GraphSession is passed to create_app() more than once; remove
  duplicate app.state.allowed_origins assignment (single source of truth is
  app.state.explorer_settings); add comment on falkordb_host/port dead config
- tests: update allowed_origins assertions to use explorer_settings dict
- .checkov.yaml: remove global CKV_K8S_21/28/30 suppressions; rely on per-file
  inline checkov:skip comments in cloudrun-service.yaml so future real K8s
  manifests are not silently exempted
2026-06-24 22:55:18 +05:30
KaifAhmad1 b2c949f7de fix(deploy): harden security in deployment templates and explorer app
- GCP: remove --allow-unauthenticated, restrict ingress to
  internal-and-cloud-load-balancing, replace wildcard ALLOWED_ORIGINS=*
  with a substitution variable (_ALLOWED_ORIGINS) so operators supply a
  real URL at deploy time; same fix in cloudrun-service.yaml
- Fly.io: replace hardcoded FALKORDB_HOST=localhost with the correct
  .internal private-network hostname pattern; update README accordingly
- docker-compose.dev.yml: add missing top-level networks: block so the
  frontend service can join the semantica network without --file layering
- K8s/Helm: add readOnlyRootFilesystem: true + runAsUser: 1000 to
  container securityContext; mount an emptyDir /tmp so uvicorn can write
  temp files
- app.py: fix _read_explorer_settings() or-chain, use in os.environ
  checks so an explicit ALLOWED_ORIGINS="" produces an empty allow-list
  instead of silently falling through to localhost defaults; remove dead
  app.state.falkordb_host/port attributes
- docs: update four locations that still documented {"status":"healthy"}
  to reflect the new {"status":"ok"} health response
- tests: update test assertion to read falkordb settings from
  app.state.explorer_settings instead of removed top-level attributes
2026-06-24 12:51:09 +05:30
Zohaib Hassnain 21ddee94f7 Add Knowledge Explorer deployment templates 2026-06-23 13:37:25 +05:00
Mohd KaifandZohaib Hassnain 46447d1f3f fix(explorer): resolve blank dashboard UI and ship frontend bundle in wheel (#638)
* fix(explorer): resolve blank dashboard UI and ship frontend bundle in wheel

Fixes #631 — the Explorer server started successfully but the browser showed
a blank page because semantica/static/ was gitignored and never present after
a fresh install or clone.

Changes:
- ci.yml / release.yml: add Node 20 setup + npm ci && npm run build before
  python -m build so every wheel contains a CI-built frontend bundle
- pyproject.toml: add package-data patterns (static/*, static/assets/*) so
  setuptools includes the bundle in the wheel; add MANIFEST.in for sdist coverage
- app.py: replace silent empty-HTML fallback with a 200 page that clearly
  explains the missing bundle and links to /docs; fix CORS allow_credentials
  to default false, gated behind EXPLORER_CORS_CREDENTIALS env var to prevent
  credentialed cross-origin requests on unauthenticated endpoints
- __init__.py: warn at startup when --host is non-loopback (unauthenticated
  network exposure)
- explorer/README.md: full rewrite covering pip-install mode (primary path,
  no Node required) and dev-server mode (contributors), CLI flags, env vars,
  workspace table, troubleshooting for the blank-page symptom
- README.md: update Knowledge Explorer section with correct command and link
  to the new setup guide

* fix(explorer): set build.target esnext to fix esbuild CI failure

esbuild >=0.28 (forced via npm overrides) conflicts with Vite 6 defaults on
Linux CI — it tries to lower destructuring syntax for the implicit browser
target list but errors out. Explicit target: 'esnext' tells esbuild to emit
native syntax unchanged, bypassing the transpilation error entirely. Safe for
a developer tool that runs in modern browsers.

* test(explorer): verify packaged frontend bundle

---------

Co-authored-by: Zohaib Hassnain <109234410+ZohaibHassan16@users.noreply.github.com>
2026-06-16 14:38:24 +05:30
63acc7a66e fix(ontology): address Qodo automated review findings from PR #524
Backend (semantica/explorer/routes/ontology.py):
- suggest-alignments: add TF-IDF character-ngram embeddings via sklearn
  (SimilarityCalculator-compatible cosine scoring) so embedding_similarity
  is populated in results; combined score = 0.4*label + 0.6*embedding when
  available, falling back to label-only when sklearn is absent
- suggest-alignments: add token-overlap prefilter before SequenceMatcher so
  zero-Jaccard pairs are skipped without computing full similarity; add
  _MAX_ENTITIES_PER_SIDE=500 per-ontology cap on top of the existing
  _MAX_ANALYSIS_NODES global cap
- suggest-alignments: remove dead try/except OntologyEngine.create_alignment
  block that always failed silently (no TripletStore configured); replace
  with a comment explaining the intentional ephemeral-only storage model
- health: replace O(alignments x entities) any() scans for alignment coverage
  with O(1) set membership checks via assessed_ids
- shacl/validate: run rdflib.Graph().parse(format='turtle') syntax check on
  the submitted Turtle before returning; invalid syntax now raises 422 instead
  of returning a misleading unavailable/success response

Frontend:
- AlignmentsTab: add pairwise alignment matrix section that groups recorded
  alignments by (source_ontology, target_ontology) pair; each cell shows
  color-coded relation badges per RELATION_COLORS; clicking a badge populates
  the create/edit form for quick editing; matrix is shown when at least two
  ontologies are loaded
- ShaclStudio: add selectedShapeId state and fullShacl ref; each shape row in
  the library is now a clickable button that extracts its Turtle block from
  the full SHACL and pre-populates the Monaco editor; a "View all" toggle
  restores the full SHACL; selected shape ID is shown in the editor header
- GraphWorkspace: fix viewMode race in external focus effect — call
  setSelectedNodeId directly instead of going through focusNode(), which
  captured a stale viewMode in its closure; remove focusNode from the
  dependency array since it is no longer called

Tests (14 passing, was 11):
- Add test_suggest_alignments_returns_embedding_similarity: asserts
  embedding_similarity is non-null when sklearn is available
- Add test_shacl_validate_rejects_invalid_turtle_syntax: asserts 422 on
  syntactically invalid Turtle
- Add test_health_alignment_coverage_uses_set_lookup: asserts alignment
  dimension score is non-zero after recording an alignment, verifying the
  O(1) set lookup path works correctly end-to-end

Co-authored-by: KaifAhmad1 <mohammadk78600@gmail.com>
Co-authored-by: ZohaibHassan16 <zohaib@hawksight.ai>
2026-05-02 16:38:09 +05:30
00ceb09960 fix(ontology): address review blockers from PR #524
Backend:
- Replace false conforms=True SHACL stub with status=unavailable always;
  live validation cannot be wired until OntologyEngine.validate_graph is
  connected to a data graph — a stub that returns conforms=True misleads
  users editing shapes
- Cap node/edge fetches in health, suggest-alignments, and SHACL generation
  at _MAX_ANALYSIS_NODES (5 000) with a logger.warning when the graph
  exceeds the limit; unbounded limit=999_999 fetches cause OOM on large graphs
- Set SHACL health dimension score to 0.0 (was 70.0) when status=unavailable;
  exclude unavailable dimensions from the total_score average so they neither
  inflate nor deflate the result
- Allow alignments to reference external/unloaded URIs (e.g. schema.org)
  without raising 404; label falls back to URI fragment or caller-supplied
  source_label/target_label fields added to OntologyAlignmentRequest
- Fix _alignment_id to use uuid.NAMESPACE_OID instead of NAMESPACE_URL;
  the composite key is not a URL
- Fix _summarize_shapes to normalise \r\n before splitting on .\n so shape
  parsing works correctly on Windows line endings

Frontend:
- Wrap handleSave/handleSuggest/handleRemove/handleAcceptSuggestion in
  useCallback in AlignmentsTab for consistency with sibling components
- Add ephemeral-storage banner in AlignmentsTab warning that alignments are
  session-memory-only and not persisted across restarts
- Fix exportReport in HealthTab to append/remove anchor from document before
  clicking and defer URL.revokeObjectURL to avoid Blob URL leak in some browsers
- Derive health dimension grid column count from health.dimensions.length
  instead of the hardcoded repeat(5, ...) that breaks if the backend adds
  or removes a dimension
- Add minimal Monarch tokenizer for the Monaco turtle language registration
  in ShaclStudio so prefix declarations, IRIs, SHACL properties, comments,
  and string literals are syntax-highlighted; previously the editor rendered
  as plain text despite theme rules being defined

Tests (11 passing, was 5):
- Rename test_shacl_validate_has_stable_contract to
  test_shacl_validate_returns_unavailable and assert status == unavailable
- Add test_shacl_validate_rejects_empty_turtle (expects 422)
- Add test_health_returns_404_for_unknown_ontology
- Add test_health_shacl_dimension_is_zero_when_unavailable with total_score check
- Add test_delete_unknown_alignment_returns_404
- Add test_alignment_upsert_is_idempotent (verifies ID stability and created_at
  preservation across updates)
- Add test_alignment_accepts_external_uri (verifies no 404 for schema.org URIs)
- Relax test_alignment_suggestions_are_ranked label assertions to substring
  checks so the test survives similarity algorithm changes

Co-authored-by: KaifAhmad1 <mohammadk78600@gmail.com>
Co-authored-by: ZohaibHassan16 <zohaib@hawksight.ai>
2026-05-02 15:53:47 +05:30
Zohaib Hassnain e8bf0e50d3 feat(ontology): add alignments health and shacl studio 2026-05-01 22:59:15 +05:00
bb956b2735 fix(explorer): address PR #515 review findings
- Align _coerce_embedding_vector inner dict-probe key list with
  _extract_node_embeddings outer key list (add 'embeddings', reorder to
  generic-first) so nested embedding dicts resolve consistently.
- Add TODO comment on _extract_node_embeddings to cache per-session
  graph revision and avoid O(N) re-scan on every semantic request.
- Add deprecation docstrings to legacy path-segment routes
  (/node/{id}/path and /node/{id}/semantic-neighborhood) documenting
  the known slash-in-ID limitation and pointing to the query-param
  alternatives.
- Extract _FakeSimilarity to module level so it is shared without
  duplication across test classes.
- Rewrite test_legacy_semantic_neighborhood_still_works_for_simple_ids
  as a fully isolated TestClient session instead of mutating the
  shared module-scoped 'client' fixture, preventing cross-test
  state pollution.
- Extract _make_slash_node_session helper to reduce boilerplate in the
  slash-safe route tests.

Co-Authored-By: ZohaibHassan16 <109234410+ZohaibHassan16@users.noreply.github.com>
Co-Authored-By: KaifAhmad1 <98801504+KaifAhmad1@users.noreply.github.com>
2026-04-29 23:17:57 +05:30
Zohaib Hassnain e3a3f6010b fix(explorer): make distance intelligence API calls slash-safe 2026-04-29 21:17:36 +05:00
KaifAhmad1 fe6ca7fccb fix(search-index): restore secondary-scan node ordering and add regression test 2026-04-19 18:46:05 +05:30
Mohd Kaif 66c8431eee Merge branch 'main' into feat/optimize-search 2026-04-19 18:25:28 +05:30
Zohaib Hassnain 6f93f429c4 perf(explorer): add indexed search for large graphs 2026-04-17 21:22:12 +05:00
Sameer6305 658de23357 fix: resolve merge conflicts with upstream main 2026-04-17 19:56:41 +05:30
Sameer6305 66e8964d22 fix(provenance): include upstream ancestors + add direction classification and markdown grouping 2026-04-17 19:33:12 +05:30
KaifAhmad1 390152c78c feat(explorer): add node distance semantics to PathResponse (#472)
- Extend PathResponse with hop_count (len(path)-1) and distance_band
  ("direct"|"near"|"mid-range"|"distant") as first-class API fields
- Add classify_path_distance() to semantica/utils/helpers.py as the
  single source of truth for hop-count thresholds; both the route and
  the visualizer import from it, eliminating duplicate threshold logic
- Populate hop_count and distance_band in find_path route via
  classify_path_distance(); remove local _classify_distance() copy
- Add highlight_path: list[str] param to KGVisualizer.visualize_network;
  path edges rendered as a distance-aware orange trace (opacity and
  stroke width scale from direct→distant: 1.0/4px to 0.35/1.5px)
- Fix bidirectional edge lookup: only forward pairs (A→B) along the
  path are added to path_edge_set; reverse back-edges in directed
  graphs are no longer incorrectly highlighted
- Add logger.warning when highlight_path contains node IDs absent from
  the layout position map, surfacing silent no-op mismatches
- Extend frontend PathResponse type in GraphInspectorPanel.tsx and
  GraphWorkspaceShell.tsx with hop_count: number and distance_band
  literal union to match the updated API contract
- Add 10 new tests: 2 API-level and 8 unit tests covering all four
  band boundaries (0, 1, 2, 3, 4, 6, 7, 20 hops); 104 explorer tests
  pass, 0 failures introduced
- Update CHANGELOG.md
2026-04-16 17:59:50 +05:30
KaifAhmad1andClaude Sonnet 4.6 523b02083f feat(explorer): add bidirectional path finding with directed=false param (#469)
- PathFinder.bfs_shortest_path() and dijkstra_shortest_path() gain a
  directed: bool = True parameter. When False, a temporary undirected
  view (graph.to_undirected()) is used for traversal only; the original
  directed edges are preserved and returned in the response.
- _make_undirected_view() helper added to PathFinder; falls back safely
  for non-NetworkX graph types.
- GET /api/graph/node/{id}/path exposes ?directed=false query param.
- PathResponse gains a directed: bool field echoing the mode used.
- Route now returns 404 on empty path (previously returned 200 with
  path: []).
- 21 new tests: 12 unit (TestBidirectionalPathFinding) + 9 API-level
  (TestBidirectionalPathRoute). All 120 tests pass.
- CHANGELOG updated under [Unreleased].

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 15:20:12 +05:30
Mohd KaifandClaude Sonnet 4.6 8eafd2d024 fix(explorer): replace KeyError/ValueError with HTTPException across all routes, fix temporal pattern method, add SPA root handler (#463)
- routes/graph.py: raise HTTPException(404) for missing nodes; wrap path_finder call in try/except → HTTPException(404)
- routes/decisions.py: raise HTTPException(404) on chain, compliance, precedents sub-routes
- routes/annotations.py: raise HTTPException(404) on create (missing node) and delete (missing annotation)
- routes/enrich.py: raise HTTPException(404/503/422) for missing nodes, unavailable services, and extraction errors
- routes/temporal.py: fix TemporalPatternDetector method name detect_patterns → detect_temporal_patterns
- explorer/app.py: root / now serves built index.html when available, falls back to minimal HTML shell; add HTMLResponse import
- tests/explorer/test_explorer_api.py: test_extract accepts 503 (spacy/transformers not installed is a valid service state)

All 45 explorer API integration tests pass.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 00:19:19 +05:30
Zohaib Hassnain dfd7785cc1 feat: overhaul graph explorer visuals and loading flow 2026-04-11 03:19:28 +05:00
Zohaib Hassnain 38b766298e feat(explorer): harden knowledge explorer backend and frontend, polish dashboard UX 2026-04-07 02:04:17 +05:00
KaifAhmad1andClaude Sonnet 4.6 f677b638e2 fix(explorer): resolve test crash, import error handling, cycle safety, and missing utils package
- test_vocabulary.py: remove sys.modules['spacy'] = MagicMock() — caused
  ValueError in pytest collection when transformers called
  importlib.util.find_spec('spacy') on a MagicMock without __spec__;
  add setup_function() reset_mock() to prevent cross-test state pollution;
  expand from 3 to 16 tests covering narrower edges, topConceptOf,
  hasTopConcept, flat scheme, empty scheme, missing param, cycle safety,
  .rdf/.owl format path, invalid file 422, and metadata envelope fallback
- vocabulary.py: /import returned HTTP 200 with {"status":"error"} on parse
  failure — now raises HTTPException(422) so clients get a proper error code;
  replace bare except with ValueError-specific catch, move add_nodes/add_edges
  outside the try block
- vocabulary.py: get_hierarchy tree assembly had no cycle detection — cyclic
  broader/narrower edges in real-world SKOS data would cause infinite recursion
  during Pydantic serialization; replaced inline loop with recursive
  _attach_children() that carries a visited set
- semantica/explorer/utils/: branch was based on main and missing rdf_parser.py
  and __init__.py (introduced in #425); copied from ebd2be3 so vocabulary.py
  import resolves correctly
- tests/explorer/test_rdf_parser.py: carried forward from #425 (32 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 19:02:24 +05:30
ZohaibHassan16 2537976e8f feat(explorer): implement SKOS vocabulary routes and integration tests 2026-03-30 17:01:11 +05:00
KaifAhmad1andClaude Sonnet 4.6 ac047f917a fix(explorer): resolve merge-artifact syntax errors and clean up all route files
app.py:
- Fix unclosed '(' in generic_error_handler (two implementations were merged,
  leaving the return JSONResponse( call with no closing paren)
- Remove duplicate 'from fastapi import FastAPI, Request' import
- Remove unused 'import traceback'
- Remove duplicate static file mount (was mounted twice: once conditionally,
  once unconditionally creating the dir — FastAPI raises on duplicate mounts)

decisions.py:
- Remove stub 'return ComplianceResponse(compliant=True)' with unclosed '('
  that was left in front of the real edge-scan implementation

temporal.py:
- Remove blocking get_nodes/get_edges calls (without asyncio.to_thread) that
  were left as dead code above the correct async versions
- Fix empty 'except Exception:' clause before 'except ImportError:' that
  caused a SyntaxError

tests/explorer/test_explorer_api.py:
- Remove all merge-artifact duplicate class definitions (TestAnalytics x2,
  TestReasoning x2, TestAnnotations x2) — Python silently used the second
  definition, hiding the first; collapsed into single canonical classes
- Fix test_snapshot_at referencing undefined 'body' (no request was made);
  merged its assertions into test_snapshot_now
- Fix test_compliance asserting isinstance(body, list) on a dict response;
  the displaced precedents-check code is now in test_precedents where it
  belongs
- Fix test_compliance_with_violation using wrong session reference
- Remove duplicate node-lookup and duplicate assertions throughout
- Add test_search_content_populated: asserts search results carry non-empty
  content (regression guard for the to_dict envelope fix)
- Add test_import_edge_metadata_preserved: asserts edge metadata survives the
  import round-trip (regression guard for the properties/metadata fallback fix)

All 51 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 00:18:08 +05:30
Mohd Kaif a0e7e9a1c9 Merge branch 'main' into fix/cg-thpag 2026-03-19 23:39:20 +05:30
88bd7d6b05 fix(explorer): resolve all PR review issues — bugs, tests, refactor
Bugs fixed:
- enrich.py: predict_links called predictor.predict_links() with wrong
  signature (graph_dict as graph_store, node_id as node_labels, top_n
  instead of top_k). Rewrote to iterate candidate nodes and call
  score_link(session.graph, src, candidate) directly.
- enrich.py: detect_duplicates called session.get_nodes() synchronously
  in an async handler, blocking the event loop. Wrapped in to_thread().
- export_import.py: temp file was leaked on export exception. Now always
  cleaned up via try/finally. Moved `import os` to module level.
- pyproject.toml: missing comma between two strings in the `all` extra
  caused a TOML syntax error breaking `pip install semantica[all]`.
- app.py: generic Exception handler swallowed HTTPException(503) raised
  by get_session dependency. Now re-raises HTTPException explicitly.
- decisions.py: compliance endpoint imported PolicyEngine then discarded
  it, always returning compliant=True. Replaced with in-graph check:
  scans for violates/non_compliant/breaches edges from the decision node.
- app.py: removed unused `import traceback`.

Refactor:
- session.py: added build_graph_dict(node_ids=None) method to eliminate
  _build_graph_dict() duplication across graph.py, analytics.py, and
  export_import.py (three identical copies).
- session.py: all 8 lazy analytics properties now initialise under _lock
  to prevent double-instantiation under concurrent requests.
- graph.py: find_path now dispatches to dijkstra_shortest_path or
  bfs_shortest_path based on the `algorithm` query param (was always BFS).
- annotations.py: removed unnecessary get_annotations() round-trip in
  create_annotation — add_annotation mutates ann_data in-place.
- temporal.py: split bare `except Exception` into ImportError (silent)
  and Exception (logs warning), so real bugs are no longer hidden.

Tests (49 total, all passing):
- Added TestEnrichExtract, TestLinkPrediction, TestDedup classes.
- Added test_compliance_with_violation to verify real violation detection.
- Added test_snapshot_at_excludes_temporal_node, test_diff assertions,
  test_export_json_subset, test_import_with_edges, test_import_unsupported_format.
- Strengthened analytics, search, and annotation assertions.
- Reasoning test now asserts response shape when status is 200.

Co-authored-by: ZohaibHassan16 <zohaibhassan16@users.noreply.github.com>
Co-authored-by: KaifAhmad1 <kaifahmad1@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 16:27:17 +05:30
ZohaibHassan16 99a4db3ece feat: implement Knowledge Explorer API backend 2026-03-15 20:29:41 +05:00