Commit Graph
270 Commits
Author SHA1 Message Date
Mohd KaifandZohaib Hassnain 46447d1f3f fix(explorer): resolve blank dashboard UI and ship frontend bundle in wheel (#638)
* fix(explorer): resolve blank dashboard UI and ship frontend bundle in wheel

Fixes #631 — the Explorer server started successfully but the browser showed
a blank page because semantica/static/ was gitignored and never present after
a fresh install or clone.

Changes:
- ci.yml / release.yml: add Node 20 setup + npm ci && npm run build before
  python -m build so every wheel contains a CI-built frontend bundle
- pyproject.toml: add package-data patterns (static/*, static/assets/*) so
  setuptools includes the bundle in the wheel; add MANIFEST.in for sdist coverage
- app.py: replace silent empty-HTML fallback with a 200 page that clearly
  explains the missing bundle and links to /docs; fix CORS allow_credentials
  to default false, gated behind EXPLORER_CORS_CREDENTIALS env var to prevent
  credentialed cross-origin requests on unauthenticated endpoints
- __init__.py: warn at startup when --host is non-loopback (unauthenticated
  network exposure)
- explorer/README.md: full rewrite covering pip-install mode (primary path,
  no Node required) and dev-server mode (contributors), CLI flags, env vars,
  workspace table, troubleshooting for the blank-page symptom
- README.md: update Knowledge Explorer section with correct command and link
  to the new setup guide

* fix(explorer): set build.target esnext to fix esbuild CI failure

esbuild >=0.28 (forced via npm overrides) conflicts with Vite 6 defaults on
Linux CI — it tries to lower destructuring syntax for the implicit browser
target list but errors out. Explicit target: 'esnext' tells esbuild to emit
native syntax unchanged, bypassing the transpilation error entirely. Safe for
a developer tool that runs in modern browsers.

* test(explorer): verify packaged frontend bundle

---------

Co-authored-by: Zohaib Hassnain <109234410+ZohaibHassan16@users.noreply.github.com>
2026-06-16 14:38:24 +05:30
KaifAhmad1andZohaib Hassnain 496b80cf2b tests: fix CodeQL lint in progress tracker regression tests
Consolidate dual import (module alias + from-import) to a single
`import ... as progress_module` alias and qualify all references.
Replace bare `BaseException` catch with `Exception` in the thread
runner helper.

Co-Authored-By: Zohaib Hassnain <zohaib179949@gmail.com>
Co-Authored-By: KaifAhmad1 <kaifahmad087@gmail.com>
2026-06-16 11:03:45 +05:30
Zohaib Hassnain 9e244ddff6 Fix CLI demo blockers 2026-06-16 04:43:45 +05:00
Sameer6305 b535839003 fix(ingest): harden public API auth validation 2026-06-10 12:47:36 +05:30
luffy2208 4d64c09ad3 fix(ingest): harden public API xml parsing 2026-06-09 14:10:40 +05:30
luffy2208 22382c2cf2 feat(ingest): add public API ingestion support 2026-06-09 06:52:58 +05:30
KaifAhmad1 dbf6ef7b0b fix(cli): resolve JSON spinner leakage and cleanup review findings
- Guard parse_cmd spinner with `fmt == "json"` (default format) to prevent
  Rich status output from polluting machine-readable stdout in piped usage
- Remove unused `Rule` import from cli.py
- Remove unused `_orig_print` variable in verify_rich_cli.py
- Unify semantica.cli import style in verify_rich_cli.py; use cli_mod.main
2026-06-04 16:57:00 +05:30
KaifAhmad1 311a7b43b1 feat(cli): modern Rich terminal styling across all modules
## Summary

Overhaul the CLI and all library modules to produce polished, modern
terminal output comparable to tools like uv, gh, and cargo. Rich was
already a declared dependency but barely used — this commit wires it
throughout every layer.

## Changes by layer

### semantica/cli.py — visual overhaul
- Add imports: `box`, `Panel`, `Rule`, `Syntax`, `Text` from Rich
- Add 7 style constants (`_BRAND`, `_KEY`, `_VAL`, `_DIM`, `_SUCCESS`,
  `_WARN_STY`, `_TABLE_BOX`) for a consistent colour palette
- `_ok()` now prefixes output with a green ✓ checkmark
- New `_info()` helper (neutral · bullet, respects --quiet)
- New `_warn()` helper (yellow ⚠ prefix, never suppressed)
- New `_pprint()` helper: renders dicts/lists as syntax-highlighted JSON
  (Rich Syntax, monokai theme) instead of raw Python repr; strings
  pass through unchanged; respects --quiet
- `info` command: banner replaced with a rounded Rich Panel showing
  version + tagline; component table uses SIMPLE_HEAD box
- All 7 table sites updated: `box=SIMPLE_HEAD`, `show_edge=False`,
  consistent `_KEY`/`_VAL` column styles (KG Stats, Reasoning Engines,
  Recent Decisions, Configured Backends, Backup Info, MCP Tools)
- `_run_build()`: `console.status(spinner="dots")` wraps the blocking
  build call; skipped under --quiet / --json
- `parse`, `extract`, `embed generate`, `reason run`, `reason explain`,
  `deduplicate`: each wraps its long-running operation in a status
  spinner, guarded by --quiet / --json
- All 30+ `console.print(result)` calls replaced with `_pprint()`
- All raw `[yellow]Warning:[/yellow]` and "not running" patterns
  replaced with the new `_warn()` / `_WARN_STY` style

### semantica/explorer/__init__.py
- Error messages use `Console(stderr=True)` with `[bold red]Error:[/bold red]`
- Graph loading wrapped in `console.status()` spinner
- Startup info replaced with a cyan-bordered Rich Panel showing URL,
  API docs, and health endpoint

### Library internals — replace print() with structured logger calls
All modules below had active `print()` calls that bypassed the logging
framework, corrupted spinners, and polluted stdout in piped/programmatic
use. All replaced with appropriate `self.logger.*` calls:

- `semantica/kg/graph_builder.py` — 23 calls: entity resolution
  progress, graph structure steps, GraphStore persistence timing, and
  the two `='*60` completion banners → `self.logger.info/debug()`
- `semantica/semantic_extract/methods.py` — 4 verbose-mode debug
  prints → `logger.debug()`
- `semantica/semantic_extract/relation_extractor.py` — progress +
  error prints → `self.logger.debug/warning()` with `exc_info`
- `semantica/semantic_extract/triplet_extractor.py` — same pattern
- `semantica/semantic_extract/semantic_network_extractor.py` — batch
  error prints → `self.logger.warning/error()`
- `semantica/semantic_extract/coreference_resolver.py` — error print
  → `self.logger.error()`
- `semantica/semantic_extract/providers.py` — debug print →
  `self.logger.debug()`

### Tooling
- `benchmarks/benchmarks_runner.py`: Rule banner, ✓/✗/⚠ status lines,
  Rule separators around regression alert
- `benchmarks/infrastructure/compare.py`: removed manual ANSI escape
  codes; comparison output is now a Rich Table with SIMPLE_HEAD;
  summary uses coloured Rule + styled SUCCESS/FAILURE messages
- `cookbook/advanced/snowflake_ingestion_examples.py`: `_section()`
  helper using Rule; tabular data rendered as Rich Table; result lines
  use ✓/✗/⚠ prefixes; logger.error already present, retained
- `docs_check.py`: `pass`/`FAIL` lines use `[bold green]` /
  `[bold red]`; summary uses styled output

## Tests
- `tests/test_cli_commands.py`: fix 3 pre-existing mock mismatches
  - `test_kg_stats_json_with_mock`: mock now uses `compute_metrics()`
    (the method the code actually calls) instead of `get_statistics()`
  - `test_dry_run_not_needed_extract_is_read_only` and
    `test_stdin_input`: mock now provides `NERExtractor`,
    `RelationExtractor`, `TripletExtractor`, `EventDetector`
    (the classes the code imports) instead of `SemanticAnalyzer`
  Result: 230/230 tests pass (was 227/230)
- `tests/verify_rich_cli.py`: new verification script; exercises all
  14 command groups (92 --help checks, table rendering, dry-run
  formatting, --json mode, _pprint helper); 111 pass, 0 fail
2026-06-04 12:34:12 +05:30
KaifAhmad1 af697a83d8 fix(cli): resolve all review findings from PR #578
P1 — runtime-breaking API mismatches:
- decision record/list/query/trace/similar/impact/check: all six decision
  commands now call decision_methods / decision_query using a GraphStore
  from _get_graph_store(cli_ctx) instead of passing config= kwargs that
  don't exist on the underlying API signatures.
- embed index: load vectors from the Parquet/JSON file into List[np.ndarray]
  before calling create_index(), which expects vectors not a file path string.

P2 — stub implementations replaced with real logic:
- backup sync: now collects local data sources via _collect_backup_sources
  and performs an incremental copy (skips files whose dst mtime >= src mtime).
- backup restore: detects .enc / tar.gz / .tar / directory, decrypts SEM1
  format when --enc, extracts tar archives with leading prefix stripped, or
  copies directory trees back to cwd.

P3 — correctness bugs:
- backup create: archive now includes actual config/ontology/store data files
  via _collect_backup_sources; manifest records the file list.
- extract: --output now works for all formats (table/rdf/yaml), not only JSON.
- backup create: empty keyfile now raises a clear error instead of silently
  producing an unencrypted archive.
- normalize: use Path.is_file() instead of Path.exists() to avoid accidentally
  reading a directory that matches the input text.
- visualize: without --output, emit to stdout; do not silently write kg.html.

Minor:
- _setup_cli_logging: replace opaque _ = (quiet, json_output, exc) tuple
  with del to suppress unused-variable lint.
- reason list: try to source engines from the reasoning module registry;
  fall back to the hardcoded list.
- deduplicate --action report: use method="pairwise" to produce individual
  pair objects with similarity scores, distinct from --action detect.
- tests: remove mixed import (from semantica.cli import main) — all 192
  runner.invoke calls now use cli_module.main as CodeQL flagged.
- tests: add two focused embed-index regression tests that verify vectors
  are loaded from the file before create_index is called.
2026-06-02 19:35:37 +05:30
Zohaib Hassnain f542fc8652 fix(cli): harden startup logging and explorer API wiring 2026-06-02 15:53:10 +05:00
Sameer6305 b22c93e9ec fix(cli): align ingest CLI with unified ingest dispatcher 2026-05-31 20:21:23 +05:30
Sameer KadamandCopilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> 98da904a06 test(cli): remove unused variable in reason list json test
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
2026-05-31 15:59:29 +05:30
Sameer6305 188c81a89c fix(cli): wire deduplicate CLI through graph store and EntityMerger 2026-05-31 15:51:05 +05:30
Sameer6305 c7d6e166ac fix(cli): align export dispatch with registry contract
Fix the export runtime mismatch where get_export_method expected the existing (task, name) registry contract but the CLI passed only the format argument.
2026-05-29 23:45:29 +05:30
KaifAhmad1 ba5038a2e1 feat(cli): implement full Semantica CLI command suite (issue #568)
Expands semantica/cli.py from a 2-command stub into a complete terminal
interface covering every capability described in issue #568, and ships
253 tests covering all new commands, flags, and error paths.

Co-Authored-By: KaifAhmad1 <kaifahmad087@gmail.com>
2026-05-28 14:43:12 +05:30
KaifAhmad1andClaude Sonnet 4.6 8feb8c00c6 fix(cli): address review findings from PR #576
- Remove incorrect # pragma: no cover from _run_with_error_handling
  generic Exception branch (test_runtime_errors_are_click_safe already
  covers it via the monkeypatched RuntimeError path)

- Add _require_ctx() guard: converts None ctx.obj into a clean
  ClickException instead of an AttributeError (protects standalone_mode=False
  / library-use callers); apply to info, kg_build, build_alias commands

- Rename serve group -> services to avoid collision with the future
  `semantica server` flat command specified in issue #568; update docstring
  to document planned subcommand layout

- Fix command-level config logging: re-call setup_logging() with the
  command-level config logging section when -c is used (setup_logging
  clears handlers before adding, so no accumulation risk)

- Fix missing log_level_override in command_ctx: global --log-level was
  silently dropped when a per-command -c config was present, breaking
  the override chain for any nested _build_runtime_config calls

- Add return-shape docstring on _run_build documenting the expected
  build_knowledge_base() return dict structure

- Add type annotation to runner fixture (-> CliRunner) so Pylance
  correctly types runner.invoke() -> Result across all test functions

- Expand test suite: 25 -> 32 tests
  * test_info_command_shows_framework_components
  * test_info_command_shows_config_path_when_supplied
  * test_log_level_global_override_stores_in_context
  * test_command_config_preserves_global_log_level_override
  * test_build_result_with_stats_shows_source_count
  * test_build_result_without_stats_shows_generic_success
  * test_build_result_none_shows_generic_success
  * test_require_ctx_raises_click_exception_on_none
  * test_require_ctx_returns_ctx_unchanged

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 15:29:39 +05:30
Sameer6305 c447bf5934 cli: harden config parsing and isolate CLI test logging 2026-05-27 13:57:18 +05:30
Sameer6305 b54d885bf2 cli: harden config parsing and logging override handling
- keep command-level config from overriding logging unless --log-level is set

- validate YAML/JSON config roots and surface parse failures as Click errors

- tighten CLI tests around isolation and cleanup
2026-05-26 23:06:00 +05:30
Sameer6305 bc9db1ff89 cli: add foundation wiring and kg build with legacy build parity
- add CLI runtime context, global config/log-level handling, and click-safe error wrapping

- implement kg build as a thin wrapper over existing orchestrator build flow

- keep hidden legacy build alias and route both build handlers through shared internal path

- add focused CLI tests for help UX, config flag compatibility, alias parity, and clean error output

- keep tests lightweight by mocking heavy build execution paths
2026-05-26 22:45:50 +05:30
KaifAhmad1 453eeb7ca9 fix: rename contributing/license pages to avoid Mintlify reserved slug conflict
mint export fails with 'file does not exist' for pages named 'contributing'
and 'license' — these are reserved by Mintlify's GitHub integration layer.
Renamed to contributing-guide.md and project-license.md and updated all
nav entries and cross-links throughout the docs.

Also adds .gitattributes LF rules to prevent CRLF issues from Windows devs.
2026-05-23 00:14:23 +05:30
Luffy2208andKaifAhmad1 98232749fb Add XML file ingestion support (#560)
* Add XML file ingestion support

* fix(xml-ingestor): add ingest_string test and document ingest() return keys

- Add test_xml_ingestor_ingests_string to cover the public ingest_string()
  method which had no test coverage
- Document all source_type return keys in the ingest() docstring so callers
  know to use result["xml"] rather than result["data"] for XML sources

* docs(changelog): add unreleased entry for XML ingestion support (#560)

---------

Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
2026-05-19 17:48:32 +05:30
KaifAhmad1 722ae06795 fix(providers): address review feedback on PR #556 + changelog
Four issues raised in code review:

- Mode.JSON retry now strips response_format from create_kwargs before
  calling json_client.chat.completions.create, preventing incompatible
  kwargs from being forwarded to a client configured for a different mode.

- Add exc_info=True to the generate_structured fallback warning in the
  manual repair loop so the gateway rejection traceback is visible in
  production logs, consistent with the other warnings added in this PR.

- Remove the duplicate is_available definition in GroqProvider. Python
  silently kept only the second definition; the first (with diagnostic
  branching) was dead code and could cause confusion on future edits.

- Validate base_url scheme in OpenAIProvider._init_client. Non-HTTP(S)
  schemes (file://, ftp://, javascript:, etc.) are now rejected with a
  ValueError at init time, preventing SSRF if base_url originates from
  configuration rather than hardcoded values.

Add 3 new tests: SSRF scheme rejection, valid-URL acceptance, and
exc_info presence on the generate_structured fallback warning (20/20 pass).

Update CHANGELOG.md with full description of all fixes under [Unreleased].
2026-05-15 20:00:44 +05:30
KaifAhmad1 ca5f42baf8 fix(ner): resolve silent pattern fallback when LLM method fails on custom gateways (#554)
Three bugs caused NERExtractor to silently return pattern-based entities
even when method="llm" was configured:

1. exc_info=True missing on method-failure warning in NERExtractor —
   the root exception was swallowed, making the gateway error invisible
   in logs even with DEBUG enabled.

2. OpenAIProvider.generate_structured always sent response_format=json_object
   to the API. Custom/enterprise gateways (Qwen, LLaMA proxies, internal
   gateways) often reject this parameter, causing both the instructor path
   and the manual repair loop to fail with the same error on every retry.

3. generate_typed manual repair loop had no fallback when generate_structured
   itself raised — it retried the same failing call up to max_retries times,
   then propagated the error, triggering _extract_fallback (pattern extraction).

Fixes:
- Add exc_info=True to the method-failure warning so the full traceback
  appears in logs and users can diagnose the root cause.
- Skip response_format=json_object in OpenAIProvider.generate_structured
  when base_url is set (custom endpoint), since standard OpenAI gateways
  don't require it and third-party ones reject it.
- In the generate_typed manual repair loop, catch generate_structured
  failures and immediately retry via plain generate() + _parse_json,
  breaking the retry-the-same-failing-call loop for custom gateways.

Also adds 17 targeted regression tests covering all three bug paths,
including the exact gateway configuration reported in the issue.
2026-05-15 19:27:45 +05:30
Luffy2208andKaifAhmad1 15d58f2b88 Added Parquet ingest support (#234) (#548)
* Added Parquet ingest support (#234)

* docs: Add Parquet ingestion support to CHANGELOG

- Add comprehensive changelog entry for PR #548
- Document ParquetIngestor class and key features
- Include author credit (@Luffy2208) and PR reference
- Follow existing changelog format and structure

---------

Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
2026-05-10 12:52:26 +05:30
Zohaib Hassnain ac5015bc3f fix(deduplication): normalize merged group keys 2026-05-05 20:25:51 +05:00
KaifAhmad1 21c2f190f8 fix: resolve Qodo review bugs and quality issues (DuplicateDetector + ConflictDetector)
- bug_001: top_k_per_entity now uses OR semantics — keep a candidate if
  EITHER entity is under quota, preventing high-quality candidates being
  silently dropped when a popular counterpart saturates its quota
- bug_002: validate max_results and top_k_per_entity at construction;
  negative or non-int values raise ValueError instead of silent empty output
- bug_003: validate min_similarity in [0.0, 1.0] at construction;
  out-of-range values raise ValueError
- bug_004: harden ConflictDetector method='relationship' normalization —
  always produces List[Dict] before calling detect_relationship_conflicts
- quality_001: update detect_duplicates + incremental_detect docstrings to
  reflect configurable sort_by field (not hardcoded 'confidence')
- quality_002: add _normalize_entity_id helper (always str) used in both
  _apply_result_limits and _build_duplicate_groups for consistent ID handling

Backward compatible: callers not using new params see no behavior change.
58 tests pass (0 failures)
2026-05-05 19:25:48 +05:30
KaifAhmad1 8ef67b8bda feat(deduplication): add max_results, top_k_per_entity, min_similarity, sort_by to DuplicateDetector
Fixes #534

- New __init__ params: max_results, top_k_per_entity, min_similarity, sort_by
- _apply_result_limits: drop below min_similarity, sort by sort_by field,
  enforce top_k_per_entity per entity, cap at max_results globally
- Wired into detect_duplicates() and incremental_detect()
- 30 new tests in TestResultLimiting; full suite 42/42 passed
2026-05-05 19:16:58 +05:30
KaifAhmad1 a01b3c36fc fix(utils): route all progress tracker stdout writes through _safe_write to prevent UnicodeEncodeError on cp1252 consoles
Closes #531

- Replace 5 direct sys.stdout.write() calls in ConsoleProgressDisplay.update()
  with self._safe_write() so emoji/block characters are encoded safely on
  Windows cp1252 consoles
- Add TestProgressTrackerEncoding regression tests (3 cases) covering
  _safe_write, pipeline header, and auto emoji-disable on cp1252
2026-05-05 16:03:19 +05:30
KaifAhmad1andZohaib Hassan e7c9f6e7f3 fix(tests): restore sys.modules after mock injection in test_retry_logic
test_retry_logic.py injected sys.modules["openai"] = MagicMock() at module
level so providers.py could be imported without the real openai package.
Those mocks were never restored, leaving openai (and spacy, instructor etc.)
as MagicMock objects for the entire test session. This caused
test_pr482_deepseek_openai tests to receive a MagicMock when importing
openai.OpenAI, making MagicMock(spec=OpenAI) raise InvalidSpecError.

Fix: save original sys.modules entries before injection and restore them
immediately after the semantica imports that needed the mocks complete.
The mock objects remain bound inside the already-imported provider module,
so test_retry_logic tests are unaffected; other test modules now see the
real packages again.

Co-authored-by: Zohaib Hassan <zohaib179949@gmail.com>
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
2026-05-05 14:21:36 +05:30
KaifAhmad1andZohaib Hassan b828ebfef0 chore: resolve CHANGELOG.md merge conflict with main
main restructured [Unreleased] into ### Added / ### Fixed sections.
Moved PR #536 semantic_extract circular import fix entry into ### Fixed
below the PR #535 ingest lazy-load entry; kept ### Added content from
main intact.

Co-authored-by: Zohaib Hassan <zohaib179949@gmail.com>
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
2026-05-05 13:56:48 +05:30
KaifAhmad1andZohaib Hassan 24e327d161 fix(tests): add from __future__ import annotations for Py3.8 compatibility
subprocess.CompletedProcess[str] as a return annotation is not subscriptable
at runtime on Python 3.8, causing test collection to abort before any tests
run. Adding PEP 563 deferred evaluation makes all annotations strings at
import time, restoring 3.8 compatibility without changing behaviour on 3.9+.

Co-authored-by: Zohaib Hassan <zohaib179949@gmail.com>
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
2026-05-05 13:40:11 +05:30
a2b6a481dc chore: resolve CHANGELOG.md merge conflict with main
main restructured [Unreleased] into ### Added / ### Fixed sections.
Moved PR #535 lazy-load fix and Ontology Hub post-review fix entries
into ### Fixed; kept ### Added content from main intact.

Co-authored-by: ZohaibHassan16 <zohaib@hawksight.ai>
Co-authored-by: KaifAhmad1 <mohammadk78600@gmail.com>
2026-05-05 12:56:06 +05:30
c7edba88ea fix(tests): set exc.name on blocker's ModuleNotFoundError to match Python import machinery
OptionalDependencyBlocker was constructing ModuleNotFoundError with only a
message string, leaving .name as None. _is_missing_dependency checks exc.name
directly (the string-scan fallback was removed when switching to
ModuleNotFoundError), so the ConfigurationError conversion never triggered and
the test asserted the wrong exception type.

Python's import machinery always sets .name to the top-level module name when
it raises ModuleNotFoundError; the blocker now does the same.

Co-authored-by: ZohaibHassan16 <zohaib@hawksight.ai>
Co-authored-by: KaifAhmad1 <mohammadk78600@gmail.com>
2026-05-05 12:45:43 +05:30
Zohaib Hassnain e1b1e63541 fix(semantic_extract): break extractor import cycle 2026-05-05 02:26:57 +05:00
Zohaib Hassnain 6b0a8e60ce fix(ingest): lazy-load optional ingestion backends 2026-05-05 02:01:27 +05:00
63acc7a66e fix(ontology): address Qodo automated review findings from PR #524
Backend (semantica/explorer/routes/ontology.py):
- suggest-alignments: add TF-IDF character-ngram embeddings via sklearn
  (SimilarityCalculator-compatible cosine scoring) so embedding_similarity
  is populated in results; combined score = 0.4*label + 0.6*embedding when
  available, falling back to label-only when sklearn is absent
- suggest-alignments: add token-overlap prefilter before SequenceMatcher so
  zero-Jaccard pairs are skipped without computing full similarity; add
  _MAX_ENTITIES_PER_SIDE=500 per-ontology cap on top of the existing
  _MAX_ANALYSIS_NODES global cap
- suggest-alignments: remove dead try/except OntologyEngine.create_alignment
  block that always failed silently (no TripletStore configured); replace
  with a comment explaining the intentional ephemeral-only storage model
- health: replace O(alignments x entities) any() scans for alignment coverage
  with O(1) set membership checks via assessed_ids
- shacl/validate: run rdflib.Graph().parse(format='turtle') syntax check on
  the submitted Turtle before returning; invalid syntax now raises 422 instead
  of returning a misleading unavailable/success response

Frontend:
- AlignmentsTab: add pairwise alignment matrix section that groups recorded
  alignments by (source_ontology, target_ontology) pair; each cell shows
  color-coded relation badges per RELATION_COLORS; clicking a badge populates
  the create/edit form for quick editing; matrix is shown when at least two
  ontologies are loaded
- ShaclStudio: add selectedShapeId state and fullShacl ref; each shape row in
  the library is now a clickable button that extracts its Turtle block from
  the full SHACL and pre-populates the Monaco editor; a "View all" toggle
  restores the full SHACL; selected shape ID is shown in the editor header
- GraphWorkspace: fix viewMode race in external focus effect — call
  setSelectedNodeId directly instead of going through focusNode(), which
  captured a stale viewMode in its closure; remove focusNode from the
  dependency array since it is no longer called

Tests (14 passing, was 11):
- Add test_suggest_alignments_returns_embedding_similarity: asserts
  embedding_similarity is non-null when sklearn is available
- Add test_shacl_validate_rejects_invalid_turtle_syntax: asserts 422 on
  syntactically invalid Turtle
- Add test_health_alignment_coverage_uses_set_lookup: asserts alignment
  dimension score is non-zero after recording an alignment, verifying the
  O(1) set lookup path works correctly end-to-end

Co-authored-by: KaifAhmad1 <mohammadk78600@gmail.com>
Co-authored-by: ZohaibHassan16 <zohaib@hawksight.ai>
2026-05-02 16:38:09 +05:30
00ceb09960 fix(ontology): address review blockers from PR #524
Backend:
- Replace false conforms=True SHACL stub with status=unavailable always;
  live validation cannot be wired until OntologyEngine.validate_graph is
  connected to a data graph — a stub that returns conforms=True misleads
  users editing shapes
- Cap node/edge fetches in health, suggest-alignments, and SHACL generation
  at _MAX_ANALYSIS_NODES (5 000) with a logger.warning when the graph
  exceeds the limit; unbounded limit=999_999 fetches cause OOM on large graphs
- Set SHACL health dimension score to 0.0 (was 70.0) when status=unavailable;
  exclude unavailable dimensions from the total_score average so they neither
  inflate nor deflate the result
- Allow alignments to reference external/unloaded URIs (e.g. schema.org)
  without raising 404; label falls back to URI fragment or caller-supplied
  source_label/target_label fields added to OntologyAlignmentRequest
- Fix _alignment_id to use uuid.NAMESPACE_OID instead of NAMESPACE_URL;
  the composite key is not a URL
- Fix _summarize_shapes to normalise \r\n before splitting on .\n so shape
  parsing works correctly on Windows line endings

Frontend:
- Wrap handleSave/handleSuggest/handleRemove/handleAcceptSuggestion in
  useCallback in AlignmentsTab for consistency with sibling components
- Add ephemeral-storage banner in AlignmentsTab warning that alignments are
  session-memory-only and not persisted across restarts
- Fix exportReport in HealthTab to append/remove anchor from document before
  clicking and defer URL.revokeObjectURL to avoid Blob URL leak in some browsers
- Derive health dimension grid column count from health.dimensions.length
  instead of the hardcoded repeat(5, ...) that breaks if the backend adds
  or removes a dimension
- Add minimal Monarch tokenizer for the Monaco turtle language registration
  in ShaclStudio so prefix declarations, IRIs, SHACL properties, comments,
  and string literals are syntax-highlighted; previously the editor rendered
  as plain text despite theme rules being defined

Tests (11 passing, was 5):
- Rename test_shacl_validate_has_stable_contract to
  test_shacl_validate_returns_unavailable and assert status == unavailable
- Add test_shacl_validate_rejects_empty_turtle (expects 422)
- Add test_health_returns_404_for_unknown_ontology
- Add test_health_shacl_dimension_is_zero_when_unavailable with total_score check
- Add test_delete_unknown_alignment_returns_404
- Add test_alignment_upsert_is_idempotent (verifies ID stability and created_at
  preservation across updates)
- Add test_alignment_accepts_external_uri (verifies no 404 for schema.org URIs)
- Relax test_alignment_suggestions_are_ranked label assertions to substring
  checks so the test survives similarity algorithm changes

Co-authored-by: KaifAhmad1 <mohammadk78600@gmail.com>
Co-authored-by: ZohaibHassan16 <zohaib@hawksight.ai>
2026-05-02 15:53:47 +05:30
Zohaib Hassnain e8bf0e50d3 feat(ontology): add alignments health and shacl studio 2026-05-01 22:59:15 +05:00
bb956b2735 fix(explorer): address PR #515 review findings
- Align _coerce_embedding_vector inner dict-probe key list with
  _extract_node_embeddings outer key list (add 'embeddings', reorder to
  generic-first) so nested embedding dicts resolve consistently.
- Add TODO comment on _extract_node_embeddings to cache per-session
  graph revision and avoid O(N) re-scan on every semantic request.
- Add deprecation docstrings to legacy path-segment routes
  (/node/{id}/path and /node/{id}/semantic-neighborhood) documenting
  the known slash-in-ID limitation and pointing to the query-param
  alternatives.
- Extract _FakeSimilarity to module level so it is shared without
  duplication across test classes.
- Rewrite test_legacy_semantic_neighborhood_still_works_for_simple_ids
  as a fully isolated TestClient session instead of mutating the
  shared module-scoped 'client' fixture, preventing cross-test
  state pollution.
- Extract _make_slash_node_session helper to reduce boilerplate in the
  slash-safe route tests.

Co-Authored-By: ZohaibHassan16 <109234410+ZohaibHassan16@users.noreply.github.com>
Co-Authored-By: KaifAhmad1 <98801504+KaifAhmad1@users.noreply.github.com>
2026-04-29 23:17:57 +05:30
Zohaib Hassnain e3a3f6010b fix(explorer): make distance intelligence API calls slash-safe 2026-04-29 21:17:36 +05:00
Mohd KaifandCopilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> 6ffed78fd9 Potential fix for pull request finding 'Module is imported with 'import' and 'import from''
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
2026-04-27 18:39:09 +05:30
KaifAhmad1 f06de0dab2 fix(context): address PR #512 review blockers and bot findings
Merge blockers (ZohaibHassan16):
- fix: distance-matrix raises HTTP 503 when metric=semantic but no
  similarity backend is available, instead of silently returning hop
  distances labeled as semantic
- fix: distance-enriched export now requires node_subset (HTTP 422 if
  omitted), preventing unbounded all-pairs O(n^2) export over full graph
- fix: DistanceExportRequest default include corrected from ["hops",
  "distance_band"] to ["source_id", "target_id", "hop_count",
  "distance_band"] so default exports are unambiguous and use the correct
  column name
- fix: confidence decay edge weight index now reads graph_dict.get("edges")
  or graph_dict.get("relationships") to handle both graph dict shapes,
  fixing always-1.0 decay when session returns relationships key

Bot findings (github-code-quality / chatgpt-codex):
- fix: remove unused Iterable import from distance_exporter.py
- fix: remove unused Response import from graph.py
- fix: move logger init before optional KG import; replace empty
  except ImportError: pass with logger.debug in distance_exporter.py
- fix: replace two bare except Exception: pass in temporal.distance_history
  with logger.warning including source, target, metric, and timestamp context
- fix: remove mixed import style in test_qual003 — use only module import
  and reference CausalChainAnalyzer through it
2026-04-27 18:28:04 +05:30
KaifAhmad1 dd016744ce feat(context): add distance intelligence across context, API, and Explorer (#502)
- ContextGraph.get_neighbors() gains include_distance_metadata flag (backward-compat)
- get_neighbor_distances() returns neighbors sorted by hop and confidence decay
- AgentContext.retrieve/find_precedents support proximity-weighted blending
- FR-4: path enrichment (decay, similarity, coherence, bottleneck, interpretation)
- FR-6: POST /api/graph/distance-matrix (hops/weighted/semantic, upper-triangle)
- FR-3: GET /api/graph/node/{id}/semantic-neighborhood
- FR-8: GET /api/decisions/causal-distance (causal-edge-only BFS)
- FR-9: GET /api/temporal/distance-history (convergence/divergence events)
- FR-10: POST /api/export/distance-enriched (CSV/JSONL, 200-node cap)
- Explorer: PathDistanceIntelPanel, Ego Mode, Structural/Semantic overlay, Heatmap
- Fix 13 Qodo review issues: API param mismatch, O(E*L) decay, breaking change,
  schema key inconsistency, datetime arithmetic, id overwrite, sweep race,
  node_subset DoS, full-matrix redundancy, effect race, silent exceptions, duplication
- 57 new tests in test_distance_intelligence.py; 18 regression tests in _smoke_review_fixes.py
2026-04-27 11:07:27 +05:30
Mohd Kaif f6198039fa Merge branch 'main' into main 2026-04-19 20:10:22 +05:30
983f5301e8 fix(providers): switch DeepSeekProvider to OpenAI SDK + fix base_url and verbose_mode (closes #482)
- Replace deepseek.Client with openai.OpenAI(base_url="https://api.deepseek.com/v1")
  in DeepSeekProvider._init_client(); the deepseek PyPI package has no Client class
- Add self.base_url = "https://api.deepseek.com/v1" to DeepSeekProvider.__init__()
  (missing from original PR; caused AttributeError on every instantiation)
- Fix verbose_mode NameError in BaseProvider.generate_typed() instructor path
- Update pyproject.toml: llm-deepseek extra now declares openai>=1.0.0
- Update _init_client warning message to reference openai library
- Add 19 tests in tests/semantic_extract/test_pr482_deepseek_openai.py
- Update CHANGELOG.md

Co-authored-by: liling <liling@users.noreply.github.com>
Co-authored-by: KaifAhmad1 <mohammadk78600@gmail.com>
2026-04-19 20:07:44 +05:30
KaifAhmad1 fe6ca7fccb fix(search-index): restore secondary-scan node ordering and add regression test 2026-04-19 18:46:05 +05:30
Mohd Kaif 66c8431eee Merge branch 'main' into feat/optimize-search 2026-04-19 18:25:28 +05:30
Zohaib Hassnain 6f93f429c4 perf(explorer): add indexed search for large graphs 2026-04-17 21:22:12 +05:00
Sameer6305 658de23357 fix: resolve merge conflicts with upstream main 2026-04-17 19:56:41 +05:30
Sameer6305 66e8964d22 fix(provenance): include upstream ancestors + add direction classification and markdown grouping 2026-04-17 19:33:12 +05:30