* feat: implement Apache Arrow and Feather file ingestion support (#235)
* fix(arrow): eliminate double full-scan and clean up reader wrapper
- Replace _read_batches with _read_batches_with_info which collects
batch metadata (total_rows, record_batches) during the same pass as
the data read, so ingest_file no longer calls _file_metadata before
_read_batches. For a limit=1 read on a large file this previously
scanned every batch twice; now it stops after the first batch.
- _file_metadata is now only invoked for include_data=False (where a
full scan is unavoidable to report accurate row counts).
- Remove the dead num_record_batches property from _ArrowReaderWrapper;
it was never called by production code and its is_table branch
materialised all batches just to count them.
- Fix _open_file exception chain: raise ... from file_err instead of
from feather_err so the most diagnostic IPC error appears in the
Python traceback chain, not the least informative fallback error.
* docs(changelog): add [Unreleased] entries for Arrow ingestion (#705)
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
* fix(explorer): resolve blank dashboard UI and ship frontend bundle in wheel
Fixes#631 — the Explorer server started successfully but the browser showed
a blank page because semantica/static/ was gitignored and never present after
a fresh install or clone.
Changes:
- ci.yml / release.yml: add Node 20 setup + npm ci && npm run build before
python -m build so every wheel contains a CI-built frontend bundle
- pyproject.toml: add package-data patterns (static/*, static/assets/*) so
setuptools includes the bundle in the wheel; add MANIFEST.in for sdist coverage
- app.py: replace silent empty-HTML fallback with a 200 page that clearly
explains the missing bundle and links to /docs; fix CORS allow_credentials
to default false, gated behind EXPLORER_CORS_CREDENTIALS env var to prevent
credentialed cross-origin requests on unauthenticated endpoints
- __init__.py: warn at startup when --host is non-loopback (unauthenticated
network exposure)
- explorer/README.md: full rewrite covering pip-install mode (primary path,
no Node required) and dev-server mode (contributors), CLI flags, env vars,
workspace table, troubleshooting for the blank-page symptom
- README.md: update Knowledge Explorer section with correct command and link
to the new setup guide
* fix(explorer): set build.target esnext to fix esbuild CI failure
esbuild >=0.28 (forced via npm overrides) conflicts with Vite 6 defaults on
Linux CI — it tries to lower destructuring syntax for the implicit browser
target list but errors out. Explicit target: 'esnext' tells esbuild to emit
native syntax unchanged, bypassing the transpilation error entirely. Safe for
a developer tool that runs in modern browsers.
* test(explorer): verify packaged frontend bundle
---------
Co-authored-by: Zohaib Hassnain <109234410+ZohaibHassan16@users.noreply.github.com>
In TOML, declaring [project.urls] inside the [project] block causes all
subsequent key-value pairs (including dependencies = [...]) to be parsed
as project.urls.* keys, producing:
ValueError: invalid pyproject.toml config: project.urls.dependencies
must be string
Fix: move [project.urls] to after the dependencies array closes and before
[project.optional-dependencies], which is the correct TOML position for a
sub-table of [project].
- description: rewritten to lead with the accountability/provenance
angle and name concrete capabilities; drops emoji which render
inconsistently across PyPI clients
- keywords: expanded from 8 to 23 terms covering modern search queries
(ai-agents, llm, graph-rag, decision-intelligence, provenance, etc.)
- classifiers: added Information Analysis, Text Processing::Linguistic,
Database Engines/Servers, Information Technology audience
- [project.urls]: new section with Homepage, Documentation, Repository,
Changelog, Bug Tracker, Discord — shown prominently on the PyPI page
and drive clicks to GitHub/docs
- optional-dependencies: added watch = [watchdog>=3.0.0]; bundled into all
* Added Parquet ingest support (#234)
* docs: Add Parquet ingestion support to CHANGELOG
- Add comprehensive changelog entry for PR #548
- Document ParquetIngestor class and key features
- Include author credit (@Luffy2208) and PR reference
- Follow existing changelog format and structure
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
- Convert mcp_server.py to package structure (semantica/mcp_server/)
- Add __init__.py and __main__.py for python -m support
- Add semantica-mcp console script entry point in pyproject.toml
- Fix API method calls (extract -> extract_entities/relations/triplets)
- Remove non-existent _result_cache imports
- Update documentation with both usage methods
Resolves pipx installation issue where semantica.mcp_server was not available.
Provides two ways to run: 'semantica-mcp' command or 'python -m semantica.mcp_server'.
* fix(deps): remove gpu extra from [all] to fix Windows installation failure
faiss-gpu has no Windows builds, so semantica[all] failed with
'No matching distribution found for faiss-gpu>=1.7.0' on Windows.
Removed gpu from both [all] lines — semantica[gpu] remains available
as an explicit opt-in for Linux GPU environments.
Closes#532
* docs(changelog): record faiss-gpu Windows installation failure fix (#532)
- Replace deepseek.Client with openai.OpenAI(base_url="https://api.deepseek.com/v1")
in DeepSeekProvider._init_client(); the deepseek PyPI package has no Client class
- Add self.base_url = "https://api.deepseek.com/v1" to DeepSeekProvider.__init__()
(missing from original PR; caused AttributeError on every instantiation)
- Fix verbose_mode NameError in BaseProvider.generate_typed() instructor path
- Update pyproject.toml: llm-deepseek extra now declares openai>=1.0.0
- Update _init_client warning message to reference openai library
- Add 19 tests in tests/semantic_extract/test_pr482_deepseek_openai.py
- Update CHANGELOG.md
Co-authored-by: liling <liling@users.noreply.github.com>
Co-authored-by: KaifAhmad1 <mohammadk78600@gmail.com>
Bump version to 0.4.0, move [Unreleased] changelog entries to [0.4.0]
(2026-04-08), and remove duplicate changelog content appended in prior
merges. Release covers temporal data model, SHACL, SKOS, Knowledge
Explorer API, Agno integration, Named Graphs, Datalog Reasoner, and
many more features landed since 0.3.0.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bugs fixed:
- enrich.py: predict_links called predictor.predict_links() with wrong
signature (graph_dict as graph_store, node_id as node_labels, top_n
instead of top_k). Rewrote to iterate candidate nodes and call
score_link(session.graph, src, candidate) directly.
- enrich.py: detect_duplicates called session.get_nodes() synchronously
in an async handler, blocking the event loop. Wrapped in to_thread().
- export_import.py: temp file was leaked on export exception. Now always
cleaned up via try/finally. Moved `import os` to module level.
- pyproject.toml: missing comma between two strings in the `all` extra
caused a TOML syntax error breaking `pip install semantica[all]`.
- app.py: generic Exception handler swallowed HTTPException(503) raised
by get_session dependency. Now re-raises HTTPException explicitly.
- decisions.py: compliance endpoint imported PolicyEngine then discarded
it, always returning compliant=True. Replaced with in-graph check:
scans for violates/non_compliant/breaches edges from the decision node.
- app.py: removed unused `import traceback`.
Refactor:
- session.py: added build_graph_dict(node_ids=None) method to eliminate
_build_graph_dict() duplication across graph.py, analytics.py, and
export_import.py (three identical copies).
- session.py: all 8 lazy analytics properties now initialise under _lock
to prevent double-instantiation under concurrent requests.
- graph.py: find_path now dispatches to dijkstra_shortest_path or
bfs_shortest_path based on the `algorithm` query param (was always BFS).
- annotations.py: removed unnecessary get_annotations() round-trip in
create_annotation — add_annotation mutates ann_data in-place.
- temporal.py: split bare `except Exception` into ImportError (silent)
and Exception (logs warning), so real bugs are no longer hidden.
Tests (49 total, all passing):
- Added TestEnrichExtract, TestLinkPrediction, TestDedup classes.
- Added test_compliance_with_violation to verify real violation detection.
- Added test_snapshot_at_excludes_temporal_node, test_diff assertions,
test_export_json_subset, test_import_with_edges, test_import_unsupported_format.
- Strengthened analytics, search, and annotation assertions.
- Reasoning test now asserts response shape when status is 200.
Co-authored-by: ZohaibHassan16 <zohaibhassan16@users.noreply.github.com>
Co-authored-by: KaifAhmad1 <kaifahmad1@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Package & distribution
- pyproject.toml: add integrations* to packages.find include so pip
install semantica[agno] ships the integration
context_store.py
- upsert_memory(): run NERExtractor after store() to index entities
into the ContextGraph
- delete_memory() / drop_table() / clear(): call AgentContext.forget()
to propagate deletions to vector/graph storage
- find_precedents(): pass limit parameter to find_precedents_advanced()
- retrieve(): pass limit as max_results to AgentContext.retrieve()
- add get_context_for_prompt() for automatic system-prompt injection
knowledge_graph.py
- __init__: wire graph_builder.graph_store = self._graph so build()
persists into the ContextGraph
- add internal AgentContext for vector retrieval (shared ContextGraph)
- search(): use AgentContext.retrieve() for vector similarity; keyword
scoring as fallback
- _ingest_text(): add paragraph-level chunking before NER/relation
extraction (parse → split → NER → relation extract → graph build)
- get_graph_context(): return structured subgraph with edge types via
ContextGraph.get_neighbors()
- load_urls(): validate scheme (http/https only) to prevent SSRF
decision_kit.py
- check_policy(): replace broken PolicyEngine.check_compliance() call
with inline _eval_rule() that evaluates simple field-op-value rules;
return compliant=False (not True) on failure — closes security bug
kg_toolkit.py
- add_to_graph(): fix add_node(node_id=, node_type=) and
add_edge(source_id=, target_id=, edge_type=) to match real API
- query_graph(): use find_nodes() (no label param) + keyword filter
- find_related(): use get_neighbors(node_id=) returning List[Dict]
- infer_facts() / export_subgraph(): use find_nodes() public API
instead of private _nodes dict
shared_context.py
- _AgentScopedStore: store shared context as self._context (not
self._ctx) so all inherited AgnoContextStore methods work correctly
tests/integrations/agno/test_kg_toolkit.py
- _FakeGraph: rewrite to match real ContextGraph signatures —
find_nodes(node_type=), add_node(node_id, node_type, **),
add_edge(source_id, target_id, edge_type, **),
get_neighbors(node_id, hops=1, ...) returning List[Dict]
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mohd Kaif