Commit Graph
434 Commits
Author SHA1 Message Date
taljeon afec253451 docs(langchain): tighten integration prose 2026-09-03 21:58:36 +09:00
Zohaib Hassnain ba85215aea docs(graphrag): fix broken string literals and clarify max_hops (#1431)
The Banking domain example built basel_cre20_text / bcbs239_text with bare indented string continuations (no parens, no backslash), raising IndentationError. Wrapped both in parentheses like the Clinical example. Separately, the guide passed max_hops= to retrieve() and stated it overrides the constructor's expansion depth: it does not. AgentContext.retrieve(max_hops=) is only consumed by _apply_proximity_metadata (a proximity-radius filter that needs anchor_node), and expansion depth is fixed by max_expansion_hops passed into ContextRetriever. Removed max_hops from the non-anchored retrieve call, annotated the anchored ones, corrected the intro and tuning sections, and noted query_with_reasoning() does take a real per-call max_hops. Also replaced an invented node/edge count comment with the real store() return keys and qualified an ingest_file() reference.
2026-09-03 17:48:07 +05:00
Mohd Kaif 5809418421 docs: tighten prose in concepts.md, guides/graphrag.md, reference/context.md (#1422)
Flagship pass establishing the crisp-prose style for the rest of
docs/: remove em dashes from explanatory prose (leave them in
simulated document/alert string literals, which are data, not our
voice), replace colon-as-dramatic-pause constructions, and fix two
broken relative links in reference/context.md ([Reasoning](reasoning)
and [Provenance](provenance) were missing their leading slash and
would 404 on the live site, the same class of bug fixed sitewide in
PR #1407). concepts.md's intro also picks up the new context/semantic
layer tagline. No code examples, tables, or technical content
changed.
2026-09-03 17:20:04 +05:30
Mohd Kaif 40efab6796 docs(index): cut marketing copy, remove em dashes, make crisp (#1421)
* docs(index): cut marketing copy, remove em dashes, make crisp

Replace the narrative hook and rhetorical-question opening with a
direct statement. Trim the persuasive framing on the problem list
and industry section to plain, factual bullets. Replace every em
dash with plain sentence structure or a colon, and drop the
repeated colon-as-dramatic-pause construction from the opening.
No content or links removed; only the framing and punctuation
changed.

* docs: update tagline to context/semantic layer for high-stakes domains

Replace "The Accountability and Context Layer for AI" with "The
Context and Semantic Layer for AI in High-Stakes Domains" across
docs.json (description, og:title) and index.md (frontmatter
description, opening sentence, and the Core Concepts step bullet).
Audit trail and accountability remain a downstream property, not
the headline framing.
2026-09-03 17:00:03 +05:30
Mohd Kaif 837654fc4f docs: restructure nav — drop FAQ/Changelog tabs, add API Reference tab (#1419)
Remove the standalone FAQ and Changelog top-level tabs. FAQ and
Community pages move into the Overview tab as their own groups
(still fully reachable, just relocated). Changelog was only an
external link to GitHub releases and had no pages of its own.

Split the API reference pages (reference/*) out of the Modules tab
into a new, dedicated API Reference tab, so Modules now holds only
the conceptual guides and API Reference holds every module's class
and function documentation.
2026-09-03 16:19:43 +05:30
Mohd Kaif 2daa937811 docs: simplify custom.css to a static, professional style (#1418)
Remove decorative hover animations (code block/card lift+glow, table
row highlighting, list item highlighting, animated nav underline,
button lift+glow) and the page-load fade-in transition. Keep the
color/typography branding, accessibility focus rings, and scrollbar
styling.
2026-09-03 15:59:36 +05:30
Zohaib Hassnain b872b29628 docs(concepts): rewrite code examples to match the actual API (#1417)
* docs(concepts): rewrite every code example against real API

* add Qodo review
2026-09-03 15:02:27 +05:00
Zohaib Hassnain 9e8db764d1 docs(quickstart): read parsed full_text (#1415)
* docs(quickstart): read parsed full_text

* correct schema
2026-09-03 14:40:58 +05:00
Zohaib Hassnain 865aad54df docs(getting-started): fix broken APIs in the Knowledge Graph and GraphRAG tabs (#1414)
* docs(getting-started): fix broken APIs in KG and GraphRAG tabs

* docs: tighten GraphRAG example

* docs: use extract_text() so the PDF example doesn't keyError
2026-09-03 14:33:23 +05:00
Zohaib HassnainandSameer Kadam 2d776b7370 docs(evals): update docs for the current evals API (#1398)
* document evals API

* docs(evals): fix evaluator behavior details

---------

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
2026-09-03 14:12:49 +05:00
Duansg a59688c6f9 additional fixes 2026-09-02 20:37:40 -07:00
Duansg 40466269b8 docs: convert relative page links to root paths to fix 404s on the live site 2026-09-02 20:20:55 -07:00
Zohaib Hassnain 38ae5b580b docs: fix two broken cookbook notebook links (#1403)
* docs: fix two dead notebook links

* docs(learning-more): describe the embeddings notebooks
2026-09-03 04:21:21 +05:00
Zohaib Hassnain 279fdbf15b docs(quickstart): qodo findings addressed (#1402) 2026-09-03 04:18:17 +05:00
Zohaib Hassnain b7b60d4a17 docs(quickstart): fix broken code against real APIs (#1401) 2026-09-03 04:05:19 +05:00
Zohaib Hassnain 25d2ea5fe9 docs: update stale latest version claims 2026-09-03 03:50:50 +05:00
Zohaib Hassnain 3c68cd12ad docs(mcp): correct tool count (#1399) 2026-09-03 03:40:52 +05:00
KevinandSameer Kadam 30a91a3a78 feat(evals): add per-metric objective support to runner (closes #1091) (#1092)
* chore: ignore .worktrees directory

* feat(evals): add eval metric and result models

* feat(evals): add evaluator registry

* feat(evals): add exact/regex/range/length evaluators

* feat(evals): add keyword/levenshtein/rouge/llm-as-judge evaluators

* feat(evals): add decision_scores composite evaluator

* feat(evals): add evaluation runner

* feat(evals): expose public API and module proxy

* fix(evals): resolve __all__ names and repair usage example

* docs(evals): add usage docs and changelog entry

* style(evals): tidy evaluator metadata and wiring comments

* fix(evals): honor expected arg and classify error metrics

* fix(evals): export get_evaluator and fix shared meta default

* fix(evals): guard provenance check against non-dict metadata

* docs: add objective layer design spec for semantica.evals

* docs: refine objective spec for consistency with AIP Evals semantics

* docs: add implementation plan for evals objective layer

* docs: fix plan tests to use module-level pytest import

* feat(evals): add per-metric objective support to runner

* docs(evals): document per-metric objectives

* docs(evals): fix minimize example threshold to demonstrate pass

* fix(evals): validate objective config shape strictly

* docs(evals): clarify objective examples and Boolean semantics

* fix(evals): honor direction-only minimize, fail fast on objectives, deep-merge case config

- minimize without threshold is now a no-op, matching maximize (issue #1091
  requires thresholds to be optional for both directions)
- objective config is parsed for every case before any target_fn/evaluator
  runs, so an invalid per-case objective rejects the run up front
- per-case evaluator config deep-merges over the global config so a case
  that overrides one setting keeps the run-level objective
- regression tests for all three, plus updated docs/CHANGELOG

Addresses 3 of 4 Qodo findings on #1092 (the 4th, 'result models defined
twice', is a false positive: types live in types.py)

* fix: finalize eval objectives review

---------

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
2026-09-02 18:47:28 +05:30
Mohd Kaif 8e73ed8d4c Merge branch 'main' into fix/semantic-layer-basics-intro 2026-09-02 12:44:19 +05:30
e040d84d59 feat(context): add ErasureCoordinator for cross-store entity erasure (#1027)
* feat(context): add ErasureCoordinator for cross-store entity erasure

purge_node() is graph-scope by design (#957), so an entity removed from the
graph can survive verbatim as an AgentMemory item and as an embedding. The
changelog names GDPR Article 17 as purge's motivation, and an Article 17
erasure the vector store can still answer queries from is not an erasure --
it is worse than none, because purge_node() returns True and writes a
tombstone attesting the content is gone.

ErasureCoordinator composes the existing public APIs to drive the cascade and
returns an ErasureReceipt recording what each store reported. Nothing in
context_graph.py or agent_memory.py changes behaviorally; ContextGraph keeps
its documented graph-scope contract instead of acquiring references that would
invert the dependency.

Honest partial reporting is the point. Stores report erased / not_found /
not_configured / unsupported / failed, and complete is False when any store
reports unsupported or failed. FAISS, Milvus and Weaviate expose no delete at
all, so erasure genuinely cannot be completed on them today -- the receipt
says so rather than reporting a success it did not achieve.

Erasure runs outward-in (vectors, memory, graph). The tombstone is the durable
attestation, so writing it first would let a crash mid-cascade leave a record
claiming more than happened; erasing the graph last leaves a partial failure
recoverable and honest.

The memory sweep pages until dry and re-queries afterwards rather than
trusting one find_by_entity() call, whose limit=10 default silently truncates
the very check a caller uses to decide the erasure is done. Unsupported vector
backends are detected by probing the wrapped backend, since the VectorStore
facade declares delete_vectors() for every backend and only raises
NotImplementedError once called.

27 tests against real ContextGraph/AgentMemory instances, including the
25-items-on-one-entity regression that fails against a naive single-call
sweep. Full tests/context/ suite: 596 passed.

Closes #1018

* docs(context): document ErasureCoordinator in the context API reference

* test(context): exercise ErasureCoordinator against a real VectorStore

The vector-leg tests asserted the three backend shapes the coordinator
expects -- delete_vectors / delete / neither -- against fakes, which is worth
exactly as much as the assumption that a real store looks like one of them.
VectorStore(backend="inmemory") runs without external services, so it can
hold that assumption to account.

Adds the end-to-end case the receipt actually attests to: a real ContextGraph,
AgentMemory and VectorStore, where the embedding is written by
AgentMemory.store() and has to be gone afterwards. That exercises the memory
leg's own delete_memory() vector cascade rather than the coordinator's model
of it.

The real backend also pins a limit worth knowing before trusting the receipt:
it pops the ids and returns True whether or not they were there, and no
backend offers a portable existence check, so `erased` on the vectors leg
means the store accepted the delete for the ids given -- not that embeddings
were really removed. The memory leg re-queries to confirm and so is the
stronger claim. Documented on STATUS_ERASED, _erase_vectors(), and both status
tables.

tests/context/: 599 passed.

* fix(context): address review findings on ErasureCoordinator

Timestamp drift (high). erase_entity() resolved erased_at up front but passed
the caller's original `at` down to purge_node(), so on the default at=None
path the coordinator and the graph each took their own now() and the receipt
attested to a different instant than the tombstone it points at -- breaking
the invariant this module states most loudly. The resolved value is now what
the graph receives. The existing test passed only because it supplied an
explicit `at`, which hides the drift; the regression test covers at=None,
which is what callers actually use.

Backend delete results. The vectors leg treated anything other than the
literal False as success, but no in-repo backend returns a bool -- Qdrant
returns {"status": <UpdateStatus>} and Pinecone {"deleted": True}, so every
dict read as success and the backend's own account of the delete was thrown
away. Results are now interpreted by shape and the payload is kept in the
receipt as backend_result, stringified so it stays JSON-serializable as an
audit record. Bool markers match by identity so a 0 count isn't read as
False; string markers match as substrings so an enum rendering as
"UpdateStatus.FAILED" isn't read as success.

Falsey vector store. The "at least one store" guard used `not vector_store`,
rejecting a valid store whose __bool__/__len__ makes an empty instance falsey
and then reporting vector_store=None when an object had been passed. It now
separates None (absent) from False (deliberately disabled) from provided, and
echoes what it received.

`at` annotations. Widened to int/float, matching the ContextGraph normalizer
they delegate to, so the coordinator stops advertising less than the API it
wraps.

tests/context/: 608 passed.

* fix(context): report memory-owned vectors that survive erasure (#1018)

A receipt could read complete while an embedding was still in the vector
store. The vector leg deleted `vector_ids` or `[entity_id]`, and the memory
leg relied on `AgentMemory.delete_memory()` to cascade to the vectors each
item owns. That cascade is best-effort: `_delete_vector_ids()` raises when a
backend returns False, `delete_memory()` catches it, logs a warning, and
still returns True. So `batch_delete` counted the item, the residual re-query
found no items, the memory leg reported `erased`, and nothing in the receipt
recorded that the embedding was refused.

Reproduced with a store that deletes the entity-keyed id and refuses the
memory-owned one: `receipt.complete` was True with the embedding still live.
That is the failure mode this module exists to prevent -- a receipt is a
compliance artifact, and one that overstates is worse than none.

Fix by deleting memory-owned vector ids through the coordinator's own vector
leg, which reports honestly, instead of trusting the memory leg's cascade.
The ids are collected before anything is deleted, while the items still exist
to be enumerated, and are unioned with any caller-supplied ids rather than
replacing them.

This needs one addition to AgentMemory: `vector_ids_for(memory_id)`, a
read-only accessor mirroring the fallback in `delete_memory` (an item stored
without tracked ids is keyed by its own memory id). Reaching into
`_vector_ids` from the coordinator would have been the internals-access
pattern this repo keeps getting bitten by. No existing AgentMemory behaviour
changes -- `delete_memory()` still cascades best-effort, so other callers are
unaffected; the coordinator simply no longer depends on that being reliable.
It does mean the vectors are attempted twice, which is a no-op on a working
store and only ever costs a log line.

Note this deviates from the PR's stated "nothing in agent_memory.py changes"
constraint. The constraint could not hold: with `_vector_ids` private and no
portable way to ask a vector store what it still holds, the coordinator had
no way to make the claim truthful without it.

Four tests: the refused-vector case (receipt must be incomplete), that
memory-owned ids reach the store, that explicit `vector_ids` do not displace
them, and the accessor's fallback. The first three were confirmed to fail
against the previous coordinator, on the `receipt.complete` assertion rather
than incidentally. 655 tests pass across tests/context and the agno
integration.

* fix(context): make erasure receipt vector failures honest

* fix(context): optimize erasure pagination handling

* fix(context): use one timestamp for batch erasure

* style: strip trailing whitespace from erasure.py and test file

* docs(changelog): correct test counts to 48 / 738 after review rounds

---------

Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
Co-authored-by: Sameer6305 <sskadam6305@gmail.com>
2026-09-01 23:07:02 +05:30
taoche e6c05df33e docs(cookbook): index semantic layer capstone 2026-09-01 19:51:49 +08:00
Sameer Kadam 46b18fbee3 fix(docs): use GraphStore facade in Neo4j quickstart (#1340)
The Neo4j example in the persistent graph store section was passing a raw
Neo4jStore straight into GraphBuilder. GraphBuilder calls add_nodes() and
add_edges() on whatever it's given, and those only exist on the GraphStore
facade, not on Neo4jStore itself. Anyone who copied the example got:

AttributeError: 'Neo4jStore' object has no attribute 'add_nodes'

Swapped the import and construction to GraphStore(backend="neo4j", uri=...,
user=..., password=...), which wraps Neo4jStore internally and actually has
the methods GraphBuilder needs.

Added a test in tests/kg/test_graph_builder_with_graph_store.py that builds
a small graph through GraphBuilder with a mocked GraphStore and checks
add_nodes/add_edges get called. Also kept a test for GraphBuilder without a
graph_store at all, so that path doesn't regress either.

Closes #1135
2026-08-31 20:34:04 +05:00
Zohaib HassnainandSameer Kadam ea0dd17bff feat(llms): add Gemini, Ollama, DeepSeek, Novita provider wrappers (#1262)
* feat(llms): add Gemini, Ollama, DeepSeek, Novita provider wrappers

* fix(llms): address Qodo review findings on provider wrappers PR

* fix(llms): address review findings

---------

Co-authored-by: Sameer Kadam <sskadam6305@gmail.com>
2026-08-30 17:00:14 +05:30
KaifAhmad1 9d98eedaa1 fix(llms): replace retired default model and tidy Anthropic wrapper
claude-3-sonnet-20240229 was retired 2025-07-21, so the wrapper's
default model and every copy-paste doc example would fail at
generate() time out of the box. Switch to claude-sonnet-4-6
everywhere (wrapper default, __init__ docstring, docs guide, tests).

Also cleans up leftover docstring typos/spacing from the previous
review pass and adds unavailable-path test coverage for
generate_structured()/generate_typed() to match generate(), clearing
ANTHROPIC_API_KEY in those tests so they don't flake on a runner that
has a real key set.
2026-08-30 13:33:18 +05:30
Zohaib Hassnain ca7f743dab fix(llms): address Qodo review findings on Anthropic wrapper 2026-08-29 22:15:19 +05:00
Zohaib Hassnain 5cf59fdd88 feat(llms): add first class Anthropic provider wrapper 2026-08-29 22:01:16 +05:00
Sameer Kadam 4bf525d409 feat(ingest): add production-ready Salesforce ingestor (#1240)
feat(ingest): add Salesforce ingestor

Adds first-class Salesforce ingestion support, following the existing
Connector + Data + Ingestor architecture already used by the
Snowflake and Databricks integrations: SalesforceConnector /
SalesforceData / SalesforceIngestor, exposed lazily from
semantica.ingest so the base install stays unaffected.

SalesforceConnector supports both auth landscapes Salesforce actually
uses in practice: username + password + security token (SOAP login,
on-prem/sandbox), and session_id + instance_url for reusing an
existing authenticated session. Production and sandbox are selected
through domain, credentials can come from environment variables, and
the connector never intentionally puts credential material into logs,
exceptions, or its own repr.

SalesforceIngestor covers ingest_sobject(), ingest_query(),
list_sobjects(), get_sobject_schema(), and export_as_documents(),
against standard sObjects, custom objects (__c), custom metadata
objects (__mdt), platform events (__e), namespaced objects, and
relationship-field traversal (Owner.Name). Pagination follows
nextRecordsUrl/query_more() automatically and stops once a caller's
limit is satisfied rather than continuing to fetch full pages past it.

Dynamically constructed SOQL is validated before it's sent: sObject
names, field names, relationship paths, ORDER BY expressions, and
numeric limits are checked, and WHERE fragments are screened against
common injection primitives after masking quoted string literals so a
value like status = 'union' doesn't false-positive. Raw SOQL passed
directly to ingest_query() stays intentionally caller-controlled,
since that method is documented as the advanced/unvalidated escape
hatch.

Salesforce-specific attributes metadata is stripped from returned
records before they're handed to the rest of the pipeline, while
relationship data, normal field values, and datetime normalization
are preserved. export_as_documents() uses the Salesforce Id as the
stable document identifier and keeps the source record in document
metadata for provenance.

Wired into the unified ingestion API via ingest_salesforce() and
ingest(source_type="salesforce", ...), registered with
MethodRegistry under sobject/query/list_sobjects/schema/documents.
Isolated behind the semantica[db-salesforce] extra
(simple-salesforce>=1.12.0), included in db-all.

JWT Bearer authentication and Bulk API 2.0 are intentionally out of
scope for this first connector; both are documented as deliberate
follow-ups rather than gaps.

fix(ingest): address Salesforce review findings

- limit now validates as a non-negative integer before use; negative,
  string, and float values raise ValidationError instead of silently
  returning an empty result, raising a bare TypeError, or building an
  invalid LIMIT 0 query
- fields is validated as a non-empty list of strings; a bare string
  (e.g. "Id") no longer gets iterated character-by-character into
  nonsense field names, and an empty list no longer builds a
  syntactically invalid SELECT
- the generic connection-failure path now raises with `from None`
  instead of chaining the original exception, so credential or
  request detail from the underlying library can't surface through a
  traceback
- the unified ingest() dispatch no longer coerces a non-dict source
  into None and silently falling back to environment credentials; an
  invalid source now raises
- _validate_order_by rewritten to validate each dot-separated
  component through _validate_field_name, rejecting malformed
  fragments like "Name." or "Owner..Name" that the previous regex let
  through
- CI conflicts from parallel merges resolved; upstream markdown
  dependency changes preserved

test(ingest): add Salesforce JWT coverage

Adds construction and connect() coverage for the JWT Bearer auth path
(consumer_key + privatekey/privatekey_file), the one auth mode that
had no dedicated tests despite handling private key material.
Also removes _SAFE_ORDER_RE, left behind as dead code once
_validate_order_by was rewritten to use _validate_field_name per
component, and fixes a test-isolation leak where an earlier test left
SALESFORCE_AVAILABLE=True behind for a later test that expected it
False when simple-salesforce isn't installed.
2026-08-29 12:55:37 +05:00
KaifAhmad1 ecb33a5b7d chore(release): prepare v0.6.7
Bump version, cut CHANGELOG's Unreleased section into 0.6.7, backfill
changelog entries for merged PRs missing from it, and refresh
version-dependent references in README/docs.
2026-08-28 15:51:17 +05:30
Kevin Zhang 100e95a098 feat(ingest): add SAP OData ingestor (#1228) (#1234)
Adds `SAPODataConnector`, `SAPODataEntity`, and `SAPIngestor` for ingesting master and transactional data from SAP OData services, mainly things like Business Partners and Sales Orders.

Tested around S/4HANA Cloud, SuccessFactors, and on-prem NetWeaver Gateway style OData endpoints.

Main pieces included:

* OAuth2 client credentials and Basic auth support. Both go through the existing `ssrf.py` checks, including the OAuth token request.
* Small EDMX parser used by `discover_service()` so we don't need to pull in `pyodata`.
* Server-side pagination support for both OData versions:

  * v2: `__next`, including plain string and `__deferred` formats
  * v4: `@odata.nextLink`
* Keeps the service path in the base URL correctly whether the URL has a trailing slash or not. This is normalized in `SAPIngestor.__init__`.
* Adds an `ingest-sap` extra with just `requests`, so there is no SAP/proprietary SDK dependency.

This is meant to be a fairly small first version of the connector without adding a lot of SAP-specific dependencies.

Closes #1228
2026-08-28 14:54:02 +05:00
cxzg007and江俊杰 cce5ea177c fix(pipeline): set_parallelism now enables dependency-layer parallel execution (closes #1223) (#1226)
PipelineBuilder.set_parallelism() validated and stored a level in
pipeline config, but ExecutionEngine._execute_steps() had no parallel
code path and no code ever read it back, so steps always ran strictly
sequentially regardless of the configured value. parallelism was also
lost across a serialize/deserialize round trip, since the nested
config key was never promoted to the top-level dict build_pipeline()
reads.

Steps are now grouped into dependency layers (declaration order
preserved within each layer). A layer runs concurrently, bounded by
ThreadPoolExecutor(max_workers=min(configured parallelism, engine
max_workers)), only when every one of the following holds: more than
one step in the layer, the shared input is a dict, every step is
opted in via the new PipelineStep.parallel_safe flag, and no step is
in delta_mode. Any layer that doesn't meet all four falls back to the
existing sequential path unchanged.

Each step's input is deep-copied before any handler in the layer
starts, so concurrent steps never share mutable state. Layer results
are merged back in declaration order, not completion order; keys
whose value is unchanged from the shared input are treated as an
echo rather than a write, so two handlers both returning {**data, ...}
don't spuriously conflict on keys neither of them actually touched.
Genuinely conflicting values for the same key raise ProcessingError
naming both the key and the two steps involved. Retry policy, step
status, result/error tracking, and progress reporting are shared
between the sequential and parallel paths so behavior stays identical
either way. On step failure, not yet started futures in the same
layer are cancelled and the error propagates, so no downstream layer
ever runs.

parallel_safe is opt-in per step because handlers that share mutable
state or depend on strict ordering are not safe to run concurrently.
ParallelismManager.execute_pipeline_steps_parallel() is deliberately
not reused here; the engine implements its own bounded layer
scheduler so retry/status semantics stay identical between the
sequential and parallel code paths instead of diverging.

fix(pipeline): address qodo review findings on PR #1226

- detect circular/unknown dependencies in parallel grouping
  (ValidationError instead of RecursionError/KeyError)
- skip unchanged echoed keys in parallel result merging to
  avoid false conflicts
- fail before COMPLETED status when a parallel step returns a
  non-dict; never retry such contract violations
- require strict bool parallel_safe in builder and engine gate
- make per-step progress tracking IDs unique across same-type
  parallel steps

---------

Co-authored-by: 江俊杰 <jiangjunjie.37@jd.com>
2026-08-28 12:33:39 +05:00
pravit-ampandPravit Ampapathini 0775b0114e test(provenance): assert stored records in KG provenance suites (#946) (#1132)
* fix(provenance): use timezone-aware UTC and assert stored records (#946)

Replace datetime.utcnow() in ProvenanceManager, ProvenanceEntry,
BridgeAxiom, and GraphBuilderWithProvenance with
datetime.now(timezone.utc), matching PipelineWithProvenance.

KG workflow and integration tests now read provenance back through
get_provenance() and assert algorithm metadata instead of generated
IDs, and call tracker methods that actually persist records.

* fix(provenance): compare provenance timestamps as instants, not strings

query_recorded_between() and audit_log() filtered and sorted on raw ISO
strings. With the timezone-aware change, a store can hold both pre-existing
naive stamps and offset-bearing ones, and the two are not string-comparable:
"...500000+00:00" sorts above "...500000", so a record at the identical
instant as a naive bound falls outside the range that should contain it.

Both now parse through _parse_timestamp() before comparing, reading naive
values as UTC. This mirrors ProvenanceTracker._parse_dt() in kg/, the class
ProvenanceManager replaces, so both sides of the migration answer a range
query the same way. Unparseable stored timestamps are skipped and logged
rather than silently dropped; unparseable bounds raise ValueError.

---------

Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
2026-08-27 15:40:38 +05:00
LeonSGP43 6032b4e0bc docs(cookbook): add index entries for notebooks 22-25
Index the four module notebooks merged via #989-#992 (Provenance
Tracking, Reasoning, Change Management, Seed Data) in the cookbook
landing page, as committed in tracking issue #1032.

Signed-off-by: LeonSGP43 <cine.dreamer.one@gmail.com>
2026-08-27 17:39:48 +08:00
cxzg007and江俊杰 5d54919804 feat(reasoning): rule-driven actions with provenance (#1096)
* feat(reasoning): rule-driven actions with provenance

Add a structured Action layer so matched rules can trigger side effects
instead of only deriving new facts, turning the reasoner into a
production-rule system.

L1 - Action type system:
- Action base class with execute(bindings, reasoner) + ?var substitution
- AssertAction (optional write-back to KnowledgeGraph), RetractAction,
  CallAction (structured replacement for the unused Rule.handler),
  EmitEventAction (delivers to a registered event sink)
- Rule.actions field; wired into Reasoner.forward_chain() and
  ReteEngine.execute_matches() (via optional bind_reasoner)

L2 - Provenance-aware actions:
- Reasoner records fired actions (rule, bindings, confidence) to
  action_log when provenance is enabled
- Fix dangling import in reasoning_provenance.py (ReasoningEngine ->
  Reasoner, infer -> infer_facts)

Backward compatible: rules using the legacy handler still fire (wrapped
as a CallAction); rules without actions behave exactly as before.

Adds tests/reasoning/test_rule_actions.py (9 tests).

Closes #1095

* fix(reasoning): address qodo review findings on rule actions

- Token-aware variable substitution to avoid ?x/?xy prefix collision
- KnowledgeGraph write-back protocol (explicit API -> canonical translation -> ValueError)
- Structured action_log entries with timestamp
- Decouple action firing from conclusion dedup via per-activation tracking
  (fires known conclusions once; retract-self no longer loops to max_iterations)
- Add Reasoner.infer_with_results preserving confidence; infer_facts delegates
- Forward provenance flag in ReasoningProvenance; drop **kwargs; propagate confidence
- Populate Rete Match.bindings from rule conditions
- Add regression tests for each fix

* fix(reasoning): persist fired action activations

* fix(reasoning): deduplicate Rete action execution

* fix(reasoning): canonicalize action activation identity

* docs(reasoning): explain action replay controls

---------

Co-authored-by: 江俊杰 <jiangjunjie.37@jd.com>
2026-08-27 13:18:07 +05:00
cxzg007and江俊杰 c9c777993b fix(pipeline): wire registered step handlers (#1215)
* fix(pipeline): wire registered step handlers

Resolve handlers registered by step type, keep explicit handlers authoritative, and prevent builder control fields from leaking into runtime kwargs.

Refs #1214

* fix(pipeline): preserve dependencies on deserialize

* fix(pipeline): dispatch falsy handlers via identity check

---------

Co-authored-by: 江俊杰 <jiangjunjie.37@jd.com>
2026-08-27 13:11:42 +05:00
Derek TapleyandCursor f0aa581318 feat(integrations): add LangChain integration — retriever, vectorstor… (#1155)
* feat(integrations): add LangChain integration — retriever, vectorstore, tools

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(langchain): address Qodo review on HybridSearch hits and tools

Read nested HybridSearch metadata so retriever/vectorstore Documents
are not empty, make the agent tools real BaseTool subclasses, and
stop slicing tool JSON into invalid payloads.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-26 18:29:22 +05:00
Mohd Kaif c415d57d16 Merge pull request #1210 from semantica-agi/citation-and-org-cleanup
docs: add citation section and fix stale org references
2026-08-24 16:20:08 +05:30
KaifAhmad1 7a6f1d0417 docs: add citation section and fix stale org references
Add a Cite Us section to the README with BibTeX citation info, and
align it with docs/citation.md (author/organization: Semantica, 2026).
Update LICENSE and docs/project-license.md copyright holder to
Semantica, and replace the stale Hawksight-AI GitHub org slug with
semantica-agi across READMEs, plugin manifests, cookbook notebooks,
and GitHub templates.
2026-08-24 16:07:22 +05:30
Mohd Kaif b9cb524514 Merge branch 'main' into fix/1185-non-tty-progress 2026-08-24 12:25:14 +05:30
Freakz2z 4c997b5017 fix(triplet_store): encode RDF4J repository paths 2026-08-24 09:02:42 +08:00
Aldrin Joseph de31b43663 fix(utils): write console progress only to an interactive stdout
ProgressTracker attached ConsoleProgressDisplay unconditionally, so any
script or CI job that piped or redirected stdout had one progress bar per
stage written into its output, escape sequences included. A plain
`python demo.py > out.txt` captured 173 bytes of progress-bar noise around
10 bytes of the program's own output.

Console progress is now attached only when stdout is an interactive
terminal, when running under Jupyter, or when SEMANTICA_FORCE_PROGRESS is
set. FileProgressDisplay is untouched, so progress logging still works in
pipelines, and SEMANTICA_DISABLE_PROGRESS keeps its existing meaning and
still takes precedence.

Both progress environment variables are now documented in the README and
the utils reference; SEMANTICA_DISABLE_PROGRESS previously existed only in
the reference page.

Deviations from the issue: the issue suggested disabling the tracker on
non-TTY stdout. This gates the display instead, because disabling the
tracker would short-circuit before FileProgressDisplay and take file
progress logging down with it, and the ~20 modules that set
`progress_tracker.enabled = True` in __init__ would need the property
setter taught about TTY state to avoid undoing it. Gating the display
leaves both alone.

Design note: the claim comment on the issue proposed an
`enabled: Optional[bool] = None` constructor opt-in; during implementation
the opt-in became SEMANTICA_FORCE_PROGRESS, which needs no signature change
and follows the NO_COLOR/FORCE_COLOR convention. Known limitation: TTY
detection runs once at tracker construction (the tracker is a process-wide
singleton), so a process that redirects stdout after first use needs the
env vars to change behaviour.

Fixes #1185
2026-08-23 22:17:31 +05:30
Freakz2z e41993a6bd fix(triplet_store): honor RDF4J repository id 2026-08-23 23:02:15 +08:00
Mohd Kaif 48c2d2a7ed Merge branch 'main' into fix/issue-888-docs-storage-backends 2026-08-23 18:02:19 +05:30
KaifAhmad1 fdafffa980 fix(docs): correct storage-backends adapter names, kwargs, and inventory
The adapter inventory and connection examples referenced classes that
don't exist in semantica.graph_store (Neo4jGraphStore, NeptuneGraphStore,
AgeGraphStore) and used constructor kwargs that don't match the actual
adapters (username vs user, host vs endpoint, url vs endpoint, etc.),
verified against each adapter's real __init__ signature and by
constructing every example against the live classes.

- Correct class names: Neo4jStore, AmazonNeptuneStore, ApacheAgeStore
- Fix kwargs for all seven examples to match actual constructors
- Fix ApacheAgeStore's connection_string to libpq keyword=value format
  instead of a postgresql:// DSN, which the adapter doesn't accept
- Reclassify Anzo from interface/BYO to built-in — AnzoStore is a real,
  exported, tested adapter
- Add the two adapters missing from the inventory: FalkorDBStore and
  OxigraphStore
- Replace the literal password='password' example with an env var
- Note a real RDF4JStore bug found while verifying the RDF4J example:
  repository_id is a named constructor parameter but the implementation
  reads it from **config instead, so it's silently ignored and the
  store always connects to the "default" repository
2026-08-23 17:57:53 +05:30
Mohd Kaif 2b077c6d0e Merge branch 'main' into codex/context-graph-markdown-round-trip 2026-08-23 17:26:05 +05:30
Mohd Kaif 4820185924 Merge branch 'main' into codex/harden-markdown-import-symlinks 2026-08-23 16:55:43 +05:30
mikemikimike fe3baad67c docs(shacl): correct legacy alias name 2026-08-23 00:02:57 +08:00
mikemikimike 50f2f82b95 feat(ontology): expose public SHACL validation API 2026-08-22 23:58:00 +08:00
Aldrin Joseph 3c99f447e6 docs(shacl): document that rdfs:range + RDFS entailment makes sh:class unfalsifiable (#1182)
* docs(shacl): warn that rdfs:range makes sh:class unfalsifiable under entailment (#1130)

* docs(shacl): self-contained pitfall example, sh:node coverage, and wrapper clarifications (#1130)
2026-08-22 19:12:56 +05:00
Saurabh Meena 8dcbee386d Merge remote-tracking branch 'origin/main' into codex/context-graph-markdown-round-trip 2026-08-22 19:38:26 +05:30
Saurabh Meena e2f850e9e4 Merge remote-tracking branch 'origin/main' into codex/harden-markdown-import-symlinks 2026-08-22 19:38:20 +05:30