* fix(provenance): use timezone-aware UTC and assert stored records (#946)
Replace datetime.utcnow() in ProvenanceManager, ProvenanceEntry,
BridgeAxiom, and GraphBuilderWithProvenance with
datetime.now(timezone.utc), matching PipelineWithProvenance.
KG workflow and integration tests now read provenance back through
get_provenance() and assert algorithm metadata instead of generated
IDs, and call tracker methods that actually persist records.
* fix(provenance): compare provenance timestamps as instants, not strings
query_recorded_between() and audit_log() filtered and sorted on raw ISO
strings. With the timezone-aware change, a store can hold both pre-existing
naive stamps and offset-bearing ones, and the two are not string-comparable:
"...500000+00:00" sorts above "...500000", so a record at the identical
instant as a naive bound falls outside the range that should contain it.
Both now parse through _parse_timestamp() before comparing, reading naive
values as UTC. This mirrors ProvenanceTracker._parse_dt() in kg/, the class
ProvenanceManager replaces, so both sides of the migration answer a range
query the same way. Unparseable stored timestamps are skipped and logged
rather than silently dropped; unparseable bounds raise ValueError.
---------
Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com>
* feat(reasoning): rule-driven actions with provenance
Add a structured Action layer so matched rules can trigger side effects
instead of only deriving new facts, turning the reasoner into a
production-rule system.
L1 - Action type system:
- Action base class with execute(bindings, reasoner) + ?var substitution
- AssertAction (optional write-back to KnowledgeGraph), RetractAction,
CallAction (structured replacement for the unused Rule.handler),
EmitEventAction (delivers to a registered event sink)
- Rule.actions field; wired into Reasoner.forward_chain() and
ReteEngine.execute_matches() (via optional bind_reasoner)
L2 - Provenance-aware actions:
- Reasoner records fired actions (rule, bindings, confidence) to
action_log when provenance is enabled
- Fix dangling import in reasoning_provenance.py (ReasoningEngine ->
Reasoner, infer -> infer_facts)
Backward compatible: rules using the legacy handler still fire (wrapped
as a CallAction); rules without actions behave exactly as before.
Adds tests/reasoning/test_rule_actions.py (9 tests).
Closes#1095
* fix(reasoning): address qodo review findings on rule actions
- Token-aware variable substitution to avoid ?x/?xy prefix collision
- KnowledgeGraph write-back protocol (explicit API -> canonical translation -> ValueError)
- Structured action_log entries with timestamp
- Decouple action firing from conclusion dedup via per-activation tracking
(fires known conclusions once; retract-self no longer loops to max_iterations)
- Add Reasoner.infer_with_results preserving confidence; infer_facts delegates
- Forward provenance flag in ReasoningProvenance; drop **kwargs; propagate confidence
- Populate Rete Match.bindings from rule conditions
- Add regression tests for each fix
* fix(reasoning): persist fired action activations
* fix(reasoning): deduplicate Rete action execution
* fix(reasoning): canonicalize action activation identity
* docs(reasoning): explain action replay controls
---------
Co-authored-by: 江俊杰 <jiangjunjie.37@jd.com>
* docs(shacl): warn that rdfs:range makes sh:class unfalsifiable under entailment (#1130)
* docs(shacl): self-contained pitfall example, sh:node coverage, and wrapper clarifications (#1130)
* fix(reasoning): refuse SPARQL query execution instead of returning empty results (#1083)
SPARQLReasoner.execute_query() never executed the query: both branches
returned an empty SPARQLQueryResult, with or without a triplet store, so
callers that trust an empty result as "no matches" silently drew wrong
conclusions. Until a real triplet-store execution path lands, the method
raises NotImplementedError with an explanation, per the issue's
suggestion. The dead cache/inference scaffolding after the execution
point is removed along with it.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(reasoning): align execute_query() docs with the NotImplementedError contract (#1087)
Review feedback: the docstring still carried a "Returns" section and the
reasoning guide showed execute_query() returning bindings, both of which
now mislead. The docstring documents Raises only, the guide demonstrates
expand_query() and points to rdflib for execution until the triplet-store
path lands, and query_cache/clear_cache() are marked as reserved for that
future execution path.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Part A - high-stakes trust blockers:
- Invalidation tombstones via ProvenanceManager.invalidate() (archive-then-append,
never mutates or deletes) instead of hard delete
- Hash-chained integrity: sequence_id/previous_checksum chain every entry to its
predecessor; new verify_chain() detects wholesale row deletion that a lone
per-row checksum cannot
- Typed Agent (AgentRecord: agent_type/is_automated) and Activity (ActivityRecord:
start/end timing), wired through all 18 *_provenance.py wrappers
- Split parent_entity_id into previous_version_id (correction) vs derived_from_id
(cross-source derivation), additive alongside the legacy combined field
- Downstream/descendant lineage traversal (get_descendants/trace_descendants,
reverse BFS) closing the dead direction="downstream" code path in the
Explorer's provenance route
- Qualified Association+hadRole and Invalidation in export_prov()
- New CLI: provenance invalidate|verify-chain|descendants
Part B - general PROV-O spec completeness:
- Qualified Generation/Usage/Derivation in export_prov()
- wasAssociatedWith, actedOnBehalfOf, wasInformedBy relations
- Bitemporal fields (valid_from/valid_until/revision_type/supersedes) plus
revision_history()/query_recorded_between(), closing the deprecated
kg.ProvenanceTracker's "no direct equivalent yet" migration gaps
- prov:Bundle/hadMember membership via bundle_id
- Configurable base_uri (--base-uri CLI flag), shared by RDFExporter's
NamespaceManager and OWLExporter's default ontology_uri so KG/OWL/PROV
exports co-resolve under one namespace instead of three hardcoded ones
Bugs fixed along the way:
- agent_id was a dead field: no track_* method read it from kwargs
- track_entities_batch silently absorbed typed kwargs into the metadata blob
- compute_checksum() had to exclude entity_id itself: hashing it made
track_entity's versioning-archive relabel permanently orphan any entry
already chained from the pre-relabel checksum, a false-positive "broken
chain" for a legitimate rename
- InMemoryStorage.get_chain_head() ignored the committed head whenever the
current transaction had staged entries, corrupting the next chain link
- several new ProvenanceEntry fields were wired into the dataclass and
export_prov() but not into SQLiteStorage's DDL/INSERT/row-mapping;
InMemoryStorage masked the gap. Added a permanent round-trip regression
test to catch this class of bug for future field additions
Flagged, not fixed (separate pre-existing issues, out of scope for #825):
- pipeline/pipeline_provenance.py imports a nonexistent module and wraps a
Pipeline dataclass with no run() method
- most *_provenance.py wrappers' backing classes are themselves missing or
incomplete (context_manager, deduplicator, normalizer, etc.)
- kg_provenance.py passes entity_type inside metadata={} instead of as a
top-level track_entity() kwarg across most of its call sites
Two review findings on #807/#812:
- retrieve() and trace_lineage() were routed through transaction()'s
BEGIN IMMEDIATE, so plain reads took SQLite's writer lock and
serialized behind every other read/write, defeating the WAL
concurrency this PR was meant to add. They now use a dedicated
_read_connection() (configured, no explicit BEGIN).
- track_entity()/track_chunk() swallowed all internal storage
exceptions unconditionally, so a single item's failure inside
track_entities_batch()/track_chunks_batch()'s shared transaction
never reached the batch loop's per-item except, inflating
tracked_count for entries that were never persisted. Both now
re-raise when called with a shared _conn (batch context) while
still degrading gracefully on standalone calls.
Added regression tests for both, corrected the CHANGELOG entry and
docs that described the prior (overly broad) behavior.
- README: get_table_lineage() takes table_name first, then catalog/schema
keyword args — the example had them in the wrong order, which would have
queried lineage for the wrong fully-qualified table when copy-pasted.
- modules.md: the ingest example used DatabricksIngestor without importing
it, causing a NameError if copy-pasted as-is.
- guides/ingest.md: corrected the claim that Databricks/Snowflake ingestors
return "the same shape as DBIngestor" — DBIngestor.execute_query() returns
a raw List[Dict] with no wrapper, unlike DatabricksData/SnowflakeData.
Makes enterprise lakehouse/warehouse ingestion (Databricks Unity Catalog +
Delta Lake, Snowflake) a first-class, prominently documented capability
across the README and guides, and adds matching runnable examples to
docs/guides/ingest.md. Also fixes several pre-existing inaccuracies caught
while auditing the ingest module docs against the actual source:
WebIngestor has no ingest_urls() (only singular ingest_url()), XMLIngestor's
XSD option is schema_path (not validate_xsd) and belongs on ingest() not the
constructor, and the "Available ingestors" list was missing DatabricksIngestor
while listing several classes not actually exported from semantica.ingest.
'By default, initializing ... with storage_path=...' read as if passing
storage_path were the default, contradicting the very next sentence
about the no-argument in-memory default. Rephrased so the in-memory
default isn't undercut by the first sentence.
* docs: improve SHACL validation guide onboarding and workflow guidance
* docs: fix SHACL validation implementation mismatches
* docs: fix stale violation URIs and drop unused imports in SHACL guide
Step 5's illustrative explain_violations() output still referenced the
old cti.example.org/data/... node URIs after Step 4's data_ttl was
rewritten to use example.org/... URIs. Also removes now-unused
export_rdf/tempfile/os imports left over from replacing dynamic RDF
export with inline Turtle strings in five of the code examples.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
Step 5's illustrative explain_violations() output still referenced the
old cti.example.org/data/... node URIs after Step 4's data_ttl was
rewritten to use example.org/... URIs. Also removes now-unused
export_rdf/tempfile/os imports left over from replacing dynamic RDF
export with inline Turtle strings in five of the code examples.
* docs: improve conflict resolution guide onboarding and workflow guidance
* docs: fix conflict resolution implementation mismatches
* docs: correct credibility-weighted example output values
Fix stale/incorrect weight and confidence figures in the conflict
resolution guide that don't match actual resolver output, and update
a leftover credibility_score field reference in Common Pitfalls.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
Fix stale/incorrect weight and confidence figures in the conflict
resolution guide that don't match actual resolver output, and update
a leftover credibility_score field reference in Common Pitfalls.
The merging example told readers to use merge_entity_group() for
already-confirmed duplicate groups, but the code right below it still
called merge_duplicates() on group.entities, which re-runs duplicate
detection redundantly. Update the call to match the stated guidance.
- Correct the unsupported-rule-key pitfall: absent keys fail compliance,
present keys (any value) pass — the previous wording had this backwards.
- Remove required_ltv/pd/lgd/dsti/credit_score: True from the mortgage
example. required_* checks equality against the given value, so True
against a real numeric field silently marks compliant decisions as
non-compliant (verified: a fully passing decision still returned False).
The min_/max_ rules already enforce presence of ltv/dsti/credit_score.
* docs: improve export guide onboarding and workflow guidance
* docs: fix export guide implementation mismatches
* docs: revert .content to .text in export guide examples
FileObject.content is raw bytes; AgentContext.store() only accepts str/list and raises ValueError on bytes, so the previous fix commit broke both domain examples.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
FileObject.content is raw bytes; AgentContext.store() only accepts str/list and raises ValueError on bytes, so the previous fix commit broke both domain examples.
* docs: improve agent memory guide onboarding and usage guidance
* docs: align Agent Memory guide with persistence implementation
* docs: fix misleading index_path persistence claim across guides
VectorStore's index_path kwarg is silently absorbed into FAISSStore's
**config and never read anywhere in faiss_store.py, so it does not make
the FAISS index persist across restarts as several docs implied. Real
persistence requires an explicit VectorStore.save()/.load() call, or
AgentContext.save()/.load() which cascades to it.
- docs/reference/context.md: rewrite the "Persist your vector store"
tip to explain the actual save()/load() mechanism instead of the
dead index_path kwarg.
- docs/guides/graphrag.md, decision-intelligence.md, ingest.md,
semantic-extraction.md: drop the dead index_path=... kwarg from
VectorStore(backend="faiss", ...) constructor calls.
Follow-up to #691, which fixed the same false claim in
docs/guides/agent-memory.md but missed these other files.
---------
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
