docs: update provenance documentation for SQLiteStorage WAL and batch tracking (#807)

This commit is contained in:
Sameer6305
2026-07-28 23:42:14 +05:30
parent 3db35a6871
commit 9458cf5b2b
2 changed files with 10 additions and 1 deletions
+4
View File
@@ -83,6 +83,10 @@ prov = ProvenanceManager(storage=SQLiteStorage("audit.db"))
For any regulated deployment — security operations, clinical data, financial risk — use `storage_path`. A SQLite file can be backed up, versioned, and queried with standard tools without requiring a server.
<Note>
`SQLiteStorage` automatically configures Write-Ahead Logging (`WAL`), `busy_timeout=5000`, and `synchronous=NORMAL`, and executes operations in atomic immediate transactions (`BEGIN IMMEDIATE`). Furthermore, `ProvenanceManager` automatically supports custom storage backends overriding only `trace_lineage(self, entity_id)` without requiring `max_depth` in their signature.
</Note>
## Recording provenance when ingesting data
The moment data enters your graph is the moment provenance must be recorded. `track_entity()` captures the source document, the timestamp, the operator or pipeline that ran the extraction, a verbatim quote from the source, and a confidence score. It returns a `ProvenanceEntry` with a SHA-256 checksum computed automatically.
+6 -1
View File
@@ -155,13 +155,15 @@ prop_entry = manager.track_property_source(
### Batch Tracking
Batch tracking methods process items in blocks (default `batch_size=1000`) inside a shared transaction per block. Only entities or chunks that successfully commit to storage are added to the returned count, preventing rolled-back entries from inflating success counts.
```python
entities = [
{"id": "entity_1", "confidence": 0.9},
{"id": "entity_2", "confidence": 0.85},
]
count = manager.track_entities_batch(entities, source="doc_1")
# Returns the number of entities successfully tracked
# Returns the number of entities successfully tracked and committed
chunks = [
{"id": "chunk_0", "start_index": 0, "end_index": 500},
@@ -322,6 +324,9 @@ manager = ProvenanceManager(storage_path="provenance.db")
`SQLiteStorage` creates the database and indexes automatically on first use.
- **Atomicity & Concurrency**: Configures Write-Ahead Logging (`PRAGMA journal_mode=WAL`), `PRAGMA busy_timeout=5000`, and `PRAGMA synchronous=NORMAL`. Each public method call opens a single connection and executes inside an immediate write transaction (`BEGIN IMMEDIATE`), ensuring read-modify-write sequences are serialized across concurrent connections without leaving open file handles across calls.
- **Backward Compatibility**: Custom storage subclasses overriding `trace_lineage(self, entity_id)` remain backward compatible; `ProvenanceManager` inspects the override signature and automatically calls it with one argument if `max_depth` is unsupported.
## Tamper-Evident Checksums
`compute_checksum` and `verify_checksum` are auto-used by `track_entity` and all other tracking methods. You can also call them directly: