mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-09-02 04:00:40 +00:00
e040d84d59
* feat(context): add ErasureCoordinator for cross-store entity erasure purge_node() is graph-scope by design (#957), so an entity removed from the graph can survive verbatim as an AgentMemory item and as an embedding. The changelog names GDPR Article 17 as purge's motivation, and an Article 17 erasure the vector store can still answer queries from is not an erasure -- it is worse than none, because purge_node() returns True and writes a tombstone attesting the content is gone. ErasureCoordinator composes the existing public APIs to drive the cascade and returns an ErasureReceipt recording what each store reported. Nothing in context_graph.py or agent_memory.py changes behaviorally; ContextGraph keeps its documented graph-scope contract instead of acquiring references that would invert the dependency. Honest partial reporting is the point. Stores report erased / not_found / not_configured / unsupported / failed, and complete is False when any store reports unsupported or failed. FAISS, Milvus and Weaviate expose no delete at all, so erasure genuinely cannot be completed on them today -- the receipt says so rather than reporting a success it did not achieve. Erasure runs outward-in (vectors, memory, graph). The tombstone is the durable attestation, so writing it first would let a crash mid-cascade leave a record claiming more than happened; erasing the graph last leaves a partial failure recoverable and honest. The memory sweep pages until dry and re-queries afterwards rather than trusting one find_by_entity() call, whose limit=10 default silently truncates the very check a caller uses to decide the erasure is done. Unsupported vector backends are detected by probing the wrapped backend, since the VectorStore facade declares delete_vectors() for every backend and only raises NotImplementedError once called. 27 tests against real ContextGraph/AgentMemory instances, including the 25-items-on-one-entity regression that fails against a naive single-call sweep. Full tests/context/ suite: 596 passed. Closes #1018 * docs(context): document ErasureCoordinator in the context API reference * test(context): exercise ErasureCoordinator against a real VectorStore The vector-leg tests asserted the three backend shapes the coordinator expects -- delete_vectors / delete / neither -- against fakes, which is worth exactly as much as the assumption that a real store looks like one of them. VectorStore(backend="inmemory") runs without external services, so it can hold that assumption to account. Adds the end-to-end case the receipt actually attests to: a real ContextGraph, AgentMemory and VectorStore, where the embedding is written by AgentMemory.store() and has to be gone afterwards. That exercises the memory leg's own delete_memory() vector cascade rather than the coordinator's model of it. The real backend also pins a limit worth knowing before trusting the receipt: it pops the ids and returns True whether or not they were there, and no backend offers a portable existence check, so `erased` on the vectors leg means the store accepted the delete for the ids given -- not that embeddings were really removed. The memory leg re-queries to confirm and so is the stronger claim. Documented on STATUS_ERASED, _erase_vectors(), and both status tables. tests/context/: 599 passed. * fix(context): address review findings on ErasureCoordinator Timestamp drift (high). erase_entity() resolved erased_at up front but passed the caller's original `at` down to purge_node(), so on the default at=None path the coordinator and the graph each took their own now() and the receipt attested to a different instant than the tombstone it points at -- breaking the invariant this module states most loudly. The resolved value is now what the graph receives. The existing test passed only because it supplied an explicit `at`, which hides the drift; the regression test covers at=None, which is what callers actually use. Backend delete results. The vectors leg treated anything other than the literal False as success, but no in-repo backend returns a bool -- Qdrant returns {"status": <UpdateStatus>} and Pinecone {"deleted": True}, so every dict read as success and the backend's own account of the delete was thrown away. Results are now interpreted by shape and the payload is kept in the receipt as backend_result, stringified so it stays JSON-serializable as an audit record. Bool markers match by identity so a 0 count isn't read as False; string markers match as substrings so an enum rendering as "UpdateStatus.FAILED" isn't read as success. Falsey vector store. The "at least one store" guard used `not vector_store`, rejecting a valid store whose __bool__/__len__ makes an empty instance falsey and then reporting vector_store=None when an object had been passed. It now separates None (absent) from False (deliberately disabled) from provided, and echoes what it received. `at` annotations. Widened to int/float, matching the ContextGraph normalizer they delegate to, so the coordinator stops advertising less than the API it wraps. tests/context/: 608 passed. * fix(context): report memory-owned vectors that survive erasure (#1018) A receipt could read complete while an embedding was still in the vector store. The vector leg deleted `vector_ids` or `[entity_id]`, and the memory leg relied on `AgentMemory.delete_memory()` to cascade to the vectors each item owns. That cascade is best-effort: `_delete_vector_ids()` raises when a backend returns False, `delete_memory()` catches it, logs a warning, and still returns True. So `batch_delete` counted the item, the residual re-query found no items, the memory leg reported `erased`, and nothing in the receipt recorded that the embedding was refused. Reproduced with a store that deletes the entity-keyed id and refuses the memory-owned one: `receipt.complete` was True with the embedding still live. That is the failure mode this module exists to prevent -- a receipt is a compliance artifact, and one that overstates is worse than none. Fix by deleting memory-owned vector ids through the coordinator's own vector leg, which reports honestly, instead of trusting the memory leg's cascade. The ids are collected before anything is deleted, while the items still exist to be enumerated, and are unioned with any caller-supplied ids rather than replacing them. This needs one addition to AgentMemory: `vector_ids_for(memory_id)`, a read-only accessor mirroring the fallback in `delete_memory` (an item stored without tracked ids is keyed by its own memory id). Reaching into `_vector_ids` from the coordinator would have been the internals-access pattern this repo keeps getting bitten by. No existing AgentMemory behaviour changes -- `delete_memory()` still cascades best-effort, so other callers are unaffected; the coordinator simply no longer depends on that being reliable. It does mean the vectors are attempted twice, which is a no-op on a working store and only ever costs a log line. Note this deviates from the PR's stated "nothing in agent_memory.py changes" constraint. The constraint could not hold: with `_vector_ids` private and no portable way to ask a vector store what it still holds, the coordinator had no way to make the claim truthful without it. Four tests: the refused-vector case (receipt must be incomplete), that memory-owned ids reach the store, that explicit `vector_ids` do not displace them, and the accessor's fallback. The first three were confirmed to fail against the previous coordinator, on the `receipt.complete` assertion rather than incidentally. 655 tests pass across tests/context and the agno integration. * fix(context): make erasure receipt vector failures honest * fix(context): optimize erasure pagination handling * fix(context): use one timestamp for batch erasure * style: strip trailing whitespace from erasure.py and test file * docs(changelog): correct test counts to 48 / 738 after review rounds --------- Co-authored-by: Pravit Ampapathini <pravit.amp@gmail.com> Co-authored-by: Sameer6305 <sskadam6305@gmail.com>