Files
semantica/tests/vector_store/test_qdrant_facade_writes.py
T
5a0d6ea431 Fix/parse and qdrant vector store (#1508)
* fix(parse): stop infinite recursion in default method dispatch

parse_document and its five sibling dispatchers (parse_web_content,
parse_structured_data, parse_email, parse_code, parse_media) were
registered in the method registry under their own task's "default"
name. Every dispatcher begins with method_registry.get(<task>, method),
so calling e.g. parse_document(file, method="default") found itself in
the registry and re-entered infinitely until RecursionError --
`semantica parse <any file>` crashed before any parsing ran.

Drop the six self-registrations. "default" remains the built-in code
path; users can still register their own "default" (or any other name)
to override it, and the existing "docling" registration is unaffected.

Verified: `semantica parse demo.pdf` now parses successfully (was
RecursionError). No repo code or tests consume
get_parse_method("document", "default"), so removing the entries
changes no behavior besides fixing the crash.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(deps): make resolution satisfiable on Python 3.9 and add parse-pdf extra

Dependency fixes so `uv lock` / `pip install semantica[all]` resolves
across the supported matrix (3.9-3.12):

- requires-python >=3.8 was unsatisfiable (numpy>=2.0.2 needs >=3.9) and
  3.9.0/3.9.1 can never resolve (every cryptography release excludes
  them) -> bump to >=3.9.2 and drop the 3.8 classifier.
- Split recently-raised floors that dropped 3.9 into marker pairs
  (3.9-capped / 3.10-unconstrained), following the pattern already used
  for scikit-learn/requests/etc.: pyarrow extras (>=24 needs 3.10),
  pre-commit 4.6, snowflake-connector 4.6, fastapi 0.129 + starlette 0.53
  (older fastapi caps starlette<0.53).
- Gate docling, litellm (its only 3.9 release pins
  python-dotenv==1.0.1, conflicting with the >=1.2.1 core floor), and
  crewai (no un-yanked 3.9 release) to >=3.10.

Also add a parse-pdf extra: the default PDFParser requires pdfplumber,
but no extra installed it, so `semantica parse file.pdf` failed on a
default install. Follows the parse-docling convention.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(vector-store): make the qdrant backend usable through VectorStore

The qdrant backend could not be used at all through the VectorStore
facade in 0.6.8 - every path raised:

1. store_vectors() only dispatched to backend add/add_vectors methods;
   QdrantStore exposes insert_vectors, so vs.store() raised
   NotImplementedError. Add an insert_vectors branch (uuid-generated
   ids, metadata -> payloads).
2. QdrantStore required an explicit create_collection() before any read
   or write, unlike FAISSStore's automatic index creation. Lazily attach
   the configured collection (config key "collection", default
   "semantica_default") on first insert/search, reusing an existing one.
3. search_points() called client.search(), removed from qdrant-client in
   favor of query_points() - use it when available, fall back otherwise.
4. store() silently dropped plain-string documents (it only extracted
   doc.metadata), so payloads lost the source text; keep them under
   payload "document".

Verified end-to-end against a Qdrant 5 server (docker): embed ->
store -> semantic search returns correctly ranked results whose payloads
carry the original documents.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(vector-store): address Qodo review findings on the qdrant write path

Four findings from the Qodo review of #1508:

1. store_vectors() returned QdrantStore.insert_vectors()' upsert status
   dict although the facade promises the stored vector IDs (decision
   storage indexes the result at position 0). Return the generated or
   caller-supplied IDs after a successful insert instead.
2. insert_vectors() pairs points with zip(vectors, ids), so a shorter
   non-empty id list silently dropped the unpaired vectors while the
   completion message still reported the full batch as inserted. Reject
   the mismatch with ValidationError before any write.
3. _ensure_default_collection() looked up the legacy "collection" config
   key, so the documented collection_name=... option was ignored and
   lazy init always fell back to semantica_default. Prefer
   collection_name, keep "collection" as an alias.
4. The new parse-pdf extra was missing from both aggregate "all"
   bundles, so semantica[all] still shipped without pdfplumber and the
   default PDFParser raised ProcessingError on first use.

Also removes the now-stale strict xfail for qdrant's write dispatch in
test_backend_facade_contract.py — that marker exists precisely to fail
once the wiring lands.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* test(vector-store): migrate qdrant mocks to the query_points API

The qdrant search path now prefers client.query_points() (qdrant-client
removed client.search), but these tests still mocked the legacy call.
A MagicMock exposes query_points too, so the code took the modern path,
read .points off an unconfigured mock, and all three tests failed on
the branch. Return the hits in response.points as the real client does.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(ci): regenerate requirements-ci.txt for the parse-pdf extra

The CI lockfile check re-resolves pyproject.toml --extra all and diffs
the pinned versions against requirements-ci.txt. The parse-pdf extra
added pdfplumber (+pdfminer-six) to the 'all' bundle without refreshing
the lockfile, so the diff failed and the build job exited 1.

Regenerated with the command from the file header. Only the two new
pins and their 'via' comments changed - every other version is
identical. Verified locally with the CI's own check command (diff of
pkg==ver lines exits clean).

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(tests): update stale default-method-registration assertion

test_parse_methods_dynamic_default_resolution asserted that "default"
resolves through the registry to parse_document, which was true only
because of the self-registration this PR removes (it's what caused the
infinite recursion in the first place). Update the assertion to match
the intended post-fix state: "default" is not registered at all, and
falls through to the built-in dispatch path unconditionally.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: yanyu <yanyu@polixir.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Mohd Kaif <98801504+KaifAhmad1@users.noreply.github.com>
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
2026-09-08 15:47:05 +05:30

127 lines
4.6 KiB
Python

"""Regression tests for the Qdrant write path behind the VectorStore facade.
Qodo review of #1508 found three bugs in the newly wired qdrant dispatch:
stored IDs were swallowed (the upsert status dict was returned instead),
mismatched ids/vectors silently truncated the write via zip(), and lazy
collection init ignored the documented ``collection_name`` option. These
tests pin all three.
qdrant-client is not installed in this environment, so QDRANT_AVAILABLE is
patched and PointStruct is replaced with a plain stand-in, following the
pattern in test_vector_store_deepdive.py and test_qdrant_store.py.
"""
from unittest.mock import MagicMock, patch
import numpy as np
import pytest
from semantica.utils.exceptions import ValidationError
from semantica.vector_store import VectorStore
from semantica.vector_store.qdrant_store import QdrantStore
VECTORS = [np.array([1.0, 0.0, 0.0]), np.array([0.0, 1.0, 0.0])]
METADATA = [{"type": "a"}, {"type": "b"}]
class _Point:
"""Stand-in for qdrant_client PointStruct that keeps the point id."""
def __init__(self, id, vector=None, payload=None):
self.id = id
self.vector = vector
self.payload = payload
def _qdrant_facade(**config):
"""VectorStore built through the real qdrant init path, with the network
client and an attached collection replaced by mocks."""
store = VectorStore(backend="qdrant", config={"dimension": 3, **config})
backend = store._backend_store
backend.client = MagicMock()
backend.collection = MagicMock()
return store
@patch("semantica.vector_store.qdrant_store.PointStruct", _Point)
@patch("semantica.vector_store.qdrant_store.QDRANT_AVAILABLE", True)
def test_facade_returns_generated_ids_not_upsert_status():
"""store_vectors() promises callers the stored vector IDs; the qdrant
branch used to leak insert_vectors()' upsert status dict instead."""
store = _qdrant_facade()
ids = store.store_vectors(VECTORS, METADATA)
assert isinstance(ids, list)
assert len(ids) == len(VECTORS)
assert all(isinstance(i, str) and i for i in ids)
upserted = store._backend_store.collection.upsert_points.call_args[0][0]
assert [p.id for p in upserted] == ids
@patch("semantica.vector_store.qdrant_store.PointStruct", _Point)
@patch("semantica.vector_store.qdrant_store.QDRANT_AVAILABLE", True)
def test_facade_returns_caller_supplied_ids_verbatim():
store = _qdrant_facade()
ids = store.store_vectors(VECTORS, METADATA, ids=["doc-a", "doc-b"])
assert ids == ["doc-a", "doc-b"]
@pytest.mark.parametrize("ids", [["doc-a"], ["doc-a", "doc-b", "doc-c"]])
@patch("semantica.vector_store.qdrant_store.PointStruct", _Point)
@patch("semantica.vector_store.qdrant_store.QDRANT_AVAILABLE", True)
def test_facade_rejects_ids_count_mismatch(ids):
"""insert_vectors() pairs points with zip(vectors, ids); a mismatched
batch must fail loudly instead of silently dropping vectors."""
store = _qdrant_facade()
with pytest.raises(ValidationError, match="must match number of vectors"):
store.store_vectors(VECTORS, METADATA, ids=ids)
store._backend_store.collection.upsert_points.assert_not_called()
@patch("semantica.vector_store.qdrant_store.PointStruct", _Point)
@patch("semantica.vector_store.qdrant_store.QDRANT_AVAILABLE", True)
def test_backend_insert_vectors_rejects_count_mismatch():
"""Direct QdrantStore callers get the same guard as facade callers."""
store = QdrantStore()
store.client = MagicMock()
store.collection = MagicMock()
with pytest.raises(ValidationError, match="must match number of vectors"):
store.insert_vectors(VECTORS, ["only-one"])
store.collection.upsert_points.assert_not_called()
def _lazy_collection_name(store):
"""Drive _ensure_default_collection and report the name it selected."""
store.client = MagicMock()
with (
patch.object(store, "create_collection") as create,
patch.object(store, "get_collection"),
):
store._ensure_default_collection(3)
return create.call_args[0][0]
def test_lazy_collection_uses_documented_collection_name():
"""The docs and facade configure qdrant with collection_name=...; lazy
init used to look up 'collection' and always fall back to the default."""
store = QdrantStore(collection_name="semantica")
assert _lazy_collection_name(store) == "semantica"
def test_lazy_collection_accepts_legacy_collection_alias():
store = QdrantStore(collection="legacy_name")
assert _lazy_collection_name(store) == "legacy_name"
def test_lazy_collection_defaults_without_config():
store = QdrantStore()
assert _lazy_collection_name(store) == "semantica_default"