mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-09-13 04:04:09 +00:00
* fix(parse): stop infinite recursion in default method dispatch
parse_document and its five sibling dispatchers (parse_web_content,
parse_structured_data, parse_email, parse_code, parse_media) were
registered in the method registry under their own task's "default"
name. Every dispatcher begins with method_registry.get(<task>, method),
so calling e.g. parse_document(file, method="default") found itself in
the registry and re-entered infinitely until RecursionError --
`semantica parse <any file>` crashed before any parsing ran.
Drop the six self-registrations. "default" remains the built-in code
path; users can still register their own "default" (or any other name)
to override it, and the existing "docling" registration is unaffected.
Verified: `semantica parse demo.pdf` now parses successfully (was
RecursionError). No repo code or tests consume
get_parse_method("document", "default"), so removing the entries
changes no behavior besides fixing the crash.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(deps): make resolution satisfiable on Python 3.9 and add parse-pdf extra
Dependency fixes so `uv lock` / `pip install semantica[all]` resolves
across the supported matrix (3.9-3.12):
- requires-python >=3.8 was unsatisfiable (numpy>=2.0.2 needs >=3.9) and
3.9.0/3.9.1 can never resolve (every cryptography release excludes
them) -> bump to >=3.9.2 and drop the 3.8 classifier.
- Split recently-raised floors that dropped 3.9 into marker pairs
(3.9-capped / 3.10-unconstrained), following the pattern already used
for scikit-learn/requests/etc.: pyarrow extras (>=24 needs 3.10),
pre-commit 4.6, snowflake-connector 4.6, fastapi 0.129 + starlette 0.53
(older fastapi caps starlette<0.53).
- Gate docling, litellm (its only 3.9 release pins
python-dotenv==1.0.1, conflicting with the >=1.2.1 core floor), and
crewai (no un-yanked 3.9 release) to >=3.10.
Also add a parse-pdf extra: the default PDFParser requires pdfplumber,
but no extra installed it, so `semantica parse file.pdf` failed on a
default install. Follows the parse-docling convention.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(vector-store): make the qdrant backend usable through VectorStore
The qdrant backend could not be used at all through the VectorStore
facade in 0.6.8 - every path raised:
1. store_vectors() only dispatched to backend add/add_vectors methods;
QdrantStore exposes insert_vectors, so vs.store() raised
NotImplementedError. Add an insert_vectors branch (uuid-generated
ids, metadata -> payloads).
2. QdrantStore required an explicit create_collection() before any read
or write, unlike FAISSStore's automatic index creation. Lazily attach
the configured collection (config key "collection", default
"semantica_default") on first insert/search, reusing an existing one.
3. search_points() called client.search(), removed from qdrant-client in
favor of query_points() - use it when available, fall back otherwise.
4. store() silently dropped plain-string documents (it only extracted
doc.metadata), so payloads lost the source text; keep them under
payload "document".
Verified end-to-end against a Qdrant 5 server (docker): embed ->
store -> semantic search returns correctly ranked results whose payloads
carry the original documents.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(vector-store): address Qodo review findings on the qdrant write path
Four findings from the Qodo review of #1508:
1. store_vectors() returned QdrantStore.insert_vectors()' upsert status
dict although the facade promises the stored vector IDs (decision
storage indexes the result at position 0). Return the generated or
caller-supplied IDs after a successful insert instead.
2. insert_vectors() pairs points with zip(vectors, ids), so a shorter
non-empty id list silently dropped the unpaired vectors while the
completion message still reported the full batch as inserted. Reject
the mismatch with ValidationError before any write.
3. _ensure_default_collection() looked up the legacy "collection" config
key, so the documented collection_name=... option was ignored and
lazy init always fell back to semantica_default. Prefer
collection_name, keep "collection" as an alias.
4. The new parse-pdf extra was missing from both aggregate "all"
bundles, so semantica[all] still shipped without pdfplumber and the
default PDFParser raised ProcessingError on first use.
Also removes the now-stale strict xfail for qdrant's write dispatch in
test_backend_facade_contract.py — that marker exists precisely to fail
once the wiring lands.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* test(vector-store): migrate qdrant mocks to the query_points API
The qdrant search path now prefers client.query_points() (qdrant-client
removed client.search), but these tests still mocked the legacy call.
A MagicMock exposes query_points too, so the code took the modern path,
read .points off an unconfigured mock, and all three tests failed on
the branch. Return the hits in response.points as the real client does.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(ci): regenerate requirements-ci.txt for the parse-pdf extra
The CI lockfile check re-resolves pyproject.toml --extra all and diffs
the pinned versions against requirements-ci.txt. The parse-pdf extra
added pdfplumber (+pdfminer-six) to the 'all' bundle without refreshing
the lockfile, so the diff failed and the build job exited 1.
Regenerated with the command from the file header. Only the two new
pins and their 'via' comments changed - every other version is
identical. Verified locally with the CI's own check command (diff of
pkg==ver lines exits clean).
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(tests): update stale default-method-registration assertion
test_parse_methods_dynamic_default_resolution asserted that "default"
resolves through the registry to parse_document, which was true only
because of the self-registration this PR removes (it's what caused the
infinite recursion in the first place). Update the assertion to match
the intended post-fix state: "default" is not registered at all, and
falls through to the built-in dispatch path unconditionally.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: yanyu <yanyu@polixir.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Mohd Kaif <98801504+KaifAhmad1@users.noreply.github.com>
Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
144 lines
5.3 KiB
Python
144 lines
5.3 KiB
Python
"""Facade-level contract tests for the cloud vector store backends.
|
|
|
|
Other tests here either mock a backend's internals or inject a fake into
|
|
``VectorStore._backend_store``. Both skip ``_init_backend_store``, which is
|
|
where the qdrant/pinecone/milvus/weaviate adapters are built, and that is how
|
|
#1316 shipped green while a qdrant-backed store could neither read nor write.
|
|
|
|
Gaps are recorded as strict xfail so they turn into XPASS once the wiring
|
|
lands, failing the suite until the stale marker is removed.
|
|
|
|
Related: #1265, #1019.
|
|
"""
|
|
|
|
from contextlib import ExitStack
|
|
from unittest.mock import MagicMock, patch
|
|
|
|
import numpy as np
|
|
import pytest
|
|
|
|
from semantica.vector_store import VectorStore
|
|
|
|
# Availability flag per backend, plus every symbol its connect/select path
|
|
# calls. The clients must be patched too: without the real SDK installed they
|
|
# are None, so a fixed _init_backend_store would still fail and these could
|
|
# never reach XPASS. Extend these if the wiring touches more symbols.
|
|
_AVAILABILITY_FLAG = {
|
|
"qdrant": "semantica.vector_store.qdrant_store.QDRANT_AVAILABLE",
|
|
"pinecone": "semantica.vector_store.pinecone_store.PINECONE_AVAILABLE",
|
|
"milvus": "semantica.vector_store.milvus_store.MILVUS_AVAILABLE",
|
|
"weaviate": "semantica.vector_store.weaviate_store.WEAVIATE_AVAILABLE",
|
|
}
|
|
|
|
_CLIENT_SYMBOLS = {
|
|
"qdrant": ("semantica.vector_store.qdrant_store.QdrantClientLib",),
|
|
"pinecone": ("semantica.vector_store.pinecone_store.PineconeClientLib",),
|
|
"milvus": (
|
|
"semantica.vector_store.milvus_store.connections",
|
|
"semantica.vector_store.milvus_store.Collection",
|
|
"semantica.vector_store.milvus_store.utility",
|
|
),
|
|
"weaviate": ("semantica.vector_store.weaviate_store.weaviate",),
|
|
}
|
|
|
|
# Pinecone refuses to connect without a key, so supply a dummy one rather than
|
|
# letting a missing credential masquerade as the wiring gap.
|
|
_EXTRA_CONFIG = {"pinecone": {"api_key": "test-key"}}
|
|
|
|
CLOUD_BACKENDS = sorted(_AVAILABILITY_FLAG)
|
|
|
|
# Backends that store locally and need no connection step.
|
|
_LOCAL_BACKENDS = {"inmemory", "faiss", "sqlite", "pgvector"}
|
|
|
|
# The facade dispatches store_vectors() to `add`, `add_vectors`, or
|
|
# `insert_vectors`. Milvus exposes add_vectors and qdrant insert_vectors, so
|
|
# both resolve; the remaining two name their write method differently and fall
|
|
# through to NotImplementedError.
|
|
_NO_WRITE_DISPATCH = {"pinecone", "weaviate"}
|
|
|
|
|
|
def _construct(backend):
|
|
"""Build a VectorStore through the real _init_backend_store path."""
|
|
config = {"dimension": 3, **_EXTRA_CONFIG.get(backend, {})}
|
|
with ExitStack() as stack:
|
|
stack.enter_context(patch(_AVAILABILITY_FLAG[backend], True))
|
|
for symbol in _CLIENT_SYMBOLS[backend]:
|
|
stack.enter_context(patch(symbol, MagicMock()))
|
|
return VectorStore(backend=backend, config=config)
|
|
|
|
|
|
def _live_handle(backend_store):
|
|
"""The attribute each adapter holds its connected resource in.
|
|
|
|
Reaching into the adapter rather than asserting through the facade is
|
|
deliberate: the facade's read methods are exactly what is broken, so there
|
|
is no public call that distinguishes "not connected" from the other gaps.
|
|
"""
|
|
for name in ("collection", "index"):
|
|
if hasattr(backend_store, name):
|
|
return getattr(backend_store, name)
|
|
return None
|
|
|
|
|
|
def _param(backend, broken_for, reason):
|
|
marks = [pytest.mark.xfail(strict=True, reason=reason)] if backend in broken_for else []
|
|
return pytest.param(backend, marks=marks)
|
|
|
|
|
|
def test_roster_covers_every_supported_backend():
|
|
"""A new backend must be classified here rather than silently uncovered."""
|
|
assert set(CLOUD_BACKENDS) | _LOCAL_BACKENDS == VectorStore.SUPPORTED_BACKENDS
|
|
|
|
|
|
@pytest.mark.parametrize("backend", CLOUD_BACKENDS)
|
|
def test_facade_constructs_an_adapter(backend):
|
|
store = _construct(backend)
|
|
|
|
assert store._backend_store is not None
|
|
assert store.backend == backend
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"backend",
|
|
[
|
|
_param(b, CLOUD_BACKENDS, "_init_backend_store never connects or selects a collection")
|
|
for b in CLOUD_BACKENDS
|
|
],
|
|
)
|
|
def test_backend_is_connected_after_construction(backend):
|
|
"""A constructed store should be usable without the caller reaching past
|
|
the facade to call connect() and get_collection() itself."""
|
|
store = _construct(backend)
|
|
|
|
assert _live_handle(store._backend_store) is not None
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"backend",
|
|
[
|
|
_param(b, _NO_WRITE_DISPATCH, "facade dispatches only to add/add_vectors")
|
|
for b in CLOUD_BACKENDS
|
|
],
|
|
)
|
|
def test_store_vectors_dispatch_resolves(backend):
|
|
"""store_vectors() should reach the backend's write method."""
|
|
store = _construct(backend)
|
|
|
|
try:
|
|
store.store_vectors([np.zeros(3)], [{}], ids=["a"])
|
|
except NotImplementedError as exc:
|
|
pytest.fail(f"no write dispatch for {backend}: {exc}")
|
|
except Exception:
|
|
# Any other error means the facade found a write method and the failure
|
|
# came from below it, which is the connection gap the test above pins.
|
|
# Whether the write succeeds needs a live server, not this test.
|
|
pass
|
|
|
|
|
|
def test_milvus_write_dispatch_already_resolves():
|
|
"""Control for _NO_WRITE_DISPATCH: if milvus changes, the xfail list is
|
|
wrong rather than the feature being broken."""
|
|
store = _construct("milvus")
|
|
|
|
assert hasattr(store._backend_store, "add_vectors")
|