Compare commits

..
Author SHA1 Message Date
KaifAhmad1 bf292ccbbc fix(docker): address terrascan findings, drop apt-get upgrade
Two terrascan/GHAS findings on the previous commit:
- AC_DOCKER_0052 (no apt-get upgrade in Dockerfiles): dropped it. It also
  wasn't fixing anything - Debian's openssl fix for CVE-2026-14456 is still
  in trixie-proposed-updates, not reachable via a normal upgrade. Pin both
  base images by digest instead (matches #1329's approach) so the docker
  Dependabot ecosystem bumps them once Debian ships a rebuilt image with
  the fix, and document why the QUIC DoS isn't reachable here regardless
  (HTTP-only via uvicorn).
- AC_DOCKER_0010 (pin pip package versions): setuptools was `>=78.1.1`;
  pinned to the exact 84.0.0 already used by pyproject.toml/requirements-ci.txt.

Also fixes two build breaks this introduces on its own: requirements-ci.txt
wasn't in .dockerignore's allowlist or container-scan.yml's path trigger,
so the COPY in the prior commit would have failed the image build outright.
2026-08-31 14:06:28 +05:30
KaifAhmad1 64d942503b fix(docker): extract requirements-ci.txt pins with Python instead of sed
The sed expression to strip requirements-ci.txt's line-continuation
backslash (`[\]$`) is valid POSIX/GNU sed - verified it exits 0 and
strips correctly - but it's easy to misread as broken (a bot reviewer
flagged it as an unterminated bracket expression), and the seemingly
more obvious `\$`/` \$` forms silently fail to match at all rather
than erroring. Swap to a small `re.findall` extraction so there's no
backslash-escaping judgment call left for a reader (bot or human) to
second-guess.
2026-08-31 14:06:02 +05:30
KaifAhmad1 471a420b10 fix(docker): resolve Trivy-flagged CVEs in the built image
Container Security Scan flagged five HIGH-severity findings against
semantica:scan:
- setuptools 70.3.0 (CVE-2025-47273, path traversal) - the base image's
  bundled copy, never touched by our own build. Upgraded explicitly.
- msgpack 1.1.2 (GHSA-6v7p-g79w-8964, OOB read/crash) - `pip install
  ".[explorer]"` re-resolved deps from scratch instead of reusing the
  audited, hash-pinned requirements-ci.txt (which already pins
  msgpack==1.2.1), so it landed on an unpatched transitive version. Now
  installs against a constraints file derived from requirements-ci.txt.
- openssl / libssl3t64 / openssl-provider-legacy (CVE-2026-14456, QUIC
  server DoS) - the Debian fix is still in trixie-proposed-updates, not
  yet promoted to trixie-security, so it can't be pulled via apt today.
  Added an apt upgrade step so the next image rebuild picks it up
  automatically once Debian ships it; documented why this image isn't
  actually exposed to it in the meantime (HTTP-only via uvicorn, no QUIC
  listener).
2026-08-31 14:06:02 +05:30
Mohd KaifandSameer6305 a4aa71ad87 fix(ci): unblock py3.9 install matrix and raise Scorecard pinning/signing (#1329)
* fix(ci): unblock py3.9 install matrix and raise Scorecard pinning/signing

pip install semantica failed on Python 3.9 across all three OSes because
spacy had no upper bound, so pip resolved spacy 3.8.16 whose thinc>=8.3.12
requirement has no cp39 wheels and no working sdist build path. Cap
spacy/thinc for python_version < '3.10' to the last wheel-compatible pair.

Also addresses the two OpenSSF Scorecard findings that were actually
fixable in code:
- Pinned-Dependencies: Dockerfile base images (node:26-alpine,
  python:3.13-slim) were unpinned by digest; pin both, and pin five
  previously-unversioned pip install calls in CI (build, safety, bandit,
  semgrep, jq, pip-audit).
- Signed-Releases: attest-build-provenance only publishes to the GH
  attestations API, which Scorecard doesn't inspect. Sign dist/* with
  Sigstore and attach the .sigstore.json bundles as release assets.

* fix(ci): correct Sigstore artifact inputs

---------

Co-authored-by: Sameer6305 <sskadam6305@gmail.com>
2026-08-31 14:04:28 +05:30
9 changed files with 60 additions and 153 deletions
+1
View File
@@ -6,6 +6,7 @@
!README.md
!LICENSE
!MANIFEST.in
!requirements-ci.txt
!semantica/
!semantica/**
!integrations/
+1 -1
View File
@@ -72,7 +72,7 @@ jobs:
diff \
<(grep -E '^[a-zA-Z0-9._-]+==' requirements-ci.txt | sed 's/ \\$//') \
<(grep -E '^[a-zA-Z0-9._-]+==' /tmp/requirements-ci-check.txt)
- run: pip install build
- run: pip install build==1.6.0
# wheel is build-time only (not in requirements-ci.txt) — install the
# same pinned version [build-system] declares so --no-isolation works.
- run: pip install wheel==0.48.0
+1
View File
@@ -12,6 +12,7 @@ on:
- 'README.md'
- 'LICENSE'
- 'MANIFEST.in'
- 'requirements-ci.txt'
- 'semantica/**'
- 'integrations/**'
- 'explorer/**'
+17 -3
View File
@@ -16,7 +16,7 @@ jobs:
cancel-in-progress: false
permissions:
contents: write # for the GitHub Release
id-token: write # for PyPI Trusted Publishing (OIDC) and attestation signing
id-token: write # for PyPI Trusted Publishing (OIDC), attestation signing, and Sigstore
attestations: write # for SLSA build provenance
# If you add another job to this workflow, give it its own explicit
# `permissions:` block rather than relying on the workflow-level default
@@ -40,7 +40,7 @@ jobs:
# build runs against the same versions CI tests against.
- name: Install pinned build dependencies
run: pip install -r requirements-ci.txt
- run: pip install build
- run: pip install build==1.6.0
# wheel is build-time only (not in requirements-ci.txt) — install the
# same pinned version [build-system] declares so --no-isolation works.
- run: pip install wheel==0.48.0
@@ -71,7 +71,21 @@ jobs:
uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4
with:
subject-path: 'dist/*'
# attest-build-provenance publishes to the GH attestations API only, which
# OpenSSF Scorecard's Signed-Releases check does not inspect - it looks for
# signature files attached as release assets. Sign here too so
# `dist/*.sigstore.json` bundles ship alongside the wheel/sdist on the
# GitHub Release itself.
- name: Sign artifacts with Sigstore
uses: sigstore/gh-action-sigstore-python@790bc6befb9d733738f18d8f895854b453640ec9 # v3.5.0
with:
inputs: |
dist/*.whl
dist/*.tar.gz
- uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3
with:
files: dist/*
files: |
dist/*.whl
dist/*.tar.gz
dist/*.sigstore.json
- uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
+1 -1
View File
@@ -52,7 +52,7 @@ jobs:
# Tooling AFTER the pinned set: installing safety/bandit/semgrep/jq
# first lets the pinned requirements overwrite their transitive deps
# (e.g. rich), which breaks the safety CLI at runtime.
pip install safety bandit semgrep jq
pip install safety==3.8.1 bandit==1.9.4 semgrep==1.175.0 jq==1.12.0
- name: Run Safety Check (Package Vulnerabilities)
run: |
+1 -1
View File
@@ -37,6 +37,6 @@ jobs:
# pyproject.toml changes under review. The schedule/workflow_dispatch
# runs stay non-blocking until a full pass over pre-existing findings
# across the whole [all] tree has been done.
- run: pip install pip-audit
- run: pip install pip-audit==2.10.1
- run: pip-audit -r requirements-ci.txt
continue-on-error: ${{ github.event_name != 'pull_request' }}
+30 -4
View File
@@ -1,5 +1,5 @@
# syntax=docker/dockerfile:1
FROM node:26-alpine AS frontend-builder
FROM node:26-alpine@sha256:2d984a15c9b54fd0aeb608b8e0d0d83529eb34d2966db27a1fb4f1edc3d298a3 AS frontend-builder
WORKDIR /app
COPY explorer/package*.json ./explorer/
@@ -9,7 +9,18 @@ RUN npm ci
COPY explorer/ ./
RUN mkdir -p /app/semantica && npm run build
FROM python:3.13-slim AS runtime
# CVE-2026-14456 (OpenSSL QUIC-server DoS, flagged against this base image's
# openssl/libssl3t64/openssl-provider-legacy): the Debian fix
# (3.5.7-1~deb13u2) is only in trixie-proposed-updates as of this writing,
# not yet promoted to trixie-security, so there's no package to pin here
# today. Deliberately NOT running `apt-get upgrade` to chase it - that
# breaks build reproducibility (terrascan AC_DOCKER_0052) and still
# wouldn't reach a proposed-updates-only package. Once Debian ships the fix
# and rebuilds this tag, the docker Dependabot ecosystem in
# .github/dependabot.yml opens a PR bumping the digest pin above. Also: this
# image only serves plain HTTP via uvicorn and never opens a QUIC listener,
# so the bug isn't reachable here regardless.
FROM python:3.13-slim@sha256:7ce4b6dfe35e55397b7cda544f8a13f191b7ae28dc5aad71fe664dbc9bc2623f AS runtime
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
@@ -22,12 +33,27 @@ WORKDIR /app
RUN groupadd --system semantica \
&& useradd --system --gid semantica --home-dir /app --shell /usr/sbin/nologin semantica
COPY pyproject.toml README.md LICENSE MANIFEST.in ./
COPY pyproject.toml README.md LICENSE MANIFEST.in requirements-ci.txt ./
COPY semantica/ ./semantica/
COPY integrations/ ./integrations/
COPY --from=frontend-builder /app/semantica/static ./semantica/static
RUN pip install --no-cache-dir ".[explorer]" \
# The base image ships an outdated setuptools (CVE-2025-47273); upgrade it
# explicitly since nothing in our own dependency tree otherwise pulls a
# newer copy. Pinned to the exact version requirements-ci.txt/pyproject.toml
# already build against, rather than a floor, per terrascan AC_DOCKER_0010.
# requirements-ci.txt itself carries the audited, CVE-checked pins for every
# transitive dependency (see security-scan.yml / security.yml) - feed them
# in as an unhashed constraints file (pip's hash-checking mode rejects the
# unhashable local source directory this installs) so the image lands on
# the same patched versions CI verified, e.g. msgpack>=1.2.1, rather than
# letting pip freely re-resolve and pick up an unpatched transitive version.
# (Extracted with Python's re module rather than sed/grep so there's no
# line-continuation-backslash stripping to get subtly wrong.)
RUN pip install --no-cache-dir "setuptools==84.0.0" \
&& python -c "import re, pathlib; pins = re.findall(r'^([A-Za-z0-9._-]+==\S+)', pathlib.Path('requirements-ci.txt').read_text(), re.M); pathlib.Path('/tmp/constraints.txt').write_text('\n'.join(pins))" \
&& pip install --no-cache-dir -c /tmp/constraints.txt ".[explorer]" \
&& rm -f /tmp/constraints.txt requirements-ci.txt \
&& chown -R semantica:semantica /app
USER semantica
+8 -1
View File
@@ -49,7 +49,14 @@ dependencies = [
"scipy>=1.13.1",
"scikit-learn>=1.7.2",
"umap-learn>=0.5.12",
"spacy>=3.4.0",
# thinc (spacy's core dep) dropped Python 3.9 wheels at 8.3.10, and later
# spacy patch releases (3.8.8+) require thinc>=8.3.9-only-on-3.10+ ranges,
# which forces a source build that fails outright on 3.9 (see Install
# Matrix run history). Capping both keeps 3.9 on the last wheel-compatible
# pair; 3.10+ is left unconstrained to always get the latest spacy/thinc.
"spacy>=3.4.0,<3.8.8; python_version < '3.10'",
"spacy>=3.4.0; python_version >= '3.10'",
"thinc<8.3.5; python_version < '3.10'",
"transformers>=4.20.0",
"torch>=1.13.1",
"sentence-transformers>=2.2.0",
@@ -1,142 +0,0 @@
"""Facade-level contract tests for the cloud vector store backends.
Other tests here either mock a backend's internals or inject a fake into
``VectorStore._backend_store``. Both skip ``_init_backend_store``, which is
where the qdrant/pinecone/milvus/weaviate adapters are built, and that is how
#1316 shipped green while a qdrant-backed store could neither read nor write.
Gaps are recorded as strict xfail so they turn into XPASS once the wiring
lands, failing the suite until the stale marker is removed.
Related: #1265, #1019.
"""
from contextlib import ExitStack
from unittest.mock import MagicMock, patch
import numpy as np
import pytest
from semantica.vector_store import VectorStore
# Availability flag per backend, plus every symbol its connect/select path
# calls. The clients must be patched too: without the real SDK installed they
# are None, so a fixed _init_backend_store would still fail and these could
# never reach XPASS. Extend these if the wiring touches more symbols.
_AVAILABILITY_FLAG = {
"qdrant": "semantica.vector_store.qdrant_store.QDRANT_AVAILABLE",
"pinecone": "semantica.vector_store.pinecone_store.PINECONE_AVAILABLE",
"milvus": "semantica.vector_store.milvus_store.MILVUS_AVAILABLE",
"weaviate": "semantica.vector_store.weaviate_store.WEAVIATE_AVAILABLE",
}
_CLIENT_SYMBOLS = {
"qdrant": ("semantica.vector_store.qdrant_store.QdrantClientLib",),
"pinecone": ("semantica.vector_store.pinecone_store.PineconeClientLib",),
"milvus": (
"semantica.vector_store.milvus_store.connections",
"semantica.vector_store.milvus_store.Collection",
"semantica.vector_store.milvus_store.utility",
),
"weaviate": ("semantica.vector_store.weaviate_store.weaviate",),
}
# Pinecone refuses to connect without a key, so supply a dummy one rather than
# letting a missing credential masquerade as the wiring gap.
_EXTRA_CONFIG = {"pinecone": {"api_key": "test-key"}}
CLOUD_BACKENDS = sorted(_AVAILABILITY_FLAG)
# Backends that store locally and need no connection step.
_LOCAL_BACKENDS = {"inmemory", "faiss", "sqlite", "pgvector"}
# The facade dispatches store_vectors() to `add` or `add_vectors`. Milvus
# exposes add_vectors so it already resolves; the other three name their write
# method differently and fall through to NotImplementedError.
_NO_WRITE_DISPATCH = {"qdrant", "pinecone", "weaviate"}
def _construct(backend):
"""Build a VectorStore through the real _init_backend_store path."""
config = {"dimension": 3, **_EXTRA_CONFIG.get(backend, {})}
with ExitStack() as stack:
stack.enter_context(patch(_AVAILABILITY_FLAG[backend], True))
for symbol in _CLIENT_SYMBOLS[backend]:
stack.enter_context(patch(symbol, MagicMock()))
return VectorStore(backend=backend, config=config)
def _live_handle(backend_store):
"""The attribute each adapter holds its connected resource in.
Reaching into the adapter rather than asserting through the facade is
deliberate: the facade's read methods are exactly what is broken, so there
is no public call that distinguishes "not connected" from the other gaps.
"""
for name in ("collection", "index"):
if hasattr(backend_store, name):
return getattr(backend_store, name)
return None
def _param(backend, broken_for, reason):
marks = [pytest.mark.xfail(strict=True, reason=reason)] if backend in broken_for else []
return pytest.param(backend, marks=marks)
def test_roster_covers_every_supported_backend():
"""A new backend must be classified here rather than silently uncovered."""
assert set(CLOUD_BACKENDS) | _LOCAL_BACKENDS == VectorStore.SUPPORTED_BACKENDS
@pytest.mark.parametrize("backend", CLOUD_BACKENDS)
def test_facade_constructs_an_adapter(backend):
store = _construct(backend)
assert store._backend_store is not None
assert store.backend == backend
@pytest.mark.parametrize(
"backend",
[
_param(b, CLOUD_BACKENDS, "_init_backend_store never connects or selects a collection")
for b in CLOUD_BACKENDS
],
)
def test_backend_is_connected_after_construction(backend):
"""A constructed store should be usable without the caller reaching past
the facade to call connect() and get_collection() itself."""
store = _construct(backend)
assert _live_handle(store._backend_store) is not None
@pytest.mark.parametrize(
"backend",
[
_param(b, _NO_WRITE_DISPATCH, "facade dispatches only to add/add_vectors")
for b in CLOUD_BACKENDS
],
)
def test_store_vectors_dispatch_resolves(backend):
"""store_vectors() should reach the backend's write method."""
store = _construct(backend)
try:
store.store_vectors([np.zeros(3)], [{}], ids=["a"])
except NotImplementedError as exc:
pytest.fail(f"no write dispatch for {backend}: {exc}")
except Exception:
# Any other error means the facade found a write method and the failure
# came from below it, which is the connection gap the test above pins.
# Whether the write succeeds needs a live server, not this test.
pass
def test_milvus_write_dispatch_already_resolves():
"""Control for _NO_WRITE_DISPATCH: if milvus changes, the xfail list is
wrong rather than the feature being broken."""
store = _construct("milvus")
assert hasattr(store._backend_store, "add_vectors")