diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml index 9f093bc7..5fb86cde 100644 --- a/.github/workflows/security.yml +++ b/.github/workflows/security.yml @@ -4,6 +4,11 @@ on: schedule: - cron: '0 0 * * 1' workflow_dispatch: + pull_request: + branches: [main] + paths: + - 'pyproject.toml' + - '.github/workflows/security.yml' permissions: contents: read @@ -16,6 +21,23 @@ jobs: - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7 with: python-version: '3.11' + # Upgrade first: actions/setup-python's baked-in setuptools has been + # behind known-vulnerable floors before (e.g. PYSEC-2026-3447 / + # CVE-2026-59890, fixed in 83.0.0) regardless of what this project's + # own [build-system] requires -- that only governs isolated build + # environments, not the ambient one pip-audit scans here. + - run: python -m pip install --upgrade pip setuptools - run: pip install pip-audit + # Install the [all] extra so pip-audit sees every optional dependency + # group (fastapi, python-multipart, etc.), not just pip-audit's own + # deps. PYSEC-2024-38 (#869) shipped in the first place because + # neither this job (bare env, no extras) nor security-scan.yml's + # Safety check (installs only [llm-litellm]) ever had fastapi or + # python-multipart installed to look at. + - run: pip install -e ".[all]" + # PR runs gate on findings, since they're scoped to actual + # pyproject.toml changes under review. The schedule/workflow_dispatch + # runs stay non-blocking until a full pass over pre-existing findings + # across the whole [all] tree has been done. - run: pip-audit - continue-on-error: true + continue-on-error: ${{ github.event_name != 'pull_request' }} diff --git a/CHANGELOG.md b/CHANGELOG.md index c43ab8ab..74333a4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Fixed along the way**: `MilvusStore`'s metadata expression builder rendered `NaN`/`Infinity` filter values as bare unquoted tokens, producing an invalid Milvus expression whose server-side rejection was then swallowed by a broad `except`, indistinguishable from "no matches"; these values are now rejected up front with a clear `ValidationError` - New/expanded test coverage in `tests/vector_store/test_backend_metadata_filtering.py` (all 7 backends, including the Pinecone dimension/zero-vector, PgVector boolean-list, FAISS `limit=0`, and Milvus `NaN` regressions) and `tests/vector_store/test_sqlite_vec_store.py` (new `TestSQLiteVecStoreFilterByMetadata`, run against the real `sqlite-vec` extension, including the array-vs-scalar intersection case) +### Security + +- **`fastapi`/`python-multipart` floors in the `explorer` extra allowed PYSEC-2024-38 (CVE-2024-24762 / GHSA-2jv5-9r88-3w3p, `python-multipart` ReDoS)** (#871, closes #869) by @agu2347 + - `explorer` declared `fastapi>=0.100.0` and `python-multipart>=0.0.6`; both floors resolve to versions carrying a ReDoS in `python-multipart`'s `Content-Type` header option parser (`parse_options_header`), reachable by any endpoint that accepts form/multipart data — an attacker-crafted header option can stall the event loop for minutes + - **Corrected during review**: the original fix raised only `fastapi>=0.109.1`, leaving `python-multipart>=0.0.6` unchanged. `python-multipart` is declared as its own direct dependency in the `explorer` extra rather than pulled in transitively via `fastapi[all]`, so a bare `fastapi` install enforces no `python-multipart` floor at all — the vulnerable `0.0.6` could still resolve with `fastapi>=0.109.1` in place. Floors raised to `fastapi>=0.109.2` / `python-multipart>=0.0.7`, the first versions of each that exclude the vulnerable range + - **Fixed along the way**: the `Security` workflow's `pip-audit` job ran only on a weekly schedule with `continue-on-error: true`, against a bare Python environment with none of Semantica's optional extras installed — it would never have seen `fastapi`/`python-multipart` regardless of which floor was pinned. `security-scan.yml`'s Safety check has the same blind spot (`pip install -e ".[llm-litellm]"` only, never `[explorer]`). `pip-audit` now also runs on `pull_request` when `pyproject.toml` changes, installs `semantica[all]`, and fails the build on any finding for that trigger; the schedule/`workflow_dispatch` runs stay non-blocking pending a full pass over any pre-existing findings across the whole `[all]` tree + - **Caught by the new gate on its first run**: `python -m pip install -e ".[all]"` pulled in `setuptools==79.0.1`, vulnerable to CVE-2026-59890/GHSA-h35f-9h28-mq5c/PYSEC-2026-3447 (Unicode-normalization bypass of `MANIFEST.in` exclude/prune patterns on macOS APFS/HFS+, letting excluded files leak into a built sdist), fixed in `83.0.0`. `[build-system] requires` had the exact same too-permissive-floor pattern this whole entry is about (`setuptools>=61.0`), and `actions/setup-python`'s baked-in `setuptools` isn't governed by that pin at all since it's outside any isolated build. Bumped `[build-system] requires` to `setuptools>=83.0.0`, and the `Security` workflow now runs `pip install --upgrade pip setuptools` before auditing so the scanned environment can't have a stale ambient copy regardless of what governs it + - Full `explorer` suite: 241 passed + ## [0.6.5] - 2026-08-11 ### Added diff --git a/pyproject.toml b/pyproject.toml index 7a65ae3c..d7fc2899 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,5 +1,5 @@ [build-system] -requires = ["setuptools>=61.0", "wheel"] +requires = ["setuptools>=83.0.0", "wheel"] build-backend = "setuptools.build_meta" [project] @@ -230,10 +230,10 @@ dev = [ # Explorer Dashboard explorer = [ - "fastapi>=0.100.0", + "fastapi>=0.109.2", "uvicorn[standard]>=0.22.0", "websockets>=15.0.1", - "python-multipart>=0.0.6", + "python-multipart>=0.0.7", "defusedxml>=0.7.1" ] explorer-lite = [