Mohd Kaif
d4cd44e7f1
* fix(docker): split explorer-extra.txt by Python version, fix broken build main's container-scan.yml has been failing since PR #1338 merged: ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not: standard-aifc from .../standard_aifc-3.13.0-py3-none-any.whl (from audioread==3.1.0->-r explorer-extra.txt (line 30)) Root cause: explorer-extra.txt was compiled with `--python-version 3.11` but is installed on the Dockerfile's actual python:3.13-slim interpreter. librosa's audioread dependency needs standard-aifc/standard-sunau only under `python_version >= "3.13"` (Python 3.13 dropped aifc/sunau from stdlib) - a file resolved for 3.11 has no hash for those packages at all, so --require-hashes fails outright once pip resolves against the real 3.13 environment instead of silently under-pinning. Splits the file in two: explorer-extra-py311.txt (ci.yml, unchanged resolution) and explorer-extra-py313.txt (Dockerfile, newly compiled for --python-version 3.13). They aren't interchangeable and shouldn't be recombined - documented in .github/requirements/README.md, including how to catch this class of bug before it ships again. * fix(ci): correct stale -o path in explorer-extra-py311.txt header Qodo review on this PR: the autogenerated header comment still said -o .github/requirements/explorer-extra.txt (the pre-rename path), which would silently regenerate the wrong file if someone copy-pasted it.
3.6 KiB
CI tool requirements
Hash-pinned pip install targets for CI/release/Dockerfile steps that install
something other than the project's own audited requirements-ci.txt set.
These exist because OpenSSF Scorecard's Pinned-Dependencies check flags any
pip install in a workflow or Dockerfile that isn't hash-verified, and
requirements-ci.txt alone doesn't cover build/release/security tooling or
the project's own local-source install.
Each .txt was generated from the adjacent .in (or, for explorer-extra-py311.txt,
explorer-extra-py313.txt, and base-deps.txt, from pyproject.toml directly) with:
uv pip compile <input> --python-version 3.11 --python-platform linux \
--constraint requirements-ci.txt --generate-hashes -o <output>.txt
(--constraint requirements-ci.txt is omitted for bootstrap.txt,
build-tools.txt, uv-tool.txt, twine.txt, pip-audit.txt, and
security-scan-tools.txt, since those install standalone tooling with no
version relationship to the project's own dependency tree.)
Regenerate a file the same way after bumping a pinned version, and re-run it
whenever requirements-ci.txt changes if the file used --constraint (see
each file's own autogenerated header comment for its exact command).
| File | Used by | Installs |
|---|---|---|
bootstrap.txt |
security.yml, security-scan.yml, benchmark.yml | pip, setuptools (upgrade before anything else) |
pep517-build.txt |
ci.yml, benchmark.yml, Dockerfile | exact [build-system] requires from pyproject.toml (setuptools, wheel) - installed with --no-build-isolation before any pip install -e . / pip install ., since --no-deps alone doesn't stop pip's PEP 517 build isolation from fetching those two unhashed |
explorer-extra-py311.txt |
ci.yml | semantica's base deps + the explorer extra, resolved for python 3.11 |
explorer-extra-py313.txt |
Dockerfile | the same, resolved for python 3.13 (the image's actual interpreter) |
pytest-tool.txt |
ci.yml | pytest, for the pre-all-extras deterministic test |
uv-tool.txt |
ci.yml | uv, to verify requirements-ci.txt is current |
build-tools.txt |
ci.yml, release.yml | build, wheel |
twine.txt |
release.yml | twine |
pip-audit.txt |
security.yml | pip-audit |
security-scan-tools.txt |
security-scan.yml | safety, bandit, semgrep, jq |
base-deps.txt |
benchmark.yml | semantica's base deps (no extras) |
benchmark-extra.txt |
benchmark.yml | the benchmark-only libs (neo4j, pdfplumber, etc.) |
explorer-extra-py31{1,3}.txt and base-deps.txt are large (they mirror
most of requirements-ci.txt) because semantica's dependencies list in
pyproject.toml isn't extras-gated - installing the package at all pulls
the full base set. That's expected, not a mistake.
explorer-extra-py311.txt and explorer-extra-py313.txt are not
interchangeable, and can't be collapsed into one file compiled for either
version: librosa's audioread dependency needs standard-aifc /
standard-sunau only under python_version >= "3.13" (Python 3.13 dropped
aifc/sunau from stdlib). A file resolved for 3.11 simply omits those
packages' hashes, so installing it with --require-hashes on a real 3.13
interpreter (the Dockerfile's base image) fails outright rather than
silently under-pinning. Any other file shared across a 3.11 and 3.13
consumer would need the same split if it hits a similar stdlib-removal
edge case - check for ERROR: In --require-hashes mode, all requirements must have their versions pinned on the other Python version before
assuming one --python-version covers every consumer.