Zohaib Hassnain
86ffa05dd4
- Reduce direct core dependencies in pyproject.toml from 44 to 22 - Move heavy and specialized packages into modular optional extras: * models-huggingface: torch, transformers * embeddings-local: sentence-transformers, fastembed, onnxruntime, tokenizers * nlp-spacy: spacy, thinc * viz: matplotlib, seaborn, plotly, ipywidgets, umap-learn (expanded) * media: librosa, opencv-python * vectorstore-faiss: faiss-cpu * documents: python-docx, openpyxl, lxml, beautifulsoup4 * ingest-git: GitPython * graph-embeddings: gensim - Update semantica[all] and semantica[vectorstore-all] to encompass all extras - Ensure lazy parser construction (DOCXParser, ExcelParser, HTMLParser, XMLParser) without error on __init__(), failing only inside .parse() with clear hints - Add stdlib xml.etree fallback in XMLParser when lxml is missing - Guard unguarded matplotlib imports in EmbeddingVisualizer and OntologyVisualizer - Standardize user-facing error messages to point to pip install 'semantica[extra]' - Bump version to 0.7.0 in pyproject.toml, semantica/__init__.py, and CITATION.cff - Add migration notes to README.md and CHANGELOG.md - Recompile CI and Docker requirements lockfiles - Add dedicated test suite tests/test_issue_1513_slim_core.py
CI tool requirements
Hash-pinned pip install targets for CI/release/Dockerfile steps that install
something other than the project's own audited requirements-ci.txt set.
These exist because OpenSSF Scorecard's Pinned-Dependencies check flags any
pip install in a workflow or Dockerfile that isn't hash-verified, and
requirements-ci.txt alone doesn't cover build/release/security tooling or
the project's own local-source install.
Each .txt was generated from the adjacent .in (or, for explorer-extra-py311.txt,
explorer-extra-py313.txt, and base-deps.txt, from pyproject.toml directly) with:
uv pip compile <input> --python-version 3.11 --python-platform linux \
--constraint requirements-ci.txt --generate-hashes -o <output>.txt
(--constraint requirements-ci.txt is omitted for bootstrap.txt,
build-tools.txt, uv-tool.txt, twine.txt, pip-audit.txt, and
security-scan-tools.txt, since those install standalone tooling with no
version relationship to the project's own dependency tree.)
Regenerate a file the same way after bumping a pinned version, and re-run it
whenever requirements-ci.txt changes if the file used --constraint (see
each file's own autogenerated header comment for its exact command).
| File | Used by | Installs |
|---|---|---|
bootstrap.txt |
security-scan.yml, benchmark.yml | pip, setuptools (upgrade before anything else) |
pep517-build.txt |
ci.yml, benchmark.yml, Dockerfile | exact [build-system] requires from pyproject.toml (setuptools, wheel) - installed with --no-build-isolation before any pip install -e . / pip install ., since --no-deps alone doesn't stop pip's PEP 517 build isolation from fetching those two unhashed |
explorer-extra-py311.txt |
ci.yml | semantica's base deps + the explorer extra, resolved for python 3.11 |
explorer-extra-py313.txt |
Dockerfile | the same, resolved for python 3.13 (the image's actual interpreter) |
pytest-tool.txt |
ci.yml | pytest, for the pre-all-extras deterministic test |
uv-tool.txt |
ci.yml | uv, to verify requirements-ci.txt is current |
build-tools.txt |
ci.yml, release.yml | build, wheel |
twine.txt |
release.yml | twine |
pip-audit.txt |
security-scan.yml | pip-audit |
security-scan-tools.txt |
security-scan.yml | bandit, semgrep, jq |
base-deps.txt |
benchmark.yml | semantica's base deps (no extras) |
benchmark-extra.txt |
benchmark.yml | the benchmark-only libs (neo4j, pdfplumber, etc.) |
explorer-extra-py31{1,3}.txt and base-deps.txt are large (they mirror
most of requirements-ci.txt) because semantica's dependencies list in
pyproject.toml isn't extras-gated - installing the package at all pulls
the full base set. That's expected, not a mistake.
explorer-extra-py311.txt and explorer-extra-py313.txt are not
interchangeable, and can't be collapsed into one file compiled for either
version: librosa's audioread dependency needs standard-aifc /
standard-sunau only under python_version >= "3.13" (Python 3.13 dropped
aifc/sunau from stdlib). A file resolved for 3.11 simply omits those
packages' hashes, so installing it with --require-hashes on a real 3.13
interpreter (the Dockerfile's base image) fails outright rather than
silently under-pinning. Any other file shared across a 3.11 and 3.13
consumer would need the same split if it hits a similar stdlib-removal
edge case - check for ERROR: In --require-hashes mode, all requirements must have their versions pinned on the other Python version before
assuming one --python-version covers every consumer.