mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-09-04 04:01:07 +00:00
Mohd Kaif
d135ad185f
* fix(ci): drop unpinnable benchmarks/requirements.txt install Scorecard flagged this pip install as unpinned-by-hash (#6082). Can't hash-pin it - benchmarks/requirements.txt doesn't exist in this repo, so there's nothing to compile a lockfile from. Dropping it instead of leaving it unpinned: the job already fails on the next real step (benchmarks/benchmarks_runner.py, also missing), so this line wasn't doing anything useful to begin with. * fix(ci): hash-pin the spacy model download in benchmark.yml Qodo review on this PR: dropping the benchmarks/requirements.txt install (the previous failure point) let the job actually reach `python -m spacy download en_core_web_sm`, which fetches an unpinned, unhashed wheel from spacy-models' GitHub releases - undoing the point of this PR by exposing a real unpinned-install path instead of a dead one. Replaced with a hash-pinned direct-URL entry in benchmark-extra.in/.txt for en_core_web_sm-3.8.0 (matches the spacy==3.8.15 already pinned in base-deps.txt). uv independently computed the same sha256 I got via a manual curl+sha256 of the release asset, and a --require-hashes dry-run install verifies clean.
78 lines
3.3 KiB
YAML
78 lines
3.3 KiB
YAML
name: Semantica Performance Suite
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
performance-test:
|
|
name: Benchmark Runner (Ubuntu/Python 3.12)
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- name: Checkout Code
|
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Set up Python 3.11
|
|
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
|
|
with:
|
|
python-version: "3.11"
|
|
cache: 'pip'
|
|
|
|
- name: Install Dependencies
|
|
env:
|
|
|
|
BENCHMARK_REAL_LIBS: "1"
|
|
run: |
|
|
pip install -r .github/requirements/bootstrap.txt --require-hashes
|
|
# --no-deps + a hash-pinned install of the same base dependency set
|
|
# (rather than a bare `pip install -e .`) so every fetched package
|
|
# is hash-verified (Scorecard Pinned-Dependencies); the local
|
|
# editable install itself has nothing to hash.
|
|
#
|
|
# --no-deps only skips *runtime* dependency resolution - `-e .`
|
|
# still does a PEP 517 build, which by default creates an isolated
|
|
# build env and fetches [build-system] requires (setuptools,
|
|
# wheel) completely outside any hash checking. Install
|
|
# pep517-build.txt (pins that exact build-system.requires) first
|
|
# and pass --no-build-isolation so pip reuses those hash-verified
|
|
# copies instead of fetching its own.
|
|
pip install -r .github/requirements/pep517-build.txt --require-hashes
|
|
pip install --no-deps --no-build-isolation -e .
|
|
pip install -r .github/requirements/base-deps.txt --require-hashes
|
|
# NOTE: benchmarks/ does not currently exist in this repo (neither
|
|
# requirements.txt nor benchmarks_runner.py below), so this job
|
|
# already fails on any real invocation - pre-existing, unrelated to
|
|
# this pinning change. The `pip install -r benchmarks/requirements.txt`
|
|
# step that used to be here is dropped rather than fixed: there's
|
|
# nothing to hash-pin without knowing what that file should
|
|
# contain, and an unpinned install here would just re-trip
|
|
# Scorecard's Pinned-Dependencies check for no real benefit, since
|
|
# the job can't run to completion regardless.
|
|
#
|
|
# `python -m spacy download en_core_web_sm` fetches an unpinned,
|
|
# unhashed wheel from spacy-models' GitHub releases - replaced with
|
|
# a hash-pinned direct-URL install of the same 3.8.0 model (matches
|
|
# the spacy==3.8.15 pinned in base-deps.txt) via benchmark-extra.txt.
|
|
pip install -r .github/requirements/benchmark-extra.txt --require-hashes
|
|
|
|
- name: Execute Benchmarks (Real Mode)
|
|
env:
|
|
BENCHMARK_REAL_LIBS: "1"
|
|
run: |
|
|
python benchmarks/benchmarks_runner.py
|
|
# Optional: Compare to baseline (requires previous run artifact)
|
|
# pytest-benchmark --storage file://benchmarks/results --benchmark-compare
|
|
|
|
- name: Upload Benchmark Results
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
if: always()
|
|
with:
|
|
name: benchmark-report-${{ github.run_id }}
|
|
path: benchmarks/results
|
|
retention-days: 30
|