ci: consolidate workflows, harden security, and clean up .github

This commit is contained in:
KaifAhmad1
2025-11-24 18:26:04 +05:30
parent 841c1c2f88
commit 204e0a92f6
13 changed files with 88 additions and 285 deletions
-78
View File
@@ -1,78 +0,0 @@
# Bot Features for Issues and Discussions
This document describes the minimal, conservative bot features that help users with issues and discussions.
## Issue Helper Bot (`.github/workflows/issue-bot.yml`)
### Features
#### 1. Conservative Auto-labeling
- **Trigger**: When a new issue is opened
- **Action**: Only adds labels for **clear, explicit** cases
- **Labels**:
- `bug` - Only if title starts with "bug:" or "[bug]" OR has multiple bug indicators
- `enhancement` - Only if explicitly mentions "feature request" or title starts with "feature:" or "[feature]"
- `documentation` - Only if explicitly mentions "documentation issue" or title starts with "docs:"
- `security` - Only if explicitly mentions "security issue" or title starts with "security:"
- `needs-triage` - Only if issue body is very short (< 50 chars) and no other labels match
- **Philosophy**: Only labels when it's very clear - avoids false positives
#### 2. Help on Request Only
- **Trigger**: When an issue contains `/help` in title or body
- **Action**: Provides helpful resources and links
- **Note**: Only responds when explicitly requested - no automatic responses
## Discussion Helper Bot (`.github/workflows/discussion-bot.yml`)
### Features
#### 1. Mark as Answered (Manual)
- **Trigger**: When a maintainer comments `/answered` on a discussion
- **Action**: Marks the discussion as answered
- **Note**: Only works for repository members/owners/collaborators
- **Philosophy**: Completely manual - maintainer decides when to mark as answered
## How It Works
Both bots use GitHub Actions workflows that:
- Run only on specific, conservative triggers
- Use GitHub's API for minimal actions (labeling, marking answered)
- Use strict pattern matching - only for very clear cases
- **No automatic responses** - only responds when explicitly requested
## Safety Features
- **Minimal automation**: Only labels very clear cases, no automatic responses
- **No external services**: All bots run using GitHub Actions only
- **Manual control**: Most actions require explicit triggers
- **Conservative matching**: Only acts on very clear patterns
- **Easy to disable**: Can be disabled in repository settings
## Customization
To customize bot behavior:
1. **Modify keywords**: Edit the regex patterns in the workflow files
2. **Change responses**: Update the response text in the scripts
3. **Add new features**: Extend the workflows with additional checks
## Disabling Bots
To temporarily disable a bot:
1. Go to repository Settings → Actions → Workflows
2. Find the bot workflow
3. Click "..." → Disable workflow
Or comment out the workflow file in `.github/workflows/`
## Best Practices
1. **Review bot comments**: Ensure they're helpful and accurate
2. **Update responses**: Keep documentation links current
3. **Monitor behavior**: Check that labels are applied correctly
4. **Community feedback**: Adjust based on user feedback
---
**Note**: These bots are designed to be helpful assistants, not replacements for human interaction. They provide initial guidance, but community members and maintainers provide the real support!
-94
View File
@@ -1,94 +0,0 @@
# CI/CD Workflows Security & Safety Review
## Security Measures Implemented
### ✅ Secrets Management
- **All secrets use GitHub Secrets**: No hardcoded credentials
- **PyPI Token**: Protected via `${{ secrets.PYPI_API_TOKEN }}`
- **GitHub Token**: Uses built-in `${{ secrets.GITHUB_TOKEN }}`
- **Conditional Publishing**: PyPI upload only if token is configured
### ✅ Access Control
- **Repository Scoping**: Workflows only run on specified branches
- **Tag-based Releases**: Only triggered on version tags (`v*`)
- **Branch Protection**: Main branch deployments require proper permissions
### ✅ Error Handling
- **Graceful Degradation**: Missing tests directory doesn't break CI
- **Non-blocking Steps**: Optional steps (coverage, type checking) won't fail entire workflow
- **Clear Messaging**: Informative messages when steps are skipped
## Safety Measures
### ✅ Backward Compatibility
- **Conditional Checks**: All new checks verify existence before running
- **No Breaking Changes**: Existing functionality preserved
- **Optional Features**: New features are additive, not required
### ✅ Project Protection
- **Test Requirements**: Tests still required if they exist
- **Linting Enforcement**: Code quality checks still enforced on source code
- **Type Safety**: Type checking runs but doesn't block (can be made required later)
### ✅ Failure Prevention
- **Directory Checks**: Verifies directories exist before operations
- **File Existence**: Checks for files before processing
- **Dependency Validation**: Handles missing dependencies gracefully
## Workflow Behavior
### Test Job
- ✅ Runs tests if `tests/` directory exists with test files
- ✅ Skips gracefully if no tests found (with informative message)
- ✅ Still fails if tests exist and fail (proper validation)
### Lint Job
- ✅ Always checks `semantica/` source code (required)
- ✅ Conditionally checks `tests/` if it exists
- ✅ Fails if source code doesn't pass linting (enforces quality)
### Type Check Job
- ✅ Runs type checking on source code
- ✅ Non-blocking (won't fail CI) but reports issues
- ✅ Can be made required later by removing `continue-on-error`
### Release Job
- ✅ Only runs on version tags
- ✅ Checks for PyPI token before publishing
- ✅ Gracefully skips if token not configured
## Security Checklist
- [x] No hardcoded secrets
- [x] All secrets use GitHub Secrets
- [x] No sensitive data in logs
- [x] Proper access controls
- [x] Secure token handling
- [x] Conditional publishing based on configuration
- [x] No unauthorized access risks
- [x] Proper error handling without exposing secrets
## Safety Checklist
- [x] Won't break existing functionality
- [x] Backward compatible
- [x] Graceful error handling
- [x] Clear error messages
- [x] Non-destructive operations
- [x] Proper validation before operations
- [x] Safe defaults
## Recommendations
1. **When tests are added**: Remove `continue-on-error` from test step
2. **When ready for production**: Make type checking required
3. **PyPI Publishing**: Configure `PYPI_API_TOKEN` secret when ready
4. **Code Coverage**: Set up Codecov account for coverage tracking
## Notes
- All workflows are safe to merge
- No breaking changes introduced
- Security best practices followed
- Project integrity maintained
+4 -3
View File
@@ -48,13 +48,14 @@ try {
exit 1
}
# Type check with mypy (non-blocking)
# Type check with mypy
Write-Host "🔬 Type checking with mypy..." -ForegroundColor Yellow
try {
mypy semantica/ 2>&1 | Out-Null
mypy semantica/
Write-Host "✅ mypy check passed" -ForegroundColor Green
} catch {
Write-Host "⚠️ Type checking issues found (non-blocking)" -ForegroundColor Yellow
Write-Host " Type checking issues found" -ForegroundColor Red
exit 1
}
Write-Host ""
+4 -3
View File
@@ -46,12 +46,13 @@ else
exit 1
fi
# Type check with mypy (non-blocking)
# Type check with mypy
echo "🔬 Type checking with mypy..."
if mypy semantica/ 2>/dev/null; then
if mypy semantica/; then
echo "✅ mypy check passed"
else
echo "⚠️ Type checking issues found (non-blocking)"
echo " Type checking issues found"
exit 1
fi
echo ""
+32 -11
View File
@@ -1,25 +1,31 @@
name: CI
# This workflow runs tests and code quality checks
# It runs on every push and pull request to main and develop branches
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
# Permissions needed for this workflow
permissions:
contents: read
jobs:
test:
name: Test Python ${{ matrix.python-version }}
# Job 1: Run Tests and Check Coverage
test-and-coverage:
name: Test & Coverage (${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
fail-fast: false # Don't stop other versions if one fails
matrix:
python-version: ['3.10', '3.11', '3.12']
steps:
- uses: actions/checkout@v4
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
@@ -43,19 +49,31 @@ jobs:
run: python -c "import semantica; print(f'Version: {semantica.__version__}')"
- name: Run tests
# Runs pytest only if tests directory exists
run: |
if [ -d "tests" ] && [ "$(find tests -name 'test_*.py' -o -name '*_test.py' | wc -l)" -gt 0 ]; then
pytest
pytest --cov=semantica --cov-report=xml --cov-report=term-missing -v
else
echo "No tests found. Skipping."
fi
continue-on-error: true
lint:
name: Lint Code
- name: Upload coverage to Codecov
# Only upload coverage for Python 3.11 to avoid duplicates
if: matrix.python-version == '3.11'
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false
# Job 2: Check Code Quality (Linting)
quality-checks:
name: Code Quality Checks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
@@ -68,14 +86,17 @@ jobs:
pip install -e ".[dev]"
- name: Check formatting (black)
# Fails if code is not formatted correctly
run: black --check semantica/
- name: Check import sorting (isort)
# Fails if imports are not sorted correctly
run: isort --check-only semantica/
- name: Run linter (flake8)
# Fails if there are syntax errors or style violations
run: flake8 semantica/
- name: Type check (mypy)
run: mypy semantica/ || echo "Type checking has errors (non-blocking)"
continue-on-error: true
# Fails if there are type errors
run: mypy semantica/
+6 -2
View File
@@ -1,7 +1,7 @@
name: Build and Deploy Documentation
# Builds and deploys documentation to GitHub Pages
# Runs on: docs changes to main branch or manual trigger
# This workflow builds the documentation site and deploys it to GitHub Pages
# It runs when changes are pushed to the 'docs' folder on the main branch
on:
push:
@@ -11,11 +11,13 @@ on:
- 'mkdocs.yml'
workflow_dispatch:
# Permissions needed to deploy to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Prevent concurrent deployments
concurrency:
group: "pages"
cancel-in-progress: false
@@ -39,9 +41,11 @@ jobs:
pip install -r requirements-docs.txt
- name: Build documentation
# Builds the static site using MkDocs
run: mkdocs build --strict
- name: Check for broken links
# Optional: checks if any links in the docs are broken
run: |
pip install linkchecker || echo "Skipping link check"
if [ -d "site" ]; then
+9 -2
View File
@@ -1,7 +1,8 @@
name: Format Code
# Automatically formats code with black and isort
# Runs on: pull requests or manual trigger
# This workflow automatically formats code to match project standards
# It runs on pull requests and pushes to main/develop branches
# If it finds formatting issues on a push, it creates a PR to fix them
on:
pull_request:
@@ -10,6 +11,7 @@ on:
branches: [main, develop]
workflow_dispatch:
# Permissions needed to write changes back to the repo
permissions:
contents: write
pull-requests: write
@@ -33,6 +35,7 @@ jobs:
pip install black isort
- name: Check formatting
# On PRs, just check and report issues (don't auto-fix)
if: github.event_name == 'pull_request'
run: |
black --check semantica/ || echo "⚠️ Code formatting issues found. Run: black semantica/"
@@ -40,14 +43,17 @@ jobs:
continue-on-error: true
- name: Format with black
# On pushes, actually run the formatter
if: github.event_name != 'pull_request'
run: black semantica/
- name: Sort imports with isort
# On pushes, actually run the import sorter
if: github.event_name != 'pull_request'
run: isort semantica/
- name: Check for changes
# See if the formatters changed any files
if: github.event_name != 'pull_request'
id: verify-changed-files
run: |
@@ -58,6 +64,7 @@ jobs:
fi
- name: Create Pull Request
# If files changed, create a PR with the fixes
if: github.event_name != 'pull_request' && steps.verify-changed-files.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v5
with:
+5 -2
View File
@@ -1,7 +1,7 @@
name: Auto-label Issues
# Automatically labels new issues based on content
# Provides help resources when requested
# This workflow automatically labels new issues based on their content
# It also provides help resources if the user asks for help
on:
issues:
@@ -9,6 +9,7 @@ on:
issue_comment:
types: [created]
# Permissions needed to add labels and comments
permissions:
contents: read
issues: write
@@ -21,6 +22,7 @@ jobs:
if: github.event.action == 'opened'
steps:
- name: Add labels based on content
# Uses a script to check title/body for keywords like "bug", "feature", etc.
uses: actions/github-script@v7
with:
script: |
@@ -75,6 +77,7 @@ jobs:
help:
name: Provide Help Resources
runs-on: ubuntu-latest
# Runs if the issue contains "/help"
if: |
github.event.action == 'opened' &&
(contains(github.event.issue.body, '/help') || contains(github.event.issue.title, '/help'))
+4 -2
View File
@@ -1,12 +1,13 @@
name: Mark Discussion as Answered
# Marks a discussion as answered when a maintainer comments /answered
# Only works for members, owners, and collaborators
# This workflow allows maintainers to mark a discussion comment as the answer
# Usage: Comment "/answered" on the correct reply
on:
discussion_comment:
types: [created]
# Permissions needed to modify discussions
permissions:
contents: read
discussions: write
@@ -15,6 +16,7 @@ jobs:
mark-answered:
name: Mark Discussion as Answered
runs-on: ubuntu-latest
# Only runs if the comment contains "/answered" and is from a maintainer
if: |
github.event.action == 'created' &&
contains(github.event.comment.body, '/answered') &&
+5 -2
View File
@@ -1,7 +1,7 @@
name: Publish to PyPI
# Publishes package to PyPI when a GitHub release is published
# Requires: PyPI API token configured in repository secrets
# This workflow publishes the package to PyPI (Python Package Index)
# It runs automatically after a GitHub Release is published
on:
release:
@@ -16,6 +16,7 @@ jobs:
name: pypi
url: https://pypi.org/project/semantica/${{ github.event.release.tag_name }}/
# Permissions needed to authenticate with PyPI
permissions:
id-token: write
contents: read
@@ -35,9 +36,11 @@ jobs:
pip install build twine
- name: Build package
# Builds the package again to ensure it's fresh
run: python -m build
- name: Publish to PyPI
# Uses Trusted Publishing (OIDC) to upload to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist/
+6 -2
View File
@@ -1,13 +1,14 @@
name: Create Release
# Creates a GitHub release when a version tag is pushed
# Tag format: v1.2.3
# This workflow creates a GitHub Release when you push a version tag
# Example tag: v1.0.0
on:
push:
tags:
- 'v*'
# Permissions needed to create a release
permissions:
contents: write
id-token: write
@@ -40,12 +41,15 @@ jobs:
echo "Version: $VERSION"
- name: Build package
# Builds the Python package (wheel and source distribution)
run: python -m build
- name: Validate package
# Checks if the package description is valid
run: twine check dist/*
- name: Create GitHub Release
# Creates the release on GitHub and attaches the built files
uses: softprops/action-gh-release@v1
with:
tag_name: ${{ github.ref_name }}
+13 -17
View File
@@ -1,7 +1,7 @@
name: Security Scan
# Scans code and dependencies for security vulnerabilities
# Runs on: push, pull requests, or manual trigger
# This workflow scans for security vulnerabilities
# It runs on pushes, pull requests, and weekly on Mondays
on:
push:
@@ -17,10 +17,10 @@ permissions:
security-events: write
jobs:
# Job 1: Check Dependencies for Vulnerabilities
dependency-scan:
name: Dependency Security Scan
runs-on: ubuntu-latest
continue-on-error: true
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -34,30 +34,25 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install safety pip-audit
continue-on-error: true
- name: Install project dependencies
run: |
pip install -e ".[dev]"
continue-on-error: true
- name: Run pip-audit
# Checks if any installed packages have known vulnerabilities
run: |
echo "Running pip-audit security scan..."
pip-audit --format json --output pip-audit-report.json 2>&1 || true
pip-audit 2>&1 || true
echo "pip-audit scan completed"
continue-on-error: true
pip-audit --format json --output pip-audit-report.json
- name: Run safety check
# Another tool to check for vulnerabilities
run: |
echo "Running safety security check..."
safety check --json --output safety-report.json 2>&1 || true
safety check 2>&1 || true
echo "Safety check completed"
continue-on-error: true
safety check --json --output safety-report.json
- name: Upload security reports
# Save the reports so you can download them later
if: always()
uses: actions/upload-artifact@v4
with:
@@ -66,8 +61,8 @@ jobs:
pip-audit-report.json
safety-report.json
retention-days: 30
if-no-files-found: ignore
# Job 2: Scan Code for Vulnerabilities
code-scan:
name: Code Security Scan
runs-on: ubuntu-latest
@@ -76,6 +71,7 @@ jobs:
uses: actions/checkout@v4
- name: Run Trivy vulnerability scanner
# Scans the file system for vulnerabilities
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
@@ -83,15 +79,15 @@ jobs:
format: 'sarif'
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'
continue-on-error: true
- name: Upload Trivy results to GitHub Security
# Shows results in the "Security" tab of your repo
if: always() && hashFiles('trivy-results.sarif') != ''
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'
continue-on-error: true
# Job 3: Scan for Secrets (Passwords, Keys)
secret-scan:
name: Secret Scanning
runs-on: ubuntu-latest
@@ -102,8 +98,8 @@ jobs:
fetch-depth: 0
- name: Run Gitleaks
# Checks if you accidentally committed passwords or keys
uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
continue-on-error: true
-67
View File
@@ -1,67 +0,0 @@
name: Run Tests
# Runs tests across multiple Python versions
# Runs on: push, pull requests, or manual trigger
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
workflow_dispatch:
permissions:
contents: read
jobs:
test:
name: Test Python ${{ matrix.python-version }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ['3.10', '3.11', '3.12']
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Cache pip packages
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
- name: Verify installation
run: python -c "import semantica; print(f'Version: {semantica.__version__}')"
- name: Run tests
run: |
if [ -d "tests" ] && [ "$(find tests -name 'test_*.py' -o -name '*_test.py' | wc -l)" -gt 0 ]; then
pytest --cov=semantica --cov-report=xml --cov-report=term-missing -v
else
echo "No tests found. Skipping test execution."
exit 0
fi
- name: Upload coverage to Codecov
if: matrix.python-version == '3.11'
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false