mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-09-08 04:00:15 +00:00
Adds Google ADK (Agent Development Kit) support to Semantica. `semantica_kg_tools()` and `semantica_decision_tools()` expose entity/relation extraction, graph updates, and decision recording as ADK `FunctionTool`s. `SemanticaSessionService` implements ADK session storage on top of a Semantica `ContextGraph`, so session state, events, and knowledge graph data can live in the same graph instead of keeping sessions in memory. There were also a number of dependency and CI fixes needed to get the integration working reliably. `google-adk` is pinned to a range that avoids the CI `websockets` conflict, the deprecated `pinecone-client` dependency was replaced with `pinecone`, Windows-only dependencies now have the appropriate platform markers, and `requirements-ci.txt` was regenerated to match. A `pip-audit` pass also required updates to `google-adk` and `starlette` for known CVEs. Some unrelated `pyproject.toml` changes had slipped in during rebases, so the previous version, dependency bounds, `ingest-sap`/LangChain entries, and package-data settings were restored. A few bugs in the initial ADK implementation were fixed during review: * `extract_relations()` was calling `RelationExtractor.extract_entities()`, which doesn't exist on that extractor. The failure was being caught and returned in the tool's `error` field, leaving callers with an empty relation list. It now calls the correct extraction path. * The repo's top-level `mcp/` package shadowed the third-party `mcp` package imported by `google.adk`, causing `google.adk` imports to fail from a normal repo checkout. The local package was moved to `semantica_mcp/mcp/`. The MCP move needed a follow-up as well. `semantica/cli.py` and four existing tests were still importing from `mcp.*`, and the modules under `semantica_mcp/mcp/` still used the old absolute imports internally. `semantica_mcp` was also missing from the setuptools package include list and had no `__init__.py`, so it wouldn't have been included in an installed package. Those imports and packaging settings are fixed now. The session service and ADK tools also had a few other problems: * `list_sessions()` returned a plain list instead of ADK's `ListSessionsResponse`. The original import for that type doesn't work against the installed `google-adk` package, so it was silently falling back to a stub. `user_id` was also incorrectly required instead of being optional. * Session node IDs were built by joining `app_name`, `user_id`, and `session_id` with unescaped colons, which allowed different identities to produce the same graph node ID. Each component is now encoded before joining. * `kg_tools.py` and `decision_tools.py` each had their own lock registry and default graph instance. Sharing a graph between the two modules therefore didn't share the lock, and using both factories without an explicit graph produced two different defaults. The shared state now lives in one module used by both. * `add_to_graph` had a `TypeError` compatibility fallback that couldn't succeed with the current `RelationExtractor` API and could hide the original extraction error. That fallback was removed. * `append_event` persisted partial streaming events even though ADK's base session service skips them. * `get_session()` ignored its `config` argument, so `num_recent_events` and `after_timestamp` had no effect. * The async session-service methods performed synchronous graph scans while holding a `threading.RLock` on the event loop thread. That work now runs in worker threads with `asyncio.to_thread()` so a slow or contended graph operation doesn't block the loop. --- Co-authored-by: Zohaib Hassnain [109234410+ZohaibHassan16@users.noreply.github.com](mailto:109234410+ZohaibHassan16@users.noreply.github.com)
173 lines
7.8 KiB
YAML
173 lines
7.8 KiB
YAML
name: CI
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths-ignore:
|
|
- 'docs/**'
|
|
- 'docs_check.py'
|
|
- '**/*.md'
|
|
pull_request:
|
|
branches: [main]
|
|
|
|
jobs:
|
|
# Detect whether this PR touches any source files (non-docs/non-markdown).
|
|
# The result drives the `build` job's `if:` condition so that:
|
|
# - docs-only PRs: `build` is skipped (satisfies the required check).
|
|
# - code PRs: `build` runs exactly as before.
|
|
# Push events (to main) keep their own paths-ignore above and never reach
|
|
# this job, so the push optimization is unaffected.
|
|
changes:
|
|
runs-on: ubuntu-latest
|
|
# Only needed for pull_request events; push events are pre-filtered above.
|
|
if: github.event_name == 'pull_request'
|
|
outputs:
|
|
src: ${{ steps.filter.outputs.src }}
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
# Fetch enough history to compute the merge base against the PR base.
|
|
fetch-depth: 0
|
|
- name: Check for source changes
|
|
id: filter
|
|
run: |
|
|
# List files changed in this PR relative to the true merge base.
|
|
# Using three-dot merge-base diff so changes on the base branch that
|
|
# are not part of this PR do not appear in the file list.
|
|
# If every changed file matches docs/** or *.md (any depth) or
|
|
# docs_check.py, this is a docs-only PR and src=false; otherwise
|
|
# src=true.
|
|
BASE="${{ github.event.pull_request.base.sha }}"
|
|
HEAD="${{ github.event.pull_request.head.sha }}"
|
|
MERGE_BASE=$(git merge-base "$BASE" "$HEAD")
|
|
CHANGED=$(git diff --name-only "$MERGE_BASE" "$HEAD")
|
|
echo "Changed files:"
|
|
echo "$CHANGED"
|
|
NON_DOCS=$(echo "$CHANGED" | grep -Ev '^(docs/|docs_check\.py|.*\.md$)' || true)
|
|
if [ -n "$NON_DOCS" ]; then
|
|
echo "src=true" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "src=false" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
build:
|
|
needs: [changes]
|
|
# For pull_request events:
|
|
# - skip only when changes ran successfully and explicitly set src=false
|
|
# (i.e. a confirmed docs-only PR).
|
|
# - run when changes succeeded with src=true (source changes present).
|
|
# - run when changes failed or was cancelled (fail-closed: missing output
|
|
# must not silently skip the build).
|
|
# For push/non-PR events: changes is skipped; always() prevents the build
|
|
# from being skipped due to a skipped needs dependency.
|
|
if: >-
|
|
always() && (
|
|
github.event_name != 'pull_request' ||
|
|
needs.changes.result != 'success' ||
|
|
needs.changes.outputs.src == 'true'
|
|
)
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
|
|
with:
|
|
python-version: '3.11'
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: explorer/package-lock.json
|
|
- name: Install Explorer frontend dependencies
|
|
working-directory: explorer
|
|
run: npm ci
|
|
- name: Install Playwright Chromium
|
|
working-directory: explorer
|
|
run: npx playwright install --with-deps chromium
|
|
- name: Test Explorer frontend
|
|
working-directory: explorer
|
|
run: |
|
|
npm run test:graph-store
|
|
npm run test:graph-workspace
|
|
npm run test:plugin-registry
|
|
npm run test:deterministic-e2e
|
|
- name: Build Explorer frontend
|
|
working-directory: explorer
|
|
run: npm run build
|
|
- name: Install Explorer backend test dependencies
|
|
run: |
|
|
# Run the deterministic backend path before the all-extras CI
|
|
# environment is installed. The Explorer extra supplies the
|
|
# production API dependencies without importing optional vector
|
|
# providers such as Pinecone during test collection.
|
|
#
|
|
# --no-deps + a separate hash-pinned install (rather than the old
|
|
# `pip install -e ".[explorer]" pytest==9.1.1`) so every fetched
|
|
# package is hash-verified (Scorecard Pinned-Dependencies); the
|
|
# local editable install itself has nothing to hash.
|
|
# .github/requirements/explorer-extra-py311.txt is
|
|
# `uv pip compile pyproject.toml --extra explorer --python-version 3.11 --constraint requirements-ci.txt --generate-hashes`
|
|
# - regenerate it the same way if pyproject.toml's base/explorer
|
|
# deps change. Resolved specifically for this job's python 3.11
|
|
# (see the Dockerfile's explorer-extra-py313.txt for why this
|
|
# can't be shared with python 3.13: audioread needs extra
|
|
# standard-aifc/standard-sunau hashes only on 3.13+).
|
|
#
|
|
# --no-deps only skips *runtime* dependency resolution - `-e .`
|
|
# still does a PEP 517 build, which by default creates an isolated
|
|
# build env and fetches [build-system] requires (setuptools,
|
|
# wheel) completely outside any hash checking. Install
|
|
# pep517-build.txt (pins that exact build-system.requires) first
|
|
# and pass --no-build-isolation so pip reuses those hash-verified
|
|
# copies instead of fetching its own.
|
|
pip install -r .github/requirements/pep517-build.txt --require-hashes
|
|
pip install --no-deps --no-build-isolation -e .
|
|
pip install -r .github/requirements/explorer-extra-py311.txt --require-hashes
|
|
pip install -r .github/requirements/pytest-tool.txt --require-hashes
|
|
- name: Test deterministic Explorer backend path
|
|
run: |
|
|
pytest -q tests/explorer/test_explorer_deterministic_rendering_e2e.py
|
|
- name: Install pinned Python dependencies
|
|
run: |
|
|
pip install -r requirements-ci.txt --require-hashes
|
|
- name: Verify requirements-ci.txt is up to date
|
|
run: |
|
|
pip install -r .github/requirements/uv-tool.txt --require-hashes
|
|
# Re-resolve with the committed file as a constraint: upstream package
|
|
# releases must NOT fail CI (deps only change when pyproject.toml
|
|
# changes intentionally). Compare only version lines (pkg==ver),
|
|
# ignoring the -c constraint comments and the `\` line continuations
|
|
# that --generate-hashes emits.
|
|
uv pip compile pyproject.toml --python-version 3.11 --extra all \
|
|
--constraint requirements-ci.txt -o /tmp/requirements-ci-check.txt
|
|
diff \
|
|
<(grep -E '^[a-zA-Z0-9._-]+==' requirements-ci.txt | sed 's/ \\$//') \
|
|
<(grep -E '^[a-zA-Z0-9._-]+==' /tmp/requirements-ci-check.txt)
|
|
# build is a dev-time dependency; wheel is build-time only (neither is
|
|
# in requirements-ci.txt) — install the same pinned versions
|
|
# [build-system] declares so --no-isolation works below.
|
|
- run: pip install -r .github/requirements/build-tools.txt --require-hashes
|
|
- name: Build package (no isolation — pinned deps)
|
|
run: python -m build --no-isolation
|
|
- name: Verify Explorer frontend is packaged
|
|
run: |
|
|
python - <<'PY'
|
|
import zipfile
|
|
from pathlib import Path
|
|
|
|
wheels = list(Path("dist").glob("*.whl"))
|
|
assert wheels, "No wheel was built"
|
|
|
|
with zipfile.ZipFile(wheels[0]) as wheel:
|
|
names = set(wheel.namelist())
|
|
|
|
assert "semantica/static/index.html" in names, "Explorer index.html missing from wheel"
|
|
assert any(name.startswith("semantica/static/assets/") for name in names), "Explorer assets missing from wheel"
|
|
|
|
print("Explorer frontend is packaged")
|
|
PY
|
|
- name: Run Google ADK Integration Tests
|
|
run: pytest tests/integrations/google_adk/
|