semantica

mengzhx/semantica

Fork 0

mirror of https://github.com/semantica-agi/semantica.git synced 2026-08-29 04:26:20 +00:00

Commit Graph

Author	SHA1	Message	Date
Shinde vinayak rao patil	d94d8f6ab8	Feat/crewai integration (#988 ) * feat(crewai): add first-class CrewAI integration (#962) Add native CrewAI support so Crew agents can share a ContextGraph and AgentContext via BaseTool subclasses and a BaseKnowledgeSource, matching the existing agno integration pattern. - SemanticaKGTool: 5 KG actions (extract_entities, extract_relations, add_to_graph, query_graph, find_related) with sync run()/async arun() - SemanticaDecisionTool: 5 decision-intelligence actions (record_decision, find_precedents, trace_causal_chain, analyze_impact, check_policy) over AgentContext - SemanticaKnowledgeSource: serializes a ContextGraph into crew knowledge storage; bridges legacy load_content() and current validate_content()/aadd() contracts for crewai>=0.80.0 - All classes degrade gracefully when crewai is absent - New pip extra crewai=... included in the all bundle - 70 new tests (stub-based present-case + subprocess degradation path) - Docs: integrations/crewai.md, docs.json nav, README matrix updates * fix(crewai): harden tools against real Semantica dataclass shapes (#962) Bugs found during live testing with crewai 1.15.16: - SemanticaKGTool.add_to_graph crashed on real Entity/Relation dataclasses ('str' object has no attribute 'end_char'): string names were passed to extract_relations(entities=...), which requires Entity objects, and the tool read .name/.source/.target instead of Entity's .text/.label and Relation's .subject/.object. Add shape-agnostic field helpers. - SemanticaDecisionTool() created an AgentContext without a knowledge_graph, so _decision_backend was never set and record_decision raised 'Decision tracking is not enabled'. Wire in a ContextGraph. - record_decision hard-failed when the agent omitted optional fields; fall back to category='general', reasoning='agent decision', outcome='recorded'. Add tests covering real Entity/Relation dataclass shapes and the live auto-created AgentContext path (now 77 crewai tests, 212 total). * fix(crewai): make find_related traverse edges undirected (#962) ContextGraph.get_neighbors only follows outgoing edges, so a node whose only edge is incoming (A -> B) reported no related concepts. Rebuild a bidirectional adjacency from find_edges() in SemanticaKGTool._find_related so 'related' honors both directions. * fix(crewai): harden tools for checkpoint serialization and correct action semantics (#962) - Exclude live graph/context/extractor state from JSON serialization (model_dump(mode="json")) so CrewAI checkpointing no longer raises PydanticSerializationError; model_post_init self-heals defaults on restore - query_graph now searches node content via graph.query() plus id/type - trace_causal_chain returns an explicit error when causal tracing is unavailable instead of substituting similarity precedents; call trace_decision_causality(..., max_depth=...) with the correct kwarg name - find_precedents propagates max_precedents/limit to the backend instead of being silently capped at 10 - Serialize add_to_graph batches under a module lock to prevent concurrent double-counting; skip nameless entities instead of creating repr()-junk nodes - aadd() runs CPU-bound serialization in a thread executor - Mirror crewai args_schema serialize/restore in the conftest stub and add serialization regression tests (crewai: 92 tests) * fix(crewai): correct check_policy coercion, guard causal tracing, and harden concurrency (#962) - _eval_rule now coerces rule values type-aware: bool("false") was truthy, so 'enabled == false' reported a violation for enabled=false, and string datums like "0.90" were compared lexicographically instead of numerically - _trace_causal_chain no longer raises AttributeError (which escaped _run) when the decision context lacks knowledge_graph; returns honest error JSON - SemanticaKnowledgeSource storage failures log an actionable ERROR; without a configured crew embedder agents previously retrieved nothing silently - add_to_graph uses a per-graph re-entrant lock (WeakKeyDictionary) instead of a process-global one: independent graphs no longer serialize each other and re-entrant extractor callbacks cannot deadlock - entity/relation confidence=None normalizes to 1.0 instead of failing the whole extraction with float(None) - add subprocess integration test against real crewai covering Crew-level serialization round-trip and checkpoint restore (stub tests cannot see it) - docs: embedder requirement for SemanticaKnowledgeSource; resume contract note * fix(crewai): surface knowledge-source save failures at ERROR when storage is wired (#962) Re-verification against real crewai showed the embedder-missing failure raises ValueError even though storage IS wired, so the old except-ValueError branch mislabeled it as 'storage not wired' and logged DEBUG — hiding the failure. Distinguish by storage presence instead of exception type: storage is None -> DEBUG keep-in-memory (legitimate standalone use); storage wired but save() raises -> actionable ERROR. Add regression test mirroring real crewai's ValueError-on-missing-embedder behavior. * fix(crewai): expose run()/arun() entry points in degraded mode (#962) The public crewai contract is run()/arun(); without crewai installed they were missing (only the private _run existed), so the documented 'usable without crewai' path raised AttributeError at the entry point. Define them in degraded mode only, leaving crewai's BaseTool implementations untouched when present. Extend the degradation subprocess test to exercise run() and arun(). * fix(crewai): standardize query shape, field-name rules, and restore-state flag - _query_graph: id/type matches now return the same schema as content matches (id/type/label/content/score) instead of a bare list - _eval_rule: non-greedy field capture so hyphen/dot/space JSON keys (e.g. "risk-score >= 0.9") are addressable in policy rules - add had_live_state/reconstructed_state so checkpoint-restored tools and knowledge sources signal that their live graph/context was lost and an empty one reconstructed; knowledge source no longer hides the loss by eagerly rebuilding its graph inside __init__ (pydantic calls __init__ during model_validate) * fix(crewai): address Qodo review — confidence errors, string trim, holistic availability - record_decision: stop calling float() in _run, so malformed confidence values surface as JSON errors (via _record_decision's handling) instead of crashing the tool - _coerce_value: return the stripped string for non-numeric literals so whitespace-padded decision_data fields match policy rules - centralize crewai availability in _availability.py so the exported CREWAI_AVAILABLE flag is holistic across tools and knowledge source (previously each module probed crewai independently and the package flag came from decision_tool only) * ci: regenerate requirements-ci.txt for the crewai extra The crewai extra in pyproject.toml brings in crewai, crewai-tools and transitive deps (chromadb, lancedb, ...). Recompile with uv==0.12.1 per CONTRIBUTING.md so the CI staleness check passes. * ci: keep crewai out of the locked CI dependency set crewai (all versions) hard-requires chromadb~=1.1.0, which carries a pre-authentication code-injection advisory (CVE-2026-45829 / GHSA-f4j7-r4q5-qw2c) with NO fixed release — even the latest 1.5.9 is affected. Keeping crewai in the 'all' extra failed pip-audit and the safety check on requirements-ci.txt. - drop crewai from the 'all' aggregate (standalone semantica[crewai] extra is unchanged and still installs crewai) - stop listing crewai-tools in the extra: the integration only uses crewai core (BaseTool, BaseKnowledgeSource) and crewai-tools pulled extra transitive deps - regenerate requirements-ci.txt: OSV/pip-audit 0 vulnerabilities, safety 0 vulnerabilities, staleness check matches * docs(crewai): document crewai extra scope and chromadb CVE-2026-45829 - CHANGELOG: extra is crewai>=0.80.0 only (no crewai-tools) and is not part of the 'all' bundle, with the chromadb CVE-2026-45829 reason - integrations/crewai/README.md: add a security warning that installing the extra pulls chromadb~=1.1.0, which is affected by the unpatched pre-auth code-injection CVE-2026-45829 ---------	2026-08-16 11:15:43 +05:00
Yunare MaiaandZohaib Hassnain	4513b61e40	ci: pin Python dependencies in requirements-ci.txt for reproducible CI (#945 ) * ci: pin Python dependencies in requirements-ci.txt for reproducible CI Adds a committed lockfile pinning all transitive dependencies at exact versions (uv pip compile, Python 3.11, all extras — 1581 lines), the Python equivalent of explorer/package-lock.json + npm ci. - CI installs from requirements-ci.txt before building the wheel - CI verifies the lockfile is byte-identical to a fresh compile (fails on staleness after pyproject.toml changes) - CONTRIBUTING documents the regeneration command Closes #938 Signed-off-by: Yunare Maia <yunare@gmail.com> * ci: address Qodo review — security scans use pinned deps, exclude gpu extras - security-scan.yml installs from requirements-ci.txt instead of "./[llm-litellm]" so Safety scans the exact CI/release dependency tree - security.yml runs pip-audit -r requirements-ci.txt for the same parity - lockfile regenerated with --extra all (the cross-platform set) instead of --all-extras, which pulled faiss-gpu/cupy from the Linux-only gpu extra and co-installed faiss-cpu + faiss-gpu in CI - uv pinned to 0.12.1 (the version that generated the lockfile) in CI and CONTRIBUTING so regeneration is deterministic Signed-off-by: Yunare Maia <yunare@gmail.com> * ci: make lockfile staleness check immune to upstream releases The previous check re-resolved pyproject.toml without constraints, so any upstream package release (e.g. boto3 1.43.69 -> 1.43.70) failed CI even when nothing in the repo changed — exactly the time-dependent drift Qodo flagged. The check now re-resolves with requirements-ci.txt as a constraint and compares only version lines, so it detects intentional pyproject.toml changes but ignores upstream releases. CONTRIBUTING updated to match. Signed-off-by: Yunare Maia <yunare@gmail.com> * ci: fix security workflows — install pip-audit; order tooling after pinned deps Security workflow: the pip-audit install step was lost in the rebase conflict merge — pip-audit was invoked but never installed (exit 127). Security-scan workflow: installing safety first let the pinned requirements-ci.txt overwrite its transitive deps (rich), breaking the safety CLI at runtime (RuntimeError: Type not yet supported). Tooling is now installed AFTER the pinned set. Signed-off-by: Yunare Maia <yunare@gmail.com> * fix(ci): address review — hashes, build isolation, release builds, docs (4/4) ZohaibHassan16's review flagged 4 supply-chain gaps; all addressed: 1. Release builds now use the lockfile: release.yml installs requirements-ci.txt and runs `python -m build --no-isolation` so the sdist/wheel is built against the exact tested dependency set. 2. Build isolation pinned: [build-system].requires is now setuptools==84.0.0 + wheel==0.48.0 (exact pins, no ranges). 3. Hashes: requirements-ci.txt regenerated with --generate-hashes (5,708 sha256 hashes, verified against PyPI). Staleness check updated to strip the `\` line continuations hashes introduce. 4. CONTRIBUTING.md documents the separate environment: hashes, never-install-into-dev note, build-system pins, --no-isolation release builds. Validated: stale-check diff clean, hash spot-check matches PyPI. Signed-off-by: Yunare Maia <yunare@gmail.com> * fix(ci): apply --no-isolation to CI build + align benchmark to Python 3.11 Follow-up to ZohaibHassan16's second review round: 1. ci.yml was still running `python -m build` with build isolation (unpinned setuptools/wheel from PyPI) — now `python -m build --no-isolation` against the pinned deps, matching release.yml. 2. benchmark.yml was on Python 3.12 while the lockfile is compiled for 3.11 — aligned to 3.11 so every workflow runs the same environment. Signed-off-by: Yunare Maia <yunare@gmail.com> * fix(ci): install pinned wheel before --no-isolation build python -m build --no-isolation failed with 'Missing dependencies: wheel==0.48.0' because wheel is build-time only — uv's lockfile excludes it, so installing requirements-ci.txt alone left the build env without it. Both ci.yml and release.yml now install wheel==0.48.0 (the same pin [build-system] declares) before building. Validated locally: wheel builds clean with --no-isolation. Signed-off-by: Yunare Maia <yunare@gmail.com> --------- Signed-off-by: Yunare Maia <yunare@gmail.com> Co-authored-by: Zohaib Hassnain <109234410+ZohaibHassan16@users.noreply.github.com>	2026-08-14 22:10:23 +05:00

Author

SHA1

Message

Date

Shinde vinayak rao patil

d94d8f6ab8

Feat/crewai integration (#988 )

* feat(crewai): add first-class CrewAI integration (#962)

Add native CrewAI support so Crew agents can share a ContextGraph and
AgentContext via BaseTool subclasses and a BaseKnowledgeSource, matching
the existing agno integration pattern.

- SemanticaKGTool: 5 KG actions (extract_entities, extract_relations,
  add_to_graph, query_graph, find_related) with sync run()/async arun()
- SemanticaDecisionTool: 5 decision-intelligence actions
  (record_decision, find_precedents, trace_causal_chain,
  analyze_impact, check_policy) over AgentContext
- SemanticaKnowledgeSource: serializes a ContextGraph into crew
  knowledge storage; bridges legacy load_content() and current
  validate_content()/aadd() contracts for crewai>=0.80.0
- All classes degrade gracefully when crewai is absent
- New pip extra crewai=... included in the all bundle
- 70 new tests (stub-based present-case + subprocess degradation path)
- Docs: integrations/crewai.md, docs.json nav, README matrix updates

* fix(crewai): harden tools against real Semantica dataclass shapes (#962)

Bugs found during live testing with crewai 1.15.16:

- SemanticaKGTool.add_to_graph crashed on real Entity/Relation dataclasses
  ('str' object has no attribute 'end_char'): string names were passed to
  extract_relations(entities=...), which requires Entity objects, and the
  tool read .name/.source/.target instead of Entity's .text/.label and
  Relation's .subject/.object. Add shape-agnostic field helpers.
- SemanticaDecisionTool() created an AgentContext without a knowledge_graph,
  so _decision_backend was never set and record_decision raised 'Decision
  tracking is not enabled'. Wire in a ContextGraph.
- record_decision hard-failed when the agent omitted optional fields; fall
  back to category='general', reasoning='agent decision',
  outcome='recorded'.

Add tests covering real Entity/Relation dataclass shapes and the live
auto-created AgentContext path (now 77 crewai tests, 212 total).

* fix(crewai): make find_related traverse edges undirected (#962)

ContextGraph.get_neighbors only follows outgoing edges, so a node whose
only edge is incoming (A -> B) reported no related concepts. Rebuild a
bidirectional adjacency from find_edges() in SemanticaKGTool._find_related
so 'related' honors both directions.

* fix(crewai): harden tools for checkpoint serialization and correct action semantics (#962)

- Exclude live graph/context/extractor state from JSON serialization
  (model_dump(mode="json")) so CrewAI checkpointing no longer raises
  PydanticSerializationError; model_post_init self-heals defaults on restore
- query_graph now searches node content via graph.query() plus id/type
- trace_causal_chain returns an explicit error when causal tracing is
  unavailable instead of substituting similarity precedents; call
  trace_decision_causality(..., max_depth=...) with the correct kwarg name
- find_precedents propagates max_precedents/limit to the backend instead of
  being silently capped at 10
- Serialize add_to_graph batches under a module lock to prevent concurrent
  double-counting; skip nameless entities instead of creating repr()-junk nodes
- aadd() runs CPU-bound serialization in a thread executor
- Mirror crewai args_schema serialize/restore in the conftest stub and add
  serialization regression tests (crewai: 92 tests)

* fix(crewai): correct check_policy coercion, guard causal tracing, and harden concurrency (#962)

- _eval_rule now coerces rule values type-aware: bool("false") was truthy, so
  'enabled == false' reported a violation for enabled=false, and string datums
  like "0.90" were compared lexicographically instead of numerically
- _trace_causal_chain no longer raises AttributeError (which escaped _run) when
  the decision context lacks knowledge_graph; returns honest error JSON
- SemanticaKnowledgeSource storage failures log an actionable ERROR; without a
  configured crew embedder agents previously retrieved nothing silently
- add_to_graph uses a per-graph re-entrant lock (WeakKeyDictionary) instead of a
  process-global one: independent graphs no longer serialize each other and
  re-entrant extractor callbacks cannot deadlock
- entity/relation confidence=None normalizes to 1.0 instead of failing the
  whole extraction with float(None)
- add subprocess integration test against real crewai covering Crew-level
  serialization round-trip and checkpoint restore (stub tests cannot see it)
- docs: embedder requirement for SemanticaKnowledgeSource; resume contract note

* fix(crewai): surface knowledge-source save failures at ERROR when storage is wired (#962)

Re-verification against real crewai showed the embedder-missing failure raises
ValueError even though storage IS wired, so the old except-ValueError branch
mislabeled it as 'storage not wired' and logged DEBUG — hiding the failure.
Distinguish by storage presence instead of exception type: storage is None ->
DEBUG keep-in-memory (legitimate standalone use); storage wired but save()
raises -> actionable ERROR. Add regression test mirroring real crewai's
ValueError-on-missing-embedder behavior.

* fix(crewai): expose run()/arun() entry points in degraded mode (#962)

The public crewai contract is run()/arun(); without crewai installed they were
missing (only the private _run existed), so the documented 'usable without
crewai' path raised AttributeError at the entry point. Define them in degraded
mode only, leaving crewai's BaseTool implementations untouched when present.
Extend the degradation subprocess test to exercise run() and arun().

* fix(crewai): standardize query shape, field-name rules, and restore-state flag

- _query_graph: id/type matches now return the same schema as content
  matches (id/type/label/content/score) instead of a bare list
- _eval_rule: non-greedy field capture so hyphen/dot/space JSON keys
  (e.g. "risk-score >= 0.9") are addressable in policy rules
- add had_live_state/reconstructed_state so checkpoint-restored tools
  and knowledge sources signal that their live graph/context was lost
  and an empty one reconstructed; knowledge source no longer hides the
  loss by eagerly rebuilding its graph inside __init__ (pydantic calls
  __init__ during model_validate)

* fix(crewai): address Qodo review — confidence errors, string trim, holistic availability

- record_decision: stop calling float() in _run, so malformed confidence
  values surface as JSON errors (via _record_decision's handling) instead
  of crashing the tool
- _coerce_value: return the stripped string for non-numeric literals so
  whitespace-padded decision_data fields match policy rules
- centralize crewai availability in _availability.py so the exported
  CREWAI_AVAILABLE flag is holistic across tools and knowledge source
  (previously each module probed crewai independently and the package
  flag came from decision_tool only)

* ci: regenerate requirements-ci.txt for the crewai extra

The crewai extra in pyproject.toml brings in crewai, crewai-tools and
transitive deps (chromadb, lancedb, ...). Recompile with
uv==0.12.1 per CONTRIBUTING.md so the CI staleness check passes.

* ci: keep crewai out of the locked CI dependency set

crewai (all versions) hard-requires chromadb~=1.1.0, which carries a
pre-authentication code-injection advisory (CVE-2026-45829 / GHSA-f4j7-r4q5-qw2c)
with NO fixed release — even the latest 1.5.9 is affected. Keeping crewai in
the 'all' extra failed pip-audit and the safety check on requirements-ci.txt.

- drop crewai from the 'all' aggregate (standalone semantica[crewai] extra is
  unchanged and still installs crewai)
- stop listing crewai-tools in the extra: the integration only uses crewai core
  (BaseTool, BaseKnowledgeSource) and crewai-tools pulled extra transitive deps
- regenerate requirements-ci.txt: OSV/pip-audit 0 vulnerabilities, safety 0
  vulnerabilities, staleness check matches

* docs(crewai): document crewai extra scope and chromadb CVE-2026-45829

- CHANGELOG: extra is crewai>=0.80.0 only (no crewai-tools) and is not
  part of the 'all' bundle, with the chromadb CVE-2026-45829 reason
- integrations/crewai/README.md: add a security warning that installing
  the extra pulls chromadb~=1.1.0, which is affected by the unpatched
  pre-auth code-injection CVE-2026-45829

---------

2026-08-16 11:15:43 +05:00

Yunare MaiaandZohaib Hassnain

4513b61e40

ci: pin Python dependencies in requirements-ci.txt for reproducible CI (#945 )

* ci: pin Python dependencies in requirements-ci.txt for reproducible CI

Adds a committed lockfile pinning all transitive dependencies at exact
versions (uv pip compile, Python 3.11, all extras — 1581 lines), the
Python equivalent of explorer/package-lock.json + npm ci.

- CI installs from requirements-ci.txt before building the wheel
- CI verifies the lockfile is byte-identical to a fresh compile (fails
  on staleness after pyproject.toml changes)
- CONTRIBUTING documents the regeneration command

Closes #938

Signed-off-by: Yunare Maia <yunare@gmail.com>

* ci: address Qodo review — security scans use pinned deps, exclude gpu extras

- security-scan.yml installs from requirements-ci.txt instead of
  "./[llm-litellm]" so Safety scans the exact CI/release dependency tree
- security.yml runs pip-audit -r requirements-ci.txt for the same parity
- lockfile regenerated with --extra all (the cross-platform set) instead
  of --all-extras, which pulled faiss-gpu/cupy from the Linux-only gpu
  extra and co-installed faiss-cpu + faiss-gpu in CI
- uv pinned to 0.12.1 (the version that generated the lockfile) in CI and
  CONTRIBUTING so regeneration is deterministic

Signed-off-by: Yunare Maia <yunare@gmail.com>

* ci: make lockfile staleness check immune to upstream releases

The previous check re-resolved pyproject.toml without constraints, so any
upstream package release (e.g. boto3 1.43.69 -> 1.43.70) failed CI even
when nothing in the repo changed — exactly the time-dependent drift Qodo
flagged. The check now re-resolves with requirements-ci.txt as a
constraint and compares only version lines, so it detects intentional
pyproject.toml changes but ignores upstream releases. CONTRIBUTING
updated to match.

Signed-off-by: Yunare Maia <yunare@gmail.com>

* ci: fix security workflows — install pip-audit; order tooling after pinned deps

Security workflow: the pip-audit install step was lost in the rebase
conflict merge — pip-audit was invoked but never installed (exit 127).

Security-scan workflow: installing safety first let the pinned
requirements-ci.txt overwrite its transitive deps (rich), breaking the
safety CLI at runtime (RuntimeError: Type not yet supported). Tooling is
now installed AFTER the pinned set.

Signed-off-by: Yunare Maia <yunare@gmail.com>

* fix(ci): address review — hashes, build isolation, release builds, docs (4/4)

ZohaibHassan16's review flagged 4 supply-chain gaps; all addressed:

1. **Release builds now use the lockfile**: release.yml installs
   requirements-ci.txt and runs `python -m build --no-isolation` so the
   sdist/wheel is built against the exact tested dependency set.
2. **Build isolation pinned**: [build-system].requires is now
   setuptools==84.0.0 + wheel==0.48.0 (exact pins, no ranges).
3. **Hashes**: requirements-ci.txt regenerated with --generate-hashes
   (5,708 sha256 hashes, verified against PyPI). Staleness check updated
   to strip the `\` line continuations hashes introduce.
4. **CONTRIBUTING.md documents the separate environment**: hashes,
   never-install-into-dev note, build-system pins, --no-isolation release
   builds.

Validated: stale-check diff clean, hash spot-check matches PyPI.
Signed-off-by: Yunare Maia <yunare@gmail.com>

* fix(ci): apply --no-isolation to CI build + align benchmark to Python 3.11

Follow-up to ZohaibHassan16's second review round:

1. ci.yml was still running `python -m build` with build isolation
   (unpinned setuptools/wheel from PyPI) — now `python -m build
   --no-isolation` against the pinned deps, matching release.yml.
2. benchmark.yml was on Python 3.12 while the lockfile is compiled for
   3.11 — aligned to 3.11 so every workflow runs the same environment.

Signed-off-by: Yunare Maia <yunare@gmail.com>

* fix(ci): install pinned wheel before --no-isolation build

python -m build --no-isolation failed with 'Missing dependencies:
wheel==0.48.0' because wheel is build-time only — uv's lockfile
excludes it, so installing requirements-ci.txt alone left the build
env without it. Both ci.yml and release.yml now install wheel==0.48.0
(the same pin [build-system] declares) before building. Validated
locally: wheel builds clean with --no-isolation.

Signed-off-by: Yunare Maia <yunare@gmail.com>

---------

Signed-off-by: Yunare Maia <yunare@gmail.com>
Co-authored-by: Zohaib Hassnain <109234410+ZohaibHassan16@users.noreply.github.com>

2026-08-14 22:10:23 +05:00