15 Commits
Author SHA1 Message Date
Derek TapleyandCursor f0aa581318 feat(integrations): add LangChain integration — retriever, vectorstor… (#1155)
* feat(integrations): add LangChain integration — retriever, vectorstore, tools

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(langchain): address Qodo review on HybridSearch hits and tools

Read nested HybridSearch metadata so retriever/vectorstore Documents
are not empty, make the agent tools real BaseTool subclasses, and
stop slicing tool JSON into invalid payloads.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-26 18:29:22 +05:00
Shinde vinayak rao patil d94d8f6ab8 Feat/crewai integration (#988)
* feat(crewai): add first-class CrewAI integration (#962)

Add native CrewAI support so Crew agents can share a ContextGraph and
AgentContext via BaseTool subclasses and a BaseKnowledgeSource, matching
the existing agno integration pattern.

- SemanticaKGTool: 5 KG actions (extract_entities, extract_relations,
  add_to_graph, query_graph, find_related) with sync run()/async arun()
- SemanticaDecisionTool: 5 decision-intelligence actions
  (record_decision, find_precedents, trace_causal_chain,
  analyze_impact, check_policy) over AgentContext
- SemanticaKnowledgeSource: serializes a ContextGraph into crew
  knowledge storage; bridges legacy load_content() and current
  validate_content()/aadd() contracts for crewai>=0.80.0
- All classes degrade gracefully when crewai is absent
- New pip extra crewai=... included in the all bundle
- 70 new tests (stub-based present-case + subprocess degradation path)
- Docs: integrations/crewai.md, docs.json nav, README matrix updates

* fix(crewai): harden tools against real Semantica dataclass shapes (#962)

Bugs found during live testing with crewai 1.15.16:

- SemanticaKGTool.add_to_graph crashed on real Entity/Relation dataclasses
  ('str' object has no attribute 'end_char'): string names were passed to
  extract_relations(entities=...), which requires Entity objects, and the
  tool read .name/.source/.target instead of Entity's .text/.label and
  Relation's .subject/.object. Add shape-agnostic field helpers.
- SemanticaDecisionTool() created an AgentContext without a knowledge_graph,
  so _decision_backend was never set and record_decision raised 'Decision
  tracking is not enabled'. Wire in a ContextGraph.
- record_decision hard-failed when the agent omitted optional fields; fall
  back to category='general', reasoning='agent decision',
  outcome='recorded'.

Add tests covering real Entity/Relation dataclass shapes and the live
auto-created AgentContext path (now 77 crewai tests, 212 total).

* fix(crewai): make find_related traverse edges undirected (#962)

ContextGraph.get_neighbors only follows outgoing edges, so a node whose
only edge is incoming (A -> B) reported no related concepts. Rebuild a
bidirectional adjacency from find_edges() in SemanticaKGTool._find_related
so 'related' honors both directions.

* fix(crewai): harden tools for checkpoint serialization and correct action semantics (#962)

- Exclude live graph/context/extractor state from JSON serialization
  (model_dump(mode="json")) so CrewAI checkpointing no longer raises
  PydanticSerializationError; model_post_init self-heals defaults on restore
- query_graph now searches node content via graph.query() plus id/type
- trace_causal_chain returns an explicit error when causal tracing is
  unavailable instead of substituting similarity precedents; call
  trace_decision_causality(..., max_depth=...) with the correct kwarg name
- find_precedents propagates max_precedents/limit to the backend instead of
  being silently capped at 10
- Serialize add_to_graph batches under a module lock to prevent concurrent
  double-counting; skip nameless entities instead of creating repr()-junk nodes
- aadd() runs CPU-bound serialization in a thread executor
- Mirror crewai args_schema serialize/restore in the conftest stub and add
  serialization regression tests (crewai: 92 tests)

* fix(crewai): correct check_policy coercion, guard causal tracing, and harden concurrency (#962)

- _eval_rule now coerces rule values type-aware: bool("false") was truthy, so
  'enabled == false' reported a violation for enabled=false, and string datums
  like "0.90" were compared lexicographically instead of numerically
- _trace_causal_chain no longer raises AttributeError (which escaped _run) when
  the decision context lacks knowledge_graph; returns honest error JSON
- SemanticaKnowledgeSource storage failures log an actionable ERROR; without a
  configured crew embedder agents previously retrieved nothing silently
- add_to_graph uses a per-graph re-entrant lock (WeakKeyDictionary) instead of a
  process-global one: independent graphs no longer serialize each other and
  re-entrant extractor callbacks cannot deadlock
- entity/relation confidence=None normalizes to 1.0 instead of failing the
  whole extraction with float(None)
- add subprocess integration test against real crewai covering Crew-level
  serialization round-trip and checkpoint restore (stub tests cannot see it)
- docs: embedder requirement for SemanticaKnowledgeSource; resume contract note

* fix(crewai): surface knowledge-source save failures at ERROR when storage is wired (#962)

Re-verification against real crewai showed the embedder-missing failure raises
ValueError even though storage IS wired, so the old except-ValueError branch
mislabeled it as 'storage not wired' and logged DEBUG — hiding the failure.
Distinguish by storage presence instead of exception type: storage is None ->
DEBUG keep-in-memory (legitimate standalone use); storage wired but save()
raises -> actionable ERROR. Add regression test mirroring real crewai's
ValueError-on-missing-embedder behavior.

* fix(crewai): expose run()/arun() entry points in degraded mode (#962)

The public crewai contract is run()/arun(); without crewai installed they were
missing (only the private _run existed), so the documented 'usable without
crewai' path raised AttributeError at the entry point. Define them in degraded
mode only, leaving crewai's BaseTool implementations untouched when present.
Extend the degradation subprocess test to exercise run() and arun().

* fix(crewai): standardize query shape, field-name rules, and restore-state flag

- _query_graph: id/type matches now return the same schema as content
  matches (id/type/label/content/score) instead of a bare list
- _eval_rule: non-greedy field capture so hyphen/dot/space JSON keys
  (e.g. "risk-score >= 0.9") are addressable in policy rules
- add had_live_state/reconstructed_state so checkpoint-restored tools
  and knowledge sources signal that their live graph/context was lost
  and an empty one reconstructed; knowledge source no longer hides the
  loss by eagerly rebuilding its graph inside __init__ (pydantic calls
  __init__ during model_validate)

* fix(crewai): address Qodo review — confidence errors, string trim, holistic availability

- record_decision: stop calling float() in _run, so malformed confidence
  values surface as JSON errors (via _record_decision's handling) instead
  of crashing the tool
- _coerce_value: return the stripped string for non-numeric literals so
  whitespace-padded decision_data fields match policy rules
- centralize crewai availability in _availability.py so the exported
  CREWAI_AVAILABLE flag is holistic across tools and knowledge source
  (previously each module probed crewai independently and the package
  flag came from decision_tool only)

* ci: regenerate requirements-ci.txt for the crewai extra

The crewai extra in pyproject.toml brings in crewai, crewai-tools and
transitive deps (chromadb, lancedb, ...). Recompile with
uv==0.12.1 per CONTRIBUTING.md so the CI staleness check passes.

* ci: keep crewai out of the locked CI dependency set

crewai (all versions) hard-requires chromadb~=1.1.0, which carries a
pre-authentication code-injection advisory (CVE-2026-45829 / GHSA-f4j7-r4q5-qw2c)
with NO fixed release — even the latest 1.5.9 is affected. Keeping crewai in
the 'all' extra failed pip-audit and the safety check on requirements-ci.txt.

- drop crewai from the 'all' aggregate (standalone semantica[crewai] extra is
  unchanged and still installs crewai)
- stop listing crewai-tools in the extra: the integration only uses crewai core
  (BaseTool, BaseKnowledgeSource) and crewai-tools pulled extra transitive deps
- regenerate requirements-ci.txt: OSV/pip-audit 0 vulnerabilities, safety 0
  vulnerabilities, staleness check matches

* docs(crewai): document crewai extra scope and chromadb CVE-2026-45829

- CHANGELOG: extra is crewai>=0.80.0 only (no crewai-tools) and is not
  part of the 'all' bundle, with the chromadb CVE-2026-45829 reason
- integrations/crewai/README.md: add a security warning that installing
  the extra pulls chromadb~=1.1.0, which is affected by the unpatched
  pre-auth code-injection CVE-2026-45829

---------
2026-08-16 11:15:43 +05:00
KaifAhmad1 2d5bd18fa4 Address review: column lineage, connection reuse, UC name validation
- get_table_lineage() gains include_column_lineage=True, resolving
  per-column upstream/downstream references via Unity Catalog's
  column-lineage API (one request per column, opt-in)
- DatabricksConnector.connect() now reuses an already-open connection
  instead of opening a second one; ingest_table()/ingest_query() only
  close the connection they opened themselves, so using the ingestor
  as a context manager no longer leaks the connection opened by
  __enter__
- get_table_schema()/get_table_lineage()/list_tables() now validate
  both catalog and schema are resolved before calling Unity Catalog,
  matching list_tables()'s existing catalog check
- 8 new regression tests (35 total)
2026-07-15 22:25:39 +05:30
KaifAhmad1 d74b650643 Add Databricks connector (Unity Catalog + Delta Lake ingestion)
Adds DatabricksIngestor to semantica/ingest/, mirroring SnowflakeIngestor's
structure and public API shape: table/query ingestion via
databricks-sql-connector, Unity Catalog metadata and lineage via
databricks-sdk, and export-as-documents for KG construction.

Closes #747
2026-07-15 22:07:25 +05:30
Mohd Kaif 25289023fe docs: replace all CardGroup/Card blocks with animated bullet points across all 50 docs pages (#648)
- Fix What's new → link in Info banner (now a proper <a> tag, always clickable)
- Replace 4-stat CardGroup on index with inline premium stats row
- Convert every <CardGroup>/<Card> block site-wide to markdown bullet lists:
  content sections → bold-title bullets with sub-bullets, nav cards → [Title](href) — description
- Add cursor-animated list item hover effects to custom.css:
  green inset left border, subtle background tint, marker color change on hover
- Affects index, getting-started, quickstart, concepts, modules, faq, architecture,
  installation, cookbook, glossary, learning-more, explorer-setup, cli-setup,
  community, contributing-guide, governance, citation, project-license,
  all integrations pages, and all 20+ reference module pages
2026-06-17 23:18:40 +05:30
Mohd Kaif a326c7d3bd Fix/mintlify theme (#645)
* fix: replace invalid Mintlify theme 'venus' with 'mint'

* docs: replace em dashes with colons across all docs files

* fix: strip UTF-8 BOM from all docs files (broke frontmatter detection)
2026-06-17 13:26:19 +05:30
Mohd Kaif 1f3cea5f0a docs: upgrade all docs pages with Mintlify premium components (#642)
Replace plain markdown lists, tables, and numbered steps with interactive
Mintlify v3 MDX components across all 50+ documentation files:

- Tabs: provider/parser/method selection guides, citation formats, component details
- Steps: setup flows, pipeline stages, connection initialization
- CardGroup/Card: feature overviews, "what you get" sections, navigation footers
- AccordionGroup: FAQ entries
- Check/Warning/Tip/Note/Info: callouts replacing plain bold text and inline notes

Files improved span the full docs surface: reference modules (context, llms,
kg, reasoning, embeddings, deduplication, provenance, parse, ontology, core,
semantic_extract), integrations (agno, docling, snowflake), graph/vector
store backends (apache_age, pgvector), and top-level guides (contributing,
governance, glossary, citation, community-projects, learning-more).
2026-06-17 12:58:44 +05:30
KaifAhmad1 77b1eaaa78 docs: fix Python 3.9+ list[dict] syntax in docling.md for 3.8 compat 2026-05-22 22:41:13 +05:30
KaifAhmad1andClaude Sonnet 4.6 946a1089c8 docs: premium redesign — Mintlify v4, dark/cream theme, full module coverage
- Migrate from mint.json to docs.json (Mintlify v4)
- Theme: maple, emerald green + near-black dark / cream light palette
  (#059669 primary, #0A0A0A dark bg, #FAF7F0 light bg)
- Typography: Lexend headings, Inter body
- 5-tab navigation: Documentation, Quick Start, API Reference, Cookbook, FAQ
- Homepage: removed badge stickers, redundant h2, added blockquote tagline,
  full 27-module reference table with semantica.mcp_server added
- quickstart.md: CodeGroup per pipeline step, pattern vs LLM options,
  AccordionGroup for patterns and troubleshooting
- faq.md: full AccordionGroup structure across 5 sections
- reference/explorer.md: NEW — FastAPI explorer, Ontology Hub, Distance
  Intelligence, CLI reference, REST API endpoints
- reference/mcp_server.md: NEW — MCP stdio server, 12 tools with I/O
  examples, 3 resources, Claude Desktop/VS Code/Windsurf/Cline config
- docs.json: explorer added to Output group, mcp_server to Utilities group
- Chat, feedback (thumbs/suggest/raise), OG/Twitter metadata, search topbar
- All reference pages reformatted with Mintlify JSX components

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 21:52:50 +05:30
Mohd KaifandClaude Sonnet 4.6 b282487b17 docs: rewrite and polish documentation site (#413)
- Rewrote index.md to match README (tagline, badges, Problem/Solution text)
- Improved getting-started, concepts, quickstart, installation, faq, use-cases, contributing, glossary, learning-more, examples, modules, architecture, cookbook, deep-dive pages: tighter prose, fixed headings/bullets, removed inconsistencies and duplicate sections
- Removed overuse of emojis from headings in integration pages (docling, snowflake)
- Fixed change_management reference page: closed unclosed JSON code block that broke the right TOC, demoted noisy sub-headings to bold text
- CSS layout: widened content area (max-width 1440px grid, left sidebar 11rem, right TOC narrowed to 11rem for broader content), tightened TOC spacing and font size, fixed word-wrap/overflow on TOC links
- Added mkdocs_local.yml for local serving without mkdocs-jupyter plugin

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 18:38:21 +05:30
KaifAhmad1andClaude Sonnet 4.6 62c7970b32 feat(integrations): add Agno agentic framework integration (#249)
Implements the full Semantica × Agno integration stack as described in
issue #249, wiring Semantica's semantic intelligence layer into Agno's
agent/team primitives via five focused components.

## New components

### integrations/agno/
- `AgnoContextStore`    — graph-backed MemoryDb (AgentMemory/storage)
- `AgnoKnowledgeGraph`  — relational AgentKnowledge with multi-hop GraphRAG
- `AgnoDecisionKit`     — Agno Toolkit: 6 decision-intelligence tools
- `AgnoKGToolkit`       — Agno Toolkit: 7 knowledge-graph tools
- `AgnoSharedContext`   — team-level shared ContextGraph with role scoping

### tests/integrations/agno/
- 110 tests, 0 failures
- conftest.py installs comprehensive agno stubs for offline testing
- Covers MemoryDb protocol, tool registration, shared memory pool,
  thread-safety, GraphRAG search, NER/relation extraction, and inference

### cookbook/integrations/
- agno_decision_intelligence.ipynb     (finance/loan underwriting)
- agno_graphrag_context.ipynb          (regulatory compliance GraphRAG)
- agno_multi_agent_shared_context.ipynb (multi-agent product strategy team)

### docs/integrations/agno.md
- Full reference documentation with examples for all 5 components

## pyproject.toml
- Added `agno = ["agno>=1.0.0"]` optional dependency
- Added agno to the `all` extra

## Design notes
- Zero breaking changes — fully additive
- Graceful degradation when agno is not installed
- Auto-creates VectorStore(backend="faiss") when none provided
- _tools always populated for inspection regardless of agno install state
- Works with both real agno package and offline stubs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 03:25:14 +05:30
KaifAhmad1 d3b579208c Comprehensive documentation cleanup and improvements
## Documentation Changes

### 📚 Major Improvements
- **Cleaned up all documentation files** - Removed redundant content and improved clarity
- **Restructured Resources section** - Removed unnecessary files, kept only essential ones
- **Added Snowflake integration** - Complete integration guide with examples
- **Improved navigation** - Better organization and user experience

### 🗂️ File Changes
- **docs/concepts.md** - Rewritten to be clean and user-friendly
- **docs/modules.md** - Updated with current modules and removed emojis
- **docs/glossary.md** - Reorganized thematically instead of alphabetically
- **docs/getting-started.md** - Made more concise and practical
- **docs/community.md** - Clean, focused community guide
- **docs/contributing.md** - Clear contribution guidelines
- **docs/faq.md** - Comprehensive FAQ with practical answers
- **docs/license.md** - Clean license explanation
- **docs/css/custom.css** - Fixed CSS syntax and organization

### 🔧 Technical Changes
- **mkdocs.yml** - Updated navigation, removed redundant files
- **docs/integrations/snowflake.md** - New comprehensive Snowflake guide
- **docs/reference/ingest.md** - Added Snowflake references
- **Removed files**: changelog.md, release-guide.md, change_management_usage.md, community-projects.md, architecture.md, governance.md, citation.md

### 🎯 Benefits
- **Better user experience** - Clean, easy to navigate documentation
- **Reduced redundancy** - No duplicate or unnecessary content
- **Professional quality** - Enterprise-ready documentation
- **Consistent style** - Uniform formatting across all files

This commit includes all documentation improvements while maintaining the main branch's stability.
2026-02-05 17:43:16 +05:30
Sameer6305 0a66e1c6ea fix: address Copilot review feedback for Snowflake ingestor 2026-02-04 00:17:25 +05:30
Sameer6305 06d5fad6b9 feat: add Snowflake ingestor for native data warehouse ingestion 2026-02-03 23:27:25 +05:30
KaifAhmad1 957c122116 docs: add Docling integration guide, clear code example, and fix parser consistency issues 2026-01-08 21:27:48 +05:30