Commit Graph
249 Commits
Author SHA1 Message Date
Mohd KaifandClaude Sonnet 4.6 fdb347fe8a feat(cookbook): add Datalog-style reasoning end-to-end notebook (#457)
End-to-end example using DatalogReasoner, GraphBuilder, ContextGraph,
GraphAnalyzer, ExplanationGenerator, DatalogFact, and DatalogRule.
Covers ancestor query, KG dependency analysis, RBAC policy, and org hierarchy.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 21:43:04 +05:30
KaifAhmad1andClaude Sonnet 4.6 c1f0cf6f34 Fix 3 bugs in notebook 13 (manual ontology + Snowflake mapping)
- Bug 1: replace dict .get() with dataclass attribute access on
  AssociativeClass (name/connects/temporal/properties)
- Bug 2: add full URI to every ontology property and use BASE_URI-prefixed
  URIs for all relationship types so TripletStore stores hr:<name>
  instead of urn:property:<name>, fixing SPARQL PREFIX hr: queries
- Bug 3: filter None values from EmploymentEvent properties dict so
  open-ended employment does not store the literal string "None" as endDate

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 15:23:19 +05:30
KaifAhmad1andClaude Sonnet 4.6 665f9c080e Add manual ontology + Snowflake mapping cookbook
Adds notebook 13 demonstrating pythonic, no-AI-inference workflow:
hand-designed ontology dict, AssociativeClass reification, explicit
row-to-graph mapping, OWL/SHACL export, and SPARQL query patterns.
Includes SPARQL 1.2 / SHACL 1.2 standards coverage notes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 15:14:19 +05:30
KaifAhmad1andClaude Sonnet 4.6 62c7970b32 feat(integrations): add Agno agentic framework integration (#249)
Implements the full Semantica × Agno integration stack as described in
issue #249, wiring Semantica's semantic intelligence layer into Agno's
agent/team primitives via five focused components.

## New components

### integrations/agno/
- `AgnoContextStore`    — graph-backed MemoryDb (AgentMemory/storage)
- `AgnoKnowledgeGraph`  — relational AgentKnowledge with multi-hop GraphRAG
- `AgnoDecisionKit`     — Agno Toolkit: 6 decision-intelligence tools
- `AgnoKGToolkit`       — Agno Toolkit: 7 knowledge-graph tools
- `AgnoSharedContext`   — team-level shared ContextGraph with role scoping

### tests/integrations/agno/
- 110 tests, 0 failures
- conftest.py installs comprehensive agno stubs for offline testing
- Covers MemoryDb protocol, tool registration, shared memory pool,
  thread-safety, GraphRAG search, NER/relation extraction, and inference

### cookbook/integrations/
- agno_decision_intelligence.ipynb     (finance/loan underwriting)
- agno_graphrag_context.ipynb          (regulatory compliance GraphRAG)
- agno_multi_agent_shared_context.ipynb (multi-agent product strategy team)

### docs/integrations/agno.md
- Full reference documentation with examples for all 5 components

## pyproject.toml
- Added `agno = ["agno>=1.0.0"]` optional dependency
- Added agno to the `all` extra

## Design notes
- Zero breaking changes — fully additive
- Graceful degradation when agno is not installed
- Auto-creates VectorStore(backend="faiss") when none provided
- _tools always populated for inspection regardless of agno install state
- Works with both real agno package and offline stubs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 03:25:14 +05:30
KaifAhmad1andClaude Sonnet 4.6 1d96b6f80e fix: address code review issues from PR #358 (#355)
- rdf_exporter.py: add isinstance(format, str) guard before .lower() so
  non-string inputs (None, int, etc.) raise ValidationError consistently
  instead of AttributeError; normalize via strip().lower() in one step
- 15_Export.ipynb: fix notebook cell using result['valid'] → result['overall_valid']
  (validate_rdf() returns overall_valid, not valid); add trailing EOF newline
- test_rdf_exporter.py: add tests for non-string format → ValidationError
  and for overall_valid key presence in validate_rdf() return value

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 03:03:44 +05:30
KaifAhmad1andClaude Sonnet 4.6 8c4e5e5968 fix: resolve TTL export alias failure and add RDF notebook example (#355)
- Add _format_aliases map in RDFExporter to accept 'ttl', 'nt', 'xml', 'rdf', 'json-ld' as shorthands for canonical format names
- Resolve alias at the start of export_to_rdf() before validation, leaving all existing callers unaffected
- Add TTL alias demo cell to cookbook/introduction/15_Export.ipynb
- Add tests/export/test_rdf_exporter.py covering alias parity, canonical formats, unsupported format error, and file export with format="ttl"

Closes #355

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 01:42:16 +05:30
KaifAhmad1 cc2e18d7ff Fix NameError: missing Type import in utils/helpers.py
- Add Type import to typing imports in helpers.py to fix retry_on_error decorator
- Remove unused Type import from config_manager.py
- Update capability gap notebook with comment about the fix
- Resolves ImportError when importing semantica modules

Fixes: NameError: name 'Type' is not defined in retry_on_error decorator
2026-02-22 15:56:05 +05:30
KaifAhmad1 7caace7c5d Refactor notebook inconsistencies and optimize ontology evaluation positioning
- Fixed duplicate setup cells and consolidated into single setup cell
- Resolved undefined variable references in corpus creation
- Moved ontology evaluation to optimal position after semantic extraction
- Enhanced ontology evaluation with extraction context integration
- Removed empty placeholder cells and improved logical flow
- Added semantica package installation requirement
- Updated pipeline sequence to follow correct data processing order
- Improved error handling and variable validation throughout notebook
2026-02-21 22:47:51 +05:30
KaifAhmad1 f5896574c6 docs: add capability gap context graphs use case and example 2026-02-19 19:22:09 +05:30
Sameer6305 06d5fad6b9 feat: add Snowflake ingestor for native data warehouse ingestion 2026-02-03 23:27:25 +05:30
KaifAhmad1 bc55dcc57a Fix stuck retries in extraction and enable configurable retry limit. Resolves #207 2026-01-25 21:27:03 +05:30
KaifAhmad1 d91619f191 Fix earnings call analysis notebook: attribute access and export logic 2026-01-20 01:51:29 +05:30
KaifAhmad1 8214acc675 optimize vector db storage in earnings call analysis 2026-01-19 16:32:22 +05:30
KaifAhmad1 1568237ce7 Add high-performance VectorStore ingestion and docs 2026-01-19 13:32:16 +05:30
KaifAhmad1 d9117b7c2f docs: update earnings call analysis notebook 2026-01-18 23:53:07 +05:30
KaifAhmad1 a39ec5fd20 Faster, class-based dedup: DuplicateDetector+EntityMerger with strict thresholds; build graph from deduplicated outputs; clean prints 2026-01-16 22:32:19 +05:30
KaifAhmad1 bbd6764215 Use deduplicated entities/relationships; optimize and clean deduplication; disable extra merging in GraphBuilder 2026-01-16 18:18:50 +05:30
KaifAhmad1 1b0b0551db Update Earnings Call Analysis notebook 2026-01-16 17:54:44 +05:30
Don Simpson 65d99f7f8a Added CloudFormation template that creates a dev cluster with a single [t3 instance](https://docs.aws.amazon.com/neptune/latest/userguide/manage-console-instances-t3.html) configured with a [public endpoint](https://docs.aws.amazon.com/neptune/latest/userguide/neptune-public-endpoints.html) and IAM Auth enabled (required for public endpoint), and creates an IAM User using least-privilege principles. See Get started with Neptune Database for free on the [Amazon Neptune pricing page](https://aws.amazon.com/neptune/pricing/).
Includes the CloudFormation template in the same directory as the [Amazon Neptune Cookbook](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/introduction/21_Amazon_Neptune_Store.ipynb) and references it as a prerequisite in the cookbook.
2026-01-15 18:48:49 -05:00
KaifAhmad1 653523efeb Update earnings call analysis notebook with relation extraction fixes
- Update notebook to use corrected RelationExtractor API
- Move provider/model parameters to initialization
- Add verbose logging for debugging
- Include working relation extraction examples
2026-01-16 03:03:11 +05:30
KaifAhmad1 a5da533d55 chore: resolve dependencies, migrate Gemini SDK, and sanitize notebooks 2026-01-14 12:37:29 +05:30
KaifAhmad1 fd9f0b2526 Add all changes 2026-01-13 00:10:44 +05:30
KaifAhmad1 58686d409b fix(cookbook): resolve TypeError in earnings call analysis step 7 #177 2026-01-12 17:32:16 +05:30
KaifAhmad1 cd1435ee10 Save changes to Earnings Call Analysis notebook 2026-01-11 23:25:28 +05:30
Don Simpson 976a20496d feat: Add Amazon Neptune Database Graph Store Support
- Implement NeptuneAuthTokenManager extending Neo4j AuthManager for IAM SigV4 signing
- Add automatic token refresh and security exception handling
- Add retry logic with backoff for transient errors (signature expired, connection closed)
- Add connection recovery with driver recreation
- Add NeptuneDriver, NeptuneSession, NeptuneTransaction wrapper classes
- Use native Neptune ~id via id() function for all CRUD operations
- Add graph-amazon-neptune optional dependency group (boto3, neo4j)
- Update cookbook with Amazon Neptune Graph Store examples
- Add comprehensive tests (61 tests covering all GraphStore interface methods)

Closes #151
2026-01-08 20:13:28 -05:00
KaifAhmad1 e3b53998c3 Fix dependency issues, align GraphRAG notebook, and update changelog 2026-01-07 19:00:30 +05:30
KaifAhmad1 04c4c9fb4c docs: clean and fix corrupted notebooks in cookbook 2026-01-07 15:41:13 +05:30
KaifAhmad1 dcd6f25f87 docs: fix KnowledgeGraph mismatch and update tests for issue #144 2026-01-07 15:01:47 +05:30
KaifAhmad1 960d7c5f8f docs: improve robustness and fix variable inconsistencies in earnings call notebook 2026-01-07 14:09:22 +05:30
KaifAhmad1 2790132e8e fix(semantic_extract): Pass API key to Groq LLM provider in extraction methods
- Add API key handling in extract_entities_llm(), extract_relations_llm(), and extract_triplets_llm()
- Add explicit api_key handling in NERExtractor and RelationExtractor
- Add llm_model parameter support in extract_triplets_llm() for consistency
- Fix relation extraction bug with type checking for subject_text/object_text
- Add environment variable fallback for API keys
- Update notebook with standard API key pattern

Fixes #147
2026-01-07 01:07:56 +05:30
KaifAhmad1 9c59f97542 fix(semantic_extract): Pass API key to Groq LLM provider in extraction methods
- Add API key handling in extract_entities_llm(), extract_relations_llm(), and extract_triplets_llm()
- Add llm_model parameter support in extract_triplets_llm() for consistency
- Fix relation extraction bug with type checking for subject_text/object_text
- Add environment variable fallback for API keys
- Include providers.py for context (GroqProvider implementation)

Fixes #145
2026-01-06 22:37:57 +05:30
KaifAhmad1 a441e935f9 Fix DoclingParser integration and align with docling API
- Fix import logic in __init__.py to properly export DoclingParser
- Rewrite docling_parser.py to use docling's native API (direct attribute access)
- Remove unsupported features (table_extraction_mode, invalid format_options)
- Use doc.tables, doc.pictures, doc.pages directly instead of dict parsing
- Update notebook with improved code and documentation
- Add proper error handling for when docling is not available

Fixes #138
2026-01-05 22:27:15 +05:30
KaifAhmad1 4e0c3bc361 Resolve DoclingParser exports, fix Windows progress encoding, and update finance cookbook #138 2026-01-05 20:04:59 +05:30
KaifAhmad1 ea02896617 feat(parse): add progress tracking to DoclingParser and update earnings call notebook
- Add 8-stage progress tracking (0-100%) with ETA to DoclingParser
- Update earnings call analysis notebook with MDA Space Q3 2025 example
- Simplify notebook code structure
- Add real-time progress visibility for PDF parsing

Closes #133
2026-01-05 14:42:12 +05:30
KaifAhmad1 e7f713d43b Fix: Handle OSError for optional dependencies and make DoclingParser standalone
- Add safe_import utility in semantica/utils/helpers.py for graceful optional dependency handling
- Update all optional imports (spacy, docling, etc.) to handle OSError (Windows DLL issues)
- Make DoclingParser standalone with docling as core dependency
- Remove DoclingParser integration from DocumentParser
- Implement lazy initialization for DoclingParser (fails on parse(), not init())
- Fix DocumentConverter initialization (remove unsupported pipeline_options parameter)
- Preserve original error messages without modification
- Update semantic_extract, split, parse, embeddings, vector_store, visualization modules
- Fix OSError handling across entire codebase for Windows compatibility
- Update 60+ files with proper optional dependency handling
2026-01-04 17:44:14 +05:30
KaifAhmad1 47809f2ef9 Fix: Make PyTorch import lazy to avoid DLL errors on Windows
- Remove top-level torch import from providers.py
- Add lazy imports in HuggingFaceLLMProvider and HuggingFaceModelLoader
- Remove hardcoded API key from notebook
- PyTorch now only loads when HuggingFace providers are instantiated

Fixes #129
2026-01-03 20:39:37 +05:30
KaifAhmad1 96a98fa037 [FEATURE] Integrate Docling for Enhanced Document Parsing
- Added DoclingParser class in semantica/parse/ module
- Created earnings call analysis notebook with Docling integration
- Added docling to pyproject.toml as optional dependency
- Maintained backward compatibility with existing parsers

Closes #124
2026-01-03 18:46:25 +05:30
KaifAhmad1 8cde40d753 Remove trading notebooks and supply chain risk management notebook
- Deleted cookbook/use_cases/trading/01_Risk_Assessment.ipynb
- Deleted cookbook/use_cases/trading/02_News_Sentiment_Analysis.ipynb
- Deleted cookbook/use_cases/supply_chain/02_Supply_Chain_Risk_Management.ipynb
- Removed empty trading directory
- Updated documentation to reflect 14 cookbooks (down from 15)
- Removed all references from README.md, docs/cookbook.md, docs/use-cases.md, docs/index.md, and STRATEGIES_SUMMARY.md
2025-12-30 23:33:27 +05:30
KaifAhmad1 9ef8a7aa18 Update Energy Market Analysis notebook: simplify code, use Semantica effectively, remove redirect_stderr, fix entity/relationship extraction 2025-12-30 21:44:53 +05:30
KaifAhmad1 af17585087 Remove Smart Grid Management notebook and update cookbook count to 15 2025-12-30 20:20:11 +05:30
KaifAhmad1 7b9bd42790 Clean up intelligence analysis notebook: remove unnecessary imports and with blocks, use Semantica built-in methods properly 2025-12-30 20:10:02 +05:30
KaifAhmad1 d8f78cd49e Refactor Criminal Network Analysis notebook: simplify code, use Semantica modules effectively, add interactive visualization, fix GraphRAG queries 2025-12-30 18:19:57 +05:30
KaifAhmad1 f4016237bd Remove healthcare use case: Drug Interactions Analysis
- Deleted cookbook/use_cases/healthcare/02_Drug_Interactions_Analysis.ipynb
- Removed Healthcare section from README.md
- Removed Healthcare section from docs/cookbook.md
- Removed Drug Interactions references from STRATEGIES_SUMMARY.md
- Updated cookbook count from 18 to 17 in all documentation
- Updated docs/index.md to reflect 17 cookbooks
2025-12-30 15:27:21 +05:30
KaifAhmad1 0c5e12f5c9 Remove Clinical Reports Processing notebook and all references
- Delete cookbook/use_cases/healthcare/01_Clinical_Reports_Processing.ipynb
- Delete cookbook/use_cases/healthcare/data/clinical_report.txt
- Remove references from README.md Healthcare section
- Remove Medical Record Analysis card from docs/use-cases.md
- Remove Clinical Reports Processing card from docs/cookbook.md
- Remove entries from STRATEGIES_SUMMARY.md table and rationale
2025-12-30 14:32:55 +05:30
KaifAhmad1 9c97d3236c Fix Fraud Detection notebook: Add real data sources, fix errors, enhance GraphRAG with Context Graph
- Add real CSV and JSON data sources for transactions and accounts
- Fix ConflictDetector, TemporalGraphQuery, and Reasoner errors
- Simplify code to use Semantica modules properly
- Enhance GraphRAG section with Context Graph and Groq LLM
- Add temporal interactive visualization using TemporalVisualizer
- Fix CSV export to use CSVExporter instead of GraphExporter
- Update README.md to mention Context Graph and Context Retriever
2025-12-30 13:45:56 +05:30
KaifAhmad1 599372a50f Update financial data integration notebook:
- Switch entity and relation extraction to ML-based methods (spaCy)
- Fix conflict detection to use detect_temporal_conflicts directly
- Fix graph building to use correct Relation attributes (subject/object/predicate)
- Improve GraphRAG with LLM-based multi-hop reasoning
- Enhance graph analytics output to show all entity types
- Update markdown descriptions with concise bullet points
2025-12-29 23:09:20 +05:30
KaifAhmad1 9f31f825ff Fix Threat Intelligence Hybrid RAG notebook: Update conflict detection, GraphRAG queries, reasoning, and visualization 2025-12-29 22:32:30 +05:30
KaifAhmad1 a674e8c039 fix: resolve Entity TypeError by adding required start_char and end_char fields across cookbook notebooks 2025-12-29 21:35:30 +05:30
KaifAhmad1 d2d6adafdb Add comprehensive progress tracking with Jupyter/Colab support
- Enhanced progress tracker with automatic Jupyter/Colab detection
- Added detailed progress tracking to all deduplication modules
- Added detailed progress tracking to all semantic_extract modules
- Progress tracker now always enabled automatically
- Shows remaining items, percentages, ETA, and processing rates
- Works in both Jupyter notebooks and Google Colab
- Dynamic update intervals based on dataset size
- Improved display handling for Colab compatibility
2025-12-29 13:11:22 +05:30
KaifAhmad1 00575b135e Add progress tracker enable check to all modules
- Added enable check to normalize module (8 files)
- Added enable check to ontology module (16 files)
- Added enable check to ingest module (4 files)
- Added enable check to graph_store module (3 files)
- Ensures progress tracking is enabled by default in all modules
- Total: 112 files updated across the codebase
2025-12-28 22:13:11 +05:30