264 Commits
Author SHA1 Message Date
KaifAhmad1 84b90b45a2 fix: align split methods with documentation and registry 2025-12-12 16:15:57 +05:30
KaifAhmad1 d3366bbcf0 Refactor Semantic Extract module: Update notebooks, docs, and implementation to use class-based interfaces 2025-12-12 13:23:37 +05:30
KaifAhmad1 afc94ad059 Remove KuzuDB backend support and cleanup references 2025-12-10 20:30:59 +05:30
KaifAhmad1 7bbf8e9881 Enhance export module, fix notebooks, and add tests
- Added comprehensive unit tests for export module (tests/test_export_module.py)

- Added simulation tests for notebooks 15 and 05 (tests/test_notebook*.py)

- Fixed GraphBuilder.build() signature usage in notebooks and simulations

- Fixed CSVExporter file path handling and CSV content verification

- Fixed VectorExporter data format in notebooks

- Updated YAMLSchemaExporter usage

- Fixed conflict detection in GraphBuilder

- Verified all export formats (JSON, CSV, RDF, GraphML, YAML, OWL, Vector, LPG)
2025-12-10 18:15:04 +05:30
KaifAhmad1 6ee19d971e feat: enhance embeddings with dynamic model switching, updated docs and tests 2025-12-10 17:32:43 +05:30
KaifAhmad1 b169ce6253 fix(conflicts): fix recursion bug in methods.py and add comprehensive tests
- Fix infinite recursion in semantica/conflicts/methods.py by removing redundant registration
- Update 04_Conflict_Resolution_Strategies.ipynb to use correct API
- Add unit tests for conflicts module in tests/conflicts/test_conflicts.py
- Add __init__.py files to tests/ and tests/conflicts/ for package structure
2025-12-10 13:36:05 +05:30
KaifAhmad1 da08354a96 Refactor Context Module: Architecture 2.0, Hierarchical Memory, and Doc Updates 2025-12-09 18:43:27 +05:30
KaifAhmad1 d5cb9b2d34 Update visualization notebooks: temporal snapshot comparison and version history; align APIs (KGVisualizer.visualize_network, EmbeddingVisualizer.visualize_2d_projection); add semantic network and multimodal/quality examples; refresh docs references 2025-12-08 23:17:03 +05:30
KaifAhmad1 06145fd4b9 docs(vector_store): finalize documentation with simplified notebooks
- Enhanced docs/reference/vector_store.md (~575 lines)
  - All 32 classes documented
  - All 10 convenience functions
  - Complete adapter documentation

- Updated cookbook/introduction/13_Vector_Store.ipynb
  - 10-step comprehensive guide

- Created cookbook/advanced/Advanced_Vector_Store_and_Search.ipynb
  - 4 focused parts (removed error handling per user request)
  - Part 1: Index selection (Flat, HNSW, IVF)
  - Part 2: Smart filtering with metadata
  - Part 3: Result fusion (RRF, weighted)
  - Part 4: Multi-tenant data isolation
  - Beginner-friendly with clear examples
  - Quick reference guide included

All vector_store documentation complete and production-ready.
2025-12-08 19:55:23 +05:30
KaifAhmad1 9fcb1c5410 docs(triple_store): complete documentation and add comprehensive notebook
- Enhanced docs/reference/triple_store.md
  - Added RDF4JAdapter section with features and transaction examples
  - Added VirtuosoAdapter section with named graphs and SQL integration
  - Added TripleStore dataclass documentation with attributes
  - Added QueryResult dataclass documentation with usage examples
  - Added QueryPlan dataclass documentation with optimization details
  - Added LoadProgress dataclass documentation with progress tracking

- Created comprehensive introduction notebook
  - Added cookbook/introduction/20_Triple_Store.ipynb
  - 10 comprehensive steps covering all triple_store functionality
  - All 11 classes demonstrated with examples
    - TripleManager, QueryEngine, BulkLoader
    - BlazegraphAdapter, JenaAdapter, RDF4JAdapter, VirtuosoAdapter
  - All 13 functions covered with practical examples
  - Multi-backend examples for all 4 store types
  - SPARQL query execution and optimization examples
  - Bulk loading with progress tracking
  - Multi-store operations and replication
  - Best practices and backend selection guide

All triple_store module exports (11 classes, 13 functions, 4 config items) now fully documented.
Documentation is 100% consistent with actual module exports.
2025-12-08 19:00:52 +05:30
KaifAhmad1 f5dfe426e9 docs(split): complete split module documentation and add comprehensive notebook
- Enhanced split_usage.md with missing class examples
  - Added OntologyAwareChunker with detailed parameters
  - Added SlidingWindowChunker with window_size and step_size
  - Added TableChunker with all configuration options
  - Updated 'Using Existing Chunkers' section with all 9 chunkers

- Enhanced docs/reference/split.md with complete class documentation
  - Added OntologyAwareChunker section (methods, parameters, examples)
  - Added SlidingWindowChunker section (methods, parameters, examples)
  - Added TableChunker section (methods, parameters, examples)
  - All sections include parameter tables and detailed examples

- Created comprehensive introduction notebook
  - Added cookbook/introduction/11_Chunking_and_Splitting.ipynb
  - 14 comprehensive steps covering all functionality
  - All 13 classes demonstrated (TextSplitter, SemanticChunker, EntityAwareChunker, etc.)
  - All 15 splitting functions covered with examples
  - Best practices, method comparisons, and performance tips included

- Removed duplicate advanced notebook
  - Deleted cookbook/advanced/11_Text_Chunking_Strategies.ipynb
  - Consolidated into comprehensive introduction notebook

All split module exports (13 classes, 15 functions, 4 config items) now fully documented.
Documentation is 100% consistent with actual module exports.
2025-12-08 18:06:08 +05:30
KaifAhmad1 31da5731b1 refactor(semantic_extract): remove build function and enhance documentation
BREAKING CHANGE: Removed build() convenience function from semantic_extract module

- Removed build() function from semantic_extract/__init__.py
- Updated __all__ exports to remove 'build'
- Resolved merge conflicts in named_entity_recognizer.py, relation_extractor.py, triple_extractor.py
- Updated semantic_extract_usage.md with class-based examples
- Updated docs/reference/semantic_extract.md with detailed parameter documentation
- Fixed 01_GraphRAG_Complete.ipynb to use individual extractor classes
- Enhanced 05_Entity_Extraction.ipynb with comprehensive examples (9 sections)
- Enhanced 06_Relation_Extraction.ipynb with complete pipeline examples (9 sections)

Users should now use individual classes (NERExtractor, RelationExtractor, TripleExtractor, etc.)
instead of the build() function for better control and flexibility.

Migration guide available in documentation.
2025-12-08 17:33:45 +05:30
KaifAhmad1 7b4b822553 Refactor seed module to class-based API, update docs and cookbooks 2025-12-08 16:14:24 +05:30
KaifAhmad1 a33e7ecb51 Cookbook: deep dive reasoning module; Rete stress test; performance comparison; mixed ReteInferenceExplanation workflow 2025-12-08 14:29:28 +05:30
KaifAhmad1 68f4eb6d2d chore: align reasoning module documentation and notebooks with implementation 2025-12-08 13:29:45 +05:30
KaifAhmad1 75ffcb1031 Standardize notebooks to handler-based pipeline API: add explicit step dependencies, use data injection for inputs, remove legacy func/args usage; update supply chain, intelligence, forensics, healthcare examples; refresh pipeline docs. 2025-12-07 23:03:39 +05:30
KaifAhmad1 902b332d9b cookbook(trading): migrate to StructuredDataParser.parse_data and dict access; fix backtesting notebook parsing and iteration; minor doc updates 2025-12-07 20:43:05 +05:30
KaifAhmad1 9837feec9b Refactor Unstructured Cookbook imports for better modularity 2025-12-07 16:15:50 +05:30
KaifAhmad1 1829b46340 Enhance Unstructured to Ontology Cookbook with LLM and Visualization 2025-12-07 16:14:11 +05:30
KaifAhmad1 694141297f Enhance Ontology Cookbook with visualization and advanced features 2025-12-07 15:58:53 +05:30
Semantica Bot 7878b4a222 cookbook/ontology: add worked example for object vs data properties; align worksFor to Schema.org; add optional hierarchy visualization; plus commit pending changes across notebooks, docs, and ontology modules 2025-12-07 15:21:16 +05:30
KaifAhmad1 6b7f230b37 docs(normalize): standardize docs to class usage; update cookbook notebooks; fix Colab link; correct text_normalizer.normalize → normalize_text 2025-12-07 00:17:57 +05:30
KaifAhmad1 dd39c544fc Remove kg_qa module and exports; update docs and notebooks to remove KG QA references and add temporary notices; adjust README Quality Assurance examples; add roadmap entry for KG QA in Q1; refine wording per request 2025-12-06 17:34:57 +05:30
KaifAhmad1 01791562f1 refactor(kg): Remove ConflictDetector and Deduplicator from kg module
- Remove ConflictDetector and Deduplicator from semantica.kg module
- Update all imports to use semantica.conflicts and semantica.deduplication
- Update all notebooks to use class-based API (no convenience functions)
- Fix method signatures: pass graph parameter to methods instead of constructor
- Update calculate_centrality calls to use specific methods (calculate_degree_centrality, etc.)
- Fix detect_communities and analyze_connectivity return value handling
- Update all documentation (kg_usage.md, docs/reference/kg.md)
- Remove conflict_detector.py and deduplicator.py from kg module
- Update registry.py to remove conflict and deduplicate task types
2025-12-06 16:17:27 +05:30
KaifAhmad1 d258ef6880 refactor(kg): Remove ConflictDetector and Deduplicator from kg module
- Remove ConflictDetector and Deduplicator from semantica.kg module
- Update all imports to use dedicated semantica.conflicts and semantica.deduplication modules
- Update all cookbook notebooks to use class-based API instead of convenience functions
- Fix calculate_centrality calls to use specific methods (calculate_degree_centrality, calculate_betweenness_centrality)
- Update detect_communities and analyze_connectivity calls to pass graph parameter
- Update documentation (kg_usage.md, docs/reference/kg.md) to reflect changes
- Remove conflict and deduplicate task types from method registry
2025-12-06 15:57:48 +05:30
KaifAhmad1 6dd9837aa6 Refactor ingest module and enhance documentation
- Removed deprecated 'build' convenience function from semantica/ingest/__init__.py to resolve conflicts and promote class-based usage.
- Updated 'docs/reference/ingest.md' to include missing main classes: FeedIngestor, EmailIngestor, DBIngestor, and MCPIngestor.
- Added 'Stream Monitoring' usage example to 'semantica/ingest/ingest_usage.md'.
- Completely rewrote 'cookbook/introduction/02_Data_Ingestion.ipynb' to provide a comprehensive, runnable guide covering all ingestion submodules and helper classes.
2025-12-06 14:32:10 +05:30
KaifAhmad1 0c10d5c876 Update Graph Store module documentation and notebooks
- Enhanced Graph Store notebook with comprehensive examples and clean formatting
- Fixed GraphStore API usage across all documentation files
- Updated examples to use keyword arguments (labels, properties, start_node_id, end_node_id, rel_type)
- Removed emojis and links from notebook for cleaner markdown
- Made summary section more concise
- Ensured consistency across cookbook notebooks, docs, and module code
2025-12-06 13:31:56 +05:30
KaifAhmad1 7da9f902e3 Update export module notebooks: comprehensive documentation and API consistency
- Enhanced introduction/15_Export.ipynb with complete module architecture documentation
- Enhanced advanced/05_Multi_Format_Export.ipynb with all export formats and classes
- Removed HTMLExporter references from intelligence notebooks (class doesn't exist)
- Fixed OWLExporter usage in healthcare notebook (removed invalid export_knowledge_graph call)
- Updated all notebooks to use only class imports, no convenience functions
- Added comprehensive documentation for all exporter classes and methods
- Improved markdown structure and learning objectives in both notebooks
2025-12-05 21:49:37 +05:30
KaifAhmad1 8b6e9a1e36 refactor: update embedding API in all notebooks and documentation
- Update all notebooks to use generate_embeddings() instead of generate()
- Update docs/reference/embeddings.md to remove references to removed components
- All notebooks now use data_type='text' parameter for embedding generation
- Updated 11 notebooks across introduction, use_cases, and advanced directories
2025-12-05 17:54:09 +05:30
KaifAhmad1 24b9fe3eb3 Refactor deduplication module documentation and notebook
- Restructured 18_Deduplication.ipynb with comprehensive module overview
- Added detailed explanations of module capabilities and architecture
- Improved markdown formatting and removed emojis
- Reorganized content to focus on module capabilities rather than individual classes
- Added clear examples for all major features
- Updated documentation for consistency across all files
2025-12-05 16:00:12 +05:30
KaifAhmad1 7d61f5b3ca Refactor core module: update config and lifecycle managers, add core_usage.md, update notebooks 2025-12-05 13:24:43 +05:30
KaifAhmad1 37d75f260e feat(context): Add comprehensive memory and context management methods
- Add memory management methods to AgentContext (exists, count, get, update, delete, clear, list, batch operations)
- Add search methods (search, find_similar, get_context, expand_query)
- Add conversation methods (get_conversation, list_conversations, delete_conversation, conversation_summary)
- Add export/import methods (export, import_data, backup, restore)
- Add statistics methods (stats, health, usage_stats)
- Add similar methods to AgentMemory, ContextRetriever, ContextGraphBuilder, EntityLinker
- Improve error messages with clear, actionable messages
- Update documentation (context_usage.md) with all new methods
- Update notebook (19_Context_Module.ipynb) - remove emojis, add new methods, clean formatting
- Improve error handling in methods.py
2025-12-05 00:44:08 +05:30
KaifAhmad1 a93700813f refactor(conflicts): Remove statistics functionality and update documentation
Resolved merge conflicts by:
- Removing statistics functionality from ConflictResolver and ConflictAnalyzer
- Removing detect_and_resolve convenience function
- Updating all documentation and examples
- Adding by_source analysis capability
- Updating method signatures to match new API
2025-12-04 22:19:04 +05:30
KaifAhmad1 e483ad164f Format GraphRAG Complete notebook: Add proper line breaks and formatting to all cells 2025-12-04 15:55:09 +05:30
KaifAhmad1 5071b60c79 Clean up cookbook: remove redundant notebooks and update documentation
- Remove 5 redundant use case notebooks:
  - renewable_energy/05_Supply_Chain_Analysis.ipynb
  - finance/05_Market_Intelligence.ipynb
  - trading/03_Real_Time_Market_Data.ipynb
  - intelligence/04_Network_Analysis_Intelligence_Reports.ipynb
  - healthcare/06_Medical_Literature_GraphRAG.ipynb
- Remove introduction/02_Configuration_Basics.ipynb
- Renumber all notebooks sequentially (01-18 for introduction, etc.)
- Update docs/cookbook.md:
  - Remove references to deleted notebooks
  - Update all notebook paths to use full GitHub URLs
  - Add missing Intelligence Analysis and Law Enforcement Forensics entries
- Remove duplicate docs/cookbook directory
2025-12-04 13:12:35 +05:30
KaifAhmad1 8b3680cd14 Refactor: Clean up 01_Welcome_to_Semantica notebook
- Remove Best Practices section
- Remove Key Concepts Explained section and all subsections
- Remove Next Steps section
- Remove Troubleshooting section
- Keep only essential introduction and framework architecture content
2025-12-03 23:36:36 +05:30
KaifAhmad1 57a1943ba6 Improve code quality in cookbook notebooks
- Remove unnecessary try-except blocks
- Simplify error handling with print statements
- Clean spacing issues (indentation, blank lines, trailing whitespace)
- Ensure consistent code formatting across all notebooks
2025-12-03 19:04:03 +05:30
KaifAhmad1 ca0b9028e0 Update cookbook documentation: Add PyPI installation instructions and update module lists
- Add PyPI installation instructions to all 72 cookbook notebooks
- Update module lists to include all 8 ingestion modules (FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, RepoIngestor, EmailIngestor, MCPIngestor)
- Reorder sections: Overview before Installation in all notebooks
- Remove duplicate content from introduction notebooks
- Update docs/cookbook.md with PyPI installation section and enhanced module descriptions
2025-12-03 17:22:48 +05:30
KaifAhmad1 7c5c9d9117 docs: organize cookbook notebooks and add Colab integration
- Add numbering to all notebooks for better sorting
  - Introduction: 01-19
  - Advanced: 01-12
  - Use cases: numbered within each category
- Add Google Colab badges to all 72 notebooks
- Clean up Welcome notebook with proper code cells
- Remove unnecessary print statements and verbose content
2025-12-03 13:00:36 +05:30
KaifAhmad1 2f0dc32276 docs: add numbering to cookbook notebooks and improve formatting
- Number all introduction notebooks (01-19)
- Number all advanced notebooks (01-12)
- Number all use case notebooks within each category
- Clean up Welcome notebook with proper code cells
- Remove unnecessary print statements
- Improve notebook organization and sorting
2025-12-03 12:42:32 +05:30
KaifAhmad1 c469f5455b feat(graph_store): Add Graph Store module to cookbook and examples
- Add new Graph_Store.ipynb introduction notebook
- Update Advanced_Graph_Analytics.ipynb with graph store persistence
- Update Fraud_Detection.ipynb with graph database storage
- Update Transaction_Network_Analysis.ipynb with blockchain graph storage
- Update Criminal_Network_Analysis.ipynb with criminal network persistence
- Update Welcome_to_Semantica.ipynb with Graph Store module documentation
- Update docs/cookbook.md, docs/examples.md, docs/CodeExamples.md
- Sync all notebooks to docs/cookbook directory
2025-11-26 16:55:55 +05:30
KaifAhmad1 2325991e6d Add intelligence cookbook notebooks: Criminal Network Analysis, Law Enforcement & Forensics, and Intelligence Analysis with Orchestrator-Worker pattern 2025-11-25 13:28:18 +05:30
KaifAhmad1 89a64a483b docs: enhance documentation structure and notebook formatting
- Update mkdocs.yml navigation to use new cookbook index
- Create comprehensive docs/cookbook.md index
- Refactor Welcome_to_Semantica.ipynb to use Markdown cells
- Enhance markdown formatting in Your_First_Knowledge_Graph.ipynb and Financial_Data_Integration.ipynb
- Update custom.css and version-selector.js for better styling
- Populate modules.md, concepts.md, and getting-started.md with detailed content
2025-11-23 14:17:36 +05:30
KaifAhmad1 750fc1f932 Add MCP ingestion examples to existing use case notebooks
- Added MCP ingestion as optional data source in Market Intelligence notebook
- Added MCP ingestion as optional data source in Clinical Reports Processing notebook
- Added MCP ingestion as optional data source in Threat Intelligence Hybrid RAG notebook
- Added MCP ingestion as optional data source in Drug Discovery Pipeline notebook
- Updated module lists and pipeline descriptions to include MCP
- Integrated MCP data merging with existing ingestion methods
2025-11-20 19:05:43 +05:30
KaifAhmad1 112a4126e2 Add three comprehensive use case notebooks to cookbook
- Healthcare GraphRAG System: Hybrid GraphRAG with materialized KGs and virtual EHRs
- Supply Chain Risk Management: Graph-based risk analysis with cascade effects
- Network Analysis Intelligence Reports: Graph analytics + AI for intelligence reporting

All notebooks use Semantica as the core framework and toolkit, demonstrating:
- End-to-end pipelines using Semantica modules
- Graph analytics, reasoning, and visualization
- Professional report generation
- Best practices and deployment recommendations
2025-11-17 23:04:17 +05:30
Mohd Kaif b6c8483529 Delete cookbook/REAL_DATA_SOURCES.md 2025-11-13 21:28:44 +05:30
KaifAhmad1 8286a8b91e Add blockchain and biomedical use cases: Transaction Network Analysis, DeFi Protocol Intelligence, Genomic Variant Analysis, and Drug Discovery Pipeline 2025-11-13 19:50:42 +05:30
KaifAhmad1 a3c461a0d6 Merge branch 'main' of https://github.com/Hawksight-AI/semantica 2025-11-13 18:28:34 +05:30
KaifAhmad1 6100d5e0d5 Complete cookbook implementation: Add all use case notebooks with real data sources
- Implemented all 25 use case notebooks across 5 domains (Trading, Healthcare, Finance, Cybersecurity, Renewable Energy)
- Added real data sources: APIs, RSS feeds, database connections, streaming sources
- Each notebook uses 20+ Semantica modules with complete end-to-end pipelines
- Multi-source ingestion with lists of URLs for feeds and APIs
- Complete workflows: ingestion → parsing → extraction → KG building → analysis → export → visualization
- All notebooks include realistic data structures and error handling
- Updated renewable energy notebooks: Environmental_Impact, Grid_Management, Resource_Optimization, Supply_Chain_Analysis
2025-11-13 18:27:33 +05:30
Mohd Kaif 817811dad6 Delete cookbook/specialized_applications directory 2025-11-13 16:54:34 +05:30