Commit Graph
74 Commits
Author SHA1 Message Date
KaifAhmad1 37bc3add62 Update documentation and changelog for Pinecone support 2026-01-26 21:44:44 +05:30
KaifAhmad1 b3a239ccb1 feat: enhance semantic extraction with BYOM support, NER aggregation, RE implementation, and Triplet improvements
- Implemented 'Bring Your Own Model' (BYOM) support for NER, Relation, and Triplet extraction
- Added NER aggregation strategies (simple, max, average)
- Implemented Relation Extraction via Sequence Classification with entity markers
- Enhanced Triplet Extraction with REBEL post-processing and lazy loading
- Updated all extractors to prioritize runtime options over config defaults
- Added extensive tests and examples (huggingface_demo.py)
- Updated documentation and CHANGELOG
2026-01-24 20:53:21 +05:30
KaifAhmad1 b96e71ae72 feat(ontology): Implement OntologyIngestor and update exports
- Added OntologyIngestor in semantica/ingest/ontology_ingestor.py
- Updated semantica/ontology/__init__.py to export OntologyIngestor
- Updated semantica/ingest/methods.py to use OntologyIngestor
- Added tests for ontology ingestion
- Cleaned up temporary files
2026-01-21 13:46:46 +05:30
KaifAhmad1 1568237ce7 Add high-performance VectorStore ingestion and docs 2026-01-19 13:32:16 +05:30
KaifAhmad1 dd7fcd3ddb [FEATURE] Performance Bottlenecks and Scaling Limitations in semantic_extract #186
- Implemented high-throughput parallel batch processing across all core extractors (NERExtractor, RelationExtractor, TripletExtractor, EventDetector, SemanticNetworkExtractor) using ThreadPoolExecutor.

- Added max_workers configuration parameter (default: 1) to all extractor extract() methods.

- Implemented parallel processing for large document chunking in _extract_entities_chunked and _extract_relations_chunked.

- Enhanced ProgressTracker to be thread-safe.

- Optimized setUpClass in tests to reduce Groq LLM initialization overhead.

- Updated documentation and usage examples.
2026-01-14 00:11:30 +05:30
KaifAhmad1 8c3f7f1f0a fix(semantic-extract): resolve incomplete output #176, relax constraints, and add Groq support 2026-01-12 17:15:21 +05:30
KaifAhmad1 f338b66274 feat: Add provenance metadata and robust fallback chains to semantic extraction
- Implemented ML/LLM -> Pattern -> Last Resort fallback chains for NER, Relation, and Triplet extractors to prevent empty results.
- Added provenance metadata (batch_index, document_id) to all extraction schemas (Entity, Relation, Triplet, etc.).
- Unified batch processing API with progress tracking across all extractors.
- Updated documentation (module usage and reference docs) to reflect new features.
- Added robustness and batch provenance tests.
2026-01-10 20:43:09 +05:30
KaifAhmad1 a5c531fd29 Fix semantic extraction empty returns, schema validation, and update docs 2026-01-09 18:39:09 +05:30
KaifAhmad1 957c122116 docs: add Docling integration guide, clear code example, and fix parser consistency issues 2026-01-08 21:27:48 +05:30
KaifAhmad1 dcd6f25f87 docs: fix KnowledgeGraph mismatch and update tests for issue #144 2026-01-07 15:01:47 +05:30
KaifAhmad1 e712949872 Enhance LLM extraction methods with auto-chunking, robust parsing and improved diagnostics (#149) 2026-01-07 03:03:19 +05:30
KaifAhmad1 d7b686f32a Release v0.1.1: Docling support, version bump, and documentation updates 2026-01-06 00:21:20 +05:30
KaifAhmad1 04eea7e7eb Update documentation: reduce code examples, add cookbook links, improve structure
- Reduced code examples in all guide pages (getting-started, quickstart, concepts, modules, examples, use-cases, learning-more)
- Added comprehensive cookbook links with descriptions (topics, difficulty, time, use cases)
- Improved structure and organization across all guide pages
- Updated use-cases.md to only include use cases with corresponding cookbooks
- Removed 'Last Updated: 2024' from all documentation files
- Enhanced navigation with better 'Next Steps' sections
2025-12-31 15:19:08 +05:30
KaifAhmad1 35391382d3 docs: configure github pages deployment and fix broken links 2025-12-31 12:37:36 +05:30
KaifAhmad1 c3555e0cfd Add LLM providers module and GraphRAG reasoning features
- Add semantica.llms module with Groq, OpenAI, HuggingFace, and LiteLLM providers
- Add query_with_reasoning() method for multi-hop reasoning with LLM-generated responses
- Update ContextRetriever and AgentContext with reasoning capabilities
- Add comprehensive documentation for LLM providers and GraphRAG reasoning
- Update README and docs with new features
- Update notebook examples to use new query_with_reasoning() method
2025-12-27 23:23:32 +05:30
KaifAhmad1 eaa1fbefa6 feat(reasoning): add dedicated reasoning tests and fix critical reasoning bugs
- Added tests/reasoning/ directory with unit and integration tests
- Fixed indentation bug in Reasoner.add_fact for dictionary-based relationships
- Fixed regex variable matching in Reasoner._match_pattern
- Fixed variable handling in SPARQLReasoner query expansion
- Cleaned up cookbook and documentation references
2025-12-23 21:26:26 +05:30
KaifAhmad1 7046c92b3a Merge main and resolve conflicts by prioritizing audited reasoning refactor 2025-12-23 15:24:07 +05:30
KaifAhmad1 1ca83dd3c9 Comprehensive reasoning module cleanup: removed InferenceEngine, updated documentation, and synchronized cookbooks project-wide 2025-12-23 15:14:52 +05:30
KaifAhmad1 a9bd3be689 Update context module docs, cleanup notebook, and refactor context files 2025-12-21 16:50:56 +05:30
KaifAhmad1 1fceb634ae chore: remove 07_Pipeline_Orchestration notebook and all references 2025-12-20 20:18:14 +05:30
KaifAhmad1 5fc188b9eb Refactor: Rename LLMEnhancer to LLMExtraction 2025-12-19 00:16:44 +05:30
KaifAhmad1 38962d3d7a Refactor triplet_store: Unified TripletStore interface, removed Virtuoso/TripletManager, added Blazegraph/Jena/RDF4J support, updated docs and notebooks 2025-12-18 21:37:11 +05:30
KaifAhmad1 8d6b38d7da Refactor Context Engineering module, rebuild advanced notebook, and update README 2025-12-18 18:30:16 +05:30
KaifAhmad1 7297d46ac8 Fix deduplication logic: Jaro-Winkler default, disjoint property handling, and docs update 2025-12-17 23:08:38 +05:30
KaifAhmad1 0a052b676d Refactor deduplication module to support simplified string-based merge strategies and update documentation 2025-12-17 22:27:16 +05:30
KaifAhmad1 b091c870bc Improve conflicts docs and notebook; align conflicts APIs 2025-12-17 19:12:16 +05:30
KaifAhmad1 72d948972b Update ontology module: 6-stage pipeline, OntologyValidator integration, and documentation updates 2025-12-17 00:10:51 +05:30
KaifAhmad1 dd6b341fb9 Refactor: Rename Adapter to Store across Vector, Graph, and Triplet stores. Update docs and tests. 2025-12-16 20:45:11 +05:30
KaifAhmad1 96702923de Remove QA components (OntologyValidator, ConflictDetector, etc) and fix residual references 2025-12-16 17:25:15 +05:30
KaifAhmad1 4fdc483935 Refactor terminology: Triple -> Triplet across codebase, docs, and notebooks 2025-12-15 16:09:20 +05:30
KaifAhmad1 f3dd7a05bd Refactor: Remove Pinecone and enhance vector store backend support
- Removed all Pinecone references, adapters, and documentation to align with open-source, self-hosted focus.
- Removed PineconeAdapter and related dependencies.
- Updated VectorStore to enforce supported backends (FAISS, Weaviate, Qdrant, Milvus, InMemory).
- Updated cookbooks (e.g., 13_Vector_Store.ipynb) to use Weaviate/FAISS examples instead of Pinecone.
- Updated core documentation (modules.md, rchitecture.md, etc.) to reflect backend changes.
- Added new tests (	est_pinecone_removal.py, 	est_vector_store_deepdive.py) to verify removal and validate remaining backends.
- Verified all vector store tests pass.
2025-12-12 20:19:17 +05:30
KaifAhmad1 6856580a7a Refactor: Rename triple_store to triplet_store across codebase
- Renamed semantica/triple_store to semantica/triplet_store
- Updated all imports and class references in core modules and adapters
- Refactored Jupyter notebooks in cookbook/
- Updated documentation files (README, docs/, etc.)
- Updated tests and verified passing status
2025-12-12 18:45:00 +05:30
KaifAhmad1 84b90b45a2 fix: align split methods with documentation and registry 2025-12-12 16:15:57 +05:30
KaifAhmad1 d3366bbcf0 Refactor Semantic Extract module: Update notebooks, docs, and implementation to use class-based interfaces 2025-12-12 13:23:37 +05:30
KaifAhmad1 afc94ad059 Remove KuzuDB backend support and cleanup references 2025-12-10 20:30:59 +05:30
KaifAhmad1 3207eb3b41 Fix export_yaml schema export bug and update docs
- Fix YAMLSchemaExporter method call in export_yaml (use export_ontology_schema)
- Add file writing logic to export_yaml for schema method
- Update docs/reference/export.md and semantica/export/export_usage.md with correct method signature
- Add test_export_methods_wrapper.py to verify schema export
- Prevent infinite recursion in method_registry lookups in methods.py
2025-12-10 18:40:32 +05:30
KaifAhmad1 6ee19d971e feat: enhance embeddings with dynamic model switching, updated docs and tests 2025-12-10 17:32:43 +05:30
KaifAhmad1 732729e707 refactor(docs): improve format and organization of context and embeddings reference 2025-12-09 21:08:55 +05:30
KaifAhmad1 9445d96fff docs: Update context and embeddings reference docs with grid cards 2025-12-09 20:23:12 +05:30
KaifAhmad1 da08354a96 Refactor Context Module: Architecture 2.0, Hierarchical Memory, and Doc Updates 2025-12-09 18:43:27 +05:30
KaifAhmad1 cf56dad82a docs: update cookbook links to absolute GitHub URLs in reference docs 2025-12-09 16:51:28 +05:30
KaifAhmad1 d5cb9b2d34 Update visualization notebooks: temporal snapshot comparison and version history; align APIs (KGVisualizer.visualize_network, EmbeddingVisualizer.visualize_2d_projection); add semantic network and multimodal/quality examples; refresh docs references 2025-12-08 23:17:03 +05:30
KaifAhmad1 06145fd4b9 docs(vector_store): finalize documentation with simplified notebooks
- Enhanced docs/reference/vector_store.md (~575 lines)
  - All 32 classes documented
  - All 10 convenience functions
  - Complete adapter documentation

- Updated cookbook/introduction/13_Vector_Store.ipynb
  - 10-step comprehensive guide

- Created cookbook/advanced/Advanced_Vector_Store_and_Search.ipynb
  - 4 focused parts (removed error handling per user request)
  - Part 1: Index selection (Flat, HNSW, IVF)
  - Part 2: Smart filtering with metadata
  - Part 3: Result fusion (RRF, weighted)
  - Part 4: Multi-tenant data isolation
  - Beginner-friendly with clear examples
  - Quick reference guide included

All vector_store documentation complete and production-ready.
2025-12-08 19:55:23 +05:30
KaifAhmad1 9fcb1c5410 docs(triple_store): complete documentation and add comprehensive notebook
- Enhanced docs/reference/triple_store.md
  - Added RDF4JAdapter section with features and transaction examples
  - Added VirtuosoAdapter section with named graphs and SQL integration
  - Added TripleStore dataclass documentation with attributes
  - Added QueryResult dataclass documentation with usage examples
  - Added QueryPlan dataclass documentation with optimization details
  - Added LoadProgress dataclass documentation with progress tracking

- Created comprehensive introduction notebook
  - Added cookbook/introduction/20_Triple_Store.ipynb
  - 10 comprehensive steps covering all triple_store functionality
  - All 11 classes demonstrated with examples
    - TripleManager, QueryEngine, BulkLoader
    - BlazegraphAdapter, JenaAdapter, RDF4JAdapter, VirtuosoAdapter
  - All 13 functions covered with practical examples
  - Multi-backend examples for all 4 store types
  - SPARQL query execution and optimization examples
  - Bulk loading with progress tracking
  - Multi-store operations and replication
  - Best practices and backend selection guide

All triple_store module exports (11 classes, 13 functions, 4 config items) now fully documented.
Documentation is 100% consistent with actual module exports.
2025-12-08 19:00:52 +05:30
KaifAhmad1 f5dfe426e9 docs(split): complete split module documentation and add comprehensive notebook
- Enhanced split_usage.md with missing class examples
  - Added OntologyAwareChunker with detailed parameters
  - Added SlidingWindowChunker with window_size and step_size
  - Added TableChunker with all configuration options
  - Updated 'Using Existing Chunkers' section with all 9 chunkers

- Enhanced docs/reference/split.md with complete class documentation
  - Added OntologyAwareChunker section (methods, parameters, examples)
  - Added SlidingWindowChunker section (methods, parameters, examples)
  - Added TableChunker section (methods, parameters, examples)
  - All sections include parameter tables and detailed examples

- Created comprehensive introduction notebook
  - Added cookbook/introduction/11_Chunking_and_Splitting.ipynb
  - 14 comprehensive steps covering all functionality
  - All 13 classes demonstrated (TextSplitter, SemanticChunker, EntityAwareChunker, etc.)
  - All 15 splitting functions covered with examples
  - Best practices, method comparisons, and performance tips included

- Removed duplicate advanced notebook
  - Deleted cookbook/advanced/11_Text_Chunking_Strategies.ipynb
  - Consolidated into comprehensive introduction notebook

All split module exports (13 classes, 15 functions, 4 config items) now fully documented.
Documentation is 100% consistent with actual module exports.
2025-12-08 18:06:08 +05:30
KaifAhmad1 31da5731b1 refactor(semantic_extract): remove build function and enhance documentation
BREAKING CHANGE: Removed build() convenience function from semantic_extract module

- Removed build() function from semantic_extract/__init__.py
- Updated __all__ exports to remove 'build'
- Resolved merge conflicts in named_entity_recognizer.py, relation_extractor.py, triple_extractor.py
- Updated semantic_extract_usage.md with class-based examples
- Updated docs/reference/semantic_extract.md with detailed parameter documentation
- Fixed 01_GraphRAG_Complete.ipynb to use individual extractor classes
- Enhanced 05_Entity_Extraction.ipynb with comprehensive examples (9 sections)
- Enhanced 06_Relation_Extraction.ipynb with complete pipeline examples (9 sections)

Users should now use individual classes (NERExtractor, RelationExtractor, TripleExtractor, etc.)
instead of the build() function for better control and flexibility.

Migration guide available in documentation.
2025-12-08 17:33:45 +05:30
KaifAhmad1 7b4b822553 Refactor seed module to class-based API, update docs and cookbooks 2025-12-08 16:14:24 +05:30
KaifAhmad1 68f4eb6d2d chore: align reasoning module documentation and notebooks with implementation 2025-12-08 13:29:45 +05:30
KaifAhmad1 75ffcb1031 Standardize notebooks to handler-based pipeline API: add explicit step dependencies, use data injection for inputs, remove legacy func/args usage; update supply chain, intelligence, forensics, healthcare examples; refresh pipeline docs. 2025-12-07 23:03:39 +05:30
KaifAhmad1 902b332d9b cookbook(trading): migrate to StructuredDataParser.parse_data and dict access; fix backtesting notebook parsing and iteration; minor doc updates 2025-12-07 20:43:05 +05:30