Compare commits

..
184 Commits
Author SHA1 Message Date
KaifAhmad1 b3f3ac413c Add Enhanced Change Management Module with comprehensive testing and documentation
- New semantica.change_management module with persistent version storage
- Core classes: TemporalVersionManager, OntologyVersionManager, ChangeLogEntry
- Storage backends: SQLite (persistent) and InMemory (fast)
- Features: SHA-256 checksums, detailed entity/relationship diffs, email validation
- Compliance: HIPAA, SOX, FDA 21 CFR Part 11 support with audit trails
- Testing: 104 tests (100% pass) - unit, integration, compliance, performance
- Performance: 17.6ms for 10k entities, 510+ ops/sec concurrent
- Documentation: Complete usage guide and API reference
- Backward compatible with simplified class names
2026-01-30 15:51:07 +05:30
Mohd Kaif ea8a250186 Update README.md 2026-01-29 17:04:38 +05:30
KaifAhmad1 1d64d58741 docs(changelog): note PRs #244, #242, #241, #239 in Unreleased 2026-01-29 00:47:00 +05:30
KaifAhmad1 3a091872ee Merge PR #244 after resolving conflicts 2026-01-29 00:32:26 +05:30
KaifAhmad1 979653e498 Finalize CSV tests after conflict resolution 2026-01-29 00:32:07 +05:30
KaifAhmad1 1a95b0d35f Resolve merge conflicts for PR #244: keep corrected PandasIngestor.from_csv implementation and expanded CSV tests 2026-01-29 00:30:14 +05:30
KaifAhmad1 589dd8c61e Merge pull request #244: Enhance CSV File Ingestion 2026-01-29 00:21:51 +05:30
KaifAhmad1 95ea8de455 CSV ingestion: fix header handling and duplicate header kwarg; add edge-case tests (tab, quoted, multiline, chunksize, NaN) 2026-01-29 00:21:40 +05:30
saloni 327792c830 test_ingest_from_csv().py 2026-01-29 00:09:12 +05:30
saloni 017a36591d pandas_ingestor.py 2026-01-29 00:07:19 +05:30
saloni e9ec904d87 Create test_ingest_from_csv().py 2026-01-28 23:39:19 +05:30
saloni b6f7542600 pandas_ingestor.py 2026-01-28 23:37:31 +05:30
saloni e8c93def07 pandas_ingestor.py 2026-01-28 23:33:29 +05:30
saloni 74cb3c6ac2 pandas_ingestor.py 2026-01-28 23:31:18 +05:30
saloni 0197062dfc pandas_ingestor.py 2026-01-28 21:26:07 +05:30
KaifAhmad1 274114ae67 Merge PR #242: add comprehensive tests for TextNormalizer; adjust punctuation normalization and preserve-case expectation 2026-01-28 20:25:34 +05:30
KaifAhmad1 4ec94b6a5d test(normalize): fix preserve-case expectation; feat(normalize): use explicit unicode mappings for punctuation normalization 2026-01-28 20:25:21 +05:30
ZohaibHassan16 eb1886bee3 test: add compre test units for TextNormalizer 2026-01-28 18:00:18 +05:00
Mohd Kaif cb91321360 Merge pull request #241 from Hawksight-AI/fix/ingest-tests-tweaks
test: register integration mark and tidy ingest test warnings
2026-01-28 14:23:00 +05:30
KaifAhmad1 d514e6b4cf test: register integration mark via pytest.ini; tidy test warnings 2026-01-28 14:18:29 +05:30
KaifAhmad1 bc875450fa test(ingest): add unit tests for file, web, and feed ingestors (#239) 2026-01-28 14:15:15 +05:30
KaifAhmad1 400a70986d test(ingest): mock boto3 client in tests; fix FeedParser._parse_date to raise ValueError on invalid input 2026-01-28 14:13:15 +05:30
Mohammed237 15b32f49be test(ingest): add unit tests for file, web, and feed ingestors 2026-01-27 18:53:05 +02:00
KaifAhmad1 3968a450a8 chore: release v0.2.5 2026-01-27 22:01:25 +05:30
KaifAhmad1 57d9c2006e feat: enhance Hugging Face integration with robust BYOM support and improved triplet/relation extraction 2026-01-27 21:55:22 +05:30
Mohd Kaif c6496d2193 Update README.md 2026-01-27 21:00:10 +05:30
Mohd Kaif 1812c8141f Update README.md 2026-01-27 20:55:49 +05:30
KaifAhmad1 b6931c45b6 Add sponsor button configuration and update sponsorship section 2026-01-27 16:43:21 +05:30
Mohd Kaif b52fe93182 Update README.md 2026-01-27 16:31:08 +05:30
Mohd Kaif c837cf1859 Update README.md 2026-01-27 16:23:14 +05:30
KaifAhmad1 65ac458b20 Update Readme with Logo Alignment 2026-01-27 16:21:02 +05:30
KaifAhmad1 a3e3b3cc2b Update README: Add Docling, AWS Neptune, and custom ontology support mentions. Remove metrics and accuracy claims. Tone down promotional language. 2026-01-27 16:18:46 +05:30
KaifAhmad1 b89658116d Update README: restructure top sections, add semantic gap explanation, balance emojis, remove traceability mentions 2026-01-27 16:06:45 +05:30
KaifAhmad1 a60a8ffe3b Update README: clarify framework positioning, add high-stakes use cases, and emphasize semantic layer building 2026-01-27 15:44:12 +05:30
Mohd Kaif 072bf92e83 Merge pull request #224 from Hawksight-AI/docs
docs: update CONTRIBUTING.md and CONTRIBUTORS.md with improved format…
2026-01-27 11:48:36 +05:30
KaifAhmad1 91f5a8b15f docs: update CONTRIBUTING.md and CONTRIBUTORS.md with improved formatting and fork mentions 2026-01-27 11:46:34 +05:30
Mohd Kaif 8ded19a2c8 Merge pull request #223 from Hawksight-AI/docs
docs: update CONTRIBUTING.md with improved formatting and documentati…
2026-01-27 11:37:02 +05:30
KaifAhmad1 ca04bfd1e9 docs: update CONTRIBUTING.md with improved formatting and documentation guidelines 2026-01-27 11:35:08 +05:30
Mohd Kaif 73732cfbb8 Merge pull request #222 from Hawksight-AI/docs
Integrate Pinecone Vector Store & Update Docs
2026-01-26 21:48:24 +05:30
KaifAhmad1 37bc3add62 Update documentation and changelog for Pinecone support 2026-01-26 21:44:44 +05:30
KaifAhmad1 5b2ad5e43c Merge branch 'abhiishekk31/main' into pr-fix: Resolve conflicts in Pinecone store implementation
Closes #219
2026-01-26 21:36:09 +05:30
Mohd Kaif 18dd0fbe09 Merge pull request #221 from Hawksight-AI/pr-fix
Pr fix
2026-01-26 21:29:30 +05:30
KaifAhmad1 ebefa61745 Merge branch 'abhiishekk31/main' into pr-fix: Resolve conflicts in Pinecone store implementation 2026-01-26 21:27:46 +05:30
KaifAhmad1 390835ec80 fix: Apply code review fixes for Pinecone integration (PR #220)
- Fix variable shadowing in fetch_vectors (use vector_id instead of id)
- Remove redundant PINECONE_AVAILABLE check in create_index
- Add Pinecone imports and exports to __init__.py
- Add 'pinecone' to SUPPORTED_BACKENDS in vector_store.py
- Add vectorstore-pinecone dependency group to pyproject.toml
- Create vectorstore-all optional dependency group
- Fix duplicate MagicMock import in test_pinecone_store.py
- Update test_pinecone_removal.py with explanatory comment
- Update all docstrings to include Pinecone in supported backends

All fixes address code review feedback and ensure proper integration.
2026-01-26 20:49:29 +05:30
Abhishek Hede 5443a221a0 Added pinecone support with required interface code 2026-01-26 09:53:21 +00:00
Mohd Kaif 6c9497cf40 Merge pull request #218 from Hawksight-AI/semantic-extract
Fix stuck retries in extraction and enable configurable retry limit
2026-01-25 21:33:43 +05:30
KaifAhmad1 bc55dcc57a Fix stuck retries in extraction and enable configurable retry limit. Resolves #207 2026-01-25 21:27:03 +05:30
Mohd Kaif 246119f48a Merge pull request #217 from Hawksight-AI/semantic-extract
Semantic Extraction Module Overhaul (BYOM, RE, Triplet, NER)
2026-01-24 20:59:14 +05:30
KaifAhmad1 b3a239ccb1 feat: enhance semantic extraction with BYOM support, NER aggregation, RE implementation, and Triplet improvements
- Implemented 'Bring Your Own Model' (BYOM) support for NER, Relation, and Triplet extraction
- Added NER aggregation strategies (simple, max, average)
- Implemented Relation Extraction via Sequence Classification with entity markers
- Enhanced Triplet Extraction with REBEL post-processing and lazy loading
- Updated all extractors to prioritize runtime options over config defaults
- Added extensive tests and examples (huggingface_demo.py)
- Updated documentation and CHANGELOG
2026-01-24 20:53:21 +05:30
Mohd Kaif 3c8bc84d18 Update README.md 2026-01-22 18:57:20 +05:30
Mohd Kaif 7f6d0fdcc4 Update README.md 2026-01-22 18:44:52 +05:30
Mohd Kaif 401ef70372 Update README.md 2026-01-22 18:43:30 +05:30
Mohd Kaif 35ce5c9b81 Update README.md 2026-01-22 18:32:57 +05:30
KaifAhmad1 b382a7df6e chore: release version 0.2.4 2026-01-22 12:50:07 +05:30
Mohd Kaif b35081e015 Delete examples/demo_ontology_ingest.py 2026-01-21 18:27:06 +05:30
Mohd Kaif 7459393eea Merge pull request #214 from Hawksight-AI/ontology
feat(ontology): Implement OntologyIngestor and update exports
2026-01-21 13:51:54 +05:30
KaifAhmad1 b96e71ae72 feat(ontology): Implement OntologyIngestor and update exports
- Added OntologyIngestor in semantica/ingest/ontology_ingestor.py
- Updated semantica/ontology/__init__.py to export OntologyIngestor
- Updated semantica/ingest/methods.py to use OntologyIngestor
- Added tests for ontology ingestion
- Cleaned up temporary files
2026-01-21 13:46:46 +05:30
KaifAhmad1 fa8544c6d6 Release v0.2.3: Update version, changelog, and documentation 2026-01-20 12:08:46 +05:30
Mohd Kaif 87649b7422 Merge pull request #213 from Hawksight-AI/docs
Fix earnings call analysis notebook: attribute access and export logic
2026-01-20 01:52:42 +05:30
KaifAhmad1 d91619f191 Fix earnings call analysis notebook: attribute access and export logic 2026-01-20 01:51:29 +05:30
Mohd Kaif 064a0db7e6 Merge pull request #212 from Hawksight-AI/docs
Optimize Vector DB Storage in Earnings Call Analysis Notebook
2026-01-19 16:37:33 +05:30
KaifAhmad1 8214acc675 optimize vector db storage in earnings call analysis 2026-01-19 16:32:22 +05:30
Mohd Kaif 2bf55485ff Merge pull request #211 from Hawksight-AI/vector-store
Vector Store Performance Optimization
2026-01-19 13:40:44 +05:30
KaifAhmad1 1568237ce7 Add high-performance VectorStore ingestion and docs 2026-01-19 13:32:16 +05:30
Mohd Kaif f6c9d50e03 Merge pull request #210 from Hawksight-AI/docs
docs: update earnings call analysis notebook
2026-01-18 23:54:44 +05:30
KaifAhmad1 d9117b7c2f docs: update earnings call analysis notebook 2026-01-18 23:53:07 +05:30
Mohd Kaif 0eabfb861e Merge pull request #209 from Hawksight-AI/kg
Fix GraphBuilder External Relationships (#208, #206)
2026-01-18 22:13:16 +05:30
KaifAhmad1 9f77dfb761 Fix GraphBuilder external relationships; refs #208 #206 2026-01-18 22:10:02 +05:30
Mohd Kaif c990d09bd3 Merge pull request #205 from Hawksight-AI/docs
docs: changelog entry for JupyterLab progress flag (#181)
2026-01-17 17:14:54 +05:30
Mohd Kaif 9ebacf43c3 Update CHANGELOG.md 2026-01-17 17:09:41 +05:30
KaifAhmad1 7958ae78f6 docs: changelog entry for JupyterLab progress flag (#181) 2026-01-17 17:03:51 +05:30
Mohd Kaif 2c61fe6cda Merge pull request #204 from Hawksight-AI/utils
feat: allow disabling Jupyter progress output (#181)
2026-01-17 16:44:03 +05:30
KaifAhmad1 92b850ac26 feat: allow disabling Jupyter progress output (#181) 2026-01-17 16:40:15 +05:30
Mohd Kaif f7bd7016c5 Merge pull request #203 from Hawksight-AI/utils
Circular import between `pipeline_builder` and `pipeline_validator`
2026-01-17 16:12:02 +05:30
KaifAhmad1 8671385cbf fix: break pipeline circular import (#192, #193) and update changelog 2026-01-17 16:02:21 +05:30
Mohd Kaif b358acfabf Merge pull request #202 from Hawksight-AI/docs
Update Coockbook
2026-01-16 23:08:03 +05:30
KaifAhmad1 a39ec5fd20 Faster, class-based dedup: DuplicateDetector+EntityMerger with strict thresholds; build graph from deduplicated outputs; clean prints 2026-01-16 22:32:19 +05:30
KaifAhmad1 bbd6764215 Use deduplicated entities/relationships; optimize and clean deduplication; disable extra merging in GraphBuilder 2026-01-16 18:18:50 +05:30
KaifAhmad1 1b0b0551db Update Earnings Call Analysis notebook 2026-01-16 17:54:44 +05:30
Mohd Kaif a6b102fa3d Merge pull request #201 from don-simpson/feature/amazon-neptune-setup
feat: Added CloudFormation template and cookbook instructions for Amazon Neptune
2026-01-16 12:37:23 +05:30
Don Simpson 65d99f7f8a Added CloudFormation template that creates a dev cluster with a single [t3 instance](https://docs.aws.amazon.com/neptune/latest/userguide/manage-console-instances-t3.html) configured with a [public endpoint](https://docs.aws.amazon.com/neptune/latest/userguide/neptune-public-endpoints.html) and IAM Auth enabled (required for public endpoint), and creates an IAM User using least-privilege principles. See Get started with Neptune Database for free on the [Amazon Neptune pricing page](https://aws.amazon.com/neptune/pricing/).
Includes the CloudFormation template in the same directory as the [Amazon Neptune Cookbook](https://github.com/Hawksight-AI/semantica/blob/main/cookbook/introduction/21_Amazon_Neptune_Store.ipynb) and references it as a prerequisite in the cookbook.
2026-01-15 18:48:49 -05:00
Mohd Kaif 9b81137b26 Merge pull request #200 from Hawksight-AI/docs
Update earnings call analysis notebook with relation extraction fixes
2026-01-16 03:05:07 +05:30
KaifAhmad1 653523efeb Update earnings call analysis notebook with relation extraction fixes
- Update notebook to use corrected RelationExtractor API
- Move provider/model parameters to initialization
- Add verbose logging for debugging
- Include working relation extraction examples
2026-01-16 03:03:11 +05:30
Mohd Kaif ba04421d9b Merge pull request #199 from Hawksight-AI/docs
Update changelog for LLM relation extraction fixes
2026-01-16 02:59:44 +05:30
KaifAhmad1 5d3fe51dbd Update changelog for LLM relation extraction fixes
- Add comprehensive changelog entry for relation extraction parsing fixes
- Document breaking changes and new test coverage
- Update with provider normalization and JSON fallback details
2026-01-16 02:58:28 +05:30
Mohd Kaif f20782f517 Merge pull request #198 from Hawksight-AI/semantic-extract
Fix LLM Relation Extraction
2026-01-16 01:22:37 +05:30
KaifAhmad1 96dc5d754a Fix LLM relation extraction parsing and add tests
- Harden LLM relation extraction result handling to parse instructor/OpenAI/Groq variations
- Add structured JSON fallback when typed generation yields zero relations
- Strip acceptance of extra kwargs like max_tokens/max_entities_prompt in relation extraction internals
- Add comprehensive unit tests with mocked LLM provider
- Add integration tests for Groq provider with environment variable API key
- Ensure relation extraction completes and returns results when model identifies relations
2026-01-16 01:19:33 +05:30
Mohd Kaif cf84526cc7 Merge pull request #197 from Hawksight-AI/semantic-extract
Robust LLM Extraction and Groq 401 Fix
2026-01-15 22:46:20 +05:30
KaifAhmad1 5ad20abeab fix(semantic_extract): fix Groq 401 error and improve LLM provider robustness with instructor.from_provider 2026-01-15 22:43:11 +05:30
Mohd Kaif ade08a65ae Merge pull request #196 from Hawksight-AI/semantic-extract
Enhance RelationExtractor with core fixes and verbose logs
2026-01-15 19:03:36 +05:30
KaifAhmad1 fb25644fa7 Enhance RelationExtractor with core fixes and verbose logs
- Fix excessive entities being passed to LLM in RelationExtractor
- Add comprehensive 'Heartbeat' verbose logs to methods.py and providers.py
- Ensure robust API key handling and explicit error reporting
2026-01-15 19:00:42 +05:30
Mohd Kaif 63899f2427 Merge pull request #195 from Hawksight-AI/semantic-extract
Robust Semantic Extraction - API Key Handling & Error Reporting
2026-01-15 18:02:01 +05:30
KaifAhmad1 fd6e058275 feat(semantic_extract): enhance error reporting and API key robustness 2026-01-15 17:59:04 +05:30
Mohd Kaif 23d8207ef5 Merge pull request #194 from Hawksight-AI/semantic-extract
Robust API Key Handling in Semantic Extract Module
2026-01-15 16:32:52 +05:30
KaifAhmad1 f2a11fc8ad fix: robust api_key handling in semantic_extract module 2026-01-15 16:29:52 +05:30
KaifAhmad1 c6316ba4bd Release 0.2.2 2026-01-15 00:42:07 +05:30
Mohd Kaif b6d630fc74 Merge pull request #191 from Hawksight-AI/semantic-extract
Improve `semantic_extract` performance and add Groq LLM smoke tests
2026-01-14 17:21:26 +05:30
Mohd Kaif 3f2cb49e50 Delete PR_DESCRIPTION.md 2026-01-14 17:17:32 +05:30
KaifAhmad1 c7814616a9 Improve semantic_extract performance and add Groq LLM smoke tests 2026-01-14 17:11:26 +05:30
Mohd Kaif 531014fbda Update version and description in pyproject.toml 2026-01-14 14:05:36 +05:30
Mohd Kaif 1cf9b34e3e Merge pull request #190 from Hawksight-AI/utils
docs: update CHANGELOG.md with recent changes
2026-01-14 12:51:40 +05:30
KaifAhmad1 2e81c86489 docs: update CHANGELOG.md with recent changes 2026-01-14 12:49:29 +05:30
Mohd Kaif 1690fec3f7 Merge pull request #189 from Hawksight-AI/utils
resolve dependencies, migrate Gemini SDK, and sanitize notebooks
2026-01-14 12:42:38 +05:30
KaifAhmad1 72a6ddb48f Merge remote-tracking branch 'origin/utils' into utils 2026-01-14 12:38:44 +05:30
KaifAhmad1 a5da533d55 chore: resolve dependencies, migrate Gemini SDK, and sanitize notebooks 2026-01-14 12:37:29 +05:30
Mohd Kaif be8856cfcf Merge pull request #188 from Hawksight-AI/semantic-extract
[SECURITY] Enhance caching security by excluding sensitive keys and using SHA-256
2026-01-14 00:25:46 +05:30
KaifAhmad1 d2e599bcb0 [SECURITY] Enhance caching security by excluding sensitive keys and using SHA-256 2026-01-14 00:22:41 +05:30
Mohd Kaif 05d0bbf86c Merge pull request #187 from Hawksight-AI/semantic-extract
Performance Bottlenecks and Scaling Limitations in semantic_extract
2026-01-14 00:15:06 +05:30
KaifAhmad1 dd7fcd3ddb [FEATURE] Performance Bottlenecks and Scaling Limitations in semantic_extract #186
- Implemented high-throughput parallel batch processing across all core extractors (NERExtractor, RelationExtractor, TripletExtractor, EventDetector, SemanticNetworkExtractor) using ThreadPoolExecutor.

- Added max_workers configuration parameter (default: 1) to all extractor extract() methods.

- Implemented parallel processing for large document chunking in _extract_entities_chunked and _extract_relations_chunked.

- Enhanced ProgressTracker to be thread-safe.

- Optimized setUpClass in tests to reduce Groq LLM initialization overhead.

- Updated documentation and usage examples.
2026-01-14 00:11:30 +05:30
Mohd Kaif 43f55e1028 Delete RELEASE_NOTES_v0.2.0.md 2026-01-13 00:33:59 +05:30
Mohd Kaif e20c522c62 Merge pull request #185 from Hawksight-AI/docs
Update Earning Call Notebook
2026-01-13 00:13:16 +05:30
KaifAhmad1 fd9f0b2526 Add all changes 2026-01-13 00:10:44 +05:30
Mohd Kaif ccaadf6299 Merge pull request #180 from Hawksight-AI/docs
Release v0.2.1: Stability Fixes
2026-01-12 17:52:43 +05:30
KaifAhmad1 428fc3b83a chore(release): bump version to 0.2.1 and update release docs 2026-01-12 17:48:07 +05:30
Mohd Kaif 09cf3ed132 Merge pull request #179 from Hawksight-AI/docs
Resolve TypeError in Earnings Call Analysis Notebook (#177)
2026-01-12 17:35:36 +05:30
KaifAhmad1 58686d409b fix(cookbook): resolve TypeError in earnings call analysis step 7 #177 2026-01-12 17:32:16 +05:30
Mohd Kaif 6d5fbc8b63 Merge pull request #178 from Hawksight-AI/semantic-extract
Resolve Incomplete Output (#176), Relax Constraints, and Add Groq Support
2026-01-12 17:18:54 +05:30
KaifAhmad1 8c3f7f1f0a fix(semantic-extract): resolve incomplete output #176, relax constraints, and add Groq support 2026-01-12 17:15:21 +05:30
Mohd Kaif 4acad23a4d Merge pull request #174 from Hawksight-AI/docs
Update Earnings Call Analysis Notebook (Finance Use Case)
2026-01-11 23:31:13 +05:30
KaifAhmad1 cd1435ee10 Save changes to Earnings Call Analysis notebook 2026-01-11 23:25:28 +05:30
KaifAhmad1 68f0a1d4d9 docs: Update PyPI version badge to shields.io 2026-01-10 23:44:35 +05:30
KaifAhmad1 a47274593b docs: Add v0.2.0 release notes 2026-01-10 23:36:51 +05:30
KaifAhmad1 87a08e0240 chore: Prepare release v0.2.0 2026-01-10 23:32:10 +05:30
Mohd Kaif 1a2604255f Merge pull request #172 from Hawksight-AI/docs
Docs Update - Neptune Store & Docling Parser
2026-01-10 21:14:29 +05:30
KaifAhmad1 94b312901b docs: Update CHANGELOG with Neptune Store and Docling Parser features
- Added Amazon Neptune Graph Store support details:
  - IAM SigV4 signing
  - Robust connection handling with retries
  - New dependency group
- Added Docling Parser integration details:
  - Multi-format support (PDF, DOCX, etc.)
  - Superior table extraction
  - Standalone parser architecture
2026-01-10 21:12:34 +05:30
Mohd Kaif 25fe95dd1a Merge pull request #171 from Hawksight-AI/semantic-extract
Enhanced Semantic Extraction with Robust Fallback Chains & Provenance Metadata
2026-01-10 21:02:21 +05:30
KaifAhmad1 f338b66274 feat: Add provenance metadata and robust fallback chains to semantic extraction
- Implemented ML/LLM -> Pattern -> Last Resort fallback chains for NER, Relation, and Triplet extractors to prevent empty results.
- Added provenance metadata (batch_index, document_id) to all extraction schemas (Entity, Relation, Triplet, etc.).
- Unified batch processing API with progress tracking across all extractors.
- Updated documentation (module usage and reference docs) to reflect new features.
- Added robustness and batch provenance tests.
2026-01-10 20:43:09 +05:30
Mohd Kaif 8b1cd47f51 Merge pull request #167 from don-simpson/feature/amazon-neptune-graph-store
feat: Add Amazon Neptune Database Graph Store Support
2026-01-09 19:27:44 +05:30
Mohd Kaif 48395b2f00 Merge pull request #170 from Hawksight-AI/docs
docs: update CHANGELOG.md
2026-01-09 18:56:55 +05:30
KaifAhmad1 91ef2939c5 docs: update CHANGELOG.md and remove PR description 2026-01-09 18:54:38 +05:30
Mohd Kaif 30d84c41ad Merge pull request #169 from Hawksight-AI/semantic-extract
Semantic Extraction Empty Returns & Schema Validation
2026-01-09 18:49:27 +05:30
KaifAhmad1 a5c531fd29 Fix semantic extraction empty returns, schema validation, and update docs 2026-01-09 18:39:09 +05:30
Don Simpson 976a20496d feat: Add Amazon Neptune Database Graph Store Support
- Implement NeptuneAuthTokenManager extending Neo4j AuthManager for IAM SigV4 signing
- Add automatic token refresh and security exception handling
- Add retry logic with backoff for transient errors (signature expired, connection closed)
- Add connection recovery with driver recreation
- Add NeptuneDriver, NeptuneSession, NeptuneTransaction wrapper classes
- Use native Neptune ~id via id() function for all CRUD operations
- Add graph-amazon-neptune optional dependency group (boto3, neo4j)
- Update cookbook with Amazon Neptune Graph Store examples
- Add comprehensive tests (61 tests covering all GraphStore interface methods)

Closes #151
2026-01-08 20:13:28 -05:00
Mohd Kaif 9bb94c2337 Merge pull request #165 from Hawksight-AI/parse
Docling Integration & Parser Documentation Fixes
2026-01-08 21:30:10 +05:30
KaifAhmad1 957c122116 docs: add Docling integration guide, clear code example, and fix parser consistency issues 2026-01-08 21:27:48 +05:30
Mohd Kaif 31ca2e4446 Merge pull request #164 from Hawksight-AI/docs
docs: update changelog with model switching fixes
2026-01-08 19:38:01 +05:30
KaifAhmad1 b08c13364b docs: update changelog with model switching fixes and tests 2026-01-08 19:35:24 +05:30
Mohd Kaif 01dd0c97ab Merge pull request #163 from Hawksight-AI/embeddings
Fix Model Switching and Dynamic Dimension Detection
2026-01-08 19:00:12 +05:30
KaifAhmad1 2bd1d06eb2 fix: resolve model switching bug and implement intrinsic dimension detection in TextEmbedder 2026-01-08 18:56:42 +05:30
Mohd Kaif 9a2f2cd2d2 Merge pull request #162 from Hawksight-AI/kg
docs: update changelog with kg module fixes #159
2026-01-08 17:43:29 +05:30
KaifAhmad1 04a210232e docs: update changelog with kg module fixes #159 2026-01-08 17:42:19 +05:30
Mohd Kaif b3baeaa74e Merge pull request #161 from Hawksight-AI/kg
Fix 'unhashable type: Entity' in GraphAnalyzer (#159)
2026-01-08 17:29:31 +05:30
KaifAhmad1 58707ff721 fix(kg): resolve 'unhashable type: Entity' in GraphAnalyzer #159
- Robust ID extraction in CentralityCalculator, CommunityDetector, and ConnectivityAnalyzer
- Support for direct Entity objects and dictionaries as node identifiers
- Improved Entity hashability in utils/types.py
- Added integration test to verify fix and prevent regression
2026-01-08 17:23:31 +05:30
Mohd Kaif 9b18bc3da3 Merge pull request #158 from Hawksight-AI/dependabot/pip/protobuf-4.25.8
chore(deps): bump protobuf from 4.25.3 to 4.25.8
2026-01-07 19:13:13 +05:30
KaifAhmad1 d8e04c29e9 Security fix: Upgrade protobuf to 4.25.8 and add PR description 2026-01-07 19:11:58 +05:30
dependabot[bot] 5764a88d7e chore(deps): bump protobuf from 4.25.3 to 4.25.8
Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 4.25.3 to 4.25.8.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v4.25.3...v4.25.8)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-version: 4.25.8
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-07 13:41:10 +00:00
Mohd Kaif cc69899b13 Merge pull request #157 from Hawksight-AI/utils
Dependency Fixes, GraphRAG Alignment, and Orchestrator Improvements
2026-01-07 19:07:26 +05:30
KaifAhmad1 e3b53998c3 Fix dependency issues, align GraphRAG notebook, and update changelog 2026-01-07 19:00:30 +05:30
Mohd Kaif de3441e76e Merge pull request #156 from Hawksight-AI/docs
Cookbook Fixes & Repository Optimization
2026-01-07 16:03:03 +05:30
KaifAhmad1 bd3c258458 chore: add .gitattributes to fix language statistics 2026-01-07 15:59:39 +05:30
Mohd Kaif 51cc445327 Merge pull request #155 from Hawksight-AI/docs
Cookbook Notebook Cleanup & Fixes
2026-01-07 15:43:37 +05:30
KaifAhmad1 04c4c9fb4c docs: clean and fix corrupted notebooks in cookbook 2026-01-07 15:41:13 +05:30
Mohd Kaif 010251ac35 Merge pull request #154 from Hawksight-AI/ontology
Fix KnowledgeGraph Documentation Mismatch
2026-01-07 15:05:48 +05:30
KaifAhmad1 dcd6f25f87 docs: fix KnowledgeGraph mismatch and update tests for issue #144 2026-01-07 15:01:47 +05:30
Mohd Kaif 55abd52b77 Merge pull request #153 from Hawksight-AI/docs
docs: improve robustness and data consistency in earnings analysis notebook
2026-01-07 14:11:41 +05:30
KaifAhmad1 960d7c5f8f docs: improve robustness and fix variable inconsistencies in earnings call notebook 2026-01-07 14:09:22 +05:30
Mohd Kaif 2489ce72b5 Delete GITHUB_ISSUE_LLM_EXTRACTION.md 2026-01-07 03:06:21 +05:30
Mohd Kaif f488dfb82a Delete PR_DESCRIPTION_LLM_EXTRACTION.md 2026-01-07 03:06:06 +05:30
Mohd Kaif 74fdd3330e Merge pull request #150 from Hawksight-AI/semantic-extract
Robust LLM Extraction - Auto-Chunking, Retries, and Diagnostics
2026-01-07 03:05:37 +05:30
KaifAhmad1 e712949872 Enhance LLM extraction methods with auto-chunking, robust parsing and improved diagnostics (#149) 2026-01-07 03:03:19 +05:30
Mohd Kaif c208f6b54e Delete PR_DESCRIPTION.md 2026-01-07 01:13:55 +05:30
Mohd Kaif d516ea69dc Merge pull request #148 from Hawksight-AI/semantic-extract
fix(semantic_extract): Pass API key to Groq LLM provider in extractio…
2026-01-07 01:11:21 +05:30
KaifAhmad1 2790132e8e fix(semantic_extract): Pass API key to Groq LLM provider in extraction methods
- Add API key handling in extract_entities_llm(), extract_relations_llm(), and extract_triplets_llm()
- Add explicit api_key handling in NERExtractor and RelationExtractor
- Add llm_model parameter support in extract_triplets_llm() for consistency
- Fix relation extraction bug with type checking for subject_text/object_text
- Add environment variable fallback for API keys
- Update notebook with standard API key pattern

Fixes #147
2026-01-07 01:07:56 +05:30
Mohd Kaif 1e22ff3a75 Merge pull request #146 from Hawksight-AI/semantic-extract
fix(semantic_extract): Pass API key to Groq LLM provider in extractio…
2026-01-06 22:43:59 +05:30
KaifAhmad1 9c59f97542 fix(semantic_extract): Pass API key to Groq LLM provider in extraction methods
- Add API key handling in extract_entities_llm(), extract_relations_llm(), and extract_triplets_llm()
- Add llm_model parameter support in extract_triplets_llm() for consistency
- Fix relation extraction bug with type checking for subject_text/object_text
- Add environment variable fallback for API keys
- Include providers.py for context (GroqProvider implementation)

Fixes #145
2026-01-06 22:37:57 +05:30
Mohd Kaif 4eb69e5048 Merge pull request #143 from Hawksight-AI/docs
Refreshed Metadata, Analytics & Community Links
2026-01-06 12:04:53 +05:30
KaifAhmad1 2b43fa4699 fix: update discord badge to a reliable static version 2026-01-06 12:00:23 +05:30
Mohd Kaif 9fb18e3ec6 Merge pull request #142 from Hawksight-AI/docs
Update Badges, Discord Links, and Version Metadata
2026-01-06 11:57:37 +05:30
KaifAhmad1 7f7c36f94d docs: update badges, discord links, and version mentions 2026-01-06 11:54:17 +05:30
Mohd Kaif 2e2f19f43d Merge pull request #141 from Hawksight-AI/docs
Release v0.1.1: Docling Integration & 2026 Sync
2026-01-06 00:28:15 +05:30
KaifAhmad1 d7b686f32a Release v0.1.1: Docling support, version bump, and documentation updates 2026-01-06 00:21:20 +05:30
Mohd Kaif e89e707e49 Merge pull request #140 from Hawksight-AI/utils
Fix DoclingParser Integration and Confirm Cross-Platform Compatibility
2026-01-05 22:31:57 +05:30
KaifAhmad1 a441e935f9 Fix DoclingParser integration and align with docling API
- Fix import logic in __init__.py to properly export DoclingParser
- Rewrite docling_parser.py to use docling's native API (direct attribute access)
- Remove unsupported features (table_extraction_mode, invalid format_options)
- Use doc.tables, doc.pictures, doc.pages directly instead of dict parsing
- Update notebook with improved code and documentation
- Add proper error handling for when docling is not available

Fixes #138
2026-01-05 22:27:15 +05:30
Mohd Kaif 3fb98aa0ed Merge pull request #139 from Hawksight-AI/utils
Resolve DoclingParser Exports, Windows Progress Encoding, and Finance Cookbook Update (#138)
2026-01-05 20:10:45 +05:30
KaifAhmad1 4e0c3bc361 Resolve DoclingParser exports, fix Windows progress encoding, and update finance cookbook #138 2026-01-05 20:04:59 +05:30
Mohd Kaif ef2c3dc841 Merge pull request #137 from Hawksight-AI/utils
feat: Add pipeline_id support and fix parsing table display
2026-01-05 18:17:24 +05:30
KaifAhmad1 96604ae398 feat: Add pipeline_id support and fix parsing table display
- Add pipeline_id parameter to all trackers, batch processors, parsers, and extractors
- Fix DoclingParser to show extraction counts in progress display
- Add 'Extracted' column showing tables, images, pages
- Emphasize Docling as core dependency in messages

Closes #136
2026-01-05 18:15:12 +05:30
Mohd Kaif 88a4b9f1d2 Merge pull request #134 from Hawksight-AI/parse
feat(parse): add progress tracking to DoclingParser and update earnin…
2026-01-05 14:46:12 +05:30
KaifAhmad1 ea02896617 feat(parse): add progress tracking to DoclingParser and update earnings call notebook
- Add 8-stage progress tracking (0-100%) with ETA to DoclingParser
- Update earnings call analysis notebook with MDA Space Q3 2025 example
- Simplify notebook code structure
- Add real-time progress visibility for PDF parsing

Closes #133
2026-01-05 14:42:12 +05:30
Mohd Kaif 01808728f4 Merge pull request #132 from Hawksight-AI/docs
[DOCS] Fix Discord invite link and reorganize README sections
2026-01-04 19:58:02 +05:30
KaifAhmad1 b03ab2458d docs: Fix Discord invite link and reorganize README sections
- Update Discord invite link from https://discord.gg/semantica to https://discord.gg/pMHguUzG
- Move Contributors section inside Contributing section (following open source best practices)
- Update Enterprise Support section to indicate future availability
- Add Evals to roadmap

Fixes #127
2026-01-04 19:56:10 +05:30
Mohd Kaif f8551c5dfb Merge pull request #131 from Hawksight-AI/utils
Fix: Handle OSError for Optional Dependencies and Make DoclingParser Standalone
2026-01-04 17:56:39 +05:30
KaifAhmad1 e7f713d43b Fix: Handle OSError for optional dependencies and make DoclingParser standalone
- Add safe_import utility in semantica/utils/helpers.py for graceful optional dependency handling
- Update all optional imports (spacy, docling, etc.) to handle OSError (Windows DLL issues)
- Make DoclingParser standalone with docling as core dependency
- Remove DoclingParser integration from DocumentParser
- Implement lazy initialization for DoclingParser (fails on parse(), not init())
- Fix DocumentConverter initialization (remove unsupported pipeline_options parameter)
- Preserve original error messages without modification
- Update semantic_extract, split, parse, embeddings, vector_store, visualization modules
- Fix OSError handling across entire codebase for Windows compatibility
- Update 60+ files with proper optional dependency handling
2026-01-04 17:44:14 +05:30
Mohd Kaif 6595f1918c Merge pull request #130 from Hawksight-AI/llms
Fix: Make PyTorch import lazy to avoid DLL errors on Windows
2026-01-03 20:43:43 +05:30
238 changed files with 31070 additions and 133682 deletions
+20
View File
@@ -0,0 +1,20 @@
# Linguist documentation and generated files
# This ensures GitHub language statistics reflect the core Python code
# Mark the entire docs directory as documentation
docs/* linguist-documentation
# Mark the cookbook directory as documentation/examples
cookbook/* linguist-documentation
# Specifically ignore large generated HTML/JSON files in cookbook
cookbook/**/*.html linguist-documentation
cookbook/**/*.json linguist-documentation
cookbook/**/*.graphml linguist-documentation
cookbook/**/*.ttl linguist-documentation
# Ensure .ipynb files are treated as documentation/examples
cookbook/**/*.ipynb linguist-documentation
# Mark data directories as documentation or vendored
**/data/* linguist-vendored
+1 -6
View File
@@ -1,8 +1,3 @@
# Funding options for Semantica
# Uncomment and add your usernames/links below
# github: [username]
# patreon: username
# ko_fi: username
# custom: ["https://your-funding-page.com"]
github: Hawksight-AI
+2
View File
@@ -32,6 +32,8 @@ For enterprise support, custom development, or consulting services:
## Sponsorship
### Sponsor this project
Support Semantica development:
- [GitHub Sponsors](https://github.com/sponsors/Hawksight-AI)
+1
View File
@@ -61,6 +61,7 @@ wheels/
.installed.cfg
*.egg
MANIFEST
.python-version
# IDE
.vscode/
+8 -1
View File
@@ -5,6 +5,7 @@ repos:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
exclude: 'neptune-setup\.yaml$'
- id: check-json
- id: check-toml
- id: check-added-large-files
@@ -49,9 +50,15 @@ repos:
hooks:
- id: yamllint
args: ['-d', '{extends: default, rules: {line-length: {max: 120}}}']
exclude: 'neptune-setup\.yaml$'
- repo: https://github.com/aws-cloudformation/cfn-lint
rev: v1.43.3
hooks:
- id: cfn-lint
files: 'neptune-setup\.yaml$'
# Removed slow hooks for faster development:
# - mypy: Type checking (can be run manually or in CI)
# - bandit: Security scanning (can be run separately)
# - pytest: Testing (should be run manually, not on every commit)
+284
View File
@@ -7,6 +7,290 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added / Changed
- **Enhanced Change Management Module**:
- New `semantica.change_management` module with persistent version storage and audit trails
- **Core Classes**: `TemporalVersionManager` (KG versioning), `OntologyVersionManager` (ontology versioning), `ChangeLogEntry` (metadata)
- **Storage**: SQLite (persistent) and in-memory backends with thread-safe operations
- **Features**: SHA-256 checksums, detailed entity/relationship diffs, structural ontology comparison, email validation
- **Compliance**: HIPAA, SOX, FDA 21 CFR Part 11 support with immutable audit trails
- **Testing**: 104 tests (100% pass) - unit, integration, compliance, performance, edge cases
- **Performance**: 17.6ms for 10k entities, 510+ ops/sec concurrent, handles 5k+ entity graphs
- **Migration**: Backward compatible, simplified class names, zero external dependencies
- CSV Ingestion Enhancements (PR #244 by @saloni0318)
- Auto-detect CSV encoding (chardet) and delimiter (csv.Sniffer)
- Tolerant decoding and malformed-row handling (`on_bad_lines='warn'`)
- Optional chunked reading for large files; metadata tracks detected values
- Expanded unit tests covering delimiters, quoted/multiline fields, header overrides, chunks, and NaN preservation
- Tests: Comprehensive units for TextNormalizer (PR #242 by @ZohaibHassan16)
- Added focused test coverage for TextNormalizer behavior across inputs
- Tests: Register integration mark and tidy ingest test warnings (PR #241 by @KaifAhmad1)
- Introduced integration test marker and reduced noisy warnings in ingest tests
- Tests (ingest): Add unit tests for file, web, and feed ingestors (PR #239 by @Mohammed2372)
- Broadened ingest test coverage across multiple source types
## [0.2.5] - 2026-01-27
### Added
- **Pinecone Vector Store Support**:
- Implemented native Pinecone support (`PineconeStore`) with full CRUD capabilities.
- Added support for serverless and pod-based indexes, namespaces, and metadata filtering.
- Integrated with `VectorStore` unified interface and registry.
- (Closes #219, Resolves #220)
- **Configurable LLM Retry Logic**:
- Exposed `max_retries` parameter in `NERExtractor`, `RelationExtractor`, `TripletExtractor` and low-level extraction methods (`extract_entities_llm`, `extract_relations_llm`, `extract_triplets_llm`).
- Defaults to 3 retries to prevent infinite loops during JSON validation failures or API timeouts.
- Propagated retry configuration through chunked processing helpers to ensure consistent behavior for long documents.
- Updated `03_Earnings_Call_Analysis.ipynb` to use `max_retries=3` by default.
### Added
- **Bring Your Own Model (BYOM) Support**:
- Enabled full support for custom Hugging Face models in `NERExtractor`, `RelationExtractor`, and `TripletExtractor`.
- Added support for custom tokenizers in `HuggingFaceModelLoader` to handle models with non-standard tokenization requirements.
- Implemented robust fallback logic for model selection: runtime options (`extract(model=...)`) now correctly override configuration defaults.
- **Enhanced NER Implementation**:
- Added configurable aggregation strategies (`simple`, `first`, `average`, `max`) to `extract_entities_huggingface` for better sub-word token handling.
- Implemented robust IOB/BILOU parsing to reconstruct entities from raw model outputs when structured output is unavailable.
- Added confidence scoring for aggregated entities.
- **Relation Extraction Improvements**:
- Implemented standard entity marker technique (wrapping subject/object with `<subj>`, `<obj>` tags) in `extract_relations_huggingface` for compatibility with sequence classification models.
- Added structured output parsing to convert raw model predictions into validated `Relation` objects.
- **Triplet Extraction Completion**:
- Added specialized parsing for Seq2Seq models (e.g., REBEL) in `extract_triplets_huggingface` to generate structured triplets directly from text.
- Implemented post-processing logic to clean and validate generated triplets.
### Fixed
- **LLM Extraction Stability**:
- Fixed infinite retry loops in `BaseProvider` by strictly enforcing `max_retries` limit during structured output generation.
- Resolved stuck execution in earnings call analysis notebooks when using smaller models (e.g., Llama 3 8B) that frequently produce invalid JSON.
- **Model Parameter Precedence**:
- Fixed issue where configuration defaults took precedence over runtime arguments in Hugging Face extractors. Runtime options now correctly override config values.
- **Import Handling**:
- Fixed circular import issues in test suites by implementing robust mocking strategies.
## [0.2.4] - 2026-01-22
### Added
- **Ontology Ingestion Module**:
- Implemented `OntologyIngestor` in `semantica.ingest` for parsing RDF/OWL files (Turtle, RDF/XML, JSON-LD, N3) into standardized `OntologyData` objects.
- Added `ingest_ontology` convenience function and integrated it into the unified `ingest(source_type="ontology")` interface.
- Added recursive directory scanning support for batch ontology ingestion.
- Exposed ingestion tools in `semantica.ontology` for better discoverability.
- Added `OntologyData` dataclass for consistent metadata handling (source path, format, timestamps).
- **Documentation**:
- **Ontology Usage Guide**: Updated `ontology_usage.md` with comprehensive examples for single-file and directory ingestion.
- **API Reference**: Updated `ontology.md` with `OntologyIngestor` class documentation and method details.
- **Tests**:
- **Comprehensive Test Suite**: Added `tests/ingest/test_ontology_ingestor.py` covering all supported formats, error handling, and unified interface integration.
- **Demo Script**: Added `examples/demo_ontology_ingest.py` for end-to-end usage demonstration.
## [0.2.3] - 2026-01-20
### Fixed
- **LLM Relation Extraction Parsing**:
- Fixed relation extraction returning zero relations despite successful API calls to Groq and other providers
- Normalized typed responses from instructor/OpenAI/Groq to consistent dict format before parsing
- Added structured JSON fallback when typed generation yields zero relations to avoid silent empty outputs
- Removed acceptance of extra kwargs (`max_tokens`, `max_entities_prompt`) from relation extraction internals
- Filtered kwargs passed to provider LLM calls to only `temperature` and `verbose`
- **API Parameter Handling**:
- Limited kwargs forwarded in chunked extraction helper to prevent parameter leakage
- Ensured minimal, safe parameters are passed to provider calls
- **Pipeline Circular Import (Issues #192, #193)**:
- Fixed circular import between `pipeline_builder` and `pipeline_validator` triggered during `semantica.pipeline` import
- Lazy-loaded `PipelineValidator` inside `PipelineBuilder.__init__` and guarded type hints with `TYPE_CHECKING`
- Ensured `from semantica.deduplication import DuplicateDetector` no longer fails even when pipeline module is imported
- **JupyterLab Progress Output (Issue #181)**:
- Added `SEMANTICA_DISABLE_JUPYTER_PROGRESS` environment variable to disable rich Jupyter/Colab progress tables
- When enabled, progress falls back to console-style output, preventing infinite scrolling and JupyterLab out-of-memory errors
### Added
- **Comprehensive Test Suite**:
- - Added unit tests (`tests/test_relations_llm.py`) with mocked LLM provider covering both typed and structured response paths
- - Added integration tests (`tests/integration/test_relations_groq.py`) for real Groq API calls with environment variable API key
- - Tests validate relation extraction completion and result parsing across different response formats
- **Amazon Neptune Dev Environment**:
- - Added CloudFormation template (`cookbook/introduction/neptune-setup.yaml`) to provision a dev Neptune cluster with public endpoint and IAM auth enabled
- - Documented deployment, cost estimates, and IAM User vs IAM Role best practices in `cookbook/introduction/21_Amazon_Neptune_Store.ipynb`
- - Added `cfn-lint` to `.pre-commit-config.yaml` for validating CloudFormation templates while excluding `neptune-setup.yaml` from generic YAML linters
- **Vector Store High-Performance Ingestion**:
- - Added `VectorStore.add_documents` for high-throughput ingestion with automatic embedding generation, batching, and parallel processing
- - Added `VectorStore.embed_batch` helper for generating embeddings for lists of texts without immediately storing them
- - Enabled default parallel ingestion in `VectorStore` with `max_workers=6` for common workloads
- - Added dedicated documentation page `docs/vector_store_usage.md` describing high-performance vector store usage and configuration
- - Added `tests/vector_store/test_vector_store_parallel.py` covering parallel vs sequential performance, error handling, and edge cases for `add_documents` and `embed_batch`
### Changed
- **Relation Extraction API**:
- - Simplified parameter interface by removing unused kwargs that were previously ignored
- - Improved error handling and verbose logging for debugging relation extraction issues
- - Enhanced robustness of post-response parsing across different LLM providers
- **Vector Store Defaults and Examples**:
- - Standardized `VectorStore` default concurrency to `max_workers=6` for parallel ingestion
- - Updated vector store reference documentation and usage guides to rely on implicit defaults instead of requiring manual `max_workers` configuration in examples
## [0.2.2] - 2026-01-15
### Added
- **Parallel Extraction Engine**:
- Implemented high-throughput parallel batch processing across all core extractors (`NERExtractor`, `RelationExtractor`, `TripletExtractor`, `EventDetector`, `SemanticNetworkExtractor`) using `concurrent.futures.ThreadPoolExecutor`.
- Added `max_workers` configuration parameter (default: 1) to all extractor `extract()` methods, allowing users to tune concurrency based on available CPU cores or API rate limits.
- **Parallel Chunking**: Implemented parallel processing for large document chunking in `_extract_entities_chunked` and `_extract_relations_chunked`, significantly reducing latency for long-form text analysis.
- **Thread-Safe Progress Tracking**: Enhanced `ProgressTracker` to handle concurrent updates from multiple threads without race conditions during batch processing.
- **Semantic Extract Performance & Regression**:
- Added edge-case regression suite covering max worker defaults, LLM prompt entity filtering, and extractor reuse.
- Added a runnable real-use-case benchmark script for batch latency across `NERExtractor`, `RelationExtractor`, `TripletExtractor`, `EventDetector`, `SemanticAnalyzer`, and `SemanticNetworkExtractor`.
- Added Groq LLM smoke tests that exercise LLM-based entities/relations/triplets when `GROQ_API_KEY` is available via environment configuration.
### Security
- **Credential Sanitization**:
- Removed hardcoded API keys from 8 cookbook notebooks to prevent secret leakage.
- Enforced environment variable usage for `GROQ_API_KEY` across all examples.
- **Secure Caching**:
- Updated `ExtractionCache` to exclude sensitive parameters (e.g., `api_key`, `token`, `password`) from cache key generation, preventing secret leakage and enabling safe cache sharing.
- Upgraded cache key hashing algorithm from MD5 to **SHA-256** for enhanced collision resistance and security.
### Changed
- **Gemini SDK Migration**:
- Migrated `GeminiProvider` to use the new `google-genai` SDK (v0.1.0+) to address deprecation warnings.
- Implemented graceful fallback to `google.generativeai` for backward compatibility.
- **Dependency Resolution**:
- Pinned `opentelemetry-api` and `opentelemetry-sdk` to `1.37.0` to resolve pip conflicts.
- Updated `protobuf` and `grpcio` constraints for better stability.
- **Entity Filtering Scope**:
- Removed entity filtering from non-LLM extraction flows to avoid accuracy regressions.
- Applied entity downselection only to LLM relation prompt construction, while matching returned entities against the full original entity list.
- **Batch Concurrency Defaults**:
- Standardized `max_workers` defaulting across `semantic_extract` and tuned for low-latency: ML-backed methods default to single-worker, while pattern/regex/rules/LLM/huggingface methods use a higher parallelism default capped by CPU.
- Raised the global `optimization.max_workers` default to 8 for better throughput on batch workloads.
### Performance
- **Bottleneck Optimization (GitHub Issue #186)**:
- **Resolved Bottleneck #1 (Sequential Processing)**: Replaced sequential `for` loops with parallel execution for both document-level batches and intra-document chunks.
- **Performance Gains**: Achieved **~1.89x speedup** in real-world extraction scenarios (tested with Groq `llama-3.3-70b-versatile` on standard datasets).
- **Initialization Optimization**: Refactored test suite to use class-level `setUpClass` for LLM provider initialization, eliminating redundant API client creation overhead.
- **Low-Latency Entity Matching**:
- Avoided heavyweight embedding stack imports on common matches by improving fast matching heuristics and short-circuiting before embedding similarity.
- Optimized entity matching to prioritize exact/substring/word-boundary matches and only fall back to embedding similarity when needed, reducing CPU overhead in LLM relation/triplet mapping.
## [0.2.1] - 2026-01-12
### Fixed
- **LLM Output Stability (Bug #176)**:
- Fixed incomplete JSON output issues by correctly propagating `max_tokens` parameter in `extract_relations_llm`.
- Implemented automatic error handling that halves chunk sizes and retries when LLM context or output limits are exceeded.
- Fixed `AttributeError` in provider integration by ensuring consistent parameter passing via `**kwargs`.
- **Constraint Relaxations**:
- Removed hardcoded `max_length` constraints from `Entity`, `Relation`, and `Triplet` classes to support long-form semantic extraction (e.g., long descriptions or names).
- Fixed orchestrator lazy property initialization and configuration normalization logic in `Orchestrator`.
- Resolved `AssertionError` in orchestrator tests by aligning test mocks with production component usage.
- Fixed dependency compatibility issues by pinning `protobuf>=5.29.1,<7.0` and `grpcio>=1.71.2`.
- Added missing dependencies `GitPython` and `chardet` to `pyproject.toml`.
- Verified and aligned `FileObject.text` property usage in GraphRAG notebooks for consistent content decoding.
### Changed
- **Chunking Defaults**:
- Increased default `max_text_length` for auto-chunking to **64,000 characters** (from 32k/16k) for OpenAI, Anthropic, Gemini, Groq, and DeepSeek providers.
- Unified chunking logic across `extract_entities_llm`, `extract_relations_llm`, and `extract_triplets_llm`.
- **Groq Support**:
- Standardized Groq provider defaults to use `llama-3.3-70b-versatile` with a 64k context window.
- Added native support for `max_tokens` and `max_completion_tokens` to prevent output truncation.
### Added
- **Testing**:
- Added `tests/reproduce_issue_176.py` to validate `max_tokens` propagation and chunking behavior across all extractors.
## [0.2.0] - 2026-01-10
### Added
- **Amazon Neptune Support**:
- Added `AmazonNeptuneStore` providing Amazon Neptune graph database integration via Bolt protocol and OpenCypher.
- Implemented `NeptuneAuthTokenManager` extending Neo4j AuthManager for AWS IAM SigV4 signing with automatic token refresh.
- Added robust connection handling: retry logic with backoff for transient errors (signature expired, connection closed) and driver recreation.
- Added `graph-amazon-neptune` optional dependency group (boto3, neo4j).
- Comprehensive test suite covering all GraphStore interface methods.
- **Docling Integration**:
- Added `DoclingParser` in `semantica.parse` for high-fidelity document parsing using the Docling library.
- Supports multi-format parsing (PDF, DOCX, PPTX, XLSX, HTML, images) with superior table extraction and structure understanding.
- Implemented as a standalone parser supporting local execution, OCR, and multiple export formats (Markdown, HTML, JSON).
- **Robust Extraction Fallbacks**:
- Implemented comprehensive fallback chains ("ML/LLM" -> "Pattern" -> "Last Resort") across `NERExtractor`, `RelationExtractor`, and `TripletExtractor` to prevent empty result lists.
- Added "Last Resort" pattern matching in `NERExtractor` to identify capitalized words as generic entities when all other methods fail.
- Added "Last Resort" adjacency-based relation extraction in `RelationExtractor` to create weak connections between adjacent entities if no relations are found.
- Added fallback logic in `TripletExtractor` to convert relations to triplets or use rule-based extraction if standard methods fail.
- **Provenance & Tracking**:
- Added count tracking to batch processing logs in `NERExtractor`, `RelationExtractor`, and `TripletExtractor`.
- Added `batch_index` and `document_id` to the metadata of all extracted entities, relations, triplets, semantic roles, and clusters for better traceability.
- **Semantic Extract Improvements**:
- Introduced `auto-chunking` for long text processing in LLM extraction methods (`extract_entities_llm`, `extract_relations_llm`, `extract_triplets_llm`).
- Added `silent_fail` parameter to LLM extraction methods for configurable error handling.
- Implemented robust JSON parsing and automatic retry logic (3 attempts with exponential backoff) in `BaseProvider` for all LLM providers.
- Enhanced `GroqProvider` with better diagnostics and connectivity testing.
- Added comprehensive entity, relation, and triplet deduplication for chunked extraction.
- Added `semantica/semantic_extract/schemas.py` with canonical Pydantic models for consistent structured output.
- **Testing**:
- Added comprehensive robustness test suite `tests/semantic_extract/test_robustness_fallback.py` for validating extraction fallbacks and metadata propagation.
- Added comprehensive unit test suite `tests/embeddings/test_model_switching.py` for verifying dynamic model transitions and dimension updates.
- Added end-to-end integration test suite for Knowledge Graph pipeline validation (GraphBuilder -> EntityResolver -> GraphAnalyzer).
- **Other**:
- Added missing dependencies `GitPython` and `chardet` to `pyproject.toml`.
- Robustified ID extraction across `CentralityCalculator`, `CommunityDetector`, and `ConnectivityAnalyzer` to handle various entity formats.
- Improved `Entity` class hashability and equality logic in `utils/types.py`.
### Changed
- **Deduplication & Conflict Logic**:
- Removed internal deduplication logic from `NERExtractor`, `RelationExtractor`, and `TripletExtractor`.
- Removed consistency/conflict checking from `ExtractionValidator` to defer to dedicated `semantica/conflicts` module.
- Removed `_deduplicate_*` methods from `semantica/semantic_extract/methods.py`.
- **Batch Processing & Consistency**:
- Standardized batch processing across all extractors (`NERExtractor`, `RelationExtractor`, `TripletExtractor`, `SemanticNetworkExtractor`, `EventDetector`, `SemanticAnalyzer`, `CoreferenceResolver`) using a unified `extract`/`analyze`/`resolve` method pattern with progress tracking.
- Added provenance metadata (`batch_index`, `document_id`) to `SemanticNetwork` nodes/edges, `Event` objects, `SemanticRole` results, `CoreferenceChain` mentions, and `SemanticCluster` (tracking source `document_ids`).
- Updated `SemanticClusterer.cluster` and `SemanticAnalyzer.cluster_semantically` to accept list of dictionaries (with `content` and `id` keys) for better document tracking during clustering.
- Removed legacy `check_triplet_consistency` from `TripletExtractor`.
- Removed `validate_consistency` and `_check_consistency` from `ExtractionValidator`.
- **Weighted Scoring**:
- Clarified weighted confidence scoring (50% Method Confidence + 50% Type Similarity) in comments.
- Explicitly labeled "Type Similarity" as "user-provided" in code comments to remove ambiguity.
- **Refactoring**:
- Fixed orchestrator lazy property initialization and configuration normalization logic in `Orchestrator`.
- Verified and aligned `FileObject.text` property usage in GraphRAG notebooks for consistent content decoding.
### Fixed
- **Critical Fixes**:
- Resolved `NameError` in `extraction_validator.py` by adding missing `Union` import.
- Resolved issues where extractors would return empty lists for valid input text when primary extraction methods failed.
- Fixed metadata initialization issue in batch processing where `batch_index` and `document_id` were occasionally missing from extracted items.
- Ensured `LLMExtraction` methods (`enhance_entities`, `enhance_relations`) return original input instead of failing or returning empty results when LLM providers are unavailable.
- **Component Fixes**:
- Fixed model switching bug in `TextEmbedder` where internal state was not cleared, preventing dynamic updates between `fastembed` and `sentence_transformers` (#160).
- Implemented model-intrinsic embedding dimension detection in `TextEmbedder` to ensure consistency between models and vector databases.
- Updated `set_model` to properly refresh configuration and dimensions during model switches.
- Fixed `TypeError: unhashable type: 'Entity'` in `GraphAnalyzer` when processing graphs with raw `Entity` objects or dictionaries in relationships (#159).
- Resolved `AssertionError` in orchestrator tests by aligning test mocks with production component usage.
- Fixed dependency compatibility issues by pinning `protobuf==4.25.3` and `grpcio==1.67.1`.
- Fixed a bug in `TripletExtractor` where the `validate_triplets` method was shadowed by an internal attribute.
- Fixed incorrect `TextSplitter` import path in the `semantic_extract.methods` module.
## [0.1.1] - 2026-01-05
### Added
- Exported `DoclingParser` and `DoclingMetadata` from `semantica.parse` for easier access.
- Added comprehensive `DoclingParser` usage examples to README and documentation.
- Added Windows-specific troubleshooting note for PyTorch DLL issues.
### Fixed
- Fixed `DoclingParser` import/export issues across platforms (Windows, Linux, Google Colab).
- Improved error messaging when optional `docling` dependency is missing.
- Fixed versioning inconsistencies across the framework.
## [0.1.0] - 2025-12-31
### Added
+263 -297
View File
@@ -1,306 +1,266 @@
# Contributing to Semantica
Thank you for your interest in contributing to Semantica! This document provides guidelines and instructions for contributing to the project.
Thank you for your interest in contributing! Every contribution, no matter how small, is valuable. 🎉
## Table of Contents
**Give us a Star** • 🍴 **[Fork Semantica](https://github.com/Hawksight-AI/semantica/fork)** • 💬 **Join our [Discord](https://discord.gg/vqRt2qbx)**
- [Code of Conduct](#code-of-conduct)
- [Getting Started](#getting-started)
- [Development Setup](#development-setup)
- [Code Style Guidelines](#code-style-guidelines)
- [Testing Requirements](#testing-requirements)
- [Commit Message Conventions](#commit-message-conventions)
- [Pull Request Process](#pull-request-process)
- [Documentation Standards](#documentation-standards)
- [Types of Contributions](#types-of-contributions)
- [Getting Help](#getting-help)
> **New to contributing?** Start with a [`good first issue`](https://github.com/Hawksight-AI/semantica/labels/good%20first%20issue) or join our [Discord](https://discord.gg/vqRt2qbx) community.
## Code of Conduct
---
This project adheres to a [Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to the maintainers.
## 🚀 Quick Start
## Getting Started
1. Find a [`good first issue`](https://github.com/Hawksight-AI/semantica/labels/good%20first%20issue)
2. [Fork Semantica](https://github.com/Hawksight-AI/semantica/fork) & clone the repository
3. Make your changes
4. Submit a pull request!
1. **Fork the repository** on GitHub
2. **Clone your fork** locally:
```bash
git clone https://github.com/your-username/semantica.git
cd semantica
```
3. **Add the upstream remote**:
```bash
git remote add upstream https://github.com/Hawksight-AI/semantica.git
```
**Need help?** Join [Discord](https://discord.gg/vqRt2qbx) or [GitHub Discussions](https://github.com/Hawksight-AI/semantica/discussions)
## Development Setup
---
### Prerequisites
## 🎯 Ways to Contribute
- Python 3.8 or higher (3.9+ recommended)
- pip package manager
- Git
### 💻 Code
### Installation
**What you can do:**
- Fix bugs
- Add new features
- Improve code quality (add type hints, docstrings, improve error messages)
- Optimize performance
1. **Create a virtual environment** (recommended):
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
```
**Where:** `semantica/` directory
2. **Install the project in editable mode with dev dependencies**:
```bash
pip install -e ".[dev]"
```
**Good first issues:** Add docstrings, type hints, or improve error messages
3. **Install pre-commit hooks**:
```bash
pre-commit install
```
---
### Verify Installation
### 📝 Documentation
**What you can do:**
- Fix typos and grammar errors
- Improve clarity and readability
- Add code examples and tutorials
- Create new cookbook notebooks
- Improve API documentation (docstrings)
- Create troubleshooting guides
- Update installation instructions
- Add missing documentation
**Where:** `README.md`, `docs/`, `cookbook/`, docstrings in code
**Good first issues:** Fix typos, add examples, create cookbook tutorials, improve docstrings
**Documentation formatting:**
- Use clear, concise language
- Include code examples where helpful
- Follow markdown best practices
- Use proper headings hierarchy
- Add links to related sections
- Include screenshots for UI-related docs
---
### 🧪 Testing
**What you can do:**
- Add unit tests
- Improve test coverage
- Add integration tests
**Where:** `tests/` directory
**Good first issues:** Add tests for specific functions or classes
---
### 🐛 Bug Reports
**What:** Report bugs you find
**How:** Use the [bug report template](https://github.com/Hawksight-AI/semantica/issues/new?template=bug_report.md)
**Include:** Description, steps to reproduce, expected vs actual behavior, environment details
---
### 💡 Feature Requests
**What:** Suggest new features or improvements
**How:** Use the [feature request template](https://github.com/Hawksight-AI/semantica/issues/new?template=feature_request.md)
**Include:** Problem statement, proposed solution, use cases
---
### 🎨 Cookbook & Examples
**What:** Create tutorials and examples
**Where:** `cookbook/` directory
**Examples:** Create new notebooks, add examples, improve existing tutorials
---
### 💬 Community Support
**What:** Help others in the community
**Where:** [Discord](https://discord.gg/vqRt2qbx), [GitHub Discussions](https://github.com/Hawksight-AI/semantica/discussions)
**Examples:** Answer questions, review PRs, share your projects
---
### 🎓 Educational Content
**What:** Create educational materials
**Examples:** Blog posts, video tutorials, talks, workshops, case studies
---
### 🔧 Other Contributions
- **Design & Graphics:** Logos, diagrams, visualizations
- **Tools & Integrations:** CLI tools, integrations with other frameworks
- **Infrastructure:** CI/CD improvements, Docker optimization
- **Security:** Report security vulnerabilities (privately)
---
## 📋 Getting Started
### 1. Fork & Clone
First, [fork Semantica](https://github.com/Hawksight-AI/semantica/fork) on GitHub, then:
```bash
python -c "import semantica; print(semantica.__version__)"
pytest --version
black --version
git clone https://github.com/your-username/semantica.git
cd semantica
git remote add upstream https://github.com/Hawksight-AI/semantica.git
```
## Code Style Guidelines
We use several tools to maintain code quality and consistency:
### Formatting
- **Black**: Code formatting (line length: 88)
```bash
black semantica/
```
- **isort**: Import sorting
```bash
isort semantica/
```
### Linting
- **flake8**: Style guide enforcement
```bash
flake8 semantica/
```
- **mypy**: Static type checking
```bash
mypy semantica/
```
### Running All Checks
### 2. Set Up Environment
```bash
# Format code
black semantica/ tests/
# Create virtual environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# Sort imports
isort semantica/ tests/
# Install dev dependencies
pip install -e ".[dev]"
# Lint
flake8 semantica/ tests/
# Type check
mypy semantica/
# Install pre-commit hooks (optional)
pre-commit install
```
Or use pre-commit hooks (automatically runs on commit):
```bash
pre-commit run --all-files
```
## Testing Requirements
### Running Tests
### 3. Create Branch
```bash
# Run all tests
pytest
# Run with coverage
pytest --cov=semantica --cov-report=html
# Run specific test file
pytest tests/test_specific.py
# Run with verbose output
pytest -v
git checkout -b feature/your-feature-name
# or
git checkout -b fix/bug-description
```
### Test Coverage
### 4. Make Changes
- Minimum coverage: **80%**
- Critical modules: **90%+**
- Coverage reports are generated in `htmlcov/`
- Follow code style (see below)
- Add tests for new features
- Update documentation
### Writing Tests
### 5. Run Checks
- Follow pytest conventions
- Use descriptive test names
- Include docstrings for complex tests
- Test both success and failure cases
- Use fixtures for common setup
Example:
```python
def test_entity_extraction():
"""Test basic entity extraction functionality."""
from semantica.semantic_extract import NamedEntityRecognizer
ner = NamedEntityRecognizer()
entities = ner.extract("Apple Inc. was founded by Steve Jobs.")
assert len(entities) > 0
assert any(e.text == "Apple Inc." for e in entities)
```bash
pytest # Run tests
black semantica/ tests/ # Format code
isort semantica/ tests/ # Sort imports
flake8 semantica/ tests/ # Lint
```
## Commit Message Conventions
Or use pre-commit hooks: `pre-commit run --all-files`
We follow [Conventional Commits](https://www.conventionalcommits.org/) specification:
### 6. Commit & Push
### Format
```
<type>(<scope>): <subject>
<body>
<footer>
```bash
git commit -m "feat(module): add new feature"
git push origin feature/your-feature-name
```
### Types
Then create a pull request on GitHub!
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation changes
- `style`: Code style changes (formatting, etc.)
- `refactor`: Code refactoring
- `test`: Adding or updating tests
- `chore`: Maintenance tasks
- `perf`: Performance improvements
- `ci`: CI/CD changes
---
### Examples
## 📐 Code Style
We use automated tools:
| Tool | Purpose | Command |
|----------|----------------------------|----------------------------|
| **Black** | Code formatting | `black semantica/ tests/` |
| **isort** | Import sorting | `isort semantica/ tests/` |
| **flake8** | Style enforcement | `flake8 semantica/ tests/` |
| **mypy** | Type checking | `mypy semantica/` |
**Run all:** `black semantica/ tests/ && isort semantica/ tests/ && flake8 semantica/ tests/ && mypy semantica/`
---
## 🧪 Testing
```bash
pytest # Run all tests
pytest --cov=semantica # With coverage
pytest tests/test_file.py # Specific file
```
**Coverage goal:** 80% minimum, 90%+ for critical modules
---
## 📝 Commit Messages
Use [Conventional Commits](https://www.conventionalcommits.org/):
```
feat(kg): add temporal graph support
Add support for temporal knowledge graphs with version tracking
and time-based queries.
Closes #123
fix(parse): handle empty PDF files
docs(readme): add installation guide
test(extract): add unit tests
```
```
fix(parse): handle empty PDF files gracefully
**Types:** `feat`, `fix`, `docs`, `test`, `refactor`, `perf`, `style`, `chore`
Previously, empty PDF files would cause a crash. Now they return
an empty document with appropriate warnings.
---
Fixes #456
```
## ✅ PR Checklist
## Pull Request Process
### Before Submitting
1. **Update your fork**:
```bash
git fetch upstream
git checkout main
git merge upstream/main
```
2. **Create a feature branch**:
```bash
git checkout -b feature/your-feature-name
# or
git checkout -b fix/bug-description
```
3. **Make your changes** and commit following our conventions
4. **Run all checks**:
```bash
pytest
black semantica/ tests/
isort semantica/ tests/
flake8 semantica/ tests/
mypy semantica/
```
5. **Push to your fork**:
```bash
git push origin feature/your-feature-name
```
### PR Checklist
Before submitting:
- [ ] Code follows style guidelines
- [ ] Tests pass locally
- [ ] New tests added for new features
- [ ] New tests added (if applicable)
- [ ] Documentation updated
- [ ] Commit messages follow conventions
- [ ] No merge conflicts
- [ ] PR description is clear and complete
### PR Description Template
---
```markdown
## Description
Brief description of changes
## 📖 Documentation Standards
## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
### Code Documentation (Docstrings)
## Related Issues
Closes #123
Related to #456
**Format:** Use Google-style docstrings
## Testing
- [ ] Tests pass locally
- [ ] Added new tests
- [ ] Updated existing tests
## Checklist
- [ ] Code follows style guidelines
- [ ] Self-review completed
- [ ] Comments added for complex code
- [ ] Documentation updated
- [ ] No new warnings generated
```
## Documentation Standards
### Code Documentation
- Use Google-style docstrings
- Include type hints
- Document all public functions and classes
- Include examples for complex functions
Example:
```python
def extract_entities(
text: str,
model: str = "transformer",
confidence_threshold: float = 0.7
) -> List[Entity]:
def extract_entities(text: str, model: str = "transformer") -> List[Entity]:
"""Extract named entities from text.
Args:
text: Input text to process
model: NER model to use (default: "transformer")
confidence_threshold: Minimum confidence score (default: 0.7)
Returns:
List of extracted Entity objects
@@ -309,92 +269,98 @@ def extract_entities(
ValueError: If text is empty or model is invalid
Example:
>>> ner = NamedEntityRecognizer()
>>> from semantica.semantic_extract import NERExtractor
>>> ner = NERExtractor(method="ml", model="en_core_web_sm")
>>> entities = ner.extract("Apple Inc. was founded in 1976.")
>>> len(entities)
2
"""
...
```
### Documentation Files
### Markdown Documentation Formatting
- Update relevant documentation in `docs/`
- Add examples to cookbook if applicable
- Update API reference if adding new public APIs
- Keep README.md up to date
**General Guidelines:**
- Use clear headings (H1 for title, H2 for main sections, H3 for subsections)
- Keep paragraphs short and focused
- Use bullet points for lists
- Add code blocks with syntax highlighting
- Include links to related documentation
## Types of Contributions
**Code Blocks:**
- Use triple backticks with language identifier: ` ```python `, ` ```bash `
- Include comments in code examples
- Show expected output when helpful
### 💻 Code Contributions
**Examples:**
- **Bug Fixes**: Resolving issues reported in the issue tracker.
- **New Features**: Implementing new capabilities (please discuss via an issue first!).
- **Refactoring**: Improving code structure and maintainability without changing behavior.
- **Algorithm Optimization**: Improving the efficiency of graph algorithms and vector search.
```markdown
## Section Title
#### ⚡ Performance and Latency
We deeply value efficiency. Contributions that make Semantica faster and lighter are highly appreciated!
Brief introduction paragraph.
- **Latency Reduction**: Optimize critical paths and RAG pipeline response times.
- **Memory Optimization**: Reduce graph/vector processing memory footprint.
- **Throughput**: Improve operations per second (bulk ingestion, parallel queries).
- **Benchmarks**: Add performance benchmarks to track regressions.
- **Async/Concurrency**: Enhance asynchronous execution and concurrency.
### Subsection
### 📚 Documentation Contributions
- Bullet point 1
- Bullet point 2
- Fix typos and grammar
- Improve clarity
- Add examples
- Create tutorials
- Translate documentation
**Code example:**
### Testing Contributions
```python
from semantica import SomeClass
- Add test coverage
- Improve test quality
- Add integration tests
- Performance benchmarks
instance = SomeClass()
result = instance.method()
```
### Other Contributions
**Note:** Additional context or warnings.
```
- Answer questions in discussions
- Help with issues
- Review pull requests
- Share use cases
- Report bugs
- Suggest features
**Best Practices:**
- Start with an overview/introduction
- Use consistent terminology
- Include "See also" links
- Add examples for complex concepts
- Keep formatting consistent across docs
## Getting Help
---
### Communication Channels
## 🆘 Getting Help
- **GitHub Discussions**: General questions and discussions
- **GitHub Issues**: Bug reports and feature requests
- **Discord**: Real-time chat and community support
- 💬 [Discord](https://discord.gg/vqRt2qbx) - Real-time chat
- 💭 [GitHub Discussions](https://github.com/Hawksight-AI/semantica/discussions) - Q&A
- 🐛 [GitHub Issues](https://github.com/Hawksight-AI/semantica/issues) - Bug reports
### Before Asking for Help
**Before asking:** Check existing documentation, search issues/discussions, review cookbook examples
1. Check existing documentation
2. Search GitHub issues and discussions
3. Review code examples in cookbook
4. Check FAQ in documentation
---
### Asking Good Questions
## 🏆 Recognition
- Provide context and environment details
- Include code examples
- Show what you've tried
- Include error messages and logs
- Be specific about what you need
## Recognition
Contributors are recognized in:
All contributors are recognized in:
- [CONTRIBUTORS.md](CONTRIBUTORS.md)
- GitHub contributors page
- Release notes for significant contributions
- Release notes
Thank you for contributing to Semantica! 🎉
We follow the [all-contributors](https://allcontributors.org) specification!
---
## 📜 Code of Conduct
This project follows a [Code of Conduct](CODE_OF_CONDUCT.md). Be respectful and inclusive.
---
## 📚 Resources
- [README.md](README.md) - Project overview
- [Cookbook](cookbook/) - Tutorials and examples
- [Documentation](docs/) - Comprehensive guides
---
**Thank you for contributing!** 🚀
Every contribution matters - whether it's a single line of code, a typo fix, a helpful answer, or a bug report. We appreciate you! 🙏
**Give us a Star** • 🍴 **[Fork Semantica](https://github.com/Hawksight-AI/semantica/fork)** • 💬 **Join our [Discord](https://discord.gg/vqRt2qbx)**
+65 -48
View File
@@ -4,44 +4,31 @@ Thank you to all the people who have contributed to Semantica! 🎉
This project follows the [all-contributors](https://allcontributors.org) specification. Contributions of any kind are welcome!
## How to Contribute
**Give us a Star** • 🍴 **Fork us** • 💬 **Join our [Discord](https://discord.gg/vqRt2qbx)**
We welcome contributions of all kinds! Whether you're:
- Writing code
- Improving documentation
- Reporting bugs
- Suggesting features
- Answering questions
- Reviewing pull requests
- Sharing use cases
- Creating examples
All contributions are valuable and appreciated!
---
## Contribution Types
We recognize all types of contributions:
- 💻 **Code**: Writing code, fixing bugs, implementing features
- 📝 **Documentation**: Writing docs, tutorials, examples
- 🧪 **Testing**: Writing tests, improving test coverage
- 🐛 **Bug Reports**: Finding and reporting bugs
- 💡 **Ideas**: Suggesting new features or improvements
- 🎨 **Design**: UI/UX improvements, graphics, branding
- 📖 **Examples**: Creating code examples and tutorials
- 🔍 **Testing**: Writing tests, improving test coverage
- 💬 **Answering Questions**: Helping others in discussions
- 📢 **Talks**: Giving talks, presentations, workshops
- 🌍 **Translation**: Translating documentation
- 🎨 **Cookbook**: Creating tutorials and examples
- 💬 **Community**: Answering questions, reviewing PRs
- 🎓 **Education**: Blog posts, video tutorials, talks, workshops
- 🔧 **Tools**: Creating tools, scripts, integrations
- 📦 **Packaging**: Improving build, release, distribution
- ⚠️ **Security**: Reporting security vulnerabilities
- 🎓 **Education**: Teaching, mentoring, tutorials
- 📹 **Video**: Creating video content, tutorials
- 🎵 **Audio**: Podcasts, audio content
- 📸 **Photography**: Screenshots, images
- 🔬 **Research**: Research, analysis, studies
- 💰 **Financial**: Sponsoring, funding
- 🏗️ **Infrastructure**: CI/CD, hosting, infrastructure
- 🚇 **Maintenance**: Maintenance, triage, project management
---
## Contributors
<!-- ALL-CONTRIBUTORS-LIST:START -->
@@ -50,48 +37,78 @@ All contributions are valuable and appreciated!
<!-- ALL-CONTRIBUTORS-LIST:END -->
---
## Recognition
### Top Contributors
All contributors are recognized in:
Contributors are recognized based on their contributions to the project. Recognition includes:
- This contributors list
- [GitHub contributors page](https://github.com/Hawksight-AI/semantica/graphs/contributors)
- Release notes for significant contributions
- Community appreciation
- Listing in this file
- GitHub contributor statistics
- Special mentions in release notes
- Featured showcases for significant contributions
### Hall of Fame
Special recognition for exceptional contributions:
- **Coming soon** - We'll feature outstanding contributors here!
---
## How to Add Yourself
If you've contributed to Semantica and want to be added to this list:
### Automatic Recognition
1. **Automatic**: If you've made a commit, you'll appear in [GitHub's contributors graph](https://github.com/Hawksight-AI/semantica/graphs/contributors)
2. **Manual**: Open a PR adding yourself to this file, or use the [@all-contributors bot](https://allcontributors.org/docs/en/bot/usage)
If you've made a commit, you'll automatically appear in [GitHub's contributors graph](https://github.com/Hawksight-AI/semantica/graphs/contributors).
Example:
```markdown
- [Your Name](https://github.com/yourusername) - 💻 📝 🐛
```
### Using All-Contributors Bot
## All Contributors Bot
We use the [all-contributors](https://allcontributors.org) bot to automatically recognize contributors. To add a contributor, comment on an issue or PR:
Comment on any issue or PR with:
```
@all-contributors please add @username for code, docs, bug
```
## Thank You!
**Examples:**
Every contribution, no matter how small, helps make Semantica better. Thank you for being part of our community!
```
@all-contributors please add @johndoe for code
@all-contributors please add @janedoe for docs, bug
@all-contributors please add @devuser for code, test, maintenance
```
### Manual Addition
Open a PR adding yourself to this file:
```markdown
- [Your Name](https://github.com/yourusername) - 💻 📝 🐛
```
---
**Want to contribute?** Check out our [Contributing Guide](CONTRIBUTING.md) to get started!
## Contribution Type Codes
When using the all-contributors bot, use these codes:
- `code` - Code contributions
- `doc` - Documentation
- `test` - Testing
- `bug` - Bug reports
- `ideas` - Feature requests/ideas
- `design` - Design work
- `example` - Cookbook/examples
- `question` - Answering questions
- `talk` - Talks/presentations
- `tool` - Tools/integrations
- `packaging` - Packaging/distribution
- `security` - Security reports
- `infra` - Infrastructure
- `maintenance` - Maintenance
See [all-contributors specification](https://allcontributors.org/docs/en/emoji-key) for complete list.
---
## Thank You!
Every contribution, no matter how small, helps make Semantica better. Thank you for being part of our community! 🙏
**Want to contribute?**
⭐ Give us a Star • 🍴 [Fork us](https://github.com/Hawksight-AI/semantica/fork) • Check out our [Contributing Guide](CONTRIBUTING.md) to get started!
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2025 Hawksight AI
Copyright (c) 2026 Hawksight AI
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+285 -181
View File
@@ -1,197 +1,237 @@
<div align="center">
<img src="semantica_logo.png" alt="Semantica Logo" width="450" height="auto">
<img src="semantica_logo.png" alt="Semantica Logo" width="460"/>
# 🧠 Semantica
### Open-Source Semantic Layer & Knowledge Engineering Framework
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![PyPI version](https://badge.fury.io/py/semantica.svg)](https://pypi.org/project/semantica/)
[![Downloads](https://pepy.tech/badge/semantica)](https://pepy.tech/project/semantica)
[![Discord](https://img.shields.io/discord/semantica?color=7289da&label=discord)](https://discord.gg/semantica)
[![PyPI](https://img.shields.io/pypi/v/semantica.svg)](https://pypi.org/project/semantica/)
[![Total Downloads](https://static.pepy.tech/badge/semantica)](https://pepy.tech/project/semantica)
[![CI](https://github.com/Hawksight-AI/semantica/workflows/CI/badge.svg)](https://github.com/Hawksight-AI/semantica/actions)
[![Discord](https://img.shields.io/badge/Discord-Join-7289da?logo=discord&logoColor=white)](https://discord.gg/RgaGTj9J)
<p align="center">
<a href="https://github.com/Hawksight-AI/semantica/stargazers">
<img src="https://img.shields.io/badge/Give%20a%20Star-%E2%AD%90-yellow?style=for-the-badge&labelColor=555555" alt="Give a Star">
</a>
&nbsp;&nbsp;
<a href="https://github.com/Hawksight-AI/semantica/fork">
<img src="https://img.shields.io/badge/Support%20Project-Fork%20Us-blue?style=for-the-badge&labelColor=555555" alt="Support Project">
</a>
</p>
### ⭐ Give us a Star • 🍴 Fork us • 💬 Join our Discord
**Open Source Framework for Semantic Layer & Knowledge Engineering**
> **Transform chaotic data into intelligent knowledge.**
*The missing fabric between raw data and AI engineering. A comprehensive open-source framework for building semantic layers and knowledge engineering systems that transform unstructured data into AI-ready knowledge — powering Knowledge Graph-Powered RAG (GraphRAG), AI Agents, Multi-Agent Systems, and AI applications with structured semantic knowledge.*
**100% Open Source****MIT Licensed****Production Ready****Community Driven**
[**Discord**](https://discord.gg/semantica)
> **Transform Choas into Intelligence. Build AI systems that are explainable, traceable, and trustworthy — not black boxes.**
</div>
## What is Semantica?
Semantica bridges the gap between raw data chaos and AI-ready knowledge. It's a **semantic intelligence platform** that transforms unstructured data into structured, queryable knowledge graphs powering GraphRAG, AI agents, and multi-agent systems.
### What Makes Semantica Different?
Unlike traditional approaches that process isolated documents and extract text into vectors, Semantica understands **semantic relationships across all content**, provides **automated ontology generation**, and builds a **unified semantic layer** with **production-grade QA**.
| **Traditional Approaches** | **Semantica's Approach** |
|:---------------------------|:-------------------------|
| Process data as isolated documents | Understands semantic relationships across all content |
| Extract text and store vectors | Builds knowledge graphs with meaningful connections |
| Generic entity recognition | General-purpose ontology generation and validation |
| Manual schema definition | Automatic semantic modeling from content patterns |
| Disconnected data silos | Unified semantic layer across all data sources |
| Basic quality checks | Production-grade QA with conflict detection & resolution |
---
## 🎯 The Problem We Solve
## 🚀 Why Semantica?
### The Semantic Gap
**Semantica** bridges the **semantic gap** between text similarity and true meaning. It's the **semantic intelligence layer** that makes your AI agents auditable, explainable, and compliant.
Organizations today face a **fundamental mismatch** between how data exists and how AI systems need it.
#### The Semantic Gap: Problem vs. Solution
Organizations have **unstructured data** (PDFs, emails, logs), **messy data** (inconsistent formats, duplicates, conflicts), and **disconnected silos** (no shared context, missing relationships). AI systems need **clear rules** (formal ontologies), **structured entities** (validated, consistent), and **relationships** (semantic connections, context-aware reasoning).
| **What Organizations Have** | **What AI Systems Require** |
|:------------------------------|:------------------------------|
| **Unstructured Data** | **Clear Rules** |
| PDFs, emails, logs | Formal ontologies |
| Mixed schemas | Graphs & Networks |
| Conflicting facts | |
| **Messy, Noisy Data** | **Structured Entities** |
| Inconsistent formats | Validated entities |
| Duplicate records | Domain Knowledge |
| Missing relationships | |
| **Disconnected, Siloed Data** | **Relationships** |
| Data in separate systems | Semantic connections |
| No shared context | Context-Aware Reasoning |
| Isolated knowledge | |
### **SEMANTICA FRAMEWORK**
Semantica operates through three integrated layers that transform raw data into AI-ready knowledge:
**Input Layer** — Universal ingestion from multiple data formats (PDFs, DOCX, HTML, JSON, CSV, databases, live feeds, APIs, streams, archives, multi-modal content) into a unified pipeline.
**Semantic Layer** — Core intelligence engine performing entity extraction, relationship mapping, ontology generation, context engineering, and quality assurance. Includes **advanced entity deduplication** (Jaro-Winkler, disjoint property handling) to ensure a clean single source of truth.
**Output Layer** — Production-ready knowledge graphs, vector embeddings, and validated ontologies that power GraphRAG systems, AI agents, and multi-agent systems.
**Powers: GraphRAG, AI Agents, Multi-Agent Systems**
### What Happens Without Semantics?
**They Break** — Systems crash due to inconsistent formats and missing structure.
**They Hallucinate** — AI models generate false information without semantic context to validate outputs.
**They Fail Silently** — Systems return wrong answers without warnings, leading to bad decisions.
**Why?** Systems have data — not semantics. They can't connect concepts, understand relationships, validate against domain rules, or detect conflicts.
Perfect for **high-stakes domains** where mistakes have real consequences.
---
## 💡 The Semantica Solution
### ⚡ Get Started in 30 Seconds
**Semantica** is an **open-source framework** that closes the semantic gap between real-world messy data and the structured semantic layers required by advanced AI systems — GraphRAG, agents, multi-agent systems, reasoning models, and more.
```bash
pip install semantica
```
### How Semantica Solves These Problems
```python
from semantica.semantic_extract import NERExtractor
from semantica.kg import GraphBuilder
**Efficient Embeddings** — Uses **FastEmbed** by default for high-performance, lightweight local embedding generation (faster than sentence-transformers).
# Extract entities and build knowledge graph
ner = NERExtractor(method="ml", model="en_core_web_sm")
entities = ner.extract("Apple Inc. was founded by Steve Jobs in 1976.")
kg = GraphBuilder().build({"entities": entities, "relationships": []})
**Universal Data Ingestion** — Handles multiple formats (PDF, DOCX, HTML, JSON, CSV, databases, APIs, streams) with unified pipeline, no custom parsers needed.
print(f"Built KG with {len(kg.get('entities', []))} entities")
```
**Automated Semantic Extraction** — NER, relationship extraction, and triplet generation with LLM enhancement discovers entities and relationships automatically.
**Knowledge Graph Construction** — Production-ready graphs with entity resolution, temporal support, and graph analytics. Queryable knowledge ready for AI applications.
**GraphRAG Engine** — Hybrid vector + graph retrieval achieves 91% accuracy (30% improvement) via semantic search + graph traversal for multi-hop reasoning. Features LLM-generated responses grounded in knowledge graph context with reasoning traces. [See Comparison Benchmark](cookbook/use_cases/advanced_rag/02_RAG_vs_GraphRAG_Comparison.ipynb)
**AI Agent Context Engineering** — Persistent memory with RAG + knowledge graphs enables context maintenance, action validation, and structured knowledge access.
**Automated Ontology Generation** — 6-stage LLM pipeline generates validated OWL ontologies with HermiT/Pellet validation, eliminating manual engineering.
**Production-Grade QA** — Conflict detection, deduplication, quality scoring, and provenance tracking ensure trusted, production-ready knowledge graphs.
**Pipeline Orchestration** — Flexible pipeline builder with parallel execution enables scalable processing via orchestrator-worker pattern.
### Core Features at a Glance
| **Feature Category** | **Capabilities** | **Key Benefits** |
|:---------------------|:-----------------|:------------------|
| **Data Ingestion** | Multiple formats (PDF, DOCX, HTML, JSON, CSV, databases, APIs, streams, archives) | Universal ingestion, no custom parsers needed |
| **Semantic Extraction** | NER, relationship extraction, triplet generation, LLM enhancement | Automated discovery of entities and relationships |
| **Knowledge Graphs** | Entity resolution, temporal support, graph analytics, query interface | Production-ready, queryable knowledge structures |
| **Ontology Generation** | 6-stage LLM pipeline, OWL generation, HermiT/Pellet validation | Automated ontology creation from documents |
| **GraphRAG** | Hybrid vector + graph retrieval, multi-hop reasoning, LLM-generated responses | 91% accuracy, 30% improvement over vector-only, reasoning traces |
| **LLM Providers** | Unified interface to 100+ LLMs (Groq, OpenAI, HuggingFace, LiteLLM) | Clean imports, multiple providers, structured output |
| **Agent Memory** | Persistent memory (Save/Load), Hybrid Retrieval (Vector+Graph), FastEmbed support | Context-aware agents with semantic understanding |
| **Pipeline Orchestration** | Parallel execution, custom steps, orchestrator-worker pattern | Scalable, flexible data processing |
| **Quality Assurance** | Conflict detection, deduplication, quality scoring, provenance | Trusted knowledge graphs ready for production |
**[📖 Full Quick Start](#-quick-start)** • **[🍳 Cookbook Examples](#-semantica-cookbook)** • **[💬 Join Discord](https://discord.gg/RgaGTj9J)** • **[⭐ Star Us](https://github.com/Hawksight-AI/semantica)**
---
## 👥 Who Is This For?
## Core Value Proposition
Semantica is designed for **developers, data engineers, and organizations** building the next generation of AI applications that require semantic understanding and knowledge graphs.
| **Trustworthy** | **Explainable** | **Auditable** |
|:------------------:|:------------------:|:-----------------:|
| Conflict detection & validation | Transparent reasoning paths | Complete provenance tracking |
| Rule-based governance | Entity relationships & ontologies | Source-level provenance |
| Production-grade QA | Multi-hop graph reasoning | Audit-ready compliance |
### Who Uses Semantica
---
**AI/ML Engineers & Data Scientists** — Build GraphRAG systems, AI agents, and multi-agent systems.
## Key Features & Benefits
**Data Engineers** — Build scalable pipelines with semantic enrichment.
### Not Just Another Agentic Framework
**Knowledge Engineers & Ontologists** — Create knowledge graphs and ontologies with automated pipelines.
**Semantica complements** LangChain, LlamaIndex, AutoGen, CrewAI, Google ADK, Agno, and other frameworks to enhance your agents with:
**Enterprise Data Teams** — Unify semantic layers, improve data quality, resolve conflicts.
| Feature | Benefit |
|:--------|:--------|
| **Auditable** | Complete provenance tracking with full audit trails |
| **Explainable** | Transparent reasoning paths with entity relationships |
| **Provenance-Aware** | Source-level provenance from documents to responses |
| **Validated** | Built-in conflict detection, deduplication, QA |
| **Governed** | Rule-based validation and semantic consistency |
| **Version Control** | Enterprise-grade change management with HIPAA/SOX/FDA compliance |
**Software & DevOps Engineers** — Build semantic APIs and infrastructure with production-ready SDK.
### Perfect For High-Stakes Use Cases
**Analysts & Researchers** — Transform data into queryable knowledge graphs for insights.
| 🏥 **Healthcare** | 💰 **Finance** | ⚖️ **Legal** |
|:-----------------:|:--------------:|:------------:|
| Clinical decisions | Fraud detection | Evidence-backed research |
| Drug interactions | Regulatory compliance | Contract analysis |
| Patient safety | Risk assessment | Case law reasoning |
**Security & Compliance Teams** — Threat intelligence, regulatory reporting, audit trails.
| 🔒 **Cybersecurity** | 🏛️ **Government** | 🏭 **Infrastructure** | 🚗 **Autonomous** |
|:-------------------:|:----------------:|:-------------------:|:-----------------:|
| Threat attribution | Policy decisions | Power grids | Decision logs |
| Incident response | Classified info | Transportation | Safety validation |
**Product Teams & Startups** — Rapid prototyping of AI products and semantic features.
### Powers Your AI Stack
- **GraphRAG Systems** — Retrieval with graph reasoning and hybrid search
- **AI Agents** — Trustworthy, accountable multi-agent systems with semantic memory
- **Reasoning Models** — Explainable AI decisions with reasoning paths
- **Enterprise AI** — Governed, auditable platforms for compliance
### Integrations
- **Docling Support** — Document parsing with table extraction (PDF, DOCX, PPTX, XLSX)
- **AWS Neptune** — Amazon Neptune graph database support with IAM authentication
- **Custom Ontology Import** — Import existing ontologies (OWL, RDF, Turtle, JSON-LD)
> **Built for environments where every answer must be explainable and governed.**
---
## 🚨 The Problem: The Semantic Gap
### Most AI systems fail in high-stakes domains because they operate on **text similarity**, not **meaning**.
### Understanding the Semantic Gap
The **semantic gap** is the fundamental disconnect between what AI systems can process (text patterns, vector similarities) and what high-stakes applications require (semantic understanding, meaning, context, and relationships).
**Traditional AI approaches:**
- Rely on statistical patterns and text similarity
- Cannot understand relationships between entities
- Cannot reason about domain-specific rules
- Cannot explain why decisions were made
- Cannot trace back to original sources with confidence
**High-stakes AI requires:**
- Semantic understanding of entities and their relationships
- Domain knowledge encoded as formal rules (ontologies)
- Explainable reasoning paths
- Source-level provenance
- Conflict detection and resolution
**Semantica bridges this gap** by providing a semantic intelligence layer that transforms unstructured data into validated, explainable, and auditable knowledge.
### What Organizations Have vs What They Need
| **Current State** | **Required for High-Stakes AI** |
|:---------------------|:-----------------------------------|
| PDFs, DOCX, emails, logs | Formal domain rules (ontologies) |
| APIs, databases, streams | Structured and validated entities |
| Conflicting facts and duplicates | Explicit semantic relationships |
| Siloed systems with no lineage | **Explainable reasoning paths** |
| | **Source-level provenance** |
| | **Audit-ready compliance** |
### The Cost of Missing Semantics
- **Decisions cannot be explained** — No transparency in AI reasoning
- **Errors cannot be traced** — No way to debug or improve
- **Conflicts go undetected** — Contradictory information causes failures
- **Compliance becomes impossible** — No audit trails for regulations
**Trustworthy AI requires semantic accountability.**
---
## 🆚 Semantica vs Traditional RAG
| Feature | Traditional RAG | Semantica |
|:--------|:----------------|:----------|
| **Reasoning** | ❌ Black-box answers | ✅ Explainable reasoning paths |
| **Provenance** | ❌ No provenance | ✅ Source-level provenance |
| **Search** | ⚠️ Vector similarity only | ✅ Semantic + graph reasoning |
| **Quality** | ❌ No conflict handling | ✅ Explicit contradiction detection |
| **Safety** | ⚠️ Unsafe for high-stakes | ✅ Designed for governed environments |
| **Compliance** | ❌ No audit trails | ✅ Audit-ready provenance |
---
## 🧩 Semantica Architecture
### 1️⃣ Input Layer — Governed Ingestion
- 📄 **Multiple Formats** — PDFs, DOCX, HTML, JSON, CSV, Excel, PPTX
- 🔧 **Docling Support** — Docling parser for table extraction
- 💾 **Data Sources** — Databases, APIs, streams, archives, web content
- 🎨 **Media Support** — Image parsing with OCR, audio/video metadata extraction
- 📊 **Single Pipeline** — Unified ingestion with metadata and source tracking
### 2️⃣ Semantic Layer — Trust & Reasoning Engine
- 🔍 **Entity Extraction** — NER, normalization, classification
- 🔗 **Relationship Discovery** — Triplet generation, semantic links
- 📐 **Ontology Induction** — Automated domain rule generation
- 🔄 **Deduplication** — Jaro-Winkler similarity, conflict resolution
-**Quality Assurance** — Conflict detection, validation
- 📊 **Provenance Tracking** — Source, time, confidence metadata
- 🧠 **Reasoning Traces** — Explainable inference paths
- 🔐 **Change Management** — Version control with audit trails, checksums, HIPAA/SOX/FDA compliance
### 3️⃣ Output Layer — Auditable Knowledge Assets
- 📊 **Knowledge Graphs** — Queryable, temporal, explainable
- 📐 **OWL Ontologies** — HermiT/Pellet validated, custom ontology import support
- 🔢 **Vector Embeddings** — FastEmbed by default
- ☁️ **AWS Neptune** — Amazon Neptune graph database support
- 🔍 **Provenance** — Every AI response links back to:
- 📄 Source documents
- 🏷️ Extracted entities & relations
- 📐 Ontology rules applied
- 🧠 Reasoning steps used
---
## 🏥 Built for High-Stakes Domains
Designed for domains where **mistakes have real consequences** and **every decision must be accountable**:
- **🏥 Healthcare & Life Sciences** — Clinical decision support, drug interaction analysis, medical literature reasoning, patient safety compliance
- **💰 Finance & Risk** — Fraud detection, regulatory compliance (SOX, GDPR, MiFID II), credit risk assessment, algorithmic trading validation
- **⚖️ Legal & Compliance** — Evidence-backed legal research, contract analysis, regulatory change management, case law reasoning
- **🔒 Cybersecurity & Intelligence** — Threat attribution, incident response, security audit trails, intelligence analysis
- **🏛️ Government & Defense** — Governed AI systems, policy decisions, classified information handling, defense intelligence
- **🏭 Critical Infrastructure** — Power grid management, transportation safety, water treatment, emergency response
- **🚗 Autonomous Systems** — Self-driving vehicles, drone navigation, robotics safety, industrial automation
---
## 👥 Who Uses Semantica?
- **🤖 AI / ML Engineers** — Building explainable GraphRAG & agents
- **⚙️ Data Engineers** — Creating governed semantic pipelines
- **📊 Knowledge Engineers** — Managing ontologies & KGs at scale
- **🏢 Enterprise Teams** — Requiring trustworthy AI infrastructure
- **🛡️ Risk & Compliance Teams** — Needing audit-ready systems
---
## 📦 Installation
> **✅ Available on PyPI!** Semantica is now published on PyPI. Install it with a single command: `pip install semantica`
**Prerequisites:** Python 3.8+ (3.9+ recommended) • pip (latest version)
### Install from PyPI (Recommended)
```bash
# Install latest version from PyPI
pip install semantica
# Or install with optional dependencies
# or
pip install semantica[all]
# Verify installation
python -c "import semantica; print(semantica.__version__)"
```
**Current Version:** [![PyPI version](https://badge.fury.io/py/semantica.svg)](https://pypi.org/project/semantica/) • [View on PyPI](https://pypi.org/project/semantica/)
### Install from Source (Development)
```bash
@@ -251,17 +291,27 @@ print(f" Ingested {len(sources)} sources")
### Document Parsing & Processing
> **Multi-format parsing** • **Text normalization** • **Intelligent chunking**
> **Multi-format parsing** • **Docling Support** • **Text normalization** • **Intelligent chunking**
```python
from semantica.parse import DocumentParser
from semantica.parse import DocumentParser, DoclingParser
from semantica.normalize import TextNormalizer
from semantica.split import TextSplitter
# Parse documents
# Standard parsing
parser = DocumentParser()
parsed = parser.parse("document.pdf", format="auto")
# Parsing with Docling (for complex layouts/tables)
# Requires: pip install docling
docling_parser = DoclingParser(enable_ocr=True)
result = docling_parser.parse("complex_table.pdf")
print(f"Text (Markdown): {result['full_text'][:100]}...")
print(f"Extracted {len(result['tables'])} tables")
for i, table in enumerate(result['tables']):
print(f"Table {i+1} headers: {table.get('headers', [])}")
# Normalize text
normalizer = TextNormalizer()
normalized = normalizer.normalize(parsed, clean_html=True, normalize_entities=True)
@@ -343,7 +393,7 @@ results = vector_store.search(query="supply chain", top_k=5)
### Graph Store & Triplet Store
> **Neo4j, FalkorDB support** • **SPARQL queries** • **RDF triplets**
> **Neo4j, FalkorDB, Amazon Neptune** • **SPARQL queries** • **RDF triplets**
```python
from semantica.graph_store import GraphStore
@@ -353,6 +403,24 @@ from semantica.triplet_store import TripletStore
graph_store = GraphStore(backend="neo4j", uri="bolt://localhost:7687", user="neo4j", password="password")
graph_store.add_nodes([{"id": "n1", "labels": ["Person"], "properties": {"name": "Alice"}}])
# Amazon Neptune Graph Store (OpenCypher via HTTP with IAM Auth)
neptune_store = GraphStore(
backend="neptune",
endpoint="your-cluster.us-east-1.neptune.amazonaws.com",
port=8182,
region="us-east-1",
iam_auth=True, # Uses AWS credential chain (boto3, env vars, or IAM role)
)
# Node Operations
neptune_store.add_nodes([
{"labels": ["Person"], "properties": {"id": "alice", "name": "Alice", "age": 30}},
{"labels": ["Person"], "properties": {"id": "bob", "name": "Bob", "age": 25}},
])
# Query Operations
result = neptune_store.execute_query("MATCH (p:Person) RETURN p.name, p.age")
# Triplet Store (Blazegraph, Jena, RDF4J)
triplet_store = TripletStore(backend="blazegraph", endpoint="http://localhost:9999/blazegraph")
triplet_store.add_triplet({"subject": "Alice", "predicate": "knows", "object": "Bob"})
@@ -363,19 +431,60 @@ results = triplet_store.execute_query("SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT
### Ontology Generation & Management
> **6-Stage LLM Pipeline** • Automatic OWL Generation • HermiT/Pellet Validation
> **6-Stage LLM Pipeline** • Automatic OWL Generation • HermiT/Pellet Validation • **Custom Ontology Import** (OWL, RDF, Turtle, JSON-LD)
```python
from semantica.ontology import OntologyGenerator
from semantica.ingest import ingest_ontology
# Generate ontology automatically
generator = OntologyGenerator(llm_provider="openai", model="gpt-4")
ontology = generator.generate_from_documents(sources=["domain_docs/"])
print(f"Classes: {len(ontology.classes)}")
# Or import your existing ontology
custom_ontology = ingest_ontology("my_ontology.ttl") # Supports OWL, RDF, Turtle, JSON-LD
print(f"Classes: {len(custom_ontology.classes)}")
```
[**Cookbook: Ontology**](https://github.com/Hawksight-AI/semantica/tree/main/cookbook/introduction/14_Ontology.ipynb)
### Change Management & Version Control
> **Enterprise-Grade Versioning** • Persistent Storage • Audit Trails • HIPAA/SOX/FDA Compliance • SHA-256 Checksums
```python
from semantica.change_management import TemporalVersionManager, OntologyVersionManager
# Knowledge Graph versioning with audit trails
kg_manager = TemporalVersionManager(storage_path="kg_versions.db")
# Create versioned snapshot
snapshot = kg_manager.create_snapshot(
knowledge_graph,
version_label="v1.0",
author="user@company.com",
description="Initial patient record"
)
# Compare versions with detailed diffs
diff = kg_manager.compare_versions("v1.0", "v2.0")
print(f"Entities added: {diff['summary']['entities_added']}")
print(f"Entities modified: {diff['summary']['entities_modified']}")
# Verify data integrity
is_valid = kg_manager.verify_checksum(snapshot)
```
**Key Features:**
- 🔐 **Persistent Storage** — SQLite and in-memory backends
- 📊 **Detailed Diffs** — Entity-level and relationship-level change tracking
-**Data Integrity** — SHA-256 checksums with tamper detection
- 🏥 **Compliance Ready** — HIPAA, SOX, FDA 21 CFR Part 11 support
-**High Performance** — 17.6ms for 10k entities, 510+ ops/sec concurrent
- 🧪 **Fully Tested** — 104 tests covering real-world scenarios
[**Documentation: Change Management**](docs/reference/change_management.md) • [**Usage Guide**](semantica/change_management/change_management_usage.md)
### Context Engineering & Memory Systems
> **Persistent Memory** • **Context Graph** • **Context Retriever** • **Hybrid Retrieval (Vector + Graph)** • **Production Graph Store (Neo4j)** • **Entity Linking** • **Multi-Hop Reasoning**
@@ -441,7 +550,7 @@ reasoned_result = context.query_with_reasoning(
### Knowledge Graph-Powered RAG (GraphRAG)
> **30% Accuracy Improvement** • Vector + Graph Hybrid Search • 91% Accuracy • **Multi-Hop Reasoning** • **LLM-Generated Responses**
> **Vector + Graph Hybrid Search** • **Multi-Hop Reasoning** • **LLM-Generated Responses** • **Semantic Re-ranking**
```python
from semantica.context import AgentContext
@@ -490,7 +599,7 @@ print(f"Confidence: {result['confidence']:.3f}")
from semantica.llms import Groq, OpenAI, HuggingFaceLLM, LiteLLM
import os
# Groq - Fast inference
# Groq
groq = Groq(
model="llama-3.1-8b-instant",
api_key=os.getenv("GROQ_API_KEY")
@@ -520,7 +629,7 @@ structured = groq.generate_structured("Extract entities from: Apple Inc. was fou
```
**Supported Providers:**
- **Groq**: Fast inference with Llama models
- **Groq**: Inference with Llama models
- **OpenAI**: GPT-3.5, GPT-4, and other OpenAI models
- **HuggingFace**: Local LLM inference with Transformers
- **LiteLLM**: Unified interface to 100+ LLM providers (OpenAI, Anthropic, Azure, Bedrock, Vertex AI, and more)
@@ -720,7 +829,7 @@ print(f"Found {len(results)} results")
#### Cybersecurity
- [**Real-Time Anomaly Detection**](cookbook/use_cases/cybersecurity/01_Real_Time_Anomaly_Detection.ipynb) - CVE RSS, Kafka streams, temporal KGs, sentence chunking
- [**Threat Intelligence Hybrid RAG**](cookbook/use_cases/cybersecurity/02_Threat_Intelligence_Hybrid_RAG.ipynb) - Security RSS, entity-aware chunking, enhanced GraphRAG, deduplication
- [**Threat Intelligence Hybrid RAG**](cookbook/use_cases/cybersecurity/02_Threat_Intelligence_Hybrid_RAG.ipynb) - Security RSS, entity-aware chunking, GraphRAG, deduplication
#### Intelligence & Law Enforcement
- [**Criminal Network Analysis**](cookbook/use_cases/intelligence/01_Criminal_Network_Analysis.ipynb) - OSINT RSS, deduplication, network centrality, graph analytics
@@ -737,12 +846,16 @@ print(f"Found {len(results)} results")
## 🔬 Advanced Features
**Docling Integration** — Document parsing with table extraction for PDFs, DOCX, PPTX, and XLSX files. Supports OCR and multiple export formats.
**AWS Neptune Support** — Amazon Neptune graph database integration with IAM authentication and OpenCypher queries.
**Custom Ontology Import** — Import existing ontologies (OWL, RDF, Turtle, JSON-LD, N3) and extend Schema.org, FOAF, Dublin Core, or custom ontologies.
**Incremental Updates** — Real-time stream processing with Kafka, RabbitMQ, Kinesis for live updates.
**Multi-Language Support** — Process multiple languages with automatic detection.
**Custom Ontology Import** — Import and extend Schema.org and custom ontologies.
**Advanced Reasoning** — Forward/backward chaining, Rete-based pattern matching, and automated explanation generation.
**Graph Analytics** — Centrality, community detection, path finding, temporal analysis.
@@ -762,6 +875,7 @@ print(f"Found {len(results)} results")
- [x] Advanced reasoning v2 (Rete, Forward/Backward Chaining)
- [ ] Quality assurance features and Quality Assurance module
- [ ] Enhanced multi-language support
- [ ] Evals
- [ ] Real-time streaming improvements
### Q2 2026
@@ -775,7 +889,7 @@ print(f"Found {len(results)} results")
| **Channel** | **Purpose** |
|:-----------:|:-----------|
| [**Discord**](https://discord.gg/semantica) | Real-time help, showcases |
| [**Discord**](https://discord.gg/pMHguUzG) | Real-time help, showcases |
| [**GitHub Discussions**](https://github.com/Hawksight-AI/semantica/discussions) | Q&A, feature requests |
### Learning Resources
@@ -783,14 +897,20 @@ print(f"Found {len(results)} results")
### Enterprise Support
| **Tier** | **Features** | **SLA** | **Price** |
|:--------:|:-----------|:-------:|:--------:|
| **Community** | Public support | Best effort | Free |
| **Professional** | Email support | 48h | Contact |
| **Enterprise** | 24/7 support | 4h | Contact |
| **Premium** | Phone, custom dev | 1h | Contact |
Enterprise support, professional services, and commercial licensing will be available in the future. For now, we offer community support through Discord and GitHub Discussions.
**Contact:** [GitHub Issues](https://github.com/Hawksight-AI/semantica/issues) with "[Enterprise]" prefix
**Current Support:**
- **Community Support** - Free support via [Discord](https://discord.gg/pMHguUzG) and [GitHub Discussions](https://github.com/Hawksight-AI/semantica/discussions)
- **Bug Reports** - [GitHub Issues](https://github.com/Hawksight-AI/semantica/issues)
**Future Enterprise Offerings:**
- Professional support with SLA
- Enterprise licensing
- Custom development services
- Priority feature requests
- Dedicated support channels
Stay tuned for updates!
## 🤝 Contributing
@@ -824,28 +944,12 @@ git push origin feature/your-feature
3. **Bug Reports** - [Create issue](https://github.com/Hawksight-AI/semantica/issues/new)
4. **Feature Requests** - [Request feature](https://github.com/Hawksight-AI/semantica/issues/new)
### Recognition
Contributors receive:
- Recognition in [CONTRIBUTORS.md](https://github.com/Hawksight-AI/semantica/blob/main/CONTRIBUTORS.md)
- GitHub badges
- Semantica swag
- Featured showcases
## 🏆 Contributors
<a href="https://github.com/Hawksight-AI/semantica/graphs/contributors">
<img src="https://contrib.rocks/image?repo=Hawksight-AI/semantica" alt="Contributors" />
</a>
## 📜 License
Semantica is licensed under the **MIT License** - see the [LICENSE](https://github.com/Hawksight-AI/semantica/blob/main/LICENSE) file for details.
<div align="center">
**Built by the Semantica Community**
[GitHub](https://github.com/Hawksight-AI/semantica) • [Discord](https://discord.gg/semantica)
</div>
[GitHub](https://github.com/Hawksight-AI/semantica) • [Discord](https://discord.gg/RgaGTj9J)
+3 -3
View File
@@ -26,10 +26,10 @@ Before releasing, ensure:
The project uses GitHub Actions for automated releases to PyPI.
1. **Tag the commit**: Create a new git tag for the version (e.g., `v0.1.0`).
1.29. **Tag the commit**: Create a new git tag for the version (e.g., `v0.2.3`).
```bash
git tag -a v0.1.0 -m "Release v0.1.0"
git push origin v0.1.0
git tag -a v0.2.3 -m "Release v0.2.3"
git push origin v0.2.3
```
2. **GitHub Action**: The `Release` workflow will automatically trigger, build the package, create a GitHub Release, and publish to PyPI using Trusted Publishing.
+5
View File
@@ -6,6 +6,11 @@ We actively support the following versions of Semantica with security updates:
| Version | Supported |
| ------- | ------------------ |
| 0.2.3 | :white_check_mark: |
| 0.2.2 | :white_check_mark: |
| 0.2.1 | :white_check_mark: |
| 0.2.0 | :white_check_mark: |
| 0.1.1 | :white_check_mark: |
| 0.1.0 | :white_check_mark: |
| < 0.1.0 | :x: |
+1 -1
View File
@@ -27,7 +27,7 @@ Start with our comprehensive documentation:
**Best for**: Real-time chat and quick questions
- [Join Discord](https://discord.gg/semantica)
- [Join Discord](https://discord.gg/pMHguUzG)
#### GitHub Issues
+14 -108
View File
@@ -42,55 +42,18 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install -q semantica"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>EventDetector</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>CoreferenceResolver</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>TripletExtractor</td><td>-</td><td>1.32s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>NERExtractor</td><td>-</td><td>0.51s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>RelationExtractor</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>SemanticNetworkExtractor</td><td>-</td><td>1.21s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detected 1 events\n",
" Event: founded - founded\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import (\n",
" EventDetector, CoreferenceResolver, TripletExtractor,\n",
@@ -118,17 +81,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Resolved 0 coreference chains\n"
]
}
],
"outputs": [],
"source": [
"coreference_resolver = CoreferenceResolver()\n",
"\n",
@@ -148,24 +103,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"DEBUG: Entity map keys: ['apple inc.', 'steve jobs', '1976', 'tim cook']\n",
"DEBUG: Match found! Subject='Apple Inc.', Object='Steve Jobs'\n",
"DEBUG: Subject Entity found: True, Object Entity found: True\n",
"DEBUG: Match found! Subject='Steve Jobs', Object='1976'\n",
"DEBUG: Subject Entity found: True, Object Entity found: True\n",
"Extracted 2 triplets\n",
" (Apple Inc., founded_by, Steve Jobs)\n",
" (Steve Jobs, located_in, 1976)\n"
]
}
],
"outputs": [],
"source": [
"triplet_extractor = TripletExtractor()\n",
"\n",
@@ -187,17 +127,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Analyzed semantic roles: 6\n"
]
}
],
"outputs": [],
"source": [
"semantic_analyzer = SemanticAnalyzer()\n",
"\n",
@@ -217,23 +149,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"DEBUG: Entity map keys: ['apple inc.', 'steve jobs', '1976', 'tim cook']\n",
"DEBUG: Match found! Subject='Apple Inc.', Object='Steve Jobs'\n",
"DEBUG: Subject Entity found: True, Object Entity found: True\n",
"DEBUG: Match found! Subject='Steve Jobs', Object='1976'\n",
"DEBUG: Subject Entity found: True, Object Entity found: True\n",
"Extracted semantic network with 4 nodes\n",
"Edges: 2\n"
]
}
],
"outputs": [],
"source": [
"semantic_network_extractor = SemanticNetworkExtractor()\n",
"\n",
@@ -254,21 +172,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"ename": "AttributeError",
"evalue": "'LLMEnhancer' object has no attribute 'enhance_extractions'",
"output_type": "error",
"traceback": [
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[1;32mIn[7], line 3\u001b[0m\n\u001b[0;32m 1\u001b[0m llm_enhancer \u001b[38;5;241m=\u001b[39m LLMEnhancer()\n\u001b[1;32m----> 3\u001b[0m enhanced_extractions \u001b[38;5;241m=\u001b[39m \u001b[43mllm_enhancer\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43menhance_extractions\u001b[49m(events, text)\n\u001b[0;32m 5\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEnhanced \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mlen\u001b[39m(enhanced_extractions)\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m extractions\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
"\u001b[1;31mAttributeError\u001b[0m: 'LLMEnhancer' object has no attribute 'enhance_extractions'"
]
}
],
"outputs": [],
"source": [
"llm_enhancer = LLMEnhancer()\n",
"\n",
@@ -22,33 +22,17 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"id": "695d435c",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install -q semantica"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -89,17 +73,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Loaded 6 raw entities and 4 raw relationships.\n"
]
}
],
"outputs": [],
"source": [
"# Our \"Raw\" Messy Data\n",
"raw_entities = [\n",
@@ -141,25 +117,10 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"id": "bd8fb13d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running Validation Check...\n",
"Validation Failed! Issues found:\n",
" - [ERROR] Target entity ID not found: startup_999 (Code: DANGLING_EDGE)\n",
" Auto-Fixing: Removing invalid relationship...\n",
" - [WARNING] Found 2 orphan nodes (no relationships). (Code: ORPHAN_NODES)\n",
"\n",
"Re-validating after fixes...\n",
"Graph is now clean and valid!\n"
]
}
],
"outputs": [],
"source": [
"# Initialize Validator\n",
"validator = GraphValidator()\n",
@@ -203,180 +164,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Scanning for duplicates...\n"
]
},
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.25s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CentralityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CommunityDetector</td><td>-</td><td>0.01s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2025-12-19 16:09:21,008 - INFO - [RUNNING] | Module: deduplication | Submodule: DuplicateDetector | Message: Detecting duplicates in 6 entities\n",
"2025-12-19 16:09:21,008 - INFO - Detecting duplicates in 6 entities (threshold: 0.7)\n",
"2025-12-19 16:09:21,011 - INFO - [RUNNING] | Module: deduplication | Submodule: DuplicateDetector | Message: Calculating similarities...\n",
"2025-12-19 16:09:21,013 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,015 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,018 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,020 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,023 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,026 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.56\n",
"2025-12-19 16:09:21,029 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,031 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,034 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,036 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,038 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,040 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.86\n",
"2025-12-19 16:09:21,043 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,046 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,049 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,052 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,055 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,058 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.61\n",
"2025-12-19 16:09:21,060 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,063 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,065 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,068 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,069 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,072 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.61\n",
"2025-12-19 16:09:21,075 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,077 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,080 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,083 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,087 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,089 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.54\n",
"2025-12-19 16:09:21,092 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,093 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,098 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,100 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,103 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,106 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.57\n",
"2025-12-19 16:09:21,109 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,112 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,115 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,117 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,121 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,124 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.59\n",
"2025-12-19 16:09:21,126 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,129 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,131 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,136 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,138 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,142 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.59\n",
"2025-12-19 16:09:21,146 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,150 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,153 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,156 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,159 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,162 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.30\n",
"2025-12-19 16:09:21,166 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,168 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,170 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,173 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,175 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,177 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.56\n",
"2025-12-19 16:09:21,179 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,180 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,184 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,186 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,187 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,188 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.66\n",
"2025-12-19 16:09:21,192 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,193 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,196 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,198 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,200 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,201 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.55\n",
"2025-12-19 16:09:21,203 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,206 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,206 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,210 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,211 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,213 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.56\n",
"2025-12-19 16:09:21,215 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,217 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,221 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,225 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,228 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,231 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.50\n",
"2025-12-19 16:09:21,234 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating similarity between entities\n",
"2025-12-19 16:09:21,237 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating string similarity...\n",
"2025-12-19 16:09:21,240 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating property similarity...\n",
"2025-12-19 16:09:21,243 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Calculating relationship similarity...\n",
"2025-12-19 16:09:21,246 - INFO - [RUNNING] | Module: deduplication | Submodule: SimilarityCalculator | Message: Aggregating similarity scores...\n",
"2025-12-19 16:09:21,249 - INFO - [COMPLETED] | Module: deduplication | Submodule: SimilarityCalculator | Message: Similarity score: 0.56\n",
"2025-12-19 16:09:21,252 - INFO - [RUNNING] | Module: deduplication | Submodule: DuplicateDetector | Message: Creating duplicate candidates...\n",
"2025-12-19 16:09:21,254 - INFO - Detected 1 duplicate candidate(s) (confidence >= 0.6)\n",
"2025-12-19 16:09:21,258 - INFO - [COMPLETED] | Module: deduplication | Submodule: DuplicateDetector | Message: Detected 1 duplicate candidates\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found potential duplicate pair (Score: 0.86):\n",
" - TechFlow AI (ID: startup_1)\n",
" - TechFlow Inc. (ID: startup_1_dup)\n",
" Merging entities...\n",
"\n",
"Checking for data conflicts...\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2025-12-19 16:09:21,261 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Detecting all conflicts\n",
"2025-12-19 16:09:21,265 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Detecting value conflicts for property: founded\n",
"2025-12-19 16:09:21,268 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Analyzing 2 entities...\n",
"2025-12-19 16:09:21,271 - INFO - [COMPLETED] | Module: conflicts | Submodule: ConflictDetector | Message: Detected 0 conflicts\n",
"2025-12-19 16:09:21,274 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Detecting value conflicts for property: revenue\n",
"2025-12-19 16:09:21,277 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Analyzing 2 entities...\n",
"2025-12-19 16:09:21,279 - WARNING - Value conflict detected: startup_1.revenue has conflicting values: ['1000000', '1200000']\n",
"2025-12-19 16:09:21,280 - INFO - [COMPLETED] | Module: conflicts | Submodule: ConflictDetector | Message: Detected 1 conflicts\n",
"2025-12-19 16:09:21,283 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Detecting value conflicts for property: name\n",
"2025-12-19 16:09:21,286 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Analyzing 2 entities...\n",
"2025-12-19 16:09:21,288 - WARNING - Value conflict detected: startup_1.name has conflicting values: ['TechFlow Inc.', 'TechFlow AI']\n",
"2025-12-19 16:09:21,291 - INFO - [COMPLETED] | Module: conflicts | Submodule: ConflictDetector | Message: Detected 1 conflicts\n",
"2025-12-19 16:09:21,295 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Detecting type conflicts\n",
"2025-12-19 16:09:21,296 - INFO - [COMPLETED] | Module: conflicts | Submodule: ConflictDetector | Message: Detected 0 type conflicts\n",
"2025-12-19 16:09:21,299 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Detecting temporal conflicts\n",
"2025-12-19 16:09:21,302 - INFO - [COMPLETED] | Module: conflicts | Submodule: ConflictDetector | Message: Detected 0 temporal conflicts\n",
"2025-12-19 16:09:21,308 - INFO - [RUNNING] | Module: conflicts | Submodule: ConflictDetector | Message: Detecting logical conflicts\n",
"2025-12-19 16:09:21,313 - INFO - [COMPLETED] | Module: conflicts | Submodule: ConflictDetector | Message: Detected 0 logical conflicts\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" Conflict detected in field 'revenue':\n",
" Values: [1000000, 1200000]\n",
" Resolved to: 1200000.0\n",
" Conflict detected in field 'name':\n",
" Values: ['TechFlow AI', 'TechFlow Inc.']\n",
"\n",
"Cleaned Data: 5 entities remaining.\n"
]
}
],
"outputs": [],
"source": [
"# 1. Detect Duplicates\n",
"print(\"Scanning for duplicates...\")\n",
@@ -435,17 +225,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Knowledge Graph Assembled Successfully!\n"
]
}
],
"outputs": [],
"source": [
"# Manual Graph Construction (since we already cleaned it)\n",
"kg = {\n",
@@ -470,76 +252,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2025-12-19 16:09:52,395 - INFO - Centrality calculator initialized\n",
"2025-12-19 16:09:52,396 - INFO - Graph analyzer initialized (temporal: True)\n",
"2025-12-19 16:09:52,398 - INFO - Analyzing graph connectivity\n",
"2025-12-19 16:09:52,398 - INFO - Finding connected components\n",
"2025-12-19 16:09:52,399 - INFO - Calculating connectivity metrics\n",
"2025-12-19 16:09:52,399 - INFO - Calculating all centrality measures\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"--- Connectivity Analysis ---\n",
" • Graph Connected? Yes\n",
" • Connected Components: 1\n",
"\n",
"--- Centrality Analysis ---\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2025-12-19 16:09:52,403 - INFO - [RUNNING] | Module: kg | Submodule: CentralityCalculator | Message: Calculating degree centrality\n",
"2025-12-19 16:09:52,403 - INFO - Calculating degree centrality\n",
"2025-12-19 16:09:52,406 - INFO - [RUNNING] | Module: kg | Submodule: CentralityCalculator | Message: Processing graph structure...\n",
"2025-12-19 16:09:52,409 - INFO - [COMPLETED] | Module: kg | Submodule: CentralityCalculator | Message: Calculated degree centrality for 4 nodes\n",
"2025-12-19 16:09:52,410 - INFO - Detecting communities using louvain algorithm\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" • Top Influencers (Degree Centrality):\n",
" - founder_1: 0.67\n",
" - startup_1: 0.67\n",
" - startup_2: 0.33\n",
"\n",
"--- Community Detection ---\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2025-12-19 16:09:52,413 - INFO - [RUNNING] | Module: kg | Submodule: CommunityDetector | Message: Detecting communities using Louvain algorithm\n",
"2025-12-19 16:09:52,414 - INFO - Detecting communities using Louvain algorithm\n",
"2025-12-19 16:09:52,417 - INFO - [RUNNING] | Module: kg | Submodule: CommunityDetector | Message: Detecting communities with NetworkX...\n",
"2025-12-19 16:09:52,422 - INFO - [COMPLETED] | Module: kg | Submodule: CommunityDetector | Message: Detected 2 communities\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" • Detected 2 communities.\n",
" Community 1: investor_1, startup_1\n",
" Community 2: startup_2, founder_1\n"
]
}
],
"outputs": [],
"source": [
"# Initialize the Master Analyzer\n",
"analyzer = GraphAnalyzer(enable_temporal=True)\n",
@@ -585,33 +300,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2025-12-19 16:09:58,419 - INFO - Querying graph at time: 2020-06-01\n",
"2025-12-19 16:09:58,420 - INFO - Querying graph at time: 2023-07-01\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"--- Time Travel: 2020 ---\n",
" Active Relationships in 2020: 1\n",
" - founder_1 --[ADVISED]--> startup_2\n",
"\n",
"--- Time Travel: 2023 ---\n",
" Active Relationships in 2023: 2\n",
" - founder_1 --[FOUNDED]--> startup_1\n",
" - investor_1 --[INVESTED_IN]--> startup_1\n"
]
}
],
"outputs": [],
"source": [
"temporal_engine = TemporalGraphQuery(temporal_granularity=\"year\")\n",
"\n",
@@ -643,23 +334,9 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"--- Provenance Report: TechFlow AI ---\n",
" Entity: startup_1\n",
" First Seen: 2025-12-19T16:13:07.782408\n",
" Sources:\n",
" - Crunchbase_API_v2 (at 2025-12-19T16:13:07.782408)\n",
" - Manual_Entry_User_Bob (at 2025-12-19T16:13:07.782408)\n"
]
}
],
"outputs": [],
"source": [
"tracker = ProvenanceTracker()\n",
"\n",
@@ -711,9 +388,7 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
},
"nbformat": 4,
"nbformat_minor": 4
}
},
"nbformat": 4,
"nbformat_minor": 5
File diff suppressed because it is too large Load Diff
@@ -602,14 +602,14 @@
" \"exports/report.txt\"\n",
"]\n",
"\n",
"print(\"\ud83d\udcca Export Summary:\")\n",
"print(\"📊 Export Summary:\")\n",
"print(\"=\" * 60)\n",
"for file in export_files:\n",
" if os.path.exists(file):\n",
" size = os.path.getsize(file)\n",
" print(f\"\u2705 {file:50} ({size:>10,} bytes)\")\n",
" print(f\" {file:50} ({size:>10,} bytes)\")\n",
" else:\n",
" print(f\"\u274c {file:50} (not found)\")\n",
" print(f\" {file:50} (not found)\")\n",
"\n",
"print(\"=\" * 60)\n",
"print(f\"Total files checked: {len(export_files)}\")\n",
@@ -623,10 +623,15 @@
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
}
File diff suppressed because it is too large Load Diff
@@ -30,32 +30,16 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install -qU semantica\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -72,22 +56,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>GraphBuilder</td><td>-</td><td>0.31s</td></tr><tr><td>🔄</td><td>Semantica is building</td><td>🧠 kg</td><td>EntityResolver</td><td>-</td><td>2.16s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>EntityMerger</td><td>-</td><td>0.05s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>MergeStrategyManager</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>Reasoner</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is reasoning</td><td>🤔 reasoning</td><td>ExplanationGenerator</td><td>-</td><td>0.01s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"outputs": [],
"source": [
"builder = GraphBuilder()\n",
"\n",
@@ -118,7 +89,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -144,19 +115,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Inferred 2 new facts:\n",
" - grandparent_of(alice, charlie)\n",
" - lives_in(alice, california)\n"
]
}
],
"outputs": [],
"source": [
"# Perform forward chaining to derive new facts\n",
"# The Reasoner can infer facts directly from the knowledge graph or a list of facts\n",
@@ -176,17 +137,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Goal 'grandparent_of(alice, charlie)' proven successfully!\n"
]
}
],
"outputs": [],
"source": [
"# Define a goal to prove\n",
"goal = \"grandparent_of(alice, charlie)\"\n",
@@ -209,18 +162,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Explanation for backward chaining proof:\n",
"Given the premises: parent_of(alice, bob), parent_of(bob, charlie), we conclude: grandparent_of(alice, charlie).\n"
]
}
],
"outputs": [],
"source": [
"generator = ExplanationGenerator()\n",
"\n",
@@ -265,7 +209,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -292,18 +236,9 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"4\n",
"4\n"
]
}
],
"outputs": [],
"source": [
"entities = [\n",
" {\"id\": \"alice\", \"type\": \"Person\", \"name\": \"Alice\"},\n",
@@ -327,7 +262,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -341,19 +276,9 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2\n",
"grandparent_of(alice, charlie)\n",
"lives_in(alice, california)\n"
]
}
],
"outputs": [],
"source": [
"for rel in relationships:\n",
" fact = f\"{rel['type']}({rel['source']}, {rel['target']})\"\n",
@@ -367,20 +292,9 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"grandparent_of(alice, charlie)\n",
"True\n",
"lives_in(alice, california)\n",
"True\n"
]
}
],
"outputs": [],
"source": [
"goals = [\n",
" \"grandparent_of(alice, charlie)\",\n",
@@ -394,18 +308,9 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Given the premises: parent_of(alice, bob), parent_of(bob, charlie), we conclude: grandparent_of(alice, charlie) using rule 'Rule 1'.\n",
"Given the premises: , we conclude: grandparent_of(alice, charlie).\n"
]
}
],
"outputs": [],
"source": [
"if derived:\n",
" exp = explainer.generate_explanation(derived[0])\n",
@@ -30,32 +30,16 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install -qU semantica\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -74,22 +58,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>GraphBuilder</td><td>-</td><td>0.13s</td></tr><tr><td>🔄</td><td>Semantica is building</td><td>🧠 kg</td><td>EntityResolver</td><td>-</td><td>7.37s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.09s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>OntologyGenerator</td><td>-</td><td>0.06s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>ClassInferrer</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>PropertyGenerator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is exporting</td><td>💾 export</td><td>RDFExporter</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is storing</td><td>🗄️ triplet_store</td><td>BulkLoader</td><td>-</td><td>3.01s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"outputs": [],
"source": [
"builder = GraphBuilder()\n",
"\n",
@@ -119,7 +90,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -136,7 +107,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -188,17 +159,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Exported knowledge graph to knowledge_graph.ttl\n"
]
}
],
"outputs": [],
"source": [
"exporter = RDFExporter()\n",
"# Export Knowledge Graph\n",
@@ -42,53 +42,20 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": null,
"id": "88491af5",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"C:\\Users\\Mohd Kaif\\AppData\\Roaming\\Python\\Python311\\site-packages\\IPython\\utils\\_process_win32.py:124: ResourceWarning: unclosed file <_io.BufferedWriter name=3>\n",
" return process_handler(cmd, _system_body)\n",
"ResourceWarning: Enable tracemalloc to get the object allocation traceback\n",
"C:\\Users\\Mohd Kaif\\AppData\\Roaming\\Python\\Python311\\site-packages\\IPython\\utils\\_process_win32.py:124: ResourceWarning: unclosed file <_io.BufferedReader name=4>\n",
" return process_handler(cmd, _system_body)\n",
"ResourceWarning: Enable tracemalloc to get the object allocation traceback\n",
"C:\\Users\\Mohd Kaif\\AppData\\Roaming\\Python\\Python311\\site-packages\\IPython\\utils\\_process_win32.py:124: ResourceWarning: unclosed file <_io.BufferedReader name=5>\n",
" return process_handler(cmd, _system_body)\n",
"ResourceWarning: Enable tracemalloc to get the object allocation traceback\n"
]
}
],
"outputs": [],
"source": [
"!pip install -qU semantica "
]
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": null,
"id": "d6401d91",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Libraries imported successfully.\n"
]
}
],
"outputs": [],
"source": [
"import sys\n",
"import os\n",
@@ -129,25 +96,10 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": null,
"id": "812158a5",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"VectorStore initialized (Backend: FAISS)\n"
]
}
],
"outputs": [],
"source": [
"try:\n",
" # Initialize FAISS Vector Store\n",
@@ -173,28 +125,10 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": null,
"id": "7c2aa896",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Connectivity check failed: Couldn't connect to localhost:7687 (resolved to ('[::1]:7687', '127.0.0.1:7687')):\n",
"Failed to establish connection to ResolvedIPv6Address(('::1', 7687, 0, 0)) (reason [WinError 10061] No connection could be made because the target machine actively refused it)\n",
"Failed to establish connection to ResolvedIPv4Address(('127.0.0.1', 7687)) (reason [WinError 10061] No connection could be made because the target machine actively refused it)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"GraphStore Connection Failed: Failed to connect to Neo4j: Could not verify connectivity to Neo4j\n",
" Switching to in-memory ContextGraph (Non-persistent fallback)\n"
]
}
],
"outputs": [],
"source": [
"try:\n",
" # Initialize Neo4j Graph Store\n",
@@ -243,18 +177,10 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": null,
"id": "f3b2eff6",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Agent Context is online and ready.\n"
]
}
],
"outputs": [],
"source": [
"if vs:\n",
" context = AgentContext(\n",
@@ -293,18 +219,10 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": null,
"id": "71adf433",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Memory Stored: mem_2ec8206c1601\n"
]
}
],
"outputs": [],
"source": [
"user_id = \"user_123\"\n",
"session_id = \"session_alpha\"\n",
@@ -332,18 +250,10 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": null,
"id": "3a98ca53",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Knowledge Ingestion Stats: {'stored_count': 2, 'memory_ids': ['mem_4e6d93ce75a2', 'mem_50eecb14c339'], 'graph_nodes': 2, 'graph_edges': 0}\n"
]
}
],
"outputs": [],
"source": [
"documents = [\n",
" {\n",
@@ -393,27 +303,10 @@
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": null,
"id": "970b605c",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\neo4j\\_sync\\driver.py:515: ResourceWarning: unclosed BoltDriver: <neo4j._sync.driver.BoltDriver object at 0x00000221C7B9FB90>.\n",
" _unclosed_resource_warn(self)\n",
"ResourceWarning: Enable tracemalloc to get the object allocation traceback\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Manual Graph Build Complete: {'node_count': 6, 'edge_count': 0}\n"
]
}
],
"outputs": [],
"source": [
"# 1. Define Nodes\n",
"entities = [\n",
@@ -450,19 +343,10 @@
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": null,
"id": "dc61c324",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Neighbors of 'project_apollo':\n"
]
}
],
"outputs": [],
"source": [
"# Helper to print graph neighbors\n",
"def inspect_node(node_id):\n",
@@ -508,41 +392,10 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": null,
"id": "69381e8c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Asking: 'Who is responsible for the Python web framework project?'...\n",
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"c:\\Users\\Mohd Kaif\\semantica\\semantica\\vector_store\\vector_store.py:480: RuntimeWarning: invalid value encountered in divide\n",
" similarities = np.dot(vectors, query_vector) / (vector_norms * query_norm)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Retrieved 3 context items:\n",
"\n",
"1. [Score: 0.54] Project Apollo is a next-gen web framework designed for high scalability....\n",
"\n",
"2. [Score: 0.32] Python 3.12 introduces significant performance improvements for async workloads....\n",
"\n",
"3. [Score: 0.21] I am working on a new project called 'Project Apollo' which uses Python and React....\n",
"\n"
]
}
],
"outputs": [],
"source": [
"query = \"Who is responsible for the Python web framework project?\"\n",
"print(f\"Asking: '{query}'...\\n\")\n",
@@ -590,18 +443,10 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": null,
"id": "0574b1b7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chat History for session_alpha:\n"
]
}
],
"outputs": [],
"source": [
"# Get recent chat history for context window\n",
"history = context.conversation(\n",
@@ -624,21 +469,10 @@
},
{
"cell_type": "code",
"execution_count": 26,
"execution_count": null,
"id": "cd472495",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"System Vital Signs:\n",
" - Total Memories: 3\n",
" - Graph Nodes: N/A\n",
" - Graph Edges: N/A\n"
]
}
],
"outputs": [],
"source": [
"stats = context.stats()\n",
"print(\"System Vital Signs:\")\n",
File diff suppressed because it is too large Load Diff
@@ -24,297 +24,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
"Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.3)\n",
"Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
"Requirement already satisfied: umap-learn>=0.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.5.9.post2)\n",
"Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
"Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
"Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
"Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
"Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
"Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.6.1)\n",
"Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
"Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
"Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
"Requirement already satisfied: ipywidgets>=8.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.1.1)\n",
"Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
"Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.14.3)\n",
"Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
"Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
"Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
"Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
"Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
"Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
"Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
"Requirement already satisfied: fastembed>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: onnxruntime>=1.17.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.22.1)\n",
"Requirement already satisfied: tokenizers>=0.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.4)\n",
"Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
"Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
"Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
"Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
"Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
"Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
"Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
"Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
"Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
"Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
"Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
"Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
"Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
"Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
"Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
"Requirement already satisfied: pydantic>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
"Requirement already satisfied: fastmcp>=0.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.14.1)\n",
"Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
"Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (14.2.0)\n",
"Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
"Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
"Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
"Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
"Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
"Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.23.1)\n",
"Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.39.1)\n",
"Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.39.1)\n",
"Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.60b1)\n",
"Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
"Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
"Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
"Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
"Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
"Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
"Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
"Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
"Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
"Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
"Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
"Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
"Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.15.0)\n",
"Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
"Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
"Requirement already satisfied: soupsieve>=1.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
"Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
"Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
"Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
"Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (4.5.1)\n",
"Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
"Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
"Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
"Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
"Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
"Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
"Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
"Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
"Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
"Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
"Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
"Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (2.41.4)\n",
"Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (0.4.2)\n",
"Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
"Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
"Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
"Requirement already satisfied: huggingface-hub<1.0,>=0.20 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (0.30.2)\n",
"Requirement already satisfied: mmh3<6.0.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (5.2.0)\n",
"Requirement already satisfied: py-rust-stemmers<0.2.0,>=0.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (0.1.5)\n",
"Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub<1.0,>=0.20->fastembed>=0.2.0->semantica) (3.16.1)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub<1.0,>=0.20->fastembed>=0.2.0->semantica) (2023.10.0)\n",
"Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
"Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
"Requirement already satisfied: authlib>=1.6.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.6.6)\n",
"Requirement already satisfied: cyclopts>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (4.4.0)\n",
"Requirement already satisfied: exceptiongroup>=1.2.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.3.0)\n",
"Requirement already satisfied: httpx>=0.28.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.28.1)\n",
"Requirement already satisfied: jsonschema-path>=0.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.3.4)\n",
"Requirement already satisfied: mcp>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.25.0)\n",
"Requirement already satisfied: openapi-pydantic>=0.5.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.5.1)\n",
"Requirement already satisfied: py-key-value-aio<0.4.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.3.0)\n",
"Requirement already satisfied: pydocket>=0.15.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.16.0)\n",
"Requirement already satisfied: pyperclip>=1.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.9.0)\n",
"Requirement already satisfied: websockets>=15.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (15.0.1)\n",
"Requirement already satisfied: py-key-value-shared==0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio<0.4.0,>=0.3.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.3.0)\n",
"Requirement already satisfied: beartype>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio<0.4.0,>=0.3.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.22.9)\n",
"Requirement already satisfied: diskcache>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (5.6.3)\n",
"Requirement already satisfied: pathvalidate>=3.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.3.1)\n",
"Requirement already satisfied: keyring>=25.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (25.6.0)\n",
"Requirement already satisfied: cachetools>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (5.5.2)\n",
"Requirement already satisfied: attrs>=23.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (24.2.0)\n",
"Requirement already satisfied: docstring-parser<4.0,>=0.15 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (0.16)\n",
"Requirement already satisfied: rich-rst<2.0.0,>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (1.3.2)\n",
"Requirement already satisfied: docutils in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich-rst<2.0.0,>=1.3.1->cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (0.21.2)\n",
"Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
"Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
"Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
"Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
"Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
"Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
"Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.28.1->fastmcp>=0.1.0->semantica) (1.0.9)\n",
"Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.28.1->fastmcp>=0.1.0->semantica) (0.16.0)\n",
"Requirement already satisfied: comm>=0.1.3 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (0.2.0)\n",
"Requirement already satisfied: ipython>=6.1.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (8.17.2)\n",
"Requirement already satisfied: traitlets>=4.3.1 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (5.13.0)\n",
"Requirement already satisfied: widgetsnbextension~=4.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipywidgets>=8.0.0->semantica) (4.0.9)\n",
"Requirement already satisfied: jupyterlab-widgets~=3.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipywidgets>=8.0.0->semantica) (3.0.9)\n",
"Requirement already satisfied: decorator in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (5.1.1)\n",
"Requirement already satisfied: jedi>=0.16 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.19.1)\n",
"Requirement already satisfied: matplotlib-inline in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.1.6)\n",
"Requirement already satisfied: pygments>=2.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.19.2)\n",
"Requirement already satisfied: stack-data in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.6.3)\n",
"Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
"Requirement already satisfied: parso<0.9.0,>=0.8.3 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.8.3)\n",
"Requirement already satisfied: pathable<0.5.0,>=0.4.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.4.4)\n",
"Requirement already satisfied: referencing<0.37.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.30.2)\n",
"Requirement already satisfied: rpds-py>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from referencing<0.37.0->jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.12.0)\n",
"Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.2.3)\n",
"Requirement already satisfied: importlib_metadata>=4.11.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (6.8.0)\n",
"Requirement already satisfied: jaraco.classes in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.4.0)\n",
"Requirement already satisfied: jaraco.functools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (4.1.0)\n",
"Requirement already satisfied: jaraco.context in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (6.0.1)\n",
"Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib_metadata>=4.11.4->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.17.0)\n",
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
"Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
"Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
"Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
"Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
"Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
"Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
"Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
"Requirement already satisfied: httpx-sse>=0.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (0.4.0)\n",
"Requirement already satisfied: jsonschema>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (4.23.0)\n",
"Requirement already satisfied: pydantic-settings>=2.5.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.11.0)\n",
"Requirement already satisfied: pyjwt>=2.10.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pyjwt[crypto]>=2.10.1->mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.10.1)\n",
"Requirement already satisfied: python-multipart>=0.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (0.0.18)\n",
"Requirement already satisfied: pywin32>=310 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (311)\n",
"Requirement already satisfied: sse-starlette>=1.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.3.6)\n",
"Requirement already satisfied: jsonschema-specifications>=2023.03.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema>=4.20.0->mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2023.7.1)\n",
"Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
"Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
"Requirement already satisfied: coloredlogs in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (15.0.1)\n",
"Requirement already satisfied: flatbuffers in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (23.5.26)\n",
"Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (1.13.3)\n",
"Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
"Requirement already satisfied: opentelemetry-semantic-conventions==0.60b1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.60b1)\n",
"Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
"Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
"Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
"Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
"Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.35.4)\n",
"Requirement already satisfied: email-validator>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic[email]>=2.11.7->fastmcp>=0.1.0->semantica) (2.2.0)\n",
"Requirement already satisfied: dnspython>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from email-validator>=2.0.0->pydantic[email]>=2.11.7->fastmcp>=0.1.0->semantica) (2.4.2)\n",
"Requirement already satisfied: cloudpickle>=3.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (3.1.2)\n",
"Requirement already satisfied: fakeredis>=2.32.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.33.0)\n",
"Requirement already satisfied: opentelemetry-exporter-prometheus>=0.60b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (0.60b1)\n",
"Requirement already satisfied: python-json-logger>=2.0.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.0.7)\n",
"Requirement already satisfied: typer>=0.15.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (0.15.2)\n",
"Requirement already satisfied: sortedcontainers>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis>=2.32.1->fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.4.0)\n",
"Requirement already satisfied: lupa>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.6)\n",
"Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-instrumentation->semantica) (1.17.2)\n",
"Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
"Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
"Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
"Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
"Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
"Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
"Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
"Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
"Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
"Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
"Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
"Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
"Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
"Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
"Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
"Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
"Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
"Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
"Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
"Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
"Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
"Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
"Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
"Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
"Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
"Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
"Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
"Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
"Requirement already satisfied: shellingham>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from typer>=0.15.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (1.5.4)\n",
"Requirement already satisfied: pynndescent>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from umap-learn>=0.5.0->semantica) (0.5.13)\n",
"Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
"Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
"Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
"Requirement already satisfied: humanfriendly>=9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coloredlogs->onnxruntime>=1.17.0->semantica) (10.0)\n",
"Requirement already satisfied: pyreadline3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from humanfriendly>=9.1->coloredlogs->onnxruntime>=1.17.0->semantica) (3.4.1)\n",
"Requirement already satisfied: more-itertools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jaraco.classes->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (10.6.0)\n",
"Requirement already satisfied: backports.tarfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jaraco.context->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (1.2.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
"Requirement already satisfied: executing>=1.2.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.0.1)\n",
"Requirement already satisfied: asttokens>=2.1.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.4.1)\n",
"Requirement already satisfied: pure-eval in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.2.2)\n",
"Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->onnxruntime>=1.17.0->semantica) (1.3.0)\n"
]
}
],
"outputs": [],
"source": [
"!pip install semantica\n"
]
@@ -330,25 +42,9 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Selected model: BAAI/bge-small-en-v1.5\n",
"Embedding dimension: 128\n"
]
}
],
"outputs": [],
"source": [
"from semantica.embeddings import TextEmbedder\n",
"\n",
@@ -379,37 +75,9 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedWriter name=3>\n",
" del self._target, self._args, self._kwargs\n",
"ResourceWarning: Enable tracemalloc to get the object allocation traceback\n",
"c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedReader name=4>\n",
" del self._target, self._args, self._kwargs\n",
"ResourceWarning: Enable tracemalloc to get the object allocation traceback\n",
"c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedReader name=5>\n",
" del self._target, self._args, self._kwargs\n",
"ResourceWarning: Enable tracemalloc to get the object allocation traceback\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 5 most similar documents:\n",
" 1. Document doc_1935 (distance: 110.917)\n",
" 2. Document doc_3860 (distance: 111.535)\n",
" 3. Document doc_277 (distance: 113.270)\n",
" 4. Document doc_1903 (distance: 113.371)\n",
" 5. Document doc_2959 (distance: 113.612)\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import FAISSStore\n",
"import numpy as np\n",
@@ -449,29 +117,9 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Technology articles from 2024:\n",
" - AI in Healthcare\n",
" - Business Strategy\n",
" - Data Science Guide\n",
" - Marketing Tips\n",
" - Machine Learning Basics\n",
"\n",
"All articles from 2024:\n",
" - AI in Healthcare (Technology)\n",
" - Business Strategy (Business)\n",
" - Data Science Guide (Technology)\n",
" - Marketing Tips (Business)\n",
" - Machine Learning Basics (Technology)\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import HybridSearch, MetadataFilter\n",
"import numpy as np\n",
@@ -529,25 +177,9 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Combined results (fair ranking):\n",
" 1. Data Science Guide (score: 0.033)\n",
" 2. Machine Learning Basics (score: 0.016)\n",
" 3. AI in Healthcare (score: 0.016)\n",
"\n",
"Combined results (prefer recent):\n",
" 1. Data Science Guide (score: 0.929)\n",
" 2. AI in Healthcare (score: 0.630)\n",
" 3. Business Strategy (score: 0.595)\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import SearchRanker\n",
"\n",
@@ -605,21 +237,9 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Company A has 10 documents\n",
"Company B has 15 documents\n",
"\n",
"Admin can delete: True\n",
"User can delete: False\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import NamespaceManager\n",
"\n",
File diff suppressed because it is too large Load Diff
+4 -226
View File
@@ -44,223 +44,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)\n",
"Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
"Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.1.4)\n",
"Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
"Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
"Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
"Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
"Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
"Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
"Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.5)\n",
"Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
"Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
"Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
"Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
"Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.12.3)\n",
"Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
"Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
"Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
"Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
"Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
"Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
"Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
"Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
"Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
"Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
"Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
"Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
"Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
"Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
"Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
"Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
"Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
"Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
"Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
"Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
"Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
"Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
"Requirement already satisfied: pydantic>=1.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
"Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
"Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (13.7.1)\n",
"Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
"Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
"Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
"Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
"Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
"Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.18.0)\n",
"Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.59b0)\n",
"Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
"Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
"Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
"Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
"Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
"Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
"Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
"Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
"Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
"Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
"Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
"Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
"Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1)\n",
"Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
"Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
"Requirement already satisfied: soupsieve>1.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
"Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
"Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
"Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
"Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (3.11.0)\n",
"Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
"Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
"Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
"Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
"Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
"Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
"Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
"Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
"Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
"Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
"Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
"Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (2.41.4)\n",
"Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.4.2)\n",
"Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
"Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
"Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
"Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
"Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
"Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
"Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
"Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
"Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
"Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
"Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
"Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
"Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.9.0->semantica) (5.1.1)\n",
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
"Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
"Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
"Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
"Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
"Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
"Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
"Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
"Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
"Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
"Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0)\n",
"Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0)\n",
"Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0)\n",
"Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
"Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
"Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
"Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
"Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6)\n",
"Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pymongo>=4.2.0->semantica) (2.4.2)\n",
"Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
"Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
"Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
"Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
"Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
"Requirement already satisfied: httpx>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1)\n",
"Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
"Requirement already satisfied: pywin32>=226 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311)\n",
"Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
"Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9)\n",
"Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0)\n",
"Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
"Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
"Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (2.19.2)\n",
"Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
"Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
"Requirement already satisfied: huggingface-hub>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2)\n",
"Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (3.16.1)\n",
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
"Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.21.4)\n",
"Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0)\n",
"Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
"Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
"Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
"Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
"Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
"Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
"Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
"Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
"Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
"Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
"Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
"Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
"Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
"Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
"Requirement already satisfied: wrapt in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2)\n",
"Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
"Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from torch>=1.12.0->semantica) (1.13.3)\n",
"Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
"Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
"Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0)\n",
"Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
"Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install semantica\n"
]
@@ -276,17 +62,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detected Type: txt\n"
]
}
],
"outputs": [],
"source": [
"import os\n",
"import tempfile\n",
@@ -260,6 +260,11 @@
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python"
}
@@ -204,8 +204,8 @@
"\n",
"text_samples = [\n",
" \"Hello, this is English text.\",\n",
" \"Bonjour, ceci est du texte fran\u00e7ais.\",\n",
" \"Hola, este es texto en espa\u00f1ol.\"\n",
" \"Bonjour, ceci est du texte français.\",\n",
" \"Hola, este es texto en español.\"\n",
"]\n",
"\n",
"for text in text_samples:\n",
@@ -238,10 +238,15 @@
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
}
+18 -513
View File
@@ -54,223 +54,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)\n",
"Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
"Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.1.4)\n",
"Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
"Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
"Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
"Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
"Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
"Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
"Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.5)\n",
"Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
"Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
"Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
"Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
"Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.12.3)\n",
"Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
"Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
"Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
"Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
"Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
"Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
"Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
"Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
"Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
"Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
"Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
"Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
"Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
"Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
"Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
"Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
"Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
"Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
"Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
"Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
"Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
"Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
"Requirement already satisfied: pydantic>=1.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
"Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
"Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (13.7.1)\n",
"Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
"Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
"Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
"Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
"Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
"Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.18.0)\n",
"Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.59b0)\n",
"Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
"Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
"Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
"Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
"Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
"Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
"Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
"Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
"Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
"Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
"Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
"Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
"Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1)\n",
"Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
"Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
"Requirement already satisfied: soupsieve>1.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
"Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
"Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
"Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
"Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (3.11.0)\n",
"Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
"Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
"Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
"Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
"Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
"Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
"Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
"Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
"Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
"Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
"Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
"Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (2.41.4)\n",
"Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.4.2)\n",
"Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
"Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
"Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
"Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
"Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
"Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
"Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
"Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
"Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
"Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
"Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
"Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
"Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.9.0->semantica) (5.1.1)\n",
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
"Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
"Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
"Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
"Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
"Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
"Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
"Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
"Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
"Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
"Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0)\n",
"Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0)\n",
"Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0)\n",
"Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
"Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
"Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
"Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
"Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6)\n",
"Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pymongo>=4.2.0->semantica) (2.4.2)\n",
"Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
"Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
"Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
"Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
"Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
"Requirement already satisfied: httpx>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1)\n",
"Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
"Requirement already satisfied: pywin32>=226 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311)\n",
"Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
"Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9)\n",
"Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0)\n",
"Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
"Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
"Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (2.19.2)\n",
"Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
"Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
"Requirement already satisfied: huggingface-hub>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2)\n",
"Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (3.16.1)\n",
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
"Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.21.4)\n",
"Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0)\n",
"Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
"Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
"Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
"Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
"Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
"Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
"Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
"Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
"Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
"Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
"Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
"Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
"Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
"Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
"Requirement already satisfied: wrapt in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2)\n",
"Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
"Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from torch>=1.12.0->semantica) (1.13.3)\n",
"Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
"Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
"Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0)\n",
"Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
"Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install semantica\n"
]
@@ -294,45 +80,9 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>NERExtractor</td><td>-</td><td>1.08s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>NamedEntityRecognizer</td><td>-</td><td>2.50s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" Extracted 13 entities:\n",
"\n",
"--------------------------------------------------------------------------------\n",
" 1. Apple Inc. | Type: ORG | Confidence: 1.00\n",
" 2. Steve Jobs | Type: PERSON | Confidence: 1.00\n",
" 3. Steve Wozniak | Type: PERSON | Confidence: 1.00\n",
" 4. Ronald Wayne | Type: PERSON | Confidence: 1.00\n",
" 5. Cupertino | Type: GPE | Confidence: 1.00\n",
" 6. California | Type: GPE | Confidence: 1.00\n",
" 7. April 1, 1976 | Type: DATE | Confidence: 1.00\n",
" 8. Tim Cook | Type: PERSON | Confidence: 1.00\n",
" 9. Steve Jobs | Type: PERSON | Confidence: 1.00\n",
"10. August 2011 | Type: DATE | Confidence: 1.00\n",
"11. Apple | Type: ORG | Confidence: 1.00\n",
"12. One | Type: CARDINAL | Confidence: 1.00\n",
"13. Cupertino | Type: GPE | Confidence: 1.00\n",
"--------------------------------------------------------------------------------\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import NERExtractor\n",
"\n",
@@ -403,43 +153,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
" Entity Visualization:\n",
"\n",
"================================================================================\n",
"\n",
"CARDINAL:\n",
" • One\n",
"\n",
"DATE:\n",
" • August 2011\n",
" • April 1, 1976\n",
"\n",
"GPE:\n",
" • California\n",
" • Cupertino\n",
"\n",
"ORG:\n",
" • Apple\n",
" • Apple Inc.\n",
"\n",
"PERSON:\n",
" • Steve Wozniak\n",
" • Ronald Wayne\n",
" • Steve Jobs\n",
" • Tim Cook\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"def highlight_entities(text, entities):\n",
" \"\"\"\n",
@@ -493,47 +209,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Comparing Extraction Methods:\n",
"\n",
"================================================================================\n",
"\n",
" Method: PATTERN\n",
"----------------------------------------\n",
"Found 4 entities:\n",
" • Apple Inc (PERSON)\n",
" • Steve Jobs (PERSON)\n",
" • Apple Inc (ORG)\n",
" • 1976 (DATE)\n",
"\n",
" Method: REGEX\n",
"----------------------------------------\n",
"Found 8 entities:\n",
" • Apple Inc (PERSON)\n",
" • was founded by Steve Jobs in Cupertino (PERSON)\n",
" • California in (PERSON)\n",
" • Apple Inc (ORG)\n",
" • Apple Inc (GPE)\n",
"\n",
" Method: ML\n",
"----------------------------------------\n",
"Found 5 entities:\n",
" • Apple Inc. (ORG)\n",
" • Steve Jobs (PERSON)\n",
" • Cupertino (GPE)\n",
" • California (GPE)\n",
" • 1976 (DATE)\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import NERExtractor\n",
"\n",
@@ -583,49 +261,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Advanced Entity Recognition Results:\n",
"\n",
"================================================================================\n",
"\n",
" Text 1: Tim Cook is the CEO of Apple Inc., based in Cupertino....\n",
" Found 3 entities:\n",
" • Tim Cook | PERSON | Confidence: 1.00\n",
" • Apple Inc. | ORG | Confidence: 1.00\n",
" • Cupertino | GPE | Confidence: 1.00\n",
"\n",
" Text 2: Microsoft Corporation, founded by Bill Gates, is headquarter...\n",
" Found 4 entities:\n",
" • Microsoft Corporation | ORG | Confidence: 1.00\n",
" • Bill Gates | PERSON | Confidence: 1.00\n",
" • Redmond | GPE | Confidence: 1.00\n",
" • Washington | GPE | Confidence: 1.00\n",
"\n",
" Text 3: Amazon was founded by Jeff Bezos in Seattle in 1994....\n",
" Found 4 entities:\n",
" • Amazon | ORG | Confidence: 1.00\n",
" • Jeff Bezos | PERSON | Confidence: 1.00\n",
" • Seattle | GPE | Confidence: 1.00\n",
" • 1994 | DATE | Confidence: 1.00\n",
"\n",
" Text 4: Google was started by Larry Page and Sergey Brin at Stanford...\n",
" Found 4 entities:\n",
" • Google | ORG | Confidence: 1.00\n",
" • Larry Page | PERSON | Confidence: 1.00\n",
" • Sergey Brin | PERSON | Confidence: 1.00\n",
" • Stanford University | ORG | Confidence: 1.00\n",
"\n",
" Total entities extracted: 15\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import NamedEntityRecognizer\n",
"\n",
@@ -685,48 +323,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Entity Classification Results:\n",
"\n",
"================================================================================\n",
"\n",
"DATE (1 entities):\n",
"----------------------------------------\n",
" • 1994\n",
"\n",
"GPE (4 entities):\n",
"----------------------------------------\n",
" • Cupertino\n",
" • Redmond\n",
" • Seattle\n",
" • Washington\n",
"\n",
"ORG (5 entities):\n",
"----------------------------------------\n",
" • Amazon\n",
" • Apple Inc.\n",
" • Google\n",
" • Microsoft Corporation\n",
" • Stanford University\n",
"\n",
"PERSON (5 entities):\n",
"----------------------------------------\n",
" • Bill Gates\n",
" • Jeff Bezos\n",
" • Larry Page\n",
" • Sergey Brin\n",
" • Tim Cook\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import EntityClassifier\n",
"\n",
@@ -774,32 +373,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Entity Confidence Scoring:\n",
"\n",
"================================================================================\n",
" High Confidence (≥0.8): 15 entities\n",
" Medium Confidence (0.5-0.8): 0 entities\n",
" Low Confidence (<0.5): 0 entities\n",
"\n",
" Confidence Distribution:\n",
"----------------------------------------\n",
"\n",
"High Confidence Examples:\n",
" • Tim Cook (PERSON) - 1.00\n",
" • Apple Inc. (ORG) - 1.00\n",
" • Cupertino (GPE) - 1.00\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import EntityConfidenceScorer\n",
"\n",
@@ -865,34 +441,9 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Custom Entity Detection:\n",
"\n",
"================================================================================\n",
"\n",
"EMAIL:\n",
" • support@apple.com\n",
" • tech@apple.com\n",
"\n",
"PHONE:\n",
" • 800-692-7753\n",
"\n",
"PRODUCT_CODE:\n",
" • SKU-12345\n",
"\n",
"URL:\n",
" • https://store.apple.com.\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import CustomEntityDetector\n",
"import re\n",
@@ -948,55 +499,9 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Batch Processing Results:\n",
"\n",
"================================================================================\n",
"\n",
" Document 1:\n",
" Text: Apple Inc. released the iPhone 15 in September 202...\n",
" Entities: 2\n",
"\n",
" Document 2:\n",
" Text: Microsoft announced Azure AI updates at Build 2023...\n",
" Entities: 3\n",
"\n",
" Document 3:\n",
" Text: Google's Sundar Pichai spoke at I/O 2023 in Mounta...\n",
" Entities: 5\n",
"\n",
" Document 4:\n",
" Text: Tesla's Elon Musk unveiled the Cybertruck in Austi...\n",
" Entities: 5\n",
"\n",
" Document 5:\n",
" Text: Amazon Web Services launched new features in North...\n",
" Entities: 2\n",
"\n",
" Batch Processing Summary:\n",
"----------------------------------------\n",
"Documents processed: 5\n",
"Total entities: 17\n",
"Average per document: 3.4\n",
"\n",
" Entity Type Distribution:\n",
" ORG: 5\n",
" GPE: 5\n",
" DATE: 3\n",
" PERSON: 2\n",
" PRODUCT: 1\n",
" LOC: 1\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"# Sample document collection\n",
"documents = [\n",
@@ -23,7 +23,7 @@
"- Configure extraction parameters for optimal results\n",
"- Extract RDF triplets with `TripletExtractor`\n",
"- Serialize triplets to RDF formats with `RDFSerializer`\n",
"- Build complete entity \u2192 relation \u2192 triplet pipelines\n",
"- Build complete entity relation triplet pipelines\n",
"\n",
"### What You'll Learn\n",
"\n",
@@ -164,7 +164,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## \ufe0f Step 2: Different Extraction Methods\n",
"## Step 2: Different Extraction Methods\n",
"\n",
"Semantica supports multiple relation extraction methods:\n",
"\n",
@@ -172,11 +172,11 @@
"\n",
"| Method | Speed | Accuracy | Use Case | Requires |\n",
"|--------|-------|----------|----------|----------|\n",
"| **pattern** | | \u2b50\u2b50\u2b50 | Common relations | Nothing |\n",
"| **dependency** | | \u2b50\u2b50\u2b50\u2b50 | Grammatical relations | spaCy |\n",
"| **cooccurrence** | | \u2b50\u2b50 | Proximity-based | Nothing |\n",
"| **huggingface** | | \u2b50\u2b50\u2b50\u2b50\u2b50 | Domain-specific | HF model |\n",
"| **llm** | | \u2b50\u2b50\u2b50\u2b50\u2b50 | Complex, custom | API key |"
"| **pattern** | | ⭐⭐⭐ | Common relations | Nothing |\n",
"| **dependency** | | ⭐⭐⭐⭐ | Grammatical relations | spaCy |\n",
"| **cooccurrence** | | ⭐⭐ | Proximity-based | Nothing |\n",
"| **huggingface** | | ⭐⭐⭐⭐⭐ | Domain-specific | HF model |\n",
"| **llm** | | ⭐⭐⭐⭐⭐ | Complex, custom | API key |"
]
},
{
@@ -214,10 +214,10 @@
" source_text = source.get('text', str(source)) if isinstance(source, dict) else getattr(source, 'text', str(source))\n",
" target_text = target.get('text', str(target)) if isinstance(target, dict) else getattr(target, 'text', str(target))\n",
" \n",
" print(f\" \u2022 {source_text} --[{rel_type}]--> {target_text}\")\n",
" print(f\" {source_text} --[{rel_type}]--> {target_text}\")\n",
" \n",
" except Exception as e:\n",
" print(f\" \ufe0f Method '{method_name}' not available: {str(e)[:50]}\")\n",
" print(f\" Method '{method_name}' not available: {str(e)[:50]}\")\n",
"\n",
"print(\"\\n\" + \"=\" * 80)"
]
@@ -279,7 +279,7 @@
" source_text = source.get('text', str(source)) if isinstance(source, dict) else getattr(source, 'text', str(source))\n",
" target_text = target.get('text', str(target)) if isinstance(target, dict) else getattr(target, 'text', str(target))\n",
" \n",
" print(f\" \u2022 {source_text} --[{rel_type}]--> {target_text} (conf: {confidence:.2f})\")\n",
" print(f\" {source_text} --[{rel_type}]--> {target_text} (conf: {confidence:.2f})\")\n",
"\n",
"print(\"\\n\" + \"=\" * 80)"
]
@@ -288,7 +288,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## \ufe0f Step 4: Relation Classification\n",
"## Step 4: Relation Classification\n",
"\n",
"Group and classify extracted relations by their predicate type."
]
@@ -309,7 +309,7 @@
"# Classify relations\n",
"classified_relations = advanced_extractor.classify_relations(all_relations)\n",
"\n",
"print(\"\ufe0f Relation Classification:\\n\")\n",
"print(\" Relation Classification:\\n\")\n",
"print(\"=\" * 80)\n",
"\n",
"for rel_type, rel_list in sorted(classified_relations.items()):\n",
@@ -323,7 +323,7 @@
" source_text = source.get('text', str(source)) if isinstance(source, dict) else getattr(source, 'text', str(source))\n",
" target_text = target.get('text', str(target)) if isinstance(target, dict) else getattr(target, 'text', str(target))\n",
" \n",
" print(f\" \u2022 {source_text} \u2192 {target_text}\")\n",
" print(f\" {source_text} {target_text}\")\n",
"\n",
"print(\"\\n\" + \"=\" * 80)"
]
@@ -455,7 +455,7 @@
"source": [
"## Step 7: Complete Extraction Pipeline\n",
"\n",
"Let's build a complete pipeline: **Entities \u2192 Relations \u2192 Triplets**\n",
"Let's build a complete pipeline: **Entities Relations Triplets**\n",
"\n",
"This demonstrates the full workflow for knowledge graph construction."
]
@@ -534,7 +534,7 @@
"3. **Try co-occurrence** for exploratory analysis\n",
"4. **Consider LLM** for complex, domain-specific relations\n",
"\n",
"### \ufe0f Optimizing Extraction\n",
"### Optimizing Extraction\n",
"\n",
"- **Set confidence thresholds** (0.6-0.7 for production)\n",
"- **Specify relation_types** to focus extraction\n",
@@ -585,7 +585,7 @@
"2. **Multiple methods available**: Choose based on accuracy vs speed needs\n",
"3. **Configuration is powerful**: Tune parameters for your domain\n",
"4. **Triplets are standardized**: Use RDF for interoperability\n",
"5. **Pipelines are efficient**: Extract entities \u2192 relations \u2192 triplets in sequence\n",
"5. **Pipelines are efficient**: Extract entities relations triplets in sequence\n",
"\n",
"### Next Steps\n",
"\n",
@@ -624,4 +624,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
@@ -39,230 +39,16 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)\n",
"Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
"Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.1.4)\n",
"Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
"Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
"Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
"Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
"Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
"Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
"Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.5)\n",
"Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
"Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
"Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
"Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
"Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.12.3)\n",
"Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
"Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
"Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
"Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
"Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
"Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
"Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
"Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
"Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
"Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
"Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
"Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
"Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
"Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
"Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
"Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
"Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
"Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
"Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
"Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
"Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
"Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
"Requirement already satisfied: pydantic>=1.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
"Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
"Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (13.7.1)\n",
"Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
"Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
"Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
"Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
"Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
"Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.18.0)\n",
"Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.59b0)\n",
"Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
"Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
"Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
"Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
"Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
"Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
"Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
"Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
"Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
"Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
"Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
"Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
"Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1)\n",
"Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
"Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
"Requirement already satisfied: soupsieve>1.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
"Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
"Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
"Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
"Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (3.11.0)\n",
"Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
"Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
"Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
"Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
"Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
"Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
"Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
"Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
"Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
"Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
"Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
"Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (2.41.4)\n",
"Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.4.2)\n",
"Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
"Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
"Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
"Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
"Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
"Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
"Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
"Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
"Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
"Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
"Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
"Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
"Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.9.0->semantica) (5.1.1)\n",
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
"Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
"Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
"Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
"Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
"Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
"Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
"Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
"Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
"Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
"Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0)\n",
"Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0)\n",
"Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0)\n",
"Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
"Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
"Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
"Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
"Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6)\n",
"Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pymongo>=4.2.0->semantica) (2.4.2)\n",
"Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
"Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
"Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
"Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
"Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
"Requirement already satisfied: httpx>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1)\n",
"Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
"Requirement already satisfied: pywin32>=226 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311)\n",
"Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
"Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9)\n",
"Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0)\n",
"Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
"Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
"Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (2.19.2)\n",
"Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
"Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
"Requirement already satisfied: huggingface-hub>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2)\n",
"Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (3.16.1)\n",
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
"Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.21.4)\n",
"Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0)\n",
"Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
"Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
"Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
"Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
"Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
"Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
"Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
"Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
"Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
"Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
"Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
"Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
"Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
"Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
"Requirement already satisfied: wrapt in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2)\n",
"Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
"Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from torch>=1.12.0->semantica) (1.13.3)\n",
"Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
"Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
"Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0)\n",
"Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
"Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install semantica\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -313,18 +99,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Original entities: 5\n",
"Resolved entities: 4\n"
]
}
],
"outputs": [],
"source": [
"from semantica.kg import EntityResolver\n",
"\n",
@@ -347,18 +124,9 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Original entities: 4\n",
"Deduplicated entities: 0\n"
]
}
],
"outputs": [],
"source": [
"from semantica.deduplication import DuplicateDetector, EntityMerger, MergeStrategy\n",
"\n",
@@ -80,262 +80,18 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)\n",
"Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
"Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.1.4)\n",
"Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
"Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
"Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
"Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
"Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
"Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
"Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.5)\n",
"Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
"Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
"Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
"Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
"Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.12.3)\n",
"Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
"Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
"Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
"Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
"Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
"Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
"Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
"Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
"Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
"Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
"Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
"Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
"Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
"Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
"Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
"Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
"Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
"Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
"Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
"Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
"Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
"Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
"Requirement already satisfied: pydantic>=1.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
"Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
"Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (13.7.1)\n",
"Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
"Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
"Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
"Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
"Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
"Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.18.0)\n",
"Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.59b0)\n",
"Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
"Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
"Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
"Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
"Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
"Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
"Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
"Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
"Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
"Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
"Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
"Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
"Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1)\n",
"Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
"Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
"Requirement already satisfied: soupsieve>1.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
"Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
"Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
"Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
"Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (3.11.0)\n",
"Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
"Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
"Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
"Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
"Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
"Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
"Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
"Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
"Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
"Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
"Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
"Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (2.41.4)\n",
"Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.4.2)\n",
"Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
"Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
"Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
"Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
"Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
"Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
"Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
"Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
"Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
"Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
"Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
"Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
"Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.9.0->semantica) (5.1.1)\n",
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
"Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
"Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
"Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
"Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
"Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
"Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
"Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
"Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
"Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
"Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0)\n",
"Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0)\n",
"Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0)\n",
"Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
"Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
"Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
"Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
"Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6)\n",
"Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pymongo>=4.2.0->semantica) (2.4.2)\n",
"Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
"Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
"Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
"Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
"Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
"Requirement already satisfied: httpx>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1)\n",
"Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
"Requirement already satisfied: pywin32>=226 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311)\n",
"Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
"Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9)\n",
"Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0)\n",
"Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
"Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
"Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (2.19.2)\n",
"Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
"Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
"Requirement already satisfied: huggingface-hub>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2)\n",
"Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (3.16.1)\n",
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
"Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.21.4)\n",
"Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0)\n",
"Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
"Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
"Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
"Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
"Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
"Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
"Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
"Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
"Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
"Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
"Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
"Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
"Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
"Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
"Requirement already satisfied: wrapt in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2)\n",
"Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
"Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from torch>=1.12.0->semantica) (1.13.3)\n",
"Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
"Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
"Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0)\n",
"Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
"Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)\n"
]
}
],
"outputs": [],
"source": [
"!pip install semantica"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"File: sample_document.txt\n",
"Content length: 281 characters\n"
]
},
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is ingesting</td><td>📥 ingest</td><td>FileIngestor</td><td>sample_document.txt</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is parsing</td><td>🔍 parse</td><td>DocumentParser</td><td>sample_document.txt</td><td>0.02s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" File name: sample_document.txt\n",
" File type: txt\n",
" Content available: True\n"
]
}
],
"outputs": [],
"source": [
"from semantica.ingest import FileIngestor\n",
"from pathlib import Path\n",
@@ -375,18 +131,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Parsed content length: 0 characters\n",
" Preview: N/A...\n"
]
}
],
"outputs": [],
"source": [
"from semantica.parse import DocumentParser\n",
"\n",
@@ -412,25 +159,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Text: ...\n",
" - Apple Inc. (Organization)\n",
" - Steve Jobs (Person)\n",
" - Steve Wozniak (Person)\n",
" - Ronald Wayne (Person)\n",
" - 1976 (Date)\n",
" - Cupertino, California (Location)\n",
" - Tim Cook (Person)\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import NamedEntityRecognizer, NERExtractor\n",
"\n",
@@ -465,31 +196,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Nodes (entities): 7\n",
" Edges (relationships): 6\n",
" Node: Apple Inc. (Organization)\n",
" Node: Steve Jobs (Person)\n",
" Node: Steve Wozniak (Person)\n",
" Node: Ronald Wayne (Person)\n",
" Node: 1976 (Date)\n",
" Node: Cupertino, California (Location)\n",
" Node: Tim Cook (Person)\n",
" Apple Inc. --[founded_by]--> Steve Jobs\n",
" Apple Inc. --[founded_by]--> Steve Wozniak\n",
" Apple Inc. --[founded_by]--> Ronald Wayne\n",
" Apple Inc. --[founded_in]--> 1976\n",
" Apple Inc. --[located_in]--> Cupertino, California\n",
" Tim Cook --[ceo_of]--> Apple Inc.\n"
]
}
],
"outputs": [],
"source": [
"from semantica.kg import GraphBuilder\n",
"import networkx as nx\n",
@@ -546,26 +255,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Total entities: 7\n",
" Total relationships: 6\n",
" - Organization: 1\n",
" - Person: 4\n",
" - Date: 1\n",
" - Location: 1\n",
" - founded_by: 3\n",
" - founded_in: 1\n",
" - located_in: 1\n",
" - ceo_of: 1\n"
]
}
],
"outputs": [],
"source": [
"from semantica.visualization import KGVisualizer\n",
"\n",
+32 -526
View File
@@ -82,223 +82,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)\n",
"Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
"Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.1.4)\n",
"Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
"Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
"Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
"Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
"Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
"Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
"Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.5)\n",
"Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
"Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
"Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
"Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
"Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.12.3)\n",
"Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
"Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
"Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
"Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
"Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
"Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
"Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
"Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
"Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
"Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
"Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
"Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
"Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
"Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
"Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
"Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
"Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
"Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
"Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
"Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
"Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
"Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
"Requirement already satisfied: pydantic>=1.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
"Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
"Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (13.7.1)\n",
"Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
"Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
"Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
"Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
"Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
"Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.18.0)\n",
"Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.59b0)\n",
"Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
"Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
"Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
"Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
"Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
"Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
"Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
"Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
"Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
"Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
"Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
"Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
"Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1)\n",
"Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
"Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
"Requirement already satisfied: soupsieve>1.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
"Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
"Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
"Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
"Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (3.11.0)\n",
"Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
"Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
"Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
"Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
"Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
"Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
"Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
"Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
"Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
"Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
"Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
"Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (2.41.4)\n",
"Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.4.2)\n",
"Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
"Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
"Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
"Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
"Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
"Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
"Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
"Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
"Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
"Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
"Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
"Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
"Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.9.0->semantica) (5.1.1)\n",
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
"Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
"Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
"Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
"Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
"Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
"Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
"Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
"Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
"Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
"Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0)\n",
"Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0)\n",
"Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0)\n",
"Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
"Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
"Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
"Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
"Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6)\n",
"Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pymongo>=4.2.0->semantica) (2.4.2)\n",
"Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
"Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
"Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
"Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
"Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
"Requirement already satisfied: httpx>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1)\n",
"Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
"Requirement already satisfied: pywin32>=226 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311)\n",
"Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
"Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9)\n",
"Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0)\n",
"Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
"Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
"Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (2.19.2)\n",
"Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
"Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
"Requirement already satisfied: huggingface-hub>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2)\n",
"Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (3.16.1)\n",
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
"Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.21.4)\n",
"Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0)\n",
"Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
"Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
"Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
"Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
"Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
"Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
"Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
"Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
"Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
"Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
"Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
"Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
"Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
"Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
"Requirement already satisfied: wrapt in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2)\n",
"Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
"Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from torch>=1.12.0->semantica) (1.13.3)\n",
"Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
"Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
"Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0)\n",
"Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
"Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install semantica\n"
]
@@ -316,15 +102,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Connected to graph database successfully!\n"
]
}
],
"outputs": [],
"source": [
"from semantica.graph_store import GraphStore\n",
"\n",
@@ -357,64 +135,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is processing</td><td>⏳ graph_store</td><td>Neo4jStore</td><td>-</td><td>0.14s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=34, offset=33>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 33, 'line': 1, 'column': 34}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'CREATE (n:Company $props) RETURN id(n) as id, n'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created company node: Apple Inc. (ID: 0)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=33, offset=32>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 32, 'line': 1, 'column': 33}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'CREATE (n:Person $props) RETURN id(n) as id, n'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created person node: Tim Cook (ID: 1)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=35, offset=34>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 34, 'line': 1, 'column': 35}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'CREATE (n:Location $props) RETURN id(n) as id, n'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created location node: Cupertino (ID: 2)\n"
]
}
],
"outputs": [],
"source": [
"# Create individual nodes with labels and properties\n",
"apple = store.create_node(\n",
@@ -438,26 +161,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=34, offset=33>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 33, 'line': 1, 'column': 34}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'CREATE (n:Company $props) RETURN id(n) as id, n'\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=34, offset=33>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 33, 'line': 1, 'column': 34}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'CREATE (n:Company $props) RETURN id(n) as id, n'\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=34, offset=33>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 33, 'line': 1, 'column': 34}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'CREATE (n:Company $props) RETURN id(n) as id, n'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created 3 company nodes in batch\n"
]
}
],
"outputs": [],
"source": [
"# Create multiple nodes in batch (more efficient for large datasets)\n",
"other_companies = store.create_nodes([\n",
@@ -484,42 +190,9 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=3, column=23, offset=54>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 54, 'line': 3, 'column': 23}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a), (b)\\n WHERE id(a) = $start_id AND id(b) = $end_id\\n CREATE (a)-[r:CEO_OF $props]->(b)\\n RETURN id(r) as id, type(r) as type, r\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=3, column=45, offset=76>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 76, 'line': 3, 'column': 45}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a), (b)\\n WHERE id(a) = $start_id AND id(b) = $end_id\\n CREATE (a)-[r:CEO_OF $props]->(b)\\n RETURN id(r) as id, type(r) as type, r\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=5, column=24, offset=165>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 165, 'line': 5, 'column': 24}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a), (b)\\n WHERE id(a) = $start_id AND id(b) = $end_id\\n CREATE (a)-[r:CEO_OF $props]->(b)\\n RETURN id(r) as id, type(r) as type, r\\n '\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created relationship: CEO_OF (ID: 1152921504606846977)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=3, column=23, offset=54>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 54, 'line': 3, 'column': 23}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a), (b)\\n WHERE id(a) = $start_id AND id(b) = $end_id\\n CREATE (a)-[r:HEADQUARTERED_IN $props]->(b)\\n RETURN id(r) as id, type(r) as type, r\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=3, column=45, offset=76>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 76, 'line': 3, 'column': 45}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a), (b)\\n WHERE id(a) = $start_id AND id(b) = $end_id\\n CREATE (a)-[r:HEADQUARTERED_IN $props]->(b)\\n RETURN id(r) as id, type(r) as type, r\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=5, column=24, offset=175>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 175, 'line': 5, 'column': 24}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a), (b)\\n WHERE id(a) = $start_id AND id(b) = $end_id\\n CREATE (a)-[r:HEADQUARTERED_IN $props]->(b)\\n RETURN id(r) as id, type(r) as type, r\\n '\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created relationship: HEADQUARTERED_IN (ID: 1152922604118474752)\n"
]
}
],
"outputs": [],
"source": [
"# Create relationships between nodes\n",
"ceo_rel = store.create_relationship(\n",
@@ -552,28 +225,9 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=26, offset=25>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 25, 'line': 1, 'column': 26}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'MATCH (n:Company) RETURN id(n) as id, n, labels(n) as labels LIMIT 10'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 4 companies:\n",
" - Apple Inc. (founded: 1976)\n",
" - Microsoft (founded: 1975)\n",
" - Google (founded: 1998)\n",
" - Amazon (founded: 1994)\n"
]
}
],
"outputs": [],
"source": [
"# Get nodes by label\n",
"companies = store.get_nodes(labels=[\"Company\"], limit=10)\n",
@@ -591,35 +245,9 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=3, column=31, offset=73>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 73, 'line': 3, 'column': 31}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a)-[r]-(b)\\n WHERE id(a) = $node_id\\n RETURN id(r) as id, type(r) as type, id(startNode(r)) as start_id, id(endNode(r)) as end_id, r\\n LIMIT 100\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=4, column=32, offset=121>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 121, 'line': 4, 'column': 32}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a)-[r]-(b)\\n WHERE id(a) = $node_id\\n RETURN id(r) as id, type(r) as type, id(startNode(r)) as start_id, id(endNode(r)) as end_id, r\\n LIMIT 100\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=4, column=62, offset=151>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 151, 'line': 4, 'column': 62}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a)-[r]-(b)\\n WHERE id(a) = $node_id\\n RETURN id(r) as id, type(r) as type, id(startNode(r)) as start_id, id(endNode(r)) as end_id, r\\n LIMIT 100\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=4, column=92, offset=181>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 181, 'line': 4, 'column': 92}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a)-[r]-(b)\\n WHERE id(a) = $node_id\\n RETURN id(r) as id, type(r) as type, id(startNode(r)) as start_id, id(endNode(r)) as end_id, r\\n LIMIT 100\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=3, column=31, offset=81>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 81, 'line': 3, 'column': 31}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a)-[r:CEO_OF]->(b)\\n WHERE id(a) = $node_id\\n RETURN id(r) as id, type(r) as type, id(a) as start_id, id(b) as end_id, r\\n LIMIT 100\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=4, column=32, offset=129>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 129, 'line': 4, 'column': 32}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a)-[r:CEO_OF]->(b)\\n WHERE id(a) = $node_id\\n RETURN id(r) as id, type(r) as type, id(a) as start_id, id(b) as end_id, r\\n LIMIT 100\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=4, column=62, offset=159>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 159, 'line': 4, 'column': 62}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a)-[r:CEO_OF]->(b)\\n WHERE id(a) = $node_id\\n RETURN id(r) as id, type(r) as type, id(a) as start_id, id(b) as end_id, r\\n LIMIT 100\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=4, column=81, offset=178>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 178, 'line': 4, 'column': 81}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH (a)-[r:CEO_OF]->(b)\\n WHERE id(a) = $node_id\\n RETURN id(r) as id, type(r) as type, id(a) as start_id, id(b) as end_id, r\\n LIMIT 100\\n '\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 2 relationships for Apple:\n",
" - HEADQUARTERED_IN: {'since': 1977}\n",
" - CEO_OF: {'since': 2011}\n",
"\n",
"Outgoing CEO_OF relationships: 1\n"
]
}
],
"outputs": [],
"source": [
"# Get relationships for a node\n",
"relationships = store.get_relationships(node_id=apple[\"id\"], direction=\"both\")\n",
@@ -652,18 +280,9 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CEO Relationships:\n",
" - Tim Cook is CEO of Apple Inc. since 2011\n"
]
}
],
"outputs": [],
"source": [
"# Execute a Cypher query to find CEO relationships\n",
"results = store.execute_query(\"\"\"\n",
@@ -681,19 +300,9 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Companies founded after 1990:\n",
" - Amazon (founded: 1994)\n",
" - Google (founded: 1998)\n"
]
}
],
"outputs": [],
"source": [
"# Parameterized query (safer and more efficient)\n",
"results = store.execute_query(\n",
@@ -721,7 +330,7 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -742,28 +351,9 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=3, column=23, offset=88>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 88, 'line': 3, 'column': 23}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH path = shortestPath((start)-[r*..5]-(end))\\n WHERE id(start) = $start_id AND id(end) = $end_id\\n RETURN path, length(path) as length\\n '\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=3, column=49, offset=114>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 114, 'line': 3, 'column': 49}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: '\\n MATCH path = shortestPath((start)-[r*..5]-(end))\\n WHERE id(start) = $start_id AND id(end) = $end_id\\n RETURN path, length(path) as length\\n '\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Shortest path found:\n",
" - Path length: 2\n",
" - Nodes in path: 3\n",
" - Relationships: 2\n"
]
}
],
"outputs": [],
"source": [
"# Find shortest path between two nodes\n",
"if tim_cook.get('id') and cupertino.get('id'):\n",
@@ -795,27 +385,9 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=17, offset=16>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 16, 'line': 1, 'column': 17}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'MATCH (n) WHERE id(n) = $id SET n += $props RETURN id(n) as id, n, labels(n) as labels'\n",
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=52, offset=51>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 51, 'line': 1, 'column': 52}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'MATCH (n) WHERE id(n) = $id SET n += $props RETURN id(n) as id, n, labels(n) as labels'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Updated node: Tim Cook\n",
" New age: 64\n",
" New title: CEO & President\n"
]
}
],
"outputs": [],
"source": [
"# Update node properties (merge mode - adds/updates properties)\n",
"if tim_cook.get('id'):\n",
@@ -849,27 +421,9 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Received notification from DBMS server: <GqlStatusObject gql_status='01N01', status_description='warn: feature deprecated with replacement. id is deprecated. It is replaced by elementId or consider using an application-generated id.', position=<SummaryInputPosition line=1, column=24, offset=23>, raw_classification='DEPRECATION', classification=<NotificationClassification.DEPRECATION: 'DEPRECATION'>, raw_severity='WARNING', severity=<NotificationSeverity.WARNING: 'WARNING'>, diagnostic_record={'_classification': 'DEPRECATION', '_severity': 'WARNING', '_position': {'offset': 23, 'line': 1, 'column': 24}, 'OPERATION': '', 'OPERATION_CODE': '0', 'CURRENT_SCHEMA': '/'}> for query: 'MATCH ()-[r]->() WHERE id(r) = $id DELETE r'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Deleted relationship (ID: 1152922604118474752)\n",
"\n",
"Tip: Use detach=True to delete a node and all its relationships\n",
" Use detach=False to only delete the node (fails if relationships exist)\n"
]
}
],
"outputs": [],
"source": [
"# Delete a relationship\n",
"if location_rel.get('id'):\n",
@@ -900,27 +454,9 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Graph Statistics:\n",
" Total nodes: 6\n",
" Total relationships: 1\n",
"\n",
"Node labels:\n",
" - Company: 4 nodes\n",
" - Person: 1 nodes\n",
" - Location: 1 nodes\n",
"\n",
"Relationship types:\n",
" - CEO_OF: 1 relationships\n"
]
}
],
"outputs": [],
"source": [
"# Get comprehensive graph statistics\n",
"stats = store.get_stats()\n",
@@ -947,23 +483,9 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Convenience functions available:\n",
" - create_node, create_nodes\n",
" - create_relationship, create_relationships\n",
" - get_nodes, get_relationships\n",
" - update_node, delete_node\n",
" - execute_query, shortest_path, get_neighbors\n",
" - run_analytics\n"
]
}
],
"outputs": [],
"source": [
"# Using convenience functions (alternative to class methods)\n",
"from semantica.graph_store import (\n",
@@ -1003,17 +525,9 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created index on Company.name for faster queries\n"
]
}
],
"outputs": [],
"source": [
"# Create an index on a node property for faster lookups\n",
"# This is especially useful for frequently queried properties\n",
@@ -1045,17 +559,9 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Connection closed successfully\n"
]
}
],
"outputs": [],
"source": [
"# Close the connection\n",
"store.close()\n",
+8 -265
View File
@@ -41,255 +41,18 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)\n",
"Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
"Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.1.4)\n",
"Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
"Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
"Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
"Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
"Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
"Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
"Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.5)\n",
"Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
"Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
"Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
"Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
"Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.12.3)\n",
"Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
"Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
"Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
"Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
"Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
"Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
"Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
"Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
"Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
"Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
"Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
"Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
"Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
"Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
"Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
"Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
"Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
"Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
"Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
"Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
"Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
"Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
"Requirement already satisfied: pydantic>=1.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
"Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
"Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (13.7.1)\n",
"Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
"Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
"Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
"Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
"Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
"Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.18.0)\n",
"Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.59b0)\n",
"Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
"Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
"Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
"Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
"Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
"Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
"Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
"Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
"Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
"Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
"Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
"Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
"Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1)\n",
"Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
"Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
"Requirement already satisfied: soupsieve>1.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
"Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
"Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
"Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
"Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (3.11.0)\n",
"Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
"Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
"Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
"Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
"Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
"Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
"Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
"Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
"Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
"Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
"Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
"Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (2.41.4)\n",
"Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.4.2)\n",
"Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
"Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
"Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
"Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
"Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
"Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
"Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
"Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
"Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
"Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
"Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
"Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
"Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.9.0->semantica) (5.1.1)\n",
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
"Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
"Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
"Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
"Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
"Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
"Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
"Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
"Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
"Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
"Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0)\n",
"Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0)\n",
"Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0)\n",
"Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
"Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
"Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
"Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
"Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6)\n",
"Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pymongo>=4.2.0->semantica) (2.4.2)\n",
"Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
"Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
"Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
"Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
"Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
"Requirement already satisfied: httpx>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1)\n",
"Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
"Requirement already satisfied: pywin32>=226 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311)\n",
"Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
"Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9)\n",
"Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0)\n",
"Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
"Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
"Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (2.19.2)\n",
"Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
"Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
"Requirement already satisfied: huggingface-hub>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2)\n",
"Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (3.16.1)\n",
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
"Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.21.4)\n",
"Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0)\n",
"Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
"Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
"Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
"Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
"Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
"Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
"Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
"Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
"Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
"Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
"Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
"Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
"Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
"Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
"Requirement already satisfied: wrapt in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2)\n",
"Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
"Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from torch>=1.12.0->semantica) (1.13.3)\n",
"Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
"Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
"Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0)\n",
"Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
"Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install semantica"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>GraphBuilder</td><td>-</td><td>0.08s</td></tr><tr><td>🔄</td><td>Semantica is building</td><td>🧠 kg</td><td>EntityResolver</td><td>-</td><td>131.06s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.04s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CentralityCalculator</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is building</td><td>🧠 kg</td><td>CommunityDetector</td><td>-</td><td>0.01s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Graph metrics:\n",
" Entities: 3\n",
" Relationships: 2\n",
" Density: 0.667\n"
]
}
],
"outputs": [],
"source": [
"from semantica.kg import GraphBuilder, GraphAnalyzer\n",
"from semantica.semantic_extract import NERExtractor, RelationExtractor\n",
@@ -329,20 +92,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Centrality scores:\n",
" e2: 0.500\n",
" e1: 1.000\n",
" e3: 0.500\n"
]
}
],
"outputs": [],
"source": [
"from semantica.kg import CentralityCalculator\n",
"\n",
@@ -367,18 +119,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detected 1 communities\n",
" Community 1: 3 entities\n"
]
}
],
"outputs": [],
"source": [
"from semantica.kg import CommunityDetector\n",
"\n",
@@ -54,25 +54,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install -q semantica\n"
]
@@ -96,72 +80,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Split into 8 chunks using recursive method\n",
"\n",
"================================================================================\n",
"\n",
"Chunk 1:\n",
" Length: 181 characters\n",
" Start: 0, End: 184\n",
" Text: Apple Inc. is a technology company founded by Steve Jobs, Steve Wozniak, and Ronald Wayne \n",
"in Cupert...\n",
"\n",
"Chunk 2:\n",
" Length: 144 characters\n",
" Start: 134, End: 281\n",
" Text: The company's current CEO is Tim Cook, who took \n",
"over from Steve Jobs in August 2011. Apple is headq...\n",
"\n",
"Chunk 3:\n",
" Length: 150 characters\n",
" Start: 231, End: 383\n",
" Text: eadquartered at One Apple Park Way in Cupertino.\n",
"\n",
"Apple develops and sells consumer electronics, com...\n",
"\n",
"Chunk 4:\n",
" Length: 151 characters\n",
" Start: 333, End: 486\n",
" Text: ter software, and online services. The company's \n",
"hardware products include the iPhone smartphone, t...\n",
"\n",
"Chunk 5:\n",
" Length: 176 characters\n",
" Start: 436, End: 614\n",
" Text: iPad tablet computer, the Mac personal computer, \n",
"the iPod portable media player, the Apple Watch sm...\n",
"\n",
"Chunk 6:\n",
" Length: 152 characters\n",
" Start: 564, End: 718\n",
" Text: al media player, and the \n",
"HomePod smart speaker.\n",
"\n",
"Apple's software includes the macOS and iOS operat...\n",
"\n",
"Chunk 7:\n",
" Length: 150 characters\n",
" Start: 668, End: 820\n",
" Text: systems, the iTunes media player, the Safari web \n",
"browser, and the iLife and iWork creativity and pr...\n",
"\n",
"Chunk 8:\n",
" Length: 126 characters\n",
" Start: 770, End: 896\n",
" Text: uctivity suites. Its online services include the \n",
"iTunes Store, the iOS App Store and Mac App Store,...\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import TextSplitter\n",
"\n",
@@ -222,39 +143,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Comparing Standard Splitting Methods:\n",
"\n",
"================================================================================\n",
"\n",
"Method: RECURSIVE\n",
"----------------------------------------\n",
" Chunks created: 8\n",
" Avg chunk size: 154 chars\n",
" First chunk: Apple Inc. is a technology company founded by Steve Jobs, Steve Wozniak, and Ron...\n",
"\n",
"Method: SENTENCE\n",
"----------------------------------------\n",
" Chunks created: 6\n",
" Avg chunk size: 148 chars\n",
" First chunk: Apple Inc. is a technology company founded by Steve Jobs, Steve Wozniak, and Ron...\n",
"\n",
"Method: PARAGRAPH\n",
"----------------------------------------\n",
" Chunks created: 3\n",
" Avg chunk size: 297 chars\n",
" First chunk: Apple Inc. is a technology company founded by Steve Jobs, Steve Wozniak, and Ron...\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"# Compare different methods\n",
"methods = [\"recursive\", \"sentence\", \"paragraph\"]\n",
@@ -297,136 +188,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Token-Based Splitting Results:\n",
"\n",
"================================================================================\n",
"\n",
"Chunk 1:\n",
" Tokens: 100\n",
" Characters: 461\n",
" Ratio: 4.61 chars/token\n",
"\n",
"Chunk 2:\n",
" Tokens: 100\n",
" Characters: 501\n",
" Ratio: 5.01 chars/token\n",
"\n",
"Chunk 3:\n",
" Tokens: 31\n",
" Characters: 150\n",
" Ratio: 4.84 chars/token\n",
"\n",
"Chunk 4:\n",
" Tokens: 20\n",
" Characters: 78\n",
" Ratio: 3.90 chars/token\n",
"\n",
"Chunk 5:\n",
" Tokens: 19\n",
" Characters: 76\n",
" Ratio: 4.00 chars/token\n",
"\n",
"Chunk 6:\n",
" Tokens: 18\n",
" Characters: 74\n",
" Ratio: 4.11 chars/token\n",
"\n",
"Chunk 7:\n",
" Tokens: 17\n",
" Characters: 70\n",
" Ratio: 4.12 chars/token\n",
"\n",
"Chunk 8:\n",
" Tokens: 16\n",
" Characters: 64\n",
" Ratio: 4.00 chars/token\n",
"\n",
"Chunk 9:\n",
" Tokens: 15\n",
" Characters: 63\n",
" Ratio: 4.20 chars/token\n",
"\n",
"Chunk 10:\n",
" Tokens: 14\n",
" Characters: 59\n",
" Ratio: 4.21 chars/token\n",
"\n",
"Chunk 11:\n",
" Tokens: 13\n",
" Characters: 55\n",
" Ratio: 4.23 chars/token\n",
"\n",
"Chunk 12:\n",
" Tokens: 12\n",
" Characters: 51\n",
" Ratio: 4.25 chars/token\n",
"\n",
"Chunk 13:\n",
" Tokens: 11\n",
" Characters: 45\n",
" Ratio: 4.09 chars/token\n",
"\n",
"Chunk 14:\n",
" Tokens: 10\n",
" Characters: 41\n",
" Ratio: 4.10 chars/token\n",
"\n",
"Chunk 15:\n",
" Tokens: 9\n",
" Characters: 37\n",
" Ratio: 4.11 chars/token\n",
"\n",
"Chunk 16:\n",
" Tokens: 8\n",
" Characters: 33\n",
" Ratio: 4.12 chars/token\n",
"\n",
"Chunk 17:\n",
" Tokens: 7\n",
" Characters: 27\n",
" Ratio: 3.86 chars/token\n",
"\n",
"Chunk 18:\n",
" Tokens: 6\n",
" Characters: 26\n",
" Ratio: 4.33 chars/token\n",
"\n",
"Chunk 19:\n",
" Tokens: 5\n",
" Characters: 20\n",
" Ratio: 4.00 chars/token\n",
"\n",
"Chunk 20:\n",
" Tokens: 4\n",
" Characters: 14\n",
" Ratio: 3.50 chars/token\n",
"\n",
"Chunk 21:\n",
" Tokens: 3\n",
" Characters: 13\n",
" Ratio: 4.33 chars/token\n",
"\n",
"Chunk 22:\n",
" Tokens: 2\n",
" Characters: 9\n",
" Ratio: 4.50 chars/token\n",
"\n",
"Chunk 23:\n",
" Tokens: 1\n",
" Characters: 2\n",
" Ratio: 2.00 chars/token\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import split_by_tokens\n",
"\n",
@@ -470,69 +234,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is splitting</td><td>✂️ split</td><td>SemanticChunker</td><td>-</td><td>0.13s</td></tr><tr><td>✅</td><td>Semantica is splitting</td><td>✂️ split</td><td>EntityAwareChunker</td><td>-</td><td>0.98s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>NERExtractor</td><td>-</td><td>0.47s</td></tr><tr><td>✅</td><td>Semantica is splitting</td><td>✂️ split</td><td>RelationAwareChunker</td><td>-</td><td>1.39s</td></tr><tr><td>✅</td><td>Semantica is extracting</td><td>🎯 semantic_extract</td><td>RelationExtractor</td><td>-</td><td>0.45s</td></tr><tr><td>✅</td><td>Semantica is splitting</td><td>✂️ split</td><td>StructuralChunker</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is splitting</td><td>✂️ split</td><td>HierarchicalChunker</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is splitting</td><td>✂️ split</td><td>SlidingWindowChunker</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is splitting</td><td>✂️ split</td><td>TableChunker</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is splitting</td><td>✂️ split</td><td>ChunkValidator</td><td>-</td><td>0.01s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Semantic Chunking Results:\n",
"\n",
"================================================================================\n",
"\n",
"Chunk 1:\n",
" Length: 133 chars\n",
" Coherence: N/A\n",
" Text: Apple Inc. is a technology company founded by Steve Jobs, Steve Wozniak, and Ronald Wayne \n",
"in Cupert...\n",
"\n",
"Chunk 2:\n",
" Length: 194 chars\n",
" Coherence: N/A\n",
" Text: Wayne \n",
"in Cupertino, California on April 1, 1976. The company's current CEO is Tim Cook, who took \n",
"o...\n",
"\n",
"Chunk 3:\n",
" Length: 136 chars\n",
" Coherence: N/A\n",
" Text: headquartered at One Apple Park Way in Cupertino. Apple develops and sells consumer electronics, com...\n",
"\n",
"Chunk 4:\n",
" Length: 295 chars\n",
" Coherence: N/A\n",
" Text: ectronics, computer software, and online services. The company's \n",
"hardware products include the iPho...\n",
"\n",
"Chunk 5:\n",
" Length: 223 chars\n",
" Coherence: N/A\n",
" Text: ital media player, and the \n",
"HomePod smart speaker. Apple's software includes the macOS and iOS opera...\n",
"\n",
"Chunk 6:\n",
" Length: 159 chars\n",
" Coherence: N/A\n",
" Text: Life and iWork creativity and productivity suites. Its online services include the \n",
"iTunes Store, th...\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import SemanticChunker\n",
"\n",
@@ -576,26 +280,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Entity-Aware Chunking Results:\n",
"\n",
"================================================================================\n",
"\n",
"Chunk 1:\n",
" Length: 892 chars\n",
" Entities: 28\n",
" Sample entities: [\"Entity(text='Apple Inc.', label='ORG', start_char=1, end_char=11, confidence=1.0, metadata={'extraction_method': 'ml', 'model': 'en_core_web_sm', 'lemma': 'Apple Inc.'})\", \"Entity(text='Steve Jobs', label='PERSON', start_char=47, end_char=57, confidence=1.0, metadata={'extraction_method': 'ml', 'model': 'en_core_web_sm', 'lemma': 'Steve Jobs'})\", \"Entity(text='Steve Wozniak', label='PERSON', start_char=59, end_char=72, confidence=1.0, metadata={'extraction_method': 'ml', 'model': 'en_core_web_sm', 'lemma': 'Steve Wozniak'})\"]\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import EntityAwareChunker\n",
"\n",
@@ -643,51 +330,9 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Relation-Aware Chunking Results:\n",
"\n",
"================================================================================\n",
"\n",
"Chunk 1:\n",
" Length: 133 chars\n",
" Triplets: 2\n",
" Relationships: 2\n",
"\n",
"Chunk 2:\n",
" Length: 144 chars\n",
" Triplets: 0\n",
" Relationships: 0\n",
"\n",
"Chunk 3:\n",
" Length: 86 chars\n",
" Triplets: 0\n",
" Relationships: 0\n",
"\n",
"Chunk 4:\n",
" Length: 244 chars\n",
" Triplets: 0\n",
" Relationships: 0\n",
"\n",
"Chunk 5:\n",
" Length: 172 chars\n",
" Triplets: 0\n",
" Relationships: 0\n",
"\n",
"Chunk 6:\n",
" Length: 108 chars\n",
" Triplets: 0\n",
" Relationships: 0\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import RelationAwareChunker\n",
"\n",
@@ -732,50 +377,9 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Structural Chunking Results:\n",
"\n",
"================================================================================\n",
"\n",
"Chunk 1:\n",
" Section: N/A\n",
" Level: N/A\n",
" Text: # Apple Inc....\n",
"\n",
"Chunk 2:\n",
" Section: N/A\n",
" Level: N/A\n",
" Text: ## History\n",
"\n",
"Apple Inc. was founded by Steve Jobs, Steve Wozniak, and Ronald Wayn...\n",
"\n",
"Chunk 3:\n",
" Section: N/A\n",
" Level: N/A\n",
" Text: ## Products\n",
"\n",
"### Hardware\n",
"\n",
"- iPhone\n",
"- iPad\n",
"- Mac\n",
"\n",
"### Software\n",
"\n",
"- macOS\n",
"- iOS\n",
"- ...\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import StructuralChunker\n",
"\n",
@@ -842,39 +446,9 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hierarchical Chunking Results:\n",
"\n",
"================================================================================\n",
"\n",
"Chunk 1:\n",
" Level: N/A\n",
" Length: 278 chars\n",
" Parent: None (root)\n",
" Children: 0\n",
"\n",
"Chunk 2:\n",
" Level: N/A\n",
" Length: 331 chars\n",
" Parent: None (root)\n",
" Children: 0\n",
"\n",
"Chunk 3:\n",
" Level: N/A\n",
" Length: 281 chars\n",
" Parent: None (root)\n",
" Children: 0\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import HierarchicalChunker\n",
"\n",
@@ -921,66 +495,9 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sliding Window Chunking Results:\n",
"\n",
"================================================================================\n",
"\n",
"Window 1:\n",
" Position: 0-134\n",
" Length: 133 chars\n",
" Overlap with previous: 0 chars\n",
"\n",
"Window 2:\n",
" Position: 100-220\n",
" Length: 120 chars\n",
" Overlap with previous: 34 chars\n",
"\n",
"Window 3:\n",
" Position: 200-279\n",
" Length: 79 chars\n",
" Overlap with previous: 20 chars\n",
"\n",
"Window 4:\n",
" Position: 300-383\n",
" Length: 81 chars\n",
" Overlap with previous: 0 chars\n",
"\n",
"Window 5:\n",
" Position: 400-486\n",
" Length: 83 chars\n",
" Overlap with previous: 0 chars\n",
"\n",
"Window 6:\n",
" Position: 500-612\n",
" Length: 112 chars\n",
" Overlap with previous: 0 chars\n",
"\n",
"Window 7:\n",
" Position: 600-718\n",
" Length: 116 chars\n",
" Overlap with previous: 12 chars\n",
"\n",
"Window 8:\n",
" Position: 700-786\n",
" Length: 86 chars\n",
" Overlap with previous: 18 chars\n",
"\n",
"Window 9:\n",
" Position: 800-896\n",
" Length: 95 chars\n",
" Overlap with previous: 0 chars\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import SlidingWindowChunker\n",
"\n",
@@ -1027,29 +544,9 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Table Chunking Results:\n",
"\n",
"================================================================================\n",
"\n",
"Chunk 1:\n",
" Type: Text\n",
" Content: Product | Category | Release Year\n",
"--- | --- | ---\n",
"iPhone | Smartphone | 2007\n",
"iPad | Tablet | 2010\n",
"Ma...\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"from semantica.split import TableChunker\n",
"\n",
@@ -1114,29 +611,9 @@
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chunk class fields: {'text': <class 'str'>, 'start_index': <class 'int'>, 'end_index': <class 'int'>, 'metadata': typing.Dict[str, typing.Any], 'id': typing.Optional[str]}\n",
"Provenance Tracking Results:\n",
"\n",
"================================================================================\n",
"Chunk ID: f0bddbe2-b717-47c7-8bdb-631a61660373\n",
"\n",
"Lineage for Chunk 1:\n",
" Source Document: apple_doc_001\n",
" File Path: data/apple.txt\n",
" Method: recursive\n",
" Timestamp: None\n",
"\n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"import sys\n",
"import os\n",
@@ -1228,26 +705,9 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Method Comparison:\n",
"\n",
"================================================================================\n",
"Method Chunks Avg Size Time (ms) \n",
"--------------------------------------------------------------------------------\n",
"recursive 8 154 0.00 \n",
"sentence 6 148 519.51 \n",
"paragraph 3 297 0.00 \n",
"token 51 129 0.00 \n",
"================================================================================\n"
]
}
],
"outputs": [],
"source": [
"import time\n",
"\n",
@@ -142,10 +142,15 @@
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
}
+16 -120
View File
@@ -55,25 +55,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install -q semantica\n"
]
@@ -97,30 +81,9 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>VectorStore</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>FAISSStore</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>HybridSearch</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>MetadataStore</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>NamespaceManager</td><td>-</td><td>0.00s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Stored 100 vectors\n",
"First 3 IDs: ['vec_0', 'vec_2', 'vec_4']\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import VectorStore\n",
"from semantica.embeddings import TextEmbedder\n",
@@ -166,18 +129,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Index search found 10 results\n",
"Distances: [0. 0.59663105 0.6359793 0.66132385 0.6909249 ]\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import VectorIndexer, FAISSStore \n",
"import numpy as np \n",
@@ -228,18 +182,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Index search found 10 results\n",
"Distances: [0. 0.59663105 0.6359793 0.66132385 0.6909249 ]\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import VectorIndexer, FAISSStore\n",
"\n",
@@ -282,24 +227,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hybrid search found 10 results\n",
"\n",
"Filtered results (science, year > 2021):\n",
"1. Category: science, Year: 2020, Score: 1.000\n",
"2. Category: technology, Year: 2021, Score: 0.702\n",
"3. Category: science, Year: 2022, Score: 0.682\n",
"4. Category: technology, Year: 2023, Score: 0.669\n",
"5. Category: science, Year: 2022, Score: 0.655\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import HybridSearch, MetadataFilter \n",
"import numpy as np\n",
@@ -351,19 +281,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 5 vectors with category='science'\n",
"\n",
"Metadata validation: True\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import MetadataStore, MetadataSchema\n",
"\n",
@@ -410,21 +330,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Fused results using RRF:\n",
"1. ID: vec_2, Fused Score: 0.033\n",
"2. ID: vec_1, Fused Score: 0.032\n",
"3. ID: vec_4, Fused Score: 0.016\n",
"4. ID: vec_3, Fused Score: 0.016\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import SearchRanker\n",
"\n",
@@ -469,21 +377,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Tenant 1: 5 vectors\n",
"Tenant 2: 5 vectors\n",
"\n",
"User1 can write: True\n",
"User2 can write: False\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import NamespaceManager\n",
"\n",
File diff suppressed because it is too large Load Diff
+7 -234
View File
@@ -76,245 +76,18 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)\n",
"Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
"Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.1.4)\n",
"Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
"Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
"Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
"Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
"Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
"Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
"Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.5)\n",
"Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
"Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
"Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
"Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
"Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.12.3)\n",
"Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
"Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
"Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
"Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
"Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
"Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
"Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
"Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
"Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
"Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
"Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
"Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
"Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
"Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
"Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
"Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
"Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
"Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
"Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
"Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
"Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
"Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
"Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
"Requirement already satisfied: pydantic>=1.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
"Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
"Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (13.7.1)\n",
"Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
"Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
"Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
"Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
"Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
"Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
"Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.18.0)\n",
"Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.38.0)\n",
"Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.59b0)\n",
"Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
"Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
"Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
"Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
"Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
"Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
"Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
"Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
"Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
"Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
"Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
"Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
"Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.14.1)\n",
"Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
"Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
"Requirement already satisfied: soupsieve>1.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
"Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
"Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
"Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
"Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (3.11.0)\n",
"Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
"Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
"Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
"Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
"Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
"Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
"Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
"Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
"Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
"Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
"Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
"Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (2.41.4)\n",
"Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=1.10.0->semantica) (0.4.2)\n",
"Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
"Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
"Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
"Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
"Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
"Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
"Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
"Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
"Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
"Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
"Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (5.5.2)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
"Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
"Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
"Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.9.0->semantica) (5.1.1)\n",
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
"Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
"Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
"Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
"Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
"Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
"Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
"Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
"Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
"Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
"Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-api>=1.12.0->semantica) (6.8.0)\n",
"Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.12.0->semantica) (3.17.0)\n",
"Requirement already satisfied: opentelemetry-semantic-conventions==0.59b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.59b0)\n",
"Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
"Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
"Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
"Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
"Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.24.6)\n",
"Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pymongo>=4.2.0->semantica) (2.4.2)\n",
"Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
"Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
"Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
"Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
"Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
"Requirement already satisfied: httpx>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.28.1)\n",
"Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
"Requirement already satisfied: pywin32>=226 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client>=1.3.0->semantica) (311)\n",
"Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
"Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (1.0.9)\n",
"Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (0.16.0)\n",
"Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
"Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
"Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (2.19.2)\n",
"Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
"Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
"Requirement already satisfied: huggingface-hub>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sentence-transformers>=2.2.0->semantica) (0.30.2)\n",
"Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (3.16.1)\n",
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
"Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.21.4)\n",
"Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub>=0.20.0->sentence-transformers>=2.2.0->semantica) (2023.10.0)\n",
"Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
"Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
"Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
"Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
"Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
"Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
"Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
"Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
"Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
"Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
"Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
"Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
"Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
"Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
"Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
"Requirement already satisfied: wrapt in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (1.17.2)\n",
"Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
"Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from torch>=1.12.0->semantica) (1.13.3)\n",
"Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
"Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
"Requirement already satisfied: authlib<2.0.0,>=1.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (1.6.0)\n",
"Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
"Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->torch>=1.12.0->semantica) (1.3.0)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install semantica"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>🔄</td><td>Semantica is building</td><td>🧠 kg</td><td>EntityResolver</td><td>-</td><td>762.30s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is exporting</td><td>💾 export</td><td>JSONExporter</td><td>output.json</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is exporting</td><td>💾 export</td><td>RDFExporter</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is exporting</td><td>💾 export</td><td>GraphExporter</td><td>output.graphml</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>OntologyGenerator</td><td>-</td><td>0.04s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>ClassInferrer</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is generating</td><td>📚 ontology</td><td>PropertyGenerator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is exporting</td><td>💾 export</td><td>OWLExporter</td><td>output.owl</td><td>0.01s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"outputs": [],
"source": [
"from semantica.export import JSONExporter\n",
"from semantica.kg import GraphBuilder\n",
@@ -350,7 +123,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -392,7 +165,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -425,7 +198,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
File diff suppressed because it is too large Load Diff
@@ -26,7 +26,7 @@
"\n",
"- Use `ConflictDetector.detect_value_conflicts(entities, property_name=...)` when you want to check one field.\n",
"- Use `ConflictDetector.detect_conflicts(entities)` when you want a broader scan (value/type/temporal/etc.).\n",
"- If you're starting from a `KnowledgeGraph`, pull entities via `kg.get(\"entities\", [])` (not `kg.entities`).\n",
"- If you're starting from a Knowledge Graph dictionary (built via `GraphBuilder`), pull entities via `kg.get(\"entities\", [])`.\n",
"\n",
"### Key Capabilities\n",
"\n",
@@ -64,32 +64,16 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install -q semantica "
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -125,17 +109,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Loaded 3 records for Employee 001\n"
]
}
],
"outputs": [],
"source": [
"# 1. Define source metadata\n",
"sources_metadata = {\n",
@@ -199,20 +175,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Registering sources...\n",
" - Registered 'hr_db' with credibility 0.95\n",
" - Registered 'linkedin_scrape' with credibility 0.6\n",
" - Registered 'public_dir' with credibility 0.4\n"
]
}
],
"outputs": [],
"source": [
"from semantica.conflicts import SourceTracker\n",
"\n",
@@ -255,45 +220,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictDetector</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictAnalyzer</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>ConflictResolver</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is resolving</td><td>⚠️ conflicts</td><td>InvestigationGuideGenerator</td><td>-</td><td>0.00s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Value conflict detected: emp_001.birth_date has conflicting values: ['1980-05-15', '1982-05-15']\n",
"Value conflict detected: emp_001.department has conflicting values: ['Software Engineering', 'Engineering']\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detected 2 conflicts:\n",
"- value_conflict: birth_date for emp_001\n",
" Values: ['1980-05-15', '1980-05-15', '1982-05-15']\n",
" Severity: medium\n",
"--- \n",
"- value_conflict: department for emp_001\n",
" Values: ['Engineering', 'Software Engineering', 'Engineering']\n",
" Severity: medium\n",
"--- \n"
]
}
],
"outputs": [],
"source": [
"from semantica.conflicts import ConflictDetector\n",
"\n",
@@ -339,20 +268,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Conflict Analysis Summary:\n",
"Total Conflicts: 2\n",
"By Type: {'value_conflict': 2}\n",
"By Severity: {'medium': 2}\n"
]
}
],
"outputs": [],
"source": [
"from semantica.conflicts import ConflictAnalyzer\n",
"\n",
@@ -401,23 +319,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--- Resolution: Voting ---\n",
"Property: birth_date | Resolved Value: 1980-05-15\n",
"Property: department | Resolved Value: Engineering\n",
"\n",
"--- Resolution: Credibility Weighted ---\n",
"Property: birth_date | Resolved Value: 1980-05-15 (Confidence: 0.79)\n",
"Property: department | Resolved Value: Engineering (Confidence: 0.69)\n"
]
}
],
"outputs": [],
"source": [
"from semantica.conflicts import ConflictResolver\n",
"\n",
@@ -462,39 +366,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"=== Investigation: emp_001_birth_date_conflict ===\n",
"Summary: Conflict Type: value_conflict | Entity: emp_001 | Property: birth_date | Conflicting Values: 1980-05-15, 1982-05-15 | Severity: medium | Sources: public_dir, hr_db, linkedin_scrape\n",
"\n",
"Investigation Steps:\n",
"1. Review conflict details and context\n",
" Action: Examine the conflict summary and identify the conflicting values\n",
"2. Identify all source documents\n",
" Action: Locate and review the following source documents: public_dir, hr_db, linkedin_scrape\n",
"3. Compare conflicting information across sources\n",
" Action: Review each source document and note the specific value and context for each\n",
"4. Assess source credibility and reliability\n",
" Action: Review source credibility scores and document metadata\n",
"5. Determine resolution approach\n",
" Action: Choose resolution strategy based on source credibility, recency, and business rules\n",
"6. Document resolution decision\n",
" Action: Record the resolved value, resolution method, and rationale\n",
"\n",
"Recommended Actions:\n",
"[ ] Compare source documents side-by-side\n",
"[ ] Check for data entry errors or typos\n",
"[ ] Verify if values represent different time periods\n",
"[ ] Review 3 conflicting source documents\n",
"[ ] Identify most authoritative source\n"
]
}
],
"outputs": [],
"source": [
"from semantica.conflicts import InvestigationGuideGenerator\n",
"\n",
+16 -130
View File
@@ -21,25 +21,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"# Install Semantica\n",
"!pip install -q semantica"
@@ -61,17 +45,9 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created 6 sample entities.\n"
]
}
],
"outputs": [],
"source": [
"entities = [\n",
" # Apple Variations\n",
@@ -140,34 +116,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>SimilarityCalculator</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>DuplicateDetector</td><td>-</td><td>0.17s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>EntityMerger</td><td>-</td><td>0.19s</td></tr><tr><td>✅</td><td>Semantica is deduplicating</td><td>🔄 deduplication</td><td>MergeStrategyManager</td><td>-</td><td>0.01s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Similarity between 'Apple Inc.' and 'Apple Inc':\n",
" Total Score: 0.7218\n",
" Breakdown: {'string': 0.98, 'property': 0.7728205128205129, 'relationship': 0.0}\n",
"\n",
"Similarity between 'Apple Inc.' and 'Microsoft':\n",
" Total Score: 0.1816\n"
]
}
],
"outputs": [],
"source": [
"from semantica.deduplication import SimilarityCalculator, SimilarityResult\n",
"# Initialize calculator with custom weights\n",
@@ -206,19 +157,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 2 duplicate pairs:\n",
" - Microsoft Corp <==> Microsoft (Score: 0.80)\n",
" - Apple Inc. <==> Apple Inc (Score: 0.74)\n"
]
}
],
"outputs": [],
"source": [
"# Import specific classes for Duplicate Detection\n",
"from semantica.deduplication import DuplicateDetector, DuplicateCandidate, DuplicateGroup\n",
@@ -247,17 +188,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"New matches found: 0\n"
]
}
],
"outputs": [],
"source": [
"existing_db = entities[:3] # The Apple entities\n",
"new_data = [entities[4]] # Microsoft\n",
@@ -279,22 +212,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 2 clusters:\n",
" Cluster 1: ['Apple Inc.', 'Apple Inc']\n",
" Cluster 2: ['Microsoft Corp', 'Microsoft']\n",
"Found 2 clusters:\n",
" Cluster 1: ['Apple Inc.', 'Apple Inc']\n",
" Cluster 2: ['Microsoft Corp', 'Microsoft']\n"
]
}
],
"outputs": [],
"source": [
"# Import specific classes for Clustering\n",
"from semantica.deduplication import ClusterBuilder, Cluster, ClusterResult\n",
@@ -331,7 +251,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -341,25 +261,9 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Performed 2 merge operations.\n",
"\n",
"--- Merged Results ---\n",
"Merged 2 entities into: 'Microsoft Corp'\n",
" - Final Properties: {'industry': 'Software', 'hq': 'Redmond'}\n",
" - Final Relationships: 0\n",
"Merged 2 entities into: 'Apple Inc.'\n",
" - Final Properties: {'industry': 'Technology', 'hq': 'Cupertino', 'founded': 1976}\n",
" - Final Relationships: 1\n"
]
}
],
"outputs": [],
"source": [
"merger = EntityMerger()\n",
"\n",
@@ -394,25 +298,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1. Detecting duplicates...\n",
"2. Merging entities...\n",
"\n",
"Original Size: 6\n",
"Cleaned Size: 4\n",
"\n",
"Final Entity Names:\n",
" - Microsoft Corp\n",
" - Apple Inc.\n",
" - Apple\n",
" - Google LLC\n"
]
}
],
"outputs": [],
"source": [
"def deduplicate_dataset(raw_entities):\n",
" print(\"1. Detecting duplicates...\")\n",
+34 -292
View File
@@ -23,26 +23,10 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"id": "257bad40",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"!pip install -q semantica"
]
@@ -59,31 +43,10 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"id": "8c845a94",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\sentence_transformers\\cross_encoder\\CrossEncoder.py:13: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)\n",
" from tqdm.autonotebook import tqdm, trange\n",
"fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n",
"fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
]
},
{
"data": {
"text/plain": [
"('inmemory', 384)"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"from semantica.vector_store import VectorStore\n",
"\n",
@@ -107,25 +70,10 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"id": "f4d788b5",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'retention_days': 30,\n",
" 'max_memories': 10000,\n",
" 'use_graph_expansion': True,\n",
" 'max_expansion_hops': 2,\n",
" 'hybrid_alpha': 0.5}"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"from semantica.context import AgentContext, ContextGraph\n",
"\n",
@@ -147,38 +95,10 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"id": "5d65eb00",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>File</th><th>Time</th></tr><tr><td>✅</td><td>Semantica is processing</td><td>🔗 context</td><td>AgentMemory</td><td>-</td><td>0.01s</td></tr><tr><td>✅</td><td>Semantica is embedding</td><td>💾 embeddings</td><td>TextEmbedder</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is indexing</td><td>📊 vector_store</td><td>VectorStore</td><td>-</td><td>0.00s</td></tr><tr><td>✅</td><td>Semantica is processing</td><td>🔗 context</td><td>ContextRetriever</td><td>-</td><td>0.02s</td></tr><tr><td>✅</td><td>Semantica is processing</td><td>🔗 context</td><td>EntityLinker</td><td>-</td><td>0.00s</td></tr></table></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"{'id': None,\n",
" 'content': 'User prefers short answers about Python.',\n",
" 'timestamp': '2025-12-18T20:46:17.265703',\n",
" 'metadata': {'type': 'preference',\n",
" 'conversation_id': 'conv_1',\n",
" 'user_id': 'user_1'}}"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"memory_id = context.store(\n",
" \"User prefers short answers about Python.\",\n",
@@ -192,35 +112,10 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"id": "3c1be718",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"C:\\Users\\Mohd Kaif\\semantica\\semantica\\vector_store\\vector_store.py:480: RuntimeWarning: invalid value encountered in divide\n",
" similarities = np.dot(vectors, query_vector) / (vector_norms * query_norm)\n"
]
},
{
"data": {
"text/plain": [
"[{'content': 'User prefers short answers about Python.',\n",
" 'score': 1.0,\n",
" 'source': 'short_term',\n",
" 'metadata': {'type': 'preference',\n",
" 'conversation_id': 'conv_1',\n",
" 'user_id': 'user_1'},\n",
" 'related_entities': []}]"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"context.store(\n",
" \"User is working on Semantica context module examples.\",\n",
@@ -234,21 +129,10 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"id": "485acf33",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"context.conversation(\"conv_1\", max_items=10)"
]
@@ -265,21 +149,10 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"id": "a264ef4d",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'{\\n \"exported_at\": \"2025-12-18T20:46:27.455602\",\\n \"count\": 2,\\n \"memories\": [\\n {\\n \"memory_id\": \"mem_5c7dba9a7373\",\\n \"content\": \"User prefers short answers about Python.\",\\n \"timestamp\": \"2025-12-18T20:46:17.265703\",\\n \"metadata\": {\\n \"type\": \"preference\",\\n \"convers'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"export_json = context.export(conversation_id=\"conv_1\", format=\"json\")\n",
"export_json[:300]"
@@ -287,24 +160,10 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"id": "b62d1859",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'conversation_id': 'conv_1',\n",
" 'message_count': 0,\n",
" 'first_message': None,\n",
" 'last_message': None}"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"import tempfile\n",
"\n",
@@ -327,24 +186,10 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"id": "72930ae7",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'stored_count': 2,\n",
" 'memory_ids': ['mem_b467da01b18a', 'mem_e3dbd310355d'],\n",
" 'graph_nodes': 0,\n",
" 'graph_edges': 0}"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"documents = [\n",
" {\n",
@@ -415,54 +260,20 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"id": "df2e5fcd",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'node': {'id': 'e_ml',\n",
" 'type': 'CONCEPT',\n",
" 'properties': {'content': 'Machine Learning',\n",
" 'id': 'e_ml',\n",
" 'text': 'Machine Learning',\n",
" 'type': 'CONCEPT'}},\n",
" 'score': 1.0,\n",
" 'content': 'Machine Learning'}]"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"kg.query(\"machine learning\")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"id": "0836feeb",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'id': 'e_ml',\n",
" 'type': 'CONCEPT',\n",
" 'content': 'Machine Learning',\n",
" 'relationship': 'used_for',\n",
" 'weight': 0.9,\n",
" 'hop': 1}]"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"kg.get_neighbors(\"e_python\", hops=2)"
]
@@ -479,22 +290,10 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"id": "24b011c0",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[('e1', 'https://semantica.dev/entity/python#programming_language', 1),\n",
" ('e2', 'https://semantica.dev/entity/pytorch#framework', 0)]"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"from semantica.context import EntityLinker\n",
"\n",
@@ -511,22 +310,10 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"id": "a282de3a",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[EntityLink(source_entity_id='e1', target_entity_id='e_py', link_type='same_as', confidence=1.0, source=None, metadata={'similarity': 1.0}),\n",
" EntityLink(source_entity_id='e1', target_entity_id='e2', link_type='related_to', confidence=0.8, source=None, metadata={})]"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"linker.link_entities(\"e1\", \"e2\", link_type=\"related_to\", confidence=0.8)\n",
"linker.get_entity_links(\"e1\")[:2]"
@@ -554,22 +341,10 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": null,
"id": "3791d6c6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[('Python powers Semantica.', 'short_term', 1.0),\n",
" ('Python', 'graph:e_python', 0.5)]"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"from semantica.context import AgentMemory, ContextRetriever\n",
"\n",
@@ -594,21 +369,10 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": null,
"id": "896e7001",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'7_days'"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"from semantica.context.config import context_config\n",
"\n",
@@ -618,21 +382,10 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": null,
"id": "e925a2e0",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'graph': ['entities_relationships', 'conversations', 'hybrid', 'custom_demo']}"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"from semantica.context.methods import build_context_graph\n",
"from semantica.context.registry import method_registry\n",
@@ -650,21 +403,10 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": null,
"id": "21fa6cb3",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'nodes': [], 'edges': [], 'statistics': {'node_count': 0, 'edge_count': 0}}"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"build_context_graph(\n",
" entities=[{\"id\": \"e1\", \"text\": \"Python\", \"type\": \"PROGRAMMING_LANGUAGE\"}],\n",
+4 -20
View File
@@ -21,25 +21,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"import sys\n",
"import os\n",
@@ -53,7 +37,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -75,7 +59,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -0,0 +1,725 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Amazon Neptune Graph Store\n",
"\n",
"## Overview\n",
"\n",
"This notebook covers the Amazon Neptune Database integration in Semantica. Amazon Neptune is a fully managed graph database service that supports both property graphs (via OpenCypher/Gremlin) and RDF graphs (via SPARQL).\n",
"\n",
"### Key Features\n",
"\n",
"- **IAM Authentication**: Secure access using AWS SigV4 signatures via AuthManager\n",
"- **OpenCypher Support**: Query using standard OpenCypher syntax\n",
"- **Bolt Protocol**: Uses Neo4j Bolt driver for efficient binary communication\n",
"- **Native ~id Support**: Leverages Neptune's native element ID handling\n",
"- **Full CRUD Operations**: Create, read, update, delete nodes and relationships\n",
"- **Automatic Retry**: Built-in retry logic with exponential backoff for transient errors\n",
"\n",
"### Prerequisites\n",
"\n",
"- An Amazon Neptune Database cluster\n",
"- AWS credentials configured (boto3, environment variables, or IAM role)\n",
"- Network access to your Neptune cluster (VPC, security groups)\n",
"\n",
"#### Quick Setup with CloudFormation\n",
"\n",
"If you don't have a Neptune cluster, use the provided CloudFormation template to create one with a public endpoint and IAM authentication:\n",
"\n",
"```bash\n",
"# Deploy the Neptune stack (takes ~15-20 minutes)\n",
"aws cloudformation create-stack \\\n",
" --stack-name semantica-neptune \\\n",
" --template-body file://neptune-setup.yaml \\\n",
" --capabilities CAPABILITY_NAMED_IAM\n",
"\n",
"# Wait for stack creation to complete\n",
"aws cloudformation wait stack-create-complete --stack-name semantica-neptune\n",
"\n",
"# Get the outputs (endpoint, port, credentials)\n",
"aws cloudformation describe-stacks --stack-name semantica-neptune \\\n",
" --query 'Stacks[0].Outputs' --output table\n",
"```\n",
"\n",
"The template creates:\n",
"- VPC with public subnets and Internet Gateway\n",
"- Neptune cluster (`db.t3.medium`) with IAM authentication enabled\n",
"- IAM user with least-privilege access for OpenCypher queries\n",
"- Security group allowing Bolt protocol (port 8182) access\n",
"\n",
"> ⚠️ **Security Note**: This template creates an IAM User with static access keys for simplicity in demo/test environments. For production use, we recommend IAM Roles (EC2 instance roles, ECS task roles, Lambda execution roles) which provide temporary credentials that are automatically rotated. The secret access key in the Cloudformation outputs is provided in plaintext to simplify initial setup - in production, use AWS Secrets Manager.\n",
"\n",
"**Outputs:**\n",
"- `NeptuneEndpoint` - Cluster hostname (use as `NEPTUNE_ENDPOINT`)\n",
"- `NeptunePort` - 8182 (use as `NEPTUNE_PORT`)\n",
"- `AwsAccessKeyId` - IAM user access key (use as `AWS_ACCESS_KEY_ID`)\n",
"- `AwsSecretAccessKey` - IAM user secret key in **plaintext** (use as `AWS_SECRET_ACCESS_KEY`)\n",
"- `AwsRegion` - Deployment region (use as `AWS_REGION`)\n",
"\n",
"**Cleanup:**\n",
"```bash\n",
"aws cloudformation delete-stack --stack-name semantica-neptune\n",
"```\n",
"\n",
"**Estimated Monthly Cost (approximately 100-105 USD/month at 100% utilization):**\n",
"\n",
"| Resource | Cost (USD) |\n",
"| --- | --- |\n",
"| Neptune db.t3.medium instance | ~96/month (0.132/hr) |\n",
"| Storage (10 GB) | ~1/month |\n",
"| I/O requests | ~1-5/month |\n",
"| Public IPv4 address | ~3.60/month (0.005/hr) |\n",
"| VPC, subnets, route tables, Internet Gateway, IAM | No Additional Charge |\n",
"\n",
"> **Free Tier**: New Neptune users get 30 days free (750 hours of db.t3.medium, 10M I/Os, 1 GB storage). Delete the stack when not in use to avoid charges.\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Installation\n",
"\n",
"```bash\n",
"# Install Semantica with Neptune support\n",
"pip install semantica\n",
"\n",
"# Required dependencies (installed automatically)\n",
"pip install boto3 neo4j\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install semantica"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuration\n",
"\n",
"Set your Neptune cluster endpoint and AWS credentials. Replace the placeholder values with your actual configuration."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"# Neptune cluster configuration - REPLACE WITH YOUR VALUES\n",
"# (Get these from CloudFormation stack outputs)\n",
"os.environ[\"NEPTUNE_ENDPOINT\"] = \"your-cluster.us-east-1.neptune.amazonaws.com\"\n",
"os.environ[\"NEPTUNE_PORT\"] = \"8182\"\n",
"os.environ[\"AWS_REGION\"] = \"us-east-1\"\n",
"\n",
"# AWS credentials for IAM Authentication\n",
"# Option 1: IAM User (static credentials from CloudFormation template)\n",
"# os.environ[\"AWS_ACCESS_KEY_ID\"] = \"AKIA...\" # From AwsAccessKeyId output\n",
"# os.environ[\"AWS_SECRET_ACCESS_KEY\"] = \"...\" # From AwsSecretAccessKey output\n",
"# Note: No AWS_SESSION_TOKEN needed for IAM users\n",
"\n",
"# Option 2: IAM Role / Temporary credentials (e.g., STS AssumeRole, EC2 instance role)\n",
"# os.environ[\"AWS_ACCESS_KEY_ID\"] = \"ASIA...\" # Temporary access key\n",
"# os.environ[\"AWS_SECRET_ACCESS_KEY\"] = \"...\" # Temporary secret key\n",
"# os.environ[\"AWS_SESSION_TOKEN\"] = \"...\" # REQUIRED for temporary credentials\n",
"\n",
"print(f\"Neptune Endpoint: {os.environ.get('NEPTUNE_ENDPOINT')}\")\n",
"print(f\"AWS Region: {os.environ.get('AWS_REGION')}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1: Initialize Neptune Store\n",
"\n",
"Initialize a connection to your Amazon Neptune cluster with IAM authentication."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from semantica.graph_store import GraphStore\n",
"\n",
"# Option 1: Using GraphStore factory (recommended)\n",
"neptune_store = GraphStore(\n",
" backend=\"neptune\",\n",
" endpoint=os.environ.get(\"NEPTUNE_ENDPOINT\"),\n",
" port=int(os.environ.get(\"NEPTUNE_PORT\", 8182)),\n",
" region=os.environ.get(\"AWS_REGION\", \"us-east-1\"),\n",
" iam_auth=True,\n",
")\n",
"\n",
"# Connect to Neptune\n",
"neptune_store.connect()\n",
"print(\"Connected to Amazon Neptune!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Development/Testing Without IAM Auth\n",
"\n",
"For development or testing environments where IAM authentication is not required (e.g., Neptune notebooks or VPC-only access), you can disable IAM signing:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# For dev/test environments without IAM authentication\n",
"neptune_store_dev = GraphStore(\n",
" backend=\"neptune\",\n",
" endpoint=os.environ.get(\"NEPTUNE_ENDPOINT\"),\n",
" port=int(os.environ.get(\"NEPTUNE_PORT\", 8182)),\n",
" region=os.environ.get(\"AWS_REGION\", \"us-east-1\"),\n",
" iam_auth=False, # Disable IAM signing for dev/test\n",
")\n",
"neptune_store_dev.connect()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Authentication Options\n",
"\n",
"IAM Authentication (recommended for production) automatically uses the AWS credential chain:\n",
"1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)\n",
"2. AWS credentials file (~/.aws/credentials)\n",
"3. IAM role (for EC2, Lambda, ECS)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2: Node Operations\n",
"\n",
"### Creating Nodes\n",
"\n",
"Nodes represent entities in your graph. Each node can have:\n",
"- **ID**: A unique identifier (custom or auto-generated UUID)\n",
"- **Labels**: Categories/types (e.g., `Person`, `Company`)\n",
"- **Properties**: Key-value pairs (e.g., `{\"name\": \"Alice\", \"age\": 30}`)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Create a single node with custom ID (id in properties)\n",
"alice = neptune_store.create_node(\n",
" labels=[\"Person\"],\n",
" properties={\"id\": \"alice\", \"name\": \"Alice\", \"age\": 30, \"role\": \"Engineer\"}\n",
")\n",
"print(f\"Created node: {alice}\")\n",
"\n",
"# Create a node with auto-generated UUID (no id in properties)\n",
"bob = neptune_store.create_node(\n",
" labels=[\"Person\"],\n",
" properties={\"name\": \"Bob\", \"age\": 25, \"role\": \"Designer\"}\n",
")\n",
"print(f\"Created node with UUID: {bob['id']}\")\n",
"\n",
"# Create a company node with auto-generated ID\n",
"acme = neptune_store.create_node(\n",
" labels=[\"Company\"],\n",
" properties={\"name\": \"Acme Corp\", \"industry\": \"Technology\", \"founded\": 2010}\n",
")\n",
"print(f\"Created company: {acme}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating Multiple Nodes (Batch)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Batch create nodes for better performance\n",
"# Include 'id' in properties for custom IDs\n",
"nodes_data = [\n",
" {\"labels\": [\"Person\"], \"properties\": {\"id\": \"charlie\", \"name\": \"Charlie\", \"age\": 35}},\n",
" {\"labels\": [\"Person\"], \"properties\": {\"id\": \"diana\", \"name\": \"Diana\", \"age\": 28}},\n",
" {\"labels\": [\"Location\"], \"properties\": {\"name\": \"San Francisco\", \"state\": \"CA\"}},\n",
"]\n",
"\n",
"created_nodes = neptune_store.create_nodes(nodes_data)\n",
"print(f\"Created {len(created_nodes)} nodes in batch\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Retrieving Nodes"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get a specific node by ID\n",
"alice_node = neptune_store.get_node(node_id=\"alice\")\n",
"print(f\"Retrieved: {alice_node}\")\n",
"\n",
"# Get nodes by label\n",
"people = neptune_store.get_nodes(labels=[\"Person\"], limit=10)\n",
"print(f\"Found {len(people)} Person nodes:\")\n",
"for person in people:\n",
" print(f\" - {person.get('properties', {}).get('name')}\")\n",
"\n",
"# Get nodes by properties\n",
"engineers = neptune_store.get_nodes(\n",
" labels=[\"Person\"],\n",
" properties={\"role\": \"Engineer\"},\n",
" limit=5\n",
")\n",
"print(f\"Found {len(engineers)} engineers\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Updating Nodes"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Update node properties (merge mode - default)\n",
"updated_alice = neptune_store.update_node(\n",
" node_id=\"alice\",\n",
" properties={\"age\": 31, \"department\": \"AI Research\"},\n",
" merge=True\n",
")\n",
"print(f\"Updated Alice: {updated_alice}\")\n",
"\n",
"# Replace all properties (merge=False)\n",
"# WARNING: This removes properties not in the update\n",
"replaced = neptune_store.update_node(\n",
" node_id=\"charlie\",\n",
" properties={\"name\": \"Charlie\", \"age\": 36},\n",
" merge=False\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Deleting Nodes"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete a node (with detach=True to also delete relationships)\n",
"deleted = neptune_store.delete_node(node_id=\"diana\", detach=True)\n",
"print(f\"Deleted diana: {deleted}\")\n",
"\n",
"# Without detach (fails if node has relationships)\n",
"# neptune_store.delete_node(node_id=\"alice\", detach=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3: Relationship Operations\n",
"\n",
"### Creating Relationships\n",
"\n",
"Relationships connect nodes and represent connections between entities."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Create a relationship between Alice and Acme\n",
"works_at = neptune_store.create_relationship(\n",
" start_node_id=\"alice\",\n",
" end_node_id=acme[\"id\"],\n",
" rel_type=\"WORKS_AT\",\n",
" properties={\"since\": 2020, \"position\": \"Senior Engineer\"}\n",
")\n",
"print(f\"Created relationship: {works_at}\")\n",
"\n",
"# Create a KNOWS relationship between people\n",
"knows_rel = neptune_store.create_relationship(\n",
" start_node_id=\"alice\",\n",
" end_node_id=bob[\"id\"],\n",
" rel_type=\"KNOWS\",\n",
" properties={\"since\": 2019}\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Retrieving Relationships"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get all relationships for a node\n",
"alice_rels = neptune_store.get_relationships(node_id=\"alice\", direction=\"both\")\n",
"print(f\"Alice has {len(alice_rels)} relationships\")\n",
"\n",
"# Get outgoing relationships only\n",
"outgoing = neptune_store.get_relationships(node_id=\"alice\", direction=\"out\")\n",
"\n",
"# Filter by relationship type\n",
"works_rels = neptune_store.get_relationships(\n",
" node_id=\"alice\",\n",
" rel_type=\"WORKS_AT\",\n",
" direction=\"out\"\n",
")\n",
"print(f\"Alice's work relationships: {len(works_rels)}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Deleting Relationships"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete a specific relationship by ID\n",
"if works_at.get(\"id\"):\n",
" deleted = neptune_store.delete_relationship(rel_id=works_at[\"id\"])\n",
" print(f\"Deleted relationship: {deleted}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4: OpenCypher Queries\n",
"\n",
"Amazon Neptune supports OpenCypher queries via the Bolt protocol. Execute complex graph patterns using standard Cypher syntax."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Simple query\n",
"results = neptune_store.execute_query(\n",
" \"MATCH (p:Person) RETURN p.name, p.age ORDER BY p.age\"\n",
")\n",
"print(\"People in the graph:\")\n",
"for record in results.get(\"records\", []):\n",
" print(f\" - {record.get('p.name')}: {record.get('p.age')} years old\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Using parameters (safer and more efficient)\n",
"results = neptune_store.execute_query(\n",
" \"MATCH (p:Person) WHERE p.age > $min_age RETURN p.name, p.age\",\n",
" parameters={\"min_age\": 25}\n",
")\n",
"print(f\"People over 25: {len(results.get('records', []))}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Find relationships between nodes\n",
"results = neptune_store.execute_query(\"\"\"\n",
" MATCH (p:Person)-[r:WORKS_AT]->(c:Company)\n",
" RETURN p.name as employee, c.name as company, r.since as start_year\n",
"\"\"\")\n",
"for record in results.get(\"records\", []):\n",
" print(f\"{record['employee']} works at {record['company']} since {record['start_year']}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Count and aggregate\n",
"results = neptune_store.execute_query(\"\"\"\n",
" MATCH (p:Person)\n",
" RETURN count(p) as total, avg(p.age) as avg_age, max(p.age) as max_age\n",
"\"\"\")\n",
"stats = results.get(\"records\", [{}])[0]\n",
"print(f\"Total: {stats.get('total')}, Avg Age: {stats.get('avg_age'):.1f}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5: Graph Analytics\n",
"\n",
"### Get Neighbors\n",
"\n",
"Traverse the graph to find connected nodes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get immediate neighbors (depth=1)\n",
"neighbors = neptune_store.get_neighbors(\n",
" node_id=\"alice\",\n",
" direction=\"both\",\n",
" depth=1\n",
")\n",
"print(f\"Alice's direct neighbors: {len(neighbors)}\")\n",
"\n",
"# Get neighbors up to 2 hops away\n",
"extended = neptune_store.get_neighbors(\n",
" node_id=\"alice\",\n",
" direction=\"out\",\n",
" depth=2\n",
")\n",
"print(f\"Nodes within 2 hops: {len(extended)}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Shortest Path\n",
"\n",
"Find the shortest path between two nodes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Find shortest path\n",
"path = neptune_store.shortest_path(\n",
" start_node_id=\"alice\",\n",
" end_node_id=\"charlie\",\n",
" max_depth=5\n",
")\n",
"\n",
"if path:\n",
" print(\"Path found!\")\n",
" print(f\" Length: {path.get('length')}\")\n",
" print(f\" Nodes: {len(path.get('nodes', []))}\")\n",
" print(f\" Relationships: {len(path.get('relationships', []))}\")\n",
"else:\n",
" print(\"No path found between nodes\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 6: Graph Statistics\n",
"\n",
"Get comprehensive statistics about your graph."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get graph statistics\n",
"stats = neptune_store.get_stats()\n",
"\n",
"print(\"Graph Statistics:\")\n",
"print(f\" Total nodes: {stats.get('node_count', 'N/A')}\")\n",
"print(f\" Total relationships: {stats.get('relationship_count', 'N/A')}\")\n",
"\n",
"print(\"\\nNode labels:\")\n",
"for label, count in stats.get('label_counts', {}).items():\n",
" print(f\" - {label}: {count}\")\n",
"\n",
"print(\"\\nRelationship types:\")\n",
"for rel_type, count in stats.get('relationship_type_counts', {}).items():\n",
" print(f\" - {rel_type}: {count}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 7: Connection Management\n",
"\n",
"Always close connections when done to free resources."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Check connection status\n",
"status = neptune_store.get_status()\n",
"print(f\"Connection status: {status}\")\n",
"\n",
"# Close the connection\n",
"neptune_store.close()\n",
"print(\"Connection closed\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Neptune-Specific Considerations\n",
"\n",
"### Native Element IDs\n",
"\n",
"Neptune uses native `~id` for element identification. Include `id` in properties to set a custom ID:\n",
"\n",
"```python\n",
"# Create a node with custom ID (include 'id' in properties)\n",
"node = neptune_store.create_node(\n",
" labels=[\"Person\"],\n",
" properties={\"id\": \"my-custom-id\", \"name\": \"Test\"}\n",
")\n",
"\n",
"# Create a node with auto-generated UUID (omit 'id' from properties)\n",
"node = neptune_store.create_node(\n",
" labels=[\"Person\"],\n",
" properties={\"name\": \"Test\"}\n",
")\n",
"\n",
"# The ID is used in id() function calls internally:\n",
"# MATCH (n) WHERE id(n) = 'my-custom-id' RETURN n\n",
"```\n",
"\n",
"### OpenCypher Considerations\n",
"\n",
"Amazon Neptune Database's OpenCypher implementation has some differences from Neo4j:\n",
"\n",
"1. **No `shortestPath()` function**: Use variable-length path patterns or `allShortestPaths()`\n",
"2. **Labels syntax**: Use `labels(n)` function to retrieve node labels\n",
"3. **Property updates**: Use `SET n += {props}` for merge behavior\n",
"\n",
"For the complete OpenCypher specification supported by Amazon Neptune Database, see the [AWS documentation](https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-opencypher.html).\n",
"\n",
"### Amazon Neptune Analytics\n",
"\n",
"For analytical (OLAP) workloads such as graph algorithms, aggregations, and large-scale traversals, consider [Amazon Neptune Analytics](https://docs.aws.amazon.com/neptune-analytics/latest/userguide/what-is-neptune-analytics.html). Neptune Analytics complements Neptune Database by providing optimized performance for analytical queries while Neptune Database is optimized for transactional (OLTP) workloads.\n",
"\n",
"### Performance Tips\n",
"\n",
"1. **Use batch operations** for creating multiple nodes/relationships\n",
"2. **Use parameters** in queries to enable query caching\n",
"3. **Limit result sets** with `LIMIT` clause"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"This notebook covered the Amazon Neptune Graph Store integration:\n",
"\n",
"- **IAM Authentication**: Secure AWS SigV4 signing\n",
"- **CRUD Operations**: Full node and relationship management\n",
"- **OpenCypher Queries**: Standard graph query language\n",
"- **Graph Analytics**: Neighbors and shortest path algorithms\n",
"- **Statistics & Monitoring**: Graph metrics and status\n",
"\n",
"### Key Takeaways\n",
"\n",
"- Neptune uses native `~id` for element identification\n",
"- IAM authentication is recommended for production\n",
"- Bolt protocol provides efficient binary query interface\n",
"- Semantica abstracts Neptune-specific syntax differences\n",
"\n",
"### Next Steps\n",
"\n",
"- [Graph Store (Neo4j/FalkorDB)](09_Graph_Store.ipynb) - Compare with other backends\n",
"- [Building Knowledge Graphs](07_Building_Knowledge_Graphs.ipynb) - Build production KGs\n",
"- [Graph Analytics](10_Graph_Analytics.ipynb) - Advanced analytics algorithms"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.9.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
+228
View File
@@ -0,0 +1,228 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: >
Amazon Neptune cluster with public endpoint, IAM authentication, and least-privilege
IAM user for Semantica cookbook. Uses db.t3.medium (most cost-effective Neptune instance type).
Parameters:
EnvironmentName:
Type: String
Default: semantica-neptune
Description: Environment name prefix for resource naming
Resources:
# =============================================================================
# VPC & NETWORKING
# =============================================================================
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
EnableDnsHostnames: true
EnableDnsSupport: true
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-vpc
InternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-igw
InternetGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
InternetGatewayId: !Ref InternetGateway
VpcId: !Ref VPC
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [0, !GetAZs '']
CidrBlock: 10.0.1.0/24
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-public-subnet-1
PublicSubnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [1, !GetAZs '']
CidrBlock: 10.0.2.0/24
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-public-subnet-2
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-public-rt
DefaultPublicRoute:
Type: AWS::EC2::Route
DependsOn: InternetGatewayAttachment
Properties:
RouteTableId: !Ref PublicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
PublicSubnet1RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnet1
PublicSubnet2RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnet2
# =============================================================================
# SECURITY GROUP
# =============================================================================
NeptuneSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Sub ${EnvironmentName}-neptune-sg
GroupDescription: Security group for Neptune cluster - allows Bolt protocol access
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 8182
ToPort: 8182
CidrIp: 0.0.0.0/0
Description: Allow Bolt protocol access from anywhere
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Description: Allow all outbound traffic
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-neptune-sg
# =============================================================================
# NEPTUNE CLUSTER
# =============================================================================
NeptuneSubnetGroup:
Type: AWS::Neptune::DBSubnetGroup
Properties:
DBSubnetGroupDescription: Subnet group for Neptune cluster
DBSubnetGroupName: !Sub ${EnvironmentName}-subnet-group
SubnetIds:
- !Ref PublicSubnet1
- !Ref PublicSubnet2
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-subnet-group
NeptuneCluster:
Type: AWS::Neptune::DBCluster
Properties:
DBClusterIdentifier: !Sub ${EnvironmentName}-cluster
DBSubnetGroupName: !Ref NeptuneSubnetGroup
VpcSecurityGroupIds:
- !Ref NeptuneSecurityGroup
EngineVersion: '1.4.6.3'
IamAuthEnabled: true
StorageEncrypted: true
DeletionProtection: false
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-cluster
NeptuneInstance:
Type: AWS::Neptune::DBInstance
Properties:
DBInstanceIdentifier: !Sub ${EnvironmentName}-instance
DBInstanceClass: db.t3.medium
DBClusterIdentifier: !Ref NeptuneCluster
PubliclyAccessible: true
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-instance
# =============================================================================
# IAM USER WITH LEAST PRIVILEGES
# =============================================================================
NeptuneUser:
Type: AWS::IAM::User
Properties:
UserName: !Sub ${EnvironmentName}-user
Tags:
- Key: Name
Value: !Sub ${EnvironmentName}-user
NeptuneUserPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: !Sub ${EnvironmentName}-neptune-access
Users:
- !Ref NeptuneUser
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: NeptuneDataAccess
Effect: Allow
Action:
- neptune-db:connect
- neptune-db:ReadDataViaQuery
- neptune-db:WriteDataViaQuery
- neptune-db:DeleteDataViaQuery
Resource: !Sub
- arn:aws:neptune-db:${AWS::Region}:${AWS::AccountId}:${ClusterResourceId}/*
- ClusterResourceId: !GetAtt NeptuneCluster.ClusterResourceId
NeptuneUserAccessKey:
Type: AWS::IAM::AccessKey
Properties:
UserName: !Ref NeptuneUser
# =============================================================================
# OUTPUTS
# =============================================================================
Outputs:
NeptuneEndpoint:
Description: Neptune cluster endpoint (hostname only) - use as NEPTUNE_ENDPOINT
Value: !GetAtt NeptuneCluster.Endpoint
NeptunePort:
Description: Neptune cluster port - use as NEPTUNE_PORT
Value: !GetAtt NeptuneCluster.Port
AwsAccessKeyId:
Description: Access key ID for the Neptune IAM user - use as AWS_ACCESS_KEY_ID
Value: !Ref NeptuneUserAccessKey
AwsSecretAccessKey:
Description: Secret access key for the Neptune IAM user - use as AWS_SECRET_ACCESS_KEY
Value: !GetAtt NeptuneUserAccessKey.SecretAccessKey
AwsRegion:
Description: AWS region where Neptune is deployed - use as AWS_REGION
Value: !Ref AWS::Region
NeptuneClusterResourceId:
Description: Neptune cluster resource ID (for IAM policy reference)
Value: !GetAtt NeptuneCluster.ClusterResourceId
VpcId:
Description: VPC ID
Value: !Ref VPC
SecurityGroupId:
Description: Neptune security group ID
Value: !Ref NeptuneSecurityGroup
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because one or more lines are too long
@@ -57,32 +57,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Note: you may need to restart the kernel to use updated packages.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~gno (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~lotly (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
"WARNING: Ignoring invalid distribution ~ython-socketio (c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
]
}
],
"outputs": [],
"source": [
"%pip install -qU semantica networkx matplotlib plotly pandas faiss-cpu beautifulsoup4 groq sentence-transformers scikit-learn\n"
]
@@ -96,7 +73,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -120,23 +97,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Ingesting from 8 feed sources...\n",
"🧠 Semantica is ingesting: 429 Client Error: Too Many Requests for url: https://feeds.finance.yahoo.com/rss/2.0/headline ❌📥 (1.0s) | 🧠 Semantica is ingesting: Ingested 10 items |███████████████| 100.0% ✅📥 [2/8] Financial Times: 10 documents\n",
"🧠 Semantica is ingesting: Ingested 10 items |███████████████| 100.0% ✅📥 | 🧠 Semantica is ingesting: Ingested 30 items |███████████████| 100.0% ✅📥 [3/8] Bloomberg: 30 documents\n",
"🧠 Semantica is ingesting: 403 Client Error: Forbidden for url: https://www.marketwatch.com/404?origin=feeds ❌📥 (3.2s) | 🧠 Semantica is ingesting: Ingested 30 items |███████████████| 100.0% ✅📥 [5/8] Seeking Alpha: 30 documents\n",
"🧠 Semantica is ingesting: Ingested 30 items |███████████████| 100.0% ✅📥 | 🧠 Semantica is ingesting: Ingested 10 items |███████████████| 100.0% ✅📥 [6/8] Investing.com: 10 documents\n",
"🧠 Semantica is ingesting: 403 Client Error: Forbidden for url: https://www.fnlondon.com/rss ❌📥 (0.5s) | 🧠 Semantica is ingesting: Ingested 20 items |███████████████| 100.0% ✅📥 [8/8] Wall Street Journal: 20 documents\n",
"Ingested 100 documents\n"
]
}
],
"outputs": [],
"source": [
"from semantica.ingest import MCPIngestor, WebIngestor, FeedIngestor, FileIngestor\n",
"import os\n",
@@ -220,17 +183,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Loaded 6 seed data items for market foundation\n"
]
}
],
"outputs": [],
"source": [
"from semantica.seed import SeedDataManager\n",
"\n",
@@ -270,19 +225,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Parsing 100 documents...\n",
"🧠 Semantica is parsing: Document file not found: Jonathan Ferro, Lisa Abramowicz and Annmarie Hordern speak daily with leaders and decision makers from Wall Street to Washington and beyond. No other program better positions investors and executives for the trading day. (Source: Bloomberg) ❌🔍 (0.0s) | 🧠 Semantica is parsing: Document file not found: <p>A look at some of the years notable trades.<\\p> ❌🔍 (0.0s)ay. (Source: Bloomberg) ❌🔍 (0.0s) Parsed 50/100 documents...\n",
"🧠 Semantica is parsing: Document file not found: Maverick Natural Resources has operations in Texas and Oklahoma. ❌🔍 (0.0s) | 🧠 Semantica is parsing: Document file not found: The president has told Bank of America and JPMorgan Chase to stop cutting conservatives off from doing business. ❌🔍 (0.0s) Parsed 100/100 documents...\n"
]
}
],
"outputs": [],
"source": [
"from semantica.parse import DocumentParser\n",
"\n",
@@ -314,23 +259,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Normalizing 100 documents...\n",
"🧠 Semantica is parsing: Document file not found: The president has told Bank of America and JPMorgan Chase to stop cutting conservatives off from doing business. ❌🔍 (0.0s) | 🧠 Normalizing text 🔄🔧 (0.0s) Normalized 50/100 documents...\n",
"🧠 Semantica is parsing: Document file not found: The president has told Bank of America and JPMorgan Chase to stop cutting conservatives off from doing business. ❌🔍 (0.0s) | 🧠 Normalizing text 🔄🔧 (0.0s) Normalized 100/100 documents...\n",
"Chunking 100 documents...\n",
" Chunked 50/100 documents (50 chunks so far)\n",
" Chunked 100/100 documents (100 chunks so far)\n",
"Created 100 chunks from 100 documents\n"
]
}
],
"outputs": [],
"source": [
"from semantica.normalize import TextNormalizer\n",
"from semantica.split import TextSplitter\n",
@@ -380,23 +311,9 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Extracting entities from 100 chunks...\n",
"🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 4 entities using ml |███████████████| 100.0% ✅🎯 Processed 20/100 chunks (56 entities found)\n",
"🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 5 entities using ml |███████████████| 100.0% ✅🎯 Processed 40/100 chunks (160 entities found)\n",
"🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 Processed 60/100 chunks (182 entities found)\n",
"🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 1 entities using ml |███████████████| 100.0% ✅🎯 Processed 80/100 chunks (215 entities found)\n",
"🧠 Normalizing text 🔄🔧 (0.0s) | 🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 Processed 100/100 chunks (264 entities found)\n",
"Extracted 66 companies/organizations, 59 markets/locations, 8 prices, 24 metrics\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import NERExtractor\n",
"\n",
@@ -436,23 +353,9 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Extracting relationships from 100 chunks...\n",
"🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 4 relations using dependency |███████████████| 100.0% ✅🎯 Processed 20/100 chunks (49 relationships found)\n",
"🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 1 relations using dependency |███████████████| 100.0% ✅🎯 Processed 40/100 chunks (111 relationships found)\n",
"🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 0 relations |███████████████| 100.0% ✅🎯█████| 100.0% ✅🎯 Processed 60/100 chunks (121 relationships found)\n",
"🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 0 relations |███████████████| 100.0% ✅🎯█████| 100.0% ✅🎯 Processed 80/100 chunks (135 relationships found)\n",
"🧠 Semantica is extracting: Extracted 2 entities using ml |███████████████| 100.0% ✅🎯 | 🧠 Semantica is extracting: Extracted 3 relations using dependency |███████████████| 100.0% ✅🎯 Processed 100/100 chunks (175 relationships found)\n",
"Extracted 175 relationships\n"
]
}
],
"outputs": [],
"source": [
"from semantica.semantic_extract import RelationExtractor\n",
"\n",
@@ -500,19 +403,9 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detecting temporal conflicts in 264 entities...\n",
"🧠 Semantica is extracting: Extracted 3 relations using dependency |███████████████| 100.0% ✅🎯 | 🧠 Semantica is resolving: Detected 0 temporal conflicts |███████████████| 100.0% [165/193] ✅⚠️ (223.9/s)████████░░░| 85.5% [165/193] 🔄⚠️ (ETA: 0.1s | 225.1/s)Detected 0 temporal conflicts\n",
"No conflicts detected\n"
]
}
],
"outputs": [],
"source": [
"from semantica.conflicts import ConflictDetector, ConflictResolver\n",
"\n",
@@ -551,20 +444,9 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Converting 264 entities to dictionaries...\n",
"Resolving duplicates in 264 entities...\n",
"🧠 Semantica is deduplicating: Merging groups... 1/1 (remaining: 0) |███████████████| 100.0% [1/1] 🔄🔄 (16.4/s) | 🧠 Semantica is deduplicating: Building merged entity... (4/4, remaining: 0 steps) |███████████████| 100.0% [4/4] 🔄🔄 (258.3/s) 🔄🔄 (ETA: 0.0s | 374.6/s)333.9/s)Converting 36 resolved entities back to Entity objects...\n",
"Deduplicated 264 entities to 36 unique entities\n"
]
}
],
"outputs": [],
"source": [
"from semantica.kg import EntityResolver\n",
"from semantica.semantic_extract import Entity\n",
@@ -605,27 +487,9 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Building knowledge graph...\n",
"🧠 Semantica is deduplicating: Building merged entity... (4/4, remaining: 0 steps) |███████████████| 100.0% [4/4] 🔄🔄 (258.3/s) | 🧠 Semantica is building: Processing relationships... 175/175 |███████████████| 100.0% [175/175] 🔄🧠 (27889.8/s) 15937.0/s)Building graph structure...\n",
"✅ Graph structure built (0.00s)\n",
"🧠 Semantica is deduplicating: Building merged entity... (4/4, remaining: 0 steps) |███████████████| 100.0% [4/4] 🔄🔄 (258.3/s) | 🧠 Semantica is building: Processing relationships... 175/175 |███████████████| 100.0% [175/175] 🔄🧠 (27889.8/s)\n",
"============================================================\n",
"✅ Knowledge Graph Build Complete\n",
" Entities: 36\n",
" Relationships: 175\n",
" Total time: 0.86s\n",
"============================================================\n",
"Graph: 36 entities, 175 relationships\n"
]
}
],
"outputs": [],
"source": [
"from semantica.kg import GraphBuilder\n",
"\n",
@@ -653,25 +517,9 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Generating embeddings for 66 companies and 59 markets...\n",
"Generated 66 company embeddings and 59 market embeddings\n"
]
}
],
"outputs": [],
"source": [
"from semantica.embeddings import EmbeddingGenerator\n",
"\n",
@@ -699,25 +547,9 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Storing 66 company vectors and 59 market vectors...\n",
"🧠 Semantica is building: Processing relationships... 175/175 |███████████████| 100.0% [175/175] 🔄🧠 (27889.8/s) | 🧠 Semantica is indexing: Storing 66 vectors 🔄📊 (0.0s)Stored 66 company vectors and 59 market vectors\n"
]
}
],
"outputs": [],
"source": [
"from semantica.vector_store import VectorStore\n",
"\n",
@@ -746,21 +578,9 @@
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"🧠 Semantica is indexing: Storing 66 vectors 🔄📊 (0.0s) | 🧠 Semantica is building: Calculating degree centrality 🔄🧠 (0.0s)Graph analytics:\n",
" - Graph density: 0.000\n",
" - Central nodes (degree): 4\n",
" - Total entities: 36\n",
" - Total relationships: 175\n"
]
}
],
"outputs": [],
"source": [
"from semantica.kg import GraphAnalyzer, CentralityCalculator\n",
"\n",
@@ -806,75 +626,9 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"================================================================================\n",
"GraphRAG Query: What technology companies are in the market and what are their key relationships?\n",
"================================================================================\n",
"\n",
"🧠 Semantica is embedding: Generated embedding (dim: 128) |███████████████| 100.0% ✅💾 | 🧠 Semantica is processing: Ranking results... 🔄🔗 (0.0s)technology companies are in the market?... 🔄🔗 (0.0s)🔄🔗 (0.0s)s)ket?... 🔄🔗 (0.0s)"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Embedding generation failed: Text cannot be empty or whitespace-only\n",
"Using random fallback embedding\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"================================================================================\n",
"Generated Answer (with Multi-hop Reasoning):\n",
"================================================================================\n",
"Based on the retrieved context and reasoning paths, I can identify some technology companies and their key relationships. However, the context provided does not directly mention specific technology companies. \n",
"\n",
"One entity that is mentioned as a GPE (Geopolitical Entity) is Russia, which is related to the year through the reasoning path \"Russia --[be]--> the year\" (Path 1). This connection suggests that Russia is associated with a specific year, but the year is not explicitly stated.\n",
"\n",
"Another entity mentioned is Ukraine, which is also a GPE. It is related to the year through the reasoning path \"Ukraine --[be]--> the year\" (Path 1), similar to Russia. \n",
"\n",
"The only technology company mentioned in the context is Nvidia, but it is not related to any other entities or relationships in the context.\n",
"\n",
"To answer the question, I would say that there are limited technology companies mentioned in the context, and their relationships are not explicitly stated. However, based on the reasoning paths, we can infer that Russia and Ukraine are associated with a specific year.\n",
"\n",
"Key relationships:\n",
"\n",
"- Russia and Ukraine are associated with a specific year.\n",
"- The year is related to Russia and Ukraine through the reasoning path \"Russia --[be]--> the year\" (Path 1) and \"Ukraine --[be]--> the year\" (Path 1).\n",
"\n",
"Note: The context does not provide sufficient information to identify specific technology companies and their relationships.\n",
"\n",
"================================================================================\n",
"\n",
"Reasoning Details:\n",
"- Confidence: 0.636\n",
"- Sources: 15\n",
"- Reasoning Paths: 49\n",
"- Total entities in graph: 36\n",
"- Total relationships in graph: 175\n",
"\n",
"Top Sources:\n",
" 1. Score: 0.736\n",
" the final days of the year is a DATE. is outstriped by Warren Buffett. is rallyed by Masayoshi Sons. say Russia (GPE). buy Donald Trumps (PERSON), New York City (GPE). is selled by Jonathan Ferro....\n",
" 2. Score: 0.728\n",
" 2026 is a DATE. is looked by Bloombergs Skylar Montgomery Koning....\n",
" 3. Score: 0.651\n",
" the year is a DATE. is beed by Russia (GPE). sell the Christmas holiday, Ukraine (GPE), Pinnacle Group and 1 more. rally the Christmas holiday, Ukraine (GPE)....\n",
" 4. Score: 0.597\n",
" Ukraine is a GPE. position the trading day. is selled by the year (DATE). is broadened by Joe Weisenthal. is rallyed by the year (DATE)....\n",
" 5. Score: 0.591\n",
" Nvidia is a GPE....\n"
]
}
],
"outputs": [],
"source": [
"from semantica.context import AgentContext\n",
"from semantica.llms import Groq\n",
@@ -937,17 +691,9 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"🧠 Semantica is processing: Ranking results... 🔄🔗 (0.0s) | 🧠 Semantica is exporting: Exporting graph to json: financial_data_kg.json 🔄💾 (0.0s)Exported financial knowledge graph to JSON, GraphML, and CSV formats\n"
]
}
],
"outputs": [],
"source": [
"from semantica.export import GraphExporter\n",
"\n",
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large Load Diff
+33
View File
@@ -138,6 +138,39 @@ async for item in feed_processor.stream_items():
knowledge_graph.add_triplets(core.generate_triplets(semantics))
```
### 🦆 Docling Clear Code Example
High-accuracy document parsing with structural understanding:
```python
from semantica.parse import DoclingParser
# 1. Initialize DoclingParser
# Docling provides superior table extraction and structure understanding
# Requires: pip install docling
parser = DoclingParser(
enable_ocr=True, # Enable OCR for scanned documents
export_format="markdown" # Options: "markdown", "html", "json"
)
# 2. Parse a complex document
# Supports PDF, DOCX, PPTX, XLSX, HTML, and images
result = parser.parse("complex_invoice.pdf")
# 3. Access structured content
print(f"Content (Markdown):\n{result['full_text']}")
# 4. Extract and iterate over tables with high precision
for i, table in enumerate(result['tables']):
print(f"\nTable {i+1}:")
print(f"Headers: {table.get('headers', [])}")
print(f"Data rows: {len(table.get('rows', []))}")
# 5. Get document metadata
metadata = result['metadata']
print(f"\nMetadata: {metadata.get('title')} ({result.get('total_pages')} pages)")
```
### 📊 Structured Data Processing Module
Handle structured and semi-structured data formats:
View File
+8 -10
View File
@@ -7,29 +7,27 @@ How to cite Semantica in academic papers and research.
## BibTeX
```bibtex
@software{semantica2025,
@software{semantica2026,
title = {Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering},
author = {Hawksight AI},
year = {2025},
url = {https://github.com/Hawksight-AI/semantica},
version = {0.1.0},
year = {2025},
year = {2026},
url = {https://github.com/Hawksight-AI/semantica},
version = {0.2.5},
doi = {10.5281/zenodo.XXXXXXX}
}
```
### APA
Hawksight AI. (2025). *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering* (Version 0.1.0) [Computer software]. https://github.com/Hawksight-AI/semantica
Hawksight AI. (2026). *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering* (Version 0.2.5) [Computer software]. https://github.com/Hawksight-AI/semantica
### MLA
Hawksight AI. *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering*. Version 0.1.0, GitHub, 2025, https://github.com/Hawksight-AI/semantica.
Hawksight AI. *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering*. Version 0.2.5, GitHub, 2026, https://github.com/Hawksight-AI/semantica.
### Chicago
Hawksight AI. *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering*. Version 0.1.0. GitHub, 2025. https://github.com/Hawksight-AI/semantica.
Hawksight AI. *Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering*. Version 0.2.5. GitHub, 2026. https://github.com/Hawksight-AI/semantica.
### IEEE
Hawksight AI, "Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering," Version 0.1.0, GitHub, 2025. [Online]. Available: https://github.com/Hawksight-AI/semantica
Hawksight AI, "Semantica: An Open Source Framework for Semantic Layers and Knowledge Engineering," Version 0.2.5, GitHub, 2026. [Online]. Available: https://github.com/Hawksight-AI/semantica
---
@@ -37,7 +35,7 @@ Hawksight AI, "Semantica: An Open Source Framework for Semantic Layers and Knowl
If you use Semantica in your work:
> "This work uses Semantica (Hawksight AI, 2025), an open-source framework for semantic layer construction and knowledge engineering."
> "This work uses Semantica (Hawksight AI, 2026), an open-source framework for semantic layer construction and knowledge engineering."
---
+4 -3
View File
@@ -6,9 +6,10 @@
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.8+-blue.svg" alt="Python 3.8+"></a>
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
<a href="https://badge.fury.io/py/semantica"><img src="https://badge.fury.io/py/semantica.svg" alt="PyPI version"></a>
<a href="https://pepy.tech/project/semantica"><img src="https://pepy.tech/badge/semantica" alt="Downloads"></a>
<a href="https://pypi.org/project/semantica/"><img src="https://img.shields.io/pypi/dm/semantica" alt="Monthly Downloads"></a>
<a href="https://pepy.tech/project/semantica"><img src="https://static.pepy.tech/badge/semantica" alt="Total Downloads"></a>
<a href="https://semantica.readthedocs.io/"><img src="https://img.shields.io/badge/docs-latest-brightgreen.svg" alt="Documentation"></a>
<a href="https://discord.gg/semantica"><img src="https://img.shields.io/discord/semantica?color=7289da&label=discord" alt="Discord"></a>
<a href="https://discord.gg/pMHguUzG"><img src="https://img.shields.io/badge/Discord-Join%20Us-7289da?style=flat&logo=discord&logoColor=white" alt="Discord"></a>
<p><strong>Open Source Framework for Semantic Layer & Knowledge Engineering</strong></p>
@@ -16,7 +17,7 @@
<p><em>The missing fabric between raw data and AI engineering. A comprehensive open-source framework for building semantic layers and knowledge engineering systems that transform unstructured data into AI-ready knowledge — powering Knowledge Graph-Powered RAG (GraphRAG), AI Agents, Multi-Agent Systems, and AI applications with structured semantic knowledge.</em></p>
<p>🆓 <strong>100% Open Source</strong> • 📜 <strong>MIT Licensed</strong> • 🚀 <strong>Production Ready</strong> • 🌍 <strong>Community Driven</strong></p>
<p>🆓 <strong>100% Open Source</strong> • 📜 <strong>MIT Licensed</strong> • 🚀 <strong>Latest Version: 0.2.3</strong> • 🚀 <strong>Production Ready</strong> • 🌍 <strong>Community Driven</strong></p>
<p>
<a href="getting-started/" class="md-button md-button--primary">Get Started</a>
+13 -2
View File
@@ -28,6 +28,14 @@ pip install semantica
This installs Semantica with all core dependencies.
### GitHub Workaround
If you encounter issues with the PyPI version, you can install directly from the main branch:
```bash
pip install git+https://github.com/Hawksight-AI/semantica.git@main
```
!!! tip "Virtual Environment"
We recommend installing Semantica in a virtual environment to avoid dependency conflicts. Use `python -m venv venv` to create one, then activate it before installing.
@@ -36,12 +44,15 @@ This installs Semantica with all core dependencies.
Verify that Semantica is installed correctly:
```bash
python -c "import semantica; print(semantica.__version__)"
python -c "from semantica.parse import DoclingParser; DoclingParser(); print('✓ Semantica ready')"
```
!!! info "Windows PyTorch Note"
If you encounter PyTorch DLL errors on Windows, ensure you have the [Microsoft Visual C++ Redistributable](https://aka.ms/vs/17/release/vc_redist.x64.exe) installed. This is a common environment-specific issue with PyTorch on Windows and not a bug in Semantica.
Expected output:
```
0.1.0
✓ Semantica ready
```
You can also check the installation:
+107
View File
@@ -0,0 +1,107 @@
# Docling Integration
Semantica features a native integration with **Docling**, the powerful document parsing library that excels at extracting structured data from complex documents like PDFs, DOCX, and PPTX.
## Overview
Docling is integrated into Semantica's `parse` module via the `DoclingParser`. This allows you to seamlessly convert unstructured documents into semantic structures that can be indexed, searched, and analyzed within the Semantica framework.
- 📖 **Semantica Docling Integration Docs**: [Reference Guide](../reference/parse.md)
- 💻 **Semantica Docling Integration GitHub**: [Source Code](https://github.com/Hawksight-AI/semantica/blob/main/semantica/parse/docling_parser.py)
- 🧑🏽‍🍳 **Semantica Docling Integration Example**: [Docling Clear Code Example](../CodeExamples.md#docling-clear-code-example)
- 📦 **Semantica Docling Integration PyPI**: [Installation Guide](../installation.md)
---
## 📖 Integration Documentation
The `DoclingParser` provides a high-level interface for document processing. It supports:
* **Multi-format support**: PDF, DOCX, PPTX, HTML, and more.
* **Table Extraction**: High-fidelity table extraction with header detection.
* **OCR Support**: Built-in Optical Character Recognition for scanned documents.
* **Markdown Export**: Clean markdown output optimized for LLM consumption.
### Basic Usage
```python
from semantica.parse import DoclingParser
# Initialize with OCR enabled
parser = DoclingParser(enable_ocr=True)
# Parse a complex document
result = parser.parse("financial_report.pdf")
# Access the structured data
print(f"Content: {result['full_text'][:200]}...")
print(f"Found {len(result['tables'])} tables")
```
For more details, see the [Parse Reference](../reference/parse.md).
---
## 🧑🏽‍🍳 Integration Example
We provide a detailed cookbook and clear code examples to help you get started quickly.
### Docling Clear Code Example
```python
from semantica.parse import DoclingParser
import json
# 1. Initialize the Docling Parser with advanced config
parser = DoclingParser(
enable_ocr=True,
export_format="markdown"
)
# 2. Parse a complex document (PDF, DOCX, etc.)
result = parser.parse("complex_invoice.pdf")
# 3. Access the clean Markdown text
print(f"--- Document Content ---\n{result['full_text']}")
# 4. Iterate through extracted tables
for i, table in enumerate(result['tables']):
print(f"\nTable {i+1} headers: {table.get('headers', [])}")
# Access table rows as a list of lists
for row in table.get('rows', [])[:3]: # Print first 3 rows
print(f" Row: {row}")
# 5. Get document metadata
metadata = result['metadata']
print(f"\n--- Metadata ---\nTitle: {metadata.get('title')}")
print(f"Total Pages: {result.get('total_pages')}")
```
See more in our [Code Examples](../CodeExamples.md).
---
## 💻 GitHub Source
The integration is open-source and available on GitHub. You can explore the implementation, contribute improvements, or report issues.
- [docling_parser.py](https://github.com/Hawksight-AI/semantica/blob/main/semantica/parse/docling_parser.py) - The core implementation of the Docling integration.
---
## 📦 PyPI & Installation
Docling is an optional but highly recommended dependency for Semantica. You can install it along with Semantica or as a separate requirement.
### Install via Semantica
```bash
pip install semantica
```
### Install Docling manually
If you are working in a custom environment:
```bash
pip install docling
```
For full installation details, see the [Installation Guide](../installation.md).
+7 -1
View File
@@ -9,7 +9,13 @@ document.addEventListener("DOMContentLoaded", function () {
// Define versions
var versions = [
{ name: "0.1.0", url: "#", current: true }
{ name: "0.2.4", url: "#", current: true },
{ name: "0.2.3", url: "#", current: false },
{ name: "0.2.2", url: "#", current: false },
{ name: "0.2.1", url: "#", current: false },
{ name: "0.2.0", url: "#", current: false },
{ name: "0.1.1", url: "#", current: false },
{ name: "0.1.0", url: "#", current: false }
];
// Create the scrollable list
+1 -1
View File
@@ -9,7 +9,7 @@ Semantica is released under the MIT License.
```
MIT License
Copyright (c) 2025 Hawksight AI
Copyright (c) 2026 Hawksight AI
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+2
View File
@@ -213,6 +213,8 @@ These modules form the intelligence core—extracting meaning, building relation
- Multi-language support
- Semantic network extraction
- Coreference resolution
- **Auto-chunking**: Automatic text splitting for long documents
- **Robust Error Handling**: Standardized LLM provider diagnostics and retry logic
**Components:**
+29 -17
View File
@@ -36,32 +36,44 @@ See the [Installation Guide](installation.md) for detailed instructions.
Building a knowledge graph involves these key steps:
1. **Ingest** your documents using `` `FileIngestor` ``
2. **Parse** documents to extract text using `` `DocumentParser` ``
3. **Extract** entities and relationships using `` `NERExtractor` `` and `` `RelationExtractor` ``
4. **Build** the graph using `` `GraphBuilder` ``
5. **Generate** embeddings (optional) using `` `TextEmbedder` ``
1. **Ingest** your documents using `FileIngestor`
2. **Parse** documents to extract text using `DocumentParser` or `DoclingParser` (for enhanced layout support)
3. **Extract** entities and relationships using `NERExtractor` and `RelationExtractor`
4. **Build** the graph using `GraphBuilder`
5. **Generate** embeddings (optional) using `TextEmbedder`
**Quick Example:**
```python
from semantica.ingest import FileIngestor
from semantica.parse import DocumentParser
from semantica.parse import DocumentParser, DoclingParser
from semantica.semantic_extract import NERExtractor, RelationExtractor
from semantica.kg import GraphBuilder
# Build your first knowledge graph
# 1. Ingest document
ingestor = FileIngestor()
parser = DocumentParser()
ner = NERExtractor()
rel_extractor = RelationExtractor()
builder = GraphBuilder()
sources = ingestor.ingest("data/sample.pdf")
# Process document and build graph
doc = ingestor.ingest_file("document.pdf")
parsed = parser.parse_document("document.pdf")
entities = ner.extract_entities(parsed.get("full_text", ""))
relationships = rel_extractor.extract_relations(parsed.get("full_text", ""), entities=entities)
kg = builder.build_graph(entities=entities, relationships=relationships)
# 2. Parse (choose your parser)
# Option A: Standard parser
parser = DocumentParser()
parsed_content = parser.parse(sources[0])
# Option B: Enhanced Docling parser (recommended for complex tables)
# docling_parser = DoclingParser()
# parsed_content = docling_parser.parse(sources[0])
# 3. Extract entities and relations
ner = NERExtractor()
entities = ner.extract(parsed_content)
relations = RelationExtractor()
relationships = relations.extract(parsed_content, entities=entities)
# 4. Build graph
builder = GraphBuilder()
graph = builder.build(entities=entities, relationships=relationships)
print(f"Built knowledge graph with {len(graph.nodes)} nodes and {len(graph.edges)} edges")
```
**For complete step-by-step examples with detailed explanations, see:**
+938
View File
@@ -0,0 +1,938 @@
# Change Management API Reference
Comprehensive API documentation for the Enhanced Change Management module in Semantica.
## Overview
The `semantica.change_management` module provides enterprise-grade version control, audit trails, and compliance tracking for knowledge graphs and ontologies. It includes persistent storage backends, detailed change tracking, data integrity verification, and standardized metadata structures.
## Module Structure
```
semantica.change_management/
├── change_log.py # Standardized metadata structures
├── version_storage.py # Storage abstraction and implementations
├── managers.py # Enhanced version managers
├── ontology_version_manager.py # Ontology version management
└── change_management_usage.md # Usage guide
```
## Quick Import
```python
from semantica.change_management import (
# Metadata
ChangeLogEntry,
# Storage
VersionStorage,
InMemoryVersionStorage,
SQLiteVersionStorage,
# Utilities
compute_checksum,
verify_checksum,
# Version Managers
BaseVersionManager,
TemporalVersionManager,
OntologyVersionManager,
VersionManager,
OntologyVersion
)
```
---
## Core Classes
### ChangeLogEntry
Standardized metadata structure for version changes with validation.
#### Class Definition
```python
@dataclass
class ChangeLogEntry:
"""
Standardized change log entry with validation.
Attributes:
timestamp: ISO 8601 formatted timestamp
author: Email address of the change author
description: Change description (max 500 characters)
change_id: Optional ID linking to external systems
"""
timestamp: str
author: str
description: str
change_id: Optional[str] = None
```
#### Methods
##### `__post_init__()`
Validates all fields after initialization.
**Raises:**
- `ValidationError`: If any field validation fails
**Example:**
```python
entry = ChangeLogEntry(
timestamp="2024-01-30T12:00:00Z",
author="user@example.com",
description="Updated entity relationships",
change_id="TICKET-123"
)
```
##### `create_now(author, description, change_id=None)` (classmethod)
Creates a change log entry with the current timestamp.
**Parameters:**
- `author` (str): Email address of the change author
- `description` (str): Change description (max 500 characters)
- `change_id` (str, optional): ID linking to external systems
**Returns:**
- `ChangeLogEntry`: New instance with current timestamp
**Example:**
```python
entry = ChangeLogEntry.create_now(
author="developer@company.com",
description="Fixed entity resolution bug",
change_id="JIRA-1234"
)
```
#### Validation Rules
- **Timestamp**: Must be valid ISO 8601 format with 'T' separator
- **Author**: Must be valid email format (RFC 5322)
- **Description**: Maximum 500 characters
- **Change ID**: Optional, no validation
---
### VersionStorage
Abstract base class for storage implementations.
#### Class Definition
```python
class VersionStorage(ABC):
"""
Abstract base class for version storage backends.
Provides interface for saving, retrieving, and managing version snapshots.
"""
```
#### Abstract Methods
##### `save(snapshot)`
Save a version snapshot.
**Parameters:**
- `snapshot` (Dict[str, Any]): Version snapshot dictionary with metadata
**Raises:**
- `ValidationError`: If snapshot data is invalid
- `ProcessingError`: If save operation fails
**Example:**
```python
snapshot = {
"label": "v1.0",
"timestamp": "2024-01-30T12:00:00Z",
"author": "user@example.com",
"description": "Initial version",
"data": {...}
}
storage.save(snapshot)
```
##### `get(label)`
Retrieve a version snapshot by label.
**Parameters:**
- `label` (str): Version label to retrieve
**Returns:**
- `Optional[Dict[str, Any]]`: Snapshot dictionary or None if not found
**Example:**
```python
snapshot = storage.get("v1.0")
if snapshot:
print(f"Retrieved: {snapshot['label']}")
```
##### `list_all()`
List all version snapshots.
**Returns:**
- `List[Dict[str, Any]]`: List of snapshot metadata dictionaries
**Example:**
```python
versions = storage.list_all()
for v in versions:
print(f"{v['label']}: {v['description']}")
```
##### `exists(label)`
Check if a version exists.
**Parameters:**
- `label` (str): Version label to check
**Returns:**
- `bool`: True if version exists, False otherwise
**Example:**
```python
if storage.exists("v1.0"):
print("Version exists")
```
##### `delete(label)`
Delete a version snapshot.
**Parameters:**
- `label` (str): Version label to delete
**Returns:**
- `bool`: True if deleted, False if not found
**Example:**
```python
if storage.delete("v1.0"):
print("Version deleted")
```
---
### InMemoryVersionStorage
In-memory version storage implementation.
#### Class Definition
```python
class InMemoryVersionStorage(VersionStorage):
"""
In-memory version storage implementation.
Fast, volatile storage for development and testing.
Data is lost when the process ends.
"""
```
#### Constructor
```python
def __init__(self):
"""Initialize in-memory storage."""
```
**Example:**
```python
storage = InMemoryVersionStorage()
```
#### Performance Characteristics
- **Save**: 0.37-16ms (10-1000 entities)
- **Get**: 0.20-16ms (10-1000 entities)
- **List**: <0.03ms
- **Thread-safe**: Yes (uses RLock)
#### Use Cases
- Development and testing
- Temporary version tracking
- High-performance scenarios where persistence is not required
---
### SQLiteVersionStorage
SQLite-based persistent version storage implementation.
#### Class Definition
```python
class SQLiteVersionStorage(VersionStorage):
"""
SQLite-based persistent version storage implementation.
Provides persistence across process restarts with ACID guarantees.
"""
```
#### Constructor
```python
def __init__(self, storage_path: str):
"""
Initialize SQLite storage.
Args:
storage_path: Path to SQLite database file
"""
```
**Parameters:**
- `storage_path` (str): Path to SQLite database file (created if doesn't exist)
**Example:**
```python
storage = SQLiteVersionStorage("versions.db")
```
#### Database Schema
```sql
CREATE TABLE versions (
label TEXT PRIMARY KEY,
timestamp TEXT NOT NULL,
author TEXT NOT NULL,
description TEXT,
checksum TEXT,
snapshot_data TEXT NOT NULL,
created_at TEXT NOT NULL
)
```
#### Performance Characteristics
- **Save**: 7-25ms (10-1000 entities)
- **Get**: 2-8ms (10-1000 entities)
- **List**: 0.6-13ms
- **Thread-safe**: Yes (uses RLock)
- **ACID**: Full transaction support
#### Use Cases
- Production deployments
- Long-term version storage
- Compliance and audit requirements
- Multi-process environments
---
### BaseVersionManager
Abstract base class for version managers.
#### Class Definition
```python
class BaseVersionManager(ABC):
"""
Abstract base class for version managers.
Provides common functionality for version management across
different data types (knowledge graphs, ontologies, etc.).
"""
```
#### Constructor
```python
def __init__(self, storage_path: Optional[str] = None):
"""
Initialize base version manager.
Args:
storage_path: Path to SQLite database file for persistent storage.
If None, uses in-memory storage.
"""
```
**Parameters:**
- `storage_path` (str, optional): Path to SQLite database file
**Example:**
```python
# In-memory storage
manager = BaseVersionManager()
# Persistent storage
manager = BaseVersionManager(storage_path="versions.db")
```
#### Abstract Methods
##### `create_snapshot(data, version_label, author, description, **options)`
Create a versioned snapshot of the data.
**Parameters:**
- `data` (Any): Data to snapshot
- `version_label` (str): Version label
- `author` (str): Email address of the author
- `description` (str): Change description
- `**options`: Additional options
**Returns:**
- `Dict[str, Any]`: Snapshot with metadata and checksum
##### `compare_versions(version1, version2, **options)`
Compare two versions and return detailed differences.
**Parameters:**
- `version1` (Any): First version (label or snapshot)
- `version2` (Any): Second version (label or snapshot)
- `**options`: Comparison options
**Returns:**
- `Dict[str, Any]`: Detailed differences
#### Concrete Methods
##### `list_versions()`
List all version snapshots.
**Returns:**
- `List[Dict[str, Any]]`: List of version metadata
**Example:**
```python
versions = manager.list_versions()
for v in versions:
print(f"{v['label']}: {v['description']}")
```
##### `get_version(label)`
Retrieve specific version by label.
**Parameters:**
- `label` (str): Version label
**Returns:**
- `Optional[Dict[str, Any]]`: Version snapshot or None
**Example:**
```python
version = manager.get_version("v1.0")
```
##### `verify_checksum(snapshot)`
Verify data integrity using checksum.
**Parameters:**
- `snapshot` (Dict[str, Any]): Snapshot to verify
**Returns:**
- `bool`: True if checksum is valid
**Example:**
```python
is_valid = manager.verify_checksum(snapshot)
```
---
### TemporalVersionManager
Enhanced temporal version management engine for knowledge graphs.
#### Class Definition
```python
class TemporalVersionManager(BaseVersionManager):
"""
Enhanced temporal version management engine for knowledge graphs.
Features:
- Persistent snapshot storage (SQLite or in-memory)
- Detailed change tracking with entity-level diffs
- SHA-256 checksums for data integrity
- Standardized metadata with author attribution
- Version comparison with backward compatibility
- Input validation and security features
"""
```
#### Constructor
```python
def __init__(self, storage_path: Optional[str] = None, **config):
"""
Initialize enhanced temporal version manager.
Args:
storage_path: Path to SQLite database file for persistent storage.
If None, uses in-memory storage
**config: Additional configuration options
"""
```
**Parameters:**
- `storage_path` (str, optional): Path to SQLite database file
- `**config`: Additional configuration options
**Example:**
```python
# In-memory storage
manager = TemporalVersionManager()
# Persistent storage
manager = TemporalVersionManager(storage_path="kg_versions.db")
```
#### Methods
##### `create_snapshot(graph, version_label, author, description, **options)`
Create and store snapshot with checksum and metadata.
**Parameters:**
- `graph` (Dict[str, Any]): Knowledge graph dict with "entities" and "relationships"
- `version_label` (str): Version string (e.g., "v1.0")
- `author` (str): Email address of the change author
- `description` (str): Change description (max 500 chars)
- `**options`: Additional options
**Returns:**
- `Dict[str, Any]`: Snapshot with metadata and checksum
**Raises:**
- `ValidationError`: If input validation fails
- `ProcessingError`: If snapshot creation fails
**Example:**
```python
graph = {
"entities": [
{"id": "e1", "name": "Entity 1", "type": "Person"},
{"id": "e2", "name": "Entity 2", "type": "Organization"}
],
"relationships": [
{"source": "e1", "target": "e2", "type": "works_for"}
]
}
snapshot = manager.create_snapshot(
graph,
version_label="v1.0",
author="user@example.com",
description="Initial knowledge graph"
)
print(f"Created: {snapshot['label']}")
print(f"Checksum: {snapshot['checksum']}")
```
##### `compare_versions(version1, version2, **options)`
Compare two versions with detailed entity and relationship diffs.
**Parameters:**
- `version1` (Union[str, Dict]): First version (label or snapshot dict)
- `version2` (Union[str, Dict]): Second version (label or snapshot dict)
- `**options`: Comparison options
**Returns:**
- `Dict[str, Any]`: Detailed differences including:
- `summary`: Aggregate statistics
- `entity_changes`: Entity-level changes
- `relationship_changes`: Relationship-level changes
**Example:**
```python
diff = manager.compare_versions("v1.0", "v2.0")
print(f"Entities added: {diff['summary']['entities_added']}")
print(f"Entities modified: {diff['summary']['entities_modified']}")
print(f"Relationships added: {diff['summary']['relationships_added']}")
# Detailed entity changes
for entity_id, changes in diff['entity_changes'].items():
print(f"Entity {entity_id}: {changes['status']}")
if changes['status'] == 'modified':
print(f" Before: {changes['before']}")
print(f" After: {changes['after']}")
```
#### Performance
- **Snapshot Creation**: 1.40-54ms (50-2000 entities)
- **Version Retrieval**: 0.65-26ms (50-2000 entities)
- **Version Comparison**: 3.46-33ms (100-1000 entities)
- **Concurrent Throughput**: 500+ operations/second
---
### OntologyVersionManager
Enhanced version management for ontologies.
#### Class Definition
```python
class OntologyVersionManager(BaseVersionManager):
"""
Enhanced version management for ontologies.
Features:
- Persistent ontology snapshot storage
- Structural comparison (classes, properties, axioms)
- SHA-256 checksums for data integrity
- Standardized metadata with author attribution
"""
```
#### Constructor
```python
def __init__(self, storage_path: Optional[str] = None, **config):
"""
Initialize enhanced version manager for ontologies.
Args:
storage_path: Path to SQLite database file for persistent storage.
If None, uses in-memory storage
**config: Additional configuration options
"""
```
**Example:**
```python
manager = OntologyVersionManager(storage_path="ontology_versions.db")
```
#### Methods
##### `create_snapshot(ontology, version_label, author, description, **options)`
Create ontology snapshot with metadata.
**Parameters:**
- `ontology` (Dict[str, Any]): Ontology dict with structure information
- `version_label` (str): Version label
- `author` (str): Email address of the author
- `description` (str): Change description
- `**options`: Additional options
**Returns:**
- `Dict[str, Any]`: Ontology snapshot with metadata
**Example:**
```python
ontology = {
"uri": "https://example.com/ontology",
"version_info": {"version": "1.0", "date": "2024-01-30"},
"structure": {
"classes": ["Person", "Organization", "Location"],
"properties": ["name", "address", "email"],
"individuals": ["JohnDoe", "ACME_Corp"],
"axioms": ["Person hasAddress exactly 1 Location"]
}
}
snapshot = manager.create_snapshot(
ontology,
version_label="ont_v1.0",
author="architect@example.com",
description="Initial ontology design"
)
```
##### `compare_versions(version1, version2, **options)`
Compare ontology versions with structural analysis.
**Parameters:**
- `version1` (Union[str, Dict]): First version
- `version2` (Union[str, Dict]): Second version
- `**options`: Comparison options
**Returns:**
- `Dict[str, Any]`: Structural differences including:
- `classes_added`, `classes_removed`
- `properties_added`, `properties_removed`
- `individuals_added`, `individuals_removed`
- `axioms_added`, `axioms_removed`, `axioms_modified`
**Example:**
```python
diff = manager.compare_versions("ont_v1.0", "ont_v2.0")
print(f"Classes added: {diff['classes_added']}")
print(f"Properties added: {diff['properties_added']}")
print(f"Axioms modified: {diff['axioms_modified']}")
```
---
## Utility Functions
### compute_checksum
Compute SHA-256 checksum for data integrity.
#### Function Signature
```python
def compute_checksum(data: Dict[str, Any]) -> str:
"""
Compute SHA-256 checksum for data.
Args:
data: Dictionary to compute checksum for
Returns:
SHA-256 checksum as hexadecimal string
"""
```
**Parameters:**
- `data` (Dict[str, Any]): Dictionary to compute checksum for
**Returns:**
- `str`: SHA-256 checksum as hexadecimal string
**Example:**
```python
from semantica.change_management import compute_checksum
data = {"entities": [...], "relationships": [...]}
checksum = compute_checksum(data)
print(f"Checksum: {checksum}")
```
**Performance:** 1.29-110ms (100-10,000 entities)
---
### verify_checksum
Verify data integrity using stored checksum.
#### Function Signature
```python
def verify_checksum(snapshot: Dict[str, Any]) -> bool:
"""
Verify data integrity using checksum.
Args:
snapshot: Snapshot dictionary with 'checksum' field
Returns:
True if checksum is valid, False otherwise
"""
```
**Parameters:**
- `snapshot` (Dict[str, Any]): Snapshot dictionary with 'checksum' field
**Returns:**
- `bool`: True if checksum is valid, False otherwise
**Example:**
```python
from semantica.change_management import verify_checksum
snapshot = manager.get_version("v1.0")
is_valid = verify_checksum(snapshot)
if not is_valid:
print("WARNING: Data integrity compromised!")
```
**Performance:** 0.82-96ms (100-10,000 entities)
---
## Legacy Classes
### VersionManager
Original ontology version manager (moved from `semantica.ontology`).
#### Import
```python
from semantica.change_management import VersionManager, OntologyVersion
```
**Note:** This class is maintained for backward compatibility. New projects should use `OntologyVersionManager`.
---
## Error Handling
### ValidationError
Raised when input validation fails.
**Common Causes:**
- Invalid email format
- Description exceeds 500 characters
- Invalid ISO 8601 timestamp
- Missing required fields
**Example:**
```python
from semantica.utils.exceptions import ValidationError
try:
entry = ChangeLogEntry(
timestamp="invalid",
author="not-an-email",
description="x" * 501
)
except ValidationError as e:
print(f"Validation failed: {e}")
```
### ProcessingError
Raised when operations fail.
**Common Causes:**
- Database connection issues
- File system errors
- Concurrent modification conflicts
**Example:**
```python
from semantica.utils.exceptions import ProcessingError
try:
storage.save(snapshot)
except ProcessingError as e:
print(f"Save failed: {e}")
```
---
## Performance Considerations
### Benchmarks
Based on comprehensive performance testing:
| Component | Small (100) | Medium (500) | Large (2000) |
|-----------|-------------|--------------|--------------|
| Snapshot Creation | 2.33ms | 10.70ms | 54.23ms |
| Version Retrieval | 1.88ms | 7.33ms | 26.04ms |
| Version Comparison | 3.46ms | 17.39ms | 32.83ms |
| Checksum Compute | 1.29ms | 5.48ms | 22.15ms |
| SQLite Save | 8.69ms | 13.37ms | 25.33ms |
| InMemory Save | 1.18ms | 10.60ms | 14.11ms |
### Optimization Tips
1. **Use appropriate storage backend:**
- Development: `InMemoryVersionStorage`
- Production: `SQLiteVersionStorage`
2. **Batch operations when possible:**
```python
for data in batch:
manager.create_snapshot(data, ...)
```
3. **Implement retention policies:**
```python
# Delete old versions periodically
for version in old_versions:
storage.delete(version['label'])
```
4. **Use concurrent operations:**
- Thread-safe: 500+ operations/second
- No performance degradation under load
---
## Compliance Features
### HIPAA Compliance
- Complete audit trails with author attribution
- Timestamp tracking for all changes
- Data integrity verification with checksums
- Secure storage with access controls
### SOX Compliance
- Immutable change records
- Detailed change descriptions
- External system linking (change IDs)
- Comprehensive audit reports
### FDA 21 CFR Part 11
- Electronic signatures (author email)
- Data integrity verification
- Audit trail generation
- Tamper detection
---
## Examples
### Complete Healthcare Example
```python
from semantica.change_management import TemporalVersionManager
# Initialize with HIPAA-compliant storage
manager = TemporalVersionManager(storage_path="hipaa_records.db")
# Patient knowledge graph
patient_kg = {
"entities": [
{"id": "patient_001", "type": "Patient", "name": "Jane Smith"},
{"id": "diagnosis_001", "type": "Diagnosis", "code": "I10"}
],
"relationships": [
{"source": "patient_001", "target": "diagnosis_001", "type": "has_diagnosis"}
]
}
# Create versioned record
snapshot = manager.create_snapshot(
patient_kg,
"patient_001_v1.0",
"dr.williams@hospital.com",
"Initial diagnosis - Essential hypertension"
)
# Verify integrity
assert manager.verify_checksum(snapshot), "Data integrity check failed"
# Generate audit report
for version in manager.list_versions():
print(f"{version['timestamp']}: {version['label']} by {version['author']}")
```
---
## See Also
- **Usage Guide**: `semantica/change_management/change_management_usage.md`
- **Performance Tests**: `tests/change_management/test_performance.py`
- **CHANGELOG**: `CHANGELOG.md`
- **GitHub**: https://github.com/Hawksight-AI/semantica
+13 -1
View File
@@ -14,7 +14,7 @@ The LLM Providers module provides:
- **Easy Provider Switching**: Change providers without code changes
- **Multiple Model Support**: Access to 100+ LLMs through LiteLLM
- **GraphRAG Integration**: Seamless integration with GraphRAG reasoning features
- **Structured Output**: Generate structured data from LLM responses
- **Structured Output**: Generate structured data with robust JSON parsing and automatic retries (3 attempts).
### Why Use the LLM Providers Module?
@@ -237,6 +237,18 @@ except ProcessingError as e:
If a provider is not available (library not installed, API key missing), a `ProcessingError` is raised with a helpful message.
### Built-in Retries
The `generate_structured` method includes built-in retry logic for all providers:
- **Attempts**: 3
- **Wait Time**: 1 second base with exponential backoff
- **Triggers**: Network errors, rate limits, and malformed JSON responses.
### Robust JSON Parsing
`BaseProvider` uses an enhanced `_parse_json` method that can handle:
- Markdown code blocks (e.g., ```json ... ```)
- Extra text before or after the JSON object
- Common LLM formatting inconsistencies.
## Examples
### Basic Text Generation
+53 -5
View File
@@ -63,6 +63,29 @@ The module uses several inference algorithms:
---
## Ontology Ingestion
Ingest existing ontology files directly into usable data structures using `OntologyIngestor`.
**Function:** `ingest_ontology(source, method="file")`
| Argument | Description |
|----------|-------------|
| `source` | File path, directory path, or list of paths |
| `method` | Ingestion method (default: "file") |
**Example:**
```python
from semantica.ontology import ingest_ontology
# Ingest file
data = ingest_ontology("ontology.ttl")
# Ingest directory
dataset = ingest_ontology("ontologies/")
```
## Main Classes
### OntologyEngine
@@ -170,6 +193,17 @@ Manages external dependencies.
| `import_external_ontology(uri, ontology)` | Load and merge external ontology |
| `evaluate_alignment(uri, ontology)` | Assess alignment and compatibility |
### OntologyIngestor
Handles ingestion of existing ontologies from files and directories.
**Methods:**
| Method | Description |
|--------|-------------|
| `ingest_ontology(file_path)` | Ingest a single ontology file |
| `ingest_directory(directory_path)` | Recursively ingest ontology files from a directory |
---
## Unified Engine Examples
@@ -222,17 +256,31 @@ ontology:
```python
from semantica.ontology import OntologyEngine
from semantica.kg import KnowledgeGraph
from semantica.kg import GraphBuilder, GraphValidator
# 1. Generate Ontology from Sample Data
engine = OntologyEngine()
ontology = engine.from_data(sample_data)
# 2. Initialize KG with Ontology
kg = KnowledgeGraph(schema=ontology)
# 2. Extract schema for validation
schema = {
"entity_types": [c["name"] for c in ontology["classes"]],
"relationship_types": [p["name"] for p in ontology["properties"]]
}
# 3. Add Data (Validated against Ontology)
kg.add_entities(full_dataset) # Will raise error if violates schema
# 3. Initialize Validator and Builder
validator = GraphValidator(schema=schema, strict=True)
builder = GraphBuilder()
# 4. Build Knowledge Graph
kg = builder.build(full_dataset)
# 5. Validate against Ontology Schema
validation_result = validator.validate(kg)
if validation_result.is_valid:
print("Knowledge Graph matches the ontology schema!")
else:
print(f"Validation issues found: {validation_result.issues}")
```
---
+38
View File
@@ -87,6 +87,7 @@ The **Parse Module** extracts structured content from raw files and data sources
### Document Parsing
- **PDF**: `pdfplumber` for precise layout preservation, table extraction, and image handling. Fallback to `PyPDF2`.
- **Docling**: `docling` integration for advanced document structure understanding and high-accuracy table extraction across PDF, DOCX, PPTX, XLSX, and HTML.
- **Office (DOCX/PPTX/XLSX)**: XML-based parsing of OpenXML formats to extract text, styles, and properties.
- **OCR**: Tesseract-based optical character recognition for image-based PDFs and image files.
@@ -128,6 +129,43 @@ print(doc.get("metadata", {}).get("title"))
print(doc.get("full_text", "")[:100])
```
### DoclingParser
Specialized parser using Docling for enhanced layout understanding and table extraction. It is independent of `DocumentParser` and optimized for complex document structures.
**Methods:**
| Method | Description |
|--------|-------------|
| `` `parse(path)` `` | Full parse including text, tables, and metadata |
| `` `parse_batch(paths)` `` | Parse multiple documents in parallel |
| `` `extract_text(path)` `` | Extract clean text in Markdown/HTML/JSON format |
| `` `extract_tables(path)` `` | High-accuracy table extraction |
| `` `extract_metadata(path)` `` | Extract document properties |
**Configuration:**
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `export_format` | `str` | `"markdown"` | Output format: `"markdown"`, `"html"`, `"json"` |
| `enable_ocr` | `bool` | `False` | Enable OCR for scanned documents |
**Example:**
```python
from semantica.parse import DoclingParser
parser = DoclingParser()
result = parser.parse("complex_table.pdf")
# Access high-accuracy tables
for table in result["tables"]:
print(table["headers"])
# Get markdown representation
print(result["full_text"])
```
### WebParser
Parses web content.
+13 -8
View File
@@ -190,8 +190,8 @@ if proof:
### Knowledge Graph Enrichment
```python
from semantica.reasoning import Reasoner, Rule
from semantica.kg import KnowledgeGraph
from semantica.reasoning import Reasoner
from semantica.kg import GraphBuilder
# 1. Define Rules
rules = [
@@ -199,14 +199,19 @@ rules = [
"IF Ancestor(?x, ?y) AND Ancestor(?y, ?z) THEN Ancestor(?x, ?z)"
]
# 2. Load Graph and Run Inference
kg = KnowledgeGraph()
reasoner = Reasoner()
inferred = reasoner.infer_facts(kg.get_all_triplets(), rules)
# 2. Build Graph and Run Inference
builder = GraphBuilder()
kg = builder.build(sources=data)
# 3. Update Graph
reasoner = Reasoner()
# Infer new facts from entities and relationships
inferred = reasoner.infer_facts(kg["entities"] + kg["relationships"], rules)
# 3. Update Graph with Inferred Facts
for fact_str in inferred:
kg.add_fact_from_string(fact_str)
# Add new inferred facts back to the graph
# For a production app, you'd parse these into entities/relationships
kg["entities"].append({"type": "InferredFact", "name": fact_str})
```
---
-1
View File
@@ -189,7 +189,6 @@ seed:
```python
from semantica.seed import SeedDataManager
from semantica.ingest import Ingestor
from semantica.kg import KnowledgeGraph
# 1. Load Foundation (Seed)
seed_manager = SeedDataManager()
+59 -12
View File
@@ -23,7 +23,7 @@ The **Semantic Extract Module** extracts structured information from unstructure
- **High Accuracy**: LLM-based extraction for complex schemas
- **Flexible Configuration**: Customize extraction for your domain
- **Confidence Scores**: Get confidence scores for all extractions
- **Batch Processing**: Efficient batch processing for large datasets
- **Batch Processing**: Efficient parallel batch processing for large datasets
- **Coreference Resolution**: Resolve pronouns to their entity references
### How It Works
@@ -184,14 +184,15 @@ Core entity extraction implementation used by notebooks and lower-level integrat
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `method` | str or list | `"ml"` | Method(s): "ml", "llm", "pattern", "regex", "huggingface" |
| `**config` | dict | `{}` | Method-specific config (e.g., `model`, `provider`) |
| `entity_types` | list | `None` | Filter for specific entity types |
| `**config` | dict | `{}` | Method-specific config (e.g., `model`, `aggregation_strategy`, `device`) |
**Methods:**
| Method | Description |
|--------|-------------|
| `extract(text)` | Alias for `extract_entities`. Get list of entities. |
| `extract_entities(text)` | Get list of entities |
| `extract(text, pipeline_id=None, **kwargs)` | Alias for `extract_entities`. Supports `max_workers`. |
| `extract_entities(text, pipeline_id=None, **kwargs)` | Get list of entities. Supports `max_workers`. |
**Example:**
@@ -202,11 +203,12 @@ from semantica.semantic_extract import NERExtractor
extractor = NERExtractor(method="ml", model="en_core_web_trf")
entities = extractor.extract("Elon Musk leads SpaceX.")
# 2. LLM (OpenAI/Gemini/etc)
# 2. LLM (OpenAI/Gemini/Groq/etc)
extractor = NERExtractor(
method="llm",
provider="openai",
model="gpt-4",
provider="groq",
model="llama-3.3-70b-versatile",
max_tokens=2048, # Increased output limit
temperature=0.0
)
@@ -226,17 +228,19 @@ Extracts relationships between entities.
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `method` | str | `"dependency"` | Method: "dependency", "pattern", "cooccurrence", "huggingface", "llm" |
| `relation_types` | list | `None` | Specific relation types to extract |
| `bidirectional` | bool | `False` | Extract bidirectional relations |
| `confidence_threshold` | float | `0.6` | Minimum confidence score |
| `max_distance` | int | `50` | Max token distance between entities |
| `**config` | dict | `{}` | Method-specific config (e.g., `model`, `device` for HuggingFace) |
**Methods:**
| Method | Description |
|--------|-------------|
| `extract(text, entities)` | Alias for `extract_relations`. Find links. |
| `extract_relations(text, entities)` | Find links |
| `extract(text, entities, pipeline_id=None, **kwargs)` | Alias for `extract_relations`. Supports `max_workers`. |
| `extract_relations(text, entities, pipeline_id=None, **kwargs)` | Find links. Supports `max_workers`. |
**Example:**
@@ -251,7 +255,7 @@ entities = ner.extract_entities(text)
# Basic relation extraction
rel_extractor = RelationExtractor()
relations = rel_extractor.extract(text, entities=entities)
# [Relation(source="Elon Musk", target="SpaceX", type="founded")]
# [Relation(subject="Elon Musk", predicate="founded", object="SpaceX")]
# With configuration
rel_extractor = RelationExtractor(
@@ -306,6 +310,7 @@ Identifies events with temporal information and participants.
| `extract_participants` | bool | `True` | Extract event participants |
| `extract_location` | bool | `True` | Extract event locations |
| `extract_time` | bool | `True` | Extract temporal information |
| `max_workers` | int | `1` | Threads for parallel batch processing |
**Methods:**
@@ -334,15 +339,18 @@ Extracts RDF triplets (Subject-Predicate-Object).
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `method` | str | `"pattern"` | Extraction method ("pattern", "rules", "huggingface", "llm") |
| `triplet_types` | list | `None` | Specific triplet types/predicates to extract |
| `include_temporal` | bool | `False` | Include time information |
| `include_provenance` | bool | `False` | Track source sentences |
| `method` | str | `"pattern"` | Extraction method ("pattern", "rules", "huggingface", "llm") |
| `**kwargs` | dict | `{}` | Configuration options (e.g., `model`, `device`) |
**Methods:**
| Method | Description |
|--------|-------------|
| `extract_triplets(text)` | Get (S, P, O) tuples |
| `extract(text, entities=None, relations=None, pipeline_id=None, **kwargs)` | Alias for `extract_triplets`. Supports `max_workers`. |
| `extract_triplets(text, entities=None, relations=None, pipeline_id=None, **kwargs)` | Get (S, P, O) tuples. Supports `max_workers`. |
**Example:**
@@ -367,6 +375,7 @@ Extracts structured semantic networks with nodes and edges.
|-----------|------|---------|-------------|
| `ner_method` | str | `None` | Method for node extraction |
| `relation_method` | str | `None` | Method for edge extraction |
| `max_workers` | int | `1` | Threads for parallel batch processing |
| `**config` | dict | `{}` | Configuration for underlying extractors |
**Methods:**
@@ -420,6 +429,44 @@ enhanced_entities = extractor.enhance_entities(text, entities)
---
## Batch Processing & Provenance
All extractors support batch processing for high-throughput extraction. You can pass a list of strings or a list of dictionaries (with `content` and `id` keys).
**Features:**
- **Progress Tracking**: Automatically shows a progress bar for large batches.
- **Provenance Metadata**: Each extracted item includes `batch_index` and `document_id` in its `metadata`.
```python
from semantica.semantic_extract import NERExtractor
documents = [
{"id": "doc_1", "content": "Apple Inc. was founded by Steve Jobs."},
{"id": "doc_2", "content": "Microsoft Corporation was founded by Bill Gates."}
]
extractor = NERExtractor()
batch_results = extractor.extract(documents)
for i, doc_entities in enumerate(batch_results):
print(f"Document {i} entities:")
for entity in doc_entities:
print(f" - {entity.text} ({entity.label})")
print(f" Provenance: Batch Index {entity.metadata['batch_index']}, Doc ID {entity.metadata.get('document_id')}")
```
## Robust Extraction Fallbacks
The framework implements robust fallback chains to prevent empty results when primary methods fail (e.g., due to model unavailability or obscure text).
- **NER**: `ML/LLM` -> `Pattern` -> `Last Resort` (Capitalized Words)
- **Relation**: `Primary` -> `Pattern` -> `Last Resort` (Adjacency)
- **Triplet**: `Primary` -> `Relation-to-Triplet` -> `Pattern`
This ensures that you almost always get *some* structured data, even if it requires falling back to simpler heuristics.
---
## Usage Examples
```python
+19 -7
View File
@@ -1,12 +1,12 @@
# Vector Store
> **Unified vector database interface supporting FAISS, Weaviate, Qdrant, and Milvus with Hybrid Search.**
> **Unified vector database interface supporting FAISS, Weaviate, Qdrant, Pinecone, and Milvus with Hybrid Search.**
---
## 🎯 Overview
The **Vector Store Module** provides a unified interface for storing and searching vector embeddings. It supports multiple backends (FAISS, Weaviate, Qdrant, Milvus) and enables semantic search, RAG, and similarity matching.
The **Vector Store Module** provides a unified interface for storing and searching vector embeddings. It supports multiple backends (FAISS, Weaviate, Qdrant, Pinecone, Milvus) and enables semantic search, RAG, and similarity matching.
### What is a Vector Store?
@@ -18,7 +18,7 @@ A **vector store** is a database optimized for storing and searching high-dimens
### Why Use the Vector Store Module?
- **Multiple Backends**: Switch between FAISS (local), Weaviate, Qdrant, and Milvus
- **Multiple Backends**: Switch between FAISS (local), Weaviate, Qdrant, Pinecone, and Milvus
- **Unified Interface**: Same API regardless of backend
- **Hybrid Search**: Combine vector similarity with metadata filtering
- **Performance**: Optimized for high-throughput search operations
@@ -38,8 +38,8 @@ A **vector store** is a database optimized for storing and searching high-dimens
- :material-database:{ .lg .middle } **Multi-Backend Support**
---
Seamlessly switch between FAISS (Local), Weaviate, Qdrant, and Milvus
Seamlessly switch between FAISS (Local), Weaviate, Qdrant, Pinecone, and Milvus
- :material-magnify-plus:{ .lg .middle } **Hybrid Search**
@@ -112,6 +112,8 @@ The main facade for all vector operations.
| Method | Description |
|--------|-------------|
| `store_vectors(vectors, metadata)` | Store embeddings |
| `add_documents(documents, metadata, batch_size, parallel)` | **(New)** Store documents with automatic embedding generation and parallelization |
| `embed_batch(texts)` | **(New)** Generate embeddings for a batch of texts |
| `search(query, k)` | Semantic search |
| `delete(ids)` | Remove vectors |
@@ -120,15 +122,24 @@ The main facade for all vector operations.
```python
from semantica.vector_store import VectorStore
# Initialize (defaults to FAISS)
# Initialize (defaults to FAISS, parallel enabled by default with 6 workers)
store = VectorStore(backend="faiss", dimension=1536)
# Store
# 1. Store pre-computed vectors
ids = store.store_vectors(
vectors=[[0.1, 0.2, ...], ...],
metadata=[{"text": "Hello"}, ...]
)
# 2. Store raw documents (High Performance)
# Automatically handles embedding generation in parallel batches (uses default 6 workers)
ids = store.add_documents(
documents=["Doc 1", "Doc 2", ...],
metadata=[{"id": 1}, {"id": 2}, ...],
batch_size=32,
parallel=True
)
# Search
results = store.search(query_vector=[0.1, 0.2, ...], k=5)
```
@@ -771,6 +782,7 @@ print(f"Context: {context}")
---
## See Also
- [High-Performance Usage Guide](../vector_store_usage.md) - **(New)** Parallel ingestion and batching guide
- [Embeddings Module](embeddings.md) - Generates the vectors
- [Context Module](context.md) - Uses vector store for memory
- [Ingest Module](ingest.md) - Source of data
+4 -4
View File
@@ -253,12 +253,12 @@ visualization:
### Exploratory Data Analysis (EDA)
```python
from semantica.ingest import Ingestor
from semantica.kg import KnowledgeGraph
from semantica.kg import GraphBuilder
from semantica.visualization import KGVisualizer, AnalyticsVisualizer
# 1. Load Data
kg = KnowledgeGraph.load("my_graph")
# 1. Build Knowledge Graph
builder = GraphBuilder()
kg = builder.build(sources=sample_data)
# 2. Visualize Structure
kg_viz = KGVisualizer()
+101
View File
@@ -0,0 +1,101 @@
# High-Performance Vector Store Usage
This guide demonstrates how to leverage the new high-performance features of the Semantica Vector Store, specifically designed for efficient batch processing and parallel ingestion of large document sets.
## 🚀 Key Features
- **Parallel Ingestion**: Utilize multi-threading to embed and store documents concurrently.
- **Batch Processing**: Automatically group documents into batches to minimize overhead.
- **Unified API**: A single `add_documents` method handles embedding generation and storage.
---
## ⚡ Quick Start: Parallel Ingestion
The fastest way to ingest documents is using the `add_documents` method. Parallelization is enabled by default with optimized settings (6 workers).
```python
from semantica.vector_store import VectorStore
import time
store = VectorStore(
backend="faiss",
dimension=768,
)
documents = [f"This is document number {i} with some content." for i in range(1000)]
metadata = [{"source": "generated", "id": i} for i in range(1000)]
start_time = time.time()
ids = store.add_documents(
documents=documents,
metadata=metadata,
batch_size=64,
parallel=True,
)
print(f"Ingested {len(ids)} documents in {time.time() - start_time:.2f}s")
```
---
## 📊 Performance Comparison
### Old Method (Sequential Loop)
*Slower due to sequential processing and overhead per single item.*
```python
for doc in documents:
emb = embedder.generate(doc)
store.store_vectors([emb], [{"text": doc}])
```
### New Method (Parallel Batching)
*Significantly faster (3x-10x) by utilizing thread pools and batch operations.*
```python
store.add_documents(documents, parallel=True)
```
---
## 🛠 Configuration & Tuning
### `max_workers`
Controls the number of concurrent threads used for embedding generation.
- **Default**: 6 (Optimized for most systems)
- **Recommendation**: You generally don't need to change this. If you have very high core counts or specific throughput needs, you can override it.
```python
store = VectorStore(max_workers=16)
```
### `batch_size`
Controls how many documents are processed in a single chunk.
- **Default**: 32
- **Recommendation**:
- **Local Models**: 32-64 usually works well.
- **API Models (OpenAI, etc.)**: Larger batches (e.g., 100-200) can reduce network latency overhead.
```python
store.add_documents(documents, batch_size=100)
```
---
## 🧩 Advanced: Manual Batch Embedding
If you need the embeddings without storing them immediately, use `embed_batch`.
```python
vectors = store.embed_batch(
texts=documents[:100],
)
print(f"Generated {len(vectors)} vectors")
```
## ⚠️ Best Practices
1. **Metadata Consistency**: Ensure your `metadata` list has the same length as your `documents` list.
2. **Error Handling**: The `add_documents` method will propagate exceptions if embedding fails. Ensure your data is clean.
3. **Memory Usage**: Very large `batch_size` combined with high `max_workers` can increase memory usage. Monitor your system resources.
+147
View File
@@ -0,0 +1,147 @@
"""
HuggingFace Local Model Usage Demo (Bring Your Own Model)
This script demonstrates how to use the 'semantica' library with local HuggingFace models
for Named Entity Recognition (NER), Relation Extraction (RE), and Triplet Extraction.
Prerequisites:
pip install transformers torch
Usage:
python examples/huggingface_demo.py
"""
import sys
import os
# Add project root to path (for running from this dir)
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from semantica.semantic_extract import NERExtractor, RelationExtractor, TripletExtractor, Entity
def demo_ner():
print("\n" + "="*50)
print("NER Demo: Bring Your Own Model (BYOM)")
print("="*50)
# 1. Initialize NERExtractor with HuggingFace method and a specific model
# Common models: "dslim/bert-base-NER", "dbmdz/bert-large-cased-finetuned-conll03-english"
model_name = "dslim/bert-base-NER"
print(f"Initializing NERExtractor with model: {model_name}...")
extractor = NERExtractor(
method="huggingface",
model=model_name,
device="cpu" # Use "cuda" for GPU
)
text = "Steve Jobs founded Apple Inc. in Cupertino, California on April 1, 1976."
print(f"\nInput text: {text}")
try:
# Note: This will download the model if not cached (approx 400MB)
print("Extracting entities (this may take a moment on first run)...")
entities = extractor.extract_entities(text)
print(f"\nExtracted {len(entities)} entities:")
for ent in entities:
print(f" - {ent.text:20} | Type: {ent.label:10} | Conf: {ent.confidence:.2f}")
except Exception as e:
print(f"Extraction failed (missing dependencies?): {e}")
def demo_relation():
print("\n" + "="*50)
print("Relation Extraction Demo: Local Model")
print("="*50)
# 1. Initialize RelationExtractor
# Note: Relation extraction usually requires a SequenceClassification model
# trained on relation datasets (e.g., TACRED, SemEval).
# For demo purposes, we'll use a generic placeholder or a widely used one.
model_name = "semantica/relation-model-v1" # This is hypothetical; replace with real model
print(f"Initializing RelationExtractor with method='huggingface'...")
extractor = RelationExtractor(
method="huggingface",
model=model_name,
device="cpu"
)
text = "Steve Jobs founded Apple Inc."
# Pre-defined entities are usually required for relation extraction
entities = [
Entity(text="Steve Jobs", label="PERSON", start_char=0, end_char=10),
Entity(text="Apple Inc.", label="ORG", start_char=19, end_char=29)
]
print(f"\nInput text: {text}")
print(f"Entities: {[e.text for e in entities]}")
try:
print("Extracting relations...")
# Note: This will fail if the model doesn't exist on HF Hub.
# In a real scenario, use a valid model ID like "some-user/bert-relation-extraction"
# For this demo, we just show the call structure.
relations = extractor.extract_relations(text, entities)
print(f"\nExtracted {len(relations)} relations:")
for rel in relations:
print(f" - {rel.subject.text} --[{rel.predicate}]--> {rel.object.text} (Conf: {rel.confidence:.2f})")
except Exception as e:
print(f"Note: Relation extraction mock run (model download might fail or be skipped): {e}")
def demo_triplet():
print("\n" + "="*50)
print("Triplet Extraction Demo: REBEL (Seq2Seq)")
print("="*50)
# 1. Initialize TripletExtractor with REBEL model
# REBEL is a popular model for end-to-end triplet extraction
model_name = "Babelscape/rebel-large"
print(f"Initializing TripletExtractor with model: {model_name}...")
extractor = TripletExtractor(
method="huggingface",
model=model_name,
device="cpu"
)
text = "Apple was founded by Steve Jobs in 1976."
print(f"\nInput text: {text}")
try:
print("Extracting triplets (this may take a moment)...")
triplets = extractor.extract_triplets(text)
print(f"\nExtracted {len(triplets)} triplets:")
for triplet in triplets:
print(f" - ({triplet.subject}, {triplet.predicate}, {triplet.object})")
except Exception as e:
print(f"Extraction failed (missing dependencies?): {e}")
if __name__ == "__main__":
print("Starting Semantica HuggingFace Usage Demo...")
print("Note: This script attempts to download models from Hugging Face Hub.")
print("Ensure you have an internet connection and 'transformers' installed.")
# Run demos
# We wrap in try-except to ensure the script doesn't crash the whole session if one fails
try:
demo_ner()
except Exception as e:
print(f"NER Demo Error: {e}")
try:
demo_relation()
except Exception as e:
print(f"Relation Demo Error: {e}")
try:
demo_triplet()
except Exception as e:
print(f"Triplet Demo Error: {e}")
+3 -1
View File
@@ -7,7 +7,7 @@ repo_name: Hawksight-AI/semantica
edit_uri: edit/main/docs/
# Copyright
copyright: Copyright &copy; 2025 Hawksight AI
copyright: Copyright &copy; 2026 Hawksight AI
# Theme Configuration
theme:
@@ -139,6 +139,8 @@ nav:
- examples.md
- Code Examples: CodeExamples.md
- learning-more.md
- Integrations:
- Docling: integrations/docling.md
- Cookbook: cookbook.md
- Resources:
- community-projects.md
+175 -269
View File
@@ -4,302 +4,208 @@ build-backend = "setuptools.build_meta"
[project]
name = "semantica"
version = "0.1.0"
description = "🧠 Semantica - An Open Source Framework for building Semantic Layers and Knowledge Engineering "
version = "0.2.5"
description = "🧠 Semantica - An Open Source Framework for building Semantic Layers and Knowledge Engineering"
readme = "README.md"
license = {text = "MIT"}
authors = [
{name = "Hawksight AI", email = "semantica-dev@users.noreply.github.com"}
]
maintainers = [
{name = "Hawksight AI", email = "semantica-dev@users.noreply.github.com"}
]
license = { text = "MIT" }
authors = [{ name = "Hawksight AI", email = "semantica-dev@users.noreply.github.com" }]
maintainers = [{ name = "Hawksight AI", email = "semantica-dev@users.noreply.github.com" }]
requires-python = ">=3.8"
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: Text Processing :: Linguistic",
"Topic :: Database :: Database Engines/Servers",
"Topic :: Internet :: WWW/HTTP :: Indexing/Search"
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
"Topic :: Software Development :: Libraries :: Python Modules"
]
keywords = [
"semantic-layer", "knowledge-engineering", "nlp", "knowledge-graph",
"embeddings", "entity-extraction", "relationship-extraction", "rdf",
"ontology", "semantic-analysis", "ai", "machine-learning"
"semantic-layer", "knowledge-graph", "nlp", "embeddings",
"entity-extraction", "relationship-extraction", "rdf", "ontology"
]
# ---------------- CORE DEPENDENCIES (SAFE DEFAULT) ----------------
dependencies = [
"numpy>=1.21.0",
"pandas>=1.3.0",
"scikit-learn>=1.0.0",
"umap-learn>=0.5.0",
"spacy>=3.4.0",
"transformers>=4.20.0",
"torch>=1.12.0",
"sentence-transformers>=2.2.0",
"rdflib>=6.2.0",
"networkx>=2.8.0",
"matplotlib>=3.5.0",
"seaborn>=0.11.0",
"plotly>=5.10.0",
"ipywidgets>=8.0.0",
"requests>=2.28.0",
"beautifulsoup4>=4.11.0",
"lxml>=4.9.0",
"pypdf2>=2.10.0",
"python-docx>=0.8.11",
"docling>=1.0.0",
"openpyxl>=3.0.10",
"pillow>=9.2.0",
"librosa>=0.9.0",
"opencv-python>=4.6.0",
"faiss-cpu>=1.7.0",
"fastembed>=0.2.0",
"onnxruntime>=1.17.0",
"tokenizers>=0.15.0",
"weaviate-client>=3.15.0",
"qdrant-client>=1.3.0",
"neo4j>=5.0.0",
"falkordb>=1.0.0",
"pymongo>=4.2.0",
"sqlalchemy>=1.4.0",
"psycopg2-binary>=2.9.0",
"pymysql>=1.0.0",
"redis>=4.3.0",
"celery>=5.2.0",
"kafka-python>=2.0.0",
"pulsar-client>=3.0.0",
"pika>=1.3.0",
"boto3>=1.24.0",
"azure-storage-blob>=12.12.0",
"google-cloud-storage>=2.5.0",
"pydantic>=2.0.0",
"fastmcp>=0.1.0",
"groq>=0.4.0",
"openai>=1.0.0",
"litellm>=1.0.0",
"click>=8.1.0",
"rich>=12.5.0",
"tqdm>=4.64.0",
"pyyaml>=6.0",
"toml>=0.10.0",
"python-dotenv>=0.20.0",
"loguru>=0.6.0",
"structlog>=22.1.0",
"prometheus-client>=0.14.0",
"opentelemetry-api>=1.12.0",
"opentelemetry-sdk>=1.12.0",
"opentelemetry-instrumentation",
"fastapi>=0.78.0",
"uvicorn>=0.18.0",
"pytest>=7.1.0",
"pytest-cov>=3.0.0",
"pytest-asyncio>=0.19.0",
"black>=22.6.0",
"isort>=5.10.0",
"flake8>=4.0.0",
"mypy>=0.971",
"pre-commit>=2.19.0"
"numpy>=1.21.0",
"pandas>=1.3.0",
"scikit-learn>=1.0.0",
"umap-learn>=0.5.0",
"spacy>=3.4.0",
"transformers>=4.20.0",
"torch>=1.12.0",
"sentence-transformers>=2.2.0",
"rdflib>=6.2.0",
"networkx>=2.8.0",
"matplotlib>=3.5.0",
"seaborn>=0.11.0",
"plotly>=5.10.0",
"ipywidgets>=8.0.0",
"requests>=2.28.0",
"GitPython>=3.1.30",
"chardet>=5.1.0",
"protobuf>=5.29.1,<7.0",
"grpcio>=1.71.2",
"beautifulsoup4>=4.11.0",
"lxml>=4.9.0",
"pypdf2>=2.10.0",
"python-docx>=0.8.11",
"openpyxl>=3.0.10",
"pillow>=9.2.0",
"librosa>=0.9.0",
"opencv-python>=4.6.0",
"faiss-cpu>=1.7.0",
"fastembed>=0.2.0",
"onnxruntime>=1.17.0",
"tokenizers>=0.15.0",
"pydantic>=2.0.0",
"click>=8.1.0",
"rich>=12.5.0",
"tqdm>=4.64.0",
"pyyaml>=6.0",
"toml>=0.10.0",
"python-dotenv>=0.20.0",
"loguru>=0.6.0",
"structlog>=22.1.0"
]
[project.urls]
Homepage = "https://github.com/Hawksight-AI/semantica"
Repository = "https://github.com/Hawksight-AI/semantica"
"Bug Tracker" = "https://github.com/Hawksight-AI/semantica/issues"
Discussions = "https://github.com/Hawksight-AI/semantica/discussions"
# ---------------- OPTIONAL DEPENDENCIES ----------------
[project.optional-dependencies]
dev = [
"pytest>=7.1.0",
"pytest-cov>=3.0.0",
"pytest-asyncio>=0.19.0",
"black>=22.6.0",
"isort>=5.10.0",
"flake8>=4.0.0",
"mypy>=0.971",
"pre-commit>=2.19.0",
"jupyter>=1.0.0",
"ipykernel>=6.15.0",
"notebook>=6.4.0"
]
viz = [
"pyvis>=0.3.0",
"graphviz>=0.20.0",
"umap-learn>=0.5.0",
"d3blocks>=1.0.0"
]
gpu = [
"torch>=1.12.0",
"faiss-gpu>=1.7.0",
"cupy>=10.0.0"
]
cloud = [
"boto3>=1.24.0",
"azure-storage-blob>=12.12.0",
"google-cloud-storage>=2.5.0",
"kubernetes>=24.0.0",
"helm>=3.10.0"
]
monitoring = [
"prometheus-client>=0.14.0",
"opentelemetry-api>=1.12.0",
"opentelemetry-sdk>=1.12.0",
"opentelemetry-instrumentation>=0.32.0",
"grafana-api>=1.0.0",
"elasticsearch>=8.5.0"
]
llm-openai = [
"openai>=1.0.0"
]
llm-gemini = [
"google-generativeai>=0.3.0"
]
llm-groq = [
"groq>=0.4.0"
]
llm-anthropic = [
"anthropic>=0.18.0"
]
llm-ollama = [
"ollama>=0.1.0"
]
llm-litellm = [
"litellm>=1.0.0"
]
# ---- LLM Providers ----
llm-openai = ["openai>=1.0.0"]
llm-groq = ["groq>=0.4.0"]
llm-gemini = ["google-genai>=0.1.0"]
llm-anthropic = ["anthropic>=0.18.0"]
llm-ollama = ["ollama>=0.1.0"]
llm-deepseek = ["deepseek>=0.1.0"]
llm-litellm = ["litellm>=1.0.0"]
llm-instructor = ["instructor>=1.0.0"]
llm-all = [
"semantica[llm-openai,llm-gemini,llm-groq,llm-anthropic,llm-ollama,llm-litellm]"
]
models-huggingface = [
"transformers>=4.20.0",
"torch>=1.12.0"
]
split-tiktoken = [
"tiktoken>=0.5.0"
]
split-community = [
"python-louvain>=0.16"
]
split-topic = [
"bertopic>=0.15.0",
"gensim>=4.3.0"
]
split-all = [
"semantica[split-tiktoken,split-community,split-topic]"
]
graph-neo4j = [
"neo4j>=5.0.0"
]
graph-falkordb = [
"falkordb>=1.0.0",
"redis>=4.3.0"
]
graph-all = [
"semantica[graph-neo4j,graph-falkordb]"
]
parse-docling = [
"docling>=1.0.0"
]
all = [
"semantica[dev,viz,gpu,cloud,monitoring,llm-all,models-huggingface,split-all,graph-all,parse-docling]"
"semantica[llm-openai,llm-groq,llm-gemini,llm-anthropic,llm-ollama,llm-deepseek,llm-litellm,llm-instructor]"
]
# ---- Document Parsing ----
parse-docling = ["docling>=1.0.0"]
# ---- Embedding / Models ----
models-huggingface = [
"transformers>=4.20.0",
"torch>=1.12.0"
]
# ---- Graph Backends ----
graph-neo4j = ["neo4j>=5.0.0"]
graph-falkordb = ["falkordb>=1.0.0", "redis>=4.3.0"]
graph-amazon-neptune = ["boto3>=1.24.0", "neo4j>=5.0.0"]
graph-all = [
"semantica[graph-neo4j,graph-falkordb,graph-amazon-neptune]"
]
# ---- Vector Store Backends ----
vectorstore-qdrant = ["qdrant-client>=1.0.0"]
vectorstore-weaviate = ["weaviate-client>=4.0.0"]
vectorstore-pinecone = ["pinecone-client>=3.0.0"]
vectorstore-milvus = ["pymilvus>=2.0.0"]
vectorstore-all = [
"semantica[vectorstore-qdrant,vectorstore-weaviate,vectorstore-pinecone,vectorstore-milvus]"
]
# ---- Infra / Queues / Workers ----
infra = [
"redis>=4.3.0",
"celery>=5.2.0",
"kafka-python>=2.0.0",
"pulsar-client>=3.0.0",
"pika>=1.3.0"
]
# ---- Cloud Providers ----
cloud = [
"boto3>=1.24.0",
"azure-storage-blob>=12.12.0",
"google-cloud-storage>=2.5.0"
]
# ---- Monitoring (FIXED) ----
monitoring = [
"prometheus-client>=0.14.0",
"opentelemetry-api>=1.30.0,<2.0.0",
"opentelemetry-sdk>=1.30.0,<2.0.0",
"opentelemetry-semantic-conventions>=0.58b0,<0.61b0",
"opentelemetry-instrumentation>=0.58b0,<0.61b0"
]
# ---- Visualization ----
viz = [
"pyvis>=0.3.0",
"graphviz>=0.20.0",
"d3blocks>=1.0.0"
]
# ---- GPU ----
gpu = [
"faiss-gpu>=1.7.0",
"cupy>=10.0.0"
]
# ---- Splitting / Chunking ----
split-tiktoken = ["tiktoken>=0.5.0"]
split-community = ["python-louvain>=0.16"]
split-topic = ["bertopic>=0.15.0", "gensim>=4.3.0"]
split-all = [
"semantica[split-tiktoken,split-community,split-topic]"
]
# ---- Dev ----
dev = [
"pytest>=7.1.0",
"pytest-cov>=3.0.0",
"pytest-asyncio>=0.19.0",
"black>=22.6.0",
"isort>=5.10.0",
"flake8>=4.0.0",
"mypy>=0.971",
"pre-commit>=2.19.0",
"jupyter>=1.0.0",
"ipykernel>=6.15.0"
]
# ---- Everything ----
all = [
"semantica[dev,viz,gpu,infra,cloud,monitoring,llm-all,models-huggingface,split-all,graph-all,vectorstore-all,parse-docling]"
]
# ---------------- ENTRYPOINTS ----------------
[project.scripts]
semantica = "semantica.cli:main"
semantica-server = "semantica.server:main"
semantica-worker = "semantica.worker:main"
# ---------------- TOOLING ----------------
[tool.setuptools.packages.find]
where = ["."]
include = ["semantica*"]
exclude = ["tests*", "docs*", "examples*"]
[tool.setuptools.package-data]
semantica = ["*.yaml", "*.yml", "*.json", "*.toml", "*.txt", "*.md"]
[tool.black]
line-length = 88
target-version = ['py38', 'py39', 'py310', 'py311', 'py312']
include = '\.pyi?$'
extend-exclude = '''
/(
# directories
\.eggs
| \.git
| \.hg
| \.mypy_cache
| \.tox
| \.venv
| build
| dist
)/
'''
[tool.isort]
profile = "black"
multi_line_output = 3
line_length = 88
known_first_party = ["semantica"]
known_third_party = ["numpy", "pandas", "scikit-learn", "spacy", "transformers", "torch"]
[tool.mypy]
python_version = "3.9"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true
strict_equality = true
show_error_codes = true
[tool.pytest.ini_options]
minversion = "7.0"
addopts = "-ra -q --strict-markers --strict-config"
testpaths = ["tests"]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
markers = [
"slow: marks tests as slow (deselect with '-m \"not slow\"')",
"integration: marks tests as integration tests",
"unit: marks tests as unit tests",
"gpu: marks tests that require GPU",
"cloud: marks tests that require cloud services"
]
[tool.coverage.run]
source = ["semantica"]
omit = [
"*/tests/*",
"*/test_*",
"*/__pycache__/*",
"*/migrations/*"
]
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"def __repr__",
"if self.debug:",
"if settings.DEBUG",
"raise AssertionError",
"raise NotImplementedError",
"if 0:",
"if __name__ == .__main__.:",
"class .*\\bProtocol\\):",
"@(abc\\.)?abstractmethod"
]
+4
View File
@@ -0,0 +1,4 @@
[pytest]
markers =
integration: marks tests as integration (deselect with '-m "not integration"')
addopts = -ra
+1 -1
View File
@@ -10,7 +10,7 @@ Main exports:
- Config: Configuration management
"""
__version__ = "0.1.0"
__version__ = "0.2.5"
__author__ = "Semantica Contributors"
__license__ = "MIT"
+63
View File
@@ -0,0 +1,63 @@
"""
Enhanced Change Management Module for Semantica
This module provides comprehensive change management capabilities including:
- Persistent version storage (SQLite and in-memory)
- Detailed change tracking and diff algorithms
- Standardized metadata and audit trails
- Data integrity verification with checksums
- Enhanced version managers for KG and ontologies
- Enterprise compliance support (HIPAA, SOX, FDA)
Public API:
ChangeLogEntry: Standardized metadata for version changes
VersionStorage: Abstract storage interface
InMemoryVersionStorage: Fast in-memory storage backend
SQLiteVersionStorage: Persistent SQLite storage backend
compute_checksum: SHA-256 checksum computation
verify_checksum: Data integrity verification
EnhancedTemporalVersionManager: Advanced KG version management
EnhancedVersionManager: Advanced ontology version management
"""
from .change_log import ChangeLogEntry
from .version_storage import (
VersionStorage,
InMemoryVersionStorage,
SQLiteVersionStorage,
compute_checksum,
verify_checksum
)
from .managers import (
BaseVersionManager,
TemporalVersionManager,
OntologyVersionManager
)
from .ontology_version_manager import VersionManager, OntologyVersion
__all__ = [
# Change metadata
"ChangeLogEntry",
# Storage backends
"VersionStorage",
"InMemoryVersionStorage",
"SQLiteVersionStorage",
# Integrity utilities
"compute_checksum",
"verify_checksum",
# Version managers
"BaseVersionManager",
"TemporalVersionManager",
"OntologyVersionManager",
# Ontology version management
"VersionManager",
"OntologyVersion"
]
__version__ = "1.0.0"
__author__ = "Semantica Team"
__description__ = "Enhanced Change Management for Semantica"
+108
View File
@@ -0,0 +1,108 @@
"""
Change Log Module
This module provides standardized metadata structures for version changes
across both ontology and knowledge graph versioning systems.
Key Features:
- Standardized ChangeLogEntry dataclass
- Email validation for authors
- Timestamp handling in ISO 8601 format
- Optional change linking and tracking
Main Classes:
- ChangeLogEntry: Standard metadata for version changes
Example Usage:
>>> from semantica.common.change_log import ChangeLogEntry
>>> entry = ChangeLogEntry(
... timestamp="2024-01-15T10:30:00Z",
... author="alice@company.com",
... description="Added Customer entity"
... )
Author: Semantica Contributors
License: MIT
"""
import re
from dataclasses import dataclass, field
from datetime import datetime
from typing import List, Optional
from ..utils.exceptions import ValidationError
@dataclass
class ChangeLogEntry:
"""
Standard metadata for version changes.
This dataclass provides a consistent structure for tracking changes
across both ontology and knowledge graph versioning systems.
Attributes:
timestamp: ISO 8601 timestamp of the change
author: Email address of the change author
description: Description of the change (max 500 characters)
change_id: Optional unique identifier for the change
related_changes: Optional list of related change IDs
"""
timestamp: str
author: str
description: str
change_id: Optional[str] = None
related_changes: List[str] = field(default_factory=list)
def __post_init__(self):
"""Validate fields after initialization."""
self._validate_timestamp()
self._validate_author()
self._validate_description()
def _validate_timestamp(self):
"""Validate timestamp is in ISO 8601 format."""
try:
# More strict validation for ISO 8601 format
if 'T' not in self.timestamp:
raise ValueError("Missing 'T' separator")
datetime.fromisoformat(self.timestamp.replace('Z', '+00:00'))
except ValueError:
raise ValidationError(f"Invalid timestamp format: {self.timestamp}. Expected ISO 8601 format.")
def _validate_author(self):
"""Validate author is a valid email address."""
email_pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
if not re.match(email_pattern, self.author):
raise ValidationError(f"Invalid email format: {self.author}")
def _validate_description(self):
"""Validate description length."""
if len(self.description) > 500:
raise ValidationError(f"Description too long: {len(self.description)} characters (max 500)")
if not self.description.strip():
raise ValidationError("Description cannot be empty")
@classmethod
def create_now(cls, author: str, description: str, change_id: Optional[str] = None,
related_changes: Optional[List[str]] = None) -> 'ChangeLogEntry':
"""
Create a ChangeLogEntry with current timestamp.
Args:
author: Email address of the change author
description: Description of the change
change_id: Optional unique identifier for the change
related_changes: Optional list of related change IDs
Returns:
ChangeLogEntry with current timestamp
"""
return cls(
timestamp=datetime.now().isoformat(),
author=author,
description=description,
change_id=change_id,
related_changes=related_changes or []
)
File diff suppressed because it is too large Load Diff
+497
View File
@@ -0,0 +1,497 @@
"""
Enhanced Version Managers Module
This module provides enhanced version management capabilities for both knowledge graphs
and ontologies, with comprehensive change tracking, persistent storage, and audit trails.
Key Features:
- Enhanced TemporalVersionManager for knowledge graphs
- Enhanced VersionManager for ontologies
- Detailed diff algorithms for entities and relationships
- Structural comparison for ontology elements
- Integration with storage backends and metadata
Main Classes:
- EnhancedTemporalVersionManager: Advanced KG version management
- EnhancedVersionManager: Advanced ontology version management
Author: Semantica Contributors
License: MIT
"""
from abc import ABC, abstractmethod
from datetime import datetime
from typing import Any, Dict, List, Optional
from .change_log import ChangeLogEntry
from .version_storage import VersionStorage, InMemoryVersionStorage, SQLiteVersionStorage, compute_checksum, verify_checksum
from ..utils.exceptions import ValidationError, ProcessingError
from ..utils.logging import get_logger
class BaseVersionManager(ABC):
"""
Abstract base class for enhanced version managers.
Provides common functionality for version management across different data types.
"""
def __init__(self, storage_path: Optional[str] = None):
"""
Initialize base version manager.
Args:
storage_path: Path to SQLite database for persistent storage.
If None, uses in-memory storage.
"""
self.logger = get_logger(self.__class__.__name__.lower())
# Initialize storage backend
if storage_path:
self.storage = SQLiteVersionStorage(storage_path)
self.logger.info(f"Initialized with SQLite storage: {storage_path}")
else:
self.storage = InMemoryVersionStorage()
self.logger.info("Initialized with in-memory storage")
@abstractmethod
def create_snapshot(self, data: Any, version_label: str, author: str, description: str, **options) -> Dict[str, Any]:
"""Create a versioned snapshot of the data."""
pass
@abstractmethod
def compare_versions(self, version1: Any, version2: Any, **options) -> Dict[str, Any]:
"""Compare two versions and return detailed differences."""
pass
def list_versions(self) -> List[Dict[str, Any]]:
"""List all version snapshots."""
return self.storage.list_all()
def get_version(self, label: str) -> Optional[Dict[str, Any]]:
"""Retrieve specific version by label."""
return self.storage.get(label)
def verify_checksum(self, snapshot: Dict[str, Any]) -> bool:
"""Verify the integrity of a snapshot using its checksum."""
return verify_checksum(snapshot)
class TemporalVersionManager(BaseVersionManager):
"""
Temporal version management engine for knowledge graphs.
Provides comprehensive version/snapshot management capabilities including
persistent storage, detailed change tracking, and audit trails.
Features:
- Persistent snapshot storage (SQLite or in-memory)
- Detailed change tracking with entity-level diffs
- SHA-256 checksums for data integrity
- Standardized metadata with author attribution
- Version comparison with backward compatibility
- Input validation and security features
"""
def __init__(self, storage_path: Optional[str] = None, **config):
"""
Initialize enhanced temporal version manager.
Args:
storage_path: Path to SQLite database file for persistent storage.
If None, uses in-memory storage
**config: Additional configuration options
"""
super().__init__(storage_path)
self.config = config
def create_snapshot(
self,
graph: Dict[str, Any],
version_label: str,
author: str,
description: str,
**options
) -> Dict[str, Any]:
"""
Create and store snapshot with checksum and metadata.
Args:
graph: Knowledge graph dict with "entities" and "relationships"
version_label: Version string (e.g., "v1.0")
author: Email address of the change author
description: Change description (max 500 chars)
**options: Additional options
Returns:
dict: Snapshot with metadata and checksum
Raises:
ValidationError: If input validation fails
ProcessingError: If storage operation fails
"""
# Validate inputs
change_entry = ChangeLogEntry(
timestamp=datetime.now().isoformat(),
author=author,
description=description
)
# Create snapshot
snapshot = {
"label": version_label,
"timestamp": change_entry.timestamp,
"author": change_entry.author,
"description": change_entry.description,
"entities": graph.get("entities", []).copy(),
"relationships": graph.get("relationships", []).copy(),
"metadata": options.get("metadata", {})
}
# Compute and add checksum
snapshot["checksum"] = compute_checksum(snapshot)
# Store snapshot
self.storage.save(snapshot)
self.logger.info(f"Created snapshot '{version_label}' by {author}")
return snapshot
def compare_versions(
self,
v1_label_or_dict,
v2_label_or_dict,
comparison_metrics: Optional[List[str]] = None,
**options,
) -> Dict[str, Any]:
"""
Compare two graph versions with detailed entity-level differences.
Args:
v1_label_or_dict: First version (label string or snapshot dict)
v2_label_or_dict: Second version (label string or snapshot dict)
comparison_metrics: List of metrics to calculate (optional, unused)
**options: Additional comparison options (unused)
Returns:
dict: Detailed version comparison results
"""
# Handle both label strings and snapshot dictionaries
if isinstance(v1_label_or_dict, str):
version1 = self.storage.get(v1_label_or_dict)
if not version1:
raise ValidationError(f"Version not found: {v1_label_or_dict}")
else:
version1 = v1_label_or_dict
if isinstance(v2_label_or_dict, str):
version2 = self.storage.get(v2_label_or_dict)
if not version2:
raise ValidationError(f"Version not found: {v2_label_or_dict}")
else:
version2 = v2_label_or_dict
# Compute detailed diff
detailed_diff = self._compute_detailed_diff(version1, version2)
# Maintain backward compatibility with summary
summary = {
"entities_added": len(detailed_diff["entities_added"]),
"entities_removed": len(detailed_diff["entities_removed"]),
"entities_modified": len(detailed_diff["entities_modified"]),
"relationships_added": len(detailed_diff["relationships_added"]),
"relationships_removed": len(detailed_diff["relationships_removed"]),
"relationships_modified": len(detailed_diff["relationships_modified"])
}
return {
"version1": version1.get("label", "unknown"),
"version2": version2.get("label", "unknown"),
"summary": summary,
**detailed_diff
}
def _compute_detailed_diff(self, version1: Dict[str, Any], version2: Dict[str, Any]) -> Dict[str, Any]:
"""
Compute detailed entity and relationship differences between versions.
Args:
version1: First version snapshot
version2: Second version snapshot
Returns:
Dict with detailed diff information
"""
entities1 = {e.get("id", str(i)): e for i, e in enumerate(version1.get("entities", []))}
entities2 = {e.get("id", str(i)): e for i, e in enumerate(version2.get("entities", []))}
relationships1 = {self._relationship_key(r): r for r in version1.get("relationships", [])}
relationships2 = {self._relationship_key(r): r for r in version2.get("relationships", [])}
# Entity differences
entity_ids1 = set(entities1.keys())
entity_ids2 = set(entities2.keys())
entities_added = [entities2[eid] for eid in entity_ids2 - entity_ids1]
entities_removed = [entities1[eid] for eid in entity_ids1 - entity_ids2]
entities_modified = []
for eid in entity_ids1 & entity_ids2:
if entities1[eid] != entities2[eid]:
changes = self._compute_entity_changes(entities1[eid], entities2[eid])
entities_modified.append({
"id": eid,
"before": entities1[eid],
"after": entities2[eid],
"changes": changes
})
# Relationship differences
rel_keys1 = set(relationships1.keys())
rel_keys2 = set(relationships2.keys())
relationships_added = [relationships2[key] for key in rel_keys2 - rel_keys1]
relationships_removed = [relationships1[key] for key in rel_keys1 - rel_keys2]
relationships_modified = []
for key in rel_keys1 & rel_keys2:
if relationships1[key] != relationships2[key]:
changes = self._compute_relationship_changes(relationships1[key], relationships2[key])
relationships_modified.append({
"key": key,
"before": relationships1[key],
"after": relationships2[key],
"changes": changes
})
return {
"entities_added": entities_added,
"entities_removed": entities_removed,
"entities_modified": entities_modified,
"relationships_added": relationships_added,
"relationships_removed": relationships_removed,
"relationships_modified": relationships_modified
}
def _relationship_key(self, relationship: Dict[str, Any]) -> str:
"""Generate a unique key for a relationship."""
source = relationship.get("source", "")
target = relationship.get("target", "")
rel_type = relationship.get("type", relationship.get("relationship", ""))
return f"{source}|{rel_type}|{target}"
def _compute_entity_changes(self, entity1: Dict[str, Any], entity2: Dict[str, Any]) -> Dict[str, Any]:
"""Compute changes between two entity versions."""
changes = {}
all_keys = set(entity1.keys()) | set(entity2.keys())
for key in all_keys:
val1 = entity1.get(key)
val2 = entity2.get(key)
if val1 != val2:
changes[key] = {"from": val1, "to": val2}
return changes
def _compute_relationship_changes(self, rel1: Dict[str, Any], rel2: Dict[str, Any]) -> Dict[str, Any]:
"""Compute changes between two relationship versions."""
changes = {}
all_keys = set(rel1.keys()) | set(rel2.keys())
for key in all_keys:
val1 = rel1.get(key)
val2 = rel2.get(key)
if val1 != val2:
changes[key] = {"from": val1, "to": val2}
return changes
class OntologyVersionManager(BaseVersionManager):
"""
Version management for ontologies with structural comparison.
Provides comprehensive version management for ontologies including
detailed structural analysis and change tracking.
Features:
- Structural comparison of ontology elements
- Detailed diff for classes, properties, individuals, axioms
- Persistent storage with metadata
- Change tracking and audit trails
"""
def __init__(self, storage_path: Optional[str] = None, **config):
"""
Initialize enhanced version manager.
Args:
storage_path: Path to SQLite database file for persistent storage.
If None, uses in-memory storage
**config: Additional configuration options
"""
super().__init__(storage_path)
self.config = config
self.versions = {} # In-memory version tracking for compatibility
def create_snapshot(
self,
ontology_data: Dict[str, Any],
version_label: str,
author: str,
description: str,
**options
) -> Dict[str, Any]:
"""
Create ontology version snapshot.
Args:
ontology_data: Ontology data dictionary
version_label: Version string (e.g., "v1.0")
author: Email address of the change author
description: Change description
**options: Additional options including metadata
Returns:
dict: Ontology version snapshot
"""
# Validate inputs
change_entry = ChangeLogEntry(
timestamp=datetime.now().isoformat(),
author=author,
description=description
)
# Create snapshot
snapshot = {
"label": version_label,
"timestamp": change_entry.timestamp,
"author": change_entry.author,
"description": change_entry.description,
"ontology_iri": ontology_data.get("uri", ""),
"version_info": ontology_data.get("version_info", {}),
"structure": ontology_data.get("structure", {}),
"metadata": options.get("metadata", {})
}
# Compute and add checksum
snapshot["checksum"] = compute_checksum(snapshot)
# Store snapshot
self.storage.save(snapshot)
# Also store in memory for compatibility
self.versions[version_label] = snapshot
self.logger.info(f"Created ontology snapshot '{version_label}' by {author}")
return snapshot
def compare_versions(self, version1: str, version2: str, **options) -> Dict[str, Any]:
"""
Compare two ontology versions with detailed structural analysis.
Args:
version1: First version label
version2: Second version label
**options: Additional comparison options
Returns:
Detailed comparison results including structural differences
"""
# Get versions from storage
v1_snapshot = self.storage.get(version1)
v2_snapshot = self.storage.get(version2)
if not v1_snapshot:
raise ValidationError(f"Version not found: {version1}")
if not v2_snapshot:
raise ValidationError(f"Version not found: {version2}")
# Basic metadata comparison
metadata_changes = {}
if v1_snapshot.get("ontology_iri") != v2_snapshot.get("ontology_iri"):
metadata_changes["ontology_iri"] = {
"from": v1_snapshot.get("ontology_iri"),
"to": v2_snapshot.get("ontology_iri")
}
if v1_snapshot.get("version_info") != v2_snapshot.get("version_info"):
metadata_changes["version_info"] = {
"from": v1_snapshot.get("version_info"),
"to": v2_snapshot.get("version_info")
}
# Structural comparison
structural_diff = self._compare_ontology_structures(v1_snapshot, v2_snapshot)
return {
"version1": version1,
"version2": version2,
"metadata_changes": metadata_changes,
**structural_diff
}
def _compare_ontology_structures(self, v1_snapshot: Dict[str, Any], v2_snapshot: Dict[str, Any]) -> Dict[str, Any]:
"""
Compare structural elements between two ontology versions.
Args:
v1_snapshot: First ontology version snapshot
v2_snapshot: Second ontology version snapshot
Returns:
Dictionary with structural differences
"""
# Extract structural information
v1_structure = v1_snapshot.get("structure", {})
v2_structure = v2_snapshot.get("structure", {})
# Compare classes
v1_classes = set(v1_structure.get("classes", []))
v2_classes = set(v2_structure.get("classes", []))
classes_added = list(v2_classes - v1_classes)
classes_removed = list(v1_classes - v2_classes)
# Compare properties
v1_properties = set(v1_structure.get("properties", []))
v2_properties = set(v2_structure.get("properties", []))
properties_added = list(v2_properties - v1_properties)
properties_removed = list(v1_properties - v2_properties)
# Compare individuals
v1_individuals = set(v1_structure.get("individuals", []))
v2_individuals = set(v2_structure.get("individuals", []))
individuals_added = list(v2_individuals - v1_individuals)
individuals_removed = list(v1_individuals - v2_individuals)
# Compare axioms/rules
v1_axioms = set(v1_structure.get("axioms", []))
v2_axioms = set(v2_structure.get("axioms", []))
axioms_added = list(v2_axioms - v1_axioms)
axioms_removed = list(v1_axioms - v2_axioms)
return {
"classes_added": classes_added,
"classes_removed": classes_removed,
"properties_added": properties_added,
"properties_removed": properties_removed,
"individuals_added": individuals_added,
"individuals_removed": individuals_removed,
"axioms_added": axioms_added,
"axioms_removed": axioms_removed,
"summary": {
"classes_added": len(classes_added),
"classes_removed": len(classes_removed),
"properties_added": len(properties_added),
"properties_removed": len(properties_removed),
"individuals_added": len(individuals_added),
"individuals_removed": len(individuals_removed),
"axioms_added": len(axioms_added),
"axioms_removed": len(axioms_removed)
}
}
@@ -42,7 +42,7 @@ from typing import Any, Dict, List, Optional
from ..utils.exceptions import ProcessingError, ValidationError
from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker
from .namespace_manager import NamespaceManager
from ..ontology.namespace_manager import NamespaceManager
@dataclass
@@ -198,14 +198,14 @@ class VersionManager:
def compare_versions(self, version1: str, version2: str) -> Dict[str, Any]:
"""
Compare two ontology versions.
Compare two ontology versions with detailed structural analysis.
Args:
version1: First version
version2: Second version
Returns:
Comparison results
Detailed comparison results including structural differences
"""
if version1 not in self.versions:
raise ValidationError(f"Version not found: {version1}")
@@ -215,19 +215,85 @@ class VersionManager:
v1 = self.versions[version1]
v2 = self.versions[version2]
# Basic comparison
changes = []
# Basic metadata comparison
metadata_changes = {}
if v1.ontology_iri != v2.ontology_iri:
changes.append("Ontology IRI changed")
metadata_changes["ontology_iri"] = {"from": v1.ontology_iri, "to": v2.ontology_iri}
if v1.version_info != v2.version_info:
changes.append("Version info changed")
metadata_changes["version_info"] = {"from": v1.version_info, "to": v2.version_info}
# Structural comparison (if ontology data is available in metadata)
structural_diff = self._compare_ontology_structures(v1, v2)
return {
"version1": version1,
"version2": version2,
"changes": changes,
"v1_iri": v1.ontology_iri,
"v2_iri": v2.ontology_iri,
"metadata_changes": metadata_changes,
**structural_diff
}
def _compare_ontology_structures(self, v1: OntologyVersion, v2: OntologyVersion) -> Dict[str, Any]:
"""
Compare structural elements between two ontology versions.
Args:
v1: First ontology version
v2: Second ontology version
Returns:
Dictionary with structural differences
"""
# Extract structural information from metadata if available
v1_structure = v1.metadata.get("structure", {})
v2_structure = v2.metadata.get("structure", {})
# Compare classes
v1_classes = set(v1_structure.get("classes", []))
v2_classes = set(v2_structure.get("classes", []))
classes_added = list(v2_classes - v1_classes)
classes_removed = list(v1_classes - v2_classes)
# Compare properties
v1_properties = set(v1_structure.get("properties", []))
v2_properties = set(v2_structure.get("properties", []))
properties_added = list(v2_properties - v1_properties)
properties_removed = list(v1_properties - v2_properties)
# Compare individuals
v1_individuals = set(v1_structure.get("individuals", []))
v2_individuals = set(v2_structure.get("individuals", []))
individuals_added = list(v2_individuals - v1_individuals)
individuals_removed = list(v1_individuals - v2_individuals)
# Compare axioms/rules if available
v1_axioms = set(v1_structure.get("axioms", []))
v2_axioms = set(v2_structure.get("axioms", []))
axioms_added = list(v2_axioms - v1_axioms)
axioms_removed = list(v1_axioms - v2_axioms)
return {
"classes_added": classes_added,
"classes_removed": classes_removed,
"properties_added": properties_added,
"properties_removed": properties_removed,
"individuals_added": individuals_added,
"individuals_removed": individuals_removed,
"axioms_added": axioms_added,
"axioms_removed": axioms_removed,
"summary": {
"classes_added": len(classes_added),
"classes_removed": len(classes_removed),
"properties_added": len(properties_added),
"properties_removed": len(properties_removed),
"individuals_added": len(individuals_added),
"individuals_removed": len(individuals_removed),
"axioms_added": len(axioms_added),
"axioms_removed": len(axioms_removed)
}
}
def get_version(self, version: str) -> Optional[OntologyVersion]:
@@ -0,0 +1,391 @@
"""
Version Storage Module
This module provides abstract storage interfaces and concrete implementations
for persistent version management in Semantica.
Key Features:
- Abstract VersionStorage interface
- In-memory storage implementation
- SQLite-based persistent storage implementation
- Checksum computation and validation
- Thread-safe operations
Main Classes:
- VersionStorage: Abstract base class for storage backends
- InMemoryVersionStorage: Dictionary-based in-memory storage
- SQLiteVersionStorage: SQLite-based persistent storage
Example Usage:
>>> from semantica.common.version_storage import SQLiteVersionStorage
>>> storage = SQLiteVersionStorage("versions.db")
>>> storage.save(snapshot)
>>> versions = storage.list_all()
Author: Semantica Contributors
License: MIT
"""
import hashlib
import json
import sqlite3
import threading
from abc import ABC, abstractmethod
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, List, Optional
from ..utils.exceptions import ProcessingError, ValidationError
from ..utils.logging import get_logger
class VersionStorage(ABC):
"""
Abstract base class for version storage backends.
This interface defines the contract that all storage implementations
must follow for version management operations.
"""
@abstractmethod
def save(self, snapshot: Dict[str, Any]) -> None:
"""
Save a version snapshot.
Args:
snapshot: Version snapshot dictionary with metadata
Raises:
ValidationError: If snapshot data is invalid
ProcessingError: If save operation fails
"""
pass
@abstractmethod
def get(self, label: str) -> Optional[Dict[str, Any]]:
"""
Retrieve a version snapshot by label.
Args:
label: Version label to retrieve
Returns:
Snapshot dictionary or None if not found
"""
pass
@abstractmethod
def list_all(self) -> List[Dict[str, Any]]:
"""
List all version snapshots.
Returns:
List of snapshot metadata dictionaries
"""
pass
@abstractmethod
def exists(self, label: str) -> bool:
"""
Check if a version exists.
Args:
label: Version label to check
Returns:
True if version exists, False otherwise
"""
pass
@abstractmethod
def delete(self, label: str) -> bool:
"""
Delete a version snapshot.
Args:
label: Version label to delete
Returns:
True if deleted, False if not found
"""
pass
class InMemoryVersionStorage(VersionStorage):
"""
In-memory version storage implementation.
This implementation stores all version data in memory using a dictionary.
Data is lost when the process ends.
"""
def __init__(self):
"""Initialize in-memory storage."""
self._storage: Dict[str, Dict[str, Any]] = {}
self._lock = threading.RLock()
self.logger = get_logger("in_memory_storage")
def save(self, snapshot: Dict[str, Any]) -> None:
"""Save snapshot to memory."""
label = snapshot.get("label")
if not label:
raise ValidationError("Snapshot must have a 'label' field")
with self._lock:
if label in self._storage:
raise ValidationError(f"Version '{label}' already exists")
# Deep copy to prevent external modifications
self._storage[label] = json.loads(json.dumps(snapshot))
self.logger.debug(f"Saved version '{label}' to memory")
def get(self, label: str) -> Optional[Dict[str, Any]]:
"""Retrieve snapshot from memory."""
with self._lock:
snapshot = self._storage.get(label)
if snapshot:
# Return deep copy to prevent external modifications
return json.loads(json.dumps(snapshot))
return None
def list_all(self) -> List[Dict[str, Any]]:
"""List all snapshots in memory."""
with self._lock:
# Return metadata only (without full graph data)
metadata_list = []
for label, snapshot in self._storage.items():
metadata = {
"label": snapshot.get("label"),
"timestamp": snapshot.get("timestamp"),
"author": snapshot.get("author"),
"description": snapshot.get("description"),
"checksum": snapshot.get("checksum"),
"entity_count": len(snapshot.get("entities", [])),
"relationship_count": len(snapshot.get("relationships", []))
}
metadata_list.append(metadata)
return metadata_list
def exists(self, label: str) -> bool:
"""Check if version exists in memory."""
with self._lock:
return label in self._storage
def delete(self, label: str) -> bool:
"""Delete version from memory."""
with self._lock:
if label in self._storage:
del self._storage[label]
self.logger.debug(f"Deleted version '{label}' from memory")
return True
return False
class SQLiteVersionStorage(VersionStorage):
"""
SQLite-based persistent version storage implementation.
This implementation stores version data in a SQLite database file,
providing persistence across process restarts.
"""
def __init__(self, storage_path: str):
"""
Initialize SQLite storage.
Args:
storage_path: Path to SQLite database file
"""
self.storage_path = Path(storage_path)
self._lock = threading.RLock()
self.logger = get_logger("sqlite_storage")
# Create directory if it doesn't exist
self.storage_path.parent.mkdir(parents=True, exist_ok=True)
# Initialize database
self._init_database()
def _init_database(self) -> None:
"""Initialize SQLite database schema."""
with self._lock:
conn = sqlite3.connect(str(self.storage_path))
try:
cursor = conn.cursor()
cursor.execute("""
CREATE TABLE IF NOT EXISTS versions (
label TEXT PRIMARY KEY,
timestamp TEXT NOT NULL,
author TEXT NOT NULL,
description TEXT NOT NULL,
checksum TEXT NOT NULL,
snapshot_data TEXT NOT NULL,
created_at TEXT NOT NULL
)
""")
conn.commit()
self.logger.debug(f"Initialized SQLite database at {self.storage_path}")
finally:
conn.close()
def save(self, snapshot: Dict[str, Any]) -> None:
"""Save snapshot to SQLite database."""
label = snapshot.get("label")
if not label:
raise ValidationError("Snapshot must have a 'label' field")
with self._lock:
conn = sqlite3.connect(str(self.storage_path))
try:
cursor = conn.cursor()
# Check if version already exists
cursor.execute("SELECT label FROM versions WHERE label = ?", (label,))
if cursor.fetchone():
raise ValidationError(f"Version '{label}' already exists")
# Insert new version
cursor.execute("""
INSERT INTO versions
(label, timestamp, author, description, checksum, snapshot_data, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
""", (
label,
snapshot.get("timestamp", ""),
snapshot.get("author", ""),
snapshot.get("description", ""),
snapshot.get("checksum", ""),
json.dumps(snapshot),
datetime.now().isoformat()
))
conn.commit()
self.logger.debug(f"Saved version '{label}' to SQLite database")
except sqlite3.Error as e:
raise ProcessingError(f"Failed to save version to database: {e}")
finally:
conn.close()
def get(self, label: str) -> Optional[Dict[str, Any]]:
"""Retrieve snapshot from SQLite database."""
with self._lock:
conn = sqlite3.connect(str(self.storage_path))
try:
cursor = conn.cursor()
cursor.execute("""
SELECT snapshot_data FROM versions WHERE label = ?
""", (label,))
row = cursor.fetchone()
if not row:
return None
return json.loads(row[0])
except sqlite3.Error as e:
raise ProcessingError(f"Failed to retrieve version from database: {e}")
finally:
conn.close()
def list_all(self) -> List[Dict[str, Any]]:
"""List all snapshots in SQLite database."""
with self._lock:
conn = sqlite3.connect(str(self.storage_path))
try:
cursor = conn.cursor()
cursor.execute("""
SELECT snapshot_data FROM versions ORDER BY timestamp DESC
""")
metadata_list = []
for row in cursor.fetchall():
snapshot = json.loads(row[0])
metadata = {
"label": snapshot.get("label"),
"timestamp": snapshot.get("timestamp"),
"author": snapshot.get("author"),
"description": snapshot.get("description"),
"checksum": snapshot.get("checksum"),
"entity_count": len(snapshot.get("entities", [])),
"relationship_count": len(snapshot.get("relationships", []))
}
metadata_list.append(metadata)
return metadata_list
except sqlite3.Error as e:
raise ProcessingError(f"Failed to list versions from database: {e}")
finally:
conn.close()
def exists(self, label: str) -> bool:
"""Check if version exists in SQLite database."""
with self._lock:
conn = sqlite3.connect(str(self.storage_path))
try:
cursor = conn.cursor()
cursor.execute("SELECT 1 FROM versions WHERE label = ?", (label,))
return cursor.fetchone() is not None
except sqlite3.Error as e:
raise ProcessingError(f"Failed to check version existence: {e}")
finally:
conn.close()
def delete(self, label: str) -> bool:
"""Delete version from SQLite database."""
with self._lock:
conn = sqlite3.connect(str(self.storage_path))
try:
cursor = conn.cursor()
cursor.execute("DELETE FROM versions WHERE label = ?", (label,))
deleted = cursor.rowcount > 0
conn.commit()
if deleted:
self.logger.debug(f"Deleted version '{label}' from SQLite database")
return deleted
except sqlite3.Error as e:
raise ProcessingError(f"Failed to delete version from database: {e}")
finally:
conn.close()
def compute_checksum(data: Dict[str, Any]) -> str:
"""
Compute SHA-256 checksum for version data.
Args:
data: Dictionary containing version data
Returns:
SHA-256 checksum as hexadecimal string
"""
# Create a deterministic JSON representation
json_str = json.dumps(data, sort_keys=True, separators=(',', ':'))
return hashlib.sha256(json_str.encode('utf-8')).hexdigest()
def verify_checksum(snapshot: Dict[str, Any]) -> bool:
"""
Verify the integrity of a snapshot using its checksum.
Args:
snapshot: Snapshot dictionary with checksum field
Returns:
True if checksum is valid, False otherwise
"""
stored_checksum = snapshot.get("checksum")
if not stored_checksum:
return False
# Create copy without checksum for verification
data_copy = snapshot.copy()
data_copy.pop("checksum", None)
computed_checksum = compute_checksum(data_copy)
return stored_checksum == computed_checksum
+119
View File
@@ -549,3 +549,122 @@ class SourceTracker:
List of traceability records
"""
return self.get_traceability_chain(entity_id, property_name)
def track_sources_batch(
self,
source_data: List[Dict[str, Any]],
pipeline_id: Optional[str] = None,
) -> Dict[str, int]:
"""
Track sources for multiple entities, properties, or relationships in batch.
Args:
source_data: List of source tracking dictionaries, each containing:
- type: "entity", "property", or "relationship"
- entity_id: Entity identifier (required for all types)
- property_name: Property name (required for "property" type)
- value: Property value (required for "property" type)
- relationship_id: Relationship identifier (required for "relationship" type)
- source: SourceReference object or dict to create SourceReference
- metadata: Optional metadata dictionary
pipeline_id: Optional pipeline ID for progress tracking
Returns:
dict: Statistics with keys:
- entities_tracked: Number of entities tracked
- properties_tracked: Number of properties tracked
- relationships_tracked: Number of relationships tracked
- total_tracked: Total number of items tracked
"""
if not source_data:
return {
"entities_tracked": 0,
"properties_tracked": 0,
"relationships_tracked": 0,
"total_tracked": 0,
}
tracking_id = self.progress_tracker.start_tracking(
module="conflicts",
submodule="SourceTracker",
message=f"Tracking sources for {len(source_data)} items",
pipeline_id=pipeline_id,
)
stats = {
"entities_tracked": 0,
"properties_tracked": 0,
"relationships_tracked": 0,
"total_tracked": 0,
}
try:
for i, item in enumerate(source_data):
item_type = item.get("type", "property")
source_ref = item.get("source")
metadata = item.get("metadata", {})
# Convert source dict to SourceReference if needed
if isinstance(source_ref, dict):
source_ref = SourceReference(**source_ref)
elif not isinstance(source_ref, SourceReference):
self.logger.warning(f"Invalid source reference in item {i}: {item}")
continue
if item_type == "entity":
entity_id = item.get("entity_id")
if entity_id:
self.track_entity_source(entity_id, source_ref, **metadata)
stats["entities_tracked"] += 1
stats["total_tracked"] += 1
elif item_type == "property":
entity_id = item.get("entity_id")
property_name = item.get("property_name")
value = item.get("value")
if entity_id and property_name is not None:
self.track_property_source(
entity_id, property_name, value, source_ref, **metadata
)
stats["properties_tracked"] += 1
stats["total_tracked"] += 1
elif item_type == "relationship":
relationship_id = item.get("relationship_id")
if relationship_id:
self.track_relationship_source(
relationship_id, source_ref, **metadata
)
stats["relationships_tracked"] += 1
stats["total_tracked"] += 1
else:
self.logger.warning(
f"Unknown type '{item_type}' in item {i}, skipping"
)
# Update progress
self.progress_tracker.update_progress(
tracking_id,
processed=i + 1,
total=len(source_data),
message=f"Tracking sources {i+1}/{len(source_data)}...",
)
message = (
f"Tracked {stats['total_tracked']} items: "
f"{stats['entities_tracked']} entities, "
f"{stats['properties_tracked']} properties, "
f"{stats['relationships_tracked']} relationships"
)
self.progress_tracker.stop_tracking(
tracking_id, status="completed", message=message
)
return stats
except Exception as e:
self.progress_tracker.stop_tracking(
tracking_id, status="failed", message=str(e)
)
raise
+2 -2
View File
@@ -178,7 +178,7 @@ class AgentContext:
vs_path = os.path.join(path, "vector_store")
self.vector_store.save(vs_path)
# 3. Save KnowledgeGraph state
# 3. Save Knowledge Graph state
if self.knowledge_graph:
if hasattr(self.knowledge_graph, "save_to_file"):
# JSON export for ContextGraph
@@ -214,7 +214,7 @@ class AgentContext:
if os.path.exists(vs_path):
self.vector_store.load(vs_path)
# 3. Load KnowledgeGraph state
# 3. Load Knowledge Graph state
if self.knowledge_graph:
kg_json_path = os.path.join(path, "knowledge_graph.json")
kg_dir_path = os.path.join(path, "knowledge_graph")
+130 -30
View File
@@ -116,7 +116,7 @@ class Semantica:
self._modules["embedding_generator"] = EmbeddingGenerator(
config=self.config.get("embedding", {})
)
except ImportError as e:
except (ImportError, OSError) as e:
self.logger.warning(f"Could not import EmbeddingGenerator: {e}")
raise ProcessingError(f"Embeddings module not available: {e}")
return self._modules["embedding_generator"]
@@ -133,11 +133,87 @@ class Semantica:
try:
from ..reasoning import GraphReasoner
self._modules["reasoner"] = GraphReasoner(config=self.config)
except ImportError as e:
except (ImportError, OSError) as e:
self.logger.warning(f"Could not import GraphReasoner: {e}")
raise ProcessingError(f"Reasoning module not available: {e}")
return self._modules["reasoner"]
@property
def graph_builder(self) -> Any:
"""
Get the framework's knowledge graph builder.
Returns:
GraphBuilder instance
"""
if "graph_builder" not in self._modules:
try:
from ..kg import GraphBuilder
self._modules["graph_builder"] = GraphBuilder(
config=self.config.get("kg", {})
)
except (ImportError, OSError) as e:
self.logger.warning(f"Could not import GraphBuilder: {e}")
raise ProcessingError(f"KG module not available: {e}")
return self._modules["graph_builder"]
@property
def document_parser(self) -> Any:
"""
Get the framework's document parser.
Returns:
DocumentParser instance
"""
if "document_parser" not in self._modules:
try:
from ..parse import DocumentParser
self._modules["document_parser"] = DocumentParser(
config=self.config.get("parse", {})
)
except (ImportError, OSError) as e:
self.logger.warning(f"Could not import DocumentParser: {e}")
raise ProcessingError(f"Parse module not available: {e}")
return self._modules["document_parser"]
@property
def file_ingestor(self) -> Any:
"""
Get the framework's file ingestor.
Returns:
FileIngestor instance
"""
if "file_ingestor" not in self._modules:
try:
from ..ingest import FileIngestor
self._modules["file_ingestor"] = FileIngestor(
config=self.config.get("ingest", {})
)
except (ImportError, OSError) as e:
self.logger.warning(f"Could not import FileIngestor: {e}")
raise ProcessingError(f"Ingest module not available: {e}")
return self._modules["file_ingestor"]
@property
def pipeline_builder(self) -> Any:
"""
Get the framework's pipeline builder.
Returns:
PipelineBuilder instance
"""
if "pipeline_builder" not in self._modules:
try:
from ..pipeline import PipelineBuilder
self._modules["pipeline_builder"] = PipelineBuilder(
config=self.config.get("pipeline", {})
)
except (ImportError, OSError) as e:
self.logger.warning(f"Could not import PipelineBuilder: {e}")
raise ProcessingError(f"Pipeline module not available: {e}")
return self._modules["pipeline_builder"]
@log_execution_time
def initialize(self) -> None:
"""
@@ -203,6 +279,7 @@ class Semantica:
sources: List of data sources (files, URLs, streams)
**kwargs: Additional processing options:
- pipeline: Custom pipeline configuration
- pipeline_id: Optional pipeline ID for progress tracking
- embeddings: Whether to generate embeddings
- graph: Whether to build knowledge graph
- normalize: Whether to normalize data
@@ -220,11 +297,19 @@ class Semantica:
# Auto-initialize if not already initialized
self._ensure_initialized()
# Extract or generate pipeline_id
pipeline_id = kwargs.get("pipeline_id")
if not pipeline_id:
# Generate a unique pipeline ID
import uuid
pipeline_id = f"kb_build_{uuid.uuid4().hex[:8]}"
# Start overall progress tracking
overall_tracking_id = self.progress_tracker.start_tracking(
module="core",
submodule="Semantica",
message=f"Building knowledge base from {len(sources)} sources",
pipeline_id=pipeline_id,
)
try:
@@ -236,6 +321,23 @@ class Semantica:
# Create processing pipeline
pipeline_config = kwargs.get("pipeline", {})
pipeline = self._create_pipeline(pipeline_config)
# Register pipeline modules for progress tracking
if hasattr(pipeline, 'steps') and pipeline.steps:
module_list = []
module_order = {}
for idx, step in enumerate(pipeline.steps):
module_name = getattr(step, 'module', None) or getattr(step, 'name', None) or str(step)
if module_name and module_name not in module_list:
module_list.append(module_name)
module_order[module_name] = idx
if module_list:
self.progress_tracker.register_pipeline_modules(
pipeline_id=pipeline_id,
module_list=module_list,
module_order=module_order
)
# Process sources
results = []
@@ -251,6 +353,7 @@ class Semantica:
module="core",
submodule="build_knowledge_base",
message=f"Processing {Path(file_str).name if file_str else 'source'}",
pipeline_id=pipeline_id,
)
try:
result = self.run_pipeline(pipeline, source)
@@ -271,7 +374,7 @@ class Semantica:
# Update overall progress
if idx % update_interval == 0 or idx == total_sources:
self.progress_tracker.update_progress(
tracking_id,
overall_tracking_id,
processed=idx,
total=total_sources,
message=f"Processing sources... {idx}/{total_sources}"
@@ -308,6 +411,9 @@ class Semantica:
message=f"Processed {len(results)} sources",
)
# Clear pipeline context when complete
self.progress_tracker.clear_pipeline_context(pipeline_id)
# Show summary
self.progress_tracker.show_summary()
@@ -319,6 +425,7 @@ class Semantica:
"metadata": {
"sources": validated_sources,
"pipeline": pipeline_config,
"pipeline_id": pipeline_id,
},
}
@@ -326,6 +433,8 @@ class Semantica:
self.progress_tracker.stop_tracking(
overall_tracking_id, status="failed", message=str(e)
)
# Clear pipeline context on failure
self.progress_tracker.clear_pipeline_context(pipeline_id)
self.logger.error(f"Failed to build knowledge base: {e}")
raise ProcessingError(f"Failed to build knowledge base: {e}")
@@ -371,7 +480,7 @@ class Semantica:
if isinstance(pipeline, Pipeline):
execution_engine = ExecutionEngine()
except ImportError:
except (ImportError, OSError):
execution_engine = None
if execution_engine is None and not hasattr(pipeline, "execute"):
@@ -528,7 +637,7 @@ class Semantica:
from ..pipeline import PipelineBuilder
self.logger.debug("Framework modules verified and available")
except ImportError as e:
except (ImportError, OSError) as e:
# Log but don't fail - modules may be optional or not installed
self.logger.warning(
f"Some framework modules could not be imported: {e}. "
@@ -632,13 +741,12 @@ class Semantica:
Pipeline object or configuration dict (if pipeline module not available)
"""
try:
from ..pipeline import PipelineBuilder
pipeline_builder = PipelineBuilder()
# Use the lazy property
builder = self.pipeline_builder
if not pipeline_config:
pipeline_builder.add_step("default_step", "default")
return pipeline_builder.build("default_pipeline")
builder.add_step("default_step", "default")
return builder.build("default_pipeline")
steps_config = pipeline_config.get("steps")
@@ -655,14 +763,14 @@ class Semantica:
}
if "parallelism" in pipeline_config:
normalized_config["parallelism"] = pipeline_config["parallelism"]
return pipeline_builder.build_pipeline(normalized_config)
return builder.build_pipeline(normalized_config)
if "steps" in pipeline_config:
return pipeline_builder.build_pipeline(pipeline_config)
return builder.build_pipeline(pipeline_config)
pipeline_builder.add_step("default_step", "default")
return pipeline_builder.build("default_pipeline")
except ImportError:
builder.add_step("default_step", "default")
return builder.build("default_pipeline")
except (ImportError, OSError, ProcessingError):
self.logger.debug("Pipeline module not available, using config directly")
return pipeline_config
@@ -692,8 +800,6 @@ class Semantica:
Dictionary containing knowledge graph structure
"""
try:
from ..kg import GraphBuilder
# Extract entities and relationships from results
graph_sources = []
for result in results:
@@ -718,11 +824,8 @@ class Semantica:
)
try:
# Build knowledge graph
graph_builder = GraphBuilder(
merge_entities=True, resolve_conflicts=True
)
knowledge_graph = graph_builder.build(graph_sources)
# Build knowledge graph using the lazy property
knowledge_graph = self.graph_builder.build(graph_sources)
self.progress_tracker.stop_tracking(kg_tracking_id, status="completed")
return knowledge_graph
except Exception as e:
@@ -731,7 +834,7 @@ class Semantica:
)
raise
except ImportError:
except (ImportError, OSError, ProcessingError):
self.logger.warning("KG module not available, returning placeholder")
return {"status": "placeholder", "results": results}
@@ -749,8 +852,6 @@ class Semantica:
Dictionary containing generated embeddings
"""
try:
from ..embeddings import EmbeddingGenerator
# Extract text content from results
texts_to_embed = []
for result in results:
@@ -778,9 +879,8 @@ class Semantica:
)
try:
# Generate embeddings
embedding_generator = EmbeddingGenerator()
embeddings_result = embedding_generator.process_batch(texts_to_embed)
# Generate embeddings using the lazy property
embeddings_result = self.embedding_generator.process_batch(texts_to_embed)
self.progress_tracker.stop_tracking(
embed_tracking_id,
status="completed",
@@ -801,7 +901,7 @@ class Semantica:
)
raise
except ImportError:
except (ImportError, OSError, ProcessingError):
self.logger.warning(
"Embeddings module not available, returning placeholder"
)
@@ -905,7 +1005,7 @@ class Semantica:
# CPU percent may not be available immediately
pass
except ImportError:
except (ImportError, OSError):
# psutil not available, use basic metrics
self.logger.debug("psutil not available, using basic metrics")
except Exception as e:
+3 -1
View File
@@ -258,13 +258,14 @@ class EmbeddingGenerator:
return max(0.0, min(1.0, similarity))
def process_batch(
self, data_items: List[Union[str, Path]], **options
self, data_items: List[Union[str, Path]], pipeline_id: Optional[str] = None, **options
) -> Dict[str, Any]:
"""
Process multiple data items for embedding generation.
Args:
data_items: List of data items
pipeline_id: Optional pipeline ID for progress tracking
**options: Processing options
Returns:
@@ -275,6 +276,7 @@ class EmbeddingGenerator:
module="embeddings",
submodule="EmbeddingGenerator",
message=f"Batch of {len(data_items)} items",
pipeline_id=pipeline_id,
)
try:
+3 -3
View File
@@ -349,21 +349,21 @@ def check_available_providers() -> Dict[str, bool]:
try:
import sentence_transformers
providers["sentence_transformers"] = True
except ImportError:
except (ImportError, OSError):
providers["sentence_transformers"] = False
# Check FastEmbed
try:
import fastembed
providers["fastembed"] = True
except ImportError:
except (ImportError, OSError):
providers["fastembed"] = False
# Check OpenAI
try:
import openai
providers["openai"] = True
except ImportError:
except (ImportError, OSError):
providers["openai"] = False
return providers
+3 -3
View File
@@ -66,7 +66,7 @@ class OpenAIStore(ProviderStore):
from openai import OpenAI
self.client = OpenAI(api_key=self.api_key)
except ImportError:
except (ImportError, OSError):
self.logger.warning("OpenAI library not installed")
def embed(self, text: str, **options) -> np.ndarray:
@@ -114,7 +114,7 @@ class BGEStore(ProviderStore):
self.model = SentenceTransformer(self.model_name)
self.logger.info(f"Loaded BGE model: {self.model_name}")
except ImportError:
except (ImportError, OSError):
self.logger.warning("sentence-transformers not available for BGE")
except Exception as e:
self.logger.warning(f"Failed to load BGE model: {e}")
@@ -210,7 +210,7 @@ class FastEmbedStore(ProviderStore):
self.model = TextEmbedding(model_name=self.model_name)
self.logger.info(f"Loaded FastEmbed model: {self.model_name}")
except ImportError:
except (ImportError, OSError):
self.logger.warning(
"fastembed not available. Install with: pip install fastembed"
)
+33 -31
View File
@@ -34,14 +34,14 @@ try:
from sentence_transformers import SentenceTransformer
SENTENCE_TRANSFORMERS_AVAILABLE = True
except ImportError:
except (ImportError, OSError):
SENTENCE_TRANSFORMERS_AVAILABLE = False
try:
from fastembed import TextEmbedding
FASTEMBED_AVAILABLE = True
except ImportError:
except (ImportError, OSError):
FASTEMBED_AVAILABLE = False
@@ -107,6 +107,7 @@ class TextEmbedder:
# Initialize models (will be None if unavailable)
self.model = None
self.fastembed_model = None
self.embedding_dimension = None
# Initialize progress tracker
self.progress_tracker = get_progress_tracker()
@@ -121,6 +122,11 @@ class TextEmbedder:
If unavailable or loading fails, falls back to hash-based embedding method.
Logs warnings but doesn't raise errors to allow graceful degradation.
"""
# Clear previous models to avoid conflicts when switching
self.model = None
self.fastembed_model = None
self.embedding_dimension = None
if self.method == "fastembed":
if FASTEMBED_AVAILABLE:
try:
@@ -129,8 +135,17 @@ class TextEmbedder:
"fastembed_model_name", self.model_name
)
self.fastembed_model = TextEmbedding(model_name=fastembed_model_name)
# Detect dimension from model
try:
# FastEmbed doesn't expose dimension directly, so we generate a test embedding
test_emb = list(self.fastembed_model.embed(["test"]))[0]
self.embedding_dimension = len(test_emb)
except Exception:
self.embedding_dimension = 384
self.logger.info(
f"Loaded FastEmbed model: {fastembed_model_name}"
f"Loaded FastEmbed model: {fastembed_model_name} (dim: {self.embedding_dimension})"
)
except Exception as e:
self.logger.warning(
@@ -149,9 +164,10 @@ class TextEmbedder:
if SENTENCE_TRANSFORMERS_AVAILABLE:
try:
self.model = SentenceTransformer(self.model_name, device=self.device)
self.embedding_dimension = self.model.get_sentence_embedding_dimension()
self.logger.info(
f"Loaded sentence-transformers model: {self.model_name} "
f"(device: {self.device})"
f"(device: {self.device}, dim: {self.embedding_dimension})"
)
except Exception as e:
self.logger.warning(
@@ -165,19 +181,12 @@ class TextEmbedder:
"Install with: pip install sentence-transformers. "
"Using fallback embedding method."
)
def get_method(self) -> str:
"""Get current embedding method."""
return self.method
def get_model_info(self) -> Dict[str, Any]:
"""Get current model information."""
return {
"method": self.method,
"model_name": self.model_name,
"device": self.device,
"normalize": self.normalize
}
# If no model loaded, use fallback dimension
if self.embedding_dimension is None:
self.embedding_dimension = self.config.get("dimension", 128)
self.logger.debug(f"Initialized text embedder with dimension: {self.embedding_dimension}")
def set_model(self, method: str, model_name: str, **config) -> None:
"""
@@ -190,6 +199,10 @@ class TextEmbedder:
"""
self.method = method.lower()
self.model_name = model_name
# Update config with new values
self.config.update(config)
if "device" in config:
self.device = config["device"]
if "normalize" in config:
@@ -340,8 +353,8 @@ class TextEmbedder:
hash_val = int(hashlib.sha256(text.encode("utf-8")).hexdigest(), 16)
rng = np.random.RandomState(hash_val % (2**32))
# Determine dimension - use config or default
dim = self.config.get("dimension", 128)
# Determine dimension - use stored dimension or default
dim = self.embedding_dimension or 128
embedding = rng.rand(dim).astype(np.float32)
# Normalize if requested
@@ -470,18 +483,7 @@ class TextEmbedder:
>>> dim = embedder.get_embedding_dimension()
>>> print(f"Embedding dimension: {dim}")
"""
if self.fastembed_model:
# FastEmbed doesn't expose dimension directly, so we generate a test embedding
try:
test_emb = self._embed_with_fastembed("test")
return len(test_emb)
except Exception:
# Default FastEmbed dimension
return 384
elif self.model:
return self.model.get_sentence_embedding_dimension()
else:
return 128 # Fallback hash-based embedding dimension
return self.embedding_dimension or 128
def get_method(self) -> str:
"""
+1 -1
View File
@@ -4,7 +4,7 @@ Semantica Evals Module
Coming Soon
"""
__version__ = "0.1.0"
__version__ = "0.1.1"
__status__ = "coming_soon"
__all__ = []
+3 -3
View File
@@ -287,7 +287,7 @@ class VectorExporter:
"""
try:
import numpy as np
except ImportError:
except (ImportError, OSError):
raise ImportError("NumPy not installed. Install with: pip install numpy")
# Extract vectors and associated data
@@ -358,7 +358,7 @@ class VectorExporter:
"""Export to binary format."""
try:
import numpy as np
except ImportError:
except (ImportError, OSError):
raise ImportError("NumPy not installed. Install with: pip install numpy")
# Extract vectors
@@ -419,7 +419,7 @@ class VectorExporter:
try:
import faiss
import numpy as np
except ImportError:
except (ImportError, OSError):
raise ImportError(
"FAISS not installed. Install with: pip install faiss-cpu or faiss-gpu"
)

Some files were not shown because too many files have changed in this diff Show More