Commit Graph
31 Commits
Author SHA1 Message Date
Mohd Kaif 25289023fe docs: replace all CardGroup/Card blocks with animated bullet points across all 50 docs pages (#648)
- Fix What's new → link in Info banner (now a proper <a> tag, always clickable)
- Replace 4-stat CardGroup on index with inline premium stats row
- Convert every <CardGroup>/<Card> block site-wide to markdown bullet lists:
  content sections → bold-title bullets with sub-bullets, nav cards → [Title](href) — description
- Add cursor-animated list item hover effects to custom.css:
  green inset left border, subtle background tint, marker color change on hover
- Affects index, getting-started, quickstart, concepts, modules, faq, architecture,
  installation, cookbook, glossary, learning-more, explorer-setup, cli-setup,
  community, contributing-guide, governance, citation, project-license,
  all integrations pages, and all 20+ reference module pages
2026-06-17 23:18:40 +05:30
Mohd Kaif a326c7d3bd Fix/mintlify theme (#645)
* fix: replace invalid Mintlify theme 'venus' with 'mint'

* docs: replace em dashes with colons across all docs files

* fix: strip UTF-8 BOM from all docs files (broke frontmatter detection)
2026-06-17 13:26:19 +05:30
Mohd Kaif 1f3cea5f0a docs: upgrade all docs pages with Mintlify premium components (#642)
Replace plain markdown lists, tables, and numbered steps with interactive
Mintlify v3 MDX components across all 50+ documentation files:

- Tabs: provider/parser/method selection guides, citation formats, component details
- Steps: setup flows, pipeline stages, connection initialization
- CardGroup/Card: feature overviews, "what you get" sections, navigation footers
- AccordionGroup: FAQ entries
- Check/Warning/Tip/Note/Info: callouts replacing plain bold text and inline notes

Files improved span the full docs surface: reference modules (context, llms,
kg, reasoning, embeddings, deduplication, provenance, parse, ontology, core,
semantic_extract), integrations (agno, docling, snowflake), graph/vector
store backends (apache_age, pgvector), and top-level guides (contributing,
governance, glossary, citation, community-projects, learning-more).
2026-06-17 12:58:44 +05:30
KaifAhmad1 850f47625f docs: address review feedback across 7 modules
- llms.md: use showcase models (llama-3.3-70b-versatile, gpt-4o) in
  provider examples and use-case tables; clarify defaults vs recommended
  in Defaults and Reproducibility section
- split.md: document that chunk_size is in characters with migration note
- ingest.md: add Note that glob patterns are not supported by ingest()
- explorer-setup.md: remove hardcoded "1.5 seconds" timing claim
- cli-setup.md: expand semantica-worker description with concrete usage
- mcp_server.md: clarify turtle/ttl are aliases for the same RDF format
- semantic_extract.md: remove emoji from code comments
2026-06-16 21:49:20 +05:30
Sameer6305 366add9fa0 docs(semantic_extract): improve onboarding and workflow guidance 2026-06-16 11:18:18 +05:30
Mohd Kaif 5d70d0c10d docs: replace Exported Classes import blocks with summary tables (all 25 modules) (#567)
* docs: replace Exported Classes import blocks with summary tables across all 25 modules

* docs: add method/parameter tables to parse, ingest, ontology, normalize, triplet_store, change_management, conflicts, export, graph_store, provenance, and semantic_extract modules
2026-05-24 15:49:58 +05:30
KaifAhmad1 beacc88b02 fix(ci): replace list[Event] with List[Event] for Python 3.8 compat 2026-05-24 14:46:50 +05:30
KaifAhmad1 37e640e7b4 docs: comprehensive audit and DX overhaul of all reference modules
llms.md:
- Only Groq/OpenAI/LiteLLM/HuggingFaceLLM are exported — remove non-exported
  Anthropic/Ollama/Gemini/DeepSeek/Novita as direct imports
- Rename HuggingFace -> HuggingFaceLLM (correct class name)
- Remove non-existent create_provider() — replace with LiteLLM provider/model pattern
- Add LiteLLM 100+ providers section with provider/model string examples
- Add Exported Classes table (class -> provider -> API key)
- Update Provider Comparison table to show correct import per provider

ontology.md:
- Remove non-existent OntologyManager — replace with OntologyEngine facade
- Remove non-existent start_explorer() — replace with CLI: semantica-explorer
- SHACLValidator -> OntologyValidator (correct exported name)
- OWLExporter -> OWLGenerator (correct exported name)
- Add Exported Classes block with all 15+ exported symbols
- Add LLMOntologyGenerator section, NamespaceManager section
- Add OntologyEvaluator section with coverage/completeness metrics
- Add ingest_ontology() section
- Add versioning moved-to note (change_management module)

kg.md:
- TemporalKnowledgeGraph does not exist — replace with TemporalGraphQuery
- DistanceCalculator does not exist — replace with SimilarityCalculator
- Add Exported Classes block with all 20+ exported symbols
- Fix temporal example to use TemporalGraphQuery + TemporalVersionManager correctly
- Add SimilarityCalculator section with NodeEmbedder integration example

provenance.md:
- ActivityTracker not exported — remove; ProvenanceManager handles tracking
- Fix track_entity() signature: add source_location, source_quote params
- Fix GraphBuilderWithProvenance import: from semantica.kg, not semantica.provenance
- Add Exported Classes block with storage backends and checksum utilities
- Add SourceReference section with DOI/page/quote fields
- Add tamper-evident checksum section (compute_checksum/verify_checksum)
- Add Enable Provenance in Extractors section
- Fix duplicate heading (W3C PROV-O Export appeared twice)

reasoning.md:
- Add Exported Classes block with all engines + data types + explanation types
- Add Quick Start section
- Add Choosing an Engine comparison table
- Add InferenceResult/Explanation/ReasoningStep type annotations in examples
- Add Tip: use DatalogReasoner for recursive rules

semantic_extract.md:
- Add Exported Classes block with NamedEntityRecognizer, EventDetector, Entity,
  Relation, Event, CoreferenceChain, EntityClassifier, TemporalEventProcessor
- Add Quick Start section (one-liner extraction pipeline)
- Rename EventExtractor -> EventDetector (correct exported name)
- Clarify NERExtractor vs NamedEntityRecognizer distinction
- Add return type annotations to EventDetector example

core.md:
- Add Exported Classes block
- Add When to Use Core vs. Individual Modules decision table
- Add Tip: LifecycleManager only for long-running apps
- Fix MethodRegistry example to import build_knowledge_base correctly

parse.md:
- Add Exported Classes block with all format-specific parsers + data types
- Add DoclingParser optional import note

utils.md:
- Add Exported Classes block with logging/validation/progress/helpers/exceptions

deduplication.md:
- Add Exported Classes block with PropertyMergeRule, MergeStrategyManager,
  method_registry, and all convenience functions

export.md:
- Add Exported Classes block with all exporters, NamespaceManager,
  SemanticNetworkYAMLExporter, and all convenience functions
2026-05-24 14:41:57 +05:30
KaifAhmad1 daa79ccef3 fix: audit and correct all remaining API mismatches in docs
- llms.md: replace non-exported Anthropic/Ollama imports with LiteLLM provider-prefix pattern; replace ReasoningEngine with Reasoner; replace create_provider with LiteLLM in YAML config example and tip
- concepts.md: replace ReasoningEngine with Reasoner/ReteEngine/GraphReasoner; fix DatalogReasoner.reason() to evaluate()/query(); replace TemporalKnowledgeGraph with TemporalGraphQuery; replace DistanceCalculator with SimilarityCalculator; replace EntityDeduplicator with DuplicateDetector/EntityMerger
- kg.md: replace non-exported build_knowledge_graph with method_registry.execute()
- semantic_extract.md: replace Anthropic import with LiteLLM
- index.md: replace Anthropic/Ollama imports with LiteLLM
- modules.md: fix TemporalKnowledgeGraph, DistanceCalculator, OntologyManager, ReasoningEngine, DatalogEngine, start_explorer, create_provider across code examples and module index table
- triplet_store.md: replace non-exported NamespacePrefixManager with semantica.ontology.NamespaceManager
2026-05-24 14:28:36 +05:30
KaifAhmad1 6f726c708f fix: remove non-existent classes and fix wrong API signatures across reference docs
- visualization.md: GraphVisualizer → KGVisualizer; fix method names (visualize_network,
  visualize_network_evolution, visualize_snapshot_comparison, visualize_temporal_patterns,
  visualize_2d_projection); remove DistanceVisualizer tab; fix start_explorer() reference
- kg.md: remove TemporalKnowledgeGraph and DistanceCalculator (don't exist); replace with
  TemporalGraphQuery and ConnectivityAnalyzer; fix query_at_time() signature
- ontology.md: remove OntologyManager, SKOSVocabulary, OntologyAligner, OntologyDiff,
  OntologyMigrator (none exist); fix SHACLValidator → OntologyValidator; fix OWLExporter
  → OWLGenerator.export_owl(); fix start_explorer() reference
- evals.md: replace entire file with coming-soon notice (module is a stub, __all__ = [])
- embeddings.md: fix EmbeddingGenerator constructor (takes config dict not model=);
  generate() → generate_embeddings(); similarity() → compare_embeddings()
- ingest.md: fix WebIngestor (rate_limit → delay, ingest() → ingest_url());
  FeedIngestor (ingest() → ingest_feed(), monitor() → monitor_feeds());
  StreamIngestor (backend= constructor → ingest_kafka/rabbitmq/kinesis/pulsar());
  DBIngestor constructor + ingest() → ingest_database(); SnowflakeIngestor.ingest() →
  ingest_query()/ingest_table(); OntologyIngestor.ingest() → ingest_ontology();
  DataSource → FileObject
- explorer.md: remove start_explorer() Python function (only CLI exists);
  replace with semantica-explorer CLI usage
- provenance.md: ActivityTracker → ProvenanceTracker in CardGroup
- semantic_extract.md: EventExtractor → EventDetector
- triplet_store.md: remove InMemoryTripletStore (doesn't exist); fix tip
- llms.md: fix providers (Anthropic/Gemini/Ollama/DeepSeek/NovitaAI → LiteLLM);
  HuggingFace → HuggingFaceLLM; remove create_provider()
2026-05-24 13:11:57 +05:30
KaifAhmad1 5eefadaa7f docs: apply full Mintlify component overhaul to all 27 reference pages and concepts.md
Replace plain markdown in every docs/reference/ file and docs/concepts.md with
rich Mintlify JSX components — CardGroup, Steps, Tabs, AccordionGroup, Tip,
Warning, Note, and CodeGroup — for a consistent, navigable, production-grade
developer experience.
2026-05-23 23:02:03 +05:30
KaifAhmad1 98bc2de20b docs: add SVG diagrams and Semantica wordmark logo
Diagrams (docs/assets/img/diagrams/):
- architecture-overview.svg: 4-column layered architecture
- pipeline-flow.svg: 8-step numbered pipeline flow
- kg-structure.svg: entity/relation graph with typed nodes and labeled edges
- graphrag-flow.svg: dual-path retrieval (vector + graph) to LLM to grounded answer
- extraction-pipeline.svg: NER/Relation/Coreference fan-out to Triplet Generator
- agent-context-flow.svg: AgentContext hub with VectorStore and ContextGraph
- reasoning-chain.svg: forward-chaining inference with explanation path

Wordmark logo (light + dark SVG variants):
- Green rounded-square S icon + Semantica text in green
- docs.json updated to use wordmark SVGs for light and dark modes

Pages updated with diagrams:
- index.md, architecture.md, quickstart.md, concepts.md
- reference/kg.md, reference/pipeline.md, reference/semantic_extract.md
- reference/context.md, reference/reasoning.md
2026-05-23 17:04:52 +05:30
KaifAhmad1 9113ef3428 docs: premium overhaul of all reference pages and core docs
- Rewrote all 26 reference module pages: removed blockquote taglines and
  horizontal rule separators, added "What You Get" bullet summaries,
  added constructor/method parameter tables, expanded thin files
  (graph_store, triplet_store, visualization, provenance) with full API
  coverage, added backend comparison tables and real-world usage patterns
- Renamed Modules tab from "API Reference" and group from "Context &
  Knowledge" to "Context & Intelligence" in docs.json
- Fixed logo: copied "Semantica Logo.png" to web-safe semantica-logo.png
  and updated all 4 references in docs.json
- Improved core docs (index, modules, concepts, quickstart, installation,
  getting-started) with better fonts, bullet points, and complete module
  listings (mcp_server, evals, core, utils previously missing)
- Rewrote community pages (community, community-projects, contributing-guide,
  use-cases, architecture, faq, learning-more, glossary) with heading
  hierarchy fixes, expanded definitions, and better structure
- Fixed markdown linter warnings: MD036 bold-as-heading, MD001 heading
  skips, MD040 missing code fence language, MD032 blank lines around lists
2026-05-23 13:10:09 +05:30
KaifAhmad1andClaude Sonnet 4.6 946a1089c8 docs: premium redesign — Mintlify v4, dark/cream theme, full module coverage
- Migrate from mint.json to docs.json (Mintlify v4)
- Theme: maple, emerald green + near-black dark / cream light palette
  (#059669 primary, #0A0A0A dark bg, #FAF7F0 light bg)
- Typography: Lexend headings, Inter body
- 5-tab navigation: Documentation, Quick Start, API Reference, Cookbook, FAQ
- Homepage: removed badge stickers, redundant h2, added blockquote tagline,
  full 27-module reference table with semantica.mcp_server added
- quickstart.md: CodeGroup per pipeline step, pattern vs LLM options,
  AccordionGroup for patterns and troubleshooting
- faq.md: full AccordionGroup structure across 5 sections
- reference/explorer.md: NEW — FastAPI explorer, Ontology Hub, Distance
  Intelligence, CLI reference, REST API endpoints
- reference/mcp_server.md: NEW — MCP stdio server, 12 tools with I/O
  examples, 3 resources, Claude Desktop/VS Code/Windsurf/Cline config
- docs.json: explorer added to Output group, mcp_server to Utilities group
- Chat, feedback (thumbs/suggest/raise), OG/Twitter metadata, search topbar
- All reference pages reformatted with Mintlify JSX components

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 21:52:50 +05:30
KaifAhmad1 b3a239ccb1 feat: enhance semantic extraction with BYOM support, NER aggregation, RE implementation, and Triplet improvements
- Implemented 'Bring Your Own Model' (BYOM) support for NER, Relation, and Triplet extraction
- Added NER aggregation strategies (simple, max, average)
- Implemented Relation Extraction via Sequence Classification with entity markers
- Enhanced Triplet Extraction with REBEL post-processing and lazy loading
- Updated all extractors to prioritize runtime options over config defaults
- Added extensive tests and examples (huggingface_demo.py)
- Updated documentation and CHANGELOG
2026-01-24 20:53:21 +05:30
KaifAhmad1 dd7fcd3ddb [FEATURE] Performance Bottlenecks and Scaling Limitations in semantic_extract #186
- Implemented high-throughput parallel batch processing across all core extractors (NERExtractor, RelationExtractor, TripletExtractor, EventDetector, SemanticNetworkExtractor) using ThreadPoolExecutor.

- Added max_workers configuration parameter (default: 1) to all extractor extract() methods.

- Implemented parallel processing for large document chunking in _extract_entities_chunked and _extract_relations_chunked.

- Enhanced ProgressTracker to be thread-safe.

- Optimized setUpClass in tests to reduce Groq LLM initialization overhead.

- Updated documentation and usage examples.
2026-01-14 00:11:30 +05:30
KaifAhmad1 8c3f7f1f0a fix(semantic-extract): resolve incomplete output #176, relax constraints, and add Groq support 2026-01-12 17:15:21 +05:30
KaifAhmad1 f338b66274 feat: Add provenance metadata and robust fallback chains to semantic extraction
- Implemented ML/LLM -> Pattern -> Last Resort fallback chains for NER, Relation, and Triplet extractors to prevent empty results.
- Added provenance metadata (batch_index, document_id) to all extraction schemas (Entity, Relation, Triplet, etc.).
- Unified batch processing API with progress tracking across all extractors.
- Updated documentation (module usage and reference docs) to reflect new features.
- Added robustness and batch provenance tests.
2026-01-10 20:43:09 +05:30
KaifAhmad1 a5c531fd29 Fix semantic extraction empty returns, schema validation, and update docs 2026-01-09 18:39:09 +05:30
KaifAhmad1 e712949872 Enhance LLM extraction methods with auto-chunking, robust parsing and improved diagnostics (#149) 2026-01-07 03:03:19 +05:30
KaifAhmad1 04eea7e7eb Update documentation: reduce code examples, add cookbook links, improve structure
- Reduced code examples in all guide pages (getting-started, quickstart, concepts, modules, examples, use-cases, learning-more)
- Added comprehensive cookbook links with descriptions (topics, difficulty, time, use cases)
- Improved structure and organization across all guide pages
- Updated use-cases.md to only include use cases with corresponding cookbooks
- Removed 'Last Updated: 2024' from all documentation files
- Enhanced navigation with better 'Next Steps' sections
2025-12-31 15:19:08 +05:30
KaifAhmad1 5fc188b9eb Refactor: Rename LLMEnhancer to LLMExtraction 2025-12-19 00:16:44 +05:30
KaifAhmad1 4fdc483935 Refactor terminology: Triple -> Triplet across codebase, docs, and notebooks 2025-12-15 16:09:20 +05:30
KaifAhmad1 d3366bbcf0 Refactor Semantic Extract module: Update notebooks, docs, and implementation to use class-based interfaces 2025-12-12 13:23:37 +05:30
KaifAhmad1 cf56dad82a docs: update cookbook links to absolute GitHub URLs in reference docs 2025-12-09 16:51:28 +05:30
KaifAhmad1 31da5731b1 refactor(semantic_extract): remove build function and enhance documentation
BREAKING CHANGE: Removed build() convenience function from semantic_extract module

- Removed build() function from semantic_extract/__init__.py
- Updated __all__ exports to remove 'build'
- Resolved merge conflicts in named_entity_recognizer.py, relation_extractor.py, triple_extractor.py
- Updated semantic_extract_usage.md with class-based examples
- Updated docs/reference/semantic_extract.md with detailed parameter documentation
- Fixed 01_GraphRAG_Complete.ipynb to use individual extractor classes
- Enhanced 05_Entity_Extraction.ipynb with comprehensive examples (9 sections)
- Enhanced 06_Relation_Extraction.ipynb with complete pipeline examples (9 sections)

Users should now use individual classes (NERExtractor, RelationExtractor, TripleExtractor, etc.)
instead of the build() function for better control and flexibility.

Migration guide available in documentation.
2025-12-08 17:33:45 +05:30
KaifAhmad1 6ebc094f56 Merge main into staging, resolve conflicts keeping local changes 2025-12-02 15:28:41 +05:30
KaifAhmad1 f1699c4f80 feat(semantic_extract): Update module with explicit parameters and docs
## Code Changes
- NamedEntityRecognizer: Added methods, confidence_threshold, merge_overlapping, include_standard_types
- RelationExtractor: Added relation_types, bidirectional, confidence_threshold, max_distance
- EventDetector: Added event_types, extract_participants, extract_location, extract_time
- TripleExtractor: Added include_temporal, include_provenance
- CoreferenceResolver: Added resolve() alias method
- Removed deprecated build() functions from all modules

## Documentation Changes
- docs/reference/semantic_extract.md: Added parameter tables and detailed examples
- docs/reference/kg.md: Updated examples after build removal
- docs/reference/embeddings.md: Updated examples after build removal
- docs/concepts.md: Updated GraphRAG and core concepts examples
- docs/LIBS_README.md: Updated all references to build functions
2025-12-02 15:22:44 +05:30
KaifAhmad1 a8edbfbfb3 docs: enhance reference documentation for all modules 2025-11-30 19:23:25 +05:30
KaifAhmad1 409f3fdd27 Enhance documentation visual appeal and fix build errors
- Added grid‑card layouts, icons, and admonitions to all reference pages
- Removed all mkdocstrings ::: directives that caused alias‑resolution errors
- Fixed export module alias issue (RDFExporter) by cleaning the docs
- Updated README‑style sections for better readability
- Added a PowerShell helper script (remove_mkdocstrings.ps1) for future clean‑ups
2025-11-23 19:23:13 +05:30
KaifAhmad1 b716b17bde Upgrade documentation: mkdocstrings, jupyter notebooks, and theme fix 2025-11-22 14:49:08 +05:30