Files
semantica/docs/reference/parse.md
T
KaifAhmad1 37e640e7b4 docs: comprehensive audit and DX overhaul of all reference modules
llms.md:
- Only Groq/OpenAI/LiteLLM/HuggingFaceLLM are exported — remove non-exported
  Anthropic/Ollama/Gemini/DeepSeek/Novita as direct imports
- Rename HuggingFace -> HuggingFaceLLM (correct class name)
- Remove non-existent create_provider() — replace with LiteLLM provider/model pattern
- Add LiteLLM 100+ providers section with provider/model string examples
- Add Exported Classes table (class -> provider -> API key)
- Update Provider Comparison table to show correct import per provider

ontology.md:
- Remove non-existent OntologyManager — replace with OntologyEngine facade
- Remove non-existent start_explorer() — replace with CLI: semantica-explorer
- SHACLValidator -> OntologyValidator (correct exported name)
- OWLExporter -> OWLGenerator (correct exported name)
- Add Exported Classes block with all 15+ exported symbols
- Add LLMOntologyGenerator section, NamespaceManager section
- Add OntologyEvaluator section with coverage/completeness metrics
- Add ingest_ontology() section
- Add versioning moved-to note (change_management module)

kg.md:
- TemporalKnowledgeGraph does not exist — replace with TemporalGraphQuery
- DistanceCalculator does not exist — replace with SimilarityCalculator
- Add Exported Classes block with all 20+ exported symbols
- Fix temporal example to use TemporalGraphQuery + TemporalVersionManager correctly
- Add SimilarityCalculator section with NodeEmbedder integration example

provenance.md:
- ActivityTracker not exported — remove; ProvenanceManager handles tracking
- Fix track_entity() signature: add source_location, source_quote params
- Fix GraphBuilderWithProvenance import: from semantica.kg, not semantica.provenance
- Add Exported Classes block with storage backends and checksum utilities
- Add SourceReference section with DOI/page/quote fields
- Add tamper-evident checksum section (compute_checksum/verify_checksum)
- Add Enable Provenance in Extractors section
- Fix duplicate heading (W3C PROV-O Export appeared twice)

reasoning.md:
- Add Exported Classes block with all engines + data types + explanation types
- Add Quick Start section
- Add Choosing an Engine comparison table
- Add InferenceResult/Explanation/ReasoningStep type annotations in examples
- Add Tip: use DatalogReasoner for recursive rules

semantic_extract.md:
- Add Exported Classes block with NamedEntityRecognizer, EventDetector, Entity,
  Relation, Event, CoreferenceChain, EntityClassifier, TemporalEventProcessor
- Add Quick Start section (one-liner extraction pipeline)
- Rename EventExtractor -> EventDetector (correct exported name)
- Clarify NERExtractor vs NamedEntityRecognizer distinction
- Add return type annotations to EventDetector example

core.md:
- Add Exported Classes block
- Add When to Use Core vs. Individual Modules decision table
- Add Tip: LifecycleManager only for long-running apps
- Fix MethodRegistry example to import build_knowledge_base correctly

parse.md:
- Add Exported Classes block with all format-specific parsers + data types
- Add DoclingParser optional import note

utils.md:
- Add Exported Classes block with logging/validation/progress/helpers/exceptions

deduplication.md:
- Add Exported Classes block with PropertyMergeRule, MergeStrategyManager,
  method_registry, and all convenience functions

export.md:
- Add Exported Classes block with all exporters, NamespaceManager,
  SemanticNetworkYAMLExporter, and all convenience functions
2026-05-24 14:41:57 +05:30

164 lines
5.3 KiB
Markdown

---
title: "Parse Module"
description: "Document parsing and text extraction — DocumentParser for standard formats and DoclingParser for complex layouts."
icon: "file-lines"
---
`semantica.parse` extracts structured text, layout, tables, and metadata from unstructured documents. `DocumentParser` handles clean machine-readable files; `DoclingParser` handles complex layouts, scanned PDFs, and multi-column documents.
## Exported Classes
```python
from semantica.parse import (
DocumentParser, # auto-detect format — delegates to format-specific parser
PDFParser, # PDF text extraction
DOCXParser, # Word .docx documents
HTMLParser, # HTML / web pages
MarkdownParser, # Markdown files
TXTParser, # plain text
JSONParser, # JSON documents
XMLParser, # XML documents
CSVParser, # CSV / TSV files
WebParser, # URL fetch + HTML parsing
EmailParser, # .eml / .msg email files
CodeParser, # source code files
# Data types
ParsedDocument, # {text, sections, tables, metadata, source_id}
DocumentMetadata, # {title, author, created_date, page_count, language, ...}
)
# Optional — requires: pip install "semantica[docling]"
from semantica.parse import DoclingParser # advanced OCR + layout analysis
```
## What You Get
- **`DocumentParser`** — standard parser for PDF, DOCX, HTML, TXT, JSON, CSV, PPTX, XLSX — auto-detects format
- **`DoclingParser`** — advanced parser for complex layouts, merged-cell tables, multi-column PDFs, and OCR (optional dep)
- **`ParsedDocument`** — structured output with `text`, `sections`, `tables`, and `metadata`
- **Format-specific parsers** — `PDFParser`, `DOCXParser`, `HTMLParser`, `WebParser`, `EmailParser`, `CodeParser`, etc.
## DocumentParser
Standard parser for clean, machine-readable documents:
```python
from semantica.parse import DocumentParser
parser = DocumentParser()
parsed = parser.parse("data/report.pdf")
print(parsed.text) # full clean text
print(parsed.metadata) # title, author, date, page_count, language, etc.
print(parsed.sections) # document structure as a list of Section objects
```
Supported formats: PDF, DOCX, HTML, TXT, JSON, CSV, PPTX, XLSX.
## DoclingParser
Advanced parser using the Docling backend — handles layouts that `DocumentParser` cannot:
```bash
pip install "semantica[docling]"
```
```python
from semantica.parse import DoclingParser
parser = DoclingParser(
extract_tables=True, # structured table extraction with cell type detection
extract_images=True, # extract image regions for downstream OCR
output_format="markdown", # "markdown" | "html" | "json"
)
parsed = parser.parse("data/annual_report.pdf")
print(parsed.text) # full clean text
print(parsed.tables) # structured TableData objects with headers and rows
print(parsed.sections) # document structure with heading hierarchy
```
Use `DoclingParser` for:
- Multi-column PDF layouts
- Tables with merged cells or complex headers
- PPTX slides with embedded charts
- XLSX spreadsheets with formulas
- Scanned documents with OCR
- Academic papers and technical reports
## OCR Support
```python
parser = DoclingParser(
ocr=True,
ocr_language=["en"], # ISO 639-1 codes; list for multi-language documents
extract_tables=True,
)
parsed = parser.parse("data/scanned_contract.pdf")
```
## Parsed Document Object
Both parsers return a `ParsedDocument` with the same structure:
```python
@dataclass
class ParsedDocument:
text: str # full extracted text
sections: List[Section] # heading-based document structure
tables: List[TableData] # structured table data (DoclingParser only)
metadata: DocumentMetadata # title, author, dates, page count
source_id: str # links back to the original DataSource
@dataclass
class DocumentMetadata:
title: Optional[str]
author: Optional[str]
created_date: Optional[datetime]
page_count: int
language: Optional[str] # ISO 639-1 code
has_tables: bool
has_images: bool
word_count: int
format: str # "pdf" | "docx" | "pptx" | ...
```
## Integration with FileIngestor
The most common pattern — ingest a directory then parse each source:
```python
from semantica.ingest import FileIngestor
from semantica.parse import DoclingParser
ingestor = FileIngestor()
parser = DoclingParser(extract_tables=True)
sources = ingestor.ingest("data/reports/")
for source in sources:
parsed = parser.parse(source)
# → parsed.text, parsed.tables, parsed.sections
```
<Note>
Docling is an optional dependency. If `docling` is not installed, `DoclingParser` raises an `ImportError` with installation instructions. `DocumentParser` is always available and requires no extras.
</Note>
<CardGroup cols={2}>
<Card title="Ingest" icon="database" href="ingest">
Load files before parsing.
</Card>
<Card title="Split" icon="scissors" href="split">
Chunk parsed text for embedding and extraction.
</Card>
<Card title="Docling Integration" icon="file-pdf" href="../integrations/docling">
Full Docling integration setup guide.
</Card>
<Card title="Semantic Extract" icon="magnifying-glass" href="semantic_extract">
Extract entities and relations from parsed text.
</Card>
</CardGroup>