mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-09-10 04:00:35 +00:00
KaifAhmad1
6f726c708f
- visualization.md: GraphVisualizer → KGVisualizer; fix method names (visualize_network, visualize_network_evolution, visualize_snapshot_comparison, visualize_temporal_patterns, visualize_2d_projection); remove DistanceVisualizer tab; fix start_explorer() reference - kg.md: remove TemporalKnowledgeGraph and DistanceCalculator (don't exist); replace with TemporalGraphQuery and ConnectivityAnalyzer; fix query_at_time() signature - ontology.md: remove OntologyManager, SKOSVocabulary, OntologyAligner, OntologyDiff, OntologyMigrator (none exist); fix SHACLValidator → OntologyValidator; fix OWLExporter → OWLGenerator.export_owl(); fix start_explorer() reference - evals.md: replace entire file with coming-soon notice (module is a stub, __all__ = []) - embeddings.md: fix EmbeddingGenerator constructor (takes config dict not model=); generate() → generate_embeddings(); similarity() → compare_embeddings() - ingest.md: fix WebIngestor (rate_limit → delay, ingest() → ingest_url()); FeedIngestor (ingest() → ingest_feed(), monitor() → monitor_feeds()); StreamIngestor (backend= constructor → ingest_kafka/rabbitmq/kinesis/pulsar()); DBIngestor constructor + ingest() → ingest_database(); SnowflakeIngestor.ingest() → ingest_query()/ingest_table(); OntologyIngestor.ingest() → ingest_ontology(); DataSource → FileObject - explorer.md: remove start_explorer() Python function (only CLI exists); replace with semantica-explorer CLI usage - provenance.md: ActivityTracker → ProvenanceTracker in CardGroup - semantic_extract.md: EventExtractor → EventDetector - triplet_store.md: remove InMemoryTripletStore (doesn't exist); fix tip - llms.md: fix providers (Anthropic/Gemini/Ollama/DeepSeek/NovitaAI → LiteLLM); HuggingFace → HuggingFaceLLM; remove create_provider()
51 lines
2.0 KiB
Markdown
51 lines
2.0 KiB
Markdown
---
|
|
title: "Evals Module"
|
|
description: "Evaluation framework for measuring Knowledge Graph quality, extraction accuracy, and pipeline performance — coming soon."
|
|
icon: "chart-line"
|
|
---
|
|
|
|
`semantica.evals` is planned as a comprehensive evaluation framework for measuring extraction accuracy, graph quality, and pipeline performance.
|
|
|
|
<Warning>
|
|
**`semantica.evals` is not yet implemented.** The module exists as a placeholder (`__all__ = []`). No classes or functions are available for import. This page describes the planned API.
|
|
</Warning>
|
|
|
|
## Planned Features
|
|
|
|
When released, `semantica.evals` will provide:
|
|
|
|
- **KG quality metrics** — completeness, consistency, schema compliance, coverage, and orphan node detection
|
|
- **Extraction accuracy** — NER precision / recall / F1 and relation extraction metrics against gold-standard datasets
|
|
- **Pipeline benchmarking** — throughput (docs/sec), per-step latency, peak memory, and error rate
|
|
- **Regression tracking** — record runs and compare metrics across commits or config changes
|
|
- **Deduplication accuracy** — merge precision, false positive / false negative rates
|
|
- **Reasoning correctness** — inference accuracy, rule coverage, and derivation depth
|
|
|
|
## Current Workaround
|
|
|
|
Until `semantica.evals` ships, use `semantica.ontology.OntologyEvaluator` for ontology quality metrics:
|
|
|
|
```python
|
|
from semantica.ontology import OntologyEvaluator
|
|
|
|
evaluator = OntologyEvaluator()
|
|
report = evaluator.evaluate(ontology, kg)
|
|
print(f"Coverage: {report.coverage:.2%}")
|
|
print(f"Completeness: {report.completeness:.2%}")
|
|
```
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Semantic Extract" icon="magnifying-glass" href="semantic_extract">
|
|
Extraction module.
|
|
</Card>
|
|
<Card title="Knowledge Graph" icon="diagram-project" href="kg">
|
|
Graph quality assessment.
|
|
</Card>
|
|
<Card title="Pipeline" icon="gear" href="pipeline">
|
|
Pipeline performance metrics.
|
|
</Card>
|
|
<Card title="Ontology Evaluator" icon="sitemap" href="ontology">
|
|
Available now for ontology quality metrics.
|
|
</Card>
|
|
</CardGroup>
|