mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
KaifAhmad1
9113ef3428
- Rewrote all 26 reference module pages: removed blockquote taglines and horizontal rule separators, added "What You Get" bullet summaries, added constructor/method parameter tables, expanded thin files (graph_store, triplet_store, visualization, provenance) with full API coverage, added backend comparison tables and real-world usage patterns - Renamed Modules tab from "API Reference" and group from "Context & Knowledge" to "Context & Intelligence" in docs.json - Fixed logo: copied "Semantica Logo.png" to web-safe semantica-logo.png and updated all 4 references in docs.json - Improved core docs (index, modules, concepts, quickstart, installation, getting-started) with better fonts, bullet points, and complete module listings (mcp_server, evals, core, utils previously missing) - Rewrote community pages (community, community-projects, contributing-guide, use-cases, architecture, faq, learning-more, glossary) with heading hierarchy fixes, expanded definitions, and better structure - Fixed markdown linter warnings: MD036 bold-as-heading, MD001 heading skips, MD040 missing code fence language, MD032 blank lines around lists
1.9 KiB
1.9 KiB
title, description, icon
| title | description | icon |
|---|---|---|
| Evals Module | Evaluation framework for measuring Knowledge Graph quality, extraction accuracy, and pipeline performance. | chart-line |
semantica.evals provides a comprehensive evaluation framework for measuring extraction accuracy, graph quality, and pipeline performance. Use it to benchmark extractors, validate pipeline output, and track quality regressions across runs.
Planned Capabilities
The Evals module will cover five evaluation areas:
| Area | What It Measures |
|---|---|
| KG Quality | Completeness, consistency, schema compliance, coverage metrics |
| Extraction Accuracy | NER precision / recall / F1, relation extraction metrics |
| Pipeline Performance | Throughput (docs/sec), latency per step, error rates |
| Deduplication | Merge accuracy, false positive / negative rates |
| Reasoning | Inference correctness, rule coverage, derivation depth |
Scope
- Offline evaluation — compare against gold-standard annotated datasets
- Regression tracking — compare pipeline runs across commits or config changes
- Live monitoring — record quality metrics during production pipeline runs
- Benchmark suites — standard NER, RE, and KG construction benchmarks