mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
docs(cookbook): add Seed Data module notebook (#992)
* docs(cookbook): add Seed Data module notebook Add cookbook/introduction/25_Seed_Data.ipynb covering the seed module with verified, executable examples: - SeedDataManager.register_source with a CSV source - load_source record enrichment (entity_type/source provenance) - create_foundation_graph entity/relationship/metadata structure - validate_quality gating The seed module ships seed_usage.md but has no cookbook coverage. All API calls and outputs were executed against semantica/seed/seed_manager.py. Signed-off-by: LeonSGP43 <LeonSGP43@users.noreply.github.com> * docs(cookbook): isolate seed CSV in a temp dir and execute notebook in Jupyter - Write companies.csv into a session-scoped tempfile.mkdtemp() directory instead of the working directory, so a user's existing companies.csv can never be silently clobbered (review finding) - Run the notebook through a fresh Jupyter kernel (restart + run all + save): real execution counts, print() cells saved as stream outputs Signed-off-by: LeonSGP43 <cine.dreamer.one@gmail.com> --------- Signed-off-by: LeonSGP43 <LeonSGP43@users.noreply.github.com> Signed-off-by: LeonSGP43 <cine.dreamer.one@gmail.com> Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
This commit is contained in:
LeonSGP
@@ -0,0 +1,314 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6eb4dfba",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"[](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/25_Seed_Data.ipynb)\n",
|
||||
"\n",
|
||||
"# Seed Data — Practical Guide\n",
|
||||
"\n",
|
||||
"The `seed` module bootstraps a knowledge graph from **trusted, pre-known data** (CSV/JSON/database/API sources) before any extraction runs. This gives extraction a foundation to link against instead of starting from an empty graph.\n",
|
||||
"\n",
|
||||
"Key pieces:\n",
|
||||
"\n",
|
||||
"- **`SeedDataManager`** — registers data sources and builds foundation graphs\n",
|
||||
"- **`create_foundation_graph()`** — turns registered sources into `entities` + `relationships` + `metadata`\n",
|
||||
"- **`validate_quality()`** — checks a foundation graph before you commit it\n",
|
||||
"\n",
|
||||
"All examples below were executed against `semantica/seed/seed_manager.py`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "32f80cc6",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2026-08-26T18:51:18.716466Z",
|
||||
"iopub.status.busy": "2026-08-26T18:51:18.716264Z",
|
||||
"iopub.status.idle": "2026-08-26T18:51:20.533828Z",
|
||||
"shell.execute_reply": "2026-08-26T18:51:20.531402Z"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install -q semantica"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "75136e5f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1) Prepare a seed CSV and register the source\n",
|
||||
"\n",
|
||||
"`register_source(name, format, location, entity_type=...)` records where trusted data lives. `verified=True` (the default) marks the source as pre-validated."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "a8089e1f",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2026-08-26T18:51:20.538772Z",
|
||||
"iopub.status.busy": "2026-08-26T18:51:20.538323Z",
|
||||
"iopub.status.idle": "2026-08-26T18:51:20.675403Z",
|
||||
"shell.execute_reply": "2026-08-26T18:51:20.674060Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"True"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import csv\n",
|
||||
"import tempfile\n",
|
||||
"from pathlib import Path\n",
|
||||
"from semantica.seed import SeedDataManager\n",
|
||||
"\n",
|
||||
"# Write the sample CSV into a session-scoped temp directory so we never\n",
|
||||
"# clobber a companies.csv that might exist in the user's working directory.\n",
|
||||
"seed_csv = Path(tempfile.mkdtemp(prefix=\"semantica-seed-\")) / \"companies.csv\"\n",
|
||||
"with open(seed_csv, \"w\", newline=\"\") as f:\n",
|
||||
" writer = csv.DictWriter(f, fieldnames=[\"id\", \"name\", \"type\", \"industry\"])\n",
|
||||
" writer.writeheader()\n",
|
||||
" writer.writerow({\"id\": \"c1\", \"name\": \"Acme\", \"type\": \"Company\", \"industry\": \"robotics\"})\n",
|
||||
" writer.writerow({\"id\": \"c2\", \"name\": \"Globex\", \"type\": \"Company\", \"industry\": \"energy\"})\n",
|
||||
"\n",
|
||||
"manager = SeedDataManager()\n",
|
||||
"manager.register_source(\"companies\", format=\"csv\", location=str(seed_csv), entity_type=\"Company\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e87221ba",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2) Load records from a registered source\n",
|
||||
"\n",
|
||||
"`load_source(name)` reads the source and enriches each record with `entity_type` and `source` provenance keys."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "f932e550",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2026-08-26T18:51:20.679424Z",
|
||||
"iopub.status.busy": "2026-08-26T18:51:20.679156Z",
|
||||
"iopub.status.idle": "2026-08-26T18:51:20.690659Z",
|
||||
"shell.execute_reply": "2026-08-26T18:51:20.688812Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div style='font-family: monospace;'><h4>🧠 Semantica - 📊 Current Progress</h4><table style='width: 100%; border-collapse: collapse;'><tr><th>Status</th><th>Action</th><th>Module</th><th>Submodule</th><th>Progress</th><th>ETA</th><th>Rate</th><th>Time</th><th>Extracted</th></tr><tr><td>✅</td><td>Semantica is seeding</td><td>🌱 seed</td><td>SeedDataManager</td><td>100.0%</td><td>-</td><td>-</td><td>0.00s</td><td>-</td></tr></table></div>"
|
||||
],
|
||||
"text/plain": [
|
||||
"<IPython.core.display.HTML object>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"🔄 Semantica is seeding: Loading seed data from CSV: /var/folders/7s/bvvstgs10y963tz6_4bbnklr0000gn/T/semantica-seed-eu9__ep1/companies.csv 🌱 seed SeedDataManager |░░░░░░░░░░░░░░░| 0.0% ETA: - Rate: - Time: 0.00s Extracted: -"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"loaded 2 records\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'id': 'c1',\n",
|
||||
" 'name': 'Acme',\n",
|
||||
" 'type': 'Company',\n",
|
||||
" 'industry': 'robotics',\n",
|
||||
" 'entity_type': 'Company',\n",
|
||||
" 'source': 'companies'}"
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"records = manager.load_source(\"companies\")\n",
|
||||
"print(f\"loaded {len(records)} records\")\n",
|
||||
"records[0]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f2ebce64",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 3) Build the foundation graph\n",
|
||||
"\n",
|
||||
"`create_foundation_graph()` converts every registered source into graph-ready entities and relationships. Entities carry `confidence: 1.0` — seed data is trusted by definition."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "09388c31",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2026-08-26T18:51:20.695259Z",
|
||||
"iopub.status.busy": "2026-08-26T18:51:20.694928Z",
|
||||
"iopub.status.idle": "2026-08-26T18:51:20.708595Z",
|
||||
"shell.execute_reply": "2026-08-26T18:51:20.707072Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['entities', 'metadata', 'relationships']"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"foundation = manager.create_foundation_graph()\n",
|
||||
"sorted(foundation.keys())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "4610a59f",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2026-08-26T18:51:20.713136Z",
|
||||
"iopub.status.busy": "2026-08-26T18:51:20.712795Z",
|
||||
"iopub.status.idle": "2026-08-26T18:51:20.718637Z",
|
||||
"shell.execute_reply": "2026-08-26T18:51:20.716835Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'id': 'c1',\n",
|
||||
" 'text': 'Acme',\n",
|
||||
" 'type': 'Company',\n",
|
||||
" 'confidence': 1.0,\n",
|
||||
" 'metadata': {'industry': 'robotics', 'source': 'companies'}}"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"foundation[\"entities\"][0]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f3a52dc7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 4) Validate quality before committing\n",
|
||||
"\n",
|
||||
"`validate_quality(foundation_graph)` returns `valid`, `errors`, `warnings`, and `metrics` so you can gate bad seed data before it pollutes the graph."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"id": "4eb7e664",
|
||||
"metadata": {
|
||||
"execution": {
|
||||
"iopub.execute_input": "2026-08-26T18:51:20.722674Z",
|
||||
"iopub.status.busy": "2026-08-26T18:51:20.722118Z",
|
||||
"iopub.status.idle": "2026-08-26T18:51:20.732003Z",
|
||||
"shell.execute_reply": "2026-08-26T18:51:20.730170Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"True"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"quality = manager.validate_quality(foundation)\n",
|
||||
"quality[\"valid\"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b534be89",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Summary\n",
|
||||
"\n",
|
||||
"| Task | API |\n",
|
||||
"|---|---|\n",
|
||||
"| Register a trusted source | `register_source(name, format, location, entity_type=...)` |\n",
|
||||
"| Load records | `load_source(name)` — adds `entity_type` / `source` keys |\n",
|
||||
"| Direct file load | `load_from_csv(path)` / `load_from_json(path)` |\n",
|
||||
"| Build the graph | `create_foundation_graph()` → `entities` / `relationships` / `metadata` |\n",
|
||||
"| Gate bad data | `validate_quality(graph)` → `valid` / `errors` / `warnings` / `metrics` |\n",
|
||||
"\n",
|
||||
"See also `semantica/seed/seed_usage.md` for `load_from_database`, `load_from_api`, and `integrate_with_extracted`."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.13.12"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user