mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
KaifAhmad1
7a6f1d0417
Add a Cite Us section to the README with BibTeX citation info, and align it with docs/citation.md (author/organization: Semantica, 2026). Update LICENSE and docs/project-license.md copyright holder to Semantica, and replace the stale Hawksight-AI GitHub org slug with semantica-agi across READMEs, plugin manifests, cookbook notebooks, and GitHub templates.
320 lines
11 KiB
Plaintext
320 lines
11 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"[](https://colab.research.google.com/github/semantica-agi/semantica/blob/main/cookbook/introduction/09_Your_First_Knowledge_Graph.ipynb)\n",
|
|
"\n",
|
|
"# 🚀 Your First Knowledge Graph\n",
|
|
"\n",
|
|
"## Overview\n",
|
|
"\n",
|
|
"This notebook walks you through creating your first knowledge graph from a simple document. You'll learn the complete end-to-end workflow from ingesting a file to visualizing the resulting knowledge graph.\n",
|
|
"\n",
|
|
"> [!TIP]\n",
|
|
"> This is the perfect starting point if you are new to Semantica. No prior knowledge of knowledge graphs is required!\n",
|
|
"\n",
|
|
"**Documentation**: [API Reference](https://semantica.readthedocs.io/reference/kg/)\n",
|
|
"\n",
|
|
"### 🎯 Learning Objectives\n",
|
|
"\n",
|
|
"- **Understand the Workflow**: Learn the `File → Parse → Extract → Graph` pipeline\n",
|
|
"- **Ingest Data**: Load documents using `FileIngestor`\n",
|
|
"- **Parse Content**: Extract text using `DocumentParser`\n",
|
|
"- **Extract Knowledge**: Identify entities using `NERExtractor`\n",
|
|
"- **Build Graph**: Construct a graph using `GraphBuilder`\n",
|
|
"- **Visualize**: See your graph come to life with `KGVisualizer`\n",
|
|
"\n",
|
|
"## Installation\n",
|
|
"\n",
|
|
"Install Semantica from PyPI:\n",
|
|
"\n",
|
|
"```bash\n",
|
|
"pip install semantica\n",
|
|
"# Or with all optional dependencies:\n",
|
|
"pip install semantica[all]\n",
|
|
"```\n",
|
|
"\n",
|
|
"---\n",
|
|
"\n",
|
|
"## 🔄 Simple End-to-End Workflow\n",
|
|
"\n",
|
|
"The complete workflow consists of four main steps:\n",
|
|
"\n",
|
|
"1. **📥 Ingest** - Load data from files or other sources\n",
|
|
"2. **📄 Parse** - Extract and structure content from documents\n",
|
|
"3. **⛏️ Extract** - Identify entities and relationships\n",
|
|
"4. **🕸️ Build Graph** - Construct the knowledge graph\n",
|
|
"\n",
|
|
"Each step is demonstrated in the code cells below.\n",
|
|
"\n",
|
|
"> [!TIP]\n",
|
|
"> **Alternative: Using Semantica Framework**\n",
|
|
"> \n",
|
|
"> For a simpler, high-level approach, you can use the `Semantica` framework class which orchestrates all these steps:\n",
|
|
"> \n",
|
|
"> ```python\n",
|
|
"> from semantica.core import Semantica\n",
|
|
"> \n",
|
|
"> framework = Semantica()\n",
|
|
"> framework.initialize()\n",
|
|
"> \n",
|
|
"> result = framework.build_knowledge_base(\n",
|
|
"> sources=[\"sample_document.txt\"],\n",
|
|
"> embeddings=True,\n",
|
|
"> graph=True\n",
|
|
"> )\n",
|
|
"> \n",
|
|
"> framework.shutdown()\n",
|
|
"> ```\n",
|
|
"> \n",
|
|
"> This notebook shows the step-by-step approach for learning. See [Core Module Usage Guide](../../../semantica/core/core_usage.md) for more details.\n",
|
|
"\n",
|
|
"---\n",
|
|
"\n",
|
|
"## 📂 Step 1: Ingest a File\n",
|
|
"\n",
|
|
"In this step, we'll use `FileIngestor` to load a document. The ingestor supports various file formats including PDF, DOCX, TXT, and more.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install semantica"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from semantica.ingest import FileIngestor\n",
|
|
"from pathlib import Path\n",
|
|
"\n",
|
|
"# Initialize the ingestor\n",
|
|
"ingestor = FileIngestor()\n",
|
|
"\n",
|
|
"# Create a sample document for demonstration\n",
|
|
"sample_text = \"\"\"\n",
|
|
"Apple Inc. is a technology company founded by Steve Jobs, Steve Wozniak, and Ronald Wayne in 1976.\n",
|
|
"The company is headquartered in Cupertino, California.\n",
|
|
"Tim Cook is the current CEO of Apple Inc.\n",
|
|
"Apple designs and manufactures consumer electronics, software, and online services.\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"sample_file = Path(\"sample_document.txt\")\n",
|
|
"sample_file.write_text(sample_text)\n",
|
|
"\n",
|
|
"print(f\"File: {sample_file}\")\n",
|
|
"print(f\"Content length: {len(sample_text)} characters\")\n",
|
|
"\n",
|
|
"# Ingest the file\n",
|
|
"file_object = ingestor.ingest_file(sample_file, read_content=True)\n",
|
|
"print(f\" File name: {file_object.name}\")\n",
|
|
"print(f\" File type: {file_object.file_type}\")\n",
|
|
"print(f\" Content available: {file_object.content is not None}\")\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 📄 Step 2: Parse the Document\n",
|
|
"\n",
|
|
"After ingesting the file, we need to parse it to extract the text content. The `DocumentParser` handles various file formats and extracts structured content.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from semantica.parse import DocumentParser\n",
|
|
"\n",
|
|
"parser = DocumentParser()\n",
|
|
"# Parse the document to extract text\n",
|
|
"parsed_document = parser.parse_document(str(sample_file))\n",
|
|
"parsed_content = parsed_document.get(\"content\", \"\")\n",
|
|
"print(f\" Parsed content length: {len(parsed_content) if parsed_content else 0} characters\")\n",
|
|
"print(f\" Preview: {parsed_content[:200] if parsed_content else 'N/A'}...\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## ⛏️ Step 3: Extract Entities\n",
|
|
"\n",
|
|
"Now we'll extract entities from the parsed text using Named Entity Recognition (NER). This identifies people, organizations, locations, dates, and other entities in the text.\n",
|
|
"\n",
|
|
"> [!NOTE]\n",
|
|
"> In a real scenario, you would use `NERExtractor` with an LLM or model backend. Here we simulate the output for demonstration purposes.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from semantica.semantic_extract import NamedEntityRecognizer, NERExtractor\n",
|
|
"\n",
|
|
"ner = NamedEntityRecognizer()\n",
|
|
"extractor = NERExtractor()\n",
|
|
"\n",
|
|
"print(f\"\\nText: {parsed_content[:100]}...\")\n",
|
|
"\n",
|
|
"# Simulated extraction results\n",
|
|
"expected_entities = [\n",
|
|
" {\"text\": \"Apple Inc.\", \"type\": \"Organization\", \"start\": 0, \"end\": 10},\n",
|
|
" {\"text\": \"Steve Jobs\", \"type\": \"Person\", \"start\": 50, \"end\": 60},\n",
|
|
" {\"text\": \"Steve Wozniak\", \"type\": \"Person\", \"start\": 62, \"end\": 75},\n",
|
|
" {\"text\": \"Ronald Wayne\", \"type\": \"Person\", \"start\": 81, \"end\": 93},\n",
|
|
" {\"text\": \"1976\", \"type\": \"Date\", \"start\": 97, \"end\": 101},\n",
|
|
" {\"text\": \"Cupertino, California\", \"type\": \"Location\", \"start\": 130, \"end\": 151},\n",
|
|
" {\"text\": \"Tim Cook\", \"type\": \"Person\", \"start\": 153, \"end\": 161},\n",
|
|
"]\n",
|
|
"\n",
|
|
"for entity in expected_entities:\n",
|
|
" print(f\" - {entity['text']} ({entity['type']})\")\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 🕸️ Step 4: Build the Knowledge Graph\n",
|
|
"\n",
|
|
"Using the extracted entities and relationships, we'll construct a knowledge graph. The graph represents entities as nodes and relationships as edges.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from semantica.kg import GraphBuilder\n",
|
|
"import networkx as nx\n",
|
|
"\n",
|
|
"builder = GraphBuilder()\n",
|
|
"\n",
|
|
"# Prepare data for graph construction\n",
|
|
"entities_data = [\n",
|
|
" {\"id\": f\"entity_{i}\", \"name\": entity[\"text\"], \"type\": entity[\"type\"]}\n",
|
|
" for i, entity in enumerate(expected_entities)\n",
|
|
"]\n",
|
|
"\n",
|
|
"relationships_data = [\n",
|
|
" {\"source\": \"entity_0\", \"target\": \"entity_1\", \"type\": \"founded_by\"},\n",
|
|
" {\"source\": \"entity_0\", \"target\": \"entity_2\", \"type\": \"founded_by\"},\n",
|
|
" {\"source\": \"entity_0\", \"target\": \"entity_3\", \"type\": \"founded_by\"},\n",
|
|
" {\"source\": \"entity_0\", \"target\": \"entity_4\", \"type\": \"founded_in\"},\n",
|
|
" {\"source\": \"entity_0\", \"target\": \"entity_5\", \"type\": \"located_in\"},\n",
|
|
" {\"source\": \"entity_6\", \"target\": \"entity_0\", \"type\": \"ceo_of\"},\n",
|
|
"]\n",
|
|
"\n",
|
|
"# Build the graph using NetworkX\n",
|
|
"kg = nx.DiGraph()\n",
|
|
"\n",
|
|
"for entity in entities_data:\n",
|
|
" kg.add_node(entity[\"id\"], name=entity[\"name\"], type=entity[\"type\"])\n",
|
|
"\n",
|
|
"for rel in relationships_data:\n",
|
|
" source_name = entities_data[int(rel[\"source\"].split(\"_\")[1])][\"name\"]\n",
|
|
" target_name = entities_data[int(rel[\"target\"].split(\"_\")[1])][\"name\"]\n",
|
|
" kg.add_edge(rel[\"source\"], rel[\"target\"], type=rel[\"type\"])\n",
|
|
"\n",
|
|
"print(f\" Nodes (entities): {len(kg.nodes)}\")\n",
|
|
"print(f\" Edges (relationships): {len(kg.edges)}\")\n",
|
|
"\n",
|
|
"for node_id in kg.nodes():\n",
|
|
" node_data = kg.nodes[node_id]\n",
|
|
" print(f\" Node: {node_data['name']} ({node_data['type']})\")\n",
|
|
"\n",
|
|
"for source, target, data in kg.edges(data=True):\n",
|
|
" source_name = kg.nodes[source]['name']\n",
|
|
" target_name = kg.nodes[target]['name']\n",
|
|
" print(f\" {source_name} --[{data['type']}]--> {target_name}\")\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 📊 Step 5: Visualize and Analyze\n",
|
|
"\n",
|
|
"Finally, we'll visualize the knowledge graph and analyze its structure. This helps you understand the relationships and entities in your data.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from semantica.visualization import KGVisualizer\n",
|
|
"\n",
|
|
"visualizer = KGVisualizer()\n",
|
|
"\n",
|
|
"print(f\" Total entities: {len(kg.nodes)}\")\n",
|
|
"print(f\" Total relationships: {len(kg.edges)}\")\n",
|
|
"\n",
|
|
"entity_types = {}\n",
|
|
"for node_id in kg.nodes():\n",
|
|
" entity_type = kg.nodes[node_id]['type']\n",
|
|
" entity_types[entity_type] = entity_types.get(entity_type, 0) + 1\n",
|
|
"\n",
|
|
"for etype, count in entity_types.items():\n",
|
|
" print(f\" - {etype}: {count}\")\n",
|
|
"\n",
|
|
"rel_types = {}\n",
|
|
"for _, _, data in kg.edges(data=True):\n",
|
|
" rel_type = data.get('type', 'unknown')\n",
|
|
" rel_types[rel_type] = rel_types.get(rel_type, 0) + 1\n",
|
|
"\n",
|
|
"for rtype, count in rel_types.items():\n",
|
|
" print(f\" - {rtype}: {count}\")\n",
|
|
"\n",
|
|
"# Cleanup\n",
|
|
"if sample_file.exists():\n",
|
|
" sample_file.unlink()\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.9"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|