mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
Enhance Ontology Cookbook with visualization and advanced features
This commit is contained in:
KaifAhmad1
@@ -1,350 +1,543 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"[](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/14_Ontology.ipynb)\n",
|
||||
"\n",
|
||||
"# Ontology\n",
|
||||
"\n",
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook demonstrates how to generate and validate ontologies using Semantica's ontology modules. You'll learn to use `OntologyEngine`, `ClassInferrer`, `PropertyGenerator`, and `OntologyValidator`.\n",
|
||||
"\n",
|
||||
"**Documentation**: [API Reference](https://semantica.readthedocs.io/reference/ontology/)\n",
|
||||
"\n",
|
||||
"### Learning Objectives\n",
|
||||
"\n",
|
||||
"- Use `OntologyEngine` to generate ontologies\n",
|
||||
"- Use `ClassInferrer` to infer classes\n",
|
||||
"- Use `PropertyGenerator` to generate properties\n",
|
||||
"- Use `OntologyValidator` to validate ontologies\n",
|
||||
"\n",
|
||||
"## Installation\n",
|
||||
"\n",
|
||||
"Install Semantica from PyPI:\n",
|
||||
"\n",
|
||||
"```bash\n",
|
||||
"pip install semantica\n",
|
||||
"# Or with all optional dependencies:\n",
|
||||
"pip install semantica[all]\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Step 1: Generate Ontology\n",
|
||||
"\n",
|
||||
"Generate ontology from entities and relationships.\n",
|
||||
"\n",
|
||||
"### Module Map\n",
|
||||
"\n",
|
||||
"- `OntologyEngine`: Unified API for generation, validation, evaluation, and OWL export\n",
|
||||
"- `OntologyGenerator`: 6-stage pipeline to build classes and properties\n",
|
||||
"- `ClassInferrer`: Derive class candidates and hierarchies from entities\n",
|
||||
"- `PropertyGenerator`: Infer object/data properties from relationships\n",
|
||||
"- `OntologyValidator`: Structural and symbolic validation (reasoners optional)\n",
|
||||
"- `NamespaceManager`: Consistent IRIs (PascalCase classes, camelCase properties)\n",
|
||||
"- `OWLGenerator`/`OWLExporter`: OWL serialization and file export\n",
|
||||
"- `RequirementsSpecManager`/`CompetencyQuestionsManager`: requirements & questions\n",
|
||||
"- `LLMGenerator`: text-to-ontology generation\n",
|
||||
"- `ReuseManager`/`DomainOntologies`: reuse known vocabularies, domain templates\n",
|
||||
"- `NamingConventions`: enforce naming guidelines\n",
|
||||
"\n",
|
||||
"### Engine Options\n",
|
||||
"\n",
|
||||
"- `base_uri`: namespace root (e.g., `http://example.org/onto#`)\n",
|
||||
"- `name`: ontology name (default: `GeneratedOntology`)\n",
|
||||
"- `version`: version string (default: `1.0`)\n",
|
||||
"\n",
|
||||
"### Detailed Guide\n",
|
||||
"\n",
|
||||
"#### Core Classes & Methods\n",
|
||||
"\n",
|
||||
"- `OntologyEngine`: `from_data(data, **options)`, `from_text(text, provider=None, model=None, **options)`, `infer_classes(entities)`, `validate(ontology)`, `evaluate(ontology)`, `to_owl(ontology, format)`, `export_owl(ontology, path, format)`\n",
|
||||
"- `OntologyGenerator`: `generate_ontology(data, **options)`, `optimize_ontology(ontology, **options)`, `remove_redundancy(ontology)`, `improve_coherence(ontology)`\n",
|
||||
"- `ClassInferrer`: `infer_classes(entities)`, `build_class_hierarchy(classes)`, `validate_classes(classes)`\n",
|
||||
"- `PropertyGenerator`: `infer_properties(entities, relationships, classes)`\n",
|
||||
"- `OntologyValidator`: `validate_ontology(ontology)` (returns `valid`, `consistent`, `errors`, `warnings`, `metrics`)\n",
|
||||
"- `NamespaceManager`: `generate_class_iri(name)`, `generate_property_iri(name)`, `register_namespace(prefix, uri)`, `get_all_namespaces()`\n",
|
||||
"\n",
|
||||
"#### Extended Modules\n",
|
||||
"\n",
|
||||
"- `OWLGenerator` vs `OWLExporter`: in-memory OWL generation vs file export\n",
|
||||
"- `RequirementsSpecManager` & `CompetencyQuestionsManager`: plan and validate coverage\n",
|
||||
"- `LLMGenerator`: bootstrap ontology from text with a provider/model\n",
|
||||
"- `ReuseManager` & `DomainOntologies`: reuse known vocabularies and templates\n",
|
||||
"- `NamingConventions`: enforce consistent, readable names\n",
|
||||
"\n",
|
||||
"#### Building Ontologies Step-by-Step\n",
|
||||
"\n",
|
||||
"1) Prepare `entities` and `relationships`\n",
|
||||
"2) Generate ontology with engine\n",
|
||||
"3) Infer classes\n",
|
||||
"4) Infer properties\n",
|
||||
"5) Validate\n",
|
||||
"6) Export OWL\n",
|
||||
"\n",
|
||||
"#### Property Types\n",
|
||||
"\n",
|
||||
"- Object properties: connect classes (domain → range)\n",
|
||||
"- Data properties: attach literals (strings, numbers, dates)\n",
|
||||
"\n",
|
||||
"#### Validator Output\n",
|
||||
"\n",
|
||||
"- `valid`, `consistent`, `errors`, `warnings`\n",
|
||||
"\n",
|
||||
"#### OWL Export Formats\n",
|
||||
"\n",
|
||||
"- `turtle`, `owl-xml`\n",
|
||||
"\n",
|
||||
"#### Namespace Management\n",
|
||||
"\n",
|
||||
"- Use stable `base_uri` and versioned IRIs\n",
|
||||
"- PascalCase for classes; camelCase for properties\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import OntologyEngine\n",
|
||||
"\n",
|
||||
"engine = OntologyEngine()\n",
|
||||
"\n",
|
||||
"entities = [\n",
|
||||
" {\"id\": \"e1\", \"type\": \"Organization\", \"name\": \"Apple Inc.\"},\n",
|
||||
" {\"id\": \"e2\", \"type\": \"Person\", \"name\": \"Tim Cook\"}\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"relationships = [\n",
|
||||
" {\"source\": \"e2\", \"target\": \"e1\", \"type\": \"CEO_of\"}\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"ontology = engine.from_data({\n",
|
||||
" \"entities\": entities,\n",
|
||||
" \"relationships\": relationships\n",
|
||||
"})\n",
|
||||
"\n",
|
||||
"print(f\"Generated ontology\")\n",
|
||||
"print(f\"Classes: {len(ontology.get('classes', []))}\")\n",
|
||||
"print(f\"Properties: {len(ontology.get('properties', []))}\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Step 2: Class Inference\n",
|
||||
"\n",
|
||||
"Infer classes from entities.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import ClassInferrer\n",
|
||||
"\n",
|
||||
"class_inferrer = ClassInferrer()\n",
|
||||
"\n",
|
||||
"classes = class_inferrer.infer_classes(entities)\n",
|
||||
"\n",
|
||||
"print(f\"Inferred {len(classes)} classes\")\n",
|
||||
"for cls in classes[:3]:\n",
|
||||
" print(f\" - {cls.get('name', cls)}\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Step 3: Property Generation\n",
|
||||
"\n",
|
||||
"Generate properties from relationships.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import PropertyGenerator\n",
|
||||
"\n",
|
||||
"property_generator = PropertyGenerator()\n",
|
||||
"\n",
|
||||
"properties = property_generator.infer_properties(entities, relationships, classes)\n",
|
||||
"\n",
|
||||
"print(f\"Generated {len(properties)} properties\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Step 4: Ontology Validation\n",
|
||||
"\n",
|
||||
"Validate the generated ontology.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import OntologyValidator\n",
|
||||
"\n",
|
||||
"validator = OntologyValidator()\n",
|
||||
"\n",
|
||||
"validation_result = validator.validate_ontology(ontology)\n",
|
||||
"\n",
|
||||
"print(f\"Ontology validation:\")\n",
|
||||
"print(f\" Valid: {validation_result.valid}\")\n",
|
||||
"print(f\" Consistent: {validation_result.consistent}\")\n",
|
||||
"print(f\" Errors: {len(validation_result.errors)}\")\n",
|
||||
"print(f\" Warnings: {len(validation_result.warnings)}\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Summary\n",
|
||||
"\n",
|
||||
"You've learned how to work with ontologies:\n",
|
||||
"\n",
|
||||
"- **OntologyEngine**: Generate ontologies from entities and relationships\n",
|
||||
"- **ClassInferrer**: Infer classes from entities\n",
|
||||
"- **PropertyGenerator**: Generate properties from relationships\n",
|
||||
"- **OntologyValidator**: Validate ontologies\n",
|
||||
"\n",
|
||||
"### Validate & Evaluate\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"result = engine.validate(ontology)\n",
|
||||
"print(\"Valid:\", result.valid, \"Consistent:\", result.consistent)\n",
|
||||
"print(\"Errors:\", len(result.errors), \"Warnings:\", len(result.warnings))\n",
|
||||
"# Optional: evaluate coverage/metrics if available\n",
|
||||
"metrics = engine.evaluate(ontology)\n",
|
||||
"print(\"Evaluation metrics keys:\", list(metrics.keys()) if isinstance(metrics, dict) else metrics)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"### Inspect Classes & Properties\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"for cls in ontology.get(\"classes\", [])[:5]:\n",
|
||||
" print(\"Class:\", cls.get(\"name\"), \"label:\", cls.get(\"label\"))\n",
|
||||
"for prop in ontology.get(\"properties\", [])[:5]:\n",
|
||||
" print(\"Property:\", prop.get(\"name\"), \"type:\", prop.get(\"type\"), \"domain:\", prop.get(\"domain\"), \"range:\", prop.get(\"range\"))\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"### Modeling Checklist\n",
|
||||
"\n",
|
||||
"- Define scope and competency questions\n",
|
||||
"- Choose stable base URI and naming conventions\n",
|
||||
"- Map entities → classes, relationships → object properties, attributes → data properties\n",
|
||||
"- Add domain/range and only essential constraints\n",
|
||||
"- Validate; export OWL; iterate with visualization\n",
|
||||
"\n",
|
||||
"Next: Learn how to export data in the Export notebook.\n"
|
||||
]
|
||||
}
|
||||
,
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Worked Example: Object vs Data Properties\n",
|
||||
"\n",
|
||||
"Differentiate object properties (link classes) from data properties (literal values)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import ClassInferrer, PropertyGenerator\n",
|
||||
"\n",
|
||||
"richer_entities = [\n",
|
||||
" {\"id\": \"p1\", \"type\": \"Person\", \"name\": \"Alice\", \"birthDate\": \"1990-01-01\"},\n",
|
||||
" {\"id\": \"c1\", \"type\": \"Company\", \"name\": \"Acme Corp\", \"foundedYear\": 2005}\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"richer_relationships = [\n",
|
||||
" {\"source_id\": \"p1\", \"target_id\": \"c1\", \"source_type\": \"Person\", \"target_type\": \"Company\", \"type\": \"WORKS_FOR\"}\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"inferrer = ClassInferrer()\n",
|
||||
"classes2 = inferrer.infer_classes(richer_entities)\n",
|
||||
"\n",
|
||||
"propgen = PropertyGenerator()\n",
|
||||
"props2 = propgen.infer_properties(richer_entities, richer_relationships, classes2)\n",
|
||||
"\n",
|
||||
"obj_props = [p for p in props2 if p.get(\"type\") == \"object\"]\n",
|
||||
"data_props = [p for p in props2 if p.get(\"type\") == \"data\"]\n",
|
||||
"\n",
|
||||
"print(\"Object properties:\", [p.get(\"name\") for p in obj_props])\n",
|
||||
"print(\"Data properties:\", [p.get(\"name\") for p in data_props])\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Vocabulary Reuse: Align to Schema.org\n",
|
||||
"\n",
|
||||
"Align a property to a known vocabulary term for interoperability."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"schema_works_for = \"https://schema.org/worksFor\"\n",
|
||||
"for p in obj_props:\n",
|
||||
" if p.get(\"name\") == \"worksFor\":\n",
|
||||
" p[\"sameAs\"] = schema_works_for\n",
|
||||
" print(\"Aligned\", p.get(\"name\"), \"to\", p[\"sameAs\"])\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Visualization (Optional)\n",
|
||||
"\n",
|
||||
"Render the class hierarchy if visualization dependencies are available."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.visualization import OntologyVisualizer\n",
|
||||
"\n",
|
||||
"viz = OntologyVisualizer()\n",
|
||||
"fig = viz.visualize_hierarchy(ontology, output=\"interactive\")\n",
|
||||
"print(\"Visualization generated\")\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"[](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/14_Ontology.ipynb)\n",
|
||||
"\n",
|
||||
"# Mastering Ontology Generation with Semantica\n",
|
||||
"\n",
|
||||
"Welcome to the comprehensive guide on Semantica's Ontology Module. This module is the powerhouse for structuring your data into meaningful knowledge graphs, providing a complete 6-stage pipeline from raw data to validated OWL ontologies.\n",
|
||||
"\n",
|
||||
"In this notebook, we will dive deep into:\n",
|
||||
"1. **The 6-Stage Generation Pipeline**: Understanding how Semantica transforms data into knowledge.\n",
|
||||
"2. **Core Components in Focus**: Detailed usage of `ClassInferrer`, `PropertyGenerator`, and `OntologyOptimizer`.\n",
|
||||
"3. **Validation & Quality**: ensuring your ontology is consistent and structurally sound.\n",
|
||||
"4. **Visualize**: exploring your ontology with interactive charts and hierarchies.\n",
|
||||
"5. **Advanced Usage**: Text-to-Ontology (LLM), Competency Questions, and Lifecycle Management.\n",
|
||||
"6. **Exporting & Interoperability**: Saving your work in standard formats like Turtle and RDF/XML.\n",
|
||||
"\n",
|
||||
"**Documentation**: [API Reference](https://semantica.readthedocs.io/reference/ontology/)\n",
|
||||
"\n",
|
||||
"## Getting Started\n",
|
||||
"\n",
|
||||
"First, let's setup our environment and initialize the `OntologyEngine`. This engine is the unified entry point for all ontology operations."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Install semantica if not already installed\n",
|
||||
"# !pip install semantica\n",
|
||||
"# !pip install plotly # Required for visualization\n",
|
||||
"\n",
|
||||
"from semantica.ontology import OntologyEngine, OntologyGenerator\n",
|
||||
"from semantica.utils.logging import get_logger\n",
|
||||
"\n",
|
||||
"# Initialize logger for visibility\n",
|
||||
"logger = get_logger(\"ontology_guide\")\n",
|
||||
"\n",
|
||||
"# Initialize the Engine\n",
|
||||
"# base_uri defines the namespace root for your ontology\n",
|
||||
"engine = OntologyEngine(base_uri=\"https://docs.semantica.dev/ontology/\")\n",
|
||||
"\n",
|
||||
"print(\"Ontology Engine initialized successfully!\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## The 6-Stage Generation Pipeline\n",
|
||||
"\n",
|
||||
"Semantica uses a sophisticated 6-stage pipeline to robustly generate ontologies. This automated process takes raw entity and relationship data and produces a high-quality OWL ontology.\n",
|
||||
"\n",
|
||||
"### The Stages:\n",
|
||||
"1. **Semantic Network Parsing**: Extracts raw concepts and connections from your inputs.\n",
|
||||
"2. **YAML-to-Definition**: Transforms concepts into structured class definitions.\n",
|
||||
"3. **Definition-to-Types**: Maps definitions to formal OWL types (e.g., `owl:Class`, `owl:ObjectProperty`).\n",
|
||||
"4. **Hierarchy Generation**: Builds a taxonomic structure (parent-child relationships) using `associatedWith` or linguistic patterns.\n",
|
||||
"5. **TTL Generation**: Serializes the in-memory structure into Turtle format logic.\n",
|
||||
"6. **Symbolic Validation**: Validates the result using reasoners like HermiT (if available) or structural checks.\n",
|
||||
"\n",
|
||||
"Let's see this in action with some sample data."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Sample Data: A simple corporate structure\n",
|
||||
"entities = [\n",
|
||||
" {\"id\": \"e1\", \"type\": \"Company\", \"name\": \"TechCorp\", \"founded\": \"2010\"},\n",
|
||||
" {\"id\": \"e2\", \"type\": \"Person\", \"name\": \"Alice\", \"role\": \"CEO\"},\n",
|
||||
" {\"id\": \"e3\", \"type\": \"Person\", \"name\": \"Bob\", \"role\": \"CTO\"},\n",
|
||||
" {\"id\": \"e4\", \"type\": \"Department\", \"name\": \"Engineering\"},\n",
|
||||
" {\"id\": \"e5\", \"type\": \"Project\", \"name\": \"Project Phoenix\"}\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"relationships = [\n",
|
||||
" {\"source\": \"e2\", \"target\": \"e1\", \"type\": \"leads\"},\n",
|
||||
" {\"source\": \"e3\", \"target\": \"e4\", \"type\": \"manages\"},\n",
|
||||
" {\"source\": \"e4\", \"target\": \"e1\", \"type\": \"part_of\"},\n",
|
||||
" {\"source\": \"e3\", \"target\": \"e5\", \"type\": \"works_on\"}\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"data = {\n",
|
||||
" \"entities\": entities,\n",
|
||||
" \"relationships\": relationships\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# Run the full pipeline\n",
|
||||
"ontology = engine.from_data(data, name=\"CorporateOntology\")\n",
|
||||
"\n",
|
||||
"print(f\"Generated Ontology: {ontology['name']}\")\n",
|
||||
"print(f\"Classes Found: {len(ontology['classes'])}\")\n",
|
||||
"print(f\"Properties Found: {len(ontology['properties'])}\")\n",
|
||||
"print(f\"Validation Status: Valid={ontology.get('validation_result', {}).get('valid', 'Unknown')}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Inspecting the Results\n",
|
||||
"\n",
|
||||
"The generated `ontology` object is a rich dictionary containing all the inferred structure. Let's peek inside to see what Classes and Properties were created."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Inspect Classes\n",
|
||||
"print(\"--- Inferred Classes ---\")\n",
|
||||
"for cls in ontology['classes']:\n",
|
||||
" print(f\"Class: {cls['name']}\")\n",
|
||||
" print(f\" URI: {cls.get('uri')}\")\n",
|
||||
" # Check if a hierarchy was inferred\n",
|
||||
" if cls.get('subClassOf'):\n",
|
||||
" print(f\" Parent: {cls['subClassOf']}\")\n",
|
||||
" print(\"\")\n",
|
||||
"\n",
|
||||
"# Inspect Properties\n",
|
||||
"print(\"--- Inferred Properties ---\")\n",
|
||||
"for prop in ontology['properties']:\n",
|
||||
" type_label = \"Object Property\" if prop['type'] == 'object' else \"Data Property\"\n",
|
||||
" print(f\"{prop['name']} [{type_label}]\")\n",
|
||||
" print(f\" Domain: {prop.get('domain')}\")\n",
|
||||
" print(f\" Range: {prop.get('range')}\")\n",
|
||||
" print(\"\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Deep Dive: Component by Component\n",
|
||||
"\n",
|
||||
"While `OntologyEngine` is great for one-shot generation, you often need fine-grained control. Let's look at the individual tools that power the engine.\n",
|
||||
"\n",
|
||||
"### 1. `ClassInferrer`: Mastering Class Discovery\n",
|
||||
"\n",
|
||||
"The `ClassInferrer` analyzes entities to find patterns. It can handle noise and only creates classes for types that appear frequently enough.\n",
|
||||
"\n",
|
||||
"* **`min_occurrences`**: Ignores types with fewer entities than this count.\n",
|
||||
"* **`build_class_hierarchy`**: Toggles automatic parent-child detection.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import ClassInferrer\n",
|
||||
"\n",
|
||||
"# Initialize inferrer with a threshold\n",
|
||||
"# We set min_occurrences=1 here to capture everything in our small example\n",
|
||||
"inferrer = ClassInferrer(min_occurrences=1)\n",
|
||||
"\n",
|
||||
"raw_entities = [\n",
|
||||
" {\"type\": \"Manager\", \"name\": \"Dave\", \"level\": 5},\n",
|
||||
" {\"type\": \"Manager\", \"name\": \"Eve\", \"level\": 4},\n",
|
||||
" {\"type\": \"Employee\", \"name\": \"Frank\"}, # Only 1 employee\n",
|
||||
" {\"type\": \"TemporaryWorker\", \"name\": \"Grace\"} \n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Infer classes\n",
|
||||
"classes = inferrer.infer_classes(raw_entities, build_hierarchy=True)\n",
|
||||
"\n",
|
||||
"print(f\"Inferred {len(classes)} classes from raw entities.\")\n",
|
||||
"for c in classes:\n",
|
||||
" print(f\"- {c['name']} (Count: {c['entity_count']})\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 2. `PropertyGenerator`: The Glue of the Ontology\n",
|
||||
"\n",
|
||||
"Properties define relationships. Semantica distinguishes between:\n",
|
||||
"* **Object Properties**: Links between two entities (e.g., `leads` between Person and Company).\n",
|
||||
"* **Data Properties**: Attributes of an entity (e.g., `founded` year of a Company).\n",
|
||||
"\n",
|
||||
"The `PropertyGenerator` automatically detects this distinction."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import PropertyGenerator\n",
|
||||
"\n",
|
||||
"prop_gen = PropertyGenerator()\n",
|
||||
"\n",
|
||||
"# We need the classes first to help property generation context\n",
|
||||
"context_classes = classes # reusing from previous step\n",
|
||||
"\n",
|
||||
"# Let's define some relationships and attributes implicitly via entities\n",
|
||||
"# Note: 'level' in Manager entities is a potential data property\n",
|
||||
"complex_entities = [\n",
|
||||
" {\"id\": \"m1\", \"type\": \"Manager\", \"name\": \"Dave\", \"level\": 5},\n",
|
||||
" {\"id\": \"e1\", \"type\": \"Employee\", \"name\": \"Frank\"}\n",
|
||||
"]\n",
|
||||
"complex_relationships = [\n",
|
||||
" {\"source\": \"m1\", \"target\": \"e1\", \"type\": \"supervises\"} # Object property\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"properties = prop_gen.infer_properties(\n",
|
||||
" entities=complex_entities,\n",
|
||||
" relationships=complex_relationships,\n",
|
||||
" classes=context_classes\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(\"--- Property Types Identified ---\")\n",
|
||||
"for p in properties:\n",
|
||||
" print(f\"Property: {p['name']}\")\n",
|
||||
" print(f\" Type: {p['type']}\")\n",
|
||||
" print(f\" Domain: {p['domain']} -> Range: {p['range']}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 3. `OntologyOptimizer`: Refining the Structure\n",
|
||||
"\n",
|
||||
"Before finalizing, it's good practice to optimize. The optimizer removes redundancies and improves coherence, such as ensuring all classes have proper labels and valid URIs."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import OntologyOptimizer\n",
|
||||
"\n",
|
||||
"optimizer = OntologyOptimizer()\n",
|
||||
"\n",
|
||||
"# Let's pretend we have a messy ontology dict\n",
|
||||
"messy_ontology = {\n",
|
||||
" \"classes\": [\n",
|
||||
" {\"name\": \"Person\", \"uri\": \"...Person\"},\n",
|
||||
" {\"name\": \"Person\", \"uri\": \"...Person\"} # Duplicate!\n",
|
||||
" ],\n",
|
||||
" \"properties\": []\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"clean_ontology = optimizer.optimize_ontology(messy_ontology, remove_redundancy=True)\n",
|
||||
"\n",
|
||||
"print(f\"Original Classes: {len(messy_ontology['classes'])}\")\n",
|
||||
"print(f\"Optimized Classes: {len(clean_ontology['classes'])}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Validation and Quality Control\n",
|
||||
"\n",
|
||||
"Semantica includes a robust `OntologyValidator`. It checks for:\n",
|
||||
"1. **Structure**: Missing fields, malformed URIs.\n",
|
||||
"2. **Consistency**: Circular hierarchies, contradictory definitions.\n",
|
||||
"3. **Metrics**: Depth of hierarchy, property usage.\n",
|
||||
"\n",
|
||||
"If you have `Owlready2` installed, it can even run a reasoner (HermiT or Pellet) to prove logical consistency."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import OntologyValidator\n",
|
||||
"\n",
|
||||
"validator = OntologyValidator(\n",
|
||||
" check_consistency=True,\n",
|
||||
" check_satisfiability=True\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Validate our previously generated 'ontology'\n",
|
||||
"result = validator.validate_ontology(ontology)\n",
|
||||
"\n",
|
||||
"print(f\"Is Valid? {result.valid}\")\n",
|
||||
"print(f\"Is Consistent? {result.consistent}\")\n",
|
||||
"\n",
|
||||
"if result.errors:\n",
|
||||
" print(\"Errors Found:\", result.errors)\n",
|
||||
"if result.warnings:\n",
|
||||
" print(\"Warnings:\", result.warnings)\n",
|
||||
" \n",
|
||||
"# Check Metrics\n",
|
||||
"print(\"Metrics:\", result.metrics)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Visualization\n",
|
||||
"\n",
|
||||
"A picture is worth a thousand triples! The `OntologyVisualizer` lets you explore your ontology's structure interactively.\n",
|
||||
"\n",
|
||||
"We can visualize:\n",
|
||||
"* **Class Hierarchies**: Tree diagrams of class inheritance.\n",
|
||||
"* **Structure Networks**: The full graph of classes and properties.\n",
|
||||
"* **Metrics Dashboards**: High-level stats at a glance."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.visualization import OntologyVisualizer\n",
|
||||
"\n",
|
||||
"viz = OntologyVisualizer()\n",
|
||||
"\n",
|
||||
"# 1. Interactive Class Hierarchy\n",
|
||||
"# Returns a Plotly figure you can interact with\n",
|
||||
"fig_hierarchy = viz.visualize_hierarchy(ontology, output=\"interactive\")\n",
|
||||
"if fig_hierarchy:\n",
|
||||
" fig_hierarchy.show()\n",
|
||||
"\n",
|
||||
"# 2. Ontology Structure Network\n",
|
||||
"# See how classes and properties connect\n",
|
||||
"fig_structure = viz.visualize_structure(ontology, output=\"interactive\")\n",
|
||||
"if fig_structure:\n",
|
||||
" fig_structure.show()\n",
|
||||
"\n",
|
||||
"# 3. Metrics Dashboard\n",
|
||||
"# View counts, depths, and statistics\n",
|
||||
"fig_metrics = viz.visualize_metrics(ontology, output=\"interactive\")\n",
|
||||
"if fig_metrics:\n",
|
||||
" fig_metrics.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Advanced Usage: Lifecycle & AI\n",
|
||||
"\n",
|
||||
"Enterprise ontologies are living artifacts. Semantica provides tools to manage their entire lifecycle and accelerate creation with AI.\n",
|
||||
"\n",
|
||||
"### 1. Text-to-Ontology (LLM Integration)\n",
|
||||
"\n",
|
||||
"Instead of manually creating entities, use the `LLMOntologyGenerator` to extract an ontology directly from text requirements or documents."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import LLMOntologyGenerator\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" # Note: Requires an API key in your environment variables\n",
|
||||
" llm_gen = LLMOntologyGenerator(provider=\"openai\", model=\"gpt-4\")\n",
|
||||
"\n",
|
||||
" text_description = \"\"\"\n",
|
||||
" A University has many Departments. Each Department offers several Courses.\n",
|
||||
" Professors teach Courses and belong to a Department.\n",
|
||||
" Students enroll in Courses.\n",
|
||||
" \"\"\"\n",
|
||||
"\n",
|
||||
" llm_ontology = llm_gen.generate_ontology_from_text(\n",
|
||||
" text=text_description,\n",
|
||||
" name=\"UniversityOntology\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" print(\"Generated Classes:\", [c['name'] for c in llm_ontology['classes']])\n",
|
||||
"except Exception:\n",
|
||||
" print(\"Skipping LLM generation: No API key or provider configured in this environment.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 2. Test-Driven Design (Competency Questions)\n",
|
||||
"\n",
|
||||
"Formalize your requirements as \"Competency Questions\" (CQs). The `CompetencyQuestionsManager` can check if your ontology contains the necessary terms to answer them."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import CompetencyQuestionsManager\n",
|
||||
"\n",
|
||||
"cq_manager = CompetencyQuestionsManager()\n",
|
||||
"\n",
|
||||
"# Define what our ontology SHOULD answer\n",
|
||||
"cq_manager.add_question(\"Who leads TechCorp?\", category=\"organizational\")\n",
|
||||
"cq_manager.add_question(\"Which projects does Bob manage?\", category=\"operational\")\n",
|
||||
"\n",
|
||||
"# Validate our 'ontology' against these questions\n",
|
||||
"validation_results = cq_manager.validate_ontology(ontology)\n",
|
||||
"\n",
|
||||
"print(f\"Answerable Questions: {validation_results['answerable']} / {validation_results['total_questions']}\")\n",
|
||||
"for q in cq_manager.questions:\n",
|
||||
" status = \"✅\" if q.answerable else \"❌\"\n",
|
||||
" print(f\"{status} {q.question}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 3. Lifecycle Management (Versioning & Reuse)\n",
|
||||
"\n",
|
||||
"Manage iterations with `VersionManager` and import external standards like FOAF or Dublin Core with `ReuseManager`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantica.ontology import VersionManager, ReuseManager\n",
|
||||
"\n",
|
||||
"# --- Versioning ---\n",
|
||||
"v_manager = VersionManager(base_uri=\"https://example.org/ontology/\")\n",
|
||||
"v1 = v_manager.create_version(\"1.0\", ontology, changes=[\"Initial creation\"])\n",
|
||||
"print(f\"Created Version: {v1.version} at {v1.ontology_iri}\")\n",
|
||||
"\n",
|
||||
"# --- Reuse ---\n",
|
||||
"reuse_manager = ReuseManager()\n",
|
||||
"\n",
|
||||
"# Check if we can reuse FOAF\n",
|
||||
"foaf_info = reuse_manager.research_ontology(\"http://xmlns.com/foaf/0.1/\")\n",
|
||||
"if foaf_info:\n",
|
||||
" print(f\"Found standard ontology: {foaf_info['name']}\")\n",
|
||||
" # We could now import this into our ontology\n",
|
||||
" ontology['imports'].append(foaf_info['uri'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exporting Your Ontology\n",
|
||||
"\n",
|
||||
"Once your ontology is built and validated, you'll want to save it. Semantica focuses on **Turtle (`.ttl`)** as the primary format, but supports others via `rdflib`.\n",
|
||||
"\n",
|
||||
"You can export to a string or directly to a file."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Turtle string representation\n",
|
||||
"ttl_output = engine.to_owl(ontology, format=\"turtle\")\n",
|
||||
"\n",
|
||||
"print(\"--- Turtle Preview (First 500 chars) ---\")\n",
|
||||
"print(ttl_output[:500])\n",
|
||||
"print(\"...\")\n",
|
||||
"\n",
|
||||
"# Save to file\n",
|
||||
"output_path = \"corporate_ontology.ttl\"\n",
|
||||
"engine.export_owl(ontology, path=output_path, format=\"turtle\")\n",
|
||||
"print(f\"Successfully saved ontology to {output_path}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Summary\n",
|
||||
"\n",
|
||||
"You have now mastered the essentials of Semantica's Ontology Module!\n",
|
||||
"\n",
|
||||
"* **Automated Generation**: Used the 6-stage pipeline to go from raw data to a structured ontology.\n",
|
||||
"* **Component Control**: Used `ClassInferrer` and `PropertyGenerator` for fine-tuned modeling.\n",
|
||||
"* **Quality Assurance**: Validated your model against strict standards.\n",
|
||||
"* **Visualization**: Explored the ontology structure interactively.\n",
|
||||
"* **Advanced Lifecycle**: Used AI generation, competency questions, and versioning.\n",
|
||||
"* **Export**: Serialized your knowledge graph for use in other semantic web tools.\n",
|
||||
"\n",
|
||||
"**Next Steps**:\n",
|
||||
"* Try customizing the `NamespaceManager` to use your organization's URL.\n",
|
||||
"* Explore `OntologyEvaluator` for deeper quality metrics.\n",
|
||||
"* Feed the generated ontology into the **Knowledge Graph** module to start reasoning over your data!"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.10"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user