{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/blockchain/01_DeFi_Protocol_Intelligence.ipynb)\n",
"\n",
"# DeFi Protocol Intelligence - Risk Assessment & Ontology Reasoning\n",
"\n",
"## Overview\n",
"\n",
"This notebook demonstrates **DeFi protocol intelligence** using Semantica with focus on **risk assessment**, **ontology-based reasoning**, and **relationship analysis**. The pipeline ingests DeFi data from multiple sources, extracts protocol entities, builds knowledge graphs, and assesses risks using graph reasoning.\n",
"\n",
"### Key Features\n",
"\n",
"- **Risk Assessment Focus**: Emphasizes KG construction and reasoning for risk evaluation\n",
"- **Ontology-Based Reasoning**: Uses domain ontologies for DeFi protocol analysis\n",
"- **Relationship Analysis**: Analyzes protocol relationships and dependencies\n",
"- **Comprehensive Data Sources**: Multiple RSS feeds, APIs, and databases\n",
"- **Modular Architecture**: Direct use of Semantica modules without core orchestrator\n",
"\n",
"### Learning Objectives\n",
"\n",
"- Ingest DeFi data from multiple sources (RSS feeds, APIs, databases)\n",
"- Extract DeFi entities (Protocols, Tokens, Pools, Transactions, Risks)\n",
"- Build and analyze DeFi knowledge graphs\n",
"- Generate and utilize DeFi ontologies\n",
"- Perform risk assessment using graph reasoning\n",
"- Store and query DeFi data using vector stores and graph stores\n",
"\n",
"### Pipeline Flow\n",
"\n",
"```mermaid\n",
"graph TD\n",
" A[Data Ingestion] --> B[Document Parsing]\n",
" B --> C[Text Processing]\n",
" C --> D[Entity Extraction]\n",
" D --> E[Relationship Extraction]\n",
" E --> F[Deduplication]\n",
" F --> G[Conflict Detection]\n",
" G --> H[Knowledge Graph]\n",
" H --> I[Embeddings]\n",
" I --> J[Vector Store]\n",
" H --> K[Ontology Generation]\n",
" K --> L[Reasoning & Risk]\n",
" J --> M[GraphRAG Queries]\n",
" L --> M\n",
" H --> N[Graph Store]\n",
" K --> O[Triplet Store]\n",
" M --> P[Visualization]\n",
" N --> P\n",
" O --> P\n",
" P --> Q[Export]\n",
"```\n",
"\n",
"## Installation\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -qU semantica networkx matplotlib plotly pandas faiss-cpu beautifulsoup4 groq sentence-transformers scikit-learn\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuration & Setup\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"GROQ_API_KEY\"] = os.getenv(\"GROQ_API_KEY\", \"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\")\n",
"\n",
"# Configuration constants\n",
"EMBEDDING_DIMENSION = 384\n",
"EMBEDDING_MODEL = \"all-MiniLM-L6-v2\"\n",
"CHUNK_SIZE = 1000\n",
"CHUNK_OVERLAP = 200\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Ingesting DeFi Data from Multiple Sources\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.ingest import FeedIngestor, FileIngestor, WebIngestor\n",
"import os\n",
"from contextlib import redirect_stderr\n",
"from io import StringIO\n",
"\n",
"os.makedirs(\"data\", exist_ok=True)\n",
"\n",
"feed_sources = [\n",
" # Crypto News RSS Feeds\n",
" (\"CoinDesk\", \"https://www.coindesk.com/arc/outboundfeeds/rss/\"),\n",
" (\"CoinTelegraph\", \"https://cointelegraph.com/rss\"),\n",
" (\"Decrypt\", \"https://decrypt.co/feed\"),\n",
" (\"The Block\", \"https://www.theblock.co/rss.xml\"),\n",
" (\"CryptoSlate\", \"https://cryptoslate.com/feed/\"),\n",
" (\"CryptoNews\", \"https://cryptonews.com/news/feed/\"),\n",
"]\n",
"\n",
"feed_ingestor = FeedIngestor()\n",
"all_documents = []\n",
"\n",
"print(f\"Ingesting from {len(feed_sources)} feed sources...\")\n",
"for i, (feed_name, feed_url) in enumerate(feed_sources, 1):\n",
" try:\n",
" with redirect_stderr(StringIO()):\n",
" feed_data = feed_ingestor.ingest_feed(feed_url, validate=False)\n",
" \n",
" feed_count = 0\n",
" for item in feed_data.items:\n",
" if not item.content:\n",
" item.content = item.description or item.title or \"\"\n",
" if item.content:\n",
" if not hasattr(item, 'metadata'):\n",
" item.metadata = {}\n",
" item.metadata['source'] = feed_name\n",
" all_documents.append(item)\n",
" feed_count += 1\n",
" \n",
" if feed_count > 0:\n",
" print(f\" [{i}/{len(feed_sources)}] {feed_name}: {feed_count} documents\")\n",
" except Exception:\n",
" continue\n",
"\n",
"if not all_documents:\n",
" defi_data = \"\"\"\n",
" Uniswap is a decentralized exchange protocol with high liquidity pools. It uses automated market makers (AMMs) for token swaps.\n",
" Aave is a lending protocol that offers variable and stable interest rates. Users can deposit assets to earn yield.\n",
" Compound is a money market protocol for lending and borrowing cryptocurrencies. It uses algorithmic interest rates.\n",
" MakerDAO uses collateralized debt positions (CDPs) for stablecoin generation. DAI is the stablecoin created.\n",
" Curve Finance is a decentralized exchange optimized for stablecoin trading with low slippage.\n",
" Yearn Finance aggregates yield farming strategies across multiple DeFi protocols.\n",
" SushiSwap is a decentralized exchange and automated market maker with yield farming features.\n",
" Balancer is a protocol for programmable liquidity and automated portfolio management.\n",
" \"\"\"\n",
" with open(\"data/defi_protocols.txt\", \"w\") as f:\n",
" f.write(defi_data)\n",
" file_ingestor = FileIngestor()\n",
" all_documents = file_ingestor.ingest(\"data/defi_protocols.txt\")\n",
"\n",
"documents = all_documents\n",
"print(f\"Ingested {len(documents)} documents\")\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.parse import DocumentParser\n",
"\n",
"parser = DocumentParser()\n",
"\n",
"print(f\"Parsing {len(documents)} documents...\")\n",
"parsed_documents = []\n",
"for i, doc in enumerate(documents, 1):\n",
" try:\n",
" parsed = parser.parse(\n",
" doc.content if hasattr(doc, 'content') else str(doc),\n",
" content_type=\"text\"\n",
" )\n",
" parsed_documents.append(parsed)\n",
" except Exception:\n",
" parsed_documents.append(doc)\n",
" if i % 50 == 0 or i == len(documents):\n",
" print(f\" Parsed {i}/{len(documents)} documents...\")\n",
"\n",
"documents = parsed_documents\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Normalizing and Chunking DeFi Documents\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.normalize import TextNormalizer\n",
"from semantica.split import TextSplitter\n",
"\n",
"normalizer = TextNormalizer()\n",
"splitter = TextSplitter(\n",
" method=\"entity_aware\",\n",
" ner_method=\"spacy\",\n",
" chunk_size=CHUNK_SIZE,\n",
" chunk_overlap=CHUNK_OVERLAP\n",
")\n",
"\n",
"print(f\"Normalizing {len(documents)} documents...\")\n",
"normalized_documents = []\n",
"for i, doc in enumerate(documents, 1):\n",
" normalized_text = normalizer.normalize(\n",
" doc.content if hasattr(doc, 'content') else str(doc),\n",
" clean_html=True,\n",
" normalize_entities=True,\n",
" remove_extra_whitespace=True,\n",
" lowercase=False\n",
" )\n",
" normalized_documents.append(normalized_text)\n",
" if i % 50 == 0 or i == len(documents):\n",
" print(f\" Normalized {i}/{len(documents)} documents...\")\n",
"\n",
"print(f\"Chunking {len(normalized_documents)} documents...\")\n",
"chunked_documents = []\n",
"for i, doc_text in enumerate(normalized_documents, 1):\n",
" try:\n",
" with redirect_stderr(StringIO()):\n",
" chunks = splitter.split(doc_text)\n",
" chunked_documents.extend(chunks)\n",
" except Exception:\n",
" simple_splitter = TextSplitter(method=\"recursive\", chunk_size=CHUNK_SIZE, chunk_overlap=CHUNK_OVERLAP)\n",
" chunks = simple_splitter.split(doc_text)\n",
" chunked_documents.extend(chunks)\n",
" if i % 50 == 0 or i == len(normalized_documents):\n",
" print(f\" Chunked {i}/{len(normalized_documents)} documents ({len(chunked_documents)} chunks so far)\")\n",
"\n",
"print(f\"Created {len(chunked_documents)} chunks from {len(normalized_documents)} documents\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Extracting DeFi Entities\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.semantic_extract import NERExtractor\n",
"\n",
"# Initialize NERExtractor with ML method only (spaCy)\n",
"# Note: ML method extracts standard NER labels (PERSON, ORG, GPE, etc.)\n",
"entity_extractor = NERExtractor(\n",
" method=[\"ml\"],\n",
" min_confidence=0.5\n",
")\n",
"\n",
"# Extract all entities (ML method doesn't support custom entity types)\n",
"# We'll filter/classify them after extraction\n",
"print(f\"Extracting entities from {len(chunked_documents)} chunks using ML (spaCy)...\")\n",
"batch_results = entity_extractor.extract(chunked_documents)\n",
"\n",
"# Flatten results (extract() returns List[List[Entity]] for batch input)\n",
"all_entities = [entity for entity_list in batch_results for entity in entity_list]\n",
"\n",
"# Use Semantica's classify_entities to group by standard labels\n",
"classified = entity_extractor.classify_entities(all_entities)\n",
"\n",
"# Filter entities for DeFi domain - look for protocol/token names in ORG entities\n",
"# and common DeFi-related terms\n",
"protocol_keywords = [\"uniswap\", \"aave\", \"compound\", \"makerdao\", \"curve\", \"yearn\", \n",
" \"sushiswap\", \"balancer\", \"protocol\", \"defi\", \"dapp\"]\n",
"token_keywords = [\"token\", \"coin\", \"crypto\", \"btc\", \"eth\", \"dai\", \"usdc\", \"usdt\"]\n",
"risk_keywords = [\"risk\", \"vulnerability\", \"exploit\", \"hack\", \"attack\", \"breach\"]\n",
"\n",
"protocols = [\n",
" e for e in all_entities \n",
" if e.label == \"ORG\" or any(kw in e.text.lower() for kw in protocol_keywords)\n",
"]\n",
"tokens = [\n",
" e for e in all_entities \n",
" if any(kw in e.text.lower() for kw in token_keywords) or e.label == \"MONEY\"\n",
"]\n",
"risks = [\n",
" e for e in all_entities \n",
" if any(kw in e.text.lower() for kw in risk_keywords)\n",
"]\n",
"\n",
"print(f\"\\n✅ Extraction complete!\")\n",
"print(f\" Total entities: {len(all_entities)}\")\n",
"print(f\" Standard labels: {list(classified.keys())}\")\n",
"print(f\" Protocols (filtered): {len(protocols)}\")\n",
"print(f\" Tokens (filtered): {len(tokens)}\")\n",
"print(f\" Risks (filtered): {len(risks)}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Extracting DeFi Relationships\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.semantic_extract import RelationExtractor\n",
"\n",
"# Use ML-based dependency parsing to avoid rate limits\n",
"relation_extractor = RelationExtractor(\n",
" method=\"dependency\", # ML/NLP method - no API calls needed\n",
" verbose=True\n",
")\n",
"\n",
"all_relationships = []\n",
"error_count = 0\n",
"print(f\"Extracting relationships from {len(chunked_documents)} chunks...\")\n",
"\n",
"for i, chunk in enumerate(chunked_documents, 1):\n",
" chunk_text = chunk.text if hasattr(chunk, 'text') else str(chunk)\n",
" try:\n",
" relationships = relation_extractor.extract_relations(\n",
" chunk_text,\n",
" entities=all_entities,\n",
" relation_types=[\"uses\", \"governs\", \"provides\", \"has_risk\", \"interacts_with\", \"depends_on\"],\n",
" verbose=True\n",
" )\n",
" all_relationships.extend(relationships)\n",
" except Exception as e:\n",
" error_count += 1\n",
" if error_count <= 3:\n",
" print(f\" Warning: Error on chunk {i}: {str(e)[:100]}\")\n",
" \n",
" if i % 20 == 0 or i == len(chunked_documents):\n",
" print(f\" Processed {i}/{len(chunked_documents)} chunks ({len(all_relationships)} relationships found)\")\n",
"\n",
"if error_count > 0:\n",
" print(f\" Note: {error_count} chunks had errors during relation extraction\")\n",
"\n",
"print(f\"Extracted {len(all_relationships)} relationships\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Resolving Duplicate Entities\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Detecting and Resolving Conflicts\n",
"\n",
"• **Entity & Relationship Conflict Detection**: Detects conflicts in both entity properties (protocol names, addresses) and relationships (protocol interactions, dependencies) from multiple data sources to ensure data consistency across the DeFi knowledge graph.\n",
"\n",
"• **Credibility-Weighted Resolution**: Uses credibility-weighted strategy that considers source reliability and extraction confidence scores, prioritizing high-confidence sources for critical DeFi protocol information while aggregating evidence from multiple sources.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.conflicts import ConflictDetector, ConflictResolver\n",
"\n",
"conflict_detector = ConflictDetector()\n",
"conflict_resolver = ConflictResolver()\n",
"\n",
"# Convert entities to dictionaries for conflict detection\n",
"entity_dicts = [\n",
" {\n",
" \"id\": e.id if hasattr(e, 'id') else e.text,\n",
" \"text\": e.text,\n",
" \"label\": e.label,\n",
" \"type\": e.label,\n",
" \"confidence\": e.confidence if hasattr(e, 'confidence') else 1.0,\n",
" \"metadata\": e.metadata if hasattr(e, 'metadata') else {}\n",
" }\n",
" for e in all_entities\n",
"]\n",
"\n",
"# Convert relationships to dictionaries for conflict detection\n",
"relationship_dicts = [\n",
" {\n",
" \"id\": f\"{r.subject.text}_{r.predicate}_{r.object.text}\",\n",
" \"source_id\": r.subject.text,\n",
" \"target_id\": r.object.text,\n",
" \"type\": r.predicate,\n",
" \"subject\": r.subject.text,\n",
" \"object\": r.object.text,\n",
" \"predicate\": r.predicate,\n",
" \"confidence\": r.confidence if hasattr(r, 'confidence') else 1.0,\n",
" \"metadata\": r.metadata if hasattr(r, 'metadata') else {}\n",
" }\n",
" for r in all_relationships\n",
"]\n",
"\n",
"# Detect conflicts in both entities and relationships\n",
"all_conflicts = []\n",
"\n",
"# 1. Detect entity conflicts (duplicate protocols, conflicting properties)\n",
"print(f\"Detecting entity conflicts in {len(entity_dicts)} entities...\")\n",
"entity_conflicts = conflict_detector.detect_entity_conflicts(entity_dicts)\n",
"all_conflicts.extend(entity_conflicts)\n",
"print(f\"Detected {len(entity_conflicts)} entity conflicts\")\n",
"\n",
"# 2. Detect relationship conflicts (conflicting protocol interactions)\n",
"print(f\"Detecting relationship conflicts in {len(relationship_dicts)} relationships...\")\n",
"relationship_conflicts = conflict_detector.detect_relationship_conflicts(relationship_dicts)\n",
"all_conflicts.extend(relationship_conflicts)\n",
"print(f\"Detected {len(relationship_conflicts)} relationship conflicts\")\n",
"\n",
"# Resolve all conflicts using credibility-weighted strategy\n",
"resolved_entities = entity_dicts.copy()\n",
"resolved_relationships = relationship_dicts.copy()\n",
"\n",
"if all_conflicts:\n",
" print(f\"Resolving {len(all_conflicts)} conflicts using credibility-weighted strategy...\")\n",
" resolved = conflict_resolver.resolve_conflicts(\n",
" all_conflicts,\n",
" strategy=\"credibility_weighted\" # Weight by source credibility and confidence\n",
" )\n",
" \n",
" # Apply resolved values back to entities and relationships\n",
" for result in resolved:\n",
" if result.resolved and result.resolved_value is not None:\n",
" if result.metadata.get(\"entity_id\"):\n",
" # Entity conflict - update entity\n",
" entity_id = result.metadata.get(\"entity_id\")\n",
" property_name = result.metadata.get(\"property_name\")\n",
" for entity in resolved_entities:\n",
" if entity.get(\"id\") == entity_id and property_name:\n",
" entity[property_name] = result.resolved_value\n",
" elif result.metadata.get(\"relationship_id\"):\n",
" # Relationship conflict - update relationship\n",
" rel_id = result.metadata.get(\"relationship_id\")\n",
" property_name = result.metadata.get(\"property_name\")\n",
" for rel in resolved_relationships:\n",
" if rel.get(\"id\") == rel_id and property_name:\n",
" rel[property_name] = result.resolved_value\n",
" \n",
" print(f\"Resolved {len([r for r in resolved if r.resolved])} conflicts\")\n",
" print(f\"Applied resolutions to {len(resolved_entities)} entities and {len(resolved_relationships)} relationships\")\n",
"else:\n",
" print(\"No conflicts detected\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Building DeFi Knowledge Graph\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.kg import GraphBuilder\n",
"\n",
"# Conflicts already resolved - disable conflict detection in GraphBuilder\n",
"graph_builder = GraphBuilder(\n",
" entity_resolution_strategy=\"fuzzy\",\n",
" resolve_conflicts=False # Conflicts already resolved in previous cell\n",
")\n",
"\n",
"kg_sources = [{\n",
" \"entities\": [\n",
" {\"id\": e.get(\"id\", e.get(\"text\")), \"text\": e.get(\"text\"), \"type\": e.get(\"type\", e.get(\"label\"))}\n",
" for e in resolved_entities\n",
" ],\n",
" \"relationships\": [\n",
" {\n",
" \"source\": r.get(\"source_id\", r.get(\"subject\")),\n",
" \"target\": r.get(\"target_id\", r.get(\"object\")),\n",
" \"type\": r.get(\"type\", r.get(\"predicate\"))\n",
" }\n",
" for r in resolved_relationships\n",
" ]\n",
"}]\n",
"\n",
"kg = graph_builder.build(kg_sources)\n",
"\n",
"entities_count = len(kg.get('entities', []))\n",
"relationships_count = len(kg.get('relationships', []))\n",
"print(f\"Graph: {entities_count} entities, {relationships_count} relationships\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generating Embeddings for Protocols and Tokens\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.embeddings import EmbeddingGenerator\n",
"\n",
"embedding_gen = EmbeddingGenerator(\n",
" provider=\"sentence_transformers\",\n",
" model=EMBEDDING_MODEL\n",
")\n",
"\n",
"print(f\"Generating embeddings for {len(protocols)} protocols and {len(tokens)} tokens...\")\n",
"protocol_texts = [p.text for p in protocols]\n",
"protocol_embeddings = embedding_gen.generate_embeddings(protocol_texts)\n",
"\n",
"token_texts = [t.text for t in tokens]\n",
"token_embeddings = embedding_gen.generate_embeddings(token_texts)\n",
"\n",
"print(f\"Generated {len(protocol_embeddings)} protocol embeddings and {len(token_embeddings)} token embeddings\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Populating Vector Store\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.vector_store import VectorStore\n",
"\n",
"vector_store = VectorStore(backend=\"faiss\", dimension=EMBEDDING_DIMENSION)\n",
"\n",
"print(f\"Storing {len(protocol_embeddings)} protocol vectors and {len(token_embeddings)} token vectors...\")\n",
"protocol_ids = vector_store.store_vectors(\n",
" vectors=protocol_embeddings,\n",
" metadata=[{\"type\": \"protocol\", \"name\": p.text, \"label\": p.label} for p in protocols]\n",
")\n",
"\n",
"token_ids = vector_store.store_vectors(\n",
" vectors=token_embeddings,\n",
" metadata=[{\"type\": \"token\", \"name\": t.text, \"label\": t.label} for t in tokens]\n",
")\n",
"\n",
"print(f\"Stored {len(protocol_ids)} protocol vectors and {len(token_ids)} token vectors\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generating DeFi Ontology\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.ontology import OntologyGenerator\n",
"\n",
"ontology_gen = OntologyGenerator(base_uri=\"https://defi.example.org/ontology/\")\n",
"ontology = ontology_gen.generate_from_graph(kg)\n",
"\n",
"print(f\"Generated DeFi ontology with {len(ontology.get('classes', []))} classes\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Reasoning and Risk Assessment\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.reasoning import Reasoner\n",
"from semantica.kg import GraphAnalyzer\n",
"\n",
"reasoner = Reasoner()\n",
"reasoner.add_rule(\"IF Protocol has_risk Risk AND Risk severity high THEN Protocol risk_level critical\")\n",
"reasoner.add_rule(\"IF Protocol depends_on Protocol AND Protocol has_risk Risk THEN Protocol inherits Risk\")\n",
"\n",
"inferred_facts = reasoner.infer_facts(kg)\n",
"\n",
"# Find paths from Protocols to Risks using GraphAnalyzer\n",
"graph_analyzer = GraphAnalyzer(kg)\n",
"protocols = [e.get(\"id\") or e.get(\"text\") for e in kg.get(\"entities\", []) if e.get(\"type\") == \"Protocol\"]\n",
"risks = [e.get(\"id\") or e.get(\"text\") for e in kg.get(\"entities\", []) if e.get(\"type\") == \"Risk\"]\n",
"\n",
"risk_paths = []\n",
"for protocol in protocols[:10]:\n",
" for risk in risks[:5]:\n",
" path = graph_analyzer.connectivity_analyzer.calculate_shortest_paths(kg, source=protocol, target=risk)\n",
" if path.get(\"exists\") and path.get(\"distance\", -1) <= 2:\n",
" risk_paths.append(path)\n",
"\n",
"print(f\"Inferred {len(inferred_facts)} facts\")\n",
"print(f\"Found {len(risk_paths)} risk paths\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Storing Knowledge Graph (Optional)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.graph_store import GraphStore\n",
"\n",
"# Optional: Store to persistent graph database\n",
"# graph_store = GraphStore(backend=\"neo4j\", uri=\"bolt://localhost:7687\", user=\"neo4j\", password=\"password\")\n",
"# graph_store.store_graph(kg)\n",
"\n",
"print(\"Graph store configured (commented out for demo)\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Storing Ontology as RDF Triplets (Optional)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.triplet_store import TripletStore\n",
"\n",
"# Store knowledge graph and ontology as RDF triplets\n",
"# Note: Requires Blazegraph running on localhost:9999\n",
"try:\n",
" triplet_store = TripletStore(backend=\"blazegraph\", endpoint=\"http://localhost:9999/blazegraph\")\n",
" result = triplet_store.store(knowledge_graph=kg, ontology=ontology)\n",
" \n",
" if result.get('success'):\n",
" print(f\"✓ Stored {result.get('processed', 0)}/{result.get('total', 0)} triplets successfully\")\n",
" else:\n",
" print(f\"⚠ Stored {result.get('processed', 0)}/{result.get('total', 0)} triplets ({result.get('failed', 0)} failed)\")\n",
"except Exception as e:\n",
" print(f\"⚠ Could not connect to Blazegraph: {str(e)[:100]}\")\n",
" print(\" To use triplet store, start Blazegraph on localhost:9999\")\n",
" print(\" Skipping triplet storage for this demo\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## GraphRAG: Hybrid Vector + Graph Queries\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.context import AgentContext\n",
"\n",
"context = AgentContext(vector_store=vector_store, knowledge_graph=kg)\n",
"\n",
"query = \"What protocols have high risk?\"\n",
"results = context.retrieve(\n",
" query,\n",
" max_results=10,\n",
" use_graph=True,\n",
" expand_graph=True,\n",
" include_entities=True,\n",
" include_relationships=True\n",
")\n",
"\n",
"print(f\"GraphRAG query: '{query}'\")\n",
"print(f\"\\nRetrieved {len(results)} results:\\n\")\n",
"for i, result in enumerate(results[:5], 1):\n",
" print(f\"{i}. Score: {result.get('score', 0):.3f}\")\n",
" print(f\" Content: {result.get('content', '')[:200]}...\")\n",
" if result.get('related_entities'):\n",
" print(f\" Related entities: {len(result['related_entities'])}\")\n",
" print()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Visualizing the DeFi Knowledge Graph\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.visualization import KGVisualizer\n",
"import plotly.graph_objects as go\n",
"\n",
"# Print graph statistics for context\n",
"num_entities = len(kg.get(\"entities\", []))\n",
"num_relationships = len(kg.get(\"relationships\", []))\n",
"print(f\"📊 Knowledge Graph Statistics:\")\n",
"print(f\" Entities: {num_entities}\")\n",
"print(f\" Relationships: {num_relationships}\")\n",
"print(f\" Density: {num_relationships / max(num_entities * (num_entities - 1) / 2, 1):.4f}\\n\")\n",
"\n",
"# Create visualizer with highly optimized settings for clarity and interactivity\n",
"visualizer = KGVisualizer(\n",
" layout=\"force\",\n",
" node_size=20, # Larger nodes for better visibility\n",
" edge_width=1.5,\n",
" color_scheme=\"vibrant\", # Colorful and distinct\n",
" k=3.0, # More spacing between nodes\n",
" iterations=150, # More iterations for stable layout\n",
" temperature=0.7,\n",
" cooling_factor=0.99\n",
")\n",
"\n",
"# Generate interactive Plotly figure with maximum interactivity\n",
"fig = visualizer.visualize_network(\n",
" kg,\n",
" output=\"interactive\",\n",
" algorithm=\"kamada_kawai\", # Best algorithm for complex graphs\n",
" node_color_by=\"type\", # Color by entity type\n",
" node_size_by=None,\n",
" hover_data=[\"type\", \"id\", \"label\"], # Rich hover information\n",
" scale=2.5, # Large scale for clear spacing\n",
" seed=42, # Reproducible\n",
" show_detailed_edges=True # Show edge labels\n",
")\n",
"\n",
"# Enhance the visualization with better interactivity and explanations\n",
"if fig:\n",
" # Update layout with comprehensive interactive features\n",
" fig.update_layout(\n",
" title={\n",
" \"text\": \"🔗 DeFi Protocol Knowledge Graph - Interactive Visualization\",\n",
" \"x\": 0.5,\n",
" \"xanchor\": \"center\",\n",
" \"font\": {\"size\": 20, \"color\": \"#2c3e50\"}\n",
" },\n",
" showlegend=True,\n",
" hovermode=\"closest\", # Show closest node on hover\n",
" margin=dict(b=40, l=40, r=40, t=80),\n",
" xaxis=dict(\n",
" showgrid=False,\n",
" zeroline=False,\n",
" showticklabels=False,\n",
" title=\"\"\n",
" ),\n",
" yaxis=dict(\n",
" showgrid=False,\n",
" zeroline=False,\n",
" showticklabels=False,\n",
" title=\"\"\n",
" ),\n",
" plot_bgcolor=\"rgba(250, 250, 250, 1)\", # Light gray background\n",
" paper_bgcolor=\"white\",\n",
" font=dict(family=\"Arial, sans-serif\", size=12),\n",
" # Add annotations for explanation\n",
" annotations=[\n",
" dict(\n",
" text=\"💡 How to interact:
\"\n",
" \"• Hover over nodes to see details
\"\n",
" \"• Click and drag to pan
\"\n",
" \"• Use mouse wheel to zoom
\"\n",
" \"• Double-click to reset view
\"\n",
" \"• Colors represent entity types\",\n",
" xref=\"paper\",\n",
" yref=\"paper\",\n",
" x=0.02,\n",
" y=0.98,\n",
" xanchor=\"left\",\n",
" yanchor=\"top\",\n",
" bgcolor=\"rgba(255, 255, 255, 0.9)\",\n",
" bordercolor=\"rgba(0, 0, 0, 0.2)\",\n",
" borderwidth=1,\n",
" font=dict(size=10, color=\"#34495e\"),\n",
" showarrow=False\n",
" )\n",
" ],\n",
" # Enhanced hover template\n",
" hoverlabel=dict(\n",
" bgcolor=\"rgba(255, 255, 255, 0.95)\",\n",
" bordercolor=\"#3498db\",\n",
" font_size=12,\n",
" font_family=\"Arial\"\n",
" ),\n",
" # Make it more responsive\n",
" autosize=True,\n",
" height=800,\n",
" width=None\n",
" )\n",
" \n",
" # Update traces for better interactivity\n",
" for trace in fig.data:\n",
" if hasattr(trace, 'marker'):\n",
" # Enhance node visibility\n",
" trace.marker.line.width = 2\n",
" trace.marker.line.color = \"white\"\n",
" trace.marker.opacity = 0.9\n",
" if hasattr(trace, 'text'):\n",
" # Make labels more readable\n",
" trace.textfont.size = 11\n",
" trace.textfont.color = \"#2c3e50\"\n",
" trace.textposition = \"middle center\"\n",
" \n",
" # Add modebar with useful tools\n",
" fig.update_layout(\n",
" modebar_add=[\n",
" \"zoom2d\",\n",
" \"pan2d\",\n",
" \"select2d\",\n",
" \"lasso2d\",\n",
" \"zoomIn2d\",\n",
" \"zoomOut2d\",\n",
" \"autoScale2d\",\n",
" \"resetScale2d\"\n",
" ]\n",
" )\n",
" \n",
" # Display the enhanced interactive graph\n",
" fig.show(config={\n",
" \"displayModeBar\": True,\n",
" \"displaylogo\": False,\n",
" \"modeBarButtonsToAdd\": [\"drawline\", \"drawopenpath\", \"drawclosedpath\", \"drawcircle\", \"drawrect\", \"eraseshape\"],\n",
" \"toImageButtonOptions\": {\n",
" \"format\": \"png\",\n",
" \"filename\": \"defi_kg\",\n",
" \"height\": 800,\n",
" \"width\": 1200,\n",
" \"scale\": 2\n",
" }\n",
" })\n",
" \n",
" print(\"\\n✅ Interactive visualization displayed!\")\n",
" print(\" Use the toolbar above to zoom, pan, and interact with the graph\")\n",
"else:\n",
" print(\"⚠️ Could not generate visualization\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Exporting Results\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantica.export import GraphExporter, RDFExporter\n",
"\n",
"# Export knowledge graph to graph formats\n",
"graph_exporter = GraphExporter()\n",
"graph_exporter.export(kg, output_path=\"defi_protocol_kg.json\", format=\"json\")\n",
"graph_exporter.export(kg, output_path=\"defi_protocol_kg.graphml\", format=\"graphml\")\n",
"\n",
"# Export ontology to RDF/TTL format using RDFExporter\n",
"rdf_exporter = RDFExporter()\n",
"rdf_string = rdf_exporter.export_to_rdf(ontology, format=\"turtle\")\n",
"with open(\"defi_ontology.ttl\", \"w\", encoding=\"utf-8\") as f:\n",
" f.write(rdf_string)\n",
"\n",
"print(\"✅ Exported knowledge graph to JSON and GraphML formats\")\n",
"print(\"✅ Exported ontology to RDF/TTL format\")\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 2
}