{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/02_Advanced_Graph_Analytics.ipynb)\n",
    "\n",
    "# Advanced Graph Analytics\n",
    "\n",
    "## Overview\n",
    "\n",
    "This notebook demonstrates advanced graph analytics using GraphAnalyzer, CentralityCalculator, CommunityDetector, ConnectivityAnalyzer, GraphValidator, Deduplicator, and **GraphStore** for persistent storage.\n",
    "\n",
    "\n",
    "**Documentation**: [API Reference](https://semantica.readthedocs.io/reference/kg/)\n",
    "\n",
    "### Learning Objectives\n",
    "\n",
    "- Use GraphAnalyzer for comprehensive graph analysis\n",
    "- Use CentralityCalculator for advanced centrality measures\n",
    "- Use CommunityDetector for community detection\n",
    "- Use ConnectivityAnalyzer for connectivity analysis\n",
    "- Use GraphValidator and Deduplicator for graph quality\n",
    "- **Use GraphStore to persist graphs to Neo4j or FalkorDB**\n",
    "\n",
    "## Installation\n",
    "\n",
    "Install Semantica from PyPI:\n",
    "\n",
    "```bash\n",
    "pip install semantica\n",
    "# Or with all optional dependencies:\n",
    "pip install semantica[all]\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Workflow: Graph Analysis → Centrality → Communities → Connectivity → Validation → Deduplication → **Persist to Graph Store**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.kg import GraphBuilder, GraphAnalyzer, CentralityCalculator, CommunityDetector, ConnectivityAnalyzer, GraphValidator\n",
    "from semantica.deduplication import DuplicateDetector, EntityMerger, MergeStrategy\n",
    "\n",
    "builder = GraphBuilder()\n",
    "analyzer = GraphAnalyzer()\n",
    "\n",
    "entities = [\n",
    "    {\"id\": \"e1\", \"type\": \"Organization\", \"name\": \"Apple Inc.\", \"properties\": {}},\n",
    "    {\"id\": \"e2\", \"type\": \"Person\", \"name\": \"Tim Cook\", \"properties\": {}},\n",
    "    {\"id\": \"e3\", \"type\": \"Location\", \"name\": \"Cupertino\", \"properties\": {}}\n",
    "]\n",
    "\n",
    "relationships = [\n",
    "    {\"source\": \"e2\", \"target\": \"e1\", \"type\": \"CEO_of\", \"properties\": {}},\n",
    "    {\"source\": \"e1\", \"target\": \"e3\", \"type\": \"located_in\", \"properties\": {}}\n",
    "]\n",
    "\n",
    "kg = builder.build(entities, relationships)\n",
    "\n",
    "metrics = analyzer.compute_metrics(kg)\n",
    "\n",
    "print(f\"Graph metrics:\")\n",
    "print(f\"  Entities: {metrics.get('entity_count', 0)}\")\n",
    "print(f\"  Relationships: {metrics.get('relationship_count', 0)}\")\n",
    "print(f\"  Density: {metrics.get('density', 0):.3f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Advanced Centrality Measures\n",
    "\n",
    "Calculate multiple centrality measures.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "centrality_calculator = CentralityCalculator()\n",
    "\n",
    "degree_centrality_result = centrality_calculator.calculate_degree_centrality(kg)\n",
    "degree_centrality = degree_centrality_result.get('centrality', {})\n",
    "betweenness_centrality_result = centrality_calculator.calculate_betweenness_centrality(kg)\n",
    "betweenness_centrality = betweenness_centrality_result.get('centrality', {})\n",
    "\n",
    "print(f\"Degree centrality: {len(degree_centrality)} entities\")\n",
    "print(f\"Betweenness centrality: {len(betweenness_centrality)} entities\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Community Detection\n",
    "\n",
    "Detect communities in the graph.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "community_detector = CommunityDetector()\n",
    "\n",
    "communities = community_detector.detect_communities(kg)\n",
    "\n",
    "print(f\"Detected {len(communities)} communities\")\n",
    "for i, community in enumerate(communities[:3], 1):\n",
    "    print(f\"  Community {i}: {len(community)} entities\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Connectivity Analysis\n",
    "\n",
    "Analyze graph connectivity.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "connectivity_analyzer = ConnectivityAnalyzer()\n",
    "\n",
    "connectivity = connectivity_analyzer.analyze_connectivity(kg)\n",
    "\n",
    "print(f\"Connectivity analysis:\")\n",
    "print(f\"  Is connected: {connectivity.get('is_connected', False)}\")\n",
    "print(f\"  Components: {len(connectivity.get('components', []))}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Graph Validation and Deduplication\n",
    "\n",
    "Validate and deduplicate the graph.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "graph_validator = GraphValidator()\n",
    "\n",
    "validation_result = graph_validator.validate(kg)\n",
    "\n",
    "print(f\"Graph validation: {validation_result.get('valid', False)}\")\n",
    "print(f\"Issues found: {len(validation_result.get('issues', []))}\")\n",
    "\n",
    "# For deduplication, use semantica.deduplication module:\n",
    "# from semantica.deduplication import DuplicateDetector, EntityMerger, MergeStrategy\n",
    "# detector = DuplicateDetector(similarity_threshold=0.8)\n",
    "# duplicate_groups = detector.detect_duplicate_groups(kg.get('entities', []))\n",
    "# merger = EntityMerger()\n",
    "# merge_operations = merger.merge_duplicates(kg.get('entities', []), strategy=MergeStrategy.KEEP_MOST_COMPLETE)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: Persist to Graph Store\n",
    "\n",
    "Store the analyzed graph in a persistent graph database using GraphStore.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.graph_store import GraphStore\n",
    "\n",
    "# Option 1: Neo4j (requires Neo4j server running)\n",
    "graph_store = GraphStore(backend=\"neo4j\", uri=\"bolt://localhost:7687\", user=\"neo4j\", password=\"password\")\n",
    "graph_store.connect()\n",
    "\n",
    "# Store entities as nodes and track node ID mapping\n",
    "node_id_map = {}\n",
    "for entity in entities:\n",
    "    node = graph_store.create_node(\n",
    "        labels=[entity[\"type\"]],\n",
    "        properties={\"name\": entity[\"name\"], \"original_id\": entity[\"id\"]}\n",
    "    )\n",
    "    node_id_map[entity[\"id\"]] = node.get(\"id\")\n",
    "    print(f\"Stored node: {entity['name']} (ID: {node.get('id')})\")\n",
    "\n",
    "# Store relationships using mapped node IDs\n",
    "for rel in relationships:\n",
    "    source_id = node_id_map.get(rel[\"source\"])\n",
    "    target_id = node_id_map.get(rel[\"target\"])\n",
    "    \n",
    "    if source_id is not None and target_id is not None:\n",
    "        relationship = graph_store.create_relationship(\n",
    "            start_node_id=source_id,\n",
    "            end_node_id=target_id,\n",
    "            rel_type=rel[\"type\"],\n",
    "            properties=rel.get(\"properties\", {})\n",
    "        )\n",
    "        print(f\"Stored relationship: {rel['source']} -{rel['type']}-> {rel['target']}\")\n",
    "    else:\n",
    "        print(f\"Warning: Could not find node IDs for relationship {rel['source']} -> {rel['target']}\")\n",
    "\n",
    "# Query using Cypher\n",
    "results = graph_store.execute_query(\"MATCH (n) RETURN n.name, labels(n) LIMIT 10\")\n",
    "print(f\"\\nQuery results: {len(results.get('records', []))} nodes\")\n",
    "\n",
    "# Get statistics\n",
    "stats = graph_store.get_stats()\n",
    "print(f\"\\nGraph store statistics:\")\n",
    "print(f\"  Node count: {stats.get('node_count', 'N/A')}\")\n",
    "print(f\"  Relationship count: {stats.get('relationship_count', 'N/A')}\")\n",
    "print(f\"  Label counts: {stats.get('label_counts', {})}\")\n",
    "\n",
    "graph_store.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "You've learned advanced graph analytics:\n",
    "\n",
    "- **GraphAnalyzer**: Comprehensive graph analysis and metrics\n",
    "- **CentralityCalculator**: Multiple centrality measures\n",
    "- **CommunityDetector**: Community detection\n",
    "- **ConnectivityAnalyzer**: Connectivity analysis\n",
    "- **GraphValidator**: Graph validation\n",
    "- **Deduplicator**: Graph deduplication\n",
    "- **GraphStore**: Persist graphs to Neo4j or FalkorDB\n",
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}