{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Notice**: The `semantica.kg_qa` module is temporarily unavailable and will be reintroduced in a future release. Any quality assessment examples in this notebook are disabled."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/use_cases/renewable_energy/02_Environmental_Impact.ipynb)\n",
    "\n",
    "# Environmental Impact Analysis Pipeline\n",
    "\n",
    "## Overview\n",
    "\n",
    "This notebook demonstrates a complete environmental impact analysis pipeline: ingest environmental data from multiple sources (EPA APIs, climate databases, sustainability feeds), extract environmental entities, build impact knowledge graph, analyze relationships, and assess environmental impact.\n",
    "\n",
    "\n",
    "**Documentation**: [API Reference](https://semantica.readthedocs.io/use-cases/)\n",
    "\n",
    "### Modules Used (20+)\n",
    "\n",
    "- **Ingestion**: FileIngestor, WebIngestor, FeedIngestor, StreamIngestor, DBIngestor, RepoIngestor, EmailIngestor, MCPIngestor\n",
    "- **Parsing**: JSONParser, CSVParser, StructuredDataParser, DocumentParser\n",
    "- **Extraction**: NERExtractor, RelationExtractor, EventDetector, SemanticAnalyzer\n",
    "- **KG**: GraphBuilder, GraphAnalyzer, CentralityCalculator, CommunityDetector\n",
    "- **Analytics**: ConnectivityAnalyzer, TemporalGraphQuery, TemporalPatternDetector\n",
    "- **Ontology**: OntologyGenerator, ClassInferrer, PropertyGenerator, OntologyValidator\n",
    "- **Reasoning**: InferenceEngine, RuleManager, ExplanationGenerator\n",
    "- **Quality**: KGQualityAssessor, ConflictDetector\n",
    "- **Export**: JSONExporter, CSVExporter, RDFExporter, OWLExporter, ReportGenerator\n",
    "- **Visualization**: KGVisualizer, OntologyVisualizer, AnalyticsVisualizer\n",
    "\n",
    "### Pipeline\n",
    "\n",
    "**Environmental Data Sources → Parse → Extract Entities → Build Impact KG → Analyze Relationships → Assess Impact → Generate Ontology → Reports → Visualize**\n",
    "\n",
    "## Installation\n",
    "\n",
    "Install Semantica from PyPI:\n",
    "\n",
    "```bash\n",
    "pip install semantica\n",
    "# Or with all optional dependencies:\n",
    "pip install semantica[all]\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Step 1: Ingest Environmental Data from Multiple Sources\n",
    "\n",
    "Ingest environmental data from EPA APIs, climate databases, and sustainability feeds.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.ingest import FileIngestor, WebIngestor, DBIngestor, FeedIngestor\n",
    "from semantica.parse import JSONParser, CSVParser, StructuredDataParser, DocumentParser\n",
    "from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector, SemanticAnalyzer\n",
    "from semantica.kg import GraphBuilder, GraphAnalyzer, CentralityCalculator, CommunityDetector\n",
    "from semantica.kg import ConnectivityAnalyzer, TemporalGraphQuery, TemporalPatternDetector\n",
    "from semantica.ontology import OntologyGenerator, ClassInferrer, PropertyGenerator, OntologyValidator\n",
    "from semantica.reasoning import InferenceEngine, RuleManager, ExplanationGenerator\n",
    "from semantica.conflicts import ConflictDetector\n",
    "from semantica.export import JSONExporter, CSVExporter, RDFExporter, OWLExporter, ReportGenerator\n",
    "from semantica.visualization import KGVisualizer, OntologyVisualizer, AnalyticsVisualizer\n",
    "import tempfile\n",
    "import os\n",
    "import json\n",
    "from datetime import datetime, timedelta\n",
    "\n",
    "file_ingestor = FileIngestor()\n",
    "web_ingestor = WebIngestor()\n",
    "db_ingestor = DBIngestor()\n",
    "feed_ingestor = FeedIngestor()\n",
    "\n",
    "json_parser = JSONParser()\n",
    "csv_parser = CSVParser()\n",
    "structured_parser = StructuredDataParser()\n",
    "document_parser = DocumentParser()\n",
    "\n",
    "# Real environmental data sources\n",
    "environmental_apis = [\n",
    "    \"https://www.epa.gov/enviro/facts-service\",  # EPA Environmental Facts Service\n",
    "    \"https://www.epa.gov/airdata\",  # EPA Air Data\n",
    "    \"https://api.github.com/repos/climate-data/aggregator\"  # Climate data aggregator\n",
    "]\n",
    "\n",
    "environmental_feeds = [\n",
    "    \"https://www.epa.gov/rss\",  # EPA RSS Feed\n",
    "    \"https://www.energy.gov/rss\",  # US Energy Department RSS\n",
    "    \"https://feeds.reuters.com/reuters/environment\"  # Reuters Environment News\n",
    "]\n",
    "\n",
    "# Real database connection for environmental data\n",
    "db_connection_string = \"postgresql://user:password@localhost:5432/environmental_db\"\n",
    "db_query = \"SELECT project_id, energy_type, co2_reduction, water_saved, land_impact, timestamp FROM environmental_impact WHERE timestamp > CURRENT_DATE - INTERVAL '1 year' ORDER BY timestamp DESC\"\n",
    "\n",
    "temp_dir = tempfile.mkdtemp()\n",
    "\n",
    "# Sample environmental impact data\n",
    "environmental_file = os.path.join(temp_dir, \"environmental_impact.json\")\n",
    "environmental_data = [\n",
    "    {\n",
    "        \"project_id\": \"PROJ-001\",\n",
    "        \"energy_type\": \"Solar\",\n",
    "        \"co2_reduction_tons\": 5000,\n",
    "        \"water_saved_gallons\": 1000000,\n",
    "        \"land_impact_acres\": 50,\n",
    "        \"carbon_offset\": 5000,\n",
    "        \"timestamp\": (datetime.now() - timedelta(days=60)).isoformat(),\n",
    "        \"region\": \"California\"\n",
    "    },\n",
    "    {\n",
    "        \"project_id\": \"PROJ-002\",\n",
    "        \"energy_type\": \"Wind\",\n",
    "        \"co2_reduction_tons\": 8000,\n",
    "        \"water_saved_gallons\": 2000000,\n",
    "        \"land_impact_acres\": 100,\n",
    "        \"carbon_offset\": 8000,\n",
    "        \"timestamp\": (datetime.now() - timedelta(days=30)).isoformat(),\n",
    "        \"region\": \"Texas\"\n",
    "    },\n",
    "    {\n",
    "        \"project_id\": \"PROJ-003\",\n",
    "        \"energy_type\": \"Hydroelectric\",\n",
    "        \"co2_reduction_tons\": 3000,\n",
    "        \"water_saved_gallons\": 500000,\n",
    "        \"land_impact_acres\": 200,\n",
    "        \"carbon_offset\": 3000,\n",
    "        \"timestamp\": datetime.now().isoformat(),\n",
    "        \"region\": \"Pacific Northwest\"\n",
    "    }\n",
    "]\n",
    "\n",
    "with open(environmental_file, 'w') as f:\n",
    "    json.dump(environmental_data, f, indent=2)\n",
    "\n",
    "file_objects = file_ingestor.ingest_file(environmental_file, read_content=True)\n",
    "parsed_data = structured_parser.parse_json(environmental_file)\n",
    "\n",
    "# Ingest from environmental APIs\n",
    "environmental_api_list = []\n",
    "for api_url in environmental_apis[:1]:\n",
    "    api_content = web_ingestor.ingest_url(api_url)\n",
    "    if api_content:\n",
    "        environmental_api_list.append(api_content)\n",
    "        print(f\"  Ingested environmental API: {api_url}\")\n",
    "\n",
    "# Ingest from environmental feeds\n",
    "environmental_feed_list = []\n",
    "for feed_url in environmental_feeds:\n",
    "    feed_data = feed_ingestor.ingest_feed(feed_url)\n",
    "    if feed_data:\n",
    "        environmental_feed_list.append(feed_data)\n",
    "        print(f\"  Ingested feed: {feed_url}\")\n",
    "\n",
    "print(f\"\\n📊 Environmental Data Ingestion Summary:\")\n",
    "print(f\"  Environmental data files: {len([file_objects]) if file_objects else 0}\")\n",
    "print(f\"  Environmental APIs: {len(environmental_api_list)}\")\n",
    "print(f\"  Environmental feeds: {len(environmental_feed_list)}\")\n",
    "print(f\"  Database sources: 1\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Extract Environmental Entities and Build Impact Knowledge Graph\n",
    "\n",
    "Extract environmental entities and build impact knowledge graph.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ner_extractor = NERExtractor()\n",
    "relation_extractor = RelationExtractor()\n",
    "event_detector = EventDetector()\n",
    "semantic_analyzer = SemanticAnalyzer()\n",
    "\n",
    "environmental_entities = []\n",
    "environmental_relationships = []\n",
    "\n",
    "# Extract from environmental data\n",
    "if parsed_data and parsed_data.data:\n",
    "    for entry in parsed_data.data if isinstance(parsed_data.data, list) else [parsed_data.data]:\n",
    "        if isinstance(entry, dict):\n",
    "            project_id = entry.get(\"project_id\", \"\")\n",
    "            energy_type = entry.get(\"energy_type\", \"\")\n",
    "            region = entry.get(\"region\", \"\")\n",
    "            \n",
    "            environmental_entities.append({\n",
    "                \"id\": project_id,\n",
    "                \"type\": \"Project\",\n",
    "                \"name\": project_id,\n",
    "                \"properties\": {\n",
    "                    \"energy_type\": energy_type,\n",
    "                    \"region\": region,\n",
    "                    \"timestamp\": entry.get(\"timestamp\", \"\")\n",
    "                }\n",
    "            })\n",
    "            \n",
    "            environmental_entities.append({\n",
    "                \"id\": energy_type,\n",
    "                \"type\": \"Energy_Source\",\n",
    "                \"name\": energy_type,\n",
    "                \"properties\": {}\n",
    "            })\n",
    "            \n",
    "            environmental_entities.append({\n",
    "                \"id\": f\"{project_id}_co2_reduction\",\n",
    "                \"type\": \"Environmental_Impact\",\n",
    "                \"name\": \"CO2 Reduction\",\n",
    "                \"properties\": {\n",
    "                    \"metric\": \"CO2\",\n",
    "                    \"value\": entry.get(\"co2_reduction_tons\", 0),\n",
    "                    \"unit\": \"tons\"\n",
    "                }\n",
    "            })\n",
    "            \n",
    "            environmental_entities.append({\n",
    "                \"id\": f\"{project_id}_water_saved\",\n",
    "                \"type\": \"Environmental_Impact\",\n",
    "                \"name\": \"Water Saved\",\n",
    "                \"properties\": {\n",
    "                    \"metric\": \"Water\",\n",
    "                    \"value\": entry.get(\"water_saved_gallons\", 0),\n",
    "                    \"unit\": \"gallons\"\n",
    "                }\n",
    "            })\n",
    "            \n",
    "            environmental_relationships.append({\n",
    "                \"source\": project_id,\n",
    "                \"target\": energy_type,\n",
    "                \"type\": \"uses\",\n",
    "                \"properties\": {}\n",
    "            })\n",
    "            \n",
    "            environmental_relationships.append({\n",
    "                \"source\": project_id,\n",
    "                \"target\": f\"{project_id}_co2_reduction\",\n",
    "                \"type\": \"reduces\",\n",
    "                \"properties\": {}\n",
    "            })\n",
    "            \n",
    "            environmental_relationships.append({\n",
    "                \"source\": project_id,\n",
    "                \"target\": f\"{project_id}_water_saved\",\n",
    "                \"type\": \"saves\",\n",
    "                \"properties\": {}\n",
    "            })\n",
    "\n",
    "builder = GraphBuilder()\n",
    "graph_analyzer = GraphAnalyzer()\n",
    "centrality_calculator = CentralityCalculator()\n",
    "community_detector = CommunityDetector()\n",
    "\n",
    "impact_kg = builder.build(environmental_entities, environmental_relationships)\n",
    "\n",
    "metrics = graph_analyzer.compute_metrics(impact_kg)\n",
    "centrality_result = centrality_calculator.calculate_degree_centrality(impact_kg)\n",
    "centrality_scores = centrality_result.get('centrality', {})\n",
    "communities = community_detector.detect_communities(impact_kg)\n",
    "\n",
    "print(f\"Extracted {len(environmental_entities)} environmental entities\")\n",
    "print(f\"Extracted {len(environmental_relationships)} relationships\")\n",
    "print(f\"Built impact knowledge graph with {len(impact_kg.get('entities', []))} entities\")\n",
    "print(f\"Graph density: {metrics.get('density', 0):.3f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Analyze Environmental Relationships\n",
    "\n",
    "Analyze environmental relationships using graph analytics.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "connectivity_analyzer = ConnectivityAnalyzer()\n",
    "temporal_query = TemporalGraphQuery()\n",
    "temporal_pattern_detector = TemporalPatternDetector()\n",
    "\n",
    "connectivity = connectivity_analyzer.analyze_connectivity(impact_kg)\n",
    "\n",
    "start_time = (datetime.now() - timedelta(days=365)).isoformat()\n",
    "end_time = datetime.now().isoformat()\n",
    "\n",
    "temporal_results = temporal_query.query_time_range(\n",
    "    graph=impact_kg,\n",
    "    query=\"Find environmental impacts in the last year\",\n",
    "    start_time=start_time,\n",
    "    end_time=end_time\n",
    ")\n",
    "\n",
    "temporal_patterns = temporal_pattern_detector.detect_temporal_patterns(\n",
    "    impact_kg,\n",
    "    pattern_type=\"trend\",\n",
    "    min_frequency=1\n",
    ")\n",
    "\n",
    "# Analyze impact by energy type\n",
    "impact_analysis = {}\n",
    "if parsed_data and parsed_data.data:\n",
    "    for entry in parsed_data.data if isinstance(parsed_data.data, list) else [parsed_data.data]:\n",
    "        if isinstance(entry, dict):\n",
    "            energy_type = entry.get(\"energy_type\", \"\")\n",
    "            \n",
    "            if energy_type not in impact_analysis:\n",
    "                impact_analysis[energy_type] = {\n",
    "                    \"co2_reduction\": [],\n",
    "                    \"water_saved\": [],\n",
    "                    \"projects\": []\n",
    "                }\n",
    "            \n",
    "            impact_analysis[energy_type][\"co2_reduction\"].append(entry.get(\"co2_reduction_tons\", 0))\n",
    "            impact_analysis[energy_type][\"water_saved\"].append(entry.get(\"water_saved_gallons\", 0))\n",
    "            impact_analysis[energy_type][\"projects\"].append(entry.get(\"project_id\", \"\"))\n",
    "\n",
    "# Calculate totals\n",
    "for energy_type, data in impact_analysis.items():\n",
    "    impact_analysis[energy_type][\"total_co2_reduction\"] = sum(data[\"co2_reduction\"])\n",
    "    impact_analysis[energy_type][\"total_water_saved\"] = sum(data[\"water_saved\"])\n",
    "    impact_analysis[energy_type][\"project_count\"] = len(data[\"projects\"])\n",
    "\n",
    "print(f\"Environmental relationships analyzed\")\n",
    "print(f\"  Connected components: {len(connectivity.get('components', []))}\")\n",
    "print(f\"  Temporal patterns: {len(temporal_patterns)}\")\n",
    "print(f\"  Energy types analyzed: {len(impact_analysis)}\")\n",
    "for energy_type, data in impact_analysis.items():\n",
    "    print(f\"    {energy_type}: {data['total_co2_reduction']:,} tons CO2 reduced, {data['total_water_saved']:,} gallons saved, {data['project_count']} projects\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Assess Environmental Impact\n",
    "\n",
    "Assess environmental impact using inference engine and generate ontology.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inference_engine = InferenceEngine()\n",
    "rule_manager = RuleManager()\n",
    "explanation_generator = ExplanationGenerator()\n",
    "ontology_generator = OntologyGenerator()\n",
    "class_inferrer = ClassInferrer()\n",
    "property_generator = PropertyGenerator()\n",
    "ontology_validator = OntologyValidator()\n",
    "\n",
    "# Environmental impact assessment rules\n",
    "inference_engine.add_rule(\"IF co2_reduction > 5000 AND water_saved > 1000000 THEN high_impact_project\")\n",
    "inference_engine.add_rule(\"IF energy_type is Solar AND co2_reduction > 3000 THEN sustainable_solar\")\n",
    "inference_engine.add_rule(\"IF multiple projects use same energy_type THEN scalable_solution\")\n",
    "\n",
    "# Assess impact\n",
    "impact_assessments = []\n",
    "if parsed_data and parsed_data.data:\n",
    "    for entry in parsed_data.data if isinstance(parsed_data.data, list) else [parsed_data.data]:\n",
    "        if isinstance(entry, dict):\n",
    "            co2_reduction = entry.get(\"co2_reduction_tons\", 0)\n",
    "            water_saved = entry.get(\"water_saved_gallons\", 0)\n",
    "            \n",
    "            impact_score = (co2_reduction / 1000) + (water_saved / 100000)\n",
    "            impact_level = \"high\" if impact_score > 10 else \"medium\" if impact_score > 5 else \"low\"\n",
    "            \n",
    "            assessment = {\n",
    "                \"project_id\": entry.get(\"project_id\", \"\"),\n",
    "                \"energy_type\": entry.get(\"energy_type\", \"\"),\n",
    "                \"co2_reduction\": co2_reduction,\n",
    "                \"water_saved\": water_saved,\n",
    "                \"impact_score\": impact_score,\n",
    "                \"impact_level\": impact_level\n",
    "            }\n",
    "            impact_assessments.append(assessment)\n",
    "            \n",
    "            inference_engine.add_fact({\n",
    "                \"project_id\": entry.get(\"project_id\", \"\"),\n",
    "                \"energy_type\": entry.get(\"energy_type\", \"\"),\n",
    "                \"co2_reduction\": co2_reduction,\n",
    "                \"water_saved\": water_saved\n",
    "            })\n",
    "\n",
    "impact_insights = inference_engine.forward_chain()\n",
    "\n",
    "# Generate environmental ontology\n",
    "impact_ontology = ontology_generator.generate_ontology({\n",
    "    \"entities\": environmental_entities,\n",
    "    \"relationships\": environmental_relationships\n",
    "})\n",
    "classes = class_inferrer.infer_classes(environmental_entities)\n",
    "properties = property_generator.infer_properties(environmental_entities, environmental_relationships, classes)\n",
    "validation_result = ontology_validator.validate_ontology(impact_ontology)\n",
    "\n",
    "print(f\"Environmental impact assessment complete\")\n",
    "print(f\"  Projects assessed: {len(impact_assessments)}\")\n",
    "print(f\"  High impact projects: {len([a for a in impact_assessments if a.get('impact_level') == 'high'])}\")\n",
    "print(f\"  Generated {len(impact_insights)} impact insights\")\n",
    "print(f\"  Ontology valid: {validation_result.valid}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Generate Reports and Visualize\n",
    "\n",
    "Generate environmental impact reports and visualize results.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "quality_assessor = KGQualityAssessor()\n",
    "json_exporter = JSONExporter()\n",
    "csv_exporter = CSVExporter()\n",
    "rdf_exporter = RDFExporter()\n",
    "owl_exporter = OWLExporter()\n",
    "report_generator = ReportGenerator()\n",
    "\n",
    "quality_score = quality_assessor.assess_overall_quality(impact_kg)\n",
    "\n",
    "json_exporter.export_knowledge_graph(impact_kg, os.path.join(temp_dir, \"environmental_impact_kg.json\"))\n",
    "csv_exporter.export_entities(environmental_entities, os.path.join(temp_dir, \"environmental_entities.csv\"))\n",
    "rdf_exporter.export_knowledge_graph(impact_kg, os.path.join(temp_dir, \"environmental_impact_kg.rdf\"))\n",
    "owl_exporter.export(impact_ontology, os.path.join(temp_dir, \"environmental_ontology.owl\"))\n",
    "\n",
    "total_co2_reduction = sum(a.get(\"co2_reduction\", 0) for a in impact_assessments)\n",
    "total_water_saved = sum(a.get(\"water_saved\", 0) for a in impact_assessments)\n",
    "\n",
    "report_data = {\n",
    "    \"summary\": f\"Environmental impact analysis identified {len(impact_assessments)} projects with {total_co2_reduction:,} tons CO2 reduction and {total_water_saved:,} gallons water saved\",\n",
    "    \"projects_analyzed\": len(impact_assessments),\n",
    "    \"total_co2_reduction\": total_co2_reduction,\n",
    "    \"total_water_saved\": total_water_saved,\n",
    "    \"high_impact_projects\": len([a for a in impact_assessments if a.get(\"impact_level\") == \"high\"]),\n",
    "    \"insights\": len(impact_insights),\n",
    "    \"quality_score\": quality_score.get('overall_score', 0)\n",
    "}\n",
    "\n",
    "report = report_generator.generate_report(report_data, format=\"markdown\")\n",
    "\n",
    "kg_visualizer = KGVisualizer()\n",
    "ontology_visualizer = OntologyVisualizer()\n",
    "analytics_visualizer = AnalyticsVisualizer()\n",
    "\n",
    "kg_viz = kg_visualizer.visualize_network(impact_kg, output=\"interactive\")\n",
    "ontology_viz = ontology_visualizer.visualize_hierarchy(impact_ontology, output=\"interactive\")\n",
    "analytics_viz = analytics_visualizer.visualize_analytics(impact_kg, output=\"interactive\")\n",
    "\n",
    "print(f\"Total modules used: 20+\")\n",
    "print(f\"Pipeline complete: Environmental Data → Parse → Extract → Build Impact KG → Analyze Relationships → Assess Impact → Generate Ontology → Reports → Visualize\")\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
