{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/introduction/04_Document_Parsing.ipynb)\n",
    "\n",
    "# Document Parsing\n",
    "\n",
    "## Overview\n",
    "\n",
    "This notebook demonstrates how to parse various document formats using Semantica's parsing modules. You'll learn to extract text, metadata, and structured data from PDFs, DOCX, CSV, JSON, XML, and HTML files.\n",
    "\n",
    "**Documentation**: [API Reference](https://semantica.readthedocs.io/reference/parse/)\n",
    "\n",
    "### Learning Objectives\n",
    "\n",
    "- Use `DocumentParser` for general document parsing\n",
    "- Use format-specific parsers: `PDFParser`, `DOCXParser`, `CSVParser`, `JSONParser`, `XMLParser`, `HTMLParser`\n",
    "- Extract text content and metadata from documents\n",
    "- Parse structured data formats\n",
    "\n",
    "## Installation\n",
    "\n",
    "Install Semantica from PyPI:\n",
    "\n",
    "```bash\n",
    "pip install semantica\n",
    "# Or with all optional dependencies:\n",
    "pip install semantica[all]\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Step 1: Document Parser\n",
    "\n",
    "Parse various document formats using the general DocumentParser.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install semantica\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.parse import DocumentParser\n",
    "import tempfile\n",
    "import os\n",
    "\n",
    "document_parser = DocumentParser()\n",
    "\n",
    "temp_dir = tempfile.mkdtemp()\n",
    "sample_txt = os.path.join(temp_dir, \"sample.txt\")\n",
    "\n",
    "with open(sample_txt, 'w') as f:\n",
    "    f.write(\"Apple Inc. is a technology company. Tim Cook is the CEO.\")\n",
    "\n",
    "text = document_parser.extract_text(sample_txt)\n",
    "metadata = document_parser.extract_metadata(sample_txt)\n",
    "\n",
    "text[:50], metadata\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: CSV Parser\n",
    "\n",
    "Parse CSV files to extract structured data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.parse import CSVParser\n",
    "\n",
    "csv_parser = CSVParser()\n",
    "csv_file = os.path.join(temp_dir, \"data.csv\")\n",
    "\n",
    "with open(csv_file, 'w') as f:\n",
    "    f.write(\"name,company,role\\n\")\n",
    "    f.write(\"Tim Cook,Apple Inc.,CEO\\n\")\n",
    "    f.write(\"Satya Nadella,Microsoft Corporation,CEO\\n\")\n",
    "\n",
    "csv_data = csv_parser.parse(csv_file)\n",
    "\n",
    "print(f\"Parsed CSV with {len(csv_data.rows)} rows\")\n",
    "print(f\"Columns: {csv_data.headers}\")\n",
    "for row in csv_data.rows[:2]:\n",
    "    print(f\"  {row}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: JSON Parser\n",
    "\n",
    "Parse JSON files to extract structured data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.parse import JSONParser\n",
    "import json\n",
    "\n",
    "json_parser = JSONParser()\n",
    "json_file = os.path.join(temp_dir, \"data.json\")\n",
    "\n",
    "data = {\n",
    "    \"companies\": [\n",
    "        {\"name\": \"Apple Inc.\", \"ceo\": \"Tim Cook\"},\n",
    "        {\"name\": \"Microsoft Corporation\", \"ceo\": \"Satya Nadella\"}\n",
    "    ]\n",
    "}\n",
    "\n",
    "with open(json_file, 'w') as f:\n",
    "    json.dump(data, f)\n",
    "\n",
    "json_data = json_parser.parse(json_file)\n",
    "\n",
    "print(f\"Parsed JSON: {json_data.data}\")\n",
    "print(f\"Companies: {len(json_data.data.get('companies', []))}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: XML Parser\n",
    "\n",
    "Parse XML files to extract structured data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.parse import XMLParser\n",
    "\n",
    "xml_parser = XMLParser()\n",
    "xml_file = os.path.join(temp_dir, \"data.xml\")\n",
    "\n",
    "xml_content = \"\"\"<?xml version=\"1.0\"?>\n",
    "<companies>\n",
    "    <company name=\"Apple Inc.\" ceo=\"Tim Cook\"/>\n",
    "    <company name=\"Microsoft Corporation\" ceo=\"Satya Nadella\"/>\n",
    "</companies>\"\"\"\n",
    "\n",
    "with open(xml_file, 'w') as f:\n",
    "    f.write(xml_content)\n",
    "\n",
    "xml_data = xml_parser.parse(xml_file)\n",
    "\n",
    "print(f\"Parsed XML with {len(xml_data.root.children)} elements\")\n",
    "print(f\"Root element: {xml_data.root.tag if xml_data.root else 'None'}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: HTML Parser\n",
    "\n",
    "Parse HTML files to extract content and structure.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.parse import HTMLParser\n",
    "\n",
    "html_parser = HTMLParser()\n",
    "html_file = os.path.join(temp_dir, \"page.html\")\n",
    "\n",
    "html_content = \"\"\"<html>\n",
    "<head><title>Sample Page</title></head>\n",
    "<body>\n",
    "    <h1>Technology Companies</h1>\n",
    "    <p>Apple Inc. is a technology company.</p>\n",
    "</body>\n",
    "</html>\"\"\"\n",
    "\n",
    "with open(html_file, 'w') as f:\n",
    "    f.write(html_content)\n",
    "\n",
    "html_data = html_parser.parse(html_file)\n",
    "\n",
    "print(f\"Parsed HTML\")\n",
    "print(f\"Title: {html_data.metadata.get('title', 'N/A')}\")\n",
    "print(f\"Text content: {html_data.text[:50]}...\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: Structured Data Parser\n",
    "\n",
    "Use StructuredDataParser for multiple formats.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from semantica.parse import StructuredDataParser\n",
    "\n",
    "structured_parser = StructuredDataParser()\n",
    "\n",
    "parsed_json = structured_parser.parse_data(json_file, data_format=\"json\")\n",
    "parsed_csv = structured_parser.parse_data(csv_file, data_format=\"csv\")\n",
    "\n",
    "print(f\"Structured parser parsed JSON: {len(parsed_json.get('data', {}).get('companies', []))} companies\")\n",
    "print(f\"Structured parser parsed CSV: {len(parsed_csv.get('rows', []))} rows\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "You've learned how to parse various document formats:\n",
    "\n",
    "- **DocumentParser**: General document parsing\n",
    "- **CSVParser**: CSV file parsing\n",
    "- **JSONParser**: JSON file parsing\n",
    "- **XMLParser**: XML file parsing\n",
    "- **HTMLParser**: HTML file parsing\n",
    "- **StructuredDataParser**: Multi-format structured data parsing\n",
    "\n",
    "Next: Learn how to normalize and clean data in the Data_Normalization notebook.\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
