{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Hawksight-AI/semantica/blob/main/cookbook/advanced/Advanced_Vector_Store_and_Search.ipynb)\n",
    "\n",
    "# Advanced Vector Store - Made Easy\n",
    "\n",
    "## What You'll Learn\n",
    "\n",
    "This notebook shows you **practical ways** to use vector stores in real applications. Each example is simple and ready to use.\n",
    "\n",
    "### Topics\n",
    "\n",
    "1. **Choosing the Right Index** - Which one to use and when\n",
    "2. **Smart Filtering** - Find exactly what you need\n",
    "3. **Combining Results** - Merge searches from different sources\n",
    "4. **Organizing Data** - Keep different users' data separate\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: semantica in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (0.0.5)"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~gno (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~lotly (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~ython-socketio (C:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Requirement already satisfied: numpy>=1.21.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.26.4)\n",
      "Requirement already satisfied: pandas>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.3)\n",
      "Requirement already satisfied: scikit-learn>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.6.1)\n",
      "Requirement already satisfied: umap-learn>=0.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.5.9.post2)\n",
      "Requirement already satisfied: spacy>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.8.11)\n",
      "Requirement already satisfied: transformers>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.53.2)\n",
      "Requirement already satisfied: torch>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.2.1)\n",
      "Requirement already satisfied: sentence-transformers>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.2.1)\n",
      "Requirement already satisfied: rdflib>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.0)\n",
      "Requirement already satisfied: networkx>=2.8.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.6.1)\n",
      "Requirement already satisfied: matplotlib>=3.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.10.1)\n",
      "Requirement already satisfied: seaborn>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.13.2)\n",
      "Requirement already satisfied: plotly>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.17.0)\n",
      "Requirement already satisfied: ipywidgets>=8.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.1.1)\n",
      "Requirement already satisfied: requests>=2.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.32.5)\n",
      "Requirement already satisfied: beautifulsoup4>=4.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.14.3)\n",
      "Requirement already satisfied: lxml>=4.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.2)\n",
      "Requirement already satisfied: pypdf2>=2.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.0.1)\n",
      "Requirement already satisfied: python-docx>=0.8.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
      "Requirement already satisfied: openpyxl>=3.0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.1.5)\n",
      "Requirement already satisfied: pillow>=9.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (10.4.0)\n",
      "Requirement already satisfied: librosa>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.11.0)\n",
      "Requirement already satisfied: opencv-python>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.9.0.80)\n",
      "Requirement already satisfied: faiss-cpu>=1.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.9.0)\n",
      "Requirement already satisfied: fastembed>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
      "Requirement already satisfied: onnxruntime>=1.17.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.22.1)\n",
      "Requirement already satisfied: tokenizers>=0.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.4)\n",
      "Requirement already satisfied: weaviate-client>=3.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.18.1)\n",
      "Requirement already satisfied: qdrant-client>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.12.2)\n",
      "Requirement already satisfied: neo4j>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.3)\n",
      "Requirement already satisfied: falkordb>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.2.2)\n",
      "Requirement already satisfied: pymongo>=4.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.15.4)\n",
      "Requirement already satisfied: sqlalchemy>=1.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.0.23)\n",
      "Requirement already satisfied: psycopg2-binary>=2.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.9.9)\n",
      "Requirement already satisfied: pymysql>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.2)\n",
      "Requirement already satisfied: redis>=4.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.4.0)\n",
      "Requirement already satisfied: celery>=5.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.3.4)\n",
      "Requirement already satisfied: kafka-python>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.3.0)\n",
      "Requirement already satisfied: pulsar-client>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (3.3.0)\n",
      "Requirement already satisfied: pika>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.3.2)\n",
      "Requirement already satisfied: boto3>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.36.0)\n",
      "Requirement already satisfied: azure-storage-blob>=12.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (12.27.1)\n",
      "Requirement already satisfied: google-cloud-storage>=2.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.18.2)\n",
      "Requirement already satisfied: pydantic>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.12.3)\n",
      "Requirement already satisfied: fastmcp>=0.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (2.14.1)\n",
      "Requirement already satisfied: click>=8.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (8.2.1)\n",
      "Requirement already satisfied: rich>=12.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (14.2.0)\n",
      "Requirement already satisfied: tqdm>=4.64.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.67.1)\n",
      "Requirement already satisfied: pyyaml>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (6.0.1)\n",
      "Requirement already satisfied: toml>=0.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.10.2)\n",
      "Requirement already satisfied: python-dotenv>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.1.1)\n",
      "Requirement already satisfied: loguru>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.7.3)\n",
      "Requirement already satisfied: structlog>=22.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (24.4.0)\n",
      "Requirement already satisfied: prometheus-client>=0.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.23.1)\n",
      "Requirement already satisfied: opentelemetry-api>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.39.1)\n",
      "Requirement already satisfied: opentelemetry-sdk>=1.12.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.39.1)\n",
      "Requirement already satisfied: opentelemetry-instrumentation in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.60b1)\n",
      "Requirement already satisfied: fastapi>=0.78.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.120.4)\n",
      "Requirement already satisfied: uvicorn>=0.18.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.38.0)\n",
      "Requirement already satisfied: pytest>=7.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.4.3)\n",
      "Requirement already satisfied: pytest-cov>=3.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.0.0)\n",
      "Requirement already satisfied: pytest-asyncio>=0.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (0.21.1)\n",
      "Requirement already satisfied: black>=22.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (23.11.0)\n",
      "Requirement already satisfied: isort>=5.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (5.12.0)\n",
      "Requirement already satisfied: flake8>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (7.3.0)\n",
      "Requirement already satisfied: mypy>=0.971 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (1.17.1)\n",
      "Requirement already satisfied: pre-commit>=2.19.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from semantica) (4.4.0)\n",
      "Requirement already satisfied: azure-core>=1.30.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (1.35.0)\n",
      "Requirement already satisfied: cryptography>=2.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (43.0.3)\n",
      "Requirement already satisfied: typing-extensions>=4.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (4.15.0)\n",
      "Requirement already satisfied: isodate>=0.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-storage-blob>=12.12.0->semantica) (0.7.2)\n",
      "Requirement already satisfied: six>=1.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from azure-core>=1.30.0->azure-storage-blob>=12.12.0->semantica) (1.16.0)\n",
      "Requirement already satisfied: soupsieve>=1.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from beautifulsoup4>=4.11.0->semantica) (2.5)\n",
      "Requirement already satisfied: mypy-extensions>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (1.0.0)\n",
      "Requirement already satisfied: packaging>=22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (24.2)\n",
      "Requirement already satisfied: pathspec>=0.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (0.11.2)\n",
      "Requirement already satisfied: platformdirs>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from black>=22.6.0->semantica) (4.5.1)\n",
      "Requirement already satisfied: botocore<1.37.0,>=1.36.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.36.26)\n",
      "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (1.0.1)\n",
      "Requirement already satisfied: s3transfer<0.12.0,>=0.11.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from boto3>=1.24.0->semantica) (0.11.3)\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (2.9.0.post0)\n",
      "Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from botocore<1.37.0,>=1.36.0->boto3>=1.24.0->semantica) (1.26.20)\n",
      "Requirement already satisfied: billiard<5.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (4.2.1)\n",
      "Requirement already satisfied: kombu<6.0,>=5.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.5.3)\n",
      "Requirement already satisfied: vine<6.0,>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (5.1.0)\n",
      "Requirement already satisfied: click-didyoumean>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.1)\n",
      "Requirement already satisfied: click-repl>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (0.3.0)\n",
      "Requirement already satisfied: click-plugins>=1.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (1.1.1)\n",
      "Requirement already satisfied: tzdata>=2022.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from celery>=5.2.0->semantica) (2025.2)\n",
      "Requirement already satisfied: colorama in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from click>=8.1.0->semantica) (0.4.6)\n",
      "Requirement already satisfied: amqp<6.0.0,>=5.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from kombu<6.0,>=5.3.2->celery>=5.2.0->semantica) (5.3.1)\n",
      "Requirement already satisfied: prompt-toolkit>=3.0.36 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from click-repl>=0.2.0->celery>=5.2.0->semantica) (3.0.40)\n",
      "Requirement already satisfied: cffi>=1.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (1.17.1)\n",
      "Requirement already satisfied: pycparser in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.12.0->semantica) (2.22)\n",
      "Requirement already satisfied: starlette<0.50.0,>=0.40.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.46.2)\n",
      "Requirement already satisfied: annotated-doc>=0.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastapi>=0.78.0->semantica) (0.0.3)\n",
      "Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.41.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (2.41.4)\n",
      "Requirement already satisfied: typing-inspection>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic>=2.0.0->semantica) (0.4.2)\n",
      "Requirement already satisfied: anyio<5,>=3.6.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (4.11.0)\n",
      "Requirement already satisfied: idna>=2.8 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (3.10)\n",
      "Requirement already satisfied: sniffio>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from anyio<5,>=3.6.2->starlette<0.50.0,>=0.40.0->fastapi>=0.78.0->semantica) (1.3.1)\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.20 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (0.30.2)\n",
      "Requirement already satisfied: mmh3<6.0.0,>=4.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (5.2.0)\n",
      "Requirement already satisfied: py-rust-stemmers<0.2.0,>=0.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastembed>=0.2.0->semantica) (0.1.5)\n",
      "Requirement already satisfied: filelock in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub<1.0,>=0.20->fastembed>=0.2.0->semantica) (3.16.1)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from huggingface-hub<1.0,>=0.20->fastembed>=0.2.0->semantica) (2023.10.0)\n",
      "Requirement already satisfied: win32-setctime>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from loguru>=0.6.0->semantica) (1.1.0)\n",
      "Requirement already satisfied: charset_normalizer<4,>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (3.4.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from requests>=2.28.0->semantica) (2025.8.3)\n",
      "Requirement already satisfied: authlib>=1.6.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.6.6)\n",
      "Requirement already satisfied: cyclopts>=4.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (4.4.0)\n",
      "Requirement already satisfied: exceptiongroup>=1.2.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.3.0)\n",
      "Requirement already satisfied: httpx>=0.28.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.28.1)\n",
      "Requirement already satisfied: jsonschema-path>=0.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.3.4)\n",
      "Requirement already satisfied: mcp>=1.24.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.25.0)\n",
      "Requirement already satisfied: openapi-pydantic>=0.5.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.5.1)\n",
      "Requirement already satisfied: py-key-value-aio<0.4.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.3.0)\n",
      "Requirement already satisfied: pydocket>=0.15.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (0.16.0)\n",
      "Requirement already satisfied: pyperclip>=1.9.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (1.9.0)\n",
      "Requirement already satisfied: websockets>=15.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fastmcp>=0.1.0->semantica) (15.0.1)\n",
      "Requirement already satisfied: py-key-value-shared==0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio<0.4.0,>=0.3.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.3.0)\n",
      "Requirement already satisfied: beartype>=0.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio<0.4.0,>=0.3.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.22.9)\n",
      "Requirement already satisfied: diskcache>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (5.6.3)\n",
      "Requirement already satisfied: pathvalidate>=3.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.3.1)\n",
      "Requirement already satisfied: keyring>=25.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (25.6.0)\n",
      "Requirement already satisfied: cachetools>=5.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (5.5.2)\n",
      "Requirement already satisfied: attrs>=23.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (24.2.0)\n",
      "Requirement already satisfied: docstring-parser<4.0,>=0.15 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (0.16)\n",
      "Requirement already satisfied: rich-rst<2.0.0,>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (1.3.2)\n",
      "Requirement already satisfied: docutils in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich-rst<2.0.0,>=1.3.1->cyclopts>=4.0.0->fastmcp>=0.1.0->semantica) (0.21.2)\n",
      "Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (0.7.0)\n",
      "Requirement already satisfied: pycodestyle<2.15.0,>=2.14.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (2.14.0)\n",
      "Requirement already satisfied: pyflakes<3.5.0,>=3.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from flake8>=4.0.0->semantica) (3.4.0)\n",
      "Requirement already satisfied: google-auth<3.0dev,>=2.26.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.36.0)\n",
      "Requirement already satisfied: google-api-core<3.0.0dev,>=2.15.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.23.0)\n",
      "Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.4.1)\n",
      "Requirement already satisfied: google-resumable-media>=2.7.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (2.7.2)\n",
      "Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-cloud-storage>=2.5.0->semantica) (1.6.0)\n",
      "Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.66.0)\n",
      "Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0.dev0,>=3.19.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (4.25.8)\n",
      "Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-api-core<3.0.0dev,>=2.15.0->google-cloud-storage>=2.5.0->semantica) (1.25.0)\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.4.1)\n",
      "Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (4.9)\n",
      "Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3.0dev,>=2.26.1->google-cloud-storage>=2.5.0->semantica) (0.6.1)\n",
      "Requirement already satisfied: httpcore==1.* in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx>=0.28.1->fastmcp>=0.1.0->semantica) (1.0.9)\n",
      "Requirement already satisfied: h11>=0.16 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpcore==1.*->httpx>=0.28.1->fastmcp>=0.1.0->semantica) (0.16.0)\n",
      "Requirement already satisfied: comm>=0.1.3 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (0.2.0)\n",
      "Requirement already satisfied: ipython>=6.1.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (8.17.2)\n",
      "Requirement already satisfied: traitlets>=4.3.1 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipywidgets>=8.0.0->semantica) (5.13.0)\n",
      "Requirement already satisfied: widgetsnbextension~=4.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipywidgets>=8.0.0->semantica) (4.0.9)\n",
      "Requirement already satisfied: jupyterlab-widgets~=3.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipywidgets>=8.0.0->semantica) (3.0.9)\n",
      "Requirement already satisfied: decorator in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (5.1.1)\n",
      "Requirement already satisfied: jedi>=0.16 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.19.1)\n",
      "Requirement already satisfied: matplotlib-inline in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.1.6)\n",
      "Requirement already satisfied: pygments>=2.4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.19.2)\n",
      "Requirement already satisfied: stack-data in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.6.3)\n",
      "Requirement already satisfied: wcwidth in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery>=5.2.0->semantica) (0.2.9)\n",
      "Requirement already satisfied: parso<0.9.0,>=0.8.3 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.8.3)\n",
      "Requirement already satisfied: pathable<0.5.0,>=0.4.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.4.4)\n",
      "Requirement already satisfied: referencing<0.37.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.30.2)\n",
      "Requirement already satisfied: rpds-py>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from referencing<0.37.0->jsonschema-path>=0.3.4->fastmcp>=0.1.0->semantica) (0.12.0)\n",
      "Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (0.2.3)\n",
      "Requirement already satisfied: importlib_metadata>=4.11.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (6.8.0)\n",
      "Requirement already satisfied: jaraco.classes in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.4.0)\n",
      "Requirement already satisfied: jaraco.functools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (4.1.0)\n",
      "Requirement already satisfied: jaraco.context in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (6.0.1)\n",
      "Requirement already satisfied: zipp>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from importlib_metadata>=4.11.4->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (3.17.0)\n",
      "Requirement already satisfied: audioread>=2.1.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (3.0.1)\n",
      "Requirement already satisfied: numba>=0.51.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.61.2)\n",
      "Requirement already satisfied: scipy>=1.6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.15.2)\n",
      "Requirement already satisfied: joblib>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.3.2)\n",
      "Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.13.1)\n",
      "Requirement already satisfied: pooch>=1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.8.2)\n",
      "Requirement already satisfied: soxr>=0.3.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.5.0.post1)\n",
      "Requirement already satisfied: lazy_loader>=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (0.4)\n",
      "Requirement already satisfied: msgpack>=1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from librosa>=0.9.0->semantica) (1.1.0)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.3.2)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (0.12.1)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (4.57.0)\n",
      "Requirement already satisfied: kiwisolver>=1.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (1.4.8)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from matplotlib>=3.5.0->semantica) (3.2.0)\n",
      "Requirement already satisfied: httpx-sse>=0.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (0.4.0)\n",
      "Requirement already satisfied: jsonschema>=4.20.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (4.23.0)\n",
      "Requirement already satisfied: pydantic-settings>=2.5.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.11.0)\n",
      "Requirement already satisfied: pyjwt>=2.10.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pyjwt[crypto]>=2.10.1->mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.10.1)\n",
      "Requirement already satisfied: python-multipart>=0.0.9 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (0.0.18)\n",
      "Requirement already satisfied: pywin32>=310 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (311)\n",
      "Requirement already satisfied: sse-starlette>=1.6.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2.3.6)\n",
      "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jsonschema>=4.20.0->mcp>=1.24.0->fastmcp>=0.1.0->semantica) (2023.7.1)\n",
      "Requirement already satisfied: pytz in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from neo4j>=5.0.0->semantica) (2024.2)\n",
      "Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from numba>=0.51.0->librosa>=0.9.0->semantica) (0.44.0)\n",
      "Requirement already satisfied: coloredlogs in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (15.0.1)\n",
      "Requirement already satisfied: flatbuffers in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (23.5.26)\n",
      "Requirement already satisfied: sympy in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from onnxruntime>=1.17.0->semantica) (1.13.3)\n",
      "Requirement already satisfied: et-xmlfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from openpyxl>=3.0.10->semantica) (2.0.0)\n",
      "Requirement already satisfied: opentelemetry-semantic-conventions==0.60b1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-sdk>=1.12.0->semantica) (0.60b1)\n",
      "Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from plotly>=5.10.0->semantica) (8.5.0)\n",
      "Requirement already satisfied: cfgv>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (3.5.0)\n",
      "Requirement already satisfied: identify>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (2.6.15)\n",
      "Requirement already satisfied: nodeenv>=0.11.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (1.9.1)\n",
      "Requirement already satisfied: virtualenv>=20.10.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pre-commit>=2.19.0->semantica) (20.35.4)\n",
      "Requirement already satisfied: email-validator>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydantic[email]>=2.11.7->fastmcp>=0.1.0->semantica) (2.2.0)\n",
      "Requirement already satisfied: dnspython>=2.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from email-validator>=2.0.0->pydantic[email]>=2.11.7->fastmcp>=0.1.0->semantica) (2.4.2)\n",
      "Requirement already satisfied: cloudpickle>=3.1.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (3.1.2)\n",
      "Requirement already satisfied: fakeredis>=2.32.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.33.0)\n",
      "Requirement already satisfied: opentelemetry-exporter-prometheus>=0.60b0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (0.60b1)\n",
      "Requirement already satisfied: python-json-logger>=2.0.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.0.7)\n",
      "Requirement already satisfied: typer>=0.15.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (0.15.2)\n",
      "Requirement already satisfied: sortedcontainers>=2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis>=2.32.1->fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.4.0)\n",
      "Requirement already satisfied: lupa>=2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from fakeredis[lua]>=2.32.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (2.6)\n",
      "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from opentelemetry-instrumentation->semantica) (1.17.2)\n",
      "Requirement already satisfied: iniconfig in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (2.0.0)\n",
      "Requirement already satisfied: pluggy<2.0,>=0.12 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from pytest>=7.1.0->semantica) (1.5.0)\n",
      "Requirement already satisfied: coverage>=7.10.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coverage[toml]>=7.10.6->pytest-cov>=3.0.0->semantica) (7.12.0)\n",
      "Requirement already satisfied: grpcio>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.68.0)\n",
      "Requirement already satisfied: grpcio-tools>=1.41.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (1.62.3)\n",
      "Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from qdrant-client>=1.3.0->semantica) (2.10.1)\n",
      "Requirement already satisfied: setuptools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from grpcio-tools>=1.41.0->qdrant-client>=1.3.0->semantica) (80.9.0)\n",
      "Requirement already satisfied: h2<5,>=3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.1.0)\n",
      "Requirement already satisfied: hyperframe<7,>=6.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (6.0.1)\n",
      "Requirement already satisfied: hpack<5,>=4.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client>=1.3.0->semantica) (4.0.0)\n",
      "Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from rich>=12.5.0->semantica) (3.0.0)\n",
      "Requirement already satisfied: mdurl~=0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from markdown-it-py>=2.2.0->rich>=12.5.0->semantica) (0.1.2)\n",
      "Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from scikit-learn>=1.0.0->semantica) (3.2.0)\n",
      "Requirement already satisfied: regex!=2019.12.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (2024.11.6)\n",
      "Requirement already satisfied: safetensors>=0.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from transformers>=4.20.0->semantica) (0.5.3)\n",
      "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
      "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.5)\n",
      "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.0.15)\n",
      "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.13)\n",
      "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.0.12)\n",
      "Requirement already satisfied: thinc<8.4.0,>=8.3.4 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (8.3.10)\n",
      "Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (1.1.3)\n",
      "Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.5.2)\n",
      "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (2.0.10)\n",
      "Requirement already satisfied: weasel<0.5.0,>=0.4.2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.4.3)\n",
      "Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (0.20.0)\n",
      "Requirement already satisfied: jinja2 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from spacy>=3.4.0->semantica) (3.1.6)\n",
      "Requirement already satisfied: blis<1.4.0,>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (1.3.3)\n",
      "Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.4.0->semantica) (0.1.5)\n",
      "Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (0.23.0)\n",
      "Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.4.0->semantica) (7.1.0)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sqlalchemy>=1.4.0->semantica) (3.2.3)\n",
      "Requirement already satisfied: shellingham>=1.3.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from typer>=0.15.1->pydocket>=0.15.5->fastmcp>=0.1.0->semantica) (1.5.4)\n",
      "Requirement already satisfied: pynndescent>=0.5 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from umap-learn>=0.5.0->semantica) (0.5.13)\n",
      "Requirement already satisfied: distlib<1,>=0.3.7 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from virtualenv>=20.10.0->pre-commit>=2.19.0->semantica) (0.3.7)\n",
      "Requirement already satisfied: validators<1.0.0,>=0.34.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (0.35.0)\n",
      "Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from weaviate-client>=3.15.0->semantica) (2.1.0)\n",
      "Requirement already satisfied: humanfriendly>=9.1 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from coloredlogs->onnxruntime>=1.17.0->semantica) (10.0)\n",
      "Requirement already satisfied: pyreadline3 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from humanfriendly>=9.1->coloredlogs->onnxruntime>=1.17.0->semantica) (3.4.1)\n",
      "Requirement already satisfied: more-itertools in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jaraco.classes->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (10.6.0)\n",
      "Requirement already satisfied: backports.tarfile in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jaraco.context->keyring>=25.6.0->py-key-value-aio[disk,keyring,memory]<0.4.0,>=0.3.0->fastmcp>=0.1.0->semantica) (1.2.0)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from jinja2->spacy>=3.4.0->semantica) (2.1.3)\n",
      "Requirement already satisfied: executing>=1.2.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.0.1)\n",
      "Requirement already satisfied: asttokens>=2.1.0 in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (2.4.1)\n",
      "Requirement already satisfied: pure-eval in c:\\users\\mohd kaif\\appdata\\roaming\\python\\python311\\site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=8.0.0->semantica) (0.2.2)\n",
      "Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\\users\\mohd kaif\\appdata\\local\\programs\\python\\python311\\lib\\site-packages (from sympy->onnxruntime>=1.17.0->semantica) (1.3.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install semantica\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 0: Setup Embeddings\n",
    "\n",
    "First, let's select our embedding provider and model. Semantica supports multiple providers like Sentence Transformers and FastEmbed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "fastembed not available. Install with: pip install fastembed. Using fallback embedding method.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selected model: BAAI/bge-small-en-v1.5\n",
      "Embedding dimension: 128\n"
     ]
    }
   ],
   "source": [
    "from semantica.embeddings import TextEmbedder\n",
    "\n",
    "# Choose provider and model\n",
    "embedder = TextEmbedder(method=\"fastembed\", model_name=\"BAAI/bge-small-en-v1.5\")\n",
    "dimension = embedder.get_embedding_dimension()\n",
    "\n",
    "print(f\"Selected model: {embedder.get_model_info()['model_name']}\")\n",
    "print(f\"Embedding dimension: {dimension}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Choosing the Right Index\n",
    "\n",
    "Think of an index like choosing a filing system:\n",
    "- **Flat**: Like a small notebook - slow but perfect\n",
    "- **HNSW**: Like a well-organized library - fast and accurate\n",
    "- **IVF**: Like a warehouse with sections - very fast for huge collections\n",
    "\n",
    "### Simple Rule\n",
    "- Less than 10,000 items? Use **Flat**\n",
    "- Between 10,000 and 1 million? Use **HNSW** ✅ (recommended)\n",
    "- More than 1 million? Use **IVF**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedWriter name=3>\n",
      "  del self._target, self._args, self._kwargs\n",
      "ResourceWarning: Enable tracemalloc to get the object allocation traceback\n",
      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedReader name=4>\n",
      "  del self._target, self._args, self._kwargs\n",
      "ResourceWarning: Enable tracemalloc to get the object allocation traceback\n",
      "c:\\Users\\Mohd Kaif\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\threading.py:986: ResourceWarning: unclosed file <_io.BufferedReader name=5>\n",
      "  del self._target, self._args, self._kwargs\n",
      "ResourceWarning: Enable tracemalloc to get the object allocation traceback\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 5 most similar documents:\n",
      "  1. Document doc_1935 (distance: 110.917)\n",
      "  2. Document doc_3860 (distance: 111.535)\n",
      "  3. Document doc_277 (distance: 113.270)\n",
      "  4. Document doc_1903 (distance: 113.371)\n",
      "  5. Document doc_2959 (distance: 113.612)\n"
     ]
    }
   ],
   "source": [
    "from semantica.vector_store import FAISSStore\n",
    "import numpy as np\n",
    "\n",
    "# Create some example vectors (like document embeddings)\n",
    "vectors = np.random.rand(5000, 768).astype('float32')\n",
    "query = np.random.rand(768).astype('float32')\n",
    "\n",
    "adapter = FAISSStore(dimension=768)\n",
    "\n",
    "# HNSW Index - Best for most cases\n",
    "index = adapter.create_index(index_type=\"hnsw\", metric=\"L2\", m=16)\n",
    "adapter.add_vectors(vectors, ids=[f\"doc_{i}\" for i in range(len(vectors))])\n",
    "\n",
    "# Search for similar vectors\n",
    "results = adapter.search_similar(query, k=5)\n",
    "\n",
    "print(\"Found 5 most similar documents:\")\n",
    "for i, result in enumerate(results, 1):\n",
    "    print(f\"  {i}. Document {result['id']} (distance: {result['distance']:.3f})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Smart Filtering with Metadata\n",
    "\n",
    "Imagine searching for \"similar articles\" but only from 2024 and only in the \"Technology\" category. That's what metadata filtering does!\n",
    "\n",
    "### Real-World Example\n",
    "You're building a document search where users want:\n",
    "- Similar documents (vector search)\n",
    "- From specific categories (metadata filter)\n",
    "- From recent years (metadata filter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Technology articles from 2024:\n",
      "  - AI in Healthcare\n",
      "  - Business Strategy\n",
      "  - Data Science Guide\n",
      "  - Marketing Tips\n",
      "  - Machine Learning Basics\n",
      "\n",
      "All articles from 2024:\n",
      "  - AI in Healthcare (Technology)\n",
      "  - Business Strategy (Business)\n",
      "  - Data Science Guide (Technology)\n",
      "  - Marketing Tips (Business)\n",
      "  - Machine Learning Basics (Technology)\n"
     ]
    }
   ],
   "source": [
    "from semantica.vector_store import HybridSearch, MetadataFilter\n",
    "import numpy as np\n",
    "\n",
    "# Create sample documents with metadata\n",
    "documents = [\n",
    "    {\"id\": 0, \"text\": \"AI in Healthcare\", \"category\": \"Technology\", \"year\": 2024},\n",
    "    {\"id\": 1, \"text\": \"Machine Learning Basics\", \"category\": \"Technology\", \"year\": 2023},\n",
    "    {\"id\": 2, \"text\": \"Business Strategy\", \"category\": \"Business\", \"year\": 2024},\n",
    "    {\"id\": 3, \"text\": \"Data Science Guide\", \"category\": \"Technology\", \"year\": 2024},\n",
    "    {\"id\": 4, \"text\": \"Marketing Tips\", \"category\": \"Business\", \"year\": 2023},\n",
    "]\n",
    "\n",
    "# Create vectors for each document\n",
    "vectors = [np.random.rand(768) for _ in documents]\n",
    "metadata = [{\"category\": d[\"category\"], \"year\": d[\"year\"]} for d in documents]\n",
    "vector_ids = [f\"doc_{d['id']}\" for d in documents]\n",
    "\n",
    "# Create search\n",
    "search = HybridSearch()\n",
    "query = np.random.rand(768)\n",
    "\n",
    "# Example 1: Find Technology articles from 2024\n",
    "filter1 = MetadataFilter().eq(\"category\", \"Technology\").eq(\"year\", 2024)\n",
    "results = search.search(query, vectors, metadata, vector_ids, filter=filter1, k=10)\n",
    "\n",
    "print(\"Technology articles from 2024:\")\n",
    "for r in results:\n",
    "    doc_id = int(r['id'].split('_')[1])\n",
    "    print(f\"  - {documents[doc_id]['text']}\")\n",
    "\n",
    "# Example 2: Find any article from 2024\n",
    "filter2 = MetadataFilter().eq(\"year\", 2024)\n",
    "results2 = search.search(query, vectors, metadata, vector_ids, filter=filter2, k=10)\n",
    "\n",
    "print(\"\\nAll articles from 2024:\")\n",
    "for r in results2:\n",
    "    doc_id = int(r['id'].split('_')[1])\n",
    "    print(f\"  - {documents[doc_id]['text']} ({documents[doc_id]['category']})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: Combining Search Results\n",
    "\n",
    "Sometimes you want to search in multiple places and combine the results. Like searching both your email and documents, then showing the best matches from both.\n",
    "\n",
    "### When to Use This\n",
    "- Searching multiple databases\n",
    "- Combining different search strategies\n",
    "- Giving more weight to certain sources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Combined results (fair ranking):\n",
      "  1. Data Science Guide (score: 0.033)\n",
      "  2. Machine Learning Basics (score: 0.016)\n",
      "  3. AI in Healthcare (score: 0.016)\n",
      "\n",
      "Combined results (prefer recent):\n",
      "  1. Data Science Guide (score: 0.929)\n",
      "  2. AI in Healthcare (score: 0.630)\n",
      "  3. Business Strategy (score: 0.595)\n"
     ]
    }
   ],
   "source": [
    "from semantica.vector_store import SearchRanker\n",
    "\n",
    "# Simulate two different searches\n",
    "# Search 1: Recent documents\n",
    "recent_results = [\n",
    "    {\"id\": \"doc_3\", \"score\": 0.95, \"source\": \"recent\"},\n",
    "    {\"id\": \"doc_0\", \"score\": 0.90, \"source\": \"recent\"},\n",
    "    {\"id\": \"doc_2\", \"score\": 0.85, \"source\": \"recent\"},\n",
    "]\n",
    "\n",
    "# Search 2: Popular documents\n",
    "popular_results = [\n",
    "    {\"id\": \"doc_1\", \"score\": 0.92, \"source\": \"popular\"},\n",
    "    {\"id\": \"doc_3\", \"score\": 0.88, \"source\": \"popular\"},\n",
    "    {\"id\": \"doc_4\", \"score\": 0.80, \"source\": \"popular\"},\n",
    "]\n",
    "\n",
    "# Method 1: Fair combination (RRF)\n",
    "ranker = SearchRanker(strategy=\"reciprocal_rank_fusion\")\n",
    "combined = ranker.rank([recent_results, popular_results])\n",
    "\n",
    "print(\"Combined results (fair ranking):\")\n",
    "for i, result in enumerate(combined[:3], 1):\n",
    "    doc_id = int(result['id'].split('_')[1])\n",
    "    print(f\"  {i}. {documents[doc_id]['text']} (score: {result['score']:.3f})\")\n",
    "\n",
    "# Method 2: Prefer recent documents (70% recent, 30% popular)\n",
    "weighted_ranker = SearchRanker(strategy=\"weighted_average\")\n",
    "weighted_combined = weighted_ranker.rank(\n",
    "    [recent_results, popular_results],\n",
    "    weights=[0.7, 0.3]\n",
    ")\n",
    "\n",
    "print(\"\\nCombined results (prefer recent):\")\n",
    "for i, result in enumerate(weighted_combined[:3], 1):\n",
    "    doc_id = int(result['id'].split('_')[1])\n",
    "    print(f\"  {i}. {documents[doc_id]['text']} (score: {result['score']:.3f})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: Keeping User Data Separate\n",
    "\n",
    "If you're building an app with multiple users or companies, you need to keep their data separate. Namespaces do this automatically.\n",
    "\n",
    "### Real Example\n",
    "You're building a SaaS app where:\n",
    "- Company A has their documents\n",
    "- Company B has their documents\n",
    "- They should never see each other's data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Company A has 10 documents\n",
      "Company B has 15 documents\n",
      "\n",
      "Admin can delete: True\n",
      "User can delete: False\n"
     ]
    }
   ],
   "source": [
    "from semantica.vector_store import NamespaceManager\n",
    "\n",
    "# Create manager\n",
    "manager = NamespaceManager()\n",
    "\n",
    "# Create separate spaces for each company\n",
    "company_a = manager.create_namespace(\"company_a\", \"Company A's documents\")\n",
    "company_b = manager.create_namespace(\"company_b\", \"Company B's documents\")\n",
    "\n",
    "# Add documents to Company A\n",
    "for i in range(10):\n",
    "    manager.add_vector_to_namespace(f\"company_a_doc_{i}\", \"company_a\")\n",
    "\n",
    "# Add documents to Company B\n",
    "for i in range(15):\n",
    "    manager.add_vector_to_namespace(f\"company_b_doc_{i}\", \"company_b\")\n",
    "\n",
    "# Get each company's documents\n",
    "a_docs = manager.get_namespace_vectors(\"company_a\")\n",
    "b_docs = manager.get_namespace_vectors(\"company_b\")\n",
    "\n",
    "print(f\"Company A has {len(a_docs)} documents\")\n",
    "print(f\"Company B has {len(b_docs)} documents\")\n",
    "\n",
    "# Set permissions (who can access what)\n",
    "company_a.set_access_control(\"admin@companya.com\", [\"read\", \"write\", \"delete\"])\n",
    "company_a.set_access_control(\"user@companya.com\", [\"read\"])  # Read-only\n",
    "\n",
    "# Check permissions\n",
    "print(f\"\\nAdmin can delete: {company_a.has_permission('admin@companya.com', 'delete')}\")\n",
    "print(f\"User can delete: {company_a.has_permission('user@companya.com', 'delete')}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quick Reference Guide\n",
    "\n",
    "### Which Index Should I Use?\n",
    "\n",
    "```python\n",
    "# Small dataset (< 10,000 items)\n",
    "index = adapter.create_index(index_type=\"flat\", metric=\"L2\")\n",
    "\n",
    "# Medium dataset (10,000 - 1,000,000 items) ✅ RECOMMENDED\n",
    "index = adapter.create_index(index_type=\"hnsw\", metric=\"L2\", m=16)\n",
    "\n",
    "# Large dataset (> 1,000,000 items)\n",
    "index = adapter.create_index(index_type=\"ivf\", metric=\"L2\", nlist=100)\n",
    "```\n",
    "\n",
    "### How Do I Filter Results?\n",
    "\n",
    "```python\n",
    "# Single condition\n",
    "filter = MetadataFilter().eq(\"category\", \"Technology\")\n",
    "\n",
    "# Multiple conditions (AND)\n",
    "filter = MetadataFilter() \\\n",
    "    .eq(\"category\", \"Technology\") \\\n",
    "    .eq(\"year\", 2024)\n",
    "\n",
    "# Greater than / Less than\n",
    "filter = MetadataFilter().gt(\"year\", 2020)\n",
    "```\n",
    "\n",
    "### How Do I Combine Results?\n",
    "\n",
    "```python\n",
    "# Fair combination\n",
    "ranker = SearchRanker(strategy=\"reciprocal_rank_fusion\")\n",
    "combined = ranker.rank([results1, results2])\n",
    "\n",
    "# Weighted combination (prefer first source)\n",
    "ranker = SearchRanker(strategy=\"weighted_average\")\n",
    "combined = ranker.rank([results1, results2], weights=[0.7, 0.3])\n",
    "```\n",
    "\n",
    "### How Do I Separate User Data?\n",
    "\n",
    "```python\n",
    "# Create namespace for each user/company\n",
    "manager = NamespaceManager()\n",
    "user_space = manager.create_namespace(\"user_123\", \"User 123's data\")\n",
    "\n",
    "# Add data to namespace\n",
    "manager.add_vector_to_namespace(\"doc_1\", \"user_123\")\n",
    "\n",
    "# Get user's data\n",
    "user_docs = manager.get_namespace_vectors(\"user_123\")\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Summary\n",
    "\n",
    "You've learned:\n",
    "\n",
    "1. ✅ **Index Selection**: Use HNSW for most cases\n",
    "2. ✅ **Smart Filtering**: Combine vector search with metadata\n",
    "3. ✅ **Result Fusion**: Merge searches from different sources\n",
    "4. ✅ **Data Isolation**: Keep users' data separate\n",
    "\n",
    "### Next Steps\n",
    "\n",
    "- Try these examples with your own data\n",
    "- Experiment with different filters\n",
    "- Build a multi-user application\n",
    "- Explore the [introduction notebook](../introduction/13_Vector_Store.ipynb) for more basics\n",
    "\n",
    "**Need Help?** Check our [documentation](https://semantica.readthedocs.io) or ask on [GitHub](https://github.com/Hawksight-AI/semantica)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
