semantica/docs/reference/llms.md

# LLM Providers Module

The `semantica.llms` module provides a unified interface for LLM providers, supporting Groq, OpenAI, HuggingFace, and LiteLLM (100+ LLMs) with clean imports and consistent API.

## Overview

The LLM Providers module abstracts away provider-specific details, providing a consistent interface for text generation across multiple LLM providers. This enables easy switching between providers and integration with GraphRAG reasoning features.

## Quick Start

```python
from semantica.llms import Groq, OpenAI, HuggingFaceLLM, LiteLLM
import os

# Groq - Fast inference
groq = Groq(model="llama-3.1-8b-instant", api_key=os.getenv("GROQ_API_KEY"))
response = groq.generate("What is AI?")

# OpenAI
openai = OpenAI(model="gpt-4", api_key=os.getenv("OPENAI_API_KEY"))
response = openai.generate("What is AI?")

# HuggingFace - Local models
hf = HuggingFaceLLM(model_name="gpt2")  # or model="gpt2"
response = hf.generate("What is AI?")

# LiteLLM - Unified interface to 100+ LLMs
litellm = LiteLLM(model="openai/gpt-4o", api_key=os.getenv("OPENAI_API_KEY"))
response = litellm.generate("What is AI?")
```

## Providers

### Groq

Fast inference provider using Groq's API.

```python
from semantica.llms import Groq

groq = Groq(
    model="llama-3.1-8b-instant",
    api_key="your-api-key"  # or use GROQ_API_KEY env var
)

response = groq.generate("Hello, world!")
structured = groq.generate_structured("Extract entities from: Apple Inc.")
```

**Parameters:**
- `model` (str): Model name (default: "llama-3.1-8b-instant")
- `api_key` (str, optional): Groq API key (default: from GROQ_API_KEY env var)
- `**kwargs`: Additional provider options

**Methods:**
- `generate(prompt: str, **kwargs) -> str`: Generate text from prompt
- `generate_structured(prompt: str, **kwargs) -> Dict[str, Any]`: Generate structured JSON output
- `is_available() -> bool`: Check if provider is available

### OpenAI

OpenAI API provider for GPT models.

```python
from semantica.llms import OpenAI

openai = OpenAI(
    model="gpt-4",
    api_key="your-api-key"  # or use OPENAI_API_KEY env var
)

response = openai.generate("Hello, world!")
```

**Parameters:**
- `model` (str): Model name (default: "gpt-3.5-turbo")
- `api_key` (str, optional): OpenAI API key (default: from OPENAI_API_KEY env var)
- `**kwargs`: Additional provider options

**Methods:**
- `generate(prompt: str, **kwargs) -> str`: Generate text from prompt
- `generate_structured(prompt: str, **kwargs) -> Dict[str, Any]`: Generate structured JSON output
- `is_available() -> bool`: Check if provider is available

### HuggingFaceLLM

Local LLM inference using HuggingFace Transformers.

```python
from semantica.llms import HuggingFaceLLM

hf = HuggingFaceLLM(
    model_name="gpt2",
    device="cuda"  # or "cpu", default: auto-detect
)

response = hf.generate("Hello, world!")
```

**Parameters:**
- `model_name` (str, optional): HuggingFace model name (default: "gpt2")
- `model` (str, optional): Alias for model_name (for consistency with other providers)
- `device` (str, optional): Device to use ("cuda" or "cpu", default: auto-detect)
- `**kwargs`: Additional provider options

**Note:** Both `model` and `model_name` are supported for consistency with other providers.

**Methods:**
- `generate(prompt: str, **kwargs) -> str`: Generate text from prompt
- `generate_structured(prompt: str, **kwargs) -> Dict[str, Any]`: Generate structured JSON output
- `is_available() -> bool`: Check if provider is available

### LiteLLM

Unified interface to 100+ LLM providers via LiteLLM library.

```python
from semantica.llms import LiteLLM

# Use any provider via LiteLLM
litellm = LiteLLM(
    model="openai/gpt-4o",  # Provider/model format
    api_key=os.getenv("OPENAI_API_KEY")
)

# Or use other providers
litellm = LiteLLM(model="anthropic/claude-sonnet-4-20250514")
litellm = LiteLLM(model="groq/llama-3.1-8b-instant")
litellm = LiteLLM(model="azure/gpt-4")

response = litellm.generate("Hello, world!")
```

**Parameters:**
- `model` (str): Model identifier in format "provider/model-name"
  - Examples: "openai/gpt-4o", "anthropic/claude-sonnet-4-20250514", "groq/llama-3.1-8b-instant", "azure/gpt-4"
- `api_key` (str, optional): API key (can use environment variables)
- `**kwargs`: Additional LiteLLM options (temperature, max_tokens, etc.)

**Methods:**
- `generate(prompt: str, **kwargs) -> str`: Generate text from prompt
- `generate_structured(prompt: str, **kwargs) -> Dict[str, Any]`: Generate structured JSON output
- `is_available() -> bool`: Check if provider is available

**Supported Providers:**
- OpenAI, Anthropic, Groq, Azure, Bedrock, Vertex AI, Cohere, Mistral, and 90+ more
- See [LiteLLM Documentation](https://docs.litellm.ai/) for full list

## Integration with GraphRAG

The LLM providers integrate seamlessly with GraphRAG reasoning:

```python
from semantica.context import AgentContext
from semantica.llms import Groq
from semantica.vector_store import VectorStore
import os

context = AgentContext(
    vector_store=VectorStore(backend="faiss"),
    knowledge_graph=kg
)

llm_provider = Groq(
    model="llama-3.1-8b-instant",
    api_key=os.getenv("GROQ_API_KEY")
)

result = context.query_with_reasoning(
    query="What IPs are associated with security alerts?",
    llm_provider=llm_provider,
    max_hops=2
)

print(f"Response: {result['response']}")
print(f"Reasoning Path: {result['reasoning_path']}")
```

## Common Parameters

All providers support common generation parameters:

- `temperature` (float): Sampling temperature (0.0-2.0)
- `max_tokens` (int): Maximum tokens to generate
- `top_p` (float): Nucleus sampling parameter
- `frequency_penalty` (float): Frequency penalty
- `presence_penalty` (float): Presence penalty

Example:

```python
response = groq.generate(
    "What is AI?",
    temperature=0.7,
    max_tokens=500,
    top_p=0.9
)
```

## Error Handling

All providers gracefully handle errors:

```python
try:
    response = groq.generate("Hello")
except ProcessingError as e:
    print(f"Generation failed: {e}")
```

If a provider is not available (library not installed, API key missing), a `ProcessingError` is raised with a helpful message.

## Examples

### Basic Text Generation

```python
from semantica.llms import Groq

groq = Groq(model="llama-3.1-8b-instant")
response = groq.generate("Explain quantum computing in simple terms.")
print(response)
```

### Structured Output

```python
from semantica.llms import OpenAI

openai = OpenAI(model="gpt-4")
result = openai.generate_structured(
    "Extract entities from: Apple Inc. was founded by Steve Jobs in 1976."
)
# Returns: {"entities": [{"name": "Apple Inc.", "type": "Organization"}, ...]}
```

### Using LiteLLM for Multiple Providers

```python
from semantica.llms import LiteLLM

# Switch between providers easily
providers = [
    LiteLLM(model="openai/gpt-4o"),
    LiteLLM(model="anthropic/claude-sonnet-4-20250514"),
    LiteLLM(model="groq/llama-3.1-8b-instant")
]

for provider in providers:
    response = provider.generate("What is AI?")
    print(f"{provider.model}: {response[:50]}...")
```

## Installation

Most providers require additional dependencies:

```bash
# Groq
pip install groq

# OpenAI
pip install openai

# HuggingFace
pip install transformers torch

# LiteLLM (supports 100+ providers)
pip install litellm
```

## See Also

- [Context Module](context.md) - GraphRAG with multi-hop reasoning
- [Semantic Extract Module](semantic_extract.md) - Entity and relationship extraction
- [GraphRAG Cookbook](../../cookbook/use_cases/advanced_rag/01_GraphRAG_Complete.ipynb) - Complete GraphRAG example