semantica/docs/reference/llms.md

LLM Providers Module

The semantica.llms module provides a unified interface for LLM providers, supporting Groq, OpenAI, HuggingFace, and LiteLLM (100+ LLMs) with clean imports and consistent API.

Overview

The LLM Providers module abstracts away provider-specific details, providing a consistent interface for text generation across multiple LLM providers. This enables easy switching between providers and integration with GraphRAG reasoning features.

Quick Start

from semantica.llms import Groq, OpenAI, HuggingFaceLLM, LiteLLM
import os

# Groq - Fast inference
groq = Groq(model="llama-3.1-8b-instant", api_key=os.getenv("GROQ_API_KEY"))
response = groq.generate("What is AI?")

# OpenAI
openai = OpenAI(model="gpt-4", api_key=os.getenv("OPENAI_API_KEY"))
response = openai.generate("What is AI?")

# HuggingFace - Local models
hf = HuggingFaceLLM(model_name="gpt2")  # or model="gpt2"
response = hf.generate("What is AI?")

# LiteLLM - Unified interface to 100+ LLMs
litellm = LiteLLM(model="openai/gpt-4o", api_key=os.getenv("OPENAI_API_KEY"))
response = litellm.generate("What is AI?")

Providers

Groq

Fast inference provider using Groq's API.

from semantica.llms import Groq

groq = Groq(
    model="llama-3.1-8b-instant",
    api_key="your-api-key"  # or use GROQ_API_KEY env var
)

response = groq.generate("Hello, world!")
structured = groq.generate_structured("Extract entities from: Apple Inc.")

Parameters:

• model (str): Model name (default: "llama-3.1-8b-instant")
• api_key (str, optional): Groq API key (default: from GROQ_API_KEY env var)
• **kwargs: Additional provider options

Methods:

• generate(prompt: str, **kwargs) -> str: Generate text from prompt
• generate_structured(prompt: str, **kwargs) -> Dict[str, Any]: Generate structured JSON output
• is_available() -> bool: Check if provider is available

OpenAI

OpenAI API provider for GPT models.

from semantica.llms import OpenAI

openai = OpenAI(
    model="gpt-4",
    api_key="your-api-key"  # or use OPENAI_API_KEY env var
)

response = openai.generate("Hello, world!")

Parameters:

• model (str): Model name (default: "gpt-3.5-turbo")
• api_key (str, optional): OpenAI API key (default: from OPENAI_API_KEY env var)
• **kwargs: Additional provider options

Methods:

• generate(prompt: str, **kwargs) -> str: Generate text from prompt
• generate_structured(prompt: str, **kwargs) -> Dict[str, Any]: Generate structured JSON output
• is_available() -> bool: Check if provider is available

HuggingFaceLLM

Local LLM inference using HuggingFace Transformers.

from semantica.llms import HuggingFaceLLM

hf = HuggingFaceLLM(
    model_name="gpt2",
    device="cuda"  # or "cpu", default: auto-detect
)

response = hf.generate("Hello, world!")

Parameters:

• model_name (str, optional): HuggingFace model name (default: "gpt2")
• model (str, optional): Alias for model_name (for consistency with other providers)
• device (str, optional): Device to use ("cuda" or "cpu", default: auto-detect)
• **kwargs: Additional provider options

Note: Both model and model_name are supported for consistency with other providers.

Methods:

• generate(prompt: str, **kwargs) -> str: Generate text from prompt
• generate_structured(prompt: str, **kwargs) -> Dict[str, Any]: Generate structured JSON output
• is_available() -> bool: Check if provider is available

LiteLLM

Unified interface to 100+ LLM providers via LiteLLM library.

from semantica.llms import LiteLLM

# Use any provider via LiteLLM
litellm = LiteLLM(
    model="openai/gpt-4o",  # Provider/model format
    api_key=os.getenv("OPENAI_API_KEY")
)

# Or use other providers
litellm = LiteLLM(model="anthropic/claude-sonnet-4-20250514")
litellm = LiteLLM(model="groq/llama-3.1-8b-instant")
litellm = LiteLLM(model="azure/gpt-4")

response = litellm.generate("Hello, world!")

Parameters:

• model (str): Model identifier in format "provider/model-name"
  • Examples: "openai/gpt-4o", "anthropic/claude-sonnet-4-20250514", "groq/llama-3.1-8b-instant", "azure/gpt-4"
• api_key (str, optional): API key (can use environment variables)
• **kwargs: Additional LiteLLM options (temperature, max_tokens, etc.)

Methods:

• generate(prompt: str, **kwargs) -> str: Generate text from prompt
• generate_structured(prompt: str, **kwargs) -> Dict[str, Any]: Generate structured JSON output
• is_available() -> bool: Check if provider is available

Supported Providers:

• OpenAI, Anthropic, Groq, Azure, Bedrock, Vertex AI, Cohere, Mistral, and 90+ more
• See LiteLLM Documentation for full list

Integration with GraphRAG

The LLM providers integrate seamlessly with GraphRAG reasoning:

from semantica.context import AgentContext
from semantica.llms import Groq
from semantica.vector_store import VectorStore
import os

context = AgentContext(
    vector_store=VectorStore(backend="faiss"),
    knowledge_graph=kg
)

llm_provider = Groq(
    model="llama-3.1-8b-instant",
    api_key=os.getenv("GROQ_API_KEY")
)

result = context.query_with_reasoning(
    query="What IPs are associated with security alerts?",
    llm_provider=llm_provider,
    max_hops=2
)

print(f"Response: {result['response']}")
print(f"Reasoning Path: {result['reasoning_path']}")

Common Parameters

All providers support common generation parameters:

• temperature (float): Sampling temperature (0.0-2.0)
• max_tokens (int): Maximum tokens to generate
• top_p (float): Nucleus sampling parameter
• frequency_penalty (float): Frequency penalty
• presence_penalty (float): Presence penalty

Example:

response = groq.generate(
    "What is AI?",
    temperature=0.7,
    max_tokens=500,
    top_p=0.9
)

Error Handling

All providers gracefully handle errors:

try:
    response = groq.generate("Hello")
except ProcessingError as e:
    print(f"Generation failed: {e}")

If a provider is not available (library not installed, API key missing), a ProcessingError is raised with a helpful message.

Examples

Basic Text Generation

from semantica.llms import Groq

groq = Groq(model="llama-3.1-8b-instant")
response = groq.generate("Explain quantum computing in simple terms.")
print(response)

Structured Output

from semantica.llms import OpenAI

openai = OpenAI(model="gpt-4")
result = openai.generate_structured(
    "Extract entities from: Apple Inc. was founded by Steve Jobs in 1976."
)
# Returns: {"entities": [{"name": "Apple Inc.", "type": "Organization"}, ...]}

Using LiteLLM for Multiple Providers

from semantica.llms import LiteLLM

# Switch between providers easily
providers = [
    LiteLLM(model="openai/gpt-4o"),
    LiteLLM(model="anthropic/claude-sonnet-4-20250514"),
    LiteLLM(model="groq/llama-3.1-8b-instant")
]

for provider in providers:
    response = provider.generate("What is AI?")
    print(f"{provider.model}: {response[:50]}...")

Installation

Most providers require additional dependencies:

# Groq
pip install groq

# OpenAI
pip install openai

# HuggingFace
pip install transformers torch

# LiteLLM (supports 100+ providers)
pip install litellm

See Also

• Context Module - GraphRAG with multi-hop reasoning
• Semantic Extract Module - Entity and relationship extraction
• GraphRAG Cookbook - Complete GraphRAG example