semantica/docs/reference/llms.md

title, description, icon

title	description	icon
LLMs Module	Unified interface for Groq, OpenAI, Anthropic, Gemini, Ollama, DeepSeek, Novita AI, LiteLLM, and HuggingFace.	microchip

The semantica.llms module provides a single consistent API across 8+ LLM providers. All providers are drop-in replacements for llm_provider= in extractors, reasoning engines, and agents.

---

Providers

from semantica.llms import Groq
import os

llm = Groq(
    model="llama-3.3-70b-versatile",   # default
    api_key=os.getenv("GROQ_API_KEY"),
    max_tokens=64000,
    temperature=0.0,
)
# Best for: high-throughput extraction, fast inference

from semantica.llms import OpenAI
import os

# pip install "semantica[llm-openai]"
llm = OpenAI(
    model="gpt-4o",
    api_key=os.getenv("OPENAI_API_KEY"),
    temperature=0.0,
)
# Best for: general purpose, function calling

from semantica.llms import Anthropic
import os

# pip install "semantica[llm-anthropic]"
llm = Anthropic(
    model="claude-opus-4-7",
    api_key=os.getenv("ANTHROPIC_API_KEY"),
    max_tokens=8192,
)
# Best for: complex reasoning, long context, safety

from semantica.llms import Gemini
import os

# pip install "semantica[llm-gemini]"
llm = Gemini(
    model="gemini-1.5-pro",
    api_key=os.getenv("GOOGLE_API_KEY"),
)
# Best for: long context, multimodal tasks

from semantica.llms import Ollama

# pip install "semantica[llm-ollama]"
llm = Ollama(
    model="llama3.2:3b",
    base_url="http://localhost:11434",
)
# Best for: local inference, air-gapped environments
# No API key required

from semantica.llms import DeepSeek
import os

llm = DeepSeek(
    model="deepseek-chat",
    api_key=os.getenv("DEEPSEEK_API_KEY"),
)
# Best for: coding tasks, analysis at low cost

from semantica.llms import LiteLLM
import os

# pip install "semantica[llm-litellm]"
llm = LiteLLM(
    model="gpt-4o",       # any LiteLLM-supported model string
    api_key=os.getenv("OPENAI_API_KEY"),
)
# Supports: OpenAI, Anthropic, Gemini, Cohere, Azure, Bedrock, and 90+ more

from semantica.llms import HuggingFace

llm = HuggingFace(
    model="mistralai/Mistral-7B-Instruct-v0.3",
    device="cuda",           # "cpu" | "cuda" | "mps"
    max_new_tokens=512,
    temperature=0.1,
)
# Bring your own model — full local control

---

Provider Factory

from semantica.llms import create_provider

llm = create_provider("groq",      model="llama-3.3-70b-versatile")
llm = create_provider("openai",    model="gpt-4o")
llm = create_provider("anthropic", model="claude-opus-4-7")
llm = create_provider("gemini",    model="gemini-1.5-pro")
llm = create_provider("ollama",    model="llama3.2")
llm = create_provider("deepseek",  model="deepseek-chat",   api_key=os.getenv("DEEPSEEK_API_KEY"))
llm = create_provider("novita",    model="deepseek/deepseek-v3.2", api_key=os.getenv("NOVITA_API_KEY"))
llm = create_provider("litellm",   model="gpt-4o")

---

Custom / Enterprise Gateways

Any OpenAI-compatible gateway — internal routing layers, Qwen proxies, LLaMA proxies:

from semantica.llms import OpenAI

llm = OpenAI(
    model="qwen2.5-72b",
    api_key=os.getenv("GATEWAY_API_KEY"),
    base_url="https://my-internal-gateway.company.com/v1",
)

`base_url` is validated at construction time. Non-HTTP(S) schemes raise `ValueError` to prevent SSRF attacks (fixed in v0.5.0).

---

Using in Extractors

from semantica.semantic_extract import NERExtractor, RelationExtractor, TripletExtractor

ner  = NERExtractor(method="llm",  llm_provider=llm, max_retries=3)
rel  = RelationExtractor(method="llm", llm_provider=llm)
trip = TripletExtractor(method="llm", llm_provider=llm)

---

Provider Comparison

Provider	Speed	Cost	Local	Context	Best for
Groq	Very fast	Low	No	128k	High-throughput extraction
OpenAI	Fast	Medium	No	128k	General purpose, function calling
Anthropic	Fast	Medium	No	200k	Complex reasoning, safety
Gemini	Fast	Low	No	1M	Long context, multimodal
Ollama	Medium	Free	Yes	Varies	Privacy, no API key
DeepSeek	Fast	Very low	No	64k	Coding, analysis
Novita AI	Fast	Low	No	Varies	DeepSeek-based tasks
LiteLLM	Varies	Varies	Varies	Varies	Multi-provider routing
HuggingFace	Slow	Free	Yes	Varies	Custom models, BYOM

For production systems, Groq delivers the best throughput-to-cost ratio for extraction pipelines. For complex multi-hop reasoning tasks, Claude Opus or GPT-4o provide the highest accuracy.

---

See Also

Use LLMs for NER and relation extraction. LLM providers in Agno multi-agent teams. LLM-backed deductive and abductive reasoning. GraphRAG uses LLMs for reasoning over knowledge graphs.