Groq

Use Groq with Blazen — construct the GroqProvider and call complete().

Groq is an OpenAI-compatible LLM provider. Construct it like any other Blazen provider — build a list of ChatMessages, call complete() (or stream()), and read back a typed ModelResponse.

At a glance

Provider idgroq
Base URLhttps://api.groq.com/openai/v1
Default modelllama-3.3-70b-versatile
API key env varGROQ_API_KEY
AuthAuthorization: Bearer <key>

Set GROQ_API_KEY in the environment and Blazen reads it automatically, or pass the key explicitly when you construct the provider.

Capabilities

CapabilitySupported
StreamingYes
Tool callingYes
Structured outputYes
VisionYes
Model listingYes
EmbeddingsNo

Usage

Construct the provider and call complete(). The default model is llama-3.3-70b-versatile; override it with with_model / model when you need a different one.

use blazen_llm::{Model, ModelRequest, ChatMessage};
use blazen_provider_groq::GroqProvider;

// Reads GROQ_API_KEY from the environment, or pass the key to `new`.
let model = GroqProvider::new(std::env::var("GROQ_API_KEY")?);

let resp = model
    .complete(ModelRequest::new(vec![ChatMessage::user("Hello")]))
    .await?;
println!("{}", resp.content.unwrap_or_default());
from blazen import GroqProvider, ProviderOptions, ChatMessage

# Omit the api_key to read GROQ_API_KEY from the environment.
model = GroqProvider(options=ProviderOptions(api_key="..."))

resp = await model.complete([ChatMessage.user("Hello")])
print(resp.content)
import { GroqProvider, ChatMessage } from "blazen";

// Omit apiKey to read GROQ_API_KEY from the environment.
const model = GroqProvider.create({ apiKey: "..." });

const resp = await model.complete([ChatMessage.user("Hello")]);
console.log(resp.content);

Streaming

async for chunk in model.stream([ChatMessage.user("Count to five")]):
    print(chunk.delta, end="")
await model.stream([ChatMessage.user("Count to five")], (chunk) => {
  if (chunk.delta) process.stdout.write(chunk.delta);
});

See also