Groq

Overview

Groq is an OpenAI-compatible provider offering the same API interface with identical parameter handling. Key features:

• Full OpenAI compatibility - Identical request/response format
• Streaming support - Server-Sent Events with delta-based updates
• Tool calling - Complete function definition and execution support
• Text completion fallback - Via litellm compatibility mode when enabled

Supported Operations

Operation	Non-Streaming	Streaming	Endpoint
Chat Completions	✅	✅	/v1/chat/completions
Responses API	✅	✅	/v1/chat/completions
Text Completions	⚠️	⚠️	Served via Chat Completions
List Models	✅	-	/v1/models
Embeddings	❌	❌	-
Image Generation	❌	❌	-
Speech (TTS)	❌	❌	-
Transcriptions (STT)	❌	❌	-
Files	❌	❌	-
Batch	❌	❌	-

Note

Text Completions (⚠️): Not supported natively by Groq. When enabled via the x-litellm-fallback context flag, text completion requests are served through Groq’s Chat Completions API.

Unsupported Operations (❌): Embeddings, Image Generation, Speech, Transcriptions, Files, and Batch are not supported by the upstream Groq API. These return an unsupported-operation error.

---

1. Chat Completions

Request Parameters

Groq supports all standard OpenAI chat completion parameters. For full parameter reference and behavior, see OpenAI Chat Completions. The following parameters are not supported and are ignored: prompt_cache_key, verbosity, store, service_tier.

Reasoning Parameter

Groq supports reasoning via the standard reasoning_effort field:

{
  "model": "llama-3.3-70b-versatile",
  "messages": [...],
  "reasoning_effort": "high"
}

Message Content

Groq uses OpenAI message format with the following content type support:

• ✅ Text content (strings)
• ❌ Images (neither URL nor base64)
• ❌ Audio input
• ❌ Files

For all other message handling, tools, responses, and streaming formats, refer to OpenAI Chat Completions.

---

2. Responses API

Groq supports the Responses API with the same parameters and message handling as Chat Completions. Responses are returned in Responses format, with output items instead of message content.

---

3. Text Completions (Litellm Fallback)

Caution

Text Completions are not natively supported by Groq. Support is only available when the x-litellm-fallback context flag is set.

When enabled, text completion requests are served through Groq’s Chat Completions API.

Limitations:

• Behaves like the chat API (different from a native text completion)
• Single choice only (n=1)
• Streaming not available

---

4. List Models

Groq’s model listing endpoint returns available models with their context lengths and capabilities.

---

Unsupported Features

Feature	Reason
Image URLs	Groq doesn’t support image inputs
Image Base64	Groq doesn’t support image inputs
Multiple Images	Groq doesn’t support image inputs
Embedding	Not offered by Groq API
Speech/TTS	Not offered by Groq API
Transcription/STT	Not offered by Groq API
Batch Operations	Not offered by Groq API
File Management	Not offered by Groq API

---

Caveats

User Field Size Limit

Severity: Low Behavior: User field > 64 characters is silently dropped Impact: Longer user identifiers are lost