LiteLLM Compatibility

Compatibility Transformations

The LiteLLM compatibility plugin provides two transformations:

1. Text-to-Chat Conversion - Automatically converts text completion requests to chat completion format for models that only support chat APIs

When either transformation is applied, responses include extra_fields.litellm_compat: true.

---

1. Text-to-Chat Conversion

Many modern AI models (like GPT-3.5-turbo, GPT-4, Claude, etc.) only support the chat completion API and don’t have native text completion endpoints. With LiteLLM compatibility enabled, you can send a text completion request to any of these models and still get a text completion response back - DeepIntShield converts the request and response shape for you.

Note

Smart Conversion: Conversion only happens when the model lacks native text completion support. If a model already supports text completion (like OpenAI’s davinci models), your request is sent to the text completion endpoint unchanged.

This lets you keep a single text completion interface across all providers, even those that only support chat completions.

What you send and get back:

• Your text prompt is delivered to the model as a user message; max_tokens, temperature, top_p, stop sequences, and fallbacks are all carried through.
• The response comes back in text completion shape - content in choices[0].text and object: "text_completion" - so your existing code reads it the same way.

Enabling LiteLLM Compatibility

Gateway UI

1. Open the DeepIntShield dashboard
2. Navigate to Settings → Client Configuration
3. Enable LiteLLM Fallbacks
4. Save your configuration

Configuration File

{
  "client": {
    "enable_litellm_fallbacks": true
  }
}

Supported Providers

LiteLLM compatibility mode works with any provider that supports chat completions but lacks native text completion support:

Provider	Native Text Completion	LiteLLM Fallback
OpenAI (GPT-4, GPT-3.5-turbo)	No	Yes
Anthropic (Claude)	No	Yes
Groq	No	Yes
Gemini	No	Yes
Mistral	No	Yes
Bedrock	Varies by model	Yes

Behavior Details

A model is treated as supporting text completion if it is listed with a text-completion capability in the model catalog. For those models your request goes straight through; for chat-only models the conversion above is applied automatically. You can check extra_fields.litellm_compat on the response to see whether conversion happened.

Transformations Reference

Transformation 1: Text-to-Chat Conversion

Applies to: Text completion requests on chat-only models

Phase	Original	Transformed
Request	Text prompt (string)	Chat message with role: "user"
Request	Array prompts	Concatenated into text content blocks
Request	text_completion request type	chat_completion request type
Request	max_tokens, temperature, top_p	Mapped to chat equivalents
Response	choices[0].message.content	choices[0].text
Response	object: "chat.completion"	object: "text_completion"

Metadata Set on Transformed Responses

When either transformation is applied:

• extra_fields.litellm_compat: Set to true
• extra_fields.provider: The provider that handled the request
• extra_fields.request_type: Reflects the original request type
• extra_fields.model_requested: The originally requested model

Error Handling

When errors occur on transformed requests:

• extra_fields.litellm_compat is set to true
• Original request type and model are preserved in error metadata

What’s Preserved

• Model selection and fallback chain
• Temperature, top_p, max_tokens, and other generation parameters
• Stop sequences and frequency/presence penalties
• Usage statistics and token counts

When to Use This

Good Use Cases:

• Migrating from LiteLLM to DeepIntShield without code changes
• Maintaining backward compatibility with text completion interfaces
• Using a unified API across providers with different capabilities

Consider Alternatives When:

• You need chat-specific features (system messages, conversation history)
• You want explicit control over message formatting
• Performance is critical (direct chat requests avoid conversion overhead)

Related Features

• Fallbacks - Automatic provider failover
• Drop-in Replacement - Use existing SDKs with DeepIntShield
• LiteLLM Integration - Using LiteLLM SDK with DeepIntShield