LiteLLM Integration

Connect your LiteLLM gateway to Capsule Security for centralized governance of every completion request and MCP tool call routed through the proxy - with inline policy enforcement before the upstream call and token/latency audit after it.

This integration uses a LiteLLM custom callback that runs in-process inside your LiteLLM proxy. Before each completion is forwarded to the upstream model provider, the callback posts the request to your Capsule tenant and blocks on an allow/deny verdict. After the upstream call settles, it posts token usage and latency as fire-and-forget telemetry.

The callback checks /chat/completions, /completions, /v1/messages, /v1/responses and Gemini generateContent. With plugin 0.1.6 or later it also checks the text inputs of /embeddings and prompts sent through LiteLLM's provider pass-through routes, such as /vertex_ai and /gemini. With plugin 0.1.8 or later it checks each record of a batch input file uploaded to /v1/files, and LiteLLM drops a denied record from the upload.

LiteLLM is operator-deployed - you run the proxy in your own infrastructure, so there is no SaaS install. The only thing exchanged at install time is a signed token that scopes the plugin to your tenant and environment.

The following hooks are configured:

Before you begin, ensure you have:

• A running LiteLLM proxy (Python 3.10 or later)
• A Capsule Security account with admin access
• Network access from your LiteLLM proxy to your Capsule agentsecurity endpoint (e.g. https://agents.capsule.security)

1. Log in to the Capsule Security portal.

2. Navigate to Integrations and locate LiteLLM.

3. Click Install - Capsule generates a JWT scoped to your tenant and LiteLLM environment. The token contains the tenant and environment claims required by the agentsecurity endpoint; treat it as a secret.

4. Copy the generated endpoint and token. You will reference them from your LiteLLM proxy environment in the next steps.

Install the published Capsule plugin into the same environment that runs your LiteLLM proxy. It requires LiteLLM 1.100 or later:

pip install capsule-litellm

Register the callback in your LiteLLM config.yaml:

litellm_settings:
  callbacks: ['capsule_litellm_plugin.capsule_callback']
  global_disable_no_log_param: true

global_disable_no_log_param stops clients from opting out of Capsule telemetry. LiteLLM skips every logging callback for a request whose body sets "no-log": true, so without it Capsule still checks such a request but receives no token usage, latency or error for it.

The callback reads its configuration from environment variables. Set these where the proxy runs:

export CAPSULE_LITELLM_ENDPOINT="https://agents.capsule.security/v1/litellm/hooks"
export CAPSULE_LITELLM_TOKEN="<token from Step 1>"

Never commit the JWT to version control. Recommended approaches:

• Read the token from an environment variable (CAPSULE_LITELLM_TOKEN)
• Source it from your secret manager (1Password, AWS Secrets Manager, HashiCorp Vault) at proxy startup
• Deliver it through the same channel that distributes the rest of your LiteLLM deployment configuration

For the callback to take effect, restart the proxy with the updated configuration:

litellm --config config.yaml

On startup the plugin logs capsule_litellm_plugin: enabled when it has a valid endpoint and token.

1. Send a completion through the proxy to generate activity:

   curl -sS http://localhost:4000/v1/chat/completions \
     -H "Authorization: Bearer $LITELLM_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

2. Log in to the Capsule Security portal.

3. Navigate to Inventory > Agents and confirm a LiteLLM agent (named after the model, e.g. LiteLLM/gpt-4o) appears. MCP servers you route through LiteLLM appear as their own agents, e.g. LiteLLM MCP/github.

4. Click on the agent and review the audit logs to verify the pre-call and post-call events are captured.

If events are not appearing:

1. Verify the endpoint is reachable from the proxy host:

   curl -sS -o /dev/null -w "%{http_code}\n" https://agents.capsule.security/v1/litellm/hooks

2. Verify the environment variables are set - the plugin logs capsule_litellm_plugin: disabled and stays inert if CAPSULE_LITELLM_ENDPOINT or CAPSULE_LITELLM_TOKEN is missing.

3. Check for timeouts - if your network has high latency to the Capsule endpoint, increase CAPSULE_LITELLM_TIMEOUT_MS or confirm CAPSULE_LITELLM_FAIL_OPEN is set appropriately.

4. Confirm the callback is loaded - LiteLLM logs registered callbacks on startup; the Capsule callback registers from capsule_litellm_plugin.capsule_callback.

5. Contact Capsule Security support if issues persist.

Capsule judges each request together with the earlier requests of the same conversation when the client sends a session id. LiteLLM resolves it from the x-litellm-session-id header (LiteLLM 1.82 or later), and from x-claude-code-session-id and other x-<client>-session-id headers (1.84 or later), so Claude Code sessions are tracked without extra setup. Plugin 0.1.2 or later forwards it.

Requests without a session id are judged on their own, and each one appears as its own session in the portal.

An agent loop re-sends the whole conversation on every request, so Capsule judges only what a request adds after the model's last answer: the results of the tools the model called, and the user's next message. A credential or prompt injection in a tool result is caught before the model reads it, and the original user prompt is not judged again on every round.

This works on /chat/completions, /v1/messages, /v1/responses and Gemini generateContent, with plugin 0.1.5 or later. Each tool call appears in the session as a Tool Call activity carrying the arguments the model passed and the result the client sent back.

Policies apply by activity type: a policy scoped to User Message judges the user's messages, and one scoped to Tool Call judges tool results. When a request carries both, each part is judged under its own type.

If you route MCP servers through LiteLLM (mcp_servers in config.yaml), the same callback checks every tool call before LiteLLM forwards it to the MCP server, whether a client calls the tool or the model does through the Responses API. A deny returns a 403 and the tool never runs. Plugin 0.1.5 or later is required.

Each MCP server appears in the inventory as its own agent, LiteLLM MCP/<server>, with its tools and an MCP server resource, so policies can be scoped to a tool or an MCP server. Tool arguments are judged like any other tool input, for example for credentials. With plugin 0.1.8 or later, the tools a client lists on LiteLLM's /mcp endpoint appear with their descriptions before any of them is called.

With plugin 0.1.8 or later, Capsule also judges what an MCP tool returns before the caller receives it, whether a client called the tool or the model did. A denied result is replaced by the block message on LiteLLM 1.104 or later; older versions record the verdict but pass the result through. A tool call the model makes through the Responses API appears in a session of its own, because LiteLLM does not pass the client's session id to it.

The Capsule callback runs in-process inside the LiteLLM proxy and observes every completion request and MCP tool call. Before deploying:

1. Protect the JWT - anyone with the token can post events on behalf of your tenant. Rotate it through the portal if it is exposed.
2. Choose CAPSULE_LITELLM_FAIL_OPEN deliberately - true (the default) prioritizes availability and lets requests through when Capsule is unreachable; false prioritizes policy enforcement and rejects requests when Capsule cannot be reached.
3. Pin a plugin version in production and review changelogs before upgrading.
4. Use TLS-only endpoints. The plugin sends a Bearer token on every request - never configure an http:// endpoint outside local development.

For help with this integration:

• Email: support@capsule.security
• Include: Your organization ID, integration status, plugin version, and any error messages

• LiteLLM Documentation
• LiteLLM Custom Callbacks
• LiteLLM MCP Gateway