# LiteLLM

You can follow the steps given below to configure a LiteLLM (opens new window) proxy to enable yCrash AI features.

LiteLLM is an LLM gateway, not a model vendor. yCrash talks to your LiteLLM proxy over an OpenAI-compatible API. The proxy routes each request to the upstream backend you configured (OpenAI, Anthropic Claude, Azure OpenAI, AWS Bedrock, Google Gemini, vLLM, and others). Upstream vendor keys stay on the LiteLLM proxy. They do not go in ycbuddy-config.json.

# Step 1: Get Your LiteLLM Proxy Details

Ask your platform / DevOps team (or open your LiteLLM Admin UI) for:

  1. The proxy base URL (for example https://litellm.mycompany.com or http://10.0.0.5:4000)
  2. A virtual API key for yCrash (typically starts with sk-)
  3. The chat model alias yCrash should use (must match a LiteLLM model_name, for example rca-chat, rca-claude, or rca-gemini)
  4. Optionally, an embedding alias on the proxy or use the local embedding model recommended below

Confirm from the yCrash server that the proxy is reachable (DNS, firewall, TLS).

Official LiteLLM references:

# Step 2: Add the LiteLLM Settings

Paste the following JSON into ycbuddy-config.json and replace the placeholders with your proxy values:

{
  "llmProvider": "litellm",
  "llmApiKey": "sk-your-litellm-virtual-key",
  "llmEndpoint": "https://litellm.mycompany.com",
  "llmModelName": "rca-chat",
  "llmEmbeddingModelName": "local-all-minilm-l6-v2-q"
}
1
2
3
4
5
6
7

# What each field means

Field Required Notes
llmProvider Yes Must be "litellm"
llmApiKey Yes LiteLLM virtual key (not the upstream OpenAI / Anthropic / Google key). You can also set LLM_API_KEY.
llmEndpoint Yes Proxy base URL. Prefer no trailing /v1 (yCrash appends /v1/chat/completions). A trailing /v1 is stripped automatically if present. You can also set LLM_ENDPOINT.
llmModelName Yes LiteLLM chat model_name alias (for example rca-chat, rca-claude, gpt-4o). Must exist on the proxy and be allowed for your virtual key.
llmEmbeddingModelName Yes Prefer local-all-minilm-l6-v2-q so RAG does not depend on a proxy embedder. You may use a LiteLLM embedding alias (for example text-embedding-3-small) if the proxy and key allow it.

Notes:

  • Keep llmProvider as "litellm" even when the alias routes to Claude, Gemini, or Bedrock. Do not switch to "claude" / "gemini" for those backends. The LiteLLM owns that routing.
  • To change backends, change (or add) the alias on the proxy and set llmModelName to that alias, then restart yCrash.
  • Chat file attachments are stored on the yCrash server (LOCAL_ONLY). yCrash does not upload to the LiteLLM Files API.

# Model aliases

llmModelName must match a LiteLLM model_name. Prefer aliases that match real upstream ids when possible (for example gpt-4o, claude-sonnet-4-6) so yCrash can apply known token limits. Opaque aliases such as rca-chat or rca-claude still route correctly but use conservative defaults unless you tune llmMaxOutputTokens / maxAllowedTokens.

Ensure the upstream model id behind each alias is current and available to the vendor key on the proxy (retired Anthropic ids return HTTP 404 from the vendor).

On the LiteLLM proxy, set:

litellm_settings:
  drop_params: true
1
2

so upstreams that reject unsupported fields still succeed.

# Step 3: Place the File and Restart

Place ycbuddy-config.json in the yCrash upload directory (opens new window) and restart the yCrash server.

Note:

Note: To check whether LLM configuration took effect successfully, refer here.

Successful startup should load the OpenAI-compatible client against your proxy (provider litellm) and initialize the storage-only file uploader.

# Optional LiteLLM Tuning

Field Default Notes
llmMaxOutputTokens 4096 Soft completion cap. Raise (e.g. 8192) for longer RCAs. Hard ceiling: 128000.
maxAllowedTokens 20000 Caps total context (prompt + reserved output) for OpenAI-compatible providers including LiteLLM. Set it at or below the upstream context window and any tpm_limit on the virtual key.
llmTemperature 0.2 yCrash allows up to 2.0 for LiteLLM. If the alias routes to Claude/Anthropic (range 0-1), keep llmTemperature at or below 1.0.
llmSupportsImages true When true, local screenshots may be sent as inline image_url for OpenAI-compatible chat. Set false for text-only aliases so the proxy does not reject vision payloads.

# Troubleshooting

Symptom Likely cause
Log: Unsupported LLM provider Typo in llmProvider (must be exactly litellm)
Log: llmEndpoint is required Missing llmEndpoint / LLM_ENDPOINT
Connection timed out / refused from yCrash yCrash server cannot reach llmEndpoint (firewall, DNS, VPN)
401 / 403 from chat Wrong virtual key, or key not allowed for llmModelName
404 / model not found from LiteLLM llmModelName does not match a proxy model_name, or the upstream vendor id behind the alias is invalid / retired
401 from upstream inside LiteLLM error body Vendor API key on the proxy is wrong (OpenAI / Anthropic / etc.) — fix LiteLLM credentials, not only ycbuddy-config.json
Embeddings / RAG errors Prefer local-all-minilm-l6-v2-q, or ensure the proxy embedding alias accepts OpenAI-style dimensions: 768

If you LLM setup still fails, check the documentation here or reach our support team (opens new window).

← LLM Configuration Troubleshooting →