1266 passed, 0 failed, 63 skipped

stage red: product gap, /v1/audio/speech 500s on empty input instead of surfacing the provider 4xx

LIT-4873: a mid-conversation role='system' reminder invalidates the prompt cache on the vertex_ai / azure_ai / bedrock_invoke Messages paths, while the same request preserves it both direct to Anthropic and through litellm's first-party anthropic path. Product bug, not a test defect: the assertion here is correct and must be restored unchanged with the fix

LIT-4873: a mid-conversation role='system' reminder invalidates the prompt cache on the vertex_ai / azure_ai / bedrock_invoke Messages paths, while the same request preserves it both direct to Anthropic and through litellm's first-party anthropic path. Product bug, not a test defect: the assertion here is correct and must be restored unchanged with the fix

LIT-4873: a mid-conversation role='system' reminder invalidates the prompt cache on the vertex_ai / azure_ai / bedrock_invoke Messages paths, while the same request preserves it both direct to Anthropic and through litellm's first-party anthropic path. Product bug, not a test defect: the assertion here is correct and must be restored unchanged with the fix

stage red: product gap, native passthrough returns no x-litellm-response-cost or x-ratelimit-* headers

set HOSTED_VLLM_API_BASE (the live vLLM server this deployment targets)

stage red: product gap (LIT-5447), retrieving an unknown response id returns 400 (model=None) instead of 404

stage red: product gap, /v1/audio/speech 500s on invalid voice instead of surfacing the provider 4xx

stage red: product gap, /v1/ocr 500s (aocr TypeError) on missing document instead of 400

stage red: product gap, /v1/audio/speech 500s on missing input instead of 400

stage red: product gap, /v1/moderations 500s (KeyError 'input') on missing input instead of 400

stage red: product gap, /v1/responses 500s (aresponses TypeError) on missing input instead of 400

stage red: product gap, /v1/messages 500s (anthropic_messages TypeError) on missing max_tokens instead of 400

stage red: product gap, /v1/messages 500s (anthropic_messages TypeError) on missing messages instead of 400

stage red: product gap, /v1/audio/speech 500s on missing model instead of 400

stage red: product gap, /v1/images/generations 500s (aimage_generation TypeError) on missing prompt instead of 400

stage red: product gap, retrieving a nonexistent vector store returns 2xx with an error envelope in the body instead of 404

stage red: product gap, vector store search 500s (asearch TypeError) on missing query instead of 400

stage red: ALB sends custom passthrough paths to backend pods, which lack ANTHROPIC_API_KEY, so the os.environ header forwards unresolved and Anthropic 401s; routing user-defined paths to the gateway needs a product/infra decision (fixed path namespace or dynamic ingress)

stage red: product gap (LIT-5446), retrieve returns a different id than the stored response (non-idempotent response-id re-encryption)

set VERTEXAI_PROJECT (the project the vertex deployment bills)

stage red: environment gap, the e2e stack neither enables the websearch_interception callback nor declares a search backend, so the request reaches bedrock's transformation and takes its by-design 400. Unskip once the ephemeral stack ships the config in this module's docstring.

Bedrock Mantle GPT-5.6 cells are opt-in; set COMPAT_MANTLE_CELLS=1 once the AWS account is allowlisted for the openai.gpt-5.6-* models

OpenAI GPT-5.6 cells are opt-in; set COMPAT_OPENAI_GPT_CELLS=1 to run them (stage suite timeouts under concurrent load)

Bedrock Mantle GPT-5.6 cells are opt-in; set COMPAT_MANTLE_CELLS=1 once the AWS account is allowlisted for the openai.gpt-5.6-* models

OpenAI GPT-5.6 cells are opt-in; set COMPAT_OPENAI_GPT_CELLS=1 to run them (stage suite timeouts under concurrent load)

stage red: Vertex returns not supported for token counting for Claude aliases

stage red: 1M long_context not green on stage Anthropic path yet (200k sonnet / model alias)

stage red: 1M long_context not green on stage Azure Foundry deployments yet

stage red: 1M long_context not green on stage Bedrock Converse deployments yet

stage red: 1M long_context not green on stage Bedrock Invoke deployments yet

stage red: 1M long_context not green on stage Vertex deployments yet

stage red: /azure passthrough drops client headers (e.g. anthropic-version); product gap

stage red: Azure Foundry tool_search_server not supported in workspace for probed models

stage red: Vertex rejects tool_search when deployment extra_headers inject context-1m beta; product/config

Bedrock Mantle GPT-5.6 cells are opt-in; set COMPAT_MANTLE_CELLS=1 once the AWS account is allowlisted for the openai.gpt-5.6-* models

OpenAI GPT-5.6 cells are opt-in; set COMPAT_OPENAI_GPT_CELLS=1 to run them (stage suite timeouts under concurrent load)

Bedrock Mantle GPT-5.6 cells are opt-in; set COMPAT_MANTLE_CELLS=1 once the AWS account is allowlisted for the openai.gpt-5.6-* models

OpenAI GPT-5.6 cells are opt-in; set COMPAT_OPENAI_GPT_CELLS=1 to run them (stage suite timeouts under concurrent load)

product gap: GET /v1/mcp/server builds each row from the in-memory registry, whose _build_mcp_server_table sets description from mcp_info['description'], so the list reports the mcp_info description while GET /v1/mcp/server/{server_id} reports the stored description column. A server created with both set to different text reads back with two different descriptions depending on the route

stage red: product gap, end-user model_max_budget rpm_limit is stored but never enforced

LIT-5211: on a fresh database the proxy's startup view creation can lose the race against schema migrations, leaving MonthlyGlobalSpend/DailyTagSpend/Last30d* views missing and these routes 500ing until the views exist

LIT-5211: on a fresh database the proxy's startup view creation can lose the race against schema migrations, leaving MonthlyGlobalSpend/DailyTagSpend/Last30d* views missing and these routes 500ing until the views exist

LIT-5211: on a fresh database the proxy's startup view creation can lose the race against schema migrations, leaving MonthlyGlobalSpend/DailyTagSpend/Last30d* views missing and these routes 500ing until the views exist

LIT-5211: on a fresh database the proxy's startup view creation can lose the race against schema migrations, leaving MonthlyGlobalSpend/DailyTagSpend/Last30d* views missing and these routes 500ing until the views exist

LIT-5211: on a fresh database the proxy's startup view creation can lose the race against schema migrations, leaving MonthlyGlobalSpend/DailyTagSpend/Last30d* views missing and these routes 500ing until the views exist

LIT-5118: the stage proxy does not run the dynamic_rate_limiter_v3 callbacks + priority_reservation config this module's docstring requires (zero limiter log lines on any pod during the 2026-08-02 run), so strict enforcement can never engage there

set HOSTED_VLLM_API_BASE (the live vLLM server this deployment targets)

set HOSTED_VLLM_API_BASE (the live vLLM server this deployment targets)

set HOSTED_VLLM_API_BASE (the live vLLM server this deployment targets)

LIT-5027: the path under test hangs. The batch rate limiter reads the input file to count tokens by awaiting litellm.afile_content with no timeout, so a slow Files API holds POST /v1/batches open past any client deadline (63.6s observed on stage against a 60s read timeout). The unattributed-spend-row contract below is never reached, so the test reports a timeout rather than the behavior it guards. Unskip once the fetch is bounded.

The GovCloud credentials this test resolves are not provisioned in the e2e lane, so it fails on configuration rather than on the partition behavior it guards: the upload returns 500 'S3 bucket_name is required' because os.environ/AWS_GOVCLOUD_BATCH_S3_BUCKET resolves to nothing. None of AWS_GOVCLOUD_ACCESS_KEY_ID, AWS_GOVCLOUD_SECRET_ACCESS_KEY, AWS_GOVCLOUD_BATCH_S3_BUCKET or AWS_GOVCLOUD_BATCH_ROLE_ARN is injected into the stack: the runner podSpec mounts 29 individual secretKeyRef entries from litellm-provider-keys and no envFrom, and the string 'govcloud' appears nowhere in project-releaser. The commercial-partition Bedrock batch tests resolve their own os.environ/ bucket and pass, so the reference syntax is sound and only the GovCloud secrets are absent. Skipped rather than weakened because the assertions below are the correct contract. Remove this marker once the four secrets are provisioned and injected; do not relax the assertions to make it pass.

LIT-4820 (https://linear.app/litellm-ai/issue/LIT-4820): GET /v1/files omits newly uploaded files. The upload succeeds and GET /v1/files/{id} returns the file, but it never appears in the listing; the returned set is stable with its newest entry ~10h old, on both the managed (/v1/files?model=) and provider-scoped (/openai/v1/files) routes. Skipped rather than weakened because the assertion below is the correct contract. Remove this marker when LIT-4820 is fixed; do not relax the assertion to make it pass.

stage red: product gap, key-scoped weave_otel spans are not delivered when the OTEL v2 callback is active

stage red: product gap, key-scoped weave_otel spans are not delivered when the OTEL v2 callback is active

LIT-7682: since #40229 the per-request routing_strategy override runs without the latency sampler, so latency-based has no signal to route on

LIT-7682: since #40229 the per-request routing_strategy override runs without the in-flight counter, so least-busy has no signal to route on

product bug LIT-4521: LLM classifier returns SIMPLE for short hard prompts (e.g. Is P equal to NP?); re-enable when classifier tier quality is fixed

stage red: product gap, a request naming a guardrail the proxy does not serve is silently served unguarded (200) instead of failing closed