litellm-e2e · build #367

1509 passed, 0 failed, 59 skipped

1.105.0rc1rc

Oct 4, 2026, 12:03 AM UTC ·2026bd8 ↗· build #367 ↗· 1.105.0rc1 ↗· run history

All 1568 tests

  1. ✓test_anthropic_chat_returns_thinking_content TestAnthropicChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-anthropic-thinking-af856bee31f7 that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-anthropic-thinking-af856bee31f7
    View test on GitHub ↗
  2. ✓test_anthropic_chat_returns_tool_call TestAnthropicChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-anthropic-tool-729a0d58068c that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-anthropic-tool-729a0d58068c
    View test on GitHub ↗
  3. ✓test_anthropic_chat_streams_real_content TestAnthropicChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-anthropic-stream-deb4b5fc032e that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-anthropic-stream-deb4b5fc032e
    View test on GitHub ↗
  4. ✓test_anthropic_chat_streams_tool_call TestAnthropicChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-anthropic-tool-stream-70081bd4eb9f that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-anthropic-tool-stream-70081bd4eb9f
    View test on GitHub ↗
  5. ✓test_anthropic_chat_structured_output_conforms_to_schema TestAnthropicChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-anthropic-schema-bb6618372595 that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-anthropic-schema-bb6618372595
    View test on GitHub ↗
  6. ✓test_anthropic_chat_vision_describes_image TestAnthropicChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-anthropic-vision-433cff1efa79 that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-anthropic-vision-433cff1efa79
    View test on GitHub ↗
  7. ✓test_anthropic_passthrough_nonstreaming_logs_costtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  8. ✓test_anthropic_passthrough_streaming_logs_costtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  9. ✓test_anthropic_passthrough_tool_call_logs_costtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  10. ✓test_anthropic_prompt_caching_reads_cache TestCacheControltest_cache_control.py›
    1. Add a deployment named e2e-anthropic-cache-d84ca041835c that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  11. ✓test_anthropic_with_extended_thinking TestChatToolResultRoundTriptest_chat_tool_round_trip_e2e.py›
    1. Add a deployment named e2e-chat-tool-loop-cbed6c6c3b56 that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-chat-tool-loop-cbed6c6c3b56
    View test on GitHub ↗
  12. ✓test_array_input_returns_vectors TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-array-5a47eb69af49 that calls openai/text-embedding-3-small
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  13. ✓test_audio_speech_returns_audio TestAudioSpeechtest_audio_speech_e2e.py›
    1. Add a deployment named e2e-speech-ccae7f1109d4 that calls openai/gpt-4o-mini-tts
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  14. ✓test_audio_speech_streams_audio_chunks TestAudioSpeechtest_audio_speech_e2e.py›
    1. Add a deployment named e2e-speech-84c669b51153 that calls openai/gpt-4o-mini-tts
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  15. ✓test_audio_transcriptions_returns_text TestAudioTranscriptionstest_audio_transcriptions_e2e.py›
    1. Add a deployment named e2e-transcribe-be162eeead8c that calls openai/gpt-4o-mini-transcribe
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  16. ✓test_azure_foundry_chat_returns_content TestAzureFoundryChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-azure-foundry-chat-e55f7dfb5683 that calls azure_ai/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-azure-foundry-chat-e55f7dfb5683
    View test on GitHub ↗
  17. ✓test_azure_openai_chat_returns_content TestAzureOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-azure-openai-chat-b10f2e914650 that calls azure/gpt-5.4-nano
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-azure-openai-chat-b10f2e914650
    View test on GitHub ↗
  18. ✓test_azure_openai_chat_returns_tool_call TestAzureOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-azure-openai-tool-e9ec901ba7b5 that calls azure/gpt-5.4-nano
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-azure-openai-tool-e9ec901ba7b5
    View test on GitHub ↗
  19. ✓test_basic_nonstream TestAzureFoundryMessagestest_messages_azure_foundry_e2e.py›
    1. Add a deployment named e2e-azure-foundry-messages-fbb7466aad43 that calls azure_ai/claude-haiku-4-5
    2. Generate a virtual key with models: e2e-azure-foundry-messages-fbb7466aad43 and user id: e2e-test-user
    View test on GitHub ↗
  20. ✓test_basic_stream TestAzureFoundryMessagestest_messages_azure_foundry_e2e.py›
    1. Add a deployment named e2e-azure-foundry-messages-b5269a8dc4db that calls azure_ai/claude-haiku-4-5
    2. Generate a virtual key with models: e2e-azure-foundry-messages-b5269a8dc4db and user id: e2e-test-user
    View test on GitHub ↗
  21. ✓test_batch_s3_keys_do_not_break_chat TestBedrockBatchDeploymentServesChattest_bedrock_provider_matrix_e2e.py›
    1. Add a deployment named e2e-bedrock-batch-chat-524c4dc8eb8e that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  22. ✓test_bedrock_converse TestChatToolResultRoundTriptest_chat_tool_round_trip_e2e.py›
    1. Add a deployment named e2e-chat-tool-loop-92fb2a80acb6 that calls bedrock/converse/us.anthropic.claude-sonnet-5-5
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-chat-tool-loop-92fb2a80acb6
    View test on GitHub ↗
  23. ✓test_bedrock_converse_chat_returns_content TestBedrockConverseChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-bedrock-chat-fa553218ff44 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-bedrock-chat-fa553218ff44
    View test on GitHub ↗
  24. ✓test_bedrock_converse_chat_returns_reasoning TestBedrockConverseChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-bedrock-thinking-c7c8599b70a3 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-bedrock-thinking-c7c8599b70a3
    View test on GitHub ↗
  25. ✓test_bedrock_converse_chat_returns_tool_call TestBedrockConverseChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-bedrock-tool-21e7941ac46f that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-bedrock-tool-21e7941ac46f
    View test on GitHub ↗
  26. ✓test_bedrock_converse_chat_streams_real_content TestBedrockConverseChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-bedrock-stream-52c93b89defa that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-bedrock-stream-52c93b89defa
    View test on GitHub ↗
  27. ✓test_bedrock_converse_chat_vision_describes_image TestBedrockConverseChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-bedrock-vision-8b771b36c3e0 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-bedrock-vision-8b771b36c3e0
    View test on GitHub ↗
  28. ✓test_bedrock_converse_reads_a_pdf_sent_by_url TestBedrockConverseChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-bedrock-document-532f6ba76ae6 that calls bedrock/us.amazon.nova-2-lite-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-bedrock-document-532f6ba76ae6
    View test on GitHub ↗
  29. ✓test_bedrock_converse_with_extended_thinking TestChatToolResultRoundTriptest_chat_tool_round_trip_e2e.py›
    1. Add a deployment named e2e-chat-tool-loop-8c17338da162 that calls bedrock/converse/us.anthropic.claude-sonnet-4-6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-chat-tool-loop-8c17338da162
    View test on GitHub ↗
  30. ✓test_bedrock_embeddings_returns_vector TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-bedrock-2338a8c2b4b8 that calls bedrock/amazon.titan-embed-text-v2:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  31. ✓test_bedrock_image_generation_returns_image TestImageGenerationtest_image_generation_e2e.py›
    1. Add a deployment named e2e-bedrock-image-7d085a0bc3ec that calls bedrock/amazon.nova-canvas-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  32. ✓test_bedrock_prompt_caching_reads_cache TestCacheControltest_cache_control.py›
    1. Add a deployment named e2e-bedrock-cache-43117276eab3 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  33. ✓test_bedrock_request_id_header_surfaces TestBedrockResponseHeaderstest_bedrock_provider_matrix_e2e.py›
    1. Add a deployment named e2e-bedrock-headers-26f3c5852e32 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  34. ✓test_bedrock_request_id_header_surfaces_on_stream TestBedrockResponseHeaderstest_bedrock_provider_matrix_e2e.py›
    1. Add a deployment named e2e-bedrock-headers-stream-16cf7b8be117 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-bedrock-headers-stream-16cf7b8be117
    View test on GitHub ↗
  35. ✓test_bedrock_rerank_scores_top_n TestReranktest_rerank_e2e.py›
    1. Add a deployment named e2e-bedrock-rerank-b57937dc96ac that calls bedrock/arn:aws:bedrock:us-east-1::foundation-model/cohere.rerank-v3-5:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /v1/rerank request to e2e-bedrock-rerank-b57937dc96ac for "What is the capital of the United States?"
    View test on GitHub ↗
  36. ✓test_bedrock_titan_embeds_token_array_input_as_its_decoded_text TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-titan-tokens-2430158f849d that calls bedrock/amazon.titan-embed-text-v2:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  37. ✓test_bridged_stream_delivers_content_finish_reason_and_done TestResponsesBridgeChatCompletionsStreamingtest_responses_bridge_streaming_e2e.py›
    1. Add a deployment named e2e-bridge-stream-50fb5539a157 that calls openai/gpt-5.3-codex
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-bridge-stream-50fb5539a157
    View test on GitHub ↗
  38. ✓test_bridged_stream_reassembles_tool_call TestResponsesBridgeChatCompletionsStreamingtest_responses_bridge_streaming_e2e.py›
    1. Add a deployment named e2e-bridge-stream-882eb349456e that calls openai/gpt-5.3-codex
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-bridge-stream-882eb349456e
    View test on GitHub ↗
  39. ✓test_bridged_stream_shares_one_chunk_id TestResponsesBridgeChatCompletionsStreamingtest_responses_bridge_streaming_e2e.py›
    1. Add a deployment named e2e-bridge-stream-f96fbcba5d9b that calls openai/gpt-5.3-codex
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-bridge-stream-f96fbcba5d9b
    View test on GitHub ↗
  40. ✓test_cache_read_tokens_bill_at_the_cache_read_rate TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-e419b318bbcf that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-together-e419b318bbcf
    4. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  41. ✓test_caller_completion_window_bills_its_rates TestSailResponsestest_sail_e2e.py›
    1. Add a deployment named e2e-sail-c8461178decc that calls sail/zai-org/GLM-5.3
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  42. ✓test_calls_without_auth_is_denied TestRealtimeHttptest_realtime_http_e2e.py›

    No steps recorded

    View test on GitHub ↗
  43. ✓test_cancel_background_response TestBackgroundResponseCanceltest_responses_retrieve_e2e.py›
    1. Add a deployment named e2e-resp-cancel-6f5e4ff9913c that calls openai/gpt-5.5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  44. ✓test_cancel_background_streaming_response_by_streamed_id TestBackgroundResponseCanceltest_responses_retrieve_e2e.py›
    1. Add a deployment named e2e-resp-cancel-stream-b5262b7adeaa that calls openai/gpt-5.5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  45. ✓test_chat_returns_real_completion[claude-haiku-4-5-anthropic] TestChatCompletionsRegressiontest_chat_completions_regression_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to claude-haiku-4-5
    View test on GitHub ↗
  46. ✓test_chat_returns_real_completion[gemini-2.5-flash-gemini] TestChatCompletionsRegressiontest_chat_completions_regression_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  47. ✓test_chat_returns_real_completion[gpt-5.5-openai] TestChatCompletionsRegressiontest_chat_completions_regression_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gpt-5.5
    View test on GitHub ↗
  48. ✓test_chat_stream_is_sse_and_ends_with_done TestChatStreamContracttest_chat_stream_contract_e2e.py›
    1. Add a deployment named e2e-chat-stream-8fcad0e0299a that calls openai/gpt-5.6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-chat-stream-8fcad0e0299a
    View test on GitHub ↗
  49. ✓test_chat_template_kwargs_reach_together TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-816a3999b0d3 that calls together_ai/Qwen/Qwen3.5-9B
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-together-816a3999b0d3
    View test on GitHub ↗
  50. ✓test_client_secret_missing_auth_is_denied TestRealtimeHttptest_realtime_http_e2e.py›
    1. Add a deployment named e2e-realtime-http-2aa9e33356e3 that calls openai/gpt-realtime
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  51. ✓test_cohere_chat_returns_content TestCohereChattest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-cohere-chat-85abb873c6f1 that calls cohere/command-r-08-2024
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-cohere-chat-85abb873c6f1
    View test on GitHub ↗
  52. ✓test_cohere_embeddings_returns_vector TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-cohere-7ee210c008e8 that calls cohere/embed-v4.0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  53. ✓test_container_file_lifecycle_through_the_gateway TestOpenAIContainerFilestest_containers_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  54. ✓test_contents_are_counted_by_the_provider_endpoint[gemini-2.5-flash-vertex] TestGeminiContentsTokenCountingtest_token_counter_gemini_contents_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  55. ✓test_contents_are_counted_by_the_provider_endpoint[gemini-2.5-flash] TestGeminiContentsTokenCountingtest_token_counter_gemini_contents_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  56. ✓test_converse_empty_messages_returns_client_error TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-943154081f5e that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  57. ✓test_converse_invalid_model_returns_error TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-0185817a3785 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  58. ✓test_converse_missing_messages_returns_error TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-0aca83a1fb0b that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  59. ✓test_converse_output_format_returns_schema_json_text TestBedrockMessagestest_messages_bedrock_e2e.py›
    1. Add a deployment named e2e-messages-bedrock-302d5fd476d4 that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  60. ✓test_converse_returns_assistant TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-f2c755f572f2 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  61. ✓test_converse_stream_returns_chunks TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-0a13709d44dd that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  62. ✓test_cost_header_and_spend_row_match_the_registry_price TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-8181339e3fc2 that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  63. ✓test_cost_header_matches_the_spend_log[chat_completions-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  64. ✓test_cost_header_matches_the_spend_log[chat_completions-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  65. ✓test_cost_header_matches_the_spend_log[chat_completions-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  66. ✓test_cost_header_matches_the_spend_log[chat_completions-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  67. ✓test_cost_header_matches_the_spend_log[chat_completions-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  68. ✓test_cost_header_matches_the_spend_log[chat_completions-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  69. ✓test_cost_header_matches_the_spend_log[messages-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  70. ✓test_cost_header_matches_the_spend_log[messages-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  71. ✓test_cost_header_matches_the_spend_log[messages-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  72. ✓test_cost_header_matches_the_spend_log[messages-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  73. ✓test_cost_header_matches_the_spend_log[messages-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  74. ✓test_cost_header_matches_the_spend_log[messages-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  75. ✓test_cost_header_matches_the_spend_log[responses-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  76. ✓test_cost_header_matches_the_spend_log[responses-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  77. ✓test_cost_header_matches_the_spend_log[responses-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  78. ✓test_cost_header_matches_the_spend_log[responses-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  79. ✓test_cost_header_matches_the_spend_log[responses-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  80. ✓test_cost_header_matches_the_spend_log[responses-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  81. ✓test_create_batch_missing_input_file_id_returns_error TestFilesBatchesContracttest_files_batches_contract_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  82. ✓test_create_client_secret TestRealtimeHttptest_realtime_http_e2e.py›
    1. Add a deployment named e2e-realtime-http-9167abaf9296 that calls openai/gpt-realtime
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  83. ✓test_create_list_retrieve_delete_lifecycle TestVectorStorestest_vector_stores_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  84. ✓test_credential_backed_messages TestCredentialBackedMessagestest_credential_messages_e2e.py›
    1. Save the provider credential e2e-cred-0c77d4dae4f0
    2. Add a deployment named e2e-cred-messages-0c77d4dae4f0 that calls anthropic/claude-haiku-4-5
    3. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  85. ✓test_custom_pricing_is_billed_at_configured_rate TestCustomPricingtest_custom_pricing_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named custom-priced-flash-a154dd15a274 that calls gemini/gemini-2.5-flash
    3. Send a /chat/completions request to custom-priced-flash-a154dd15a274
    View test on GitHub ↗
  86. ✓test_custom_pricing_is_isolated_from_sibling_deployment TestCustomPricingtest_custom_pricing_e2e.py›
    1. Add a deployment named custom-priced-flash-ddef0ed307ef that calls gemini/gemini-2.5-flash
    2. Add a deployment named base-flash-17a6cae4f9a0 that calls gemini/gemini-2.5-flash
    3. List the deployments from /model/info
    View test on GitHub ↗
  87. ✓test_deleted_response_is_no_longer_retrievable TestStoredResponseLifecycletest_responses_retrieve_e2e.py›
    1. Add a deployment named e2e-resp-delete-cede6d399276 that calls openai/gpt-5.5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  88. ✓test_embeddings_call_logs_its_cost TestOpenAIPassthroughSpendtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  89. ✓test_embeddings_returns_vector TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-37a3d97307df that calls openai/text-embedding-3-small
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  90. ✓test_empty_image_returns_error TestImageEdittest_image_edits_e2e.py›
    1. Add a deployment named e2e-image-edit-6dc41b5d5792 that calls openai/gpt-image-1
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  91. ✓test_empty_input_returns_client_error TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-val-9ed8107715c1 that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  92. ✓test_empty_messages_returns_client_error TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-defeab417cfd that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  93. ✓test_empty_prompt_returns_client_error TestImageGenerationtest_image_generation_e2e.py›
    1. Add a deployment named e2e-image-4120df031c06 that calls openai/gpt-image-1-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  94. ✓test_empty_prompt_returns_error TestImageEdittest_image_edits_e2e.py›
    1. Add a deployment named e2e-image-edit-f6e391cf914b that calls openai/gpt-image-1
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  95. ✓test_file_attach_poll_and_search TestVectorStorestest_vector_stores_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  96. ✓test_flagged_model_keeps_prompt_cache_across_system_reminder TestAnthropicChatMidConversationSystemtest_chat_mid_conversation_system_e2e.py›
    1. Add a deployment named e2e-chat-midsys-71bcb3de1ae4 that calls anthropic/claude-opus-4-8
    2. Generate a virtual key with models: e2e-chat-midsys-71bcb3de1ae4 and user id: e2e-test-user
    View test on GitHub ↗
  97. ✓test_flagged_model_keeps_prompt_cache_across_system_reminder TestBedrockInvokeChatMidConversationSystemtest_chat_mid_conversation_system_e2e.py›
    1. Add a deployment named e2e-chat-midsys-cbfe758fbebf that calls bedrock/invoke/us.anthropic.claude-sonnet-5
    2. Generate a virtual key with models: e2e-chat-midsys-cbfe758fbebf and user id: e2e-test-user
    View test on GitHub ↗
  98. ✓test_gemini TestChatToolResultRoundTriptest_chat_tool_round_trip_e2e.py›
    1. Add a deployment named e2e-chat-tool-loop-7885dd772dd5 that calls gemini/gemini-3.5-flash-lite
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-chat-tool-loop-7885dd772dd5
    View test on GitHub ↗
  99. ✓test_gemini_chat_returns_content_and_logs_cost TestGeminiChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-gemini-chat-517153d8d6ea that calls gemini/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-gemini-chat-517153d8d6ea
    4. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  100. ✓test_gemini_passthrough_nonstreaming_logs_costtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  101. ✓test_gemini_passthrough_streaming_logs_costtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  102. ✓test_gemini_passthrough_tool_call_logs_costtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  103. ✓test_generate_content_returns_response_cost_header TestGoogleNativeGenerateContenttest_google_native_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-google-native-d3e9f79c3997 that calls gemini/gemini-2.5-flash
    View test on GitHub ↗
  104. ✓test_image_edit_returns_image TestImageEdittest_image_edits_e2e.py›
    1. Add a deployment named e2e-image-edit-31354392d09e that calls openai/gpt-image-1
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  105. ✓test_image_generation_returns_image TestImageGenerationtest_image_generation_e2e.py›
    1. Add a deployment named e2e-image-c0c23a03bc9c that calls openai/gpt-image-1-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  106. ✓test_input_items_list_the_stored_prompt TestStoredResponseLifecycletest_responses_retrieve_e2e.py›
    1. Add a deployment named e2e-resp-items-b472fc79e64c that calls openai/gpt-5.5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  107. ✓test_invalid_chunking_returns_error TestVectorStorestest_vector_stores_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  108. ✓test_invalid_max_completion_tokens_return_client_errors TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-7f94a15474b9 that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  109. ✓test_invalid_n_returns_client_error TestImageGenerationtest_image_generation_e2e.py›
    1. Add a deployment named e2e-image-0bf839b37824 that calls openai/gpt-image-1-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  110. ✓test_invalid_role_returns_client_error TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-21baafe714dd that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  111. ✓test_invalid_size_returns_client_error TestImageGenerationtest_image_generation_e2e.py›
    1. Add a deployment named e2e-image-1a76505b7ca4 that calls openai/gpt-image-1-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  112. ✓test_invalid_temperatures_return_client_errors TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-419cfc499651 that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  113. ✓test_invoke_invalid_temperature_returns_client_error TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-19f3a306f5bf that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  114. ✓test_invoke_missing_max_tokens_returns_error TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-aafadf20b0c8 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  115. ✓test_invoke_missing_messages_returns_error TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-8d52bc9bf70b that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  116. ✓test_invoke_regional_id_completes TestBedrockInvokeRegionalModelIdstest_bedrock_provider_matrix_e2e.py›
    1. Add a deployment named e2e-bedrock-invoke-11144900a84b that calls bedrock/invoke/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-bedrock-invoke-11144900a84b
    View test on GitHub ↗
  117. ✓test_invoke_regional_id_streams TestBedrockInvokeRegionalModelIdstest_bedrock_provider_matrix_e2e.py›
    1. Add a deployment named e2e-bedrock-invoke-stream-66256498fb61 that calls bedrock/invoke/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-bedrock-invoke-stream-66256498fb61
    View test on GitHub ↗
  118. ✓test_invoke_returns_message TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-898de35301d9 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  119. ✓test_invoke_stream_returns_chunks TestBedrockNativetest_bedrock_native_e2e.py›
    1. Add a deployment named e2e-bedrock-native-8b5bb988c622 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  120. ✓test_messages_logs_cost_matching_the_response_header TestAnthropicMessagestest_messages_e2e.py›
    1. Add a deployment named e2e-messages-cost-261e021d24ce that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  121. ✓test_messages_output_format_returns_schema_json TestAnthropicMessagestest_messages_e2e.py›
    1. Add a deployment named e2e-messages-7b7338616ee3 that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  122. ✓test_messages_returns_completion TestAnthropicMessagestest_messages_e2e.py›
    1. Add a deployment named e2e-messages-608dea21efc5 that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  123. ✓test_messages_streams_completion TestAnthropicMessagestest_messages_e2e.py›
    1. Add a deployment named e2e-messages-f18911e92f7c that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  124. ✓test_messages_tool_use TestAnthropicMessagestest_messages_e2e.py›
    1. Add a deployment named e2e-messages-f03b61654f2b that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  125. ✓test_missing_file_returns_error TestAudioTranscriptionstest_audio_transcriptions_e2e.py›
    1. Add a deployment named e2e-transcribe-37f440cb7bef that calls openai/gpt-4o-mini-transcribe
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  126. ✓test_missing_input_returns_error TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-missin-5002be9a4fc3 that calls openai/text-embedding-3-small
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  127. ✓test_missing_messages_returns_error TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-5b6b39790b25 that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  128. ✓test_missing_model_returns_client_error TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-898a79027f89 that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  129. ✓test_missing_model_returns_client_error TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  130. ✓test_missing_model_returns_client_error TestResponsestest_responses_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  131. ✓test_missing_model_returns_error TestAudioTranscriptionstest_audio_transcriptions_e2e.py›
    1. Add a deployment named e2e-transcribe-e21053f9e3bc that calls openai/gpt-4o-mini-transcribe
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  132. ✓test_missing_model_returns_error TestAnthropicMessagestest_messages_e2e.py›
    1. Add a deployment named e2e-messages-eeefcf08df78 that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  133. ✓test_mistral TestChatToolResultRoundTriptest_chat_tool_round_trip_e2e.py›
    1. Add a deployment named e2e-chat-tool-loop-34b51b2126cd that calls mistral/mistral-medium-3.5
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-chat-tool-loop-34b51b2126cd
    View test on GitHub ↗
  134. ✓test_mistral_embeddings_returns_vector TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-mistral-8105ae38939a that calls mistral/mistral-embed
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  135. ✓test_model_info_reports_custom_pricing TestCustomPricingtest_custom_pricing_e2e.py›
    1. Add a deployment named custom-priced-flash-9060c01102be that calls gemini/gemini-2.5-flash
    2. List the deployments from /model/info
    View test on GitHub ↗
  136. ✓test_moderations_flags_violent_content TestModerationstest_moderations_e2e.py›
    1. Add a deployment named e2e-moderation-e5a4e83933b8 that calls openai/omni-moderation-latest
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  137. ✓test_moderations_passes_benign_content TestModerationstest_moderations_e2e.py›
    1. Add a deployment named e2e-moderation-b0f077a26eee that calls openai/omni-moderation-latest
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  138. ✓test_multi_turn_history_is_honored TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-a7f85f4374ad that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-chat-sec-a7f85f4374ad
    View test on GitHub ↗
  139. ✓test_nova_sonic_response_create_completes TestNovaSonicRealtimerealtime/test_realtime_bedrock_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-nova-sonic-6b8c6973b1a9 that calls bedrock/amazon.nova-2-sonic-v1:0
    View test on GitHub ↗
  140. ✓test_nova_stream_relays_text_usage_and_stop TestBedrockMessagestest_messages_bedrock_e2e.py›
    1. Add a deployment named e2e-messages-bedrock-132e20dbf105 that calls bedrock/us.amazon.nova-2-lite-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  141. ✓test_ocr_reads_the_document_and_bills_its_pages[azure-ai-image] TestOcrDocumentContenttest_ocr_rust_e2e.py›
    1. Add a deployment named ocr-content-azure-ai-image-54faea45265a that calls azure_ai/mistral-document-ai-2512
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  142. ✓test_ocr_reads_the_document_and_bills_its_pages[cohere-image] TestOcrDocumentContenttest_ocr_rust_e2e.py›
    1. Add a deployment named ocr-content-cohere-image-b2da5d6aa718 that calls cohere/parse-v5.0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  143. ✓test_ocr_reads_the_document_and_bills_its_pages[mistral-image] TestOcrDocumentContenttest_ocr_rust_e2e.py›
    1. Add a deployment named ocr-content-mistral-image-862d488d0526 that calls mistral/mistral-ocr-latest
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  144. ✓test_ocr_reads_the_document_and_bills_its_pages[mistral-pdf] TestOcrDocumentContenttest_ocr_rust_e2e.py›
    1. Add a deployment named ocr-content-mistral-pdf-7748d7f7da87 that calls mistral/mistral-ocr-latest
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  145. ✓test_ocr_reads_the_document_and_bills_its_pages[vertex-mistral-image] TestOcrDocumentContenttest_ocr_rust_e2e.py›
    1. Add a deployment named ocr-content-vertex-mistral-image-97d6a24bfe08 that calls vertex_ai/mistral-ocr-2505
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  146. ✓test_openai_chat_logs_cost TestOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-openai-cost-a5e1cba746bd that calls openai/gpt-5.6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-openai-cost-a5e1cba746bd
    4. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  147. ✓test_openai_chat_reasoning_reports_reasoning_tokens TestOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-openai-reasoning-11fb629b2dfe that calls openai/gpt-5.6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-openai-reasoning-11fb629b2dfe
    View test on GitHub ↗
  148. ✓test_openai_chat_returns_tool_call TestOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-openai-tool-4b829caa0a71 that calls openai/gpt-5.6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-openai-tool-4b829caa0a71
    View test on GitHub ↗
  149. ✓test_openai_chat_streams_real_content TestOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-openai-chat-4f445028132f that calls openai/gpt-5.6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-openai-chat-4f445028132f
    View test on GitHub ↗
  150. ✓test_openai_chat_streams_tool_call TestOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-openai-tool-stream-40127d9b604e that calls openai/gpt-5.6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-openai-tool-stream-40127d9b604e
    View test on GitHub ↗
  151. ✓test_openai_chat_structured_output_conforms_to_schema TestOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-openai-schema-f2d8a51956df that calls openai/gpt-5.6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-openai-schema-f2d8a51956df
    View test on GitHub ↗
  152. ✓test_openai_chat_vision_describes_image TestOpenAIChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-openai-vision-69e33ecc2b04 that calls openai/gpt-4o
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-openai-vision-69e33ecc2b04
    View test on GitHub ↗
  153. ✓test_openai_family_model_id_completes_with_max_tokens TestBedrockOpenAIFamilyDefaultRoutetest_bedrock_provider_matrix_e2e.py›
    1. Add a deployment named e2e-bedrock-openai-family-bcc88b6cb113 that calls bedrock/global.openai.gpt-6-sol
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-bedrock-openai-family-bcc88b6cb113
    View test on GitHub ↗
  154. ✓test_openai_prefix_chat_returns_completion_and_logs_its_cost TestOpenAIProviderPrefixChattest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  155. ✓test_openai_prompt_caching_reads_cache TestCacheControltest_cache_control.py›
    1. Add a deployment named e2e-openai-cache-3c3af99afcc2 that calls openai/gpt-5.6
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-openai-cache-3c3af99afcc2
    View test on GitHub ↗
  156. ✓test_openai_service_tier_is_echoed TestServiceTiertest_provider_features_e2e.py›
    1. Add a deployment named e2e-service-tier-fd0d617d293a that calls openai/gpt-5.5
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-service-tier-fd0d617d293a
    View test on GitHub ↗
  157. ✓test_output_format_returns_schema_json TestAzureFoundryMessagestest_messages_azure_foundry_e2e.py›
    1. Add a deployment named e2e-azure-foundry-messages-8c94212e3125 that calls azure_ai/claude-haiku-4-5
    2. Generate a virtual key with models: e2e-azure-foundry-messages-8c94212e3125 and user id: e2e-test-user
    View test on GitHub ↗
  158. ✓test_passthrough_denies_model_outside_key_allowlist TestPassthroughModelAllowlisttest_passthrough_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash
    View test on GitHub ↗
  159. ✓test_passthrough_prefix_lists_batches_from_openai TestOpenAIPassthroughPrefixtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  160. ✓test_passthrough_prefix_uploads_a_file_to_openai TestOpenAIPassthroughPrefixtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  161. ✓test_pipecat_tool_smoke[azure]realtime/test_realtime_pipecat_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  162. ✓test_pipecat_tool_smoke[gemini]realtime/test_realtime_pipecat_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  163. ✓test_pipecat_tool_smoke[openai]realtime/test_realtime_pipecat_e2e.py›
    1. Add a deployment named openai-realtime-8902105410cc that calls openai/gpt-realtime-2
    2. Add a deployment named azure-realtime-5504f9e77d69 that calls azure/gpt-realtime
    3. Add a deployment named gemini-realtime-b45446d3b824 that calls gemini/gemini-3.1-flash-live-preview
    4. Add a deployment named vertex-realtime-3c81be151232 that calls vertex_ai/gemini-live-2.5-flash-native-audio
    5. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  164. ✓test_plain_call_returns_a_message TestSailMessagestest_sail_e2e.py›
    1. Add a deployment named e2e-sail-4509bd8f2f66 that calls sail/zai-org/GLM-5.3
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  165. ✓test_polly_generative_voice_returns_mp3 TestAwsPollySpeechtest_audio_speech_e2e.py›
    1. Add a deployment named e2e-speech-polly-c856f1e14d3e that calls aws_polly/generative
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  166. ✓test_realtime_upgrade_reaches_openai_through_the_passthrough_prefix TestOpenAIPassthroughWebsockettest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  167. ✓test_reasoner_returns_reasoning_by_default TestDeepSeekReasoningDisabletest_deepseek_reasoning_e2e.py›
    1. Add a deployment named e2e-deepseek-reasoner-5135496db6af that calls deepseek/deepseek-v4-pro
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-deepseek-reasoner-5135496db6af
    View test on GitHub ↗
  168. ✓test_reasoning_effort_none_disables_reasoning TestDeepSeekReasoningDisabletest_deepseek_reasoning_e2e.py›
    1. Add a deployment named e2e-deepseek-reasoner-c3aea5b06059 that calls deepseek/deepseek-v4-pro
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-deepseek-reasoner-c3aea5b06059
    View test on GitHub ↗
  169. ✓test_reasoning_effort_none_reaches_together TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-f4d5290fcab5 that calls together_ai/Qwen/Qwen3.5-9B
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-together-f4d5290fcab5
    View test on GitHub ↗
  170. ✓test_reasoning_items_replay_into_structured_output_after_tool_call TestResponsesOpenAIHostedFeaturestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-reasoning-a653f41edc2d that calls openai/gpt-5.4-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  171. ✓test_reasoning_streams_as_reasoning_content_deltas TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-3ad484a500a4 that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-together-3ad484a500a4
    View test on GitHub ↗
  172. ✓test_reasoning_surfaces_as_reasoning_content TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Read the model cost map
    2. Add a deployment named e2e-together-ef3b4d400fca that calls together_ai/zai-org/GLM-5.3-Flash
    3. Generate a virtual key with user id: e2e-test-user
    4. Send a /chat/completions request to e2e-together-ef3b4d400fca
    View test on GitHub ↗
  173. ✓test_replayed_reasoning_content_reaches_together TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-fcf7276300d1 that calls together_ai/MiniMaxAI/MiniMax-M3
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-together-fcf7276300d1
    View test on GitHub ↗
  174. ✓test_reply_carries_assistant_text_and_usage[chat_completions-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  175. ✓test_reply_carries_assistant_text_and_usage[chat_completions-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  176. ✓test_reply_carries_assistant_text_and_usage[chat_completions-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Add a deployment named e2e-matrix-gpt-4o-mini-env_ref-b678249632b3 that calls openai/gpt-4o-mini
    2. Save the provider credential e2e-matrix-gpt-4o-mini-e4f2be3df2b7
    3. Add a deployment named e2e-matrix-gpt-4o-mini-stored_credential-e4f2be3df2b7 that calls openai/gpt-4o-mini
    4. Add a deployment named e2e-matrix-gpt-5.4-mini-env_ref-29995a4a497e that calls openai/gpt-5.4-mini
    5. Save the provider credential e2e-matrix-gpt-5.4-mini-4b4c7e293b15
    6. Add a deployment named e2e-matrix-gpt-5.4-mini-stored_credential-4b4c7e293b15 that calls openai/gpt-5.4-mini
    7. Add a deployment named e2e-matrix-claude-haiku-4-5-env_ref-767ccfa95ee5 that calls anthropic/claude-haiku-4-5
    8. Save the provider credential e2e-matrix-claude-haiku-4-5-540800d74bba
    9. Add a deployment named e2e-matrix-claude-haiku-4-5-stored_credential-540800d74bba that calls anthropic/claude-haiku-4-5
    10. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  177. ✓test_reply_carries_assistant_text_and_usage[chat_completions-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  178. ✓test_reply_carries_assistant_text_and_usage[chat_completions-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  179. ✓test_reply_carries_assistant_text_and_usage[chat_completions-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  180. ✓test_reply_carries_assistant_text_and_usage[messages-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  181. ✓test_reply_carries_assistant_text_and_usage[messages-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  182. ✓test_reply_carries_assistant_text_and_usage[messages-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  183. ✓test_reply_carries_assistant_text_and_usage[messages-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  184. ✓test_reply_carries_assistant_text_and_usage[messages-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  185. ✓test_reply_carries_assistant_text_and_usage[messages-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  186. ✓test_reply_carries_assistant_text_and_usage[responses-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  187. ✓test_reply_carries_assistant_text_and_usage[responses-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  188. ✓test_reply_carries_assistant_text_and_usage[responses-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  189. ✓test_reply_carries_assistant_text_and_usage[responses-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  190. ✓test_reply_carries_assistant_text_and_usage[responses-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  191. ✓test_reply_carries_assistant_text_and_usage[responses-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  192. ✓test_required_tool_arguments_and_correlated_result[nonstream] TestOpenAIMessagesToolContinuationtest_messages_e2e.py›
    1. Add a deployment named e2e-bridge-tool-32d4eab2495d that calls openai/gpt-5.6
    2. Generate a virtual key with models: e2e-bridge-tool-32d4eab2495d and user id: e2e-test-user
    View test on GitHub ↗
  193. ✓test_required_tool_arguments_and_correlated_result[stream] TestOpenAIMessagesToolContinuationtest_messages_e2e.py›
    1. Add a deployment named e2e-bridge-tool-30b1b3386873 that calls openai/gpt-5.6
    2. Generate a virtual key with models: e2e-bridge-tool-30b1b3386873 and user id: e2e-test-user
    View test on GitHub ↗
  194. ✓test_rerank_scores_top_n TestReranktest_rerank_e2e.py›
    1. Add a deployment named e2e-rerank-913a97b9bd84 that calls cohere/rerank-v3.5
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /v1/rerank request to e2e-rerank-913a97b9bd84 for "What is the capital of the United States?"
    View test on GitHub ↗
  195. ✓test_response_format_json_schema_shapes_the_reply TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-198af9712cd4 that calls together_ai/Qwen/Qwen3.5-9B
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-together-198af9712cd4
    View test on GitHub ↗
  196. ✓test_responses_anthropic_returns_completion TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-d902d5d3abff that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  197. ✓test_responses_anthropic_returns_function_call TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-65ae28a59a1a that calls anthropic/claude-haiku-4-5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  198. ✓test_responses_azure_openai_returns_completion TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-azure-openai-523a57f07cab that calls azure/gpt-5.4-nano
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  199. ✓test_responses_azure_openai_returns_function_call TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-azure-openai-tool-a8528bc20f38 that calls azure/gpt-5.4-nano
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  200. ✓test_responses_bedrock_returns_completion TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-ed8baff5b3d6 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  201. ✓test_responses_bedrock_returns_function_call TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-c4750647c114 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  202. ✓test_responses_logs_cost TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-d130f8d112c8 that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  203. ✓test_responses_returns_completion TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-2db974388a82 that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  204. ✓test_responses_returns_function_call TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-c871fd41210c that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  205. ✓test_responses_streaming_returns_completion TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-0012d376bb5b that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  206. ✓test_responses_upgrade_is_accepted_on_the_openai_prefix TestOpenAIPassthroughWebsockettest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  207. ✓test_responses_vertex_returns_completion TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-vertex-a3552237add1 that calls vertex_ai/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  208. ✓test_responses_vertex_returns_function_call TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-vertex-tool-71686eb76f73 that calls vertex_ai/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  209. ✓test_responses_vision_describes_image TestResponsestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-94ef9a27adf1 that calls openai/gpt-4o
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  210. ✓test_retrieve_invalid_batch_id_returns_error TestFilesBatchesContracttest_files_batches_contract_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  211. ✓test_rust_ocr_response[azure-ai] TestRustOcrGatewaytest_ocr_rust_e2e.py›
    1. Add a deployment named rust-ocr-azure-ai-f251b17a534d that calls azure_ai/mistral-document-ai-2512
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /v1/ocr request to rust-ocr-azure-ai-f251b17a534d
    View test on GitHub ↗
  212. ✓test_rust_ocr_response[azure-document-intelligence] TestRustOcrGatewaytest_ocr_rust_e2e.py›
    1. Add a deployment named rust-ocr-azure-document-intelligence-2b13f451fb63 that calls azure_ai/doc-intelligence/prebuilt-layout
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /v1/ocr request to rust-ocr-azure-document-intelligence-2b13f451fb63
    View test on GitHub ↗
  213. ✓test_rust_ocr_response[mistral] TestRustOcrGatewaytest_ocr_rust_e2e.py›
    1. Add a deployment named rust-ocr-mistral-2255b056032d that calls mistral/mistral-ocr-latest
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /v1/ocr request to rust-ocr-mistral-2255b056032d
    View test on GitHub ↗
  214. ✓test_rust_ocr_response[vertex-mistral] TestRustOcrGatewaytest_ocr_rust_e2e.py›
    1. Add a deployment named rust-ocr-vertex-mistral-290d3d668f61 that calls vertex_ai/mistral-ocr-2505
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /v1/ocr request to rust-ocr-vertex-mistral-290d3d668f61
    View test on GitHub ↗
  215. ✓test_service_account_key_reads_container_file_by_native_id TestAzureContainerFilestest_containers_e2e.py›
    1. Add a deployment named e2e-containers-decoy-043b6a4c92eb that calls azure/gpt-5.4-nano
    2. Add a deployment named e2e-containers-043b6a4c92eb that calls azure/gpt-5.4-nano
    View test on GitHub ↗
  216. ✓test_service_account_key_reads_container_file_created_by_a_streamed_response TestAzureContainerFilestest_containers_e2e.py›
    1. Add a deployment named e2e-containers-decoy-58f90462483b that calls azure/gpt-5.4-nano
    2. Add a deployment named e2e-containers-58f90462483b that calls azure/gpt-5.4-nano
    View test on GitHub ↗
  217. ✓test_service_tier_bills_the_matching_completion_window[auto-base] TestSailChatCompletionstest_sail_e2e.py›
    1. Add a deployment named e2e-sail-6f5eff1e47f1 that calls sail/zai-org/GLM-5.3
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  218. ✓test_service_tier_bills_the_matching_completion_window[balanced-balanced] TestSailChatCompletionstest_sail_e2e.py›
    1. Add a deployment named e2e-sail-998a5ce3f346 that calls sail/zai-org/GLM-5.3
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  219. ✓test_shell_tool_stream_surfaces_shell_call_and_its_output TestResponsesOpenAIHostedFeaturestest_responses_e2e.py›
    1. Add a deployment named e2e-responses-shell-f42e88dfb9d1 that calls openai/gpt-5.5
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  220. ✓test_stream_delivers_text_usage_and_a_terminal_event[chat_completions-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  221. ✓test_stream_delivers_text_usage_and_a_terminal_event[chat_completions-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  222. ✓test_stream_delivers_text_usage_and_a_terminal_event[chat_completions-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  223. ✓test_stream_delivers_text_usage_and_a_terminal_event[chat_completions-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  224. ✓test_stream_delivers_text_usage_and_a_terminal_event[chat_completions-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  225. ✓test_stream_delivers_text_usage_and_a_terminal_event[chat_completions-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  226. ✓test_stream_delivers_text_usage_and_a_terminal_event[messages-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  227. ✓test_stream_delivers_text_usage_and_a_terminal_event[messages-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  228. ✓test_stream_delivers_text_usage_and_a_terminal_event[messages-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  229. ✓test_stream_delivers_text_usage_and_a_terminal_event[messages-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  230. ✓test_stream_delivers_text_usage_and_a_terminal_event[messages-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  231. ✓test_stream_delivers_text_usage_and_a_terminal_event[messages-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  232. ✓test_stream_delivers_text_usage_and_a_terminal_event[responses-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  233. ✓test_stream_delivers_text_usage_and_a_terminal_event[responses-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  234. ✓test_stream_delivers_text_usage_and_a_terminal_event[responses-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  235. ✓test_stream_delivers_text_usage_and_a_terminal_event[responses-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  236. ✓test_stream_delivers_text_usage_and_a_terminal_event[responses-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  237. ✓test_stream_delivers_text_usage_and_a_terminal_event[responses-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  238. ✓test_stream_generate_content_frames_sse_the_way_google_sdks_expect TestGoogleNativeGenerateContenttest_google_native_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-google-native-a66df7f0d395 that calls gemini/gemini-2.5-flash
    View test on GitHub ↗
  239. ✓test_streamed_responses_call_logs_its_cost TestOpenAIPassthroughSpendtest_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  240. ✓test_streams_text_deltas TestTogetherMessagestest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-b324a400d790 that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /v1/messages request to e2e-together-b324a400d790
    View test on GitHub ↗
  241. ✓test_success_response_matches_chat_completion_contract TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-6c93a3718bbe that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  242. ✓test_temperature_boundaries_succeed TestChatCompletionsContracttest_chat_completions_contract_e2e.py›
    1. Add a deployment named e2e-chat-sec-c649bd28ccce that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  243. ✓test_text_completion_returns_text TestCompletionsEndpointtest_completions_endpoint_e2e.py›
    1. Add a deployment named e2e-completions-3edf471f4d0a that calls openai/gpt-5.4-nano
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  244. ✓test_text_conversation[azure]realtime/test_realtime_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  245. ✓test_text_conversation[gemini]realtime/test_realtime_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  246. ✓test_text_conversation[openai]realtime/test_realtime_e2e.py›
    1. Add a deployment named openai-realtime-b51981bcb497 that calls openai/gpt-realtime-2
    2. Add a deployment named azure-realtime-2c2ea7dbeac8 that calls azure/gpt-realtime
    3. Add a deployment named gemini-realtime-41f8b89fffcf that calls gemini/gemini-3.1-flash-live-preview
    4. Add a deployment named vertex-realtime-0e222e99c2c3 that calls vertex_ai/gemini-live-2.5-flash-native-audio
    5. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  247. ✓test_text_conversation[vertex_ai]realtime/test_realtime_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  248. ✓test_thinking_disabled_disables_reasoning TestDeepSeekReasoningDisabletest_deepseek_reasoning_e2e.py›
    1. Add a deployment named e2e-deepseek-reasoner-112d1386d078 that calls deepseek/deepseek-v4-pro
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-deepseek-reasoner-112d1386d078
    View test on GitHub ↗
  249. ✓test_tool_call_is_returned TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-12ebba7ee668 that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-together-12ebba7ee668
    View test on GitHub ↗
  250. ✓test_tool_call_is_returned_named_and_addressable[chat_completions-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  251. ✓test_tool_call_is_returned_named_and_addressable[chat_completions-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  252. ✓test_tool_call_is_returned_named_and_addressable[chat_completions-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  253. ✓test_tool_call_is_returned_named_and_addressable[chat_completions-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  254. ✓test_tool_call_is_returned_named_and_addressable[chat_completions-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  255. ✓test_tool_call_is_returned_named_and_addressable[chat_completions-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  256. ✓test_tool_call_is_returned_named_and_addressable[messages-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  257. ✓test_tool_call_is_returned_named_and_addressable[messages-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  258. ✓test_tool_call_is_returned_named_and_addressable[messages-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  259. ✓test_tool_call_is_returned_named_and_addressable[messages-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  260. ✓test_tool_call_is_returned_named_and_addressable[messages-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  261. ✓test_tool_call_is_returned_named_and_addressable[messages-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  262. ✓test_tool_call_is_returned_named_and_addressable[responses-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  263. ✓test_tool_call_is_returned_named_and_addressable[responses-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  264. ✓test_tool_call_is_returned_named_and_addressable[responses-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  265. ✓test_tool_call_is_returned_named_and_addressable[responses-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  266. ✓test_tool_call_is_returned_named_and_addressable[responses-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  267. ✓test_tool_call_is_returned_named_and_addressable[responses-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  268. ✓test_tool_call_is_streamed TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-0106781e0145 that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-together-0106781e0145
    View test on GitHub ↗
  269. ✓test_tool_call_round_trip[azure]realtime/test_realtime_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  270. ✓test_tool_call_round_trip[gemini]realtime/test_realtime_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  271. ✓test_tool_call_round_trip[openai]realtime/test_realtime_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  272. ✓test_tool_call_round_trip[vertex_ai]realtime/test_realtime_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  273. ✓test_tool_result_round_trip TestTogetherChatCompletionstest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-1d58b657bdfb that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-together-1d58b657bdfb
    View test on GitHub ↗
  274. ✓test_tool_result_round_trip TestTogetherMessagestest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-d92b1b0b870f that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /v1/messages request to e2e-together-d92b1b0b870f
    View test on GitHub ↗
  275. ✓test_tool_result_round_trip_reaches_the_model[chat_completions-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  276. ✓test_tool_result_round_trip_reaches_the_model[chat_completions-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  277. ✓test_tool_result_round_trip_reaches_the_model[chat_completions-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  278. ✓test_tool_result_round_trip_reaches_the_model[chat_completions-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  279. ✓test_tool_result_round_trip_reaches_the_model[chat_completions-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  280. ✓test_tool_result_round_trip_reaches_the_model[chat_completions-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  281. ✓test_tool_result_round_trip_reaches_the_model[messages-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  282. ✓test_tool_result_round_trip_reaches_the_model[messages-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  283. ✓test_tool_result_round_trip_reaches_the_model[messages-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  284. ✓test_tool_result_round_trip_reaches_the_model[messages-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  285. ✓test_tool_result_round_trip_reaches_the_model[messages-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  286. ✓test_tool_result_round_trip_reaches_the_model[messages-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  287. ✓test_tool_result_round_trip_reaches_the_model[responses-claude-haiku-4-5-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  288. ✓test_tool_result_round_trip_reaches_the_model[responses-claude-haiku-4-5-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  289. ✓test_tool_result_round_trip_reaches_the_model[responses-gpt-4o-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  290. ✓test_tool_result_round_trip_reaches_the_model[responses-gpt-4o-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  291. ✓test_tool_result_round_trip_reaches_the_model[responses-gpt-5.4-mini-env_ref] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  292. ✓test_tool_result_round_trip_reaches_the_model[responses-gpt-5.4-mini-stored_credential] TestConversationalMatrixtest_conversational_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  293. ✓test_tool_use_block_is_returned TestTogetherMessagestest_together_ai_e2e.py›
    1. Add a deployment named e2e-together-f9a6efa0bfdf that calls together_ai/zai-org/GLM-5.3-Flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /v1/messages request to e2e-together-f9a6efa0bfdf
    View test on GitHub ↗
  294. ✓test_tool_use_nonstream TestAzureFoundryMessagestest_messages_azure_foundry_e2e.py›
    1. Add a deployment named e2e-azure-foundry-messages-b8e02139a86a that calls azure_ai/claude-haiku-4-5
    2. Generate a virtual key with models: e2e-azure-foundry-messages-b8e02139a86a and user id: e2e-test-user
    View test on GitHub ↗
  295. ✓test_tool_use_stream TestAzureFoundryMessagestest_messages_azure_foundry_e2e.py›
    1. Add a deployment named e2e-azure-foundry-messages-9feda6bda562 that calls azure_ai/claude-haiku-4-5
    2. Generate a virtual key with models: e2e-azure-foundry-messages-9feda6bda562 and user id: e2e-test-user
    View test on GitHub ↗
  296. ✓test_unflagged_model_converts_system_reminder_and_succeeds TestAnthropicChatMidConversationSystemtest_chat_mid_conversation_system_e2e.py›
    1. Add a deployment named e2e-chat-midsys-8cb683ca59c9 that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with models: e2e-chat-midsys-8cb683ca59c9 and user id: e2e-test-user
    View test on GitHub ↗
  297. ✓test_unflagged_model_converts_system_reminder_and_succeeds TestBedrockInvokeChatMidConversationSystemtest_chat_mid_conversation_system_e2e.py›
    1. Add a deployment named e2e-chat-midsys-aec6053dccd0 that calls bedrock/invoke/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with models: e2e-chat-midsys-aec6053dccd0 and user id: e2e-test-user
    View test on GitHub ↗
  298. ✓test_unflagged_model_converts_system_reminder_and_succeeds TestBedrockInvokeMidConversationSystemtest_messages_mid_conversation_system_e2e.py›
    1. Add a deployment named e2e-midsys-3c86414386b4 that calls bedrock/invoke/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with models: e2e-midsys-3c86414386b4 and user id: e2e-test-user
    View test on GitHub ↗
  299. ✓test_unflagged_model_converts_system_reminder_and_succeeds TestAzureFoundryMidConversationSystemtest_messages_mid_conversation_system_native_providers_e2e.py›
    1. Add a deployment named e2e-midsys-c965e122b2bf that calls azure_ai/claude-opus-4-7
    2. Generate a virtual key with models: e2e-midsys-c965e122b2bf and user id: e2e-test-user
    View test on GitHub ↗
  300. ✓test_unflagged_model_converts_system_reminder_and_succeeds TestVertexMidConversationSystemtest_messages_mid_conversation_system_native_providers_e2e.py›
    1. Add a deployment named e2e-midsys-37247811c93a that calls vertex_ai/claude-sonnet-4-6
    2. Generate a virtual key with models: e2e-midsys-37247811c93a and user id: e2e-test-user
    View test on GitHub ↗
  301. ✓test_unknown_service_tier_is_dropped_and_billed_asap[5] TestSailChatCompletionstest_sail_e2e.py›
    1. Add a deployment named e2e-sail-fade3afa57bd that calls sail/zai-org/GLM-5.3
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  302. ✓test_unknown_service_tier_is_dropped_and_billed_asap[bogus] TestSailChatCompletionstest_sail_e2e.py›
    1. Add a deployment named e2e-sail-1569bb7f5567 that calls sail/zai-org/GLM-5.3
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  303. ✓test_upload_without_purpose_returns_error TestFilesBatchesContracttest_files_batches_contract_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  304. ✓test_upstream_handshake_refusal_is_an_error_event_and_policy_close[azure-bad-key]realtime/test_realtime_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named azure-realtime-refused-8a3934c2883f that calls azure/gpt-realtime
    View test on GitHub ↗
  305. ✓test_verbose_json_returns_word_timestamps TestWhisperTranscriptionFormatstest_audio_transcriptions_e2e.py›
    1. Add a deployment named e2e-whisper-verbose-8e6ad6c7cbf5 that calls openai/whisper-1
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  306. ✓test_vertex_chat_returns_content TestVertexChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-vertex-chat-6a10588841b7 that calls vertex_ai/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-vertex-chat-6a10588841b7
    View test on GitHub ↗
  307. ✓test_vertex_chat_returns_tool_call TestVertexChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-vertex-tool-3f14c96bf7f9 that calls vertex_ai/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-vertex-tool-3f14c96bf7f9
    View test on GitHub ↗
  308. ✓test_vertex_chat_streams_real_content TestVertexChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-vertex-stream-0e7ecd3e42af that calls vertex_ai/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a streaming /chat/completions request to e2e-vertex-stream-0e7ecd3e42af
    View test on GitHub ↗
  309. ✓test_vertex_chat_vision_describes_image TestVertexChatCompletionstest_chat_completions_regression_e2e.py›
    1. Add a deployment named e2e-vertex-vision-bf271e80138b that calls vertex_ai/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-vertex-vision-bf271e80138b
    View test on GitHub ↗
  310. ✓test_vertex_embeddings_honor_requested_dimensions TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-vertex-dims-009543add863 that calls vertex_ai/text-embedding-005
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  311. ✓test_vertex_embeddings_returns_vector TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-vertex-ec2fdd212c2c that calls vertex_ai/text-embedding-005
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  312. ✓test_vertex_multimodal_embeddings_honor_dimensions_and_are_costed TestEmbeddingsEndpointtest_embeddings_endpoint_e2e.py›
    1. Add a deployment named e2e-embeddings-vertex-mm-6b8a80d9ae53 that calls vertex_ai/multimodalembedding@001
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  313. ✓test_vertex_passthrough_via_managed_model_logs_cost TestVertexPassthroughSpendTrackingtest_vertex_passthrough_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  314. ✓test_vertex_prompt_caching_reads_cache TestCacheControltest_cache_control.py›
    1. Add a deployment named e2e-vertex-cache-9b32aec89255 that calls vertex_ai/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    3. Wait for the request's spend log in /spend/logs
    4. List the deployments from /model/info
    View test on GitHub ↗
  315. ✓test_vtt_format_returns_webvtt_transcript TestWhisperTranscriptionFormatstest_audio_transcriptions_e2e.py›
    1. Add a deployment named e2e-whisper-vtt-361aeb15c890 that calls openai/whisper-1
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  316. −test_empty_input_returns_error TestAudioSpeechtest_audio_speech_e2e.py›

    No steps recorded

    stage red: product gap, /v1/audio/speech 500s on empty input instead of surfacing the provider 4xx

    View test on GitHub ↗
  317. −test_flagged_model_keeps_prompt_cache_across_system_reminder TestBedrockInvokeMidConversationSystemtest_messages_mid_conversation_system_e2e.py›

    No steps recorded

    LIT-4873: a mid-conversation role='system' reminder invalidates the prompt cache on the vertex_ai / azure_ai / bedrock_invoke Messages paths, while the same request preserves it both direct to Anthropic and through litellm's first-party anthropic path. Product bug, not a test defect: the assertion here is correct and must be restored unchanged with the fix

    View test on GitHub ↗
  318. −test_flagged_model_keeps_prompt_cache_across_system_reminder TestAzureFoundryMidConversationSystemtest_messages_mid_conversation_system_native_providers_e2e.py›

    No steps recorded

    LIT-4873: a mid-conversation role='system' reminder invalidates the prompt cache on the vertex_ai / azure_ai / bedrock_invoke Messages paths, while the same request preserves it both direct to Anthropic and through litellm's first-party anthropic path. Product bug, not a test defect: the assertion here is correct and must be restored unchanged with the fix

    View test on GitHub ↗
  319. −test_flagged_model_keeps_prompt_cache_across_system_reminder TestVertexMidConversationSystemtest_messages_mid_conversation_system_native_providers_e2e.py›

    No steps recorded

    LIT-4873: a mid-conversation role='system' reminder invalidates the prompt cache on the vertex_ai / azure_ai / bedrock_invoke Messages paths, while the same request preserves it both direct to Anthropic and through litellm's first-party anthropic path. Product bug, not a test defect: the assertion here is correct and must be restored unchanged with the fix

    View test on GitHub ↗
  320. −test_gemini_passthrough_returns_the_same_header_contract_as_the_managed_routetest_passthrough_e2e.py›

    No steps recorded

    stage red: product gap, native passthrough returns no x-litellm-response-cost or x-ratelimit-* headers

    View test on GitHub ↗
  321. −test_hosted_vllm_chat_returns_content TestHostedVllmChattest_chat_completions_regression_e2e.py›

    No steps recorded

    set HOSTED_VLLM_API_BASE (the live vLLM server this deployment targets)

    View test on GitHub ↗
  322. −test_invalid_response_id_returns_error TestResponsesRetrievetest_responses_retrieve_e2e.py›

    No steps recorded

    stage red: product gap (LIT-5447), retrieving an unknown response id returns 400 (model=None) instead of 404

    View test on GitHub ↗
  323. −test_invalid_voice_returns_error TestAudioSpeechtest_audio_speech_e2e.py›

    No steps recorded

    stage red: product gap, /v1/audio/speech 500s on invalid voice instead of surfacing the provider 4xx

    View test on GitHub ↗
  324. −test_missing_document_returns_error TestRustOcrGatewaytest_ocr_rust_e2e.py›

    No steps recorded

    stage red: product gap, /v1/ocr 500s (aocr TypeError) on missing document instead of 400

    View test on GitHub ↗
  325. −test_missing_input_returns_error TestAudioSpeechtest_audio_speech_e2e.py›

    No steps recorded

    stage red: product gap, /v1/audio/speech 500s on missing input instead of 400

    View test on GitHub ↗
  326. −test_missing_input_returns_error TestModerationstest_moderations_e2e.py›

    No steps recorded

    stage red: product gap, /v1/moderations 500s (KeyError 'input') on missing input instead of 400

    View test on GitHub ↗
  327. −test_missing_input_returns_error TestResponsestest_responses_e2e.py›

    No steps recorded

    stage red: product gap, /v1/responses 500s (aresponses TypeError) on missing input instead of 400

    View test on GitHub ↗
  328. −test_missing_max_tokens_returns_error TestAnthropicMessagestest_messages_e2e.py›

    No steps recorded

    stage red: product gap, /v1/messages 500s (anthropic_messages TypeError) on missing max_tokens instead of 400

    View test on GitHub ↗
  329. −test_missing_messages_returns_error TestAnthropicMessagestest_messages_e2e.py›

    No steps recorded

    stage red: product gap, /v1/messages 500s (anthropic_messages TypeError) on missing messages instead of 400

    View test on GitHub ↗
  330. −test_missing_model_returns_error TestAudioSpeechtest_audio_speech_e2e.py›

    No steps recorded

    stage red: product gap, /v1/audio/speech 500s on missing model instead of 400

    View test on GitHub ↗
  331. −test_missing_prompt_returns_error TestImageGenerationtest_image_generation_e2e.py›

    No steps recorded

    stage red: product gap, /v1/images/generations 500s (aimage_generation TypeError) on missing prompt instead of 400

    View test on GitHub ↗
  332. −test_retrieve_invalid_id_returns_error TestVectorStorestest_vector_stores_e2e.py›

    No steps recorded

    stage red: product gap, retrieving a nonexistent vector store returns 2xx with an error envelope in the body instead of 404

    View test on GitHub ↗
  333. −test_search_missing_query_returns_error TestVectorStorestest_vector_stores_e2e.py›

    No steps recorded

    stage red: product gap, vector store search 500s (asearch TypeError) on missing query instead of 400

    View test on GitHub ↗
  334. −test_static_and_x_pass_headers_reach_upstream TestPassthroughHeaderstest_passthrough_headers_e2e.py›

    No steps recorded

    stage red: ALB sends custom passthrough paths to backend pods, which lack ANTHROPIC_API_KEY, so the os.environ header forwards unresolved and Anthropic 401s; routing user-defined paths to the gateway needs a product/infra decision (fixed path namespace or dynamic ingress)

    View test on GitHub ↗
  335. −test_store_and_retrieve_by_id TestResponsesRetrievetest_responses_retrieve_e2e.py›

    No steps recorded

    stage red: product gap (LIT-5446), retrieve returns a different id than the stored response (non-idempotent response-id re-encryption)

    View test on GitHub ↗
  336. −test_vertex_partner_model_streams_openai_shaped_chunks[vertex_ai/mistral-small-2503] TestVertexPartnerChatCompletionstest_chat_completions_regression_e2e.py›

    No steps recorded

    the e2e Vertex project has no access to mistral-small-2503 (404 publisher model not found)

    View test on GitHub ↗
  337. −test_vertex_partner_model_streams_openai_shaped_chunks[vertex_ai/openai/gpt-oss-120b-maas] TestVertexPartnerChatCompletionstest_chat_completions_regression_e2e.py›

    No steps recorded

    never served by the e2e Vertex project (60s read timeout, no headers)

    View test on GitHub ↗
  338. −test_web_search_server_tool_is_served TestBedrockWebSearchServerTooltest_bedrock_web_search_server_tool_e2e.py›

    No steps recorded

    stage red: environment gap, the e2e stack neither enables the websearch_interception callback nor declares a search backend, so the request reaches bedrock's transformation and takes its by-design 400. Unskip once the ephemeral stack ships the config in this module's docstring.

    View test on GitHub ↗
  339. ✓test_a_client_built_for_another_proxy_reads_management_routes_back_from_that_proxy TestReplicasFortest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  340. ✓test_a_route_no_replica_serves_is_refused_rather_than_read_back_vacuously TestReplicasFortest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  341. ✓test_actor_key_cleanup_reports_failure_and_continues TestBoundManagementCallertest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  342. ✓test_an_exported_list_wins_over_the_base_url_rule TestParseControlPlaneReplicaUrlstest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  343. ✓test_any_other_endpoints_follow_the_base_url_rule TestStackEndpointsControlReplicastest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  344. ✓test_collapses_repeated_gateway_addresses_to_one_replica TestParseReplicaUrlstest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  345. ✓test_control_and_data_replica_readers_keep_the_caller TestSplitCallerPropagationtest_proxy_client.py›
    1. Read the key's settings back from /key/info
    2. Read /key/info on every proxy replica until it settles
    3. Read /key/info on every proxy replica until they all agree
    4. Read /v1/models on every proxy replica until they all agree
    View test on GitHub ↗
  346. ✓test_credentials_are_absent_from_binding_and_header_diagnostics TestBoundManagementCallertest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  347. ✓test_direct_delegated_and_replica_reads_keep_the_bound_caller[dashboard_session] TestBoundManagementCallertest_proxy_client.py›
    1. Read the key's settings back from /key/info
    2. Read /key/info on every proxy replica until they all agree
    3. Read /key/info on every proxy replica until it settles
    4. Read the key's settings back from /key/info
    View test on GitHub ↗
  348. ✓test_direct_delegated_and_replica_reads_keep_the_bound_caller[direct_jwt] TestBoundManagementCallertest_proxy_client.py›
    1. Read the key's settings back from /key/info
    2. Read /key/info on every proxy replica until they all agree
    3. Read /key/info on every proxy replica until it settles
    4. Read the key's settings back from /key/info
    View test on GitHub ↗
  349. ✓test_direct_delegated_and_replica_reads_keep_the_bound_caller[virtual_key] TestBoundManagementCallertest_proxy_client.py›
    1. Read the key's settings back from /key/info
    2. Read /key/info on every proxy replica until they all agree
    3. Read /key/info on every proxy replica until it settles
    4. Read the key's settings back from /key/info
    View test on GitHub ↗
  350. ✓test_each_replica_gets_its_own_full_budget TestAwaitConvergedEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  351. ✓test_expired_shaped_token_is_sent_once_without_renewal TestSplitCallerPropagationtest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  352. ✓test_explicit_override_wins_without_rebinding_or_changing_master TestBoundManagementCallertest_proxy_client.py›
    1. Read the key's settings back from /key/info
    View test on GitHub ↗
  353. ✓test_fails_on_the_replica_that_never_lists_the_model[gateway-1] TestAwaitServableEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  354. ✓test_fails_on_the_replica_that_never_lists_the_model[gateway-2] TestAwaitServableEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  355. ✓test_falls_back_to_the_data_plane_address_when_unset TestParseReplicaUrlstest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  356. ✓test_gateway_pods_behind_one_router_read_management_routes_back_from_the_router TestReplicasFortest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  357. ✓test_management_operations_send_the_selected_credential[dashboard_session-add_team_member]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  358. ✓test_management_operations_send_the_selected_credential[dashboard_session-block_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  359. ✓test_management_operations_send_the_selected_credential[dashboard_session-connection_test]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  360. ✓test_management_operations_send_the_selected_credential[dashboard_session-create_customer]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  361. ✓test_management_operations_send_the_selected_credential[dashboard_session-create_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  362. ✓test_management_operations_send_the_selected_credential[dashboard_session-create_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  363. ✓test_management_operations_send_the_selected_credential[dashboard_session-create_tag]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  364. ✓test_management_operations_send_the_selected_credential[dashboard_session-create_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  365. ✓test_management_operations_send_the_selected_credential[dashboard_session-create_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  366. ✓test_management_operations_send_the_selected_credential[dashboard_session-customer_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  367. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_customer]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  368. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_key_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  369. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  370. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_model_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  371. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  372. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_tag]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  373. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_team_member]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  374. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  375. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_user_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  376. ✓test_management_operations_send_the_selected_credential[dashboard_session-delete_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  377. ✓test_management_operations_send_the_selected_credential[dashboard_session-generate_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  378. ✓test_management_operations_send_the_selected_credential[dashboard_session-key_alias_count]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  379. ✓test_management_operations_send_the_selected_credential[dashboard_session-key_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  380. ✓test_management_operations_send_the_selected_credential[dashboard_session-key_list]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  381. ✓test_management_operations_send_the_selected_credential[dashboard_session-llm_only_key]test_proxy_client.py›
    1. Generate a virtual key with allowed routes: llm_api_routes
    View test on GitHub ↗
  382. ✓test_management_operations_send_the_selected_credential[dashboard_session-org_info_status]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  383. ✓test_management_operations_send_the_selected_credential[dashboard_session-org_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  384. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.create_credential]test_proxy_client.py›
    1. Save the provider credential owned
    View test on GitHub ↗
  385. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.create_model]test_proxy_client.py›
    1. Add a deployment named owned that calls synthetic
    View test on GitHub ↗
  386. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.create_team]test_proxy_client.py›
    1. Create a team with team alias: owned
    View test on GitHub ↗
  387. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.create_toolset]test_proxy_client.py›
    1. Create an MCP toolset with the tools
    View test on GitHub ↗
  388. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.delete_credential]test_proxy_client.py›
    1. Delete the provider credential
    View test on GitHub ↗
  389. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.delete_customers]test_proxy_client.py›
    1. Delete the end users owned
    View test on GitHub ↗
  390. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.delete_key]test_proxy_client.py›
    1. Delete the virtual key
    View test on GitHub ↗
  391. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.delete_model]test_proxy_client.py›
    1. Delete the deployment
    View test on GitHub ↗
  392. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.delete_team]test_proxy_client.py›
    1. Delete the team
    View test on GitHub ↗
  393. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.delete_toolset]test_proxy_client.py›
    1. Delete the MCP toolset
    View test on GitHub ↗
  394. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.delete_user]test_proxy_client.py›
    1. Delete the internal user
    View test on GitHub ↗
  395. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.generate_key]test_proxy_client.py›
    1. Generate a virtual key with default settings
    View test on GitHub ↗
  396. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.key_info]test_proxy_client.py›
    1. Read the key's settings back from /key/info
    View test on GitHub ↗
  397. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.memory_summary]test_proxy_client.py›
    1. Read memory usage from /debug/memory/summary on every proxy replica
    View test on GitHub ↗
  398. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.model_cost_map]test_proxy_client.py›
    1. Read the model cost map
    View test on GitHub ↗
  399. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.model_info]test_proxy_client.py›
    1. List the deployments from /model/info
    View test on GitHub ↗
  400. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.probe]test_proxy_client.py›
    1. Call the management route /user/info
    View test on GitHub ↗
  401. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.spend_logs]test_proxy_client.py›
    1. Read /spend/logs
    View test on GitHub ↗
  402. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.update_model]test_proxy_client.py›
    1. Update a deployment's settings to model: synthetic
    View test on GitHub ↗
  403. ✓test_management_operations_send_the_selected_credential[dashboard_session-proxy.update_toolset]test_proxy_client.py›
    1. Update an MCP toolset with toolset id: owned
    View test on GitHub ↗
  404. ✓test_management_operations_send_the_selected_credential[dashboard_session-regenerate_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  405. ✓test_management_operations_send_the_selected_credential[dashboard_session-reset_key_spend]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  406. ✓test_management_operations_send_the_selected_credential[dashboard_session-tag_list]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  407. ✓test_management_operations_send_the_selected_credential[dashboard_session-team_info_status]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  408. ✓test_management_operations_send_the_selected_credential[dashboard_session-team_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  409. ✓test_management_operations_send_the_selected_credential[dashboard_session-team_list_ids]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  410. ✓test_management_operations_send_the_selected_credential[dashboard_session-update_key_models]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  411. ✓test_management_operations_send_the_selected_credential[dashboard_session-update_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  412. ✓test_management_operations_send_the_selected_credential[dashboard_session-update_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  413. ✓test_management_operations_send_the_selected_credential[dashboard_session-update_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  414. ✓test_management_operations_send_the_selected_credential[dashboard_session-update_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  415. ✓test_management_operations_send_the_selected_credential[dashboard_session-update_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  416. ✓test_management_operations_send_the_selected_credential[dashboard_session-user_count]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  417. ✓test_management_operations_send_the_selected_credential[dashboard_session-user_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  418. ✓test_management_operations_send_the_selected_credential[dashboard_session-user_list_ids]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  419. ✓test_management_operations_send_the_selected_credential[direct_jwt-add_team_member]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  420. ✓test_management_operations_send_the_selected_credential[direct_jwt-block_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  421. ✓test_management_operations_send_the_selected_credential[direct_jwt-connection_test]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  422. ✓test_management_operations_send_the_selected_credential[direct_jwt-create_customer]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  423. ✓test_management_operations_send_the_selected_credential[direct_jwt-create_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  424. ✓test_management_operations_send_the_selected_credential[direct_jwt-create_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  425. ✓test_management_operations_send_the_selected_credential[direct_jwt-create_tag]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  426. ✓test_management_operations_send_the_selected_credential[direct_jwt-create_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  427. ✓test_management_operations_send_the_selected_credential[direct_jwt-create_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  428. ✓test_management_operations_send_the_selected_credential[direct_jwt-customer_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  429. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_customer]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  430. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_key_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  431. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  432. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_model_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  433. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  434. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_tag]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  435. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_team_member]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  436. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  437. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_user_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  438. ✓test_management_operations_send_the_selected_credential[direct_jwt-delete_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  439. ✓test_management_operations_send_the_selected_credential[direct_jwt-generate_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  440. ✓test_management_operations_send_the_selected_credential[direct_jwt-key_alias_count]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  441. ✓test_management_operations_send_the_selected_credential[direct_jwt-key_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  442. ✓test_management_operations_send_the_selected_credential[direct_jwt-key_list]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  443. ✓test_management_operations_send_the_selected_credential[direct_jwt-llm_only_key]test_proxy_client.py›
    1. Generate a virtual key with allowed routes: llm_api_routes
    View test on GitHub ↗
  444. ✓test_management_operations_send_the_selected_credential[direct_jwt-org_info_status]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  445. ✓test_management_operations_send_the_selected_credential[direct_jwt-org_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  446. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.create_credential]test_proxy_client.py›
    1. Save the provider credential owned
    View test on GitHub ↗
  447. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.create_model]test_proxy_client.py›
    1. Add a deployment named owned that calls synthetic
    View test on GitHub ↗
  448. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.create_team]test_proxy_client.py›
    1. Create a team with team alias: owned
    View test on GitHub ↗
  449. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.create_toolset]test_proxy_client.py›
    1. Create an MCP toolset with the tools
    View test on GitHub ↗
  450. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.delete_credential]test_proxy_client.py›
    1. Delete the provider credential
    View test on GitHub ↗
  451. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.delete_customers]test_proxy_client.py›
    1. Delete the end users owned
    View test on GitHub ↗
  452. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.delete_key]test_proxy_client.py›
    1. Delete the virtual key
    View test on GitHub ↗
  453. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.delete_model]test_proxy_client.py›
    1. Delete the deployment
    View test on GitHub ↗
  454. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.delete_team]test_proxy_client.py›
    1. Delete the team
    View test on GitHub ↗
  455. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.delete_toolset]test_proxy_client.py›
    1. Delete the MCP toolset
    View test on GitHub ↗
  456. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.delete_user]test_proxy_client.py›
    1. Delete the internal user
    View test on GitHub ↗
  457. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.generate_key]test_proxy_client.py›
    1. Generate a virtual key with default settings
    View test on GitHub ↗
  458. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.key_info]test_proxy_client.py›
    1. Read the key's settings back from /key/info
    View test on GitHub ↗
  459. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.memory_summary]test_proxy_client.py›
    1. Read memory usage from /debug/memory/summary on every proxy replica
    View test on GitHub ↗
  460. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.model_cost_map]test_proxy_client.py›
    1. Read the model cost map
    View test on GitHub ↗
  461. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.model_info]test_proxy_client.py›
    1. List the deployments from /model/info
    View test on GitHub ↗
  462. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.probe]test_proxy_client.py›
    1. Call the management route /user/info
    View test on GitHub ↗
  463. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.spend_logs]test_proxy_client.py›
    1. Read /spend/logs
    View test on GitHub ↗
  464. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.update_model]test_proxy_client.py›
    1. Update a deployment's settings to model: synthetic
    View test on GitHub ↗
  465. ✓test_management_operations_send_the_selected_credential[direct_jwt-proxy.update_toolset]test_proxy_client.py›
    1. Update an MCP toolset with toolset id: owned
    View test on GitHub ↗
  466. ✓test_management_operations_send_the_selected_credential[direct_jwt-regenerate_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  467. ✓test_management_operations_send_the_selected_credential[direct_jwt-reset_key_spend]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  468. ✓test_management_operations_send_the_selected_credential[direct_jwt-tag_list]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  469. ✓test_management_operations_send_the_selected_credential[direct_jwt-team_info_status]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  470. ✓test_management_operations_send_the_selected_credential[direct_jwt-team_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  471. ✓test_management_operations_send_the_selected_credential[direct_jwt-team_list_ids]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  472. ✓test_management_operations_send_the_selected_credential[direct_jwt-update_key_models]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  473. ✓test_management_operations_send_the_selected_credential[direct_jwt-update_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  474. ✓test_management_operations_send_the_selected_credential[direct_jwt-update_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  475. ✓test_management_operations_send_the_selected_credential[direct_jwt-update_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  476. ✓test_management_operations_send_the_selected_credential[direct_jwt-update_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  477. ✓test_management_operations_send_the_selected_credential[direct_jwt-update_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  478. ✓test_management_operations_send_the_selected_credential[direct_jwt-user_count]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  479. ✓test_management_operations_send_the_selected_credential[direct_jwt-user_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  480. ✓test_management_operations_send_the_selected_credential[direct_jwt-user_list_ids]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  481. ✓test_management_operations_send_the_selected_credential[master-add_team_member]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  482. ✓test_management_operations_send_the_selected_credential[master-block_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  483. ✓test_management_operations_send_the_selected_credential[master-connection_test]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  484. ✓test_management_operations_send_the_selected_credential[master-create_customer]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  485. ✓test_management_operations_send_the_selected_credential[master-create_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  486. ✓test_management_operations_send_the_selected_credential[master-create_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  487. ✓test_management_operations_send_the_selected_credential[master-create_tag]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  488. ✓test_management_operations_send_the_selected_credential[master-create_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  489. ✓test_management_operations_send_the_selected_credential[master-create_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  490. ✓test_management_operations_send_the_selected_credential[master-customer_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  491. ✓test_management_operations_send_the_selected_credential[master-delete_customer]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  492. ✓test_management_operations_send_the_selected_credential[master-delete_key_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  493. ✓test_management_operations_send_the_selected_credential[master-delete_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  494. ✓test_management_operations_send_the_selected_credential[master-delete_model_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  495. ✓test_management_operations_send_the_selected_credential[master-delete_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  496. ✓test_management_operations_send_the_selected_credential[master-delete_tag]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  497. ✓test_management_operations_send_the_selected_credential[master-delete_team_member]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  498. ✓test_management_operations_send_the_selected_credential[master-delete_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  499. ✓test_management_operations_send_the_selected_credential[master-delete_user_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  500. ✓test_management_operations_send_the_selected_credential[master-delete_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  501. ✓test_management_operations_send_the_selected_credential[master-generate_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  502. ✓test_management_operations_send_the_selected_credential[master-key_alias_count]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  503. ✓test_management_operations_send_the_selected_credential[master-key_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  504. ✓test_management_operations_send_the_selected_credential[master-key_list]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  505. ✓test_management_operations_send_the_selected_credential[master-llm_only_key]test_proxy_client.py›
    1. Generate a virtual key with allowed routes: llm_api_routes
    View test on GitHub ↗
  506. ✓test_management_operations_send_the_selected_credential[master-org_info_status]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  507. ✓test_management_operations_send_the_selected_credential[master-org_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  508. ✓test_management_operations_send_the_selected_credential[master-proxy.create_credential]test_proxy_client.py›
    1. Save the provider credential owned
    View test on GitHub ↗
  509. ✓test_management_operations_send_the_selected_credential[master-proxy.create_model]test_proxy_client.py›
    1. Add a deployment named owned that calls synthetic
    View test on GitHub ↗
  510. ✓test_management_operations_send_the_selected_credential[master-proxy.create_team]test_proxy_client.py›
    1. Create a team with team alias: owned
    View test on GitHub ↗
  511. ✓test_management_operations_send_the_selected_credential[master-proxy.create_toolset]test_proxy_client.py›
    1. Create an MCP toolset with the tools
    View test on GitHub ↗
  512. ✓test_management_operations_send_the_selected_credential[master-proxy.delete_credential]test_proxy_client.py›
    1. Delete the provider credential
    View test on GitHub ↗
  513. ✓test_management_operations_send_the_selected_credential[master-proxy.delete_customers]test_proxy_client.py›
    1. Delete the end users owned
    View test on GitHub ↗
  514. ✓test_management_operations_send_the_selected_credential[master-proxy.delete_key]test_proxy_client.py›
    1. Delete the virtual key
    View test on GitHub ↗
  515. ✓test_management_operations_send_the_selected_credential[master-proxy.delete_model]test_proxy_client.py›
    1. Delete the deployment
    View test on GitHub ↗
  516. ✓test_management_operations_send_the_selected_credential[master-proxy.delete_team]test_proxy_client.py›
    1. Delete the team
    View test on GitHub ↗
  517. ✓test_management_operations_send_the_selected_credential[master-proxy.delete_toolset]test_proxy_client.py›
    1. Delete the MCP toolset
    View test on GitHub ↗
  518. ✓test_management_operations_send_the_selected_credential[master-proxy.delete_user]test_proxy_client.py›
    1. Delete the internal user
    View test on GitHub ↗
  519. ✓test_management_operations_send_the_selected_credential[master-proxy.generate_key]test_proxy_client.py›
    1. Generate a virtual key with default settings
    View test on GitHub ↗
  520. ✓test_management_operations_send_the_selected_credential[master-proxy.key_info]test_proxy_client.py›
    1. Read the key's settings back from /key/info
    View test on GitHub ↗
  521. ✓test_management_operations_send_the_selected_credential[master-proxy.memory_summary]test_proxy_client.py›
    1. Read memory usage from /debug/memory/summary on every proxy replica
    View test on GitHub ↗
  522. ✓test_management_operations_send_the_selected_credential[master-proxy.model_cost_map]test_proxy_client.py›
    1. Read the model cost map
    View test on GitHub ↗
  523. ✓test_management_operations_send_the_selected_credential[master-proxy.model_info]test_proxy_client.py›
    1. List the deployments from /model/info
    View test on GitHub ↗
  524. ✓test_management_operations_send_the_selected_credential[master-proxy.probe]test_proxy_client.py›
    1. Call the management route /user/info
    View test on GitHub ↗
  525. ✓test_management_operations_send_the_selected_credential[master-proxy.spend_logs]test_proxy_client.py›
    1. Read /spend/logs
    View test on GitHub ↗
  526. ✓test_management_operations_send_the_selected_credential[master-proxy.update_model]test_proxy_client.py›
    1. Update a deployment's settings to model: synthetic
    View test on GitHub ↗
  527. ✓test_management_operations_send_the_selected_credential[master-proxy.update_toolset]test_proxy_client.py›
    1. Update an MCP toolset with toolset id: owned
    View test on GitHub ↗
  528. ✓test_management_operations_send_the_selected_credential[master-regenerate_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  529. ✓test_management_operations_send_the_selected_credential[master-reset_key_spend]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  530. ✓test_management_operations_send_the_selected_credential[master-tag_list]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  531. ✓test_management_operations_send_the_selected_credential[master-team_info_status]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  532. ✓test_management_operations_send_the_selected_credential[master-team_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  533. ✓test_management_operations_send_the_selected_credential[master-team_list_ids]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  534. ✓test_management_operations_send_the_selected_credential[master-update_key_models]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  535. ✓test_management_operations_send_the_selected_credential[master-update_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  536. ✓test_management_operations_send_the_selected_credential[master-update_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  537. ✓test_management_operations_send_the_selected_credential[master-update_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  538. ✓test_management_operations_send_the_selected_credential[master-update_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  539. ✓test_management_operations_send_the_selected_credential[master-update_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  540. ✓test_management_operations_send_the_selected_credential[master-user_count]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  541. ✓test_management_operations_send_the_selected_credential[master-user_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  542. ✓test_management_operations_send_the_selected_credential[master-user_list_ids]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  543. ✓test_management_operations_send_the_selected_credential[virtual_key-add_team_member]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  544. ✓test_management_operations_send_the_selected_credential[virtual_key-block_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  545. ✓test_management_operations_send_the_selected_credential[virtual_key-connection_test]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  546. ✓test_management_operations_send_the_selected_credential[virtual_key-create_customer]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  547. ✓test_management_operations_send_the_selected_credential[virtual_key-create_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  548. ✓test_management_operations_send_the_selected_credential[virtual_key-create_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  549. ✓test_management_operations_send_the_selected_credential[virtual_key-create_tag]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  550. ✓test_management_operations_send_the_selected_credential[virtual_key-create_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  551. ✓test_management_operations_send_the_selected_credential[virtual_key-create_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  552. ✓test_management_operations_send_the_selected_credential[virtual_key-customer_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  553. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_customer]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  554. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_key_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  555. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  556. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_model_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  557. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  558. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_tag]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  559. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_team_member]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  560. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  561. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_user_strict]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  562. ✓test_management_operations_send_the_selected_credential[virtual_key-delete_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  563. ✓test_management_operations_send_the_selected_credential[virtual_key-generate_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  564. ✓test_management_operations_send_the_selected_credential[virtual_key-key_alias_count]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  565. ✓test_management_operations_send_the_selected_credential[virtual_key-key_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  566. ✓test_management_operations_send_the_selected_credential[virtual_key-key_list]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  567. ✓test_management_operations_send_the_selected_credential[virtual_key-llm_only_key]test_proxy_client.py›
    1. Generate a virtual key with allowed routes: llm_api_routes
    View test on GitHub ↗
  568. ✓test_management_operations_send_the_selected_credential[virtual_key-org_info_status]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  569. ✓test_management_operations_send_the_selected_credential[virtual_key-org_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  570. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.create_credential]test_proxy_client.py›
    1. Save the provider credential owned
    View test on GitHub ↗
  571. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.create_model]test_proxy_client.py›
    1. Add a deployment named owned that calls synthetic
    View test on GitHub ↗
  572. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.create_team]test_proxy_client.py›
    1. Create a team with team alias: owned
    View test on GitHub ↗
  573. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.create_toolset]test_proxy_client.py›
    1. Create an MCP toolset with the tools
    View test on GitHub ↗
  574. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.delete_credential]test_proxy_client.py›
    1. Delete the provider credential
    View test on GitHub ↗
  575. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.delete_customers]test_proxy_client.py›
    1. Delete the end users owned
    View test on GitHub ↗
  576. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.delete_key]test_proxy_client.py›
    1. Delete the virtual key
    View test on GitHub ↗
  577. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.delete_model]test_proxy_client.py›
    1. Delete the deployment
    View test on GitHub ↗
  578. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.delete_team]test_proxy_client.py›
    1. Delete the team
    View test on GitHub ↗
  579. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.delete_toolset]test_proxy_client.py›
    1. Delete the MCP toolset
    View test on GitHub ↗
  580. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.delete_user]test_proxy_client.py›
    1. Delete the internal user
    View test on GitHub ↗
  581. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.generate_key]test_proxy_client.py›
    1. Generate a virtual key with default settings
    View test on GitHub ↗
  582. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.key_info]test_proxy_client.py›
    1. Read the key's settings back from /key/info
    View test on GitHub ↗
  583. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.memory_summary]test_proxy_client.py›
    1. Read memory usage from /debug/memory/summary on every proxy replica
    View test on GitHub ↗
  584. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.model_cost_map]test_proxy_client.py›
    1. Read the model cost map
    View test on GitHub ↗
  585. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.model_info]test_proxy_client.py›
    1. List the deployments from /model/info
    View test on GitHub ↗
  586. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.probe]test_proxy_client.py›
    1. Call the management route /user/info
    View test on GitHub ↗
  587. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.spend_logs]test_proxy_client.py›
    1. Read /spend/logs
    View test on GitHub ↗
  588. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.update_model]test_proxy_client.py›
    1. Update a deployment's settings to model: synthetic
    View test on GitHub ↗
  589. ✓test_management_operations_send_the_selected_credential[virtual_key-proxy.update_toolset]test_proxy_client.py›
    1. Update an MCP toolset with toolset id: owned
    View test on GitHub ↗
  590. ✓test_management_operations_send_the_selected_credential[virtual_key-regenerate_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  591. ✓test_management_operations_send_the_selected_credential[virtual_key-reset_key_spend]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  592. ✓test_management_operations_send_the_selected_credential[virtual_key-tag_list]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  593. ✓test_management_operations_send_the_selected_credential[virtual_key-team_info_status]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  594. ✓test_management_operations_send_the_selected_credential[virtual_key-team_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  595. ✓test_management_operations_send_the_selected_credential[virtual_key-team_list_ids]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  596. ✓test_management_operations_send_the_selected_credential[virtual_key-update_key_models]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  597. ✓test_management_operations_send_the_selected_credential[virtual_key-update_key]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  598. ✓test_management_operations_send_the_selected_credential[virtual_key-update_mcp_server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  599. ✓test_management_operations_send_the_selected_credential[virtual_key-update_org]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  600. ✓test_management_operations_send_the_selected_credential[virtual_key-update_team]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  601. ✓test_management_operations_send_the_selected_credential[virtual_key-update_user]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  602. ✓test_management_operations_send_the_selected_credential[virtual_key-user_count]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  603. ✓test_management_operations_send_the_selected_credential[virtual_key-user_info]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  604. ✓test_management_operations_send_the_selected_credential[virtual_key-user_list_ids]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  605. ✓test_management_read_backs_poll_the_control_replicas_only TestReplicasFortest_proxy_client.py›
    1. Read /key/info on every proxy replica until they all agree
    View test on GitHub ↗
  606. ✓test_mcp_admin_routes_read_back_from_every_data_plane_replica TestReplicasFortest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  607. ✓test_monolith_reads_management_routes_back_from_every_replica TestReplicasFortest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  608. ✓test_names_the_replica_that_never_converges_with_its_last_read TestAwaitConvergedEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  609. ✓test_names_the_replica_that_never_converges_with_what_it_last_served TestAwaitEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  610. ✓test_partial_updates_preserve_explicit_null_at_the_http_boundary[server]test_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  611. ✓test_partial_updates_preserve_explicit_null_at_the_http_boundary[toolset]test_proxy_client.py›
    1. Update an MCP toolset with toolset id: owned
    View test on GitHub ↗
  612. ✓test_passes_once_every_replica_lists_the_model TestAwaitServableEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  613. ✓test_polls_until_the_deadline_before_giving_up TestAwaitEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  614. ✓test_split_deployment_reads_management_routes_back_from_the_control_plane TestReplicasFortest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  615. ✓test_splits_and_trims_the_gateway_addresses TestParseReplicaUrlstest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  616. ✓test_strict_key_cleanup_accepts_missing_only_when_requested TestBoundManagementCallertest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  617. ✓test_successful_team_and_model_polling_uses_the_bound_caller TestSplitCallerPropagationtest_proxy_client.py›
    1. Add a deployment named owned that calls synthetic
    View test on GitHub ↗
  618. ✓test_the_stacks_own_endpoints_take_its_exported_control_list TestStackEndpointsControlReplicastest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  619. ✓test_unset_with_a_split_control_plane_polls_its_base_alone TestParseControlPlaneReplicaUrlstest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  620. ✓test_unset_with_one_shared_base_follows_the_data_plane_replicas TestParseControlPlaneReplicaUrlstest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  621. ✓test_waits_for_the_lagging_replica_and_returns_every_settled_answer TestAwaitEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  622. ✓test_waits_for_the_replica_that_lags_behind_the_write TestAwaitConvergedEverywheretest_proxy_client.py›

    No steps recorded

    View test on GitHub ↗
  623. ✓test_a_drained_group_does_not_block_a_different_group TestModelAccessGroupBudgetbudgets/test_model_access_group_budget_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.4-nano
    1. Generate a virtual key with models: e2e-mag-free-e9801ef4f868 and user id: e2e-test-user
    View test on GitHub ↗
  624. ✓test_a_key_that_spent_nothing_is_blocked_by_the_shared_pool TestModelAccessGroupBudgetbudgets/test_model_access_group_budget_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.4-nano
    1. Generate a virtual key with models: e2e-mag-budget-e9801ef4f868 and user id: e2e-test-user
    View test on GitHub ↗
  625. ✓test_alias_shares_rpm_bucket_with_model_group TestModelGroupAliasRateLimitratelimit/test_model_group_alias_rate_limit_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to e2e-alias-rl-target with the prompt "reply with one word 57119e94b555"
    3. Send a /chat/completions request to e2e-alias-rl-target with the prompt "reply with one word a34576e45e1f"
    4. Send a /chat/completions request to e2e-alias-rl-target with the prompt "reply with one word b7cf5f3bf6be"
    5. Send a /chat/completions request to e2e-alias-rl-alias with the prompt "reply with one word 053918ea1b10"
    View test on GitHub ↗
  626. ✓test_bare_key_blocks_over_its_own_budget TestBudgetBlocksPerLevelbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06
    View test on GitHub ↗
  627. ✓test_bare_key_budget_resets_after_window TestBudgetResetPerLevelbudgets/test_budget_reset_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06 and budget duration: 30s
    View test on GitHub ↗
  628. ✓test_blocked_member_does_not_block_peer TestTeamMemberBudgetIsolationbudgets/test_team_member_budget_isolation_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • gpt-5.5
    1. Generate a virtual key with user id: efac2e25-6e68-41a5-a701-8c341936d04d and team id: f6ca967a-6521-4fcc-93b5-c155edb2fb39
    2. Generate a virtual key with user id: aeb848dc-e6e4-4b42-bf3a-6a4cfdee0a47 and team id: f6ca967a-6521-4fcc-93b5-c155edb2fb39
    3. Send a /chat/completions request to gpt-5.5
    4. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  629. ✓test_budget_crud_roundtripbudgets/test_budget_crud_e2e.py›
    • spend-budgets
    • budget_management
    1. Generate a virtual key with budget id: 7c2aa61e-7215-40cc-a4d2-54c18366eb6e
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  630. ✓test_budget_delete_removes_itbudgets/test_budget_crud_e2e.py›
    • spend-budgets
    • budget_management

    No steps recorded

    View test on GitHub ↗
  631. ✓test_budget_duration_schedules_reset_on_keybudgets/test_budget_crud_e2e.py›
    • spend-budgets
    • key_management
    1. Generate a virtual key with max budget: 10.0 and budget duration: 30d
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  632. ✓test_budget_fallback_reroutes_anthropic_messages_to_openaibudgets/test_budget_fallback_e2e.py›
    • spend-budgets
    • messages
    • nonstream
    • anthropic
    • openai
    • claude-haiku-4-5
    • gpt-5.5
    1. Generate a virtual key with model max budget: claude-haiku-4-5: budget limit: 1e-06 and time period: 30d and budget fallbacks: claude-haiku-4-5: gpt-5.5
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  633. ✓test_burst_of_concurrent_calls_loses_no_spendspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.6-luna
    1. Add a deployment named e2e-reconciliation-1053881cdd5f that calls openai/gpt-5.6-luna
    2. Create a team with team alias: e2e-spend-b59edaaf307a
    3. Generate a virtual key with models: e2e-reconciliation-1053881cdd5f and team id: 0d9b7b67-aeae-4f05-9ee9-17503977328c
    4. Send a /chat/completions request to e2e-reconciliation-1053881cdd5f
    5. Create a team with team alias: e2e-spend-c8913eb8381a
    6. Generate a virtual key with models: e2e-reconciliation-1053881cdd5f and team id: 0172c5e0-4841-40b3-8331-7616354fc547
    7. Send a /chat/completions request to e2e-reconciliation-1053881cdd5f
    8. Wait for at least 7 of the key's spend logs in /spend/logs
    9. Read the key's settings back from /key/info
    10. Wait for at least 7 of the key's spend logs in /spend/logs
    11. Read the key's settings back from /key/info
    View test on GitHub ↗
  634. ✓test_burst_rate_limit_does_not_freeze_fresh_key TestRedisCircuitBreakerPathratelimit/test_redis_circuit_breaker_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • anthropic/claude-haiku-4-5-20251001
    1. Add a deployment named e2e-cb-model-95cc4078b3f3 that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with models: e2e-cb-model-95cc4078b3f3, key alias: e2e-cb-hot-616f38cd7a3f and rpm limit: 1
    3. Generate a virtual key with models: e2e-cb-model-95cc4078b3f3 and key alias: e2e-cb-cool-082222440f60
    4. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst a02c320071dc"
    5. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst 7e797ca55fbb"
    6. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst ac9acba1b322"
    7. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst 94e5778dd036"
    8. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst df5e7d4ae41c"
    9. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst 8fee8f534216"
    10. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst 4e0ca0d0ae47"
    11. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst 5e3eca85a6eb"
    12. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst 671660062123"
    13. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst c9195beb8be8"
    14. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst 6d5f794848f0"
    15. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "burst 0d9104b0e56a"
    16. Send a /chat/completions request to e2e-cb-model-95cc4078b3f3 with the prompt "fresh a03cdc64e2a5"
    View test on GitHub ↗
  635. ✓test_cache_hit_is_zero_cost_and_suffixedspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • nonstream
    • gemini
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  636. ✓test_cache_hit_reduces_tpm_by_non_cached_only TestTpmExcludesCachedTokensratelimit/test_tpm_excludes_cached_tokens_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • anthropic/claude-haiku-4-5-20251001
    • prompt_caching
    1. Add a deployment named e2e-tpm-cache-a3d535ef294b that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with models: e2e-tpm-cache-a3d535ef294b and tpm limit: 100000
    View test on GitHub ↗
  637. ✓test_cache_write_tokens_billed_at_cache_creation_rate TestCacheCostAccountingspend_tracking/test_cache_cost_accounting_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.6-luna
    • prompt_caching
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named cache-write-priced-a2ac2273b71d that calls openai/gpt-5.6-luna
    3. Send a /chat/completions request to cache-write-priced-a2ac2273b71d
    View test on GitHub ↗
  638. ✓test_capture_rate_reports_or_names_the_missing_billing_keyspend_tracking/test_spend_routes.py›

    No steps recorded

    View test on GitHub ↗
  639. ✓test_chat_completion_writes_nonzero_spend_rowspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • chat_completions
    • nonstream
    • gemini
    • gemini-2.5-flash
    1. List the deployments from /model/info
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to gemini-2.5-flash
    4. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  640. ✓test_cold_counter_reseed_keeps_counter_equal_to_db_spendbudgets/test_spend_counter_reseed_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with models: claude-haiku-4-5 and max budget: 1.0
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  641. ✓test_component_cost_headers_sum_to_total TestCostHeadersspend_tracking/test_cost_headers_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.5
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named header-priced-fc2610fdb5de that calls openai/gpt-5.5
    View test on GitHub ↗
  642. ✓test_cost_breakdown_reports_component_costs TestCacheCostAccountingspend_tracking/test_cache_cost_accounting_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.5
    • prompt_caching
    • reasoning
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named breakdown-priced-81fae462d67d that calls openai/gpt-5.5
    3. Send a /chat/completions request to breakdown-priced-81fae462d67d
    View test on GitHub ↗
  643. ✓test_each_model_on_a_shared_key_gets_its_own_rowspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • gemini
    • claude-haiku-4-5
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Send a /chat/completions request to claude-haiku-4-5
    4. Wait for at least 2 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  644. ✓test_edge_wired_chat_writes_nonzero_spend_rowspend_tracking/test_provider_edge_spend_e2e.py›
    • spend-budgets
    • chat_completions
    • nonstream
    • openai
    • openai/gpt-5.5
    1. List the deployments from /model/info
    2. Add a deployment named openai-text-embedding-3-small that calls openai/text-embedding-3-small
    3. Add a deployment named openai-responses-codex that calls openai/gpt-5.3-codex
    4. Add a deployment named openai-gpt-4o-mini that calls openai/gpt-4o-mini
    5. Generate a virtual key with user id: e2e-test-user
    6. Add a deployment named e2e-edge-openai-32dad6fc8195 that calls openai/gpt-5.5
    7. Send a /chat/completions request to e2e-edge-openai-32dad6fc8195
    8. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  645. ✓test_embedding_writes_nonzero_spend_rowspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • embeddings
    • nonstream
    • openai
    • openai-text-embedding-3-small
    1. Generate a virtual key with user id: e2e-test-user
    2. Send an /embeddings request to openai-text-embedding-3-small for "vectorize this sentence e4e204e2a5b4"
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  646. ✓test_end_user_budget_blocks_attributed_calls TestBudgetBlocksPerLevelbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with models: claude-haiku-4-5
    View test on GitHub ↗
  647. ✓test_end_user_header_attributes_responses_row[x-litellm-customer-id]spend_tracking/test_spend_tracking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  648. ✓test_end_user_header_attributes_responses_row[x-litellm-end-user-id]spend_tracking/test_spend_tracking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  649. ✓test_end_user_spend_attributed_on_rowspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • nonstream
    • gemini
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  650. ✓test_every_streamed_chunk_carries_the_served_tier TestServiceTierPricingspend_tracking/test_service_tier_pricing_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named tier-echo-stream-30f21ce33ff5 that calls openai/gpt-5.5
    3. Send a streaming /chat/completions request to tier-echo-stream-30f21ce33ff5
    View test on GitHub ↗
  651. ✓test_every_write_path_row_joins_the_key TestKeyAttributionspend_tracking/test_key_attribution_e2e.py›
    • spend-budgets
    • spend_reporting
    • anthropic
    • gemini
    • openai
    • claude-haiku-4-5
    • gemini-2.5-flash
    • openai-gpt-4o-mini
    • openai-responses-codex
    • openai-text-embedding-3-small
    1. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  652. ✓test_failure_call_writes_failure_status_rowspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.5
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-spend-failure-36b3b11e6cd4 that calls openai/gpt-5.5
    3. Send a /chat/completions request to e2e-spend-failure-36b3b11e6cd4
    4. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  653. ✓test_failure_rows_share_normalized_error_across_provider_wordingspend_tracking/test_spend_tracking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-norm-openai-183027346e33 that calls openai/gpt-5.5
    3. Send a /chat/completions request to e2e-norm-openai-183027346e33
    4. Add a deployment named e2e-norm-anthropic-183027346e33 that calls anthropic/claude-haiku-4-5
    5. Send a /chat/completions request to e2e-norm-anthropic-183027346e33
    6. Wait for at least 2 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  654. ✓test_generous_mode_lets_priority_borrow_past_reservation TestDynamicRateLimitPriorityratelimit/test_dynamic_rate_limit_priority_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • anthropic/claude-haiku-4-5-20251001
    1. Add a deployment named e2e-dynpri-generous-ade513c10108 that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with models: e2e-dynpri-generous-ade513c10108, key alias: e2e-dynpri-generous-dev-5044d7d57b9f and metadata: priority: dev
    3. Generate a virtual key with models: e2e-dynpri-generous-ade513c10108, key alias: e2e-dynpri-generous-prod-2b4ac7e0a544 and metadata: priority: prod
    4. Read the key's settings back from /key/info
    5. Send a /chat/completions request to e2e-dynpri-generous-ade513c10108 with the prompt "reply with one word 04895b60070c"
    6. Send a /chat/completions request to e2e-dynpri-generous-ade513c10108 with the prompt "reply with one word 33f21e52f15e"
    7. Send a /chat/completions request to e2e-dynpri-generous-ade513c10108 with the prompt "reply with one word a0c7c523b5f8"
    8. Send a /chat/completions request to e2e-dynpri-generous-ade513c10108 with the prompt "reply with one word 8008423285a5"
    9. Send a /chat/completions request to e2e-dynpri-generous-ade513c10108 with the prompt "reply with one word ea2709bbd856"
    View test on GitHub ↗
  655. ✓test_headers_report_limit_and_remaining TestKeyRateLimitsratelimit/test_rate_limit_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with models: claude-haiku-4-5, tpm limit: 100000 and rpm limit: 5
    2. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word e27cb2b80998"
    View test on GitHub ↗
  656. ✓test_health_check_rows_keep_the_service_account_key TestKeyAttributionspend_tracking/test_key_attribution_e2e.py›
    • spend-budgets
    • health
    • gemini
    • gemini-2.5-flash
    1. Read /spend/logs between 2026-10-04 00:25:17.663509+00:00 and 2026-10-04 00:27:22.014260+00:00
    2. Read /spend/logs between 2026-10-04 00:25:17.663509+00:00 and 2026-10-04 00:27:27.096042+00:00
    3. Read /spend/logs between 2026-10-04 00:25:17.663509+00:00 and 2026-10-04 00:27:32.147559+00:00
    View test on GitHub ↗
  657. ✓test_intercepted_search_is_billed_under_the_request_session TestWebSearchInterceptionSessionspend_tracking/test_websearch_interception_session_e2e.py›
    1. Add a deployment named e2e-websearch-session-8fb37097b4c2 that calls bedrock/invoke/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with models: e2e-websearch-session-8fb37097b4c2 and user id: e2e-test-user
    3. Send a /v1/messages request to e2e-websearch-session-8fb37097b4c2
    4. Wait for at least 2 of the session's spend logs in /spend/logs
    View test on GitHub ↗
  658. ✓test_key_budget_reset_at_advances_after_windowbudgets/test_budget_reset_advances_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06 and budget duration: 30s
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  659. ✓test_key_spend_blocks_at_capbudgets/test_budget_reset_advances_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06 and budget duration: 30s
    View test on GitHub ↗
  660. ✓test_key_spend_equals_sum_of_logsspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • nonstream
    • gemini
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Wait for at least 2 of the key's spend logs in /spend/logs
    4. Read the key's settings back from /key/info
    View test on GitHub ↗
  661. ✓test_key_with_budget_duration_schedules_reset_at_creationbudgets/test_budget_reset_advances_e2e.py›
    • spend-budgets
    • key_management
    1. Generate a virtual key with max budget: 3e-06 and budget duration: 30s
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  662. ✓test_long_window_blocks_after_short_window_resetsbudgets/test_multi_window_budget_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • gpt-5.5
    1. Generate a virtual key with models: gpt-5.5 and budget limits: budget duration: 30s and max budget: 1e-09, budget duration: 1d and max budget: 5e-07
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  663. ✓test_member_budget_blocks_without_touching_teammates TestBudgetBlocksPerLevelbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with user id: af1aea89-ff8e-4740-85bc-94d8317c2403 and team id: aa72d163-aa1e-4dcd-9ea6-73142dcf78a6
    2. Generate a virtual key with user id: 1cd14d7f-381e-4af3-80c0-68b6026f55d3 and team id: aa72d163-aa1e-4dcd-9ea6-73142dcf78a6
    View test on GitHub ↗
  664. ✓test_member_spend_attributed_to_team_and_user TestTeamMemberBudgetbudgets/test_team_member_budget_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with user id: 7bbf3c73-9f32-4914-b061-00624307bbbe and team id: ac74bb60-3bf7-40bc-9cdc-1c6931eb5aac
    2. Send a /chat/completions request to claude-haiku-4-5
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  665. ✓test_member_spend_over_budget_is_blocked TestTeamMemberBudgetbudgets/test_team_member_budget_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5

    No steps recorded

    View test on GitHub ↗
  666. ✓test_messages_bridge_keeps_cache_tokens TestCacheCostAccountingspend_tracking/test_cache_cost_accounting_e2e.py›
    • spend-budgets
    • messages
    • nonstream
    • openai
    • openai/gpt-5.3-codex
    • prompt_caching
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named bridge-cache-priced-fc4c14e95a11 that calls openai/gpt-5.3-codex
    3. Send a /v1/messages request to bridge-cache-priced-fc4c14e95a11
    View test on GitHub ↗
  667. ✓test_messages_stream_records_the_served_tier TestServiceTierPricingspend_tracking/test_service_tier_pricing_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named tier-messages-stream-1366130913b5 that calls openai/gpt-5.5
    3. Send a streaming /v1/messages request to tier-messages-stream-1366130913b5
    View test on GitHub ↗
  668. ✓test_missing_end_date_is_rejected TestTeamDailyActivityspend_tracking/test_team_daily_activity_e2e.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  669. ✓test_missing_start_date_is_rejected TestTeamDailyActivityspend_tracking/test_team_daily_activity_e2e.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  670. ✓test_model_group_shares_rpm_bucket_with_alias TestModelGroupAliasRateLimitratelimit/test_model_group_alias_rate_limit_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to e2e-alias-rl-alias with the prompt "reply with one word 1d50310b7adf"
    3. Send a /chat/completions request to e2e-alias-rl-alias with the prompt "reply with one word b05a710da0b3"
    4. Send a /chat/completions request to e2e-alias-rl-alias with the prompt "reply with one word 8f8fefc82efb"
    5. Send a /chat/completions request to e2e-alias-rl-target with the prompt "reply with one word fd2ca3a8badf"
    View test on GitHub ↗
  671. ✓test_model_max_budget_isolates_per_modelbudgets/test_model_max_budget_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • gemini
    • claude-haiku-4-5
    • gemini-2.5-flash
    1. Generate a virtual key with model max budget: claude-haiku-4-5: budget limit: 1e-06 and time period: 30d and gemini-2.5-flash: budget limit: 1000.0 and time period: 30d
    View test on GitHub ↗
  672. ✓test_multi_window_key_resets_each_window_independentlybudgets/test_budget_reset_advances_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with budget limits: budget duration: 30s and max budget: 3e-06, budget duration: 1m and max budget: 1.0
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  673. ✓test_one_request_lands_the_same_spend_on_every_surface TestSpendSurfaceConsistencyspend_tracking/test_spend_surface_consistency_e2e.py›
    1. List the deployments from /model/info
    2. Add a deployment named e2e-spend-surfaces-883a49dd3544 that calls openai/gpt-5.6-luna
    3. Create a team with team alias: e2e-spend-surfaces-883a49dd3544
    4. Send a /chat/completions request to e2e-spend-surfaces-883a49dd3544
    5. Wait for the request's spend log in /spend/logs
    6. Read the key's settings back from /key/info
    View test on GitHub ↗
  674. ✓test_org_budget_blocks_keys_under_it TestBudgetBlocksPerLevelbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with team id: 260608aa-66d4-4f39-8449-68ef2ca0dda2
    View test on GitHub ↗
  675. ✓test_org_budget_resets_after_window TestBudgetResetPerLevelbudgets/test_budget_reset_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with team id: 1e737d16-8f9d-4342-bba1-352b0d3411ba
    View test on GitHub ↗
  676. ✓test_personal_key_blocks_over_its_own_budget TestKeyBudgetBlocksAcrossKeyKindsbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06 and user id: 808123b2-39bf-4ca0-b560-6534c901c9b8
    2. Generate a virtual key with user id: 808123b2-39bf-4ca0-b560-6534c901c9b8
    View test on GitHub ↗
  677. ✓test_personal_key_resets_after_window TestKeyBudgetResetAcrossKeyKindsbudgets/test_budget_reset_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06, budget duration: 30s and user id: 46f11a54-701f-4136-b97c-d87a3bb60f8a
    View test on GitHub ↗
  678. ✓test_personal_key_user_budget_resets_after_window TestBudgetResetPerLevelbudgets/test_budget_reset_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with user id: 898e4123-6c13-4b24-a0b7-9dc4562932b1
    View test on GitHub ↗
  679. ✓test_pre_call_rejection_row_attributes_provider_and_model_idspend_tracking/test_spend_tracking_e2e.py›
    1. Add a deployment named e2e-spend-precall-0c4a2521aa38 that calls openai/gpt-5.5
    2. Generate a virtual key with models: e2e-spend-precall-0c4a2521aa38 and rpm limit: 1
    3. Send a /chat/completions request to e2e-spend-precall-0c4a2521aa38
    4. Wait for at least 2 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  680. ✓test_priority_tier_bills_priority_rates TestServiceTierPricingspend_tracking/test_service_tier_pricing_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.6-luna
    • reasoning
    1. List the deployments from /model/info
    2. Generate a virtual key with user id: e2e-test-user
    3. Add a deployment named tier-priced-c44583d7c31e that calls openai/gpt-5.6-luna
    4. Send a /chat/completions request to tier-priced-c44583d7c31e
    View test on GitHub ↗
  681. ✓test_request_tags_round_tripspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • nonstream
    • gemini
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  682. ✓test_reset_wait_never_yields_non_budget_errorbudgets/test_budget_reset_advances_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06 and budget duration: 30s
    View test on GitHub ↗
  683. ✓test_responses_stream_records_the_served_tier TestServiceTierPricingspend_tracking/test_service_tier_pricing_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named tier-responses-stream-9dac54c35571 that calls openai/gpt-5.5
    3. Send a streaming /v1/responses request to tier-responses-stream-9dac54c35571
    View test on GitHub ↗
  684. ✓test_rpm_limit_blocks_over_limit TestKeyRateLimitsratelimit/test_rate_limit_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with models: claude-haiku-4-5 and rpm limit: 3
    2. Read the key's settings back from /key/info
    3. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word b5bfccb5869f"
    4. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word d2fe9b4a8472"
    5. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 7a211d45ddb8"
    6. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word b247d6bfa163"
    View test on GitHub ↗
  685. ✓test_rpm_limit_one_blocks_second_call TestRedisBackedRateLimitratelimit/test_redis_backed_ratelimit_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • anthropic/claude-haiku-4-5-20251001
    1. Add a deployment named e2e-redis-rpm-1eb9e859d360 that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with models: e2e-redis-rpm-1eb9e859d360, key alias: e2e-redis-rpm-fd50b8733e3c and rpm limit: 1
    3. Read the key's settings back from /key/info
    4. Send a /chat/completions request to e2e-redis-rpm-1eb9e859d360 with the prompt "ping 29085acc5fec"
    5. Send a /chat/completions request to e2e-redis-rpm-1eb9e859d360 with the prompt "pong 1eefaf3a9d9e"
    View test on GitHub ↗
  686. ✓test_rpm_limit_resets_after_window TestKeyRateLimitsratelimit/test_rate_limit_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with models: claude-haiku-4-5 and rpm limit: 1
    2. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word f00712e6c3e0"
    3. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word c03c65326e9c"
    4. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word d72516168b5f"
    5. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 4a99fb0b3a6b"
    6. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 994e5e4cc0c8"
    7. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 2483c17acf47"
    8. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 1a9cf0d7847a"
    9. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word bd7653f888d8"
    10. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 9988bfbc0d16"
    11. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 86eb1b69b407"
    12. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 45650156130d"
    13. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word e1ebfb8c6c94"
    14. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 2dbeccca67fa"
    15. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 4dc46a3355a8"
    16. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word b0c9f4485bdb"
    View test on GitHub ↗
  687. ✓test_schema_listed_spend_routes_are_responsivespend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  688. ✓test_short_window_blocks_then_resetsbudgets/test_multi_window_budget_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • gpt-5.5
    1. Generate a virtual key with models: gpt-5.5 and budget limits: budget duration: 30s and max budget: 1e-09, budget duration: 1m and max budget: 1.0
    View test on GitHub ↗
  689. ✓test_soft_budget_does_not_blockbudgets/test_soft_budget_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 1000.0 and soft budget: 1e-09
    View test on GitHub ↗
  690. ✓test_spend_calculate_returns_nonzero_costspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • spend_reporting
    • gemini
    • gemini-2.5-flash

    No steps recorded

    View test on GitHub ↗
  691. ✓test_spend_logs_by_key_return_every_row_with_the_alias TestKeyAttributionspend_tracking/test_key_attribution_e2e.py›
    • spend-budgets
    • spend_reporting
    • anthropic
    • gemini
    • openai
    • claude-haiku-4-5
    • gemini-2.5-flash
    • openai-gpt-4o-mini
    • openai-responses-codex
    • openai-text-embedding-3-small
    1. Wait for at least 9 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  692. ✓test_spend_logs_endpoint_returns_spendspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • spend_reporting
    • nonstream
    • gemini
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  693. ✓test_spend_logs_v2_pagination_caps_pages_and_keeps_totalspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • spend_reporting
    • nonstream
    • gemini
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Wait for at least 2 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  694. ✓test_spend_route_responsive[/customer/daily/activity]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  695. ✓test_spend_route_responsive[/customer/list]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • customer_management

    No steps recorded

    View test on GitHub ↗
  696. ✓test_spend_route_responsive[/end_user/daily/activity]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  697. ✓test_spend_route_responsive[/global/activity]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  698. ✓test_spend_route_responsive[/global/activity/exceptions]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  699. ✓test_spend_route_responsive[/global/activity/exceptions/deployment]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  700. ✓test_spend_route_responsive[/global/activity/model]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  701. ✓test_spend_route_responsive[/global/all_end_users]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  702. ✓test_spend_route_responsive[/global/spend]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  703. ✓test_spend_route_responsive[/global/spend/all_tag_names]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  704. ✓test_spend_route_responsive[/global/spend/keys]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  705. ✓test_spend_route_responsive[/global/spend/logs]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  706. ✓test_spend_route_responsive[/global/spend/models]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  707. ✓test_spend_route_responsive[/global/spend/provider]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  708. ✓test_spend_route_responsive[/global/spend/report]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  709. ✓test_spend_route_responsive[/global/spend/tags]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  710. ✓test_spend_route_responsive[/global/spend/teams]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  711. ✓test_spend_route_responsive[/key/list]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • key_management

    No steps recorded

    View test on GitHub ↗
  712. ✓test_spend_route_responsive[/organization/daily/activity]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  713. ✓test_spend_route_responsive[/organization/list]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • organization_management

    No steps recorded

    View test on GitHub ↗
  714. ✓test_spend_route_responsive[/spend/keys]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting
    1. List the deployments from /model/info
    2. Add a deployment named openai-text-embedding-3-small that calls openai/text-embedding-3-small
    3. Add a deployment named openai-responses-codex that calls openai/gpt-5.3-codex
    4. Add a deployment named openai-gpt-4o-mini that calls openai/gpt-4o-mini
    View test on GitHub ↗
  715. ✓test_spend_route_responsive[/spend/logs]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  716. ✓test_spend_route_responsive[/spend/logs/session/ui]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  717. ✓test_spend_route_responsive[/spend/logs/ui]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  718. ✓test_spend_route_responsive[/spend/logs/v2]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  719. ✓test_spend_route_responsive[/spend/tags]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  720. ✓test_spend_route_responsive[/spend/users]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  721. ✓test_spend_route_responsive[/tag/daily/activity]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  722. ✓test_spend_route_responsive[/team/daily/activity]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  723. ✓test_spend_route_responsive[/team/list]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • team_management

    No steps recorded

    View test on GitHub ↗
  724. ✓test_spend_route_responsive[/user/daily/activity]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  725. ✓test_spend_route_responsive[/user/daily/activity/aggregated]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • spend_reporting

    No steps recorded

    View test on GitHub ↗
  726. ✓test_spend_route_responsive[/user/list]spend_tracking/test_spend_routes.py›
    • spend-budgets
    • user_management

    No steps recorded

    View test on GitHub ↗
  727. ✓test_streamed_call_records_and_bills_the_served_tier TestServiceTierPricingspend_tracking/test_service_tier_pricing_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named tier-priced-stream-bac815899242 that calls openai/gpt-5.6-luna
    3. Send a streaming /chat/completions request to tier-priced-stream-bac815899242
    View test on GitHub ↗
  728. ✓test_streaming_cache_read_billed_at_cache_read_rate TestCacheCostAccountingspend_tracking/test_cache_cost_accounting_e2e.py›
    • spend-budgets
    • stream
    • openai
    • openai/gpt-5.5
    • prompt_caching
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named stream-cache-priced-dbde7c3e2084 that calls openai/gpt-5.5
    3. Send a /chat/completions request to stream-cache-priced-dbde7c3e2084
    4. Send a streaming /chat/completions request to stream-cache-priced-dbde7c3e2084
    View test on GitHub ↗
  729. ✓test_streaming_chat_completion_tracks_spendspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • chat_completions
    • stream
    • gemini
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a streaming /chat/completions request to gemini-2.5-flash
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  730. ✓test_streaming_messages_via_responses_bridge_tracks_spendspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • messages
    • stream
    • openai
    • openai-responses-codex
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a streaming /v1/messages request to openai-responses-codex
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  731. ✓test_tag_budget_blocks_tagged_requestsbudgets/test_tag_budget_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  732. ✓test_tag_spend_matches_sum_of_tagged_logsspend_tracking/test_spend_tracking_e2e.py›
    • spend-budgets
    • nonstream
    • gemini
    • gemini-2.5-flash
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Wait for at least 2 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  733. ✓test_team_budget_blocks_every_team_key TestBudgetBlocksPerLevelbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with team id: 9c0008ee-a396-4d92-90bb-3399905d0739
    View test on GitHub ↗
  734. ✓test_team_budget_resets_after_window TestBudgetResetPerLevelbudgets/test_budget_reset_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with team id: 7b6a01ee-e1e4-480c-b9d3-92935dd33279
    View test on GitHub ↗
  735. ✓test_team_key_blocks_over_its_own_budget TestKeyBudgetBlocksAcrossKeyKindsbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06 and team id: 66a78cbc-2111-4b3c-bd74-7dcb5ded0b1a
    2. Generate a virtual key with team id: 66a78cbc-2111-4b3c-bd74-7dcb5ded0b1a
    View test on GitHub ↗
  736. ✓test_team_key_resets_after_window TestKeyBudgetResetAcrossKeyKindsbudgets/test_budget_reset_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06, budget duration: 30s and team id: 653bf54d-373f-46d4-83a1-2fe6b33a40aa
    View test on GitHub ↗
  737. ✓test_team_long_window_blocks_after_short_window_resetsbudgets/test_team_multi_window_budget_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with models: claude-haiku-4-5 and team id: b054669b-a8ce-4e67-96e3-5939c48c539d
    View test on GitHub ↗
  738. ✓test_team_member_budget_reset_at_advancesbudgets/test_budget_reset_advances_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with user id: 3b4665c9-e184-477b-9a1d-242fbeba8288 and team id: 86c2a5ea-f854-4f16-a835-461798a307e5
    View test on GitHub ↗
  739. ✓test_team_member_budget_reset_keeps_advancingbudgets/test_team_member_budget_reset_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with user id: c81ea46f-1a02-4972-aff5-7bc92fcea2be and team id: 25df41dc-d1ec-4595-b43f-edf43451a380
    View test on GitHub ↗
  740. ✓test_team_member_key_blocks_over_its_own_budget TestKeyBudgetBlocksAcrossKeyKindsbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06, user id: c7ac4688-112a-4cde-a441-3045cef3b03d and team id: 3d1cefcc-d2c5-43b3-8586-93126446d4fb
    2. Generate a virtual key with user id: c7ac4688-112a-4cde-a441-3045cef3b03d and team id: 3d1cefcc-d2c5-43b3-8586-93126446d4fb
    View test on GitHub ↗
  741. ✓test_team_member_key_resets_after_window TestKeyBudgetResetAcrossKeyKindsbudgets/test_budget_reset_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with max budget: 3e-06, budget duration: 30s, user id: e386739d-676b-483f-adb3-0813122a95d1 and team id: 20fdd861-80dc-45bd-89c0-8fbb9c3bb41d
    View test on GitHub ↗
  742. ✓test_team_short_window_blocks_then_resetsbudgets/test_team_multi_window_budget_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with models: claude-haiku-4-5 and team id: 46ad9d70-fdcb-4ae5-897a-ede7e576648f
    View test on GitHub ↗
  743. ✓test_terminal_batch_cost_row_joins_the_retrieving_key TestKeyAttributionspend_tracking/test_key_attribution_e2e.py›
    • spend-budgets
    • batches
    • batch
    • openai
    • openai-gpt-4o-mini
    1. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  744. ✓test_the_budget_read_reports_the_spend_drawn_against_the_pool TestModelAccessGroupBudgetbudgets/test_model_access_group_budget_e2e.py›
    • spend-budgets
    • budget_management
    • openai
    • openai/gpt-5.4-nano

    No steps recorded

    View test on GitHub ↗
  745. ✓test_the_key_that_drained_the_pool_stays_blocked TestModelAccessGroupBudgetbudgets/test_model_access_group_budget_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • openai/gpt-5.4-nano
    1. Generate a virtual key with models: e2e-mag-budget-e9801ef4f868
    2. Add a deployment named e2e-mag-budgeted-e9801ef4f868 that calls openai/gpt-5.4-nano
    3. Add a deployment named e2e-mag-unbudgeted-e9801ef4f868 that calls openai/gpt-5.4-nano
    View test on GitHub ↗
  746. ✓test_tpm_limit_blocks_over_limit TestKeyRateLimitsratelimit/test_rate_limit_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with models: claude-haiku-4-5 and tpm limit: 60
    2. Read the key's settings back from /key/info
    3. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 261d53e92d06"
    4. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word d3220adbbe19"
    5. Send a /chat/completions request to claude-haiku-4-5 with the prompt "reply with one word 6f6393e7ced5"
    View test on GitHub ↗
  747. ✓test_user_budget_blocks_a_second_key TestUserBudgetAcrossKeysbudgets/test_user_budget_across_keys_e2e.py›
    • spend-budgets
    • nonstream
    • openai
    • gpt-5.5
    1. Generate a virtual key with user id: 6b4d8f18-6d0b-4e9a-a773-4ef94da38a2d
    View test on GitHub ↗
  748. ✓test_user_budget_enforced_across_their_personal_keys TestBudgetBlocksPerLevelbudgets/test_budget_enforcement_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • claude-haiku-4-5
    1. Generate a virtual key with user id: 8731b813-861e-4506-aaed-0065cc4fb893
    2. Generate a virtual key with user id: 8731b813-861e-4506-aaed-0065cc4fb893 and team id: 5bcfef88-becf-4db6-b4a8-6a29d93b24cc
    View test on GitHub ↗
  749. ✓test_user_daily_activity_reports_alias_and_email TestKeyAttributionspend_tracking/test_key_attribution_e2e.py›
    • spend-budgets
    • spend_reporting
    • anthropic
    • gemini
    • openai
    • claude-haiku-4-5
    • gemini-2.5-flash
    • openai-gpt-4o-mini
    • openai-responses-codex
    • openai-text-embedding-3-small

    No steps recorded

    View test on GitHub ↗
  750. ✓test_valid_date_range_returns_results_and_metadata TestTeamDailyActivityspend_tracking/test_team_daily_activity_e2e.py›
    • spend-budgets
    • spend_reporting
    • openai
    • openai/gpt-5.6-luna
    1. Add a deployment named e2e-reconciliation-33a23072ba35 that calls openai/gpt-5.6-luna
    2. Create a team with team alias: e2e-spend-1a2f4565f43b
    3. Generate a virtual key with models: e2e-reconciliation-33a23072ba35 and team id: e4eb2e83-1d4a-47d6-9e56-6cd172a708f0
    4. Send a /chat/completions request to e2e-reconciliation-33a23072ba35
    5. Create a team with team alias: e2e-spend-e20c7d7be9bf
    6. Generate a virtual key with models: e2e-reconciliation-33a23072ba35 and team id: 8a8054cf-be56-4a66-91c5-99ab6c4d0c7b
    7. Send a /chat/completions request to e2e-reconciliation-33a23072ba35
    8. Wait for at least 7 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  751. −test_end_user_model_max_budget_enforces_per_model_rpmbudgets/test_model_max_budget_e2e.py›
    • spend-budgets
    • nonstream
    • gemini
    • gemini-2.5-flash

    No steps recorded

    stage red: product gap, end-user model_max_budget rpm_limit is stored but never enforced

    View test on GitHub ↗
  752. −test_strict_mode_blocks_saturated_priority_but_serves_the_other TestDynamicRateLimitPriorityratelimit/test_dynamic_rate_limit_priority_e2e.py›
    • spend-budgets
    • nonstream
    • anthropic
    • anthropic/claude-haiku-4-5-20251001

    No steps recorded

    LIT-5118: the stage proxy does not run the dynamic_rate_limiter_v3 callbacks + priority_reservation config this module's docstring requires (zero limiter log lines on any pod during the 2026-08-02 run), so strict enforcement can never engage there

    View test on GitHub ↗
  753. ✓test_access_group_info_reports_membership TestModelAccessGroupRoutestest_model_tag_accessgroup_e2e.py›
    1. Add a deployment named e2e-mgmt-agmodel-614b46cca0e7 that calls openai/gpt-5.5
    View test on GitHub ↗
  754. ✓test_actor_subject_and_database_role[internal_user_viewer] TestJwtManagementtest_jwt_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  755. ✓test_actor_subject_and_database_role[internal_user] TestJwtManagementtest_jwt_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  756. ✓test_actor_subject_and_database_role[organization_admin] TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-team-ec5471d59f1a and organization id: a207494c-0adc-4293-aa6e-e8ac10fbddfd
    View test on GitHub ↗
  757. ✓test_actor_subject_and_database_role[proxy_admin_viewer] TestJwtManagementtest_jwt_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  758. ✓test_actor_subject_and_database_role[proxy_admin] TestJwtManagementtest_jwt_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  759. ✓test_actor_subject_and_database_role[team_admin] TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-team-784a92417a90 and organization id: 07ebcb42-a0ec-4f76-b71d-536de70373c4
    View test on GitHub ↗
  760. ✓test_actor_subject_and_database_role[team_member] TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-team-39e376352a7e and organization id: 4d9e521b-d299-40a6-9d07-807160bd389b
    View test on GitHub ↗
  761. ✓test_actor_subject_and_database_role[unrelated_user] TestJwtManagementtest_jwt_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  762. ✓test_admin_creates_reads_updates_clears_and_deletes_a_key[direct_jwt] TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-team-2c16a005fae2 and organization id: dc143ed1-6d1a-4c19-89ab-45464139fe7a
    View test on GitHub ↗
  763. ✓test_admin_creates_reads_updates_clears_and_deletes_a_key[virtual_key] TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-team-58d70f119c0b and organization id: 9718f104-7c7c-4640-aba4-869286fac169
    View test on GitHub ↗
  764. ✓test_admin_viewer_reads_but_cannot_update TestJwtManagementtest_jwt_management_e2e.py›
    1. Read the key's settings back from /key/info
    View test on GitHub ↗
  765. ✓test_approve_activates_submission_and_persists TestMcpServerSubmissiontest_config_misc_endpoints_e2e.py›
    1. Generate a virtual key with team id: de3372bd-379b-4ccb-bcfb-692005bd2660
    View test on GitHub ↗
  766. ✓test_bedrock_mantle_responses_connection_succeeds TestModelTestConnectiontest_model_test_connection_e2e.py›

    No steps recorded

    View test on GitHub ↗
  767. ✓test_block_persists_to_key_info TestKeyRoutestest_management_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  768. ✓test_block_then_unblock_persists_to_model_info TestModelRoutestest_model_tag_accessgroup_e2e.py›
    1. Add a deployment named e2e-mgmt-model-block-bc3520fcf294 that calls openai/gpt-5.5
    View test on GitHub ↗
  769. ✓test_block_then_unblock_persists_to_team_info TestTeamManagementRoutestest_team_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  770. ✓test_bulk_update_applies_max_budget_to_target_key TestKeyManagementRoutestest_key_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5 and max budget: 5.0
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  771. ✓test_callbacks_list_reports_active_logging_callbacks TestInventoryRoutestest_config_misc_endpoints_e2e.py›

    No steps recorded

    View test on GitHub ↗
  772. ✓test_clearing_the_description_with_null_reads_back_null TestMcpServerLifecycletest_mcp_lifecycle_e2e.py›
    1. Read /v1/mcp/server/4357574d-16ce-499e-8faa-c6fda5313ba8 on every proxy replica until it settles
    View test on GitHub ↗
  773. ✓test_clearing_the_description_with_null_reads_back_null TestMcpToolsetLifecycletest_mcp_lifecycle_e2e.py›
    1. Create an MCP toolset with the tools server id: d822ca00-c022-470a-be9c-0c233668113f and tool name: search_datadog_logs, server id: d822ca00-c022-470a-be9c-0c233668113f and tool name: get_datadog_metr
    2. Update an MCP toolset with toolset id: 4f801d43-436a-4d4b-a67e-f88154df769b
    3. Read /v1/mcp/toolset/4f801d43-436a-4d4b-a67e-f88154df769b on every proxy replica until it settles
    View test on GitHub ↗
  774. ✓test_config_update_persists_router_setting_to_get TestRouterSettingstest_config_misc_endpoints_e2e.py›

    No steps recorded

    View test on GitHub ↗
  775. ✓test_create_echoes_every_field_written TestKeyLifecycletest_key_lifecycle_e2e.py›
    1. Add a deployment named e2e-key-lifecycle-5fd471f7b174 that calls gpt-4o-mini
    View test on GitHub ↗
  776. ✓test_create_persists_and_is_read_back TestFallbackManagementtest_config_misc_endpoints_e2e.py›
    1. Add a deployment named e2e-fallback-primary-961666f212bf that calls openai/gpt-5.5
    2. Add a deployment named e2e-fallback-secondary-2538416f15ef that calls openai/gpt-5.5
    View test on GitHub ↗
  777. ✓test_create_persists_both_tools_under_the_exact_names_written TestMcpToolsetLifecycletest_mcp_lifecycle_e2e.py›
    1. Create an MCP toolset with the tools server id: 0e854cfe-885d-46bc-8460-f98f38774f11 and tool name: search_datadog_logs, server id: 0e854cfe-885d-46bc-8460-f98f38774f11 and tool name: get_datadog_metr
    2. Read /v1/mcp/toolset/ca3e6b7c-ce20-4459-a7ed-2b48465d05b1 on every proxy replica until it settles
    3. Read /v1/mcp/toolset on every proxy replica until it settles
    View test on GitHub ↗
  778. ✓test_create_persists_every_field_on_every_replica TestMcpServerLifecycletest_mcp_lifecycle_e2e.py›
    1. Read /v1/mcp/server/ab454f4a-a65c-4965-8a87-f4f1d9985511 on every proxy replica until it settles
    View test on GitHub ↗
  779. ✓test_created_budget_appears_in_budget_list TestBudgetManagementtest_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  780. ✓test_created_key_appears_in_key_list_inventory TestKeyRoutestest_management_e2e.py›
    1. Generate a virtual key with key alias: e2e-mgmt-keylist-aeccdcfb225f
    View test on GitHub ↗
  781. ✓test_created_team_appears_in_team_list TestTeamRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  782. ✓test_created_users_appear_in_user_list TestUserRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  783. ✓test_creating_a_key_from_the_dashboard_persists_and_works TestDashboardKeyRoutestest_management_e2e.py›
    1. Read the key's settings back from /key/info
    View test on GitHub ↗
  784. ✓test_credential_migration_check_reports_residual_scan TestInventoryRoutestest_config_misc_endpoints_e2e.py›

    No steps recorded

    View test on GitHub ↗
  785. ✓test_customer_create_persists_to_info TestCustomertest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  786. ✓test_delete_forbidden_for_non_admin_key TestKeyManagementRoutestest_key_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  787. ✓test_delete_persists_and_revokes_team_bound_key TestTeamRoutestest_management_e2e.py›
    1. Generate a virtual key with team id: 0e6438bd-83a9-4efe-80c8-f01838bdfba6
    View test on GitHub ↗
  788. ✓test_delete_removes_from_model_info_catalog TestModelRoutestest_management_e2e.py›
    1. Add a deployment named e2e-mgmt-model-edac42ed22af that calls openai/gpt-5.5
    2. List the deployments from /model/info
    View test on GitHub ↗
  789. ✓test_delete_removes_from_organization_info TestOrganizationRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  790. ✓test_delete_removes_the_customer TestCustomerManagementtest_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  791. ✓test_delete_removes_the_server_from_every_replica TestMcpServerLifecycletest_mcp_lifecycle_e2e.py›
    1. Check that /v1/mcp/server/acd4d858-c269-4178-9959-0bf64a29b5fc returns 404 on every proxy replica
    2. Read /v1/mcp/server on every proxy replica until it settles
    View test on GitHub ↗
  792. ✓test_delete_removes_the_toolset_from_every_replica TestMcpToolsetLifecycletest_mcp_lifecycle_e2e.py›
    1. Create an MCP toolset with the tools server id: 276c933e-44b7-477e-825a-87022d4d463f and tool name: search_datadog_logs, server id: 276c933e-44b7-477e-825a-87022d4d463f and tool name: get_datadog_metr
    2. Delete the MCP toolset
    3. Check that /v1/mcp/toolset/73077a1f-5465-43c8-9a2a-cc0ef76f8b05 returns 404 on every proxy replica
    4. Read /v1/mcp/toolset on every proxy replica until it settles
    View test on GitHub ↗
  793. ✓test_delete_removes_the_user_from_inventory TestUserRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  794. ✓test_delete_revokes_info_and_chat_on_every_replica TestKeyLifecycletest_key_lifecycle_e2e.py›
    1. Read /key/info on every proxy replica until they all agree
    View test on GitHub ↗
  795. ✓test_delete_revokes_the_key_on_chat TestKeyRoutestest_management_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash
    View test on GitHub ↗
  796. ✓test_editing_a_key_from_the_dashboard_persists_and_is_enforced TestDashboardKeyRoutestest_management_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash, key alias: e2e-mgmt-uiedit-d2837bacfd22, tpm limit: 100 and rpm limit: 200
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  797. ✓test_end_user_new_persists_to_end_user_info TestCustomerManagementtest_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  798. ✓test_estimate_computes_cost_from_token_counts TestCostEstimatetest_config_misc_endpoints_e2e.py›

    No steps recorded

    View test on GitHub ↗
  799. ✓test_explicit_null_clears_the_budget_and_its_reset_time TestKeyLifecycletest_key_lifecycle_e2e.py›
    1. Read /key/info on every proxy replica until they all agree
    View test on GitHub ↗
  800. ✓test_gdpr_check_derives_verdict_from_the_request TestComplianceRoutestest_config_misc_endpoints_e2e.py›

    No steps recorded

    View test on GitHub ↗
  801. ✓test_generate_forbidden_for_non_admin_key TestKeyManagementRoutestest_key_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5
    View test on GitHub ↗
  802. ✓test_generate_persists_to_key_info_and_scopes_chat TestKeyRoutestest_management_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash, key alias: e2e-mgmt-key-e6002cb32e25, tpm limit: 424242 and rpm limit: 424243
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  803. ✓test_health_reports_the_calling_key_healthy TestKeyManagementRoutestest_key_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5
    View test on GitHub ↗
  804. ✓test_info_reflects_the_fields_the_key_was_created_with TestKeyManagementRoutestest_key_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5, gemini-2.5-flash, key alias: e2e-mgmt-keyinfo-aea40b8cc287, tpm limit: 131313 and rpm limit: 141414
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  805. ✓test_info_returns_created_team_fields TestTeamManagementRoutestest_team_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  806. ✓test_is_null_finds_the_budget_left_uncapped TestBudgetListV1test_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  807. ✓test_is_refused_for_a_non_admin_key TestBudgetListV1test_budget_customer_user_org_e2e.py›
    1. Generate a virtual key with default settings
    View test on GitHub ↗
  808. ✓test_key_delete_by_alias_writes_audit_row TestKeyDeletionAuditLogtest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  809. ✓test_key_delete_by_key_writes_audit_row TestKeyDeletionAuditLogtest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  810. ✓test_key_serves_its_model_and_is_denied_others TestKeyLifecycletest_key_lifecycle_e2e.py›

    No steps recorded

    View test on GitHub ↗
  811. ✓test_llm_only_key_forbidden_from_management_writes TestManagementRoutePermissionstest_management_e2e.py›
    1. Generate a virtual key with allowed routes: llm_api_routes
    View test on GitHub ↗
  812. ✓test_member_add_and_delete_persist_to_team_info TestTeamRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  813. ✓test_member_add_forbidden_to_plain_member TestTeamManagementRoutestest_team_management_e2e.py›
    1. Generate a virtual key with user id: d8dde7a3-0cd8-4c0e-a929-32e95634697d and team id: a429f921-88c8-4944-9494-9a0bf895c104
    View test on GitHub ↗
  814. ✓test_member_add_records_membership TestOrganizationMembershiptest_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  815. ✓test_member_cannot_write_and_another_team_cannot_read_the_key TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-jwt-394f3a35ea5b and team id: e2e-jwt-team-394f3a35ea5b
    2. Create a team with team alias: 9e878514954c and team id: e2e-jwt-team-9e878514954c
    View test on GitHub ↗
  816. ✓test_member_delete_forbidden_to_plain_member TestTeamManagementRoutestest_team_management_e2e.py›
    1. Generate a virtual key with user id: 00a44a1f-56c0-4f9e-88b6-06e61d610d3c and team id: 149c08b5-0d51-40f0-a618-4b4425fb55d7
    View test on GitHub ↗
  817. ✓test_member_delete_persists_to_team_info TestTeamManagementRoutestest_team_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  818. ✓test_member_update_persists_role_and_budget TestTeamManagementRoutestest_team_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  819. ✓test_multi_group_actor_keeps_exact_memberships TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-team-3fb8e74227e8 and organization id: d0f0c5f9-8e13-4651-8ec8-c04216fcd29b
    2. Create a team with team alias: e2e-team-4812cb7b7bd8 and organization id: 448b5d71-bbe4-44c2-8035-3b30b0b34090
    View test on GitHub ↗
  820. ✓test_new_access_group_tags_the_deployment TestModelAccessGroupRoutestest_model_tag_accessgroup_e2e.py›
    1. Add a deployment named e2e-mgmt-agmodel-ed259ea2f160 that calls openai/gpt-5.5
    View test on GitHub ↗
  821. ✓test_new_is_denied_to_non_admin_keys TestTeamManagementRoutestest_team_management_e2e.py›
    1. Generate a virtual key with default settings
    2. Generate a virtual key with user id: a8e45023-8eb4-453b-ba90-00eccc1a7020
    View test on GitHub ↗
  822. ✓test_new_is_refused_for_a_non_admin_key TestBudgetManagementtest_budget_customer_user_org_e2e.py›
    1. Generate a virtual key with default settings
    View test on GitHub ↗
  823. ✓test_new_persists_mapping_and_is_read_back TestJwtKeyMappingtest_config_misc_endpoints_e2e.py›
    1. Generate a virtual key with default settings
    View test on GitHub ↗
  824. ✓test_new_persists_to_customer_info TestCustomerManagementtest_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  825. ✓test_new_persists_to_model_info_catalog TestModelRoutestest_management_e2e.py›
    1. Add a deployment named e2e-mgmt-model-a657125f7a59 that calls openai/gpt-5.5
    2. List the deployments from /model/info
    View test on GitHub ↗
  826. ✓test_new_persists_to_organization_info TestOrganizationRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  827. ✓test_new_persists_to_tag_list TestTagRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  828. ✓test_new_persists_to_team_info_and_binds_keys TestTeamRoutestest_management_e2e.py›
    1. Generate a virtual key with team id: 6e4f3945-6a8a-4e7f-bfec-282b972f43b6
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  829. ✓test_new_persists_to_user_info TestUserRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  830. ✓test_new_user_is_readable_via_user_info TestUserManagementtest_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  831. ✓test_non_admin_key_cannot_add_global_model TestModelRoutestest_model_tag_accessgroup_e2e.py›
    1. Generate a virtual key with default settings
    2. List the deployments from /model/info
    View test on GitHub ↗
  832. ✓test_oidc_browser_profile_identity_mapping TestJwtManagementtest_jwt_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  833. ✓test_partial_setup_removes_previously_created_identities[group] TestJwtManagementtest_jwt_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  834. ✓test_partial_setup_removes_previously_created_identities[user] TestJwtManagementtest_jwt_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  835. ✓test_partial_update_changes_only_the_named_field TestKeyLifecycletest_key_lifecycle_e2e.py›
    1. Read /key/info on every proxy replica until they all agree
    View test on GitHub ↗
  836. ✓test_project_detachment_preserves_key_scope_and_refreshes_auth TestKeyManagementRoutestest_key_management_e2e.py›
    1. Add a deployment named e2e-detach-3c3a724b1f73 that calls openai/synthetic-detachment
    2. Generate a virtual key with models: e2e-detach-3c3a724b1f73, max budget: 5.0, team id: ad90cabb-124f-4ec7-8c2f-aec0e59cbc4b, project id: 25b6a17a-8ac2-4727-b187-aa402419f644, organization id: f2acfbf0
    3. Read the key's settings back from /key/info
    View test on GitHub ↗
  837. ✓test_read_reflects_the_create_on_every_replica TestKeyLifecycletest_key_lifecycle_e2e.py›
    1. Read /key/info on every proxy replica until they all agree
    View test on GitHub ↗
  838. ✓test_refuses_a_sort_field_and_a_parameter_it_does_not_support TestBudgetListV1test_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  839. ✓test_regenerate_rotates_to_a_working_new_key TestKeyRegenerationtest_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5
    View test on GitHub ↗
  840. ✓test_regenerate_with_grace_period_keeps_old_key_until_revoked TestKeyRegenerationtest_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5
    View test on GitHub ↗
  841. ✓test_register_submits_pending_server TestMcpServerSubmissiontest_config_misc_endpoints_e2e.py›
    1. Generate a virtual key with team id: 7a0e8e5f-9424-409f-a1bc-4b8636693bf1
    View test on GitHub ↗
  842. ✓test_reset_spend_zeroes_recorded_spend_and_lifts_the_budget_block TestKeyManagementRoutestest_key_management_e2e.py›
    1. Generate a virtual key with models: claude-haiku-4-5 and max budget: 3e-06
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  843. ✓test_sorts_pages_and_filters_the_budgets_it_created TestBudgetListV1test_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  844. ✓test_successful_actor_cleanup_removes_owned_state TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-team-2b26fd81b102 and organization id: bf579a72-9ea6-4065-af02-ac5c7f452abc
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  845. ✓test_tag_delete_removes_from_list TestTagRoutestest_model_tag_accessgroup_e2e.py›

    No steps recorded

    View test on GitHub ↗
  846. ✓test_tag_list_reports_created_tag TestTagRoutestest_model_tag_accessgroup_e2e.py›

    No steps recorded

    View test on GitHub ↗
  847. ✓test_team_admin_cannot_change_a_setting_that_is_not_enabled[budget_duration] TestTeamAdminWithTpmLimitEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 124eb498-cf03-4b5c-9955-6c799367e43f
    View test on GitHub ↗
  848. ✓test_team_admin_cannot_change_a_setting_that_is_not_enabled[max_budget] TestTeamAdminWithTpmLimitEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 61b1efe9-e90a-4bc9-98d8-742578e250cf
    View test on GitHub ↗
  849. ✓test_team_admin_cannot_change_a_setting_that_is_not_enabled[metadata] TestTeamAdminWithTpmLimitEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 80c041a3-4f89-4a66-b02c-723362834b08
    View test on GitHub ↗
  850. ✓test_team_admin_cannot_change_a_setting_that_is_not_enabled[models] TestTeamAdminWithTpmLimitEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: f9ac961c-1733-4e27-acfa-bf0c14378d18
    View test on GitHub ↗
  851. ✓test_team_admin_cannot_change_a_setting_that_is_not_enabled[rpm_limit] TestTeamAdminWithTpmLimitEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 80ee747b-66ff-4927-8a0e-d9fac6150478
    View test on GitHub ↗
  852. ✓test_team_admin_cannot_change_a_setting_that_is_not_enabled[team_alias] TestTeamAdminWithTpmLimitEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 202ee819-378b-4932-b83e-7b98fd4c3ae0
    View test on GitHub ↗
  853. ✓test_team_admin_cannot_change_any_team_setting TestTeamAdminWithNoEditableFieldstest_team_management_e2e.py›
    1. Generate a virtual key with user id: 49cfe4d5-d177-4d87-a937-1608999d0490
    View test on GitHub ↗
  854. ✓test_team_admin_cannot_raise_an_org_team_budget_under_the_org_cap TestTeamAdminWithRpmLimitAndMaxBudgetEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 88939454-446d-48c6-a00f-4449da26e123
    View test on GitHub ↗
  855. ✓test_team_admin_cannot_raise_or_remove_the_budget[raise] TestTeamAdminWithRpmLimitAndMaxBudgetEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: bfdf54c1-1ef4-4975-94ff-82cca1270bb9
    View test on GitHub ↗
  856. ✓test_team_admin_cannot_raise_or_remove_the_budget[remove] TestTeamAdminWithRpmLimitAndMaxBudgetEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 6c4cc69b-f8c9-4409-992e-0dddacda8a25
    View test on GitHub ↗
  857. ✓test_team_admin_resending_the_budget_settings_keeps_the_next_budget_reset TestTeamAdminWithTpmLimitEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 03efbd65-ca13-45bc-849d-f3ea4bee8992
    View test on GitHub ↗
  858. ✓test_team_admin_saves_a_new_rpm_limit_and_a_tighter_budget[first-budget] TestTeamAdminWithRpmLimitAndMaxBudgetEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 598710a8-1707-4003-b914-09059019e9a1
    View test on GitHub ↗
  859. ✓test_team_admin_saves_a_new_rpm_limit_and_a_tighter_budget[lower] TestTeamAdminWithRpmLimitAndMaxBudgetEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: c12997e0-edb4-47be-b16a-07739af62881
    View test on GitHub ↗
  860. ✓test_team_admin_saves_the_settings_form_with_a_new_tpm_limit TestTeamAdminWithTpmLimitEnabledtest_team_management_e2e.py›
    1. Generate a virtual key with user id: 8bc79174-9175-4f10-b6e0-5fc294c6f7b8
    View test on GitHub ↗
  861. ✓test_team_delete_succeeds_for_team_larger_than_db_pool TestTeamRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  862. ✓test_team_delete_writes_audit_row_for_team_keys TestKeyDeletionAuditLogtest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  863. ✓test_team_member_delete_writes_audit_row_for_member_keys TestKeyDeletionAuditLogtest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  864. ✓test_tool_list_returns_catalog_with_consistent_total TestInventoryRoutestest_config_misc_endpoints_e2e.py›

    No steps recorded

    View test on GitHub ↗
  865. ✓test_two_actor_sets_keep_tenants_and_keys_isolated TestJwtManagementtest_jwt_management_e2e.py›
    1. Create a team with team alias: e2e-team-d12c9be0edd2 and organization id: 348aac55-1408-4e71-a93c-bed438580bf3
    2. Create a team with team alias: e2e-team-dccd8b363f94 and organization id: 61a89ea6-8d64-4ba5-bd75-1ccad49c5595
    3. Read the key's settings back from /key/info
    View test on GitHub ↗
  866. ✓test_unblock_flips_key_info_blocked_back TestKeyManagementRoutestest_key_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  867. ✓test_update_accepts_per_model_budgets_including_punctuated_names TestBudgetManagementtest_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  868. ✓test_update_alias_persists_to_organization_info TestOrganizationRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  869. ✓test_update_forbidden_for_non_admin_key TestKeyManagementRoutestest_key_management_e2e.py›
    1. Generate a virtual key with models: gpt-5.5
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  870. ✓test_update_max_budget_persists_to_budget_info TestBudgetManagementtest_budget_customer_user_org_e2e.py›

    No steps recorded

    View test on GitHub ↗
  871. ✓test_update_models_persists_and_flips_enforcement TestKeyRoutestest_management_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  872. ✓test_update_persists_input_cost_to_model_info TestModelRoutestest_management_e2e.py›
    1. Add a deployment named e2e-mgmt-model-40a76ebbef1e that calls gpt-4o-mini
    2. List the deployments from /model/info
    3. Update a deployment's settings to model: gpt-4o-mini and input cost per token: 2.22e-06
    4. List the deployments from /model/info
    View test on GitHub ↗
  873. ✓test_update_persists_to_team_info TestTeamRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  874. ✓test_update_persists_to_user_info TestUserRoutestest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  875. ✓test_updating_only_the_alias_keeps_every_other_field_on_every_replica TestMcpServerLifecycletest_mcp_lifecycle_e2e.py›
    1. Read /v1/mcp/server/a4b4eeda-d4e4-493b-b880-a440f6977049 on every proxy replica until it settles
    View test on GitHub ↗
  876. ✓test_updating_only_the_description_keeps_the_tools_and_name TestMcpToolsetLifecycletest_mcp_lifecycle_e2e.py›
    1. Create an MCP toolset with the tools server id: a61b123b-ae46-4434-8685-a4a7fda0f219 and tool name: search_datadog_logs, server id: a61b123b-ae46-4434-8685-a4a7fda0f219 and tool name: get_datadog_metr
    2. Update an MCP toolset with toolset id: 90f5efd6-0c13-4486-b6a4-74e01ed0cc13 and description: edited
    3. Read /v1/mcp/toolset/90f5efd6-0c13-4486-b6a4-74e01ed0cc13 on every proxy replica until it settles
    View test on GitHub ↗
  877. ✓test_updating_the_tools_to_one_entry_reads_back_exactly_that_entry TestMcpToolsetLifecycletest_mcp_lifecycle_e2e.py›
    1. Create an MCP toolset with the tools server id: 1cd456c0-7f74-4d05-81ff-f784bffe6d9f and tool name: search_datadog_logs, server id: 1cd456c0-7f74-4d05-81ff-f784bffe6d9f and tool name: get_datadog_metr
    2. Update an MCP toolset with toolset id: 9dc3b4c8-c112-4c84-b7c5-9228f425917c and tools: server id: 1cd456c0-7f74-4d05-81ff-f784bffe6d9f and tool name: search_datadog_logs
    3. Read /v1/mcp/toolset/9dc3b4c8-c112-4c84-b7c5-9228f425917c on every proxy replica until it settles
    View test on GitHub ↗
  878. ✓test_user_delete_writes_audit_row_for_user_keys TestKeyDeletionAuditLogtest_management_e2e.py›

    No steps recorded

    View test on GitHub ↗
  879. ✓test_workflow_runs_list_returns_consistent_count TestInventoryRoutestest_config_misc_endpoints_e2e.py›

    No steps recorded

    View test on GitHub ↗
  880. −test_created_server_is_listed_with_every_field TestMcpServerLifecycletest_mcp_lifecycle_e2e.py›

    No steps recorded

    product gap: GET /v1/mcp/server builds each row from the in-memory registry, whose _build_mcp_server_table sets description from mcp_info['description'], so the list reports the mcp_info description while GET /v1/mcp/server/{server_id} reports the stored description column. A server created with both set to different text reads back with two different descriptions depending on the route

    View test on GitHub ↗
  881. ✓test_a_failed_first_turn_is_reported_as_the_first_turn_probe_unit_tests/test_http_probe.py›

    No steps recorded

    View test on GitHub ↗
  882. ✓test_a_history_with_no_server_tool_block_is_not_a_pass_probe_unit_tests/test_http_probe.py›

    No steps recorded

    View test on GitHub ↗
  883. ✓test_a_pending_tool_use_is_answered_with_the_id_the_model_returned_probe_unit_tests/test_http_probe.py›

    No steps recorded

    View test on GitHub ↗
  884. ✓test_a_turn_truncated_before_the_result_block_is_not_a_pass_probe_unit_tests/test_http_probe.py›

    No steps recorded

    View test on GitHub ↗
  885. ✓test_a_turn_with_no_pending_tool_use_gets_a_plain_follow_up_probe_unit_tests/test_http_probe.py›

    No steps recorded

    View test on GitHub ↗
  886. ✓test_accepts_a_replayed_server_tool_pair_probe_unit_tests/test_http_probe.py›

    No steps recorded

    View test on GitHub ↗
  887. ✓test_basic_messaging_non_streaming_anthropicbasic_messaging_non_streaming/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  888. ✓test_basic_messaging_non_streaming_azurebasic_messaging_non_streaming/test_azure.py›
    1. Add a deployment named claude-haiku-4-5 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named claude-sonnet-4-5 that calls anthropic/claude-sonnet-4-5
    3. Add a deployment named claude-opus-4-7 that calls anthropic/claude-opus-4-7
    4. Add a deployment named claude-haiku-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    5. Add a deployment named claude-sonnet-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    6. Add a deployment named claude-opus-4-7-bedrock-invoke that calls bedrock/us.anthropic.claude-opus-4-7
    7. Add a deployment named claude-haiku-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    8. Add a deployment named claude-sonnet-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    9. Add a deployment named claude-opus-4-7-bedrock-converse that calls bedrock/converse/us.anthropic.claude-opus-4-7
    10. Add a deployment named claude-haiku-4-5-vertex that calls vertex_ai/claude-haiku-4-5
    11. Add a deployment named claude-sonnet-4-5-vertex that calls vertex_ai/claude-sonnet-4-5
    12. Add a deployment named claude-opus-4-7-vertex that calls vertex_ai/claude-opus-4-7
    13. Add a deployment named claude-haiku-4-5-azure that calls azure_ai/claude-haiku-4-5
    14. Add a deployment named claude-sonnet-4-5-azure that calls azure_ai/claude-sonnet-4-5
    15. Add a deployment named claude-opus-4-7-azure that calls azure_ai/claude-opus-4-7
    16. Add a deployment named gpt-5-6-sol-openai that calls openai/gpt-5.6-sol
    17. Add a deployment named gpt-5-6-terra-openai that calls openai/gpt-5.6-terra
    18. Add a deployment named gpt-5-6-luna-openai that calls openai/gpt-5.6-luna
    19. Add a deployment named gpt-5-6-sol-azure-openai that calls azure/gpt-5.6-sol
    20. Add a deployment named gpt-5-6-terra-azure-openai that calls azure/gpt-5.6-terra
    21. Add a deployment named gpt-5-6-luna-azure-openai that calls azure/gpt-5.6-luna
    22. Add a deployment named gpt-5-6-sol-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-sol
    23. Add a deployment named gpt-5-6-terra-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-terra
    24. Add a deployment named gpt-5-6-luna-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-luna
    View test on GitHub ↗
  889. ✓test_basic_messaging_non_streaming_azure_openaibasic_messaging_non_streaming/test_azure_openai.py›
    1. Add a deployment named claude-haiku-4-5 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named claude-sonnet-4-5 that calls anthropic/claude-sonnet-4-5
    3. Add a deployment named claude-opus-4-7 that calls anthropic/claude-opus-4-7
    4. Add a deployment named claude-haiku-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    5. Add a deployment named claude-sonnet-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    6. Add a deployment named claude-opus-4-7-bedrock-invoke that calls bedrock/us.anthropic.claude-opus-4-7
    7. Add a deployment named claude-haiku-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    8. Add a deployment named claude-sonnet-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    9. Add a deployment named claude-opus-4-7-bedrock-converse that calls bedrock/converse/us.anthropic.claude-opus-4-7
    10. Add a deployment named claude-haiku-4-5-vertex that calls vertex_ai/claude-haiku-4-5
    11. Add a deployment named claude-sonnet-4-5-vertex that calls vertex_ai/claude-sonnet-4-5
    12. Add a deployment named claude-opus-4-7-vertex that calls vertex_ai/claude-opus-4-7
    13. Add a deployment named claude-haiku-4-5-azure that calls azure_ai/claude-haiku-4-5
    14. Add a deployment named claude-sonnet-4-5-azure that calls azure_ai/claude-sonnet-4-5
    15. Add a deployment named claude-opus-4-7-azure that calls azure_ai/claude-opus-4-7
    16. Add a deployment named gpt-5-6-sol-openai that calls openai/gpt-5.6-sol
    17. Add a deployment named gpt-5-6-terra-openai that calls openai/gpt-5.6-terra
    18. Add a deployment named gpt-5-6-luna-openai that calls openai/gpt-5.6-luna
    19. Add a deployment named gpt-5-6-sol-azure-openai that calls azure/gpt-5.6-sol
    20. Add a deployment named gpt-5-6-terra-azure-openai that calls azure/gpt-5.6-terra
    21. Add a deployment named gpt-5-6-luna-azure-openai that calls azure/gpt-5.6-luna
    22. Add a deployment named gpt-5-6-sol-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-sol
    23. Add a deployment named gpt-5-6-terra-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-terra
    24. Add a deployment named gpt-5-6-luna-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-luna
    View test on GitHub ↗
  890. ✓test_basic_messaging_non_streaming_bedrock_conversebasic_messaging_non_streaming/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  891. ✓test_basic_messaging_non_streaming_bedrock_invokebasic_messaging_non_streaming/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  892. ✓test_basic_messaging_non_streaming_vertex_aibasic_messaging_non_streaming/test_vertex_ai.py›
    1. Add a deployment named claude-haiku-4-5 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named claude-sonnet-4-5 that calls anthropic/claude-sonnet-4-5
    3. Add a deployment named claude-opus-4-7 that calls anthropic/claude-opus-4-7
    4. Add a deployment named claude-haiku-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    5. Add a deployment named claude-sonnet-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    6. Add a deployment named claude-opus-4-7-bedrock-invoke that calls bedrock/us.anthropic.claude-opus-4-7
    7. Add a deployment named claude-haiku-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    8. Add a deployment named claude-sonnet-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    9. Add a deployment named claude-opus-4-7-bedrock-converse that calls bedrock/converse/us.anthropic.claude-opus-4-7
    10. Add a deployment named claude-haiku-4-5-vertex that calls vertex_ai/claude-haiku-4-5
    11. Add a deployment named claude-sonnet-4-5-vertex that calls vertex_ai/claude-sonnet-4-5
    12. Add a deployment named claude-opus-4-7-vertex that calls vertex_ai/claude-opus-4-7
    13. Add a deployment named claude-haiku-4-5-azure that calls azure_ai/claude-haiku-4-5
    14. Add a deployment named claude-sonnet-4-5-azure that calls azure_ai/claude-sonnet-4-5
    15. Add a deployment named claude-opus-4-7-azure that calls azure_ai/claude-opus-4-7
    16. Add a deployment named gpt-5-6-sol-openai that calls openai/gpt-5.6-sol
    17. Add a deployment named gpt-5-6-terra-openai that calls openai/gpt-5.6-terra
    18. Add a deployment named gpt-5-6-luna-openai that calls openai/gpt-5.6-luna
    19. Add a deployment named gpt-5-6-sol-azure-openai that calls azure/gpt-5.6-sol
    20. Add a deployment named gpt-5-6-terra-azure-openai that calls azure/gpt-5.6-terra
    21. Add a deployment named gpt-5-6-luna-azure-openai that calls azure/gpt-5.6-luna
    22. Add a deployment named gpt-5-6-sol-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-sol
    23. Add a deployment named gpt-5-6-terra-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-terra
    24. Add a deployment named gpt-5-6-luna-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-luna
    View test on GitHub ↗
  893. ✓test_basic_messaging_non_streaming_vertex_ai_gptbasic_messaging_non_streaming/test_vertex_ai_gpt.py›
    1. Add a deployment named claude-haiku-4-5 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named claude-sonnet-4-5 that calls anthropic/claude-sonnet-4-5
    3. Add a deployment named claude-opus-4-7 that calls anthropic/claude-opus-4-7
    4. Add a deployment named claude-haiku-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    5. Add a deployment named claude-sonnet-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    6. Add a deployment named claude-opus-4-7-bedrock-invoke that calls bedrock/us.anthropic.claude-opus-4-7
    7. Add a deployment named claude-haiku-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    8. Add a deployment named claude-sonnet-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    9. Add a deployment named claude-opus-4-7-bedrock-converse that calls bedrock/converse/us.anthropic.claude-opus-4-7
    10. Add a deployment named claude-haiku-4-5-vertex that calls vertex_ai/claude-haiku-4-5
    11. Add a deployment named claude-sonnet-4-5-vertex that calls vertex_ai/claude-sonnet-4-5
    12. Add a deployment named claude-opus-4-7-vertex that calls vertex_ai/claude-opus-4-7
    13. Add a deployment named claude-haiku-4-5-azure that calls azure_ai/claude-haiku-4-5
    14. Add a deployment named claude-sonnet-4-5-azure that calls azure_ai/claude-sonnet-4-5
    15. Add a deployment named claude-opus-4-7-azure that calls azure_ai/claude-opus-4-7
    16. Add a deployment named gpt-5-6-sol-openai that calls openai/gpt-5.6-sol
    17. Add a deployment named gpt-5-6-terra-openai that calls openai/gpt-5.6-terra
    18. Add a deployment named gpt-5-6-luna-openai that calls openai/gpt-5.6-luna
    19. Add a deployment named gpt-5-6-sol-azure-openai that calls azure/gpt-5.6-sol
    20. Add a deployment named gpt-5-6-terra-azure-openai that calls azure/gpt-5.6-terra
    21. Add a deployment named gpt-5-6-luna-azure-openai that calls azure/gpt-5.6-luna
    22. Add a deployment named gpt-5-6-sol-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-sol
    23. Add a deployment named gpt-5-6-terra-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-terra
    24. Add a deployment named gpt-5-6-luna-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-luna
    View test on GitHub ↗
  894. ✓test_basic_messaging_streaming_anthropicbasic_messaging_streaming/test_anthropic.py›
    1. Add a deployment named claude-haiku-4-5 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named claude-sonnet-4-5 that calls anthropic/claude-sonnet-4-5
    3. Add a deployment named claude-opus-4-7 that calls anthropic/claude-opus-4-7
    4. Add a deployment named claude-haiku-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    5. Add a deployment named claude-sonnet-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    6. Add a deployment named claude-opus-4-7-bedrock-invoke that calls bedrock/us.anthropic.claude-opus-4-7
    7. Add a deployment named claude-haiku-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    8. Add a deployment named claude-sonnet-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    9. Add a deployment named claude-opus-4-7-bedrock-converse that calls bedrock/converse/us.anthropic.claude-opus-4-7
    10. Add a deployment named claude-haiku-4-5-vertex that calls vertex_ai/claude-haiku-4-5
    11. Add a deployment named claude-sonnet-4-5-vertex that calls vertex_ai/claude-sonnet-4-5
    12. Add a deployment named claude-opus-4-7-vertex that calls vertex_ai/claude-opus-4-7
    13. Add a deployment named claude-haiku-4-5-azure that calls azure_ai/claude-haiku-4-5
    14. Add a deployment named claude-sonnet-4-5-azure that calls azure_ai/claude-sonnet-4-5
    15. Add a deployment named claude-opus-4-7-azure that calls azure_ai/claude-opus-4-7
    16. Add a deployment named gpt-5-6-sol-openai that calls openai/gpt-5.6-sol
    17. Add a deployment named gpt-5-6-terra-openai that calls openai/gpt-5.6-terra
    18. Add a deployment named gpt-5-6-luna-openai that calls openai/gpt-5.6-luna
    19. Add a deployment named gpt-5-6-sol-azure-openai that calls azure/gpt-5.6-sol
    20. Add a deployment named gpt-5-6-terra-azure-openai that calls azure/gpt-5.6-terra
    21. Add a deployment named gpt-5-6-luna-azure-openai that calls azure/gpt-5.6-luna
    22. Add a deployment named gpt-5-6-sol-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-sol
    23. Add a deployment named gpt-5-6-terra-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-terra
    24. Add a deployment named gpt-5-6-luna-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-luna
    View test on GitHub ↗
  895. ✓test_basic_messaging_streaming_azurebasic_messaging_streaming/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  896. ✓test_basic_messaging_streaming_azure_openaibasic_messaging_streaming/test_azure_openai.py›

    No steps recorded

    View test on GitHub ↗
  897. ✓test_basic_messaging_streaming_bedrock_conversebasic_messaging_streaming/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  898. ✓test_basic_messaging_streaming_bedrock_invokebasic_messaging_streaming/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  899. ✓test_basic_messaging_streaming_vertex_aibasic_messaging_streaming/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  900. ✓test_basic_messaging_streaming_vertex_ai_gptbasic_messaging_streaming/test_vertex_ai_gpt.py›

    No steps recorded

    View test on GitHub ↗
  901. ✓test_bedrock_503_is_retryable_but_not_rate_limit_shaped_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  902. ✓test_count_tokens_anthropiccount_tokens/test_anthropic.py›
    1. Count tokens with /v1/messages/count_tokens for claude-haiku-4-5
    2. Count tokens with /v1/messages/count_tokens for claude-sonnet-4-5
    3. Count tokens with /v1/messages/count_tokens for claude-opus-4-7
    View test on GitHub ↗
  903. ✓test_count_tokens_azurecount_tokens/test_azure.py›
    1. Count tokens with /v1/messages/count_tokens for claude-haiku-4-5-azure
    2. Count tokens with /v1/messages/count_tokens for claude-sonnet-4-5-azure
    3. Count tokens with /v1/messages/count_tokens for claude-opus-4-7-azure
    View test on GitHub ↗
  904. ✓test_count_tokens_bedrock_conversecount_tokens/test_bedrock_converse.py›
    1. Count tokens with /v1/messages/count_tokens for claude-haiku-4-5-bedrock-converse
    2. Count tokens with /v1/messages/count_tokens for claude-sonnet-4-5-bedrock-converse
    3. Count tokens with /v1/messages/count_tokens for claude-opus-4-7-bedrock-converse
    View test on GitHub ↗
  905. ✓test_count_tokens_bedrock_invokecount_tokens/test_bedrock_invoke.py›
    1. Count tokens with /v1/messages/count_tokens for claude-haiku-4-5-bedrock-invoke
    2. Count tokens with /v1/messages/count_tokens for claude-sonnet-4-5-bedrock-invoke
    3. Count tokens with /v1/messages/count_tokens for claude-opus-4-7-bedrock-invoke
    View test on GitHub ↗
  906. ✓test_driver_error_message_is_classified_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  907. ✓test_find_regressions_empty_old_matrix_is_safe_builder_unit_tests/test_matrix_builder.py›

    No steps recorded

    View test on GitHub ↗
  908. ✓test_find_regressions_flags_pass_to_fail_builder_unit_tests/test_matrix_builder.py›
    1. Add a deployment named claude-haiku-4-5 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named claude-sonnet-4-5 that calls anthropic/claude-sonnet-4-5
    3. Add a deployment named claude-opus-4-7 that calls anthropic/claude-opus-4-7
    4. Add a deployment named claude-haiku-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    5. Add a deployment named claude-sonnet-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    6. Add a deployment named claude-opus-4-7-bedrock-invoke that calls bedrock/us.anthropic.claude-opus-4-7
    7. Add a deployment named claude-haiku-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    8. Add a deployment named claude-sonnet-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    9. Add a deployment named claude-opus-4-7-bedrock-converse that calls bedrock/converse/us.anthropic.claude-opus-4-7
    10. Add a deployment named claude-haiku-4-5-vertex that calls vertex_ai/claude-haiku-4-5
    11. Add a deployment named claude-sonnet-4-5-vertex that calls vertex_ai/claude-sonnet-4-5
    12. Add a deployment named claude-opus-4-7-vertex that calls vertex_ai/claude-opus-4-7
    13. Add a deployment named claude-haiku-4-5-azure that calls azure_ai/claude-haiku-4-5
    14. Add a deployment named claude-sonnet-4-5-azure that calls azure_ai/claude-sonnet-4-5
    15. Add a deployment named claude-opus-4-7-azure that calls azure_ai/claude-opus-4-7
    16. Add a deployment named gpt-5-6-sol-openai that calls openai/gpt-5.6-sol
    17. Add a deployment named gpt-5-6-terra-openai that calls openai/gpt-5.6-terra
    18. Add a deployment named gpt-5-6-luna-openai that calls openai/gpt-5.6-luna
    19. Add a deployment named gpt-5-6-sol-azure-openai that calls azure/gpt-5.6-sol
    20. Add a deployment named gpt-5-6-terra-azure-openai that calls azure/gpt-5.6-terra
    21. Add a deployment named gpt-5-6-luna-azure-openai that calls azure/gpt-5.6-luna
    22. Add a deployment named gpt-5-6-sol-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-sol
    23. Add a deployment named gpt-5-6-terra-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-terra
    24. Add a deployment named gpt-5-6-luna-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-luna
    View test on GitHub ↗
  909. ✓test_find_regressions_ignores_green_to_grey_builder_unit_tests/test_matrix_builder.py›

    No steps recorded

    View test on GitHub ↗
  910. ✓test_find_regressions_ignores_improvements_and_steady_green_builder_unit_tests/test_matrix_builder.py›

    No steps recorded

    View test on GitHub ↗
  911. ✓test_find_regressions_ignores_new_cells_without_baseline_builder_unit_tests/test_matrix_builder.py›

    No steps recorded

    View test on GitHub ↗
  912. ✓test_find_regressions_ignores_red_to_red_builder_unit_tests/test_matrix_builder.py›

    No steps recorded

    View test on GitHub ↗
  913. ✓test_find_regressions_matches_by_id_not_name_builder_unit_tests/test_matrix_builder.py›

    No steps recorded

    View test on GitHub ↗
  914. ✓test_find_regressions_reports_multiple_sorted_builder_unit_tests/test_matrix_builder.py›

    No steps recorded

    View test on GitHub ↗
  915. ✓test_passing_outcome_is_never_retryable_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  916. ✓test_passthrough_anthropicpassthrough/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  917. ✓test_passthrough_bedrock_conversepassthrough/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  918. ✓test_passthrough_bedrock_invokepassthrough/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  919. ✓test_passthrough_vertex_aipassthrough/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  920. ✓test_pdf_input_anthropicpdf_input/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  921. ✓test_pdf_input_azurepdf_input/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  922. ✓test_pdf_input_bedrock_conversepdf_input/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  923. ✓test_pdf_input_bedrock_invokepdf_input/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  924. ✓test_pdf_input_vertex_aipdf_input/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  925. ✓test_picks_the_newest_published_version_at_least_min_age_old_pr_gate_unit_tests/test_pr_gate_version_resolver.py›

    No steps recorded

    View test on GitHub ↗
  926. ✓test_prompt_caching_1h_anthropicprompt_caching_1h/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  927. ✓test_prompt_caching_1h_azureprompt_caching_1h/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  928. ✓test_prompt_caching_1h_bedrock_converseprompt_caching_1h/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  929. ✓test_prompt_caching_1h_bedrock_invokeprompt_caching_1h/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  930. ✓test_prompt_caching_1h_vertex_aiprompt_caching_1h/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  931. ✓test_prompt_caching_5m_anthropicprompt_caching_5m/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  932. ✓test_prompt_caching_5m_azureprompt_caching_5m/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  933. ✓test_prompt_caching_5m_bedrock_converseprompt_caching_5m/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  934. ✓test_prompt_caching_5m_bedrock_invokeprompt_caching_5m/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  935. ✓test_prompt_caching_5m_vertex_aiprompt_caching_5m/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  936. ✓test_raises_when_the_only_old_enough_version_is_unpublished_pr_gate_unit_tests/test_pr_gate_version_resolver.py›

    No steps recorded

    View test on GitHub ↗
  937. ✓test_reports_the_status_when_the_replayed_history_is_rejected_probe_unit_tests/test_http_probe.py›

    No steps recorded

    View test on GitHub ↗
  938. ✓test_shapes_are_classified_independently[[claude-opus-4-7-bedrock-converse] tool_search probe failed: status 503: {"error":{"message":"litellm.ServiceUnavailableError: BedrockException - {\\"message\\":\\"Bedrock is unable to process your request.\\"}"}}-False-True]_driver_unit_tests/test_retry_classification.py›
    1. Add a deployment named claude-haiku-4-5 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named claude-sonnet-4-5 that calls anthropic/claude-sonnet-4-5
    3. Add a deployment named claude-opus-4-7 that calls anthropic/claude-opus-4-7
    4. Add a deployment named claude-haiku-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    5. Add a deployment named claude-sonnet-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    6. Add a deployment named claude-opus-4-7-bedrock-invoke that calls bedrock/us.anthropic.claude-opus-4-7
    7. Add a deployment named claude-haiku-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    8. Add a deployment named claude-sonnet-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    9. Add a deployment named claude-opus-4-7-bedrock-converse that calls bedrock/converse/us.anthropic.claude-opus-4-7
    10. Add a deployment named claude-haiku-4-5-vertex that calls vertex_ai/claude-haiku-4-5
    11. Add a deployment named claude-sonnet-4-5-vertex that calls vertex_ai/claude-sonnet-4-5
    12. Add a deployment named claude-opus-4-7-vertex that calls vertex_ai/claude-opus-4-7
    13. Add a deployment named claude-haiku-4-5-azure that calls azure_ai/claude-haiku-4-5
    14. Add a deployment named claude-sonnet-4-5-azure that calls azure_ai/claude-sonnet-4-5
    15. Add a deployment named claude-opus-4-7-azure that calls azure_ai/claude-opus-4-7
    16. Add a deployment named gpt-5-6-sol-openai that calls openai/gpt-5.6-sol
    17. Add a deployment named gpt-5-6-terra-openai that calls openai/gpt-5.6-terra
    18. Add a deployment named gpt-5-6-luna-openai that calls openai/gpt-5.6-luna
    19. Add a deployment named gpt-5-6-sol-azure-openai that calls azure/gpt-5.6-sol
    20. Add a deployment named gpt-5-6-terra-azure-openai that calls azure/gpt-5.6-terra
    21. Add a deployment named gpt-5-6-luna-azure-openai that calls azure/gpt-5.6-luna
    22. Add a deployment named gpt-5-6-sol-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-sol
    23. Add a deployment named gpt-5-6-terra-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-terra
    24. Add a deployment named gpt-5-6-luna-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-luna
    View test on GitHub ↗
  939. ✓test_shapes_are_classified_independently[claude CLI timed out after 120s-True-False]_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  940. ✓test_shapes_are_classified_independently[status 400: {"error":"bad request"}-False-False]_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  941. ✓test_shapes_are_classified_independently[status 429: {"error":{"message":"Rate limit reached"}}-True-False]_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  942. ✓test_shapes_are_classified_independently[status 503 service unavailable-False-True]_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  943. ✓test_shapes_are_classified_independently[status 529: {"type":"overloaded_error"}-False-True]_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  944. ✓test_shapes_are_classified_independently[throttling exception from provider-True-False]_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  945. ✓test_shapes_are_classified_independently[upstream overloaded, try again later-False-True]_driver_unit_tests/test_retry_classification.py›

    No steps recorded

    View test on GitHub ↗
  946. ✓test_skips_a_version_npm_has_unpublished_pr_gate_unit_tests/test_pr_gate_version_resolver.py›

    No steps recorded

    View test on GitHub ↗
  947. ✓test_structured_outputs_anthropicstructured_outputs/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  948. ✓test_structured_outputs_azurestructured_outputs/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  949. ✓test_structured_outputs_bedrock_conversestructured_outputs/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  950. ✓test_structured_outputs_bedrock_invokestructured_outputs/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  951. ✓test_structured_outputs_vertex_aistructured_outputs/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  952. ✓test_thinking_anthropicthinking/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  953. ✓test_thinking_azurethinking/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  954. ✓test_thinking_bedrock_conversethinking/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  955. ✓test_thinking_bedrock_invokethinking/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  956. ✓test_thinking_vertex_aithinking/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  957. ✓test_thinking_with_tool_use_anthropicthinking_with_tool_use/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  958. ✓test_thinking_with_tool_use_azurethinking_with_tool_use/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  959. ✓test_thinking_with_tool_use_bedrock_conversethinking_with_tool_use/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  960. ✓test_thinking_with_tool_use_bedrock_invokethinking_with_tool_use/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  961. ✓test_thinking_with_tool_use_vertex_aithinking_with_tool_use/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  962. ✓test_tool_search_anthropictool_search/test_anthropic.py›
    1. Send a /v1/messages request to claude-haiku-4-5
    2. Send a /v1/messages request to claude-sonnet-4-5
    3. Send a /v1/messages request to claude-opus-4-7
    View test on GitHub ↗
  963. ✓test_tool_search_bedrock_conversetool_search/test_bedrock_converse.py›
    1. Send a /v1/messages request to claude-haiku-4-5-bedrock-converse
    2. Send a /v1/messages request to claude-sonnet-4-5-bedrock-converse
    3. Send a /v1/messages request to claude-opus-4-7-bedrock-converse
    View test on GitHub ↗
  964. ✓test_tool_search_bedrock_invoketool_search/test_bedrock_invoke.py›
    1. Add a deployment named claude-haiku-4-5 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named claude-sonnet-4-5 that calls anthropic/claude-sonnet-4-5
    3. Add a deployment named claude-opus-4-7 that calls anthropic/claude-opus-4-7
    4. Add a deployment named claude-haiku-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    5. Add a deployment named claude-sonnet-4-5-bedrock-invoke that calls bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    6. Add a deployment named claude-opus-4-7-bedrock-invoke that calls bedrock/us.anthropic.claude-opus-4-7
    7. Add a deployment named claude-haiku-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    8. Add a deployment named claude-sonnet-4-5-bedrock-converse that calls bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    9. Add a deployment named claude-opus-4-7-bedrock-converse that calls bedrock/converse/us.anthropic.claude-opus-4-7
    10. Add a deployment named claude-haiku-4-5-vertex that calls vertex_ai/claude-haiku-4-5
    11. Add a deployment named claude-sonnet-4-5-vertex that calls vertex_ai/claude-sonnet-4-5
    12. Add a deployment named claude-opus-4-7-vertex that calls vertex_ai/claude-opus-4-7
    13. Add a deployment named claude-haiku-4-5-azure that calls azure_ai/claude-haiku-4-5
    14. Add a deployment named claude-sonnet-4-5-azure that calls azure_ai/claude-sonnet-4-5
    15. Add a deployment named claude-opus-4-7-azure that calls azure_ai/claude-opus-4-7
    16. Add a deployment named gpt-5-6-sol-openai that calls openai/gpt-5.6-sol
    17. Add a deployment named gpt-5-6-terra-openai that calls openai/gpt-5.6-terra
    18. Add a deployment named gpt-5-6-luna-openai that calls openai/gpt-5.6-luna
    19. Add a deployment named gpt-5-6-sol-azure-openai that calls azure/gpt-5.6-sol
    20. Add a deployment named gpt-5-6-terra-azure-openai that calls azure/gpt-5.6-terra
    21. Add a deployment named gpt-5-6-luna-azure-openai that calls azure/gpt-5.6-luna
    22. Add a deployment named gpt-5-6-sol-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-sol
    23. Add a deployment named gpt-5-6-terra-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-terra
    24. Add a deployment named gpt-5-6-luna-bedrock-mantle that calls bedrock_mantle/openai.gpt-5.6-luna
    25. Send a /v1/messages request to claude-haiku-4-5-bedrock-invoke
    26. Send a /v1/messages request to claude-sonnet-4-5-bedrock-invoke
    27. Send a /v1/messages request to claude-opus-4-7-bedrock-invoke
    View test on GitHub ↗
  965. ✓test_tool_search_history_bedrock_invoketool_search/test_bedrock_invoke.py›
    1. Send a /v1/messages request to claude-haiku-4-5-bedrock-invoke
    2. Send a /v1/messages request to claude-sonnet-4-5-bedrock-invoke
    3. Send a /v1/messages request to claude-opus-4-7-bedrock-invoke
    View test on GitHub ↗
  966. ✓test_tool_use_anthropictool_use/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  967. ✓test_tool_use_azuretool_use/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  968. ✓test_tool_use_azure_openaitool_use/test_azure_openai.py›

    No steps recorded

    View test on GitHub ↗
  969. ✓test_tool_use_bedrock_conversetool_use/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  970. ✓test_tool_use_bedrock_invoketool_use/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  971. ✓test_tool_use_streaming_anthropictool_use_streaming/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  972. ✓test_tool_use_streaming_azuretool_use_streaming/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  973. ✓test_tool_use_streaming_azure_openaitool_use_streaming/test_azure_openai.py›

    No steps recorded

    View test on GitHub ↗
  974. ✓test_tool_use_streaming_bedrock_conversetool_use_streaming/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  975. ✓test_tool_use_streaming_bedrock_invoketool_use_streaming/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  976. ✓test_tool_use_streaming_vertex_aitool_use_streaming/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  977. ✓test_tool_use_streaming_vertex_ai_gpttool_use_streaming/test_vertex_ai_gpt.py›

    No steps recorded

    View test on GitHub ↗
  978. ✓test_tool_use_vertex_aitool_use/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  979. ✓test_tool_use_vertex_ai_gpttool_use/test_vertex_ai_gpt.py›

    No steps recorded

    View test on GitHub ↗
  980. ✓test_vision_anthropicvision/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  981. ✓test_vision_azurevision/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  982. ✓test_vision_bedrock_conversevision/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  983. ✓test_vision_bedrock_invokevision/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  984. ✓test_vision_vertex_aivision/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  985. ✓test_web_search_anthropicweb_search/test_anthropic.py›

    No steps recorded

    View test on GitHub ↗
  986. ✓test_web_search_azureweb_search/test_azure.py›

    No steps recorded

    View test on GitHub ↗
  987. ✓test_web_search_bedrock_converseweb_search/test_bedrock_converse.py›

    No steps recorded

    View test on GitHub ↗
  988. ✓test_web_search_bedrock_invokeweb_search/test_bedrock_invoke.py›

    No steps recorded

    View test on GitHub ↗
  989. ✓test_web_search_vertex_aiweb_search/test_vertex_ai.py›

    No steps recorded

    View test on GitHub ↗
  990. −test_basic_messaging_non_streaming_bedrock_mantlebasic_messaging_non_streaming/test_bedrock_mantle.py›

    No steps recorded

    Bedrock Mantle GPT-5.6 cells are opt-in; set COMPAT_MANTLE_CELLS=1 once the AWS account is allowlisted for the openai.gpt-5.6-* models

    View test on GitHub ↗
  991. −test_basic_messaging_non_streaming_openaibasic_messaging_non_streaming/test_openai.py›

    No steps recorded

    OpenAI GPT-5.6 cells are opt-in; set COMPAT_OPENAI_GPT_CELLS=1 to run them (stage suite timeouts under concurrent load)

    View test on GitHub ↗
  992. −test_basic_messaging_streaming_bedrock_mantlebasic_messaging_streaming/test_bedrock_mantle.py›

    No steps recorded

    Bedrock Mantle GPT-5.6 cells are opt-in; set COMPAT_MANTLE_CELLS=1 once the AWS account is allowlisted for the openai.gpt-5.6-* models

    View test on GitHub ↗
  993. −test_basic_messaging_streaming_openaibasic_messaging_streaming/test_openai.py›

    No steps recorded

    OpenAI GPT-5.6 cells are opt-in; set COMPAT_OPENAI_GPT_CELLS=1 to run them (stage suite timeouts under concurrent load)

    View test on GitHub ↗
  994. −test_count_tokens_vertex_aicount_tokens/test_vertex_ai.py›

    No steps recorded

    stage red: Vertex returns not supported for token counting for Claude aliases

    View test on GitHub ↗
  995. −test_long_context_1m_anthropiclong_context_1m/test_anthropic.py›

    No steps recorded

    stage red: 1M long_context not green on stage Anthropic path yet (200k sonnet / model alias)

    View test on GitHub ↗
  996. −test_long_context_1m_azurelong_context_1m/test_azure.py›

    No steps recorded

    stage red: 1M long_context not green on stage Azure Foundry deployments yet

    View test on GitHub ↗
  997. −test_long_context_1m_bedrock_converselong_context_1m/test_bedrock_converse.py›

    No steps recorded

    stage red: 1M long_context not green on stage Bedrock Converse deployments yet

    View test on GitHub ↗
  998. −test_long_context_1m_bedrock_invokelong_context_1m/test_bedrock_invoke.py›

    No steps recorded

    stage red: 1M long_context not green on stage Bedrock Invoke deployments yet

    View test on GitHub ↗
  999. −test_long_context_1m_vertex_ailong_context_1m/test_vertex_ai.py›

    No steps recorded

    stage red: 1M long_context not green on stage Vertex deployments yet

    View test on GitHub ↗
  1000. −test_passthrough_azurepassthrough/test_azure.py›

    No steps recorded

    stage red: /azure passthrough drops client headers (e.g. anthropic-version); product gap

    View test on GitHub ↗
  1001. −test_tool_search_azuretool_search/test_azure.py›

    No steps recorded

    stage red: Azure Foundry tool_search_server not supported in workspace for probed models

    View test on GitHub ↗
  1002. −test_tool_search_vertex_aitool_search/test_vertex_ai.py›

    No steps recorded

    stage red: Vertex rejects tool_search when deployment extra_headers inject context-1m beta; product/config

    View test on GitHub ↗
  1003. −test_tool_use_bedrock_mantletool_use/test_bedrock_mantle.py›

    No steps recorded

    Bedrock Mantle GPT-5.6 cells are opt-in; set COMPAT_MANTLE_CELLS=1 once the AWS account is allowlisted for the openai.gpt-5.6-* models

    View test on GitHub ↗
  1004. −test_tool_use_openaitool_use/test_openai.py›

    No steps recorded

    OpenAI GPT-5.6 cells are opt-in; set COMPAT_OPENAI_GPT_CELLS=1 to run them (stage suite timeouts under concurrent load)

    View test on GitHub ↗
  1005. −test_tool_use_streaming_bedrock_mantletool_use_streaming/test_bedrock_mantle.py›

    No steps recorded

    Bedrock Mantle GPT-5.6 cells are opt-in; set COMPAT_MANTLE_CELLS=1 once the AWS account is allowlisted for the openai.gpt-5.6-* models

    View test on GitHub ↗
  1006. −test_tool_use_streaming_openaitool_use_streaming/test_openai.py›

    No steps recorded

    OpenAI GPT-5.6 cells are opt-in; set COMPAT_OPENAI_GPT_CELLS=1 to run them (stage suite timeouts under concurrent load)

    View test on GitHub ↗
  1007. ✓test_a_success_status_without_a_deletion_confirmation_is_rejected[bGl0ZWxsbV9wcm94eTtmaWxlLTE=] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1008. ✓test_a_success_status_without_a_deletion_confirmation_is_rejected[file-1] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1009. ✓test_accepted_cancellation_waits_through_stale_provider_status[in_progress-batch-1] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1010. ✓test_accepted_cancellation_waits_through_stale_provider_status[in_progress-bGl0ZWxsbV9wcm94eTtiYXRjaC0x] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1011. ✓test_accepted_cancellation_waits_through_stale_provider_status[validating-batch-1] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1012. ✓test_accepted_cancellation_waits_through_stale_provider_status[validating-bGl0ZWxsbV9wcm94eTtiYXRjaC0x] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1013. ✓test_active_batch_is_cancelled_through_its_provider TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1014. ✓test_any_other_delete_failure_still_raises[failure0] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1015. ✓test_any_other_delete_failure_still_raises[failure1] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1016. ✓test_any_other_delete_failure_still_raises[failure2] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1017. ✓test_anything_but_still_cancelling_at_the_deadline_still_fails[last0-did not finish within 120.0s, last status in_progress] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1018. ✓test_anything_but_still_cancelling_at_the_deadline_still_fails[last1-after cancellation failed: HTTP 403] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1019. ✓test_azure_form_serializes_native_expiry_for_the_proxy TestAzureFileExpirytest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1020. ✓test_batch_create_over_rpm_returns_mapped_429 TestBatchRateLimitErrorMappingtest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-batch-rl-map-de9fcda351ed, tpm limit: 1000000 and rpm limit: 2
    View test on GitHub ↗
  1021. ✓test_batch_key_model_access_deniedtest_batches_e2e.py›
    1. Generate a virtual key with models: openai-batch-ead71e024fa4 and user id: e2e-test-user
    View test on GitHub ↗
  1022. ✓test_batch_lifecycle[azure-encoded]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1023. ✓test_batch_lifecycle[azure-model_param]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1024. ✓test_batch_lifecycle[azure-provider_fallback]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1025. ✓test_batch_lifecycle[azure-unified]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1026. ✓test_batch_lifecycle[bedrock-unified]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1027. ✓test_batch_lifecycle[openai-encoded]test_batches_e2e.py›
    1. Call the management route /health/liveliness
    2. Add a deployment named openai-batch-ead71e024fa4 that calls openai/gpt-4o-mini
    3. Add a deployment named azure-batch-ead71e024fa4 that calls azure/gpt-5.4-mini-batch
    4. Add a deployment named vertex-batch-ead71e024fa4 that calls vertex_ai/gemini-2.5-flash
    5. Add a deployment named bedrock-batch-ead71e024fa4 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    6. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1028. ✓test_batch_lifecycle[openai-model_param]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1029. ✓test_batch_lifecycle[openai-provider_fallback]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1030. ✓test_batch_lifecycle[openai-unified]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1031. ✓test_batch_lifecycle[vertex_ai-encoded]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1032. ✓test_batch_lifecycle[vertex_ai-model_param]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1033. ✓test_batch_lifecycle[vertex_ai-provider_fallback]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1034. ✓test_batch_lifecycle[vertex_ai-unified]test_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1035. ✓test_batch_still_cancelling_at_the_deadline_and_its_input_file_are_left_and_reported TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1036. ✓test_batch_that_completed_before_cleanup_deletes_output_and_error_files[False] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1037. ✓test_batch_that_completed_before_cleanup_deletes_output_and_error_files[True] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1038. ✓test_cancellation_conflict_is_accepted_only_when_batch_became_inactive[completed] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1039. ✓test_cancellation_conflict_is_accepted_only_when_batch_became_inactive[in_progress] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1040. ✓test_cancelling_batch_is_polled_until_terminal_without_cancelling_again TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1041. ✓test_cleanup_is_idempotent_when_file_is_already_deleted TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1042. ✓test_completed_batch_downloads_output_and_books_cost TestBatchTerminalStatetest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1043. ✓test_default_resource_cleanup_keeps_existing_best_effort_behavior TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1044. ✓test_delete_refused_because_a_batch_still_references_the_file_is_left_and_reported TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1045. ✓test_deletes_raw_files_through_the_upload_provider[azure-encoded] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1046. ✓test_deletes_raw_files_through_the_upload_provider[azure-model_param] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1047. ✓test_deletes_raw_files_through_the_upload_provider[azure-provider_fallback] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1048. ✓test_deletes_raw_files_through_the_upload_provider[azure-unified] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1049. ✓test_deletes_raw_files_through_the_upload_provider[bedrock-unified] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1050. ✓test_deletes_raw_files_through_the_upload_provider[openai-encoded] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1051. ✓test_deletes_raw_files_through_the_upload_provider[openai-model_param] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1052. ✓test_deletes_raw_files_through_the_upload_provider[openai-provider_fallback] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1053. ✓test_deletes_raw_files_through_the_upload_provider[openai-unified] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1054. ✓test_deletes_raw_files_through_the_upload_provider[vertex_ai-encoded] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1055. ✓test_deletes_raw_files_through_the_upload_provider[vertex_ai-model_param] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1056. ✓test_deletes_raw_files_through_the_upload_provider[vertex_ai-provider_fallback] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1057. ✓test_deletes_raw_files_through_the_upload_provider[vertex_ai-unified] TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1058. ✓test_endpoint_mismatch_fails_batch_and_cancel_conflicts TestBatchFailurePathstest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  1059. ✓test_enqueued_allowance_accepts_batch_over_key_rpm TestBatchEnqueuedTokenLimittest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-batch-enq-rpm-cf846cd9ac69, rpm limit: 2 and metadata: batch enqueued token limit: 100000
    View test on GitHub ↗
  1060. ✓test_exhausted_allowance_blocks_until_cancel_refunds TestBatchEnqueuedTokenLimittest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-batch-enq-size-5aac6d0a067d and metadata: batch enqueued token limit: 1
    2. Generate a virtual key with user id: e2e-batch-enq-refund-2346877a9870 and metadata: batch enqueued token limit: 36
    View test on GitHub ↗
  1061. ✓test_failed_delete_is_reported_after_remaining_resources_are_cleaned TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1062. ✓test_file_content_matches_upload TestBatchFileContenttest_batches_e2e.py›
    1. Add a deployment named e2e-file-content-2405ca8aa0d4 that calls openai/gpt-4o-mini
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1063. ✓test_file_lifecycle_signs_s3_with_s3_credentials TestBedrockBatchSplitS3Credentialstest_batches_e2e.py›
    1. Add a deployment named bedrock-split-s3-batch-ead71e024fa4 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1064. ✓test_file_upload_and_delete_outputstest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1065. ✓test_foreign_encoded_file_id_routes_by_file_model TestBatchFailurePathstest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1066. ✓test_gemini_file_upload TestGeminiFilestest_batches_e2e.py›
    1. Add a deployment named gemini-files-ead71e024fa4 that calls gemini/gemini-2.5-flash
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1067. ✓test_inactive_batch_needs_no_cancellation[cancelled] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1068. ✓test_inactive_batch_needs_no_cancellation[completed] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1069. ✓test_inactive_batch_needs_no_cancellation[expired] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1070. ✓test_inactive_batch_needs_no_cancellation[failed] TestBatchCancellationtest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1071. ✓test_list_page_cursors_address_only_the_callers_own_files TestOpenAIFilestest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-file-list-d135215942bd
    View test on GitHub ↗
  1072. ✓test_malformed_jsonl_upload_rejected TestBatchFailurePathstest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1073. ✓test_managed_delete_accepts_the_deleted_file_object TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1074. ✓test_native_jsonl_round_trips_untouched_and_starts_a_batch TestVertexNativePassthroughtest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1075. ✓test_other_providers_keep_their_existing_upload_fields[bedrock] TestAzureFileExpirytest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1076. ✓test_other_providers_keep_their_existing_upload_fields[openai] TestAzureFileExpirytest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1077. ✓test_other_providers_keep_their_existing_upload_fields[vertex_ai] TestAzureFileExpirytest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1078. ✓test_passthrough_upload_is_rejected_outside_a_native_vertex_batch[no-target-model] TestVertexNativePassthroughtest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1079. ✓test_passthrough_upload_is_rejected_outside_a_native_vertex_batch[non-vertex-target-model] TestVertexNativePassthroughtest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1080. ✓test_passthrough_upload_is_rejected_outside_a_native_vertex_batch[openai-shaped-rows] TestVertexNativePassthroughtest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1081. ✓test_permanent_error_is_not_retried TestCleanupRetriestest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1082. ✓test_persistent_error_has_bounded_retries TestCleanupRetriestest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1083. ✓test_retrieve_round_trips_metadata TestOpenAIFilestest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1084. ✓test_success_response_must_confirm_deletion TestFileCleanuptest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1085. ✓test_transient_error_retries_and_returns_success[failure0] TestCleanupRetriestest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1086. ✓test_transient_error_retries_and_returns_success[failure1] TestCleanupRetriestest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1087. ✓test_transient_error_retries_and_returns_success[failure2] TestCleanupRetriestest_batch_cleanup.py›

    No steps recorded

    View test on GitHub ↗
  1088. ✓test_unified_batch_create_with_assume_role TestBedrockBatchAssumeRoletest_batches_e2e.py›
    1. Add a deployment named bedrock-sts-batch-ead71e024fa4 that calls bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1089. ✓test_unified_create_and_retrieve_via_chained_gateway TestBatchSecondHoptest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named openai-batch-hop-ead71e024fa4 that calls litellm_proxy/openai-batch-ead71e024fa4
    View test on GitHub ↗
  1090. ✓test_unified_file_content_downloads[azure] TestBatchFileContenttest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1091. ✓test_unified_file_content_downloads[bedrock] TestBatchFileContenttest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1092. ✓test_unified_file_content_downloads[vertex_ai] TestBatchFileContenttest_batches_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1093. ✓test_unified_file_upload_and_batch_create_in_govcloud TestBedrockBatchGovCloudtest_batches_e2e.py›
    1. Add a deployment named bedrock-govcloud-batch-ead71e024fa4 that calls bedrock/amazon.nova-lite-v1:0
    2. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1094. −test_batch_runs_to_completion_with_a_downloadable_output[model_query] TestHostedVllmBatchtest_batches_e2e.py›

    No steps recorded

    set HOSTED_VLLM_API_BASE (the live vLLM server this deployment targets)

    View test on GitHub ↗
  1095. −test_batch_runs_to_completion_with_a_downloadable_output[target_model_names] TestHostedVllmBatchtest_batches_e2e.py›

    No steps recorded

    set HOSTED_VLLM_API_BASE (the live vLLM server this deployment targets)

    View test on GitHub ↗
  1096. −test_failing_line_lands_in_the_error_file_not_the_batch_status TestHostedVllmBatchtest_batches_e2e.py›

    No steps recorded

    set HOSTED_VLLM_API_BASE (the live vLLM server this deployment targets)

    View test on GitHub ↗
  1097. −test_rate_limited_batch_create_leaves_no_unattributed_spend_rowtest_batches_e2e.py›

    No steps recorded

    LIT-5027: the path under test hangs. The batch rate limiter reads the input file to count tokens by awaiting litellm.afile_content with no timeout, so a slow Files API holds POST /v1/batches open past any client deadline (63.6s observed on stage against a 60s read timeout). The unattributed-spend-row contract below is never reached, so the test reports a timeout rather than the behavior it guards. Unskip once the fetch is bounded.

    View test on GitHub ↗
  1098. −test_uploaded_file_appears_in_list TestOpenAIFilestest_batches_e2e.py›

    No steps recorded

    LIT-4820 (https://linear.app/litellm-ai/issue/LIT-4820): GET /v1/files omits newly uploaded files. The upload succeeds and GET /v1/files/{id} returns the file, but it never appears in the listing; the returned set is stable with its newest entry ~10h old, on both the managed (/v1/files?model=) and provider-scoped (/openai/v1/files) routes. Skipped rather than weakened because the assertion below is the correct contract. Remove this marker when LIT-4820 is fixed; do not relax the assertion to make it pass.

    View test on GitHub ↗
  1099. ✓test_a_body_that_does_not_match_its_declared_boundary_stays_opaque TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1100. ✓test_a_chunk_split_inside_an_event_survives_replay TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1101. ✓test_a_chunked_non_sse_response_stays_buffered TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1102. ✓test_a_declared_part_content_type_separates_otherwise_identical_uploads TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1103. ✓test_a_downstream_disconnect_mid_relay_records_only_the_delivered_chunks TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1104. ✓test_a_file_part_without_a_filename_is_not_mistaken_for_a_plain_field TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1105. ✓test_a_filename_carrying_a_per_run_marker_keys_the_same_next_run TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1106. ✓test_a_fresh_boundary_replays_the_same_upload TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1107. ✓test_a_length_change_the_canonicalizer_absorbs_does_not_move_the_key TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1108. ✓test_a_mid_frame_cut_tears_a_data_line_whose_marker_is_split_across_chunks TestStreamCuttest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1109. ✓test_a_mid_stream_upstream_failure_records_the_delivered_chunks_and_the_truncation TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1110. ✓test_a_non_streamed_response_keeps_the_buffered_shape TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1111. ✓test_a_redacted_field_still_matches_the_live_request_that_carried_the_secret TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1112. ✓test_a_repeated_field_cannot_collide_with_a_literal_indexed_name TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1113. ✓test_a_secret_named_field_never_reaches_the_stored_request TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1114. ✓test_a_separator_inside_a_filename_cannot_forge_a_different_split TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1115. ✓test_a_streamed_response_records_its_chunk_boundaries TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1116. ✓test_a_structurally_different_upload_misses[extra-file-part] TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1117. ✓test_a_structurally_different_upload_misses[file-content] TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1118. ✓test_a_structurally_different_upload_misses[file-name] TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1119. ✓test_a_structurally_different_upload_misses[form-field] TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1120. ✓test_a_structurally_different_upload_misses[repeated-form-field] TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1121. ✓test_a_truncated_recording_replays_as_a_truncated_stream TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1122. ✓test_an_empty_declared_boundary_falls_back_instead_of_splitting_on_dashes TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1123. ✓test_bedrock_never_wires_a_bundle_because_the_edge_cannot_sign_into_one[record] TestApiBaseSeamtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1124. ✓test_bedrock_never_wires_a_bundle_because_the_edge_cannot_sign_into_one[replay] TestApiBaseSeamtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1125. ✓test_concurrent_streams_each_record_their_own_chunks TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1126. ✓test_content_drift_returns_the_miss_status_naming_both_keys TestReplayModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1127. ✓test_exhausted_key_returns_the_miss_status TestReplayModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1128. ✓test_failed_provider_attempt_is_counted TestProviderRequestObservationtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1129. ✓test_forwards_to_the_provider_and_writes_one_interaction_file TestRecordModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1130. ✓test_fully_consumed_recording_leaves_nothing TestReplayLeftovertest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1131. ✓test_identical_requests_replay_their_responses_in_recorded_order TestReplayModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1132. ✓test_inert_outside_replay_mode TestReplayLeftovertest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1133. ✓test_invalid_mode_raises_naming_the_value TestApiBaseSeamtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1134. ✓test_live_counts_repeated_marker_calls_without_recording TestProviderRequestObservationtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1135. ✓test_live_mode_returns_none TestApiBaseSeamtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1136. ✓test_never_stores_headers_so_credentials_never_touch_disk TestRecordModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1137. ✓test_non_json_bodies_match_by_canonical_digest_without_storing_them TestReplayModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1138. ✓test_parallel_identical_calls_serve_each_recording_exactly_once TestConcurrentReplaytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1139. ✓test_partially_consumed_recording_names_the_leftover TestReplayLeftovertest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1140. ✓test_query_params_are_part_of_the_identity TestReplayModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1141. ✓test_record_and_replay_count_each_matching_call TestProviderRequestObservationtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1142. ✓test_record_mode_boots_one_shared_edge_and_prepares_the_bundle TestApiBaseSeamtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1143. ✓test_record_mode_relays_the_stream_chunked_like_replay_will TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1144. ✓test_replay_of_a_stream_makes_no_provider_connection TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1145. ✓test_replay_reproduces_the_recorded_split_points TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1146. ✓test_replay_serves_a_directly_recorded_interaction TestHandleEdgeRequestPuretest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1147. ✓test_request_identity_ignores_auth_headers TestReplayModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1148. ✓test_serves_recorded_bytes_with_zero_provider_hits TestReplayModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1149. ✓test_several_file_parts_separate_when_their_contents_swap TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1150. ✓test_strips_volatile_response_headers_and_serves_the_filtered_copy TestRecordModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1151. ✓test_test_without_recordings_has_no_leftover TestReplayLeftovertest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1152. ✓test_the_boundary_parameter_is_read_the_way_the_client_meant_it[lookalike-parameter] TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1153. ✓test_the_boundary_parameter_is_read_the_way_the_client_meant_it[uppercase-parameter] TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1154. ✓test_the_stored_request_carries_fields_and_file_identity_but_no_secrets TestMultipartIdentitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1155. ✓test_the_usage_chunk_replays_in_its_recorded_position TestStreamingFidelitytest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1156. ✓test_two_binary_field_values_of_one_length_stay_apart TestMultipartIdentityEdgestest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1157. ✓test_unknown_mount_404s_naming_the_known_mounts TestHandleEdgeRequestPuretest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1158. ✓test_unknown_mount_raises_naming_the_known_mounts TestApiBaseSeamtest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1159. ✓test_unreachable_provider_records_and_serves_a_502 TestRecordModetest_provider_edge.py›

    No steps recorded

    View test on GitHub ↗
  1160. ✓test_429_honors_server_reset_and_preserves_duplicate_eventstest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1161. ✓test_all_attempts_failed_is_a_failure_not_a_passtest_span_selection.py›

    No steps recorded

    View test on GitHub ↗
  1162. ✓test_chat_completions_emits_one_log_event TestDataDogLogDeliverytest_datadog_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-dd-chat-0f533c1b8bf3 and key alias: dd-chat-0f533c1b8bf3
    View test on GitHub ↗
  1163. ✓test_chat_completions_exports_complete_trace TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-otel-trace-chat-3e18de3fbe35 and key alias: otel-trace-chat-3e18de3fbe35
    View test on GitHub ↗
  1164. ✓test_chat_completions_failure_writes_one_object TestS3LogDeliverytest_s3_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Add a deployment named s3-err-da8bafc697c5 that calls anthropic/claude-haiku-4-5
    3. Generate a virtual key with models: s3-err-da8bafc697c5, user id: e2e-s3-err-key-d8c39c09f87e and key alias: s3-err-key-d8c39c09f87e
    View test on GitHub ↗
  1165. ✓test_chat_completions_object_key_follows_the_partition_granularity TestS3LogDeliverytest_s3_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-s3-layout-98481de6da1b and key alias: s3-layout-98481de6da1b
    View test on GitHub ↗
  1166. ✓test_chat_completions_stream_emits_one_log_event TestDataDogLogDeliverytest_datadog_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-dd-stream-chat-811d1eab7baa and key alias: dd-stream-chat-811d1eab7baa
    3. Send a streaming /chat/completions request to claude-haiku-4-5
    4. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1167. ✓test_chat_completions_stream_exports_complete_trace TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-otel-stream-chat-078ef8742ec5 and key alias: otel-stream-chat-078ef8742ec5
    3. Send a streaming /chat/completions request to claude-haiku-4-5
    View test on GitHub ↗
  1168. ✓test_chat_completions_stream_records_real_ttft TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-otel-ttft-chat-336126df19c1 and key alias: otel-ttft-chat-336126df19c1
    3. Send a streaming /chat/completions request to claude-haiku-4-5
    View test on GitHub ↗
  1169. ✓test_chat_completions_writes_one_success_object TestS3LogDeliverytest_s3_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-s3-chat-8b6a75f5fa4d and key alias: s3-chat-8b6a75f5fa4d
    View test on GitHub ↗
  1170. ✓test_chat_completions_writes_one_success_record TestGcsLogDeliverytest_gcs_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-gcs-chat-ce6c5178ad15 and key alias: gcs-chat-ce6c5178ad15
    View test on GitHub ↗
  1171. ✓test_distinct_key_aliases_produce_distinct_series TestPrometheusPerKeyCardinalitytest_prometheus_cardinality_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash, user id: e2e-e2e-prom-1bf2894ff51f and key alias: e2e-prom-1bf2894ff51f
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Generate a virtual key with models: gemini-2.5-flash, user id: e2e-e2e-prom-c456acf7930a and key alias: e2e-prom-c456acf7930a
    4. Send a /chat/completions request to gemini-2.5-flash
    5. Generate a virtual key with models: gemini-2.5-flash, user id: e2e-e2e-prom-84c370ee1c8f and key alias: e2e-prom-84c370ee1c8f
    6. Send a /chat/completions request to gemini-2.5-flash
    7. Call the management route /metrics
    View test on GitHub ↗
  1172. ✓test_empty_polling_does_not_start_a_final_search_after_its_deadlinetest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1173. ✓test_failed_chat_completions_emits_one_error_event TestDataDogFailureDeliverytest_datadog_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Add a deployment named dd-err-52bc8dbb5556 that calls anthropic/claude-haiku-4-5
    3. Generate a virtual key with models: dd-err-52bc8dbb5556, user id: e2e-dd-err-key-48ea106ece94 and key alias: dd-err-key-48ea106ece94
    View test on GitHub ↗
  1174. ✓test_failed_chat_completions_error_span_attributes TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Add a deployment named otel-err-313d693a6c66 that calls anthropic/claude-haiku-4-5
    3. Generate a virtual key with models: otel-err-313d693a6c66, user id: e2e-otel-err-da5600775621 and key alias: otel-err-da5600775621
    View test on GitHub ↗
  1175. ✓test_failed_messages_error_span_attributes TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Add a deployment named otel-err-86ef8d5df228 that calls anthropic/claude-haiku-4-5
    3. Generate a virtual key with models: otel-err-86ef8d5df228, user id: e2e-otel-err-f825f7ded6a8 and key alias: otel-err-f825f7ded6a8
    View test on GitHub ↗
  1176. ✓test_failure_diagnostics_hide_credentials_without_changing_auth_headerstest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1177. ✓test_invalid_reset_uses_search_interval[-1]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1178. ✓test_invalid_reset_uses_search_interval[]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1179. ✓test_invalid_reset_uses_search_interval[inf]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1180. ✓test_invalid_reset_uses_search_interval[invalid]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1181. ✓test_invalid_reset_uses_search_interval[nan]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1182. ✓test_late_delivery_cannot_pass_without_a_complete_settle_windowtest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1183. ✓test_late_retry_cannot_receive_a_fresh_request_timeouttest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1184. ✓test_messages_emits_one_log_event TestDataDogLogDeliverytest_datadog_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-dd-messages-fa2f748339af and key alias: dd-messages-fa2f748339af
    View test on GitHub ↗
  1185. ✓test_messages_exports_complete_trace TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-otel-trace-messages-07486797e4c5 and key alias: otel-trace-messages-07486797e4c5
    View test on GitHub ↗
  1186. ✓test_messages_stream_emits_one_log_event TestDataDogLogDeliverytest_datadog_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-dd-stream-messages-0b17006b37f5 and key alias: dd-stream-messages-0b17006b37f5
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1187. ✓test_messages_stream_exports_complete_trace TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-otel-stream-messages-91817fc9d891 and key alias: otel-stream-messages-91817fc9d891
    View test on GitHub ↗
  1188. ✓test_messages_stream_records_real_ttft TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-otel-ttft-messages-a2e62f685611 and key alias: otel-ttft-messages-a2e62f685611
    View test on GitHub ↗
  1189. ✓test_non_json_native_metadata_reaches_langsmith TestLangsmithBatchSerializationtest_langsmith_batch_serialization_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1190. ✓test_non_quota_failures_are_not_retried_or_treated_as_empty_results[-1]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1191. ✓test_non_quota_failures_are_not_retried_or_treated_as_empty_results[401]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1192. ✓test_non_quota_failures_are_not_retried_or_treated_as_empty_results[403]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1193. ✓test_non_quota_failures_are_not_retried_or_treated_as_empty_results[500]test_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1194. ✓test_other_operations_are_not_countedtest_span_selection.py›

    No steps recorded

    View test on GitHub ↗
  1195. ✓test_polling_quota_retries_share_the_original_deadlinetest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1196. ✓test_queue_time_histogram_records_an_observation TestPrometheusRequestQueueTimetest_prometheus_queue_time_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash, user id: e2e-e2e-queue-time-1af2abe4f1d0 and key alias: e2e-queue-time-1af2abe4f1d0
    2. Send a /chat/completions request to gemini-2.5-flash
    3. Call the management route /metrics
    View test on GitHub ↗
  1197. ✓test_rate_limit_wait_stops_at_deadline_without_issuing_another_requesttest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1198. ✓test_responses_emits_one_log_event TestDataDogLogDeliverytest_datadog_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: gpt-5.5, user id: e2e-dd-responses-1abf8659f665 and key alias: dd-responses-1abf8659f665
    View test on GitHub ↗
  1199. ✓test_responses_exports_complete_trace TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: gpt-5.5, user id: e2e-otel-trace-responses-7bfa65793153 and key alias: otel-trace-responses-7bfa65793153
    View test on GitHub ↗
  1200. ✓test_responses_stream_emits_one_log_event TestDataDogLogDeliverytest_datadog_log_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: gpt-5.5, user id: e2e-dd-stream-responses-f65abb9dfb1f and key alias: dd-stream-responses-f65abb9dfb1f
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1201. ✓test_responses_stream_exports_complete_trace TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: gpt-5.5, user id: e2e-otel-stream-responses-cc0482fc57c1 and key alias: otel-stream-responses-cc0482fc57c1
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1202. ✓test_responses_stream_records_real_ttft TestOtelTraceCompletenesstest_otel_trace_e2e.py›
    1. Call the management route /health/readiness/details
    2. Generate a virtual key with models: gpt-5.5, user id: e2e-otel-ttft-responses-2fb02fb108f6 and key alias: otel-ttft-responses-2fb02fb108f6
    View test on GitHub ↗
  1203. ✓test_retried_attempt_span_is_excludedtest_span_selection.py›

    No steps recorded

    View test on GitHub ↗
  1204. ✓test_retry_after_is_not_shortened_by_an_earlier_resettest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1205. ✓test_served_span_is_the_only_one_when_nothing_was_retriedtest_span_selection.py›

    No steps recorded

    View test on GitHub ↗
  1206. ✓test_settlement_detects_a_duplicate_on_the_final_searchtest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1207. ✓test_settlement_keeps_confirmed_events_through_empty_searchestest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1208. ✓test_settlement_quota_retries_keep_the_remaining_readback_budgettest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1209. ✓test_several_failed_attempts_still_leave_one_served_spantest_span_selection.py›

    No steps recorded

    View test on GitHub ↗
  1210. ✓test_team_callback_delivers_and_isolates TestTeamLangfuseCallbacktest_team_langfuse_callback_e2e.py›
    1. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-lf-team-key-fa250ea8a232, team id: 16f3650a-0bbf-4dd0-a248-b8e69b1682a9 and key alias: lf-team-key-fa250ea8a232
    2. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-lf-solo-key-211ed7f16daa and key alias: lf-solo-key-211ed7f16daa
    View test on GitHub ↗
  1211. ✓test_two_served_spans_still_failtest_span_selection.py›

    No steps recorded

    View test on GitHub ↗
  1212. ✓test_zero_reset_cannot_create_a_busy_retry_looptest_datadog_reader.py›

    No steps recorded

    View test on GitHub ↗
  1213. −test_chat_completions_delivers_one_call_with_spend TestWeaveLogDeliverytest_weave_log_e2e.py›

    No steps recorded

    stage red: product gap, key-scoped weave_otel spans are not delivered when the OTEL v2 callback is active

    View test on GitHub ↗
  1214. −test_failed_chat_completions_delivers_one_error_call TestWeaveLogDeliverytest_weave_log_e2e.py›

    No steps recorded

    stage red: product gap, key-scoped weave_otel spans are not delivered when the OTEL v2 callback is active

    View test on GitHub ↗
  1215. ✓test_a_flat_ceiling_fails_a_degraded_phase_that_is_cheaper_than_its_baseline TestAbsoluteBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1216. ✓test_a_missing_baseline_is_a_violation_rather_than_a_silent_pass TestRatioBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1217. ✓test_a_route_that_never_ran_is_absent_so_a_one_sided_run_cannot_pass_unnoticed TestPerEndpointtest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1218. ✓test_a_run_inside_every_budget_reports_nothing TestViolationstest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1219. ✓test_a_run_that_drove_no_traffic_reports_a_total_failure_ratio TestAggregatetest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1220. ✓test_a_run_with_no_failures_writes_no_csv_and_reports_no_errors TestErrorBreakdowntest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1221. ✓test_a_single_sample_has_no_interval_and_reports_zero TestCpuUtilizationtest_proxy_usage.py›

    No steps recorded

    View test on GitHub ↗
  1222. ✓test_a_value_exactly_at_the_ceiling_is_allowed TestAbsoluteBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1223. ✓test_a_value_past_the_ceiling_reports_the_measurement_and_the_ceiling TestAbsoluteBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1224. ✓test_a_value_under_the_ceiling_is_not_a_violation TestAbsoluteBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1225. ✓test_a_zero_measurement_is_not_a_violation TestAbsoluteBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1226. ✓test_all_planned_turns_completing_reports_zero_failures TestSummarizePlannedTurnstest_session_anomaly.py›

    No steps recorded

    View test on GitHub ↗
  1227. ✓test_an_empty_window_reports_zero_rather_than_indexing_nothing TestRssPercentilestest_proxy_usage.py›

    No steps recorded

    View test on GitHub ↗
  1228. ✓test_an_even_split_takes_the_lower_middle_sample_as_locust_itself_does TestPercentilestest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1229. ✓test_both_budget_shapes_report_together TestViolationstest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1230. ✓test_cost_per_request_separates_runs_that_cores_busy_reports_identically TestCpuUtilizationtest_proxy_usage.py›

    No steps recorded

    View test on GitHub ↗
  1231. ✓test_diagnosis_caps_the_list_and_says_how_many_it_left_out TestErrorBreakdowntest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1232. ✓test_diagnosis_leads_with_the_most_common_error TestErrorBreakdowntest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1233. ✓test_diagnosis_says_so_when_locust_recorded_nothing TestErrorBreakdowntest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1234. ✓test_each_route_keeps_its_own_requests_failures_and_median TestPerEndpointtest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1235. ✓test_every_blown_budget_is_reported_not_just_the_first TestViolationstest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1236. ✓test_exhausted_attempts_return_the_last_failure TestRetriedtest_session_anomaly.py›

    No steps recorded

    View test on GitHub ↗
  1237. ✓test_first_try_success_never_sleeps TestRetriedtest_session_anomaly.py›

    No steps recorded

    View test on GitHub ↗
  1238. ✓test_growth_exactly_at_the_ceiling_is_allowed TestRatioBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1239. ✓test_growth_past_the_ceiling_reports_both_values_and_the_ratio TestRatioBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1240. ✓test_growth_within_the_ceiling_is_not_a_violation TestRatioBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1241. ✓test_locust_failure_rows_become_the_error_breakdown TestErrorBreakdowntest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1242. ✓test_median_is_the_middle_sample_not_the_mean_a_slow_tail_would_drag TestPercentilestest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1243. ✓test_no_requests_reports_zero_cost_rather_than_dividing_by_zero TestCpuUtilizationtest_proxy_usage.py›

    No steps recorded

    View test on GitHub ↗
  1244. ✓test_no_samples_reports_zero_rather_than_dividing_by_an_empty_histogram TestPercentilestest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1245. ✓test_ordinary_locust_chatter_is_not_reported_as_a_warning TestGeneratorSaturationtest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1246. ✓test_partial_total_between_batch_flushes_is_not_accepted_as_final TestSettledSpendtest_session_anomaly.py›

    No steps recorded

    View test on GitHub ↗
  1247. ✓test_percentiles_merge_the_histograms_of_every_stats_entry TestPercentilestest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1248. ✓test_repeated_cpu_warnings_collapse_to_one_and_reach_the_diagnosis TestGeneratorSaturationtest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1249. ✓test_session_aborted_on_first_turn_counts_all_its_planned_turns_as_failed TestSummarizePlannedTurnstest_session_anomaly.py›

    No steps recorded

    View test on GitHub ↗
  1250. ✓test_several_stats_entries_for_one_route_fold_into_a_single_row TestPerEndpointtest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1251. ✓test_shrinking_is_never_a_violation TestRatioBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1252. ✓test_spend_that_never_becomes_nonzero_raises TestSettledSpendtest_session_anomaly.py›

    No steps recorded

    View test on GitHub ↗
  1253. ✓test_spend_that_never_stabilizes_raises TestSettledSpendtest_session_anomaly.py›

    No steps recorded

    View test on GitHub ↗
  1254. ✓test_summary_reports_every_percentile_in_human_units TestCpuUtilizationtest_proxy_usage.py›

    No steps recorded

    View test on GitHub ↗
  1255. ✓test_tail_percentiles_come_from_the_slow_end_of_the_histogram TestAggregatetest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1256. ✓test_the_summary_names_every_route_with_its_counts TestPerEndpointtest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1257. ✓test_the_tail_percentiles_reach_the_peak_the_median_hides TestRssPercentilestest_proxy_usage.py›

    No steps recorded

    View test on GitHub ↗
  1258. ✓test_the_tail_percentiles_reach_the_slow_samples_the_median_hides TestPercentilestest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1259. ✓test_the_unit_and_decimals_carry_into_the_message TestRatioBudgettest_phase_budget.py›

    No steps recorded

    View test on GitHub ↗
  1260. ✓test_throughput_spans_from_the_earliest_start_when_locust_reports_several_entries TestAggregatetest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1261. ✓test_throughput_spans_the_whole_window_and_latency_comes_from_the_histogram TestAggregatetest_locust_load.py›

    No steps recorded

    View test on GitHub ↗
  1262. ✓test_transient_failures_then_success_returns_the_success TestRetriedtest_session_anomaly.py›

    No steps recorded

    View test on GitHub ↗
  1263. ✓test_utilization_is_the_counter_delta_over_the_interval_not_the_counter_itself TestCpuUtilizationtest_proxy_usage.py›

    No steps recorded

    View test on GitHub ↗
  1264. ✓test_apply_guardrail_blocks_banned_and_allows_clean TestApplyGuardrailEndpointtest_apply_guardrail_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1265. ✓test_bedrock_post_call_blocks_denied_model_output TestBedrockGuardrailtest_bedrock_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1266. ✓test_bedrock_post_call_blocks_denied_streamed_output_and_passes_clean_streams TestBedrockGuardrailtest_bedrock_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a streaming /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1267. ✓test_bedrock_pre_call_blocks_harmful_prompt TestBedrockGuardrailtest_bedrock_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1268. ✓test_bedrock_pre_call_blocks_on_messages TestBedrockGuardrailtest_bedrock_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1269. ✓test_bedrock_pre_call_blocks_on_responses TestBedrockGuardrailtest_bedrock_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1270. ✓test_blocks_execution_request_but_allows_explanation TestBlockCodeExecutionGuardrailtest_block_code_execution_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-blockcode-backend-dc4ebb8e54d4 that calls gemini/gemini-2.5-flash
    3. Send a /chat/completions request to e2e-blockcode-backend-dc4ebb8e54d4
    View test on GitHub ↗
  1271. ✓test_chat_blocks_category TestOpenAIModerationCategoryMatrixtest_openai_moderation_category_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-mod-cat-chat-bd6dc6b2a576 that calls gemini/gemini-2.5-flash
    3. Send a /chat/completions request to e2e-mod-cat-chat-bd6dc6b2a576
    View test on GitHub ↗
  1272. ✓test_child_condition_match_applies_child_and_inherited_parent_guardrails TestPolicyInheritedGuardrailtest_policy_inherited_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1273. ✓test_child_condition_miss_still_applies_inherited_parent_guardrail TestPolicyInheritedGuardrailtest_policy_inherited_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1274. ✓test_during_call_blocks_stream_before_first_chunk TestBedrockDuringCallStreamingtest_streaming_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a streaming /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1275. ✓test_flag_returns_guardrail_information_for_the_guardrail_that_ran TestGuardrailInformationResponsetest_guardrail_information_response_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-guardrail-info-backend-880ac0e787f2 that calls openai/gpt-4.1-mini
    3. Send a /chat/completions request to e2e-guardrail-info-backend-880ac0e787f2
    View test on GitHub ↗
  1276. ✓test_global_guardrail_blocks_key_without_team_opt_out TestTeamDisableGlobalGuardrailtest_team_disable_global_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1277. ✓test_http_failure_is_not_hidden_by_a_later_success[400]test_guardrails_client.py›

    No steps recorded

    View test on GitHub ↗
  1278. ✓test_http_failure_is_not_hidden_by_a_later_success[401]test_guardrails_client.py›

    No steps recorded

    View test on GitHub ↗
  1279. ✓test_http_failure_is_not_hidden_by_a_later_success[429]test_guardrails_client.py›

    No steps recorded

    View test on GitHub ↗
  1280. ✓test_http_failure_is_not_hidden_by_a_later_success[500]test_guardrails_client.py›

    No steps recorded

    View test on GitHub ↗
  1281. ✓test_key_attached_content_filter_blocks_banned_video_prompt TestKeyAttachedGuardrailOnVideostest_key_guardrail_video_e2e.py›
    1. Generate a virtual key with user id: e2e-guardrails-user and metadata: guardrails: e2e-video-filter-bbe25f0a0f16
    2. Add a deployment named e2e-guard-video-ebb3c626b0e9 that calls vertex_ai/veo-3.1-fast-generate-001
    3. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1282. ✓test_masking_run_is_recorded_on_the_spend_log TestPresidioSpendLogRecordtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  1283. ✓test_messages_blocks_category TestOpenAIModerationCategoryMatrixtest_openai_moderation_category_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-mod-cat-msg-37bec3496b27 that calls anthropic/claude-haiku-4-5
    3. Send a /v1/messages request to e2e-mod-cat-msg-37bec3496b27
    View test on GitHub ↗
  1284. ✓test_missing_exact_guardrail_returns_failure_evidence_at_deadline[]test_guardrails_client.py›

    No steps recorded

    View test on GitHub ↗
  1285. ✓test_missing_exact_guardrail_returns_failure_evidence_at_deadline[global-filter]test_guardrails_client.py›

    No steps recorded

    View test on GitHub ↗
  1286. ✓test_missing_exact_guardrail_returns_failure_evidence_at_deadline[tool-permission-sibling]test_guardrails_client.py›

    No steps recorded

    View test on GitHub ↗
  1287. ✓test_moderation_blocks_flagged_input TestOpenAIModerationGuardrailtest_openai_moderation_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-moderation-backend-fa32405a85b4 that calls gemini/gemini-2.5-flash
    3. Send a /chat/completions request to e2e-moderation-backend-fa32405a85b4
    View test on GitHub ↗
  1288. ✓test_moderation_blocks_flagged_input_on_messages TestOpenAIModerationGuardrailtest_openai_moderation_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-moderation-msg-backend-e0d9f90b9680 that calls gemini/gemini-2.5-flash
    3. Send a /v1/messages request to e2e-moderation-msg-backend-e0d9f90b9680
    View test on GitHub ↗
  1289. ✓test_non_stream_block_is_schema_valid_json TestResponsesPreCallBlocktest_responses_pre_call_block_stream_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-responses-block-07a2f38b2bc1 that calls openai/gpt-4.1-mini
    View test on GitHub ↗
  1290. ✓test_post_call_masks_pii_in_model_output TestPresidioPostCallMaskingtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1291. ✓test_pre_call_allows_permitted_tool TestToolPermissionPreCalltest_tool_permission_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1292. ✓test_pre_call_blocks_tool_outside_the_allow_list TestToolPermissionPreCalltest_tool_permission_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1293. ✓test_pre_call_masks_pii_on_chat_completions TestPresidioPreCallMaskingtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1294. ✓test_pre_call_masks_pii_on_messages TestPresidioPreCallMaskingtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /v1/messages request to gemini-2.5-flash
    View test on GitHub ↗
  1295. ✓test_responses_blocks_category TestOpenAIModerationCategoryMatrixtest_openai_moderation_category_matrix_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-mod-cat-resp-9b78965a9600 that calls openai/gpt-4o-mini
    View test on GitHub ↗
  1296. ✓test_spend_log_stores_masked_output_on_anthropic_messages TestPresidioSpendLogStoresMaskedOutputtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1297. ✓test_spend_log_stores_masked_output_on_chat_completions TestPresidioSpendLogStoresMaskedOutputtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1298. ✓test_spend_log_stores_masked_output_on_responses TestPresidioSpendLogStoresMaskedOutputtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1299. ✓test_spend_log_stores_masked_output_on_streaming_anthropic_messages TestPresidioSpendLogStoresMaskedOutputtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a streaming /v1/messages request to gemini-2.5-flash
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1300. ✓test_spend_log_stores_masked_output_on_streaming_chat_completions TestPresidioSpendLogStoresMaskedOutputtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a streaming /chat/completions request to gemini-2.5-flash
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1301. ✓test_stream_block_is_sse_with_completed_assistant_message TestResponsesPreCallBlocktest_responses_pre_call_block_stream_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-responses-block-d101b60dcc8d that calls openai/gpt-4.1-mini
    View test on GitHub ↗
  1302. ✓test_team_with_disable_flag_bypasses_global_guardrail TestTeamDisableGlobalGuardrailtest_team_disable_global_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-guardrails-user and team id: 6a0953ae-43c6-48c5-96f5-dc601e7a3c16
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1303. ✓test_ui_default_scope_masks_a_card_number_the_model_generates_on_chat_completions TestPresidioCreditCardOutputMaskingtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1304. ✓test_ui_default_scope_masks_a_card_number_the_model_generates_on_streaming_anthropic_messages TestPresidioCreditCardOutputMaskingtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a streaming /v1/messages request to gemini-2.5-flash
    View test on GitHub ↗
  1305. ✓test_ui_default_scope_masks_a_card_number_the_model_generates_on_streaming_chat_completions TestPresidioCreditCardOutputMaskingtest_presidio_masking_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a streaming /chat/completions request to gemini-2.5-flash
    View test on GitHub ↗
  1306. ✓test_waits_for_requested_guardrail_after_an_unrelated_global_guardrailtest_guardrails_client.py›

    No steps recorded

    View test on GitHub ↗
  1307. ✓test_without_flag_response_has_no_guardrail_information TestGuardrailInformationResponsetest_guardrail_information_response_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-guardrail-info-backend-5b808474c3dc that calls openai/gpt-4.1-mini
    3. Send a /chat/completions request to e2e-guardrail-info-backend-5b808474c3dc
    View test on GitHub ↗
  1308. −test_request_naming_an_unknown_guardrail_fails_closedtest_guardrail_dispatch_e2e.py›

    No steps recorded

    stage red: product gap, a request naming a guardrail the proxy does not serve is silently served unguarded (200) instead of failing closed

    View test on GitHub ↗
  1309. ✓test_429_on_first_deployment_succeeds_on_retry TestReliabilityRetriestest_reliability_retries_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Generate a virtual key with models: gpt-5.5, user id: e2e-test-user and rpm limit: 1
    3. Add a deployment named reliability-retry-429-62d94dd6dbf5 that calls openai/gpt-5.5
    View test on GitHub ↗
  1310. ✓test_429_trips_cooldown_then_recovers TestReliabilityCooldownstest_reliability_cooldowns_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Generate a virtual key with models: gpt-5.5, user id: e2e-test-user and rpm limit: 1
    3. Add a deployment named reliability-cooldown-429-9bb869ac56be that calls openai/gpt-5.5
    View test on GitHub ↗
  1311. ✓test_5xx_on_first_deployment_succeeds_on_retry TestReliabilityRetriestest_reliability_retries_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-5xx-upstream-d36a3bea64d0 that calls openai/gpt-5.5
    3. Add a deployment named reliability-retry-5xx-fa79bdcd291c that calls openai/reliability-5xx-upstream-d36a3bea64d0
    4. Add a deployment named reliability-retry-5xx-fa79bdcd291c that calls openai/gpt-5.5
    View test on GitHub ↗
  1312. ✓test_5xx_routes_to_fallback TestReliabilityFallbackstest_reliability_fallbacks_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-fail-554cae1ebf60 that calls openai/gpt-5.5
    View test on GitHub ↗
  1313. ✓test_5xx_trips_cooldown_then_recovers TestReliabilityCooldownstest_reliability_cooldowns_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-cooldown-5xx-upstream-b7d78d9809d6 that calls openai/gpt-5.5
    3. Add a deployment named reliability-cooldown-5xx-6a8a3922f5e4 that calls openai/reliability-cooldown-5xx-upstream-b7d78d9809d6
    4. Add a deployment named reliability-cooldown-5xx-6a8a3922f5e4 that calls openai/gpt-5.5
    View test on GitHub ↗
  1314. ✓test_alias_api_key_never_overrides_the_tier_credential TestAliasParamForwardingtest_auto_router_regressions_e2e.py›
    1. Add a deployment named e2e-cred-tier-03a5f22032eb that calls anthropic/claude-haiku-4-5
    2. Add a deployment named e2e-cred-alias-03a5f22032eb that calls auto_router/complexity_router
    3. Generate a virtual key with models: e2e-cred-alias-03a5f22032eb, e2e-cred-tier-03a5f22032eb and user id: e2e-auto-router-regressions
    4. Send a /chat/completions request to e2e-cred-alias-03a5f22032eb
    5. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1315. ✓test_auth_failure_on_first_deployment_succeeds_on_retry TestReliabilityRetriestest_reliability_retries_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-retry-auth-10540a663249 that calls openai/gpt-5.5
    View test on GitHub ↗
  1316. ✓test_auth_failure_trips_cooldown_then_recovers TestReliabilityCooldownstest_reliability_cooldowns_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-cooldown-auth-6f46cf0148ea that calls openai/gpt-5.5
    View test on GitHub ↗
  1317. ✓test_body_tagged_chat_reaches_the_untagged_tier_after_marker_rewrite TestUntaggedTierDeploymentstest_auto_router_regressions_e2e.py›
    1. Generate a virtual key with models: e2e-autoroute-88f2a972db0e, e2e-tier-88f2a972db0e, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    2. Send a /chat/completions request to e2e-autoroute-88f2a972db0e
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1318. ✓test_body_tagged_chat_routes_through_the_marker_to_its_tier TestTagSplitRoutingtest_auto_router_regressions_e2e.py›
    1. Add a deployment named e2e-autoroute-617b0a7c3ca3 that calls anthropic/claude-sonnet-5
    2. Add a deployment named e2e-tier-617b0a7c3ca3 that calls anthropic/claude-haiku-4-5
    3. Add a deployment named e2e-autoroute-617b0a7c3ca3 that calls auto_router/complexity_router
    4. Generate a virtual key with models: e2e-autoroute-617b0a7c3ca3, e2e-tier-617b0a7c3ca3, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    5. Send a /chat/completions request to e2e-autoroute-617b0a7c3ca3
    6. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1319. ✓test_body_tagged_responses_with_list_input_routes_to_the_tier TestResponsesApiTagRoutingtest_auto_router_regressions_e2e.py›
    1. Generate a virtual key with models: e2e-autoroute-89d4fd6b6bab, e2e-tier-89d4fd6b6bab, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1320. ✓test_cached_conversation_stays_on_deployment_holding_its_cache TestReliabilityPromptCachingAffinitytest_reliability_prompt_caching_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Read the router settings from /router/settings
    3. Add a deployment named reliability-cache-97a3a1d96827 that calls anthropic/claude-haiku-4-5
    4. Add a deployment named reliability-cache-97a3a1d96827 that calls openai/gpt-5.5
    View test on GitHub ↗
  1321. ✓test_client_hanging_up_never_benches_the_deployment TestReliabilityCancelOnDisconnecttest_reliability_cancel_on_disconnect_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Check whether the general setting cancel_on_disconnect is on
    3. Add a deployment named reliability-cooldown-disconnect-2d0618e4af5e that calls azure/gpt-5.4-nano
    4. Add a deployment named reliability-cooldown-disconnect-2d0618e4af5e that calls openai/gpt-5.5
    View test on GitHub ↗
  1322. ✓test_content_policy_routes_to_fallback TestReliabilityFallbackstest_reliability_fallbacks_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-policyfail-609805e254c9 that calls azure/gpt-5.4-nano
    View test on GitHub ↗
  1323. ✓test_context_window_refusal_on_first_deployment_succeeds_on_retry TestReliabilityRetriestest_reliability_retries_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-retry-context-72c737c70022 that calls openai/gpt-3.5-turbo
    3. Add a deployment named reliability-retry-context-72c737c70022 that calls openai/gpt-5.5
    View test on GitHub ↗
  1324. ✓test_context_window_routes_to_fallback TestReliabilityFallbackstest_reliability_fallbacks_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-ctxfail-995d4f7e6354 that calls openai/gpt-3.5-turbo
    View test on GitHub ↗
  1325. ✓test_cost_based_picks_cheapest_deployment TestReliabilityRoutingStrategiestest_reliability_routing_strategies_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-cost-02f6f443220b that calls openai/gpt-5.5
    View test on GitHub ↗
  1326. ✓test_exact_cache_returns_cached TestReliabilityCachetest_reliability_cache_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named e2e-cache-ee3ad198ef15 that calls openai/gpt-5.6
    View test on GitHub ↗
  1327. ✓test_failing_requests_do_not_grow_rss_or_stored_request TestReliabilityMemorytest_reliability_memory_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-memory-509b220f2ed8 that calls openai/gpt-5.5
    3. Add a deployment named reliability-memory-fb-509b220f2ed8 that calls openai/gpt-5.5
    4. Wait for the request's spend log in /spend/logs
    5. Read memory usage from /debug/memory/summary on every proxy replica
    View test on GitHub ↗
  1328. ✓test_header_tagged_messages_routes_through_the_marker_to_an_untagged_tier TestUntaggedTierDeploymentstest_auto_router_regressions_e2e.py›
    1. Add a deployment named e2e-autoroute-88f2a972db0e that calls auto_router/complexity_router
    2. Add a deployment named e2e-tier-88f2a972db0e that calls anthropic/claude-haiku-4-5
    3. Add a deployment named e2e-autoroute-88f2a972db0e that calls anthropic/claude-sonnet-5
    4. Generate a virtual key with models: e2e-autoroute-88f2a972db0e, e2e-tier-88f2a972db0e, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    5. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1329. ✓test_header_tagged_responses_with_string_input_routes_to_the_tier TestResponsesApiTagRoutingtest_auto_router_regressions_e2e.py›
    1. Add a deployment named e2e-autoroute-89d4fd6b6bab that calls anthropic/claude-sonnet-5
    2. Add a deployment named e2e-tier-89d4fd6b6bab that calls anthropic/claude-haiku-4-5
    3. Add a deployment named e2e-autoroute-89d4fd6b6bab that calls auto_router/complexity_router
    4. Generate a virtual key with models: e2e-autoroute-89d4fd6b6bab, e2e-tier-89d4fd6b6bab, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    5. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1330. ✓test_request_timeout_exceeds_deadline TestReliabilityTimeoutstest_reliability_timeouts_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-timeout-471612f291cb that calls openai/gpt-5.5
    View test on GitHub ↗
  1331. ✓test_responses_input_reaches_the_semantic_auto_router TestSemanticAutoRouterResponsestest_auto_router_regressions_e2e.py›
    1. Add a deployment named e2e-semantic-embedding-58c84e6844be that calls openai/text-embedding-3-small
    2. Add a deployment named e2e-semantic-target-58c84e6844be that calls anthropic/claude-haiku-4-5
    3. Add a deployment named e2e-semantic-fallback-58c84e6844be that calls anthropic/claude-sonnet-5
    4. Add a deployment named e2e-semantic-router-58c84e6844be that calls auto_router/e2e-semantic-router-58c84e6844be
    5. Generate a virtual key with models: e2e-semantic-router-58c84e6844be, e2e-semantic-target-58c84e6844be, e2e-semantic-fallback-58c84e6844be and user id: e2e-auto-router-regressions
    6. Wait for at least 2 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1332. ✓test_simple_shuffle_honors_weights TestReliabilityRoutingStrategiestest_reliability_routing_strategies_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-shuffle-a58784c96f3a that calls openai/gpt-5.5
    View test on GitHub ↗
  1333. ✓test_stream_timeout_exceeds_deadline TestReliabilityTimeoutstest_reliability_timeouts_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-stream-timeout-7c2e7af539cb that calls openai/gpt-5.5
    View test on GitHub ↗
  1334. ✓test_tagged_call_straight_at_an_untagged_deployment_stays_denied TestUntaggedTierDeploymentstest_auto_router_regressions_e2e.py›
    1. Generate a virtual key with models: e2e-tier-88f2a972db0e, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    2. Send a /chat/completions request to e2e-tier-88f2a972db0e
    View test on GitHub ↗
  1335. ✓test_timeout_on_first_deployment_succeeds_on_retry TestReliabilityRetriestest_reliability_retries_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-retry-78fa736aa336 that calls openai/gpt-5.5
    View test on GitHub ↗
  1336. ✓test_timeout_routes_to_fallback TestReliabilityFallbackstest_reliability_fallbacks_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-tofail-cecfcd7469ef that calls openai/gpt-5.5
    View test on GitHub ↗
  1337. ✓test_timeout_trips_cooldown_then_recovers TestReliabilityCooldownstest_reliability_cooldowns_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-cooldown-timeout-ee91ac892e5d that calls openai/gpt-5.5
    View test on GitHub ↗
  1338. ✓test_trivial_ask_behind_keyword_heavy_system_prompt_stays_on_the_cheap_tier TestComplexityHeuristicScopetest_auto_router_regressions_e2e.py›
    1. Add a deployment named e2e-heuristic-cheap-2e01a9043e1a that calls anthropic/claude-haiku-4-5
    2. Add a deployment named e2e-heuristic-strong-2e01a9043e1a that calls openai/gpt-5.6
    3. Add a deployment named e2e-heuristic-router-2e01a9043e1a that calls auto_router/complexity_router
    4. Generate a virtual key with models: e2e-heuristic-router-2e01a9043e1a, e2e-heuristic-cheap-2e01a9043e1a, e2e-heuristic-strong-2e01a9043e1a and user id: e2e-auto-router-regressions
    5. Send a /chat/completions request to e2e-heuristic-router-2e01a9043e1a
    6. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1339. ✓test_untagged_chat_is_always_served_by_the_plain_deployment TestTagSplitRoutingtest_auto_router_regressions_e2e.py›
    1. Generate a virtual key with models: e2e-autoroute-617b0a7c3ca3, e2e-tier-617b0a7c3ca3, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    2. Send a /chat/completions request to e2e-autoroute-617b0a7c3ca3
    3. Wait for at least 5 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1340. ✓test_untagged_messages_is_served_by_the_plain_deployment TestTagSplitRoutingtest_auto_router_regressions_e2e.py›
    1. Generate a virtual key with models: e2e-autoroute-617b0a7c3ca3, e2e-tier-617b0a7c3ca3, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    2. Send a /v1/messages request to e2e-autoroute-617b0a7c3ca3
    3. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1341. ✓test_untagged_responses_is_served_by_the_plain_deployment TestResponsesApiTagRoutingtest_auto_router_regressions_e2e.py›
    1. Generate a virtual key with models: e2e-autoroute-89d4fd6b6bab, e2e-tier-89d4fd6b6bab, user id: e2e-auto-router-regressions and router settings: enable tag filtering: True
    2. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1342. ✓test_usage_based_picks_deployment_with_tpm_headroom TestReliabilityRoutingStrategiestest_reliability_routing_strategies_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Add a deployment named reliability-usage-14704eadc21c that calls openai/gpt-5.5
    View test on GitHub ↗
  1343. ✓test_workers_idle_under_rss_budget_before_traffic TestReliabilityMemorytest_reliability_memory_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1344. ✓test_zero_priced_alias_still_logs_spend_at_the_tier_rate TestStrategyAliasPricingtest_auto_router_regressions_e2e.py›
    1. Add a deployment named e2e-priced-tier-289a307f1e36 that calls anthropic/claude-haiku-4-5
    2. Add a deployment named e2e-priced-alias-289a307f1e36 that calls auto_router/complexity_router
    3. Generate a virtual key with models: e2e-priced-alias-289a307f1e36, e2e-priced-tier-289a307f1e36 and user id: e2e-auto-router-regressions
    4. Send a /chat/completions request to e2e-priced-alias-289a307f1e36
    5. Wait for at least 1 of the key's spend logs in /spend/logs
    View test on GitHub ↗
  1345. −test_latency_based_routes_around_deployment_that_times_out TestReliabilityRoutingStrategiestest_reliability_routing_strategies_e2e.py›

    No steps recorded

    LIT-7682: since #40229 the per-request routing_strategy override runs without the latency sampler, so latency-based has no signal to route on

    View test on GitHub ↗
  1346. −test_least_busy_avoids_deployment_with_request_in_flight TestReliabilityRoutingStrategiestest_reliability_routing_strategies_e2e.py›

    No steps recorded

    LIT-7682: since #40229 the per-request routing_strategy override runs without the in-flight counter, so least-busy has no signal to route on

    View test on GitHub ↗
  1347. −test_llm_classifier_runs_and_routes_by_semantic_tier TestComplexityRouterLlmClassifiertest_complexity_router_e2e.py›

    No steps recorded

    product bug LIT-4521: LLM classifier returns SIMPLE for short hard prompts (e.g. Is P equal to NP?); re-enable when classifier tier quality is fixed

    View test on GitHub ↗
  1348. −test_sibling_replica_serves_backup_within_redis_read_interval TestReliabilityCooldownstest_reliability_cooldowns_e2e.py›

    No steps recorded

    this cell trips a deployment through one gateway and reads the bench from another, so LITELLM_PROXY_REPLICA_URLS has to name at least two distinct gateways, got ('http://e2e-bk-367-router.litellm.svc.cluster.local',)

    View test on GitHub ↗
  1349. ✓test_a_kept_header_is_identity TestKeyDistinctnesstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1350. ✓test_a_volatile_header_is_not_identity TestKeyDistinctnesstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1351. ✓test_file_content_is_identity TestKeyDistinctnesstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1352. ✓test_form_fields_are_identity TestKeyDistinctnesstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1353. ✓test_generated_ids_in_the_path_do_not_change_the_key TestKeyStabilitytest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1354. ✓test_key_names_method_path_and_digest TestKeyShapetest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1355. ✓test_names_that_carry_credentials[api_base-False] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1356. ✓test_names_that_carry_credentials[api_key-True] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1357. ✓test_names_that_carry_credentials[aws_secret_access_key-True] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1358. ✓test_names_that_carry_credentials[aws_session_token-True] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1359. ✓test_names_that_carry_credentials[langfuse_secret_key-True] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1360. ✓test_names_that_carry_credentials[max_completion_tokens-False] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1361. ✓test_names_that_carry_credentials[model-False] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1362. ✓test_names_that_carry_credentials[openai_api_key-True] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1363. ✓test_names_that_carry_credentials[static_headers-True] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1364. ✓test_names_that_carry_credentials[vertex_credentials-True] TestSecretFieldstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1365. ✓test_query_params_are_identity TestKeyDistinctnesstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1366. ✓test_requests_differing_only_inside_canonicalized_fields_stay_distinct TestKeyDistinctnesstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1367. ✓test_rewrites_exactly_the_volatile_shapes[0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef-<sha256>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1368. ✓test_rewrites_exactly_the_volatile_shapes[2026-08-19-<date>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1369. ✓test_rewrites_exactly_the_volatile_shapes[2026-08-19T20:57:13.363499+00:00-<timestamp>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1370. ✓test_rewrites_exactly_the_volatile_shapes[9f1c8a2e-4b3d-4f6a-8f2f-0a1b2c3d4e5f-<uuid>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1371. ✓test_rewrites_exactly_the_volatile_shapes[batch_688a8b7f9a08819096e0f7c88fcd07c5-<id>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1372. ✓test_rewrites_exactly_the_volatile_shapes[chatcmpl-C0LO6rRkfJlpJ2mqW9BHYo4Sm8FWl-<id>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1373. ✓test_rewrites_exactly_the_volatile_shapes[e2e-chat-stream-4d5152a995b7-e2e-chat-stream-<marker>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1374. ✓test_rewrites_exactly_the_volatile_shapes[file-XyZ12345abc-<id>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1375. ✓test_rewrites_exactly_the_volatile_shapes[gpt-4o-mini-gpt-4o-mini] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1376. ✓test_rewrites_exactly_the_volatile_shapes[max_tokens-max_tokens] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1377. ✓test_rewrites_exactly_the_volatile_shapes[Reply ok. 4d5152a995b7-Reply ok. <marker>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1378. ✓test_rewrites_exactly_the_volatile_shapes[sk-1234-sk-1234] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1379. ✓test_rewrites_exactly_the_volatile_shapes[sk-3mCXCTGmYuEEIU2i2qmVE3Xq6tSK1O0X6ZIRP1Lpw8ZlbNjt-<key>] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1380. ✓test_rewrites_exactly_the_volatile_shapes[zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz] TestPlaceholderstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1381. ✓test_secret_set_versus_unset_stays_distinct TestKeyDistinctnesstest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1382. ✓test_serialization_order_is_not_identity TestKeyStabilitytest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1383. ✓test_volatile_material_does_not_change_the_key TestKeyStabilitytest_fixture_canonical.py›

    No steps recorded

    View test on GitHub ↗
  1384. ✓test_429_is_never_retried TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1385. ✓test_a_non_streaming_outcome_carries_no_arrivals TestStreamEventArrivalstest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1386. ✓test_body_that_is_not_json_is_still_a_validation_failure TestClassifyEmptyBodytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1387. ✓test_each_event_is_stamped_at_the_moment_its_line_arrives TestStreamEventArrivalstest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1388. ✓test_empty_2xx_body_is_a_success TestClassifyEmptyBodytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1389. ✓test_malformed_guardrail_mode_remains_a_validation_failure[5] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1390. ✓test_malformed_guardrail_mode_remains_a_validation_failure[mode1] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1391. ✓test_malformed_guardrail_mode_remains_a_validation_failure[mode2] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1392. ✓test_non_transient_status_returns_immediately[200] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1393. ✓test_non_transient_status_returns_immediately[201] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1394. ✓test_non_transient_status_returns_immediately[400] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1395. ✓test_non_transient_status_returns_immediately[401] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1396. ✓test_non_transient_status_returns_immediately[404] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1397. ✓test_non_transient_status_returns_immediately[422] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1398. ✓test_non_transient_status_returns_immediately[500] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1399. ✓test_non_transient_status_returns_immediately[502] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1400. ✓test_non_transient_status_returns_immediately[503] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1401. ✓test_non_transient_status_returns_immediately[504] TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1402. ✓test_overloaded_529_retries_with_backoff_then_returns_the_success TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1403. ✓test_partial_body_omits_unset_fields_and_sends_explicit_none_as_null TestWireBodytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1404. ✓test_persistent_transient_is_bounded_and_returns_the_last_response TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1405. ✓test_plain_body_drops_none_fields TestWireBodytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1406. ✓test_qualification_disables_retries_and_restores_the_default TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1407. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[mode2-False] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1408. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[mode2-True] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1409. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[mode3-False] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1410. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[mode3-True] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1411. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[mode4-False] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1412. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[mode4-True] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1413. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[None-False] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1414. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[None-True] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1415. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[post_call-False] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1416. ✓test_supported_guardrail_modes_preserve_neighbor_attribution_and_masked_response[post_call-True] TestSpendLogDecodingtest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1417. ✓test_transient_set_is_only_statuses_the_proxy_cannot_emit TestTransientRetryPolicytest_e2e_http.py›

    No steps recorded

    View test on GitHub ↗
  1418. ✓test_attach_is_idempotent TestResultPropertiestest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1419. ✓test_bar-logging] TestPackageFromNodeid::test_package_is_the_first_dir_under_the_suite_root[logging/test_x.py::TestFootest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1420. ✓test_bar-logging] TestPackageFromNodeid::test_package_is_the_first_dir_under_the_suite_root[tests/e2e/logging/test_x.py::TestFootest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1421. ✓test_bar-quota_management] TestPackageFromNodeid::test_package_is_the_first_dir_under_the_suite_root[quota_management/spend_tracking/test_x.pytest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1422. ✓test_bar-root] TestPackageFromNodeid::test_package_is_the_first_dir_under_the_suite_root[tests/e2e/test_fixture_mode.pytest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1423. ✓test_both_invocation_shapes_collapse_to_the_same_components[./logging/test_x.py] TestSuitePartstest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1424. ✓test_both_invocation_shapes_collapse_to_the_same_components[logging/test_x.py] TestSuitePartstest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1425. ✓test_both_invocation_shapes_collapse_to_the_same_components[tests/e2e/logging/test_x.py] TestSuitePartstest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1426. ✓test_both_invocation_shapes_collapse_to_the_same_components[tests\\e2e\\logging\\test_x.py] TestSuitePartstest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1427. ✓test_every_test_carries_package_covers_and_source TestResultPropertiestest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1428. ✓test_ids_are_unique_order_preserving_and_non_empty_strings TestDedupeCoverstest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1429. ✓test_known_values_normalize-root] TestPackageFromNodeid::test_package_is_the_first_dir_under_the_suite_root[test_fixture_mode.py::TestParseFixtureModetest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1430. ✓test_line_is_emitted_one_based TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1431. ✓test_nothing_linkable_yields_empty_rather_than_a_guess[-40] TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1432. ✓test_nothing_linkable_yields_empty_rather_than_a_guess[../conftest.py-40] TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1433. ✓test_nothing_linkable_yields_empty_rather_than_a_guess[/app/e2e/a2a/test_x.py-40] TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1434. ✓test_nothing_linkable_yields_empty_rather_than_a_guess[a2a/test_x.py-None] TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1435. ✓test_nothing_linkable_yields_empty_rather_than_a_guess[C:\\app\\e2e\\a2a\\test_x.py-40] TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1436. ✓test_path_is_repo_relative_however_pytest_was_started[a2a/test_a2a_agent_e2e.py] TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1437. ✓test_path_is_repo_relative_however_pytest_was_started[tests/e2e/a2a/test_a2a_agent_e2e.py] TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1438. ✓test_top_level_suite_file_keeps_its_single_component TestSuitePartstest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1439. ✓test_top_level_suite_file_sits_directly_under_the_suite_root TestSourceFromLocationtest_junit_properties.py›

    No steps recorded

    View test on GitHub ↗
  1440. −test_suite_root_names_this_file_s_real_home TestSuiteRoottest_junit_properties.py›

    No steps recorded

    no checkout above this file (the runner image copies tests/e2e/ to /app/e2e)

    View test on GitHub ↗
  1441. ✓test_a_missing_admin_credential_fails_loudly_instead_of_skipping[ ]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1442. ✓test_a_missing_admin_credential_fails_loudly_instead_of_skipping[]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1443. ✓test_a_refused_create_fails_the_test_with_the_idps_own_wordstest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1444. ✓test_cleanup_failure_is_visibletest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1445. ✓test_connection_details_come_from_the_environmenttest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1446. ✓test_create_without_a_resource_id_fails[]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1447. ✓test_create_without_a_resource_id_fails[http://keycloak/groups/]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1448. ✓test_created_id_is_the_last_segment_of_the_location_headertest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1449. ✓test_expired_admin_credentials_do_not_abort_remaining_cleanupstest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1450. ✓test_new_users_are_born_fully_set_uptest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1451. ✓test_oidc_launcher_removes_client_on_exit_and_termination[group-False]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1452. ✓test_oidc_launcher_removes_client_on_exit_and_termination[normal-False]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1453. ✓test_oidc_launcher_removes_client_on_exit_and_termination[parent-False]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1454. ✓test_oidc_launcher_removes_client_on_exit_and_termination[parent-True]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1455. ✓test_partial_provisioning_removes_the_group_when_user_creation_failstest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1456. ✓test_provisioning_records_zero_one_or_multiple_groups[groups0]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1457. ✓test_provisioning_records_zero_one_or_multiple_groups[groups1]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1458. ✓test_provisioning_records_zero_one_or_multiple_groups[groups2]test_idp.py›

    No steps recorded

    View test on GitHub ↗
  1459. ✓test_realm_urls_match_keycloaks_own_layouttest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1460. ✓test_strict_cleanup_reports_each_failure_and_continuestest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1461. ✓test_successful_provisioning_cleans_up_user_before_grouptest_idp.py›

    No steps recorded

    View test on GitHub ↗
  1462. ✓test_encrypted_stored_value_is_not_a_bearer_token TestSessionTokentest_session_token_e2e.py›
    1. Generate a virtual key with key alias: e2e-session-d1a3a630bd63 and metadata: logging: callback name: langfuse and callback vars: default settings
    2. Read the key's settings back from /key/info
    View test on GitHub ↗
  1463. ✓test_expired_session_token_is_denied TestSessionTokentest_session_token_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1464. ✓test_expired_token_is_rejected TestJwtAuthtest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-34302445706f and team id: e2e-jwt-team-34302445706f
    2. Send a /chat/completions request to gpt-5.5
    View test on GitHub ↗
  1465. ✓test_liveness_reports_alive_without_auth TestHealthLifecycletest_health_lifecycle_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1466. ✓test_master_key_authenticates_and_grants_admin_route TestMasterKeyAuthtest_master_key_auth_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1467. ✓test_non_matching_master_key_is_denied TestMasterKeyAuthtest_master_key_auth_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1468. ✓test_plain_virtual_key_still_works_with_jwt_auth_enabled TestJwtAuthtest_jwt_auth_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Send a /chat/completions request to gpt-5.5
    View test on GitHub ↗
  1469. ✓test_readiness_details_require_auth_and_expose_diagnostics TestHealthLifecycletest_health_lifecycle_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1470. ✓test_readiness_is_reachable_without_credentials TestHealthLifecycletest_health_lifecycle_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1471. ✓test_readiness_reports_connected_db TestHealthLifecycletest_health_lifecycle_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1472. ✓test_signed_token_for_another_application_is_rejected TestJwtAuthtest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-6e1363258589 and team id: e2e-jwt-team-6e1363258589
    2. Send a /chat/completions request to gpt-5.5
    View test on GitHub ↗
  1473. ✓test_signed_token_from_the_wrong_issuer_is_rejected TestJwtAuthtest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-52d246aa3657 and team id: e2e-jwt-team-52d246aa3657
    2. Send a /chat/completions request to gpt-5.5
    View test on GitHub ↗
  1474. ✓test_tampered_signature_is_rejected TestJwtAuthtest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-0e147af602dd and team id: e2e-jwt-team-0e147af602dd
    2. Send a /chat/completions request to gpt-5.5
    View test on GitHub ↗
  1475. ✓test_team_header_with_the_alias_of_a_team_the_caller_is_not_in_is_rejected_like_an_unknown_value TestJwtTeamHeadertest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-alias-3e4966296b10 and team id: e2e-jwt-team-3e4966296b10
    2. Create a team with team alias: e2e-jwt-alias-6ba3354ae0f6 and team id: e2e-jwt-other-6ba3354ae0f6
    View test on GitHub ↗
  1476. ✓test_team_header_with_the_team_alias_binds_the_same_team_as_the_team_id TestJwtTeamHeadertest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-alias-659ca339f6b4 and team id: e2e-jwt-team-659ca339f6b4
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  1477. ✓test_team_model_alias_is_listed_by_v1_models_under_the_same_token_that_routes_it[anthropic_shape] TestJwtTeamHeadertest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-aliased-eba2cfaeb4a7, models: gpt-5.5, team id: e2e-jwt-team-eba2cfaeb4a7 and model aliases: e2e-jwt-model-alias-eba2cfaeb4a7: gpt-5.5
    2. Send a /chat/completions request to e2e-jwt-model-alias-eba2cfaeb4a7
    View test on GitHub ↗
  1478. ✓test_team_model_alias_is_listed_by_v1_models_under_the_same_token_that_routes_it[openai_shape] TestJwtTeamHeadertest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-aliased-4837d19c23c4, models: gpt-5.5, team id: e2e-jwt-team-4837d19c23c4 and model aliases: e2e-jwt-model-alias-4837d19c23c4: gpt-5.5
    2. Send a /chat/completions request to e2e-jwt-model-alias-4837d19c23c4
    View test on GitHub ↗
  1479. ✓test_token_naming_a_team_that_does_not_exist_is_rejected TestJwtAuthtest_jwt_auth_e2e.py›
    1. Send a /chat/completions request to gpt-5.5
    View test on GitHub ↗
  1480. ✓test_unexpired_session_token_reaches_admin_route TestSessionTokentest_session_token_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1481. ✓test_valid_token_for_an_existing_team_is_accepted_and_attributed TestJwtAuthtest_jwt_auth_e2e.py›
    1. Create a team with team alias: e2e-jwt-62f623f2de4e and team id: e2e-jwt-team-62f623f2de4e
    2. Send a /chat/completions request to gpt-5.5
    3. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  1482. ✓test_allowed_model_is_permitted TestAccessControltest_access_control_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash and user id: e2e-test-user
    View test on GitHub ↗
  1483. ✓test_bearer_invalid_token_is_denied TestChatAuthHeaderstest_chat_auth_headers_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1484. ✓test_disallowed_model_is_denied_403 TestAccessControltest_access_control_e2e.py›
    1. Generate a virtual key with models: gemini-2.5-flash and user id: e2e-test-user
    View test on GitHub ↗
  1485. ✓test_empty_bearer_token_is_denied TestChatAuthHeaderstest_chat_auth_headers_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1486. ✓test_group_grants_every_deployment_in_it[bare name the group's wildcard covers-<lambda>] TestKeyScopedToAccessGrouptest_model_access_group_e2e.py›
    1. Add a deployment named openai/gpt-5.4* that calls openai/gpt-5.4*
    2. Add a deployment named e2e-ag-member-33f89fac2427 that calls openai/gpt-5.4-nano
    3. Add a deployment named e2e-ag-outsider-33f89fac2427 that calls openai/gpt-5.4-nano
    4. Generate a virtual key with models: e2e-ag-33f89fac2427 and user id: e2e-test-user
    View test on GitHub ↗
  1487. ✓test_group_grants_every_deployment_in_it[exactly-named deployment in the group-<lambda>] TestKeyScopedToAccessGrouptest_model_access_group_e2e.py›
    1. Generate a virtual key with models: e2e-ag-33f89fac2427 and user id: e2e-test-user
    View test on GitHub ↗
  1488. ✓test_group_grants_every_deployment_in_it[provider-prefixed name the group's wildcard covers-<lambda>] TestKeyScopedToAccessGrouptest_model_access_group_e2e.py›
    1. Generate a virtual key with models: e2e-ag-33f89fac2427 and user id: e2e-test-user
    View test on GitHub ↗
  1489. ✓test_group_grants_nothing_outside_it[deployment outside the group-<lambda>] TestKeyScopedToAccessGrouptest_model_access_group_e2e.py›
    1. Generate a virtual key with models: e2e-ag-33f89fac2427 and user id: e2e-test-user
    View test on GitHub ↗
  1490. ✓test_group_grants_nothing_outside_it[name no provider claims-<lambda>] TestKeyScopedToAccessGrouptest_model_access_group_e2e.py›
    1. Generate a virtual key with models: e2e-ag-33f89fac2427 and user id: e2e-test-user
    View test on GitHub ↗
  1491. ✓test_group_grants_nothing_outside_it[provider model outside the group's wildcard-<lambda>] TestKeyScopedToAccessGrouptest_model_access_group_e2e.py›
    1. Generate a virtual key with models: e2e-ag-33f89fac2427 and user id: e2e-test-user
    View test on GitHub ↗
  1492. ✓test_group_grants_the_team_nothing_outside_it TestTeamScopedToAccessGrouptest_model_access_group_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1493. ✓test_group_grants_the_teams_own_wildcard TestTeamScopedToAccessGrouptest_model_access_group_e2e.py›
    1. Generate a virtual key with team id: df210965-ffdb-4a05-a86e-26ef5f6f18d6
    2. Add a deployment named openai/gpt-5.6* that calls openai/gpt-5.6*
    3. Read /team/info?team_id=df210965-ffdb-4a05-a86e-26ef5f6f18d6 on every proxy replica until it settles
    View test on GitHub ↗
  1494. ✓test_llm_api_routes_group_grants_every_llm_endpoint TestAccessControltest_access_control_e2e.py›
    1. Generate a virtual key with allowed routes: llm_api_routes
    2. Send an /embeddings request to openai-text-embedding-3-small for "route group 691e58bd9857"
    View test on GitHub ↗
  1495. ✓test_llm_only_key_forbidden_from_management_route_403 TestAccessControltest_access_control_e2e.py›
    1. Generate a virtual key with allowed routes: llm_api_routes
    View test on GitHub ↗
  1496. ✓test_missing_authorization_header_is_denied TestChatAuthHeaderstest_chat_auth_headers_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1497. ✓test_not_bearer_scheme_is_denied TestChatAuthHeaderstest_chat_auth_headers_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1498. ✓test_token_without_bearer_prefix_is_denied TestChatAuthHeaderstest_chat_auth_headers_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1499. ✓test_unknown_model_returns_400 TestAccessControltest_access_control_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1500. ✓test_valid_key_allows_and_invalid_key_denied TestVirtualKeyAuthtest_access_control_e2e.py›
    1. Add a deployment named e2e-auth-chat-bed6360f3ac5 that calls anthropic/claude-haiku-4-5-20251001
    2. Generate a virtual key with user id: e2e-test-user
    3. Send a /chat/completions request to e2e-auth-chat-bed6360f3ac5
    View test on GitHub ↗
  1501. ✓test_a_streamed_response_round_trips_through_the_bundle TestRecordAndLoadtest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1502. ✓test_bundle_at_the_limit_is_still_fresh TestFreshnesstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1503. ✓test_corrupt_manifest_is_unreadable TestFreshnesstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1504. ✓test_fresh_directory_gets_a_fresh_manifest TestPrepareBundletest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1505. ✓test_interaction_filename_orders_and_slugs TestSlugstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1506. ✓test_interactions_group_per_test TestRecordAndLoadtest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1507. ✓test_load_bundle_rejects_a_foreign_format_version TestRecordAndLoadtest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1508. ✓test_load_returns_interactions_in_recorded_order TestRecordAndLoadtest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1509. ✓test_missing_manifest_is_unreadable_with_recording_hint TestFreshnesstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1510. ✓test_naive_recorded_at_is_read_as_utc TestFreshnesstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1511. ✓test_record_wipes_the_previous_bundle_instead_of_reading_it TestPrepareBundletest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1512. ✓test_refuses_a_path_that_is_a_file TestPrepareBundletest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1513. ✓test_refuses_to_wipe_a_directory_that_is_not_a_bundle TestPrepareBundletest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1514. ✓test_same_tail_in_different_files_never_collides TestSlugstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1515. ✓test_slug_for_test_is_deterministic TestSlugstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1516. ✓test_stale_bundle_reports_age_and_limit TestFreshnesstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1517. ✓test_unknown_format_version_is_unreadable TestFreshnesstest_fixture_bundle.py›

    No steps recorded

    View test on GitHub ↗
  1518. ✓test_fresh_replay_bundle_collects TestCollectionGatetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1519. ✓test_invalid_mode_names_the_value_and_the_choices TestCollectionGatetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1520. ✓test_known_values_normalize[ REPLAY -replay] TestParseFixtureModetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1521. ✓test_known_values_normalize[-live] TestParseFixtureModetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1522. ✓test_known_values_normalize[live-live] TestParseFixtureModetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1523. ✓test_known_values_normalize[record-record] TestParseFixtureModetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1524. ✓test_known_values_normalize[replay-replay] TestParseFixtureModetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1525. ✓test_live_and_record_never_block_collection[] TestCollectionGatetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1526. ✓test_live_and_record_never_block_collection[live] TestCollectionGatetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1527. ✓test_live_and_record_never_block_collection[record] TestCollectionGatetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1528. ✓test_live_mode_prints_nothing TestReportHeadertest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1529. ✓test_names_this_test_and_strips_the_phase TestCurrentTestKeytest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1530. ✓test_record_and_replay_name_the_bundle TestReportHeadertest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1531. ✓test_replay_with_no_bundle_says_how_to_record_one TestCollectionGatetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1532. ✓test_sequence_is_a_pure_function_of_test_and_ordinal TestDeterministicMarkertest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1533. ✓test_stale_replay_bundle_fails_naming_its_age TestCollectionGatetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1534. ✓test_unknown_value_is_invalid_with_the_original_spelling TestParseFixtureModetest_fixture_mode.py›

    No steps recorded

    View test on GitHub ↗
  1535. ✓test_cell_claimed_only_by_a_skipped_test_is_uncoveredtest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1536. ✓test_collection_counts_only_markers_on_tests_that_would_runtest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1537. ✓test_compute_coverage_counts_covered_p0_and_gapstest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1538. ✓test_json_render_exposes_module_coverage_for_grafana_jobstest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1539. ✓test_llm_cells_roll_up_by_core_endpointtest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1540. ✓test_load_registry_rejects_duplicate_idstest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1541. ✓test_logging_and_guardrail_roll_up_into_one_moduletest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1542. ✓test_loki_render_exposes_exact_stdout_lines_for_lokitest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1543. ✓test_orphan_marker_is_reported_not_countedtest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1544. ✓test_prometheus_render_exposes_module_coverage_timeseriestest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1545. ✓test_real_registry_loads_and_ids_are_uniquetest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1546. ✓test_skipped_marker_outside_the_registry_is_still_an_orphantest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1547. ✓test_text_render_uses_plain_coverage_languagetest_collector.py›

    No steps recorded

    View test on GitHub ↗
  1548. ✓test_access_group_scopes_tool_selection TestMcpAccessGroupToolSelectiontest_mcp_access_group_e2e.py›
    1. Read /v1/mcp/server on every proxy replica until it settles
    2. Generate a virtual key with user id: e2e-mcp-ag-granted-23ba4fb82a3d and object permission: mcp access groups: e2e-mcp-grp-2286b569391d
    3. Generate a virtual key with user id: e2e-mcp-ag-other-2d284be0ea3b and object permission: mcp access groups: e2e-mcp-grp-absent-0a165834a95e
    View test on GitHub ↗
  1549. ✓test_alias_grant_persists_verbatim_and_lists_tools TestMcpKeyGrantByAliastest_mcp_key_access_e2e.py›
    1. Read /v1/mcp/server on every proxy replica until it settles
    2. Generate a virtual key with user id: e2e-mcp-allowed-226e3100aa60 and object permission: mcp servers: e2e_dd_mcp_8f41cbe51e1e
    3. Read the key's settings back from /key/info
    View test on GitHub ↗
  1550. ✓test_call_tool_denied_without_permission TestMcpKeyWithoutAccessIsDeniedtest_mcp_key_access_e2e.py›
    1. Read /v1/mcp/server on every proxy replica until it settles
    2. Generate a virtual key with user id: e2e-mcp-allowed-761cf4122cad and object permission: mcp servers: dadc5a2e-5026-4986-9802-54b6caab1396
    3. Generate a virtual key with user id: e2e-mcp-denied-f993b526cebe
    View test on GitHub ↗
  1551. ✓test_content_filter_blocks_banned_keyword_in_tool_args TestMcpToolCallGuardrailtest_mcp_guardrail_e2e.py›
    1. Generate a virtual key with user id: e2e-mcp-guard-f1197982b017 and object permission: mcp servers: 1691736e-5ddd-45e7-953c-2f47ff3c6f56
    View test on GitHub ↗
  1552. ✓test_key_granted_a_toolset_lists_exactly_its_tools TestMcpToolsetEnforcementtest_mcp_toolset_enforcement_e2e.py›
    1. Read /v1/mcp/server on every proxy replica until it settles
    2. Generate a virtual key with user id: e2e-mcp-catalog-2c55533de76b and object permission: mcp servers: 359c5aac-2bbf-4a6a-8d6d-0c7276215106
    3. Create an MCP toolset with the tools server id: 359c5aac-2bbf-4a6a-8d6d-0c7276215106 and tool name: _dd_gff, server id: 359c5aac-2bbf-4a6a-8d6d-0c7276215106 and tool name: _dd_guc
    4. Generate a virtual key with user id: e2e-mcp-toolset-19adabfd11a3 and object permission: mcp servers: 359c5aac-2bbf-4a6a-8d6d-0c7276215106 and mcp toolsets: c1ab480b-d843-4f22-b6a3-79550081bebc
    View test on GitHub ↗
  1553. ✓test_list_tools_denied_without_permission TestMcpKeyWithoutAccessIsDeniedtest_mcp_key_access_e2e.py›
    1. Read /v1/mcp/server on every proxy replica until it settles
    2. Generate a virtual key with user id: e2e-mcp-allowed-8d3e2fe0d6e2 and object permission: mcp servers: 482093f2-7f2d-42e3-8ef0-5df2c663217a
    3. Generate a virtual key with user id: e2e-mcp-denied-a359178144ce
    View test on GitHub ↗
  1554. ✓test_route_restricted_health_matches_server_grants TestMcpHealthVisibilitytest_mcp_key_access_e2e.py›
    1. Read /v1/mcp/server on every proxy replica until it settles
    2. Generate a virtual key with user id: e2e-mcp-allowed-bb59ce8b4185 and object permission: mcp servers: c1bddcac-0ab2-4e67-97c9-4bd5da9aa8fc
    3. Generate a virtual key with user id: e2e-mcp-health-075f911d21df, allowed routes: /v1/mcp/server, /v1/mcp/server/health and object permission: mcp servers: c1bddcac-0ab2-4e67-97c9-4bd5da9aa8fc
    4. Generate a virtual key with user id: e2e-mcp-health-efcdc78e7eb9, allowed routes: /v1/mcp/server, /v1/mcp/server/health and object permission: mcp servers: 98554303-f2d9-4248-be91-8a6b065834de
    5. Generate a virtual key with user id: e2e-mcp-health-4f461804ab80, allowed routes: /v1/mcp/server, /v1/mcp/server/health and object permission: default settings
    View test on GitHub ↗
  1555. ✓test_search_logs_finds_seeded_completion TestDatadogMcpRoundTriptest_mcp_datadog_e2e.py›
    1. Call the management route /health/readiness/details
    2. Read /v1/mcp/server on every proxy replica until it settles
    3. Generate a virtual key with models: claude-haiku-4-5, user id: e2e-dd-mcp-08fc08449595 and object permission: mcp servers: 803c479f-bd9f-489a-8511-b88bcaf70c0f
    4. Send a /chat/completions request to claude-haiku-4-5
    View test on GitHub ↗
  1556. −test_chat_completion_uses_linear_with_authorization_bearer_header TestMcpChatCompletionOauthtest_mcp_chat_completion_oauth_e2e.py›

    No steps recorded

    set E2E_LINEAR_STORAGE_STATE to a Linear session captured via mcp/linear_session_capture.py

    View test on GitHub ↗
  1557. −test_chat_completion_uses_linear_with_x_litellm_api_key_header TestMcpChatCompletionOauthtest_mcp_chat_completion_oauth_e2e.py›

    No steps recorded

    set E2E_LINEAR_STORAGE_STATE to a Linear session captured via mcp/linear_session_capture.py

    View test on GitHub ↗
  1558. ✓test_discovery_card_is_proxy_fronted TestA2AAgentLifecycletest_a2a_agent_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1559. ✓test_malformed_protocol_version_rejected TestA2AAgentLifecycletest_a2a_agent_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1560. ✓test_message_send_runs_completion_bridge TestA2AAgentLifecycletest_a2a_agent_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    2. Wait for the request's spend log in /spend/logs
    View test on GitHub ↗
  1561. ✓test_pinned_v0_3_serves_flat_message_shape TestA2AAgentLifecycletest_a2a_agent_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1562. ✓test_pinned_v1_0_serves_nested_message_shape TestA2AAgentLifecycletest_a2a_agent_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1563. ✓test_real_world_agent_replies_to_property_query TestA2AAgentLifecycletest_a2a_agent_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1564. ✓test_register_persists TestA2AAgentLifecycletest_a2a_agent_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1565. ✓test_semver_protocol_version_registers_and_serves TestA2AAgentLifecycletest_a2a_agent_e2e.py›
    1. Generate a virtual key with user id: e2e-test-user
    View test on GitHub ↗
  1566. ✓test_unsupported_protocol_version_rejected TestA2AAgentLifecycletest_a2a_agent_e2e.py›

    No steps recorded

    View test on GitHub ↗
  1567. −llm_translation/realtime/test_realtime_pipecat_audio_e2e.pyllm_translation/realtime/test_realtime_pipecat_audio_e2e.py›

    No steps recorded

    collection skipped

  1568. ✓test_readers_share_exclusive_waits_and_a_waiting_exclusive_beats_later_readerstest_stack_lock.py›

    No steps recorded

    View test on GitHub ↗