To have your issue addressed in a timely and efficient manner please help us understand and reproduce it better by providing the following:
- Exact command involving one of the scripts under the benchmarks folder
- Any any non-default environment variables
- The results of the benchmark before and after the regression
- A vllm docker image where the original performance is observed
- A vllm docker image where the decreased performance is observed
- Any additional info
- Exact command leading to the crash
- Any any non-default environment variables
- Full vLLM run logs (plain text, no screenshots)
- A vllm docker image where the issue occurs
- Any additional info