[Misc] Add qwen2.5-vl BNB support

[Isotr0py]

FIX #12604 (comment)

• Add tp compatible BNB support to Qwen2.5-VL models.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[Isotr0py]

Outputs under TP=2:

WARNING 02-08 07:22:12 config.py:621] bitsandbytes quantization is not fully optimized yet. The speed can be slower than non-quantized models.
INFO 02-08 07:22:12 config.py:1401] Defaulting to use mp for distributed inference
INFO 02-08 07:22:13 llm_engine.py:234] Initializing a V0 LLM engine (v0.1.dev4472+g6e1fc61) with config: model='Qwen/Qwen2.5-VL-3B-Instruct', speculative_config=None, tokenizer='Qwen/Qwen2.5-VL-3B-Instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=LoadFormat.BITSANDBYTES, tensor_parallel_size=2, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=bitsandbytes, enforce_eager=False, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=Qwen/Qwen2.5-VL-3B-Instruct, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=False, chunked_prefill_enabled=False, use_async_output_proc=True, disable_mm_preprocessor_cache=False, mm_processor_kwargs={'min_pixels': 784, 'max_pixels': 1003520, 'fps': 1}, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[8,4,2,1],"max_capture_size":8}, use_cached_outputs=False, 
WARNING 02-08 07:22:14 multiproc_worker_utils.py:300] Reducing Torch parallelism from 2 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.

...

Processed prompts:   0%|                                                           | 0/4 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s/kaggle/working/vllm/vllm/model_executor/models/qwen2_5_vl.py:938: UserWarning: Specified kernel cache directory could not be created! This disables kernel caching. Specified directory is /root/.cache/torch/kernels. This warning will appear only once per process. (Triggered internally at ../aten/src/ATen/native/cuda/jit_utils.cpp:1442.)
  sizes = grid_thw.prod(-1) // merge_size // merge_size
Processed prompts: 100%|█████████████████████████████████████████████████| 4/4 [00:26<00:00,  6.57s/it, est. speed input: 194.25 toks/s, output: 9.74 toks/s]
The image depicts the Tokyo Skytree, a tall broadcasting tower located in the Sumida Ward of Tokyo, Japan. The photo is taken from a low angle, looking up at the tower through a canopy of cherry blossoms. The cherry blossoms are in full bloom, creating a beautiful contrast with the clear blue sky and
The image depicts the Tokyo Skytree, a tall broadcasting tower located in Sumida, Tokyo, Japan. The photo is taken from a low angle, looking up towards the tower, framed by cherry blossoms in full bloom. The cherry blossoms are in full bloom, creating a beautiful contrast with the clear blue sky and
The image depicts the Tokyo Skytree, a tall broadcasting tower located in Sumida, Tokyo, Japan. The photo is taken from a low angle, looking up towards the tower, framed by cherry blossoms in full bloom. The cherry blossoms are in the foreground, creating a beautiful contrast with the blue sky and the
The image depicts the Tokyo Skytree, a tall broadcasting tower located in Sumida, Tokyo, Japan. The photo is taken from a low angle, looking up at the tower through a canopy of cherry blossoms. The cherry blossoms are in full bloom, with pink flowers covering the branches of the trees. The sky
INFO 02-08 07:23:21 multiproc_worker_utils.py:141] Terminating local vLLM worker processes

[jeejeelee]

Thanks for fixing, LGTM

[MotorBottle]

Tested working.