PLUS-WAVE's Blog

简体中文

English

简体中文

English

外观

3.2 Qwen2.5-VL

约 689 字大约 2 分钟

2025-05-06

3.2.1 部署

  1. Clone & 创建虚拟环境

    git clone https://github.com/QwenLM/Qwen2.5-VL
cd Qwen2.5-VL
mamba create --name Qwen2_5_VL python=3.12
mamba activate Qwen2_5_VL

  2. 安装模型依赖库

    pip install git+https://github.com/huggingface/transformers accelerate
# 根据自己的版本来选择 https://github.com/Dao-AILab/flash-attention/releases
pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.4cxx11abiFALSE-cp312-cp312-linux_x86_64.whl
pip install -r requirements_web_demo.txt

  3. 下载预训练模型

    # 下载预训练权重
git lfs install
git clone https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct

  4. 运行 gradio_demo.py 文件

    # 启动 web 页面
python web_demo_mm.py -c Qwen2.5-VL-3B-Instruct --flash-attn2

    注意更改 --server-port 为 SSH 连接时设定的端口转发，如 ssh -L 8001:localhost:8001 server_name，那就设置 server-port 为 8001

3.2.2 flash_attn undefined symbol

完整报错：

Traceback (most recent call last):
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2011, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 40, in <module>
    from ...modeling_flash_attention_utils import flash_attn_supports_top_left_mask, is_flash_attn_available
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/transformers/modeling_flash_attention_utils.py", line 36, in <module>
    from flash_attn.bert_padding import index_first_axis, pad_input, unpad_input  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/flash_attn/__init__.py", line 3, in <module>
    from flash_attn.flash_attn_interface import (
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/flash_attn/flash_attn_interface.py", line 15, in <module>
    import flash_attn_2_cuda as flash_attn_gpu
ImportError: /mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/flash_attn_2_cuda.cpython-312-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/mnt/data/home/xxxx/matching/Qwen2.5-VL/web_demo_mm.py", line 14, in <module>
    from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration, TextIteratorStreamer
  File "<frozen importlib._bootstrap>", line 1412, in _handle_fromlist
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 1999, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2013, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.qwen2_5_vl.modeling_qwen2_5_vl because of the following error (look up to see its traceback):
/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/flash_attn_2_cuda.cpython-312-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

要使用 flash-attention 官方编译好的直接安装，不要自己编译；根据自己的 python 版本，CUDA 版本，torch 版本，选择 cxx11abiFALSE 的复制下载链接，直接 pip install <链接>

https://github.com/Dao-AILab/flash-attention/releases

3.2.3 gradio 报错 ‘bool’ is not iterable

在运行 gradio_demo.py 启动 Qwen2.5-VL 的 web GUI 时报错：

ERROR:    Exception in ASGI application
Traceback (most recent call last):
...
  File "/mnt/data/home/xxxx/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/gradio_client/utils.py", line 880, in get_type
    if "const" in schema:
       ^^^^^^^^^^^^^^^^^
TypeError: argument of type 'bool' is not iterable

https://github.com/hiyouga/LLaMA-Factory/issues/7132

pydantic 版本导致的问题，退回 2.10.6 版本

pip install pydantic==2.10.6

3.2.4 gradio Check your network or proxy/数据无法传送

完整报错：

Traceback (most recent call last):
  File "/mnt/data/home/wangjiawei/matching/Qwen2.5-VL/web_demo_mm.py", line 297, in <module>
    main()
  File "/mnt/data/home/wangjiawei/matching/Qwen2.5-VL/web_demo_mm.py", line 293, in main
    _launch_demo(args, model, processor)
  File "/mnt/data/home/wangjiawei/matching/Qwen2.5-VL/web_demo_mm.py", line 282, in _launch_demo
    demo.queue().launch(
  File "/mnt/data/home/wangjiawei/miniforge3/envs/Qwen2_5_VL/lib/python3.12/site-packages/gradio/blocks.py", line 2716, in launch
    raise Exception(
Exception: Couldn't start the app because 'http://127.0.0.1:6007/gradio_api/startup-events' failed (code 503). Check your network or proxy settings to ensure localhost is accessible.

可能是因为有 HTTP 和 HTTPS 的代理，需要设置 no_proxy

export no_proxy=127.0.0.1

注意：如果不设置 no_proxy，并且把 share 设置为 True，那么还是可以打开网页的，但是可能会遇见数据无法传送到服务器端的情况（即在 web 上输入后，一直在传输）