Downloading vllm_nccl_cu12-2.18.1.0.4.0.tar.gz (6.2 kB) Preparing metadata (setup.py) ... done 此处略去 ... Successfully installed diskcache-5.6.3 dnspython-2.6.1 email_validator-2.1.1 fastapi-0.111.0 fastapi-cli-0.0.3 h11-0.14.0 httpcore-1.0.5 httptools-0.6.1 httpx-0.27.0 inte...
nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cud...
after i delete the /root/.config/vllm/nccl/cu12/libnccl.so.2.18.1, collect_env.py can run. rm -rf /root/.config/vllm/nccl/cu12/libnccl.so.2.18.1 vllm git:(main) vllm git:(main) python3 collect_env.py Collecting environment information... INFO 04-15 11:02:25 pynccl.py:58...
nvidia-cuda-cupti-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cuda-runtime-cu11, nvidia-cudnn-cu11, nvidia-cufft-cu11, nvidia-curand-cu11, nvidia-cusolver-cu11, nvidia-cusparse-cu11, nvidia-nccl-cu11, nvidia-nvtx-cu11, sympy, triton, typing-extensions ...
卸载torch pip uninstall torch torchvision 重装老版本 pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118 如果运行vllm推理 or 启动openai.api_server时报错 NCCL Error 则在启动命令加上--enforce-eage(作为服务启动) 或者在LLM类实例化时增加 enforce_eage=True (代码推理) ...
nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 8.9.2.26 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu12 2.18.1 ...
PyTorch version: 2.1.2+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A OS: Ubuntu 22.04.4 LTS (x86_64) GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Clang version: Could not collect CMake version: version 3.29.0 Libc version: ...
nccl-cu12 2.20.5 nvidia-nvjitlink-cu12 12.6.20 nvidia-nvtx-cu12 12.1.105 omegaconf 2.3.0 onnxruntime 1.16.0 openai 1.39.0 openai-whisper 20230306 opencv-contrib-python 4.10.0.84 opencv-python 4.10.0.84 optimum 1.21.3 optuna 2.10.1 orjson 3.10.7 oss2 2.18.6 outlines 0.0.46 overrides ...