This repository has been archived by the owner on Jun 3, 2024. It is now read-only. vllm-project/vllm-ncclPublic archive NotificationsYou must be signed in to change notification settings Fork2 Star17 Code Issues2 Pull requests Actions ...
Repository files navigation README Apache-2.0 license NOTE: This repo is deprecated with this fix to the main vLLM repo. vllm-nccl Manages vllm-nccl dependency Define package_name, nccl_version, vllm_nccl_verion run python setup.py sdist run twine upload dist/*About...
assert nccl_version == "2.18.1", f"only support nccl 2.18.1, got {version}"url = f"https://storage.googleapis.com/vllm-public-assets/nccl/{cuda_name}/libnccl.so.{nccl_version}" url = f"https://github.com/vllm-project/vllm-nccl/releases/download/v0.1.0/{cuda_name}-libnccl.so...
nccl_major_version, cuda_major_version = os.environ["VLLM_INSTALL_NCCL"].split("+")assert nccl_major_version in ["2.20", "2.18", "2.17", "2.16"], f"Unsupported nccl major version: {nccl_major_version}"assert cuda_major_version in ["11", "12"], f"Unsupported cuda major version:...
# the `vllm_nccl` package must be installed from source distribution # pip is too smart to store a wheel in the cache, and other CI jobs # will directly use the wheel from the cache, which is not what we want. # we need to remove it manually RUN --mount=type=cache,target=/root...
only nvidia-nccl is necessary pip list | grep nccl to check if you have two versions, you should remove the unnecessary one only nvidia-nccl is necessary what's vllm-nccl-cu12 for? wjj19950828mentioned this issueJun 13, 2024 Sign up for freeto join this conversation on GitHub. Already ...
[pip3] vllm-nccl-cu11==2.18.1.0.4.0 [conda] Could not collectROCM Version: Could not collect Neuron SDK Version: N/A vLLM Version: 0.4.1 vLLM Build Flags: CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled GPU Topology:
NCCL version 2.20.5+cuda11.0 INFO 08-06 18:38:36 custom_all_reduce_utils.py:232] reading GPU P2P access cache from /home/wjc/.cache/vllm/gpu_p2p_access_cache_for_0,1.json (VllmWorkerProcess pid=3615391) INFO 08-06 18:38:36 custom_all_reduce_utils.py:232] reading GPU P2P access...
Your current environment vllm 0.4.0.post1 docker image how ran: docker run -d \ --runtime=nvidia \ --gpus '"device=0,1"' \ --shm-size=10.24gb \ -p 5002:5002 \ -e NCCL_IGNORE_DISABLED_P2P=1 \ -v /etc/passwd:/etc/passwd:ro \ -v /etc/group:...
the version of nccl in environment is "nvidia-nccl-cu11 2.20.5" 3、 when deploy LLM model by vllm, there have Errors as follows: 2024-06-18 22:03:51 | INFO | stdout | (RayWorkerWrapper pid=1043334) ERROR 06-18 22:03:50 worker_base.py:148] File "/opt/anaconda3/envs/vllm4/...