I guess we are using the system NCCL installation to be able to pip install nvidia-nccl-cu12 during the runtime. If we would use the third_party/nccl module I assume we would link NCCL into the PyTorch binaries. If so, we should make sure to update the install_cuda.sh NCCL version ...
CUDA_ROOT=${ENV_HOME}/dep/cuda-9.0-cudnn7.6.5 MPI_ROOT=${ENV_HOME}/dep/openmpi-4.0.3-cuda9.0-ucx1.7.0 UCX_ROOT=${ENV_HOME}/dep/ucx-1.7.0 NCCL_ROOT=${ENV_HOME}/dep/nccl_2.5.6-1-cuda9.0 export CUDA_HOME=${CUDA_ROOT} export MPI_ROOT=${MPI_ROOT} export NCCL_ROOT=${NCCL...
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch) Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) Collecting nvidia-nccl-cu12==2.20.5 (from torch) Downloading nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl.metadata (1.8 kB...
多卡的话,安装 NCCLhttps://developer.nvidia.com/nccl/nccl-download 配置pip 源vim ~/.pip/pip.conf 代码语言:javascript 复制 [global]index-url=https://pypi.tuna.tsinghua.edu.cn/simple 安装virtualenv包pip install virtualenv 创建虚拟环境virtualenv yourenvname,激活source yourenvname/bin/activate 安装pad...
specify the location where TensorRT is installed. [Default is /usr/lib/x86_64-linux-gnu]:/home/hp/bin/TensorRT-5.0.2.6-cuda10.0-cudnn7.3/targets/x86_64-linux-gnu Please specify the locally installed NCCL version you want to use. [Default is to use https://github.com/nvidia/nccl]: ...
nvidia-curand-cu12 10.3.2.106 pypi_0 pypi nvidia-cusolver-cu12 11.4.5.107 pypi_0 pypi nvidia-cusparse-cu12 12.1.0.106 pypi_0 pypi nvidia-nccl-cu12 2.18.1 pypi_0 pypi nvidia-nvjitlink-cu12 12.3.101 pypi_0 pypi nvidia-nvtx-cu12 12.1.105 pypi_0 pypi ...
Using cached nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB) Collecting nvidia-nccl-cu11==2.14.3 (from torch>=2.0.0->vllm) Using cached nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB) ...
Downloading nvidia_cusparse_cu11-11.7.5.86-py3-none-manylinux1_x86_64.whl (204.1 MB) ━━━ 204.1/204.1 MB 10.4 MB/s eta 0:00:00 Collecting nvidia-nccl-cu11==2.16.5 (from tensorflow[and-cuda]) Downloading nvidia_nccl_cu11-2.16.5-py3-none-manylinux1_x86_64.whl (210.3 MB) ...
CUDNN_INSTALL_PATH=/usr/local/cuda-10.0 --action_env TF_CUDNN_VERSION=7 --action_env TF_NCCL_VERSION=1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=6.1 --action_env LD_LIBRARY_PATH=:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64 --action_env TF_CUDA_CLANG=0 --action_env...
#Pull the Docker image with CUDA 11.8.docker run --gpus all -it --rm --shm-size=8g nvcr.io/nvidia/pytorch:22.12-py3 Inside the docker, run: pip uninstall torch pip install vllm bashirsouidcommentedJun 22, 2023 Oh, silly me, I missed seeing in the docs that CUDA 12 wasn't support...