2.1 安装vllm较为简单 pip install vllm==0.7.2 pip install vllm==0.7.2 --index-url https://mirrors.aliyun.com/pypi/simple/ 2.2 安装flash-attn踩了比较多的坑 可参考FlashAttention(flash-attn)安装 open-r1的项目要求是cuda12.4【附录1】
pip3 install tensorrt_llm -U --pre --extra-index-url https://pypi.nvidia.com !pip install -r TensorRT-LLM/examples/phi/requirements.txt !pip install flash_attn pytest !curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash !apt-get install git-lfs 然...
python3 -m pip configsetglobal.index-url https://mirrors.cloud.tencent.com/pypi/simple python3 -m pip install --upgrade pip python3 -m pip installtorch==2.6.0 torchvision torchaudio xformers --no-cache python3 -m pip install flash-attn --no-build-isolationcdvllm/requirements python3 -...
!pip3 install tensorrt_llm -U --pre --extra-index-url https://pypi.nvidia.com !pip install -r TensorRT-LLM/examples/phi/requirements.txt !pip install flash_attn pytest !curl -s https://packagecloud.io/install/repositories/github/git-lfs/ | bash !apt-get install git-lfs 1. 2. 3. 4...
!pip3 install tensorrt_llm -U --pre --extra-index-url https://pypi.nvidia.com !pip install -r TensorRT-LLM/examples/phi/requirements.txt !pip install flash_attn pytest !curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash ...
!pip3 install tensorrt_llm -U --pre --extra-index-url https://pypi.nvidia.com !pip install -r TensorRT-LLM/examples/phi/requirements.txt !pip install flash_attn pytest !curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash ...
安装flash-attn(注意可能需要设置代理或手动下载whl文件): bash pip install ninja MAX_JOBS=8 pip install flash-attn --no-build-isolation # 如果失败,尝试从GitHub下载whl文件或使用代理 pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.2/flash_attn-2.5.2+cu122torch2....
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple pip install modelscope==1.11.0 pip install openai==1.17.1 pip install torch==2.1.2+cu121 pip install tqdm==4.64.1 pip install transformers==4.39.3 # 下载flash-attn 请等待大约10分钟左右~ ...
pip install vllm==0.4.0.post1 flash-attn==2.5.7 aohan237 commentedon Apr 23, 2024 aohan237 Ka0Ri commentedon Apr 23, 2024 Ka0Ri This is my setup: # ampere 'dspy-ai', # 'torch==2.1.2+cu118', # 2.2 net supported due to vllm see: https://github.com/vllm-project/vllm/iss...
git clone https://github.com/Dao-AILab/flash-attention cd flash_attn git checkout v2.5.6 python3 setup.py bdist_wheel --dist-dir=dist then compile and install flash_attn. Sign up for freeto join this conversation on GitHub.Already have an account?Sign in to comment ...