flash_attn安装 1. cuda-nvcc安装 /nvidia/cuda-nvcc 2. torch安装 # / # 找到对应cuda版本的torch进行安装 pip3 install torch torchvision torchaudio --index-url /whl/cu121 3. flash_attn安装 访问该网站,找到对应torch、python、cuda版本的flash_attn进行下载,并上传到服务器 /Dao-AILab/flash-attention...
RuntimeError: FlashAttention is only supported onCUDA11.6 and above. Note: make sure nvcc has a supported version by running nvcc -V. torch.__version__ = 2.1.2+cu121 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata...
linux transformers 4.39.0 tokenizers 0.15.2 torch 2.1.2+cu121 flash-attn 2.3.3 在使用vllm运行xverse/XVERSE-13B-256K时(代码如下): qwen_model = AutoModelForSequenceClassification.from_pretrained( args.pre_train, trust_remote_code=True, attn_implementation="flash_attention_2", torch_dtype=torc...
│ exit code: 1 ╰─> [9 lines of output] fatal: not a git repository (or any of the parent directories): .git torch.__version__ = 2.1.2+cu121 running bdist_wheel Guessing wheel URL: https://github.com/Dao-AILab/flash-attention/releases/download/v2.4.2/flash_attn-2.4.2+cu122to...
torch.__version__ = 2.4.1+cu121 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed× Encountered error while generating package metadata. ╰─> See above for output.note: This is an issue with the ...
估计也没有谁会需要。放在这里纯当是自己也备份一下,以后万一需要重装也不必重新编译了。 python:3.11.6 cuda:12.6 torch:2.4.0+cu121 flash_attn:2.6.3 xformers:0.0.27.post2 https://pan.baidu.com/s/1XTWx060Ded8blUU5lsOoNw vz9f
最后解决: 先卸载原本的torch: pip uninstall torch torchvision torchaudio 然后安装12.1的: pip install torch torchvision torchaudio -f https://download.pytorch.org/whl/cu121/torch_stable.html 最后加载成功codellama 本站仅提供存储服务,所有内容均由用户发布,如发现有害或侵权内容,请点击举报。打开...
估计也没有谁会需要。放在这里纯当是自己也备份一下,以后万一需要重装也不必重新编译了。 python:3.11.6 cuda:12.6 torch:2.4.0+cu121 flash_attn:2.6.3 xformers:0.0.27.post2 https://pan.baidu.com/s/1XTWx060Ded8blUU5lsOoNw vz9f
Build cuda_12.1.r12.1/compiler.32415258_0 最后解决: 先卸载原本的torch: pip uninstall torch torchvision torchaudio 然后安装12.1的: pip install torch torchvision torchaudio -f https://download.pytorch.org/whl/cu121/torch_stable.html 最后加载成功codellama...
Your current environment PyTorch version: 2.3.0+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A OS: Ubuntu 20.04.5 LTS (x86_64) GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 Clang version...