and I run nvcc -version,can find cuda is cuda11.8,and I have installed ninja&packaging&setuptools&torch2.0.0&wheel I run it on jetson orin 64G,it's aarch64 dongkuang changed the title install vllm,but show this error,please help me,thank you! install vllm,but show CUDA runtime is ...
self.distribution.run_command(command) File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools\dist.py", line 989, in run_command super().run_command(command) File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-pac...
RuntimeError: Error in model execution (input dumped to /tmp/err_execute_model_input_20241211-200011.pkl): CUDA error: no kernel image is available for execution on the device 这个问题去stackoverflow[2]了一下,大概率是vllm编译的时候没有支持对应的显卡架构,还记得重点么,没错,大概率就是不支持...
RuntimeError: Error in model execution (input dumped to /tmp/err_execute_model_input_20241211-200011.pkl): CUDA error: no kernel image is available for execution on the device 这个问题去stackoverflow[2]了一下,大概率是vllm编译的时候没有支持对应的显卡架构,还记得重点么,没错,大概率就是不支持...
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' Traceback (most recent call last): ... torch._C._cuda_init() RuntimeError: The NVIDIA driver on your system is too old (found version 11080). Please update your GPU driver by downloading and installing a new version from...
当tensor_parallel_size=2被使用时，输出结果为：
不确定是什么造成了上面的混乱。抱歉。
当tensor_parallel_size=2被使用时，输出结果为：
不确定是什么造成了上面的混乱。抱歉。
"so vLLM may not be able to run correctly", sys.platform) VLLM_TARGET_DEVICE = "empty" MAIN_CUDA_VERSION = "12.1" def is_sccache_available() -> bool: return which("sccache") is not None def is_ccache_available() -> bool: ...