no resolution:cublasLt runs into an error on 8 bit quantizedLightning-AI/lit-llama#315 low vramException: cublasLt ran into an error!deep-diver/LLM-As-Chatbot#16 no resolution:Error while loading a model on 8bithuggingface/transformers#21371 ...
Exception: cublasLt ran into an error! I am running on Ubuntu 22.04 with a 8GB GPU The error as follows when giving the prompt: cuBLAS API failed with status 15 A: torch.Size([24, 4096]), B: torch.Size([4096, 4096]), C: (24, 4096); (lda, ldb, ldc): (c_int(768), c_in...
But when I ran the setup.py again I met this error: OSError: /home/anhay20171/anaconda3/envs/alphapose_0/lib/python3.7/site-packages/torch/lib/../../nvidia/cublas/lib/libcublas.so.11: undefined symbol: cublasLtGetStatusString, version libcublasLt.so.11 During handling of the above ...