"koboldcpp.exe --usecublas mmq rowsplit normal --contextsize 4096 --blasbatchsize 512 --threads 9 --highpriority --model 70B.q2_k.gguf" ? --gpulayers, --tensor_split are not needed in this case? Author candre23 commented Feb 18, 2024 Updated for 1.58. Still 103b, three P40s, ...
koboldcpp_cublas: $(DONOTHING) endif ifdef HIPBLAS_BUILD koboldcpp_hipblas: ggml_v4_cublas.o ggml_v3_cublas.o ggml_v2_cublas.o ggml_v1.o expose.o gpttype_adapter_cublas.o sdcpp_cublas.o whispercpp_default.o llavaclip_cublas.o llava.o ggml-backend_cublas.o $(HIP_OBJS) $(OBJS...
I'm getting this too, for my 6600xt rocBLAS error: Cannot read D:\WinTemp\_MEI96962\/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1032 List of available TensileLibrary Files : Using the new release of koboldcpp_rocm I just installed ROCm HIP SDK for windows...