MPT support in llama.cpp by jploski · Pull Request #3417...
ggml_init_cublas: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6 llama_model_loader: loaded meta data with 19 key-value pairs and 195 tensors from E:\hf\mosaicml-mpt-7b-chat-gguf