quack.py vllm_arctic_480b.py vllm_aya_8b.py vllm_codeqwen_110b_v1_5.py vllm_deepseek_coder_33b.py vllm_duckdb_nsql_7b.py vllm_llama3_70b.py vllm_llama3_8b.py vllm_seallm_7b_v2_5.py vllm_sqlcoder_7b_2.pyBreadcrumbs llm-hosting / vllm_llama3_70b.py Latest...
2024.04.19: Support for inference, fine-tuning, and deployment of Llama3 series models. This includes: Llama-3-8B, Llama-3-8B-Instruct, Llama-3-70B, and Llama-3-70B-Instruct. use this script to train. 2024.04.18: Supported models: wizardlm2-7b-awq, wizardlm2-8x22b, yi-6b-chat-aw...
It is built on the excellent work of llama.cpp, bitsandbytes, qlora, gptq, AutoGPTQ, awq, AutoAWQ, vLLM, llama-cpp-python, gptq_for_llama, chatglm.cpp, redpajama.cpp, gptneox.cpp, bloomz.cpp, etc. Latest update 🔥 [2024/03] LangChain added support for bigdl-llm; see the...
Llama 3 8B/70B llama3 LLaVA-1.5 7B/13B vicuna Mistral/Mixtral 7B/8x7B/8x22B mistral OLMo 1B/7B - PaliGemma 3B gemma Phi-1.5/Phi-2 1.3B/2.7B - Phi-3 4B/7B/14B phi Qwen/Qwen1.5/Qwen2 (Code/MoE) 0.5B/1.5B/4B/7B/14B/32B/72B/110B qwen StarCoder 2 3B/7B/15B - XVERSE 7B...
2024.04.19: Support for inference, fine-tuning, and deployment of Llama3 series models. This includes: Llama-3-8B, Llama-3-8B-Instruct, Llama-3-70B, and Llama-3-70B-Instruct. use this script to train. 2024.04.18: Supported models: wizardlm2-7b-awq, wizardlm2-8x22b, yi-6b-chat-aw...
2024.04.19: Support for inference, fine-tuning, and deployment of Llama3 series models. This includes: Llama-3-8B, Llama-3-8B-Instruct, Llama-3-70B, and Llama-3-70B-Instruct. use this script to train. 2024.04.18: Supported models: wizardlm2-7b-awq, wizardlm2-8x22b, yi-6b-chat-aw...
2024.04.19: Support for inference, fine-tuning, and deployment of Llama3 series models. This includes: Llama-3-8B, Llama-3-8B-Instruct, Llama-3-70B, and Llama-3-70B-Instruct. use this script to train. 2024.04.18: Supported models: wizardlm2-7b-awq, wizardlm2-8x22b, yi-6b-chat-aw...
2024.04.19: Support for inference, fine-tuning, and deployment of Llama3 series models. This includes: Llama-3-8B, Llama-3-8B-Instruct, Llama-3-70B, and Llama-3-70B-Instruct. use this script to train. 2024.04.18: Supported models: wizardlm2-7b-awq, wizardlm2-8x22b, yi-6b-chat-aw...
2024.04.19: Support for inference, fine-tuning, and deployment of Llama3 series models. This includes: Llama-3-8B, Llama-3-8B-Instruct, Llama-3-70B, and Llama-3-70B-Instruct. use this script to train. 2024.04.18: Supported models: wizardlm2-7b-awq, wizardlm2-8x22b, yi-6b-chat-aw...
outlines_llama3_8b.py quack.py vllm_arctic_480b.py vllm_aya_8b.py vllm_codeqwen_110b_v1_5.py vllm_deepseek_coder_33b.py vllm_duckdb_nsql_7b.py vllm_llama3_70b.py vllm_llama3_8b.py vllm_seallm_7b_v2_5.py vllm_sqlcoder_7b_2.pyBreadcrumbs llm-hosting/...