Use PEFT or Full-parameter to finetune 400+ LLMs (Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, ...) or 150+ MLLMs (Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2.5, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSe
Use PEFT or Full-parameter to finetune 400+ LLMs (Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, ...) or 150+ MLLMs (Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2.5, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSe
Hi, firstly again thank you so much for this very fast and memory-friendly finetune library! Below I will share my thoughts about full finetuning. Firstly, I made some experiments to test speed and memory consumption between Unsloth and vanilla HuggingFace transformers Trainer for full finetune ...
I am working on fine-tuning LLMs (6B to 40B parameters) using the LoRA framework on an instruction tuning dataset comprising of instructions corresponding to ~20 tasks (a mix of factual as well as open-ended tasks). The input to the model consists of a conversation snippet between two ...
Use PEFT or Full-parameter to finetune 400+ LLMs (Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, ...) or 150+ MLLMs (Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2.5, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSe
🔥2024.05.20: Support for inferencing and fine-tuning cogvlm2-llama3-chinese-chat-19B, cogvlm2-llama3-chat-19B. you can refer to cogvlm2 Best Practice. 🔥2024.05.17: Support peft=0.11.0. Meanwhile support 3 new tuners: BOFT, Vera and Pissa. use --sft_type boft/vera to use...
🔥2024.05.20: Support for inferencing and fine-tuning cogvlm2-llama3-chinese-chat-19B, cogvlm2-llama3-chat-19B. you can refer to cogvlm2 Best Practice. 🔥2024.05.17: Support peft=0.11.0. Meanwhile support 3 new tuners: BOFT, Vera and Pissa. use --sft_type boft/vera to use...
🔥2024.05.20: Support for inferencing and fine-tuning cogvlm2-llama3-chinese-chat-19B, cogvlm2-llama3-chat-19B. you can refer to cogvlm2 Best Practice. 🔥2024.05.17: Support peft=0.11.0. Meanwhile support 3 new tuners: BOFT, Vera and Pissa. use --sft_type boft/vera to use...
🔥2024.03.12: Support inference and fine-tuning for deepseek-vl series. Best practices can be found here. 🔥2024.03.11: Support GaLore for effectively reducing memory usage to 1/2 of the original in full-parameter training. 🔥2024.03.10: End-to-end best practices from fine-tuning to ...
🔥2024.05.20: Support for inferencing and fine-tuning cogvlm2-llama3-chinese-chat-19B, cogvlm2-llama3-chat-19B. you can refer to cogvlm2 Best Practice. 🔥2024.05.17: Support peft=0.11.0. Meanwhile support 3 new tuners: BOFT, Vera and Pissa. use --sft_type boft/vera to use...