Unified Efficient Fine-Tuning of 100+ LLMs (ACL 2024) - LLaMA-Factory/examples/deepspeed/ds_z3_config.json at b29d5560f1359a3868d917048aeba1a069ba12a9 · hiyouga/LLaMA-Factory
deepspeed: examples/deepspeed/ds_z3_config.json ### dataset dataset: mllm_demo,identity template: qwen2_vl cutoff_len: 2048 max_samples: 1000 overwrite_cache: true preprocessing_num_workers: 16 ### output output_dir: saves/qwen2_vl-7b/full/sft logging_steps: 10 save_steps: 500 plot_lo...
Add support for datasets in local json files (#597) Jun 17, 2023 .pre-commit-config.yaml DeepSpeed Chat Release (#264) Apr 12, 2023 CODEOWNERS Cleanup CODEOWNERS (#953) Jan 25, 2025 CODE_OF_CONDUCT.md Initial CODE_OF_CONDUCT.md commit ...
33 changes: 33 additions & 0 deletions 33 examples/lora_multi_gpu/ds_zero3.sh Original file line numberDiff line numberDiff line change @@ -0,0 +1,33 @@ #!/bin/bash deepspeed --num_gpus 4 ../../src/train_bash.py \ --deepspeed ../deepspeed/ds_z3_config.json \ --stage sft...
deepspeed --num_gpus 4 ../../src/train_bash.py \ --deepspeed ds_z3_config.json \ --stage sft \ --do_train \ --model_name_or_path meta-llama/Llama-2-7b-hf \ --dataset alpaca_gpt4_en \ --dataset_dir ../../data \ --template default \ --finetuning_type full \ --output...