os.environ['CUDA_VISIBLE_DEVICES'] ='0'importtorchfromswift.llmimport( DatasetName, InferArguments, ModelType, SftArguments, infer_main, sft_main, app_ui_main, merge_lora ) model_type = ModelType.qwen1half_0_5b sft_args = SftArguments( model_type=model_type, train_dataset_sample=2000, ...
class ModelKeys: model_type: str = None module_list: str = None embedding: str = None mlp: str = None down_proj: str = None attention: str = None o_proj: str = None q_proj: str = None k_proj: str = None v_proj: str = None qkv_proj: str = None qk_proj: str = None...
在项目中创建一个inference_train.py文件,写入以下代码: importosos.environ['CUDA_VISIBLE_DEVICES']='0'fromswift.llmimportDatasetName,ModelType,SftArguments,sft_mainsft_args=SftArguments(model_type=ModelType.glm4_9b_chat,dataset=[f'{DatasetName.alpaca_zh}#500',f'{DatasetName.alpaca_en}#500',f'...
--model_type model_base --model_id_or_path /data/train/model-base --sft_type full --tuner_backend peft --template_type AUTO --dtype AUTO --output_dir output --ddp_backend nccl --dataset /data/project/data/QA-chinese/process.jsonl --num_train_epochs 2 --max_length 20...
使用swift infer --model_type llama3_2-11b-vision-instruct进行体验. 2024.09.26: 支持llama3.2系列模型的训练到部署. 使用swift infer --model_type llama3_2-1b-instruct进行体验. 2024.09.25: 支持got-ocr2的训练到部署. 最佳实践可以查看这里. 2024.09.24: 支持llama3_1-8b-omni的训练与部署. 使用...
glm4_9b_chat template_type = get_default_template_type(model_type) model_id_or_path = None model, tokenizer = get_model_tokenizer(model_type, model_id_or_path=model_id_or_path, model_kwargs={'device_map': 'cuda:0'}) model.generation_config.max_new_tokens = 128 model = Swift....
ROG Swift OLED PG39WCDM gaming monitor ― 39-inch (3440 x 1440) curved OLED panel, 240 Hz (above 144Hz), 0.03 ms, G-SYNC® compatible, custom heatsink, uniform brightness, ROG Smart KVM, 90 W Type-C®, ASUS DisplayWidget Center ...
ROG Swift OLED PG39WCDM gaming monitor ― 39-inch (3440 x 1440) curved OLED panel, 240 Hz (above 144Hz), 0.03 ms, G-SYNC® compatible, custom heatsink, uniform brightness, ROG Smart KVM, 90 W Type-C®, ASUS DisplayWidget Center 39-inch curved ultrawide (3440 x 1440) OLED gami...
--model_type minicpm-v-v2_6-chat \ --model_id_or_path OpenBMB/MiniCPM-V-2_6 \ --sft_type lora \ --dataset coco-en-mini#20000 \ --deepspeed default-zero2 如果要使用自定义数据集,只需按以下方式进行指定: --dataset train.jsonl \ ...
skipIf(SKIP_TEST, 'To avoid citest error: OOM') def test_inference_vllm(self): model_type = ModelType.qwen_7b_chat llm_engine = get_vllm_engine(model_type, torch.float16) 0 comments on commit b1e919b Please sign in to comment. ...