https://ollama.com/ollam/unichat-llama3-chinese-8b, by xx025 模型微调 可使用以下框架, 对模型进行SFT, LoRA, DPO, PPO等方式的微调 Llama-Factory 70B为llama3官方模板,8B模型对话模板: _register_template(name="llama3-unichat",format_user=StringFormatter(slots=["Human:{{content}}\nAssistant:"])...
"_name_or_path": "/data/Meta-Llama-3-8B", "architectures": [ "LlamaForCausalLM" ], "attention_bias": false, "attention_dropout": 0.0, "bos_token_id": 128000, "eos_token_id": 128001, "hidden_act": "silu", "hidden_size": 4096, "initializer_range": 0.02, "in...
Unichat官网链接地址:https://github.com/UnicomAI/Unichat-llama3-Chinese 网站默认显示语言:英语 Unichat 介绍 Unichat-llama3-Chinese是中国联通AI创新中心发布的首个基于Meta Llama 3模型的中文指令微调模型。该模型经过中文数据训练,实现了高质量的中文问答功能,支持长达28K的上下文输入,并计划发布支持长度64K的版本。
可以huggingface .bin->llama.cpp GGUF->Ollama这个路径来做。 从huggingface_hub下载下来后,使用python3 ../llama.cpp/convert.py [local_dir] --vocab-type bpe --outfile llama3-8b-chinese.gguf进行转换,然后再按照Ollama的文档来就行: $ cat ./Modelfile FROM ./llama3-8b-chinese.gguf # set the...
"model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin", "model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin", "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin", "model.layers.3.self_attn.k_proj.weight"...