最近在使用官网的教程训练chatGLM3,但是出现了“RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn”错误,查阅了官方的文档,目前这个问题还没什么解决方案 但是其中有人回复说:是注释掉503行的model.gradient_checkpointing_enable() 。个人验证确实是可以成功的,那么问题来...
🐛 Describe the bug Hello, when I am using DDP to train a model, I found that using multi-task loss and gradient checkpointing at the same time can lead to gradient synchronization failure between GPUs, which in turn causes the parameters...
i am getting this error as soon as i enable gradient_checkpointing=True CUDA Version: 12.4 torch '2.1.2+cu121'Activity 545999961 commented on Nov 1, 2024 545999961 on Nov 1, 2024 Collaborator You could check if DeepSpeed is being used, as gradient checkpointing needs to be used in conju...
from transformers import AutoModelForCausalLM, TraininArgumentsmodel = AutoModelForCausalLM.from_pretrained( model_id, use_cache=False, # False if gradient_checkpointing=True **default_args)model.gradient_checkpointing_enable()LoRA LoRA是微软团队开发的一种技术，用于加速大型语言模型的微调。他...
We currently have a few issues like #831 and #480 where gradient checkpointing + DDP does not work with the RewardTrainer. Let's use this issue to collect the various training modes we'd like to support and track the status of their fixe...
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024) - TypeError: ChatGLMPreTrainedModel._set_gradient_checkpointing() got an unexpected keyword argument 'enable' 不太懂怎么解决?我尝试了更新transformers还是报这个错 · Issue #1391 · hiyouga/LLaM
unet.enable_gradient_checkpointing() File "I:\Git\AI\SDWebUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1207, in __getattr__ raise AttributeError("'{}' object has no attribute '{}'".format( AttributeError: 'UNet2DConditionModel' object has no attribute 'enable_gradient_...
Security1 Insights New issue Closed Description hayougei Activity Sign up for freeto join this conversation on GitHub.Already have an account?Sign in to comment Assignees No one assigned Labels invalidThis doesn't seem right Projects No projects ...