In your case, the tokenizer need not be saved as it you have not changed the tokenizer or added new tokens. Huggingface tokenizer provides an option of adding new tokens or redefining the special tokens such as [MASK], [CLS], etc. If you do such modifications, then you may have to ...
31 Huggingface saving tokenizer 29 Saving and reload huggingface fine-tuned transformer 8 is there a way to save only the model with huggingface trainer? 2 key dataset lost during training using the Hugging Face Trainer 27 saving finetuned model locally 1 Why there are no logs and which...
Huggingface_hub version: 0.16.4 Safetensors version: 0.3.1 PyTorch version (GPU?): 2.0.1+cu117 (True) Tensorflow version (GPU?): not installed (NA) Flax version (CPU?/GPU?/TPU?): not installed (NA) Jax version: not installed
Support saving models trained with DeepSpeed in Trainer callbacks huggingface/transformers#31338 Open ferrazzipietro mentioned this issue Jun 18, 2024 [BUG] AttributeError: 'Accelerator' object has no attribute 'deepspeed_config' microsoft/DeepSpeed#4143 Closed Sign...
to 1.xx.# Now, either I follow the HugginFace docks:model.save_pretrained("./huggingface_model")# -> saves /notebooks/huggingface_model/adapter_model.bin 16mb.# or an alternative I found on SO:trainer.save_model("./torch_model")# -> saves /notebooks/torch_model/pytorch_model.bin ...
By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy. Not the answer you're looking for? Browse other questions tagged python nlp google-colaboratory bert-language-model huggingface-transformers or ask your ow...
Support saving models trained with DeepSpeed in Trainer callbackshuggingface/transformers#31338 Open ferrazzipietromentioned this issueJun 18, 2024 [BUG] AttributeError: 'Accelerator' object has no attribute 'deepspeed_config'microsoft/DeepSpeed#4143 ...
By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy. Not the answer you're looking for? Browse other questions tagged python nlp google-colaboratory bert-language-model huggingface-transformers or ask your own questi...
By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy. Not the answer you're looking for? Browse other questions tagged python nlp google-colaboratory bert-language-model huggingface-transformers or ask your ow...