I also tried settinguse_cache=Falseinmodel.generate(), as hinted inhttps://huggingface.co/OpenAssistant/falcon-40b-sft-top1-560/discussions/1, but still gibberish output. I also tried running this withtorch==2.0.1, but same error behavior. ...
balanced 8bit memory Verified 7c0dfec HuggingFaceDocBuilderDev commented Mar 24, 2023 • edited The documentation is not available anymore as the PR was closed or merged. KohakuBlueleaf commented Mar 26, 2023 I'm not sure if this is caused by your work but I met this problem: I'm ...
pip install "peft==0.2.0" !pip install "transformers==4.27.2" "datasets==2.9.0" "accelerate==0.17.1" "evaluate==0.4.0" "bitsandbytes==0.37.1" loralib --upgrade --quiet or !pip install -q -U bitsandbytes !pip install -q -U git+https://github.com/huggingfac...
I have followed the code given in the huggingface docs: device_map = { "transformer.word_embeddings": 0, "transformer.word_embeddings_layernorm": 0, "lm_head": "cpu", "transformer.h": 0, "transformer.ln_f": 0, } quantization_config = BitsAndBytesConfig(llm_int8_enable_fp32_cpu_o...
huggingface/transformersPublic NotificationsYou must be signed in to change notification settings Fork25.8k Star130k New issue Load T5 model in 8 bit fails#25443 1 of 4 tasks PansaLegrandopened this issueAug 10, 2023· 4 comments System Info ...
System Info transformers version: 4.28.0.dev0 Platform: Linux-4.18.0-305.65.1.el8_4.x86_64-x86_64-with-glibc2.28 Python version: 3.10.4 Huggingface_hub version: 0.13.3 PyTorch version (GPU?): 2.0.0+cu117 (True) Tensorflow version (GPU?):...
deepspeed @ https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/deepspeed-0.11.2_cuda121-cp310-cp310-win_amd64.whl defusedxml==0.7.1 dill==0.3.7 einops==0.7.0 en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3...
Platform: Windows-10-10.0.22621-SP0 Python version: 3.10.8 Huggingface_hub version: 0.12.1 PyTorch version (GPU?): 1.13.1+cu117 (True) Tensorflow version (GPU?): not installed (NA) Flax version (CPU?/GPU?/TPU?): not installed (NA) ...
deepspeed @ https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/deepspeed-0.11.2_cuda121-cp310-cp310-win_amd64.whl defusedxml==0.7.1 dill==0.3.7 einops==0.7.0 en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3...