Environment: Windows 11 (venv) C:\sd\HunyuanDiT>pip install deepspeed==0.6.3 Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Collecting deepspeed==0.6.3 Downloading deepspeed-0.6.3.tar.gz (554 kB) ━━━..
File "C:\Users\i\AppData\Local\Temp\pip-install-97anxpmj\deepspeed_629338d4deb54654aba44efd0bf8dab4\setup.py", line 156, in <module> abort(f"Unable to pre-compile {op_name}") File "C:\Users\i\AppData\Local\Temp\pip-install-97anxpmj\deepspeed_629338d4deb54654aba44efd0bf8dab4\set...
我们使用这个脚本DeepSpeedExamples/Megatron-LM/scripts/pretrain_gpt2_model_parallel.sh来进行2卡的模型并行训练,除了2卡数据并行相关的修改之外我们还需要去掉这个脚本里面的--deepspeed参数,因为要使用上DeepSpeed还需要执行deepspeed的config配置文件。和deepspeed相关的训练特性,我们留到下一篇文章中探索。 使用bash scri...
本文记录了博主遇到问题“ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.”的解决方案。更新于2019.03.12。 背景: 博主需要安装一个程序,该程序用到了PyTorch中的一个包(torch.utils.ffi),但是博主安装的PyTorch版本是1.0.1,也就因此导致了上面的错误。 原因: 在PyTorch 1.0.1中...
deepspeed 0.8.3 dill 0.3.7 einops 0.7.0 exceptiongroup 1.2.0 fastapi 0.78.0 filelock 3.13.1 flash-attn 2.4.2 frozenlist 1.4.1 fsspec 2023.10.0 gitdb 4.0.11 GitPython 3.1.41 h11 0.14.0 hjson 3.1.0 huggingface-hub 0.20.2 idna 3.6 ...
[WARNING] unable to import torch, please install it if you want to pre-compile any deepspeed ops. DS_BUILD_OPS=1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed ...
!pip install transformers[deepspeed] I had to include transformers[deepspeed] yesterday, and earlier today I had to cherrypick commits to make things work.. Development is going so fast, hard to keep up with every change 😅 😄 1 Contributor younesbelkada commented Jun 28, 2023 Hi @DJ...