Describe the bug When the deepspeed.zero.Init wraps not only the model but also the deepspeed.initialize call, a RecursionError is raised. This happens in deepspeed 0.6.5 but NOT in 0.6.4. It blocks the integration with Lightning Lite wh...
Doing so will maketests/deepspeed/test_deepspeed.py::TestDeepSpeedWithLauncher::test_basic_distributed_zero3_fp16fail, with the same error as stated. Please try running with:CUDA_VISIBLE_DEVICES="0,1" RUN_SLOW="yes" ACCELERATE_USE_DEEPSPEED="yes" pytest -sv tests/deepspeed/test_deepspeed.py...
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library. - [ZeRO-3] Partitioned init with `deepspeed.zero.Init()` (#1190) · son-koku/gpt-neox@7267a74
[rank2]: raise ValueError("Please use scripts/pissa_init.py to initialize PiSSA in DeepSpeed ZeRO-3.") [rank2]: ValueError: Please use scripts/pissa_init.py to initialize PiSSA in DeepSpeed ZeRO-3. [rank1]: Traceback (most recent call last): [rank1]: File "/root/autodl-tmp/LLaMA-F...
--lora_dropout 0.1 --lora_target all --pissa_init True --pissa_convert True --deepspeed /app/LLaMA-Factory/examples/deepspeed/ds_z3_config_jc.json Expected behavior 是需要采用scripts/pissa_init.py将模型转换之后再执行上述操作吗? Others No response...