Example command: DNNL_MAX_CPU_ISA=AVX512_CORE_BF16 ONEDNN_VERBOSE=1 MKL_ENABLE_INSTRUCTIONS=AVX512_E4 MKL_VERBOSE=1 OMP_NUM_THREADS=8 MKL_NUM_THREADS=8 TORCHINDUCTOR_FREEZING=0 python benchmarks/dynamo/huggingface.py --accuracy --amp -dcpu -n50 --no-skip --dashboard --batch-size 1...
The official example scripts My own modified scripts Tasks An officially supported task in theexamplesfolder (such as GLUE/SQuAD, ...) My own task or dataset (give details below) Reproduction I'm trying to fine tune GPT2 using accelerate + HF training + FSDP with the intention of training...