...apex.optimizers.FusedAdam` to replace `torch.optim.AdamW...
🚀 The feature, motivation and pitch After running several benchmarks 1 and 2 it appears that apex.optimizers.FusedAdam is 10-15% faster than torch.optim.AdamW (in an ensemble of the HF Trainer loop). I'm proposing to replace torch.optim...