In this paper, we observe that local minima of modern deep networks are more than being flat or sharp. Instead, at a local minimum there exist many asymmetric directions such that the loss increases abruptly along one side, and slowly along the opposite side - we formally define such minima...
Asymmetric Valleys: Beyond Sharp and Flat Local Minima | Semantic Scholar 虽然看起来有点道理,但友情提示现在DL的理论解释目前还没有共识的,不要太过相信。 2022年更新评论:该观点已经广为人知,但已经有…
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective Stanford University code Differentially Private Federated Learning on Heterogeneous Data Stanford University code Towards Understanding Biased Client Selection in Federated Learning Carnegie Mellon University code FLIX: A Simple and Commu...
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective Stanford University code Differentially Private Federated Learning on Heterogeneous Data Stanford University code Towards Understanding Biased Client Selection in Federated Learning Carnegie Mellon University code FLIX: A Simple and Commu...