Aimed at explaining the surprisingly good generalization behavior of overparameterized deep networks, recent works have developed a variety of generalization bounds for deep learning, all based on the fundamental learning-theoretic technique of uniform convergence. While it is well-known that many of ...
MAML,即Model-Agnostic Meta-Learning,是一种元学习算法,旨在通过少量训练样本快速适应新任务。MAML的关键思想是通过元学习优化模型参数,使其能够在遇到新任务时经过少量的梯度更新就能达到良好的性能。具体来说,MAML通过以下步骤进行训练: 元训练阶段: 在元训练阶段,MAML从多个任务中采样任务集。 对于每个任务,使用当前...
Deep learning 的 generalization 是目前一个非常火热的问题。众所周知深度学习不同于其他机器学习的模型，...
A. Generalization bounds for deep learning. Preprint at https://arxiv.org/abs/2012.04115 (2020). Cristianini, N., Shawe-Taylor, J., Elisseeff, A. & Kandol A. J. On kernel-target alignment. In Advances in Neural Information Processing Systems Vol. 14 (eds Dietterich, T., Becker, S....
exploring generalization in deep learning 大概想法就是flat minima的pac bayesian bound 证明在nips那篇...
Deep neural networks (DNNs) exhibit an exceptional generalization capability in practice. This work aims to capture the effect of depth and its potential benefit for learning within the paradigm of information-theoretic generalization bounds. We derive two novel hierarchical bounds on the generalization ...
Mao, “Tighter information-theoretic generalization bounds from supersamples,” arXiv preprint arXiv:2302.02432, 2023. [4] J. Negrea, M. Haghifam, G. K. Dziugaite, A. Khisti, and D. M. Roy, “Information-theoretic generalization bounds for sgld via data-dependent estimates,” Advances ...
We derive generalization bounds for learning algorithms based on their robustness: the property that if a testing sample is "similar" to a training sample, then the testing error is close to the training error. This provides a novel approach, different from complexity or stability arguments, to ...
Algorithm-dependent generalization error bounds are central to statistical learning theory. A learning algorithm may use a large hypothesis space, but the limited number of iterations controls its model capacity and generalization error. The impacts of stochastic gradient methods on generalization error fo...
This depends on the way the bounds are derived. However, this is not of major practical importance, since the essence of the theory remains the same. Due to the importance of the VC dimension, efforts have been made to compute it for certain classes of networks. In [Baum 89] it has ...