The second proposes as-if theories: Expected utility theory and Bayesian statistics are turned into theories of mind, describing an optimal solution of a problem but not its psychological process. The third studies the adaptive toolbox (formal models of heuristics) that describes mental processes in...
It's supposed to help with vanishing and exploding gradients.It's a common problem during training: sometimes the gradients of the parameters turn out too big or too small. Changing them on the training iteration either has very little effect on the loss function (and so the model converges ...