We consider finite-difference gradient approximation based on normally distributed random Gaussian vectors and prove that gradient descent scheme based on this approximation converges to the stationary point of the smoothed function. We also consider convergence to the stationary point of the original (...
SOLNP+ is an advanced derivative-free solver designed for constrained nonlinear optimization problems. Building on the original SOLNP algorithm proposed in 1989, SOLNP+ incorporates innovative techniques such as finite difference gradient approximation, augmented Lagrangian, and Sequential Quadratic Programming ...
Following a newly introduced algorithm framework for zeroth-order stochastic approximation methods, we first propose algorithms {\\bf CG-ZOSA} and {\\bf RG-ZOSA} for smooth DR-submodular optimization based on the coordinate-wise gradient estimator and the randomized gradient estimator, respectively. ...
We discuss ways to obtain analytical gradients within the scalar zeroth-order regular approximation (ZORA) to the Dirac–Fock equation within an ab initio context. Simply employing the relativistic density within the non-relativistic gradient package is in error by 10 5. We introduce a new ...
Zeroth-order optimizationDerivative-free methodsStochastic algorithmsPolyak-Łojasiewicz inequalityConvex programmingFinite differences approximationRandom search90C5665K0590C30We propose and analyze a randomized zeroth-order optimization method based on approximating the exact gradient by finite differences computed...
The proposed algorithm utilizes the well-known Gaussian smoothing technique, which yields unbiased zeroth-order gradient estimators of a related partially smooth surrogate problem (in which one of the two nonsmooth terms in the original problem's objective is replaced by a smooth approximation). This...
The density gradient further causes excitation of electron plasma wave (EPW) at the main beam frequency. The excited EPW couples with input beam thereby generating second harmonics. The well known WKB approximation and moment theory are employed for obtaining non-linear differential equation for laser...
The apparent kinetics of zeroth order surface-catalysed reactions are quantitatively investigated for several configurations involving laminar or turbulent boundary layer flow, with or without pressure gradient. Nine cases of interest are solved exactly in terms of the incomplete beta function and ...
Conditional gradient methodsNewton methodIn this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing constrained optimization, high-dimensional setting and saddle-point avoiding. To handle constrained optimization, we...
Furthermore, we present experimental results for neural networks on MNIST and CIFAR that show faster convergence in training loss and test accuracy, and a smaller distance of the gradient approximation to the true gradient in sparse SZO compared to dense SZO....