This is the first comprehensive reference on trust-region methods, a class of numerical algorithms for the solution of nonlinear convex optimization methods. Its unified treatment covers both unconstrained and constrained problems and reviews a large part of the specialized literature on the subject. It also provides an up-to-date view of numerical optimization. Written primarily for …

A FORTRAN subroutine is described for minimizing a sum of squares of functions of many variables. Such problems arise in non-linear data fitting, and in the solution of non-linear algebraic equations. The subroutine is based on an algorithn due to Marquardt, but with modifications which improve the performance of the method in certain circumstances, yet which require negligible extra computer time …

We provide a natural gradient method that represents the steepest descent direction based on the underlying structure of the parameter space. Although gradient methods cannot make large changes in the values of the parameters, we show that the natural gradient is moving toward choosing a greedy optimal action rather than just a better action. These greedy optimal actions are those that would be …

Most algorithms for the least-squares estimation of non-linear parameters have centered about either of two approaches. On the one hand, the model may be expanded as a Taylor series and corrections to the several parameters calculated at each iteration on the assumption of local linearity. On the other hand, various modifications of the method of steepest-descent have been used. Both methods not …