Next: Nonlinear Least-Squares TRS Dogleg, Previous: Nonlinear Least-Squares TRS Levenberg-Marquardt, Up: Nonlinear Least-Squares TRS Overview [Index]
This method applies a so-called geodesic acceleration correction to the standard Levenberg-Marquardt step \delta_k (Transtrum et al, 2011). By interpreting \delta_k as a first order step along a geodesic in the model parameter space (ie: a velocity \delta_k = v_k), the geodesic acceleration a_k is a second order correction along the geodesic which is determined by solving the linear least squares system
[J_k; sqrt(mu_k) D_k] a_k = - [f_vv(x_k); 0]
where f_{vv} is the second directional derivative of
the residual vector in the velocity direction v,
f_{vv}(x) = D_v^2 f = \sum_{\alpha\beta} v_{\alpha} v_{\beta} \partial_{\alpha} \partial_{\beta} f(x),
where \alpha and \beta are summed over the p
parameters. The new total step is then \delta_k' = v_k + {1 \over 2}a_k.
The second order correction a_k can be calculated with a modest additional
cost, and has been shown to dramatically reduce the number of iterations
(and expensive Jacobian evaluations) required to reach convergence on a variety
of different problems. In order to utilize the geodesic acceleration, the user must supply a
function which provides the second directional derivative vector
f_{vv}(x), or alternatively the library can use a finite
difference method to estimate this vector with one additional function
evaluation of f(x + h v) where h is a tunable step size
(see the h_fvv parameter description).