39.11 Troubleshooting
When developing a code to solve a nonlinear least squares problem,
here are a few considerations to keep in mind.
- The most common difficulty is the accurate implementation of the Jacobian
matrix. If the analytic Jacobian is not properly provided to the
solver, this can hinder and many times prevent convergence of the method.
When developing a new nonlinear least squares code, it often helps
to compare the program output with the internally computed finite
difference Jacobian and the user supplied analytic Jacobian. If there
is a large difference in coefficients, it is likely the analytic
Jacobian is incorrectly implemented.
- If your code is having difficulty converging, the next thing to
check is the starting point provided to the solver. The methods
of this chapter are local methods, meaning if you provide a starting
point far away from the true minimum, the method may converge to
a local minimum or not converge at all. Sometimes it is possible
to solve a linearized approximation to the nonlinear problem,
and use the linear solution as the starting point to the nonlinear
problem.
- If the various parameters of the coefficient vector x
vary widely in magnitude, then the problem is said to be badly scaled.
The methods of this chapter do attempt to automatically rescale
the elements of x to have roughly the same order of magnitude,
but in extreme cases this could still cause problems for convergence.
In these cases it is recommended for the user to scale their
parameter vector x so that each parameter spans roughly the
same range, say [-1,1]. The solution vector can be backscaled
to recover the original units of the problem.