go home Home | Main Page | Modules | Namespace List | Class Hierarchy | Alphabetical List | Data Structures | File List | Namespace Members | Data Fields | Globals | Related Pages
Public Types | Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | Private Attributes
itk::QuasiNewtonLBFGSOptimizer Class Reference
Components » Optimizers

#include <itkQuasiNewtonLBFGSOptimizer.h>

Detailed Description

ITK version of the lbfgs algorithm ...

This class is an ITK version of the netlib lbfgs_ function. It gives exactly the same results, if used in combination with the itk::MoreThuenteLineSearchOptimizer.

The optimizer solves the unconstrained minimization problem

\[ \min F(x), \quad x = ( x_1,x_2,\ldots,x_N ), \]

using the limited memory BFGS method. The routine is especially effective on problems involving a large number of variables. In a typical iteration of this method an approximation $H_k$ to the inverse of the Hessian is obtained by applying $M$ BFGS updates to a diagonal matrix $H_0$, using information from the previous $M$ steps. The user specifies the number $M$ (Memory), which determines the amount of storage required by the routine.

The algorithm is described in "On the limited memory BFGS method for large scale optimization", by D. Liu and J. Nocedal, Mathematical Programming B 45 (1989) 503-528.

The steplength is determined at each iteration by means of a line search routine. The itk::MoreThuenteLineSearchOptimizer works well.

Definition at line 58 of file itkQuasiNewtonLBFGSOptimizer.h.

+ Inheritance diagram for itk::QuasiNewtonLBFGSOptimizer:

Public Types

typedef SmartPointer< const Self > ConstPointer
 
typedef Superclass::CostFunctionType CostFunctionType
 
typedef Superclass::DerivativeType DerivativeType
 
typedef Array< double > DiagonalMatrixType
 
typedef LineSearchOptimizerType::Pointer LineSearchOptimizerPointer
 
typedef LineSearchOptimizer LineSearchOptimizerType
 
typedef Superclass::MeasureType MeasureType
 
typedef Superclass::ParametersType ParametersType
 
typedef SmartPointer< Self > Pointer
 
typedef Array< double > RhoType
 
typedef Superclass::ScaledCostFunctionType ScaledCostFunctionType
 
typedef Superclass::ScalesType ScalesType
 
typedef QuasiNewtonLBFGSOptimizer Self
 
enum  StopConditionType {
  MetricError , LineSearchError , MaximumNumberOfIterations , InvalidDiagonalMatrix ,
  GradientMagnitudeTolerance , ZeroStep , Unknown
}
 
typedef std::vector< ParametersType > SType
 
typedef ScaledSingleValuedNonLinearOptimizer Superclass
 
typedef std::vector< DerivativeType > YType
 
- Public Types inherited from itk::ScaledSingleValuedNonLinearOptimizer
typedef SmartPointer< const Self > ConstPointer
 
typedef Superclass::CostFunctionType CostFunctionType
 
typedef Superclass::DerivativeType DerivativeType
 
typedef Superclass::MeasureType MeasureType
 
typedef Superclass::ParametersType ParametersType
 
typedef SmartPointer< Self > Pointer
 
typedef ScaledCostFunctionType::Pointer ScaledCostFunctionPointer
 
typedef ScaledSingleValuedCostFunction ScaledCostFunctionType
 
typedef NonLinearOptimizer::ScalesType ScalesType
 
typedef ScaledSingleValuedNonLinearOptimizer Self
 
typedef SingleValuedNonLinearOptimizer Superclass
 

Public Member Functions

virtual const char * GetClassName () const
 
virtual const DerivativeType & GetCurrentGradient ()
 
virtual unsigned long GetCurrentIteration () const
 
virtual double GetCurrentStepLength () const
 
virtual MeasureType GetCurrentValue () const
 
virtual double GetGradientMagnitudeTolerance () const
 
virtual bool GetInLineSearch () const
 
virtual unsigned long GetMaximumNumberOfIterations () const
 
virtual unsigned int GetMemory () const
 
virtual const StopConditionType & GetStopCondition ()
 
 itkGetModifiableObjectMacro (LineSearchOptimizer, LineSearchOptimizerType)
 
virtual void ResumeOptimization (void)
 
virtual void SetGradientMagnitudeTolerance (double _arg)
 
virtual void SetLineSearchOptimizer (LineSearchOptimizerType *_arg)
 
virtual void SetMaximumNumberOfIterations (unsigned long _arg)
 
virtual void SetMemory (unsigned int _arg)
 
void StartOptimization (void) override
 
virtual void StopOptimization (void)
 
- Public Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
virtual const char * GetClassName () const
 
const ParametersType & GetCurrentPosition (void) const override
 
virtual bool GetMaximize () const
 
virtual const ScaledCostFunctionType * GetScaledCostFunction ()
 
virtual const ParametersType & GetScaledCurrentPosition ()
 
bool GetUseScales (void) const
 
virtual void InitializeScales (void)
 
virtual void MaximizeOff ()
 
virtual void MaximizeOn ()
 
void SetCostFunction (CostFunctionType *costFunction) override
 
virtual void SetMaximize (bool _arg)
 
virtual void SetUseScales (bool arg)
 

Static Public Member Functions

static Pointer New ()
 
- Static Public Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
static Pointer New ()
 

Protected Member Functions

virtual void ComputeDiagonalMatrix (DiagonalMatrixType &diag_H0)
 
virtual void ComputeSearchDirection (const DerivativeType &gradient, ParametersType &searchDir)
 
virtual void LineSearch (const ParametersType searchDir, double &step, ParametersType &x, MeasureType &f, DerivativeType &g)
 
void PrintSelf (std::ostream &os, Indent indent) const override
 
 QuasiNewtonLBFGSOptimizer ()
 
virtual void SetInLineSearch (bool _arg)
 
virtual void StoreCurrentPoint (const ParametersType &step, const DerivativeType &grad_dif)
 
virtual bool TestConvergence (bool firstLineSearchDone)
 
 ~QuasiNewtonLBFGSOptimizer () override
 
- Protected Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
virtual void GetScaledDerivative (const ParametersType &parameters, DerivativeType &derivative) const
 
virtual MeasureType GetScaledValue (const ParametersType &parameters) const
 
virtual void GetScaledValueAndDerivative (const ParametersType &parameters, MeasureType &value, DerivativeType &derivative) const
 
void PrintSelf (std::ostream &os, Indent indent) const override
 
 ScaledSingleValuedNonLinearOptimizer ()
 
void SetCurrentPosition (const ParametersType &param) override
 
virtual void SetScaledCurrentPosition (const ParametersType &parameters)
 
 ~ScaledSingleValuedNonLinearOptimizer () override
 

Protected Attributes

unsigned int m_Bound
 
DerivativeType m_CurrentGradient
 
unsigned long m_CurrentIteration
 
double m_CurrentStepLength
 
MeasureType m_CurrentValue
 
bool m_InLineSearch
 
unsigned int m_Point
 
unsigned int m_PreviousPoint
 
RhoType m_Rho
 
SType m_S
 
bool m_Stop
 
StopConditionType m_StopCondition
 
YType m_Y
 
- Protected Attributes inherited from itk::ScaledSingleValuedNonLinearOptimizer
ScaledCostFunctionPointer m_ScaledCostFunction
 
ParametersType m_ScaledCurrentPosition
 

Private Member Functions

void operator= (const Self &)
 
 QuasiNewtonLBFGSOptimizer (const Self &)
 

Private Attributes

double m_GradientMagnitudeTolerance
 
LineSearchOptimizerPointer m_LineSearchOptimizer
 
unsigned long m_MaximumNumberOfIterations
 
unsigned int m_Memory
 

Member Typedef Documentation

◆ ConstPointer

typedef SmartPointer< const Self > itk::QuasiNewtonLBFGSOptimizer::ConstPointer

Definition at line 65 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ CostFunctionType

typedef Superclass::CostFunctionType itk::QuasiNewtonLBFGSOptimizer::CostFunctionType

Definition at line 72 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ DerivativeType

typedef Superclass::DerivativeType itk::QuasiNewtonLBFGSOptimizer::DerivativeType

Definition at line 71 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ DiagonalMatrixType

typedef Array< double > itk::QuasiNewtonLBFGSOptimizer::DiagonalMatrixType

Definition at line 80 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ LineSearchOptimizerPointer

typedef LineSearchOptimizerType::Pointer itk::QuasiNewtonLBFGSOptimizer::LineSearchOptimizerPointer

Definition at line 83 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ LineSearchOptimizerType

typedef LineSearchOptimizer itk::QuasiNewtonLBFGSOptimizer::LineSearchOptimizerType

Definition at line 81 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ MeasureType

typedef Superclass::MeasureType itk::QuasiNewtonLBFGSOptimizer::MeasureType

Definition at line 74 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ ParametersType

typedef Superclass::ParametersType itk::QuasiNewtonLBFGSOptimizer::ParametersType

Definition at line 70 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ Pointer

typedef SmartPointer< Self > itk::QuasiNewtonLBFGSOptimizer::Pointer

Definition at line 64 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ RhoType

typedef Array< double > itk::QuasiNewtonLBFGSOptimizer::RhoType

Definition at line 77 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ ScaledCostFunctionType

typedef Superclass::ScaledCostFunctionType itk::QuasiNewtonLBFGSOptimizer::ScaledCostFunctionType

Definition at line 73 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ ScalesType

typedef Superclass::ScalesType itk::QuasiNewtonLBFGSOptimizer::ScalesType

Definition at line 75 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ Self

typedef QuasiNewtonLBFGSOptimizer itk::QuasiNewtonLBFGSOptimizer::Self

Definition at line 62 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ SType

typedef std::vector< ParametersType > itk::QuasiNewtonLBFGSOptimizer::SType

Definition at line 78 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ Superclass

typedef ScaledSingleValuedNonLinearOptimizer itk::QuasiNewtonLBFGSOptimizer::Superclass

Definition at line 63 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ YType

typedef std::vector< DerivativeType > itk::QuasiNewtonLBFGSOptimizer::YType

Definition at line 79 of file itkQuasiNewtonLBFGSOptimizer.h.

Member Enumeration Documentation

◆ StopConditionType

enum itk::QuasiNewtonLBFGSOptimizer::StopConditionType
Enumerator
MetricError 
LineSearchError 
MaximumNumberOfIterations 
InvalidDiagonalMatrix 
GradientMagnitudeTolerance 
ZeroStep 
Unknown 

Definition at line 85 of file itkQuasiNewtonLBFGSOptimizer.h.

Constructor & Destructor Documentation

◆ QuasiNewtonLBFGSOptimizer() [1/2]

itk::QuasiNewtonLBFGSOptimizer::QuasiNewtonLBFGSOptimizer ( )
protected

◆ ~QuasiNewtonLBFGSOptimizer()

itk::QuasiNewtonLBFGSOptimizer::~QuasiNewtonLBFGSOptimizer ( )
inlineoverrideprotected

Definition at line 135 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ QuasiNewtonLBFGSOptimizer() [2/2]

itk::QuasiNewtonLBFGSOptimizer::QuasiNewtonLBFGSOptimizer ( const Self &  )
private

Member Function Documentation

◆ ComputeDiagonalMatrix()

virtual void itk::QuasiNewtonLBFGSOptimizer::ComputeDiagonalMatrix ( DiagonalMatrixType &  diag_H0)
protectedvirtual

Compute H0

Override this method if not satisfied with the default choice.

◆ ComputeSearchDirection()

virtual void itk::QuasiNewtonLBFGSOptimizer::ComputeSearchDirection ( const DerivativeType &  gradient,
ParametersType &  searchDir 
)
protectedvirtual

Compute -Hg

COMPUTE -H*G USING THE FORMULA GIVEN IN: Nocedal, J. 1980, "Updating quasi-Newton matrices with limited storage", Mathematics of Computation, Vol.24, No.151, pp. 773-782.

◆ GetClassName()

virtual const char * itk::QuasiNewtonLBFGSOptimizer::GetClassName ( ) const
virtual

Reimplemented from itk::ScaledSingleValuedNonLinearOptimizer.

Reimplemented in elastix::QuasiNewtonLBFGS< TElastix >.

◆ GetCurrentGradient()

virtual const DerivativeType & itk::QuasiNewtonLBFGSOptimizer::GetCurrentGradient ( )
virtual

◆ GetCurrentIteration()

virtual unsigned long itk::QuasiNewtonLBFGSOptimizer::GetCurrentIteration ( ) const
virtual

Get information about optimization process:

◆ GetCurrentStepLength()

virtual double itk::QuasiNewtonLBFGSOptimizer::GetCurrentStepLength ( ) const
virtual

◆ GetCurrentValue()

virtual MeasureType itk::QuasiNewtonLBFGSOptimizer::GetCurrentValue ( ) const
virtual

◆ GetGradientMagnitudeTolerance()

virtual double itk::QuasiNewtonLBFGSOptimizer::GetGradientMagnitudeTolerance ( ) const
virtual

Setting: the mininum gradient magnitude.

The optimizer stops when: ||CurrentGradient|| < GradientMagnitudeTolerance * max(1, ||CurrentPosition||)

◆ GetInLineSearch()

virtual bool itk::QuasiNewtonLBFGSOptimizer::GetInLineSearch ( ) const
virtual

◆ GetMaximumNumberOfIterations()

virtual unsigned long itk::QuasiNewtonLBFGSOptimizer::GetMaximumNumberOfIterations ( ) const
virtual

Setting: the maximum number of iterations

◆ GetMemory()

virtual unsigned int itk::QuasiNewtonLBFGSOptimizer::GetMemory ( ) const
virtual

◆ GetStopCondition()

virtual const StopConditionType & itk::QuasiNewtonLBFGSOptimizer::GetStopCondition ( )
virtual

◆ itkGetModifiableObjectMacro()

itk::QuasiNewtonLBFGSOptimizer::itkGetModifiableObjectMacro ( LineSearchOptimizer  ,
LineSearchOptimizerType   
)

◆ LineSearch()

virtual void itk::QuasiNewtonLBFGSOptimizer::LineSearch ( const ParametersType  searchDir,
double &  step,
ParametersType &  x,
MeasureType &  f,
DerivativeType &  g 
)
protectedvirtual

Perform a line search along the search direction. On calling, x, f, and g should contain the current position, the cost function value at this position, and the derivative. On return the step, x (new position), f (value at x), and g (derivative at x) are updated.

Reimplemented in elastix::QuasiNewtonLBFGS< TElastix >.

◆ New()

static Pointer itk::QuasiNewtonLBFGSOptimizer::New ( )
static

◆ operator=()

void itk::QuasiNewtonLBFGSOptimizer::operator= ( const Self &  )
private

◆ PrintSelf()

void itk::QuasiNewtonLBFGSOptimizer::PrintSelf ( std::ostream &  os,
Indent  indent 
) const
inlineoverrideprotected

Definition at line 138 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ ResumeOptimization()

virtual void itk::QuasiNewtonLBFGSOptimizer::ResumeOptimization ( void  )
virtual

◆ SetGradientMagnitudeTolerance()

virtual void itk::QuasiNewtonLBFGSOptimizer::SetGradientMagnitudeTolerance ( double  _arg)
virtual

◆ SetInLineSearch()

virtual void itk::QuasiNewtonLBFGSOptimizer::SetInLineSearch ( bool  _arg)
protectedvirtual

◆ SetLineSearchOptimizer()

virtual void itk::QuasiNewtonLBFGSOptimizer::SetLineSearchOptimizer ( LineSearchOptimizerType *  _arg)
virtual

Setting: the line search optimizer

◆ SetMaximumNumberOfIterations()

virtual void itk::QuasiNewtonLBFGSOptimizer::SetMaximumNumberOfIterations ( unsigned long  _arg)
virtual

◆ SetMemory()

virtual void itk::QuasiNewtonLBFGSOptimizer::SetMemory ( unsigned int  _arg)
virtual

Setting: the memory. The number of iterations that are used to estimate the Hessian. 5 by default. 0 results in (normalised) gradient descent search directions

◆ StartOptimization()

void itk::QuasiNewtonLBFGSOptimizer::StartOptimization ( void  )
override

◆ StopOptimization()

virtual void itk::QuasiNewtonLBFGSOptimizer::StopOptimization ( void  )
virtual

◆ StoreCurrentPoint()

virtual void itk::QuasiNewtonLBFGSOptimizer::StoreCurrentPoint ( const ParametersType &  step,
const DerivativeType &  grad_dif 
)
protectedvirtual

Store s = x_k - x_k-1 and y = g_k - g_k-1 in m_S and m_Y, and store 1/(ys) in m_Rho.

◆ TestConvergence()

virtual bool itk::QuasiNewtonLBFGSOptimizer::TestConvergence ( bool  firstLineSearchDone)
protectedvirtual

Check if convergence has occured; The firstLineSearchDone bool allows the implementation of TestConvergence to decide to skip a few convergence checks when no line search has performed yet (so, before the actual optimisation begins)

Reimplemented in elastix::QuasiNewtonLBFGS< TElastix >.

Field Documentation

◆ m_Bound

unsigned int itk::QuasiNewtonLBFGSOptimizer::m_Bound
protected

Definition at line 156 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_CurrentGradient

DerivativeType itk::QuasiNewtonLBFGSOptimizer::m_CurrentGradient
protected

Definition at line 140 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_CurrentIteration

unsigned long itk::QuasiNewtonLBFGSOptimizer::m_CurrentIteration
protected

Definition at line 142 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_CurrentStepLength

double itk::QuasiNewtonLBFGSOptimizer::m_CurrentStepLength
protected

Definition at line 145 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_CurrentValue

MeasureType itk::QuasiNewtonLBFGSOptimizer::m_CurrentValue
protected

Definition at line 141 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_GradientMagnitudeTolerance

double itk::QuasiNewtonLBFGSOptimizer::m_GradientMagnitudeTolerance
private

Definition at line 205 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_InLineSearch

bool itk::QuasiNewtonLBFGSOptimizer::m_InLineSearch
protected

Is true when the LineSearchOptimizer has been started.

Definition at line 148 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_LineSearchOptimizer

LineSearchOptimizerPointer itk::QuasiNewtonLBFGSOptimizer::m_LineSearchOptimizer
private

Definition at line 206 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_MaximumNumberOfIterations

unsigned long itk::QuasiNewtonLBFGSOptimizer::m_MaximumNumberOfIterations
private

Definition at line 204 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_Memory

unsigned int itk::QuasiNewtonLBFGSOptimizer::m_Memory
private

Definition at line 207 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_Point

unsigned int itk::QuasiNewtonLBFGSOptimizer::m_Point
protected

Definition at line 154 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_PreviousPoint

unsigned int itk::QuasiNewtonLBFGSOptimizer::m_PreviousPoint
protected

Definition at line 155 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_Rho

RhoType itk::QuasiNewtonLBFGSOptimizer::m_Rho
protected

Definition at line 150 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_S

SType itk::QuasiNewtonLBFGSOptimizer::m_S
protected

Definition at line 151 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_Stop

bool itk::QuasiNewtonLBFGSOptimizer::m_Stop
protected

Definition at line 144 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_StopCondition

StopConditionType itk::QuasiNewtonLBFGSOptimizer::m_StopCondition
protected

Definition at line 143 of file itkQuasiNewtonLBFGSOptimizer.h.

◆ m_Y

YType itk::QuasiNewtonLBFGSOptimizer::m_Y
protected

Definition at line 152 of file itkQuasiNewtonLBFGSOptimizer.h.


Generated on 1667476801 for elastix by doxygen 1.9.4 elastix logo