Poblano Toolbox

Optimization Input Parameters

Input parameters are passed to the different optimization methods using Matlab inputParser objects. Some parameters are shared across all methods and others are specific to a particular method. Below are descriptions of the shared input parameters and examples of how to set and use these parameters in the optimization methods. The Poblano function poblano_params is used by the optimization methods to set the input parameters.

Contents


Parameters Shared Across All Methods

Display Parameters

The following parameters control the information that is displayed during a run of a Poblano optimizer.

Display                 Controls amount of printed output {'iter'}
  'iter'                Display information every iteration
  'final'               Display information only after final iteration
  'off'                 Display no information

Example using default value of 'iter':

ncg(@example1, pi/4);
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     0         1       0.70710678       0.70710678
     1         6      -0.99999998       0.00017407
     2         7      -1.00000000       0.00000000

The per iteration information displayed contains the iteration number (Iter), total number of function evaluations performed thus far (FuncEvals), the function value at the current iterate (F(X)), and the norm of the gradient at the current iterate scaled by the problem size (||G(X)||/N). After the final iteration is performed, the total iteration and function evaluations, along with the function value and scaled gradient norm at the solution found is displayed. Below is an example of the information displayed using the default parameters.

Stopping Criteria Parameters

The following parameters control the stopping criteria of the optimization methods.

MaxIters                Maximum number of iterations allowed {100}
MaxFuncEvals            Maximum number of function evaluations allowed {100}
StopTol                 Gradient norm stopping tolerance, i.e., the
                        method stops when the norm of the gradient is less
                        than StopTol times the number of variables {1e-5}
RelFuncTol              Relative function value change stopping tolerance,
                        i.e., the method stops when the relative change
                        of the function value from one iteration to the
                        next is less than RelFuncTol {1e-6}

Trace Parameters

The following parameters control the information that is saved and output for each iteration.

TraceX                  Flag to save a history of X (iterates) {false}
TraceFunc               Flag to save a history of the function values
                        of the iterates {false}
TraceRelFunc            Flag to save a history of the relative difference
                        between the function values at the current and
                        previous iterates {false}
TraceGrad               Flag to save a history of the gradients of the
                        iterates {false}
TraceGradNorm           Flag to save a history of the norm of the
                        gradients of the iterates {false}
TraceFuncEvals          Flag to save a history of the number of function
                        evaluations performed at each iteration {false}

Line Search Parameters

The following parameters control the behavior of the line search method used in the optimization methods.

LineSearch_xtol         Stopping tolerance for minimum change input
                        variable {1e-15}
LineSearch_ftol         Stopping tolerance for sufficient decrease
                        condition {1e-4}
LineSearch_gtol         Stopping tolerance for directional derivative
                        condition {1e-2}
LineSearch_stpmin       Minimum step to take {1e-15}
LineSearch_stpmax       Maximum step to take {1e15}
LineSearch_maxfev       Maximum number of iterations {20}
LineSearch_initialstep  Initial step to be taken in the line search {1}


Method-Specific Parameters

Nonlinear Conjugate Gradent Minimization (ncg) Method details

Update                  Conjugate direction update {'PR'}
  'FR'                  Fletcher-Reeves
  'PR'                  Polak-Ribiere
  'HS'                  Hestenes-Stiefel
  'SD'                  Steepest Decsent
RestartIters            Number of iterations to run before conjugate
                        direction restart {20}
RestartNW               Flag to use restart heuristic of Nocedal and
                        Wright {false}
RestartNWTol            Tolerance for Nocedal and Wright restart
                        heuristic {0.1}

Limited-memory BFGS Minimization (lbfgs) Method details

M                       Limited memory parameter {5}

Truncated Newton Minimization (tn) Method details

CGSolver                Matlab conjugate gradient (CG) method used to
                        solve for search direction {'symmlq'}
  'symmlq'              Symmetric LQ method
  'pcg'                 Classical CG method
CGIters                 Maximum number of CG iterations allowed {5}
CGTolType               CG stopping tolerance type used {'quadratic'}
  'quadratic'           ||R|| / ||G|| <  min(0.5,||G||)
  'superlinear'         ||R|| / ||G|| <  min(0.5,sqrt(||G||))
  'fixed'               ||R|| < CGTol
                        where R is the residual and G is the gradient
                        of FUN at X
CGTol                   CG stopping tolerance when CGTolType is
                        'fixed' {1e-6}
HessVecFDStep           Hessian vector product finite difference step {1e-10}
  0                     Use iterate-based step: 1e-8*(1+||X||)
  >0                    Fixed value to use as the difference step


Default Parameters

The default input parameters are returned using the sole input of 'defaults' to one of the Poblano optimization methods:

ncg_default_params = ncg('defaults')
lbfgs_default_params = lbfgs('defaults')
tn_default_params = tn('defaults')
ncg_default_params = 

                   Display: 'iter'
           LineSearch_ftol: 0.0001
           LineSearch_gtol: 0.01
    LineSearch_initialstep: 1
         LineSearch_maxfev: 20
         LineSearch_method: 'more-thuente'
         LineSearch_stpmax: 1e+15
         LineSearch_stpmin: 1e-15
           LineSearch_xtol: 1e-15
              MaxFuncEvals: 100
                  MaxIters: 100
                RelFuncTol: 1e-06
              RestartIters: 20
                 RestartNW: 0
              RestartNWTol: 0.1
                   StopTol: 1e-05
                 TraceFunc: 0
            TraceFuncEvals: 0
                 TraceGrad: 0
             TraceGradNorm: 0
              TraceRelFunc: 0
                    TraceX: 0
                    Update: 'PR'


lbfgs_default_params = 

                   Display: 'iter'
           LineSearch_ftol: 0.0001
           LineSearch_gtol: 0.01
    LineSearch_initialstep: 1
         LineSearch_maxfev: 20
         LineSearch_method: 'more-thuente'
         LineSearch_stpmax: 1e+15
         LineSearch_stpmin: 1e-15
           LineSearch_xtol: 1e-15
                         M: 5
              MaxFuncEvals: 100
                  MaxIters: 100
                RelFuncTol: 1e-06
                   StopTol: 1e-05
                 TraceFunc: 0
            TraceFuncEvals: 0
                 TraceGrad: 0
             TraceGradNorm: 0
              TraceRelFunc: 0
                    TraceX: 0


tn_default_params = 

                   CGIters: 5
                  CGSolver: 'symmlq'
                     CGTol: 1e-06
                 CGTolType: 'quadratic'
                   Display: 'iter'
             HessVecFDStep: 1e-10
           LineSearch_ftol: 0.0001
           LineSearch_gtol: 0.01
    LineSearch_initialstep: 1
         LineSearch_maxfev: 20
         LineSearch_method: 'more-thuente'
         LineSearch_stpmax: 1e+15
         LineSearch_stpmin: 1e-15
           LineSearch_xtol: 1e-15
              MaxFuncEvals: 100
                  MaxIters: 100
                RelFuncTol: 1e-06
                   StopTol: 1e-05
                 TraceFunc: 0
            TraceFuncEvals: 0
                 TraceGrad: 0
             TraceGradNorm: 0
              TraceRelFunc: 0
                    TraceX: 0


Passing Parameters to Methods

As mentioned above, input parameters are passed to the Poblano optimization methods using Matlab inputParser objects. Below are several examples of passing parameters to Poblano methods. For more detailed description of inputParser objects, see the Matlab documentation.

Case 1: Using default input parameters.

To use the default methods, simply pass the function handle to the function/gradient method and a starting point to the optimization method (i.e., do not pass any input parameters into the method).

ncg(@(x) example1(x,3), pi/4);
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     0         1       0.70710678       2.12132034
     1        14      -0.99998885       0.01416497
     2        16      -1.00000000       0.00000147

Case 2: Passing parameter-value pairs into a method.

Instead of passing a structure of input parameters, pairs of parameters and values may be passed as well. In this case, all parameters not specified as input use their default values. Below is an example of specifying a parameter in this way.

ncg(@(x) example1(x,3), pi/4,'Display','final');
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     2        16      -1.00000000       0.00000147

Case 3: Passing input parameters as fields in a structure.

Input parameters can be passed as fields in a structure. Note, though, that since each optimization method uses method specific parameters, it is suggested to start from a structure of the default parameters for a particular method. Once the structure of default parameters has been created, the individual parameters (i.e., fields in the structure) can be changed. Below is an example of this for the ncg method.

params = ncg('defaults');
params.MaxIters = 1;
ncg(@(x) example1(x,3), pi/4, params);
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     0         1       0.70710678       2.12132034
     1        14      -0.99998885       0.01416497

Case 4: Using parameters from one run in another run.

One of the outputs returned by the Poblano optimization methods is the inputParser object of the input parameters used in that run. That object contains a field called Results, which can be passed as the input parameters to another run. For example, this is helpful when running comparisons of methods where only one parameter is changed. Shown below is such an example, where default parameters are used in one run, and the same parameters with just a single change are used in another run.

out = ncg(@(x) example1(x,3), pi./[4 5 6]');
params = out.Params.Results;
params.Display = 'final';
ncg(@(x) example1(x,3), pi./[4 5 6]',params);
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     0         1       2.65816330       0.77168096
     1         7      -0.63998759       0.78869570
     2        11      -0.79991790       0.60693819
     3        14      -0.99926100       0.03843827
     4        16      -0.99999997       0.00023739
     5        18      -1.00000000       0.00000000
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     5        18      -1.00000000       0.00000000

Poblano Toolbox