Optim initial step length guess

Tip

This example is also available as a Jupyter notebook: optim_initialstep.ipynb

This example shows how to use the initial step length procedures with Optim. We solve the Rosenbrock problem with two different procedures.

First, run Newton with the (default) initial guess and line search procedures.

using Optim, LineSearches
import OptimTestProblems.MultivariateProblems
UP = MultivariateProblems.UnconstrainedProblems
prob = UP.examples["Rosenbrock"]

algo_st = Newton(alphaguess = InitialStatic(), linesearch = HagerZhang())
res_st = Optim.optimize(prob.f, prob.g!, prob.h!, prob.initial_x, algo_st)

 * Status: success

 * Candidate solution
    Final objective value:     1.527062e-28

 * Found with
    Algorithm:     Newton's Method

 * Convergence measures
    |x - x'|               = 7.05e-08 ≰ 0.0e+00
    |x - x'|/|x'|          = 7.05e-08 ≰ 0.0e+00
    |f(x) - f(x')|         = 1.24e-15 ≰ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 8.11e+12 ≰ 0.0e+00
    |g(x)|                 = 4.85e-13 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    24
    f(x) calls:    53
    ∇f(x) calls:   53
    ∇f(x)ᵀv calls: 0
    ∇²f(x) calls:  25
    ∇²f(x)v calls: 0

We can now try with the initial step length guess from Hager and Zhang.

algo_hz = Newton(alphaguess = InitialHagerZhang(α0=1.0), linesearch = HagerZhang())
res_hz = Optim.optimize(prob.f, prob.g!, prob.h!, prob.initial_x, algo_hz)

 * Status: success

 * Candidate solution
    Final objective value:     4.008523e-28

 * Found with
    Algorithm:     Newton's Method

 * Convergence measures
    |x - x'|               = 4.00e-10 ≰ 0.0e+00
    |x - x'|/|x'|          = 4.00e-10 ≰ 0.0e+00
    |f(x) - f(x')|         = 3.32e-18 ≰ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 8.29e+09 ≰ 0.0e+00
    |g(x)|                 = 7.97e-13 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    27
    f(x) calls:    76
    ∇f(x) calls:   50
    ∇f(x)ᵀv calls: 0
    ∇²f(x) calls:  28
    ∇²f(x)v calls: 0

From the result we see that this has reduced the number of function and gradient calls, but increased the number of iterations.

This page was generated using Literate.jl.