Optim line search

This example shows how to use LineSearches with Optim. We solve the Rosenbrock problem with two different line search algorithms.

First, run Newton with the default line search algorithm:

using Optim, LineSearches
import OptimTestProblems.MultivariateProblems
UP = MultivariateProblems.UnconstrainedProblems
prob = UP.examples["Rosenbrock"]

algo_hz = Newton(linesearch = HagerZhang())
res_hz = Optim.optimize(prob.f, prob.g!, prob.h!, prob.initial_x, algo_hz)

 * Status: success

 * Candidate solution
    Final objective value:     1.527062e-28

 * Found with
    Algorithm:     Newton's Method

 * Convergence measures
    |x - x'|               = 7.05e-08 ≰ 0.0e+00
    |x - x'|/|x'|          = 7.05e-08 ≰ 0.0e+00
    |f(x) - f(x')|         = 1.24e-15 ≰ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 8.11e+12 ≰ 0.0e+00
    |g(x)|                 = 4.85e-13 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    24
    f(x) calls:    53
    ∇f(x) calls:   53
    ∇f(x)ᵀv calls: 0
    ∇²f(x) calls:  25
    ∇²f(x)v calls: 0

Now we can try Newton with the cubic backtracking line search, which reduced the number of objective and gradient calls.

algo_bt3 = Newton(linesearch = BackTracking(order=3))
res_bt3 = Optim.optimize(prob.f, prob.g!, prob.h!, prob.initial_x, algo_bt3)

 * Status: success

 * Candidate solution
    Final objective value:     1.232595e-30

 * Found with
    Algorithm:     Newton's Method

 * Convergence measures
    |x - x'|               = 1.76e-09 ≰ 0.0e+00
    |x - x'|/|x'|          = 1.76e-09 ≰ 0.0e+00
    |f(x) - f(x')|         = 1.13e-17 ≰ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 9.14e+12 ≰ 0.0e+00
    |g(x)|                 = 4.44e-14 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    25
    f(x) calls:    36
    ∇f(x) calls:   26
    ∇f(x)ᵀv calls: 0
    ∇²f(x) calls:  26
    ∇²f(x)v calls: 0

This page was generated using Literate.jl.