Initial step length guess

InitialStatic(; alpha=1.0, scaled=false)

Provide a static initial step length.

Keyword alpha corresponds to the static step length (default 1.0). If scaled = true, then the initial step length is scaled with the l_2 norm of the step direction.

InitialPrevious(; alpha=1.0, alphamin=0.0, alphamax=Inf)

Use the previous step length as the initial guess, clamped to the bounds [alphamin, alphamax].

If state.alpha is NaN, then the fallback value alpha is returned.

InitialQuadratic(; α0=1.0, αmin=1e-12, αmax=1.0, ρ=0.25, snap2one=(0.75, Inf))

Quadratic interpolation for the initial step length guess.

This is meant for methods that do not produce well-scaled search directions, such as Gradient Descent and (variations of) Conjugate Gradient methods. See the discussion around Nocedal and Wright, 2nd ed, (3.60).

This procedure has several arguments, with the following defaults.

• α0 = 1.0. The initial step size at the first iteration.
• αmin = 1e-12. The minimum initial step size. (Default arbitrary.)
• αmax = 1.0. The maximum initial step size.
• ρ = 0.25. Maximum decrease from previous iteration, αinit ≥ α_{k-1}. (Default arbitrary.)
• snap2one = (0.75, Inf). Set all values within this (closed) interval to 1.0. (Default arbitrary.)

If αmax ≠ 1.0, then consider ensuring that snap2one[2] < αmax.

InitialConstantChange(; α0=1.0, αmin=1e-12, αmax=1.0, ρ=0.25, snap2one=(0.75, Inf))

Constant first-order change approximation to determine the initial step length.

This requires that the optimization algorithm stores dphi0 from the previous iteration (dphi0_previous = real(dot(∇f_{k-1}, s_{k-1}))), where s is the step direction.

This is meant for methods that do not produce well-scaled search directions, such as Gradient Descent and (variations of) Conjugate Gradient methods. See the discussion in Nocedal and Wright, 2nd ed, p. 59 on "Initial Step Length".

This procedure has several arguments, with the following defaults.

• α0 = 1.0. The initial step size at the first iteration.
• αmin = 1e-12. The minimum initial step size. (Default arbitrary.)
• αmax = 1.0. The maximum initial step size.
• ρ = 0.25. Maximum decrease from previous iteration, αinit ≥ α_{k-1}. (Default arbitrary.)
• snap2one = (0.75, Inf). Set all values within this (closed) interval to 1.0. (Default arbitrary.)

If αmax ≠ 1.0, then consider ensuring that snap2one[2] < αmax.

InitialHagerZhang(; ψ0=0.01, ψ1=0.2, ψ2=2.0, ψ3=0.1, αmax=Inf, α0=1.0,
                    quadstep=true, verbose=false)

Initial step size algorithm from:

W. W. Hager and H. Zhang (2006) Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM Transactions on Mathematical Software 32: 113–137.

If α0 is NaN, then procedure I0 is called at the first iteration; otherwise we select according to procedure I1-2, with starting value α0.