Fomo nonmonotone extension

[d-monnet]

Add non-monotone strategy to fomo solver.
Modifies conditions on momentum contribution.

Related paper can be shared upon request.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.66%. Comparing base (3d05513) to head (1e1368e).
Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #267      +/-   ##
==========================================
- Coverage   88.33%   86.66%   -1.67%     
==========================================
  Files           7        7              
  Lines        1106     1125      +19     
==========================================
- Hits          977      975       -2     
- Misses        129      150      +21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Package name	latest	stable
FletcherPenaltySolver.jl
Percival.jl

[tmigot]

Thanks @d-monnet for the PR! I made a first batch of comments.
You should also add unit tests for this variant.

[tmigot]

@d-monnet Would you have a simple example or an illustration of the nonmonotone vs. monotone?

[d-monnet]

@d-monnet Would you have a simple example or an illustration of the nonmonotone vs. monotone?

Not a simple example, but here are the perf profile (iterations) for R2, TR, and fomo with r2 and tr steps.
Profiles were obtained on 74 unconstrained problems of ADNLPProblems, run with Float64 with atol=rtol = 1e-4 and default parameters.

Looks like we have a bug with the labels when using label kwargs with profile_solvers by the way: 2 by 2 comparison labels are wrong.

[tmigot]

@d-monnet Would you have a simple example or an illustration of the nonmonotone vs. monotone?

Not a simple example, but here are the perf profile (iterations) for R2, TR, and fomo with r2 and tr steps. Profiles were obtained on 74 unconstrained problems of ADNLPProblems, run with Float64 with atol=rtol = 1e-4 and default parameters.

Looks like we have a bug with the labels when using label kwargs with profile_solvers by the way: 2 by 2 comparison labels are wrong.

That's awesome! In case, you still have the result of this benchmark, maybe you can find an example of a problem that has been solved by nonmonotone that has not been solved by monotone?
I think we should plan a relatively short tutorial on this 1D methods and such an example would be really great.

[tmigot]

Thanks @d-monnet it looks good to me! Just one last typo.

When updating your branch to the current code, in JSO, we use git rebase instead of git merge. This would avoid duplicating the changes in your PR.

[d-monnet]

Thanks @d-monnet it looks good to me! Just one last typo.

When updating your branch to the current code, in JSO, we use git rebase instead of git merge. This would avoid duplicating the changes in your PR.

That's what I was planning to do, but I messed something up with VS Code interface...

[d-monnet]

@d-monnet Would you have a simple example or an illustration of the nonmonotone vs. monotone?

Not a simple example, but here are the perf profile (iterations) for R2, TR, and fomo with r2 and tr steps. Profiles were obtained on 74 unconstrained problems of ADNLPProblems, run with Float64 with atol=rtol = 1e-4 and default parameters.
Looks like we have a bug with the labels when using label kwargs with profile_solvers by the way: 2 by 2 comparison labels are wrong.

That's awesome! In case, you still have the result of this benchmark, maybe you can find an example of a problem that has been solved by nonmonotone that has not been solved by monotone? I think we should plan a relatively short tutorial on this 1D methods and such an example would be really great.

I'll look into it and start working on a tutorial.