Docs for MOO #823

ParasPuneetSingh · 2024-09-20T16:35:02Z

Checklist

Appropriate tests were added
Any code changes were done in a way that does not break public API
All documentation related to code changes were updated
The new code follows the
contributor guidelines, in particular the SciML Style Guide and
COLPRAC.
Any new documentation only uses public API

Additional context

Add any other context about the problem here.

Added documentation for MOO in BBO

MOO docs update.

docs/src/optimization_packages/blackboxoptim.md

Vaibhavdixit02 · 2024-09-22T04:05:57Z

https://github.com/SciML/Optimization.jl/actions/runs/10977080544/job/30478629355?pr=823 documentation failure seems real, can you take a look?

updated project.toml for the docs.

Added compat for BBO.

docs/src/optimization_packages/blackboxoptim.md

Vaibhavdixit02 · 2024-09-23T16:08:29Z

docs/src/optimization_packages/blackboxoptim.md

+end
+mof = MultiObjectiveOptimizationFunction(multi_obj_func_2)
+prob = Optimization.OptimizationProblem(mof, u0; lb = [0.0, 0.0], ub = [2.0, 2.0])
+sol = solve(prob_2, opt, NumDimensions=2, FitnessScheme=ParetoFitnessScheme{2}(is_minimizing=true))


Suggested change

sol = solve(prob_2, opt, NumDimensions=2, FitnessScheme=ParetoFitnessScheme{2}(is_minimizing=true))

sol = solve(prob, opt, NumDimensions=2, FitnessScheme=ParetoFitnessScheme{2}(is_minimizing=true))

Vaibhavdixit02 · 2024-09-23T16:08:56Z

docs/src/optimization_packages/evolutionary.md

+  return [f1, f2]
+end
+initial_guess = [1.0, 1.0]
+function gradient_multi_objective(x, p=nothing)


Since this isn't used please remove it

It is used in the MultiObjectiveOptimizationFunction call below and is passed as the jac arg.

But Evolutionary doesn't need derivatives right?

@ParasPuneetSingh bump

Yeah just checked the testing once again and it is not needed, so will remove it.

Vaibhavdixit02 · 2024-09-23T16:09:50Z

docs/src/optimization_packages/metaheuristics.md

+npartitions = 100
+
+# reference points (Das and Dennis's method)
+weights = gen_ref_dirs(nobjectives, npartitions)


This function is missing here

Vaibhavdixit02 · 2024-09-23T16:10:16Z

@ParasPuneetSingh please take a look at the review above

Added required packages for MOO docs.

added required packages for MOO

Corrected function names for MOO docs.

Removed unnecessary FowardDiff function.

Vaibhavdixit02 · 2024-09-26T14:05:38Z

Can you rebase on master there was an incorrect compat there that made the docs environment fail here I have updated it

docs/src/optimization_packages/metaheuristics.md

docs/src/optimization_packages/evolutionary.md

Added the package for the algorithms.

ChrisRackauckas · 2024-11-09T19:28:45Z

docs/src/optimization_packages/blackboxoptim.md

@@ -67,3 +67,21 @@ prob = Optimization.OptimizationProblem(f, x0, p, lb = [-1.0, -1.0], ub = [1.0,
 sol = solve(prob, BBO_adaptive_de_rand_1_bin_radiuslimited(), maxiters = 100000,
    maxtime = 1000.0)
 ```
+
+## Multi-objective optimization
+The optimizer for Multi-Objective Optimization is `BBO_borg_moea()`. Your objective function should return a tuple of the objective values and you should indicate the fitness scheme to be (typically) Pareto fitness and specify the number of objectives. Otherwise, the use is similar, here is an example:


Why a tuple? That is not going to scale well.

Yeah corrected that mistake to vector, the struct uses a vector of objective functions.

ChrisRackauckas · 2024-11-09T19:29:05Z

docs/src/optimization_packages/blackboxoptim.md

+end
+mof = MultiObjectiveOptimizationFunction(multi_obj_func)
+prob = Optimization.OptimizationProblem(mof, u0; lb = [0.0, 0.0], ub = [2.0, 2.0])
+sol = solve(prob, opt, NumDimensions=2, FitnessScheme=ParetoFitnessScheme{2}(is_minimizing=true))


NumDimensions, FitnessScheme, those don't match Julia style.

Those should be in the opt.

NumDimensions should be computed from the MultiObjectiveOptimizationFunction, FitnessScheme should be in the opt.

ChrisRackauckas · 2024-11-09T19:29:41Z

docs/src/optimization_packages/evolutionary.md

+function func(x, p=nothing)::Vector{Float64}
+  f1 = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2  # Rosenbrock function
+  f2 = -20.0 * exp(-0.2 * sqrt(0.5 * (x[1]^2 + x[2]^2))) - exp(0.5 * (cos(2π * x[1]) + cos(2π * x[2]))) + exp(1) + 20.0  # Ackley function
+  return [f1, f2]


it's an array now? Is there an in-place form?

ChrisRackauckas · 2024-11-09T19:31:00Z

docs/src/optimization_packages/metaheuristics.md

+    # In this example, we have no constraints
+    gx = [0.0]  # Inequality constraints (not used)
+    hx = [0.0]  # Equality constraints (not used)
+    return [f1, f2], gx, hx


This is now a third API?

Added evolutionary to the package.

updated algorithm call.

Correction of changeing tuple to vector.

ChrisRackauckas · 2024-11-10T04:22:48Z

docs/src/optimization_packages/blackboxoptim.md

+function multi_obj_func(x, p)
+    f1 = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2  # Rosenbrock function
+    f2 = -20.0 * exp(-0.2 * sqrt(0.5 * (x[1]^2 + x[2]^2))) - exp(0.5 * (cos(2π * x[1]) + cos(2π * x[2]))) + exp(1) + 20.0  # Ackley function
+    return (f1, f2)


you didn't actually change it though?

corrected algorithm calls.

Vaibhavdixit02 · 2025-01-16T13:49:10Z

@ParasPuneetSingh please finish this up.

Adding argument mapping for num_dimensions and fitness_scheme.

syntax change for num_dimensions and fitness_scheme passing in solve().

ParasPuneetSingh · 2025-05-16T07:15:06Z

@ChrisRackauckas please review this at the earliest.

ChrisRackauckas · 2025-05-16T15:00:39Z

docs/src/optimization_packages/blackboxoptim.md

+    f2 = -20.0 * exp(-0.2 * sqrt(0.5 * (x[1]^2 + x[2]^2))) - exp(0.5 * (cos(2π * x[1]) + cos(2π * x[2]))) + exp(1) + 20.0  # Ackley function
+    return (f1, f2)
+end
+mof = MultiObjectiveOptimizationFunction(multi_obj_func)


Suggested change

mof = MultiObjectiveOptimizationFunction(multi_obj_func)

mof = MultiObjectiveOptimizationFunction(multi_obj_func, obj_prototype = zeros(2))

ChrisRackauckas · 2025-05-16T15:01:01Z

docs/src/optimization_packages/blackboxoptim.md

+function multi_obj_func(x, p)
+    f1 = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2  # Rosenbrock function
+    f2 = -20.0 * exp(-0.2 * sqrt(0.5 * (x[1]^2 + x[2]^2))) - exp(0.5 * (cos(2π * x[1]) + cos(2π * x[2]))) + exp(1) + 20.0  # Ackley function
+    return (f1, f2)


Show in-place form too?

ChrisRackauckas · 2025-05-16T15:14:12Z

This is not ready. There's a few steps here. First of all, we document the OptimizationFunction interface https://github.com/SciML/SciMLBase.jl/blob/master/src/scimlfunctions.jl#L1913-L2030 . Right now, MultiObjectiveOptimizationFunction's interface is not documented. Let's start there.

https://github.com/SciML/SciMLBase.jl/blob/master/src/scimlfunctions.jl#L2063-L2066

Write a similar docstring that describes the interface on MultiObjectiveOptimizationFunction. The closest interface should be the NonlinearLeastSquares, since NonlinearLeastSquares is a multi-objective optimization that just has a canonical coalesce via coalesce(cost,u,p) = norm(cost,2).

So okay, what should this thing be? It should be as follows. You have an out of place form cost = f(u,p) and an in-place form f!(cost,u,p). You'll have to have a required cost_prototype which is the vector to use for the in-place cost. This means that length(cost) is equal to the number of objectives. isinplace should then be counting for in-place when you have 3 values, thus matching the same setup as NonlinearFunction. You then need to describe the argument structures expected for each of the derivative definitions, which is more or less matching that of NonlinearLeastSquares. And there should be an optional coalesce(cost,u,p) = norm(cost,2) which when provided allows the MOO problem to be solved with a scalar optimization problem under the definition of coalesce(f(u,p).

Once you have that down, we merge, and then let's use that defined interface. You'll need to MultiobjectiveOptimizationFunction(f; cost_prototype = zeros(2)) some definitions, and then the functions here should not require the user pass in the dimension, since that can be known from the function definition itself.

Draft code for Euler's method using Gradient Descent.

Draft-1 tests for the ODE solver-euler's method using gradient descent.

Initial project.toml for ODE package

Updated the code to match the interface structure of OptimizationNLopt.jl

Updated runtests.jl to match the updated OptimizationODE.jl code.

ParasPuneetSingh added 3 commits September 7, 2024 17:16

MOO Docs updated blackboxoptim.md

2646d29

Added documentation for MOO in BBO

Update evolutionary.md

79a297b

MOO docs update.

Update metaheuristics.md

1394ec4

MOO docs update.

Vaibhavdixit02 force-pushed the master branch from 3307b06 to 799f7de Compare September 20, 2024 21:51

Vaibhavdixit02 reviewed Sep 22, 2024

View reviewed changes

docs/src/optimization_packages/blackboxoptim.md Outdated Show resolved Hide resolved

Update docs/src/optimization_packages/blackboxoptim.md

5af0e2b

ParasPuneetSingh added 2 commits September 22, 2024 10:22

Update Project.toml

37aa52b

updated project.toml for the docs.

Update Project.toml

81f5e8a

Added compat for BBO.

Vaibhavdixit02 reviewed Sep 22, 2024

View reviewed changes

docs/src/optimization_packages/blackboxoptim.md Outdated Show resolved Hide resolved

Update docs/src/optimization_packages/blackboxoptim.md

238dcbe

Vaibhavdixit02 reviewed Sep 23, 2024

View reviewed changes

ParasPuneetSingh added 4 commits September 24, 2024 02:06

Update metaheuristics.md

9198cb4

Added required packages for MOO docs.

Update Project.toml

be6388b

added required packages for MOO

Update blackboxoptim.md

5b3a0b5

Corrected function names for MOO docs.

Update evolutionary.md

6a1dceb

Removed unnecessary FowardDiff function.

Vaibhavdixit02 closed this Sep 26, 2024

Vaibhavdixit02 reopened this Sep 26, 2024

Vaibhavdixit02 reviewed Sep 27, 2024

View reviewed changes

docs/src/optimization_packages/metaheuristics.md Outdated Show resolved Hide resolved

Update docs/src/optimization_packages/metaheuristics.md

c9d892f

Vaibhavdixit02 closed this Oct 4, 2024

Vaibhavdixit02 reopened this Oct 4, 2024

Vaibhavdixit02 closed this Oct 26, 2024

Vaibhavdixit02 reopened this Oct 26, 2024

Vaibhavdixit02 reviewed Oct 26, 2024

View reviewed changes

docs/src/optimization_packages/evolutionary.md Outdated Show resolved Hide resolved

Update docs/src/optimization_packages/evolutionary.md

564f53a

Vaibhavdixit02 closed this Oct 28, 2024

Vaibhavdixit02 reopened this Oct 28, 2024

Update metaheuristics.md

b1fe7da

Added the package for the algorithms.

ChrisRackauckas reviewed Nov 9, 2024

View reviewed changes

ParasPuneetSingh added 3 commits November 10, 2024 01:43

Update Project.toml

89abf73

Added evolutionary to the package.

Update metaheuristics.md

5aa1289

updated algorithm call.

Update blackboxoptim.md

fb0181b

Correction of changeing tuple to vector.

ChrisRackauckas reviewed Nov 10, 2024

View reviewed changes

Update metaheuristics.md

6d4c6ba

corrected algorithm calls.

ParasPuneetSingh added 2 commits January 30, 2025 00:05

Update OptimizationBBO.jl

d8fea39

Adding argument mapping for num_dimensions and fitness_scheme.

Update runtests.jl

032e5b9

syntax change for num_dimensions and fitness_scheme passing in solve().

ChrisRackauckas reviewed May 16, 2025

View reviewed changes

ParasPuneetSingh added 5 commits May 18, 2025 12:38

Create OptimizationODE.jl

ee54b25

Draft code for Euler's method using Gradient Descent.

Create runtests.jl

a3b6e70

Draft-1 tests for the ODE solver-euler's method using gradient descent.

Create Project.toml

0129ed0

Initial project.toml for ODE package

Update OptimizationODE.jl

b2c757e

Updated the code to match the interface structure of OptimizationNLopt.jl

Update runtests.jl

dc2ad91

Updated runtests.jl to match the updated OptimizationODE.jl code.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docs for MOO #823

Docs for MOO #823

ParasPuneetSingh commented Sep 20, 2024

Vaibhavdixit02 commented Sep 22, 2024

Vaibhavdixit02 Sep 23, 2024

Vaibhavdixit02 Sep 23, 2024

ParasPuneetSingh Sep 23, 2024

Vaibhavdixit02 Sep 23, 2024

Vaibhavdixit02 Sep 25, 2024

ParasPuneetSingh Sep 25, 2024

Vaibhavdixit02 Sep 23, 2024

Vaibhavdixit02 commented Sep 23, 2024

Vaibhavdixit02 commented Sep 26, 2024

ChrisRackauckas Nov 9, 2024

ParasPuneetSingh Nov 9, 2024

ChrisRackauckas Nov 9, 2024

ChrisRackauckas May 16, 2025

ChrisRackauckas May 16, 2025

ChrisRackauckas Nov 9, 2024

ChrisRackauckas Nov 9, 2024

ChrisRackauckas Nov 10, 2024

Vaibhavdixit02 commented Jan 16, 2025

ParasPuneetSingh commented May 16, 2025

ChrisRackauckas May 16, 2025

ChrisRackauckas May 16, 2025

ChrisRackauckas commented May 16, 2025

	sol = solve(prob_2, opt, NumDimensions=2, FitnessScheme=ParetoFitnessScheme{2}(is_minimizing=true))
	sol = solve(prob, opt, NumDimensions=2, FitnessScheme=ParetoFitnessScheme{2}(is_minimizing=true))

	mof = MultiObjectiveOptimizationFunction(multi_obj_func)
	mof = MultiObjectiveOptimizationFunction(multi_obj_func, obj_prototype = zeros(2))

Docs for MOO #823

Are you sure you want to change the base?

Docs for MOO #823

Conversation

ParasPuneetSingh commented Sep 20, 2024

Checklist

Additional context

Vaibhavdixit02 commented Sep 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vaibhavdixit02 commented Sep 23, 2024

Vaibhavdixit02 commented Sep 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vaibhavdixit02 commented Jan 16, 2025

ParasPuneetSingh commented May 16, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChrisRackauckas commented May 16, 2025