Skip to content

Latest commit

 

History

History
460 lines (460 loc) · 15.5 KB

synthetic-easy.md

File metadata and controls

460 lines (460 loc) · 15.5 KB

The overall comparison on optimization performance among all baselines in our Repo. For each problem in Synthetic easy suites, we test each algorithm for $51$ independent runs and average the test results for presentation. ''Obj'' (smaller is better) indicates the final global best value. ''Gap'' (smaller is better) is the optimization gap away from the SOTA optimizer, which is CMA-ES algorithm in this paper. ''FEs'' indicates how many function evaluations an optimizer takes to get the ''Obj''. Under the consideration of runtime, we assign BayesianOptimizer 100 FEs. For RNN-OI, we also set the maxFEs to 100 respecting the original paper. Therefore, their ''FEs'' and ''Gap'' are not comparable.

Problem Schaffers_high_cond Weierstrass Lunacek_bi_Rastrigin Linear_Slope Schwefel Sphere
metric Obj Gap FEs Obj Gap FEs Obj Gap FEs Obj Gap FEs Obj Gap FEs Obj Gap FEs
DE_DDQN 3.852e-3
(1.485e-2)		 0.000 2.000e+4
(0.000e+0)		 9.312e+0
(2.418e+0)		 0.938 2.000e+4
(0.000e+0)		 4.120e+1
(6.536e+0)		 0.306 2.000e+4
(0.000e+0)		 3.181e-2
(2.249e-1)		 0.001 9.682e+3
(1.987e+3)		 1.699e+0
(3.369e-1)		 0.001 2.000e+4
(0.000e+0)		 9.038e-7
(6.332e-6)		 0.000 1.071e+4
(1.354e+3)
DEDQN 2.866e+1
(5.593e+0)		 1.965 2.000e+4
(0.000e+0)		 2.158e+1
(5.262e+0)		 2.702 2.000e+4
(0.000e+0)		 1.659e+2
(1.747e+1)		 1.659 2.000e+4
(0.000e+0)		 4.782e+1
(1.480e+1)		 0.920 2.000e+4
(0.000e+0)		 5.633e+3
(2.673e+3)		 16.140 2.000e+4
(0.000e+0)		 2.716e+1
(7.467e+0)		 2.367 2.000e+4
(0.000e+0)
RL_HPSDE 4.995e+0
(1.942e+0)		 0.342 2.025e+4
(0.000e+0)		 1.092e+1
(2.543e+0)		 1.170 2.025e+4
(0.000e+0)		 6.806e+1
(6.534e+0)		 0.598 2.025e+4
(0.000e+0)		 0.000e+0
(0.000e+0)		 0.000 6.039e+3
(1.136e+3)		 2.422e+0
(2.279e-1)		 0.003 2.025e+4
(0.000e+0)		 1.017e-1
(5.665e-2)		 0.009 2.025e+4
(0.000e+0)
LDE 1.768e-1
(1.175e-1)		 0.012 2.000e+4
(0.000e+0)		 4.679e+0
(1.092e+0)		 0.273 2.000e+4
(0.000e+0)		 3.789e+1
(5.600e+0)		 0.270 2.000e+4
(0.000e+0)		 1.057e-8
(6.120e-9)		 0.000 1.978e+4
(2.304e+2)		 1.243e+0
(1.548e-1)		 -0.001 2.000e+4
(0.000e+0)		 7.837e-9
(1.322e-9)		 0.000 8.523e+3
(2.488e+2)
QLPSO 4.108e+0
(3.331e+0)		 0.282 2.000e+4
(0.000e+0)		 6.339e+0
(4.667e+0)		 0.511 2.000e+4
(0.000e+0)		 3.752e+1
(9.055e+0)		 0.266 2.000e+4
(0.000e+0)		 2.607e+1
(2.213e+1)		 0.502 1.613e+4
(7.843e+3)		 1.594e+0
(3.814e-1)		 0.000 2.000e+4
(0.000e+0)		 2.757e+0
(3.715e+0)		 0.240 2.000e+4
(0.000e+0)
RLEPSO 4.534e+0
(3.453e+0)		 0.311 2.000e+4
(5.539e-1)		 2.664e+0
(1.793e+0)		 -0.017 2.000e+4
(6.360e-1)		 2.617e+1
(8.404e+0)		 0.143 2.000e+4
(8.254e-1)		 0.000e+0
(0.000e+0)		 0.000 2.669e+3
(6.557e+2)		 1.648e+0
(3.037e-1)		 0.001 2.001e+4
(1.288e+1)		 8.306e-9
(1.031e-9)		 0.000 7.102e+3
(1.206e+3)
RL_PSO 1.191e+1
(3.738e+0)		 0.817 2.000e+4
(0.000e+0)		 1.476e+1
(3.209e+0)		 1.721 2.000e+4
(0.000e+0)		 6.723e+1
(9.598e+0)		 0.589 2.000e+4
(0.000e+0)		 5.468e+1
(1.212e+1)		 1.052 2.000e+4
(0.000e+0)		 2.890e+0
(2.406e-1)		 0.004 2.000e+4
(0.000e+0)		 2.384e+0
(1.219e+0)		 0.208 2.000e+4
(0.000e+0)
DE 3.017e-2
(5.529e-2)		 0.002 2.000e+4
(0.000e+0)		 8.707e+0
(1.814e+0)		 0.851 2.000e+4
(0.000e+0)		 4.148e+1
(4.506e+0)		 0.309 2.000e+4
(0.000e+0)		 0.000e+0
(0.000e+0)		 0.000 1.449e+3
(3.374e+2)		 1.779e-1
(1.777e-1)		 -0.004 1.846e+4
(2.800e+3)		 8.179e-9
(1.301e-9)		 0.000 4.579e+3
(1.884e+2)
JDE21 9.219e-1
(1.302e+0)		 0.063 2.001e+4
(0.000e+0)		 4.948e+0
(2.253e+0)		 0.311 2.001e+4
(0.000e+0)		 3.782e+1
(7.600e+0)		 0.270 2.001e+4
(0.000e+0)		 1.492e-8
(2.313e-8)		 0.000 1.684e+4
(1.615e+3)		 4.500e-1
(2.469e-1)		 -0.003 2.001e+4
(0.000e+0)		 5.376e-9
(2.571e-9)		 -0.000 6.005e+3
(9.560e+2)
MadDE 9.613e-1
(3.516e-1)		 0.066 2.000e+4
(0.000e+0)		 2.557e+0
(8.181e-1)		 -0.032 2.000e+4
(0.000e+0)		 4.152e+1
(5.500e+0)		 0.310 2.000e+4
(0.000e+0)		 0.000e+0
(0.000e+0)		 0.000 5.950e+3
(5.298e+2)		 8.503e-1
(1.675e-1)		 -0.002 2.000e+4
(0.000e+0)		 8.102e-9
(1.456e-9)		 0.000 1.946e+4
(1.506e+2)
NL_SHADE_LBC 9.342e-2
(6.006e-2)		 0.006 2.000e+4
(0.000e+0)		 3.008e+0
(1.011e+0)		 0.032 2.000e+4
(0.000e+0)		 3.359e+1
(6.153e+0)		 0.224 2.000e+4
(0.000e+0)		 0.000e+0
(0.000e+0)		 0.000 7.745e+3
(1.428e+3)		 1.830e-1
(1.372e-1)		 -0.004 2.000e+4
(0.000e+0)		 8.113e-9
(1.492e-9)		 0.000 1.443e+4
(8.883e+1)
PSO 5.863e+0
(1.234e+0)		 0.402 2.000e+4
(0.000e+0)		 9.250e+0
(1.949e+0)		 0.930 2.000e+4
(0.000e+0)		 6.097e+1
(7.073e+0)		 0.521 2.000e+4
(0.000e+0)		 1.479e+0
(4.123e+0)		 0.028 2.639e+3
(6.340e+3)		 2.701e+0
(2.142e-1)		 0.004 2.000e+4
(0.000e+0)		 1.832e+0
(7.114e-1)		 0.160 2.000e+4
(0.000e+0)
GL_PSO 2.149e-1
(1.471e-1)		 0.015 2.000e+4
(0.000e+0)		 4.710e+0
(3.824e+0)		 0.277 2.000e+4
(0.000e+0)		 4.391e+1
(5.309e+0)		 0.336 2.000e+4
(0.000e+0)		 4.473e-6
(1.596e-6)		 0.000 2.000e+4
(0.000e+0)		 1.133e+0
(5.111e-1)		 -0.001 2.000e+4
(0.000e+0)		 1.336e-6
(8.689e-7)		 0.000 2.000e+4
(0.000e+0)
sDMS_PSO 1.030e+1
(3.491e+0)		 0.706 2.010e+4
(0.000e+0)		 4.945e+0
(1.591e+0)		 0.311 2.010e+4
(0.000e+0)		 5.092e+1
(8.540e+0)		 0.412 2.010e+4
(0.000e+0)		 2.136e+0
(4.018e+0)		 0.041 2.010e+4
(0.000e+0)		 2.240e+0
(2.694e-1)		 0.002 2.010e+4
(0.000e+0)		 1.455e+0
(6.409e-1)		 0.127 2.010e+4
(0.000e+0)
SAHLPSO 1.196e+1
(6.608e+0)		 0.820 2.000e+4
(0.000e+0)		 8.623e+0
(4.808e+0)		 0.839 2.000e+4
(0.000e+0)		 8.963e+1
(1.936e+1)		 0.832 2.000e+4
(0.000e+0)		 0.000e+0
(0.000e+0)		 0.000 1.408e+3
(2.104e+3)		 1.898e+1
(3.977e+1)		 0.050 2.000e+4
(0.000e+0)		 5.000e+0
(2.425e+0)		 0.436 2.000e+4
(0.000e+0)
CMAES 7.578e-4
(3.335e-3)		 0.000 1.512e+4
(3.625e+3)		 2.782e+0
(5.052e-1)		 0.000 2.000e+4
(0.000e+0)		 1.296e+1
(6.876e+0)		 0.000 2.000e+4
(0.000e+0)		 0.000e+0
(0.000e+0)		 0.000 4.549e+2
(5.536e+1)		 1.434e+0
(3.287e-1)		 0.000 2.000e+4
(0.000e+0)		 7.678e-9
(1.619e-9)		 0.000 4.760e+3
(1.968e+2)
Random_search 1.458e+1
(2.649e+0)		 1.000 2.000e+4
(0.000e+0)		 9.740e+0
(1.960e+0)		 1.000 2.000e+4
(0.000e+0)		 1.051e+2
(1.086e+1)		 1.000 2.000e+4
(0.000e+0)		 5.197e+1
(6.343e+0)		 1.000 2.000e+4
(0.000e+0)		 3.503e+2
(2.907e+2)		 1.000 2.000e+4
(0.000e+0)		 1.147e+1
(2.506e+0)		 1.000 2.000e+4
(0.000e+0)
BayesianOptimizer 3.541e+1
(8.821e+0)		 2.428 1.000e+2
(0.000e+0)		 2.552e+1
(6.897e+0)		 3.268 1.000e+2
(0.000e+0)		 8.425e+1
(1.208e+1)		 0.773 1.000e+2
(0.000e+0)		 2.088e+1
(1.832e+1)		 0.402 6.800e+1
(4.013e+1)		 3.355e+2
(1.252e+3)		 0.958 1.000e+2
(0.000e+0)		 7.805e-2
(6.928e-2)		 0.007 1.000e+2
(0.000e+0)
RNN-OI 5.550e+1
(1.421e-14)		 3.806 1.000e+2
(0.000e+0)		 5.917e+1
(0.000e+0)		 8.105 1.000e+2
(0.000e+0)		 2.109e+2
(2.842e-14)		 2.147 1.000e+2
(0.000e+0)		 2.299e+2
(0.000e+0)		 4.424 1.000e+2
(0.000e+0)		 2.552e+4
(0.000e+0)		 73.135 1.000e+2
(0.000e+0)		 9.372e+1
(1.421e-14)		 8.169 1.000e+2
(0.000e+0)