PSD-NOMAD crash due to "std::out_of_range"

[idwwwoqq808]

I was repeating an optimization experiment using PSD-NOMAD. It repeated for a few times without any problem and then crashed with "std::out_of_range" error. Is there a way to solve it?
This problem need to repeat execution several times to trigger.
File Content
nomad4_param.txt

DIMENSION 60
BB_EXE "$python3 obj_func.py"
BB_OUTPUT_TYPE  OBJ
LOWER_BOUND ( 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 )
UPPER_BOUND ( 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 100 100 100 100 100 100 100 100 100 100 )
BB_INPUT_TYPE ( B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B I I I I I I I I I I )
HISTORY_FILE nomad4_history.txt
MAX_BB_EVAL 248
DISPLAY_DEGREE 2
PSD_MADS_OPTIMIZATION true
PSD_MADS_NB_SUBPROBLEM 8
CACHE_FILE nomad4.cache
DIRECTION_TYPE ORTHO 2N

nomad4.cache (externally generated)

CACHE_HITS 1
BB_OUTPUT_TYPE OBJ
(   1  1  1  0  0  1  1  1  1  0  1  1  0  1  1  1  1  1  1  1  0  0  1  0  0  0  1  0  0  0  1  1  1  1  0  1  1  1  1  0  0  0  0  1  0  1  1  0  0  0 -50 14 -19 -37 52  8 -49 79 23 91) EVAL_OK ( 1.3862614188120563 )

Code
obj_func.py

import numpy as np
import sys

sys.path.insert(0,'/home/idwwwoqq808/workspace/optimizer/EnsOpt')
from test_func  import Test_Function2
obj_func = Test_Function2()
x_file = sys.argv[1]
x_vec = np.loadtxt(x_file)
fx = obj_func(x_vec)
print(fx)

test_func.py

import numpy as np
import time, copy, sys, random
import ioh #Github: https://iohprofiler.github.io/IOHproblem/PBO
def GriewankConvex(x:np.ndarray):
    term1 = np.power(x,2)
    term3 = copy.deepcopy(term1)
    term1 = term1.sum()/4e3

    term2 = np.cos(x/np.sqrt(np.arange(1, 1+x.size)))
    term2 = term2.cumprod()[-1]

    term3 = np.sqrt(term3.sum()) / 33.0

    fx = term1 - term2 +1 + term3
    return fx
class Test_Function2:
    _cont_granu = 1e-2

    def __init__(self):
        self.dim = 60
        self.sub_idx = [ np.arange(0,30), np.arange(10,40), np.arange(20,50), np.arange(50,self.dim) ]
        inv_granu = np.rint(1/self._cont_granu).astype(int)
        self.lb = np.hstack((np.zeros(50), np.ones(10)*(-inv_granu) )).astype(int)
        self.ub = np.hstack((np.ones(50), np.ones(10)*inv_granu)).astype(int)
        self.var_type = ['categorical']*50 + ['discrete']*10
        assert self.lb.size == self.dim
        assert self.ub.size == self.dim

    def __call__(self,X):
        if not (type(X) is np.ndarray):
            X = np.array(X)

        if 1==len(X.shape):
            try:
                Y = self.ComboFunc(X)
            except Exception as e:
                print(X)
                raise e
        else:
            Y = np.zeros(X.shape[0])
            for i in range(X.shape[0]):
                Y[i] = self.ComboFunc(X[i])
        return Y

    def ComboFunc(self, x:np.ndarray):
        x_part0, x_part1, x_part2, x_part3 = self.SplitInputVector(x)
        func0 = ioh.get_problem(
            "LABS", # Objective is to maximize
            instance = 1,
            dimension = 30,
            problem_class = ioh.ProblemClass.INTEGER
        )
        term0 = -1/3 * (func0(x_part0.tolist()) - 3.3)
        func1 = ioh.get_problem(
            "OneMaxRuggedness2", # Objective is to maximize
            instance = 0,
            dimension = 30,
            problem_class = ioh.ProblemClass.INTEGER
        )
        term1 = 0 -1/17 * (func1(x_part1.tolist()) -23)
        func2 = ioh.get_problem(
            "MIS", # Objective is to maximize
            instance = 0,
            dimension = 30,
            problem_class = ioh.ProblemClass.INTEGER
        )
        term2 = -1e-3 * (func2(x_part2.tolist()) -10)
        func3 = GriewankConvex
        term3 = 1/3 * func3(x_part3)
        return term0 + term1 + term2 + term3

    def SplitInputVector(self, x:np.ndarray) -> Union[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
        x_part0 = np.rint(x[self.sub_idx[0]]).astype(int)
        x_part1 = np.rint(x[self.sub_idx[1]]).astype(int)
        x_part2 = np.rint(x[self.sub_idx[2]]).astype(int)
        x_part3 = np.rint(x[self.sub_idx[3]]) * self._cont_granu
        return x_part0, x_part1, x_part2, x_part3

Terminal Output

All variables are granular. MAX_EVAL is set to 1000000 to prevent algorithm from circling around best solution indefinitely
Warning: Dimension 60 is greater than (or equal to) 50. Models are disabled.
Version 4.3.1 
 
NOMAD 4 has been created by 
    Viviane Rochon Montplaisir 
    Christophe Tribes 

The copyright of NOMAD 4 is owned by 
    Charles Audet 
    Sebastien Le Digabel 
    Viviane Rochon Montplaisir 
    Christophe Tribes 

NOMAD 4 has been funded by Rio Tinto, Hydro-Québec, Huawei-Canada, 
 NSERC (Natural Sciences and Engineering Research Council of Canada), 
 InnovÉÉ (Innovation en Énergie Électrique) and 
 IVADO (The Institute for Data Valorization) 

Download  : https://www.gerad.ca/nomad or 
 https://github.com/bbopt/nomad 
License   : see LICENSE file 
User guide: https://nomad-4-user-guide.readthedocs.io 
Help      : run nomad -h KEYWORD on the command line 
Examples  : see 'examples' directory 

Please report bugs to nomad@gerad.ca or 
create an issue at https://github.com/bbopt/nomad 


terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted (core dumped)

Environment
Python version: 3.10.6
OS: Ubuntu 22.04

[TianningGao]

Forgot to mention that PSD-MADS is called from python:

# already has "$NOMAD_HOME/bin" in "PATH" environment variable.
import subprocess, sys
cmd = 'nomad nomad4_param.txt'
tmp = subprocess.run(cmd, stdout=sys.stdout, stderr=sys.stderr, shell=True)

[ctribes]

I see that std::map is the reason the out_of_range exception was triggered.
In the release version we use map for accessing different of Eval values (BB and MODEL) for a given EvaluationPoint.
Could you do the same experiment without using quadratic MODEL to see if there is the same problem ?
DIRECTION_TYPE ORTHO 2N
QUAD_MODEL_SEARCH no
EVAL_QUEUE_SORT DIR_LAST_SUCCESS

[ctribes]

On my side I will take some time to reproduce the problem with the files you shared. Thanks

[TianningGao]

OK, I will try it.
Just being curious, in the terminal output there is a warning saying that model is disabled due to high dimensions. Does PSD-MADS still use models even if this warning occurs?

[ctribes]

In fact, quad models are disabled only for the search step of Mads. Other use of models are not disabled.

I am really not sure this is the cause of the exception you have. I will dig into your example.

[ctribes]

I did some tests on your example (ran ~ 10 times) on OSX and Linux (both gcc) but I did not get the out_of_range exception.

You can increase the display degree to have more information prior to the crash, maybe this can help to figure out what is going on.

[TianningGao]

I tried adding the parameters you mentioned but it still crashed after repeating the experiment 29 times. I will increase display degree to see what NOMAD outputs.

[ctribes]

PSD Mads have been revisited. So far (version 4.5.0), the last tests show no crash.