SNOBFIT randomly fails

[ilario]

• Operating System: Arch Linux
• Python version: 3.10.8
• summit version used: 0.8.8

Description

It did not use to happen, likely it started due to the new version of everything that I am using.

When I ask for suggested_experiments to SNOBFIT, it some times gives me the answers and some times it gives an error, and this happens repeating exactly this same code:

strategySNOBFIT = SNOBFIT(domain_constr)
strategySNOBFIT.reset()
strategySNOBFIT.suggest_experiments(2, prev_res=prev_res_extreme)

the more experiments I require, the more likely it is to fail.

This is the error:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[75], line 2
      1 strategySNOBFIT.reset()
----> 2 strategySNOBFIT.suggest_experiments(2, prev_res=prev_res_extreme)

File /home/ilario/.venv/lib/python3.10/site-packages/summit/strategies/snobfit.py:142, in SNOBFIT.suggest_experiments(self, num_experiments, prev_res, **kwargs)
    140 while not valid_next_experiments and c_iter < inner_iter_tol:
    141     valid_next_experiments = False
--> 142     next_experiments, xbest, fbest, param = self._inner_suggest_experiments(
    143         num_experiments=num_experiments,
    144         prev_res=prev_res,
    145         prev_param=inner_prev_param,
    146     )
    147     # Invalid experiments hidden from data returned to user but stored internally elswehere
    148     invalid_experiments = next_experiments.loc[
    149         next_experiments[("constraint", "DATA")] == False
    150     ]

File /home/ilario/.venv/lib/python3.10/site-packages/summit/strategies/snobfit.py:337, in SNOBFIT._inner_suggest_experiments(self, num_experiments, prev_res, prev_param)
    334 dx = (bounds[:, 1] - bounds[:, 0]) * self._dx_dim
    336 # Run SNOBFIT for one iteration
--> 337 request, xbest, fbest, param = self.snobfit(x0, y0, config, dx, prev_param)
    339 # Generate DataSet object with variable values of next experiments
    340 next_experiments = {}

File /home/ilario/.venv/lib/python3.10/site-packages/summit/strategies/snobfit.py:609, in SNOBFIT.snobfit(self, x, f, config, dx, prev_param)
    607     fnan = numpy.concatenate((fnan, ind.flatten()))
    608 if fnan.size > 0:
--> 609     f = snobnan(fnan, f, near, inew)
    611 fbest, jbest = min_(f[:, 0])
    612 xbest = x[jbest]

File /home/ilario/.venv/lib/python3.10/site-packages/SQSnobFit/_snobnan.py:78, in snobnan(fnan, f, near, inew)
     75          ind1 = numpy.concatenate((ind1, [i]), 0)
     77 if ind1.size > 0:
---> 78     ind = numpy.delete(ind, ind1, 0)
     80 if ind.size <= 0:
     81     f[l,0] = fmx + 1.e-3*(fmx-fmn)

File <__array_function__ internals>:180, in delete(*args, **kwargs)

File /home/ilario/.venv/lib/python3.10/site-packages/numpy/lib/function_base.py:5128, in delete(arr, obj, axis)
   5126 else:
   5127     keep = ones(N, dtype=bool)
-> 5128     keep[obj,] = False
   5130 slobj[axis] = keep
   5131 new = arr[tuple(slobj)]

IndexError: arrays used as indices must be of integer (or boolean) type

What I Did

python -m venv .venv     
source .venv/bin/activate
pip install -U summit

This installed these versions of the Python packages:

GPy-1.10.0 SQCommon-0.3.2 SQSnobFit-0.4.5 attrs-22.1.0 autograd-1.5 botorch-0.7.3 cma-2.7.0 contourpy-1.0.6 cycler-0.11.0 cython-0.29.32 decorator-5.1.1 fastprogress-0.2.6 fonttools-4.38.0 future-0.18.2 gpyopt-1.2.6 gpytorch-1.9.0 h5py-3.7.0 iniconfig-1.1.1 joblib-1.2.0 kiwisolver-1.4.4 linear-operator-0.2.0 llvmlite-0.38.1 matplotlib-3.6.2 multipledispatch-0.6.0 numba-0.55.2 numpy-1.22.4 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 opt-einsum-3.3.0 packaging-21.3 pandas-1.5.2 paramz-0.9.5 pillow-9.3.0 pluggy-1.0.0 py-1.11.0 pymoo-0.4.2.2 pyparsing-3.0.9 pyrff-2.0.2 pyro-api-0.1.2 pyro-ppl-1.8.3 pytest-6.2.5 python-dateutil-2.8.2 pytz-2022.6 scikit-learn-1.1.3 scipy-1.9.3 six-1.16.0 skorch-0.9.0 summit-0.8.8 tabulate-0.9.0 threadpoolctl-3.1.0 toml-0.10.2 torch-1.13.0 tqdm-4.64.1 typing-extensions-4.4.0 wheel-0.38.4

Open VSCodium and indicate to use .venv as an environment.

Run some summit stuff.

[marcosfelt]

I know we've had this problem historically, but I unfortunately don't remember the details since I am not an expert on SNOBFIT. If you can give an example for me to reproduce the bug, I can try to look into it when I have time.