[Question] Why is min_trials = 1 by default?

[becktepe]

I am wondering why the default value of the min_trials parameter of the ConfigSelector is 1 for multi-fidelity optimization.

Let's consider the following Hyperband brackets:

--------------------------------------------------------------------------------
Stage 0
  #Configs: [  64,   16,    4,    1]
  Budgets:  [  16,   62,  250, 1000]
--------------------------------------------------------------------------------
Stage 1
  #Configs: [  22,    5,    1]
  Budgets:  [  62,  250, 1000]
--------------------------------------------------------------------------------
Stage 2
  #Configs: [   8,    2]
  Budgets:  [ 250, 1000]
--------------------------------------------------------------------------------
Stage 3
  #Configs: [   4]
  Budgets:  [1000]
--------------------------------------------------------------------------------

In Stage 0, when we have evaluated all configurations on budget 16, the surrogate model uses all 64 datapoints. However, when we transition to budget 62, i.e., evaluated the first configuration of this budget, with min_trials = 1 we fit our surrogate model only on this datapoint and successively use it to sample the following 15 datapoints.

I don't understand, why we should set min_trials = 1 in this case as we would throw away a lot of datapoints when transitioning to a higher fidelity.

[mwever]

In case you are referring to BOHB here, in Stage 0 after evaluating all configurations on budget 16 there should be no more sampling in that stage but only in the subsequent stage, i.e., Stage 1. However, as already 16 configurations have been evaluated in Stage 0, this should actually be fine. So on the one hand, what you describe, should be a non-issue. But setting min_trials = 1 could still be problematic in Stage 0 if it really means that only one or two random configurations are sampled before fitting the surrogate.

[becktepe]

Ah I see, this was a misunderstanding from my side. Thank you!