[GENERAL SUPPORT]: Initial Sobol Points Selection for 50-Dimensional Parameter Search in Vanilla BO

[vinaysaini94]

Question

Hello,

I am working on an optimization problem with a 50-dimensional search space, where each parameter ranges from [1,100]. I am using Vanilla Bayesian Optimization (BO) to search for the optimal parameter values that maximize throughput.

Given the high-dimensional nature of my problem, I would like to know:

How many initial Sobol points should I sample to provide a good starting set for the BO optimizer?
Are there any general heuristics or best practices for choosing the number of initial points in such a high-dimensional setting?
Any insights or references would be greatly appreciated!

Thanks

Please provide any relevant code snippet if applicable.

Code of Conduct

• I agree to follow this Ax's Code of Conduct

[ltiao]

Hey there!

Generally, it can make sense to sample initial Sobol points proportional to the dimensionality. However, this can become prohibitively expensive in high-dimensional cases like yours if your blackbox function evaluation is costly.

In short, it depends on your total function evaluation budget, which in turn depends on the cost of function evaluations:

• If function evaluations are cheap, you might consider using some multiple of the dimensionality as the number of initial points (although beware that the model inference cost scales cubically and might become prohibitively slow)
• If evaluations are expensive, you might consider taking either some multiple 0 < k < 1 of the dimensionality or something like 10% of your total evaluation budget, whichever is smaller, i.e. min(k*d, 0.1n)

Good luck with your optimization problem!