[GENERAL SUPPORT]: Why Does Computation Time First Decrease, Then Increase with Batch Size in based on GPEI?

[vinaysaini94]

Question

We are running an experiment on a mechanical system to determine the best parameter values while minimizing computation time. Our experiment involves the latest dimensional scaled vanilla BO-based GPEI batch trials, where we vary the batch size per trial while keeping the total number of iterations constant at 100.

Setup:

Batch sizes tested: 1, 2, 3, 4, and 5
The mechanical system only allows sequential execution due to design constraints
Final parameter values across all batch sizes are comparable
Observations on Computation Time:

Batch size 1: 100.1 min
Batch size 2: 99.9 min
Batch size 3: 98.21 min (lowest)
Batch size 4: 110.51 min (increase)
Batch size 5: 120.21 min (highest)
We noticed that computation time initially decreased from batch size 1 to 3 but then increased significantly from batch size 4 to 5. Given that all trials execute sequentially on the mechanical system, we are trying to understand the reason behind this pattern.

Question:
Why does the computation time first decrease from batch size 1 to 3 and then increase from batch size 4 to 5, despite comparable final parameter values? Could this be due to system-specific overhead, memory constraints, or some inefficiency in batch processing?

Any insights or explanations would be greatly appreciated!

Please provide any relevant code snippet if applicable.

Code of Conduct

• I agree to follow this Ax's Code of Conduct

[Balandat]

What's the variance in these observations if you were to repeatedly run the optimization? It seems to me that this will likely be high enough that the observation here is subject to a lot of noise you can't really conclude anything from these numbers without running more replications to estimate the average generation time.

[vinaysaini94]

@Balandat
Yes, I have taken repeated measurements.

Here are the details (values in brackets represent the standard deviation of six measurements for each batch size):

Batch size 1: 100.1 (±5.74)
Batch size 2: 99.9 (±7.95)
Batch size 3: 98.21 (±4.58)
Batch size 4: 110.51 (±7.24)
Batch size 5: 120.21 (±7.83)