Set Observation Budget

When you are creating an experiment there is a keyword argument observation_budget you can optionally set.

Setting an observation budget allows our algorithm to better trade off between exploration and exploitation in order to effectively seek out a global maximum in your objective function. We expect observation_budget to be the minimum number of observations you will report, and have seen convergence in many of experiments when the budget is set to 10-20 times the number of parameters in the experiment.

You may continue to keep reporting observations beyond the budget and we will continue to provide suggestions.

Examples

Below is a basic example of how to include an observation budget.

# Run `pip install sigopt` to download the python API client
from sigopt import Connection
from sigopt.examples import franke_function

conn = Connection(client_token=SIGOPT_API_TOKEN)

experiment = conn.experiments().create(
    name='Franke Optimization (Python)',
    project='sigopt-examples',
    metrics=[dict(name='function_value', objective='maximize')],
    parameters=[
        dict(name='x', type='double', bounds=dict(min=0.0, max=1.0)),
        dict(name='y', type='double', bounds=dict(min=0.0, max=1.0)),
    ],
    observation_budget=30
)

print("Created experiment: https://app.sigopt.com/experiment/" + experiment.id)

This sample code provides an example of how observation budget can be used. For the associated blog post, see here.

Notes

Default observation budget

When an observation budget is not reported, we anticipate at least 10 times the number of parameters in your experiment.

Parallelism

We do not provide an official recommendation for an observation budget of an experiment run with a high degree of parallelism. In general, the minimum number of observations will grow beyond the recommended 10-20 times the number of parameters as the degree of parallelism increases.

Updates

Observation budgets can be updated during the experiment.