Create an Experiment

An Experiment is a set of parameters and a metric that you would like to maximize.

Here, we will show code snippets and go over the two most important fields you need to get your experiment up and running. You can also view the full experiment create api reference for a deeper dive.

Find your SigOpt API token on the API tokens page.

Basic Example

Every experiment must have a name and at least one parameter. This example creates an experiment named “Classifier Accuracy” that has one double parameter ranging from 0.0 to 50.0. You can optionally put the experiment in a Project. The sigopt-examples project will be used for this example.

from sigopt import Connection

conn = Connection(client_token="SIGOPT_API_TOKEN")
conn.set_api_url("https://api.sigopt.com")

experiment = conn.experiments().create(
  name="Classifier Accuracy",
  parameters=[
    dict(
      name="gamma",
      bounds=dict(
        min=0,
        max=50
        ),
      type="double"
      )
    ],
  metrics=[
    dict(
      name="Accuracy"
      )
    ],
  observation_budget=30,
  parallel_bandwidth=1,
  project="sigopt-examples"
  )

Name

Choose a memorable and descriptive name for this experiment.

Parameters

A Parameter can be an int, a double, or a categorical.

Double

A double parameter can take on any value between the minimum and maximum values specified by the bounds.

{
  "name": "gamma",
  "type": "double",
  "bounds": {
    "min": 0.001,
    "max": 1.0
  }
}

Integer

An int parameter is very similar to a double parameter, but it can only take on integer values.

{
  "name": "degree",
  "type": "int",
  "bounds": {
    "min": 1,
    "max": 15
  }
}

Categorical

A categorical parameter provides a list of options to choose from. A categorical parameter is analogous to an enum type in many programming languages.

{
  "name": "kernel",
  "type": "categorical",
  "categorical_values": ["rbf", "poly", "sigmoid"]
}

Observation Budget

We suggest setting an Observation Budget for each experiment, which tells SigOpt how many observations you expect to submit (how many distinct models you can evaluate during the optimization). Learn more about this concept here.

Parallel Bandwidth

Many will find it useful to be able to optimize a model across a cluster of machines. In this situation, SigOpt can serve as a distributed scheduler that each worker connects to for its next parameter configuration to evaluate. When running an experiment across parallel workers, set the parallel_bandwidth so SigOpt can provide better suggestions. Learn more about running parallel experiments here.

Next Steps

When you've created an experiment that you're satisfied with, you're ready to start optimizing!