Experiment Create

https://api.sigopt.com/v1/experiments

Creates a new Experiment.

Request Method: POST

Parameters

NameTypeRequired?Description
namestringYA user-specified name for this experiment.
parametersarray<Parameter>YAn array of Parameter objects.
conditionalsarray<Conditional>NSee conditionals.
metadataMetadataNOptional user-provided object. See Using Metadata for more information.
metricsarray<Metric>NAn array of Metric objects for simultaneous maximization. If the array is of length one, the standard maximization problem is conducted. This array can have no more than 2 entries.
num_solutionsintNThe number of (diverse) solutions SigOpt will search for. This feature is only available for special plans, and does not need to be set unless the desired number of solutions is greater than 1. An observation budget is required if the number of solutions is greater than 1. No categorical variables are allowed in multiple solution experiments.
observation_budgetintNThe number of Observations you plan to create for this experiment. This is required when the length of metrics is greater than 1, and optional for a single metric experiment. Deviating from this value, especially by failing to reach it, may result in suboptimal performance for your experiment.
parallel_bandwidthintNThe number of simultaneously open Suggestions you plan to maintain during this experiment. The default value for this is 1, i.e., a sequential experiment. The maximum value for this is dependent on your plan. This field is optional, but setting it correctly may improve performance.
projectstringNThe id of the project to put this experiment in.
typestringNA type for this experiment. Used for experimental and alpha features only.

Deprecated Parameters

These parameters should no longer be used because there are better alternatives.

NameTypeRequired?Description
metricMetricNInformation about the metric that this experiment is optimizing.

Response

Experiment object.

Examples

One double parameter

experiment = conn.experiments().create(
  name="Classifier Accuracy",
  parameters=[
    dict(
      name="gamma",
      bounds=dict(
        min=0.001,
        max=1
        ),
      type="double"
      )
    ],
  observation_budget=20,
  parallel_bandwidth=1
  )
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1560895206,
  "development": false,
  "id": "2",
  "linear_constraints": [],
  "metadata": null,
  "metric": {
    "name": null,
    "object": "metric",
    "objective": "maximize",
    "value_baseline": null
  },
  "metrics": [
    {
      "name": null,
      "object": "metric",
      "objective": "maximize",
      "value_baseline": null
    }
  ],
  "name": "Classifier Accuracy",
  "num_solutions": null,
  "object": "experiment",
  "observation_budget": 20,
  "parallel_bandwidth": 1,
  "parameters": [
    {
      "bounds": {
        "max": 1,
        "min": 0.001,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "gamma",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "double"
    }
  ],
  "progress": {
    "best_observation": null,
    "first_observation": null,
    "last_observation": null,
    "object": "progress",
    "observation_budget_consumed": 0,
    "observation_count": 0
  },
  "project": "classification-models",
  "state": "active",
  "type": "offline",
  "updated": 1560895206,
  "user": null
}

One integer parameter

experiment = conn.experiments().create(
  name="Classifier Accuracy",
  parameters=[
    dict(
      name="degree",
      bounds=dict(
        min=1,
        max=5
        ),
      type="int"
      )
    ],
  observation_budget=20,
  parallel_bandwidth=1
  )
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1560895206,
  "development": false,
  "id": "2",
  "linear_constraints": [],
  "metadata": null,
  "metric": {
    "name": null,
    "object": "metric",
    "objective": "maximize",
    "value_baseline": null
  },
  "metrics": [
    {
      "name": null,
      "object": "metric",
      "objective": "maximize",
      "value_baseline": null
    }
  ],
  "name": "Classifier Accuracy",
  "num_solutions": null,
  "object": "experiment",
  "observation_budget": 20,
  "parallel_bandwidth": 1,
  "parameters": [
    {
      "bounds": {
        "max": 5,
        "min": 1,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "degree",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "int"
    }
  ],
  "progress": {
    "best_observation": null,
    "first_observation": null,
    "last_observation": null,
    "object": "progress",
    "observation_budget_consumed": 0,
    "observation_count": 0
  },
  "project": "classification-models",
  "state": "active",
  "type": "offline",
  "updated": 1560895206,
  "user": null
}

One categorical parameter

Categorical values can be provided as an array of objects (shown in a later example), or as an array of strings (shown here).

experiment = conn.experiments().create(
  name="Classifier Accuracy",
  parameters=[
    dict(
      name="gamma",
      bounds=dict(
        min=0.001,
        max=1
        ),
      type="double"
      ),
    dict(
      name="kernel",
      categorical_values=[
        "rbf",
        "poly",
        "sigmoid"
        ],
      type="categorical"
      )
    ],
  observation_budget=40,
  parallel_bandwidth=1
  )
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1560895206,
  "development": false,
  "id": "2",
  "linear_constraints": [],
  "metadata": null,
  "metric": {
    "name": null,
    "object": "metric",
    "objective": "maximize",
    "value_baseline": null
  },
  "metrics": [
    {
      "name": null,
      "object": "metric",
      "objective": "maximize",
      "value_baseline": null
    }
  ],
  "name": "Classifier Accuracy",
  "num_solutions": null,
  "object": "experiment",
  "observation_budget": 20,
  "parallel_bandwidth": 1,
  "parameters": [
    {
      "bounds": null,
      "categorical_values": [
        {
          "enum_index": 1,
          "name": "rbf",
          "object": "categorical_value"
        },
        {
          "enum_index": 2,
          "name": "poly",
          "object": "categorical_value"
        },
        {
          "enum_index": 3,
          "name": "sigmoid",
          "object": "categorical_value"
        }
      ],
      "conditions": {},
      "default_value": null,
      "name": "kernel",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "categorical"
    }
  ],
  "progress": {
    "best_observation": null,
    "first_observation": null,
    "last_observation": null,
    "object": "progress",
    "observation_budget_consumed": 0,
    "observation_count": 0
  },
  "project": "classification-models",
  "state": "active",
  "type": "offline",
  "updated": 1560895206,
  "user": null
}

All types of parameters, experiment type, and metric

experiment = conn.experiments().create(
  name="Support Vector Classifier Accuracy",
  parameters=[
    dict(
      name="degree",
      bounds=dict(
        min=1,
        max=5
        ),
      type="int"
      ),
    dict(
      name="gamma",
      bounds=dict(
        min=0.001,
        max=1
        ),
      type="double"
      ),
    dict(
      name="kernel",
      categorical_values=[
        dict(
          name="rbf"
          ),
        dict(
          name="poly"
          ),
        dict(
          name="sigmoid"
          )
        ],
      type="categorical"
      )
    ],
  metrics=[
    dict(
      name="Accuracy",
      objective="maximize"
      )
    ],
  observation_budget=60,
  parallel_bandwidth=1,
  type="offline"
  )
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1560895206,
  "development": false,
  "id": "1",
  "linear_constraints": [],
  "metadata": null,
  "metric": {
    "name": "Accuracy",
    "object": "metric",
    "objective": "maximize",
    "value_baseline": null
  },
  "metrics": [
    {
      "name": "Accuracy",
      "object": "metric",
      "objective": "maximize",
      "value_baseline": null
    }
  ],
  "name": "Support Vector Classifier Accuracy",
  "num_solutions": null,
  "object": "experiment",
  "observation_budget": 60,
  "parallel_bandwidth": null,
  "parameters": [
    {
      "bounds": {
        "max": 5,
        "min": 1,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "degree",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "int"
    },
    {
      "bounds": {
        "max": 1,
        "min": 0.001,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "gamma",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "double"
    },
    {
      "bounds": null,
      "categorical_values": [
        {
          "enum_index": 1,
          "name": "rbf",
          "object": "categorical_value"
        },
        {
          "enum_index": 2,
          "name": "poly",
          "object": "categorical_value"
        },
        {
          "enum_index": 3,
          "name": "sigmoid",
          "object": "categorical_value"
        }
      ],
      "conditions": {},
      "default_value": null,
      "name": "kernel",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "categorical"
    }
  ],
  "progress": {
    "best_observation": null,
    "first_observation": null,
    "last_observation": null,
    "object": "progress",
    "observation_budget_consumed": 0,
    "observation_count": 0
  },
  "project": "classification-models",
  "state": "active",
  "type": "offline",
  "updated": 1560895206,
  "user": null
}

Two metrics

experiment = conn.experiments().create(
  name="Profit vs. Robustness",
  parameters=[
    dict(
      name="mixing speed",
      bounds=dict(
        min=0,
        max=3
        ),
      type="double"
      ),
    dict(
      name="personnel",
      bounds=dict(
        min=10,
        max=25
        ),
      type="int"
      )
    ],
  metrics=[
    dict(
      name="profit",
      objective="maximize"
      ),
    dict(
      name="robustness",
      objective="maximize"
      )
    ],
  observation_budget=120,
  parallel_bandwidth=1
  )
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1560895206,
  "development": false,
  "id": "2",
  "linear_constraints": [],
  "metadata": null,
  "metric": {
    "name": "profit",
    "object": "metric",
    "objective": "maximize",
    "value_baseline": null
  },
  "metrics": [
    {
      "name": "profit",
      "object": "metric",
      "objective": "maximize",
      "value_baseline": null
    },
    {
      "name": "robustness",
      "object": "metric",
      "objective": "maximize",
      "value_baseline": null
    }
  ],
  "name": "Profit vs. Robustness",
  "num_solutions": null,
  "object": "experiment",
  "observation_budget": 120,
  "parallel_bandwidth": 1,
  "parameters": [
    {
      "bounds": {
        "max": 3,
        "min": 0,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "mixing speed",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "double"
    },
    {
      "bounds": {
        "max": 25,
        "min": 10,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "personnel",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "int"
    }
  ],
  "progress": {
    "best_observation": null,
    "first_observation": null,
    "last_observation": null,
    "object": "progress",
    "observation_budget_consumed": 0,
    "observation_count": 0
  },
  "project": "classification-models",
  "state": "active",
  "type": "offline",
  "updated": 1560895206,
  "user": null
}

Two solutions

experiment = conn.experiments().create(
  name="Classifier Accuracy",
  parameters=[
    dict(
      name="layer_size",
      bounds=dict(
        min=10,
        max=100
        ),
      type="int"
      ),
    dict(
      name="learning_rate",
      bounds=dict(
        min=0.001,
        max=1
        ),
      type="double"
      )
    ],
  metrics=[
    dict(
      name="Accuracy",
      objective="maximize"
      )
    ],
  num_solutions=2,
  observation_budget=60,
  parallel_bandwidth=1
  )
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1560895206,
  "development": false,
  "id": "2",
  "linear_constraints": [],
  "metadata": null,
  "metric": {
    "name": "Accuracy",
    "object": "metric",
    "objective": "maximize",
    "value_baseline": null
  },
  "metrics": [
    {
      "name": "Accuracy",
      "object": "metric",
      "objective": "maximize",
      "value_baseline": null
    }
  ],
  "name": "Classifier Accuracy",
  "num_solutions": 2,
  "object": "experiment",
  "observation_budget": 60,
  "parallel_bandwidth": 1,
  "parameters": [
    {
      "bounds": {
        "max": 100,
        "min": 10,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "layer_size",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "int"
    },
    {
      "bounds": {
        "max": 1,
        "min": 0.001,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "learning_rate",
      "object": "parameter",
      "precision": null,
      "tunable": true,
      "type": "double"
    }
  ],
  "progress": {
    "best_observation": null,
    "first_observation": null,
    "last_observation": null,
    "object": "progress",
    "observation_budget_consumed": 0,
    "observation_count": 0
  },
  "project": "classification-models",
  "state": "active",
  "type": "offline",
  "updated": 1560895206,
  "user": null
}