Experiment Create
https://api.sigopt.com/v1/experiments
Creates a new Experiment.
Request Method: POST
Parameters
| Name | Type | Required? | Description |
|---|---|---|---|
name | string | Y | A user-specified name for this experiment. |
parameters | array<Parameter> | Y | An array of Parameter objects. |
conditionals | array<Conditional> | N | See conditionals. |
linear_constraints | array<Constraint> | N | See constraints. |
metadata | Metadata | N | Optional user-provided object. See Using Metadata for more information. |
metrics | array<Metric> | N | An array of Metric objects to be optimized or stored for analysis. If the array is of length one, the standard optimization problem is conducted. This array can have no more than 2 optimized entries and no more than 50 entries in total. |
num_solutions | int | N | The number of (diverse) solutions SigOpt will search for. This feature is only available for special plans, and does not need to be set unless the desired number of solutions is greater than 1. An observation budget is required if the number of solutions is greater than 1. No categorical variables are allowed in multiple solution experiments. |
observation_budget | int | N | The number of Observations you plan to create for this experiment. This is required when the length of metrics is greater than 1, and optional for a single metric experiment. Deviating from this value, especially by failing to reach it, may result in suboptimal performance for your experiment. |
parallel_bandwidth | int | N | The number of simultaneously open Suggestions you plan to maintain during this experiment. The default value for this is 1, i.e., a sequential experiment. The maximum value for this is dependent on your plan. This field is optional, but setting it correctly may improve performance. |
project | string | N | The id of the project to put this experiment in. |
type | string | N | A type for this experiment. Used for experimental and alpha features only. |
Deprecated Parameters
These parameters should no longer be used because there are better alternatives.
| Name | Type | Required? | Description |
|---|---|---|---|
metric | Metric | N | Information about the metric that this experiment is optimizing. |
Response
Experiment object.Examples
One double parameter
experiment = conn.experiments().create(
name="Classifier Accuracy",
parameters=[
dict(
name="gamma",
bounds=dict(
min=0.001,
max=1
),
type="double"
)
],
metrics=[
dict(
name="Accuracy",
objective="maximize",
strategy="optimize"
)
],
observation_budget=20,
parallel_bandwidth=1
)
print("Created experiment: https://app.sigopt.com/experiment/" + experiment.id)
Response
{
"client": "1",
"conditionals": [],
"created": 1414800000,
"development": false,
"id": "2",
"linear_constraints": [],
"metadata": null,
"metric": {
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
},
"metrics": [
{
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
}
],
"name": "Classifier Accuracy",
"num_solutions": null,
"object": "experiment",
"observation_budget": 20,
"parallel_bandwidth": 1,
"parameters": [
{
"bounds": {
"max": 1,
"min": 0.001,
"object": "bounds"
},
"categorical_values": null,
"conditions": {},
"default_value": null,
"name": "gamma",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "double"
}
],
"progress": {
"best_observation": null,
"first_observation": null,
"last_observation": null,
"object": "progress",
"observation_budget_consumed": 0,
"observation_count": 0
},
"project": "classification-models",
"state": "active",
"type": "offline",
"updated": 1446422400,
"user": null
}One integer parameter
experiment = conn.experiments().create(
name="Classifier Accuracy",
parameters=[
dict(
name="degree",
bounds=dict(
min=1,
max=5
),
type="int"
)
],
metrics=[
dict(
name="Accuracy",
objective="maximize",
strategy="optimize"
)
],
observation_budget=20,
parallel_bandwidth=1
)
print("Created experiment: https://app.sigopt.com/experiment/" + experiment.id)
Response
{
"client": "1",
"conditionals": [],
"created": 1414800000,
"development": false,
"id": "2",
"linear_constraints": [],
"metadata": null,
"metric": {
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
},
"metrics": [
{
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
}
],
"name": "Classifier Accuracy",
"num_solutions": null,
"object": "experiment",
"observation_budget": 20,
"parallel_bandwidth": 1,
"parameters": [
{
"bounds": {
"max": 5,
"min": 1,
"object": "bounds"
},
"categorical_values": null,
"conditions": {},
"default_value": null,
"name": "degree",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "int"
}
],
"progress": {
"best_observation": null,
"first_observation": null,
"last_observation": null,
"object": "progress",
"observation_budget_consumed": 0,
"observation_count": 0
},
"project": "classification-models",
"state": "active",
"type": "offline",
"updated": 1446422400,
"user": null
}One categorical parameter
Categorical values can be provided as an array of objects (shown in a later example), or as an array of strings (shown here).
experiment = conn.experiments().create(
name="Classifier Accuracy",
parameters=[
dict(
name="gamma",
bounds=dict(
min=0.001,
max=1
),
type="double"
),
dict(
name="kernel",
categorical_values=[
"rbf",
"poly",
"sigmoid"
],
type="categorical"
)
],
metrics=[
dict(
name="Accuracy",
objective="maximize",
strategy="optimize"
)
],
observation_budget=40,
parallel_bandwidth=1
)
print("Created experiment: https://app.sigopt.com/experiment/" + experiment.id)
Response
{
"client": "1",
"conditionals": [],
"created": 1414800000,
"development": false,
"id": "2",
"linear_constraints": [],
"metadata": null,
"metric": {
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
},
"metrics": [
{
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
}
],
"name": "Classifier Accuracy",
"num_solutions": null,
"object": "experiment",
"observation_budget": 20,
"parallel_bandwidth": 1,
"parameters": [
{
"bounds": null,
"categorical_values": [
{
"enum_index": 1,
"name": "rbf",
"object": "categorical_value"
},
{
"enum_index": 2,
"name": "poly",
"object": "categorical_value"
},
{
"enum_index": 3,
"name": "sigmoid",
"object": "categorical_value"
}
],
"conditions": {},
"default_value": null,
"name": "kernel",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "categorical"
}
],
"progress": {
"best_observation": null,
"first_observation": null,
"last_observation": null,
"object": "progress",
"observation_budget_consumed": 0,
"observation_count": 0
},
"project": "classification-models",
"state": "active",
"type": "offline",
"updated": 1446422400,
"user": null
}All types of parameters, experiment type, and metric
experiment = conn.experiments().create(
name="Support Vector Classifier Accuracy",
parameters=[
dict(
name="degree",
bounds=dict(
min=1,
max=5
),
type="int"
),
dict(
name="gamma",
bounds=dict(
min=0.001,
max=1
),
type="double"
),
dict(
name="kernel",
categorical_values=[
dict(
name="rbf"
),
dict(
name="poly"
),
dict(
name="sigmoid"
)
],
type="categorical"
)
],
metrics=[
dict(
name="Accuracy",
objective="maximize",
strategy="optimize"
)
],
observation_budget=60,
parallel_bandwidth=1,
type="offline"
)
print("Created experiment: https://app.sigopt.com/experiment/" + experiment.id)
Response
{
"client": "1",
"conditionals": [],
"created": 1414800000,
"development": false,
"id": "1",
"linear_constraints": [],
"metadata": null,
"metric": {
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
},
"metrics": [
{
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
}
],
"name": "Support Vector Classifier Accuracy",
"num_solutions": null,
"object": "experiment",
"observation_budget": 60,
"parallel_bandwidth": null,
"parameters": [
{
"bounds": {
"max": 5,
"min": 1,
"object": "bounds"
},
"categorical_values": null,
"conditions": {},
"default_value": null,
"name": "degree",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "int"
},
{
"bounds": {
"max": 1,
"min": 0.001,
"object": "bounds"
},
"categorical_values": null,
"conditions": {},
"default_value": null,
"name": "gamma",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "double"
},
{
"bounds": null,
"categorical_values": [
{
"enum_index": 1,
"name": "rbf",
"object": "categorical_value"
},
{
"enum_index": 2,
"name": "poly",
"object": "categorical_value"
},
{
"enum_index": 3,
"name": "sigmoid",
"object": "categorical_value"
}
],
"conditions": {},
"default_value": null,
"name": "kernel",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "categorical"
}
],
"progress": {
"best_observation": null,
"first_observation": null,
"last_observation": null,
"object": "progress",
"observation_budget_consumed": 0,
"observation_count": 0
},
"project": "classification-models",
"state": "active",
"type": "offline",
"updated": 1446422400,
"user": null
}Two metrics
experiment = conn.experiments().create(
name="Profit vs. Robustness",
parameters=[
dict(
name="mixing speed",
bounds=dict(
min=0,
max=3
),
type="double"
),
dict(
name="personnel",
bounds=dict(
min=10,
max=25
),
type="int"
)
],
metrics=[
dict(
name="profit",
objective="maximize",
strategy="optimize"
),
dict(
name="robustness",
objective="maximize",
strategy="optimize"
)
],
observation_budget=120,
parallel_bandwidth=1
)
print("Created experiment: https://app.sigopt.com/experiment/" + experiment.id)
Response
{
"client": "1",
"conditionals": [],
"created": 1414800000,
"development": false,
"id": "2",
"linear_constraints": [],
"metadata": null,
"metric": null,
"metrics": [
{
"name": "profit",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
},
{
"name": "robustness",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
}
],
"name": "Profit vs. Robustness",
"num_solutions": null,
"object": "experiment",
"observation_budget": 120,
"parallel_bandwidth": 1,
"parameters": [
{
"bounds": {
"max": 3,
"min": 0,
"object": "bounds"
},
"categorical_values": null,
"conditions": {},
"default_value": null,
"name": "mixing speed",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "double"
},
{
"bounds": {
"max": 25,
"min": 10,
"object": "bounds"
},
"categorical_values": null,
"conditions": {},
"default_value": null,
"name": "personnel",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "int"
}
],
"progress": {
"best_observation": null,
"first_observation": null,
"last_observation": null,
"object": "progress",
"observation_budget_consumed": 0,
"observation_count": 0
},
"project": "classification-models",
"state": "active",
"type": "offline",
"updated": 1446422400,
"user": null
}Two solutions
experiment = conn.experiments().create(
name="Classifier Accuracy",
parameters=[
dict(
name="layer_size",
bounds=dict(
min=10,
max=100
),
type="int"
),
dict(
name="learning_rate",
bounds=dict(
min=0.001,
max=1
),
type="double"
)
],
metrics=[
dict(
name="Accuracy",
objective="maximize",
strategy="optimize"
)
],
num_solutions=2,
observation_budget=60,
parallel_bandwidth=1
)
print("Created experiment: https://app.sigopt.com/experiment/" + experiment.id)
Response
{
"client": "1",
"conditionals": [],
"created": 1414800000,
"development": false,
"id": "2",
"linear_constraints": [],
"metadata": null,
"metric": {
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
},
"metrics": [
{
"name": "Accuracy",
"object": "metric",
"objective": "maximize",
"strategy": "optimize",
"threshold": null
}
],
"name": "Classifier Accuracy",
"num_solutions": 2,
"object": "experiment",
"observation_budget": 60,
"parallel_bandwidth": 1,
"parameters": [
{
"bounds": {
"max": 100,
"min": 10,
"object": "bounds"
},
"categorical_values": null,
"conditions": {},
"default_value": null,
"name": "layer_size",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "int"
},
{
"bounds": {
"max": 1,
"min": 0.001,
"object": "bounds"
},
"categorical_values": null,
"conditions": {},
"default_value": null,
"name": "learning_rate",
"object": "parameter",
"precision": null,
"prior": null,
"tunable": true,
"type": "double"
}
],
"progress": {
"best_observation": null,
"first_observation": null,
"last_observation": null,
"object": "progress",
"observation_budget_consumed": 0,
"observation_count": 0
},
"project": "classification-models",
"state": "active",
"type": "offline",
"updated": 1446422400,
"user": null
}