diff --git a/tutorials/submitit.ipynb b/tutorials/submitit.ipynb
new file mode 100644
index 00000000000..9dd49f37a8d
--- /dev/null
+++ b/tutorials/submitit.ipynb
@@ -0,0 +1,7610 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Hyperparameter Optimization on Slurm via SubmitIt\n",
+ "\n",
+ "This notebook serves as a quickstart guide for using the Ax library with the SubmitIt library in an ask-tell loop. [SubmitIt](https://github.com/facebookincubator/submitit/) is a Python toolbox for submitting jobs to [Slurm](https://slurm.schedmd.com/quickstart.html). \n",
+ "\n",
+ "The notebook demonstrates how to use the Ax client in an ask-tell loop where each trial is scheduled to run on a Slurm cluster asynchronously.\n",
+ "\n",
+ "To use this script, run it on a slurm node either as an interactive notebook or export it as a Python script and run it as a Slurm job.\n",
+ "\n",
+ "## Importing Necessary Libraries\n",
+ "Let's start by importing the necessary libraries."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import time\n",
+ "from ax.service.ax_client import AxClient, ObjectiveProperties\n",
+ "from ax.utils.notebook.plotting import render\n",
+ "from ax.service.utils.report_utils import exp_to_df\n",
+ "from submitit import AutoExecutor, LocalJob, DebugJob"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Defining the Function to Optimize\n",
+ "We'll define a simple function to optimize. This function takes two parameters, and returns a single metric."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def evaluate(parameters):\n",
+ " x = parameters[\"x\"]\n",
+ " y = parameters[\"y\"]\n",
+ " return {\"result\": (x - 3)**2 + (y - 4)**2}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Note: SubmitIt's [CommandFunction](https://github.com/facebookincubator/submitit/blob/main/docs/examples.md#working-with-commands) allows you to define commands to run on the node and then redirects the standard output.\n",
+ "\n",
+ "## Setting up Ax\n",
+ "We'll use Ax's Service API for this example. We start by initializing an AxClient and creating an experiment."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "[INFO 01-11 17:57:00] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.\n",
+ "[INFO 01-11 17:57:00] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x. If that is not the expected value type, you can explicitly specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n",
+ "[INFO 01-11 17:57:00] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter y. If that is not the expected value type, you can explicitly specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n",
+ "[INFO 01-11 17:57:00] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x', parameter_type=FLOAT, range=[-10.0, 10.0]), RangeParameter(name='y', parameter_type=FLOAT, range=[-10.0, 10.0])], parameter_constraints=[ParameterConstraint(1.0*x + 1.0*y <= 2.0)]).\n",
+ "[INFO 01-11 17:57:00] ax.modelbridge.dispatch_utils: Using Models.BOTORCH_MODULAR since there are more ordered parameters than there are categories for the unordered categorical parameters.\n",
+ "[INFO 01-11 17:57:00] ax.modelbridge.dispatch_utils: Calculating the number of remaining initialization trials based on num_initialization_trials=None max_initialization_trials=None num_tunable_parameters=2 num_trials=None use_batch_trials=False\n",
+ "[INFO 01-11 17:57:00] ax.modelbridge.dispatch_utils: calculated num_initialization_trials=5\n",
+ "[INFO 01-11 17:57:00] ax.modelbridge.dispatch_utils: num_completed_initialization_trials=0 num_remaining_initialization_trials=5\n",
+ "[INFO 01-11 17:57:00] ax.modelbridge.dispatch_utils: `verbose`, `disable_progbar`, and `jit_compile` are not yet supported when using `choose_generation_strategy` with ModularBoTorchModel, dropping these arguments.\n",
+ "[INFO 01-11 17:57:00] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+BoTorch', steps=[Sobol for 5 trials, BoTorch for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.\n"
+ ]
+ }
+ ],
+ "source": [
+ "ax_client = AxClient()\n",
+ "ax_client.create_experiment(\n",
+ " name=\"my_experiment\",\n",
+ " parameters=[\n",
+ " {\"name\": \"x\", \"type\": \"range\", \"bounds\": [-10.0, 10.0]},\n",
+ " {\"name\": \"y\", \"type\": \"range\", \"bounds\": [-10.0, 10.0]},\n",
+ " ],\n",
+ " objectives={\"result\": ObjectiveProperties(minimize=True)},\n",
+ " parameter_constraints=[\"x + y <= 2.0\"], # Optional.\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Other commonly used [parameters types](https://ax.dev/docs/glossary.html#parameter) include `choice` parameters and `fixed` parameters. \n",
+ "\n",
+ "Tip 1: you can specify additional information for parameters such as `log_scale`, if a parameter operates at a log-scale and `is_ordered` for choice parameters that have a meaningful ordering.\n",
+ "\n",
+ "Tip 2: Ax is an excellent choice for multi-objective optimization problems when there are multiple competing objectives and the goal is to find all Pareto-optimal solutions.\n",
+ "\n",
+ "Tip 3: One can define constraints on both the parameters and the outcome.\n",
+ "\n",
+ "## Setting up SubmitIt\n",
+ "We'll use SubmitIt's `AutoExecutor` for this example. We start by initializing an `AutoExecutor`, and setting a few commonly used parameters."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Log folder and cluster. Specify cluster='local' or cluster='debug' to run the jobs locally during development.\n",
+ "# When we're are ready for deployment, switch to cluster='slurm' \n",
+ "executor = AutoExecutor(folder=\"/tmp/submitit_runs\", cluster='local') \n",
+ "executor.update_parameters(timeout_min=60) # Timeout of the slurm job. Not including slurm scheduling delay.\n",
+ "executor.update_parameters(cpus_per_task=2)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Other commonly used Slurm parameters include `partition`, `ntasks_per_node`, `cpus_per_task`, `cpus_per_gpu`, `gpus_per_node`, `gpus_per_task`, `qos`, `mem`, `mem_per_gpu`, `mem_per_cpu`, `account`.\n",
+ "\n",
+ "## Running the Optimization Loop\n",
+ "Now, we're ready to run the optimization loop. We'll use an ask-tell loop, where we ask Ax for a suggestion, evaluate it using our function, and then tell Ax the result.\n",
+ "\n",
+ "The example loop schedules new jobs whenever there is availability. For tasks that take a similar amount of time regardless of the parameters, it may make more sense to wait for the whole batch to finish before scheduling the next (so ax can make better informed parameter choices).\n",
+ "\n",
+ "Note that `get_next_trials` may not use all available `num_parallel_jobs` if it doesn't have good parameter candidates to run."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "[INFO 01-11 17:57:00] ax.service.ax_client: Generated new trial 0 with parameters {'x': -1.756784, 'y': -4.021679}.\n",
+ "[INFO 01-11 17:57:00] ax.service.ax_client: Generated new trial 1 with parameters {'x': -9.300127, 'y': -4.654682}.\n",
+ "[INFO 01-11 17:57:00] ax.service.ax_client: Generated new trial 2 with parameters {'x': 4.881288, 'y': -7.929573}.\n"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "[WARNING 01-11 17:57:03] ax.service.utils.report_utils: Column reason missing for all trials. Not appending column.\n",
+ "[INFO 01-11 17:57:03] ax.service.utils.report_utils: No results present for the specified metrics `[Metric('result')]`. Returning arm parameters and metadata only.\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " trial_index | \n",
+ " arm_name | \n",
+ " trial_status | \n",
+ " generation_method | \n",
+ " x | \n",
+ " y | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 0 | \n",
+ " 0_0 | \n",
+ " RUNNING | \n",
+ " Sobol | \n",
+ " -1.756784 | \n",
+ " -4.021679 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 1 | \n",
+ " 1_0 | \n",
+ " RUNNING | \n",
+ " Sobol | \n",
+ " -9.300127 | \n",
+ " -4.654682 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 2 | \n",
+ " 2_0 | \n",
+ " RUNNING | \n",
+ " Sobol | \n",
+ " 4.881288 | \n",
+ " -7.929573 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " trial_index | \n",
+ " arm_name | \n",
+ " trial_status | \n",
+ " generation_method | \n",
+ " result | \n",
+ " x | \n",
+ " y | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 0 | \n",
+ " 0_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 86.974325 | \n",
+ " -1.756784 | \n",
+ " -4.021679 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 1 | \n",
+ " 1_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 226.196643 | \n",
+ " -9.300127 | \n",
+ " -4.654682 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 2 | \n",
+ " 2_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 145.853961 | \n",
+ " 4.881288 | \n",
+ " -7.929573 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 3 | \n",
+ " 3_0 | \n",
+ " RUNNING | \n",
+ " Sobol | \n",
+ " NaN | \n",
+ " 2.752141 | \n",
+ " -8.223596 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 4 | \n",
+ " 4_0 | \n",
+ " RUNNING | \n",
+ " Sobol | \n",
+ " NaN | \n",
+ " 9.275037 | \n",
+ " -7.347285 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " trial_index | \n",
+ " arm_name | \n",
+ " trial_status | \n",
+ " generation_method | \n",
+ " result | \n",
+ " x | \n",
+ " y | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 0 | \n",
+ " 0_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 86.974325 | \n",
+ " -1.756784 | \n",
+ " -4.021679 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 1 | \n",
+ " 1_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 226.196643 | \n",
+ " -9.300127 | \n",
+ " -4.654682 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 2 | \n",
+ " 2_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 145.853961 | \n",
+ " 4.881288 | \n",
+ " -7.929573 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 3 | \n",
+ " 3_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 149.477736 | \n",
+ " 2.752141 | \n",
+ " -8.223596 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 4 | \n",
+ " 4_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 168.136982 | \n",
+ " 9.275037 | \n",
+ " -7.347285 | \n",
+ "
\n",
+ " \n",
+ " | 5 | \n",
+ " 5 | \n",
+ " 5_0 | \n",
+ " RUNNING | \n",
+ " BoTorch | \n",
+ " NaN | \n",
+ " 0.590279 | \n",
+ " -1.398661 | \n",
+ "
\n",
+ " \n",
+ " | 6 | \n",
+ " 6 | \n",
+ " 6_0 | \n",
+ " RUNNING | \n",
+ " BoTorch | \n",
+ " NaN | \n",
+ " -2.248477 | \n",
+ " 1.686329 | \n",
+ "
\n",
+ " \n",
+ " | 7 | \n",
+ " 7 | \n",
+ " 7_0 | \n",
+ " RUNNING | \n",
+ " BoTorch | \n",
+ " NaN | \n",
+ " 1.439472 | \n",
+ " -3.621688 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " trial_index | \n",
+ " arm_name | \n",
+ " trial_status | \n",
+ " generation_method | \n",
+ " result | \n",
+ " x | \n",
+ " y | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 0 | \n",
+ " 0_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 86.974325 | \n",
+ " -1.756784 | \n",
+ " -4.021679 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 1 | \n",
+ " 1_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 226.196643 | \n",
+ " -9.300127 | \n",
+ " -4.654682 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 2 | \n",
+ " 2_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 145.853961 | \n",
+ " 4.881288 | \n",
+ " -7.929573 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 3 | \n",
+ " 3_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 149.477736 | \n",
+ " 2.752141 | \n",
+ " -8.223596 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 4 | \n",
+ " 4_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 168.136982 | \n",
+ " 9.275037 | \n",
+ " -7.347285 | \n",
+ "
\n",
+ " \n",
+ " | 5 | \n",
+ " 5 | \n",
+ " 5_0 | \n",
+ " COMPLETED | \n",
+ " BoTorch | \n",
+ " 34.952299 | \n",
+ " 0.590279 | \n",
+ " -1.398661 | \n",
+ "
\n",
+ " \n",
+ " | 6 | \n",
+ " 6 | \n",
+ " 6_0 | \n",
+ " COMPLETED | \n",
+ " BoTorch | \n",
+ " 32.899584 | \n",
+ " -2.248477 | \n",
+ " 1.686329 | \n",
+ "
\n",
+ " \n",
+ " | 7 | \n",
+ " 7 | \n",
+ " 7_0 | \n",
+ " COMPLETED | \n",
+ " BoTorch | \n",
+ " 60.525376 | \n",
+ " 1.439472 | \n",
+ " -3.621688 | \n",
+ "
\n",
+ " \n",
+ " | 8 | \n",
+ " 8 | \n",
+ " 8_0 | \n",
+ " RUNNING | \n",
+ " BoTorch | \n",
+ " NaN | \n",
+ " 0.308729 | \n",
+ " 1.691271 | \n",
+ "
\n",
+ " \n",
+ " | 9 | \n",
+ " 9 | \n",
+ " 9_0 | \n",
+ " RUNNING | \n",
+ " BoTorch | \n",
+ " NaN | \n",
+ " 0.304300 | \n",
+ " 1.695700 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " trial_index | \n",
+ " arm_name | \n",
+ " trial_status | \n",
+ " generation_method | \n",
+ " result | \n",
+ " x | \n",
+ " y | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 0 | \n",
+ " 0_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 86.974325 | \n",
+ " -1.756784 | \n",
+ " -4.021679 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 1 | \n",
+ " 1_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 226.196643 | \n",
+ " -9.300127 | \n",
+ " -4.654682 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 2 | \n",
+ " 2_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 145.853961 | \n",
+ " 4.881288 | \n",
+ " -7.929573 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 3 | \n",
+ " 3_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 149.477736 | \n",
+ " 2.752141 | \n",
+ " -8.223596 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 4 | \n",
+ " 4_0 | \n",
+ " COMPLETED | \n",
+ " Sobol | \n",
+ " 168.136982 | \n",
+ " 9.275037 | \n",
+ " -7.347285 | \n",
+ "
\n",
+ " \n",
+ " | 5 | \n",
+ " 5 | \n",
+ " 5_0 | \n",
+ " COMPLETED | \n",
+ " BoTorch | \n",
+ " 34.952299 | \n",
+ " 0.590279 | \n",
+ " -1.398661 | \n",
+ "
\n",
+ " \n",
+ " | 6 | \n",
+ " 6 | \n",
+ " 6_0 | \n",
+ " COMPLETED | \n",
+ " BoTorch | \n",
+ " 32.899584 | \n",
+ " -2.248477 | \n",
+ " 1.686329 | \n",
+ "
\n",
+ " \n",
+ " | 7 | \n",
+ " 7 | \n",
+ " 7_0 | \n",
+ " COMPLETED | \n",
+ " BoTorch | \n",
+ " 60.525376 | \n",
+ " 1.439472 | \n",
+ " -3.621688 | \n",
+ "
\n",
+ " \n",
+ " | 8 | \n",
+ " 8 | \n",
+ " 8_0 | \n",
+ " COMPLETED | \n",
+ " BoTorch | \n",
+ " 12.573169 | \n",
+ " 0.308729 | \n",
+ " 1.691271 | \n",
+ "
\n",
+ " \n",
+ " | 9 | \n",
+ " 9 | \n",
+ " 9_0 | \n",
+ " COMPLETED | \n",
+ " BoTorch | \n",
+ " 12.576597 | \n",
+ " 0.304300 | \n",
+ " 1.695700 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
result: 86.9743 (SEM: None)
x: -1.75678
y: -4.02168",
+ "Arm 1_0
result: 226.197 (SEM: None)
x: -9.30013
y: -4.65468",
+ "Arm 2_0
result: 145.854 (SEM: None)
x: 4.88129
y: -7.92957",
+ "Arm 3_0
result: 149.478 (SEM: None)
x: 2.75214
y: -8.2236",
+ "Arm 4_0
result: 168.137 (SEM: None)
x: 9.27504
y: -7.34729",
+ "Arm 5_0
result: 34.9523 (SEM: None)
x: 0.590279
y: -1.39866",
+ "Arm 6_0
result: 32.8996 (SEM: None)
x: -2.24848
y: 1.68633",
+ "Arm 7_0
result: 60.5254 (SEM: None)
x: 1.43947
y: -3.62169"
+ ],
+ "type": "scatter",
+ "x": [
+ -1.756784152239561,
+ -9.30012697353959,
+ 4.881288334727287,
+ 2.752141449600458,
+ 9.275037422776222,
+ 0.590278666557456,
+ -2.2484768683250875,
+ 1.4394719125089814
+ ],
+ "xaxis": "x",
+ "y": [
+ -4.021678697317839,
+ -4.654681961983442,
+ -7.929573114961386,
+ -8.22359612211585,
+ -7.347285468131304,
+ -1.3986611527486303,
+ 1.6863286966529074,
+ -3.6216880101600566
+ ],
+ "yaxis": "y"
+ },
+ {
+ "hoverinfo": "text",
+ "legendgroup": "In-sample",
+ "marker": {
+ "color": "black",
+ "opacity": 0.5,
+ "symbol": 1
+ },
+ "mode": "markers",
+ "name": "In-sample",
+ "showlegend": false,
+ "text": [
+ "Arm 0_0
result: 86.9743 (SEM: None)
x: -1.75678
y: -4.02168",
+ "Arm 1_0
result: 226.197 (SEM: None)
x: -9.30013
y: -4.65468",
+ "Arm 2_0
result: 145.854 (SEM: None)
x: 4.88129
y: -7.92957",
+ "Arm 3_0
result: 149.478 (SEM: None)
x: 2.75214
y: -8.2236",
+ "Arm 4_0
result: 168.137 (SEM: None)
x: 9.27504
y: -7.34729",
+ "Arm 5_0
result: 34.9523 (SEM: None)
x: 0.590279
y: -1.39866",
+ "Arm 6_0
result: 32.8996 (SEM: None)
x: -2.24848
y: 1.68633",
+ "Arm 7_0
result: 60.5254 (SEM: None)
x: 1.43947
y: -3.62169"
+ ],
+ "type": "scatter",
+ "x": [
+ -1.756784152239561,
+ -9.30012697353959,
+ 4.881288334727287,
+ 2.752141449600458,
+ 9.275037422776222,
+ 0.590278666557456,
+ -2.2484768683250875,
+ 1.4394719125089814
+ ],
+ "xaxis": "x2",
+ "y": [
+ -4.021678697317839,
+ -4.654681961983442,
+ -7.929573114961386,
+ -8.22359612211585,
+ -7.347285468131304,
+ -1.3986611527486303,
+ 1.6863286966529074,
+ -3.6216880101600566
+ ],
+ "yaxis": "y2"
+ }
+ ],
+ "layout": {
+ "annotations": [
+ {
+ "font": {
+ "size": 14
+ },
+ "showarrow": false,
+ "text": "Mean",
+ "x": 0.25,
+ "xanchor": "center",
+ "xref": "paper",
+ "y": 1,
+ "yanchor": "bottom",
+ "yref": "paper"
+ },
+ {
+ "font": {
+ "size": 14
+ },
+ "showarrow": false,
+ "text": "Standard Error",
+ "x": 0.8,
+ "xanchor": "center",
+ "xref": "paper",
+ "y": 1,
+ "yanchor": "bottom",
+ "yref": "paper"
+ }
+ ],
+ "autosize": false,
+ "height": 450,
+ "hovermode": "closest",
+ "legend": {
+ "orientation": "h",
+ "x": 0,
+ "y": -0.25
+ },
+ "margin": {
+ "b": 100,
+ "l": 35,
+ "pad": 0,
+ "r": 35,
+ "t": 35
+ },
+ "template": {
+ "data": {
+ "bar": [
+ {
+ "error_x": {
+ "color": "#2a3f5f"
+ },
+ "error_y": {
+ "color": "#2a3f5f"
+ },
+ "marker": {
+ "line": {
+ "color": "#E5ECF6",
+ "width": 0.5
+ },
+ "pattern": {
+ "fillmode": "overlay",
+ "size": 10,
+ "solidity": 0.2
+ }
+ },
+ "type": "bar"
+ }
+ ],
+ "barpolar": [
+ {
+ "marker": {
+ "line": {
+ "color": "#E5ECF6",
+ "width": 0.5
+ },
+ "pattern": {
+ "fillmode": "overlay",
+ "size": 10,
+ "solidity": 0.2
+ }
+ },
+ "type": "barpolar"
+ }
+ ],
+ "carpet": [
+ {
+ "aaxis": {
+ "endlinecolor": "#2a3f5f",
+ "gridcolor": "white",
+ "linecolor": "white",
+ "minorgridcolor": "white",
+ "startlinecolor": "#2a3f5f"
+ },
+ "baxis": {
+ "endlinecolor": "#2a3f5f",
+ "gridcolor": "white",
+ "linecolor": "white",
+ "minorgridcolor": "white",
+ "startlinecolor": "#2a3f5f"
+ },
+ "type": "carpet"
+ }
+ ],
+ "choropleth": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "type": "choropleth"
+ }
+ ],
+ "contour": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "colorscale": [
+ [
+ 0,
+ "#0d0887"
+ ],
+ [
+ 0.1111111111111111,
+ "#46039f"
+ ],
+ [
+ 0.2222222222222222,
+ "#7201a8"
+ ],
+ [
+ 0.3333333333333333,
+ "#9c179e"
+ ],
+ [
+ 0.4444444444444444,
+ "#bd3786"
+ ],
+ [
+ 0.5555555555555556,
+ "#d8576b"
+ ],
+ [
+ 0.6666666666666666,
+ "#ed7953"
+ ],
+ [
+ 0.7777777777777778,
+ "#fb9f3a"
+ ],
+ [
+ 0.8888888888888888,
+ "#fdca26"
+ ],
+ [
+ 1,
+ "#f0f921"
+ ]
+ ],
+ "type": "contour"
+ }
+ ],
+ "contourcarpet": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "type": "contourcarpet"
+ }
+ ],
+ "heatmap": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "colorscale": [
+ [
+ 0,
+ "#0d0887"
+ ],
+ [
+ 0.1111111111111111,
+ "#46039f"
+ ],
+ [
+ 0.2222222222222222,
+ "#7201a8"
+ ],
+ [
+ 0.3333333333333333,
+ "#9c179e"
+ ],
+ [
+ 0.4444444444444444,
+ "#bd3786"
+ ],
+ [
+ 0.5555555555555556,
+ "#d8576b"
+ ],
+ [
+ 0.6666666666666666,
+ "#ed7953"
+ ],
+ [
+ 0.7777777777777778,
+ "#fb9f3a"
+ ],
+ [
+ 0.8888888888888888,
+ "#fdca26"
+ ],
+ [
+ 1,
+ "#f0f921"
+ ]
+ ],
+ "type": "heatmap"
+ }
+ ],
+ "heatmapgl": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "colorscale": [
+ [
+ 0,
+ "#0d0887"
+ ],
+ [
+ 0.1111111111111111,
+ "#46039f"
+ ],
+ [
+ 0.2222222222222222,
+ "#7201a8"
+ ],
+ [
+ 0.3333333333333333,
+ "#9c179e"
+ ],
+ [
+ 0.4444444444444444,
+ "#bd3786"
+ ],
+ [
+ 0.5555555555555556,
+ "#d8576b"
+ ],
+ [
+ 0.6666666666666666,
+ "#ed7953"
+ ],
+ [
+ 0.7777777777777778,
+ "#fb9f3a"
+ ],
+ [
+ 0.8888888888888888,
+ "#fdca26"
+ ],
+ [
+ 1,
+ "#f0f921"
+ ]
+ ],
+ "type": "heatmapgl"
+ }
+ ],
+ "histogram": [
+ {
+ "marker": {
+ "pattern": {
+ "fillmode": "overlay",
+ "size": 10,
+ "solidity": 0.2
+ }
+ },
+ "type": "histogram"
+ }
+ ],
+ "histogram2d": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "colorscale": [
+ [
+ 0,
+ "#0d0887"
+ ],
+ [
+ 0.1111111111111111,
+ "#46039f"
+ ],
+ [
+ 0.2222222222222222,
+ "#7201a8"
+ ],
+ [
+ 0.3333333333333333,
+ "#9c179e"
+ ],
+ [
+ 0.4444444444444444,
+ "#bd3786"
+ ],
+ [
+ 0.5555555555555556,
+ "#d8576b"
+ ],
+ [
+ 0.6666666666666666,
+ "#ed7953"
+ ],
+ [
+ 0.7777777777777778,
+ "#fb9f3a"
+ ],
+ [
+ 0.8888888888888888,
+ "#fdca26"
+ ],
+ [
+ 1,
+ "#f0f921"
+ ]
+ ],
+ "type": "histogram2d"
+ }
+ ],
+ "histogram2dcontour": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "colorscale": [
+ [
+ 0,
+ "#0d0887"
+ ],
+ [
+ 0.1111111111111111,
+ "#46039f"
+ ],
+ [
+ 0.2222222222222222,
+ "#7201a8"
+ ],
+ [
+ 0.3333333333333333,
+ "#9c179e"
+ ],
+ [
+ 0.4444444444444444,
+ "#bd3786"
+ ],
+ [
+ 0.5555555555555556,
+ "#d8576b"
+ ],
+ [
+ 0.6666666666666666,
+ "#ed7953"
+ ],
+ [
+ 0.7777777777777778,
+ "#fb9f3a"
+ ],
+ [
+ 0.8888888888888888,
+ "#fdca26"
+ ],
+ [
+ 1,
+ "#f0f921"
+ ]
+ ],
+ "type": "histogram2dcontour"
+ }
+ ],
+ "mesh3d": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "type": "mesh3d"
+ }
+ ],
+ "parcoords": [
+ {
+ "line": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "parcoords"
+ }
+ ],
+ "pie": [
+ {
+ "automargin": true,
+ "type": "pie"
+ }
+ ],
+ "scatter": [
+ {
+ "fillpattern": {
+ "fillmode": "overlay",
+ "size": 10,
+ "solidity": 0.2
+ },
+ "type": "scatter"
+ }
+ ],
+ "scatter3d": [
+ {
+ "line": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "marker": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "scatter3d"
+ }
+ ],
+ "scattercarpet": [
+ {
+ "marker": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "scattercarpet"
+ }
+ ],
+ "scattergeo": [
+ {
+ "marker": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "scattergeo"
+ }
+ ],
+ "scattergl": [
+ {
+ "marker": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "scattergl"
+ }
+ ],
+ "scattermapbox": [
+ {
+ "marker": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "scattermapbox"
+ }
+ ],
+ "scatterpolar": [
+ {
+ "marker": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "scatterpolar"
+ }
+ ],
+ "scatterpolargl": [
+ {
+ "marker": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "scatterpolargl"
+ }
+ ],
+ "scatterternary": [
+ {
+ "marker": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "type": "scatterternary"
+ }
+ ],
+ "surface": [
+ {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ },
+ "colorscale": [
+ [
+ 0,
+ "#0d0887"
+ ],
+ [
+ 0.1111111111111111,
+ "#46039f"
+ ],
+ [
+ 0.2222222222222222,
+ "#7201a8"
+ ],
+ [
+ 0.3333333333333333,
+ "#9c179e"
+ ],
+ [
+ 0.4444444444444444,
+ "#bd3786"
+ ],
+ [
+ 0.5555555555555556,
+ "#d8576b"
+ ],
+ [
+ 0.6666666666666666,
+ "#ed7953"
+ ],
+ [
+ 0.7777777777777778,
+ "#fb9f3a"
+ ],
+ [
+ 0.8888888888888888,
+ "#fdca26"
+ ],
+ [
+ 1,
+ "#f0f921"
+ ]
+ ],
+ "type": "surface"
+ }
+ ],
+ "table": [
+ {
+ "cells": {
+ "fill": {
+ "color": "#EBF0F8"
+ },
+ "line": {
+ "color": "white"
+ }
+ },
+ "header": {
+ "fill": {
+ "color": "#C8D4E3"
+ },
+ "line": {
+ "color": "white"
+ }
+ },
+ "type": "table"
+ }
+ ]
+ },
+ "layout": {
+ "annotationdefaults": {
+ "arrowcolor": "#2a3f5f",
+ "arrowhead": 0,
+ "arrowwidth": 1
+ },
+ "autotypenumbers": "strict",
+ "coloraxis": {
+ "colorbar": {
+ "outlinewidth": 0,
+ "ticks": ""
+ }
+ },
+ "colorscale": {
+ "diverging": [
+ [
+ 0,
+ "#8e0152"
+ ],
+ [
+ 0.1,
+ "#c51b7d"
+ ],
+ [
+ 0.2,
+ "#de77ae"
+ ],
+ [
+ 0.3,
+ "#f1b6da"
+ ],
+ [
+ 0.4,
+ "#fde0ef"
+ ],
+ [
+ 0.5,
+ "#f7f7f7"
+ ],
+ [
+ 0.6,
+ "#e6f5d0"
+ ],
+ [
+ 0.7,
+ "#b8e186"
+ ],
+ [
+ 0.8,
+ "#7fbc41"
+ ],
+ [
+ 0.9,
+ "#4d9221"
+ ],
+ [
+ 1,
+ "#276419"
+ ]
+ ],
+ "sequential": [
+ [
+ 0,
+ "#0d0887"
+ ],
+ [
+ 0.1111111111111111,
+ "#46039f"
+ ],
+ [
+ 0.2222222222222222,
+ "#7201a8"
+ ],
+ [
+ 0.3333333333333333,
+ "#9c179e"
+ ],
+ [
+ 0.4444444444444444,
+ "#bd3786"
+ ],
+ [
+ 0.5555555555555556,
+ "#d8576b"
+ ],
+ [
+ 0.6666666666666666,
+ "#ed7953"
+ ],
+ [
+ 0.7777777777777778,
+ "#fb9f3a"
+ ],
+ [
+ 0.8888888888888888,
+ "#fdca26"
+ ],
+ [
+ 1,
+ "#f0f921"
+ ]
+ ],
+ "sequentialminus": [
+ [
+ 0,
+ "#0d0887"
+ ],
+ [
+ 0.1111111111111111,
+ "#46039f"
+ ],
+ [
+ 0.2222222222222222,
+ "#7201a8"
+ ],
+ [
+ 0.3333333333333333,
+ "#9c179e"
+ ],
+ [
+ 0.4444444444444444,
+ "#bd3786"
+ ],
+ [
+ 0.5555555555555556,
+ "#d8576b"
+ ],
+ [
+ 0.6666666666666666,
+ "#ed7953"
+ ],
+ [
+ 0.7777777777777778,
+ "#fb9f3a"
+ ],
+ [
+ 0.8888888888888888,
+ "#fdca26"
+ ],
+ [
+ 1,
+ "#f0f921"
+ ]
+ ]
+ },
+ "colorway": [
+ "#636efa",
+ "#EF553B",
+ "#00cc96",
+ "#ab63fa",
+ "#FFA15A",
+ "#19d3f3",
+ "#FF6692",
+ "#B6E880",
+ "#FF97FF",
+ "#FECB52"
+ ],
+ "font": {
+ "color": "#2a3f5f"
+ },
+ "geo": {
+ "bgcolor": "white",
+ "lakecolor": "white",
+ "landcolor": "#E5ECF6",
+ "showlakes": true,
+ "showland": true,
+ "subunitcolor": "white"
+ },
+ "hoverlabel": {
+ "align": "left"
+ },
+ "hovermode": "closest",
+ "mapbox": {
+ "style": "light"
+ },
+ "paper_bgcolor": "white",
+ "plot_bgcolor": "#E5ECF6",
+ "polar": {
+ "angularaxis": {
+ "gridcolor": "white",
+ "linecolor": "white",
+ "ticks": ""
+ },
+ "bgcolor": "#E5ECF6",
+ "radialaxis": {
+ "gridcolor": "white",
+ "linecolor": "white",
+ "ticks": ""
+ }
+ },
+ "scene": {
+ "xaxis": {
+ "backgroundcolor": "#E5ECF6",
+ "gridcolor": "white",
+ "gridwidth": 2,
+ "linecolor": "white",
+ "showbackground": true,
+ "ticks": "",
+ "zerolinecolor": "white"
+ },
+ "yaxis": {
+ "backgroundcolor": "#E5ECF6",
+ "gridcolor": "white",
+ "gridwidth": 2,
+ "linecolor": "white",
+ "showbackground": true,
+ "ticks": "",
+ "zerolinecolor": "white"
+ },
+ "zaxis": {
+ "backgroundcolor": "#E5ECF6",
+ "gridcolor": "white",
+ "gridwidth": 2,
+ "linecolor": "white",
+ "showbackground": true,
+ "ticks": "",
+ "zerolinecolor": "white"
+ }
+ },
+ "shapedefaults": {
+ "line": {
+ "color": "#2a3f5f"
+ }
+ },
+ "ternary": {
+ "aaxis": {
+ "gridcolor": "white",
+ "linecolor": "white",
+ "ticks": ""
+ },
+ "baxis": {
+ "gridcolor": "white",
+ "linecolor": "white",
+ "ticks": ""
+ },
+ "bgcolor": "#E5ECF6",
+ "caxis": {
+ "gridcolor": "white",
+ "linecolor": "white",
+ "ticks": ""
+ }
+ },
+ "title": {
+ "x": 0.05
+ },
+ "xaxis": {
+ "automargin": true,
+ "gridcolor": "white",
+ "linecolor": "white",
+ "ticks": "",
+ "title": {
+ "standoff": 15
+ },
+ "zerolinecolor": "white",
+ "zerolinewidth": 2
+ },
+ "yaxis": {
+ "automargin": true,
+ "gridcolor": "white",
+ "linecolor": "white",
+ "ticks": "",
+ "title": {
+ "standoff": 15
+ },
+ "zerolinecolor": "white",
+ "zerolinewidth": 2
+ }
+ }
+ },
+ "title": {
+ "text": "result"
+ },
+ "width": 950,
+ "xaxis": {
+ "anchor": "y",
+ "autorange": false,
+ "domain": [
+ 0.05,
+ 0.45
+ ],
+ "exponentformat": "e",
+ "range": [
+ -10,
+ 10
+ ],
+ "tickfont": {
+ "size": 11
+ },
+ "tickmode": "auto",
+ "title": {
+ "text": "x"
+ },
+ "type": "linear"
+ },
+ "xaxis2": {
+ "anchor": "y2",
+ "autorange": false,
+ "domain": [
+ 0.6,
+ 1
+ ],
+ "exponentformat": "e",
+ "range": [
+ -10,
+ 10
+ ],
+ "tickfont": {
+ "size": 11
+ },
+ "tickmode": "auto",
+ "title": {
+ "text": "x"
+ },
+ "type": "linear"
+ },
+ "yaxis": {
+ "anchor": "x",
+ "autorange": false,
+ "domain": [
+ 0,
+ 1
+ ],
+ "exponentformat": "e",
+ "range": [
+ -10,
+ 10
+ ],
+ "tickfont": {
+ "size": 11
+ },
+ "tickmode": "auto",
+ "title": {
+ "text": "y"
+ },
+ "type": "linear"
+ },
+ "yaxis2": {
+ "anchor": "x2",
+ "autorange": false,
+ "domain": [
+ 0,
+ 1
+ ],
+ "exponentformat": "e",
+ "range": [
+ -10,
+ 10
+ ],
+ "tickfont": {
+ "size": 11
+ },
+ "tickmode": "auto",
+ "type": "linear"
+ }
+ }
+ },
+ "text/html": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "best_parameters, (means, covariances) = ax_client.get_best_parameters()\n",
+ "print(f'Best set of parameters: {best_parameters}')\n",
+ "print(f'Mean objective value: {means}')\n",
+ "# The covariance is only meaningful when multiple objectives are present.\n",
+ "\n",
+ "render(ax_client.get_contour_plot())\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "axenv",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/website/pages/tutorials/index.js b/website/pages/tutorials/index.js
index 52a1b8ad737..618740522f0 100644
--- a/website/pages/tutorials/index.js
+++ b/website/pages/tutorials/index.js
@@ -100,6 +100,15 @@ class TutorialHome extends React.Component {
Ax and integration with an external ML library.
+