bundle/01_bundle_intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a7318b28-758a-41f3-a5cb-2b634dfe0100",
   "metadata": {},
   "source": [
    "Copyright (c) MONAI Consortium  \n",
    "Licensed under the Apache License, Version 2.0 (the \"License\");  \n",
    "you may not use this file except in compliance with the License.  \n",
    "You may obtain a copy of the License at  \n",
    "&nbsp;&nbsp;&nbsp;&nbsp;http://www.apache.org/licenses/LICENSE-2.0  \n",
    "Unless required by applicable law or agreed to in writing, software  \n",
    "distributed under the License is distributed on an \"AS IS\" BASIS,  \n",
    "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  \n",
    "See the License for the specific language governing permissions and  \n",
    "limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45839c34-faf2-4f14-b28a-fd6ff635db34",
   "metadata": {},
   "source": [
    "## Setup environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1bf88e03-1c87-4901-9cfb-9c626d454b98",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -c \"import monai\" || pip install -q \"monai-weekly[ignite,pyyaml]\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2814d671-6db5-4a89-9237-46ed4a950594",
   "metadata": {},
   "source": [
    "## Setup imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "280efd0a-74dd-41c7-8a2b-0de382dc0657",
   "metadata": {},
   "outputs": [],
   "source": [
    "from monai.config import print_config\n",
    "\n",
    "print_config()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e2cb6cb-8fc2-41cc-941b-ff2e37c4c043",
   "metadata": {},
   "source": [
    "# MONAI Bundles\n",
    "\n",
    "Bundles are essentially _self-descriptive networks_. They combine a network definition with the metadata about what they are meant to do, what they are used for, the nature of their inputs and outputs, and scripts (possibly with associated data) to train and infer using them. \n",
    "\n",
    "The key objective with bundles is to provide a structured format for using and distributing your network along with all the added information needed to understand the network in context. This makes it easier for you and others to use the network, adapt it to different applications, reproduce your experiments and results, and simply document your work.\n",
    "\n",
    "The bundle documentation and specification can be found here: https://docs.monai.io/en/stable/bundle_intro.html\n",
    "\n",
    "## Bundle Structure\n",
    "\n",
    "A bundle consists of a named directory containing specific subdirectories for different parts. From the specification we have a basic outline of directories in this form (* means optional file):\n",
    "\n",
    "```\n",
    "ModelName\n",
    "┣━ configs\n",
    "┃  ┗━ metadata.json\n",
    "┣━ models\n",
    "┃  ┣━ model.pt\n",
    "┃  ┣━ *model.ts\n",
    "┃  ┗━ *model.onnx\n",
    "┣━ docs\n",
    "┃  ┣━ *README.md\n",
    "┃  ┗━ *license.txt\n",
    "┗━ *scripts\n",
    "```\n",
    "\n",
    "Here the `metadata.json` file will contain the name of the bundle, plain language description of what it does and intended purpose, a description of what the input and output values are for the network's forward pass, copyright information, and otherwise anything else you want to add. Further configuration files go into `configs` which will be JSON or YAML documents representing scripts in the form of Python object instantiations.\n",
    "\n",
    "The `models` directory contains the stored weights for your network which can be in multiple forms. The weight dictionary `model.pt` must be present but the Torchscript `model.ts` and ONNX `model.onnx` files representing the same network are optional. \n",
    "\n",
    "The `docs` directory will contain the readme file and any other documentation you want to include. Notebooks and images are good things to include for demonstrating use of the bundle.\n",
    "\n",
    "A further `scripts` directory can be included which would contain Python definitions of any sort to be used in the JSON/YAML script files. This directory should be a valid Python module if present, ie. contains a `__init__.py` file.\n",
    "\n",
    "## Instantiating a new bundle\n",
    "\n",
    "This notebook will introduce the concepts of the bundle and how to define your own. MONAI provides a number of bundle-related programs through the `monai.bundle` module using the Fire library. We can use `init_bundle` to start creating a bundle from scratch:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a1d9d107-58d6-4ed8-9cf1-6e9103e78a92",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[01;34mTestBundle\u001b[00m\n",
      "├── \u001b[01;34mconfigs\u001b[00m\n",
      "│   ├── inference.json\n",
      "│   └── metadata.json\n",
      "├── \u001b[01;34mdocs\u001b[00m\n",
      "│   └── README.md\n",
      "├── LICENSE\n",
      "└── \u001b[01;34mmodels\u001b[00m\n",
      "\n",
      "3 directories, 4 files\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "python -m monai.bundle init_bundle TestBundle\n",
    "# you may need to install tree with \"sudo apt install tree\"\n",
    "which tree && tree TestBundle || true"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99c6a04e-4859-4123-9433-6632bbd6ff0d",
   "metadata": {},
   "source": [
    "Our new blandly-named bundle, `TestBundle`, doesn't have much in it currently. It has the directory structure so we can start putting definitions in the right places. The first thing we should do is fill in relevant information to the `metadata.json` file so that anyone who has our bundle knows what it is. The default is a template of common fields:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3e19c030-4e03-4a96-a127-ee0daa604052",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"version\": \"0.0.1\",\n",
      "    \"changelog\": {\n",
      "        \"0.0.1\": \"Initial version\"\n",
      "    },\n",
      "    \"monai_version\": \"1.4.0\",\n",
      "    \"pytorch_version\": \"2.5.0\",\n",
      "    \"numpy_version\": \"1.24.4\",\n",
      "    \"required_packages_version\": {},\n",
      "    \"task\": \"Describe what the network predicts\",\n",
      "    \"description\": \"A longer description of what the network does, use context, inputs, outputs, etc.\",\n",
      "    \"authors\": \"Your Name Here\",\n",
      "    \"copyright\": \"Copyright (c) Your Name Here\",\n",
      "    \"network_data_format\": {\n",
      "        \"inputs\": {},\n",
      "        \"outputs\": {}\n",
      "    }\n",
      "}"
     ]
    }
   ],
   "source": [
    "!cat TestBundle/configs/metadata.json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "827c759b-9ae6-4ec1-a83d-9077bf23bafd",
   "metadata": {},
   "source": [
    "We'll replace this with some more information that reflects our bundle being a demo:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a56e4833-171c-432c-8145-f325fad3bfcb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting TestBundle/configs/metadata.json\n"
     ]
    }
   ],
   "source": [
    "%%writefile TestBundle/configs/metadata.json\n",
    "\n",
    "{\n",
    "    \"version\": \"0.0.1\",\n",
    "    \"changelog\": {\n",
    "        \"0.0.1\": \"Initial version\"\n",
    "    },\n",
    "    \"monai_version\": \"1.2.0\",\n",
    "    \"pytorch_version\": \"2.0.0\",\n",
    "    \"numpy_version\": \"1.23.5\",\n",
    "    \"required_packages_version\": {},\n",
    "    \"name\": \"TestBundle\",\n",
    "    \"task\": \"Demonstration Bundle Network\",\n",
    "    \"description\": \"This is a demonstration bundle meant to showcase features of the MONAI bundle system only and does nothing useful\",\n",
    "    \"authors\": \"Your Name Here\",\n",
    "    \"copyright\": \"Copyright (c) Your Name Here\",\n",
    "    \"network_data_format\": {\n",
    "        \"inputs\": {},\n",
    "        \"outputs\": {}\n",
    "    },\n",
    "    \"intended_use\": \"This is suitable for demonstration only\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da6aa796-d4ae-423c-9215-957ad968b845",
   "metadata": {},
   "source": [
    "## Configuration Files\n",
    "\n",
    "Configuration files define how to instantiate a number of Python objects and run simple routines. These files, whether JSON or YAML, are Python dictionaries containing expression lists or the arguments to be passed to a named constructor.\n",
    "\n",
    "The provided `inference.json` file is a demo of applying a network to a series of JPEG images. This illustrates some of the concepts around typical bundles, specifically how to declare MONAI objects to put a workflow together, but we're going to ignore that for now and create some YAML configuration files instead which do some very basic things. \n",
    "\n",
    "Whether you're working with JSON or YAML the config files are doing the same thing which is define a series of object instantiations with the expectation that this constitutes a workflow. Typically for training or inference with a network this would be defining data sources, loaders, transform sequences, and finally a subclass of the [Ignite Engine](https://docs.monai.io/en/stable/engines.html#workflow). A class like `SupervisedTrainer` is the driving program for training a network, so creating an instance of this along with its associated arguments then calling its `run()` method constitutes a workflow or \"program\". \n",
    "\n",
    "You don't have to use any specific objects types though so you're totally free to design your workflows to be whatever you like, but typically as demonstrated in the MONAI Model Zoo they'll be Ignite-based workflows doing training or inference. We'll start with a very simple workflow which actually just imports Pytorch and MONAI then prints diagnostic information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "63322909-1a24-426e-a744-39452cdff14f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing TestBundle/configs/test_config.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile TestBundle/configs/test_config.yaml\n",
    "\n",
    "imports: \n",
    "- $import torch\n",
    "- $import monai\n",
    "\n",
    "device: $torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n",
    "\n",
    "shape: [4, 4]\n",
    "\n",
    "test_tensor: '$torch.rand(*@shape).to(@device)'\n",
    "\n",
    "test_config:\n",
    "- '$monai.config.print_config()'\n",
    "- '$print(\"Test tensor:\", @test_tensor)'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6c3d978-10d1-47ce-9171-2e4a4f7dbac1",
   "metadata": {},
   "source": [
    "This file demonstrates a number of key concepts:\n",
    "\n",
    "* `imports` is a sequence of strings starting with `$` which indicate the string should be interpreted as a Python expression. These will be interpreted at the start of the execution so that modules can be imported into the running namespace. `imports` should be a sequence of such expressions.\n",
    "* `device` is an object definition created by evaluating the given expression, in this case creating a Pytorch device object.\n",
    "* `shape` is a list of literal values in YAML format we'll use elsewhere.\n",
    "* `test_tensor` is another object created by evaluating an expression, this one uses references to `shape` and `device` with the `@` syntax.\n",
    "* `test_config` is a list of expressions which are evaluated in order to act as the \"main\" or entry point for the program, in this case printing config information and then our created tensor.\n",
    "\n",
    "As mentioned `$` and `@` are sigils with special meaning. A string starting with `$` is treated as a Python expression and is evaluated as such when needed, these need to be enclosed in quotes only when JSON/YAML need that to parse correctly. A variable starting with `@` is treated as reference to something we've defined in the script, eg `@shape`, and will only work for such definitions. Accessing a member of a definition before being interpreted can be done with `#`, so something like `@foo#bar` will access the `bar` member of a definition `foo`. More information on the usage of these can be found at https://docs.monai.io/en/latest/config_syntax.html.\n",
    "\n",
    "We can run this \"program\" on the command line now using the bundle submodule and a few arguments to specify the metadata file and configuration file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7968ceb4-89ef-40a9-ac9b-f048c6cca73b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-09-18 08:15:12,299 - INFO - --- input summary of monai.bundle.scripts.run ---\n",
      "2024-09-18 08:15:12,299 - INFO - > config_file: './TestBundle/configs/test_config.yaml'\n",
      "2024-09-18 08:15:12,299 - INFO - > meta_file: './TestBundle/configs/metadata.json'\n",
      "2024-09-18 08:15:12,299 - INFO - > run_id: 'test_config'\n",
      "2024-09-18 08:15:12,299 - INFO - ---\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MONAI version: 1.4.0\n",
      "Numpy version: 1.24.4\n",
      "Pytorch version: 2.5.0a0+872d972e41.nv24.08\n",
      "MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False\n",
      "MONAI rev id: ab3ee18ba32bb9bd67cd3a55717684bbd343c94c\n",
      "MONAI __file__: /workspace/Code/MONAI/monai/__init__.py\n",
      "\n",
      "Optional dependencies:\n",
      "Pytorch Ignite version: 0.4.11\n",
      "ITK version: 5.4.0\n",
      "Nibabel version: 5.2.1\n",
      "scikit-image version: 0.23.2\n",
      "scipy version: 1.14.0\n",
      "Pillow version: 10.4.0\n",
      "Tensorboard version: 2.16.2\n",
      "gdown version: 5.2.0\n",
      "TorchVision version: 0.20.0a0\n",
      "tqdm version: 4.66.5\n",
      "lmdb version: 1.5.1\n",
      "psutil version: 6.0.0\n",
      "pandas version: 2.2.2\n",
      "einops version: 0.8.0\n",
      "transformers version: 4.40.2\n",
      "mlflow version: 2.16.0\n",
      "pynrrd version: 1.0.0\n",
      "clearml version: 1.16.4\n",
      "\n",
      "For details about installing the optional dependencies, please visit:\n",
      "    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies\n",
      "\n",
      "Test tensor: tensor([[0.7786, 0.5913, 0.6661, 0.2732],\n",
      "        [0.6756, 0.3978, 0.7760, 0.7936],\n",
      "        [0.0093, 0.6293, 0.1988, 0.0280],\n",
      "        [0.8460, 0.5727, 0.6339, 0.7825]], device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "# convenient to define the bundle's root in a variable\n",
    "BUNDLE=\"./TestBundle\"\n",
    "\n",
    "# loads the test_config.yaml file and runs the test_config program it defines\n",
    "python -m monai.bundle run test_config \\\n",
    "    --meta_file \"$BUNDLE/configs/metadata.json\" \\\n",
    "    --config_file \"$BUNDLE/configs/test_config.yaml\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a28777d-b44a-4c78-b81b-a946b7f4ec30",
   "metadata": {},
   "source": [
    "Here the `run` routine is invoked and the name of the \"main\" sequence of expressions is given (`test_config`). MONAI will then load and interpret the config then evaluate the expressions of `test_config` in order. Definitions in the configuratoin which aren't needed to do this are ignored, so you can provide multiple expression lists that run different parts of your script without having to create everything. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1c00118-a695-4629-a454-3fda51c57232",
   "metadata": {},
   "source": [
    "## Object Instantiation\n",
    "\n",
    "Creating objects is a key concept in config files which would be cumbersome if done only through expressions as has been demonstrated here. Instead, an object can be defined by a dictionary of values naming first the type with `_target_` and then providing the constructor arguments as named values. The following is a simple example creat a `Dataset` class with a very simple set of values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "cb762695-8c5d-4f42-9c29-eb6260990b0c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing TestBundle/configs/test_object.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile TestBundle/configs/test_object.yaml\n",
    "\n",
    "datadicts: '$[{i: (i * i)} for i in range(10)]'  # create a fake dataset as a list of dicts\n",
    "\n",
    "test_dataset:  # creates an instance of an object because _target_ is present\n",
    "  _target_: Dataset  # name of type to create is monai.data.Dataset (loaded implicitly from MONAI)\n",
    "  data: '@datadicts'  # argument data provided by a definition\n",
    "  transform: '$None'  # argument transform provided by a Python expression\n",
    "\n",
    "test:\n",
    "- '$print(\"Dataset\", @test_dataset)'\n",
    "- '$print(\"Size\", len(@test_dataset))'\n",
    "- '$print(\"Transform member\", @test_dataset.transform)'\n",
    "- '$print(\"Values\", list(@test_dataset))'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2cd1287c-f287-4831-bfc7-4cbdc394a3a1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-09-18 08:15:22,338 - INFO - --- input summary of monai.bundle.scripts.run ---\n",
      "2024-09-18 08:15:22,338 - INFO - > config_file: './TestBundle/configs/test_object.yaml'\n",
      "2024-09-18 08:15:22,338 - INFO - > meta_file: './TestBundle/configs/metadata.json'\n",
      "2024-09-18 08:15:22,338 - INFO - > run_id: 'test'\n",
      "2024-09-18 08:15:22,338 - INFO - ---\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset <monai.data.dataset.Dataset object at 0x7f1777fd7b20>\n",
      "Size 10\n",
      "Transform member <monai.transforms.compose.Compose object at 0x7f1777fd5a20>\n",
      "Values [{0: 0}, {1: 1}, {2: 4}, {3: 9}, {4: 16}, {5: 25}, {6: 36}, {7: 49}, {8: 64}, {9: 81}]\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "BUNDLE=\"./TestBundle\"\n",
    "\n",
    "# prints normal values\n",
    "python -W ignore -m monai.bundle run test \\\n",
    "    --meta_file \"$BUNDLE/configs/metadata.json\" \\\n",
    "    --config_file \"$BUNDLE/configs/test_object.yaml\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6326d601-23f0-444b-821c-9596bd8c8296",
   "metadata": {},
   "source": [
    "The `test_dataset` definition is roughly equivalent to the expression `Dataset(data=datadicts, transform=None)`. Like regular Python we don't need to provide values for arguments having defaults, but we can only give argument values by name and not by position. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93c091b0-6140-4539-bb1e-36bf78445365",
   "metadata": {},
   "source": [
    "## Command Line Definitions\n",
    "\n",
    "Command line arguments can be provided to add or modify definitions in the script you're running. Using `--` before the name of the variable allows you to set their value with the next argument, but this must be a valid Python expression. You can also set individual members of definitions with `#` but be sure to put quotes around the argument in Bash. \n",
    "\n",
    "We can demo this with an even simpler script:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "391ec82b-43a2-4b6f-8307-e3c853986719",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing TestBundle/configs/test_cmdline.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile TestBundle/configs/test_cmdline.yaml\n",
    "\n",
    "shape: [8, 8]\n",
    "area: '$@shape[0]*@shape[1]'\n",
    "\n",
    "test:\n",
    "- '$print(\"Height\", @shape[0])'\n",
    "- '$print(\"Width\", @shape[1])'\n",
    "- '$print(\"Area\", @area)'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "229617a0-1120-4054-9232-1991cfa21ae9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-09-18 08:15:32,092 - INFO - --- input summary of monai.bundle.scripts.run ---\n",
      "2024-09-18 08:15:32,092 - INFO - > config_file: './TestBundle/configs/test_cmdline.yaml'\n",
      "2024-09-18 08:15:32,092 - INFO - > meta_file: './TestBundle/configs/metadata.json'\n",
      "2024-09-18 08:15:32,092 - INFO - > run_id: 'test'\n",
      "2024-09-18 08:15:32,092 - INFO - ---\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Height 8\n",
      "Width 8\n",
      "Area 64\n",
      "2024-09-18 08:15:41,817 - INFO - --- input summary of monai.bundle.scripts.run ---\n",
      "2024-09-18 08:15:41,817 - INFO - > config_file: './TestBundle/configs/test_cmdline.yaml'\n",
      "2024-09-18 08:15:41,817 - INFO - > meta_file: './TestBundle/configs/metadata.json'\n",
      "2024-09-18 08:15:41,817 - INFO - > run_id: 'test'\n",
      "2024-09-18 08:15:41,817 - INFO - > shape#0: 4\n",
      "2024-09-18 08:15:41,817 - INFO - ---\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Height 4\n",
      "Width 8\n",
      "Area 32\n",
      "2024-09-18 08:15:51,574 - INFO - --- input summary of monai.bundle.scripts.run ---\n",
      "2024-09-18 08:15:51,574 - INFO - > config_file: './TestBundle/configs/test_cmdline.yaml'\n",
      "2024-09-18 08:15:51,574 - INFO - > meta_file: './TestBundle/configs/metadata.json'\n",
      "2024-09-18 08:15:51,574 - INFO - > run_id: 'test'\n",
      "2024-09-18 08:15:51,574 - INFO - > area: 32\n",
      "2024-09-18 08:15:51,574 - INFO - ---\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Height 8\n",
      "Width 8\n",
      "Area 32\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "BUNDLE=\"./TestBundle\"\n",
    "\n",
    "# prints normal values\n",
    "python -W ignore -m monai.bundle run test \\\n",
    "    --meta_file \"$BUNDLE/configs/metadata.json\" \\\n",
    "    --config_file \"$BUNDLE/configs/test_cmdline.yaml\"\n",
    "\n",
    "# half the height\n",
    "python -W ignore -m monai.bundle run test \\\n",
    "    --meta_file \"$BUNDLE/configs/metadata.json\" \\\n",
    "    --config_file \"$BUNDLE/configs/test_cmdline.yaml\" \\\n",
    "    '--shape#0' 4\n",
    "\n",
    "# area definition replaces existing expression with a lie\n",
    "python -W ignore -m monai.bundle run test \\\n",
    "    --meta_file \"$BUNDLE/configs/metadata.json\" \\\n",
    "    --config_file \"$BUNDLE/configs/test_cmdline.yaml\" \\\n",
    "    --area 32"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87683aa7-0322-48cb-9919-f3b3b2546763",
   "metadata": {},
   "source": [
    "## Multiple Files\n",
    "\n",
    "Multiple config files can be specified which will create a final script composed of definitions in the first file added to or updated with those in subsequent files. Remember that the files are essentially creating Python dictionaries of definitions that are interpreted later, so later files are just updating that dictionary when loaded. Definitions in one file can be referenced in others:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "55c034c5-b03f-4ac1-8aa0-a7b768bbbb7e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing TestBundle/configs/multifile1.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile TestBundle/configs/multifile1.yaml\n",
    "\n",
    "width: 8\n",
    "height: 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "2511798a-cd44-4aec-954c-c766b29f0a43",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing TestBundle/configs/multifile2.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile TestBundle/configs/multifile2.yaml\n",
    "\n",
    "area: '$@width*@height'\n",
    "\n",
    "test:\n",
    "- '$print(\"Area\", @area)'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "dc6adf63-c4b5-4f97-805a-2321dc1e8d2c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-09-18 08:16:01,272 - INFO - --- input summary of monai.bundle.scripts.run ---\n",
      "2024-09-18 08:16:01,273 - INFO - > config_file: ['./TestBundle/configs/multifile1.yaml', './TestBundle/configs/multifile2.yaml']\n",
      "2024-09-18 08:16:01,273 - INFO - > meta_file: './TestBundle/configs/metadata.json'\n",
      "2024-09-18 08:16:01,273 - INFO - > run_id: 'test'\n",
      "2024-09-18 08:16:01,273 - INFO - ---\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Area 64\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "BUNDLE=\"./TestBundle\"\n",
    "\n",
    "# area definition replaces existing expression with a lie\n",
    "python -W ignore -m monai.bundle run test \\\n",
    "    --meta_file \"$BUNDLE/configs/metadata.json\" \\\n",
    "    --config_file \"['$BUNDLE/configs/multifile1.yaml','$BUNDLE/configs/multifile2.yaml']\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1afcbac7-1e65-4078-8465-24d5c8e08102",
   "metadata": {},
   "source": [
    "The value for `config_file` in this example is a Python list containing 2 strings. It takes a bit of care to get the Bash syntax right so that this expression isn't mangled (eg. avoid spaces to prevent tokenisation and use \"\" quotes so that other quotes aren't interpreted), but is otherwise a simple mechanism.\n",
    "\n",
    "This mechanism, and the ability to add/modify definitions on the command line, is important for a number of reasons:\n",
    "\n",
    "* It lets you write a \"common\" configuration file containing definitions to be used with other config files and so reduce duplication.\n",
    "* It lets different expressions or setups to be defined with different combinations of files, again avoiding having to duplicate then modify scripts for different experiments.\n",
    "* Adding/changing definitions also allows quick minor changes or batching of different operations on the command line or in shell scripts, eg. doing a parameter sweep by looping through possible values and passing them as arguments. \n",
    "\n",
    "## Summary and Next\n",
    "\n",
    "We have here described the basics of bundles:\n",
    "\n",
    "* Directory structure\n",
    "* Metadata file\n",
    "* Configuration files\n",
    "* Command line usage\n",
    "\n",
    "In the next tutorial we will actually create a bundle for a real network that does something and demonstrate features for working with networks."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}