1266 update bundle and model zoo tutorials (#1272)

Fixes #1266 and #1043  .

### Description
This PR does the following works:
1. updates tutorials in `bundle` and `model_zoo` via using the new
BundleWorkflow interface.
2. To avoid maintaining the spleen bundle in both `tutorials` and
`model-zoo`, this PR changes to use download API to download the spleen
bundle.
3. remove "MONAI model-zoo" related words in tutorials that can also use
NGC bundles.

### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Avoid including large-size files in the PR.
- [x] Clean up long text outputs from code cells in the notebook.
- [x] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [x] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [ ] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

---------

Signed-off-by: Yiheng Wang <[email protected]>

Author: yiheng-wang-nv

bundle/README.md

@@ -4,9 +4,6 @@ This folder contains the `getting started` tutorial and below code examples of t
 ### [introducing_config](./introducing_config)
 A simple example to introduce the MONAI bundle config and parsing.
 
-### [spleen segmentation](./spleen_segmentation)
-A bundle example for volumetric (3D) segmentation of the spleen from CT image.
-
 ### [customize component](./custom_component)
 Example shows the use cases of bringing customized python components, such as transform, network, and metrics, in a configuration-based workflow.
 

bundle/custom_component/README.md

@@ -1,7 +1,15 @@
 # Description
 This example mainly shows a typical use case that brings customized python components (such as transform, network, metrics) in a configuration-based workflow.
 
-Please note that this example depends on the `spleen_segmentation` bundle example and executes via overriding the config file of it.
+Please note that this example depends on the `spleen_ct_segmentation` bundle example and executes via overriding the config file of it.
+
+## Download bundle
+
+Download the `spleen_ct_segmentation` bundle for this example.
+
+```shell
+python -m monai.bundle download --name spleen_ct_segmentation --bundle_dir "./"
+```
 
 ## commands example
 To run the workflow with customized components, `PYTHONPATH` should be revised to include the path to the customized component:
@@ -12,7 +20,7 @@ And please make sure the folder `custom_component/scripts` is a valid python mod
 
 Override the `train` config with the customized `transform` and execute training:
 ```bash
-python -m monai.bundle run training --meta_file <spleen_configs_path>/metadata.json \
+python -m monai.bundle run --meta_file <spleen_configs_path>/metadata.json \
     --config_file "['<spleen_configs_path>/train.json','configs/custom_train.json']" \
     --logging_file <spleen_configs_path>/logging.conf
 ```

bundle/get_started.md

@@ -22,10 +22,14 @@ You can find the usage examples of MONAI bundle key features and syntax in this
 - Override config content at runtime.
 - Hybrid programming with config and python code.
 
-## Download dataset
+## Download bundle and dataset
 
-Downloads and extracts the dataset for this example.
-The dataset comes from http://medicaldecathlon.com/.
+Download the `spleen_ct_segmentation` bundle for this example.
+```shell
+python -m monai.bundle download --name spleen_ct_segmentation --bundle_dir "./"
+```
+
+The dataset for this example comes from http://medicaldecathlon.com/.
 Here specify a directory with the `MONAI_DATA_DIRECTORY` environment variable to save downloaded dataset and outputs, if no environment, save to the temorary directory.
 
 ```python
@@ -46,7 +50,7 @@ if not os.path.exists(data_dir):
 
 ## Define train config - Set imports and input / output environments
 
-Now let's start to define the config file for a regular training task. MONAI bundle support both `JSON` and `YAML` format, here we use `JSON` as the example. the [whole config for training](spleen_segmentation/configs/train.json) is available and can be a reference.
+Now let's start to define the config file for a regular training task. MONAI bundle support both `JSON` and `YAML` format, here we use `JSON` as the example. After downloading the bundle, the train config file in `spleen_ct_segmentation/configs/train.json` is available for reference.
 
 According to the predefined syntax of MONAI bundle, `$` indicates an expression to evaluate and `@` refers to another object in the config content. For more details about the syntax in bundle config, please check: https://docs.monai.io/en/latest/config_syntax.html.
 
@@ -305,24 +309,32 @@ Just show an example of `macro text replacement` to simplify the config content
 
 We can define a `metadata` file in the bundle, which contains the metadata information relating to the model, including what the shape and format of inputs and outputs are, what the meaning of the outputs are, what type of model is present, and other information. The structure is a dictionary containing a defined set of keys with additional user-specified keys.
 
-A typical [metadata example](spleen_segmentation/configs/metadata.json) is available.
+After downloading the bundle, a typical metadata example in `spleen_ct_segmentation/configs/metadata.json` is available for reference.
 
 ## Execute training with bundle script - `run`
 
 There are several predefined scripts in MONAI bundle module, here we leverage the `run` script and specify the ID of trainer in the config.
 
-Just define the entry point expressions in the config to execute in order, and specify the `runner_id` in CLI script.
+We can define the following three sections:
+
+"run" determines the section of the expected config expression to run.
+"initialize" determines the section of the expected config expression to initialize before running.
+"finalize" determines the section of the expected config expression to finalize after running.
+
+In this example, only "initialize" and "run" are utilized:
 
 ```json
-"training": [
+"initialize": [
     "$monai.utils.set_determinism(seed=123)",
-    "$setattr(torch.backends.cudnn, 'benchmark', True)",
+    "$setattr(torch.backends.cudnn, 'benchmark', True)"
+],
+"run": [
     "$@train#trainer.run()"
 ]
 ```
 
 ```shell
-python -m monai.bundle run training --config_file configs/train.json
+python -m monai.bundle run --config_file configs/train.json
 ```
 
 ## Execute training with bundle script - Override config at runtime
@@ -331,11 +343,11 @@ To override some config items at runtime, users can specify the target `id` and
 
 Please note that "#" and "$" may be meaningful syntax for some `shell` and `CLI` tools, so may need to add escape character or quotes for them in the command line, like: `"\$torch.device('cuda:1')"`. For more details: https://github.com/google/python-fire/blob/v0.4.0/fire/parser.py#L60.
 ```shell
-python -m monai.bundle run training --config_file configs/train.json --device "\$torch.device('cuda:1')"
+python -m monai.bundle run --config_file configs/train.json --device "\$torch.device('cuda:1')"
 ```
 Override content from another config file.
 ```shell
-python -m monai.bundle run training --config_file configs/train.json --network "%configs/test.json#network"
+python -m monai.bundle run --config_file configs/train.json --network "%configs/inference.json#network"
 ```
 
 ## Execute other bundle scripts
@@ -365,12 +377,6 @@ python -m monai.bundle verify_net_in_out network --meta_file <metadata path> --c
 ```
 The acceptable data shape in the metadata can support `"*"` for any size, or use an expression with Python mathematical operators and one character variables to represent dependence on an unknown quantity, for example, `"2**p"` represents a size which must be a power of 2, `"2**p*n"` must be a multiple of a power of 2. `"spatial_shape": [ "32 * n", "2 ** p * n", "*"]`.
 
-
-5. Download a bundle from Github release or URL.
-```shell
-python -m monai.bundle download --name <bundle_name> --version "0.1.0" --bundle_dir "./"
-```
-
 ## Hybrid programming with config and python code
 
 A MONAI bundle supports flexible customized logic, there are several ways to achieve this:

bundle/hybrid_programming/README.md

@@ -10,6 +10,7 @@ python -m scripts.inference run \
   --config_file "['configs/data_loading.json','configs/net_inferer.yaml','configs/post_processing.json']" \
   --ckpt_path <path_to_checkpoint>
 ```
+Before running above command, please check the script first and modify `datalist` and `output_dir` according to the actual paths.
 
 Parse the config file content in Python and overriding the default values, using the entry point defined in `scrips/train_demo.py`.
 ```bash

bundle/introducing_config/README.md

@@ -152,7 +152,7 @@ plt.show()
 
 ## Topics not covered but possible in the config
 
-- Running customized Python components (made available on the `PYTHONPATH`, more examples [in the model_zoo](https://github.com/Project-MONAI/model-zoo)).
+- Running customized Python components (made available on the `PYTHONPATH`).
 - Overriding the component in `example.yaml` using, for example, `--id=new_value` in the command line.
 - Multiple configuration files and cross-file references.
 - Replacing in terms of plain texts instead of Python objects ([tutorial](../get_started.md)).

bundle/spleen_segmentation/configs/evaluate.json

@@ -1,5 +1,14 @@
-This folder contains typical usage examples of the bundles in MONAI model-zoo.
+This folder contains tutorials and examples to show how to use an existing bundle of our released models, including:
+
+- Adapt a bundle to another dataset.
+- Integrate a bundle into your own application.
+- Load the pre-trained weights from a bundle and do transfer-learning.
+- Extend the features of workflow in MONAI bundles based on `event-handler` mechanism.
 
 ## Getting Started
 
-To download and get started with the models, please see also https://monai.io/model-zoo.
+To download and get started with the models, `monai.bundle` API is recommended. The following is an example to download the `spleen_ct_segmentation` bundle:
+
+```shell
+python -m monai.bundle download --name spleen_ct_segmentation --bundle_dir "./"
+```

bundle/spleen_segmentation/configs/inference.json

@@ -2,7 +2,7 @@
 Adapt a bundle to another dataset.
 
 # Overview
-This tutorial shows how to adapt an example monai bundle from MONAI model-zoo to a new dataset.
+This tutorial shows how to adapt an example monai bundle to a new dataset.
 
 ## Data
 The new dataset is BTCV challenge dataset (https://www.synapse.org/#!Synapse:syn3193805/wiki/217752). It has 24 Training + 6 Validation CT abdominal scans.
@@ -19,9 +19,9 @@ In this experiment, we focus on spleen only, which has intensity = 1 in the labe
 
 Step 4: `cp -avr ./data/RawData/Training/img ./data/btcv_spleen/imagesTr`
 
-## Download example monai bundle from model-zoo
+## Download example monai bundle
 ```
-python -m monai.bundle download --name spleen_ct_segmentation --version "0.1.1" --bundle_dir "./"
+python -m monai.bundle download --name spleen_ct_segmentation --bundle_dir "./"
 ```
 
 ## Training configuration
@@ -33,7 +33,7 @@ Actual Model Input: 96 x 96 x 96
 ## Modify ./spleen_ct_segmentation/configs/train.json from the downloaded example
 | Old json config | Updated json config |
 | --- | --- |
-| "bundle_root": "/workspace/data/tutorials/bundle/spleen_segmentation", | "bundle_root": "./spleen_ct_segmentation", |
+| "bundle_root": ".", | "bundle_root": "./spleen_ct_segmentation", |
 | "dataset_dir": "/workspace/data/Task09_Spleen",| "data_file_base_dir": "./data/btcv_spleen", |
 | "images": "$list(sorted(glob.glob(@dataset_dir + '/imagesTr/*.nii.gz')))",| "data_list_file_path": "./data/dataset_0.json", |
 | "labels": "$list(sorted(glob.glob(@dataset_dir + '/labelsTr/*.nii.gz')))",| "train_datalist": "$monai.data.load_decathlon_datalist(@data_list_file_path, is_segmentation=True, data_list_key='training', base_dir=@data_file_base_dir)", |
@@ -59,13 +59,13 @@ Mean Dice = 0.9294.
 Execute training:
 
 ```
-python -m monai.bundle run training --meta_file ./spleen_ct_segmentation/configs/metadata.json --config_file ./spleen_ct_segmentation/configs/train.json --logging_file ./spleen_ct_segmentation/configs/logging.conf
+python -m monai.bundle run --meta_file ./spleen_ct_segmentation/configs/metadata.json --config_file ./spleen_ct_segmentation/configs/train.json --logging_file ./spleen_ct_segmentation/configs/logging.conf
 ```
 
 Override the `train` config to execute evaluation:
 
 ```
-python -m monai.bundle run evaluating --meta_file ./spleen_ct_segmentation/configs/metadata.json --config_file "['./spleen_ct_segmentation/configs/train.json','./spleen_ct_segmentation/configs/evaluate.json']" --logging_file ./spleen_ct_segmentation/configs/logging.conf
+python -m monai.bundle run --meta_file ./spleen_ct_segmentation/configs/metadata.json --config_file "['./spleen_ct_segmentation/configs/train.json','./spleen_ct_segmentation/configs/evaluate.json']" --logging_file ./spleen_ct_segmentation/configs/logging.conf
 ```
 
 

bundle/spleen_segmentation/configs/logging.conf

@@ -22,12 +22,12 @@ The script is tested with:
 You can conda environments to install the dependencies.
 
 ```bash
-pip install monai==0.9.1 scikit-learn==0.24.2
+pip install scikit-learn==0.24.2
 ```
 
-or you can just use nvidia docker.
+or you can just use MONAI docker.
 ```bash
-docker pull projectmonai/monai:0.9.1
+docker pull projectmonai/monai:latest
 ```
 
 For more information please check out [the installation guide](https://docs.monai.io/en/latest/installation.html).
@@ -42,7 +42,11 @@ python ./ensemble.py -h
 ### Get started
 1. Prepare your bundle.
 
-    First download a bundle from [model-zoo](https://github.com/Project-MONAI/model-zoo/releases/tag/hosting_storage_v1) to somewhere as your `bundle_root_path`.
+    First download a bundle to somewhere as your `bundle_root_path`:
+
+    ```shell
+    python -m monai.bundle download --name spleen_ct_segmentation --bundle_dir "./"
+    ```
 
 2. Prepare your data.
 

bundle/spleen_segmentation/configs/metadata.json

@@ -48,7 +48,6 @@ class EnsembleTrainTask:
     """
     To construct an n-fold training and ensemble infer on any dataset.
     Just specify the bundle root path and data root path.
-    Bundle can be download from https://github.com/Project-MONAI/model-zoo/releases/tag/hosting_storage_v1
     Date root path also need a dataset.json which should be like:
         {
             "training": [
@@ -256,7 +255,7 @@ def run_command(self, cmd, env):
 if __name__ == "__main__":
     """
     Usage
-        first download a bundle from model-zoo to somewhere as your bundle_root path
+        first download a bundle to somewhere as your bundle_root path
         split your data into train and test datalist
             train datalist: [
                 {

bundle/spleen_segmentation/configs/multi_gpu_train.json

@@ -1,9 +1,9 @@
 # Overview
-This tutorial shows how to extend the features of workflow in the model-zoo bundles based on `event-handler` mechanism.
+This tutorial shows how to extend the features of workflow in MONAI bundles based on `event-handler` mechanism.
 Here we try to add the execution time computation logic in the spleen segmentation bundle.
 
 ## Event-handler mechanism
-The bundles in the `model-zoo` are constructed by MONAI workflow, which can enable quick start of training and evaluation experiments.
+Bundles that constructed by MONAI workflow enable quick start of training and evaluation experiments.
 The MONAI workflow is compatible with pytorch-ignite `Engine` and `Event-Handler` mechanism: https://pytorch-ignite.ai/.
 
 So we can easily extend new features to the workflow by defining a new independent event handler and attaching to the workflow engine.
@@ -91,9 +91,9 @@ like the existing [StatsHandler](https://docs.monai.io/en/stable/handlers.html#m
 MONAI contains a convenient utility `monai.handlers.from_engine` to support most of the typical `output_transform` callables.
 For more details, please refer to: https://docs.monai.io/en/stable/handlers.html#monai.handlers.utils.from_engine.
 
-## Download example MONAI bundle from model-zoo
+## Download example MONAI bundle
 ```
-python -m monai.bundle download --name spleen_ct_segmentation --version "0.1.1" --bundle_dir "./"
+python -m monai.bundle download --name spleen_ct_segmentation --bundle_dir "./"
 ```
 
 ## Extend the workflow to print the execution time for every iteration, every epoch and total time
@@ -152,5 +152,5 @@ And please make sure the folder `spleen_ct_segmentation/scripts` is a valid pyth
 Execute training:
 
 ```
-python -m monai.bundle run training --meta_file configs/metadata.json --config_file configs/train.json --logging_file configs/logging.conf
+python -m monai.bundle run --meta_file configs/metadata.json --config_file configs/train.json --logging_file configs/logging.conf
 ```

bundle/spleen_segmentation/configs/train.json

@@ -1,21 +1,21 @@
 # Description
-Load the pre-trained weights from a MONAI model-zoo model and execute the transfer-learning in a typical PyTorch training program based on it.
+Load the pre-trained weights from a MONAI bundle and execute the transfer-learning in a typical PyTorch training program based on it.
 
 # Overview
-This tutorial shows how to load a pretrained U-net from MONAI model-zoo and train it on BTCV dataset using pytorch. The pretrained U-net was trained on volumetric (3D) segmentation of CT spleen from MSD dataset.
+This tutorial shows how to load a pretrained U-net from a MONAI bundle and train it on BTCV dataset using pytorch. The pretrained U-net was trained on volumetric (3D) segmentation of CT spleen from MSD dataset.
 
 ## Data
 The description of data and data preparation can be found in [adapt_bundle_to_another_dataset](../adapt_bundle_to_another_dataset)
 
 ## Pretrained model
-The pretrained model is from MONAI model-zoo spleen_ct_segmentation.
+The pretrained model is from MONAI spleen_ct_segmentation bundle.
 It was trained using the runner-up [1] awarded pipeline of the "Medical Segmentation Decathlon Challenge 2018" using the UNet architecture [2] with 32 training images and 9 validation images.
 
 It can automatically download the bundle and load the model with the following code, which can be found in `train.py`.
 ```
 import monai
 pretrained_model = monai.bundle.load(
-    name="spleen_ct_segmentation", bundle_dir="./", version="0.1.1"
+    name="spleen_ct_segmentation", bundle_dir="./"
 )
 model.load_state_dict(pretrained_model)
 ```

bundle/spleen_segmentation/docs/README.md

@@ -72,7 +72,7 @@ def main(tempdir, load_pretrained_ckpt=False):
         save_model = "./models/model_from_scratch.pt"
         max_epochs = 600
 
-    # create UNet and load pretrained weights from monai model-zoo
+    # create UNet and load pretrained weights
     device = torch.device("cuda:0")
     model = UNet(
         spatial_dims=3,
@@ -85,8 +85,7 @@ def main(tempdir, load_pretrained_ckpt=False):
     ).to(device)
 
     if load_pretrained_ckpt:
-        print("Load model from monai model-zoo.")
-        pretrained_model = monai.bundle.load(name="spleen_ct_segmentation", bundle_dir="./", version="0.1.1")
+        pretrained_model = monai.bundle.load(name="spleen_ct_segmentation", bundle_dir="./")
         model.load_state_dict(pretrained_model)
 
     # define transforms for image and segmentation