From d8ae8faeb8d78d9afdd33094a322cf2e243ce002 Mon Sep 17 00:00:00 2001 From: Joey Wang Date: Tue, 1 Mar 2022 04:23:57 -0800 Subject: [PATCH] Doc v3.4.1 joey --- README.md | 2 +- docs/hugectr_user_guide.md | 2 +- notebooks/README.md | 8 ++--- notebooks/continuous_training.ipynb | 2 +- notebooks/hugectr2onnx_demo.ipynb | 2 +- notebooks/hugectr_criteo.ipynb | 2 +- notebooks/movie-lens-example.ipynb | 2 +- notebooks/news-example.ipynb | 2 +- onnx_converter/README.md | 2 +- release_notes.md | 32 +++++++++++++++++-- samples/criteo/README.md | 4 +-- samples/criteo_multi_slots/README.md | 4 +-- samples/dcn/README.md | 4 +-- samples/deepfm/README.md | 4 +-- samples/din/README.md | 4 +-- samples/dlrm/README.md | 4 +-- samples/ncf/README.md | 4 +-- samples/wdl/README.md | 4 +-- sparse_operation_kit/ReadMe.md | 2 +- sparse_operation_kit/notebooks/ReadMe.md | 4 +-- .../notebooks/benchmark.ipynb | 4 +-- .../demo_with_nvtabular_datareader.ipynb | 2 +- .../notebooks/sparse_operation_kit_demo.ipynb | 2 +- 23 files changed, 65 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index fbca9edead..ca8f5aee4b 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ If you'd like to quickly train a model using the Python interface, do the follow 1. Start a NGC container with your local host directory (/your/host/dir mounted) by running the following command: ``` - docker run --gpus=all --rm -it --cap-add SYS_NICE -v /your/host/dir:/your/container/dir -w /your/container/dir -it -u $(id -u):$(id -g) nvcr.io/nvidia/merlin/merlin-training:22.02 + docker run --gpus=all --rm -it --cap-add SYS_NICE -v /your/host/dir:/your/container/dir -w /your/container/dir -it -u $(id -u):$(id -g) nvcr.io/nvidia/merlin/merlin-training:22.03 ``` **NOTE**: The **/your/host/dir** directory is just as visible as the **/your/container/dir** directory. The **/your/host/dir** directory is also your starting directory. diff --git a/docs/hugectr_user_guide.md b/docs/hugectr_user_guide.md index cfb89bb203..e308a5a242 100644 --- a/docs/hugectr_user_guide.md +++ b/docs/hugectr_user_guide.md @@ -77,7 +77,7 @@ The following sample command pulls and starts the Merlin Training container: ```shell # Run the container in interactive mode -$ docker run --gpus=all --rm -it --cap-add SYS_NICE nvcr.io/nvidia/merlin/merlin-training:22.02 +$ docker run --gpus=all --rm -it --cap-add SYS_NICE nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Building HugeCTR from Scratch diff --git a/notebooks/README.md b/notebooks/README.md index b3378b9892..487423afe4 100755 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -7,12 +7,12 @@ The quickest way to run a notebook here is with a docker container, which provid ### Pull the NGC Docker To start the [sparse_operation_kit_demo.ipynb](../sparse_operation_kit/notebooks/sparse_operation_kit_demo.ipynb) notebook, pull this docker image: ``` -docker pull nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.02 +docker pull nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.03 ``` To start the other notebooks, pull the docker image using the following command: ``` -docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 +docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Clone the HugeCTR Repository @@ -25,11 +25,11 @@ git clone https://github.com/NVIDIA/HugeCTR 1. Launch the container in interactive mode (mount the HugeCTR root directory into the container for your convenience) by running this command: ``` - docker run --runtime=nvidia --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr -p 8888:8888 nvcr.io/nvidia/merlin/merlin-training:22.02 + docker run --runtime=nvidia --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr -p 8888:8888 nvcr.io/nvidia/merlin/merlin-training:22.03 ``` Launch the container in interactive mode (mount the HugeCTR root directory into the container for your convenience) by running this command to run [sparse_operation_kit_demo.ipynb](../sparse_operation_kit/notebooks/sparse_operation_kit_demo.ipynb) notebook : ``` - docker run --runtime=nvidia --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr -p 8888:8888 nvcr.io/nvstaging/merlin/merlin-tensorflow-training:22.02 + docker run --runtime=nvidia --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr -p 8888:8888 nvcr.io/nvstaging/merlin/merlin-tensorflow-training:22.03 ``` 2. Start Jupyter using these commands: diff --git a/notebooks/continuous_training.ipynb b/notebooks/continuous_training.ipynb index 7bdd1d7369..191208299b 100644 --- a/notebooks/continuous_training.ipynb +++ b/notebooks/continuous_training.ipynb @@ -38,7 +38,7 @@ "## Installation\n", "\n", "### 1.1 Get HugeCTR from NGC\n", - "The continuous training module is preinstalled in the [Merlin Training Container](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-training): `nvcr.io/nvidia/merlin/merlin-training:22.02`.\n", + "The continuous training module is preinstalled in the [Merlin Training Container](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-training): `nvcr.io/nvidia/merlin/merlin-training:22.03`.\n", "\n", "You can check the existence of required libraries by running the following Python code after launching this container.\n", "```bash\n", diff --git a/notebooks/hugectr2onnx_demo.ipynb b/notebooks/hugectr2onnx_demo.ipynb index 3bee0c5591..8aa4c81536 100755 --- a/notebooks/hugectr2onnx_demo.ipynb +++ b/notebooks/hugectr2onnx_demo.ipynb @@ -41,7 +41,7 @@ "\n", "## 1. Access the HugeCTR to ONNX Converter\n", "\n", - "1. Please make sure that you start the notebook inside the running NGC docker container: `nvcr.io/nvidia/merlin/merlin-training:22.02`. The module of the ONNX converter is installed to the system path `/usr/local/lib/python3.8/dist-packages`. As for HugeCTR Python interface, a dynamic link to the `hugectr.so` library is installed to the system path `/usr/local/hugectr/lib/`. You can access the ONNX converter as well as HugeCTR Python interface anywhere within the container." + "1. Please make sure that you start the notebook inside the running NGC docker container: `nvcr.io/nvidia/merlin/merlin-training:22.03`. The module of the ONNX converter is installed to the system path `/usr/local/lib/python3.8/dist-packages`. As for HugeCTR Python interface, a dynamic link to the `hugectr.so` library is installed to the system path `/usr/local/hugectr/lib/`. You can access the ONNX converter as well as HugeCTR Python interface anywhere within the container." ] }, { diff --git a/notebooks/hugectr_criteo.ipynb b/notebooks/hugectr_criteo.ipynb index 760920c91e..c8dd943b9b 100755 --- a/notebooks/hugectr_criteo.ipynb +++ b/notebooks/hugectr_criteo.ipynb @@ -48,7 +48,7 @@ "\n", "\n", "### 1.1 Get HugeCTR from NGC\n", - "The HugeCTR Python module is preinstalled in the [Merlin Training Container](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-training): `nvcr.io/nvidia/merlin/merlin-training:22.02`.\n", + "The HugeCTR Python module is preinstalled in the [Merlin Training Container](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-training): `nvcr.io/nvidia/merlin/merlin-training:22.03`.\n", "\n", "You can check the existence of required libraries by running the following Python code after launching this container.\n", "```bash\n", diff --git a/notebooks/movie-lens-example.ipynb b/notebooks/movie-lens-example.ipynb index c73cae25ad..a6d6bc2b7d 100755 --- a/notebooks/movie-lens-example.ipynb +++ b/notebooks/movie-lens-example.ipynb @@ -59,7 +59,7 @@ "\n", "\n", "### 1.1 Docker containers\n", - "Please make sure that you have started the notebook inside the running NGC docker container: `nvcr.io/nvidia/merlin/merlin-training:22.02`. The HugeCTR Python interface have been installed to the system path `/usr/local/hugectr/lib/`. Besides, this system path is added to the environment variable `PYTHONPATH`, which means that you can use the HugeCTR Python interface within the docker container environment.\n", + "Please make sure that you have started the notebook inside the running NGC docker container: `nvcr.io/nvidia/merlin/merlin-training:22.03`. The HugeCTR Python interface have been installed to the system path `/usr/local/hugectr/lib/`. Besides, this system path is added to the environment variable `PYTHONPATH`, which means that you can use the HugeCTR Python interface within the docker container environment.\n", "\n", "### 1.2 Hardware\n", "This notebook requires a Pascal, Volta, Turing, Ampere or newer GPUs, such as P100, V100, T4 or A100. " diff --git a/notebooks/news-example.ipynb b/notebooks/news-example.ipynb index c7b652c75c..df34cb1587 100644 --- a/notebooks/news-example.ipynb +++ b/notebooks/news-example.ipynb @@ -29,7 +29,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Please remember to run this jupyter notebook in the [merlin-training:22.02](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-training) docker container." + "Please remember to run this jupyter notebook in the [merlin-training:22.03](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-training) docker container." ] }, { diff --git a/onnx_converter/README.md b/onnx_converter/README.md index 450180d1da..b3e4666322 100644 --- a/onnx_converter/README.md +++ b/onnx_converter/README.md @@ -34,7 +34,7 @@ There are several ways to install this package. **Use NGC Container** -In the docker image: `nvcr.io/nvidia/merlin/merlin-training:22.02`, hugectr2onnx is already installed and you can directrly import this package via: +In the docker image: `nvcr.io/nvidia/merlin/merlin-training:22.03`, hugectr2onnx is already installed and you can directrly import this package via: ```python import hugectr2onnx ``` diff --git a/release_notes.md b/release_notes.md index 351200a82f..91ba6d72d2 100755 --- a/release_notes.md +++ b/release_notes.md @@ -1,4 +1,32 @@ # Release Notes +## What's New in Version 3.4.1 ++ **Support mixed precision inference for dataset with multiple labels**: We enable FP16 for the `Softmax` layer and support mixed precision for multi-label inference. For more information, please refer to [Inference API](docs/python_interface.md#inference-api). + ++ **Support multi-GPU offline inference with Python API**: We support multi-GPU offline inference with the Python interface, which can leverage [Hierarchical Parameter Server](docs/hugectr_parameter_server.md) and enable concurrent execution on multiple devices. For more information, please refer to [Inference API](docs/python_interface.md#inference-api) and [Multi-GPU Offline Inference Notebook](notebooks/multi_gpu_offline_inference.ipynb). + ++ **Introduction to metadata.json**: We add the introduction to `_metadata.json` for Parquet datasets. For more information, please refer to [Parquet](docs/python_interface.md#parquet). + ++ **Documents and tool for workspace size per GPU estimation**: we add a tool named [embedding_workspace_calculator](tools/embedding_workspace_calculator) to help calculate `workspace_size_per_gpu_in_mb` required by hugectr.SparseEmbedding. For more information, please refer to [embedding_workspace_calculator/README.md](tools/embedding_workspace_calculator/README.md) and [QA 24](docs/QAList.md#24-how-to-set-workspace_size_per_gpu_in_mb-and-slot_size_array). + ++ **Improved Debugging Capability**: The old logging system, which was flagged as deprecated for some time has been removed. All remaining log messages and outputs have been revised and migrated to the new logging system (base/debug/logging.hpp/cpp). During this revision, we also adjusted log levels for log messages throughout the entire codebase to improve visibility of relevant information. + ++ **Support HDFS Parameter Server in Training**: + + Decoupled HDFS in Merlin containers to make the HDFS support more flexible. Users can now compile HDFS related functionalities optionally. + + Now supports loading and dumping models and optimizer states from HDFS. + + Added a [notebook](notebooks/training_with_hdfs.ipynb) to show how to use HugeCTR with HDFS. + ++ **Support Multi-hot Inference on Hugectr Backend**: We support categorical input in multi-hot format for HugeCTR Backend inference. + ++ **Multi-label inference with mixed precision**: Mixed precision training is enabled for softmax layer. + ++ **Python Script and documentation demonstrating how to analyze model files**: In this release, we provide a script to retreive vocabulary information from model file. Please find more details on the [readme](tools/model_analyzer/README.md) + ++ **Bug Fixing**: + + Mirror strategy bug in SOK (see in https://github.com/NVIDIA-Merlin/HugeCTR/issues/291) + + Can't import sparse operation kit in nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.03 (see in https://github.com/NVIDIA-Merlin/HugeCTR/issues/296) + + HPS: Fixed access violation that can occur during initialization when not configuring a volatile DB. + + ## What's New in Version 3.4 + **Supporting HugeCTR Development with Merlin Unified Container**: From Merlin v22.02 we encourage you to develop HugeCTR under Merlin Unified Container (release container) according to the instructions in [Contributor Guide](docs/hugectr_contributor_guide.md) to keep consistent. @@ -202,6 +230,6 @@ + HugeCTR uses NCCL to share data between ranks, and NCCL may require shared system memory for IPC and pinned (page-locked) system memory resources. When using NCCL inside a container, it is recommended that you increase these resources by issuing: `-shm-size=1g -ulimit memlock=-1` See also [NCCL's known issue](https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/troubleshooting.html#sharing-data). And the [GitHub issue](https://github.com/NVIDIA-Merlin/HugeCTR/issues/243). -+ Softmax layer currently does not support fp16 mode. - + KafkaProducers startup will succeed, even if the target Kafka broker is unresponsive. In order to avoid data-loss in conjunction with streaming model updates from Kafka, you have to make sure that a sufficient number of Kafka brokers is up, working properly and reachable from the node where you run HugeCTR. + ++ The number of data files in the file list should be no less than the number of data reader workers. Otherwise, different workers will be mapped to the same file and data loading does not progress as expected. diff --git a/samples/criteo/README.md b/samples/criteo/README.md index 3ec97e268b..433d2d0835 100644 --- a/samples/criteo/README.md +++ b/samples/criteo/README.md @@ -11,11 +11,11 @@ HugeCTR is available as buildable source code, but the easiest way to install an 1. Pull the HugeCTR NGC Docker by running the following command: ```bash - $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` 2. Launch the container in interactive mode with the HugeCTR root directory mounted into the container by running the following command: ```bash - $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Build the HugeCTR Docker Container on Your Own ### diff --git a/samples/criteo_multi_slots/README.md b/samples/criteo_multi_slots/README.md index bc466af479..8a87d067f9 100644 --- a/samples/criteo_multi_slots/README.md +++ b/samples/criteo_multi_slots/README.md @@ -11,11 +11,11 @@ HugeCTR is available as buildable source code, but the easiest way to install an 1. Pull the HugeCTR NGC Docker by running the following command: ```bash - $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` 2. Launch the container in interactive mode with the HugeCTR root directory mounted into the container by running this command: ```bash - $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Build the HugeCTR Docker Container on Your Own ### diff --git a/samples/dcn/README.md b/samples/dcn/README.md index b52c8c6371..f3de28a45f 100644 --- a/samples/dcn/README.md +++ b/samples/dcn/README.md @@ -11,11 +11,11 @@ HugeCTR is available as buildable source code, but the easiest way to install an 1. Pull the HugeCTR NGC Docker by running the following command: ```bash - $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` 2. Launch the container in interactive mode with the HugeCTR root directory mounted into the container by running the following command: ```bash - $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Build the HugeCTR Docker Container on Your Own ### diff --git a/samples/deepfm/README.md b/samples/deepfm/README.md index a556c245e0..b33ab0f18a 100644 --- a/samples/deepfm/README.md +++ b/samples/deepfm/README.md @@ -11,11 +11,11 @@ HugeCTR is available as buildable source code, but the easiest way to install an 1. Pull the HugeCTR NGC Docker by running the following command: ```bash - $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` 2. Launch the container in interactive mode with the HugeCTR root directory mounted into the container by running the following command: ```bash - $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Build the HugeCTR Docker Container on Your Own ### diff --git a/samples/din/README.md b/samples/din/README.md index c1f3ea6c11..387d8dc263 100644 --- a/samples/din/README.md +++ b/samples/din/README.md @@ -11,11 +11,11 @@ HugeCTR is available as buildable source code, but the easiest way to install an 1. Pull the HugeCTR NGC Docker by running the following command: ```bash - $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` 2. Launch the container in interactive mode with the HugeCTR root directory mounted into the container by running the following command: ```bash - $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Build the HugeCTR Docker Container on Your Own ### diff --git a/samples/dlrm/README.md b/samples/dlrm/README.md index a7e7e352f7..094a47f5bb 100644 --- a/samples/dlrm/README.md +++ b/samples/dlrm/README.md @@ -16,11 +16,11 @@ HugeCTR is available as buildable source code, but the easiest way to install an 1. Pull the HugeCTR NGC Docker by running the following command: ```bash - $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` 2. Launch the container in interactive mode with the HugeCTR root directory mounted into the container by running the following command: ```bash - $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Build the HugeCTR Docker Container on Your Own ### diff --git a/samples/ncf/README.md b/samples/ncf/README.md index 2d64a41f4a..5109f63787 100644 --- a/samples/ncf/README.md +++ b/samples/ncf/README.md @@ -11,11 +11,11 @@ HugeCTR is available as buildable source code, but the easiest way to install an 1. Pull the HugeCTR NGC Docker by running the following command: ```bash - $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` 2. Launch the container in interactive mode with the HugeCTR root directory mounted into the container by running the following command: ```bash - $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Build the HugeCTR Docker Container on Your Own ### diff --git a/samples/wdl/README.md b/samples/wdl/README.md index 00b9368460..27f63198ef 100644 --- a/samples/wdl/README.md +++ b/samples/wdl/README.md @@ -11,11 +11,11 @@ HugeCTR is available as buildable source code, but the easiest way to install an 1. Pull the HugeCTR NGC Docker by running the following command: ```bash - $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker pull nvcr.io/nvidia/merlin/merlin-training:22.03 ``` 2. Launch the container in interactive mode with the HugeCTR root directory mounted into the container by running the following command: ```bash - $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.02 + $ docker run --gpus=all --rm -it --cap-add SYS_NICE -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-training:22.03 ``` ### Build the HugeCTR Docker Container on Your Own ### diff --git a/sparse_operation_kit/ReadMe.md b/sparse_operation_kit/ReadMe.md index 30ffdd4cfe..756919e1b9 100644 --- a/sparse_operation_kit/ReadMe.md +++ b/sparse_operation_kit/ReadMe.md @@ -18,7 +18,7 @@ Due to SOK is compatible with DP training provided by common synchronized traini There are several ways to install this package.
### *Install this module along with HugeCTR* ### -In the docker image: `nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.02`, SparseOpeationKit is already installed, and you can directrly import this module via: +In the docker image: `nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.03`, SparseOpeationKit is already installed, and you can directrly import this module via: ```python import sparse_opeation_kit as sok ``` diff --git a/sparse_operation_kit/notebooks/ReadMe.md b/sparse_operation_kit/notebooks/ReadMe.md index aa48964492..c8c7b4f4d0 100644 --- a/sparse_operation_kit/notebooks/ReadMe.md +++ b/sparse_operation_kit/notebooks/ReadMe.md @@ -5,7 +5,7 @@ This directory contains a set of Jupyter Notebook demos for SparseOperationKit. Before trying the notebooks here, you have to follow [these instructions](../../notebooks/README.md#Quickstart) to prepare the operating environment. Summarized bellow: + Pull the NGC Docker ```shell -$ docker pull nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.02 +$ docker pull nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.03 ``` + Clone the HugeCTR Repo ```shell @@ -14,7 +14,7 @@ $ git clone https://github.com/NVIDIA/HugeCTR hugectr + Start the Jupyter Notebook - Launch the container in interactive mode and mount the HugeCTR root directory into the container for your convenience by running this command: ```shell - $ docker run --runtime=nvidia --rm -it -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr -p 8888:8888 nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.02 + $ docker run --runtime=nvidia --rm -it -u $(id -u):$(id -g) -v $(pwd):/hugectr -w /hugectr -p 8888:8888 nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.03 ``` - Start Jupyter using these commands: ```shell diff --git a/sparse_operation_kit/notebooks/benchmark.ipynb b/sparse_operation_kit/notebooks/benchmark.ipynb index 271c8c4267..6b08aa9c70 100755 --- a/sparse_operation_kit/notebooks/benchmark.ipynb +++ b/sparse_operation_kit/notebooks/benchmark.ipynb @@ -36,10 +36,10 @@ "\n", "## Requirement\n", "\n", - "This notebook is designed to run with the Merlin Tensorflow docker image nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.02, which can be obtained from the NVIDIA GPU cloud [Merlin page](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-tensorflow-training).\n", + "This notebook is designed to run with the Merlin Tensorflow docker image nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.03, which can be obtained from the NVIDIA GPU cloud [Merlin page](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-tensorflow-training).\n", "\n", "```\n", - "docker run --runtime=nvidia --net=host --rm -it -v $(pwd):/hugectr -w /hugectr -p 8888:8888 nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.02\n", + "docker run --runtime=nvidia --net=host --rm -it -v $(pwd):/hugectr -w /hugectr -p 8888:8888 nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.03\n", "```\n", "\n", "Then from within the container, start the Jupyter notebook server with:\n", diff --git a/sparse_operation_kit/notebooks/demo_with_nvtabular_datareader.ipynb b/sparse_operation_kit/notebooks/demo_with_nvtabular_datareader.ipynb index 2ea2ab127c..bb5d7d267d 100755 --- a/sparse_operation_kit/notebooks/demo_with_nvtabular_datareader.ipynb +++ b/sparse_operation_kit/notebooks/demo_with_nvtabular_datareader.ipynb @@ -82,7 +82,7 @@ "### Get SOK from NGC\n", "\n", "The SparseOperationKit is preinstalled in the [Merlin Tensorflow Training Container](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-tensorflow-training). You can simply start the notebook inside the running NGC docker container:\n", - "`nvcr.io/nvidia/merlin/merlin-training:22.02`\n", + "`nvcr.io/nvidia/merlin/merlin-training:22.03`\n", "\n", "And you can check the existence of required libraries by running the following Python code after launching this container.\n", "\n", diff --git a/sparse_operation_kit/notebooks/sparse_operation_kit_demo.ipynb b/sparse_operation_kit/notebooks/sparse_operation_kit_demo.ipynb index a8494ffef4..0e512bac04 100644 --- a/sparse_operation_kit/notebooks/sparse_operation_kit_demo.ipynb +++ b/sparse_operation_kit/notebooks/sparse_operation_kit_demo.ipynb @@ -43,7 +43,7 @@ "\n", "\n", "+ **Get SOK from NGC**
\n", - "The SparseOperationKit is preinstalled in the [Merlin Tensorflow Training Container](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-tensorflow-training): `nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.02`.
\n", + "The SparseOperationKit is preinstalled in the [Merlin Tensorflow Training Container](https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-tensorflow-training): `nvcr.io/nvidia/merlin/merlin-tensorflow-training:22.03`.
\n", "You can check the existence of required libraries by running the following Python code after launching this container.\n", "```shell\n", "$ python3 -c \"import sparse_operation_kit as sok\"\n",