Optimizing PyTorch models with Neural Network Compression Framework of OpenVINO by 8-bit quantization.
This tutorial demonstrates how to use NNCF 8-bit quantization to optimize the PyTorch model for inference with OpenVINO Toolkit. For more advanced usage refer to these examples.
This notebook is based on 'ImageNet training in PyTorch' example. To make downloading and training fast, we use a ResNet-18 model with the Tiny ImageNet dataset.
It consists of the following steps:
- Transform the original FP32 model to INT8
- Use fine-tuning to restore the accuracy
- Export optimized and original models to ONNX and then to OpenVINO
- Measure and compare the performance of the models
If you have not done so already, please follow the Installation Guide to install all required dependencies.