Skip to content

Latest commit

 

History

History
166 lines (121 loc) · 9.82 KB

README.md

File metadata and controls

166 lines (121 loc) · 9.82 KB
Raw

Benchmark on Deep Learning Frameworks and GPUs

Performance of popular deep learning frameworks and GPUs are compared, including the effect of adjusting the floating point precision (the new Volta architecture allows performance boost by utilizing half/mixed-precision calculations.)

Deep Learning Frameworks

Note: Docker images available from NVIDIA GPU Cloud were used so as to make benchmarking controlled and repeatable by anyone.

  • PyTorch 0.3.0

    • docker pull nvcr.io/nvidia/pytorch:17.12

  • PyTorch 1.0.0 (CUDA 10.0, cuDNN 7.4.2)

    • docker pull nvcr.io/nvidia/pytorch:19.01-py3 (note: requires login API key to NGC registry)

  • Caffe2 0.8.1

    • docker pull nvcr.io/nvidia/caffe2:17.12

  • TensorFlow 1.4.0 (note: this is TensorFlow 1.4.0 compiled against CUDA 9 and CuDNN 7)

    • docker pull nvcr.io/nvidia/tensorflow:17.12

  • TensorFlow 1.5.0

  • TensorFlow 1.12.0 (CUDA 10.0, cuDNN 7.4.2)

    • docker pull nvcr.io/nvidia/tensorflow:19.01-py3 (note: requires login API key to NGC registry)

  • MXNet 1.0.0 (anyone interested?)

    • docker pull nvcr.io/nvidia/mxnet:17.12

  • CNTK (anyone interested?)

    • docker pull nvcr.io/nvidia/cntk:17.12

GPUs

Model Architecture Memory CUDA Cores Tensor Cores F32 TFLOPS F16 TFLOPS* Retail Cloud
Tesla V100 Volta 16GB HBM2 5120 640 15.7 125 $3.06/hr (p3.2xlarge)
Titan V Volta 12GB HBM2 5120 640 15 110 $2999 N/A
1080 Ti Pascal 11GB GDDR5 3584 0 11 N/A $699 N/A
2080 Ti Turing 11GB GDDR6 4352 544 13.4 56.9 $1299 N/A

*: F16 (single precision) TFLOPS on TensorCores.

CUDA / CuDNN

  • CUDA 9.0.176
  • CuDNN 7.0.0.5
  • NVIDIA driver 387.34.

Except where noted.

Networks

  • VGG16
  • Resnet152
  • Densenet161
  • Any others you might be interested in?

Benchmark Results

PyTorch 0.3.0

The results are based on running the models with images of size 224 x 224 x 3 with a batch size of 16. "Eval" shows the duration for a single forward pass averaged over 20 passes. "Train" shows the duration for a pair of forward and backward passes averaged over 20 runs. In both scenarios, 20 runs of warm up is performed and those are not counted towards the measured numbers.

Titan V gets a significant speed up when going to half precision by utilizing its Tensor cores, while 1080 Ti gets a small speed up with half precision computation. Similarly, the numbers from V100 on an Amazon p3 instance is shown. It is faster than Titan V and the speed up when going to half-precision is similar to that of Titan V.

32-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
Titan V 31.3ms 108.8ms 48.9ms 180.2ms 52.4ms 174.1ms
1080 Ti 39.3ms 131.9ms 57.8ms 206.4ms 62.9ms 211.9ms
V100 (Amazon p3, CUDA 9.0.176, CuDNN 7.0.0.3) 26.2ms 83.5ms 38.7ms 136.5ms 48.3ms 142.5ms
2080 Ti 30.5ms 102.9ms 41.9ms 157.0ms 47.3ms 160.0ms

16-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
Titan V 14.7ms 74.1ms 26.1ms 115.9ms 32.2ms 118.9ms
1080 Ti 33.5ms 117.6ms 46.9ms 193.5ms 50.1ms 191.0ms
V100 (Amazon p3, CUDA 9.0.176, CuDNN 7.0.0.3) 12.6ms 58.8ms 21.7ms 92.9ms 35.7ms 102.3ms
2080 Ti 23.6ms 99.3ms 31.3ms 133.0ms 35.5ms 135.8ms

PyTorch 1.0.0

32-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
2080 Ti (CUDA 10.0.130, CuDNN 7.4.2.24) 28.0ms 95.5ms 41.8ms 142.5ms 45.4ms 148.4ms

16-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
2080 Ti (CUDA 10.0.130, CuDNN 7.4.2.24) 19.1ms 68.1ms 25.0ms 98.6ms 30.1ms 110.8ms

Tensorflow 1.4.0

32-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
Titan V 31.8ms 157.2ms 50.3ms 269.8ms
1080 Ti 43.4ms 131.3ms 69.6ms 300.6ms
2080 Ti 31.3ms 99.4ms 43.2ms 187.7ms

16-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
Titan V 16.1ms 96.7ms 28.4ms 193.3ms
1080 Ti 38.6ms 121.1ms 53.9ms 257.0ms
2080 Ti 24.9ms 81.8ms 31.9ms 155.5ms

TensorFlow 1.5.0

32-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
V100 24.0ms 71.7ms 39.4ms 199.8ms

16-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
V100 13.6ms 49.4ms 22.6ms 147.4ms

TensorFlow 1.12.0

32-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
2080 Ti (CUDA 10.0.130, CuDNN 7.4.2.24) 28.8ms 90.8ms 43.6ms 191.0ms

16-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
2080 Ti (CUDA 10.0.130, CuDNN 7.4.2.24) 18.7ms 58.6ms 25.8ms 133.5ms

Caffe2 0.8.1

32-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
Titan V 57.5ms 185.4ms 74.4ms 214.1ms
1080 Ti 47.0ms 158.9ms 77.9ms 223.9ms

16-bit

Model vgg16 eval vgg16 train resnet152 eval resnet152 train densenet161 eval densenet161 train
Titan V 41.6ms 156.1ms 56.9ms 172.7ms
1080 Ti 40.1ms 137.8ms 61.7ms 184.1ms

Comparison Graphs

Comparison of Titan V vs 1080 Ti, PyTorch 0.3.0 vs Tensorflow 1.4.0 vs Caffe2 0.8.1, and FP32 vs FP16 in terms of images processed per second:

vgg16-eval vgg16-train resnet152-eval resnet152-train

Contributors

  • Yusaku Sako
  • Bartosz Ludwiczuk (thank you for supplying the V100 numbers)