Skip to content

Latest commit

 

History

History
143 lines (143 loc) · 236 KB

all_hyper_params.md

File metadata and controls

143 lines (143 loc) · 236 KB
Raw
division submitter verified by submitter benchmark system system_name accelerators_per_node accelerator_model_name global_batch_size label smoothing lars_epsilon lars_opt_base_learning_rate lars_opt_end_learning_rate lars_opt_learning_rate_decay_poly_power lars_opt_learning_rate_decay_steps lars_opt_learning_rate_warmup_epochs lars_opt_momentum lars_opt_weight_decay lr_boundaries lr_decay_start_steps lr_rates max_image_size max_samples max_sequence_length min_image_size min_selfplay_games_per_generation model_bn_span num_image_candidates num_warmup_steps opt_adam_beta_1 opt_adam_beta_2 opt_adam_epsilon opt_base_learning_rate opt_epsilon opt_lamb_beta_1 opt_lamb_beta_2 opt_lamb_weight_decay_rate opt_learning_rate_alt_decay_func opt_learning_rate_alt_warmup_func opt_learning_rate_decay_boundary_epochs opt_learning_rate_decay_factor opt_learning_rate_decay_interval opt_learning_rate_decay_steps opt_learning_rate_remain_steps opt_learning_rate_training_steps opt_learning_rate_warmup_epochs opt_learning_rate_warmup_factor opt_learning_rate_warmup_steps opt_name opt_weight_decay sgd_opt_base_learning_rate sgd_opt_end_learning_rate sgd_opt_learning_rate_decay_poly_power sgd_opt_learning_rate_decay_steps sgd_opt_momentum sgd_opt_weight_decay start_warmup_step train_batch_size truncated norm initialization
closed Google unverified bert gpu-v100-8-TF2.0 8xV100 8 TPU 760 512 420 0.00035221 1.00E-06 0.91063 0.96497 0.01 8000 420 0
closed Google unverified bert tpu-v3-32-TF2.0 TPU v3-32 8 TPU v3 768 512 420 0.00035221 1.00E-06 0.91063 0.96497 0.01 8000 420 0
closed Google unverified bert tpu-v3-8192-JAX TPU v3-8192 8 TPU v3 8192 512 287 0.00288293 1.00E-06 0.88 0.88 0.0166629 560 287 -76
closed Google unverified bert tpu-v3-8192-TF TPU v3-8192 8 TPU v3 8192 512 287 0.00288293 1.00E-06 0.88 0.88 0.0166629 560 287 -76
closed Google unverified bert tpu-v4-128-TF tpu-v4-128 64 TPU 3072 512 100 0.0015 1.00E-06 0.9 0.999 0.01 1141 100 0
closed Google unverified bert tpu-v4-16-TF tpu-v4-16 8 TPU 256 512 1200 0.0015 1.00E-06 0.9 0.999 0.01 14796 1200 0
closed Google unverified bert tpu-v4-512-TF tpu-v4-512 256 TPU 3072 512 100 0.0015 1.00E-06 0.9 0.999 0.01 1141 100 -76
closed Microsoft unverified bert ND40rs_v2_ort ND40rs_v2_ort 8 Tesla V100-SXM2-32GB ERROR 0 0.0005 1.00E-06 0.9 0.999 -1 -1 -1 0
closed NVIDIA verified bert dgx1_ngc20.06_pytorch dgx1_ngc20.06_pytorch 8 NVIDIA V100-SXM2-16GB (300W) 288 0 0.0004 1.00E-06 0.9 0.999 0.01 13889 0 0
closed NVIDIA verified bert dgx2h_n32_ngc20.06_pytorch dgx2h_n32_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 9216 0 0.002 1.00E-06 0.86 0.975 0.01 750 0 0
closed NVIDIA verified bert dgx2h_n96_ngc20.06_pytorch dgx2h_n96_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 27648 0 0.003 1.00E-06 0.87 0.98 0.01 550 0 0
closed NVIDIA verified bert dgx2h_ngc20.06_pytorch dgx2h_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 432 0 0.0004 1.00E-06 0.9 0.999 0.01 8103 0 0
closed NVIDIA verified bert dgxa100_n128_ngc20.06_pytorch dgxa100_n128_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 10240 0 0.002 1.00E-06 0.86 0.975 0.01 750 0 0
closed NVIDIA verified bert dgxa100_n256_ngc20.06_pytorch dgxa100_n256_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 20480 51 0.004 1.00E-06 0.878 0.974 0.01 513 51 0
closed NVIDIA verified bert dgxa100_n2_ngc20.06_pytorch dgxa100_n2_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 432 0 0.0004 1.00E-06 0.9 0.999 0.01 8103 0 0
closed NVIDIA verified bert dgxa100_n32_ngc20.06_pytorch dgxa100_n32_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 5120 0 0.0017 1.00E-06 0.87 0.97 0.01 990 0 0
closed NVIDIA verified bert dgxa100_ngc20.06_pytorch dgxa100_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 256 0 0.00035 1.00E-06 0.9 0.999 0.01 13700 0 0
closed Google unverified dlrm tpu-v4-128-TF tpu-v4-128 64 TPU 65536 38000 0.7 32 2000 0.7 2 40000
closed Google unverified dlrm tpu-v4-16-TF tpu-v4-16 8 TPU 65536 38000 0.7 32 2000 0.7 2 40000
closed Google unverified dlrm tpu-v4-512-TF tpu-v4-512 256 TPU 65536 38000 0.7 32 2000 0.7 2 40000
closed Inspur verified dlrm nf5488_pytorch NF5488 8 NVIDIA A100 55296 49315 24 2750 24 2 27772
closed Intel verified dlrm 1-node-4s-cpx-pytorch 1-node-4s-cpx-pytorch 0 N/A 32768 70000 18 8000 18 2 30000
closed Intel verified dlrm 1-node-8s-cpx-1-pytorch 1-node-8s-cpx-1-pytorch 0 N/A 32768 70000 18 8000 18 2 30000
closed Intel verified dlrm 1-node-8s-cpx-2-pytorch 1-node-8s-cpx-2-pytorch 0 N/A 32768 70000 18 8000 18 2 30000
closed Intel verified dlrm 2-nodes-8s-cpx-pytorch 2-nodes-8s-cpx-pytorch 0 N/A 32768 70000 18 8000 18 2 30000
closed Intel verified dlrm 4-nodes-16s-cpx-pytorch 4-nodes-16s-cpx-pytorch 0 N/A 32768 70000 18 8000 18 2 30000
closed NVIDIA verified dlrm dgx2h_ngc20.06_merlin_hugectr dgx2h_ngc20.06_merlin_hugectr 16 NVIDIA V100-SXM3-32GB (450W) 55296 49315 24 0 2750 24 2 27772
closed NVIDIA verified dlrm dgx2h_ngc20.06_pytorch dgx2h_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 55296 49315 24 2750 24 2 27772
closed NVIDIA verified dlrm dgxa100_ngc20.06_merlin_hugectr dgxa100_ngc20.06_merlin_hugectr 8 NVIDIA A100-SXM4-40GB (400W) 55296 49315 24 0 2750 24 2 27772
closed NVIDIA verified dlrm dgxa100_ngc20.06_pytorch dgxa100_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 55296 49315 24 2750 24 2 27772
Closed Fujitsu unverified gnmt gx_V100x8_ngc19.05 PRIMERGY GX2570M5 8 Nvidia Tesla V100 2048 65 0.0028 TRUE TRUE 0.5 859 4 6053 200
closed Google unverified gnmt tpu-v3-1024-TF TPU v3-1024 8 TPU v3 8192 48 0.0045 0.5 241 4 1924 150
closed Google unverified gnmt tpu-v4-128-TF tpu-v4-128 64 TPU 8192 48 0.0045 0.5 241 4 1924 150
closed Google unverified gnmt tpu-v4-16-TF tpu-v4-16 8 TPU 4096 48 0.003 0.5 300 4 3000 150
closed Google unverified gnmt tpu-v4-512-TF tpu-v4-512 256 TPU 8192 48 0.0045 0.5 241 4 1924 150
closed Inspur verified gnmt nf5488_pytorch NF5488 8 NVIDIA A100 3072 75 0.9 0.999 1.00E-08 0.002875 TRUE TRUE 0.5 506 4 4054 200
closed NVIDIA verified gnmt dgx1_ngc20.06_pytorch dgx1_ngc20.06_pytorch 8 NVIDIA V100-SXM2-16GB (300W) 2048 75 0.9 0.999 1.00E-08 0.002 TRUE TRUE 0.5 809 4 6453 200
closed NVIDIA verified gnmt dgx2h_n16_ngc20.06_pytorch dgx2h_n16_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 8192 75 0.9 0.999 1.00E-08 0.004 TRUE TRUE 0.5 283 4 2259 200
closed NVIDIA verified gnmt dgx2h_n64_ngc20.06_pytorch dgx2h_n64_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 16384 75 0.9 0.999 1.00E-08 0.005 TRUE TRUE 0.5 201 4 1605 200
closed NVIDIA verified gnmt dgx2h_ngc20.06_pytorch dgx2h_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 3072 75 0.9 0.999 1.00E-08 0.002875 TRUE TRUE 0.5 506 4 4054 200
closed NVIDIA verified gnmt dgxa100_n128_ngc20.06_pytorch dgxa100_n128_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 16384 75 0.9 0.999 1.00E-08 0.005 TRUE TRUE 0.5 201 4 1605 200
closed NVIDIA verified gnmt dgxa100_n2_ngc20.06_pytorch dgxa100_n2_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 3072 75 0.9 0.999 1.00E-08 0.002875 TRUE TRUE 0.5 506 4 4054 200
closed NVIDIA verified gnmt dgxa100_n32_ngc20.06_pytorch dgxa100_n32_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 8192 75 0.9 0.999 1.00E-08 0.004 TRUE TRUE 0.5 283 4 2259 200
closed NVIDIA verified gnmt dgxa100_ngc20.06_pytorch dgxa100_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 3072 75 0.9 0.999 1.00E-08 0.002875 TRUE TRUE 0.5 506 4 4054 200
closed DellEMC unverified maskrcnn DSS8440 DSS8440 8 Nvidia Tesla V100S-PCIe-32GB 64 1333 800 8000 0.08 0.1 [18000, 24000] 0.0001 800
Closed Fujitsu unverified maskrcnn gx_V100x8_ngc19.05 PRIMERGY GX2570M5 8 Nvidia Tesla V100 48 1333 800 6000 0.06 0.1 [24000, 32000] 9.60E-05 625
closed Google unverified maskrcnn tpu-v3-1024-TF TPU v3-1024 8 TPU v3 256 1333 800 1000 0.24 0.1 [6000, 8000] 0.00012 2000
closed Google unverified maskrcnn tpu-v4-128-TF tpu-v4-128 64 TPU 256 1333 800 1000 0.24 0.1 [5625, 7500] 0.000133333 1800
closed Google unverified maskrcnn tpu-v4-16-TF tpu-v4-16 8 TPU 128 1333 800 1000 0.16 0.1 [9750, 13000] 0.00016 1000
closed Google unverified maskrcnn tpu-v4-512-TF tpu-v4-512 256 TPU 256 1333 800 1000 0.24 0.1 [6000, 8000] 0.00012 2000
closed NVIDIA verified maskrcnn dgx1_ngc20.06_pytorch dgx1_ngc20.06_pytorch 8 NVIDIA V100-SXM2-16GB (300W) 48 1333 800 6000 0.06 0.1 [24000, 32000] 9.60E-05 625
closed NVIDIA verified maskrcnn dgx2h_n12_ngc20.06_pytorch dgx2h_n12_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 192 1333 800 1000 0.24 0.1 [7000, 9333] 0.000192 1250
closed NVIDIA verified maskrcnn dgx2h_n4_ngc20.06_pytorch dgx2h_n4_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 128 1333 800 2000 0.16 0.1 [9000, 12000] 0.000256 625
closed NVIDIA verified maskrcnn dgx2h_ngc20.06_pytorch dgx2h_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 96 1333 800 6000 0.12 0.1 [12000, 16000] 0.000192 625
closed NVIDIA verified maskrcnn dgxa100_n24_ngc20.06_pytorch dgxa100_n24_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 192 1333 800 1000 0.24 0.1 [7000, 9333] 0.000192 1250
closed NVIDIA verified maskrcnn dgxa100_n2_ngc20.06_pytorch dgxa100_n2_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 96 1333 800 6000 0.12 0.1 [12000, 16000] 0.000192 625
closed NVIDIA verified maskrcnn dgxa100_n4_ngc20.06_pytorch dgxa100_n4_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 128 1333 800 4000 0.16 0.1 [9000, 12000] 0.000256 625
closed NVIDIA verified maskrcnn dgxa100_ngc20.06_pytorch dgxa100_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 64 1333 800 8000 0.08 0.1 [18000, 24000] 0.0001 800
Tencent unverified maskrcnn GN10X-8xV100 GN10X 8 NVIDIA V100-SXM2-32GB None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None
closed Google unverified minigo tpu-v4-128-TF tpu-v4-128 64 TPU 8192 0.0001 512
closed Intel verified minigo 8-nodes-32s-cpx-tensorflow 8-nodes-32s-cpx-tensorflow 0 N/A 4096 [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed NVIDIA verified minigo dgx1_ngc20.06_tensorflow dgx1_ngc20.06_tensorflow 8 NVIDIA V100-SXM2-16GB (300W) [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed NVIDIA verified minigo dgx2_ngc20.06_tensorflow dgx2_ngc20.06_tensorflow 16 NVIDIA V100-SXM3-32GB (350W) [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed NVIDIA verified minigo dgx2h_n32_ngc20.06_tensorflow dgx2h_n32_ngc20.06_tensorflow 16 NVIDIA V100-SXM3-32GB (450W) [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed NVIDIA verified minigo dgx2h_n96_ngc20.06_tensorflow dgx2h_n96_ngc20.06_tensorflow 16 NVIDIA V100-SXM3-32GB (450W) [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed NVIDIA verified minigo dgxa100_n224_ngc20.06_tensorflow dgxa100_n224_ngc20.06_tensorflow 8 NVIDIA A100-SXM4-40GB (400W) [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed NVIDIA verified minigo dgxa100_n2_ngc20.06_tensorflow dgxa100_n2_ngc20.06_tensorflow 8 NVIDIA A100-SXM4-40GB (400W) [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed NVIDIA verified minigo dgxa100_n32_ngc20.06_tensorflow dgxa100_n32_ngc20.06_tensorflow 8 NVIDIA A100-SXM4-40GB (400W) [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed NVIDIA verified minigo dgxa100_ngc20.06_tensorflow dgxa100_ngc20.06_tensorflow 8 NVIDIA A100-SXM4-40GB (400W) [128, 10000, 20000] [0.016, 0.16, 0.016, 0.0016] 8192 0.0001 4096
closed Alibaba unverified resnet v100-1280 PAI_EFLOPS 8 NVIDIA V100-SXM2-32GB 57600 23.5 0.0001 2 71 22 0.0002
closed Alibaba unverified resnet v100-512 PAI_EFLOPS 8 NVIDIA V100-SXM2-32GB 33280 21.2 0.0001 2 58 16 0.0002
closed Alibaba unverified resnet v100-256 PAI_EFLOPS 8 NVIDIA V100-SXM2-32GB 33280 21.2 0.0001 2 58 16 0.0002
closed DellEMC unverified resnet 2xC4140 2xC4140 4 Nvidia Tesla V100-SXM2-32GB 1664 0.0001 2 208 3 3 0.0001 2 42 0.9 2.50E-05
closed DellEMC unverified resnet 4xC4140 4xC4140 4 Nvidia Tesla V100-SXM2-32GB 3328 0.0001 2 208 6 3.1 0.0001 2 42 0.95 2.50E-05
closed DellEMC unverified resnet C4140 C4140 4 Nvidia Tesla V100-SXM2-32GB 832 0.0001 2 208 3 3 0.0001 2 42 0.9 2.50E-05
closed DellEMC unverified resnet DSS8440 DSS8440 8 Nvidia Tesla V100S-PCIe-32GB 1664 0.0001 2 208 3 3 0.0001 2 42 0.9 2.50E-05
Open Fujitsu unverified resnet gx_V100x8_fjmxnet PRIMERGY GX2570M5 8 Nvidia Tesla V100 1664 0 7.4 0.0001 2 35 2 0.9 0.0001 208
Closed Fujitsu unverified resnet gx_V100x8_ngc20.03 PRIMERGY GX2570M5 8 Nvidia Tesla V100 1664 0 7.4 0.0001 2 37 2 0.9 0.0001 208
closed Google unverified resnet gpu-v100-8-TF2.0 8xV100 8 TPU 2496 0 9.5 0.0001 2 17956 5 0.9 0.0002 312
closed Google unverified resnet tpu-v3-32-TF2.0 TPU v3-32 8 TPU v3 4096 0 13 0.0001 2 12169 2 0.9 0.0002 128
closed Google unverified resnet tpu-v3-8192-JAX TPU v3-8192 8 TPU v3 65536 0 23.0809773 0.0001 2 1087.25 33 0.953721378 0.0001 64
closed Google unverified resnet tpu-v3-8192-TF TPU v3-8192 8 TPU v3 65536 0 24.699 0.0001 2 1133 31 0.951807 0.0001 64
closed Google unverified resnet tpu-v4-128-TF tpu-v4-128 64 TPU 8192 0 17 0.0001 2 6720 5 0.9 0.0002 128
closed Google unverified resnet tpu-v4-16-TF tpu-v4-16 8 TPU 2048 0 8.5 0.0001 2 22530 5 0.9 0.0002 128
closed Google unverified resnet tpu-v4-512-TF tpu-v4-512 256 TPU 8192 0 17 0.0001 2 6095 5 0.9 0.0002 64
closed Inspur verified resnet nf5488_mxnet NF5488 8 NVIDIA A100 3264 0.0001 2 408 5 6 0.0001 2 42 0.9 2.50E-05
closed Intel verified resnet 1-node-8s-cpx-1-tensorflow 1-node-8s-cpx-1-tensorflow 0 N/A 4096 0 10 0.0001 2 42 5 0.9 0.0002 128
closed Intel verified resnet 1-node-8s-cpx-2-mxnet 1-node-8s-cpx-2-mxnet 0 N/A 4096 0 10 0.0001 2 5 0.9 0.0002 128 0.0002
closed NVIDIA verified resnet dgx1_ngc20.06_mxnet dgx1_ngc20.06_mxnet 8 NVIDIA V100-SXM2-16GB (300W) 1664 0.0001 2 208 3 3 0.0001 2 42 0.9 2.50E-05
closed NVIDIA verified resnet dgx2h_n32_ngc20.06_mxnet dgx2h_n32_ngc20.06_mxnet 16 NVIDIA V100-SXM3-32GB (450W) 33280 0 21.2 0.0001 2 58 16 0.94 2.50E-05 65 0.0001 2
closed NVIDIA verified resnet dgx2h_n96_ngc20.06_mxnet dgx2h_n96_ngc20.06_mxnet 16 NVIDIA V100-SXM3-32GB (450W) 59904 0 26 0.0001 2 90 33 0.94 2.50E-05 39 0.0001 2
closed NVIDIA verified resnet dgx2h_ngc20.06_mxnet dgx2h_ngc20.06_mxnet 16 NVIDIA V100-SXM3-32GB (450W) 3328 0.0001 2 208 5 6 0.0001 2 42 0.9 2.50E-05
closed NVIDIA verified resnet dgxa100_n192_ngc20.06_mxnet dgxa100_n192_ngc20.06_mxnet 8 NVIDIA A100-SXM4-40GB (400W) 44544 0 23.5 0.0001 2 70 22 0.94 2.50E-05 29 0.0001 2
closed NVIDIA verified resnet dgxa100_n230_ngc20.06_mxnet dgxa100_n230_ngc20.06_mxnet 8 NVIDIA A100-SXM4-40GB (400W) 53360 0 25 0.0001 2 82 29 0.94 2.50E-05 29 0.0001 2
closed NVIDIA verified resnet dgxa100_n2_ngc20.06_mxnet dgxa100_n2_ngc20.06_mxnet 8 NVIDIA A100-SXM4-40GB (400W) 3264 0.0001 2 204 5 6 0.0001 2 42 0.9 2.50E-05
closed NVIDIA verified resnet dgxa100_n96_ngc20.06_mxnet dgxa100_n96_ngc20.06_mxnet 8 NVIDIA A100-SXM4-40GB (400W) 34560 0 21.6 0.0001 2 59 17 0.94 2.50E-05 45 0.0001 2
closed NVIDIA verified resnet dgxa100_ngc20.06_mxnet dgxa100_ngc20.06_mxnet 8 NVIDIA A100-SXM4-40GB (400W) 3264 0.0001 2 408 5 6 0.0001 2 42 0.9 2.50E-05
SIAT unverified resnet modelarts_1024_tensorflow_open modelarts_1024_1 8 Ascend910 32768 0 25 0.0001 2 639.6 7.6 0.95 0.0001
SIAT unverified resnet modelarts_128_mindspore_open modelarts_128_1 8 Ascend910 4096
SIAT unverified resnet modelarts_256_mindspore_open modelarts_256_1 8 Ascend910 8192
SIAT unverified resnet modelarts_512_mindspore_close modelarts_512_0 8 Ascend910 32768 0 26.379 0.0001 2 2028 16 0.9202 0.0001
SIAT unverified resnet modelarts_512_tensorflow_close modelarts_512_0 8 Ascend910 32768 0 20.81 0.0001 2 2067 11 0.9395 0.0001
SIAT unverified resnet modelarts_512_tensorflow_open modelarts_512_1 8 Ascend910 16384 0 21 0.0001 2 1123 5.6 0.95 0.0001
closed DellEMC unverified ssd 2xC4140 2xC4140 4 Nvidia Tesla V100-SXM2-32GB 1024 1 128 0.1875 [44, 55] 0 601 0.00016
closed DellEMC unverified ssd 4xC4140 4xC4140 4 Nvidia Tesla V100-SXM2-32GB 896 1 56 0.09 [44, 55] 0 654 0.00013
closed DellEMC unverified ssd C4140 C4140 4 Nvidia Tesla V100-SXM2-32GB 512 1 128 0.0475 [44, 55] 0 1202 0.00016
closed DellEMC unverified ssd DSS8440 DSS8440 8 Nvidia Tesla V100S-PCIe-32GB 1024 1 128 0.1875 [44, 55] 0 572 0.00016
Closed Fujitsu unverified ssd gx_V100x8_ngc19.05 PRIMERGY GX2570M5 8 Nvidia Tesla V100 960 1 120 0.135 [36, 40] 0 650 0.00016
closed Google unverified ssd tpu-v3-4096-JAX TPU v3-4096 8 TPU v3 4096 50 16 16 [64.0, 80.0] 0.0002048 750 0.0005
closed Google unverified ssd tpu-v3-8192-TF TPU v3-8192 8 TPU v3 4096 50 64 0.00125 [72, 90] 128 721 0.0005
closed Google unverified ssd tpu-v4-128-TF tpu-v4-128 64 TPU 1024 50 64 0.0031 [40, 50] 32 577 0.0005
closed Google unverified ssd tpu-v4-16-TF tpu-v4-16 8 TPU 1024 50 64 0.0031 [40, 50] 32 577 0.0005
closed Google unverified ssd tpu-v4-512-TF tpu-v4-512 256 TPU 2048 50 64 0.0025 [56, 70] 64 1155 0.0005
closed Inspur verified ssd nf5488_pytorch NF5488 8 NVIDIA A100 912 1 114 0.09 [44, 55] [44, 55] 0 650 0.00013
closed NVIDIA verified ssd dgx1_ngc20.06_pytorch dgx1_ngc20.06_pytorch 8 NVIDIA V100-SXM2-16GB (300W) 960 1 120 0.0875 [44, 55] [44, 55] 0 650 0.00016
closed NVIDIA verified ssd dgx2h_n4_ngc20.06_pytorch dgx2h_n4_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 1536 1 24 0.14 [44, 55] [44, 55] 0 850 0.00017
closed NVIDIA verified ssd dgx2h_n64_ngc20.06_mxnet dgx2h_n64_ngc20.06_mxnet 16 NVIDIA V100-SXM3-32GB (450W) 2048 1 16 0.2925 [60, 75] 0 1488 4.00E-05
closed NVIDIA verified ssd dgx2h_ngc20.06_pytorch dgx2h_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 896 1 56 0.09 [44, 55] [44, 55] 0 650 0.00013
closed NVIDIA verified ssd dgxa100_n128_ngc20.06_mxnet dgxa100_n128_ngc20.06_mxnet 8 NVIDIA A100-SXM4-40GB (400W) 2048 1 16 0.2925 [60, 75] 0 1488 4.00E-05
closed NVIDIA verified ssd dgxa100_n2_ngc20.06_pytorch dgxa100_n2_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 896 1 56 0.09 [44, 55] [44, 55] 0 650 0.00013
closed NVIDIA verified ssd dgxa100_n64_ngc20.06_mxnet dgxa100_n64_ngc20.06_mxnet 8 NVIDIA A100-SXM4-40GB (400W) 2048 1 16 0.2925 [60, 75] 0 1488 4.00E-05
closed NVIDIA verified ssd dgxa100_n8_ngc20.06_mxnet dgxa100_n8_ngc20.06_mxnet 8 NVIDIA A100-SXM4-40GB (400W) 1536 1 24 0.14 [44, 55] 0 839 0.00017
closed NVIDIA verified ssd dgxa100_ngc20.06_pytorch dgxa100_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 912 1 114 0.09 [44, 55] [44, 55] 0 650 0.00013
closed DellEMC unverified transformer C4140 C4140 4 Nvidia Tesla V100-SXM2-32GB 61440 64 0.9 0.98 1.00E-09 0.001976 2000
Closed Fujitsu unverified transformer gx_V100x8_ngc19.05 PRIMERGY GX2570M5 8 Nvidia Tesla V100 81920 64 0.9 0.98 1.00E-09 0.00125 1100
closed Google unverified transformer tpu-v3-8192-JAX TPU v3-8192 8 TPU v3 2048 256 0.908657573 0.919871912 1.00E-09 3.309215769 664
closed Google unverified transformer tpu-v3-8192-TF TPU v3-8192 8 TPU v3 2048 80 0.88 0.92 1.00E-09 1.875 831
closed Google unverified transformer tpu-v4-128-TF tpu-v4-128 64 TPU 512 80 0.9 0.98 1.00E-09 1.875 1107
closed Google unverified transformer tpu-v4-16-TF tpu-v4-16 8 TPU 512 80 0.9 0.98 1.00E-09 1.875 1107
closed Google unverified transformer tpu-v4-512-TF tpu-v4-512 256 TPU 2048 80 0.88 0.94 1.00E-09 1.9 831
closed Inspur verified transformer nf5488_pytorch NF5488 8 NVIDIA A100 110592 64 0.9 0.98 1.00E-09 0.0019 750
closed NVIDIA verified transformer dgx1_ngc20.06_pytorch dgx1_ngc20.06_pytorch 8 NVIDIA V100-SXM2-16GB (300W) 81920 64 0.9 0.98 1.00E-09 0.0019 750
closed NVIDIA verified transformer dgx2h_n10_ngc20.06_pytorch dgx2h_n10_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 737280 76 0.86 0.92 1.00E-09 0.001732 400
closed NVIDIA verified transformer dgx2h_n60_ngc20.06_pytorch dgx2h_n60_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 737280 76 0.86 0.92 1.00E-09 0.001732 400
closed NVIDIA verified transformer dgx2h_ngc20.06_pytorch dgx2h_ngc20.06_pytorch 16 NVIDIA V100-SXM3-32GB (450W) 131072 80 0.9 0.98 1.00E-09 0.001976 1000
closed NVIDIA verified transformer dgxa100_n10_ngc20.06_pytorch dgxa100_n10_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 737280 76 0.86 0.92 1.00E-09 0.001732 400
closed NVIDIA verified transformer dgxa100_n20_ngc20.06_pytorch dgxa100_n20_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 737280 76 0.86 0.92 1.00E-09 0.001732 400
closed NVIDIA verified transformer dgxa100_n2_ngc20.06_pytorch dgxa100_n2_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 110592 64 0.9 0.98 1.00E-09 0.0019 750
closed NVIDIA verified transformer dgxa100_n60_ngc20.06_pytorch dgxa100_n60_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 737280 76 0.86 0.92 1.00E-09 0.001732 400
closed NVIDIA verified transformer dgxa100_ngc20.06_pytorch dgxa100_ngc20.06_pytorch 8 NVIDIA A100-SXM4-40GB (400W) 110592 64 0.9 0.98 1.00E-09 0.0019 750
Tencent unverified transformer GN10X-8xV100 GN10X 8 NVIDIA V100-SXM2-32GB None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None None