DLRM suggest benchmark

[JunxiChhen]

Hi,
I read about your doc and the example of benchmark only has CNN related model.
And what benchmark tool do you suggest to use with DLRM and running on AMD platform using ZenDNN?

Thanks

[ratan-prasad]

ZenDNN library is intended to be used in conjunction with the frameworks like TensorFlow, PyTorch, ONNXRT and cannot be used independently.
ZenDNN release v3.3 is available at https://developer.amd.com/zendnn/

We recommend to refer the official DLRM https://github.com/facebookresearch/dlrm.git for benchmarking.

[JunxiChhen]

Thank you! It is really helpful. BTW, how about the suggest benchmark of Bert-Large?

[JunxiChhen]

ZenDNN library is intended to be used in conjunction with the frameworks like TensorFlow, PyTorch, ONNXRT and cannot be used independently. ZenDNN release v3.3 is available at https://developer.amd.com/zendnn/

We recommend to refer the official DLRM https://github.com/facebookresearch/dlrm.git for benchmarking.

When I ran the official dlrm benchmark with ZenDNN v3.3 pytorch (python3.8), there is an issue:
RuntimeError: could not execute a primitive
And when I use previous ZenDNN version (v3.3), there is a core dumped issue:
Segmentation fault (core dumped)
The issue happened when min-batch-size equal and larger than 128 on my test vm (m6a.16xlarge).

These issues didn't happen when I didn't use ZenDNN.
So are there any tricks to avoid them?

The benchmark cmd:
export OMP_NUM_THREADS=8 && export GOMP_CPU_AFFINITY=0-7 && numactl --physcpubind=0-7 -m 0 python dlrm_s_pytorch.py --mini-batch-size=128 --test-num-workers=0 --num-batches=100 --data-generation=random --arch-mlp-bot=512-512-64 --arch-mlp-top=1024-1024-1024-1 --arch-sparse-feature-size=64 --arch-embedding-size=1000000-1000000-1000000-1000000-1000000-1000000-1000000-1000000 --num-indices-per-lookup=100 --arch-interaction-op=dot --numpy-rand-seed=727 --print-freq=100 --print-time --inference-only

[JunxiChhen]

Segmentation fault (core dumped) occurred using ZenDNN3.2 pytorch when using criteo data benchmark on dlrm_s_pytorch.py (in https://github.com/facebookresearch/dlrm.git)

[ratan-prasad]

Thank you! It is really helpful. BTW, how about the suggest benchmark of Bert-Large?

Hi, Please refer the Bert: https://github.com/google-research/bert

[ratan-prasad]

Segmentation fault (core dumped) occurred using ZenDNN3.2 pytorch when using criteo data benchmark on dlrm_s_pytorch.py (in https://github.com/facebookresearch/dlrm.git)
Hi, we found the issue when running official dlrm benchmark and criteo data benchmark.
We will submit the fix very soon.

[JunxiChhen]

Segmentation fault (core dumped) occurred using ZenDNN3.2 pytorch when using criteo data benchmark on dlrm_s_pytorch.py (in https://github.com/facebookresearch/dlrm.git)
Hi, we found the issue when running official dlrm benchmark and criteo data benchmark.
We will submit the fix very soon.

Thank you. And how about the "RuntimeError: could not execute a primitive" using zendnn3.3?

[ratan-prasad]

Segmentation fault (core dumped) occurred using ZenDNN3.2 pytorch when using criteo data benchmark on dlrm_s_pytorch.py (in https://github.com/facebookresearch/dlrm.git)
Hi, we found the issue when running official dlrm benchmark and criteo data benchmark.
We will submit the fix very soon.

Thank you. And how about the "RuntimeError: could not execute a primitive" using zendnn3.3?

That is also fixed. The fix will be submitted very soon.