Investigate not great Weka performance

[knikolla]

Possibly network? Consult with Hakan and Justin.

If so, we need to come up with a plan to directly connect the Weka storage to the H100 nodes.

[sanastas]

It seems problematic that Weka is in a test cluster, and (AFAIK) H100s are going to be in the production cluster. But there need to be something else performance-wise, as MLPerf Storage benchmark results are not as presented elsewhere.

[sanastas]

This is also relevant here: https://docs.google.com/document/d/1gAixZ5Xkmym_LyGck9U3iDi6j-GXlvZMSRZB-r-cuP0/edit?usp=sharing