Fix test failure in test_fuse_pointwise

[pfultz2]

No description provided.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #4033      +/-   ##
===========================================
- Coverage    92.11%   92.03%   -0.08%     
===========================================
  Files          530      530              
  Lines        24472    24498      +26     
===========================================
+ Hits         22541    22546       +5     
- Misses        1931     1952      +21     

Files with missing lines	Coverage Δ
src/fuse_pointwise.cpp	98.51% <100.00%> (+0.03%)	⬆️
src/instruction.cpp	88.30% <100.00%> (+0.04%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[migraphx-bot]

Test	Batch	Rate new 71f31d	Rate old fce74e	Diff	Compare
torchvision-resnet50	64	3,258.01	3,237.29	0.64%	✅
torchvision-resnet50_fp16	64	6,913.17	6,875.20	0.55%	✅
torchvision-densenet121	32	2,448.94	2,444.78	0.17%	✅
torchvision-densenet121_fp16	32	4,212.94	4,185.93	0.65%	✅
torchvision-inceptionv3	32	1,629.49	1,617.74	0.73%	✅
torchvision-inceptionv3_fp16	32	2,722.57	2,707.57	0.55%	✅
cadene-inceptionv4	16	760.87	755.79	0.67%	✅
cadene-resnext64x4	16	815.25	813.93	0.16%	✅
slim-mobilenet	64	7,476.90	7,439.91	0.50%	✅
slim-nasnetalarge	64	209.76	208.65	0.53%	✅
slim-resnet50v2	64	3,349.26	3,332.41	0.51%	✅
bert-mrpc-onnx	8	1,147.40	1,142.60	0.42%	✅
bert-mrpc-tf	1	472.12	462.31	2.12%	✅
pytorch-examples-wlang-gru	1	338.95	343.13	-1.22%	✅
pytorch-examples-wlang-lstm	1	472.04	486.93	-3.06%	🔴
torchvision-resnet50_1	1	804.94	803.37	0.19%	✅
cadene-dpn92_1	1	411.34	414.94	-0.87%	✅
cadene-resnext101_1	1	388.89	392.67	-0.96%	✅
onnx-taau-downsample	1	396.44	395.19	0.32%	✅
dlrm-criteoterabyte	1	32.35	32.26	0.27%	✅
dlrm-criteoterabyte_fp16	1	51.35	51.26	0.17%	✅
agentmodel	1	10,353.67	10,477.47	-1.18%	✅
unet_fp16	2	59.58	59.43	0.25%	✅
resnet50v1_fp16	1	1,026.56	1,041.11	-1.40%	✅
resnet50v1_int8	1	1,060.12	1,068.70	-0.80%	✅
bert_base_cased_fp16	64	1,175.66	1,169.98	0.49%	✅
bert_large_uncased_fp16	32	358.15	356.28	0.53%	✅
bert_large_fp16	1	200.02	200.05	-0.02%	✅
distilgpt2_fp16	16	2,243.54	2,229.84	0.61%	✅
yolov5s	1	541.61	545.53	-0.72%	✅
tinyllama	1	43.88	43.60	0.63%	✅
vicuna-fastchat	1	45.07	44.86	0.47%	✅
whisper-tiny-encoder	1	419.26	418.31	0.23%	✅
whisper-tiny-decoder	1	410.94	402.69	2.05%	✅
llama2_7b	1	19.11	19.05	0.32%	✅
qwen1.5-7b	1	23.55	23.43	0.50%	✅
phi3-3.8b	1	26.63	26.54	0.36%	✅
mask-rcnn	1	12.81	12.81	0.05%	✅
llama3-8b	1	21.77	21.66	0.52%	✅
whisper-large-encoder	1	10.22	10.18	0.46%	✅
whisper-large-decoder	1	101.23	100.97	0.26%	✅
mistral-7b	1	23.77	23.68	0.38%	✅
FLUX.1-schnell	1	769.09	776.17	-0.91%	✅
nan	nan	nan	nan	nan%	❌

This build is not recommended to merge 🔴

[migraphx-bot]

     ✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

❌bert-mrpc-tf: ERROR - check error output

2025-05-28 04:48:08.324382: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1748425693.702622 185219 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 62973 MB memory: -> device: 0, name: AMD Instinct MI250X/MI250, pci bus id: 0000:b3:00.0
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1748425694.571469 185219 mlir_graph_optimization_pass.cc:401] MLIR V1 optimization pass is not enabled
2025-05-28 04:48:24.087893: E external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:250] bitcode module is required by this HLO module but was not found at ./opencl.bc
2025-05-28 04:48:24.087945: E external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:250] bitcode module is required by this HLO module but was not found at ./opencl.bc
2025-05-28 04:48:24.088001: E external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:250] bitcode module is required by this HLO module but was not found at ./opencl.bc
2025-05-28 04:48:24.088060: E external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:250] bitcode module is required by this HLO module but was not found at ./opencl.bc
2025-05-28 04:48:24.088089: E external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:250] bitcode module is required by this HLO module but was not found at ./opencl.bc
2025-05-28 04:48:24.088274: E external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:250] bitcode module is required by this HLO module but was not found at ./opencl.bc
2025-05-28 04:48:24.088328: E external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:250] bitcode module is required by this HLO module but was not found at ./opencl.bc
2025-05-28 04:48:24.088382: E external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:250] bitcode module is required by this HLO module but was not found at ./opencl.bc
error: Failure when generating HSACO
error: Failure when generating HSACO
error: Failure when generating HSACO
error: Failure when generating HSACO
error: Failure when generating HSACO
error: Failure when generating HSACO
error: Failure when generating HSACO
error: Failure when generating HSACO
2025-05-28 04:48:24.089427: E tensorflow/compiler/mlir/tools/kernel_gen/tf_framework_c_interface.cc:228] INTERNAL: Generating device code failed.
2025-05-28 04:48:24.090721: W tensorflow/core/framework/op_kernel.cc:1829] UNKNOWN: JIT compilation failed.
2025-05-28 04:48:24.090740: I tensorflow/core/framework/local_rendezvous.cc:405] Local rendezvous is aborting with status: UNKNOWN: JIT compilation failed.
[[{{node import/bert/embeddings/LayerNorm/moments/SquaredDifference}}]]
2025-05-28 04:48:24.090753: I tensorflow/core/framework/local_rendezvous.cc:405] Local rendezvous is aborting with status: UNKNOWN: JIT compilation failed.
[[{{node import/bert/embeddings/LayerNorm/moments/SquaredDifference}}]]
[[import/loss/output/_21]]
2025-05-28 04:48:24.090770: I tensorflow/core/framework/local_rendezvous.cc:424] Local rendezvous recv item cancelled. Key hash: 11217777527359497193
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/client/session.py", line 1407, in _do_call
return fn(*args)
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/client/session.py", line 1390, in _run_fn
return self._call_tf_sessionrun(options, feed_dict, fetch_list,
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/client/session.py", line 1483, in _call_tf_sessionrun
return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
(0) UNKNOWN: JIT compilation failed.
[[{{node import/bert/embeddings/LayerNorm/moments/SquaredDifference}}]]
[[import/loss/output/_21]]
(1) UNKNOWN: JIT compilation failed.
[[{{node import/bert/embeddings/LayerNorm/moments/SquaredDifference}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 340, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 324, in main
y_out = sess.run(y, feed_dict=tf_dict)
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/client/session.py", line 977, in run
result = self._run(None, fetches, feed_dict, options_ptr,
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/client/session.py", line 1220, in _run
results = self._do_run(handle, final_targets, final_fetches,
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/client/session.py", line 1400, in _do_run
return self._do_call(_run_fn, feeds, fetches, targets, options,
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/client/session.py", line 1426, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.UnknownError: Graph execution error:

Detected at node 'import/bert/embeddings/LayerNorm/moments/SquaredDifference' defined at (most recent call last):
Node: 'import/bert/embeddings/LayerNorm/moments/SquaredDifference'
Detected at node 'import/bert/embeddings/LayerNorm/moments/SquaredDifference' defined at (most recent call last):
Node: 'import/bert/embeddings/LayerNorm/moments/SquaredDifference'
2 root error(s) found.
(0) UNKNOWN: JIT compilation failed.
[[{{node import/bert/embeddings/LayerNorm/moments/SquaredDifference}}]]
[[import/loss/output/_21]]
(1) UNKNOWN: JIT compilation failed.
[[{{node import/bert/embeddings/LayerNorm/moments/SquaredDifference}}]]
0 successful operations.
0 derived errors ignored.

Original stack trace for 'import/bert/embeddings/LayerNorm/moments/SquaredDifference':

     ✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     ✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     ✅ agentmodel: PASSED: MIGraphX meets tolerance

🔴unet: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ resnet50v1: PASSED: MIGraphX meets tolerance

     ✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ bert_large: PASSED: MIGraphX meets tolerance

     ✅ yolov5s: PASSED: MIGraphX meets tolerance

     ✅ tinyllama: PASSED: MIGraphX meets tolerance

     ✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     ✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance

     ✅ llama2_7b: PASSED: MIGraphX meets tolerance

     ✅ qwen1.5-7b: PASSED: MIGraphX meets tolerance

     ✅ phi3-3.8b: PASSED: MIGraphX meets tolerance

🔴mask-rcnn: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ llama3-8b: PASSED: MIGraphX meets tolerance

     ✅ whisper-large-decoder: PASSED: MIGraphX meets tolerance

     ✅ mistral-7b: PASSED: MIGraphX meets tolerance

     ✅ FLUX.1-schnell: PASSED: MIGraphX meets tolerance