Skip to content

Implement performance checking script: i8 vs f16 non-fused conv kernels #1727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 10 commits into from
80 changes: 80 additions & 0 deletions mlir/utils/performance/perf-test-i8-f16-conv.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
#!/bin/bash

# Shell script that captures the performance difference between data types fp16 and int8 to validate expected kernel performance.
# The script is currently comparing only fp16 vs int8 performance of non-fused kernels (only convolution).
# Usage: ./performance-checking.sh --d <model> --p <model_path> [--r <number_of_iterations>]

MODEL_NAME="resnet50-fp16"
MODEL_PATH="/mnt/sc_nas_share/migraphx/models/mlperf/resnet50_v1.onnx"
RUNS=5

export PATH="$HOME/AMDMIGraphX/build/bin:$PATH" # path to migraphx-driver

while [[ $# -gt 0 ]]; do
case "$1" in
--d)
MODEL_NAME="$2"
shift 2
;;
--p)
MODEL_PATH="$2"
shift 2
;;
--r)
RUNS="$2"
shift 2
;;
--help)
echo "Usage: $0 --d <model> --p <model_path> [--r <number_of_iterations>]"
exit
;;
*)
echo "Option $1 doesn't exist"
exit 1
;;
esac
done

mkdir -p "$MODEL_NAME"

# Compile model to generate .mxr files
MIGRAPHX_MLIR_DUMP_TO_MXR="$MODEL_NAME" \
MIGRAPHX_ENABLE_NHWC=1 \
MIGRAPHX_ENABLE_HIPBLASLT_GEMM=1 \
MIGRAPHX_MLIR_USE_SPECIFIC_OPS="convolution,~fused,~dot" \
migraphx-driver compile "$MODEL_PATH" --fp16 --exhaustive-tune

# Convert each .mxr to .py
ls "$MODEL_NAME"/*.mxr | xargs -I {} -n 1 migraphx-driver read "{}" --py -o "{}".py

# Benchmark a set of .py testcases
run_benchmark() {
local label="$1"
echo "$label:" >> "$MODEL_NAME/times"

for testcase in "$MODEL_NAME"/*.py; do
test_name=$(basename "$testcase")
total_time=0

compiled="$testcase.mxdb"
migraphx-driver compile "$testcase" --mlir -o "$compiled" > /dev/null
Copy link
Member

@umangyadav umangyadav Jun 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this script is in rocMLIR ? Looks like better place is inside migraphx-benchmark-utils
https://github.com/ROCm/migraphx-benchmark-utils

Copy link
Contributor Author

@dorde-antic dorde-antic Jun 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On MLIR Standup (in january) we talked to put it in /mlir/utils/performance. But I agree with what you have said. Should I open PR there and put this script then? @umangyadav

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And would it be in this directory: https://github.com/ROCm/migraphx-benchmark-utils/tree/main/scripts and to add it to readme table? @umangyadav

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can close this one


for ((i = 1; i <= RUNS; i++)); do
migraphx-driver time "$compiled" > "$MODEL_NAME/results.out"
run_time=$(awk -F'[/ ]' '/Total time/{print substr($3, 1, length($3)-2)}' "$MODEL_NAME/results.out")
echo "$run_time"
total_time=$(awk -v total="$total_time" -v run="$run_time" 'BEGIN {print total + run}')
done

avg_time=$(awk -v total="$total_time" -v runs="$RUNS" 'BEGIN {print total / runs}')
echo "$test_name,$avg_time" >> "$MODEL_NAME/times"
done
}

echo "NEW RUN" >> "$MODEL_NAME/times"
run_benchmark "FP16"

# Modify the testcases for int8 quantization
sed -i -e "s/half_type/int8_type/" -e 's/convolution/quant_convolution/' "$MODEL_NAME"/*.py

run_benchmark "INT8"