Handle default inlining on funcOp and callOp

[raikonenfnu]

After llvm/llvm-project#119970 landed upstream, we are running issues compiling some of our models. It throws off:

_ IREE compile and run: open-llama-3b-v2-f16::open-llama-3b-v2-f16.mlirbc::open-llama-3b-v2-f16.mlirbc::gpu_rocm::real_weights_prefill _
Error invoking iree-compile
Error code: 1
Stderr diagnostics:
open-llama-3b-v2-f16.mlirbc:0:0: error: attempting to parse a byte at the end of the bytecode
open-llama-3b-v2-f16.mlirbc:0:0: note: in bytecode version 6 produced by: MLIR19.0.0git


/home/esaimana/iree_tests_cache/artifacts/sd3_vae/model.mlirbc:0:0: error: attempting to parse a byte at the end of ...esaimana/iree_tests_cache/artifacts/sd3_vae/model.mlirbc:0:0: note: in bytecode version 6 produced by: MLIR19.0.0git

error: expected mlir::ArrayAttr, but got: "__auto.t5...ome/nod/iree_tests_cache/artifacts/sd3_clip/model.mlirbc:0:0: note: in bytecode version 6 produced by: MLIR19.0.0git

A good way to repro this issue is to do:

wget https://sharkpublic.blob.core.windows.net/sharkpublic/sai/sd3-mmdit/model.mlirbc -O mmdit.mlirbc
/path/to/iree-opt mmdit.mlirbc -o /dev/null

expected output looks something like:
error: expected mlir::ArrayAttr, but got: "__auto.t5...ome/nod/iree_tests_cache/artifacts/sd3_clip/model.mlirbc:0:0: note: in bytecode version 6 produced by: MLIR19.0.0git

wget https://sharkpublic.blob.core.windows.net/sharkpublic/sai/sdxl-scheduled-unet/model.mlirbc
/path/to/iree-opt mmdit.mlirbc -o /dev/null

Expected output looks something like:
attempting to parse a byte at the end of note: in bytecode version 6 produced by: MLIR19.0.0git

[ScottTodd]

I find it strange that adding a new optional attribute broke parsing, but only for MLIR bytecode and not MLIR text. Is that the expected behavior?

---

Sample logs: https://github.com/iree-org/iree/actions/runs/12358432483

We can update our tests as needed... and this could be a good forcing function.

• The test_models tests are using https://github.com/nod-ai/SHARK-TestSuite/tree/main/iree_tests/pytorch/models and https://github.com/nod-ai/SHARK-TestSuite/tree/main/iree_tests/sharktank.
  • Those have been due for a migration to https://github.com/iree-org/iree-test-suites (see this issue: Organize full model integration tests in some central location iree-test-suites#49).
  • The llama and punet tests there also overlap with other tests so they could be deleted.
  • Regeneration steps for the remaining files are listed at https://github.com/nod-ai/SHARK-TestSuite/tree/main/iree_tests#pytorch-models.
• The test_regression_suite tests are pulling files from Azure like here:

  iree/experimental/regression_suite/shark-test-suite-models/sdxl/test_clip.py

  Lines 55 to 58 in 1894af3

  	sdxl_clip_mlir = fetch_source_fixture(
  	"https://sharkpublic.blob.core.windows.net/sharkpublic/sai/sdxl-prompt-encoder/model.mlirbc",
  	group="sdxl_clip",
  	)

  • The regeneration process is not documented for any of those files and they haven't been updated since they were added 5 months ago.

[ScottTodd]

Sent nod-ai/SHARK-TestSuite#417 and nod-ai/SHARK-TestSuite#418 to delete some old tests that are (almost) redundant at this point.

[raikonenfnu]

Cross posting from our convo on discord:

looks like the root issue It is to do with the inline attribute(per the reverted commit), i.e we'd need the util.globalOp to have that noinline attribute on i.e:

  util.global private @_params.unet.time_embedding.linear_1.bias {noinline} = #stream.parameter.named<"model"::"unet.time_embedding.linear_1.bias"> : tensor<1280xf16>
  util.global private @_params.unet.time_embedding.linear_2.weight {noinline} = #stream.parameter.named<"model"::"unet.time_embedding.linear_2.weight"> : tensor<1280x1280xf16>
  util.global private @_params.unet.time_embedding.linear_2.bias {noinline} = #stream.parameter.named<"model"::"unet.time_embedding.linear_2.bias"> : tensor<1280xf16>
  util.global private @_params.unet.add_embedding.linear_1.weight {noinline} = #stream.parameter.named<"model"::"unet.add_embedding.linear_1.wei

I think we just need to regenerate the IR without modifying any of the code and should generate the {noinline} attribute now by default! :slight_smile:

I tested the hypothesis above locally, i.e:

(with reverted noinline) iree-compile model.mlirbc --compile-to=input -o model_input.mlir (the IR generated her has {noinline} autogenerated)

(with unrevert the noinline patch) iree-opt model.mlirbc -> same error
(with unrevert the noinline patch) iree-opt model_input.mlir -> works!

[raikonenfnu]

I find it strange that adding a new optional attribute broke parsing, but only for MLIR bytecode and not MLIR text. Is that the expected behavior?

Not familiar with these side of the stack so not sure also, all I know is bytecode == smaller memory footprint + non-readable. 😅

The regeneration process is not documented for any of those files and they haven't been updated since they were added 5 months ago.

Yeah that's unfortunate, what do you think if I just do an iree-opt on it to get some noinline attributes on it and then convert it to back to an mlirbc? Or would that be too hacky?

[benvanik]

util.global should use inlining_policy, e.g.

util.global private @_params.classifier.weight {inlining_policy = #util.inline.never} : tensor<30x20xf32>

[raikonenfnu]

util.global should use inlining_policy

Ah interesting, need to figure out if this gets autogenerated now from sharktank (unless here knows). But in that case, we may need to either:

a) regenerate with correct inlining policy (a bit more challenging because regeneration process is not documented)
or
b) iree-opt to get text, insert it with some regex, and re-encode it back to mlirbc

LMK if you guys have a preference on which solution works better, or if there' something else we can do to resolve this

[raikonenfnu]

Hey @ScottTodd, fyi I have regenerated the MLIR and uploaded them to the Azure, plus regenerated the mlirbc in SHARK-TestSuite to the latest version here https://github.com/nod-ai/SHARK-TestSuite/tree/raikonen/integrate/llvm20241220.