Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shortfin LLM CI Fails on export after IREE nightly 1227 #729

Open
renxida opened this issue Dec 30, 2024 · 2 comments
Open

Shortfin LLM CI Fails on export after IREE nightly 1227 #729

renxida opened this issue Dec 30, 2024 · 2 comments

Comments

@renxida
Copy link
Contributor

renxida commented Dec 30, 2024

Shortfin LLM CI fails in sharktank.examples.export_paged_llm_v1 with error:

TypeError: get(): incompatible function arguments
Supported: get(value: str, context: mlir.ir.Context | None = None) -> iree.compiler._mlir_libs._mlir.ir.FlatSymbolRefAttr

when

            iree-base-compiler==3.1.0rc20241227 \
            iree-base-runtime==3.1.0rc20241227 \
            iree-turbine==3.1.0rc20241227

@nithinsubbiah mentions in slack that this issue also seems to affect SDXL export functionality

To resolve this:

  1. Pin IREE to version 1226 temporarily
  2. Address IREE 1227 compatibility issue

This is likely due to iree-org/iree and not iree-turbine: iree-turbine has not been committed to between Dec 20 and Dec 30. IREE had a llvm version bump on Dec 26

@renxida
Copy link
Contributor Author

renxida commented Dec 30, 2024

renxida added a commit to renxida/shark-ai that referenced this issue Dec 30, 2024
renxida added a commit to renxida/shark-ai that referenced this issue Dec 30, 2024
@renxida
Copy link
Contributor Author

renxida commented Dec 30, 2024

https://github.com/nod-ai/shark-ai/actions/runs/12550021461?pr=730

Pinning the iree-base-compiler, iree-base-runtime, and iree-turbine verisons to 1226 fixes the export failure but the server still fails to start.

Also, almost all of the sharktank tests fails when i do this.

marbre pushed a commit to marbre/shark-ai that referenced this issue Jan 2, 2025
renxida added a commit to renxida/shark-ai that referenced this issue Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant