ORT dist - Use dispatch() for SD 1.5, SD Turbo and Whisper Base

[ibelem]

compute() will be deprecated and removed in favor of dispatch().

Update latest dev version of ORT dists which support dispatch() for SD 1.5, SD Turbo and Whisper Base demos.

• SD 1.5 / ORT 1.21.0-dev.20241109-d3ad76b2cf
• SD Turbo / ORT 1.21.0-dev.20241109-d3ad76b2cf
• Whisper Base / ORT 1.21.0-dev.20241109-d3ad76b2cf
• Image Classification / Transformers.js dist built with ORT 1.21.0-dev.20241109-d3ad76b2cf
• Segment Anything / Test dists by using dispatch() in https://github.com/microsoft/webnn-developer-preview/pull/52/files#diff-9940e30cb48d0308e828926531652ef2298a6c43260f69c1ba58fee8ea255435
• Phi-3 Mini / Test dists by using dispatch() in https://github.com/microsoft/webnn-developer-preview/pull/52/files#diff-9940e30cb48d0308e828926531652ef2298a6c43260f69c1ba58fee8ea255435, will update to ORT dev version after merging [WebNN] Fixed WebNN Module undefined issue onnxruntime#22795

@fdwr PTAL

[fdwr]

Thanks Belem. The dev change makes sense, but stable being empty?

[eyaler]

@ibelem @fdwr I am seeing a considerable slowdown in the sd-turbo performance with unet inference times going from 100ms to 1000ms. this happens when changing from 1.20.0-dev.20240927-b81e76b9a6 (or 1.20.0-dev.20240919-bd60add8ce used here before this commit) to 1.20.0-dev.20240928-1bda91fc57 (or later including 1.21.0-dev.20241109-d3ad76b2cf of this commit and the latest 1.21.0-dev.20241127-b930b4ab5b). The main change seems to be the dispatch() change in ORT. I did not change anything in the code other than the ORT version.

[ibelem]

eyaler Thanks for the report! What's your detailed test environment? We looked at the performance gap between compute() and dispatch(), but didn't see a 10x performance drop when comparing the daily performance test reports for unet models. Did you clear the cache/memory when compare the performance?

• CPU model
• GPU model
• GPU driver version
• Memory

[eyaler]

@ibelem

AMD Ryzen 7 6800H
RTX 3070 Ti Laptop (8 GB)
Nvidia Studio driver 566.14 (latest)
64GB RAM
Chrome 131.0.6778.86

not sure about clearing cache/memory - what is the recommended procedure? i did switch ORT versions back and forth multiple times and could consistently see the performance differences correlated with the compute/dispatch change for the UNET (as well as the VAE encoder in my im2im fork)

[ibelem]

Thanks @eyaler , I just wanted to check if you are under clean test environment (e.g. no other tabs openned or no other backgound heavy applications are running) since there is a knonw memory increasing issue for Tab (blink process).

We don't have RTX 3070 Ti Laptop but the RTX 4070S, the first and second inference times are:

1	2	3	4
167.30	26.60	24.80	23.10
93.80	25.90	25.20	23.80

ONNX Runtime Web: 1.21.0-dev.20241122-a2ba3cb547 dispatch()

Considering the performance gap between 3070Ti Laptop and 4070S, the ~26ms is the expected results on 4070S.

Processor	12th Gen Intel(R) Core(TM) i9-12900K   3.20 GHz
Installed RAM	32.0 GB (31.7 GB usable)
GPU: RTX 4070 SUPER 
GPU Driver: 32.0.15.6603
Edition	Windows 11 Enterprise
Version	23H2
OS build	22631.4460
Tested on Chrome Canary 133.0.6873.0

Have you tried on https://microsoft.github.io/webnn-developer-preview/demos/sd-turbo/ directly? Thanks!