VAE tiling seems to skip the last several tiles during processing

[lostdisc]

With both SD 1.5 and SDXL, running with VAE tiling seems to skip the last several tiles during processing.

With SD 1.5, VAE tiling has 16 tiles, but skips to the end after doing 8. The relevant part of the verbose output looks like this:

[DEBUG] ggml_extend.hpp:549  - tile work buffer size: 0.77 MB
[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
[INFO ] ggml_extend.hpp:563  - processing 16 tiles
  |===>                                              | 1/16 - 0.00it/s[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 1/16 - 2.18s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |======>                                           | 2/16 - 2.20s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=========>                                        | 3/16 - 2.25s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |============>                                     | 4/16 - 2.17s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |===============>                                  | 5/16 - 2.17s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |==================>                               | 6/16 - 2.13s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 7/16 - 2.14s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 8/16 - 2.22s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================================| 16/16 - 2.22s/it
[DEBUG] stable-diffusion.cpp:1080 - computing vae [mode: DECODE] graph completed, taking 21.88s
[INFO ] stable-diffusion.cpp:1487 - latent 1 decoded, taking 21.88s
[INFO ] stable-diffusion.cpp:1491 - decode_first_stage completed, taking 21.88s
[INFO ] stable-diffusion.cpp:1614 - txt2img completed in 57.88s
save result image to 'output.png'

With SDXL, VAE tiling has 64 tiles, but skips to the end after doing 48. The relevant part of the verbose output looks like this:

[DEBUG] ggml_extend.hpp:513  - tile work buffer size: 0.77 MB
[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
[INFO ] ggml_extend.hpp:527  - processing 64 tiles
  |>                                                 | 1/64 - 0.00it/s[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |>                                                 | 1/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=>                                                | 2/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==>                                               | 3/64 - 1.75s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 4/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 5/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====>                                             | 6/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====>                                            | 7/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |======>                                           | 8/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======>                                          | 9/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======>                                          | 10/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |========>                                         | 11/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========>                                        | 12/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========>                                       | 13/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========>                                       | 14/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===========>                                      | 15/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============>                                     | 16/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=============>                                    | 17/64 - 1.72s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============>                                   | 18/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============>                                   | 19/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===============>                                  | 20/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================>                                 | 21/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================>                                | 22/64 - 1.68s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================>                                | 23/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================>                               | 24/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================>                              | 25/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====================>                             | 26/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 27/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 28/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |======================>                           | 29/64 - 1.73s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======================>                          | 30/64 - 1.74s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |========================>                         | 31/64 - 1.73s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 32/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 33/64 - 1.58s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========================>                       | 34/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===========================>                      | 35/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============================>                     | 36/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============================>                     | 37/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=============================>                    | 38/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============================>                   | 39/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===============================>                  | 40/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================================>                 | 41/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================================>                 | 42/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================================>                | 43/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================>               | 44/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================================>              | 45/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================================>              | 46/64 - 1.72s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====================================>             | 47/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================================>            | 48/64 - 1.74s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================================| 64/64 - 1.74s/it
[DEBUG] stable-diffusion.cpp:1045 - computing vae [mode: DECODE] graph completed, taking 83.93s
[INFO ] stable-diffusion.cpp:1416 - latent 1 decoded, taking 83.93s
[INFO ] stable-diffusion.cpp:1420 - decode_first_stage completed, taking 83.93s
[INFO ] stable-diffusion.cpp:1539 - txt2img completed in 346.14s
save result image to 'output.png'

And for visual comparison... First, here is an SD 1.5 image generated with non-tiled VAE:

And here is how it comes out with tiled VAE:

The colors look a little different if you flip between the two, but I don't see an obvious boundary in the tiled version, FWIW.

[olivbrau]

I also noticed that, but have not yet found bad result : my images are correct.
May be it is only a simple error in % calculation ? (because of a bad total ?)

[stduhpf]

I'm pretty sure #484 fixes that. This was not the initial goal of the PR, but I can't reproduce this issue with these changes merged in.

[lostdisc]

Neat, thanks!