Even better images in Astro

[kurtextrem]

Body

Summary

Astro's <Image /> component implements many must-have optimizations already. However, there is one thing it does not to at the moment: Auto optimizing for high and low DPR screens to achieve a better image quality and higher efficiency (less downloaded kB), where this proposal comes in handy.

Background & Motivation

After reading the blog post Better images, I thought: Why not go a step further than automatically converting to WebP, by converting images to different qualities and dimensions automatically.

By crafting a specific <img> sizes and srcset attribute, we can make sure:

• devices with high DPR screens get a high DPR image, that is capped to 2x (see a study about this here)
• reduce the image quality of high DPR images by 50% as described by Jake Archibald's "Serving Sharp Images to High Density Screens"
• devices with low DPR screens get a low DPR image (= we do not serve something heavier than needed)

By adding this further step of automatic conversion, we can achieve images that weight less on smaller screens, vary images based on the DPR of the screen and at the same time keep sharp images on devices that need them. No server needed, a build step is enough for this (perfect for Astro).

Goals

• We can serve lower quality images and sharp, higher quality images dynamically...
• We can compress higher quality images more aggressively which allows them to stay performant even if they have twice the px, as innovated by Jake Archibald
• We can do this without changing the HTML output too drastically by applying the "boolean <img>" technique as described my my blog post

Example

Taken from the mentioned Astro blog post, the very first example. I could imagine it to look like this:

Config

import { defineConfig } from "astro/config"

export default defineConfig({
	experimental: { 
       assets: true,
       assetDefaultBreakpoint: '480px', // <-- string; This is needed to separate smaller screens
       assetDefaultQuality: 75, // number; 75% of the original file
       assetHighDPRSmallQuality: 35, //  number; 35% of the original file
       assetHighDPRLargeQuality: 50, //  number; 50% of the original file 
    },
})

mdx

---
// Import the <Image /> component
import { Image } from "astro:assets"
// Import a reference to the image itself
import myImage from "../assets/penguin.png"
---

<Image src={myImage} alt="A very cool penguin!" 
   autoDPR={{ 
      small: { w: 100, h: 100 }, large: { w: 300, h: 300 },
      quality: { default: 75, small: 35, large: 50 }, breakpoint: '480px' 
   }} />

• The autoDPR attribute would be the toggle to enable the automatic conversion
• The value of the attribute can be used to:
  • override the dimensions for small and large screens (based on the breakpoint configured above)
  • override the defaults for the quality and for the breakpoint for individual images

output

<!-- Result: -->
<img
	src="/_astro/penguin.XXXXXX.webp"
	width="300"
	height="300"
	loading="lazy"
	decoding="async"
	alt="A very cool penguin!"
    sizes="
    (max-width: 480px) and (-webkit-device-pixel-ratio: 1) 1px,
    (min-width: 481px) and (-webkit-device-pixel-ratio: 1) 2px,
    (max-width: 480px) and (-webkit-min-device-pixel-ratio: 2) 3px,
    (min-width: 481px) and (-webkit-min-device-pixel-ratio: 2) 16px"
    srcset="
    /_astro/penguin.XXXXXX-100x100-1x-q75.webp 1w,
    /_astro/penguin.XXXXXX-300x300-1x-q75.webp 2w,
    /_astro/penguin.XXXXXX-100x100-2x-q35.webp 15w,
    /_astro/penguin.XXXXXX-300x300-2x-q50.webp 16w"
/>

As the file names show, for each image, we'd have to generate 4 other images. This can extend processing time for a lot of images, so being selective with autoDPR makes sense here (e.g. the savings for a 15x15 image are probably not worth the increase in HTML output).
Each image quality can be configured with the options above, where after the hash of the file (here XXXXXX), follow the dimensions, the DPR target and the quality of the image. Of course, this could also be factored into the hash to make it more compact.

---

Really looking forward to your feedback!

[lilnasy]

I'm new to the idea of 2x, 4x as sizes. Is there a benefit to them if you already have, say, 200w, 400w, 800w, 1200w in the source set?

[markpinero]

AFAIK, no. Based on this blog post:

It’s smart enough to know that on a low-resolution desktop screen if the image will be displayed at 800 pixels wide, it should choose an asset from the list that is at least 800 pixels wide.

It will also know that if the image will be displayed at 320 pixels wide on a retina screen, it should choose an asset that is at least 640 pixels wide. So you don’t have to worry about 1x and 2x assets. All you have to do is provide a good set of images, and the browser will do the rest.

[kurtextrem]

Yes, there is a difference. Let's take a look at an example:
On mobile, the element has a width of 400x400 (400w), on desktop 800x800 (800w).

If you specify 400w, 800w: on non-high-DPR devices, it will pick 400w and 800w as expected.
But now the high DPR ratios come into play. How do you specify an image for a desktop 4k monitor without making devices with a DPR of 3-4x (e.g. Samsung phones!) also pick that image? If you have solved that, how do you make sure browsers don't download pictures that contain more pixels than humans can see (see linked study, cap to 2x)?
That's only possible through the "boolean" image technique from my blog post - at least on the <img> tag (<picture> is easier to handle).

https://kurtextrem.de/posts/modern-way-of-img#best-of-both-worlds-the-boolean-img-tag <-- Here I run through all the details

[tony-sull]

Is there a benefit to using an <img> tag in this way compared to using a <picture> element?

I've used the 200w, 400w syntax on images before for basic responsive images but for anything more complex I end up using <picture> with additional formats as well - now you have me very curious of scenarios where I still may want to reach for <img> instead!

---

In general for both @astrojs/image and now astro:assets I've always been a bit hesititant to get too complex with the API. We need to document the full API surface and can ideally provide clear best practices when to reach for various image techniques.

If nothing else, I'd want to make sure this specific use case for <img> is doable with the underlying getImage() API to build the different image assets, even if the built-in <Image> component doesn't support it directly.

[kurtextrem]

Personally, I'd say it makes sense to keep using the <img> tag whenever you want the input as close as possible to the output (e.g. when devs expect if you use <Image>, you get <img>). However, it is from my POV also perfectly valid to entirely go with <picture> instead of <img> as output for <Image>.

[alexanderniebuhr]

Really agree with the improvements here. I would also like to stress that a new "version" of images in Astro should be adapter agnostic. So it should work in all runtimes, regardless if it is Node or V8 based. So we would need to make sure that everything we design, is supported by all these runtimes and potential tooling needs.

Question would all the suggested ideas work in these Environments, e.g. with jSquash?
Or would we need to come up with custom tooling to support it.

[kurtextrem]

As long as the library we're using in some environment supports a "quality" option and width/height, we're good to go. There is no other magic or similar needed thankfully. As far as I can tell jSquash meets those conditions perfectly (so does Squoosh itself).

[alexanderniebuhr]

great, as far as I followed discussions:

• Squoosh is deprecated & to big for V8 on Cloudflare
• Sharp does not run on V8
• jSquash should work, but haven't seen it implemented other than in examples
• Maybe there are other alternatives

[spaceemotion]

I was looking at adding AVIF support for our images today, but I think we have to stick with WebP for now. The native <Picture /> component only supports a unified quality setting for all formats, which actually makes it hard to dial in a good value. As such, our webp images are smaller than the AVIF because they require a higher quality.

It would be great if there's a way to specify quality options per format instead.