Add safe area and document coordinate systems #3890

madsmtm · 2024-08-26T03:13:56Z

Resolves #2308, by adding Window::safe_area, which describes the area of the surface that is unobstructed by notches, bezels etc. The drawing code in the examples have been updated to draw a star inside the safe area, and the plain background outside of it.

Also renamed Window::inner_position (introduced in #430) to Window::surface_position, and changed it to from screen coordinates to window coordinates, to better align how these coordinate systems work together.

Finally, I've added some SVG images and documentation to describe how these coordinate systems work together. The images should be auto-adjusting to the page theme when viewed on docs.rs. See also #3891, whether we use window coordinates or surface coordinates is currently a bit confusing.

Completing implementation on all platforms is tracked in #3910

Fixes #1122
Fixes #2308
Fixes #3742
Fixes #2066
Fixes #2347
Fixes #2235

kchibisov

Can macOS backend provide safe area when the transparent decorations are used? The ones where the buttons are over the main surface.

src/window.rs

kchibisov · 2024-08-26T11:46:01Z

docs/res/coordinate-systems-desktop.svg

positions are points, outer_size is window_size.

outer_size is not window_size currently, and I'm not sure I want it to be? It's already defined on Window, so it seems a bit excessive to repeat "window". If anything, I'd rather call it just size, but that might lead people to use that over surface_size.

What do you mean by "positions are points"?

outer_size is not window_size currently, and I'm not sure I want it to be? It's already defined on Window, so it seems a bit excessive to repeat "window". If anything, I'd rather call it just size, but that might lead people to use that over surface_size.

it's a size with all the decorations, which is what we have and that's what we're reporting, so it's effectively a window size.

If you think of window as a part without decorations, probably then not, but in the current state it's window_size from what I can say.

What do you mean by "positions are points"?

I mean, you show them with lines as in they are areas, but you should just draw them as a point.

I mean, you show them with lines as in they are areas, but you should just draw them as a point.

Ah, yeah, but I wasn't sure how to represent what the point was relative to?

maybe color coding, so different coordinate spaces will be differently colored, though, you already do so?

I have pushed a new commit that updates the diagram to do something like this:

src/window.rs

madsmtm · 2024-08-26T16:05:10Z

Can macOS backend provide safe area when the transparent decorations are used? The ones where the buttons are over the main surface.

Didn't even know we had WindowAttributesExtMacOS::with_titlebar_transparent, but yeah, that should be possible.

I was kinda hoping I could define safe_area to not include CSDs, but seems like I'll have to update the docs to include it as well :/.

kchibisov · 2024-08-26T16:41:12Z

I was kinda hoping I could define safe_area to not include CSDs, but seems like I'll have to update the docs to include it as well :/.

You just need to y offset, and that's about it from what I can say.

Added `Window::safe_area`, which describes the area of the surface that is unobstructed by notches, bezels etc. The drawing code in the examples have been updated to draw a star inside the safe area, and the plain background outside of it. Also renamed `Window::inner_position` to `Window::surface_position`, and changed it to from screen coordinates to window coordinates, to better align how these coordinate systems work together. Finally, added some SVG images and documentation to describe how all of this works.

Dunno, I can imagine that it will happen on some platforms

madsmtm · 2024-09-10T07:28:30Z

I have ensured that safe_area includes the title bar when with_titlebar_transparent + with_titlebar_hidden is set on macOS

Very bare-bones, probably not correct

madsmtm · 2024-09-10T09:06:09Z

src/window.rs

+    ///
+    /// - **Android / Orbital / Wayland / Web / Windows / X11:** Unimplemented, returns `((0, 0),
+    ///   surface_size)`.
+    fn safe_area(&self) -> (PhysicalPosition<u32>, PhysicalSize<u32>);


Note that I considered letting this be safe_area_insets, i.e. return { left, right, top, bottom } instead, since I think that's generally a nicer API for users. I decided against it for now, since we'd have to define yet another type, and then I'd rather re-use our existing types.

The position and size is generally better, since it's easier to forward stuff like that into the e.g. glViewport stuff, since you generally want an offset and not left/right.

kchibisov · 2024-09-11T09:56:40Z

examples/window.rs

+        let mut buffer = self.surface.buffer_mut()?;
+
+        // Fill the whole surface with a plain background
+        buffer.fill(match self.theme {
+            Theme::Light => 0xffffffff, // White
+            Theme::Dark => 0xff181818,  // Dark gray
+        });
+
+        // Draw a star (without anti-aliasing) inside the safe area
+        let surface_size = self.window.surface_size();
+        let (origin, size) = self.window.safe_area();
+        let in_star = |x, y| -> bool {
+            // Shamelessly adapted from https://stackoverflow.com/a/2049593.
+            let sign = |p1: (i32, i32), p2: (i32, i32), p3: (i32, i32)| -> i32 {
+                (p1.0 - p3.0) * (p2.1 - p3.1) - (p2.0 - p3.0) * (p1.1 - p3.1)
+            };
+
+            let pt = (x as i32, y as i32);
+            let v1 = (0, size.height as i32 / 2);
+            let v2 = (size.width as i32 / 2, 0);
+            let v3 = (size.width as i32, size.height as i32 / 2);
+            let v4 = (size.width as i32 / 2, size.height as i32);
+
+            let d1 = sign(pt, v1, v2);
+            let d2 = sign(pt, v2, v3);
+            let d3 = sign(pt, v3, v4);
+            let d4 = sign(pt, v4, v1);
+
+            let has_neg = (d1 < 0) || (d2 < 0) || (d3 < 0) || (d4 < 0);
+            let has_pos = (d1 > 0) || (d2 > 0) || (d3 > 0) || (d4 > 0);

-        let color = match self.theme {
-            Theme::Light => WHITE,
-            Theme::Dark => DARK_GRAY,
+            !(has_neg && has_pos)
        };
+        for y in 0..size.height {
+            for x in 0..size.width {
+                if in_star(x, y) {
+                    let index = (origin.y + y) * surface_size.width + (origin.x + x);
+                    buffer[index as usize] = match self.theme {
+                        Theme::Light => 0xffe8e8e8, // Light gray
+                        Theme::Dark => 0xff525252,  // Medium gray
+                    };
+                }
+            }


That looks a bit complicated, maybe just different coloring for something inside the unsafe area should be fine?

The nice thing about the star is that then you can very clearly see whether it lines up with the edges or not.

For example on Mac Catalyst, the surface is actually obscured by the title bar, but I couldn't see this just by colouring the unsafe part differently.

kchibisov · 2024-09-11T09:58:27Z

src/platform_impl/linux/wayland/window/mod.rs

@@ -338,6 +337,11 @@ impl CoreWindow for Window {
        super::logical_to_physical_rounded(window_state.outer_size(), scale_factor)
    }

+    fn safe_area(&self) -> (PhysicalPosition<u32>, PhysicalSize<u32>) {
+        // FIXME: Include CSDs drawn by Winit


Drop this thing, they don't draw into the surface like you have on macOS, so it's all irrelevant for them, since they can not physically overlap, since they are on a completely different surface.

Hmm, should we then update surface_position to include the CSDs drawn by Winit?

On Wayland, I think it won't change anything since CSDs are in negative part of the window, so the surface is still at (0,0), it's just CSDs are in (-border, -header), so don't have to do anything, since position is at (0,0).

Wow, that's wild! How does the compositor allow you to create and draw surfaces outside the bounds that you are otherwise restricted to?

You're not restricted to anything on Wayland and negative geometry is how on protocol level was defined for this exact reason I guess, so you don't change the way normal stuff works.

So, everything we're doing is how it's done by everyone else and nothing more to it.

And in the end of the day, user has full control over drawing, and they can draw things as big as they want, compositor may decide cropping what they draw, but it's not an error by any means.

src/platform_impl/android/mod.rs

src/window.rs

src/lib.rs

kchibisov · 2024-09-11T11:17:41Z

Just thinking more about it and that you should generally always call safe_area inside your rendering is that, I actually can not do that to achieve good behavior, since the only place where I can call it to achieve what I want (sync resize of terminal content and window) is during the SurfaceResized event, since I need to alter the terminal viewport size which I'll send via signal downstream during it, otherwise if I do that unconditionally in rendering part, I'd need more frames.

So RedrawRequsted is not the greatest place for that, since you generally know that you got resized way before that, and that. It also doesn't mean that you have to redraw, so I'd probably try to put it into its own event, then the users can decide whether they want to redraw or not, instead of us doing so for them, what do you think?

madsmtm · 2024-09-11T11:23:31Z

Hmm, if you set the terminal viewport to only be the safe area, what does Alacritty intend to draw in the unsafe area, then?

kchibisov · 2024-09-11T11:27:40Z

Hmm, if you set the terminal viewport to only be the safe area, what does Alacritty intend to draw in the unsafe area, then?

just padding with background color, it's just the safe area changes the size of the content, which we should forward down the line for us? It's not really an issue for regular GUIs, I guess.

To support on Alacritty

madsmtm · 2024-09-11T11:45:19Z

Okay, so if it's something that only affects Alacritty, then I'd argue that we don't need the event, since Alacritty only works on desktop anyhow, and I'm fine for now with guaranteeing for that case that on macOS, a change to the safe area will always correspond with a SurfaceResized ; see 62a95a5.

I'm not in general against an event for it, but it's kinda cumbersome to set up, and I'd like to see a better use-case for it - and besides, it can be added after this PR.

kchibisov · 2024-09-11T12:03:14Z

I don't think that the issue is specific to alacritty though, it's specific to any application that computes layout and then decides whether to render or not.

Re-running your layout calculation each time during rendering is not that great, unless you're egui or something, but for retained UIs they probably want to react to state a bit earlier.

I'd also not mention alacritty in the source, since it makes it sound like its problem, where in reality any program that forwards sizes to some external process will work the same, like vnc, etc.

In general though, I'd assume that on mobile it's not really an issue since you have rotation event, when it'll change the safe area and you can just call it once, since it doesn't change over time.

The thing about safe area though, is that it can not really change for the given monitor, because it's a hardware property, what changes is the state of winit decorations and the state where this safe area should be honored, so if talking about event, you don't have to hook into the actual change to the bits you're quering for safe area, but to send this event from places where system wants you to honor/stop honoring the safe area.

In case of macOS, it'll be when you enter simple fullscreen mode, and in case of ios/android you can just always send it once you open your application or rotation happens (if rotation changes coordinate system).

So I don't see it being cumbersome to setup, since the area itself can not change.

madsmtm · 2024-09-11T12:54:46Z

I'm mostly against such an event because if we want it to be useful, we also have to emit it every time we do SurfaceResized, on every platform, and that's cumbersome.

Again, I believe it does not have to be handled in this PR, so I've opened #3911, let's discuss it there instead.

kchibisov · 2024-09-11T13:06:25Z

I'm mostly against such an event because if we want it to be useful, we also have to emit it every time we do SurfaceResized, on every platform, and that's cumbersome.

It doesn't have to be synced though, you should emit it only when the area changes, resize doesn't mean that it changed, because it may not apply at all.

I won't block on event, since it doesn't really matter for the majority of the clients though.

madsmtm added DS - android S - api Design and usability S - docs Awareness, docs, examples, etc. DS - ios C - needs discussion Direction must be ironed out labels Aug 26, 2024

madsmtm mentioned this pull request Aug 26, 2024

Coordinate systems #3891

Open

madsmtm force-pushed the madsmtm/safe-area branch from 7899334 to 67113a7 Compare August 26, 2024 03:17

madsmtm mentioned this pull request Aug 26, 2024

Rename "inner size" to "surface size" #3889

Merged

3 tasks

dannydulai mentioned this pull request Aug 26, 2024

macbook pro notch obscures terminal in simple fullscreen mode alacritty/alacritty#5612

Closed

kchibisov reviewed Aug 26, 2024

View reviewed changes

src/window.rs Show resolved Hide resolved

kchibisov reviewed Aug 26, 2024

View reviewed changes

Base automatically changed from madsmtm/inner-to-surface to master September 4, 2024 13:04

madsmtm force-pushed the madsmtm/safe-area branch from 67113a7 to 98a9914 Compare September 10, 2024 06:41

madsmtm added 5 commits September 10, 2024 08:45

Attempt to show better show that positions are relative to something

335a1c6

Remove extra comment that was more confusing than helpful

3cd2fa6

Improve docs, allow safe area to include transparent title bar

9bcd3ff

Use possibly negative values for surface position

bec2ed1

Dunno, I can imagine that it will happen on some platforms

Fix safe_area on macOS < 11

81c92eb

madsmtm added 6 commits September 10, 2024 09:37

Fix outer_size on iOS

d5d8e30

Remove draw.io files from typos check

b1fee88

Fix doc link

5df9eb8

Fix SurfaceResized on iOS

da7942e

Always redraw when the safe area changes

9f080a5

Properly set up typos check

bc12e13

madsmtm force-pushed the madsmtm/safe-area branch from ed83b5f to bc12e13 Compare September 10, 2024 08:25

Add Android, Orbital, Web, Windows, Wayland and X11 implementations

287fd8d

Very bare-bones, probably not correct

madsmtm requested review from notgull, daxpedda and MarijnS95 as code owners September 10, 2024 09:04

madsmtm requested a review from kchibisov September 10, 2024 09:04

madsmtm commented Sep 10, 2024

View reviewed changes

madsmtm added 3 commits September 10, 2024 11:11

Fix size of desktop coordinate systems image

feb686a

Note on android that there's events for safe area insets

e878c59

Add docs on also emitting RedrawRequested when safe area changes

c7523d4

kchibisov requested changes Sep 11, 2024

View reviewed changes

madsmtm added 3 commits September 11, 2024 12:49

Move docs on safe area to # Drawing on the window

01552d5

Remove comment about CSD in safe_area on wayland

45326fb

Remove fixmes, use issue to track instead

c651490

madsmtm mentioned this pull request Sep 11, 2024

Finish safe area implementation #3910

Open

8 tasks

Change note on negative surface position

c1ea82a

madsmtm requested a review from kchibisov September 11, 2024 10:59

Make safe area position vs. size a bit clearer

39f40c7

kchibisov reviewed Sep 11, 2024

View reviewed changes

src/lib.rs Outdated Show resolved Hide resolved

Add dot and arrow to the start and end of position indicators

131eaa1

macOS: Guarantee that SurfaceResized is emitted on safe area change

62a95a5

To support on Alacritty

madsmtm mentioned this pull request Sep 11, 2024

Add event for changes to the safe area #3911

Open

Don't mention Alacritty

3b3928e

Comment before mobile picture

992778a

lucasmerlin mentioned this pull request Sep 16, 2024

iOS safe area emilk/egui#4915

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add safe area and document coordinate systems #3890

Add safe area and document coordinate systems #3890

madsmtm commented Aug 26, 2024 •

edited

Loading

kchibisov left a comment

kchibisov Aug 26, 2024

madsmtm Aug 26, 2024

kchibisov Aug 26, 2024

madsmtm Aug 26, 2024

kchibisov Aug 26, 2024

madsmtm Sep 10, 2024

madsmtm commented Aug 26, 2024

kchibisov commented Aug 26, 2024

madsmtm commented Sep 10, 2024

madsmtm Sep 10, 2024

kchibisov Sep 11, 2024

kchibisov Sep 11, 2024

madsmtm Sep 11, 2024

kchibisov Sep 11, 2024

madsmtm Sep 11, 2024

kchibisov Sep 11, 2024 •

edited

Loading

madsmtm Sep 11, 2024

kchibisov Sep 11, 2024

kchibisov commented Sep 11, 2024

madsmtm commented Sep 11, 2024

kchibisov commented Sep 11, 2024

madsmtm commented Sep 11, 2024

kchibisov commented Sep 11, 2024

madsmtm commented Sep 11, 2024

kchibisov commented Sep 11, 2024

Add safe area and document coordinate systems #3890

Are you sure you want to change the base?

Add safe area and document coordinate systems #3890

Conversation

madsmtm commented Aug 26, 2024 • edited Loading

kchibisov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

madsmtm commented Aug 26, 2024

kchibisov commented Aug 26, 2024

madsmtm commented Sep 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kchibisov Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kchibisov commented Sep 11, 2024

madsmtm commented Sep 11, 2024

kchibisov commented Sep 11, 2024

madsmtm commented Sep 11, 2024

kchibisov commented Sep 11, 2024

madsmtm commented Sep 11, 2024

kchibisov commented Sep 11, 2024

madsmtm commented Aug 26, 2024 •

edited

Loading

kchibisov Sep 11, 2024 •

edited

Loading