RFC: A position aware composited transform follower

[matthew-carroll]

Flutter provides two widgets that, when used together, cause one widget to always appear near another widget.

This RFC identifies a limitation of the existing widgets and recommends an extension.

The widget being followed is wrapped with a CompositedTransformTarget widget. The widget doing the following is wrapped with a CompositedTransformFollower widget.

One use-case where these widgets are useful is when a text selection drag handle sits beneath the caret.

As shown above, the drag handle always sits a few pixels below the caret. This relative position should remain true, no matter where the caret goes, or where the user scrolls. For example, even when the user scrolls the content off-screen, the selection handle should remain beneath the caret.

A composited transform follower provides two benefits:

• A (possible) performance improvement. The content target can be rebuilt without rebuilding the follower, and the follower can be rebuilt without rebuilding the target. Additionally, if neither the target nor the follower change, but the target's position changes, the compositor can update the UI without re-running any layout or painting behavior.
• A simplified solution for developers. Without such a tool, developers would need to be aware of all possible ways in which a widget might move around the screen. For example, if a widget sits within a Scrollable, then to align an overlay widget with a piece of content in that scrollable, the developer would need to listen for all scroll changes and then rebuild the overlay widget. This problem is compounded if a Scrollable sits within another Scrollable. Alignment requires full knowledge of the UI, and full access to all controllers that move content around the screen. This problem is generally intractable.

The problem with CompositedTransformFollower is that its offset is static and so is its child.

Consider, for example, a popover toolbar for text editing. When it fits on the screen, everything works fine.

The problem with this example is that the popover toolbar should re-orient itself when it gets too close to the screen boundary, but a CompositedTransformFollower doesn't allow for this.

Instead of scrolling offscreen, the toolbar should jump to the bottom of the selected content.

The above use-case is not currently possible with a CompositedTransformFollower because the developer can't make any decisions based on the target's offset or bounds.

Flutter should consider introducing a similar widget to CompositedTransformFollower that facilitates such decisions.

The following is a jumping off point to design such a widget.

return NewCompositedTransformFollower(
  link: layerLink,
  builder: (context, targetBounds) {
   // Rebuild the follower every time the target moves. Always rebuilding the follower reduces
   // the possible performance as compared to the original CompositedTransformFollower, but
   // in a use-case like this, it's unavoidable, one way or another.

    // This is an example of how one might build a popover toolbar.
    if (targetBounds.top > minToolbarSpace) {
      // By default, show the toolbar above the content.
      return Transform.translate(
        offset: Offset(0, targetBounds.top - (toolbarHeight + toolbarGap)),
        child: PopoverToolbar(
          orientation: PopoverToolbar.above,
        ),
      );
    } else if (targetBounds.bottom < minToolbarSpace) {
      // Show the toolbar below the content.
      return Transform.translate(
        offset: Offset(0, targetBounds.bottom + toolbarGap),
        child: PopoverToolbar(
          orientation: PopoverToolbar.below,
        ),
      );
    } else {
      // There's not enough room for the toolbar above or below. Show nothing.
      return const SizedBox();
    }
  },
);

There may be a challenge in the timing of this solution. The standard Flutter order of operations is:

1. Build widgets
2. Layout and paint
3. Composite

If the compositor is the only part that knows about a change to the target's position, then it's unclear how the follower can rebuild.

That said, any overlay widget associated with a piece of content will want to follow that content around, without concern for what moves that content, but with consideration for screen boundaries. Flutter should make this use-case easier for developers, and should provide a generic approach.

[creativecreatorormaybenot]

Hi, I implemented exactly this for a flutter_portal adjustment (not yet merged into the main repo). This is possible with the current framework API. Let me know how I can best contribute the solution.

[creativecreatorormaybenot]

The gist is creating a new follower layer that works with the leader offset.

[matthew-carroll]

Is your current implementation open source? Can I see it somewhere?

[venkatd]

@matthew-carroll small world :)

We sponsored @creativecreatorormaybenot to improve flutter_portal and he managed to solve some of the challenges related to positioning in the compositing step.

The code is here but it hasn't been merged into the main repo yet
https://github.com/creativecreatorormaybenot/flutter_portal/tree/portal-anchor-refactor

But @creativecreatorormaybenot can point you to the places to look in the code

If relevant, we are happy to co-sponsor extracting the component you need out through Flutter Bounty Hunters.

[matthew-carroll]

@venkatd @creativecreatorormaybenot it would definitely save me some time if either of you could point me to specific hook where you're accomplishing widget rebuilds based on global position changes. I looked through some of the code recently. A lot of it was an alternative to the standard Overlay. For this proposal, I'm laser focused on how we can make widget build decisions based on any/all changes to another widget's screen offset.

[creativecreatorormaybenot]

Problem mapping

To understand the solution I came up with, I want to quickly map the problem described in your original comment to the specific classes used.

Specifically, the selected text is wrapped in a CompositedTransformTarget and linked to the popover toolbar that is wrapped in a CompositedTransformFollower. The linking is achieved using a LayerLink.

The problem now is that the popover toolbar should change its offset relative to the selected text based on:

• The position of the selected text (the position of the target).
• The available portion of the screen the popover toolbar (the follower) should be allowed to be displayed in (the available rect).

Issue to solve

The solution to this seems quite easy:

• The available rect is static, so this part is trivial.

However, the issue is that the follower (popover toolbar) automatically follows the target (selected text) via the LayerLink and the CompositedTransformTarget and CompositedTransformFollower implementation.
However, we are not informed about the position of the transform target and can therefore not base our relative decisions (how to position and orient the follower) on it.

Solution

This means the solution involves some way of informing our follower about the target's position when making our layout decisions. This is not possible by default (it is hidden in the implementation of the linking).

→ We need to modify some parts of CompositedTransformTarget, CompositedTransformFollower, and/or LayerLink (which could also be the classes used by these classes).

Implementation

This is possible. I tried to explain it to you already @matthew-carroll - I am not sure if I am really missing the point here or if you did not look into it closely / were not able to grasp it on the high-level, so I will just try to be as detailed as possible and explain the modifications that I made.

The implementation that I built was for flutter_portal and this probably has different specific API requirements than the code you want to write. I cannot write the code you need to use @matthew-carroll in this comment because I do not know your requirements. I will try to explain on the high-level while the same core mechanic can be used in any of these kinds of problems.

CustomCompositedTransformFollower

We need a custom implementation of CompositedTransformFollower because the default implementation only computes the child layout based on an offset, targetAnchor, and followerAnchor, which are all static values (Offset and Alignment) that do not respect the position of the target (nor the available rect).

We basically need two things here:

1. The available rect.
2. The heuristic to position our follower based on the target's position.

In my flutter_portal implementation, this is given via:

1. The overlayLink, which simply stores a reference to the render object that defines the available rect.
2. The custom anchor that defines how to position the child / follower based on the target's offset. This is a simple abstract contract in the case of flutter_portal, but you can use whatever you want.

To help you out here @matthew-carroll, I will try to propose what this could look like for the example problem from your comment:

1. The render object of the viewport (the "screen" in your comment) which defines what would be off-screen and what not. This is the available rect.
2. Some logic that determines when the popover toolbar should be above or below the selected text. Note that this is trivial at this point because you have the viewport (see 1.) and can react to the position of the selected text (see below for that).

If not everything is clear until this point, we might want to jump onto a call to go through it 🙂

Note that we just pass 1. and 2. along with the LayerLink to the render object, so nothing fancy here.

CustomRenderFollowerLayer

If we are using different properties in the render object widget, we obviously also need a custom render object. We just base this off of the RenderFollowerLayer implementation.

The only parts we modify here is using our custom follower layer (see below) when setting the layer instead of the one from the framework and pass our heuristic for the follower positioning based on the target's position to that layer.

The heuristic can be seen here. It is based on what we had in 1. above combined with what we have in 1. above.

The code:

final theater = overlayLink.theater!;

final theaterShift = -globalToLocal(
  leaderOffset,
  ancestor: theater,
);

final targetRect = Offset.zero & targetSize;
final theaterRect = theaterShift & theater.size;

return anchor.getSourceOffset(
  // The size is set in performLayout of the RenderProxyBoxMixin.
  sourceSize: size,
  targetRect: targetRect,
  theaterRect: theaterRect,
);

Can be broken down into the more abstract form of this answer:

final availableRectRenderObject = ..;
final shift = -globalToLocal(leaderOffset, ancestor: availableRectRenderObject);
final availableRect = shift & availableRectRenderObject.size;

return heuristicForFollowerPosition(
  size,
  availableRect,
  ..,
);

Basically, here we have everything we need. We now only need to obtain the leaderOffset.

CustomFollowerLayer

The default FollowerLayer implementation does not give us any access to the leaderOffset, which we need in our heuristic.

This means that we need to write our own version of the FollowerLayer (which is a ContainerLayer) class from the framework that passes the leader offset back to our custom render object, which can then use it in the heuristic (see above).

This was not trivial for me to figure out and all of this definitely was painful to do, but in the end, we barely need to change anything.

There are two places where we need to change how the offset of the follower (the linked offset) is calculated:

• https://github.com/creativecreatorormaybenot/flutter_portal/blob/ab6079b723868f5b1de8e1c0b9fbd9e0d6e431f3/lib/src/custom_follower.dart#L300
• https://github.com/creativecreatorormaybenot/flutter_portal/blob/ab6079b723868f5b1de8e1c0b9fbd9e0d6e431f3/lib/src/custom_follower.dart#L400

The rest of the code is the same as in FollowerLayer. I had to change some of the transform logic obviously, but that is trivial.

In order to change how the offset is calculated, we need to pass our heuristic from above to our custom layer. This is pretty obvious, but I thought I would mention it anyway. This is how I passed the heuristic in the flutter_portal case:

• https://github.com/creativecreatorormaybenot/flutter_portal/blob/ab6079b723868f5b1de8e1c0b9fbd9e0d6e431f3/lib/src/custom_follower.dart#L281

Done

That should be it. This should be all of the tools needed to pull of exactly what you were asking for. And it stays eaxctly as performant and efficient as previously because the heuristic is applied right in the layer. There are no rebuilds or anything (not even close to that) associated with it. This is the purest solution to this problem IMO.

Please let me know if this was helpful to you.