[BUG] MediaElement does not respect clipping of Views that contain it in Android, so videos show where they should not (works properly in Windows/iOS)

[jonmdev]

Is there an existing issue for this?

• I have searched the existing issues

Did you read the "Reporting a bug" section on Contributing file?

• I have read the "Reporting a bug" section on Contributing file: https://github.com/CommunityToolkit/Maui/blob/main/CONTRIBUTING.md#reporting-a-bug

Current Behavior

I wrote a simple demo project which shows the issue as below. This project loads (in App.xaml.cs) a hierarchy of:

--ContentPage
----AbsoluteLayout
-------Border1
----------AbsoluteLayout1
----------------MediaElement1
-------Border2
----------AbsoluteLayout2
----------------MediaElement2

It then applies Border1 to the left half of the screen and Border2 to the right side of the screen.

The MediaElements are translated so that they would hypothetically still fill the screen if the Borders weren't clipping them.

This creates the following expected effect in Windows, with the left side of screen the left side of clipA, and the right side of the screen, the right side of clipB:

In iOS we also get the desired split screen effect:

In Android, if we only add one Border to the hierarchy at a time, we see they both perform the same as Windows/iOS as expected:

...//

But when we enable both sides in Android, only the topmost layer is shown across the whole screen. ie. It stops respecting the clipping of the Border.

This creates unmanageable behavior whenever one clipped video is superimposed on another. For example, I have a real world application similar to this which is broken on Android due to this problem.

I have not been able to test iOS.

Any thoughts on the cause or a solution?

Thanks.

Expected Behavior

The Android behavior should be the same as the Windows behavior - we should see both half clips side by side when they are both enabled in Android.

Steps To Reproduce

Open Repo, go to App.xaml.cs for relevant code. Can toggle left and right sides of screen videos with toggles at top:

        bool SHOW_LEFT_HALF_VIDEO = true;
        bool SHOW_RIGHT_HALF_VIDEO = true;

Full code of this document is as follows:

using System.Diagnostics;
using CommunityToolkit.Maui.Views;

namespace MediaElementOverlapBug {
    public partial class App : Application {

        //============================================================
        //TOGGLE WHICH OF TWO HALF SCREEN VIDEOS TO ADD
        //============================================================
        //== SHOULD SHOW ONE VIDEO ON EACH HALF OF SCREEN
        //== WORKS FINE IN WINDOWS
        //== BUT IN ANDROID, WILL ONLY SHOW THE RIGHT SIDE ON FULL SCREEN (the highest overlaying video will cover any underlying videos irrespective of clipping rules)
        //== UNSURE ABOUT IOS

        bool SHOW_LEFT_HALF_VIDEO = true;
        bool SHOW_RIGHT_HALF_VIDEO = true;


        List<MediaElement> mediaElements = new();
        List<Border> borders = new();
        List<AbsoluteLayout> absoluteLayouts = new();
        public App() {
            ContentPage mainPage = new ContentPage();
            MainPage = mainPage;

            AbsoluteLayout abs = new();
            mainPage.Content = abs;


            for (int i = 0; i < 2; i++) {
                Border border = new();
                border.StrokeThickness = 0;
                borders.Add(border);

                AbsoluteLayout absolute = new();
                absoluteLayouts.Add(absolute);
                border.Content = absolute;

                MediaElement mediaElement = new();
                if (i == 0) {
                    mediaElement.Source = MediaSource.FromResource("bunny.mp4");
                }
                else {
                    mediaElement.Source = MediaSource.FromResource("woods.mp4");
                }
                mediaElement.Loaded += delegate {
                    mediaElement.Play();
                };
                mediaElement.ShouldShowPlaybackControls = true;
                mediaElement.ShouldAutoPlay = true;
                mediaElement.Aspect = Aspect.AspectFill;
                mediaElement.ShouldLoopPlayback = true;
                mediaElement.Play();

                mediaElements.Add(mediaElement);
                absolute.Add(mediaElement);
            }


            //=============================
            //ADD VIDEOS TO HIERARCHY
            //=============================
            if (SHOW_LEFT_HALF_VIDEO) {
                abs.Add(borders[0]);
            }
            if (SHOW_RIGHT_HALF_VIDEO) {
                abs.Add(borders[1]);
            }


            mainPage.SizeChanged += delegate {
                if (mainPage.Width > 0) {
                    for (int i = 0; i < borders.Count; i++) {
                        borders[i].HeightRequest = mainPage.Height;
                        borders[i].WidthRequest = mainPage.Width * 0.5;

                        mediaElements[i].HeightRequest = mainPage.Height;
                        mediaElements[i].WidthRequest = mainPage.Width;

                        if (i == 1) {
                            borders[i].TranslationX = 0.5 * mainPage.Width;
                            mediaElements[i].TranslationX = -0.5 * mainPage.Width;
                        }
                    }
                }
            };
        }
    }
}

Link to public reproduction project repository

https://github.com/jonmdev/MediaElementOverlapBug

Environment

- .NET MAUI CommunityToolkit: 3.x? current
- OS: Android
- .NET MAUI: 8.0

[vhugogarcia]

The video appears now to clip correctly on Android on the latest version of the Media Element. See screenshots below. I'm closing this issue. We can re-open it in case it appears again.

[jonmdev]

FYI @vhugogarcia that is not correct. You should be seeing the Big Buck Bunny on one side of the screen and forest video on the other side of the screen. You are not seeing the Big Buck Bunny cartoon.

This is because Surface Textures cannot be overlapped or clipped properly in Android. It must be a TextureView.

There was extensive discussion of adding a mode and some effort but then it seemed to stop.

See:

• [BUG] ExoPlayer uses SurfaceView by default, needs way to optionally use TextureView to fix glitchy behavior and extend usage to transparent background #1773
• [Proposal] Add new MediaElement constructor to allow creation of ExoPlayer as TextureView in Android (which will then allow overlaying, opacity, transparency, etc.) #1891

Personally I solved this for myself by just copying and pasting the MediaElement class into my code and re-writing it so I can use a TextureView.

Microsoft should still add a TextureView mode for Android.

[vhugogarcia]

Sounds good. Thanks for the detailed feedback 😎👍