Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(macOS): add native elements support for macOS/Skia #18793

Open
wants to merge 37 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
66daa65
feat: Add native embedding to netXX-desktop/macOS
spouliot Jun 28, 2024
f13004a
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Jul 4, 2024
14c0f99
chore: rework logging code
spouliot Jul 4, 2024
f606e5f
chore: change sample native element background to red
spouliot Jul 5, 2024
4d62ae0
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Jul 12, 2024
7496a4f
chore: keep a ref when attaching, drop it when detaching
spouliot Jul 12, 2024
bd0aae9
chore: fix flipped view and plugin svg clipping code (not yet working)
spouliot Jul 16, 2024
1a55f3a
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Jul 16, 2024
313e2cf
chore: fix clipping removal
spouliot Jul 16, 2024
49e741c
chore: implement SVG parsing for clipping
spouliot Jul 17, 2024
9ac7d6a
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Aug 9, 2024
c297452
fix: fix native element visibility (reset by clipping)
spouliot Aug 9, 2024
dc317ea
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Aug 12, 2024
12e7d07
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Aug 15, 2024
daf304d
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Aug 28, 2024
45ec5e5
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Sep 12, 2024
d4500f1
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Oct 4, 2024
619b34b
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Oct 7, 2024
d7a9213
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Oct 21, 2024
9cff94a
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Oct 29, 2024
df15280
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Oct 31, 2024
11be12e
chore: cleanup without native element clipping
spouliot Nov 3, 2024
c260496
chore: cleanup without native element clipping
spouliot Nov 5, 2024
5b7c0fc
fix: change how we handle the keyboard
spouliot Nov 5, 2024
76e4883
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Nov 5, 2024
b52c713
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Nov 6, 2024
2b31ad2
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Nov 6, 2024
4b9f24e
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Nov 8, 2024
1df5a8b
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Nov 8, 2024
f80f7ff
chore: fix xcode project post merge (crashed Xcode)
spouliot Nov 8, 2024
124cb84
chore: better/simpler basic clipping
spouliot Nov 8, 2024
84d13cf
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Nov 12, 2024
026226b
fix(macOS): native clip on the provided path/svg
spouliot Nov 13, 2024
610789c
chore: fix UnoNativeMac.xcodeproj/project.pbxproj
spouliot Nov 14, 2024
bd3d611
Merge branch 'master' into dev/spouliot/macos-desktop-native-embedding
spouliot Nov 14, 2024
f20b0b8
fix: reverse clip and set fillrule to oddeven
spouliot Nov 14, 2024
ab73de5
fix: parse floating point values in svg path
spouliot Nov 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/Uno.UI.Runtime.Skia.MacOS/MacOSMetalRenderer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ internal static class MacOSMetalRenderer
// FIXME: contribute some extra API (e.g. using `nint` or `IntPtr`) to SkiaSharp to avoid reflection
// net8+ alternative -> https://steven-giesel.com/blogPost/05ecdd16-8dc4-490f-b1cf-780c994346a4
var get = typeof(GRContext).GetMethod("GetObject", BindingFlags.Static | BindingFlags.NonPublic);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up: I compiled SkiaSharp 3 a while ago to try it with Uno and this blew up in my face. I think they may have removed GetObject or something. The software renderer worked though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I'll PR something to SkiaSharp so that reflection is not needed for 3.x

var context = (GRContext?)get?.Invoke(null, new object[] { ctx, true });
var context = (GRContext?)get?.Invoke(null, [ctx, true]);
if (context is null)
{
// Macs since 2012 have Metal 2 support and macOS 10.14 Mojave (2018) requires Metal
Expand Down
157 changes: 157 additions & 0 deletions src/Uno.UI.Runtime.Skia.MacOS/MacOSNativeElementHostingExtension.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
#nullable enable

using Windows.Foundation;
using Windows.UI.Core;
using Microsoft.UI.Xaml.Controls;

using Uno.Foundation.Extensibility;
using Uno.Foundation.Logging;

namespace Uno.UI.Runtime.Skia.MacOS;

internal class MacOSNativeElement : Microsoft.UI.Xaml.FrameworkElement
{
public nint NativeHandle { get; internal set; }

internal bool Detached { get; set; }
}

internal class MacOSNativeElementHostingExtension : ContentPresenter.INativeElementHostingExtension
{
private readonly ContentPresenter _presenter;

Check notice on line 21 in src/Uno.UI.Runtime.Skia.MacOS/MacOSNativeElementHostingExtension.cs

View check run for this annotation

Codacy Production / Codacy Static Code Analysis

src/Uno.UI.Runtime.Skia.MacOS/MacOSNativeElementHostingExtension.cs#L21

Remove the field '_presenter' and declare it as a local variable in the relevant methods.
private readonly MacOSWindowNative? _window;

private MacOSNativeElementHostingExtension(ContentPresenter contentPresenter)
{
_presenter = contentPresenter;
_window = _presenter.XamlRoot?.HostWindow?.NativeWindow as MacOSWindowNative;
}

public static void Register() => ApiExtensibility.Register<ContentPresenter>(typeof(ContentPresenter.INativeElementHostingExtension), o => new MacOSNativeElementHostingExtension(o));

public void ArrangeNativeElement(object content, Rect arrangeRect, Rect clipRect)
{
if (content is MacOSNativeElement element)
{
if (element.Detached)
{
this.Log().Debug($"Cannot arrange element `{nameof(content)}` of type {content.GetType().FullName} since it was detached.");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also a Debug.Assert?

}
else
{
NativeUno.uno_native_arrange(element.NativeHandle, arrangeRect.Left, arrangeRect.Top, arrangeRect.Width, arrangeRect.Height, clipRect.Left, clipRect.Top, clipRect.Width, clipRect.Height);
}
}
else if (this.Log().IsEnabled(LogLevel.Debug))
{
this.Log().Debug($"Object `{nameof(content)}` is a {content.GetType().FullName} and not a MacOSNativeElement subclass.");
}
}

public void AttachNativeElement(object content)
{
if (content is MacOSNativeElement element)
{
NativeUno.uno_native_attach(element.NativeHandle);
}
else if (this.Log().IsEnabled(LogLevel.Debug))
{
this.Log().Debug($"Object `{nameof(content)}` is a {content.GetType().FullName} and not a MacOSNativeElement subclass.");
}
}

public void ChangeNativeElementOpacity(object content, double opacity)
{
if (content is MacOSNativeElement element)
{
// https://developer.apple.com/documentation/appkit/nsview/1483560-alphavalue?language=objc
// note: no marshaling needed as CGFloat is double for 64bits apps
NativeUno.uno_native_set_opacity(element.NativeHandle, opacity);
}
else if (this.Log().IsEnabled(LogLevel.Debug))
{
this.Log().Debug($"Object `{nameof(content)}` is a {content.GetType().FullName} and not a MacOSNativeElement subclass.");
}
}

public void ChangeNativeElementVisibility(object content, bool visible)
{
if (content is MacOSNativeElement element)
{
// https://developer.apple.com/documentation/appkit/nsview/1483369-hidden?language=objc
NativeUno.uno_native_set_visibility(element.NativeHandle, visible);
}
else if (this.Log().IsEnabled(LogLevel.Debug))
{
this.Log().Debug($"Object `{nameof(content)}` is a {content.GetType().FullName} and not a MacOSNativeElement subclass.");
}
}

public object? CreateSampleComponent(string text)
{
if (_window is null)
{
if (this.Log().IsEnabled(LogLevel.Debug))
{
this.Log().Debug($"CreateSampleComponent failed as no MacOSWindowNative instance could be found.");
}
return null;
}

var handle = NativeUno.uno_native_create_sample(_window.Handle, text);
return new MacOSNativeElement()
{
NativeHandle = handle,
AccessKey = text // FIXME: debug helper, to be removed
};
}

public void DetachNativeElement(object content)
{
if (content is MacOSNativeElement element)
{
if (element.Detached)
{
this.Log().Debug($"Object `{nameof(content)}` of type {content.GetType().FullName} was already detached.");
}
else
{
NativeUno.uno_native_detach(element.NativeHandle);
element.Detached = true;
}
}
else if (this.Log().IsEnabled(LogLevel.Debug))
{
this.Log().Debug($"Object `{nameof(content)}` is a {content.GetType().FullName} and not a MacOSNativeElement subclass.");
}
}

public bool IsNativeElement(object content) => content is MacOSNativeElement;

public bool IsNativeElementAttached(object owner, object nativeElement)
{
if (nativeElement is MacOSNativeElement element)
{
return NativeUno.uno_native_is_attached(element.NativeHandle);
}
else if (this.Log().IsEnabled(LogLevel.Debug))
{
this.Log().Debug($"Object `{nameof(owner)}` is a {owner.GetType().FullName} and not a MacOSNativeElement subclass.");
}
return false;
}

public Size MeasureNativeElement(object content, Size childMeasuredSize, Size availableSize)
{
if (content is MacOSNativeElement element)
{
NativeUno.uno_native_measure(element.NativeHandle, childMeasuredSize.Width, childMeasuredSize.Height, availableSize.Width, availableSize.Height, out var width, out var height);
return new Size(width, height);
}
else if (this.Log().IsEnabled(LogLevel.Debug))
{
this.Log().Debug($"Object `{nameof(content)}` is a {content.GetType().FullName} and not a MacOSNativeElement subclass.");
}
return Size.Empty;
}
}
27 changes: 18 additions & 9 deletions src/Uno.UI.Runtime.Skia.MacOS/MacOSWindowHost.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@
using Windows.UI.Core;
using Windows.UI.Input;
using Microsoft.UI.Xaml;
using Microsoft.UI.Xaml.Input;

using Window = Microsoft.UI.Xaml.Window;

using Uno.Foundation.Extensibility;
using Uno.Foundation.Logging;
using Uno.UI.Helpers;
using Uno.UI.Hosting;

namespace Uno.UI.Runtime.Skia.MacOS;
Expand Down Expand Up @@ -71,7 +73,7 @@ private void UpdateWindowSize(double nativeWidth, double nativeHeight)
SizeChanged?.Invoke(this, new Size(nativeWidth, nativeHeight));
}

private void Draw(SKSurface surface)
private void Draw(double nativeWidth, double nativeHeight, SKSurface surface)
{
using var canvas = surface.Canvas;
using (new SKAutoCanvasRestore(canvas, true))
Expand All @@ -80,7 +82,15 @@ private void Draw(SKSurface surface)

if (RootElement?.Visual is { } rootVisual)
{
RootElement.XamlRoot?.Compositor.RenderRootVisual(surface, rootVisual, null);
int width = (int)nativeWidth;
int height = (int)nativeHeight;
var path = SkiaRenderHelper.RenderRootVisualAndReturnPath(width, height, rootVisual, surface);
if (path is { })
{
var svg = path.ToSvgPathData();
// remove first path since it cover the whole area
spouliot marked this conversation as resolved.
Show resolved Hide resolved
NativeUno.uno_window_clip_svg(_nativeWindow.Handle, svg.Substring(svg.IndexOf('Z') + 1));
}
}
}

Expand Down Expand Up @@ -115,7 +125,7 @@ private void MetalDraw(double nativeWidth, double nativeHeight, nint texture)

surface.Canvas.Scale(scale, scale);

Draw(surface);
Draw(nativeWidth, nativeHeight, surface);

_context?.Flush();
}
Expand Down Expand Up @@ -155,7 +165,7 @@ private unsafe void SoftDraw(double nativeWidth, double nativeHeight, nint* data
_rowBytes = info.RowBytes;
}

Draw(_surface!);
Draw(nativeWidth, nativeHeight, _surface!);

*data = _bitmap.GetPixels(out var bitmapSize);
*size = (int)bitmapSize;
Expand Down Expand Up @@ -287,8 +297,7 @@ private static int OnRawKeyDown(nint handle, VirtualKey key, VirtualKeyModifiers
}
var args = CreateArgs(key, mods, scanCode, unicode);
keyDown.Invoke(window!, args);
// we tell macOS it's always handled as WinUI does not mark as handled some keys that would make it beep in common cases
return 1;
return FocusManager.GetFocusedElement() == null ? 0 : 1;
}
catch (Exception e)
{
Expand All @@ -315,7 +324,7 @@ private static int OnRawKeyUp(nint handle, VirtualKey key, VirtualKeyModifiers m
}
var args = CreateArgs(key, mods, scanCode, unicode);
keyUp.Invoke(window!, args);
return args.Handled ? 1 : 0;
return 1;
}
catch (Exception e)
{
Expand Down Expand Up @@ -423,8 +432,8 @@ internal static unsafe int OnMouseEvent(nint handle, NativeMouseEventData* data)
}

mouseEvent(window, BuildPointerArgs(*data));
// let the window be activated (becoming the keyWindow) when clicked
return data->EventType == NativeMouseEvents.Down ? 0 : 1;
// always let the native side know about the mouse events, e.g. setting keyWindow, embedded native controls
return 0;
}
catch (Exception e)
{
Expand Down
1 change: 1 addition & 0 deletions src/Uno.UI.Runtime.Skia.MacOS/MacSkiaHost.cs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ static MacSkiaHost()
MacOSFileSavePickerExtension.Register();
MacOSFolderPickerExtension.Register();
MacOSLauncherExtension.Register();
MacOSNativeElementHostingExtension.Register();
MacOSNativeWindowFactoryExtension.Register();
MacOSSystemNavigationManagerPreviewExtension.Register();
MacOSSystemThemeHelperExtension.Register();
Expand Down
30 changes: 29 additions & 1 deletion src/Uno.UI.Runtime.Skia.MacOS/NativeUno.cs
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ internal static unsafe partial void uno_set_window_events_callbacks(
internal static partial string uno_window_get_title(nint window);

[LibraryImport("libUnoNativeMac.dylib", StringMarshalling = StringMarshalling.Utf8)]
internal static partial nint uno_window_set_title(nint window, string title);
internal static partial void uno_window_set_title(nint window, string title);

[LibraryImport("libUnoNativeMac.dylib")]
internal static unsafe partial void uno_set_window_close_callbacks(
Expand Down Expand Up @@ -277,6 +277,9 @@ internal static unsafe partial void uno_set_window_close_callbacks(
[LibraryImport("libUnoNativeMac.dylib")]
internal static partial void uno_window_set_min_size(nint window, double width, double height);

[LibraryImport("libUnoNativeMac.dylib", StringMarshalling = StringMarshalling.Utf8)]
internal static partial void uno_window_clip_svg(nint window, string? svg);

[LibraryImport("libUnoNativeMac.dylib", StringMarshalling = StringMarshalling.Utf8)]
internal static partial string? /* const char* _Nullable */ uno_pick_single_folder(string? prompt, string? identifier, int suggestedStartLocation);

Expand Down Expand Up @@ -320,4 +323,29 @@ internal static unsafe partial void uno_set_window_close_callbacks(
[LibraryImport("libUnoNativeMac.dylib")]
[return: MarshalAs(UnmanagedType.I1)]
internal static partial bool uno_cursor_set(CoreCursorType cursorType);

[LibraryImport("libUnoNativeMac.dylib", StringMarshalling = StringMarshalling.Utf8)]
internal static partial nint uno_native_create_sample(nint window, string text);

[LibraryImport("libUnoNativeMac.dylib")]
internal static partial void uno_native_arrange(nint element, double arrangeLeft, double arrangeTop, double arrangeWidth, double arrangeHeight, double clipLeft, double clipTop, double clipWidth, double clipHeight);

[LibraryImport("libUnoNativeMac.dylib")]
internal static partial void uno_native_attach(nint element);

[LibraryImport("libUnoNativeMac.dylib")]
internal static partial void uno_native_detach(nint element);

[LibraryImport("libUnoNativeMac.dylib")]
[return: MarshalAs(UnmanagedType.I1)]
internal static partial bool uno_native_is_attached(nint element);

[LibraryImport("libUnoNativeMac.dylib")]
internal static partial void uno_native_set_opacity(nint element, double opacity);

[LibraryImport("libUnoNativeMac.dylib")]
internal static partial void uno_native_set_visibility(nint element, [MarshalAs(UnmanagedType.I1)] bool visible);

[LibraryImport("libUnoNativeMac.dylib")]
internal static partial void uno_native_measure(nint element, double childWidth, double childHeight, double availableWidth, double availableHeight, out double width, out double height);
}
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
/* Begin PBXBuildFile section */
D116C63E2AC79876004B975F /* UNOCursor.m in Sources */ = {isa = PBXBuildFile; fileRef = D116C63D2AC79876004B975F /* UNOCursor.m */; };
D13AB8A82B5839B200693F8E /* libSkiaSharp.dylib in Frameworks */ = {isa = PBXBuildFile; fileRef = D13AB8A72B5839B200693F8E /* libSkiaSharp.dylib */; };
D18D4FAC2C2DE804003E4BBF /* UNONative.m in Sources */ = {isa = PBXBuildFile; fileRef = D18D4FAB2C2DE804003E4BBF /* UNONative.m */; };
D1A0651E2A7066F700101BE6 /* UNOMetalViewDelegate.m in Sources */ = {isa = PBXBuildFile; fileRef = D1A0651D2A7066F700101BE6 /* UNOMetalViewDelegate.m */; };
D1A065202A8467B200101BE6 /* UNOApplication.h in Headers */ = {isa = PBXBuildFile; fileRef = D1A0651F2A8467B200101BE6 /* UNOApplication.h */; };
D1A065222A84688000101BE6 /* UNOApplication.m in Sources */ = {isa = PBXBuildFile; fileRef = D1A065212A84688000101BE6 /* UNOApplication.m */; };
Expand All @@ -24,6 +25,8 @@
D116C63D2AC79876004B975F /* UNOCursor.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = UNOCursor.m; sourceTree = "<group>"; };
D13AB8A72B5839B200693F8E /* libSkiaSharp.dylib */ = {isa = PBXFileReference; lastKnownFileType = "compiled.mach-o.dylib"; name = libSkiaSharp.dylib; path = UnoNativeMac/libSkiaSharp.dylib; sourceTree = "<group>"; };
D13AB8AC2B58566400693F8E /* Config.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = Config.xcconfig; sourceTree = "<group>"; };
D18D4FAA2C2DE76F003E4BBF /* UNONative.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = UNONative.h; sourceTree = "<group>"; };
D18D4FAB2C2DE804003E4BBF /* UNONative.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = UNONative.m; sourceTree = "<group>"; };
D1A0651C2A70664A00101BE6 /* UNOMetalViewDelegate.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = UNOMetalViewDelegate.h; sourceTree = "<group>"; };
D1A0651D2A7066F700101BE6 /* UNOMetalViewDelegate.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = UNOMetalViewDelegate.m; sourceTree = "<group>"; };
D1A0651F2A8467B200101BE6 /* UNOApplication.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = UNOApplication.h; sourceTree = "<group>"; };
Expand Down Expand Up @@ -90,6 +93,8 @@
D116C63D2AC79876004B975F /* UNOCursor.m */,
D1A0651C2A70664A00101BE6 /* UNOMetalViewDelegate.h */,
D1A0651D2A7066F700101BE6 /* UNOMetalViewDelegate.m */,
D18D4FAA2C2DE76F003E4BBF /* UNONative.h */,
D18D4FAB2C2DE804003E4BBF /* UNONative.m */,
D1CC768E2ABA1368002A44F0 /* UNOPickers.h */,
D1CC768C2ABA1337002A44F0 /* UNOPickers.m */,
D1FE7A2C2B75C8E100ACFC76 /* UNOSoftView.h */,
Expand Down Expand Up @@ -176,6 +181,7 @@
D1A065252A8AC23800101BE6 /* UNOWindow.m in Sources */,
D1CC768B2AB9D464002A44F0 /* UNOClipboard.m in Sources */,
D116C63E2AC79876004B975F /* UNOCursor.m in Sources */,
D18D4FAC2C2DE804003E4BBF /* UNONative.m in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
};
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
//
// UNONative.h
//

#pragma once

#import "UnoNativeMac.h"
#import "UNOWindow.h"

NS_ASSUME_NONNULL_BEGIN

@protocol UNONativeElement

@property (nonatomic) bool visible;

-(void) detach;

@end

@interface UNORedView : NSView<UNONativeElement>

@end

NSView* uno_native_create_sample(NSWindow *window, const char* _Nullable text);

void uno_native_arrange(NSView *element, double arrangeLeft, double arrangeTop, double arrangeWidth, double arrangeHeight, double clipLeft, double clipTop, double clipWidth, double clipHeight);

void uno_native_attach(NSView* element);

void uno_native_detach(NSView* element);

bool uno_native_is_attached(NSView* element);

void uno_native_measure(NSView* element, double childWidth, double childHeight, double availableWidth, double availableHeight, double* width, double* height);

void uno_native_set_opacity(NSView* element, double opacity);

void uno_native_set_visibility(NSView* element, bool visible);

NS_ASSUME_NONNULL_END
Loading
Loading