Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add an SDL example to the C bindings #250

Merged
merged 6 commits into from
Aug 8, 2023

Conversation

DataTriny
Copy link
Member

@DataTriny DataTriny commented Jun 18, 2023

Introduces a cross-platform examples using the SDL2 library.

Tested on Windows and Linux. Please help to validate on macOS.

You can use this test release to try the new example.

@mwcampbell
Copy link
Contributor

On Windows, I assume you used SDL2-devel-2.26.5-VC.zip. It wasn't immediately obvious that this is the archive to use, since the win32 zip file with just the DLL was at the top on the GitHub release page. Would be good to add a clarification in the example README.

The macOS .dmg does include the CMake support, under SDL2.framework/Resources/CMake. But I got some compile errors. I can give you remote access to my Mac if that would help.

@mwcampbell
Copy link
Contributor

Here are the compilation errors I got on macOS:

/Users/matt/tmp/accesskit_c-v0.4.0/examples/sdl/hello_world.c:69:39: error: field has incomplete type 'accesskit_macos_subclassing_adapter' (aka 'struct accesskit_macos_subclassing_adapter')
  accesskit_macos_subclassing_adapter adapter;
                                      ^
/Users/matt/tmp/accesskit_c-v0.4.0/include/accesskit.h:624:16: note: forward declaration of 'struct accesskit_macos_subclassing_adapter'
typedef struct accesskit_macos_subclassing_adapter
               ^
/Users/matt/tmp/accesskit_c-v0.4.0/examples/sdl/hello_world.c:87:39: error: incomplete definition of type 'struct _NSWindow'
      (void *)wmInfo.info.cocoa.window->contentView(), source, source_userdata,
              ~~~~~~~~~~~~~~~~~~~~~~~~^
/Users/matt/tmp/SDL2.framework/Headers/SDL_syswm.h:87:16: note: forward declaration of 'struct _NSWindow'
typedef struct _NSWindow NSWindow;
               ^
2 errors generated.

The first one looks easy; the adapter field should be a pointer. But the second one implies that we may need a bit of Objective-C code to make this work, unless perhaps we add an alternate macOS adapter constructor that takes an NSWindow* rather than an NSView*.

@DataTriny
Copy link
Member Author

New test release

Please help to test again on Mac. Are the instructions in the README accurate?

@DataTriny
Copy link
Member Author

It would be nice to also try the shared library by using -DBUILD_SHARED_LIBS=ON when configuring the project. I guess you'd then have to move libaccesskit.dylib alongside the hello_world executable.

@mwcampbell
Copy link
Contributor

I see only one problem with the SDL example on macOS: For some reason, Tab doesn't work. Looking into that now.

The instructions in the README are correct. A couple of minor changes though: the word "On" in the platform headings is redundant IMO, and is mildly annoying for a screen reader user trying to skim. Also, the name "Mac OS X" is outdated; it's now "macOS".

@mwcampbell
Copy link
Contributor

Don't forget to rebase this branch now that #247 is merged.

@DataTriny
Copy link
Member Author

DataTriny commented Jun 24, 2023

Oh I forgot to mention that hit-testing on Linux is not working either... I think something odd is happening with libwnck that Orca is using.
The conclusion I came to was that the PID associated to the X11 window was not the same as the one associated with the "X11 application". This leads to Orca filtering out the SDL window for mouse review. I honestly don't think I'm doing something wrong in the example about this specifically.

@mwcampbell
Copy link
Contributor

It's clear enough that Linux support is a work in progress. I see no reason to block this PR on that.

@mwcampbell
Copy link
Contributor

Update on the problem with Tab on macOS: I now see that pressing Tab updates the internal focus in the example program, but for some reason I haven't yet figured out, VoiceOver isn't getting a focus change event. This does work in the winit example on macOS.

@mwcampbell
Copy link
Contributor

I figured out that accessibilityFocusedUIElement is never called on our subclassed view. I don't know why yet; I think SDL might be doing something weird with keyboard focus on macOS.

@mwcampbell
Copy link
Contributor

I think the problem is that in SDL on macOS, the keyboard focus is on the window, not the view. So I need to see if overriding accessibilityFocusedUIElement on the window rather than the view fixes the problem. We always have to override accessibilityChildren and accessibilityHitTest on the view, not the window, to make sure we preserve the OS-defined behavior for window decorations. So I guess this will mean that when SubclassingAdapter::for_window is called, we have to subclass both the window and the view. I'll try this out after I get a haircut.

@mwcampbell
Copy link
Contributor

I tried writing code to subclass the window yesterday afternoon, but that somehow has the unwanted effect of preventing the window from appearing at all. I don't believe I currently have the expertise in Objective-C and Cocoa to debug this. So I can't tell if overriding accessibilityFocusedUIElement on the window would even solve the problem.

Another thing I can try is forcing the view, rather than the window, to be the first responder (the object that has keyboard focus in Cocoa). But I don't know if that will have unwanted side effects. Presumably the SDL developers had a good reason for doing things the way they did.

@DataTriny
Copy link
Member Author

Thanks @mwcampbell for the detailed report. This is unfortunate. The bug will probably show up with pygame too, as it also uses SDL under the hood.

Should I find another library to build the cross-platform with, or are you confident it can be fixed in a reasonable timeframe? FWIW I've used SFML in the past, but it's less popular.

@rkk-ableton
Copy link

rkk-ableton commented Jun 27, 2023

@mwcampbell accessibilityFocusedUIElement is a property of NSApp; it should only be overridden on the application.

@mwcampbell
Copy link
Contributor

@DataTriny If it's straightforward to use SFML with CMake, please go ahead and try that on another branch, and let's see if that has the same problem on macOS. Another option I was considering is GLFW, though I don't know how much OpenGL setup you have to do just to get a blank window on the screen with GLFW.

@mwcampbell
Copy link
Contributor

To more directly answer your question, I might not be able to do any more work with the SDL example until about a week from now.

@mwcampbell
Copy link
Contributor

Finally getting back to the issue with focus on macOS. Contrary to what @rkk-ableton wrote, I did a quick experiment which shows that the accessibilityFocusedUIElement method does get called on the window. Since trying to subclass the window at runtime (as we do for the view) breaks things badly, I'm instead going to try using class_addMethod in the ObjC runtime to add an accessibilityFocusedUIElement to the SDL window class at runtime. This is less safe than subclassing, because there's no way to remove the method from the class later. So I'll add a note explaining that when using this technique, the AccessKit code must never be unloaded from the process.

@mwcampbell
Copy link
Contributor

The macOS focus issue is solved by #266. The change to the SDL example is simple. Just call this right before creating the macOS adapter:

accesskit_macos_add_focus_forwarder_to_window_class("SDLWindow");

This does require knowing the name of SDL's NSWindow subclass. I didn't want to get the class name from the window itself at runtime, because if I do that, I get the name of the subclass created at runtime by Cocoa's Key-Value Observing (KVO) feature. I figured it would be more robust to add the method to the SDL window class. Also, getting the class name from the window instance at runtime would suggest that this function does something to the window instance, which it doesn't; I wanted to make it clear that it's permanently modifying the class.

@mwcampbell
Copy link
Contributor

@DataTriny Now that I've merged #266, please rebase this branch and add the one line to the SDL example. Then I'll do one more build and test, and assuming that works, we should be ready to merge this and do a release.

@mwcampbell
Copy link
Contributor

I built and ran this version on macOS, and it works. Let's ship it.

@mwcampbell mwcampbell merged commit 1f5cd1f into AccessKit:main Aug 8, 2023
5 checks passed
@mwcampbell mwcampbell mentioned this pull request Aug 7, 2023
@DataTriny DataTriny deleted the c_bindings_sdl_example branch August 8, 2023 17:07
lunixbochs pushed a commit to talonvoice/accesskit that referenced this pull request Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants