vsgQt build fails an macOS

[rainergericke]

An old error is back on the Mac:

NSView* cannot be casted to a c++ class pointer, since it is an OBJC class pointer. This problem returned with commit:c64cf39805f6011e836125979456fd0f5b02299d

Source file: ViewerWindow.cpp line 238

[robertosfield]

Unfortunately I don't have a macOS system or expertise to help resolve this, to get vsgQt working on macOS we'll tackle this as I have had to abandon Qt's won Vulkan support as I design choices Trollteh have adopted road-blocked proper multi-window support.

I have written up details of the new work on the VSG forum

I presume there must be a way of passing window pointers around on macOS but need those with macOS systems and expertise to step up and figure out how we can resolve the issue.

[rainergericke]

Seems since Qt 6.5.2 the build under macOS is possible, after line 155 in file Window.cpp has been changed to
traits->nativeWindow = winId();
That was the good news. The bad news is the tools don't work as needed.
Starting vsgqtviewer leads to the message:
vsgqtviewer teapot.vsgt
Failed to load a valid scenene graph, Please specify a 3d model or image file on the command line.
Seems VSG_FILE_PATH is not evaluated.
Next trial:
vsgqtviewer $VSG_FILE_PATH/models/teapot.vsgt
shows this:

Bringing the white window to the front shows this:

The 3d object can be turned, scaled and moved by mouse action, but the program does not react to window close events and must be closed by ^C from the terminal.

vsgqtwindows $VSG_FILE_PATH/models/teapot.vsgt

shows this:

Sorry, I am not able to present a complete solution, but I hope this is helpful for others.

[robertosfield]

That's good news that at least part of the problem is resolvable. Could you get a PR for the change you've made so far.

vsgqtviewer teapot.vsgt
Failed to load a valid scenene graph, Please specify a 3d model or image file on the command line.
Seems VSG_FILE_PATH is not evaluated.

If you have set VSG_FILE_PATH to vsgExamples/data then to load the teapot.vsgt you'll need to use:

vsgqtviewer models/teapot.vsgt

From the screenshots it looks like there are several issues outstanding, first up the viewport state doesn't look to be syncing up to the dimensions of the window. The close button should be the Qt window related so I'm surprised it's not working, perhaps the vsg::MacOS_Window.mm is affect how events are handled?

Does pressing escape work?

[rainergericke]

Right!

vsgqtviewer models/teapot.vsgt

works. So reading the environment variables is not an issue, fortunately.

The escape key is not recognized.

[robertosfield]

Thanks for the PR, now merged.

Th escape key and close button not working, but the camera manipulator working is an odd combination of non working/working events. Unfortunately I don't have a Mac so can't provide any direct assistance on figuring out what is amiss.

[rainergericke]

Is the additional empty window an intended feature?

[rainergericke]

I just found out: the white window reacts on window close. Then the app terminates. The 3d window only reacts on window resize and move, not on window close.

[robertosfield]

Is the additional empty window an intended feature?

I didn't pick up on that there was two separate windows. I think that suggests that the MacOS_Window.mm isn't using the window passed into MacOS_Window.mm via WindowTraits::nativeWindow isn't being used as intended so it's creating it's own window rather than just adding vkSurface/Swapchain etc. to the Window.

Could you have a look at how Win32_Window.cpp, Xcb_Window.cpp and MacOS_Window.mm all respond to WindowTraits::nativeWindow? I suspect more work needs to be done in MacOS_Window.mm to support this usage case.

[rainergericke]

My last activities about qt are 10 years ago, so I am not really an expert. I tried to understand the code in vsgqtviewer. In my understanding two windows are created, the mainWindow (QMainWindow) and the window (QWindow+vsg). mainWindow is the white window that hears to window close event. window contains the vsg related part. Should windows act as widget inside mainWindow?

[robertosfield]

I have only ever flirted with Qt during my career, working on vsgQt is the closest I've got, but even that is a need to know basis to get Vulkan/VSG integration working, rather than fully knowledge of Qt.

It looks to me like QMainWindow is a window with several features that are common to most applications main windows like menu and status bars, whereas QWindow is more bare bones that you'd need to add these features to.

It looks like QMainWindow can have widgets within that can be a QWindow, but these QWindow themselves would be wrapped. I can't claim to understand why it's done like is is.

I don't know if the separate QMainWindow and QWindow is actually an issue in your case, it may be that Qt itself is probably embedding the QWindow within the QMainWindow. The first things I'd investigate is whether MacOS_Window.mm is creating it's own window or whether it's use the one provided by QWindow.

[rainergericke]

After reading the Qt6 docu, I finally understand your code, I think it is ok. 'window' is set as a widget in 'mainWindow', good so far. My next idea was to remove the manWindow and only to use 'window' from MacOS_Window.mm, not a solution, just for testing.

changed lines in main.cpp of vsgqtviewer (122 to 137):
//QMainWindow* mainWindow = new QMainWindow();

// create the viewer that will manage all the rendering of the views
auto viewer = vsgQt::Viewer::create();

auto window = createWindow(viewer, windowTraits, vsg_scene, nullptr, "First Window");

//auto widget = QWidget::createWindowContainer(window, mainWindow);

//mainWindow->setCentralWidget(widget);

window->setGeometry(windowTraits->x, windowTraits->y, windowTraits->width, windowTraits->height);

window->show();

The program output is the same as bevor, in fact we get two windows, a QWindow() and a NSWindow()!

Actually I dont have any idea how to solve this problem.

[rainergericke]

I have found a possibility to use objc objects directly from c++. It is based on the metal-cpp headers. In the method
_void Window::initializeWindow()_I tried:
#if defined(VK_USE_PLATFORM_WIN32_KHR)
traits->nativeWindow = reinterpret_cast(winId());
#elif defined(VK_USE_PLATFORM_XLIB_KHR)
traits->nativeWindow = static_cast<::Window>(winId());
#elif defined(VK_USE_PLATFORM_XCB_KHR)
traits->nativeWindow = static_cast<xcb_window_t>(winId());
#elif defined(VK_USE_PLATFORM_MACOS_MVK)
traits->nativeWindow = reinterpret_castNS::Window*(winId()); // cast to Cocoa window
traits->nativeWindow = reinterpret_castNS::View*(winId()); // cast to Cocoa view
traits->nativeWindow = reinterpret_castMTK::View*(winId()); // cast to View with Metal layer
traits->nativeWindow = winId()); // no casting
#endif
The effect is always the same, not satisfactory at all.

The Qt6 examples based based on QVulkanWindow work very well on the Mac. Could that be an alternative?

[robertosfield]

On Wed, 9 Aug 2023 at 09:24, Rainer Gericke ***@***.***> wrote: I have found a possibility to use objc objects directly from c++. It is based on the metal-cpp headers. In the method _void Window::initializeWindow()_I tried: *#if defined(VK_USE_PLATFORM_WIN32_KHR) traits->nativeWindow = reinterpret_cast(winId()); #elif defined(VK_USE_PLATFORM_XLIB_KHR) traits->nativeWindow = static_cast<::Window>(winId()); #elif defined(VK_USE_PLATFORM_XCB_KHR) traits->nativeWindow = static_cast<xcb_window_t>(winId()); #elif defined(VK_USE_PLATFORM_MACOS_MVK) traits->nativeWindow = reinterpret_castNS::Window*(winId()); // cast to Cocoa window traits->nativeWindow = reinterpret_castNS::View*(winId()); // cast to Cocoa view traits->nativeWindow = reinterpret_castMTK::View*(winId()); // cast to View with Metal layer traits->nativeWindow = winId()); // no casting #endif* The effect is always the same, not satisfactory at all.

What is the statius of VulkanSceneGraph/src/vsg/platform/MacOS_Window.mm? Does this recieve the was nativeWindow? Does it still attempt to set up its own Window?

The Qt6 examples based based on QVulkanWindow work very well on the Mac. Could that be an alternative? I had to abandon QVulkanWindow as it it's design binds the vkDevice to the

Window which prevents sharing vkDevice between Windows even though they are allocated on the same GPU. It's a really crappy bit of design/implementation from Qt.

…

Message ID: ***@***.***>

[rainergericke]

I understand.

What is the statius of VulkanSceneGraph/src/vsg/platform/MacOS_Window.mm?
Does this recieve the was nativeWindow? Does it still attempt to set up its
own Window?

MacOS_Window.mm generates an NSApplication instance, when it shall create just a window. NSApplication is equivalent to QMainWindow. Maybe the reason for this strange behavior.

[robertosfield]

I have just done quick code review on MacOS_Window.mm and there is no handling of WindowTraits::nativeWindow, so it's clear the the extra Window that is being created on the VSG is happening because MacOS_Window.mm doesn't yet support the usage case when a window is provided by the caller, the relevant code is in:

https://github.com/vsg-dev/VulkanSceneGraph/blob/master/src/vsg/platform/macos/MacOS_Window.mm#L711

Both the Win32_Window.cpp and Xcb_Window.cpp support using an application supplied nativeWindow.

Something similar will be needed in MacOS_Window.mm.