Blender Integration

[BigRoy]

Issue

Implement Blender as Avalon integration so that the tools can operate and prepare an example studio configuration to show a Creator + Publish + Loader action as initial setup.

---

If anyone knows anyone with a good Blender background or Blender + pipeline related experience I'd be happy to get in touch with them and ping them about some Blender workflow things.

I'm focusing currently on Blender 2.8+ (on beta now) and am working on this in my spare time.

[BigRoy]

Here's some implementation details I was thinking about. I mentioned doing some initial work already on the Avalon gitter here.

Containers and Instances

Using the new Blender 2.8 Collections as the object set-like container for loaded content and using them as the set for publish instances. Though I have started a discussion on custom properties on Collections as I wasn't able to completely get it to a user-friendly state.

Avalon menu

I was unable in Blender 2.8 to get the Avalon menu in the top menu bar (in the File Menu) so for now I pushed it into the View Menus, see:

Pyblish QML + Blender

Even though Pyblish doesn't have a Blender host integration it seemed to run fine on the first tries. See:

Reference gitter conversation here

Blender 2.8 + PyQt5

I have been unable so far to run PyQt5 stable inside Blender 2.8 (it's using Python 3.7) as it seems to hang the Blender UI and holds the main thread. I've been able to get it working without holding the UI but then it crashed on reopen or performing actions.

This still needs further investigation. Reading over Blender forums it should be possible to operate Qt inside of Blender - albeit not super easy of the bat.

[mottosso]

I have been unable so far to run PyQt5 stable inside Blender

Aside from listening in on the application quit signal; what else do you need PyQt for inside of Blender? I'd imagine we could find similar signals native to Blender and let Qt exist only external to Blender, like how it does from Maya.

[BigRoy]

Aside from listening in on the application quit signal; what else do you need PyQt for inside of Blender? I'd imagine we could find similar signals native to Blender and let Qt exist only external to Blender, like how it does from Maya.

I think you're referring to the Pyblish QML logic as I mentioned PyQt5. But I was intending to run Qt for the Avalon tools like the Creator, Loader, and Scene Inventory (Manager). I went for PyQt5 since Blender is on Python 3.7 so I felt there was no need to use PyQt4 or PySide as PyQt5 could be pip installed to Python 3.7.

[BigRoy]

I've pushed the current state of the Blender integration work so other coulds fiddle around if they wanted:

• Avalon Core: https://github.com/BigRoy/core/tree/blender2_8
• Colorbleed-config: https://github.com/BigRoy/colorbleed-config/tree/blender2_8

---

PyQt5 install for Blender 2.8

Note that Blender will need PyQt5 to even come close to starting Avalon tools. (Even for Pyblish QML as it will try to import Qt in the host to show a Splash Screen. This dependency could be removed/fixed in Pyblish QML but the dependency is currently there.)

1. You can pip install PyQt5 for Blender 2.8's Python.
2. On Windows for me it failed to launch out of the box. Instead I had to take the python3.dll and move them into the Blender folder as that was a missing DLL that Blender does not have and PyQt5 seems to require. Also mentioned on Avalon Gitter here
3. Now from PyQt5 import QtWidgets should work if you run it in the script editor.

Install avalon on Blender launch

You'll need to set the BLENDER_USER_SCRIPTS environment variable before launching blender and point it to: path/to/avalon-core/setup/blender so that Blender can pickup the startup folder and script.

For example, I've added this in the Blender .toml:

BLENDER_USER_SCRIPTS="{AVALON_CORE}/setup/blender"

[BigRoy]

I am able to run a Qt interface without hanging Blender and both running in parallel. Here's an example of the code:

class QtModalOperator(bpy.types.Operator):
    """A base class for Operators that run a Qt interface."""

    def modal(self, context, event):

        if self._app:
            self._app.processEvents()
            return {'PASS_THROUGH'}

        return {"FINISHED"}

    def execute(self, context):
        """Execute the Operator.

        The child class must implement execute() and call super to trigger this
        class' execute() at the beginning. The execute() method must finally
        return {'RUNNING_MODAL"}

        Note that the Qt code should *not* call QApplication.exec_() as it
        seems that magically the Qt application already processes straight
        away in Blender. Maybe due to:
        https://stackoverflow.com/questions/28060218/where-is-pyqt-event
        -loop-running


        """
        from avalon.vendor.Qt import QtWidgets

        self._app = QtWidgets.QApplication.instance()
        if not self._app:
            self._app = QtWidgets.QApplication(["blender"])


class CreatorOperator(QtModalOperator):
    """Launch Avalon Creator.."""

    bl_idname = "object.avalon_creator"
    bl_label = "Create.."

    def execute(self, context):
        # Initialize Qt operator execution
        super(CreatorOperator, self).execute(context)

        from ..tools import creator
        creator.show()
        return {'RUNNING_MODAL'}

However, trying to access Blender's bpy.context fails with an error stating: context is incorrect. So probably I'll need to investigate some more

[mottosso]

I think you're referring to the Pyblish QML logic as I mentioned PyQt5.

Gah, yes, you are right. Of course we need Avalon GUIs too. :)

PyQt5 could be pip installed to Python 3.7.

You probably need to build it. Else you can get subtle and difficult-to-debug errors like that context error. Worst case you'll need to build not only PyQt but also Qt. :S It depends on how Blender was built, they need the same compiler version and flags throughout. The same is true for Maya.

[BigRoy]

You probably need to build it. Else you can get subtle and difficult-to-debug errors like that context error. Worst case you'll need to build not only PyQt but also Qt. :S It depends on how Blender was built, they need the same compiler version and flags throughout. The same is true for Maya.

The context error is not related to Qt but I believe it's related to calling bpy.context (the Blender context) from a Thread, which is where the Qt application code is running in. I believe in a Modal operator in Blender you should be using the Context that's passed to the operator, but this context is not passed to our Creator/Loader plug-ins, they just do:

import bpy

context = bpy.context

Which is invalid when "inside the context of the Operator" - if that makes sense?

[BigRoy]

So I have made some very good progress regarding this - basically the UIs work without hanging Blender and I am able to use the Creator to create an instance and able to Load an Alembic through the pipeline too + show it in the Scene Inventory. Again, still a draft.

Resolved Blender Qt hanging blender issues

To get the Qt UIs to run they needed to trigger as modal operators, see here.

Resolved Blender context is incorrect issue

The bpy.context invalid error was due to me trying to access bpy.context.selected_objects which is not available in the thread of Qt, so I fixed it and was able to continue development.

Note:
The unfortunate thing really is that whenever any error happens (anything you raise that isn't explicitly captured) from the "Qt thread" then Blender will crash. In some odd cases it will continue to work but not show anything, for example I had an empty Scene Inventory at first when an error happens in the QAbstractItemModel (The InventoryModel).

Note that with the errors you are able to see them (just briefly) in the System Console that comes with Blender, so you are able to debug - but you need to restart Blender. I actually believe the same behavior happens when you run PyQt5 standalone in Python 3, and for example you raise an error from a method that is triggered by a QPushButton then it crashes the Python console - but I haven't checked.

How to more explicitly get these errors to show and run it "safer" for developing purposes I am not sure.

---

Again the current state of the code can be found here:

• Avalon Core: https://github.com/BigRoy/core/tree/blender2_8
• Colorbleed-config: https://github.com/BigRoy/colorbleed-config/tree/blender2_8

[BigRoy]

Note that with the errors you are able to see them (just briefly) in the System Console that comes with Blender, so you are able to debug - but you need to restart Blender. I actually believe the same behavior happens when you run PyQt5 standalone in Python 3, and for example you raise an error from a method that is triggered by a QPushButton then it crashes the Python console - but I haven't checked.

Actually, other people seem to confirm this PyQt5 behavior - see here and here.

Using those workarounds Blender continues to "survive" and the errors are printed to the System Console. It's helpful at least for development/debugging even though according to those Stackoverflow answers it's bad practice. :)

[mottosso]

I actually believe the same behavior happens when you run PyQt5 standalone in Python 3, and for example you raise an error from a method that is triggered by a QPushButton then it crashes the Python console - but I haven't checked.

I suspect this isn't a crash, but a change in how exceptions are handled in PyQt5 under Python 3 versus 2. From what I understand, it's a change to the sys.excepthook which involves a sys.exit() per default. Overriding that to do nothing should/could restore Python 2 behavior, and avoid it looking like a crash.

Edit: I had a look at those links you posted, and basically repeated what you already found out for yourself. :) Good to have a third opinion at least!

[mottosso]

Having a look at threading-related matters for Pyblish QML.

For completeness, the issue relates to Pyblish QML running in a separate Python process, and communicate with Blender via subprocess.Popen.stdout. Blender listens on stdout of the external process from a separate thread, such that whenever QML says "Run this plug-in for me" it's doing that from that separate thread.

The result is your plug-in and all of the calls made to Blender operates entirely in this separate thread.

Blender, like Maya, Houdini, Nuke and pretty much any DCC I can think of, doesn't like being interacted with from a different thread than the so-called "main thread".

The way Pyblish QML manages this in Maya and friends is via an application supplied, aptly named executeInMainThreadWithResult. But Blender doesn't appear to have that, so we must roll our own.

The problem isn't unique to Pyblish QML either; anything running in a thread, wanting to integrate without blocking user input, has the same problem. Looking here shows us one approach.

In a nutshell:

1. Pyblish QML establishes a Queue in Blender
2. The external Python process populates this Queue with a command, and waits for a reply..
3. Blender empties the Queue at a given interval, e.g. every millisecond
4. On a command being present, it is executed and passed back to QML

It'd work, but is a little complicated.

[BigRoy]

There was another discussion about running Qt interfaces in Blender on Avalon Gitter without locking up the Blender GUI
Linking this for future reference

[jasperges]

Just a quick update on my progress.

Running Avalon apps from within Blender

App execution from within Blender seems to work fine for the most part. The apps don't block the Blender UI and also when you access bpy.context from an Avalon app, it's populated correctly. If you set the environment variable AVALON_DEBUG you get an extra 'test app' in the Avalon menu in Blender. The menu is now also in the top bar (I guess the ability to do so was added later in Blender). There are still some (small) issues:

• When having multiple apps open at the same time, loading a work file with Work Files, can close all app windows.
• I've run into an issue in which I am unable to close Pyblish QML. Even when I quit Blender I still have to wait a bit before Pyblish closes.
• A reference to pop-up windows (like the 'Save changes?' in Work Files) is needed, else they are garbage collected and immediately disappear. I now use self.message for example instead of just message. So this one is fixed in my branch.

I didn't have time to thoroughly investigate the first 2 issues. I also don't consider them to be 'criticial', so I will leave them alone for now.

Containers and Instances

Just like @BigRoy did here, I also use collections to store custom data for Avalon. As he found out on Blender Artists there is no simple way to make these available to the user, so they can be changed. You can register 'custom properties', but you have to know in advance what kind of information you will store. Doing it this way will make it very rigid and you will not be able to simply add studio specific properties. If you dynamically add a property (for example collection['avalon'] = {'asset': 'my_asset', 'dependencies': ['a', 'b', 'c']} it's very flexible (you can then store dicts, lists, string, numbers, strings and bools), but you can't easily expose this in the UI.

What I intend to do is add all the information to a 'dynamically assigned' avalon property (so all Avalon related things are nicely grouped) and add an Avalon panel that allows you to interact with these properties.

Config

I started to make a config based on Colorlbeed's config that integrates Blender 2.80. It's all very much work in progress and so far only used to test things and figure out how stuff could work.

Testing

If you would like to see what I've done so far and give it a go, you can find it here. Reporting any issues or giving suggestions will be greatly appreciated!

• Avalon Core on Gitlab or Github
• Sonar config on Gitlab

---

As a final remark: the next couple of weeks I won't be available. After that I plan to continue working on this.

[jasperges]

Just created a PR: #466

[jasperges]

So far the use of Qt apps inside of Blender actually seem to work quite okay. But there are still some issues. The main issue is that (when I tested it quickly) it's totally unusable on macOS. That is quite unfortunate, but I expect most people are using Linux or Windows, so I will leave it alone for now.

Furthermore there is a difference between Linux and Windows: in Windows only the Global Context is available if you do bpy.context from within a Qt app, but on Linux it seems you have the Screen Context.

Another problem is how the Qt window is kept 'alive'. A modal operator is abused to trigger QApplication.processEvents() all the time. When opening a new file it seems the modal handler is removed and the timed events are no longer triggered, which leave the Qt app in a 'dead' state.
With the new bpy.app.timers introduced in Blender 2.80 this this should be fixed.
At first I thought this didn't work properly, but actually it does. When you register a timer to trigger the processEvents() you also only have the Global Context on Linux. So I will change the code so a timer is used and all should be good. Also add persistent=True and the Qt app will keep running even when you open a new file. Yeah!
(Maybe this even fixes the macOS issue, but I can't remember if that worked or not.)

So in the planned documentation for the Blender part I would like to make it clear, that you should only expect to have the Global Context available. Some library functions can be added to 'ease the pain', for example for getting the selected objects.

---

To do some quick testing with different methods, I made this snippet. Just save it, load it in the Script Editor in Blender and do Alt + p or press the Run Script button. Just make the changes you want and run it again.

[jasperges]

Just committed some changes (#466). The most important one is that instead of a timed modal operator I switched to using bpy.app.timers to trigger QApplication.processEvents(). This has several benefits:

• It works on Linux, Windows and macOS. Yes, macOS now also works. The only thing I noticed is that the Blender interface gets very laggy if you set the interval to less then 0.01.
• If you use bpy.context you only get the Global Context, also on Linux. So it's consistent across OSes.
• When you load a new file the apps keep running, because I use persistent=True.

So things are looking good as far as I am concerned. :)