Separate import into steps and other changes

[davidrios]

Hello, sorry for the mega PR, it ended up just happening 😅

The basic idea of this PR is to separate the import into distinct steps, because there're a few issues with processing order that can't be solved with the standard import flow. There are flexibility gains as well, which will be described shortly.

I'll also put some speed numbers from my machine on an export set that I'm testing which has around 19k files total and 30 map tiles. Unity doesn't use more than 1 thread, so these should be roughly representative in general. These are the implemented steps:

1. The standard processing that Unity does automatically when one adds files to the Assets folder. This now only does some minimal required processing, so its speed is pretty much entirely dependent on Unity, it's the same even without the plugin installed. It takes around 4 minutes here.
2. This is the processing of WMOs and M2s and it's available via a button in a newly implemented wow.unity window. One clicks it and it goes through all available wow assets and do the post-processing. It first processes the raw models, creating materials, etc, and after that populates the WMOs with doodads. Its implementation has some quirks for performance reasons, but it ended up being pretty fast, it takes around 1:30 minutes here to process ~700 objs and ~900 materials. This button can be clicked as many times as one likes, it won't reprocess stuff that is already processed, so subsequent clicks run very fast. It has the added flexibility that it won't mess up stuff that was already changed, for example if one has customized some models and later decides to add additional exported models/tiles, it'll only process the newly added things. It also uses a cancellable progress bar, so it's safe to stop and start it again at any time. I've implemented a popup that reminds the user to execute this step after the automatic Unity import just in case.
3. This step creates and sets up the materials for the terrain. To do it one needs to select the desired tiles from the Project window, they'll show up in the wow.unity window, then one clicks the Setup Terrain button. It provides the flexibility to process as many or few tiles as desired. It's also cancellable and idempotent like asset processing. The implementation is as well optimized as I could get it, but I think it's not too bad at around 7 minutes to process 30 tiles and create materials for over 7.5k chunks. The implementation lives in the new ADTUtility module
4. The final step is placing doodads in the map tile prefabs. It works the same, select the tiles in the Project window and click Place Doodads. It's pretty fast, takes less than a minute for 30 tiles, unless the other assets haven't been processed yet, which it'll do automatically given that they need to exist to be able to be placed. This could easily be changed to place the doodads in a prefab instanced in the scene instead as well.

Other notables changes:

• General model post-processing logic moved to the AssetConversionManager module. It goes through all wow assets, processing the M2s and queuing the WMOs and collision setup for subsequent processing. This is done for performance reasons, as it uses AssetDatabase.StartAssetEditing() for speed, but live modification of existing prefabs doesn't play well with that.
• Post-processing logic for individual items is implemented in M2Utility, WMOUtility and ItemCollectionUtility, they don't operate on the whole asset list by themselves anymore, exposing a function to process a single item instead. ADTUtility differs in that it operates on a list instead of individual tiles for performance and convenience reasons.
• Doodad placement is now setup in a nested hierarchy based on the set name, for example Set_A, Set_B, etc. They are disabled by default in the prefab (except Set_$DefaultGlobal), and when they are placed by the script in the map tile they have the specific sets for that instance enabled. One can also do that manually after placing the prefab in a scene. I have opened a PR in the wow.export project that makes this even better, as it'll always export a single WMO with all doodad sets instead of different copies of the same WMO for each doodad set combination, so this makes it really convenient to customize the WMO prefab once and have it updated everywhere in the map.
• Changed how material creation is handled. It doesn't use the exported mtl files anymore, instead it reads the information from each model's JSON metadata, so that it can create different materials from the same texture depending on rendering settings, resulting in more accurate rendering. The MaterialUtility model now exposes different methods to create materials for M2s, WMOs and map tiles. Most of the logic for M2 and WMO remains shared, as they are very similar, but there are some rendering flags that differ and are now handled.
• Materials are saved at the same place their texture lives for better organization, except for map textures which are saved to a subfolder of the maps folder.
• For M2/WMO materials: Use Lit instead of Simple Lit shader to handle smoothness information that's available for some materials. Implement emissive textures for WMO windows. Other minor fixes.

[briochie]

Hey @davidrios no problem - thank you for making this! I don't have time to dive into these changes right away, but I do want to take a look and play around with it. I appreciate you circling back on this.

Once I get a chance to mess with it I will give you any feedback I have. :)

[briochie]

@davidrios thank you for the continued work 🙏

I have still been crazy busy in my personal life and have been focused on other projects. I recently built a hurdy gurdy. It sounds awful.

I will commit to getting back to this sometime in September. I’ve been trying to block off months or weeks for certain projects and hobbies, and plan to fully try out the changes you’ve made here.

[davidrios]

Don't worry about it, I'm just doing my thing anyway 😅
Wow, that hurdy gurdy thing sounds interesting! How do one get to do that?

[briochie]

I tried making one out of a pine box and some hardware store parts years ago, but gave up on it. Then I learned about this project:

https://www.printables.com/model/492694-electric-hurdy-gurdy

I asked a friend to 3D print it for me, along with the remixes so I could make it acoustic. It looks cool, but sounds terrible when trying to use the keys.