Skip to content

Asset cooking (pre 1.2.0)

Jump to bottom Edit New page
Xymanek edited this page Dec 19, 2021 · 2 revisions

IMPORTANT: This guide is applicable only X2MBC versions prior to 1.2.0. The guide for current version can be found at https://github.com/X2CommunityCore/X2ModBuildCommon/wiki/Asset-cooking

Asset cooking is a process that can yield great benefits but it's an advanced technique so to use it correctly (and to avoid making the situation worse) you must understand the theory behind it. Note that that article was written before a workable way to cook mod assets was found.

IMPORTANT: Do not skip or skim the link above. You will not understand what this article talks about

Also be aware that if you didn't fix the TLE issue with your SDK, the cooker will simply crash without any output. You can find the instructions on the wiki

How is mod asset cooking is implemented?

Unreal Engine has no concept of "projects" before version 4 - it does support DLC/mod ("usermode") cooking but FXS has made some changes to the cooker so that functionality no longer works as described in the UE3 docs. As such a different approach was taken. The mod assets are "injected" into the SDK's content and then we cook, for all intents and purposes, the game itself. However, before doing that, we also make some modifications to avoid cooking all the game's packages for a few reasons:

  • to save time during the build (duh)
  • to avoid "polluting" (filling with unneeded content) the .tfcs that will be generated (and shipped) for your mod

As such any assets referenced by those which need to be cooked will be included in your SF packages or TFCs.

Use cases for cooking

Cooking mod assets works well in 2 cases:

  • When all the assets are custom (made by you)
  • When you reference game's assets which are not part of a seekfree standalone package

In the first case you can expect ~10x package size reduction on average. In the second case, inlining the SF standalone missing assets into your mod is likely to take less space than including the original .upk from SDK (the editor).

In case your assets reference objects that reside within packages that were already shipped as SF standalone with the game, the only benefit would be a slight loading time boost for players with a HDD, but the price you pay is bloating your mod by duplicating game's assets.

Cooking your assets

Heads up: Cooking has been tested only with the full_content version of the SDK. Cooking will likely fail otherwise, even if you plan to cook exclusively custom assets

As mentioned before, not all assets are good candidates for cooking. As such, the build script expects a new folder in your mod - ContentForCook. The maps and packages there will be run through the cooker and only the SF versions will be shipped with your mod - not the originals.

Important: assets in the ContentForCook directory cannot reference objects from your packages inside the Content folder - the cooker will simply not find the latter. This limitation is on purpose - it prevents you from accidentally duplicating your own assets

After you prepared your assets for cooking, you need to tell the build script what you want to do with them. To do that, you must first setup ContentOptions.json and then set the cooking options (setting any of them enables asset cooking, otherwise it's skipped).

  • sfStandalone - each element of the array is the name of a package (sans the extension, .upk) which will be made SeekFree Standalone
  • sfMaps - each element of the array is the name of a map (sans the extension, .umap) which will be made SeekFree
  • sfCollectionMaps - extremely advanced, see section below

Remember - not every package needs to be seekfree. It's perfectly fine (and intended) to have packages which are meant to be only referenced by other packages (or levels) and never loaded directly. Since the cooker will "pack" everything together anyway, you can separate your packages however you wish - by content type (e.g. specific packages for animations, materials, meshes, etc.), by feature/purpose/segment/part, by max size, etc.

Collection maps

DANGER: If you misuse this feature, your assets will simply not load at all

Sometimes you want to be able to load arbitrary assets from several packages at the same time. A simple example would be several custom UI elements which you expect the player to see while in strategy, with a package per each UI element. The simplest solution would be to make each of those packages SF standalone and add them to the MapContent (see Textures loaded by UI). However, there are 2 slight issues with this approach:

  • Players using an HDD will experience longer loading time (due to the HDD jumping around to read the different packages - the exact scenario SeekFree approach was designed to avoid)
  • (Different use case) If more than 1 of these packages reference the same shared asset, said asset will be duplicated - inlined into each SF package

A "simple" approach would be to create a "collection package" that will have nothing but an ObjectReferencer (or a set of them) to cause the cooker to inline the objects from the individual "source" packages. The degree to which this approach is laborious and error-prone cannot be understated as you have to list each and every object individually. Instead, we can use UE3's PackagesToForceCookPerMap to cause our packages to be made SF and packaged together into a single file - the "map". In this case, the assets will always be available as long as the map is loaded.

Here's an example from the Covert Infiltration mod:

"sfCollectionMaps": [
	{
		"name": "StrategyCollection_CI",
		"packages": [
			"GeoscapeMesh_CI",
			"UILibrary_CI_ChainPreview",
			"UILibrary_CovertInfiltration"
		]
	}
]

This will create a single SF map - StrategyCollection_CI - which will include everything from the 3 packages.

You don't need to provide a CollectionMap.umap (e.g. StrategyCollection_CI.umap in the example above) - the build script will provide it for you, since in most cases it will be empty. If you do want something in the map itself, just create the file yourself inside the ContentForCook directory and it will be used instead

Important: the assets inside the collection will be available only while the map is loaded - otherwise the game will simply have no idea where to look for these assets

Loading collection maps

MapContent

Continuing the above example, we want our StrategyCollection_CI to be always loaded while, well, in strategy. We can use our trusty Content.MapContent config while abusing the fact that maps are also unreal packages:

[Content.MapContent]
.Map=Avenger_Root
.Package=StrategyCollection_CI

Other approaches

Any way to load a map will work for this scenario. For example, you can manually control the loading state using

`MAPS.AddStreamingMap();
`MAPS.RemoveStreamingMapByName();

NOTE: While there is no reason to believe that the above will not work, it has not been tested

Cooking output

The cooking process will add a new folder to the final version of the built mod - CookedPCConsole. It will contain the seekfree packages (all extensions will be changed to .upk, even for maps) and the TFCs - if you provide/reference any textures that go there (e.g. UI textures do not).

However, the actual cooking process produces more files, which do not need to be shipped with your mod, such as GlobalPersistentCookerData.upk and others. These are required to maintain the cooking process, "cache" it (avoid recooking SF packages when the original(s) did not change) or are simply unavoidable due to how the cooker is implemented (e.g. they would be used if we were cooking the actual game). These files (together with the original copies of the SF packages + TFCs) reside in BuildCache\PublishedCookedPCConsole of your mod project. Delete this folder if you want to force a full recook

Pitfalls and peculiarities of cooking

First of all, cooking assets is an extremely new practice and you should make sure to test

  • your assets in-game to make sure they work as expected
  • the file sizes of cooked packages to make sure that any inlining of base game assets (e.g. by mistake) isn't negating the compression advantages

It's also a good idea to defer cooking until your asset(s) are ready and have been tested in-game - this makes it easier to figure out whether the problem is with the asset itself or with the (usage of) cooker.

Opening cooked packages

Recall that as part of cooking process the editor-only data is removed from the packages - this makes opening them with the editor impossible (just like it's impossible to open a ModShaderCache despite it using a .upk extension). It's a good idea to provide a link to the original packages, e.g. a GitHub repository

Textures loaded by UI

When the UI (scaleform/flash) attempts to load from disk a texture (either as a visual for your custom UI or an image/icon for research, item, ability, etc.) it will fail if the texture resides in a SF standalone package and is inside a grouping (e.g. SomePackage.SomeGrouping.SomeTexture instead of SomePackage.SomeTexture). There are two ways to avoid this problem

  1. Move the texture to the root of the package
  2. Make sure the texture is already loaded in memory when the UI attempts to load it

FXS always uses either of these options, if not both at the same time (which is why they probably were not aware of this issue).

Using the first option is the simplest, albeit likely tedious as you have to go and change all the usages. Using the 2nd option will depend on how/when your textures are used.

For example, if the texture is a part of a custom UI that the player will interact with while in strategy you can force your package to be always loaded while in strategy, by adding the following code to XComContent.ini:

[Content.MapContent]
.Map=Avenger_Root
.Package=YourPackageWithTexture

IMPORTANT: If you use +Map, your config will be ignored if the player doesn't have DLC2

Note: this is the approach that FXS uses for their UI

Add a custom footer
Add a custom sidebar
Clone this wiki locally