EXPERIMENTAL! Introduced a custom LevelDB impl with regions and better key format

[dktapps]

This new impl (which is not loadable by vanilla) is targeted at very large worlds, which experience significant I/O performance issues due to a variety of issues described in #6580.

Two main changes are made in RegionizedLevelDB:

• First, multiple LevelDBs are used, which cover a fixed NxN chunk segment of terrain, similar to Anvil in Java. However, there's no technical constraint on these region sizes, unlike Anvil (which is limited to 32x32 chunks). Several experimental sizes are supported by default in WorldProviderManager.
• Second, bigEndianLong(morton2d(chunkX, chunkZ)) is used for chunk keys instead of littleEndianInt(chunkX).littleEndianInt(chunkZ). This new scheme has much better cache locality than Mojang's version, which reduces overlap and costly DB compactions.

The following new provider options are available as a result of this change:

• custom-leveldb-regions-128
• custom-leveldb-regions-256

128 will probably be the sweet spot. 256 will generate 4x fewer regions (might be better for space & file handle usage), but will probably experience more costly compactions (though still far less expensive than multi-GB worlds in regular leveldb format).

Note that the different variations of custom-leveldb-regions-* are not cross-compatible.
Conversion between the different formats is necessary if you want to change formats.

Related issues & PRs

Related to #6580

Changes

Backwards compatibility

Should be fully backwards compatible. While significant changes were made to LevelDB, this was mainly in the interest of extracting a base class for RegionizedLevelDB to inherit from.

Follow-up

• Add warnings for users with large worlds that their world performance will probably suffer if using leveldb
• Make regionized leveldb the default
• Add an easier way to export a world in a specific format from PM (instead of using convert-world.php which requires a src install)

Tests

TBD

[dktapps]

Turns out 128x128 has a significantly larger performance benefit than I expected when converting worlds.

On account of the average region size at 128x128 chunks being about 50 MB, this means that level 3 compactions are avoided in the vast majority of cases. (256x256 hovers at about 200 MB, which exceeds the threshold for level 3 compactions).

In particular, converting to custom-leveldb-regions-128 from any format is several times faster than custom-leveldb-regions-256 for a 10 GB world (4 hours vs 90-100 mins).

[dktapps]

For posterity: I hesitate to keep going with this change, because I think LevelDB is just fundamentally unsuited to the task of storing worlds, regardless of whether a single DB is used or multiple.

While Mojang's usage of it also couldn't be any more suboptimal (about the worst possible cache locality on account of the poor key structure), LevelDB is also just not suited to this job.

Compactions are always going to be costly for storing blob data (usually a few KB per key in Bedrock), because keys and values are stored together. Values have to be copied during sorting, which greatly increases the I/O cost of compactions. RocksDB BlobDB would be a much better solution for this.

Splitting up into many DBs reduces the amount of live data at any given time, meaning that only live DBs will get compaction work done. However, since compaction is all about sorting data (which doesn't really benefit Bedrock anyway since iteration & range scans are not really required beyond the scope of a single chunk), I'm questioning whether it even makes sense to use a KV-store at all.

I'm also hesitant to once again introduce a custom world format which is only supported by PocketMine-MP. The onus would be on PMMP to maintain support for it forever just like PMAnvil, except this would be rather more complicated.

If we do decide to introduce a custom format into mainline, we really need to get this right.