From e66c1538a1c1b47d93111a3aaec5bacfecc5ab43 Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Tue, 7 Jan 2025 15:51:20 +1100 Subject: [PATCH 01/11] Fix typo. --- doc/topics/selection-algorithm-details.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index ff8cc4cbf..d35157062 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -109,7 +109,7 @@ When these two conditions are met for a tile's descendants, then we should not r We decide whether a _Kick_ is necessary by first traversing the tile's children normally, as in a _REFINE_. Calling `_visitTileIfNeeded` on each child tile returns a `TraversalDetails` instance with three fields: * `allAreRenderable`: `true` if every selected tile is renderable. `false` if even one selected tile is not yet renderable. -* `anyWereRenderedLastFrame`: `true` if any selected tile was rendered last frame. `false` is none of the selected tiles were rendered last frame. +* `anyWereRenderedLastFrame`: `true` if any selected tile was rendered last frame. `false` if none of the selected tiles were rendered last frame. * `notYetRenderableCount`: The number of selected tiles for which `isRenderable` is false. The `TraversalDetails` for all children are combined in the intuitive way: `allAreRenderable` values are combined with a boolean AND, `anyWereRenderedLastFrame` values are combined with a boolean OR, and `notYetRenderableCount` values are combined by summing. From edd488c6ed60d848d4cb1556c8c337739767608e Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Wed, 8 Jan 2025 22:46:16 +1100 Subject: [PATCH 02/11] Add tile loading sections. --- .../include/Cesium3DTilesSelection/Tileset.h | 8 ++-- doc/topics/selection-algorithm-details.md | 44 ++++++++++++++++++- 2 files changed, 46 insertions(+), 6 deletions(-) diff --git a/Cesium3DTilesSelection/include/Cesium3DTilesSelection/Tileset.h b/Cesium3DTilesSelection/include/Cesium3DTilesSelection/Tileset.h index 1c062ea2f..9858ff275 100644 --- a/Cesium3DTilesSelection/include/Cesium3DTilesSelection/Tileset.h +++ b/Cesium3DTilesSelection/include/Cesium3DTilesSelection/Tileset.h @@ -494,14 +494,14 @@ class CESIUM3DTILESSELECTION_API Tileset final { /** * @brief Medium priority tiles that are needed to render the current view - * the appropriate level-of-detail. + * at the appropriate level-of-detail. */ Normal = 1, /** - * @brief High priority tiles that are causing extra detail to be rendered - * in the scene, potentially creating a performance problem and aliasing - * artifacts. + * @brief High priority tiles whose absence is causing extra detail to be + * rendered in the scene, potentially creating a performance problem and + * aliasing artifacts. */ Urgent = 2 }; diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index d35157062..fd7d8f7bb 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -139,12 +139,52 @@ The `loadingDescendantLimit` works like a heuristic for deciding when intermedia Once the current tile finishes loading and is rendered, only then will the tiles deeper in the subtree be given an opportunity to load and render. This ensures that the user sees the model sooner, at the cost of loading more tiles. The idea is to strike a tunable balance between user feedback and loading efficiency. +## Tile Content Loading + +It is important to understand the distinction between a [Tile](@ref Cesium3DTilesSelection::Tile) and its _content_. In the 3D Tiles specification, a [Tile](https://github.com/CesiumGS/3d-tiles/blob/main/specification/PropertiesReference_3dtiles.adoc#tile) is a node in the tileset's bounding-volume hierarchy (BVH), and has a bounding volume, a transform, a geometric error value, and more. Tiles are arranged in a tree, so every `Tile` has a parent `Tile` (except the root), and zero or more children. Tiles also have a `content.uri` property which points to the externally-stored _content_ for the tile. This external content is usually some sort of renderable object, such as a glTF model or a legacy container format like [b3dm](https://github.com/CesiumGS/3d-tiles/blob/main/specification/TileFormats/Batched3DModel/README.adoc#tileformats-batched3dmodel-batched-3d-model). It can also be a further subtree of the BVH rooted at this node, which is known as an [external tileset](https://github.com/CesiumGS/3d-tiles/tree/main/specification#core-external-tilesets). + +Because _content_ is by far the largest part of a tile, and the most time-consuming to load, Cesium Native aims to only load it when when it is needed. The loading happens through a small state machine maintained in the `_loadState` property of each tile. The possible load states are captured in the [TileLoadState](@ref Cesium3DTilesSelection::TileLoadState) enumeration. + +A `Tile` starts in the `Unloaded` state. A `Tile` in this state is added to the `_workerThreadLoadQueue` the first time it is visited during the selection algorithm. Any tiles that remain in this queue at the end of the selection (that is, that are not [kicked](#kicking)) are [prioritized](#load-prioritization) for loading. + +The number of tiles that may load simultaneously is controlled by the [TilesetOptions::maximumSimultaneousTileLoads](\ref Cesium3DTilesSelection::TilesetOptions::maximumSimultaneousTileLoads) property. We can picture tile loading as a swimming pool with `maximumSimultaneousTileLoads` swim lanes. The highest priority tiles jump in the pool, each in their own lane, and race for the other end. When a tile reaches the other side (finishes asynchronous loading), the next highest priority tile can jump in the pool in that now-unoccupied lane and start swimming. + +The swim across the pool includes all of the steps of the tile loading process that do not need to happen on the main thread, including: + +* Initiating an HTTP request for the tile content and receiving the response. +* Parsing the content from the response, such as parsing the glTF or external tileset JSON. +* Decoding compressed geometry and textures. +* Generating extra data needed for rendering, such as normals and raster overlay texture coordinates. + +When a tile jumps in the pool, its `TileLoadState` is changed to `ContentLoading`. When it reaches the other end, the state is changed to `ContentLoaded`. + +The next time the selection algorithm runs, and it sees a tile in the `ContentLoaded` state, the tile is added to the `_mainThreadLoadQueue`. Just like with the `_workerThreadLoadQueue`, tiles may be kicked from this queue as the selection algorithm proceeds, and those that remain are prioritized as described in the next section. This time, though, the loading happens synchronously, on the same thread that is running the selection algorithm. To avoid monopolizing too much main thread time, which could have a severe impact on interactivity, the highest priority tiles from this queue are processed only until the [TilesetOptions::mainThreadLoadingTimeLimit](\ref Cesium3DTilesSelection::TilesetOptions::mainThreadLoadingTimeLimit) has elapsed. Additional tiles will need to wait until the next frame. + +The primary task that is completed during main thread loading is to call [IPrepareRendererResources::prepareInMainThread](\ref Cesium3DTilesSelection::IPrepareRendererResources::prepareInMainThread). See [Rendering 3D Tiles](#rendering-3d-tiles) for details. + +Once a tile has completed its main-thread loading, it enters the `Done` state and is ready to be rendered. + +## Load Prioritization {#load-prioritization} + +Tiles that need to be loaded are prioritized so that the most important tiles are loaded first. Load priority consists of a priority _group_ plus a priority value within that group. The group is chosen according to the reason that this tile is needed by the selection algorithm. The groups are as follows: + +* `Preload` - Low priority tiles that aren't needed right now, but are being preloaded for the future. +* `Normal` - Medium priority tiles that are needed to render the current view at the appropriate level-of-detail. +* `Urgent` - High priority tiles whose absence is are causing extra detail to be rendered in the scene, potentially creating a performance problem and aliasing artifacts. + +Within the group, a tile with a lower priority value will be loaded before a tile with a higher priority value. The priority value is computed as follows: + +``` +(1.0 - dot(tileDirection, cameraDirection)) * distance +``` + +Where `distance` is the distance from the camera to the closest point on the tile, `tileDirection` is the unit vector from the camera to the center of the tile's bounding volume, and `cameraDirection` is the look direction of of the camera. The idea is that tiles that are near the center of the screen and closer to the camera are loaded first. + ## Additional Topics Not Yet Covered {#additional-topics} Here are some additional selection algorithm topics that are not yet covered here, but should be in the future: -* Load prioritization -* Forbid Holes +* Forbid Holes * Unconditionally-Refined Tiles * Occlusion Culling * External Tilesets and Implicit Tiles From 3d2f173f38406fbabeb48965af050e54ce205b29 Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Wed, 8 Jan 2025 23:05:19 +1100 Subject: [PATCH 03/11] Add incomplete tile loading state diagram. --- doc/diagrams/tile-load-states.mmd | 14 ++++++++++++++ doc/topics/selection-algorithm-details.md | 2 ++ 2 files changed, 16 insertions(+) create mode 100644 doc/diagrams/tile-load-states.mmd diff --git a/doc/diagrams/tile-load-states.mmd b/doc/diagrams/tile-load-states.mmd new file mode 100644 index 000000000..4280a78eb --- /dev/null +++ b/doc/diagrams/tile-load-states.mmd @@ -0,0 +1,14 @@ +stateDiagram-v2 + unloaded : Unloaded + contentLoading : Content Loading + contentLoaded : Content Loaded + done : Done + failed : Failed + failedTemp : Failed Temporarily + unloading : Unloading + [*] --> unloaded + unloaded --> contentLoading: Start Async Loading + contentLoading --> contentLoaded: Done Async Loading + contentLoaded --> done: Main Thread Loading + contentLoaded --> unloaded: Unload Tile Content + done --> unloaded: Unload Tile Content diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index fd7d8f7bb..a71f6d62a 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -145,6 +145,8 @@ It is important to understand the distinction between a [Tile](@ref Cesium3DTile Because _content_ is by far the largest part of a tile, and the most time-consuming to load, Cesium Native aims to only load it when when it is needed. The loading happens through a small state machine maintained in the `_loadState` property of each tile. The possible load states are captured in the [TileLoadState](@ref Cesium3DTilesSelection::TileLoadState) enumeration. +@mermaid{tile-load-states} + A `Tile` starts in the `Unloaded` state. A `Tile` in this state is added to the `_workerThreadLoadQueue` the first time it is visited during the selection algorithm. Any tiles that remain in this queue at the end of the selection (that is, that are not [kicked](#kicking)) are [prioritized](#load-prioritization) for loading. The number of tiles that may load simultaneously is controlled by the [TilesetOptions::maximumSimultaneousTileLoads](\ref Cesium3DTilesSelection::TilesetOptions::maximumSimultaneousTileLoads) property. We can picture tile loading as a swimming pool with `maximumSimultaneousTileLoads` swim lanes. The highest priority tiles jump in the pool, each in their own lane, and race for the other end. When a tile reaches the other side (finishes asynchronous loading), the next highest priority tile can jump in the pool in that now-unoccupied lane and start swimming. From db43ce4a7b15c365b5f9f697c1afb70883eb9e8d Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Fri, 10 Jan 2025 15:40:56 +1100 Subject: [PATCH 04/11] Add remaining states to load state diagram. --- doc/diagrams/tile-load-states.mmd | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/diagrams/tile-load-states.mmd b/doc/diagrams/tile-load-states.mmd index 4280a78eb..7ac5a751a 100644 --- a/doc/diagrams/tile-load-states.mmd +++ b/doc/diagrams/tile-load-states.mmd @@ -9,6 +9,10 @@ stateDiagram-v2 [*] --> unloaded unloaded --> contentLoading: Start Async Loading contentLoading --> contentLoaded: Done Async Loading + contentLoading --> failed: Load Failed + contentLoading --> failedTemp: Retry Later + failedTemp --> contentLoading: Start Async Loading contentLoaded --> done: Main Thread Loading contentLoaded --> unloaded: Unload Tile Content - done --> unloaded: Unload Tile Content + done --> unloading: Start Unloading + unloading --> unloaded: Done Unloading From ef651ae55d6740ec05851d16f81ca9f174b3ab59 Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Fri, 10 Jan 2025 17:24:37 +1100 Subject: [PATCH 05/11] Add more load state details. --- doc/topics/selection-algorithm-details.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index a71f6d62a..01e7daf23 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -147,9 +147,9 @@ Because _content_ is by far the largest part of a tile, and the most time-consum @mermaid{tile-load-states} -A `Tile` starts in the `Unloaded` state. A `Tile` in this state is added to the `_workerThreadLoadQueue` the first time it is visited during the selection algorithm. Any tiles that remain in this queue at the end of the selection (that is, that are not [kicked](#kicking)) are [prioritized](#load-prioritization) for loading. +A `Tile` starts in the `Unloaded` state. A `Tile` in this state is added to the `_workerThreadLoadQueue` the first time it is visited during the selection algorithm. Any tiles that remain in this queue at the end of the selection (that is, that are not [kicked](#kicking)) are prioritized for loading as described in the [next section](#load-prioritization). -The number of tiles that may load simultaneously is controlled by the [TilesetOptions::maximumSimultaneousTileLoads](\ref Cesium3DTilesSelection::TilesetOptions::maximumSimultaneousTileLoads) property. We can picture tile loading as a swimming pool with `maximumSimultaneousTileLoads` swim lanes. The highest priority tiles jump in the pool, each in their own lane, and race for the other end. When a tile reaches the other side (finishes asynchronous loading), the next highest priority tile can jump in the pool in that now-unoccupied lane and start swimming. +The number of tiles that may load simultaneously is controlled by the [TilesetOptions::maximumSimultaneousTileLoads](\ref Cesium3DTilesSelection::TilesetOptions::maximumSimultaneousTileLoads) property. We can picture tile loading as a swimming pool with `maximumSimultaneousTileLoads` swim lanes. The highest priority tiles jump in the pool, each in their own lane, and race for the other end. When a tile reaches the other side (finishes asynchronous loading), the next highest priority tile can jump in the pool in that now-unoccupied lane and start swimming. Multiple tiles can never share a swim lane. The swim across the pool includes all of the steps of the tile loading process that do not need to happen on the main thread, including: @@ -160,19 +160,23 @@ The swim across the pool includes all of the steps of the tile loading process t When a tile jumps in the pool, its `TileLoadState` is changed to `ContentLoading`. When it reaches the other end, the state is changed to `ContentLoaded`. -The next time the selection algorithm runs, and it sees a tile in the `ContentLoaded` state, the tile is added to the `_mainThreadLoadQueue`. Just like with the `_workerThreadLoadQueue`, tiles may be kicked from this queue as the selection algorithm proceeds, and those that remain are prioritized as described in the next section. This time, though, the loading happens synchronously, on the same thread that is running the selection algorithm. To avoid monopolizing too much main thread time, which could have a severe impact on interactivity, the highest priority tiles from this queue are processed only until the [TilesetOptions::mainThreadLoadingTimeLimit](\ref Cesium3DTilesSelection::TilesetOptions::mainThreadLoadingTimeLimit) has elapsed. Additional tiles will need to wait until the next frame. +The next time the selection algorithm runs, and it sees a tile in the `ContentLoaded` state, the tile is added to the `_mainThreadLoadQueue`. Just like with the `_workerThreadLoadQueue`, tiles may be kicked from this queue as the selection algorithm proceeds, and those that remain are ranked by priority. This time, though, the loading happens synchronously, on the same thread that is running the selection algorithm. To avoid monopolizing too much main thread time, which could have a severe impact on interactivity, the highest priority tiles from this queue are processed only until the [TilesetOptions::mainThreadLoadingTimeLimit](\ref Cesium3DTilesSelection::TilesetOptions::mainThreadLoadingTimeLimit) has elapsed. Additional tiles will need to wait until the next frame. The primary task that is completed during main thread loading is to call [IPrepareRendererResources::prepareInMainThread](\ref Cesium3DTilesSelection::IPrepareRendererResources::prepareInMainThread). See [Rendering 3D Tiles](#rendering-3d-tiles) for details. Once a tile has completed its main-thread loading, it enters the `Done` state and is ready to be rendered. +If something goes wrong during the `ContentLoading` state, such as an HTTP error because the tile's content URL is invalid or the server is down, the tile enters the `Failed` state. Failed tiles will usualy show up as missing data or "holes" in the model. Content loads can also fail temporarily, such as when an access token expires and needs to be refreshed. Such tiles will transition to the `FailedTemporarily` state, and the async loading process will be restarted the next time the tile is needed. + +Finally, there is the `Unloading` state. When a tile is no longer needed, it usually transitions directly and synchronously from the `ContentLoaded` or `Done` state to the `Unloaded` state. However, if the tile being unloaded happens at the same time to be the source of an active raster overlay "upsampling" operation, then unloading its content would lead to undefined behavior. Instead, we only unload its renderer resources (by calling [IPrepareRendererResources::free](\ref Cesium3DTilesSelection::IPrepareRendererResources::free)) but do _not_ delete its content. We put it in the `Unloading` state to mark that this has been done, and transition it to the `Unloaded` state when it is safe to do so. For more details, see the [pull request](https://github.com/CesiumGS/cesium-native/pull/554) where this mechanism was added. + ## Load Prioritization {#load-prioritization} -Tiles that need to be loaded are prioritized so that the most important tiles are loaded first. Load priority consists of a priority _group_ plus a priority value within that group. The group is chosen according to the reason that this tile is needed by the selection algorithm. The groups are as follows: +Tiles that need to be loaded are prioritized so that the most important tiles are loaded first. Load priority consists of a priority _group_ plus a priority _value_ within that group. The group is chosen according to the reason that this tile is needed by the selection algorithm. The groups are as follows: * `Preload` - Low priority tiles that aren't needed right now, but are being preloaded for the future. * `Normal` - Medium priority tiles that are needed to render the current view at the appropriate level-of-detail. -* `Urgent` - High priority tiles whose absence is are causing extra detail to be rendered in the scene, potentially creating a performance problem and aliasing artifacts. +* `Urgent` - High priority tiles whose absence is causing extra detail to be rendered in the scene, potentially creating a performance problem and aliasing artifacts. Within the group, a tile with a lower priority value will be loaded before a tile with a higher priority value. The priority value is computed as follows: From 79fa57cdb790183624428160010ee41bb9708347 Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Mon, 13 Jan 2025 09:06:35 +1100 Subject: [PATCH 06/11] Add Forbid Holes doc. --- doc/topics/selection-algorithm-details.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index 01e7daf23..041b1dbea 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -186,11 +186,18 @@ Within the group, a tile with a lower priority value will be loaded before a til Where `distance` is the distance from the camera to the closest point on the tile, `tileDirection` is the unit vector from the camera to the center of the tile's bounding volume, and `cameraDirection` is the look direction of of the camera. The idea is that tiles that are near the center of the screen and closer to the camera are loaded first. +## Forbid Holes {#forbid-holes} + +With the default settings, the tile selection algorithm prioritizes getting the needed detail to the screen as quickly as possible. The downside of this approach is that it can sometimes lead to "holes" - or parts of the model that are visibly missing - when the camera moves. This happens when the camera movement reveals a part of the model that wasn't previously visible, and the tiles necessary to show that part of the model are not yet loaded. + +This default behavior can be changed by setting [TilesetOptions::forbidHoles](\ref Cesium3DTilesSelection::TilesetOptions::forbidHoles) to `true`. When holes are forbidden, loading will take longer, because some extra tiles will need to be loaded in order to fill the potential holes, and camera movement may still reveal areas of lower detail, but it will never reveal a part of the model that is missing entirely. This may offer a better user experience for many applications. + +_Forbid Holes_ mode operates via a small change in `_visitTileIfNeeded`. Normally, when a tile is culled, we either don't load it at all, or we load with it with the lower `Preload` [priority](#load-prioritization). But when holes are forbidden, a culled tile is instead loaded with `Normal` priority, and it is also represented in the `TraversalDetails` returned from the method. This means that the subtree containing this tile will be [kicked](#kicking) if this tile is not yet loaded. This guarantees the subtree will not be rendered until this tile is loaded, which in turn guarantees that if the camera is moved, so that this tile suddenly becomes visible, it will be possible to render it immediately. There will not be a hole. + ## Additional Topics Not Yet Covered {#additional-topics} Here are some additional selection algorithm topics that are not yet covered here, but should be in the future: -* Forbid Holes * Unconditionally-Refined Tiles * Occlusion Culling * External Tilesets and Implicit Tiles From bb6e769c1cc80fbf5bc76a53f68e312f08639355 Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Mon, 13 Jan 2025 11:01:49 +1100 Subject: [PATCH 07/11] Add unconditionally-refined tile selction. --- Cesium3DTilesSelection/src/Tileset.cpp | 4 +++- doc/topics/selection-algorithm-details.md | 26 ++++++++++++++++++++++- 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/Cesium3DTilesSelection/src/Tileset.cpp b/Cesium3DTilesSelection/src/Tileset.cpp index 5d4f052ca..d27de5e53 100644 --- a/Cesium3DTilesSelection/src/Tileset.cpp +++ b/Cesium3DTilesSelection/src/Tileset.cpp @@ -1305,11 +1305,13 @@ Tileset::TraversalDetails Tileset::_visitTile( // Determine whether to REFINE or RENDER. Note that even if this tile is // initially marked for RENDER here, it may later switch to REFINE as a // result of `mustContinueRefiningToDeeperTiles`. - VisitTileAction action = VisitTileAction::Render; + VisitTileAction action; if (unconditionallyRefine) action = VisitTileAction::Refine; else if (!meetsSse && !ancestorMeetsSse) action = VisitTileAction::Refine; + else + action = VisitTileAction::Render; const TileSelectionState lastFrameSelectionState = tile.getLastSelectionState(); diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index 041b1dbea..8c07c1197 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -194,11 +194,35 @@ This default behavior can be changed by setting [TilesetOptions::forbidHoles](\r _Forbid Holes_ mode operates via a small change in `_visitTileIfNeeded`. Normally, when a tile is culled, we either don't load it at all, or we load with it with the lower `Preload` [priority](#load-prioritization). But when holes are forbidden, a culled tile is instead loaded with `Normal` priority, and it is also represented in the `TraversalDetails` returned from the method. This means that the subtree containing this tile will be [kicked](#kicking) if this tile is not yet loaded. This guarantees the subtree will not be rendered until this tile is loaded, which in turn guarantees that if the camera is moved, so that this tile suddenly becomes visible, it will be possible to render it immediately. There will not be a hole. +## Unconditionally-Refined Tiles + +A tile that is "unconditionally refined" will always be _REFINED_, it will never be _RENDERED_. We can think of such a tile as having infinite geometric and screen-space error. Unconditionally-refined tiles are used in the following situations: + +1. The `_pRootTile` held by the `TilesetContentManager`. This is root tile of the entire bounding-volume hierarchy. For a standard 3D Tiles `tileset.json` tileset, the root tile defined in the `tileset.json` is the single child of this `_pRootTile`. +2. Tiles whose "content" is an [external tileset](#external-tilesets-and-implicit-tilesets). +3. Tiles that have a geometric error that is higher than their parent's. + +In all three cases, the tile flag is set with a call to [Tile::setUnconditionallyRefine](\ref Cesium3DTilesSelection::Tile::setUnconditionallyRefine). + +Consider a tileset which has a root tile with some renderable content, and four children. Three of the children have normal renderable content as well, but the forth is an external tileset. This means that it provides more nodes for the bounding-volume hierarchy, but it does not have any renderable content itself. Even once that fourth tile is done loading, we can't render it. If we did, we would end up creating a visible hole in the tileset that would not be filled until the children of the tile finished loading as well. + +Normally, a tile, no matter how large its screen-space error, can be rendered in some cases, such as when other tiles are not loaded yet. While we can conceptually think of unconditionally-refined tiles as simply having very large error, the handling of them within the selection algorithm goes a bit deeper, in order to ensure they are never rendered. + +To start, [Tile::isRenderable](\ref Cesium3DTilesSelection::Tile::isRenderable) will return false for an unconditionally-refined tile. This will ensure the tile is [Kicked](#kicking) and thus not rendered. + +> [!note] +> There is one exception where `isRenderable` will return true for an unconditionally-refined tile: when that tile also does not have any children, and never will. It would be an unusual - perhaps buggy! - tileset that has such a situation, but when it does occur, we must allow the tile to render so that its sibilings, if any, may render. + +Also, when the children of an unconditionally-refined tile are kicked out of the render list, those tiles will _not_ also be kicked out of the load queue, even if the [Loading Descendant Limit](#loading-descendant-limit) is exceeded. Because the unconditionally-refined tile will never become renderable, failing to load its children would result in the entire subtree never becoming renderable. + +When [Forbid Holes](#forbid-holes) is enabled, `_visitTileIfNeeded` will always visit unconditionally-refined tiles, even if they're culled. This is necessary because the non-renderable, unconditionally-refined tile would otherwise block renderable siblings from rendering, too. By visiting the unconditionally-refined tile, we allow its children to load, and thereby allow the subtree to become renderable. + +## External Tilesets and Implicit Tiles {#external-tilesets-and-implicit-tilesets} + ## Additional Topics Not Yet Covered {#additional-topics} Here are some additional selection algorithm topics that are not yet covered here, but should be in the future: * Unconditionally-Refined Tiles * Occlusion Culling -* External Tilesets and Implicit Tiles * Additive Refinement From e5ac2f9cc338055f58ede7995b8d7f29748c35bf Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Mon, 13 Jan 2025 12:26:04 +1100 Subject: [PATCH 08/11] Add Implicit Tilesets section. --- doc/topics/selection-algorithm-details.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index 8c07c1197..78d18dc40 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -199,7 +199,7 @@ _Forbid Holes_ mode operates via a small change in `_visitTileIfNeeded`. Normall A tile that is "unconditionally refined" will always be _REFINED_, it will never be _RENDERED_. We can think of such a tile as having infinite geometric and screen-space error. Unconditionally-refined tiles are used in the following situations: 1. The `_pRootTile` held by the `TilesetContentManager`. This is root tile of the entire bounding-volume hierarchy. For a standard 3D Tiles `tileset.json` tileset, the root tile defined in the `tileset.json` is the single child of this `_pRootTile`. -2. Tiles whose "content" is an [external tileset](#external-tilesets-and-implicit-tilesets). +2. Tiles whose "content" is an [external tileset](https://github.com/CesiumGS/3d-tiles/tree/main/specification#core-external-tilesets). 3. Tiles that have a geometric error that is higher than their parent's. In all three cases, the tile flag is set with a call to [Tile::setUnconditionallyRefine](\ref Cesium3DTilesSelection::Tile::setUnconditionallyRefine). @@ -215,14 +215,22 @@ To start, [Tile::isRenderable](\ref Cesium3DTilesSelection::Tile::isRenderable) Also, when the children of an unconditionally-refined tile are kicked out of the render list, those tiles will _not_ also be kicked out of the load queue, even if the [Loading Descendant Limit](#loading-descendant-limit) is exceeded. Because the unconditionally-refined tile will never become renderable, failing to load its children would result in the entire subtree never becoming renderable. -When [Forbid Holes](#forbid-holes) is enabled, `_visitTileIfNeeded` will always visit unconditionally-refined tiles, even if they're culled. This is necessary because the non-renderable, unconditionally-refined tile would otherwise block renderable siblings from rendering, too. By visiting the unconditionally-refined tile, we allow its children to load, and thereby allow the subtree to become renderable. +Finally, when [Forbid Holes](#forbid-holes) is enabled, `_visitTileIfNeeded` will always visit unconditionally-refined tiles, even if they're culled. This is necessary because the non-renderable, unconditionally-refined tile would otherwise block renderable siblings from rendering, too. By visiting the unconditionally-refined tile, we allow its children to load, and thereby allow the subtree to become renderable. -## External Tilesets and Implicit Tiles {#external-tilesets-and-implicit-tilesets} +## Implicit Tilesets {#implicit-tilesets} + +The tile selection algorithm selects tiles from an explicit representation of the bounding-volume hierarchy. Every tile in the tileset is represented as a [Tile](\ref Cesium3DTilesSelection::Tile) instance. Starting with 3D Tiles 1.1, the bounding-volume hierarchy may instead be defined _implicitly_ using [Implicit Tiling](https://github.com/CesiumGS/3d-tiles/tree/main/specification#core-implicit-tiling). This is much more efficient representation when the bounding-volume hierarchy has a uniform subdivision structure. + +> [!note] +> The older `layer.json` / [quantized-mesh-1.0](https://github.com/CesiumGS/quantized-mesh) terrain format also uses a form of implicit tiling. + +Cesium Native supports implicit tiling by lazily transforming the implicit representation into an explicit one as individual tiles are needed. This happens in the `TilesetContentManager::createLatentChildrenIfNecessary` method, called for each tile near the top of `_visitTileIfNeeded`. This method attempts to create explicit tile instances for the implicitly-defined children of the current tile by invoking [TilesetContentLoader::createTileChildren](\ref Cesium3DTilesSelection::TilesetContentLoader::createTileChildren). + +Implicit loaders, such as `ImplicitQuadtreeLoader`, `ImplicitOctreeLoader`, and `LayerJsonTerrainLoader`, implement this method by determining in their own way whether this tile has any children, and creating them if it does. In some cases, extra asynchronous work, like downloading subtree availability files, may be necessary to determine if children exist. In that case, the `createTileChildren` will return [TileLoadResultState::RetryLater](\ref Cesium3DTilesSelection::TileLoadResultState::RetryLater) to signal that children may exist, but they can't be created yet. The selection algorithm will try again next frame if the tile's children are still needed. ## Additional Topics Not Yet Covered {#additional-topics} Here are some additional selection algorithm topics that are not yet covered here, but should be in the future: -* Unconditionally-Refined Tiles * Occlusion Culling * Additive Refinement From abc187d8cd4e738d9fe9b88bf1fcb750db564c83 Mon Sep 17 00:00:00 2001 From: Kevin Ring Date: Mon, 13 Jan 2025 13:39:28 +1100 Subject: [PATCH 09/11] TilesetContentLoader, more implicit loading details. --- doc/topics/selection-algorithm-details.md | 24 +++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index 78d18dc40..e0c1ff47e 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -217,6 +217,21 @@ Also, when the children of an unconditionally-refined tile are kicked out of the Finally, when [Forbid Holes](#forbid-holes) is enabled, `_visitTileIfNeeded` will always visit unconditionally-refined tiles, even if they're culled. This is necessary because the non-renderable, unconditionally-refined tile would otherwise block renderable siblings from rendering, too. By visiting the unconditionally-refined tile, we allow its children to load, and thereby allow the subtree to become renderable. +## TilesetContentLoader {#tileset-content-loader} + +The process of loading content and children for a tile is delegated to a pluggable interface called [TilesetContentLoader](\ref Cesium3DTilesSelection::TilesetContentLoader). This means that the Cesium Native 3D Tiles selection algorithm is not limited to 3D Tiles. Anything that can be portrayed in Cesium Native's 3D Tiles and glTF object model can be loaded, selected, and rendered by Cesium Native. Every `Tile` is associated with a loader, and that loader is used to load that `Tile`'s content. Child `Tiles` may use the same or a different loader. + +The following `TilesetContentLoader` types are currently provided: + +* `TilesetJsonLoader` - The standard loader for explicit 3D Tiles based on `tileset.json`. Individual tile content is loaded via [GltfConverters](\ref Cesium3DTilesContent::GltfConverters). +* `CesiumIonTilesetLoader` - Loads a 3D Tiles asset or `layer.json` / `quantized-mesh-1.0` terrain asset from Cesium ion, by delegating to one of the other loaders as appropriate. Automatically handles refreshing the token when it expires. +* `LayerJsonTerrainLoader` - Loads terrain described by a `layer.json` and individual terrain tiles in `quantized-mesh-1.0` format. +* `ImplicitQuadtreeLoader` - Loads a 3D Tiles 1.1 implicit quadtree. +* `ImplicitOctreeLoader` - Loads a 3D Tiles 1.1 implicit octree. +* `EllipsoidTilesetLoader` - Generates tiles on-the-fly by tessellating an ellipsoid, such as the WGS84 ellipsoid. Does not load any data from the disk or network. + +Other loaders can be added by users of the library. + ## Implicit Tilesets {#implicit-tilesets} The tile selection algorithm selects tiles from an explicit representation of the bounding-volume hierarchy. Every tile in the tileset is represented as a [Tile](\ref Cesium3DTilesSelection::Tile) instance. Starting with 3D Tiles 1.1, the bounding-volume hierarchy may instead be defined _implicitly_ using [Implicit Tiling](https://github.com/CesiumGS/3d-tiles/tree/main/specification#core-implicit-tiling). This is much more efficient representation when the bounding-volume hierarchy has a uniform subdivision structure. @@ -224,9 +239,14 @@ The tile selection algorithm selects tiles from an explicit representation of th > [!note] > The older `layer.json` / [quantized-mesh-1.0](https://github.com/CesiumGS/quantized-mesh) terrain format also uses a form of implicit tiling. -Cesium Native supports implicit tiling by lazily transforming the implicit representation into an explicit one as individual tiles are needed. This happens in the `TilesetContentManager::createLatentChildrenIfNecessary` method, called for each tile near the top of `_visitTileIfNeeded`. This method attempts to create explicit tile instances for the implicitly-defined children of the current tile by invoking [TilesetContentLoader::createTileChildren](\ref Cesium3DTilesSelection::TilesetContentLoader::createTileChildren). +Cesium Native supports implicit tiling by lazily transforming the implicit representation into an explicit one as individual tiles are needed. This happens in the `TilesetContentManager::createLatentChildrenIfNecessary` method, called for each tile near the top of `_visitTileIfNeeded`. This method attempts to create explicit tile instances for the implicitly-defined children of the current tile by invoking [TilesetContentLoader::createTileChildren](\ref Cesium3DTilesSelection::TilesetContentLoader::createTileChildren). Thus, the `TilesetContentLoader` interface is not only responsible for loading tile content, it is also responsible for creating additional `Tile` instances in the bounding-volume hierarchy as needed. + +Implicit [loaders](#tileset-content-loader), such as `ImplicitQuadtreeLoader`, `ImplicitOctreeLoader`, and `LayerJsonTerrainLoader`, implement this method by determining in their own way whether this tile has any children, and creating them if it does. In some cases, extra asynchronous work, like downloading subtree availability files, may be necessary to determine if children exist. In that case, the `createTileChildren` will return [TileLoadResultState::RetryLater](\ref Cesium3DTilesSelection::TileLoadResultState::RetryLater) to signal that children may exist, but they can't be created yet. The selection algorithm will try again next frame if the tile's children are still needed. -Implicit loaders, such as `ImplicitQuadtreeLoader`, `ImplicitOctreeLoader`, and `LayerJsonTerrainLoader`, implement this method by determining in their own way whether this tile has any children, and creating them if it does. In some cases, extra asynchronous work, like downloading subtree availability files, may be necessary to determine if children exist. In that case, the `createTileChildren` will return [TileLoadResultState::RetryLater](\ref Cesium3DTilesSelection::TileLoadResultState::RetryLater) to signal that children may exist, but they can't be created yet. The selection algorithm will try again next frame if the tile's children are still needed. +Currently, a `Tile` instance, once created, will not be destroyed until the entire [Tileset](\ref Cesium3DTilesSelection::Tileset) is destroyed. This is true for `Tile` instances created explicitly from `tileset.json` as well as `Tile` instances created lazily by the implicit loaders. This is convenient because we don't need to worry about a `Tile` instance vanishing unexpectedly, but it can cause a slow increase in memory usage over time. + +> [!note] +> The above refers to `Tile` instances, _not_ their content. Content is unloaded when it is no longer needed. This is important because content is by far the largest portion of a tile. ## Additional Topics Not Yet Covered {#additional-topics} From 5ce9cbb3fe6e4240a06155e7d8da6e60abcfecd7 Mon Sep 17 00:00:00 2001 From: Ashley Rogers Date: Wed, 15 Jan 2025 17:41:09 -0500 Subject: [PATCH 10/11] Copyedit --- doc/topics/selection-algorithm-details.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/topics/selection-algorithm-details.md b/doc/topics/selection-algorithm-details.md index e0c1ff47e..8b913c2d6 100644 --- a/doc/topics/selection-algorithm-details.md +++ b/doc/topics/selection-algorithm-details.md @@ -141,7 +141,7 @@ Once the current tile finishes loading and is rendered, only then will the tiles ## Tile Content Loading -It is important to understand the distinction between a [Tile](@ref Cesium3DTilesSelection::Tile) and its _content_. In the 3D Tiles specification, a [Tile](https://github.com/CesiumGS/3d-tiles/blob/main/specification/PropertiesReference_3dtiles.adoc#tile) is a node in the tileset's bounding-volume hierarchy (BVH), and has a bounding volume, a transform, a geometric error value, and more. Tiles are arranged in a tree, so every `Tile` has a parent `Tile` (except the root), and zero or more children. Tiles also have a `content.uri` property which points to the externally-stored _content_ for the tile. This external content is usually some sort of renderable object, such as a glTF model or a legacy container format like [b3dm](https://github.com/CesiumGS/3d-tiles/blob/main/specification/TileFormats/Batched3DModel/README.adoc#tileformats-batched3dmodel-batched-3d-model). It can also be a further subtree of the BVH rooted at this node, which is known as an [external tileset](https://github.com/CesiumGS/3d-tiles/tree/main/specification#core-external-tilesets). +It is important to understand the distinction between a [Tile](@ref Cesium3DTilesSelection::Tile) and its _content_. In the 3D Tiles specification, a [Tile](https://github.com/CesiumGS/3d-tiles/blob/main/specification/PropertiesReference_3dtiles.adoc#tile) is a node in the tileset's bounding-volume hierarchy (BVH), and has a bounding volume, a transform, a geometric error value, and more. Tiles are arranged in a tree, so every `Tile` has a parent `Tile` (except the root) and zero or more children. Tiles also have a `content.uri` property which points to the externally-stored _content_ for the tile. This external content is usually some sort of renderable object, such as a glTF model or a legacy container format like [b3dm](https://github.com/CesiumGS/3d-tiles/blob/main/specification/TileFormats/Batched3DModel/README.adoc#tileformats-batched3dmodel-batched-3d-model). It can also be a further subtree of the BVH rooted at this node, which is known as an [external tileset](https://github.com/CesiumGS/3d-tiles/tree/main/specification#core-external-tilesets). Because _content_ is by far the largest part of a tile, and the most time-consuming to load, Cesium Native aims to only load it when when it is needed. The loading happens through a small state machine maintained in the `_loadState` property of each tile. The possible load states are captured in the [TileLoadState](@ref Cesium3DTilesSelection::TileLoadState) enumeration. @@ -224,7 +224,7 @@ The process of loading content and children for a tile is delegated to a pluggab The following `TilesetContentLoader` types are currently provided: * `TilesetJsonLoader` - The standard loader for explicit 3D Tiles based on `tileset.json`. Individual tile content is loaded via [GltfConverters](\ref Cesium3DTilesContent::GltfConverters). -* `CesiumIonTilesetLoader` - Loads a 3D Tiles asset or `layer.json` / `quantized-mesh-1.0` terrain asset from Cesium ion, by delegating to one of the other loaders as appropriate. Automatically handles refreshing the token when it expires. +* `CesiumIonTilesetLoader` - Loads a 3D Tiles asset or `layer.json` / `quantized-mesh-1.0` terrain asset from [Cesium ion](https://cesium.com/platform/cesium-ion/), by delegating to one of the other loaders as appropriate. Automatically handles refreshing the token when it expires. * `LayerJsonTerrainLoader` - Loads terrain described by a `layer.json` and individual terrain tiles in `quantized-mesh-1.0` format. * `ImplicitQuadtreeLoader` - Loads a 3D Tiles 1.1 implicit quadtree. * `ImplicitOctreeLoader` - Loads a 3D Tiles 1.1 implicit octree. From 067e1cf7fbad08226f5fd01e90e7bb074ffe2b34 Mon Sep 17 00:00:00 2001 From: Ashley Rogers Date: Thu, 16 Jan 2025 13:57:54 -0500 Subject: [PATCH 11/11] Organize under Developer Resources --- doc/topics/developer.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/doc/topics/developer.md b/doc/topics/developer.md index 48f23d1d3..5db41f388 100644 --- a/doc/topics/developer.md +++ b/doc/topics/developer.md @@ -1,6 +1,13 @@ # Developer Resources {#developer-resources} +## Getting Started + * \subpage developer-setup -* \subpage multithreading * \subpage style-guide -* \subpage contributing \ No newline at end of file +* \subpage contributing + +## Architecture of Cesium Native + +* \subpage multithreading +* \subpage selection-algorithm-details +* \subpage rendering-3d-tiles \ No newline at end of file