Skip to content

Commit

Permalink
Updates to docs for gaussian splatting (#2782)
Browse files Browse the repository at this point in the history
* additions to docs

---------

Co-authored-by: Justin Kerr <[email protected]>
  • Loading branch information
AdamRashid96 and kerrj authored Jan 18, 2024
1 parent 6f2b87e commit a8d5dc9
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions docs/nerfology/methods/splat.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,23 @@
# Gaussian Splatting
<h4>Real-Time Radiance Field Rendering</h4>


```{button-link} https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
:color: primary
:outline:
Paper Website
```

[3D Gaussian Splatting](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) was proposed in SIGGRAPH 2023 from INRIA, and is a completely different method of representing radiance fields by explicitly storing a collection of 3D volumetric gaussians. These can be "splatted", or projected, onto a 2D image provided a camera pose, and rasterized to obtain per-pixel colors. Because rasterization is very fast on GPUs, this method can render much faster than neural representations of radiance fields.

### Installation

```{button-link} https://docs.gsplat.studio/
:color: primary
:outline:
GSplat
```

Nerfstudio uses [gsplat](https://github.com/nerfstudio-project/gsplat) as its gaussian rasterization backend, an in-house re-implementation which is designed to be more developer friendly. This can be installed with `pip install gsplat`. The associated CUDA code will be compiled the first time gaussian splatting is executed. Some users with PyTorch 2.0 have experienced issues with this, which can be resolved by either installing gsplat from source, or upgrading torch to 2.1.

### Data
Expand All @@ -13,6 +29,12 @@ Because gaussian splatting trains on *full images* instead of bundles of rays, t
### Running the Method
To run gaussian splatting, run `ns-train gaussian-splatting --data <data>`. Just like NeRF methods, the splat can be interactively viewed in the web-viewer, loaded from a checkpoint, rendered, and exported.

#### Quality and Regularization
The default settings provided maintain a balance between speed, quality, and splat file size, but if you care more about quality than training speed or size, you can decrease the alpha cull threshold
(threshold to delete translucent gaussians) and disable culling after 15k steps like so: `ns-train gaussian-splatting --pipeline.model.cull_scale_thresh=0.005 --pipeline.model.continue_cull_post_densification=False --data <data>`

A common artifact in splatting is long, spikey gaussians. [PhysGaussian](https://xpandora.github.io/PhysGaussian/) proposes an example of a scale-regularizer that encourages gaussians to be more evenly shaped. To enable this, set the `use_scale_regularization` flag to `True`.

### Details
For more details on the method, see the [original paper](https://arxiv.org/abs/2308.04079). Additionally, for a detailed derivation of the gradients used in the gsplat library, see [here](https://arxiv.org/abs/2312.02121).

Expand Down

0 comments on commit a8d5dc9

Please sign in to comment.