Neural Maps: NeRF and 3D Gaussian Splatting

Neural maps rebuild a scene from photos so well that new views look real. NeRF stores the scene as a function of color and density at every point and renders by marching along each camera ray. 3D Gaussian splatting stores it as millions of small colored blobs and renders by drawing them in depth order, which is much faster.

Method Camera position Number of Gaussians (splatting) Samples per ray (NeRF)

Volume rendering

NeRF renders each pixel by walking along its ray, adding the color $\mathbf{c}_i$ of each sample weighted by its density $\sigma_i$ and by how much light still gets through, the transmittance $T_i$:

$$C = \sum_i T_i\,\big(1 - e^{-\sigma_i \delta_i}\big)\,\mathbf{c}_i, \qquad T_i = \exp\!\Big(-\sum_{j

Gaussian splatting uses the same blending formula, but instead of sampling a field along each ray, it sorts the Gaussians by depth and blends each one's opacity $\alpha_i$ directly:

$$C = \sum_i \mathbf{c}_i\,\alpha_i \prod_{j

What to notice

This page works in 2D, rendering a one-row image of a flat scene. NeRF with few samples per ray misses thin objects and looks blotchy; more samples cost more time per pixel. Splatting with few Gaussians looks blobby, but each frame only costs one sort and one pass over the blobs, which is why it renders in real time. In robotics, both are used for realistic simulation, mapping, and as dense maps that a robot can query for what it would see.