Gaussian splatting

Data

class GaussianDataStandard
std::vector<Gaussian3D> klartraum::loadGaussiansSpz(const std::string &path, bool flipY = false)

Loads and unpacks an SPZ file into Gaussians: linear scales, opacities in [0, 1], rotations as x, y, z, w quaternions.

With flipY the scene is mirrored across the Y axis (e.g. for Nerfstudio exports).

Throws:

std::runtime_error – if the file cannot be read.

struct Gaussian3D

One Gaussian of a scene, with the same fields as spz::UnpackedGaussian.

struct GaussianSoABuffers

GPU storage of a 3D Gaussian model as a structure of arrays, as both splatting backends read it.

Seven buffers (position vec3, rotation vec4 as x, y, z, w, linear scale vec3, colour and alpha vec4, and the three spherical-harmonics streams of 15 floats per Gaussian, coefficient-major: sh[b * count + i]) plus the number of Gaussians. It only bundles handles and loads nothing: the buffers may be static uploads (e.g. GaussianDataStandard) or written every frame by compute elements (e.g. GaussianTransform), so a backend can be connected to buffers from any source.

struct BufferRef

A buffer as a consumer connects to it.

Either element itself (slot -1) or the buffer at input slot of element, e.g. one that a compute element writes. In the second case the consumer depends on that element, so it runs first.

Backends

klartraum::createGaussianSplatting() builds either backend from the same inputs.

namespace klartraum

Enums

enum class GsplatBackend

Selects the Gaussian splatting backend that createGaussianSplatting() builds.

Values:

enumerator Compute
enumerator Raster

Functions

std::shared_ptr<ComputeGraphElement> createGaussianSplatting(VulkanContext &vulkanContext, GsplatBackend backend, std::shared_ptr<ImageViewSrc> imageViewSrc, std::shared_ptr<CameraUboType> cameraUBO, std::shared_ptr<GaussianDataStandard> model, GsplatConfig config = GsplatConfig{})

Creates a Gaussian splatting backend that renders model into imageViewSrc.

Both backends take the same inputs and produce a single image-producing ComputeGraphElement that KlartraumEngine::add() accepts, so switching backend does not change the surrounding graph.

The caller owns the GaussianDataStandard (e.g. std::make_shared<GaussianDataStandard>(vulkanContext, path)); the same instance can be passed to several backends or viewports, so the model is loaded only once.

std::shared_ptr<ComputeGraphElement> createGaussianSplatting(VulkanContext &vulkanContext, GsplatBackend backend, std::shared_ptr<ImageViewSrc> imageViewSrc, std::shared_ptr<CameraUboType> cameraUBO, const GaussianSoABuffers &buffers, GsplatConfig config = GsplatConfig{})

Creates a Gaussian splatting backend for Gaussians from any source, e.g. the outputs of a GaussianTransform or GaussianMerge.

The backend then depends on the elements that produce the buffers.

class VulkanGaussianSplatting : public virtual klartraum::RenderGraphElement, public virtual klartraum::ComputeGraphGroup
class VulkanGaussianSplattingRaster : public klartraum::RenderGraphElement, public klartraum::ComputeGraphGroup

Gaussian splatting backend that sorts the Gaussians once and draws them with hardware rasterization.

The counterpart to VulkanGaussianSplatting, which rasterizes in compute shaders. It reads the same Gaussian data and connects:

The internal RenderPass is the group’s output element: the single image-producing element through which the compute graph reaches all others.

struct GsplatConfig

Draw components of the raster backend

class GaussianSplatRasterizer : public klartraum::DrawComponent

Draw component of the raster Gaussian splatting backend (vertex and fragment shader path).

It binds

  • set 0: the camera uniform buffer;

  • set 1: the per-splat storage buffers and the sorted index buffer, in the order passed to the constructor (binding = vector index);

generates the quad corners from gl_VertexIndex (no vertex buffer), and issues one vkCmdDrawIndirect that reads the instance count from the draw-arguments buffer. That buffer is filled by the compute culling and sorting, which BufferToGraphicsBarrier orders before this render pass. Splats are blended with premultiplied “over”, so they composite correctly back to front.

class GaussianSplatMeshRasterizer : public klartraum::DrawComponent

Mesh-shader draw component of the raster Gaussian splatting backend, an alternative to GaussianSplatRasterizer’s vertex shader path.

One mesh workgroup emits a batch of quads from the precomputed 2D splat records, read through the sorted-index permutation, and from the visible count (so the last, partial workgroup emits the right number). The backend uses it only when VK_EXT_mesh_shader is supported, otherwise the vertex path. Its outputs match the vertex shader’s, so the same fragment shader (gsplat_raster.frag) is used. It draws with vkCmdDrawMeshTasksIndirectEXT; gsplat_mesh_args.comp computes the group count from the visible count.

Operations on Gaussians

class GaussianTransform : public klartraum::GeneralComputation<GaussianTransformPushConstants>

Writes transformed copies of Gaussians every time the graph runs.

Positions, orientations and sizes move by a transform buffer (see TransformBuffer), SH coefficients rotate with the scene (shaders/gaussians/transform_gaussians.comp). The output is per-path, so a frame in flight keeps its Gaussians while the next one is written.

inline GaussianTransformResult klartraum::createGaussianTransform(VulkanContext &vulkanContext, const GaussianSoABuffers &source, const BufferRef &transform)

Creates a GaussianTransform that applies transform to the Gaussians in source.

struct GaussianTransformResult

Result of createGaussianTransform(): the element and the Gaussians it outputs.

class GaussianMerge : public klartraum::GeneralComputation<GaussianMergePushConstants>

Concatenates two sets of Gaussians every time the graph runs.

A’s Gaussians come first, so one backend renders and depth-sorts both (shaders/gaussians/merge_gaussians.comp). The output is per-path.

inline GaussianMergeResult klartraum::createGaussianMerge(VulkanContext &vulkanContext, const GaussianSoABuffers &a, const GaussianSoABuffers &b)

Creates a GaussianMerge that concatenates the Gaussians of a and b; those of a come first.

struct GaussianMergeResult

Result of createGaussianMerge(): the element and the merged Gaussians it outputs.

class TransformBuffer : public klartraum::GeneralComputation<void>

Builds a transform buffer for GaussianTransform from seven scalars.

Reads translation x, y, z, rotation pitch, yaw, roll in degrees (about X, then Y, then Z) and a uniform scale, each from the first float of its buffer, e.g. a HostFloat the CPU animates. Writes translation, scale, the rotation quaternion and the rotation matrices of the three SH bands (shaders/gaussians/make_transform.comp). One invocation per run.

inline TransformBufferResult klartraum::createTransformBuffer(VulkanContext &vulkanContext, const std::array<BufferRef, 7> &parameters)

Creates a TransformBuffer from seven parameters in the order x, y, z, pitch, yaw, roll, scale.

struct TransformBufferResult

Result of createTransformBuffer(): the element and the transform buffer it writes.