Gaussian splatting¶
Data¶
-
class GaussianDataStandard¶
-
std::vector<Gaussian3D> klartraum::loadGaussiansSpz(const std::string &path, bool flipY = false)¶
Loads and unpacks an SPZ file into Gaussians: linear scales, opacities in [0, 1], rotations as x, y, z, w quaternions.
With
flipYthe scene is mirrored across the Y axis (e.g. for Nerfstudio exports).- Throws:
std::runtime_error – if the file cannot be read.
-
struct Gaussian3D¶
One Gaussian of a scene, with the same fields as
spz::UnpackedGaussian.
-
struct GaussianSoABuffers¶
GPU storage of a 3D Gaussian model as a structure of arrays, as both splatting backends read it.
Seven buffers (position vec3, rotation vec4 as x, y, z, w, linear scale vec3, colour and alpha vec4, and the three spherical-harmonics streams of 15 floats per Gaussian, coefficient-major:
sh[b * count + i]) plus the number of Gaussians. It only bundles handles and loads nothing: the buffers may be static uploads (e.g. GaussianDataStandard) or written every frame by compute elements (e.g. GaussianTransform), so a backend can be connected to buffers from any source.
-
struct BufferRef¶
A buffer as a consumer connects to it.
Either
elementitself (slot -1) or the buffer at inputslotofelement, e.g. one that a compute element writes. In the second case the consumer depends on that element, so it runs first.
Backends¶
klartraum::createGaussianSplatting() builds either backend from the
same inputs.
-
namespace klartraum¶
Enums
-
enum class GsplatBackend¶
Selects the Gaussian splatting backend that createGaussianSplatting() builds.
Values:
-
enumerator Compute¶
-
enumerator Raster¶
-
enumerator Compute¶
Functions
Creates a Gaussian splatting backend that renders
modelintoimageViewSrc.Both backends take the same inputs and produce a single image-producing ComputeGraphElement that KlartraumEngine::add() accepts, so switching
backenddoes not change the surrounding graph.The caller owns the GaussianDataStandard (e.g.
std::make_shared<GaussianDataStandard>(vulkanContext, path)); the same instance can be passed to several backends or viewports, so the model is loaded only once.
Creates a Gaussian splatting backend for Gaussians from any source, e.g. the outputs of a GaussianTransform or GaussianMerge.
The backend then depends on the elements that produce the buffers.
-
enum class GsplatBackend¶
-
class VulkanGaussianSplatting : public virtual klartraum::RenderGraphElement, public virtual klartraum::ComputeGraphGroup¶
-
class VulkanGaussianSplattingRaster : public klartraum::RenderGraphElement, public klartraum::ComputeGraphGroup¶
Gaussian splatting backend that sorts the Gaussians once and draws them with hardware rasterization.
The counterpart to VulkanGaussianSplatting, which rasterizes in compute shaders. It reads the same Gaussian data and connects:
GaussianDist: culling, depth keys and compaction (
gsplat_dist.comp);a radix sort of the depth keys;
BufferToGraphicsBarrier, which orders the indirect draw after the compute work;
a RenderPass with a GaussianSplatRasterizer (or, where mesh shaders are supported, a GaussianSplatMeshRasterizer) that draws instanced quads with premultiplied “over” blending.
The internal RenderPass is the group’s output element: the single image-producing element through which the compute graph reaches all others.
-
struct GsplatConfig¶
Draw components of the raster backend¶
-
class GaussianSplatRasterizer : public klartraum::DrawComponent¶
Draw component of the raster Gaussian splatting backend (vertex and fragment shader path).
It binds
set 0: the camera uniform buffer;
set 1: the per-splat storage buffers and the sorted index buffer, in the order passed to the constructor (binding = vector index);
generates the quad corners from
gl_VertexIndex(no vertex buffer), and issues onevkCmdDrawIndirectthat reads the instance count from the draw-arguments buffer. That buffer is filled by the compute culling and sorting, which BufferToGraphicsBarrier orders before this render pass. Splats are blended with premultiplied “over”, so they composite correctly back to front.
-
class GaussianSplatMeshRasterizer : public klartraum::DrawComponent¶
Mesh-shader draw component of the raster Gaussian splatting backend, an alternative to GaussianSplatRasterizer’s vertex shader path.
One mesh workgroup emits a batch of quads from the precomputed 2D splat records, read through the sorted-index permutation, and from the visible count (so the last, partial workgroup emits the right number). The backend uses it only when VK_EXT_mesh_shader is supported, otherwise the vertex path. Its outputs match the vertex shader’s, so the same fragment shader (
gsplat_raster.frag) is used. It draws withvkCmdDrawMeshTasksIndirectEXT;gsplat_mesh_args.compcomputes the group count from the visible count.
Operations on Gaussians¶
-
class GaussianTransform : public klartraum::GeneralComputation<GaussianTransformPushConstants>¶
Writes transformed copies of Gaussians every time the graph runs.
Positions, orientations and sizes move by a transform buffer (see TransformBuffer), SH coefficients rotate with the scene (shaders/gaussians/transform_gaussians.comp). The output is per-path, so a frame in flight keeps its Gaussians while the next one is written.
-
inline GaussianTransformResult klartraum::createGaussianTransform(VulkanContext &vulkanContext, const GaussianSoABuffers &source, const BufferRef &transform)¶
Creates a GaussianTransform that applies
transformto the Gaussians insource.
-
struct GaussianTransformResult¶
Result of createGaussianTransform(): the element and the Gaussians it outputs.
-
class GaussianMerge : public klartraum::GeneralComputation<GaussianMergePushConstants>¶
Concatenates two sets of Gaussians every time the graph runs.
A’s Gaussians come first, so one backend renders and depth-sorts both (shaders/gaussians/merge_gaussians.comp). The output is per-path.
-
inline GaussianMergeResult klartraum::createGaussianMerge(VulkanContext &vulkanContext, const GaussianSoABuffers &a, const GaussianSoABuffers &b)¶
Creates a GaussianMerge that concatenates the Gaussians of
aandb; those ofacome first.
-
struct GaussianMergeResult¶
Result of createGaussianMerge(): the element and the merged Gaussians it outputs.
-
class TransformBuffer : public klartraum::GeneralComputation<void>¶
Builds a transform buffer for GaussianTransform from seven scalars.
Reads translation x, y, z, rotation pitch, yaw, roll in degrees (about X, then Y, then Z) and a uniform scale, each from the first float of its buffer, e.g. a HostFloat the CPU animates. Writes translation, scale, the rotation quaternion and the rotation matrices of the three SH bands (shaders/gaussians/make_transform.comp). One invocation per run.
-
inline TransformBufferResult klartraum::createTransformBuffer(VulkanContext &vulkanContext, const std::array<BufferRef, 7> ¶meters)¶
Creates a TransformBuffer from seven parameters in the order x, y, z, pitch, yaw, roll, scale.
-
struct TransformBufferResult¶
Result of createTransformBuffer(): the element and the transform buffer it writes.