Compute graph

A directed acyclic graph of elements. Each element records its own Vulkan command buffers; the graph connects them with semaphores.

Graph

class ComputeGraph
class ComputeGraphElement

Abstract base class representing an element in a compute graph.

ComputeGraphElement serves as the foundation for building computational graphs in the Klartraum framework. Each element can have multiple inputs and outputs, forming a directed acyclic graph structure for computation pipelines.

See also

ComputeGraph which is used to compile a compute graph from these elements.

Note

This class is designed to work with shared_ptr (ComputeGraphElementPtr) for automatic memory management and safe reference handling. Generally, you should not create instances of this class directly, but rather derive from it to implement specific compute graph elements and use the VulkanContext::create to instantiate them.

Subclassed by klartraum::GeneralComputation< GaussianMergePushConstants >, klartraum::GeneralComputation< GaussianTransformPushConstants >, klartraum::GeneralComputation< ImageResamplePushConstants >, klartraum::GeneralComputation< void >, klartraum::BufferElementInterface, klartraum::BufferToGraphicsBarrier, klartraum::ComputeGraphGroup, klartraum::CopyBuffer, klartraum::GeneralComputation< P >, klartraum::ImageSrc, klartraum::ImageViewSrcInterface, klartraum::ImageViewSrcTransition, klartraum::NoOp, klartraum::OnnxNetwork, klartraum::RenderGraphElement, klartraum::UniformBufferObjectInterface

Public Functions

template<int index = 0>
inline void setInput(ComputeGraphElementPtr input, int slot = -1)

Sets the input element for this ComputeGraphElement at the specified index.

If slot is used, this indicates that the input is not given by input directly, but rather by the slot element of the given element.

If slot is specified, retrieves the input element from the provided input’s slot and checks its validity. Updates the internal input and source output slot mappings.

Template Parameters:

index – The index at which to set the input (default is 0).

Parameters:
  • input – The ComputeGraphElementPtr to set as input.

  • slot – The slot of the input element to use (default is -1, meaning no slot). See explanation of slots above.

inline ComputeGraphElementPtr getOutputElement(int index = 0)

get the output with specified index

Generally simply returns the input at the given index. In the klartraum computegraph framework, each input is passed through the node, regardless of whether it is modified or not.

Parameters:

index –

Returns:

ComputeGraphElementPtr

Groups

Elements that are built from several graph elements.

class ComputeGraphGroup : public virtual klartraum::ComputeGraphElement

Base class for elements built from several graph elements, such as OnnxNetwork or the Gaussian splatting backends.

getInputs() returns the group’s output elements, so compiling a graph traverses from the group through its output elements back to all elements of the group.

Subclassed by klartraum::OnnxNetwork, klartraum::VulkanGaussianSplatting, klartraum::VulkanGaussianSplattingRaster

class RenderGraphElement : public virtual klartraum::ComputeGraphElement

Base class for graph elements that render with a camera; getCameraUBO() returns their camera uniform buffer input.

Subclassed by klartraum::RenderPass, klartraum::VulkanGaussianSplatting, klartraum::VulkanGaussianSplattingRaster

Buffers and tensors

class BufferElementInterface : public klartraum::ComputeGraphElement

Subclassed by klartraum::TemplatedBufferElementInterface< R >, klartraum::TemplatedBufferElementInterface< VulkanBuffer< T > >, klartraum::TemplatedBufferElementInterface< BufferType >, klartraum::TensorElementInterface

template<typename BufferType>
class TemplatedBufferElementInterface : public klartraum::BufferElementInterface

Buffer element whose buffers have the type BufferType; getBuffer() returns the buffer of one path.

Subclassed by klartraum::BufferElement< VulkanBuffer< T > >, klartraum::BufferElement< BufferType >, klartraum::BufferElementSinglePath< BufferType >

template<typename BufferType>
class BufferElement : public klartraum::TemplatedBufferElementInterface<BufferType>
template<typename BufferType>
class BufferElementSinglePath : public klartraum::TemplatedBufferElementInterface<BufferType>

Buffer element with one buffer for all paths.

The constructor arguments are passed on to the buffer; getBuffer() returns the same buffer for every path.

using klartraum::DispatchIndirectCommandBufferElement = BufferElement<VulkanBuffer<VkDispatchIndirectCommand>>

Buffer element holding the arguments of an indirect compute dispatch (vkCmdDispatchIndirect).

using klartraum::DrawIndirectCommandBufferElement = BufferElement<VulkanBuffer<VkDrawIndirectCommand>>

Buffer element holding the arguments of an indirect draw (vkCmdDrawIndirect).

template<typename T>
class HostValues : public klartraum::BufferElement<VulkanBuffer<T>>

A few values the CPU sets and shaders read, e.g. animation values.

set() only stores the values on the CPU. Right before a path of the graph is submitted, _update() copies them into that path’s storage buffer, so a value set between frames is seen by the next frame and paths still in flight keep theirs. Shaders read it as an array of T.

class TensorElementInterface : public klartraum::BufferElementInterface

Untemplated interface for tensor elements.

Subclassed by klartraum::TensorElement< DataType >

Public Functions

virtual const std::vector<uint32_t> &getDimensions() const = 0

Get tensor dimensions.

virtual void setDimensions(const std::vector<uint32_t> &newDimensions) = 0

Set the dimension data and update the dimensions buffer.

template<typename DataType>
class TensorElement : public klartraum::TensorElementInterface

TensorElement represents a tensor with separate dimension and data buffers.

This class manages GPU tensors with:

  • A dimensions buffer containing 4 uint32_t values [width, height, depth, batch]

  • A data buffer containing the actual tensor data of the specified type

The class follows the ComputeGraphElement pattern and manages multiple paths for different rendering/compute contexts.

Subclassed by klartraum::TensorElementSinglePath< DataType >

Public Functions

inline TensorElement(VulkanContext &vulkanContext, uint32_t batch, uint32_t depth, uint32_t height, uint32_t width, VkBufferUsageFlags dataUsageFlags = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT, VkBufferUsageFlags dimUsageFlags = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT)

Construct a TensorElement with specified dimensions.

Parameters:
  • vulkanContext – The Vulkan context for buffer creation

  • batch – Tensor batch dimension

  • depth – Tensor depth dimension

  • height – Tensor height dimension

  • width – Tensor width dimension

  • dataUsageFlags – Vulkan usage flags for the data buffer

  • dimUsageFlags – Vulkan usage flags for the dimensions buffer

inline TensorElement(VulkanContext &vulkanContext, const std::vector<uint32_t> &dimensions, VkBufferUsageFlags dataUsageFlags = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT, VkBufferUsageFlags dimUsageFlags = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT)

Construct a TensorElement with size of a dimensions vector.

Parameters:
  • vulkanContext – The Vulkan context for buffer creation

  • dimensions – Vector containing [width, height, depth, batch]

  • dataUsageFlags – Vulkan usage flags for the data buffer

  • dimUsageFlags – Vulkan usage flags for the dimensions buffer

inline virtual VulkanBuffer<DataType> &getDataBuffer(uint32_t pathId)

Get the data buffer for a specific path.

inline virtual VulkanBuffer<uint32_t> &getDimensionsBuffer()

Get the dimensions buffer.

inline virtual VkBuffer &getDataVkBuffer(uint32_t pathId) override

Get the Vulkan buffer handle for data buffer.

inline virtual VkBuffer &getDimensionsVkBuffer() override

Get the Vulkan buffer handle for dimensions buffer.

inline virtual const std::vector<uint32_t> &getDimensions() const override

Get tensor dimensions.

inline virtual void setDimensions(const std::vector<uint32_t> &newDimensions) override

Set the dimension data and update the dimensions buffer.

inline uint32_t getDataElementCount() const

Get number of data elements.

inline size_t getDataBufferMemSize() const

Get total memory size for data buffers.

inline size_t getDimensionsBufferMemSize() const

Get total memory size for dimensions buffers.

inline void setData(uint32_t pathId, const std::vector<DataType> &data)

Copy data to the data buffer for a specific compute path.

inline void setRecordToZero(bool recordDataToZero, bool recordDimensionsToZero = false)

Set whether to zero buffers during recording.

template<typename DataType>
class TensorElementSinglePath : public klartraum::TensorElement<DataType>

TensorElementSinglePath represents a tensor with a single buffer.

This class is designed for constant/fixed tensors (weights, biases, initializers) that don’t change during execution and therefore only need a single buffer instead of multiple buffers for different paths.

Public Functions

inline TensorElementSinglePath(VulkanContext &vulkanContext, uint32_t batch, uint32_t depth, uint32_t height, uint32_t width, VkBufferUsageFlags dataUsageFlags = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT, VkBufferUsageFlags dimUsageFlags = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT)

Construct a TensorElementSinglePath with specified dimensions.

Parameters:
  • vulkanContext – The Vulkan context for buffer creation

  • batch – Tensor batch dimension

  • depth – Tensor depth dimension

  • height – Tensor height dimension

  • width – Tensor width dimension

  • dataUsageFlags – Vulkan usage flags for the data buffer

  • dimUsageFlags – Vulkan usage flags for the dimensions buffer

inline TensorElementSinglePath(VulkanContext &vulkanContext, const std::vector<uint32_t> &dimensions, VkBufferUsageFlags dataUsageFlags = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT, VkBufferUsageFlags dimUsageFlags = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT)

Construct a TensorElementSinglePath with specified dimensions.

Parameters:
  • vulkanContext – The Vulkan context for buffer creation

  • dimensions – Vector containing tensor dimensions

  • dataUsageFlags – Vulkan usage flags for the data buffer

  • dimUsageFlags – Vulkan usage flags for the dimensions buffer

inline virtual VulkanBuffer<DataType> &getDataBuffer(uint32_t pathId = -1) override

Get the data buffer for a specific path.

inline virtual VkBuffer &getDataVkBuffer(uint32_t pathId = -1) override

Get the Vulkan buffer handle for data buffer.

template<typename UniformBufferObjectType>
class UniformBufferObject : public klartraum::UniformBufferObjectInterface

Computations

template<typename P = void>
class GeneralComputation : public klartraum::ComputeGraphElement
template<typename A, typename R, typename U = void, typename P = void>
class BufferTransformation : public klartraum::TemplatedBufferElementInterface<R>
class CopyBuffer : public klartraum::ComputeGraphElement

CopyBuffer element that copies data from source buffer to destination buffer.

This element performs a direct buffer copy operation using vkCmdCopyBuffer. Its main purpose is to copy a buffer so that consecutive operations on the data do not conflict between different operations.

Public Functions

inline CopyBuffer(VulkanContext &vulkanContext)

Construct a new CopyBuffer element.

Parameters:

vulkanContext – The Vulkan context for this element

class ImageResample : public klartraum::GeneralComputation<ImageResamplePushConstants>

Resamples an image into another image of any size.

Input 0 is the source image, input 1 the destination; both are ImageViewSrc elements (or elements passing one through a slot) whose format stores as rgba8, such as OffscreenTarget or the swapchain. The whole destination is written, so its previous contents are discarded.

The source is read in VK_IMAGE_LAYOUT_GENERAL, transitioned from srcLayout (e.g. TRANSFER_SRC_OPTIMAL for an OffscreenTarget a splatting backend rendered into). Both images are left in GENERAL.

enum class klartraum::ResampleFilter : uint32_t

Filter that ImageResample uses to sample the source image.

Values:

enumerator Nearest
enumerator Bilinear

Rendering

class RenderPass : public klartraum::ImageViewSrc, public klartraum::RenderGraphElement
class ImageViewSrc : public virtual klartraum::ImageViewSrcInterface

Subclassed by klartraum::OffscreenTarget, klartraum::RenderPass

Synchronization and placeholders

class ImageViewSrcTransition : public klartraum::ComputeGraphElement

A compute graph element that performs image layout transitions for Vulkan images.

This class handles the transition of image layouts using VkImageMemoryBarrier. It’s designed to work with ImageViewSrc elements in a compute graph and records the necessary commands to transition an image from one layout to another.

Default transition is from VK_IMAGE_LAYOUT_GENERAL to VK_IMAGE_LAYOUT_PRESENT_SRC_KHR.

class BufferToGraphicsBarrier : public klartraum::ComputeGraphElement

Orders an indirect draw after the compute work that wrote its buffers.

The graph only orders compute work after compute work (GeneralComputation emits compute-to-compute memory barriers between its own dispatches). An indirect draw that reads buffers written by compute shaders (sorted indices, indirect draw arguments) needs a buffer barrier whose destination is the indirect-draw and vertex-shader stages of the graphics pipeline. This element supplies it: placed between the compute elements that write the buffers and the RenderPass that draws, the graph schedules it in between.

class NoOp : public klartraum::ComputeGraphElement

NoOp (No Operation) element that passes inputs through unchanged.

This element performs no computation and simply passes its inputs through to its outputs. It’s useful for representing operations like ONNX Constant nodes that provide data without computation, or for debugging/testing purposes.

Public Functions

inline NoOp(VulkanContext &vulkanContext)

Construct a new NoOp element.

Parameters:

vulkanContext – The Vulkan context for this element