Setting up pipeline barriers

In Vulkan, commands may be reordered when a command buffer is being processed, subject to certain restrictions. This is known as command buffer reordering, and it can help to improve performance by allowing the driver to optimize the order in which commands are executed.

The good news is that Vulkan provides a mechanism called pipeline barriers to ensure that dependent commands are executed in the correct order. They are used to explicitly specify dependencies between commands, preventing them from being reordered, and at what stages they might overlap. This recipe will explain what pipeline barriers are and what their properties mean. It will also show you how to create and install pipeline barriers.

Getting ready

Consider two draw calls issued in sequence. The first one writes to a color attachment, while the second draw call samples from that attachment in the fragment shader:

vkCmdDraw(...); // draws into color attachment 0
vkCmdDraw(...); // reads from color attachment 0

Figure 2.8 helps visualize how those two commands may be processed by the device. In the diagram, commands are processed from top to bottom and progress on the pipeline from left to right. Clock cycles are a loose term, because processing may take multiple clock cycles, but are used to indicate that – in general – some tasks must happen after others.

In the example, the second vkCmdDraw call starts executing at C2, after the first draw call. This offset is not enough, as the second draw call needs to read the color attachment at the Fragment Shader stage, which is not produced by the first draw call until it reaches the Color Attach Output stage. Without synchronization, this setup may cause data races:

Figure 2.8 – Two consecutive commands recorded on the same command buffer being processed without synchronization

A pipeline barrier is a feature that is recorded into the command buffer and that specifies the pipeline stages that need to have been completed for all commands that appear before the barrier and before the command buffer continues processing. Commands recorded before the barrier are said to be in the first synchronization scope or first scope. Commands recorded after the barrier are said to be part of the second synchronization scope or second scope.

The barrier also allows fine-grained control to specify at which stage commands after the barrier must wait until commands in the first scope finish processing. That’s because commands in the second scope don’t need to wait until commands in the first scope are done. They can start processing as soon as possible, as long as the conditions specified in the barrier are met.

In the example in Figure 2.8, the first draw call, in the first scope, needs to write to the attachment before the second draw call can access it. The second draw call does not need to wait until the first draw call finishes processing the Color Attach Output stage. It can start right away, as long as its fragment stage happens after the first draw call is done with its Color Attach Output stage, as shown in Figure 2.9:

Figure 2.9 – Two consecutive commands recorded on the same command buffer being processed with synchronization

There are three types of barriers:

1. Memory barriers are global barriers and apply to all commands in the first and second scopes.
2. Buffer memory barriers are barriers that apply only to commands that access a portion of the buffer, as it’s possible to specify to which portion of the buffer the barrier applies (offset + range).
3. Image memory barriers are barriers that apply only to commands that access a subresource of an image. It’s possible to add barriers based on mip level, sections of the image, or array layers. This is an especially important barrier as it is also used to transition an image from one layout to another. For instance, while generating mipmaps and blitting from one mip level to the next, the levels need to be in the correct layout. The previous level needs to be in the VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL layout, as it will be read from, while the next mip level needs to be in the VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL layout, as it will be written to.

How to do it…

Pipeline barriers are recorded with the vkCmdPipelineBarrier command, in which you can provide several barriers of multiple types at the same time. The following code snippet shows how to create a barrier used to create a dependency between the two draw calls in Figure 2.9:

VkCommandBuffer commandBuffer;  // Valid Command Buffer
VkImage image;                  // Valid image
const VkImageSubresourceRange subresource = {
    .aspectMask =.baseMipLevel = 0,
    .levelCount = VK_REMAINING_MIP_LEVELS,
    .baseArrayLayer = 0,
    .layerCount = 1,
};
const VkImageMemoryBarrier imageBarrier = {
    .sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
    .srcAccessMask =
        VK_ACCESS_2_COLOR_ATTACHMENT_WRITE_BIT_KHR,
    .dstAccessMask = VK_ACCESS_2_SHADER_READ_BIT_KHR,
    .oldLayout = VK_IMAGE_LAYOUT_ATTACHMENT_OPTIMAL,
    .newLayout = VK_IMAGE_LAYOUT_READ_ONLY_OPTIMAL,
    .srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
    .dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
    .image = image,
    .subresourceRange = &subresource,
};
vkCmdPipelineBarrier(
    commandBuffer,
    VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
    VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT, 0, 0,
    nullptr, 0, nullptr, 1, &memoryBarrier);

The barrier needs to be recorded between the two draw calls:

vkCmdDraw(...); // draws into color attachment 0
vkCmdPipelineBarrier(...);
vkCmdDraw(...); // reads from color attachment 0

Pipeline barriers are tricky but absolutely fundamental in Vulkan. Make sure you understand what they offer and how they operate before continuing to read the other recipes.