For this chapter, you will need to make sure you have VS 2022 installed along with the Vulkan SDK. Basic familiarity with the C++ programming language and an understanding of OpenGL or any other graphics API will be useful. Please revisit Chapter 1, Vulkan Core Concepts, under the Technical requirements section for details on setting up and building executables for this chapter. The recipe for this chapter can be run by launching Chapter02_MultiDrawIndirect.exe executable.

Memory allocation and management are crucial in Vulkan, as almost none of the details of memory usage are managed by Vulkan. Except for deciding the exact memory address where memory should be allocated, all other details are the responsibility of the application. This means the programmer must manage memory types, their sizes, and alignments, as well as any sub-allocations. This approach gives applications more control over memory management and allows developers to optimize their programs for specific uses. This recipe will provide some fundamental information about the types of memory provided by the API as well as a summary of how to allocate and bind that memory to resources.

Getting ready

Graphics cards come in two variants, integrated and discrete. Integrated graphics cards share the same memory as the CPU, as shown in Figure 2.1:

Figure 2.1 – Typical memory architecture for discrete graphics cards

Discrete graphics cards have their own memory (device memory) separate from the main memory (host memory), as shown in Figure 2.2:

Figure 2.2 – Typical memory architecture for integrated graphics cards

Vulkan provides different types of memory:

• Device-local memory: This type of memory is optimized for use by the GPU and is local to the device. It is typically faster than host-visible memory but is not accessible from the CPU. Usually, resources such as render targets, storage images, and buffers are stored in this memory.
• Host-visible memory: This type of memory is accessible from both the GPU and the CPU. It is typically slower than device-local memory but allows for efficient data transfer between the GPU and CPU. Reads from GPU to CPU happen across Peripheral Component Interconnect Express (PCI-E) lanes in the case of non-integrated GPU. It’s typically used to set up staging buffers, where data is stored before being transferred to device-local memory, and uniform buffers, which are constantly updated from the application.
• Host-coherent memory: This type of memory is like host-visible memory but provides guaranteed memory consistency between the GPU and CPU. This type of memory is typically slower than both device-local and host-visible memory but is useful for storing data that needs to be frequently updated by both the GPU and CPU.

Figure 2.3 summarizes the three aforementioned types of memory. Device-local memory is not visible from the host, while host-coherent and host-visible are. Copying data from the CPU to the GPU can be done using mapped memory for those two types of memory allocations. For device-local memory, it’s necessary to copy the data from the CPU to host-visible memory first using mapped memory (the staging buffer), and then perform a copy of the data from the staging buffer to the destination, the device-local memory, using a Vulkan function:

Figure 2.3 – Types of memory and their visibility from the application in Vulkan

Images are usually device-local memory, as they have their own layout that isn’t readily interpretable by the application. Buffers can be of any one of the aforementioned types.

How to do it…

A typical workflow for creating and uploading data to a buffer includes the following steps:

1. Create a buffer object of type VkBuffer by using the VkBufferCreateInfo structure and calling vkCreateBuffer.
2. Retrieve the memory requirements based on the buffer’s properties by calling vkGetBufferMemoryRequirements. The device may require a certain alignment, which could affect the necessary size of the allocation to accommodate the buffer’s contents.
3. Create a structure of type VkMemoryAllocateInfo, specify the size of the allocation and the type of memory, and call vkAllocateMemory.
4. Call vkBindBufferMemory to bind the allocation with the buffer object.
5. If the buffer is visible from the host, map a pointer to the destination with vkMapMemory, copy the data, and unmap the memory with vkUnmapMemory.
6. If the buffer is a device-local buffer, copy the data to a staging buffer first, then perform the final copy from the staging buffer to the device-local memory using the vkCmdCopyBuffer function.

As you can see, that’s a complex procedure that can be simplified by using the VMA library, an open source library that provides a convenient and efficient way to manage memory in Vulkan. It offers a high-level interface that abstracts the complex details of memory allocation, freeing you from the burden of manual memory management.

To use VMA, you first need to create an instance of the library and store a handle in a variable of type VmaAllocator. To create one, you need a Vulkan physical device and a device.

How to do it…

Creating a VMA library instance requires instancing two different structures. One stores pointers to API functions that VMA needs to find other function pointers and another structure that provides a physical device, a device, and an instance for creating an allocator:

VkPhysicalDevice physicalDevice;  // Valid Physical Device
VkDevice device; // Valid Device
VkInstance instance; // Valid Instance
const uint32_t apiVersion = VK_API_VERSION_1_3;
const VmaVulkanFunctions vulkanFunctions = {
    .vkGetInstanceProcAddr = vkGetInstanceProcAddr,
    .vkGetDeviceProcAddr = vkGetDeviceProcAddr,
#if VMA_VULKAN_VERSION >= 1003000
    .vkGetDeviceBufferMemoryRequirements =
        vkGetDeviceBufferMemoryRequirements,
    .vkGetDeviceImageMemoryRequirements =
        vkGetDeviceImageMemoryRequirements,
#endif
};
VmaAllocator allocator = nullptr;
const VmaAllocatorCreateInfo allocInfo = {
    .physicalDevice = physicalDevice,
    .device = device,
    .pVulkanFunctions = &vulkanFunctions,
    .instance = instance,
    .vulkanApiVersion = apiVersion,
};
vmaCreateAllocator(&allocInfo, &allocator);

The allocator needs pointers to a few Vulkan functions so that it can work based on the features you would like to use. In the preceding case, we provide only the bare minimum for allocating and deallocating memory. The allocator needs to be freed once the context is destroyed with vmaDestroyAllocator.

A buffer in Vulkan is simply a contiguous block of memory that holds some data. The data can be vertex, index, uniform, and more. A buffer object is just metadata and does not directly contain data. The memory associated with a buffer is allocated after a buffer has been created.

Table 2.1 summarizes the most important usage types of buffers and their access type:

Table 2.1 – Buffer types

Creating buffers is easy, but it helps to know what types of buffers exist and what their requirements are before setting out to create them. In this chapter, we will provide a template for creating buffers.

Getting ready

In the repository, Vulkan buffers are managed by the VulkanCore::Buffer class, which provides functions to create and upload data to the device, as well as a utility function to use a staging buffer to upload data to device-only heaps.

How to do it…

Creating a buffer using VMA is simple:

1. All you need are buffer creation flags ( –a value of 0 for the flags is correct for most cases), the size of the buffer in bytes, its usage (this is how you define how the buffer will be used), and assign those values to an instance of the VkBufferCreateInfo structure:

   VkDeviceSize size;  // The requested size of the buffer
   VmaAllocator allocator;  // valid VMA Allocator
   VkUsageBufferFlags use;  // Transfer src/dst/uniform/SSBO
   VkBuffer buffer;        // The created buffer
   VkBufferCreateInfo createInfo = {
       .sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO,
       .pNext = nullptr,
       .flags = {},
       .size = size,
       .usage = use,
       .sharingMode = VK_SHARING_MODE_EXCLUSIVE,
       .queueFamilyIndexCount = {},
       .pQueueFamilyIndices = {},
   };

   You will also need a set of VmaAllocationCreateFlagBits values:

   const VmaAllocationCreateFlagBits allocCreateInfo = {
       VMA_ALLOCATION_CREATE_MAPPED_BIT,
       VMA_MEMORY_USAGE_CPU_ONLY,
   };

2. Then, call vmaCreateBuffer to obtain the buffer handle and its allocation:

   VmaAllocation allocation;  // Needs to live until the
                              // buffer is destroyed
   VK_CHECK(vmaCreateBuffer(allocator, &createInfo,
                            &allocCreateInfo, &buffer,
                            &allocation, nullptr));

3. The next step is optional but useful for debugging and optimization:

   VmaAllocationInfo allocationInfo;
   vmaGetAllocationInfo(allocator, allocation,
                        &allocationInfo);

Some creation flags affect how the buffer can be used, so you might need to make adjustments to the preceding code depending on how you intend to use the buffers you create in your application.

Uploading data from the application to the GPU depends on the type of buffer. For host-visible buffers, it’s a direct copy using memcpy. For device-local buffers, we need a staging buffer, which is a buffer that is visible both by the CPU and the GPU. In this recipe, we will demonstrate how to upload data from your application to the device-visible memory (into a buffer’s memory region on the device).

Getting ready

If you haven’t already, please refer to the Understanding Vulkan’s memory model recipe.

How to do it…

The upload process depends on the type of buffer:

1. For host-visible memory, it’s enough to retrieve a pointer to the destination using vmaMapMemory and copy the data using memcpy. The operation is synchronous, so the mapped pointer can be unmapped as soon as memcpy returns.

   It’s fine to map a host-visible buffer as soon as it is created and leave it mapped until its destruction. That is the recommended approach, as you don’t incur the overhead of mapping the memory every time it needs to be updated:

   VmaAllocator allocator;   // Valid VMA allocator
   VmaAllocation allocation; // Valid VMA allocation
   void *data;               // Data to be uploaded
   size_t size;              // Size of data in bytes
   void *map = nullptr;
   VK_CHECK(vmaMapMemory(allocator, allocation,
                         &map));
   memcpy(map, data, size);
   vmaUnmapMemory(allocator_, allocation_);
   VK_CHECK(vmaFlushAllocation(allocator_,
                               allocation_, offset,
                               size));

2. Uploading data to a device-local memory needs to be (1) copied to a buffer that is visible from the host first (called a staging buffer) and then (2) copied from the staging buffer to the device-local memory using vkCmdCopyBuffer, as depicted in Figure 2.4. Note that this requires a command buffer:

Figure 2.4 – Staging buffers

3. Once the data is residing on the device (on the host-visible buffer), copying it to the device-only buffer is simple:

   VkDeviceSize srcOffset;
   VkDeviceSize dstOffset;
   VkDeviceSize size;
   VkCommandBuffer commandBuffer; // Valid Command Buffer
   VkBuffer stagingBuffer; // Valid host-visible buffer
   VkBuffer buffer; // Valid device-local buffer
   VkBufferCopy region(srcOffset, dstOffset, size);
   vkCmdCopyBuffer(commandBuffer, stagingBuffer, buffer, 1, &region);

Uploading data from your application to a buffer is accomplished either by a direct memcpy operation or by means of a staging buffer. We showed how to perform both uploads in this recipe.