Understanding the glTF scene format
Many 3D file formats have been developed over the years, and for this book, we chose to use glTF. It has become increasingly popular in recent years; it has an open specification, and it supports a physically based rendering (PBR) model by default.
We chose this format because of its open specification and easy-to-understand structure. We can use several models provided by Khronos on GitHub to test our implementation and compare our results with other frameworks.
It is a JSON-based format and we built a custom parser for this book. The JSON data will be deserialized into a C++ class, which we are going to use to drive the rendering.
We now provide an overview of the main sections of the glTF format. At its root, we have a list of scenes, and each scene can have multiple nodes. You can see this in the following code:
"scene": 0,
"scenes": [
{
"nodes": [
0,
1,
2,
3,
4,
5
]
}
],
Each node contains an index that is present in the mesh array:
"nodes": [
{
"mesh": 0,
"name": "Hose_low"
},
]
The data for the scene is stored in one or more buffers, and each section of the buffer is described by a buffer view:
"buffers": [
{
"uri": "FlightHelmet.bin",
"byteLength": 3227148
}
],
"bufferViews": [
{
"buffer": 0,
"byteLength": 568332,
"name": "bufferViewScalar"
},
]
Each buffer view references the buffer that contains the actual data and its size. An accessor points into a buffer view by defining the type, offset, and size of the data:
"accessors": [
{
"bufferView": 1,
"byteOffset": 125664,
"componentType": 5126,
"count": 10472,
"type": "VEC3",
"name": "accessorNormals"
}
]
The mesh array contains a list of entries, and each entry is composed of one or more mesh primitives. A mesh primitive contains a list of attributes that point into the accessors array, the index of the indices accessor, and the index of the material:
"meshes": [
{
"primitives": [
{
"attributes": {
"POSITION": 1,
"TANGENT": 2,
"NORMAL": 3,
"TEXCOORD_0": 4
},
"indices": 0,
"material": 0
}
],
"name": "Hose_low"
}
]
The materials object defines which textures are used (diffuse color, normal map, roughness, and so on) and other parameters that control the rendering of the material:
"materials": [
{
"pbrMetallicRoughness": {
"baseColorTexture": {
"index": 2
},
"metallicRoughnessTexture": {
"index": 1
}
},
"normalTexture": {
"index": 0
},
"occlusionTexture": {
"index": 1
},
"doubleSided": true,
"name": "HoseMat"
}
]
Each texture is specified as a combination of an image and a sampler:
"textures": [
{
"sampler": 0,
"source": 0,
"name": "FlightHelmet_Materials_RubberWoodMat_Nor
mal.png"
},
],
"images": [
{
"uri": "FlightHelmet_Materials_RubberWoodMat_Nor
mal.png"
},
],
"samplers": [
{
"magFilter": 9729,
"minFilter": 9987
}
]
The glTF format can specify many other details, including animation data and cameras. Most of the models that we are using in this book don’t make use of these features, but we will highlight them when that’s the case.
The JSON data is deserialized into a C++ class, which is then used for rendering. We omitted glTF extensions in the resulting object as they are not used in this book. We are now going through a code example that shows how to read a glTF file using our parser. The first step is to load the file into a glTF object:
char gltf_file[512]{ };
memcpy( gltf_file, argv[ 1 ], strlen( argv[ 1] ) );
file_name_from_path( gltf_file );
glTF::glTF scene = gltf_load_file( gltf_file );
We now have a glTF model loaded into the scene variable.
The next step is to upload the buffers, textures, and samplers that are part of our model to the GPU for rendering. We start by processing textures and samplers:
Array<TextureResource> images;
images.init( allocator, scene.images_count );
for ( u32 image_index = 0; image_index
< scene.images_count; ++image_index ) {
glTF::Image& image = scene.images[ image_index ];
TextureResource* tr = renderer.create_texture(
image.uri.data, image.uri.data );
images.push( *tr );
}
Array<SamplerResource> samplers;
samplers.init( allocator, scene.samplers_count );
for ( u32 sampler_index = 0; sampler_index
< scene.samplers_count; ++sampler_index ) {
glTF::Sampler& sampler = scene.samplers[ sampler_index ];
SamplerCreation creation;
creation.min_filter = sampler.min_filter == glTF::
Sampler::Filter::LINEAR ? VK_FILTER_LINEAR :
VK_FILTER_NEAREST;
creation.mag_filter = sampler.mag_filter == glTF::
Sampler::Filter::LINEAR ? VK_FILTER_LINEAR :
VK_FILTER_NEAREST;
SamplerResource* sr = renderer.create_sampler( creation
);
samplers.push( *sr );
}
Each resource is stored in an array. We go through each entry in the array and create the corresponding GPU resource. We then store the resources we just created in a separate array that will be used in the rendering loop.
Now let’s see how we process the buffers and buffer views, as follows:
Array<void*> buffers_data;
buffers_data.init( allocator, scene.buffers_count );
for ( u32 buffer_index = 0; buffer_index
< scene.buffers_count; ++buffer_index ) {
glTF::Buffer& buffer = scene.buffers[ buffer_index ];
FileReadResult buffer_data = file_read_binary(
buffer.uri.data, allocator );
buffers_data.push( buffer_data.data );
}
Array<BufferResource> buffers;
buffers.init( allocator, scene.buffer_views_count );
for ( u32 buffer_index = 0; buffer_index
< scene.buffer_views_count; ++buffer_index ) {
glTF::BufferView& buffer = scene.buffer_views[
buffer_index ];
u8* data = ( u8* )buffers_data[ buffer.buffer ] +
buffer.byte_offset;
VkBufferUsageFlags flags =
VK_BUFFER_USAGE_VERTEX_BUFFER_BIT |
VK_BUFFER_USAGE_INDEX_BUFFER_BIT;
BufferResource* br = renderer.create_buffer( flags,
ResourceUsageType::Immutable, buffer.byte_length,
data, buffer.name.data );
buffers.push( *br );
}
First, we read the full buffer data into CPU memory. Then, we iterate through each buffer view and create its corresponding GPU resource. We store the newly created resource in an array that will be used in the rendering loop.
Finally, we read the mesh definition to create its corresponding draw data. The following code provides a sample for reading the position buffer. Please refer to the code in chapter1/main.cpp for the full implementation:
for ( u32 mesh_index = 0; mesh_index < scene.meshes_count;
++mesh_index ) {
glTF::Mesh& mesh = scene.meshes[ mesh_index ];
glTF::MeshPrimitive& mesh_primitive = mesh.primitives[
0 ];
glTF::Accessor& position_accessor = scene.accessors[
gltf_get_attribute_accessor_index(
mesh_primitive.attributes, mesh_primitive.
attribute_count, "POSITION" ) ];
glTF::BufferView& position_buffer_view =
scene.buffer_views[ position_accessor.buffer_view
];
BufferResource& position_buffer_gpu = buffers[
position_accessor.buffer_view ];
MeshDraw mesh_draw{ };
mesh_draw.position_buffer = position_buffer_gpu.handle;
mesh_draw.position_offset = position_accessor.
byte_offset;
}
We have grouped all the GPU resources needed to render a mesh into a MeshDraw data structure. We retrieve the buffers and textures as defined by the Accessor object and store them in a MeshDraw object to be used in the rendering loop.
In this chapter, we load a model at the beginning of the application, and it’s not going to change. Thanks to this constraint, we can create all of our descriptor sets only once before we start rendering:
DescriptorSetCreation rl_creation{};
rl_creation.set_layout( cube_rll ).buffer( cube_cb, 0 );
rl_creation.texture_sampler( diffuse_texture_gpu.handle,
diffuse_sampler_gpu.handle, 1 );
rl_creation.texture_sampler( roughness_texture_gpu.handle,
roughness_sampler_gpu.handle, 2 );
rl_creation.texture_sampler( normal_texture_gpu.handle,
normal_sampler_gpu.handle, 3 );
rl_creation.texture_sampler( occlusion_texture_gpu.handle,
occlusion_sampler_gpu.handle, 4 );
mesh_draw.descriptor_set = gpu.create_descriptor_set(
rl_creation );
For each resource type, we call the relative method on the DescriptorSetCreation object. This object stores the data that is going to be used to create the descriptor set through the Vulkan API.
We have now defined all the objects we need for rendering. In our render loop, we simply have to iterate over all meshes, bind each mesh buffer and descriptor set, and call draw:
for ( u32 mesh_index = 0; mesh_index < mesh_draws.size;
++mesh_index ) {
MeshDraw mesh_draw = mesh_draws[ mesh_index ];
gpu_commands->bind_vertex_buffer( sort_key++,
mesh_draw.position_buffer, 0,
mesh_draw.position_offset );
gpu_commands->bind_vertex_buffer( sort_key++,
mesh_draw.tangent_buffer, 1,
mesh_draw.tangent_offset );
gpu_commands->bind_vertex_buffer( sort_key++,
mesh_draw.normal_buffer, 2,
mesh_draw.normal_offset );
gpu_commands->bind_vertex_buffer( sort_key++,
mesh_draw.texcoord_buffer, 3,
mesh_draw.texcoord_offset );
gpu_commands->bind_index_buffer( sort_key++,
mesh_draw.index_buffer, mesh_draw.index_offset );
gpu_commands->bind_descriptor_set( sort_key++,
&mesh_draw.descriptor_set, 1, nullptr, 0 );
gpu_commands->draw_indexed( sort_key++,
TopologyType::Triangle, mesh_draw.count, 1, 0, 0,
0 );
}
We are going to evolve this code over the course of the book, but it’s already a great starting point for you to try and load a different model or to experiment with the shader code (more on this in the next section).
There are several tutorials online about the glTF format, some of which are linked in the Further reading section. The glTF spec is also a great source of details and is easy to follow. We recommend you refer to it if something about the format is not immediately clear from reading the book or the code.
In this section, we have analyzed the glTF format and we have presented examples of the JSON objects most relevant to our renderer. We then demonstrated how to use the glTF parser, which we added to our framework, and showed you how to upload geometry and texture data to the GPU. Finally, we have shown how to use this data to draw the meshes that make up a model.
In the next section, we explain how the data we just parsed and uploaded to the GPU is used to render our model using a physically-based rendering implementation.