Framebuffers — new strategies and syntax in mental ray 3.6

Chapter 25 demonstrates how rendering components can be saved into separate files using framebuffers. This page describes the simplified scene file syntax for the definition of framebuffers introduced in mental ray version 3.6 as well as the use of geometry shaders in framebuffer definition.

A review of framebuffer use prior to version 3.6

In releases of mental ray prior to 3.6, framebuffers defined by the user are identified by integer indices, and named simply by preceding the framebuffer index with the string “fb”. These names are defined in the options block of the scene, for example, in lines 7-9 in this set of options statements from scene file buffer_5.mi from Chapter 25.

 1  options "opt"
 2      object space
 3      contrast .1 .1 .1 1
 4      samples 0 2
 5      finalgather on
 6      finalgather accuracy 50 2 .5
 7      frame buffer 0 "+rgba"
 8      frame buffer 1 "+rgba"
 9      frame buffer 2 "+rgba"
10  end options

During rendering, shaders can access framebuffers using the API functions mi_fb_put() and mi_fb_get(). For example, Chapter 25 defines the shader framebuffer_put that simply passes along the color in the result pointer, but with the side effect of storing the color in a framebuffer with mi_fb_put() in line 7:

 1  miBoolean framebuffer_put(
 2      miColor *result, miState *state, struct framebuffer_put *params)
 3  {
 4      *result = *mi_eval_color(&params->color);
 5  
 6      if (state->type == miRAY_EYE)
 7          mi_fb_put(state, *mi_eval_integer(&params->index), result);
 8  
 9      return miTRUE;
10  }

When rendering is done, framebuffers are written to the files defined by “output statements” in the camera:

 1  camera "cam"
 2      output "fb0" "tif"
 3          "buffer_5_diffuse.tif"
 4      output "fb1" "tif"
 5          "buffer_5_specular.tif"
 6      output "fb2" "tif"
 7          "buffer_5_indirect.tif"
 8      output "rgba" "tif"
 9          "buffer_5.tif"
10      focal 90
11      aperture 33.3
12      aspect 1.4
13      resolution 420 300
14      environment
15          "one_color" (
16              "color" .1 .1 .2 )
17  end camera

Though this framebuffer structure is still valid in version 3.6, it continues to sufffer from two primary problems:

  1. The separate definition of the framebuffer index in the options and camera elements of the scene is inherently error-prone.
  2. The name of a framebuffer does not describe its contents or intended use.

In the following sections, we’ll address problem #1 with geometry shaders make_indexed_framebuffers and make_named_framebuffers for framebuffer definition. The new “named framebuffers” of version 3.6 will help with problem #2.

Framebuffer creation Framebuffer writing
Indexed make_indexed_framebuffers framebuffer_put (from Chapter 25)
Named make_named_framebuffers named_framebuffer_put

Defining framebuffers in a geometry shader

We typically think of geometry shaders as constructing geometric primitives to be added to the scene that mental ray will render. However, geometry shaders also have access to other elements in the scene database, and (with appropriate care) can modify those elements. Here we are depending upon the fact that geometry shaders only run once before actual rendering begins and primary rays are sent from the camera into the scene.

The geometry shader make_indexed_framebuffers doesn’t make geometry at all—it modifies an option element by adding framebuffers.

 1  declare shader
 2      geometry "make_indexed_framebuffers" (
 3          string "option_block_name",
 4          array struct "framebuffers" {
 5              integer "index",
 6              string "type",
 7              string "filename"
 8          }
 9      )
10      version 1
11      apply geometry
12  end declare

Because a scene may have more than one element of type options, the parameter option_block_name in line 3 identifies the option element to be modified. The second parameter is an example of the combined use of two mechanism that allow multiple values to be passed to a shader as a single parameter. The framebuffers parameter, defined in lines 4-8, is an array of structs. Grouping all the information required to create and use a single framebuffer in a struct simplifies the definition of a list of framebuffers. (This is similar to the array of colors that we developed for environment shaders in Chapter 21.

Specifying the array of structs for make_indexed_framebuffers in a scene file follows the same syntactic pattern for an array of any type: a comma-separated list of items enclosed by square brackets. In the case of a struct, each item is surrounded by curly braces, with the fields of the struct defined by key/value pairs:

 1  instance "framebuffer_creation"
 2       geometry "make_indexed_framebuffers" (
 3          "option_block_name" "opt",
 4          "framebuffers" [
 5              { "index" 0,
 6                "type" "+rgba",
 7                "filename" "buffer_6_diffuse.tif" },
 8              { "index" 1,
 9                "type" "+rgba",
10                "filename" "buffer_6_specular.tif" },
11              { "index" 2,
12                "type" "+rgba",
13                "filename" "buffer_6_indirect.tif" }
14          ]
15      )
16  end instance

In the C shader code, a corresponding struct must be declared for the struct used in the list of framebuffers, duplicating the struct of the scene file and its field values specifying the framebuffer’s index, type and filename:

 1  typedef struct {
 2      miInteger index;
 3      miTag type;
 4      miTag filename;
 5  } indexed_framebuffer;

Now that we’ve declared the struct for a single framebuffer, we can declare the parameter struct that will include an array of the these individual framebuffers. This is the same type of parameter struct that is used to define array parameters, with the single scene file array parameter expanded in C to three fields that define the offset, count, and pointer to the initial element of the array. (See the description of light arrays in Chapter 12, section 1, and the on-line documentation section Using and Writing Shaders / Shader Parameter Declarations.)

 1  struct make_indexed_framebuffers {
 2      miTag option_block_name;
 3      int i_fb;
 4      int n_fb;
 5      indexed_framebuffer fb[1];
 6  };

However, the complexity of an array of structs suggests that we divide the shader into two parts, separately evaluating the shader’s parameters to create the arguments that will define the actions of the shader. We’ll store the evaluated parameters into a separate struct, fb_args:

 1  typedef struct {
 2      int index;
 3      char* type;
 4      char* filename;
 5      char* ext;
 6      char* name;
 7  } fb_args;

For clarity, we’re using the convention that a shader’s inputs are called its parameters, whereas the inputs to a function are called its arguments.

Utility function get_indexed_framebuffer_args evaluates the shader’s parameters. It assumes that the memory for an array of fb_args has already been allocated. The array — a pointer to fb_args — is passed as the first argument.

 1  void get_indexed_framebuffer_args(
 2      fb_args args[], int count, miState *state,
 3      struct make_indexed_framebuffers *params)
 4  {
 5      int i_fb = *mi_eval_integer(&params->i_fb), i;
 6      mi_info("Making %i user framebuffers:", count);
 7  
 8      for (i = 0; i < count; ++i) {
 9          char name[16];
10          indexed_framebuffer fb = params->fb[i + i_fb];
11          args[i].index = *mi_eval_integer(&fb.index);
12          args[i].type = tag_to_string(*mi_eval_tag(&fb.type), NULL);
13          args[i].filename = tag_to_string(*mi_eval_tag(&fb.filename), NULL);
14          args[i].ext = get_extension(args[i].filename);
15          sprintf(name, "fb%d", args[i].index);
16          args[i].name = mi_mem_strdup(name);
17          mi_progress("   Buffer %d: filename '%s', type '%s', ext '%s', name '%s'",
18                      args[i].index, args[i].filename, args[i].type,
19                      args[i].ext, args[i].name);
20      }
21  }

Notice that we are printing out the results of our parameter evaluation in lines 17-19. We can easily confirm the accuracy of reading the parameters from the scene file by viewing the progress messages. Now we can be sure that any errors we encounter after this point result from the use of the parameters in the shader and not simply from their acquisition.

Using get_indexed_framebuffer_args, we can now define the shader function for make_indexed_framebuffers.

 1  miBoolean make_indexed_framebuffers (
 2      miTag *result, miState *state, struct make_indexed_framebuffers *params)
 3  {
 4      char* option_block_name =
 5          tag_to_string(*mi_eval_tag(&params->option_block_name), "opt");
 6      miOptions* options = (miOptions*)(state->options);
 7  
 8      int count = *mi_eval_integer(&params->n_fb), i;
 9      fb_args* args = (fb_args*)mi_mem_allocate(count * sizeof(fb_args));
10      get_indexed_framebuffer_args(args, count, state, params);
11  
12      mi_api_options_begin(mi_mem_strdup(option_block_name));
13      for (i = 0; i < count; ++i)
14          mi_api_framebuffer(
15              options, args[i].index, mi_mem_strdup(args[i].type));
16      mi_api_options_end();
17  
18      for (i = 0; i < count; ++i) {
19          /* A framebuffer might not be output to file; check for a filename: */
20          if (args[i].filename && strlen(args[i].filename) > 0) {
21              mi_api_framebuffer_add(
22                  state->camera->buffertag, mi_mem_strdup(args[i].name),
23                  mi_mem_strdup(args[i].ext), 0,
24                  mi_mem_strdup(args[i].filename));
25          }
26      }
27      mi_mem_release(args);
28      return miTRUE;
29  }

The size of the array of fb_args allocated in line 9 is based on the number of framebuffers defined by parameter n_fb. Notice that we use mi_mem_allocate in line 9 and mi_mem_release in line 27, rather than malloc and free, to allow for mental ray memory management. We store the framebuffer creation arguments in array args in line 10. Framebuffer creation then takes place in two phases: addition of the framebuffers to the options element in lines 12-16, and the actual creation of the framebuffers in lines 18-26.

A new syntax for framebuffers

The construction of indexed framebuffers in the geometry shader of the previous section consolidates the scene file definitions of framebuffers into a single syntactic unit. However, it does not address our other problem of a fixed set of framebuffer names. As of mental ray release 3.6, framebuffer definition can instead be consolidated in the scene file itself within the camera element. The new framebuffer statement in the camera object defines all attributes of each framebuffer. Framebuffers can also have arbitrary names; these names are then used to determine the index for mi_fb_put.

For example, the framebuffer definitions in the first section, above, could be replaced by these statements within the camera:

 1  camera "cam"
 2      framebuffer "primary"
 3          datatype "rgba"
 4          filtering on
 5          primary on
 6          filename "buffer_7.tif"
 7      framebuffer "diffuse"
 8          datatype "rgba"
 9          filtering on
10          user on
11          filename "buffer_7_diffuse.tif"
12      framebuffer "specular"
13          datatype "rgba"
14          filtering on
15          user on
16          filename "buffer_7_specular.tif"
17      framebuffer "indirect"
18          datatype "rgba"
19          filtering on
20          user on
21          filename "buffer_7_indirect.tif"
22      focal 90
23      aperture 33.3
24      aspect 1.4
25      resolution 420 300
26      environment
27          "one_color" (
28              "color" .1 .1 .2 )
29  end camera

The full set of attributes for the framebuffer are described in the on-line documentation in Scene Description Language / Scene Entities / Camera / Frame Buffers.

The shader framebuffer_put defined in the book can be modified to use the new named framebuffer mechanism. The declaration of the new shader named_framebuffer_put has only been changed from framebuffer_put in the use of a name instead of an index to specify the framebuffer.

 1  declare shader
 2      color "named_framebuffer_put" (
 3          color "color",
 4          string "name" )
 5      version 1
 6      apply material
 7  end declare

Newer mechanisms in mental ray use C++ namespaces and classes. In named_framebuffer_put, access to framebuffers is provided by class Access_fb in namespace mi::shader. (See Upgrading / Upgrading from mental ray 3.5 to 3.6 in the on-line mental ray documentation.)

 1  miBoolean named_framebuffer_put(
 2      miColor *result, miState *state, struct named_framebuffer_put *params)
 3  {
 4      char *buffer_name = miaux_tag_to_string(*mi_eval_tag(&params->name), NULL);
 5      *result = *mi_eval_color(&params->color);
 6  
 7      mi::shader::Access_fb framebuffers(state->camera->buffertag);
 8  
 9      size_t buffer_index;
10      if (buffer_name && framebuffers->get_index(buffer_name, buffer_index))
11          mi_fb_put(state, buffer_index, result);
12  
13      return miTRUE;
14  }

The syntax of line 7 may be unfamiliar to C programmers. It declares a variable named framebuffers that is an instance of class mi::shader::Access_fb (the Access_fb class in the mi::shader namespace). The constructor function of Access_fb that has a single miTag argument is then run with state->camera->buffertag to initialize the class instance. To use mi_fb_put in line 11 we need the index of the framebuffer, so in line 10 we call the get_index method of Access_fb on instance framebuffers.

But what about other framebuffers? When we defined the geometry shader make_indexed_framebuffers that created framebuffers, we had the shader also print out a description of the framebuffer list. We can use an init function for named_framebuffer_put that will only run a single time when the shaders is first used to display information about all the framebuffers that have been defined.

 1  miBoolean named_framebuffer_put_init(
 2      miState *state, void *params, miBoolean *instance_init_required)
 3  {
 4      mi::shader::Access_fb framebuffers(state->camera->buffertag);
 5  
 6      size_t count = 0;
 7      framebuffers->get_buffercount(count);
 8  
 9      if (count > 0)
10          mi_info("Checking the %i existing framebuffers:", count);
11      else
12          mi_warning("No framebuffers found.");
13  
14      for (unsigned int i = 0; i < count; i++) {
15          const char *name = NULL;
16          if (framebuffers->get_buffername(i, name)) {
17              size_t index;
18              if (framebuffers->get_index(name, index))
19                  mi_info("  framebuffer '%s' has index %i", name, (int)index);
20              else
21                  mi_info("  framebuffer '%s' has no index", name);
22          }
23      }
24      return miTRUE;
25  }

Printing out diagnostic information before rendering begins can help verify that you have defined the framebuffers correctly, which is not as easily done once rendering begins.

Defining named framebuffers in a geometry shader

Creating framebuffers in a geometry shader using the new Access_fb class is very similar in structure to the method shown above for indexed framebuffers. Once again, we have an array of structs as an input to our geometry shader, but with a name instead of a number to identify each one.

 1  declare shader
 2      geometry "make_named_framebuffers" (
 3          array struct "framebuffers" {
 4              string "name",
 5              string "type",
 6              boolean "filtering",
 7              string "filename"
 8          }
 9      )
10      version 1
11      apply geometry
12  end declare

Using the shader in a scene file employs the same array of structs as with the indexed version.

 1  instance "framebuffer_creation"
 2       geometry "make_named_framebuffers" (
 3          "framebuffers" [
 4              { "name" "diffuse",
 5                "type" "rgba",
 6                "filtering" on,
 7                "filename" "buffer_8_diffuse.tif" },
 8              { "name" "specular",
 9                "type" "rgba",
10                "filtering" on,
11                "filename" "buffer_8_specular.tif" },
12              { "name" "indirect",
13                "type" "rgba",
14                "filtering" on,
15                "filename" "buffer_8_indirect.tif" }
16          ]
17      )
18  end instance

We need a struct to define the information for a single framebuffer that is the C++ version of the structs in the scene file.

 1  typedef struct {
 2      miInteger name;
 3      miTag type;
 4      miBoolean filtering;
 5      miTag filename;
 6  } named_framebuffer;

The parameter struct for make_named_framebuffers defines the same array of structs for the framebuffers. However, we are not modifying the options element, so we do not need to specify its name as a parameter.

 1  struct make_named_framebuffers {
 2      int i_fb;
 3      int n_fb;
 4      named_framebuffer fb[1];
 5  };

Evaluating the parameters for each framebuffer produces arguments specific to the new framebuffer mechanism, stored in struct fb_args.

 1  typedef struct {
 2      char* name;
 3      char* type;
 4      miBoolean filtering;
 5      char* filename;
 6  } fb_args;

We’ll divide up the shader into two phases as before: evaluate all the parameters, then use those parameters to create framebuffers.

 1  std::vector<fb_args> get_named_framebuffer_args(
 2      miState *state, struct make_named_framebuffers *params)
 3  {
 4      int n_fb = *mi_eval_integer(&params->n_fb);
 5      int i_fb = *mi_eval_integer(&params->i_fb);
 6  
 7      mi_info("Making %i user framebuffers:", n_fb);
 8  
 9      std::vector<fb_args> args(n_fb);
10      for (int i = 0; i < n_fb; ++i) {
11          named_framebuffer p = params->fb[i + i_fb];
12          args[i].name = miaux_tag_to_string(*mi_eval_tag(&p.name), NULL);
13          args[i].type = miaux_tag_to_string(*mi_eval_tag(&p.type), NULL);
14          args[i].filtering = *mi_eval_boolean(&p.filtering);
15          args[i].filename = miaux_tag_to_string(*mi_eval_tag(&p.filename), NULL);
16          mi_progress("   Buffer %s: filename '%s', type '%s', filtering %s",
17                      args[i].name, args[i].filename, args[i].type,
18                      args[i].filtering ? "on" : "off");
19      }
20      return args;
21  }

Notice that the list of framebuffer arguments are stored in the standard library vector container defined in line 9 of get_named_framebuffer_args. Once we’ve collected information for all the framebuffers, we use the Edit_fb class to set the attributes of the framebuffer list in the database.

 1  miBoolean make_named_framebuffers (
 2      miTag *result, miState *state, struct make_named_framebuffers *params)
 3  {
 4      std::vector<fb_args> args = get_named_framebuffer_args(state, params);
 5      mi::shader::Edit_fb framebuffers(state->camera->buffertag);
 6  
 7      for (unsigned int i = 0; i < args.size(); ++i) {
 8          framebuffers->set(args[i].name, "datatype", args[i].type);
 9          framebuffers->set(args[i].name, "filtering",
10                            args[i].filtering ? true : false);
11          framebuffers->set(args[i].name, "user", true);
12          // A framebuffer could be used by output shader and not
13          // written to file, so check for a filename:
14          if (args[i].filename && strlen(args[i].filename) > 1) {
15              mi_info ("  ... writes to %s", args[i].filename);
16              framebuffers->set(args[i].name, "filename", args[i].filename);
17          }
18      }
19  
20      return miTRUE;
21  }

These new mechanisms for framebuffers differ from version 3.5 in the way that framebuffers are defined and accessed in shaders, but the techniques of Chapter 25 for creating compositing layers are still applicable.

buffer_8.jpg
buffer_8.tif

buffer_8_diffuse.jpg
buffer_8_diffuse.tif

buffer_8_specular.jpg
buffer_8_specular.tif

buffer_8_indirect.jpg
buffer_8_indirect.tif

 1  instance "framebuffer_creation"
 2       geometry "make_named_framebuffers" (
 3          "framebuffers" [
 4              { "name" "diffuse",
 5                "type" "rgba",
 6                "filtering" on,
 7                "filename" "buffer_8_diffuse.tif" },
 8              { "name" "specular",
 9                "type" "rgba",
10                "filtering" on,
11                "filename" "buffer_8_specular.tif" },
12              { "name" "indirect",
13                "type" "rgba",
14                "filtering" on,
15                "filename" "buffer_8_indirect.tif" }
16          ]
17      )
18  end instance
19  
20  declare phenomenon
21      "global_phong_components" (
22          color "diffuse" default .5 .5 .5,
23          color "specular" default .3 .3 .3,
24          scalar "exponent" default 30,
25          array light "lights" )
26  
27      shader "diffuse"
28          "phong" (
29              "diffuse" = interface "diffuse",
30              "lights" = interface "lights",
31              "specular" 0 0 0 )
32  
33      shader "specular"
34          "phong" (
35              "specular" = interface "specular",
36              "exponent" = interface "exponent",
37              "lights" = interface "lights",
38              "diffuse" 0 0 0 )
39  
40      shader "indirect"
41          "average_radiance" ()
42  
43      shader "indirect_diffuse"
44          "op_mul_cc" (
45              "A" = "indirect",
46              "B" = interface "diffuse" )
47  
48      shader "write_diffuse"
49          "named_framebuffer_put" (
50              "color" = "diffuse",
51              "name" "diffuse" )
52  
53      shader "write_specular"
54          "named_framebuffer_put" (
55              "color" = "specular",
56              "name" "specular" )
57  
58      shader "write_indirect"
59          "named_framebuffer_put" (
60              "color" = "indirect_diffuse",
61              "name" "indirect" )
62  
63      shader "add_direct"
64          "add_colors" (
65              "x" = "write_diffuse",
66              "y" = "write_specular" )
67  
68      shader "add_indirect"
69          "add_colors" (
70              "x" = "add_direct",
71              "y" = "write_indirect" )
72  
73      root = "add_indirect"
74  end declare

Source files

Source files (gzip tar) WMRS_framebuffers.tgz
Source files (zip) WMRS_framebuffers.zip
24 April 2008 23:09:18