One of the big problems with OpenGL shaders is that the runtime compiler doesn’t provide an include mechanism to let you easily share common code between multiple vertex and fragment shaders. At the present, if you have a common function that you’d like to use in more than one shader your only real option is to maintain separate copies of the function in each of the shader source files you want to use it in.

This is a point of pain for several reasons, the biggest of which is maintainability. If you find a bug or make an enhancement to the common function you have to go back and update each individual copy. Also, the code clutters the shaders and detracts your focus from the code specific to that particular file.

This blog post details the method I use to separate out common shader code into separate files and access it from multiple OpenGL shaders.

Introducing m4, the unix preprocessor

m4 is a standard UNIX tool orginally developed by Kernighan and Ritchie in 1977. The most common version today is GNU m4, which found on most UNIX-based systems, including OS X and Linux. m4 is not particularly well-known to most users, but is a well regarded workhorse and a foundation for most UNIX services. As an example, m4 is used extensively in GNU Autoconf.

m4 is a command line tool that acts as a preprocessor, similar in concept to the regular C preprocessor but with a few syntactic differences. You can use m4 to define your own macros for text expansion and can also insert the contents of one or more files into another by use of the m4 include directive.

Using m4 with your shaders

In this example, let’s imagine that you have some common lighting code you’d like to be able to share between multiple different shaders. The goal is to maintain the lighting code in one file and be able to call that code from the shaders that need to use it.

In this example we have two vertex shaders, creatively named shader1.vs and shader2.vs, that will need to use the lighting code. We’ll use m4 to pull the lighting code into the shader files when we build our project, and store off merged versions of the shader text files in our application bundle (see below).

The final shader output files will look more like conventional shaders: the lighting function will be defined up above the shader’s main() method and appear in full in both shader1.vs and shader2.vs, so the runtime compiler will have no problem building and loading them when you go to use them in your application.

Showtime

First off, let’s look at lighting.vs

struct directional_light {
    vec3 direction;
    vec3 halfplane;
    vec4 ambient_color;
    vec4 diffuse_color;
    vec4 specular_color;
};

struct material_properties {
    vec4 ambient_color;
    vec4 diffuse_color;
    vec4 specular_color;
    float   specular_exponent;
};

const float c_zero = 0.0;
const float c_one = 1.0;

material_properties material;
directional_light light;

vec4 calc_directional_light(in vec3 normal) {
    vec4 computed_color = vec4(c_zero, c_zero, c_zero, c_zero);
    float ndot1;
    float ndoth;
    ndot1 = max(c_zero, dot(normal, light.direction));
    ndoth = max(c_zero, dot(normal, light.halfplane));
    
    computed_color += (light.ambient_color * material.ambient_color);
    computed_color += (ndot1 * light.diffuse_color * material.diffuse_color);
    if( ndoth > c_zero)
    {
        computed_color += (pow(ndoth, material.specular_exponent) * material.specular_color* light.specular_color);
    }
    return computed_color;
}

Including all of that in each shader file directly would add a lot more text to wade through! So let’s use m4 instead. Here’s what we’ll put in shader1.vs

include('Lighting.vs') // < - much more compact!

/**
 *  Uniform attributes
 */
uniform mat4    u_mvp_matrix;       // Modelview matrix
uniform mat4    u_project_matrix;   // Projection matrix

/**
 *  Assigned attributes specific to this vertex
 */
attribute vec4  a_position;
attribute vec4  a_vertColor;
attribute vec3  a_normal;

void main()
{
    // Do stuff
    ...
    // Now do our lighting calculation
    vec4 color = calc_directional_light(a_normal);
    // Do more stuff 
}

Pulling it all together

Finally, we'll add a Run Script step as the last part of our Xcode target to do the final shader merges automatically at the end of each build. Here's the contents of the script step:

cd build/$CONFIGURATION-iphoneos/myIphoneApp.app
for f in `ls *.vs`; do m4 $f > $f.tmp;mv $f.tmp $f; done;
for f in `ls *.fs`; do m4 $f > $f.tmp;mv $f.tmp $f; done;

Assuming that your application's name was myIphoneApp, this script would:

1. dive into the application bundle in your build directory
2. get a listing of all of the vertex shader files
3. run the following process on each shader file found:
   1. run m4 on the file
   2. store the output to a temporary file
   3. replace the original file with the temporary file
4. repeat steps 1 thought 3 for each fragment shader file

The preprocessing step simply acts as a pass-through on files that don't contain m4-specific directives, so it won't do any harm to files that aren't including lighting.vs.

That's all there is to it! I hope that this is helpful to people. If anyone has any suggestions for improvements I would be very interested in hearing your feedback.