Parallax Mapping (or, tangent space revisited)

The eye gets used to what it's seeing quite soon.

Normal mapping is actually an order of magnitude more realistic that classic texturing.

But the illusion is fragile. The illusion of depth is quite easy to see through as soon as the viewing angle changes - the real flatness of the surface becomes apparent, or at least easily detectable, soon enough.

Something more is needed, something basically unrelated to normal mapping, but ultimately similar in tools to it.

Parallax mapping, or, the simplest form of Virtual Displacement Mapping.


Parallax mapping is a shot at making a surface's features behave 3D-correctly. The idea is to move the sampling point of the various textures used (the various color/albedo etc. textures, plus the normal textures) using the viewing angle and the depth at the sampled point as input.

This basically needs two things.

First, it needs tangent space, which is where the similarity with tangent-space normal maps comes from.

Then, it needs another texture, the displacement map. This is actually the simplest of the textures used - it is a simple grayscale image that shows the height difference, in the direction of the normal at that point (hence in tangent space), between the shape we want to represent represented and the actual mesh used.

Using the displacement map, you sample and find the height/depth at that point. First gotcha, it is rather difficult to have a common ground between maps of different models, taking into account both model size, and whether the features are above or below the surface. Enter bias and scale. These two should be per-object properties, that transform the [0,1] sample from the map to what is meaningful for the particular object.

As soon as you get the displacement, you can multiply it with the surface-parallel component of the view direction in tangent space. The result, is the new sampling point for our textures.

In other words, the answer to the question, "If I take into account my viewing angle and the fact that the point I am looking at is below the actual surface, what point would I actually see?"

The result is truly beautiful and realistic, your details actually recess or protrude depending on the viewing angle, giving the actual illusion of depth to the objects you view.

There are disadvantages of course, the most important one being that, at its simple form described, you only do a first-order approximation instead of solving for the actual sampling point which, especially on oblique angles, introduces artifacts (texture swim) if you try to add any more than a little bit of depth.

Many methods exist to improve parallax mapping (I am aware of at least 5 OOTTOMH), most of them doing some kind of ray-tracing to approximate the actual point. The most interesting ones deal with self-occlusion and self-shadowing.




Note that the simplest version of the parallax effect can be calculated out of the box in a shader that works in tangent space - in fact, all you have to do is integrate something like the following:


Before wondering what on earth I am doing with the alpha of the normal texture, you should be aware that I am using a cooked normal/displacement map that contains the normals in the rgb channels, and the displacement in the alpha channel. You could of course use any configuration that suits your fancy engine.

//You should have all these numbers out of the box in a classic, tangent-space normal mapped pixel shader
float2 get_parallax_texuv(float2 texuv, float2 tan_view_dir, float scale, float bias)
{
//Displacement is cooked in the alpha channel of the normal map.
    float displacement = (normalTexture.Sample(normalSampler, texuv).a * scale + bias);
    return texuv + (tan_view_dir * displacement);
}

And there you have it: an approximation of surface subject to the parallax effect!