*2 notes* &

# Playing with OpenGL 4 Tessellation

I’ve been interested in the cutting edge GPU features, perhaps most of all hardware tessellation that is provided by graphics cards supporting OpenGL 4 (or DX11). As I happen to own GTX 460 and some programming skills, I set out to learn tessellation in practise.

A word of warning: this post is not a tutorial, so if you have no idea what (hardware) tessellation is, you should check the article linked below for introduction and probably also google a bit.

**Starting point**

I’ve found a couple of tutorials on the net, especially useful was Triangle Tessellation with OpenGL 4.0 which I ended up using to provide the boiler plate shader code. For the C++ side, I used Wendy engine.

To business: Below is a shot of several trivial cubes. They only have the minimal amount of geometry: 8 vertices and 12 triangles. Coloring is done procedurally with a fragment shader.

My goal was to make these boxes appear round and smooth (but not spherical like done in the tutorial linked above) and make it in a generic way, trivially applicable to more complex objects.

**Algorithm**

I came up with an algorithm that utilizes the corner normals of the triangle to calculate a curvature factor and then extrudes the vertices generated by tessellation according to that. In short, the more the direction of the two normals of two edge end points differ, the more the edge should be curved outwards. A value for the difference in normal direction is easily calculated by a dot product in the control shader.

In the evaluation shader, the interpolated edge curvature values are multiplied by distance to the corresponding edge. This has the goal and effect that the curvature is zero at the vertices and thus the underlying core vertex geometry is not altered, keeping the mesh scale the same. Finally, the tessellated vertices are moved in the direction of the surface normal by the calculated curvature factor, multiplied by an exaggeration factor.

The details of the implementation can be checked from the shader sources: control, evaluation. Below is the result with high level of tessellation and moderate extruding factor.

It works nicely. Below is a shot with low tessellation level (i.e. only a few additional triangles are created).

By exaggerating the curvature factor, the triangles start to “pop-out”:

**Limitations**

For the algorithm to work on a given mesh, there are some constraints:

- There must be only one normal per vertex (“smooth normals”), otherwise there will be gaps as edges in adjacent triangles would be curved differently.
- There cannot be surfaces on top of other surfaces, because the bottom surface would partially or even wholly cover the geometry on top of it when bulging outwards.