informa
/
Programming
Features

Messing with Tangent Space

What is tangent space, and more important, what is its relevance to game design? In this in-depth feature, FXLabs programmer Siddharth Hegde discusses both tangent space and its practical uses, and why it is such an important consideration in many of today’s pixel shaders.

1. Assumptions

I will assume that you are familiar with some basic vector math and 3D coordinate systems. I will also use the left handed coordinate system (OpenGL uses the right handed system) throughout this document. For the pixel and vertex shaders I will assume that you understand the syntax of DirectX shaders. Many people seem to find assembly shaders quite intimidating. The main point of this article is to help you understand the concepts and not provide code. The code is provided only to assist you in understanding the concepts.

2. Introduction

In this article I will cover what tangent space is and how to convert a point between world space and tangent space. The main reason I wrote this article was so that a person new to pixel shaders will be able to quickly understand a fairly important concept and make his own implementations.

The second section was written to demonstrate a practical use of world space to tangent space conversion. That is when you will actually understand why tangent space is so important in many of today's pixel shaders. I am sure you will find many fancier tutorials for per-pixel lighting on the internet, but this one is meant to get you warmed up on tangent space.

That being said, lets get started...

3. Tangent space

1. What is tangent space

As far as the name goes, tangent space is also known as texture space in some cases.

Our 3D world can be split up in to many different coordinate systems. You already should be familiar with some of these – world space, object space, camera space. If you aren’t I think you are jumping the gun and would strongly recommend you go back to these topics first.

Tangent space is just another such coordinate system, with it’s own origin. This is the coordinate system in which the texture coordinates for a face are specified. The tangent space system will most likely vary for any two faces.

In this coordinate system visualize the X axis pointing in the direction in which the U value increases and the Y axis in the direction in which V value increases. In most texturing only U and V change across a face tangent space can be thought of a 2D coordinate system aligned to the plane of a face.

So what about the Z axis in tangent space?
The u and v values can be used to define a 2D space, but to do calculations in a 3D world we need a Z axis as well. The Z axis can be though of as the face normal, always perpendicular to the face itself.

For now we will call this n (for normal). The value of n will always stay constant at 0 across a face.


Figure 1: Tangent space axis for a face in a plane

Take a look at Figure 1. It will help you visualize the tangent space coordinate system. It shows a quad made up of four vertices and two faces assumed to have a simple texture unwrap applied to it.

Figure 2 shows the tangent space axis for a face on a cube defined by the vertices 1, 2, 3. Once again we assume that this cube has a very simple texture unwrap applied to it.
In the bottom left you will also see the axis for the world space coordinate system marked by x,y,z.

The u, v, n axis represent the direction in which u, v, n values increase across the face, just as the x, y, z values represent the direction in which the x, y, z values increase in the world space coordinate system.

Figure 2: Tangent space axis for a face in a box


Let us take another example. In Figure 2 we try to visualize the tangent space for a top face of a box.


Why Use Tangent Space

Both, tangent space and world space represent a 3D space, just as inches and meters both represent length. If any meaningful calculations need to be done, all values must be in the same unit system or coordinate systems.

At this point you are probably wondering…

So why use tangent space in the first place? Why not just declare all positions and vectors in world space?

Certain per pixel lighting techniques and many other shaders require normals and other height information declared at each pixel point. This means that we have one normal vector at each texel and the n axis will vary for each texel. Think of it as a bumpy surface defined on a flat plane.

Now if these normals were declared in the world space coordinate system, we would have to rotate these normals every time the model is rotated even a small amount. Remember that the lights, camera and other objects also involved in these calculations will be defined in world space and will move independent of the object. This would mean thousands or even millions of object to world matrix transformations will need to take place at the pixel level. We all know matrix transformations don't come cheap.

Instead of doing this, we declare the thousands of surface normals in the tangent space coordinate system. Then we just need to transform the other objects (mostly consisting of lights, the camera, etc) to the same tangent space coordinate system and do our calculations there. The count of these objects won’t exceed 10 in most cases.
So it’s 1000000s vs 10s of calculations. Hmmm… It’s going to take a while to decide which one would be better.

Think of the box in figure 2 rotating. Even if we rotate it, the tangent space axis will remain aligned with respect to the face. Then instead of converting thousand or millions of surface normals to world space we convert tens or hundreds of light/camera/vertex positions to tangent space and do all our calculations in tangent space as required.

To add to the list of advantages of tangent space, the matrices required for the transformations can be pre calculated. They only need to be recalculated when the positions of the vertices change with respect to each other.

Hopefully you understand why we need tangent space and a transformation from world space to tangent space. If not, for now think of it as something that we just need to have and you will begin to understand when you read the second section.

3. Deriving a World space to Tangent space transformation matrix

In order to convert between coordinate system A to coordinate system B, we need to define the basis vectors for B with respect to A and use them in a matrix.

What are basis vectors?

Every n-dimensional coordinate system can be defined in terms of n basis vectors. A 3 dimensional coordinate system can be defined in terms of 3 basis vectors. A 2 dimensional coordinate system can be defined in terms of 2 basis vectors. The rule is that every vector that makes up a set of basis vectors is always perpendicular to the other basis vectors.

It’s basically one of those fancy mathematical terms given to three three vectors (in a 3D world) of unit size and perpendicular to each other. These three vectors can point in any direction as long as they are always perpendicular to each other. If you would like to visualize a set of basis vectors, think of an axis drawn in any 3D application. Look at Figure 1 & 2. The vectors that are on the axis (x, y, z) define one set of basis vectors. The vectors u, v, n also define another set.

Method of deriving world to tangent space transformation matrix

And why all the fuss over these basis vectors?
In order to achieve our goal, the first step is to define the u, v and n basis vectors in terms of the world space coordinate system. Once we do that, the rest is just a cakewalk.

From here on, I will refer to the u, v, n vectors as the Tangent (T), Bi-Normal(B) and Normal (N). Why? Because this is what they are actually called.

As I said before we need to find the T, B, N basis vectors with respect to world space. This is the first step in the process.

A simple way of thinking of this would be:
I have a vector that points in the direction in which the u value increases across a face, then what will be it’s value in the world space coordinate system.

Another way of thinking of it would be:
To find the rate of change of the u, v, n components across a face with respect to the x, y, z components of world space. The T vector is actually the rate of change of the u component with respect to world space.

Whatever way you choose to visualize it in, we will have the T, B, N vectors.
The next step is to build a matrix of the form:

 

Tx

Ty

Tz

Mwt =

Bx

By

Bz

 

Nx

Ny

Nz

Multiplying a world space position (Posws) with Mwt will give us the position of the same point in terms of tangent space (Posts)


Posts = Posws X Mwt

As mentioned before the T, B, N vectors (and all basis vectors) will always be at right angles to each other. All we need to do is derive any two vectors and the third will be a cross product of the previous two. In most cases, the face normal will already be calculated. This means we just need one more vector to complete our matrix.

Creating the tangent space matrix for a face

In this section I will take a single face and then derive Mwt for the face.

Figure 3: A sample face

Let’s assume the vertices of the face in Figure 3 have the following values

Vertex

Position (x, y, z)

Texture coordinate (u, v)

V1

(0, 20, 0)

(0, 0)

V2

(20, 20, 0)

(1, 0)

V3

(0, 0, 0)

(0, 1)


As stated in the previous section,

  1. The tangent vector (u axis) will point in the direction of the increasing U component across the face.
  2. The bi-normal vector (v axis) will always point in the direction of increasing V component.
  3. The n component is always assumed to be constant and hence is equal to the normal vector (n axis), and will point in the same direction as the face's normal.

In order to derive T, B, N calculate the difference of the u, v, n components across the face with respect to the x, y, z components of world space.

Step 1: Calculate any two edge vectors. Edge


E2-1 = V2 - V1 = (20-0, 20-20, 0-0) & (1-0, 0-0) = (20, 0, 0) & (1, 0)
E
3-1 = V3 - V1 = (0-0, 0-20, 0-0) & (0-0, 1-0) = (0, -20, 0) & (0, 0)

Edge E2-1 and E3-1 is actually the difference of x, y, z and u, v components between V1, V2 and V1, V3

Step 2: Calculate the T vector


In order to get the tangent vector T we need the rate of change the x, y, z components with respect to the u component.
Hence:

T = E2-1.xyz / E2-1.u = (20/1 , 0/1 , 0/1) = (20, 0, 0)

The T vector is then normalized. This is done because we do not need the scalar component of the T, B, N vectors.

T = Normalize(T) = (1, 0, 0)

Note: This works fine for E2-1 in this case, but what if the E2-1.u was equal to 0. There is a good chance that this can happen, depending on how your artist has mapped the mesh. If the value of E2-1.u value is 0, then we perform the same calculation on E3-1. It does not matter which edge we use, all we are looking for is the direction of increasing u component across the face.

Step 3: Calculate the Normal vector (N)


Now that we have the tangent vector, we can get the N vector. This is a standard calculation for a face normal that we normally do.

A simple cross product of the edge vectors E2-1 and E3-1 can get us the face normal.

N = CrossProduct(E2-1 , E3-1) = CrossProduct ( (20, 0, 0) , (0, -20, 0) ) = (0, 0, -400)
And just as with the T vector, N is also normalized.

N = Normalize(N) = (0, 0, -1)

Step 4: Calculate the Bi-Normal vector (B)


As mentioned earlier the T, B, N vectors are always at right angles to each other, hence we can derive the third from by just doing a cross product of the previous two. Therefore

B = CrossProduct ( T , N ) = CrossProduct ( (1, 0, 0) , (0, 0, -1) ) = (0, 1, 0)

Step 5: Build Mwt from T, B, N


Now that we have T, B, N, we can build Mwt

 

Tx

Ty

Tz

=

1

0

0

Mwt =

Bx

By

Bz

0

1

0

 

Nx

Ny

Nz

0

0

-1

And volla! There we have our matrix.


Creating the tangent space matrix for a vertex

Ok, so you are all excited about your new magic matrix. But if you haven’t realized yet, there is no way we can send face data to the graphics card. When we draw a face on the screen, we create vertices in a certain order to define a face, but there is no real face data.
So we have one last step. Splitting up the transformation matrices that we generated for each face, to each vertex that defines the face. Some of you who have generated vertex normals will be familiar with some techniques to do this. You can use any of these methods for the T, B vectors just as you do for the N vector. For the rest I will describe a basic technique here. You can always use more complex methods if the need calls for it.

A fairly simple and common method is to calculate the average of the vectors from all the faces that share that vertex.
Example:
Taking the quad from Figure 1, lets say we have generated the Mwt matrix for the two faces. Lets call the two Mwt matrices M1wt for Face1,2,3 and M2wt for Face2,4,3.

In order to calculate T, B, N for vertex 1, we do as follows


V1.Mwt.T = M1wt.T / 1
V1.M
wt.B = M1wt.B / 1
V1.M
wt.N = M1wt.N / 1

We divide by 1 because V1 is used only by 1 face.

T, B, N values for vertex 2 are as follows


V2.Mwt.T = (M1wt.T + M2wt.T) / 2
V2.M
wt.B = (M1wt.B + M2wt.B) / 2
V2.M
wt.N = (M1wt.N + M2wt.N) / 2

Here we divide by 2 because V2 is shared between 2 faces.
Note: that the T, B, N vectors need to be re-normalized after taking their average.

Putting everything together in pseudo code

I will put everything I discussed in the previous two sections together into some pseudo code. I’ve kept the pseudo code as close to C/C++ as possible, so the majority will find it easy to understand.

Declarations:

Data types:

Vector3, Vector2, Vertex, Face, Matrix3, Mesh

  • Every vertex has two sub sections. P holds the position and T holds the texture coordinates at the vertex.
  • Every face consists of three integers that index the vertex list for the three vertices that define the face.
  • Matrix3 is a 3X3 matrix which contains 3 vectors, the Tangent, Binormal and Normal.
  • Mesh is made of a list of vertices and list of faces that index this vertex list. A mesh can also hold n matrices for the n vertices it has.
  • [] signifies an array

Vector3 =
{
Float X, Y, Z ;
}
Vector2 =
{
Float U, V ;
}
Vertex =
{
Vector3 P ;
Vector2 T ;
}

Face =
{
Integer A, B, C ;
}

Matrix3 =
{
Vector3 T, B, N ;
}
Mesh =
{
Vertex VertexList[] ;
Face FaceList[] ;
Matrix3 TSMarixList[] ;
}

Predefined Functions:


Normalize(Vector3) - Returns a normalized vector
Cross(Vector3, Vector3) - Returns the cross product of two vectors

Standard vector arithmetic is assumed to work on a per component / sub component level.

Pseudo Code:

TangentSpaceFace

Description: The TangentSpaceFace function calculates the tangent space matrix for a Face F and returns a 3X3 tangent space matrix for that face.
Note: In order to keep things simple, I have not taken care of special cases where the difference between two vertices will result in the u component being 0.

Matrix3 TangentSpaceFace(Face F, Vertex3 VertexList[])
{
Matrix3 ReturnValue ;

Vertex E21 = VertexList[F.B] - VertexList[F.A] ;
Vertex E31 = VertexList[F.C] - VertexList[F.A] ;

ReturnValue.N = Cross(E21.P, E31.P) ;
ReturnValue.N = Normalize(ReturnValue.N) ;

ReturnValue.T = E21.P / E21.T.u ;
ReturnValue.T = Normalize(ReturnValue.T) ;

ReturnValue.B = Cross(ReturnValue.T, ReturnValue.N) ;

return ReturnValue ;
}

TangentSpaceVertex
Description: This function calculates the tangent space matrix for a vertex with index position VertexIndex based on the list of faces that index this vertex from FaceList. FaceMatrix is an array of tangent space matrices for the face list passed in.

Matrix3 TangentSpaceVertex(Integer VertexIndex, Face FaceList[], Matrix3 FaceMatrix[])
{
Integer Count = 0 ;
Integer i = 0 ;
Matrix3 ReturnValue = ((0, 0, 0), (0, 0, 0), (0, 0, 0)) ;

ForEach ( Face in FaceList as CurrFace )
{
If(CurrFace.A == VertexIndex or CurrFace.B == VertexIndex or CurrFace.C == VertexIndex)
{
ReturnValue.T += FaceMatrix[i].T ;
ReturnValue.B += FaceMatrix[i].B ;
ReturnValue.N += FaceMatrix[i].N ;
++Count ;
}

++i ;
}

if (Count > 0)
{
ReturnValue.T = ReturnValue.T / Count ;
ReturnValue.B = ReturnValue.B / Count ;
ReturnValue.N = ReturnValue.N / Count ;

ReturnValue.T = Normalize(ReturnValue.T) ;
ReturnValue.B = Normalize(ReturnValue.B) ;
ReturnValue.N = Normalize(ReturnValue.N) ;
}

return ReturnValue ;
}

TangentSpaceMesh
Description: This calculates the tangent space matrices for the vertices in the Mesh M

TangentSpaceMesh(Mesh M)
{

Matrix3 FaceMatrix[M.FaceList.Count()] ;
Interger i ;

ForEach Face in M.FaceList as ThisFace
{
FaceMatrix[i] = TangentSpaceFace(ThisFace, M.VertexList) ;
}

For(i=0; i {
M.TSMarixList[i] = TangentSpaceVertex(i, M.FaceList, FaceMatrix) ;
}

}


4. Problems in practice

And it all comes crashing down…

So, that was all theory, but generally when you take something from theory and implement it in practice you end up with some problems. The same goes for the tangent space generation process.

Most problems arise from the way texture coordinates may be applied for an object during the texture unwrap stage. In most cases the artist would have done this to save on texture memory. The best solution is to him to fix the texture mapping at his end, rather than you write special cases for each object. In any case I have taken some commonly faces problems and explained how you can fix them. Most these solutions are less than perfect and should be avoided.

Upside down mapping

Take a look at Figure 4. It has two adjacent faces defined by the vertices 1,2,3 and 2,6,3 with inverted v channels. This will cause the lighting calculations in a per-pixel lighting shader to give wrong output.

Latest Jobs

Sucker Punch Productions

Bellevue, Washington
08.27.21
Combat Designer

Xbox Graphics

Redmond, Washington
08.27.21
Senior Software Engineer: GPU Compilers

Insomniac Games

Burbank, California
08.27.21
Systems Designer

Deep Silver Volition

Champaign, Illinois
08.27.21
Senior Environment Artist
More Jobs   

CONNECT WITH US

Register for a
Subscribe to
Follow us

Game Developer Account

Game Developer Newsletter

@gamedevdotcom

Register for a

Game Developer Account

Gain full access to resources (events, white paper, webinars, reports, etc)
Single sign-on to all Informa products

Register
Subscribe to

Game Developer Newsletter

Get daily Game Developer top stories every morning straight into your inbox

Subscribe
Follow us

@gamedevdotcom

Follow us @gamedevdotcom to stay up-to-date with the latest news & insider information about events & more

Figure 4: Six faces of a plane. Two with the texture mapping straight and two with the mapping inverted on the v channel