Parallax Mapping with D3D and HLSL



Concept: Basic texture mapping has the problem of looking flat and not accuratly depicting the geometry which the texture displays. For instance if you want a stone wall made of protruding stones. Texture mapping isn't going to look very convincing as it will just give u a flat plane with a picture of a stone wall. What you would actually expect is for the stone to be sticking out of the wall. Parallax mapping will help give this illusion. Note this is different than normal mapping where it is a lighting trick. Parallax mapping is texture trick, however, it does require normal mapping to look decent, or else the lighting will be off. So heres three pictures showing texture mapping, normal mapping, and then parallax + normal mapping. (Err, when I took these pictures, I was forgetting to take the texture color into account, hopefully I'll update the pics someday)

-> ->
Texture Mapping -> Normal Mapping -> Parallax Mapping


How it works:


Basically you want to offset your texture coordinate by the viewing vector while taking into account the height of the surface which is trying to be simulated. You can get the height from a height map. Now note that this is only an approximation. Its a clever hack but it does have its problems such as texture swimming if you offset the texture too much.

Implementation: So how do we offset the texture coordinate by the view vector?

In Vertex Shader:
- Get eye vector in world space
- Transform eye vector into tangent space (Don't normalize the vector though)
- Pass eye vector to pixel shader

In Pixel Shader:
-Normalize view vector
-Get height value at current texture coordinate
-Offset texture by scaled and biased view vector


Shader Code:

struct VS_IN
{
	float4 pos      : POSITION;
	float3 normal   : NORMAL;
	float2 texCoord : TEXCOORD0;
	float3 tangent  : TANGENT;
	float3 binormal : BINORMAL;
};

struct VS_OUT
{
	float4 pos 	: POSITION;
	float2 texCoord : TEXCOORD0;
	float3 eyeVec   : TEXCOORD1;
};

VS_OUT ParallaxVS(VS_IN inVertex)
{
	VS_OUT outVertex;
	
	//basic stuff
	outVertex.pos  = mul(inVertex.pos, s_matWorldViewProj);
	outVertex.tex  = inVertex.tex;
		
	//get eye vector
	float3 worldVert = mul(inVertex.pos, s_matWorld);
	float3 eye       = s_matViewInv[3] - worldVert;
	
	//get tangent matrix
	float3x3 matTangent;
	matTangent[0] = normalize(mul(inVertex.tangent,  (float3x3)s_matWorld));
	matTangent[1] = normalize(mul(inVertex.binormal, (float3x3)s_matWorld));
	matTangent[2] = normalize(mul(inVertex.normal,   (float3x3)s_matWorld));

	//bring eye into tangent space
	outVertex.toEye = mul(matTangent, eye);
		
	return outVertex;
}

float4 ParallaxPS(VS_OUT inPixel)
{
	//get height value
	float height = tex2D(heightMapSampler, inPixel.tex).x;
	 
	//get look vector
	float2 look = normalize(inPixel.toEye).xy;  //only use the xy componentents
	
	//offset texture coord by the look vector times a scaled and biased height
	float2 newTC  = inPixel.tex + (look*(height*0.04 - 0.02f));
	
	//texture color
	float4 diffuse = tex2D(diffuseTextureSampler, newTC);  //note that we are using the offsetted texture coordinate now
	
	return diffuse;
}



Pretty simple and inexpensive. You can add in a light vector and a normal map there and you would get normal mapping as well. Make sure you use the offsetted texture coordinates for your normal map too.

Now, if you haven't ever done normal mapping or anything requiring tangent information, you might be wondering how to get the tangent and binormal vectors. Well there are a few tutorials out there about calculating tangent data, but if you are using DirectX with the DX mesh functions then theres an easy way.

MS provides three functions to calculate tangent data. From my experience only one of these actually gives usable results and only when given the right parameters. So anyways, heres what you need to do. Clone you mesh and add in vertex data for tangent and binormals.

	
	//create a new vert decleration
	D3DVERTEXELEMENT9 vertDecl[] = 
	{
		{ 0, 0,  D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 0 },
		{ 0, 12, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_NORMAL,   0 },
		{ 0, 24, D3DDECLTYPE_FLOAT2, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 0 },
		{ 0, 32, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TANGENT,  0 },
		{ 0, 44, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_BINORMAL, 0 },
		D3DDECL_END()
	};

	//clonedMesh is where the new mesh with the new vertexDecl will be, and new mesh will be passed into the ComputeTangent function. Compute tangent
	//might modify some vertices to get correct tangent data so it needs this mesh.
	LPD3DXMESH clonedMesh, newMesh;

	//clone your mesh with new data and store it in clonedMesh
	yourMesh->CloneMesh(D3DXMESH_VB_MANAGED, vertDecl, d3dDevice, &clonedMesh);
	
	//release old mesh
	yourMesh->Release();

	//And heres what you really need
	D3DXComputeTangentFrameEx( clonedMesh, D3DDECLUSAGE_TEXCOORD, 0, D3DDECLUSAGE_TANGENT, 0,
                               D3DX_DEFAULT, 0, D3DDECLUSAGE_NORMAL, 0,
                               D3DXTANGENT_CALCULATE_NORMALS,
			       NULL, -1, 0, -1, &newMesh, NULL );


	//don't need clonedMesh anymore
	clonedMesh->Release();
	
	//set new mesh to the one that was created
	mesh_    = newMesh;
	newMesh  = 0;
	


Thats all you need if you are using vertex declerations. Now if your for some reason using fvfs, then you need to add a few lines.


	//store tangent and binormal data in texture coordinates
	DWORD newFVF = D3DFVF_XYZ | D3DFVF_NORMAL | D3DFVF_TEX3 | D3DFVF_TEXCOORDSIZE2(0) | D3DFVF_TEXCOORDSIZE3(1) | D3DFVF_TEXCOORDSIZE3(2);
	
	//create the new mesh
	LPD3DXMESH fvfMesh;
	
	//you need to clone the fvf before you do anything else
	yourMesh->CloneMeshFVF(D3DXMESH_VB_MANAGED , newFVF, d3dDevice, &fvfMesh);
	
	//Now you need to use this fvfMesh to do the rest of what was done above with the vertexDecl method. 
	// D3DXComputeTangentFrameEx won't work with fvf stuff. It requires the correct vertexDecl, so you 
	//still have to do what was described above this code.  Also, note that if you do it this way
	//then in your input vertex struct for your vertex shader, you need to change the TANGENT and BINORMAL
	// symmantics to TEXCOORD1 and TEXCOORD2 or else DX will give you an error if run using the debug runtime.
	// Also some graphics cards (ATI >_<) don't like it if those are wrong. 
	


Refrence: Welsh, Terry, "Parallax Mapping", ShaderX3: Advanced Rendering with DirectX and OpenGL, pages 89-95.


RiZ '06