HLSL trick

If you write a HLSL Shader Model 2 you know you are limited to 32 const registers. Here is a trick that helped me to save one.

Suppose you want to write in pixel shader:

 float4x4 l_Color=
   {
      tex2D(samTex0, In.Tex0),
      tex2D(samTex1, In.Tex1),
      tex2D(samTex2, In.Tex2),
      tex2D(samTex3, In.Tex3)
   };
 ...
   l_Color[g_BlendLayer] = foo(l_Color[g_BlendLayer]); 
   //OOPS! illegal syntax
   out.Color = combine (Color);

But you cannot write in ShaderModel 2 “array[index] = blah“.

Second try:

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
   switch (BlendLayer)
   {
      case 0: l_Color[0] = blended; break;
      case 1: l_Color[1] = blended; break;
      case 2: l_Color[2] = blended; break;
      case 3: l_Color[3] = blended; break;
   }
   //OOPS! illegal syntax
   ...

Of course, there is no switch/case in ShaderModel 2…

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
   switch (BlendLayer)
   {
      case 0: l_Color[0] = blended; break;
      case 1: l_Color[1] = blended; break;
      case 2: l_Color[2] = blended; break;
      case 3: l_Color[3] = blended; break;
   }
   //OOPS! illegal syntax
   ...

Same problem: “array[index] = blah“.

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
  if (0 == BlendLayer) l_Color[0] = blended; else
  if (1 == BlendLayer) l_Color[1] = blended; else
  if (2 == BlendLayer) l_Color[2] = blended; else
  if (3 == BlendLayer) l_Color[3] = blended; 
   //OOPS! Too bad
   ...

Although this code is syntactically correct, it is wrong: condition are bad, and nested ifs are even worse (just look on disassembly). Too much instructions.

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
  if (0 == BlendLayer) l_Color[0] = blended; 
  if (1 == BlendLayer) l_Color[1] = blended; 
  if (2 == BlendLayer) l_Color[2] = blended; 
  if (3 == BlendLayer) l_Color[3] = blended; 
   //Much better!
   ...

This code (same as previous, but without “else”s, is much better. But still, I get “error X5589: Invalid const register num: 32. Max allowed is 31.”

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
  if (0 == BlendLayer--) l_Color[0] = blended; 
  if (0 == BlendLayer--) l_Color[1] = blended; 
  if (0 == BlendLayer--) l_Color[2] = blended; 
  if (0 == BlendLayer--) l_Color[3] = blended; 
   //It works!
   ...

Learning the assembly code really helps.
Here is how you get .asm files from your HLSL shaders,
using Microsoft’s fxc tool:

fxc /Gfp /Zi /T ps_2_0 /Fc out.asm l:\efx\mini.fx /E PS_Test

or, to get nice HTML:

fxc /Gfp /Zi /T ps_2_0 /Cc /Fc out.asm.html l:\efx\mini.fx /E PS_Test

Advertisement

Tags: coding, HLSL, optimizations, PixelShader, shaders