HLSL trick

If you write a HLSL Shader Model 2 you know you are limited to 32 const registers. Here is a trick that helped me to save one.

Suppose you want to write in pixel shader:

 float4x4 l_Color=
   {
      tex2D(samTex0, In.Tex0),
      tex2D(samTex1, In.Tex1),
      tex2D(samTex2, In.Tex2),
      tex2D(samTex3, In.Tex3)
   };
 ...
   l_Color[g_BlendLayer] = foo(l_Color[g_BlendLayer]); 
   //OOPS! illegal syntax
   out.Color = combine (Color);

But you cannot write in ShaderModel 2 “array[index] = blah“.

Second try:

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
   switch (BlendLayer)
   {
      case 0: l_Color[0] = blended; break;
      case 1: l_Color[1] = blended; break;
      case 2: l_Color[2] = blended; break;
      case 3: l_Color[3] = blended; break;
   }
   //OOPS! illegal syntax
   ...

Of course, there is no switch/case in ShaderModel 2…

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
   switch (BlendLayer)
   {
      case 0: l_Color[0] = blended; break;
      case 1: l_Color[1] = blended; break;
      case 2: l_Color[2] = blended; break;
      case 3: l_Color[3] = blended; break;
   }
   //OOPS! illegal syntax
   ...

Same problem: “array[index] = blah“.

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
  if (0 == BlendLayer) l_Color[0] = blended; else
  if (1 == BlendLayer) l_Color[1] = blended; else
  if (2 == BlendLayer) l_Color[2] = blended; else
  if (3 == BlendLayer) l_Color[3] = blended; 
   //OOPS! Too bad
   ...

Although this code is syntactically correct, it is wrong: condition are bad, and nested ifs are even worse (just look on disassembly). Too much instructions.

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
  if (0 == BlendLayer) l_Color[0] = blended; 
  if (1 == BlendLayer) l_Color[1] = blended; 
  if (2 == BlendLayer) l_Color[2] = blended; 
  if (3 == BlendLayer) l_Color[3] = blended; 
   //Much better!
   ...

This code (same as previous, but without “else”s, is much better. But still, I get “error X5589: Invalid const register num: 32. Max allowed is 31.”

   ...
  float4 blended = foo(l_Color[g_BlendLayer]);
  if (0 == BlendLayer--) l_Color[0] = blended; 
  if (0 == BlendLayer--) l_Color[1] = blended; 
  if (0 == BlendLayer--) l_Color[2] = blended; 
  if (0 == BlendLayer--) l_Color[3] = blended; 
   //It works!
   ...

Learning the assembly code really helps.
Here is how you get .asm files from your HLSL shaders,
using Microsoft’s fxc tool:

fxc /Gfp /Zi /T ps_2_0 /Fc out.asm l:\efx\mini.fx /E PS_Test

or, to get nice HTML:

fxc /Gfp /Zi /T ps_2_0 /Cc /Fc out.asm.html l:\efx\mini.fx /E PS_Test
Advertisements

Tags: , , , ,

2 Responses to “HLSL trick”

  1. HLSL trick #2 « Komap’s Blog Says:

    […] Komap’s Blog Just another WordPress.com weblog « HLSL trick […]

  2. JohnJack Says:

    Good tip man. I’m aware of the limited compare functions on SM2.0 hardware, but using the comparison against zero like a switch was genius. I learnt OpenGL’s assembly, but never took any interest in Direct3D’s assembly, or I might’ve learnt this sooner.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: