Ray Marching volumetric rendering generates strange noise

Hi,


I've been trying to implement a volumetric rendering algorithm based on the Ray Marching technique.
I have been trying to implement something very simple, as in the following tutorial:
http://graphicsrunner.blogspot.com.br/2009/01/volume-rendering-101.html

My ultimate goal is to load medical (DICOM) data and display it in Unity... Unfortunately, I am just a beginner in shader programming, and although I've had some interesting results lately, they are still glitchy, so I hope you guys can help me find what am I doing wrong here...


Okay, so my approach follows the idea of the tutorial I've linked above: a multi-pass shader, where I render the back-faces and the front faces of a cube, storing the positions of the generated fragments in each pass to calculate the "rays" that intercept that cube. Then I use these rays and their orientations to sample a 3D texture and perform the "Ray Marching" algorithm to try to generate the final image "inside" the cube.

Things are working, but not perfectly: my redering generates some artifacts. I am generating a 3D texture of size 128x128x128, it is all black and inside of it there is a 64x64x64 white cube... Here's the generating code:

Code (CSharp):

1. using UnityEngine;
2.  
3. public class Texture3DTester : MonoBehaviour {
4.     public static Texture3D GenerateProceduralTexture( int texSize )
5.     {
6.         Texture3D generatedTex = new Texture3D(texSize, texSize, texSize, TextureFormat.RGBA32, false);
7.  
8.         Color[] texColors = new Color[texSize * texSize * texSize];
9.         for (int z = 0; z < texSize; z++)
10.         {
11.             for (int y = 0; y < texSize; y++)
12.             {
13.                 for (int x = 0; x < texSize; x++)
14.                 {
15.                     if (x < 64 && y < 64 && z < 64)
16.                     {
17.                         float intensity = 1;
18.                         texColors[x + y * texSize + z * texSize * texSize] = new Color(intensity, intensity, intensity, x / (float)texSize);
19.                     }
20.                     else
21.                         texColors[x + y * texSize + z * texSize * texSize] = new Color(0, 0, 0, 0);
22.                 }
23.             }
24.         }
25.         generatedTex.SetPixels(texColors);
26.         generatedTex.Apply();
27.         return generatedTex;
28.     }
29.  
30.  
31.  
32.  
33.  
34.     void Start () {
35.         // Generate texture
36.         const int texSize = 128;
37.         Texture3D volumetricTex = GenerateProceduralTexture(texSize);
38.  
39.         // Update the renderer/shader/material with the loaded data
40.         var renderer = this.GetComponent<MeshRenderer>();
41.         Shader shader = renderer.material.shader;
42.  
43.         renderer.material.mainTexture = volumetricTex;
44.     }
45. }
46.  

Then I wrote a shader to do a very basic rendering of this on screen, based on the multi-pass idea I've described earlier:

Code (CSharp):

1. Shader "Vinicius/Volumetric Ray Marching"
2. {
3.     // SHADER PROPERTIES
4.     Properties
5.     {
6.         /** Main 3D texture (generated via C# script). */
7.         [NoScaleOffset] _MainTex ("3D Texture (DICOM)", 3D) = "white" {}
8.     }
9.    
10.    
11.     // SHADER DEFINITION
12.     SubShader
13.     {
14.         Tags { "Queue" = "Transparent" }
15.        
16.         // FIRST PASS: used to render the back-faces of a cube defined with minimum bounds (-0.5,-0.5,-0.5) and
17.         // maximum bounds (+0.5,+0.5,+0.5) -- bounds given in object-space. These coordinates are shifted to
18.         // 3D texture coordinates ranging from (0,0,0) to (1,1,1). Resulting fragments store these shifted
19.         // coordinates, which are used to identify the point where a ray "leaves" the cube which represents the 3D volumetric data being
20.         // rendered, and is used to sample and render its associated 3D texture in a Ray-Marching procedure.
21.         // The point where the ray "enters" the cube will be calculated in this shader's third pass.
22.         Pass
23.         {
24.             // Cull front: only back faces will be drawn in this pass
25.             Cull Front
26.            
27.             CGPROGRAM
28.             #pragma vertex vert
29.             #pragma fragment frag
30.            
31.             #include "UnityCG.cginc"
32.  
33.             // Fragment shader's input
34.             struct fragmentInput
35.             {
36.                 float4 clipPos : SV_POSITION;    // The clip-space position (standard output of the vertex shader).
37.                 float4 rayHitBack : TEXCOORD1;   // The position where the ray "hits" the back of the cube, converted in 3D texture coordinates.
38.             };
39.            
40.             // Vertex shader
41.             fragmentInput vert ( float4 vertexPos : POSITION )
42.             {
43.                 fragmentInput output;
44.                 output.clipPos = UnityObjectToClipPos( vertexPos );
45.                 output.rayHitBack = (vertexPos / vertexPos.w) + 0.5;   // converts object-space coordinates X, Y and Z from range [-0.5,+0.5] to range [0,1], so that we can use them as 3D texture coordinates
46.                 return output;
47.             }
48.  
49.             // Fragment shader
50.             fixed4 frag ( fragmentInput fragmentData ) : SV_Target
51.             {
52.                 // Store the texture coordinates of the ray's "far-hit point" in each fragment...
53.                 // In future passes we'll access this information after calculating the "near-hit point" to define the ray
54.                 // that crosses the cube and is used to sample the 3D texture and render the volumetric data.
55.                 return fragmentData.rayHitBack;
56.             }
57.             ENDCG
58.         }
59.        
60.         // Second pass: store the contents we've drawn on the first pass in a variable (sampler2D) whose name is specified below.
61.         GrabPass { "_VolumetricBackHitPoint" }
62.        
63.         // Third pass: draw the front-faces of the cube and convert each fragment to a 3D texture coordinate, just like we did in the first pass...
64.         // This coordinates will be the point where the ray "enters" the cube. For each fragment, we can access the "_VolumetricBackHitPoint" and check the corresponding
65.         // fragment generated in this texture to discover the point where the ray "leaves" the cube. By having both the points where the ray "enters" and where the ray "leaves"
66.         // the cube, we can sample the texture by marching inside it, in several interactions, and build the resulting fragment by combining the texture data as colors.
67.         Pass
68.         {
69.             // Cull front: only front faces will be drawn in this pass
70.             Cull Back
71.            
72.             CGPROGRAM
73.             #pragma vertex vert
74.             #pragma fragment frag
75.            
76.             #include "UnityCG.cginc"
77.            
78.             sampler2D _VolumetricBackHitPoint;
79.             sampler3D _MainTex;
80.  
81.             // Fragment shader's input (which is also the output of the vertex shader)
82.             struct fragmentInput
83.             {
84.                 float4 clipPos : SV_POSITION;
85.                 float4 rayHitFront : TEXCOORD1;
86.                 float4 rayHitBackUV : TEXCOORD2;
87.             };
88.            
89.             // Vertex shader
90.             fragmentInput vert ( float4 vertexPos : POSITION )
91.             {
92.                 fragmentInput output;
93.                 output.clipPos = UnityObjectToClipPos( vertexPos );     // Same as first pass' vertex shader...
94.                 output.rayHitFront = (vertexPos / vertexPos.w) + 0.5;   // Same as first pass' vertex shader...
95.                 output.rayHitBackUV = ComputeGrabScreenPos( output.clipPos );   // Unity function for computing the UV coordinates used to sample the "_VolumetricBackHitPoint" texture to retrieve the "leaving point" of a ray
96.                 return output;
97.             }
98.            
99.             bool isInsideInterval( float val, float minVal, float maxVal )
100.             {
101.                 return ( val >= minVal && val <= maxVal );
102.             }
103.  
104.             // Fragment shader
105.             fixed4 frag ( fragmentInput fragmentData ) : SV_Target
106.             {
107.                 // Copute the position where the ray "leaves" the cube, and use it with the position where the ray "enters" the cube to calculate the ray's direction
108.                 float4 rayHitBack = tex2Dproj( _VolumetricBackHitPoint, fragmentData.rayHitBackUV );   // position where the ray leaves the cube
109.                 float4 rayDirection = rayHitBack - fragmentData.rayHitFront;                           // vector from the point where the ray enters the cube to the point where it leaves the cube
110.                
111.                 // Ray Marching procedure
112.                 const int MAX_STEPS = 50;                                    // Number of iterations in the Ray Marching
113.                 float4 curPos = float4( fragmentData.rayHitFront.xyz, 0 );   // Current position we're sampling inside the 3D texture's cube
114.                 float4 step = float4( rayDirection.xyz / MAX_STEPS, 0 );     // After each iteration, we will add "step" to "curPos", so we can step a little bit further inside the 3D texture's cube
115.                 float4 result = float4(0,0,0,0);                             // Resulting pixel color
116.                
117.                 for ( int s = 0; s < MAX_STEPS; s++ )
118.                 {
119.                     // Breaks the loop if we are trying to sample a position outside of the 3D texture's cube
120.                     if ( curPos.x < 0 || curPos.y < 0 || curPos.z < 0
121.                         || curPos.x >= 1 || curPos.y >= 1 || curPos.z >= 1 )
122.                         break;
123.                    
124.                     // Sample the texture and blend it with the resulting color!
125.                     float4 curSample = tex3Dlod(_MainTex, curPos);   // Sample texture
126.                     result += (1.0f - result.a)*curSample;              // Alpha-based standard blending of the resulting pixel colorsz
127.                     if ( result.a > 0.95 )                              // Stop whenever alpha reaches a value that is too high (current fragment is too opaque already, so data behind it will probably not affect the final result at all)
128.                         break;
129.                    
130.                     // Go a little bit further inside the 3D texture cube, "marching" over our ray... but break out of the loop if we have marched outside of the cube.
131.                     curPos.xyz += step;
132.                     if ( isInsideInterval( curPos.x, 0, 1 ) == false || isInsideInterval( curPos.y, 0, 1 ) == false || isInsideInterval( curPos.z, 0, 1 ) == false )
133.                         break;
134.                 }
135.                
136.                 // Return resulting color
137.                 return result;
138.             }
139.             ENDCG
140.         }
141.     }
142. }
143.  

Then I go to Unity Editor, create a cube, place the "Texture3DTester" script on it and apply a material with my shader to that cube. The Ray Marching algorithm seems to be working fine, except for the faces of the cube which are facing from the positive to the negative values of each axis (see screenshot below)... on those faces, the shader generates some strange "noise" effect. And I don't really know why is this happening..


(White cube is drawn correctly, but some faces of the cube seem to be drawing some "noise")

The problem always appear on the same faces. I can rotate around the cube and see the inside texture correctly, unless I see them from the "troublesome" faces.. In these faces I get noise generated.


(Seen from the opposite side of the first screenshot)



So, I was wondering if somebody with more expertise could try to elucidate that for me..? Why do I get this noise? What am I doing wrong..?

 

The noise is likely a combination of your wrap clamping and float precision.

My suggestions would be to set the wrapMode on your texture to clamp, that should get rid of some of the noise when viewing from the "front".

generatedTex.wrapMode = TextureWrapMode.clamp;

Then change your test at the start of the loop from doing:
x < 0.0 || x >= 1.0
to
x < 0.0 || x > 1.0

It's a minor change, but it might prevent the noise when viewing from the "back".

As for why, when viewing from the "back" side, the ray might sometimes be starting perfectly at "1.0", which your code immediately breaks the loop at. When viewing from the "front" side you might sometimes be getting values of exactly "0.0" which with out the wrap setting set to clamp will, well, wrap, and thus you're getting the "bottom" and sides of the texture.

 

Hi bgolus,

Thank you very much for your help, it solved my problem and I was able to load and render the medical data in Unity.



Much appreciated!

 

Hey vinicius-ras,

I'am trying the same. But I do not know which data I have to extract from a DICOM-File to apply the Ray Marching. How did you solve it?

I have access to the Pixel Data with its bytes[] or HU-Values. But with the Ray Marching I use, I could not generate a 3D Model.

Thanks.