Taking a screenshot with depth information

Good day everyone,
this is my first post in the Unity Community forums and I hope it is respecting the rules I have absorbed so far.
I am open for all kinds of constructive advice. Since I am a beginner in Unity I do not know if this post
is in the correct section, please move it, if it is not.
Now, my endeavor is to take a screenshot which contains the depth information of a camera's view.

Open questions:
Do I need a shader for that?
(My knowledge about shaders is very limited.)

According to the forum entry
http://answers.unity3d.com/questions/877170/render-scene-depth-to-a-texture.html
I need a shader.
But as far as I understood the depth is a byproduct that can be accessed when the frame is rendered by a camera so why do I need a shader?

Do I get something fundamentally wrong?
(Especially with regards to my code.)

Why is the screenshot I take "blank"?
(See provided picture material.)

The following code snippet shows what I coded so far:

Code (CSharp):

1.  
2. using System.Collections;
3. using System.IO;
4. using UnityEngine;
5.  
6. public class DepthTextureScreenShot : MonoBehaviour
7. {
8.  
9.     public Camera camera;
10.  
11.     public int captureWidth = 1920;
12.     public int captureHeight = 1080;
13.     public enum Format { RAW, JPG, PNG, PPM };
14.     public Format format = Format.PNG;
15.     public string folder;
16.  
17.     private Rect rect;
18.     private RenderTexture renderTexture;
19.     private RenderTexture depthTexture;
20.     private Texture2D depthScreenShot;
21.  
22.     private int counter = 0; // image #
23.  
24.     void LateUpdate()
25.     {
26.  
27.         if (Input.GetKeyDown("o"))
28.         {
29.            
30.             rect = new Rect(0, 0, captureWidth, captureHeight);
31.             depthScreenShot = new Texture2D(captureWidth, captureHeight, TextureFormat.ARGB32, false);
32.             renderTexture = new RenderTexture(captureWidth, captureHeight, 0);
33.             depthTexture = new RenderTexture(captureWidth, captureHeight, 24, RenderTextureFormat.Depth);
34.            
35.             RenderTexture.active = depthTexture;
36.             Debug.Log(depthTexture.IsCreated());
37.             camera.depthTextureMode = DepthTextureMode.Depth;
38.             camera.SetTargetBuffers(depthTexture.colorBuffer, depthTexture.depthBuffer);
39.             camera.targetTexture = depthTexture;
40.             camera.Render();
41.  
42.             depthScreenShot.ReadPixels(rect, 0, 0);
43.             depthScreenShot.Apply();
44.        
45.             string fileName = UniqueFilename((int)rect.width, (int)rect.height, "l");
46.  
47.             WriteScreenShotToFile(depthScreenShot, fileName);
48.  
49.             //cleaning up
50.             camera.targetTexture = null;
51.             RenderTexture.active = null;
52.             Destroy(depthScreenShot);
53.         }
54.     }
55.  
56.  
57.     void WriteScreenShotToFile(Texture2D screenShot, string filename)
58.     {
59.         // pull in our file header/data bytes for the specified image format (has to be done from main thread)
60.         byte[] fileHeader = null;
61.         byte[] fileData = null;
62.         if (format == Format.RAW)
63.         {
64.             fileData = screenShot.GetRawTextureData();
65.         }
66.         else if (format == Format.PNG)
67.         {
68.             fileData = screenShot.EncodeToPNG();
69.         }
70.         else if (format == Format.JPG)
71.         {
72.             fileData = screenShot.EncodeToJPG();
73.         }
74.         else // ppm
75.         {
76.             // create a file header for ppm formatted file
77.             string headerStr = string.Format("P6\n{0} {1}\n255\n", rect.width, rect.height);
78.             fileHeader = System.Text.Encoding.ASCII.GetBytes(headerStr);
79.             fileData = screenShot.GetRawTextureData();
80.         }
81.         new System.Threading.Thread(() =>
82.         {
83.             // create file and write optional header with image bytes
84.             var f = System.IO.File.Create(filename);
85.             if (fileHeader != null) f.Write(fileHeader, 0, fileHeader.Length);
86.             f.Write(fileData, 0, fileData.Length);
87.             f.Close();
88.             Debug.Log(string.Format("Wrote screenshot {0} of size {1}", filename, fileData.Length));
89.         }).Start();
90.  
91.     }
92.  
93.     private string UniqueFilename(int width, int height, string camPointOfView)
94.     {
95.         // if folder not specified by now use a good default
96.         if (folder == null || folder.Length == 0)
97.         {
98.             folder = Application.dataPath;
99.             if (Application.isEditor)
100.             {
101.                 // put screenshots in folder above asset path so unity doesn't index the files
102.                 var stringPath = folder + "/..";
103.                 folder = Path.GetFullPath(stringPath);
104.             }
105.             folder += "/screenshots";
106.  
107.             // make sure directory exists
108.             System.IO.Directory.CreateDirectory(folder);
109.  
110.             // count number of files of specified format in folder
111.             string mask = string.Format("depth_{0}x{1}*.{2}", width, height, format.ToString().ToLower());
112.             counter = Directory.GetFiles(folder, mask, SearchOption.TopDirectoryOnly).Length;
113.         }
114.  
115.         // use width, height, and counter for unique file name
116.         var filename = string.Format("{0}/depth_{1}x{2}_{3}_{4}.{5}", folder, width, height, counter, camPointOfView, format.ToString().ToLower());
117.  
118.         // up counter for next call
119.         ++counter;
120.  
121.         // return unique filename
122.         return filename;
123.     }
124. }
125.  

Thank you for your time!

Kind regards
BL

 

Check this out.
https://github.com/unity3d-jp/FrameCapturer

That's setup to capture the gbuffers when using deferred rendering, but it might help you on the right path.

The first issue you're running into is just rendering the camera and copying the contents of the render texture won't get you what you want. Part of the problem is Unity doesn't provide a way to grab the depth directly, so setting up a camera and rendering out will only ever get you the color information. The way people work around this is to render the depth value to the color with a replacement shader pass, but that's not necessary here. Unity handles rendering the scene depth and creating a texture of it for you. The only thing you need to do is grab a copy of that depth texture. Setting the depthTextureMode on the camera tells Unity to generate that texture for you.

If you look at that frame capture tool I linked to above, you'll see it uses command buffers. Command buffers let you run bits of code at different times in the rendering pipeline. In this case you'd want to enable depth rendering on the main camera, then use a command buffer with the event CameraEvent.AfterDepthTexture to run a shader that copies _CameraDepthTexture to your own render texture, then save that to a texture.


Next problem is the depth texture is not linear, and is a 32 bit float. The range is nicely a 0.0 to 1.0 range, but it goes from the camera's near clip plane to the far clip plane and 0.5 is not half way between the near and far. Saving to a png or jpg is going to loose a lot of data since it only has 8 bits of precision. You likely want to save to at least an exr file, though that's still only 16 bit. Saving to raw is an option, but may be a hassle to use. You may want to transform the depth from non-linear to linear, and rescale that linear depth to a range you care about. That would all need to be done in the shader you use the copy the depth with.

 

is there a known way to do that in the HDRP? I'm trying for a while now to capture anything else than a single frame through the toolbox, but I'd like to have a depth capture as well as a 360° capture (not together), but the hdrp doesnt seem to like any extra components requesting capture data from the camera.

 

For anyone in the future that lands here from a google search, you may want to know about this
https://docs.unity3d.com/Packages/com.unity.simulation.capture@0.0/manual/index.html