Bug WebGL headless screenshots not rendering properly

Good evening,

I have been trying to run a Unity WebGL build in headless mode (through puppeteer) while saving 'screenshots' of the game, but the camera rendering doesn't seem to be working. The resulting images are all black.

It works as expected when not in headless mode (but still WebGL).
It also works properly in standalone builds (e.g., windows, mac), through -batchMode.

Here's the faulty code:

Code (CSharp):

1.  
2. // Problem seems to be in the following 2 lines
3. RenderTexture.active = camera.targetTexture;
4. camera.Render();
5.  
6. // same dimensions, both in headless and not headless
7. Debug.Log("CAMERA TARGET TEXTURE WIDTH: " + camera.targetTexture.width);
8. Debug.Log("CAMERA TARGET TEXTURE HEIGHT: " + camera.targetTexture.height);
9.  
10. tempTexture2D = new Texture2D(camera.targetTexture.width, camera.targetTexture.height, TextureFormat.RGB24, false);
11. tempTexture2D.ReadPixels(new Rect(0, 0, camera.targetTexture.width, camera.targetTexture.height), 0, 0);
12. tempTexture2D.Apply();
13.                
14. // RGBA(0.000, 0.000, 0.000, 1.000): totally black, when in WebGL headless mode. Works fine otherwise.
15. Debug.Log(tempTexture2D.GetPixels(100, 100, 1, 1)[0].ToString());
16.  
17. // Encode texture into JPG
18. byte[] bytes = tempTexture2D.EncodeToJPG();
19.  
20. // byte count is almost half when in headless mode
21. Debug.Log("IMG " + frameNumber + " byte count: " + bytes.Length);
22. // save to persistentData (indexedDB in WebGL)
23. // that data is then read on client side and encoded again

The problem is on the unity side, most likely with the camera.Render because the resulting texture is all black and thus lowering the number of bytes in WebGL headless mode. Not sure if using something like xvfb-run would solve the problem, but I can't test it through MacOS anymore.

Any ideas?

Thanks!

 

I found some differences between the headfull and headless versions (respective pictures below). Which of the entries would be the problem? Can I disable hardware acceleration from my WebGL build?

 

Also just managed to get some output from unity when in headless mode.


PAGE LOG: Loading player data from data.unity3d
PAGE LOG: Initialize engine version: 2018.4.10f1 (a0470569e97b)
PAGE LOG: Creating WebGL 2.0 context.
PAGE LOG: Renderer: WebKit WebGL
PAGE LOG: Vendor: WebKit
PAGE LOG: Version: OpenGL ES 3.0 (WebGL 2.0 (OpenGL ES 3.0 Chromium))
PAGE LOG: GLES: 3
PAGE LOG: EXT_color_buffer_float GL_EXT_color_buffer_float EXT_float_blend GL_EXT_float_blend EXT_texture_filter_anisotropic GL_EXT_texture_filter_anisotropic OES_texture_float_linear GL_OES_texture_float_linear WEBGL_compressed_texture_etc GL_WEBGL_compressed_texture_etc WEBGL_compressed_texture_etc1 GL_WEBGL_compressed_texture_etc1 WEBGL_compressed_texture_s3tc GL_WEBGL_compressed_texture_s3tc WEBGL_debug_renderer_info GL_WEBGL_debug_renderer_info WEBGL_debug_shaders GL_WEBGL_debug_shaders WEBGL_lose_context GL_WEBGL_lose_context
PAGE LOG: OPENGL LOG: Creating OpenGL ES 3.0 graphics device ; Context level <OpenGL ES 3.0> ; Context handle 1
PAGE LOG: UnloadTime: 0.340000 ms
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_OPERATION : glFramebufferTexture2D: <- error from previous GL command
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_OPERATION : GetShaderiv: <- error from previous GL command
PAGE LOG: WebGL: INVALID_OPERATION: renderbufferStorageMultisample: samples out of range
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glClear: framebuffer incomplete
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glDrawElements: framebuffer incomplete
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glDrawArrays: framebuffer incomplete
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glBlitFramebufferCHROMIUM: framebuffer incomplete

 

Running chromium with --use-gl=swiftshader seems to be better, but it still doesn't solve the issue:


To be clear, the arguments I'm passing to chromium are the following:

args:[
'--headless', // if headfull, unity3D will still show black when using swiftshader.
'--hide-scrollbars',
'--mute-audio',
'--no-sandbox',
'--use-gl=swiftshader' // tested with and without
]


Could the problem be the WebGL hardware acceleration altogether? Can I disable it from my WebGL build?

This issue seems to be related with: Rendering WebGL image in headless chrome without a GPU

But that seems to be working properly, at least under MacOS: https://github.com/Apidcloud/WebGLHeadlessRendering

So I'm still assuming it has something to do with Unity3D. Even if headfull, it will become black when using swift shader. The gpu stats are the following:

 

Managed to get it working by disabling the anti-aliasing in unity's quality settings, even without using `swiftshader`.