Search Unity

Unity GPU lightmapper preview

Discussion in 'Global Illumination' started by KEngelstoft, Sep 26, 2018.

  1. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    Baked LOD will not be available for the GPU lightmapper in 2019.x, you should use CPU lightmapper if you need this.
     
  2. Vagabond_

    Vagabond_

    Joined:
    Aug 26, 2014
    Posts:
    903
    @KEngelstoft i finally found the reason for getting black parts of the meshes all over the scene in all the projects i tested !

    It is the "Pushoff" value. I never thought that this might cause any issue because it is set by default to 0.0001 which seems correct to me as far as i am aware of other renderers, but it does produce issues with the CPU and GPU lightmappers !

    Note that the objects in the images below are placed around a couple of KMs away from the origin !

    Result with default tolerance of 0.0001 - incorrect ( this value is set to all the presets and that's the reason i did not get good results with any preset loaded ) Could this be a floating point precision issues with Radeon Rays API ? But it is also valid for the CPU Lightmapper !

    GPU
    upload_2019-8-16_22-23-19.png

    CPU
    upload_2019-8-16_22-29-36.png



    Result with tolerance set to 0.001 - correct - I created a custom preset and did change the value !

    GPU

    upload_2019-8-16_22-16-40.png


    CPU
    upload_2019-8-16_22-30-39.png
     

    Attached Files:

    Lars-Steenhoff likes this.
  3. Aze_

    Aze_

    Joined:
    Mar 1, 2015
    Posts:
    24
    Hi again guys,

    I've tried again to bake with GPU PLM in latest 2019.2.1f1: same error using my RTX 2080 or my GTX 1070 with any lowest setting possible. I've tried forcing to use another GPU (Intel Graphics) to render the Editor scene; it doesn't help.

    I still the same error:

    Code (CSharp):
    1. (Filename: C:\buildslave\unity\build\Editor/Src/GI/Progressive/OpenCL/OpenCLCheck.cpp Line: 134)
    2. OpenCL Error: 'GetBuffer(kRRBuf_lightRayIndexToPathRayIndexBuffer).EnqueueClearBuffer(openCLState)' returned -4 (CL_MEM_OBJECT_ALLOCATION_FAILURE)!
    3. OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_NDRANGE_KERNEL on GeForce RTX 2080 (Device 0).
    It works perfecly fine on 2019.1.14f1.

    Any ideas to help with this? I've got latest nVidia drivers; should I roll back the one recommended in the very first post? Seems like a pretty old driver so haven't tried that yet.

    Thanks,
    Charles
     
  4. Vagabond_

    Vagabond_

    Joined:
    Aug 26, 2014
    Posts:
    903
    Hi, i am using GTX 1060 with this latest 2019.2.1f1 and it is working great.
    Have you tried the suggestion of cleaning the compute cache folder ?
    I assume you are on windows.
    Just go in that location and delete this folder - "ComputeCache": I actually deleted everything in the NVIDIA folder !
    C:\Users\"Your User Name"\AppData\Roaming\NVIDIA\ComputeCache
     
    Aze_ likes this.
  5. Aze_

    Aze_

    Joined:
    Mar 1, 2015
    Posts:
    24
    Hi, thank you for replying!

    I did indeed delete this folder; and I also deleted all NVIDIA folders in all 3 Roaming / Local / LocalLow and reinstalled entirely latest drivers and didn't help either :(

    I just discovered that in a very small level it works - but strangely it doesn't work in a real level, where it works well as soon as I downgrade back 2019.1.14f1

    I will try this again, and maybe delete parts of the level gradually to see if a specific object could create this error (or level size, but would be weird that it's only "too big for my RTX2080" even with 32x32 lightmap size and resolution set to 1 from 2019.2.1f1, whereas I can bake 2k resolution 28 on that same level on 2019.1.14f1... confused :D
     
  6. Vagabond_

    Vagabond_

    Joined:
    Aug 26, 2014
    Posts:
    903
    It does not make sense to me. It should be an issue. Unfortunately i can not think of a fix.
    Have you tried other OpenCL based applications like Blender renderer benchmarks and the Lux renderer, just to be sure that they are working ?
    Also there are some things that you can make to allow driver ( or OS - not sure ) to allow using more memory for OpenCL like this "GPU_FORCE_64BIT_PTR=1" go and search the web about "Environment Variables". I know there are some entries that can be added in order to improve memory allocation - well mostly for AMD cards, but you can go and search the web for these things !
    Also do you have the latest windows updates ?

    Another thing you can try ( i would try it for sure, but i doubt Unity would recommend this ), is to go to 2019.1 Unity install folder and copy the OpenCL.dll file and paste it to the 2019.2 folder. There might be some things changed if eventually the 19.2 version is using a dll compiled with a newer Radeon Rays SDK version ( just guessing ) ! Just be sure, back up the OpenCL.dll from the 19.2 folder just in case so you do not have to re install it if it does not work !
     
    Aze_ likes this.
  7. Aze_

    Aze_

    Joined:
    Mar 1, 2015
    Posts:
    24
    Thanks a lot for these suggestions and your lengthy reply :) I'm going to try this and keep you posted on what happens :)
     
    Vagabond_ likes this.
  8. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    Are you using the Optix denoiser? Does it work if you disable it?
     
  9. Aze_

    Aze_

    Joined:
    Mar 1, 2015
    Posts:
    24
    Hello,
    Thanks for your reply.

    I tried with Filter set to None too, and all settings to a minimum, it doesn't help unfortunately. I've tested the DLL replacement suggested above, but there isn't any OpenCL.dll it seems so I tried with various other DLLs, didn't help either.

    EDIT: It happens on multiple machines with various settings, it maybe is project-related to my project only as with other scenes it seems to work fine. I did clear all caches / delete lighting data, delete all NVIDIA folders, reinstall latest drivers, etc.

    Thanks,
    Charles
     
  10. Vagabond_

    Vagabond_

    Joined:
    Aug 26, 2014
    Posts:
    903
    Does the Project Architecture matter - i mean, if the Windows Project is set to 32 bit try to set it to 64 bit !
     
    Aze_ likes this.
  11. icefallgames

    icefallgames

    Joined:
    Dec 6, 2014
    Posts:
    25
    Unfortunately seemingly tiny changes to a level can cause the GPU lightmapper to blow up and fail. We just had an issue (with 2019.2.0f1) where suddenly a level started failing to bake, even when the settings were all turned way down. After losing hours and hours of dev time, we finally narrowed it down to too many light probe groups (someone happened to try disabling all light probe groups in the Light Probes tab of light explorer, and suddenly the bakes worked fine).
     
  12. Aze_

    Aze_

    Joined:
    Mar 1, 2015
    Posts:
    24
    Thanks, I'm going to try this and tell you how it goes :)
     
  13. Gametyme

    Gametyme

    Joined:
    May 7, 2014
    Posts:
    554
    For me it works on small scenes but anything big it falls back to cpu baker. This is using a gtx1080ti.
     
    Last edited: Aug 21, 2019
  14. drupaljoe

    drupaljoe

    Joined:
    Feb 4, 2019
    Posts:
    18
    @icefallgames @Aze_ We are also getting the same issue with the GPU lightmapper not able to bake when there are any Light Probes on the map. This seems to be a regression in the latest Unity. Is there a bug tracked for this?

    I tried to bake the same map with CPU lightmapper, and it doesn't work either and seems to run out of memory after a while.
     
  15. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    Hi, memory management for many light probe groups will land in 2020.1. For 2019.x please combine probe positions into one light probe group.
     
  16. drupaljoe

    drupaljoe

    Joined:
    Feb 4, 2019
    Posts:
    18
    @KEngelstoft Thanks! That completely fixed the issue and I baked without errors. Do you happen to know what the lack of baking on LOD Groups in GPU Lightmapper actually means? Does it mean any static object with a LOD Group component gets no indirect lighting baked?

    For any onlookers who want to save time, copy/pasting of Light Probes between groups is not trivial. To do it you have to select all the light probes, hit CTRL-C, go to the new light probe group and select one of the light probes in that group before pressing CTRL-V. We had hundreds or probes so this helped save a lot of time.
     
    KEngelstoft likes this.
  17. drupaljoe

    drupaljoe

    Joined:
    Feb 4, 2019
    Posts:
    18
    I'm also assuming lightmapping on Terrains isn't working yet? I get good results with CPU but bad with GPU.
     
  18. PhaseQuad

    PhaseQuad

    Joined:
    Jun 14, 2017
    Posts:
    29
    [PathTracer] InitializeLightmapData job with hash: b9f2fdc6080521d91d4e0858a1dc35e5 failed with exit code 2. How fix this?

    This is a nightmare. I not can create lightmaps more one month for one city map, why I have this error? On CPU I have more 20 hours for baking, on GPU I have this error.

    Please get the script, which ignore this error or something, I can pay for that, because I really need create lightmaps.
     
  19. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    Baked lightmaps for LOD is not implemented for GPU lightmapper so all the LODs will be there from the POV of the light rays, giving you severe overlap problems. We are working on a solution but for now you have to stick with CPU if you need this.
     
  20. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    Perhaps you are using an old version, there is no known issues with terrain on the latest 2019.3 version. If you find something, please report a bug. Thanks!
     
  21. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    This error code means out of memory, not GPU memory but physical main memory (RAM). Try closing other applications you have running while baking or use a lower lightmap resolution.
     
  22. drupaljoe

    drupaljoe

    Joined:
    Feb 4, 2019
    Posts:
    18
    @KEngelstoft Apologies, it looks like it might be a general lighting issue and not just related to the GPU. Thought I'd show you a screenshot anyway. Do you know why the shadows from trees would be blocky like that on the terrain?

    There is also several instances in my game of where mixed lights do not seem to render any direct light either. I'm Forward rendering and the pixel light count is at 4.
     

    Attached Files:

  23. PhaseQuad

    PhaseQuad

    Joined:
    Jun 14, 2017
    Posts:
    29
    Oookaay. Thank you. I spent one month for understanding this error... Why not write in console: Out of memory? This is standart message in other engines and programs.
    This problem is randomly, I will start baking 3-4 times and will have this error on start or on finish. 16GB ram, about 12 free and 4GB vram - how much need for normal baking?

    Okay, I have more problems:
    1) why scene is dark after baking?
    2) why some objects have a white or black spots, how fast fix this problem? I have not this problem in Unreal, so I think that problem not in meshes. Also, sometimes this spots dissapear, how it works?

    Sorry. but I really tired. After one hard month I have this poroblems again and again...Unfortunately I not can upgrade my PC at this time.
     

    Attached Files:

    Last edited: Aug 27, 2019
  24. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    Hi, I finally got an eGPU to test with today and it works with the Editor. I found that if an OpenCL device is ignored for lightmapping, for instance because it has too little memory, it will not count when specifying device index on the command line, so you have to subtract the number of ignored devices from the index yourself, so -OpenCL-PlatformAndDeviceIndices 0 3 becomes -OpenCL-PlatformAndDeviceIndices 0 2 in your case. Hope this helps.
     
    Last edited: Aug 29, 2019
  25. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    Hi @Stygian, I finally managed to get hold of a RX 5700 and reproduced your issue. The OpenCL compiler in the Navi driver is more strict than previous versions so a few fixes had to be made to our kernels. The case number is 1180454 for reference and I think the fix will land in 2019.3 or 2020.1. The Radeon VII kernel compiler is less strict so that card works with the current versions of Unity (and it has 16 GB of memory so it is a good choice for the GPU lightmapper).
     
    Lars-Steenhoff likes this.
  26. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    2,100
    Just got a Radeon VII in a sonnet e-gpu for my MacBook 2016.
    It's working fine in Mojave and unity 2019.3.0b2.

    On some scenes I get LightmapRasterize.cpp(574) in RasterTriangle - Triangle failed to raster! v1
    What does this mean?
     
  27. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,080
    Hi Lars, it means that a triangle is either very close to zero area or somehow shaped in a way that the rasterization code doesn't like. What is printed after v1, there should be a lot of indices and original triangle()...?
    Please share the offending scene / object with us in a bug report we can fix it. This is not GPU lightmapper specific code so CPU lightmapper will likely hit the same problem.
     
  28. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    2,100
    Thanks I will check for the model
     
  29. Ghost_Tales

    Ghost_Tales

    Joined:
    Sep 22, 2018
    Posts:
    5
    Hello there, recently got around to baking parts of my project. Ran into a problem. No matter how low I set the lighting settings, the light mapper fails to bake in GPU mode. I have a 980Ti, so the memory should not be an issue. The errors I get are the following,

    "OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_NDRANGE_KERNEL on GeForce GTX 980 Ti (Device 0)"

    "OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE"

    "OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_WRITE_BUFFER on GeForce GTX 980 Ti (Device 0)"

    Any suggestions or help would be appreciated.

    My Unity Version is 2019.2.5f1