Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

Question Max allocation is 2.75 GB on a 11GB GPU

Discussion in '2019.3 Beta' started by Weidz_, Jul 10, 2019.

  1. Weidz_

    Weidz_

    Joined:
    Feb 15, 2018
    Posts:
    41
    I have issues trying to use the GPU lightmapper in HDR,¨I have been trying for a few months now, with each new update but always fail, whatever I try to bake that is either a bit large or has more tha just a few meshes will automatically failback to CPU with the following message;

    OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_NDRANGE_KERNEL on GeForce RTX 2080 Ti (Device 0).

    Or sometimes

    OpenCL Error. Falling back to CPU lightmapper. Error callback from context: Max allocation size supported by this device is 2.75 GB. 3.00 GB requested.

    Thing is, this is a 11GB GPU, how can it be restricted to so little VRAM ?
     
  2. Kuba

    Kuba

    Moderator

    Joined:
    Jan 13, 2009
    Posts:
    416
    Hey @Weidz_,

    The restriction is for a single allocation, not for the total memory used by the GPU lightmapper. Are you perhaps using a 4k atlas size?

    Cheers!
     
  3. Weidz_

    Weidz_

    Joined:
    Feb 15, 2018
    Posts:
    41
    I am trying to, indeed
     
  4. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,366
    Hi! We are working on reducing the memory footprint of the GPU lightmapper. For now, please consider using 2K lightmap resolution or use a 16GB or higher graphics card.
     
  5. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,443
    seems like all lightmap sizes fall back to cpu.. (tested on 2018.4.x, and 2020.1.0a8, using gtx1080, rtx2080ti)

    OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE
    OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_NDRANGE_KERNEL on GeForce RTX 2080 Ti (Device 0).
     
    P_Jong likes this.
  6. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,366
    Perhaps you have many Light Probe Groups in the scene (in the current versions, memory usage will increase). This will be fixed in a future alpha version of 2020.1.
     
  7. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,443
    still fails on 2020.1.0a9..
    *although seems to bake if disable 9 out of the 10 light probe groups.. **nevermind, if failed halfway..
    upload_2019-10-23_11-25-43.png

    tested with this scene, adjusted all bake settings to quite minimum.
    https://assetstore.unity.com/packages/3d/environments/urban/archvizpro-interior-vol-6-120489

    2020.1.0a9 release notes mentioned these:
    • GI: Improved memory allocation strategy for GPU lightmapper on Nvidia GPUs (Windows only).
    • GI: Reduced GPU memory usage when baking lighting with the GPU lightmapper by using stackless BVH traversal.
    • GI: Reduced memory usage for light probe occlusion baking when baking with the GPU lightmapper.
    • GI: Compressed transparency textures for GPU lightmapper, 75% memory reduction for transparency textures by using rgba32 instead of floats.

    some of the errors received with:
    Code (CSharp):
    1. OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_OUT_OF_RESOURCES error executing CL_COMMAND_READ_BUFFER on GeForce GTX 1080 (Device 0).
    2. OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_OUT_OF_RESOURCES
    3. OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_NDRANGE_KERNEL on GeForce GTX 1080 (Device 0).
    4. OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE
    5. OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_COPY_IMAGE on GeForce GTX 1080 (Device 0).
    6. OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_READ_IMAGE on GeForce GTX 1080 (Device 0).
     
    Last edited: Oct 23, 2019
  8. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,366
    Yes, some improvements landed already but there are more improvements coming, we are not quite there yet... I have a feature in flight for 2020.x that makes it unnecessary to disable the Light Probe groups. Another one makes larger lightmaps viable.

    We are testing on the archviz6 scene ourselves and I know that the scene in the Editor itself takes up quite a lot of memory. A short term solution: if you have a second GPU you can run the Editor on one and the GPU lightmapper on the 2nd. This way there is enough memory for the lightmapper. Details on how to do this can be found here https://forum.unity.com/threads/progressive-gpu-lightmapper-preview.561103/unread
     
    mgear and konsic like this.
  9. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    It seems to me a 4gb card cannot bake out a 4096 lightmap. Why not dice up the problem so it's tackled with multiple 512 lightmaps, then merge them after? Is this too difficult a job for Unity or is the answer simply to upgrade my 980 (which I can't afford to do) ?
     
    forestrf and Tudor like this.
  10. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,443
    ok, gpu bake works now (for archviz6 scene) using 2020.1.a15

    @KEngelstoft what mainly affects to that baking performance mrays value?

    (i mean if someone is getting 500mrays with 2070, seems that the scene complexity matters - but then again, one would expect to shoot "max amount" of rays from the device, regardless of what you are shooting at?)

    upload_2019-12-11_17-56-31.png
     
  11. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    @mgear I've upgraded to 2070S and the amount of rays will vary depending on options chosen.
     
  12. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,443
    tested with super simple scene and pretty minimal settings, still not going much over 220mrays.. i guess i'll test updating driver next.

    Also baking still seems to fail, if cancel bake, then bake again, errors - need to restart unity to make it work.
    (didnt happen with lower values though)

    OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE

    OpenCL Error. Falling back to CPU lightmapper. Error callback from context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_NDRANGE_KERNEL on GeForce RTX 2080 Ti (Device 0).
     
  13. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Yep I used to restart if it fell back to CPU or it'd stay on CPU.
     
  14. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,366
    We are working on this.
     
  15. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,366
    Performance is impacted by geometric scene complexity, transparency texture count / resolution, lightmap resolution and spatial coherence and many other factors. It is very scene dependent.
     
    P_Jong and mgear like this.
  16. Immersive-Matthew

    Immersive-Matthew

    Joined:
    Mar 24, 2020
    Posts:
    137
    Nice to see this issue being solve. I am wondering if there is a May 2020 update for this as CPU versus my 1080Ti is really slowing down development?
     
  17. Tudor

    Tudor

    Joined:
    Sep 27, 2012
    Posts:
    150
    So is it so hard for Unity to handle in 8 months or...? Same gpu mem problem in latest unity. Why is it that in 1 year Mr. F has singlehandedly made the Bakery light baking asset that just works and puts every single iteration of Unity's lighting to shame, and you still can't tile your renderer? I myself wrote 2 atlassers since then.

    This is the kind of stuff this forum is riddled with that gives Unity such a bad name. 8 months come on, how many countries do you have offices in? How many people? And I can't even bake a quick demo scene on the gpu without an hour of reverse engineering unity and getting to a bad compromise.

    "Progressive GPU" has been in "Preview" for how many YEARS now?
     
    Last edited: Aug 23, 2020
  18. sky_is_tumbling

    sky_is_tumbling

    Joined:
    Feb 26, 2018
    Posts:
    8
    it has literally been 2+ years and this STILL is happening, what the hell is going on unity??
    progressive has been in this unfinished state for almost half a decade already and it's still suffering from most nonsensical problems.

    i'm over here trying just to bake a small scene and i'm having to deal with all of this stuff like searching up how to unlock open cl max memory JUST for a lightmap, it's insane.
     
    Immersive-Matthew likes this.
  19. KEngelstoft

    KEngelstoft

    Unity Technologies

    Joined:
    Aug 13, 2013
    Posts:
    1,366
    Lightmap space tiling was added in 2021.2.0a19, this will make baking 4K lightmaps use much less memory.
     
  20. sky_is_tumbling

    sky_is_tumbling

    Joined:
    Feb 26, 2018
    Posts:
    8
    it doesn't really change much as the lightmapper still uses only a quarter of your gpu memory, i have a 13 GB RTX 2060 GPU and it always says it runs out of memory despite not using even half of all the provided memory.
     
    Weidz_ likes this.
  21. KevinZaWarrior

    KevinZaWarrior

    Joined:
    Jun 1, 2017
    Posts:
    1
    This is still an issue in 2023
    I have a 3060ti that is only using a few hundred megabytes when I have a 8gb gpu
     
  22. Da_Drood

    Da_Drood

    Joined:
    Mar 30, 2021
    Posts:
    4
    The issue is still there in 2024, with Unity 2022.3 using a rtx 2080ti. Seriously?