Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Progressive CPU Lightmapper is SLOW

Discussion in 'General Graphics' started by John-B, Mar 17, 2022.

  1. John-B

    John-B

    Joined:
    Nov 14, 2009
    Posts:
    1,259
    I've been trying the Progressive CPU light mapping option, and it is VERY SLOW. Generating a light map for the scene using Enlighten (Deprecated), as I've done in the past, takes about 45 minutes. I don't really know how long it takes using Progressive CPU as I've not yet seen it finish. After starting and stopping several times, I let it go and walked away. I came back to my machine about 3 hours later to find the progress indicator about where it was when I left, about a 1/4 of the way finished. And while it's running, I can do pretty much nothing on my machine, everything grinds to a halt. Not the case with Enlighten. Is this normal? How much slower should I expect Progressive CPU to be compared to Enlighten? Orders of magnitude slower?

    Apparently neither of the light mappers continue working when the computer goes to sleep, so apparently walking away and letting it render is not the way to do it.

    I have a late 2015 iMac, and am using the standard shader with a normal map. The static objects in the scene all share one material. There are 4 lights in the scene. This project is in Unity 2021.1.10. I have an M1 Max Mac Studio on order, will that speed the lightmapping process up?
     
    Last edited: Mar 18, 2022
  2. UnityLighting

    UnityLighting

    Joined:
    Mar 31, 2015
    Posts:
    3,866
    First you must know that the CPU lightmapper is just for high-end cpu because of its quality
    Enlighten is faster in very low preset without Final Gather. With a high preset and Final Gather it is much slower than CPU lightmapper

    2two things will affect on your bake time:
    Indirect Samples
    Lightmap resolution


    So first use most lower lightmap resolution and use the default indirect samples and 1024 lightmap size to see the lightmap quality. If the quality was not good, so increase the lightmap resolution...
    If you had noise, so increase indirect samples

    You must increase lightmap resolution and indirect samples progressively step by step

    If you have not a high end cpu, you can only use Enlighten in low preset
     
  3. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    I've had my Mac Studio with an M1 Max running Unity 2021.2.19 (Apple Silicon) for a few weeks now, and I'm not seeing much improvement in Lightmap baking speed. In Activity monitor, when a Lightmap is baking using Enlighten, %CPU hovers around 10%, down to as low as 1%, and %GPU around 5%, occasionally bouncing up to 10-20%. I've haven't done any timed comparisons, but it seems as slow as ever.
     
  4. spiney199

    spiney199

    Joined:
    Feb 11, 2021
    Posts:
    7,606
    Is it not possible to use the Progressive GPU lightmapper on the new Mac's? Or does the lack of a dedicated GPU put a pin in that?
     
  5. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,627
    Did you try gpu lightmapper again? GPU Lightmapper had a ton of bugs on silicon macs, but I think most of them were fixed during the 2021.2 cycle.
     
  6. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    I tried Progressive GPU (Preview), the only other option available. I let it run over night, and when I came back this morning, it was right where it left off, with 3 hours (or is that 3 minutes) remaining. So apparently it still hangs if the computer goes to sleep while it's rendering.

    FWIW, in the Lighting panel, it says, "Use the Progressive Lightmapper for Baked GI in new projects. Use Enlighten Baked GI for backwards compatibility only."

    Running the Progressive GPU, it uses around 200% CPU and 30-60% GPU.
     
    Last edited: Apr 28, 2022
  7. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    I updated to 2021.3.0f1, and am trying Progressive GPU again. Indirect samples set to 128, very low Lightmap resolution. When I started baking, the initial estimate was 19 days (I didn't know there was a Background Tasks panel with time estimates). After running a while, it's down to 8 hours. That still seems excessive. Especially since it looks like very little of my machine's GPU power is being used. I assumed since GPU was in the name, it would take advantage of the the 24 graphics cores in my machine. Hopefully, it will finish this time and not freeze.

    Contrary to Unity's recommendation, I don't think the Apple silicon version of the Progressive light mapper is ready for use in new projects. I don't know how the results compare, since I've not gotten Progressive to finish, but it's not practical for me to wait all day or all night to see the results.
     
    Last edited: May 1, 2022
  8. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,627
    You mean lightmap parameters? This does nothing for progressive lightmapper. What is the actual resolution used? (number of the "Lightmap resolution" setting)
     
  9. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I have a suspicion you're not tailoring meshes well for lightmapping. Depending on the game, different meshes very much should have different lightmap settings, especially based on their size in the scene. Small objects should just be probe lit.

    This per-object care is mandatory when you do not have AAA scale massive render farms for baking and only want to use a single home computer.
     
  10. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,627
    I mean, even with a render farm, Unity can only use one computer unfortunately.
     
  11. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Yeah but my point here is... their workflow only works on renderfarms (not using Unity). So they'd have to be doing settings per mesh, to reduce time. There is no other way if using PLM.
     
  12. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,627
    I mean, I would still argue that it's more of a matter of RAM usage (not hand adjusting everything will probably lead to a ton of lightmaps eating a ton of ram) rather than lack of render farm (or equivalent power).

    The coupling of PLM and Enlighten setting super frustrates me (and the whole lightmap parameters thing is ill-designed), makes giving advice for settings really awkward. Even Unity themselves are often confused.

    @jabaro a screenshot of your lighting settings would help.
     
  13. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    Yes, I do have some tailoring and some tweaking to do. I'm just getting started with light mapping in this project, and it's been a while since I've done it. The big difference in speed between Enlighten and Progressive GPU makes Enlighten more suitable for me, so I can see the results of a change I make within an hour versus waiting a day.

    Not to mention the hanging. I've yet to have a Progressive Lightmap finish, so I don't know if taking 10x longer means it's going to be 10x better. And Enlighten is hit or miss. With Enlighten, sometimes when I walk away and come back, the Lightmap is finished, but often as not, the process is hung, not responding, and so must start over.

    Lightmap Settings.jpg
     
  14. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,627
    Having progressive updates ticked causes issues sometimes.

    Also start with a low Lightmap Resolution (like 2-3) just to see if it then finishes the bake really quick.
     
  15. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    Progressive is on the back burner right now, but now I'm having trouble with Enlighten. It's not finishing, but not hanging. I click Generate Lighting and I get a progress bar, and it shows up in the Background Tasks panel. Then a few seconds or minutes later, the task is gone. The button in the Lighting panel still says Cancel, but nothing's happening. No light maps are generated. I have to quit and restart Unity to get it going again.
     
  16. Max-om

    Max-om

    Joined:
    Aug 9, 2017
    Posts:
    499
    Get a PC with a Nvidia 3090, 24 gigs VRAM, I haven't had a scene yet that can't bake with that GPU. Though even if the hardware can bake a large scene doesn't mean it will perform well. You need to lower scale for unimportant objects, use probes for smaller objects. And most important use bake tags to tag objecs, Unity will not put close by items on same lightmap resulting in more set pass calls. You need to manually fix this with bake tags. Also put LODS on same lightmap will result in fewer set pass for far away objects.
     
  17. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    Seeing as how I just got a Mac Studio, that's not a very helpful suggestion. No reason why an M1 Max with 10 CPU Cores and 24 GPU cores shouldn't excel at a GPU-intensive task like this. When it only uses 5-10% of the GPU's capability, that suggests to me it's the software, not the hardware. Unity is not fully taking advantage my machine's power.

    It's not just speed, it's reliability. I've given up on the Progressive light mapper for now, but Enlighten has gotten flaky. This is something new, maybe since I recently updated to 2021.3.0f1. It doesn't crash or hang, it counts down the task list and appears to finish, but nothing happens, nothing is updated. The only way to make it generate a light map again is to quit/restart Unity (fortunately that process is FAST on this machine). I can only generate 1-3 light maps before having to restart. That really slows things down.
     
  18. Max-om

    Max-om

    Joined:
    Aug 9, 2017
    Posts:
    499
    24 GPU cores translates to how many cuda cores?
     
  19. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    Doesn't matter how many cores there are if they are mostly just sitting there doing nothing.
     
  20. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    20,950
    Unfortunately the best answer to how to improve the performance of the official lightmap baking systems is to not use the official lightmap baking systems but to instead pick up a copy of Bakery. The big caveat with that of course is that it's only capable of running on PCs with NVIDIA hardware.

    https://assetstore.unity.com/packages/tools/level-design/bakery-gpu-lightmapper-122218

    Better yet there is a secondary asset that you can pick up for it that allows you to preview everything in realtime.

    https://assetstore.unity.com/packages/tools/level-design/bakery-real-time-preview-164462
     
    Last edited: May 9, 2022
  21. Max-om

    Max-om

    Joined:
    Aug 9, 2017
    Posts:
    499
    i see close to 100 percent utilization of my 3090 when baking. Only problem is there is an insane overhead when starting on a new lightmap. I don't know what unity is doing but computer idles for a long time until it starts actually baking. This is a relative new behaviour, it used to be much faster between lightmaps.
     
  22. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    I just bought a new Mac, so.
     
  23. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    Interesting (to me) that the Activity Monitor app shows very different info depending on the window. Attached is a screen capture of the first 4 minutes of an Enlighten Lightmap generation. In the main Activity Monitor window, it shows Unity using very little CPU or GPU. However, CPU history shows it maxing out. Enlighten apparently doesn't make much use of those GPU cores.
    Act Monitor Enlighten.jpg
     
  24. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    20,950
    A lightmapper has to be designed to use the GPU and to my knowledge Enlighten is purely CPU.
     
    jabaro and Max-om like this.
  25. jabaro

    jabaro

    Joined:
    Aug 28, 2012
    Posts:
    28
    I've been curious to see how CPU/GPU usage compares, so I gave the Progressive light mapper another try. Progressive is the exact opposite of Enlighten, it maxes out the GPU, but barely uses the CPU cores at all. I don't know if it's the recent Unity update, or reducing the resolution and other quality settings, but it finished my first attempt in under 3 minutes! Even upping the resolution, it still finished in about 6 minutes. FWIW, the "Bake Performance" was Infinity mrays/sec!!! That's pretty fast. Compared to my previous attempts where Progressive ran for hours, then crashed or hung.

    I'm seeing some weird shadow artifacts, but I'm hoping those will go away with higher resolution/quality settings. At least now I can do some experimenting.

    Progressive GPU.jpg
     
  26. John-B

    John-B

    Joined:
    Nov 14, 2009
    Posts:
    1,259
    I just noticed, the Progressive Lightmapper on my Mac Studio is REALLY fast. Can't get faster than infinity!

    Infinity.jpg
     
  27. John-B

    John-B

    Joined:
    Nov 14, 2009
    Posts:
    1,259
    I just started a new scene, and the Progressive Lightmapper is getting progressively worse. It started out crashing Unity when it was finished rendering, now it crashes a few seconds after it starts. I have no idea if it has something to do with the scene, but for now the Progressive Lightmapper is no longer usable, at least in this project/scene.
     
  28. RevC2

    RevC2

    Joined:
    May 2, 2021
    Posts:
    48
    Hey John, can you post some screenshots or at least describe what scene you are working with? Progressive GPU lightmapper can indeed be prone to "OpenCL errors" on big scenes reverting to CPU but with preatty conservative settings you should be able at least to get decent bakes. If every scene fails to work with lightmapper maybe it's an hardware/system problem or just bad authored models/settings somewhere in the project. Where I work every day we make projects quickly as hell (1, 2 projects a month) and I still haven't found a scene that will totally not bake no matter what with good authored models, fair settings and a well mantained system. I'm asking this 'cause just yesterday, a coworker of mine couldn't bake lightmaps in the latest projects we are working on. I tried on my pc, it worked normally, everybody tested on the other machines and everything worked (little bit slow, but it baked nonetheless), just on that computer it would not bake no matter what
     
  29. Reahreic

    Reahreic

    Joined:
    Mar 23, 2011
    Posts:
    254
    I've noticed that when Unity is asked to generate UV's for meshes, it will run the CPU at max and not touch the GPU until it's finished that part of the process. There used to be a feedback message indicating something to that effect, but it appears missing nowadays. A GPU based clustering/packing solution would immediately fix 98% of the unity's lightmapper is so slow threads.


    The Guide
    I always bake out a preliminary lightmap with the absolute lowest settings to get the clustering/packing part out of the way as it's slow as hell, even on a high-end workstation CPU.

    In order to get any reasonable lightmapping results from progressive, it's mandatory to perform the following for every MeshRenderer component in the scene:
    1. Manually generate Lightmap UV's in your 3D asset program, save them to the UV2 channel. Do NOT let Unity 'Generate Lightmap UV's'! (I reject any mesh that the 3D artist hasn't generated lightmap UV's for.)
    2. Manually configure the lightmap scale. The value should be relative to the highest importance mesh [0-1] and will require lots of iterative testing to fine tune the results for even half decent clustering/packing.
    3. Use Light Probes instead of 'Contribute to Global Illumination' for all small/unimportant objects.
    4. Use Lightmap Tags to group nearby items together. (Think of this as combining similar for better culling/batching)
    5. Disable any lightmap filtering.
    Then when the above is done:
    1. Reduce Lightmap Parameters > Anti Aliasing values to their lowest for every lightmap parameter asset in he scene.
    2. Set Direct, Indirect, Environment samples to their absolute lowest values.
    3. Set bounces to [min:1, max:1]
    4. Set Lightmap Resolution to it's lowest value, increasing it iteratively with each bake until you're happy with the clustering/packing & resolution of the generated map. (This property goes hand in hand with lightmap scale & tags in getting the maps to cluster/pack nicely.)
    5. Uncheck Ambient Occlusion
    Lastly, once you are happy with the number, clustering/packing, and size of your baked maps:
    1. Set the Anti Aliasing values to your desired value for each Lightmap Parameters asset file.
    2. Set your Lighting > Lightmapper Settings as follows, then iteratively increase their values until you are happy with the bake quality:
      1. Direct Samples: 32
      2. Indirect Samples: 64
      3. Environment Samples: 16
      4. Bounces [min:2, max:3] (higher values will increase workload massively)
      5. Set lightmap filtering to:
        1. Direct Denoiser: Advanced, OpenImageDenoise, Gausian, 1
        2. Indirect Denoiser: Advanced, OpenImageDenoise, Gausian, 3
        3. Ambient Occlusion Denoiser: Advanced, OpenImageDenoise, Gausian, 2
      6. Set Ambient Occlusion as desired.
    3. Set your computer to never sleep, idle, or use any power savings modes aside from max performance.
    4. Close everything except Unity, kill all background tasks, windows scheduled tasks, network connection, Bluetooth, backup tasks, windows visual (Aero) effects, disable all but your primary monitor, set your windows screen resolution as low as you can, etc. You want the leanest possible windows runtime.
    NOTE: If you add a new lightmapped mesh to the scene or change the lightmapping status of an existing mesh you will need to re-do the clustering portion. The good news is that once clustered, progressive GPU is nice and fast results even with samples set in the 4k-16k range and resolution in the 128-256 range.

    Hopefully this helps future Googlers.

    ---

    Soapbox:
    Sadly Unity corporate is incentivized to not give a $#!t about many parts of their engine. This is especially true for lightmapping as they offload the (non-trivial) labor cost onto the Bakery Developer and get to take a cut of Bakery's sales too. FYI The bakery dev is the true MVP, it's well worth the $50.
     
    stormg_unity likes this.
  30. Max-om

    Max-om

    Joined:
    Aug 9, 2017
    Posts:
    499
    Yep, i have a 3090 and the baking is super fast per lightmap. But the time between lightmaps is insane
     
  31. Reahreic

    Reahreic

    Joined:
    Mar 23, 2011
    Posts:
    254
    It's at the point where I may lobby for a 64core Threadripper on the grounds that the $8000 price tag will be ROI'd in less than a 6 months.
     
  32. Max-om

    Max-om

    Joined:
    Aug 9, 2017
    Posts:
    499
    I have a 16 core 5950x and its slow as hell. It will be slow even if 4 times faster (if it scales linear)