Search Unity

Bug HDRP Shadows have big issues.

Discussion in 'High Definition Render Pipeline' started by jjejj87, Aug 13, 2022.

  1. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Hello, I am posting this to let the devs know of the current issues with the HDRP shadows and want to suggest some fixes. I labelled it bug as the shadows are not physically correct, but technically I don't think it is a bug, but more of a technical shortcoming.

    Please look at the following screenshot
    GI.jpg

    The screenshots are from HDRP 15.0.0, Unity2023.1.a5
    I used the latest public to show that this issue has been around since day 1 of HDRP and it still is.
    I am not trying to say the devs did something wrong, but I just want to point out that probably 99% of the devs are not aware of this issue.

    The top is with Raytraced GI, and the bottom is adjusted photoshop. The only difference is that the shadows on the lawn was brightened to be more physically correct.

    So, my issue, is that in HDRP, shadows on darker material is super dark. And while this doesn't sound like the wrong way to do it, the actual results are obviously not physically correct. So, darker the material, the darker the shadows (only on top facing shadows though, eg. ground). Btw, no terrains were used. 100% mesh.

    Here is another example

    GI2.jpg
    Again, the dark materials receive very dark shadows. The bottom screenshot is photoshop adjusted.

    Now, I have ACES enabled, and it does make it look worse. But even with it off, the shadows are much much darker than it should be. Now, I deliberately turned on Raytraced GI to demonstrate, now I am posting path traced and GI off completely.

    GI3.jpg

    We can clearly see from path tracing that GI algorythm is working perfectly and is still correct, except the dark material shows darker shadows than it should (on the lawn grass). Now look at the bottom screenshot without any GI.

    You will notice that the dark shadows on darker materials persist.

    From this we can deduce that this is not related GI.

    So, the HDRP shadow, has some issues. My guess is that they multiply the colors with shadow color and material color - hence the current result, but I think there needs to be a gradient curve to match it to reality a bit more. The rendering of shadows in relation to base material color is not physically correct. Now, I know I can change directional lighting shadow color, but it also makes shadows on brighter materials brighter, hence the issue remains. And also, it is still physically incorrect.

    Suggested Solution: Add a editable curve in the post processing so that we can manually change how much it multiplies or just fix it in shader level (this is the preferred as the current one is physically incorrect to begin with)

    Right now, it is impossible to achieve photo realism due to this issue.

    Thanks.
    So, can someone look into this?
     
    PutridEx likes this.
  2. pierred_unity

    pierred_unity

    Unity Technologies

    Joined:
    May 25, 2018
    Posts:
    433
    Hey, how does your soil material look like, especially the albedo (and potentially AO)? It looks very dark.

    Judging by the very high exposure in your first image, you seemed to have settled for a very dark soil, maybe unintentionally.

    upload_2022-8-14_0-42-4.png

    With such a dark albedo it's really not surprising you're getting these dark shadows and get unsatisfying results.

    upload_2022-8-14_1-19-16.png
    upload_2022-8-14_1-8-2.png
    upload_2022-8-14_1-22-2.png

    I've picked these images above because they don't have any particularly strong grading/tonemapping, they closely match the exposure of your time of day, and they display various types of soil. As you can see their shadows are already very dark. These photos are as photorealistic as it gets, by definition. So if you still feel their shadows are too dark, then it's only an artistic consideration; and the solution is tonemapping/grading, or even pushing the indirect lighting multipliers (though that one would not be physically "correct" anymore, though it's often a great way to deliver more pleasing indirect lighting).

    And if you compare these with your first path traced image (with ACES tonemapping I believe), you'll see the sunlit soil in these photos is much brighter than your terrain in a similar exposure, which means you're not really starting on the right foot if you want to extrapolate how bright shadows are meant to be, knowing that even photos with brighter soils already present very dark shadows.
     
  3. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    I think u are slightly missing the point. I am saying that the rate at which shadows added to the base material is off. And the screenshots were to get to the point that it is not GI but shadows alone. The above screenshots only used ACES and the texture is just a photoscanned texture from Megascans.

    Here is what Unity is doing (Top is the exact same image u posted and the bottom is made in Photoshop)
    Gi8.jpg

    To just explain in simple terms, I am going to assume that the shadow color is just added to the base texture (Probably has clamping and normalization but let's just keep it simple)

    GI9.jpg
    Let's assume that the current shadows are applied with a 1:1 curve like the top image. The x axis being the darkness of base material and y axis being the opacity of shadows.

    Now, all I am saying is that this does not seem physically correct. And it probably can be fixed by having a curve like the second one. I understand that this may be going against some math that the shadow rendering is based on, but it is clearly not correct at the moment.
     
  4. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Here, I took a first person perspective of the same scene. As you can see the ground material is not even that dark. Clearly, something is off. I've never seen anything like this in real life.
    GI10.jpg

    Here is what it should look like (Made in Photoshop)
    GI11.jpg
    Now, I can achieve similar results with very bright ground textures, but it has to be almost white, which is sort of not viable.
     
    Last edited: Aug 14, 2022
  5. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Here is another screenshot. This is raytracing + ACES only
    The ground shadows are just super dark. look at the brick table in the shadows.
    Clearly, something is off.
    GI13.jpg
    The table should be as dark as the ground or the ground should be as bright as the table. The discontinuity of shadow opacity is clearly an issue - hence why I think the way shadows are "added" to the base material needs adjusting.
     
    PutridEx and ftejada like this.
  6. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    Interesting. Can you post that texture or a screen capture of it? Also the brick texture is apparently lit correctly.
     
  7. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    Also, is that a terrain or a plane mesh?
     
  8. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Here is another one.
    I've put the white doorway in the shadows. It almost look like lighting glitch at this point.
    This is all raytraced in case u are wondering.
    GI14.jpg
     
    Last edited: Aug 14, 2022
  9. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    No terrains as terrain raytracing is not supported. All 100% mesh.

    I heard it is supported now, but when I started building this scene, it did not, so this scene does not have any terrains.
     
  10. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    I don't think I have the right to upload it as it is megascans. But it is just normal texture. Just slightly dark green texture...

    EDIT: I blurred the texture but you can see the colors.
    tex.jpg

    I can go into Unreal Engine 5 and replicate the scene.
    I bet it won't do this.
     
    Last edited: Aug 14, 2022
    pierred_unity likes this.
  11. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    In case u are still not convinced, go outside and put a black paper on the ground so that half of it is in direct sunlight and half of it in the shadows and see if it looks like Unity.

    It does not.

    A simple curve would solve this. This is a big issue really for physically correct rendering.
     
    ftejada likes this.
  12. cLick1338

    cLick1338

    Joined:
    Feb 23, 2017
    Posts:
    74
    ACES skews results way too much (in my experience it can be so agressive it crushes dark areas into black). (Per-channel tonemappers should be a thing of the past but that's a whole another discussion)

    I'd try a bit more scientific approach - line up a few planes with varying degrees of albedo; Maybe some spheres with the same materials on top; Also use neutral tonemapping and a set exposure.
     
  13. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Here is without any ACES. Same results, except it looks worse
    GI-noACES.jpg
     
    ftejada likes this.
  14. pierred_unity

    pierred_unity

    Unity Technologies

    Joined:
    May 25, 2018
    Posts:
    433
    I can see that your base exposure seems off by a couple of stops to begin with seeing how dark your sky is while you're exposing for the shade: your scene is very underexposed to begin with. So if you absolutely want to use ACES without any additional grading, it is expected that the shadows will be very dark, as ACES will crush the lower end of your range. One of the root causes here could be that you're using a fixed exposure for all these shots and expect optimal results everywhere, as you seem to be using the same exposure in many of your shot, with no room for adaptation.

    I've done a quick test scene with the texture you shared, because it raised my eyebrows to see such odd results (I've never seen this with correctly calibrated lighting/exposure/materials). I'm not seeing the extreme problems you see: the neutral tonemapper works as expected and the scene is exposed correctly with values between 10 and 13EV below, which are expected for a sunny day, with some room for darker areas. Also, my texture looks significantly brighter in the sun light than in your scene, which might indicate some import issues with some of the maps (e.g. normals, AO, metalness), on top of the core issues with exposure/ACES mentioned above.

    upload_2022-8-14_18-42-47.png

    Even with ACES on, it is nowhere near the blackness you're getting, while maintaining sufficient details in the sky.

    upload_2022-8-14_18-45-4.png

    Then it's very easy to slightly lift the shadows, if you wish to restore some details.
    upload_2022-8-14_18-45-8.png

    Alternatively, you can expose for the shadows more (while still using ACES), with a slightly lower exposure value, and restore the highlights with the Lift Gamma Gain adjustment. ACES is very receptive to overexposure.

    upload_2022-8-14_20-14-21.png

    It's one of the many ways this can be solved (using Color Curves would also be a way to do it).


    If you imply that a camera is expected to have the same behavior as a human brain in Unity out of the box, that's a whole different discussion, and this could be the root cause of your problem here: your expectation of how a camera works, how tonemapping works, and your definition of "photorealism". You will not get good results if you're not using a dynamic exposure for such an extremely contrasted scene, if you're using a very "clampy" tonemapper (ACES), and if you're expecting a camera to behave like a human eye; all at the same time.

    Also, as a reminder, grading/tonemapping are artistic considerations: if one wants to have a strong filmic look, ACES can be a great starting point, but it requires to be very careful with exposure, and usually forces one to "expose for the shadows" with a tad of gain reduction to preserve the highlights. If you want to recreate a "human eye" look, then definitely avoid ACES (though not impossible with many adjustments on top of it and general overexposition), but rather start from the Neutral Tonemapper, or even use your own custom LUT generated in another software if you feel confident with tonemapping/colorgrading.

    You can try to implement some of the solutions mentioned above, I am certain you will get better results.

    The best way for you to learn how this all works is by playing with a DSLR/mirrorless camera in manual mode for exposure. I believe you'll then better understand the complexity around exposure/tonemapping when dealing with real-world situations.

    And I think your "black paper" example is probably the closest thing to "local tonemapping" that I can think off, and it's been tried in several games already (e.g. Crysis 2, RDR2, God Of War, but with limited results, depending on your taste or expectations of "filmic look"), and it's certainly not done by hacking the shadows/indirect lighting as you propose. You can read this article for instance to learn more about it, it's certainly a complex topic with many moving parts. A local tonemapper could be something we introduce in the future, but no ETA. ;)
     
    Last edited: Aug 14, 2022
  15. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Thanks for the replies, but I still think u are missing the point

    My screenshots indicate that the shadows on flat ground with darker textures are excessively dark.

    Then I say with screenshots that

    1) It is not GI related -> The pathtracing results put an end to this speculation
    2) Tonemapping and grading is not related -> Have posted without one
    3) Nor is it exposure related -> You posted the very screenshot that has the same issue

    Even on your screenshot, the shadows are off. Maybe you disagree but look at this
    GI-Re.jpg

    Now, to make sure that this is not related to exposure or ACES grading

    I replicated your scene
    and added some balls to show my point
    gi2.jpg
    Do u see how unnatural the white balls in the shade look?

    gi3.jpg
    Now in first person

    Do u see the issue now?
    Either the wall on left has to be darker or the ground shadows need to be brighter, but since the shadows are just working fine, this is just a straight up issue with how shadow color blends with base material, and it should compensate it more so that dark materials don't receive ridiculously dark shadows

    I am attaching more photos for reference




    Here is another one with darker shadows, but still it is not like unity's


    Here is some Unreal Engine shadows. I haven't tested this myself, and just forked it from web, so take it with a grain of salt :)


    Now another photo that features darker shadows on darker object. You can see the shadows are dark like unity, but the shadow caster, (house, tree, bush) are equally dark, but in Unity, they are much brighter, indicating either one is not rendered correctly.


    Now, if this is the intended behavior, then I have to say that the current HDRP framework is not physically correct fundamentally. I say this because then I won't be able to replicate a real scene with matching shadows. Even with pathtracing.

    Maybe it is just me, but HDRP shadows immediately ring a "uncomfortable alarm" to me. I bet those who've seen the screenshot will have noticed as well.

    Please look into this deeper. I understand that this sort of thing is probably industry standard, and is probably common at the moment, and is accepted widely, but this soon will be noticed by other users. Especially by the Archviz and cinema users. This is hard to not notice when one is working with a good reference and the Unity HDRP refuses to render correctly.

    Maybe someone experienced in Maya can render a similar scene and see how it renders. A good ground truth to put this dispute to an end. I could be just wrong anyway.

    All in all, I understand that GI and shadows work in passes, and that we are not working with infinitely strong raytracing GI to resolve this. But I think this is more related to a shortcoming on the shadows or at least it can be helped by adding another curve to the shadows until we reach complete photo realism.
     
    Last edited: Aug 14, 2022
  16. Gooren

    Gooren

    Joined:
    Nov 20, 2015
    Posts:
    332
    Have you tried changing Specular Occlusion Mode on your materials? I experienced some shadows weirdness with it enabled. Specially when combined with micro shadows. That combination even created literally unlitable places here and there (in some small crevices etc.)
     
  17. Deleted User

    Deleted User

    Guest

    @jjejj87 the balls are looking odd maybe because of lack of reflection probes... Maybe try adding a reflection probe or raytraced reflections to see whether the shadow issue stands out?? Yes the shadows on dark materials definitely looks off but as @pierred_unity said looks like an exposure, tonemapping and other post processing issue not being set up properly... Maybe try setting indirect light intensity to 2 (even in unreal 5 i mostly used to set indirect intensity at two to get rid of darkness, so must be the same solution for unity as well...)
     
  18. Deleted User

    Deleted User

    Guest

    @jjejj87
    ACES:-
    Screenshot (220).png
    Neutral:-
    Screenshot (221).png

    Neutral + Indirect Lighting Multiplier =2 :-
    Screenshot (222).png

    I am using screen space GI and reflection probe, tone mapping is making a huge difference don't know why its not affecting yours, must be a bug!! I think the look of 3rd image is what u are trying to achieve i guess!!
    EDIT- If this is still not helping, check out the shadows debug mode
     
    Last edited by a moderator: Aug 15, 2022
  19. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Yes I can use other elements of lighting to bump up a specific region (eg. reflection probes) or cranking up indirect values, but this forces me to work outside of physically correct realm and also reflection probe approach makes working with dynamic environment or open world very painful.

    Also, the ground texture has close to zero metallic and smoothness to begin with, which means for all other objects that are metallic, it is going to be hard to maintain a physically correct look.
     
  20. ElevenGame

    ElevenGame

    Joined:
    Jun 13, 2016
    Posts:
    146
    Maybe just make a test by trying to implement such a curve into the HDRP? After all initially the render pipelines were supposed to be starting points to modify and extend. I personally have not dug into the lighting or light mixing (shader)code of HDRP, but if you could achieve a (however hacky) implementation of your concept, it would be easy to argue, why it works or doesn't.
     
  21. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Thing is that picture doesn't have physically correct values AT ALL. Unreal doesn't have physically correct lighting for a sun by default, you have to do all that setting up yourself. So what you are seeing there is unrealistic but eyeballed.

    Really your problem with the Unity shot is exposure. I recommend you use fixed and see where it takes you with dir light strength using real lux values.
     
  22. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    257
    This is 100% the issue here, no exposure setting is going to hide/fix this.

    To explain your current situation (for example "Do u see how unnatural the white balls in the shade look?"), due to the lack of reflection probes in open world, the specular of the walls/spheres in the shadow are currently shaded by the sky, making them appear way too bright. Your grass is not suffering from this because of the near zero metallic/smoothness you mentioned. To have a completely correct diffuse + specular you have the following options available:

    1. Use fully ray traced lighting (path tracing), this is the most correct option and only suitable for offline.

    2. Use a hybrid approach, by using real time ray tracing for the first bounce, and using baked lighting (global illumination + reflection probes) for the last bounce, this is suitable for real time using a RTX graphics card.

    3. Use a hybrid approach, by using screen space global illumination / reflections for the first bounce, and using baked lighting (global illumination + reflection probes) for the last bounce, this is suitable for real time using pretty much every graphics card, but is limited to screen space.

    4. Use fully baked lighting (global illumination + reflection probes), this is suitable for real time and works on pretty much every graphics card, but has a lot of limitations.
     
  23. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Hippo, I am using physically accurate brightness for sun, fixed exposure and no ACES. This is pretty much HDRP standard. Also, if Unity was physically correct, then this wouldn't have happened, it may have looked a bit stylized, but not physically wrong. I think the big issue right now is that we can't match real world lighting even with path tracing.

    I am actually going to try to take a photo outside soon, and see if pathtracing in Unity can replicate it. I bet even then the devs will not be convinced. The shadows are clearly wrong.

    1. I tried and same issues
    2. But then, why would I even use raytracing...mixing baking and raytracing....
    3. This is not good as the quality fluctuates...
    4. I don't want to bake lighting non this 1km^2 scene, and the map will get bigger and bigger

    I think the issue here is very simple

    It is not exposure, nor brightness settings, nor GI or anything
    It is just straight up shadow color blending being incorrect.
     
  24. SoufianeKHIAT

    SoufianeKHIAT

    Unity Technologies

    Joined:
    Jan 16, 2020
    Posts:
    7
    Hello,
    May I ask how you can conclude that? By using eyes comparison or image from camera comparison?

    None of them are a "reproduction of reality". For instance each camera vendor have an OETF (Opto-Electric Transfert Function) which convert radiance (Let say Physically Based data with units) to electric signal, which can varying per exposition, each vendor hand-craft this function, which let produce a interesting discussion between photographer why they prefer this or this camera vendor, it's an art-direction decision made by manufacturer.

    Here what is surprising, is shadow is just "lack of light" so to validate you can check diffuse only lighting to not be polluted by Specular-Lighting.

    On Image-Industries it was a hard problem to solve util ACES-Pipeline was create with RRT-Image. We have to keep in mind, all images are exposed, all images are curve-{contrasted, lighten, darken, ...} and tonemapped.

    Our eyes sometime is our enemy to compare, physical information.
     
  25. pierred_unity

    pierred_unity

    Unity Technologies

    Joined:
    May 25, 2018
    Posts:
    433
    Your definition of "Physically correct" seems to vary quite a bit. On one hand, you don't want to increase the indirect lighting to boost the shadows because it would be "outside of physically correct realm", yet on the other hand, the hack you propose would go against the core principle of physcally-based rendering. Shadows are a lack of direct lighting, thus they are only affected by indirect lighting, and the amount of indirect lighting received by a surface has nothing to do with how bright or dark that surface is, before it's being lit. What you propose would literally consist in getting stronger indirect lighting on darker pixels because you're convinced that "shadows are too dark". There is no "force field" around dark surfaces in the shade that allows them to somehow attract more indirect light than brighter surfaces. I hope you can understand the contradiction there, between "not willing to boost indirect lighting" and "wanting dark shadows or surfaces to get more indirect lighting".

    I've been sharing your proposal internally with rendering engineers from various backgrounds, and we will not be changing the way indirect lighting is affecting the scene any time soon, and certainly not by changing the way pixels in the shadows sample their indirect lighting. We've all seen in our career similar discussions between artists and engineers (myself being in your position more than a decade ago when I was a newbie lighting artist discovering PBR and HDR workflows and complaining about similar issues). I've been on both sides of this discussion, so I can see where you're coming from.


    If you feel shadows are too dark, and that's perfectly valid as an artistic consideration, I tried to explain to you several simple methods to fix your issue, explaining to you that if the lighting and materials are set up correctly, the way shadows appear depends on your exposure and the tonemapping/grading being used. If you're still not willing to accept or try these, I doubt we'll be able to convince you.
     
    Last edited: Aug 16, 2022
    ElevenGame and m0nsky like this.
  26. Ruchir

    Ruchir

    Joined:
    May 26, 2015
    Posts:
    934
    I do agree they don't seem to be in the same shadow condition in this image o_O
    Although I do believe increasing the luminance of the sky will do the trick (maybe)
     
    Last edited: Aug 16, 2022
  27. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    luminance of the sun is max (130000) also the exposure is as pierre put (Auto, 13, 0), no tone mapping or any grading.
     
  28. Deleted User

    Deleted User

    Guest

    @jjejj87 I don't know what's your setup but i am getting the effect which u want in a pretty straight forward way by default without reflection probes, any GI or cranking up the Indirect lighting
    ACES:-
    Screenshot (223).png
    Neutral:-
    Screenshot (224).png
    EDIT:- As u had mentioned earlier i have used more darker texture with metallic value set to max and increased smoothness value
     
  29. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    I think you guys are still misunderstanding the point so I went outside (luckily I live in a climate region very close to the default HDRP physically based sky scene)

    gi1.jpg

    Then I put down some objects that resemble the scene we are working with. I used a PS5 box which is very white and moderately smooth (maybe like .45?). The paving is an orange paving, and was took with an ipad. Now all cameras do some exposure and post processing, which is fine, but it won't grade one area of shadow one way and another part differently - correct?

    Then I added in two more adjusted photos (only exposure and gamma) to simulate Unity's tonemapping.
    gi2.jpg

    You can see that even when the photo is processed, it does not necessarily make shadows on darker materials darker nor does it grade it different compared to the white box's shadowed side.
    There is obvious natural continuity of shadow strength. Except for the last photo which I photoshopped to simulate current HDRP shadows.

    But let's assume that I am doing something wrong, and got all the exposure, grading and settings wrong in Unity - I mean it is possible. So, let's apply the same processing to an actual photo and see if we get similar results to Unity.
    gi3.jpg

    Now let's look back at Unity
    GI10.jpg
    GI14.jpg


    I am open to suggestions and recommendations, and really my goal is to achieve photorealism, but no matter what I do, I just can't replicate or match real life shadows - and this only seem to happen with objects facing up, with slightly dark (but not really dark, you got the texture as well).

    I am also not against the idea of bumping up indirect - the screenshots were with the value of 1.5 to begin with.

    But when I was playing around with it, I noticed that the shadows on ground don't react to indirect value like other shadows on walls.
     
  30. Deleted User

    Deleted User

    Guest

    Images with more better lighting and exposure
    ACES:-
    Screenshot (225).png
    Neutral:-
    Screenshot (226).png
     
  31. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Here is a comparison screenshot
    Indirect values are 0, 1, 3, 20.
    Beyond the value of 3 and above the indirect difference of shadows on wall and ground appears. Shadows on ground receive way less indirect for some reason. I don't know if this is meaningful though
    indirect.jpg
     
  32. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Yes, except the ground is metallic shiny now, that kind of metallic shininess is just a side effect of how reflections work in Unity. As you can see in my photo's taken this morning, ground objects like soil or bricks don't shine like that in real life. I can achieve something similar but then every other object is going to look like it is made with metal - reflecting blue sky. Of course, this pertains to objects in shadows so either reflection probes have to be present in every shadow or the materials all need to be excessively metallic/smooth.

    Not a solution but a possible workaround that comes with more problems. Put another metallic object in the same shadows and now you have to do more adjustments...
    ref.jpg
    Another way to tell that your way is not physically correct, is to pathtrace it, and we don't get those results. Hence why I say this is not GI but a simple shadow issue.

    Regardless, I could be wrong here and maybe seeing what's not really there. But it is also true that I cannot replicate a lighting condition - a normal bright sunny day.

    I will continue to build my scene. I hope the devs look at this with open eyes. If you can, try to replicate the actual photo I posted without changing shadow opacity. I will continue to look into this as well.

    I am convinced that introducing a curve to handle how shadow adds to base material will fix this. As for the reason behind it, well, I can't replicate a real life lighting for flat grounds.

    Thanks.
     
    Last edited: Aug 16, 2022
  33. Deleted User

    Deleted User

    Guest

    The texture receives shadows perfectly with default metallic (which was zero by default) and smoothness value (which is according to the mask map) as well.. u mentioned earlier that in my first test i had very less metallic/smoothness to start with.... Maybe i misunderstood something:D
    Thats why I later tested it with max Metallic & smoothness, in nature no ground material is 100% Mt & Sm!!
     
  34. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Yes, using eyes to see if something is wrong is actually something that we can do.
    Then we look at the underlying technicalities to see if the suspicion is valid. I don't really see anything wrong with it, do you?

    And no, the "lack of light on the shadow" is not what I am basing this on. I think I have provided enough data to indicate that there could be something wrong.

    1) A unity scene
    2) A replicated unity scene of Pierre's scene
    3) A photoshop adjusted image to indicate what seems to be off for comparison
    4) An actual photo with similar settings

    I don't know what exactly I have to put on the table to discuss that the shadows are not rendering correctly? I find it funny that you find it "surprising" for me to suggest that the shadows could be wrong. What exactly would not make you not surprised?

    Since you asked, I am going to re-re-iterate how I concluded "that"
    1. I saw that the shadows on ground were excessively dark
    2. I also saw that only shadows on flat grounds did this
    3. Then I put brighter objects in the shadows to see how it would receive shadows
    4. As shown in the screenshots above the objects with brighter materials received milder shadows and looked like they were floating
    5. So naturally, with my eyes, because I have eyes, and see the world everyday, I question myself.
    6. Is this correct? Is this what I see in the real world? Is this a bug? Is this physically correct? Am I doing something wrong? Why are materials on flat ground receiving much darker shadows?
    7. So I begin testing
    8. I start path tracing to get a ground truth of the scene, every other aspect of lighting raytraced.
    9. I still see the same issue
    10. I then look into ACES and tonemapping to see if it is distorting color excessively.
    11. Issue persists
    12. Then I look into exposure to see if that is the cause.
    13. I match Pierre's parameters
    14. Issues persists
    15. Then I go outside, replicate the scene in real life take some photos
    16. And try to replicate it in Unity
    17. Issue persists and most importantly, I cannot get the same shadows like in reallife, not without changing shadow opacity
    18. I then apply post processing to the photos to match Unity. So essentially I am reversing the process to see if it can be done.
    19. I cannot and the shadows maintain a smooth continuity in its opacity even on dark objects.
    20. So I post here with my findings.
    If you were actually taking this seriously, even by a bit, you could have tried to replicate the scene, and showed me that it can be done and that what I have was the results of my settings, material or whatever. Or lead to me documentation or paper that I can reference to. Or a possible workaround. Or like Pierre did, say what he can and cannot do.

    Instead you are surprised because I looked at the scene and noticed something with my eyes.

    But here is the best part.
    I still can't replicate the real life lighting (just the shadows on ground with slightly darker material).

    Can you?

    If you can, please show me. I really need to fix it.

    Please, just look at it with an open mind.
     
    Last edited: Aug 16, 2022
  35. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    257
    I think people are all for looking at this open minded and would love to catch an actual bug here, but there are a lot of contradictions, vague measurements and guesstimate/hacky proposals that will get nowhere. You can't take JPEG's with an iPad, change the contrast in photoshop to mimic a color space and what not.

    If you believe Unity's path tracer is incorrect, use an HDRI, throw the exact scene into pbrt-v4 / Mitsuba and verify.

    You can not say the path traced GI is correct but the shadows are wrong. Shadows don't exist, that's a rasterization thing. At that point it can only be an bxdf problem with your grass for which the math can be ran through desmos and verified.

    I truly believe the solution to your problem is in my previous post. Yes it takes a lot of work to bake lighting and set up reflection probes for an open world game, but you can't just skip part of how the pipeline is designed and not agree with the results.
     
    Last edited: Aug 17, 2022
    pierred_unity likes this.
  36. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    I did not say the path tracer is incorrect, I said that the shadow issue persists (shadows much darker on darker materials) with both path tracer on and off, hence why the path tracer is not the issue.

    Shadows don't exist, that's a rasterization thing -> Exactly why I said, just introduce a curve to the shadows.

    Here is the base material of the ground complete white (255, 255, 255), then mid (122, 122, 122) and then black (0,0,0)

    gi.jpg
    Am I the only one who thinks the shadows are off?
    Now I am starting question myself.... :)

    I mean, I see black and grey asphalt everyday, and shadows don't get dark like that...and bright objects in shadows don't pop like that either. I've tried extreme combo's of exposure, grading and all but I can't get the shadows on darker materials to be lit correctly.

    Regardless, I think we've come to point where further posting is not going to convince the devs, so I am going to try it in Unreal and maybe blender. I have shared this with fellow devs and have mixed responses as well. I will get back with the results soon. I should be able to try a few things this weekend.
     
  37. JarkkoUnity

    JarkkoUnity

    Unity Technologies

    Joined:
    Feb 1, 2021
    Posts:
    17
    The human eye has MUCH higher dynamic range than monitors (even HDR monitors), so with your eyes you are able to see much more details in shadows than is possible to replicate with monitors. We are talking about ~20 stops for human eye vs ~10 stops for monitors, which is 1000x+ difference. So, it's a bit unfair to compare the details you see with eyes in the real world to the details you see on a monitor: “I see black and grey asphalt everyday, and shadows don't get dark like that”. One way to fake the high dynamic range of eye on monitors is to use “local tone mapping” that @pierred_unity also mentioned, which artificially brightens the darker regions and darkens the brighter regions of the image to map values to the representable range on monitors, while preserving the details. This feature hasn’t yet been implemented in Unity though and is not as straightforward as it may sound.

    Also, mobile devices do bunch of post processing to improve the quality of images captured with very constrained sensors & optics of those devices, so assuming “..it won't grade one area of shadow one way and another part differently - correct?” is not correct. I believe the closest real world reference you could have is with raw images of a DSLR camera. Even for raw DLSR images there must be some post processing to make the images viewable, but probably nothing more advanced such as local tone mapping (should be bare minimum processing).

    I saw you mentioned that you use the max 130000 lux for the Sun illuminance. This is at least one thing contributing to your problem and you should lower it to more proper level below 100000. Sun has “extraterrestrial solar illuminance” of around 128000 lux, which is the Sun illuminance ABOVE atmosphere in space before it gets attenuated by the Earth’s atmosphere. When the Sun is straight above, the Sun light travels shortest distance through the atmosphere and attenuates to around 100000 lux at the ground level. If you use overly bright Sun light, it increases the contrast between shadows (lit only by the sky + GI) and Sun-lit areas (lit by the sky + GI + Sun) making the shadows appear darker, so bringing those values to proper balance results in more realistic image. Physically Based Sky in Unity automatically attenuates the Sun light based on the Sun angle due to the atmospheric attenuation and you can use Unity’s “Lux Meter” to check that the lighting values are as expected and tweak appropriately if not. I recall the illuminance of unobscured clear sky (no clouds on an open field excluding the Sun) with the Sun located straight up should be around 20000Lux, which you should also verify in your project. Ideally you would use real world light meter to get some good luminance & illuminance reference values.

    Once you have the lighting values in check, you should check that the material properties are correct. Particularly that the albedo values for the ground are around “bare soil” values of 0.17 or “green grass” values of 0.25, and those buildings are around “new concrete” values of 0.55. If you use overly dark values for the ground (e.g. “fresh asphalt” range of 0.04) and/or too bright values for the buildings (e.g. “fresh snow” range of 0.8 or above) that will also contribute to the high contrast issues you are having. You can of course diverge from these values but they give some ballpark reference values to target and verify.

    And finally as others already mentioned, the camera exposure settings and global tone mapping that’s used to map those HDR luminance values to be representable on monitor has big impact on the final image. ACES filmic tone mapping crushes blacks and may not be ideal for your case, and games may prefer to use gentler “neutral” curve for the dark parts of the curve to preserve more details.

    Hope that helps.
     
    Last edited: Aug 17, 2022
  38. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Thanks for the detailed reply, I will update my scene with suggested values and report back this weekend! I hope this resolves it.
     
  39. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Okay, after this long post, I sat down and thought to myself.

    "Am I just seeing things that no one sees?"

    And then I looked at the Unity scene I am working with and my reference photos

    And there was a disparity. A clear one. I showed the scene to a 12 year old girl and immediately she says why is there a black carpet on the floor.

    Now, I come back and read all the posts, and clearly, most people think otherwise and say that I am using my eyes to say that there is something wrong.

    Sure I could be wrong, or not. Regardless, a valid point.

    But if a 12 year old girl will notice it right away, surely my players will too.

    And at that point it won't matter how physically correct or incorrect it is.

    So, I started to double check everything.

    And then concluded, that the diffuse textures, even if it is color corrected, had to be brighter. While the devs saying that it is physically correct, and fellow devs claiming that it cannot be verified with just "eyes", I come to accept that one way or another, the darker textures just have to be brighter to have shadows to match my reference photos.

    I mean, if it looks good, who would know that it is not a match to the reference photo? Right?
    gi1.jpg
    So I changed the texture a bit so that it looks like something that belongs to the scene, and as expected, the shadows started to brighten up.

    BTW, I added the white tunnel box in the scene to keep track of completely occluded areas. I want shadows to brighten up, but I didn't want to lose the ability to be completely dark as well.

    Now, to me, the shadows are still super dark and there is an obvious discontinuity, but like we discussed above, maybe this is physically correct.

    But it is also true, despite my endeavor, that I could not replicate a real photo's lighting. And I couldn't make the ground brighter further, as that would be really stupid.

    Then a thought struck me.

    What if.

    What if it is physically correct, except even with pathtracing, it is lacking?
    What if the "physically correct" only applies to the currently implemented rendering tech? What if it is that simple?
    Is there a possibility that it is not just the approximation or simplification but a straight up missing aspect to the pathtracing tech? Hard to imagine - it is pathtracing afterall, but also very possible. It could even be something no one researched.

    So I stopped assuming that the pathtracing is 100% correct (against reality) and decided to look at this old school.

    I've been adding proxy lights for many years before raytracing came along...why not now?

    So, what can I do to make the shadows brighter (so that there is no obvious discontinuity) and still maintain 100% dark shadows where there is no indirect bounce light.

    Then it came to me.

    What if I add emissive, a very slight emissive, enough to make the shadows brighter (because shadows render correctly on brighter material) but not enough to make the texture seem brighter or shine.

    Here is the result

    First is ACES, brightened textures
    Second is +white emissive at 7
    Third is +orange emissive at 7 (just a color tint for artistic touch)

    gi2.jpg
    And I checked under dynamic day/night cycle and no issues. The discontinuity of shadows have gone for me.

    Here is the final comparison shot with more appropriate shadow angle diameter settings so that the shadows are not razor sharp
    gi3.jpg

    I think my journey ends here as I have what I needed.

    As for whether the shadows need to be fixed because it is not physically correct or anything, I am 50/50 about it. As for whether the devs should look into this, I am also 50/50 about it. Their job is not researching this to begin with, their job is more close to implementing a proven algorithm within a viable hardware spec and the platform. It might just be coming along in the next few years. The next big paper by some doctor at some university. So I digress as I am not exactly sure how HDRP devs work. But I appreciate their interest and replies to this topic.

    As for JarkkoUnity's response, I would like to say that while what you point out is valid, and that there are many in between processing that pollutes what we perceive (our eyes-brain and cameras) and that physically, the ground shadows could be darker in reality and that we are processing it (or the camera) to match it. That could be.

    But the issue is, in the end, is that we see it with our eyes on the monitor at the end. So, if a 3d renderer doesn't present an image that we think we are seeing (the processed) then most likely your approach may not be valid. Because, if that were the case, then we would immediately see photos taken with our cellphones and say, "Hey why is the shadow on the ground darker?". People notice.

    Also, another aspect to think about is all the usual fx we add to the final image, bloom, color correction, motion blur and exposure. If your argument is correct, then we shouldn't be adding those effects as the rendered image without those effects will be processed in our brain to have those effects when we see it on the monitor.

    I have over simplified the issue here, but I just wanted to tell you that what you said (very common statement in visual fields) may not exactly apply here.

    So, maybe we should really think what we are trying to achieve with physically correct rendering. If the assumption of human eyes at the end remains, and not scientific research, then maybe it is worth reconsidering.

    But then again, I am only speaking from my experience.

    I also did some testing in Unreal Engine 5 with Lumen, and the shadows on the ground was equally dark, but the shadows on the wall were darker, which did not present that much of a shadow discontinuity. So, darker than I think it should be, but no discontinuity between flat ground and vertical walls. But my UE5 experience is limited to few weeks of playing around with it, so take it with grain of salt.

    All in all, I was able to get something very close to the reference photo after all. It isn't exactly verbatim, but I did get it.

    I think I am ready to continue working on my DX12 raytraced open world game. Hopefully I can achieve photo realistic graphics. I also hope anyone else having issues can use my hack to workaround it, for artistic reasons or what.

    Cheers.

    Also, I think I found a bug during my testings.
    Directional shadows sort of break with pixel displacement + depth offset
    bug.jpg
     
    Last edited: Aug 18, 2022
    ElevenGame likes this.
  40. pierred_unity

    pierred_unity

    Unity Technologies

    Joined:
    May 25, 2018
    Posts:
    433
    I wonder what's the most impressive in this whole thread: the total disregard for advice or the stubborn ingenuity. You've definitely raised the hype for your next photorealistic game!

    In the end, it's great a solution was found that works for yourself though. Who knows, calibrating materials properly, lowering the camera exposure to expose the scene correctly, and tuning the tonemapping/grading would have worked, as explained several times, but I suppose this would have meant listening to feedback and recognising a certain lack of understanding for PBR.

    But eh, if using emissive material does the job, so be it. Ultimately it's the final pixel value that counts, right? :D

    (Obviously, for anyone reading this, using emissive materials to brighten shadows is absolutely NOT recommended.)
     
    Last edited: Aug 19, 2022
    RevC2, koirat, shikhrr and 3 others like this.
  41. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    1,117
    Well, like I keep saying, while the theory you guys suggested sounded good, it didn't make any difference. The only suggestion you guys made was, make the texture brighter and change grading to make shadows brighter. So I did listen to feedback except it didn't help. Why? Because then all shadows are brighter. I might as well as just change the shadow opacity at that point. But then how would I render a complete dark shadow? Like the white tunnel box I added?

    The issue still remains that any texture below grey (122, 122, 122) starts to have a black carpet like shadows is also not addressed.

    I've posted this before, but this is with indirect bumped up to 20
    indirect.jpg
    You see how the interiors (only windows as light source) are brighter than the outside shadows? And the house wall on the shadowed side? I mean come on...this is a dead giveaway.
     
    Last edited: Aug 19, 2022
    GameModeOn likes this.
  42. Deleted User

    Deleted User

    Guest

    U need to understand that this is neither the issue of how shadows are getting applied on other materials nor the issue of texture color being quite dark... This is a issue of tonemapping and camera exposure... This can only be solved completely by using local tonemapping and exposure as @pierred_unity said, because currently these effects get applied to the whole screen instead being locally as per the brightness weight on the screen!! This causes loss of details as it makes several parts of the scene to look extremely dark or extremely bright, especially when dark and bright parts are together on screen... UE5 has this and if u still don't understand what it is, u can check their docs to get better understanding... Currently u can only bump your texture brightness or use custom tonemapping or play around with exposure in unity as a workaround which can solve 60-80% of this problem in most scenarios...
     
  43. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Seems like a lot of advice from people with AAA backgrounds and backgrounds in photography were completely ignored. Who cares about tech details when you can monologue and and make it "work" (while failing under lighting conditions not yet tested).

    Don't forget to blame HDRP for it somehow.
     
  44. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    595
    I'm glad after all your lengthy Inputs and Inputs from others, you at least achieved what you wanted .

    Some additional help for future reference is monitoring your lighting is utilising the Render Pipeline Debugger.

    -PBR validate
    -Monitor Scopes ( 2022)
    -false colour
    -Exposure modes( Ev mode / Final Exposure)

    Can all help assist your eyes and perspectives from your monitor.

    Especially the EV modes as you can measure those outside with some care and degree of accuracy to do you best to create a control.

    When I first saw the post I was curious the colour of the walls whether you used just a primitive above 241 RGB.
    Exposure targets can be thripped pretty wildly with extremes in contrast.

    A photographer for example, would often shoot a stop lower than the metering when shooting snow, because it's so white and bright it throws off contract meter, similar with darks.

    So if you're placing primitives or very near limits for 50SRGB PBR validation and ACES or heavy post contrast, exposure/end results can be a little tricky to balance.

    I'd generally go into scene view with the exposure rendering debugger on set to ev100, you'll be able to see all the EV in the scene via screen colour and mouse pos.
     
    Ruchir likes this.
  45. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
    I really struggle to get shadowmaps looking good. Either they look ok close up and pretty bad at a distance, or the other way around... Shadows also flickers a lot.

    It is not an option to switch to high quality shadows or super high resolution due to performance restrictions.

    I think Unity team should prioritze such fundemental functionality.

    Just sayin'
     
  46. bonickhausen

    bonickhausen

    Joined:
    Jan 20, 2014
    Posts:
    115
    my dude, if your goal is to make a DX12 raytraced open world game yet you're struggling with PBR you're gonna have a bad time
     
    Gooren likes this.