Search Unity

De-Lighting Tool De-Lighting Tool

Discussion in 'Graphics Experimental Previews' started by Cyril-Jover, Jul 7, 2017.

  1. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi ssh4 !
    In the documentation it is written this:
    "In the Base Map, the alpha channel is used to mark the pixels that should be de-lighted ".
    XNormal has a bug on previous version. The last one is good and the default settings are ok, just need to be world/object space bent normal maps.
     
  2. ParadoxSolutions

    ParadoxSolutions

    Joined:
    Jul 27, 2015
    Posts:
    272
    Can the De-Lighting Tool de-light regular imported textures or do they have to come from a scanned 3D mesh?
     
  3. comankh

    comankh

    Joined:
    Aug 10, 2013
    Posts:
    2
    Hello Cyril,
    I'm trying to run attached example and get driver crashes at around 45%.
    Are there specific hardware requirements concerning GPU?

    Crashes on both:
    GTX770 Lightning LE 2GB
    Quadro FX 3800M 1GB

    i7cpu, 24 or 16 GB ram respectively, win7, latest drivers
     
    Last edited: Aug 21, 2017
  4. comankh

    comankh

    Joined:
    Aug 10, 2013
    Posts:
    2
    ok, figured it out by downscaling textures to 2k, so I guess it is GPU RAM issue.
     
  5. Dash-POWER

    Dash-POWER

    Joined:
    Aug 21, 2017
    Posts:
    1
    Hi,
    Thank you for the tool. Looks very interesting. I have tried the stone example and it works pretty well but I've created my scan and I'm getting strange color results. I'm using all maps which are necessary (albedo, normalOS, BentNormalOS, AO, mask). I really don't know where could be problem. I understand that the black shadows areas are impossible to delight because there aren't color surface data but in other cases I think the tool could make better results.

    PS: I want to ask also how is the right axis orientation? X+ Y+ Z+ or X+ Y- Z+? Tried both but almost same results like on attached image.
     

    Attached Files:

  6. caubebutchi

    caubebutchi

    Joined:
    Dec 12, 2010
    Posts:
    68
    I have issues with shader . what's kind of shader use it ? . Standard or not ? Please send to me the shader De lighting.
    Thanks alot
     
  7. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi comankh !
    There is still few bugs. I'll communicate when those crashes will be fixed. Sorry that you have to downscale to make it works. I've got the same bug as you but with 8k ... but I've got a 4GB graphic card. This first version wasn't fully optimized, and processing big maps should be longer but shouldn't crash ! Maybe in the next version.
     
  8. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi caubebutchi !
    I'm not sure I understand your problem. Have you got a shader error when you use the tool ? Could you make a screenshot of the error ?
     
  9. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Dash-POWER !
    Could you test to make those two masks:
    Masks.jpg


    Activate the eye-dropper:
    upload_2017-8-28_15-38-9.png

    And place the reference zone like this:
    upload_2017-8-28_15-40-26.png


    1/ Process the De-Lighting with Mask 1 then save the result as Result_1.png
    2/ Process the De-Lighting with Mask 2 then save the result as Result_2.png

    Using the reference zone should make the two de-lighting consistent.

    3/ Mix the results using photoshop

    I hope it helps you. Don't hesitate to share your de-lighting results !
     
  10. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    "I want to ask also how is the right axis orientation? X+ Y+ Z+ or X+ Y- Z+?"
    => The axis order or direction don't really matter. The most important thing is Normal map and Bent Normal map have the same
     
  11. caubebutchi

    caubebutchi

    Joined:
    Dec 12, 2010
    Posts:
    68
    Sry my English. I mean, could you tell me what did shader you used for 3D mesh ?
     
  12. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    When your texture is de-lighted you can use it with a Unity Standard Shader in the slot "Albedo".
     
  13. andyfedak

    andyfedak

    Joined:
    Jul 2, 2017
    Posts:
    1
    Hey there Cyril, thanks for putting together such a brilliant tool and workflow. It's fantastic, and one of a kind.

    I had one question from your Siggraph talk, in it you demoed a layered shader workflow which could be tuned within the unity engine in realtime. Is this something which is available to use as well? The results seem excellent.



    Thanks for your help!
    -andy
     
  14. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Andy ! and thanks !
    Yes we're working on a layered material. This is currently work in progress and I can't give the release date right know. It depend on other features that should be released too. When I know more about the date , I' ll tell !!! I promise !!!

    Cheers,

    Cyril
     
  15. kite3h

    kite3h

    Joined:
    Aug 27, 2012
    Posts:
    93
    Best Delighting Tool I've ever seen.

    Bent normal or derivative normal or what ever , It is very important point to delight.
     
  16. kite3h

    kite3h

    Joined:
    Aug 27, 2012
    Posts:
    93
    This tool must be the best delighting tool I have ever seen. Paradoxically, however, the best result of delighting is that photmetry seems to show no need for albedo. This is similar to the roughly hand-painted result. In fact, I have blended the two before doing the height blend with the layered material on the old modeling data, which was simply past diffuse. The results are very similar to those of this delighting tool. For example, if you have building modeling, you have layered height blending the concrete scanning material on the walls. Of course, the speed of work is fast enough to be opponent. The only problem is that it is a crushing of the normal map caused by the normal enlargement and reduction. This is because no blurring can blend the normal map. But it seems to be a big help for people with a lot of money. To indie
     
  17. nbac

    nbac

    Joined:
    Jul 7, 2015
    Posts:
    248
    this is awesome!
    is the hard shadow removal part in the latest build now?
    it is a bit off topic, but there is on thing not solved for me yet (photgrammetry whise not especially delighting tool).
    it is capturing the glossiness / reflectivty of an object.
    you could capture the reflectivity of an object if you would take a picture with and without a polarize filter and extract the difference of those pictures. kind of a cheap hack but you could get pretty close with this i guess.
    but this is not how you aquire the data you take hundreds of (free hand) pictures and always switching the filter on and of is "impossible". so you end up with a texture, filtered or not filtered. would it be possible to take just one photo with and without filter to create a template to extract the data? which is recognizing patterns or similarities and can process this on the data?
     
  18. nbac

    nbac

    Joined:
    Jul 7, 2015
    Posts:
    248
    and what exactly is the position map (world pos?!) i cant find this terminology in any baker? what is the equivalent in xnormal?
     
  19. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi nbac !
    Yes, playing with polarized filters (on the camera and the light) helps a lot to separate what's the reflective part and the diffuse part in a picture. We had the same idea as you : get a roughness sample (using polarizers) on a part of the object and then propagate this measurement on the whole object (using matching patterns, recognition ...). A big material roughness database can also help a lot to achieve this process. We are working on this subject but we didn't plan to ship a dedicated tool/feature right now.
     
  20. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi the position map is the 3d position of each parts of the surface (texel) this is used to correct some artifact that can occur when there is UV seams on the mesh. You can find details on this in the documentation that is provided with the tool.
    You can bake it in Knald and Substance designer. I think there is also a way to bake it in 3dsmax using vray and a specific vray material. This position map can be baked on the high poly or the low poly mesh, it doesn't make a big difference at the end. it's better to bake it in 32 bit exr, but 8bit tga can work too.

    Cheers,

    Cyril
     
    mrtan007 likes this.
  21. nbac

    nbac

    Joined:
    Jul 7, 2015
    Posts:
    248
    thank you very much for your reply. i have my first results. it is not very good and i want to give it a try with those optional maps. ill keep you updated. everything else - tool handling etc. is superb. and the result could suffer from a bad input on my side (even if it is looking pretty solid).
     
  22. testunity01

    testunity01

    Joined:
    Sep 15, 2017
    Posts:
    7
    hey there!
    I can't install delight tool. Can you help me out?
    I have Unity 2017.1.1f1 Personal 64bit installed. I downloaded the delight tool zip from github, unzip everything on a temp folder on my desktop and double-click on DeLighting-Tool-Data-Rock01.unitypackage. It opens the maps in unity.
    Then if I go to Windows->experimental I don't see the delight tool there.
    What did I do wrong?

    On github it says: "Open the project with Unity" but I don't see any project to load except the unitypackage(?)
    Thank you! :)
     
  23. JakubSmaga

    JakubSmaga

    Joined:
    Aug 5, 2015
    Posts:
    416
    You need to open/create your own project, then import unitypackage.
     
  24. testunity01

    testunity01

    Joined:
    Sep 15, 2017
    Posts:
    7
    Yes, that was I did.
    Launch Unity,
    create a new 3d project (i tried with a 2D one as well, same results),
    once created: asset -> import package -> custom package...
    select DeLighting-Tool-Data-Rock01.unitypackage
    A popup Windows appears with the maps to import, click ok.
    In the bottom, in the assets section I have a folder with all the maps.
    then I go to Window -> Experimental -> ?? Only "look dev" here... no delight tool :/
     
  25. JakubSmaga

    JakubSmaga

    Joined:
    Aug 5, 2015
    Posts:
    416
    I've downloaded this unitypackage and it's just a texture sample, for De-Lighting tool just go here and find "Installation" section. https://github.com/Unity-Technologi...ightingTool/Documentation/De-LightingTool.pdf
     
  26. testunity01

    testunity01

    Joined:
    Sep 15, 2017
    Posts:
    7
    Download Tool Project
    The De-Lighting Tool project can be downloaded at:
    Unity Github .

    The link goes to: https://github.com/Unity-Technologies/DeLightingTool
    This is where I downloaded the .zip. I clicked on "clone or download" and create a .zip file.

    Am I missing something here?
     
  27. JakubSmaga

    JakubSmaga

    Joined:
    Aug 5, 2015
    Posts:
    416
    Click on "Clone or download" and select "Download ZIP".
    Now create an empty folder.
    Extract all the files that you have in your ZIP file into this folder and open the project with Unity.
     
  28. testunity01

    testunity01

    Joined:
    Sep 15, 2017
    Posts:
    7
    Oh you have to select the folder! This is why it didn't work. Sorry for the noob question and thank you for your kind help :)
     
    JakubSmaga likes this.
  29. nbac

    nbac

    Joined:
    Jul 7, 2015
    Posts:
    248
    i gave it a second pass in photoshop it is kind of okay now. ist it just the very shadowed input or am i doing something wrong? i double checked the documentation and i think i am doing it right.
     
  30. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi testunity01,
    when then project is unzipped and you've got a folder named "DeLightingTool-master", you just have to open Unity, select "File/Open Project" and coose the folder. It should work, if not, don't hesitate to tell me.

    Cheers,

    Cyril Jover
     
  31. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi nbac !
    Your Normal Map is in tangeant space, it should be baked in world/object space.
    Your AO seems to be strange on the right. In the texture and in the normal, we can see a vertical wrinkle, but it doesn't appear in the AO.
    In baking softwares, there's often a maximum distance or a radius parameter for AO and Bent Normal baking. You should put the maximum value for this distance/radius parameter.
    I hope it was helpful !

    Cheers,

    Cyril
     
  32. nbac

    nbac

    Joined:
    Jul 7, 2015
    Posts:
    248
    ill give it a try it is not a wrinkle it is the shadow edge - the thing i want to get rid of. next time i bring a occluder to the photo session.
     
  33. nbac

    nbac

    Joined:
    Jul 7, 2015
    Posts:
    248
    this fixed it! very nice - still some manual tweaking but a perfect start! one hell of a tool!
     
  34. testunity01

    testunity01

    Joined:
    Sep 15, 2017
    Posts:
    7
    Hey there!
    What software would you recommand to export all the maps? Designer? Knald? Which is better? (I'm using 16k textures) Thank you :)
     
  35. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi testunity01 !
    I don't work with 16k texture, so sorry, I can't answer.
    Keep in mind that it will be hard to work with more than 4k with the De-Lighting Tool. I've processed 8k but never 16k.

    - xNormal does almost everything (except position map) and it's free
    - Designer does everything and is faster than xNormal. It can be batched using commandline, so it's good for pipeline integration. Designer is a complete and powerful tool that does more than "just" baking.
    - Knald does everything and is, by far, the fastest.

    Most of the time, I use Knald.

    Cheers,

    Cyril
     
  36. testunity01

    testunity01

    Joined:
    Sep 15, 2017
    Posts:
    7
    Would the limitation come from Unity or your tool? It's really fast with 4k so it could take some time with 16k but it should work I guess! I'll have to try it to be sure :)
     
  37. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi testunity01,
    it comes from the tool and the amount of video memory required. People that tries more than 4k start to have problems. A clustered version of the tool could handle this problem but it isn't plan for now.

    Cheers,

    Cyril
     
  38. testunity01

    testunity01

    Joined:
    Sep 15, 2017
    Posts:
    7
    if this is a question of money you can turn this into a kickstarter. This kind of tool is becoming very handy these days and there's not many else available. It really could become a game changer IMO.
     
  39. Grihan

    Grihan

    Joined:
    Aug 27, 2013
    Posts:
    8
    Last weekend I captured some surfaces, in not good conditions for photogrammetry: a sunny day with strong shadows. But just wanted to try the delighting tool with a different subject, a planar surface.
    Here you can see the result in 2 different close-ups of a 8k texture:

     
  40. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Grihan !
    Cool results ! even with this kind of lighting conditions ! Strong shadows are definitely one of the major problem to solve. I've made a test to remove them and it's pretty promising. You can see it in this video at minute 42:


    Another missing feature is the automatic separation of the materials. Today it's required to use a mask to separate the main material from others, but it could be much better to detect different materials automatically without user input.

    Thanks a lot for sharing your results, it gives a lot of energy to add those missing features,
    Cheers
     
  41. Grihan

    Grihan

    Joined:
    Aug 27, 2013
    Posts:
    8
    Thanks to you for the tool! I can make some new tests, forcing extreme situations if you need. I learn from every captured subject, so you only have to ask for it.

    It would be great. In Spain, specially in Madrid area, cloudy days are difficult to appear. When I wake up in the morning and see that it is a cloudy day, I take my equipment and go out as fast as I can to take pictures. Always when I don't have to go at work. So yes, strong shadows solution would be great! ;)
     
  42. craven7

    craven7

    Joined:
    Aug 21, 2015
    Posts:
    2
    Hey, so I gave it a go, but I am having some issues.
    The image is pretty much destoyed (esp the grass and dirt bit) but i am also getting artefacts on the bark (the very bright spots).
    Also I read from a previous post that you can bake ALL the maps in Knald, but I can't seem to find the position and bent normals maps.
    Diffuse_postdl.png Diffuse_predl.png
     
  43. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi craven7 !
    You're right, it doen't seem to be a good result. Can you verify these few things:

    - your textures match the naming convention: DLAO, DLBC, DLN, DLBN ...
    - your color map has got the alpha channel black for the background and white for the baked pixels
    - your normal map and bent normal map are baked in world or object space

    What's your version of Knald ?
     
  44. craven7

    craven7

    Joined:
    Aug 21, 2015
    Posts:
    2
    Hey Cyril, thanks for the reply, but I already managed to fix it.
    The main issue was that the maps were all in sRGB except from the colour map which actually needed to be . I didn't read the documentation and that's why I got the bad results. Here is what I got after correcting all the issues.
    This is the result of 3 different masks, one for the bark, one for the dirt, and one for the grass.
    Is this how we should go about deighting textures that have more than one "material"? Creating different masks, and merging them afterwards in to Photoshop, or is there an easiest way?
    Btw this is my first scan, so it's not the best results as far as texture quality(shaking camera n stuff so it's a bit blurry, that's not from the tool)

    Oh also, i found out that I have an outdated version of knald, so yeah that's why i didnt have the bent normals option, :p
    Albedo_Delighted.png
     
  45. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi craven7 !
    Well this is better !!!

    "
    This is the result of 3 different masks, one for the bark, one for the dirt, and one for the grass.
    Is this how we should go about deighting textures that have more than one "material"? Creating different masks, and merging them afterwards in to Photoshop, or is there an easiest way?
    "
    => Yes, this is the way it should be done. In many cases, getting light informations from one material is enough to remove the lighting of the whole object. Maybe we are going to be able to edit the mask directly in the tool, and all the merge is done by the tool too. I work also on auto-masking and shadow removal too. I hope this process will be easier in next versions.

    Cheers
     
  46. rafaga

    rafaga

    Joined:
    Mar 20, 2009
    Posts:
    3
    Hi, I just download the tool, that looks very interesting, but have an issue:

    So I follow the naming convention, drag the folder with all the maps (baked in Substance Designer and xNormal).
    After set up the maps appear this info in the console:

    Unable to find required resource at 'Editor Default Resources/Delighter/LatLongArrayGather.compute'
    UnityEditor.EditorApplication:Internal_CallUpdateFunctions()

    NullReferenceException: Object reference not set to an instance of an object
    UnityEditor.DelightingInternal.DelightingService.GatherLLArray (Int32 outWidth, Int32 outHeight) (at Assets/DeLightting_Tool/DeLightingTool/Editor/Internal/DelightingService.cs:721)
    UnityEditor.DelightingInternal.DelightingService+<DoProcess>c__Iterator0.MoveNext () (at Assets/DeLightting_Tool/DeLightingTool/Editor/Internal/DelightingService.cs:352)
    UnityEditor.Experimental.AsyncOperationEnumerator`1[UnityEditor.Delighting+IProcessOperation].MoveNext () (at Assets/DeLightting_Tool/DeLightingTool/EditorGUITools/Editor/Miscellaneous/AsyncOperationEnumerator.cs:33)
    UnityEditor.EditorApplication:Internal_CallUpdateFunctions()

    InvalidCastException: Cannot cast from source type to destination type.
    UnityEditor.DelightingInternal.DelightingViewModel.Update () (at Assets/DeLightting_Tool/DeLightingTool/Editor/ViewModel/DelightingViewModel.cs:238)
    UnityEditor.EditorApplication.Internal_CallUpdateFunctions () (at /Users/builduser/buildslave/unity/build/Editor/Mono/EditorApplication.cs:109)

    So I follow the naming convention, drag the folder with all the maps (baked in Substance Designer and xNormal).
    I'm on mac and unity 2017.1.f1

    Any clue?
     
  47. fredericv

    fredericv

    Unity Technologies

    Joined:
    Feb 20, 2017
    Posts:
    8
    Hi,

    Can you check that the mentionned file exists in your project folder?
    (Editor Default Resources/Delighter/LatLongArrayGather.compute).


     
  48. Niklas-Blume

    Niklas-Blume

    Joined:
    Jun 5, 2017
    Posts:
    2
    I imported the delighting tool to unity 2017.1.0f3. Then I put the Base, Normal, Bent Normal and Ambient textures in the slots. When I hit Compute the progress bar disappears somewhere at 10% and I keep getting this Error:

    InvalidCastException: Cannot cast from source type to destination type.
    UnityEditor.DelightingInternal.DelightingViewModel.Update () (at Assets/DeLightingTool-master/Assets/DeLightingTool/Editor/ViewModel/DelightingViewModel.cs:238)
    UnityEditor.EditorApplication.Internal_CallUpdateFunctions () (at C:/buildslave/unity/build/Editor/Mono/EditorApplication.cs:109)

    Could this be triggered because the original textures are 8k and I set the max size in Unity to 2k? I have a decent gaming Laptop.
     
  49. Niklas-Blume

    Niklas-Blume

    Joined:
    Jun 5, 2017
    Posts:
    2
    Found out the problem, I did not correctly import the delighting folders.
     
  50. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    upload_2017-11-7_13-17-5.png
    Hi everyone, The Official De-lighting Tool has been released on the asset store.

    Everything you need is:

    1 - A Unity version equal or higher to 5.6 (https://store.unity.com/)

    2 - Download the De-Lighting Tool (free) from this link: http://u3d.as/WBF

    3 - Take a look at the tutorial (
    ) and at the documentation that is included in the package


    note: the personal (free) edition of Unity works perfectly with the tool, and the De-Lighting Tool package can be directly downloaded within Unity using the Asset Store Panel.
     
    JakubSmaga likes this.