Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

De-Lighting Tool De-Lighting Tool

Discussion in 'Graphics Experimental Previews' started by Cyril-Jover, Jul 7, 2017.

  1. mfranquelo

    mfranquelo

    Joined:
    Sep 2, 2017
    Posts:
    1
    I've finished another test, still with many artifacts across the delighted image (it works on some areas though)
    Any ideas of what could be failing?

    Worked:

    ferapontov_delighting.jpg

    Artifacts:

    ferapontov_delighting_artifacts.jpg

    Best,
    Manuel

    Factum Arte.
     
  2. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Manuel,
    There is a lot of reason this could fail.
    - Did you use the same exposure settings for all the photo used for the photogrammetry
    - Is your Normalmap very noisy. Do you feel it is accurate
    Can you show the different input maps ?
     
  3. mNano-Games

    mNano-Games

    Joined:
    Dec 5, 2017
    Posts:
    1
    @Cyril-Jover
    Do you now when Knaldtech is planning to release this beta version of Knald with Position baking? Is it possible to recive this beta if I'm already customer/owner of Knald license? Currenly we have bottleneck in pipeline because there is no Position bake in Kanld, baking this map in another software like Substance is not effective for production purposes. Could you PM me any info with who should we contact about reciving Kanld beta version that you mentioned?
     
  4. xplorer80

    xplorer80

    Joined:
    Dec 10, 2017
    Posts:
    1
    Hello ! i'm all new to unity and i would love to test the delighting tool, unfortunately i don't know how to install it.

    i'd like to ask you please how i should install the delighting tool that i've downloaded on github, in unit.
    i'm a bit lost.
    should it simply be copied and pasted into a unity folder perhaps, just like for the c4d plugins ?

    i tried to apply what you replied once :

    "when then project is unzipped and you've got a folder named "DeLightingTool-master", you just have to open Unity, select "File/Open Project" and choose the folder. It should work"

    but i had a trouble with it, please see the picture attached.

    delighting bug.jpg

    thanks
     
  5. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi ! just press the continue button !
    Cheers
     
    xplorer80 likes this.
  6. syntheticlounge

    syntheticlounge

    Joined:
    Dec 18, 2017
    Posts:
    3
    Make sure you have the 'DelightingTool-Master' zip file and NOT the Delighting-Tool-Data-Rock01 file.

    I can tell some people got the Data zip only (which has a pdf with the link to the master but the difference of the zips are not made clear as both have 'delighting tool' in the name). Hope that helps anyone who has trouble installing the package.

    Btw, Great Tool! Truly appreciate the work that went into it! Very helpful.
     
    xplorer80 likes this.
  7. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Thanks a lot !!!
     
    syntheticlounge likes this.
  8. Janoshx

    Janoshx

    Joined:
    Dec 18, 2014
    Posts:
    4
    Should this be working with mac? I see the texture as pink and usually i just see some flickering until views in unity goes just scrambled.

    I have macbook pro late 2013 / i7@2.3GHz / 16G ram / nvidia 750M 2G ram 500G SSD
    On my pc this works no problem
     
  9. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Janoshx !
    There a problem with mac and compute shaders, so it's PC only for the moment ! Sorry !
     
  10. monarkmedia

    monarkmedia

    Joined:
    Jan 10, 2018
    Posts:
    3
    UNITY_DelightTest-01.jpg Hello Cyril,

    I'm having some issues getting to the final delit albedo. I run through all the steps without error but the output just ends up being completely black. I feel that I am missing something...I'm using substance designer to bake out the required maps. I assembled the model in agisoft photoscan. Cleaned up the topology and uv's in zbrush and generated the original texture back in photoscan using the clean model and uv's. Perhaps I'm not baking the correct map parameters from substance? Any tips would be super helpful!
     
  11. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi monarkmedia !
    You're right, it's strange !!! Can you post a screenshot in Debug Mode (you're in Advanced mode in this one)
     
  12. syntheticlounge

    syntheticlounge

    Joined:
    Dec 18, 2017
    Posts:
    3
    Hi Cyril,

    Thanks to you and your team for a great tool!

    Having a few issues though. I'm getting some odd errors with the compute. There's certain areas of saturated off colors pixels and then on the ground area of the mesh I'm seeing boxes of shades.

    Any idea what this might be?

    Here's images of my textures and the result. Appreciate any help/guidance!!

    Christian
    Delighting_question_Odd-Errors.jpg
     
  13. monarkmedia

    monarkmedia

    Joined:
    Jan 10, 2018
    Posts:
    3
    Thanks for getting back to me, Cyril. I've attempted again. This time I've made extra effort to bake correct maps from substance designer i.e. world space normals from mesh for Bent and Normal etc. as specified by the PDF guide.. A few things worth noting:

    -I'm noticing that my DLBC map is coming in with sRGB activated (circled). Kinda noobie but not sure how to fix that.
    -When baking, I'm baking a single high-res model (for use in offline render engine) as oppose to baking from high res model to low res model.
    -My world bent map (from mesh) appears to be nearly identical to my world normal map (from mesh)
    -Not much happening with my AO map (from mesh) either...this is a simple bowl shape and admittedly I don't know much about how it should look.
    -Lastly, this model and it's original base color map were generated from Agisoft using a turntable method and masking out the relevant photo information to build the final mesh...not sure if that could potentially have any effect.

    This tool looks really incredible and I look forward to getting it figured out :) Thanks!
     

    Attached Files:

  14. monarkmedia

    monarkmedia

    Joined:
    Jan 10, 2018
    Posts:
    3
    After relentless trial and error, I'm fairly certain the issue has something to do with my DLBC map though I really don't know what. The Rock01 example files work fine so I don't think its a software/install issue...any suggestions?
     
  15. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi monarkmedia, Can you share me your data ?
     
  16. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Christian !
    Do you use the name convention _DLBC, _DLAO ... ?

    You can look at this tutorial to know exactly what to do :


    Cheers,

    Cyril
     
  17. syntheticlounge

    syntheticlounge

    Joined:
    Dec 18, 2017
    Posts:
    3
    I did not use the name conventions but put the right textures into the right inputs. I believe I followed all the instructions right. But I will check the video and get back to you. Thanks Cyril!
     
  18. Janoshx

    Janoshx

    Joined:
    Dec 18, 2014
    Posts:
    4
    Hi

    I am trying to follow your guide and pretty much everything goes ok but there is one part where i struggle...

    After i make tiffs from RAW files with dcraw i am not sure if do everything correctly in photoshop. I convert to 32 bits, pick the color from gray card(I get the x-rite next week but i believe common gray card should do), fill layer with 0.6 gray and set it to multiply. Then i make new filled layer with the color i picked and set it to divide.

    Now my image is pretty strange looking and the reason is that when i took the picture i have used wrong white balance in location. My image is very very warm... but since i am using raw file it should not matter... so I go forward. The gray card is now gray not "reddish". Now i should convert the image back to 8-bit mode. How do you do this exactly? Do you just go to top menu Image/Mode/8-bit per channel? If you do what are the settings you are using there? That menu opens HDR toning menu in photoshop CC20018 and there is plenty of settings... So which settings do you use? If i use default settings, in my case the highlights will be blown white...

    If i would adjust the white balance in lightroom how i should do the linear color space conversion in photoshop?

    I have usually done my white balancing in lightroom but the export from lightroom is slow as S***. Also in lightroom scenario the image is not resulted in linear color space but still in gamma.

    I was hoping that your workflow might speed things up a bit and i get linear color space at the same time.

    The delighting tool in unity works properly if i "eyeball" the result but i want to learn to do this in linear space in proper way.
     
  19. me109

    me109

    Joined:
    Apr 23, 2015
    Posts:
    1
    How do you increase the export resolution of the solved unlit diffuse texture? it outputs only at 1k right now..

    cheers
     
  20. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,654
    This run on cpu or gpu?
     
  21. Stormy102

    Stormy102

    Joined:
    Jan 17, 2014
    Posts:
    495
    @Cyril-Jover Any plans to make the De-Lighting tool available as a package in the Unity package manager?
     
    Thall33 and JakubSmaga like this.
  22. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    2,119
    I did some 3d capture with an iSense. it produces an obj and a color texture map.
    It seems that delighting tool could extract all necessary information from just these two data. Maybe a future version. In the meantime, how do I extract bent normal, normals and AO from just these ?
     
  23. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi syntheticlounge,
    sorry for the late answer !!!
    Putting the right data in the right inputs is not enough. You need also to setup the right parameters for the texture import. Using the naming convention forces Unity (when the de-lighting tool is installed) to import correctly the textures.
    Contact me again if you don't have better results, I'll try to react quicker this time !

    Cheers,

    Cyril
     
  24. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Janoshx,
    Sorry for the late answer !
    Can you show the results you've got? It's pretty difficult to know what goes wrong.
    Don't worry about the original white balance, I should be corrected after the process.
    My advice is to use neutral white balance for all your shots, this way your color will be corrected only once.

    Cheers,

    Cyril
     
  25. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi me109,
    The output is the same resolution as the input. Check the resolution of your input data and its import parameters to see if it is bigger than 1k.
    Don't hesitate to tell me if it is not the case.

    Cheers,

    Cyril
     
  26. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi neoshaman,
    This is running on GPU.

    Cheers,

    Cyril
     
    neoshaman likes this.
  27. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Stormy102,
    No plan for that right now, but it could be nice.
    I'll post here if we take this decision. Thanks for your support !!!

    Cheers,

    Cyril
     
    Stormy102 likes this.
  28. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi laurentlavigne,
    you can compute all the maps you need with Knald and Substance Designer. xNormal is nice too ... and free ! You just can't compute position map with this one and it is pretty slow, but sometimes it doesn't matter so much !

    Cheers,

    Cyril
     
    laurentlavigne likes this.
  29. XXXRarach

    XXXRarach

    Joined:
    Apr 5, 2018
    Posts:
    1
    Hi, I would be very interested in your further conversation on the topic of Faces delighting, could you please elaborate on this, and to what conclusion did you come. I am trying to use Knald to generate required textures, but without success yet. I am also trying to improve our photogrammetry scans from 40DSLRs and studio Lights, to remove even the small soft shadows from the face, but am not getting any luck with Unity Delighting tool, the colors after delighting are, damaged at best, it might be that I am using a wrong direction of normals and bent normals generated, or there is another problem there.

    Another issue is the absence of Position baking in Knald still - it is still not in the newest available version.

    I also tried XNormals, but without much success either. Any recommendation here? We would like to use your tool in our production pipeline in Bohemia Interactive, so any help and/or advice is highly appreciated :)
     
  30. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    2,119
    Thanks
     
  31. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi XXXRarach !
    As I said to 8infinite8, a clean sphere shape is the worst case for this de-lighting technique. I explain this in this video:


    Because the face is close to a smooth sphere, the results are not good with the official version of the de-lighting tool. But, because you managed to have an homogeneous lighting, the tool can be tweaked to achieve good results even in this case. I've made a test with 8infinite8 on a face scan, and the results were pretty good. Maybe I'll do a specific version to handle this case.I'll post here when there is some news about this.

    Cheers,

    Cyril
     
  32. bmc131

    bmc131

    Joined:
    Mar 23, 2018
    Posts:
    1
    I have input a 4K texture and delighted, but when I choose export it gives me a 2K texture. Can I set the output to be 4K?
     
  33. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi bmc131,
    are you using the naming convention (DLBC, DLAO ...) ?
     
  34. EstherZuidgeest

    EstherZuidgeest

    Joined:
    Jan 20, 2014
    Posts:
    3
    Hi Cyril!
    I hope you can shed some light on some artifacts im getting from my delighting. One side note to this model is that we moved the model around instead of the camera, this was done with a neutral white background.

    I've attached images to see the result, artifacts and our scan setup.

    On another note:
    There are some mistakes in the substance texture baking documentation. AO settings, there you should put the bent normal map in the normal map slot. Otherwise you only get ambient occlusion from your low res mesh.
    In the documentation its said to export everything except the position map to tga. Unfortunately TGA doesn't store alpha properly. Only when i stored my base color+alpha as a .png i get good results in the delighter.
     

    Attached Files:

  35. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70

    Hi EstherZuidgeest !
    Your normal map doesn't look good, I think you baked the lowpoly normal and not the projection of HD normal to Low Poly.
    Can you tell me what parameters you've used ?
     
  36. EstherZuidgeest

    EstherZuidgeest

    Joined:
    Jan 20, 2014
    Posts:
    3
    I've used the World Space Normal without any changes to the parameters.
    I've now tried the ' normal map from mesh' with map type set to world space.
    This map gives much better results in the delighter and got rid of the artifacts! thanks. :)
     
  37. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi Esther !
    Really glad to hear that !!!
    I've got another question:
    In the archive you provided, you've got the nice idea to join the ScanSituation.png file that shows the buddha sit on the floor. In your Texture, I think I've seen the bottom of the statue that is not visible in ScanSituation.png. Did you shot the bottom separately and combined the results ?
    In this case, it is better to remove the lighting of the two photogrammetry separately, then combine the results.

    Cheers,

    Cyril
     
  38. metteigel

    metteigel

    Joined:
    Aug 6, 2012
    Posts:
    33
    So i get some strange colors when i try to delight dark areas. Is there a way to fix this ?

     
  39. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70

    Hi metteigel,
    Yes, this is strange !
    Could you post what's your baked maps look like ?
    Did you color correct your photo input ?

    Cheers,

    Cyril
     
    metteigel likes this.
  40. xavierck

    xavierck

    Joined:
    Apr 28, 2018
    Posts:
    1
    I'm trying out the De-Lighting Tool on a fresh install of Unity, and getting this error:

    upload_2018-5-3_10-30-8.png

    Any idea what I can do to fix this problem?

    Thanks in advance for any help you can provide!
     
    MarkusGod likes this.
  41. metteigel

    metteigel

    Joined:
    Aug 6, 2012
    Posts:
    33
    Hey Cryril,
    here are the Maps. Not Correction or something was made to this Point. Straight out of the Marmoset Baker.

    I also added the Bakes (and pm'ed you) to my Dropbox if you want to check them out.





     
    Last edited: May 3, 2018
  42. xrooshka

    xrooshka

    Joined:
    Mar 5, 2014
    Posts:
    57
    Hi there! I'm doing some tests with photogrammetry and I've begun with some asphalt road texture. I tried de-lighting tool. Result is black upload_2018-5-6_1-20-18.png
    What am I doing wrong?

    [UPD]

    I've used Unity 2018.1.0f2, De-lighting v1.0.14 was downloaded from the Assetstore. This screenshot is made on HDRP project, same thing with build in render pipeline.

    One thing I admit. Computing on project with buildin render pipeline was quicker than the same on HDRP project. But maybe I'm mistaken. Rather or not the result is black )))

    [UPD#2]

    upload_2018-5-6_4-57-55.png
    It works in 2017.4.2! Works well. So it looks like no support for Unity 2018. Looks strange to me, I hope ot will be fixed soon.
     
    Last edited: May 6, 2018
  43. tmammela

    tmammela

    Joined:
    Sep 28, 2015
    Posts:
    27
    Hi, I have been trying to use this tool for delighting small parts of a city. 3D data is textured CityGML derived from photogrammetry done from satellite photos.

    Yes it's not what this tools made for, but it actually works well enough, but there is one breaking problem: I get white triangles that are not supposed to be there. I have double and triple checked geometry, normals, baking quality (used both SD and Marmoset) and I can't get rid of them. Any pointers that could help me?

    upload_2018-5-8_12-31-42.png

    If I use 8 bit textures, then I get huge amounts of noise but the triangles don't appear.

    upload_2018-5-8_12-29-6.png
     
    Last edited: May 8, 2018
  44. EstherZuidgeest

    EstherZuidgeest

    Joined:
    Jan 20, 2014
    Posts:
    3
    Your normal map and bent normal map look quite different. Normally they look quite similar. They should both be object or world space.

    I also have a question of my own. I've been using the tool quite succesfully, at least so I think. But i have a little doubt since in the documentation it says your LUT should cover mostly the top left traingle of the square area. Here is the documentation example and then my own results.

    LUT.jpg
     
  45. GameDevCouple_I

    GameDevCouple_I

    Joined:
    Oct 5, 2013
    Posts:
    2,183
    Just wanted to chime in and say that bent normals are not always looking the same, it very much depends on the geometry etc. they will definately be similar but there can be stark differences, especially on curved surfaces
     
  46. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Answer part 1/2

    Hi Metteigel,

    Here are the best results I’ve got using the data you send me:

    Rockwall.jpg

    Some parts are "ok":

    2018-05-30_20-16-54.gif

    2018-05-30_22-54-26.gif

    Some are not really good :

    2018-05-30_22-59-14.gif

    I'll send the file to you by private message



    Here are my suggestions to improve this result:

    About Position map:
    When you're baking on a plane, a position map is not mandatory. Use it when there is a lot of UV Clusters.


    About the Bent Normal map:
    The baked Bent Normal map seems very strange. The axis orientation is different between Normal map and Bent Normal map. Even after switching axis, the Bent Normal look strange for me. I'm not sure that Marmoset is the best to bake this kind of map (I haven’t tested). Instead of using your Bent Normal, I've used the Normal map in the Bent Normal Texture field.
    - The Result could be much better with a proper Bent Normal Map !!!
    - I suggest you to bake all your maps with the same tool.
    In your case, because you don't really need position map, you can use xNormal. It's a good tool and it is free.


    About the Mask map:
    There is moss on the rock, and it's not homogeneously distributed on the whole surface. So we can consider that there is two different materials here, the rock and the moss. In this case, it is recommended to create a mask that separate the main material from the other. The mask should be red everywhere except on the main material (here the rock) where it should be black.

    Rockwall_mask.jpg


    This is shown in those doc/video:

    doc: https://github.com/unity-technologi...ightingtool/documentation/de-lightingtool.pdf

    Tuto:
     
  47. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Answer part 2/2

    About Ambient Occlusion:
    When some parts of the scanned object are very dark because they are very occluded(like holes or cracks), the reconstruction of the mesh is often wrong.

    AO_Miss.jpg

    Instead of having holes, there is flat surfaces (like a blocked hole), and in those areas, the AO should be much darker than it is: there is a mismatch between real AO and reconstructed AO.
    To compensate this problem, the parameter Separate dark areas can be used. It will “inject” the darkest parts of the Base Color map in the AO map. Don't push it too far, the better results I’ve had (in similar cases) were with values lower than 0.3.

    I hope my answers help you, don't hesitate to give me feedback.
    Cheers,

    Cyril
     
  48. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi xrooshka,
    and thanks for the multiple updates, I'll investigate because it was tested on 2018 (non HDRP) and was perfectly working. It is highly recommended to keep the De-Lighting Tool as a separate project. In this case, switching to HDRP doesn't have interest, but it is mandatory to have the color space set to linear.

    Cheers,

    Cyril
     
  49. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi tmammela,
    Can you send me your data (mesh high/low + textures) that I can check what's going on ?

    Cheers,

    Cyril
     
  50. Cyril-Jover

    Cyril-Jover

    Unity Technologies

    Joined:
    Oct 28, 2015
    Posts:
    70
    Hi EstherZuidgeest,

    first, thanks a lot for your help in answering users question !

    You're right your LUT look strange but, in fact, I think it is ok.

    LUT1.jpg

    I think 95% of your points is in the green area. Even if the red part seems to be quite as big as the green one, points can be accumulated in the same area. In fact, all the points of a 4k Base color map could be accumulated in this small LUT. The points in the red part could be the result of 1000 bad points. In comparison to 16 millions points of a 4k texture, it is very low.

    I can guess most of your points are accumulated in the green area because we can see some kind of a consistent color gradient there. In the other parts, it looks like much random.
    Your de-lighting result seems to be good even if I feel that it is maybe a bit too bright. Placing the color chart in the direct light may lead to problems with color correction (some parts may be overexposed). Maybe it comes from this.
    upload_2018-5-31_1-28-58.png
    Based on this picture (from your website) I think the average albedo luminance should be somewhere between the 2nd and the 3rd grey reference (from top to bottom).
    Excellent photogrammetry workflow documentation BTW, everyone should look at it:
    http://www.estherzuidgeest.nl/portfolio/2-uncategorised/35-photogrammetry.




    So why is there some points in the red area?

    It can happen when:
    - all the maps are not baked with the same tool or with the same settings.

    - maps have been resized. There is no specific resizing feature for Vector map in authoring tools (e.g. Photoshop). Normal Maps are Vector Map and not Color Map, and image resizing is mainly designed for Color Maps.In some cases It can generate few artefacts.

    - Some non-consistent parts contribute to the LUT. Cluster borders or some part of the background can generate bad points in the LUT if they are not correctly masked (or black in alpha).




    Why the left part of the green triangle is missing ?

    AO_values.jpg

    The green triangle is filled from the left to the right using AO values. The left part is the lowest AO values and the right part is the highest.

    upload_2018-5-31_0-53-1.png

    2018-05-31_01-16-06.png

    As we can see on the baked AO map, there is no highly occluded parts, so it doesn't cover the full range of possible AO values. It approximately starts from 0.5 to 1, that's why there is nothing on the left part of the triangle.

    I hope I answer clearly to your questions, don't hesitate to ask me more details,

    Cheers,

    Cyril
     
    Last edited: May 31, 2018