Search Unity

Photogrammetry in Unity: Making Real-World Objects into Digital Assets

Discussion in 'General Graphics' started by SebLagarde, Mar 15, 2018.

  1. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    435
    This thread allow to continue the discussion about the blog post: Photogrammetry in Unity: Making Real-World Objects into Digital Assets



    It will be use to update users on the status of the demo as new update for new platforms will come later.

    The demo focus on Photogrammetry. To illustrate how good Photogrammetry authored assets work fine in various lighting condition the demo come with 3 different lighting variants.
    The other goal was to be close to a game production workflow for the level and the the main target is PS4 at 1080p at 30fps.

    The first and third navigation mode and the character animation (Download from the assets store in state) are not the focus of this demo and are available simply to give a game look feelings.

    This demo is compatible with 2018.1b10 and above.

    Current 2018.1b10 link:

    The project for can be downloaded here.
    The pre-built PC stand alone player can be downloaded here.
    A PlayStation 4 pre-built version of this project is available to registered developers on the Unity forum on the PlayStation 4 developer site here.
    The documentation about the demo can be downloaded here.

    Current 2018.2b8 link (Use HDRP package 2.0.3-preview):
    The project for can be downloaded here.
    The pre-built PC stand alone player can be downloaded here.

    Current 2018.2f2 link (Use HDRP package 2.0.5-preview):
    Improvement: Replace fake light shafts by volumetric fog + some cosmetic improvement
    The project for can be downloaded here.
    The pre-built PC stand alone player can be downloaded here.

    A video of the demo is available here

    [EDIT]: Fontainebleau demo have been updated and publish on github for 2018.3 and later version at this link: https://github.com/Unity-Technologies/FontainebleauDemo
     
    Last edited: Jan 18, 2019
  2. Disruptive-Digital-Studio

    Disruptive-Digital-Studio

    Joined:
    Oct 24, 2015
    Posts:
    2
    Hello, opening the project in Unity 2017.3.1 i have this error : "ShaderIncludePath' could not be found. Are you missing an assembly reference?", do you have any solution ?
     
  3. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    435
    Hi, I update the introducing text to make it more clear. This demo is compatible with 2018.1b10 and above. Sorry.
     
    antoripa likes this.
  4. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,061
    Have you guys tried implementing AutoLOD or some other form of mesh combination approach for this? Static geometry could all be combined into a single mesh and culled with compute shader. Trees and plants could be combined too and animated through shader.

    I'd be curious to see how far we could go with performance in this demo. Seems to me like there are still many opportunities for improvements, and it would be great to have a public project that focuses on these sorts of things. Though I'm guessing the Book of the Dead demo will be more seriously focused on performance than this one

    Are there plans to facilitate these sorts of drawcall-reducing optimizations in the HDPipeline? I think I've seen vague hints at a "new batcher" on the github repo
     
    Last edited: Mar 15, 2018
    JamesArndt likes this.
  5. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,061
    One more thing; this is 100% subjective, but....

    I feel like the PostFX decisions in this demo don't really do it justice. The image looks very blurry/muddy and overburdened with visual pollution. It looks like someone stuck their fingers into the camera lens. I thought Book of the Dead was kinda guilty of this too, but to a lesser extent.

    With some very simple tweaks that have no negative impact on performance, we can
    • sharpen the TAA (or use SMAA)
    • remove the DOF
    • make the bloom look more subtle and tasteful
    • tone down the vignette
    • remove chromatic aberration (or as I prefer to call it: "chromatic abomination")
    The game now looks sharp, clean and beautiful!

    Here's a comparison:
    Blurry, "crank everything to 11" post-processing

    Clean, elegant post-processing
     
    Last edited: Mar 15, 2018
    satchell, Rich_A, cfree and 4 others like this.
  6. VavylonaEntertainment

    VavylonaEntertainment

    Joined:
    Oct 21, 2015
    Posts:
    143
    PhilSA what would be the difference with shader wind? Wouldnt it look more fake?
     
  7. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,061
    Actually I'm pretty sure individual trees are already animated through shader in this demo right now, but the thing I'm talking about is to turn all trees into a single mesh (a single draw call) that is animated through shader as if they were individual trees

    It would give the exact same end result visually, but with a potentially immense performance boost

    By using this sort of technique in the past, I've seen my framerate go from 15 fps to 200 fps
     
    Last edited: Mar 15, 2018
  8. Thall33

    Thall33

    Joined:
    Sep 18, 2013
    Posts:
    53
    Thank you Unity for releasing this, I spent the better part of 6 hours pulling it a part and love the integration of photogrammerty assets into a custom terrain mesh that works so seamlessly! The value of the new vertex animation for use with vegetation is a big win too! I'm looking forward to using these new technics in our studios next game!

    Figuring out a solid workflow for iteration and testing the custom terrain meshes will take some time, I was wondering if there will be any videos, or docs about this? It would be so appreciated!
     
  9. NorthStar79

    NorthStar79

    Joined:
    May 8, 2015
    Posts:
    57
    Wouldn't it harm occlusion culling and cause some other performance issues? how do you manage that?
     
  10. VavylonaEntertainment

    VavylonaEntertainment

    Joined:
    Oct 21, 2015
    Posts:
    143
    Where can i read about shader-animated and wind-animated?
     
  11. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,061
    this definitely isn't compatible with Unity's default occlusion system, but you can write shaders whose job is to handle occlusion for this new approach (example)

    However, I never reached that point in my own project, and @SebLagarde probably knows way more than me about this
     
  12. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    832
    Wow, that looks awesome. I guess the next step to give it more life would be particles like dust motes, butterflies, and so on.

    Oh, and firebreathing dragons :D

    unka-12.jpg

    unka-09.jpg

    @Unity Team: thank you very much, this is most awesome and exciting!!!
     
  13. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    832
    What's the purpose of these invisible walls I keep on running against?

    invisible.jpg
     
  14. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,591
    Ah your dragons hehehe. Anyway i tested this scene and its look cool but question is: do you have kind of baker or something ti batch grass i to sectors?

    We checked wind and its really cool. Our trees works on it from today.

    Scene is really cool but i now it crash everytime when i try to hit play :/
     
  15. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    832
    Was the same for me most of the time with Unity 2018.1.0b11. I figured it doesn't crash when I use the main menu function to load the various day scenes exclusively, i. e. hit Load Levels -> Day.
     
  16. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    832
    I've just read the naming convention. You are using

    _MSK = (R) Metalness
    (G) Ambiant Occlusion
    (B) Emissive Mask
    (A) Smoothness

    Is there some convenient tool to combine these to a single image (other than photoshop)?
     
  17. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,591
    Nope but same masks I used before unity 2018. All my asset got this convention all the time because instead of 3 or 4 textures you use only 1. This kind of format is used in unreal too. Adam Goodrich had such tool somewhere, I help him a bit in this. Ask him maybe he will share it to you:).

    I use menu and got crash during scene load. It works only when i manualy drag and drop scenes to load.
     
    TreeMSI and Mark_01 like this.
  18. antoripa

    antoripa

    Joined:
    Oct 19, 2015
    Posts:
    1,149
    Just ping me in the afternoon. We have a channel packer in Cascade.
    Nice demo, nice shaders.
     
    Rowlan likes this.
  19. antoripa

    antoripa

    Joined:
    Oct 19, 2015
    Posts:
    1,149
    Colliders delimit the walkable area. Pretty common approach
     
    JamesArndt and Rowlan like this.
  20. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    832
    yeah, thought so. but it's rather dense. one could walk further without noticing you're in a limited scene :)
     
  21. asethi

    asethi

    Joined:
    Jul 24, 2012
    Posts:
    16
    I just downloaded the Fontainebleau Demo from repo and opened it in 2018.1.0b11 but none of the scenes open correctly. Can't even see the Scene correctly. I'm using Windows Desktop with 12 GB RAM and Nvidia 1050Ti Graphics Card. Is there anything i'm missing, because all other high end projects open correctly or even the current one on which i'm working works smoothly. Please would like to experience your demo correctly and learn about the workflow in detail. Below is a snaphot of one the scenes, where majority of the graphics is not rendering and there are few errors of Scripts Missing. Tried even running the Video Timeline scene, it plays only sound and nothing else is rendered.
    upload_2018-3-22_23-29-34.png
     
  22. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    832
    Yeah, read the documentation, I made the same mistakes because I didn't find one. It's in the first post. In short: Use the Load Levels menu instead of loading the scenes directly. There's some multi-scene functionality being used, i. e. one scene for the graphics, the other for day / sunset / night.
     
    asethi likes this.
  23. asethi

    asethi

    Joined:
    Jul 24, 2012
    Posts:
    16
    Thanks @Rowlan for the swift response. I read the documentation and followed it, but it crashes the editor every time after giving many warnings about missing behaviors and scripts. I have just snapped the crash a minute ago.
    https://pasteboard.co/Hd72WRm.png
     
    antoripa likes this.
  24. antoripa

    antoripa

    Joined:
    Oct 19, 2015
    Posts:
    1,149
    I got same issue. Exploring the code I could get all working properly starting with this scene setting
    Just check that latest is the active scene as it has all the managers

    upload_2018-3-22_19-20-10.png
     
    asethi and Rowlan like this.
  25. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    832
    For me it always crashes when I load the Video_Loader scene and hit play. When I use the menu Load Levels -> Day, it works without problems.
     
    asethi likes this.
  26. asethi

    asethi

    Joined:
    Jul 24, 2012
    Posts:
    16
    Awesome it works now. Thanks a lot!
     
    antoripa likes this.
  27. asethi

    asethi

    Joined:
    Jul 24, 2012
    Posts:
    16
    Load levels is not working till now, but I will keep looking into it. Thanks!
     
  28. antoripa

    antoripa

    Joined:
    Oct 19, 2015
    Posts:
    1,149
    Yes, it does. And starts faster as well
     
    Rowlan and asethi like this.
  29. asethi

    asethi

    Joined:
    Jul 24, 2012
    Posts:
    16
    Yeah it does. Have to explore each module and code as you said to get the hang of it.
     
    antoripa likes this.
  30. antoripa

    antoripa

    Joined:
    Oct 19, 2015
    Posts:
    1,149
    In my humble opinion, If you are a graphics programmer, I would suggest checking code for pipeline and shaders. It is a good exercise to start getting familiar with the HD SRP as not documented so far. That is what I have done
     
    Rowlan and asethi like this.
  31. asethi

    asethi

    Joined:
    Jul 24, 2012
    Posts:
    16
    Yeah, got it. I'm not a core graphics programmer, but a Generalist. In my current workplace, we are gonna start a project for Real-Time rendering use case with Photogrammtery Workflow, SRP and Multi-Scene functionality in Team designing and development workflow. Hence, wanted to check how Unity has used all these three and incorporate / learn from it.
     
    antoripa likes this.
  32. Kory-therapy

    Kory-therapy

    Joined:
    Jun 5, 2013
    Posts:
    17
    Is their any youtube videos on how to get this to work?
     
  33. rahuxx

    rahuxx

    Joined:
    May 8, 2009
    Posts:
    533
    can we use this asset for commercial app?
     
    AntonioModer likes this.
  34. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    435
    Yes the license allow it, as long as it is a Unity product :)
     
  35. chanfort

    chanfort

    Joined:
    Dec 15, 2013
    Posts:
    591
    Hi Seb, good to hear this. Is it Unity Companion Package License v1.0 (i.e. the one which is used for most of Unity things on GitHub)?
     
    AntonioModer and rahuxx like this.
  36. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    435
    AntonioModer, rahuxx and chanfort like this.
  37. tinyant

    tinyant

    Joined:
    Aug 28, 2015
    Posts:
    101
    HI Rowlan , Did you create your custom shader for the Dragon's Fire? that's so amazing!
     
  38. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    832
    That's what comes out of the box with the Unka asset. It's just particles. HDRP makes it look good.
     
    Alverik and tinyant like this.
  39. jakes-du-preez

    jakes-du-preez

    Joined:
    Aug 14, 2013
    Posts:
    30
    Hi Anybody got this running in the new released version og 2018.1. I extracted to a new folder and then opend the folder with 2018.1 but get error.

    error CS2001: Source file `Assets/ScriptableRenderPipeline/ScriptableRenderPipeline/HDRenderPipeline/HDRP/Editor/Lighting/Reflection/Volume/SerializedReflectionProxyVolumeComponent.cs' could not be found


    Thank you
     
  40. DanKing100

    DanKing100

    Joined:
    May 8, 2018
    Posts:
    1
    Testing it out now...
     
  41. Grimreaper358

    Grimreaper358

    Joined:
    Apr 8, 2013
    Posts:
    535
    I agree that the post looks muddy, I actually went in and changed the post-processing settings when I opened the project a while back.

    Chromatic Abortion wasn't a part of it, the reason it's muddy is the bloom.

    This is the default post-processing with
    Bloom On - Left Bloom Off - Right
    Untitled-1.jpg

    This is the version I slightly edit
    2018-05-11_11-12-15.jpg

    Just the bloom off and it's much clearer.
     
  42. Sephyrus

    Sephyrus

    Joined:
    Dec 26, 2015
    Posts:
    2
    It was mentioned in the documentation of this project that you can vertex paint to determine where each material is placed for the Layered Lit shader however it was not mentioned how to do it, nor have I seen an option while fiddling with it.
     
    JamesArndt likes this.
  43. JamesArndt

    JamesArndt

    Joined:
    Dec 1, 2009
    Posts:
    2,669
    I'm not sure if this is what they used, but I have used it myself in recent projects. It's made by Unity Technologies so I'm assuming this will become a built-in painting tool for the Editor.

    https://www.assetstore.unity3d.com/en/?stay#!/content/111427

     
  44. JamesArndt

    JamesArndt

    Joined:
    Dec 1, 2009
    Posts:
    2,669
    This project is awesome. I am considering using some of the forest assets like the rocks and trees in a project. This project though is not using the HD rendering pipeline so I don't have access to the layered shader and stuff. I am wondering if I can find a shader that does something similar for the rocks and ground and trees. Basically allow me to vertex paint blending on these with a layered material not within the HD Render Pipeline. I know I have to strip out the channels from the mask texture to use them in a regular standard shader. I would love to just start with the HD render pipeline and use those materials but unfortunately I already authored a lot of art content using the regular Standard shader in 2018.
     
  45. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    435
    >It was mentioned in the documentation of this project that you can vertex paint to determine where each >material is placed for the Layered Lit shader however it was not mentioned how to do it

    We mention it in the doc of the project, we use
    https://github.com/slipster216/VertexPaint

    We have updated project for latest Unity beta:

    Current 2018.2b8 link (Use HDRP package 2.0.3-preview):
    The project for can be downloaded here.
    The pre-built PC stand alone player can be downloaded here.

    (Edit: updated the link, they were wrong)
     
    Last edited: Jun 16, 2018
    tinyant, Mark_01 and antoripa like this.
  46. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    435
    We have updated project for latest Unity 2018.2 release:

    Current 2018.2f2 link (Use HDRP package 2.0.5-preview):

    Improvement: Replace fake lens flare by volumetric + some cosmetic improvement
    The project for can be downloaded here.
    The pre-built PC stand alone player can be downloaded here.
     
  47. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,015
    Did you find a way (and a shader) to do this?
     
  48. relativegames

    relativegames

    Joined:
    Nov 22, 2013
    Posts:
    23
    On UWP this project gives a build error and in editor the image is completely black :|
    UPDATE: You need to comment out all the #pragma only_renderers lines from all shaders. Next going to try to put it on my xbox
    UPDATE2:Anything with LWSRP or HDSRP craashes on UWP :/
     
    Last edited: Sep 30, 2018
  49. keeponshading

    keeponshading

    Joined:
    Sep 6, 2018
    Posts:
    318
    Fontainebleau. Book of the Dead. FPSSample for vertical slicing.
    SRP with HDRP and LWRP. ShaderGraph for booth, now VFX Graph.

    Never learned so much in such a short time from real framerates compared to marketing videos others show.)
    2018. UberAmazing and probably the only way to rock SRP LWRP and HDRP.

    Do you plan another update of Fontainebleau for
    > Unity 2018.2.14f1
    > HDRP 5.1.0-preview
    ?


    We could attach it then as Level02 and Book of the Dead as Level03 to FPSSample as photogrammetry examples. Only because we can.)



     
    Last edited: Nov 6, 2018
  50. laurenth-unity

    laurenth-unity

    Unity Technologies

    Joined:
    Sep 29, 2016
    Posts:
    37
    Thanks for your enthusiasm :)
    Is the 2018.2.0f2 version no longer compatible with 2018.2.14f1 ?

    The current plan is to update the project for unity 2018.3 and HDRP 4.1.0

    HDRP 5.X.X are now for 2019.1