Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Official Feedback Wanted: Streaming Virtual Texturing

Discussion in 'Graphics Experimental Previews' started by AljoshaD, Mar 18, 2020.

  1. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    Hi Lars, yes I expect that to work. I'm curious, how do you expect to use VT in your mobile projects? How large in terms of storage size do you think your project can be?
     
  2. PerunCreative

    PerunCreative

    Joined:
    Nov 2, 2015
    Posts:
    113
    Hi,

    is there ETA for URP support? Is it planned for 2020.2 or it will arrive after that?
     
    futurlab_xbox likes this.
  3. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,521
    Last edited: May 19, 2020
  4. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    URP is high on our priority list. We'd like to support it as soon as possible but we first want to make sure Streaming VT has all the features you need to ship a game and the quality is good. We might have URP support in a preview package during the 2020.2 cycle, it's not entirely clear yet at the moment.

    VT support has landed in HDRP master and we'll share an updated sample project that will use the HDRP 9-preview package that will be promoted (and available in the package manager) soon for 2020.1.b11. This will allow for more experimentation with Streaming VT but shipping a title is not advised before it is out of preview in the HDRP 10 package for 2020.2. We won't have assetbundle support in 2020 so all VT streaming data is stored in the Player build. This is a critical feature for many projects and it's our main focus now.
     
    futurlab_xbox likes this.
  5. Peter-Bailey

    Peter-Bailey

    Joined:
    Oct 12, 2012
    Posts:
    36
    Will VT be used to increase the resolution of dynamic shadow maps? Shadow quality is sorely lacking in Unity and has been for many years. It's about due for an update, wouldn't you say? What would be the point of having super hi-res textures if your shadow maps are still low-res with abrupt transitions between cascades? It would be nice to be able to achieve super sharp, pixel perfect shadows with little to no jaggies that we can soften at our leisure. Like applying a smooth-step filter to the shadow map.

    https://www.eurogamer.net/articles/...eal-engine-5-playstation-5-tech-demo-analysis

    "Really, the core method here, and the reason there is such a jump in shadow fidelity, is virtual shadow maps. This is basically virtual textures but for shadow maps. Nanite enables a number of things we simply couldn't do before, such as rendering into virtualised shadow maps very efficiently. We pick the resolution of the virtual shadow map for each pixel such that the texels are pixel-sized, so roughly one texel per pixel, and thus razor sharp shadows. This effectively gives us 16K shadow maps for every light in the demo where previously we'd use maybe 2K at most. High resolution is great, but we want physically plausible soft shadows, so we extended some of our previous work on denoising ray-traced shadows to filter shadow map shadows and give us those nice penumbras."
     
  6. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    Hi Tim, my team is responsible for building the VT system in Unity. I can't say much about how other teams within Unity will leverage VT but these things become straightforward once VT is available to build features on.
     
    futurlab_xbox likes this.
  7. Peter-Bailey

    Peter-Bailey

    Joined:
    Oct 12, 2012
    Posts:
    36
    Cool! Thanks!
     
  8. Onat-H

    Onat-H

    Joined:
    Mar 11, 2015
    Posts:
    195
    Do you already know when HDRP 9 will be available for 2020.1 in the PM? It would be great to use a more recent version of the pipeline instead of the "test project - version".
     
  9. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    2,683
    It's already available for a couple of weeks (if not month).
    upload_2020-6-4_13-54-46.png
     
    Onat-H likes this.
  10. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    We hope to promote a new HDRP 9 preview package within the next two weeks that has VT support. This will be available through the package manager. The VT docs will be online soon as well. We'll share an upgraded sample project at that point.
     
    futurlab_xbox and Onat-H like this.
  11. Onat-H

    Onat-H

    Joined:
    Mar 11, 2015
    Posts:
    195
    Great, thanks :)
     
  12. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    I'm currently testing this with bleeding edge 2020.2 alpha and HDRP from github repo. I like the new setup but the debug tiles menu entry is now gone. How do I enable this debug mode now?
     
  13. Grimreaper358

    Grimreaper358

    Joined:
    Apr 8, 2013
    Posts:
    789
    Any issues you noticed with it? I tested the original project from the first post but haven't messed with it since then. I'll have to make a project and test see.
     
  14. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    There's 9.0.0-preview.35 SRPs released on Package Manager today, so maybe worth trying with it :) I've only tested with bleeding edge 10.x myself.

    I couldn't make it work in build with DOTS hybrid rendering and didn't figure out how to get debug tiles showing up but other than that, the new setup is more streamlined which is kinda cool.
     
  15. Grimreaper358

    Grimreaper358

    Joined:
    Apr 8, 2013
    Posts:
    789

    Ah, alright. Yea, I mainly use 10.x.x from Github as well. I just never tried VT with it and the new setup yet. So I asked to see if there are any issues and I could just enable it in one of my projects without having to make a new one just to test it. They are all test projects but I wouldn't wanna enable a feature and then have to go back and do cleanup if it might break something for a little while.
     
  16. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    wow you guys are fast! :) We indeed promoted a new 9-preview package today. We are doing some final testing of our upgraded sample but it should be ready to share here soon.

    We polished the namespace a bit so some things have moved (in 2020.1.b13 and up). You can see the VT tile debug overlay by toggling UnityEngine.Rendering.VirtualTexturing.Debugging.debugTilesEnabled

    Edit: Hybrid Renderer / Dots doesn't work with VT at the moment.
     
    Last edited: Jul 3, 2020
    futurlab_xbox and rz_0lento like this.
  17. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    I did get it running in the editor but build rejected the shader graph due to no srp batcher compatible passes but this could be an issue with my modified bleeding edge HDRP. I can try again with just released HDRP 9.0.0-preview.
     
  18. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    9-preview will only work with 2020.1.

    Edit: VT does not work with hybrid renderer / Dots in 2020.1 and 2020.2. Strictly speaking it works with hybrid renderer but the VT data is not built when you put materials in subscenes.
     
    Last edited: Jul 3, 2020
    futurlab_xbox and rz_0lento like this.
  19. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    I'm confused now what works and with what :D
    You are backporting fix for 2020.1 but also backporting a another fix for 2020.2? So basically it shouldn't work on either atm?

    For the ref, when I tried bleeding edge HDRP from hdrp/staging on 2020.2.a16, I also had this PR merged to it: https://github.com/Unity-Technologies/Graphics/pull/910. Again it did work in editor, just not in build (maybe something got stripped from SG on build).

    But yeah, I'm not in a hurry, I'll wait.
     
  20. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    VT won't work with hybrid renderer/dots in 2020.1 and 2020.2. I've edited my earlier messages about this.
    The fix I mentioned does allow hybrid v2 to work with VT and will land in 2020.1 and 2020.2 soon. However, the materials with VT that are added to a subscene are not picked up by the VT builder so the data can't be found in the player. In the editor the VT tiles are extracted from regular Unity Texture2D assets so they don't need to be built. We are thinking about a workaround until we have full subscene/asset bundle support but right now VT won't work with subscenes/dots.
     
    futurlab_xbox likes this.
  21. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    We updated the sample project for 2020.1. You can download it here. It works with beta 14 and up.

    The sample uses the HDRP 9-preview.33 package that you can find in the package manager when enabling preview packages in the project settings.

    The Streaming Virtual Texturing docs are online here.

    We look forward to your feedback!
     
    Last edited: Jul 3, 2020
    futurlab_xbox, m0nsky and BATTLEKOT like this.
  22. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    What happens if I plug a Custom Render Texture asset into a VT Stack node in shader graph? It can be plugged into a regular texture2D sampler node, but the downside is that it doesn't use compression like a regular texture file. So, I was wondering if it could take advantage of virtual texturing in terms of memory consumption.
     
  23. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    Apparently VT's are partially supported by DXR reflections now? It's bit sketchy still but when it works, it's looking like it should. I hope this is not a glitch and it's supposed to be supported :D

    It didn't work on 9.0.0-preview.33 but it does seem to work on 10.x from github while using DXR reflections performance mode (quality mode just results in black reflection).
     
  24. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    A custom render texture asset won't work with Streaming Virtual Texturing. We are working on Procedural Virtual Texturing where you can render into texture tiles but we still have some work to do. We are doing a survey to collect requirements about PVT. You can find that here.

    With regards to DXR/Raytracing, we are testing this now. It should work indeed but you'll need a good prefetching strategy because the VT system will only automatically stream what is directly visible by the camera. With a prefetching strategy like the experimental script in the sample you at least have some low res content in the cache for texture areas that are not directly visible by the camera.
     
    futurlab_xbox, m0nsky and rz_0lento like this.
  25. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    257
    I've been using DXR + Virtual Texturing for a few days now. I would like to experiment with this using a 16k PBR material that uses a displacement map and/or tessellation. Is there any temporary workaround (as I'm limited by Shader Graph here) I can use to apply VT to the Lit/LitTessellation shader?
     
    Last edited: Jul 15, 2020
    rz_0lento likes this.
  26. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    Nice!! Thanks for sharing your work!

    We aren't supporting the built in shaders unfortunately.

    Technically, you can hack it into the lit shader yourself. You can see how it works by inspecting a shader generated with ShaderGraph. The VT functionality can be found in com.unity.render-pipelines.core/ShaderLibrary/TextureStack.hlsl
    But it is certainly not advised to ship with this hack. Implementing all edge cases of the lit shader is a daunting task. Also, we will very likely change the shader library. It is not supported as public API.
     
  27. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    257




    I decided to go down the shader graph route and parallax occlusion mapping + virtual texturing seems to work like a charm.



    Currently, it doesn't seem to be possible to connect my virtual texture sample node to the input heightmap node because it's expecting a T2. I would love to see this if this isn't too much to ask.
     
    Last edited: Jul 9, 2020
  28. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    I'll ask the ShaderGraph team. It's not being worked on at the moment though. It will be pretty expensive because the heightmap would be sampled multiple times with VT, which is more expensive than a regular texture sample.
     
    futurlab_xbox and m0nsky like this.
  29. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    I'm also curious about URP support. I'm assuming this is now happening during 2021 cycle? Also same for procedural VT's?
     
    futurlab_xbox likes this.
  30. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    Indeed, URP support and Procedural VTs are coming in 2021. We might have URP support in some preview form earlier but it depends on getting Streaming VT robust and feature complete first.
     
    futurlab_xbox likes this.
  31. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    I do have some questions about this thing:

    CPU Cache is obvious but I'm not quite sure what to take away from the GPU cache settings. I understand the allocated size but I'm not sure what the format and type does here. I have few guesses:

    1) Format is something you specify for Granite to place them to texture stack for all the types, in this case all VT sRGB textures would be packed using RGBA_BC7 in 256MB and normal maps go to RG_BC5

    2) Format only picks existing textures that set to use RGBA_BC7 and place them in this 256MB slot etc.

    Reading this line from doc: "Virtual Texturing allocates the GPU cache of a given graphics format when it renders a Material that has a Texture Stack with a texture that uses that format." makes me believe option 2 is what is correct?
    But knowing that textures may have wildly different texture packing options, this would make way less sense so I'm really wondering if I just read the doc wrong and option 1 is what really happens?
     
  32. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    The Streaming Virtual Texturing system will allocate a GPU cache (i.e. a cache texture) per texture format. You can set exactly how much GPU memory you want to give each format.

    When you render with a material that is using VT, the system will examine the formats of the textures used in the VT properties. This gives us a list of texture formats used in the VT property. For each format, Granite/the SVT system will look for an existing GPU cache and if no cache exists it will create one with the size you set in the HDRP asset.

    Lets say you have a VT property (we also call this a Texture Stack) with two textures. One is color using the BC7 graphics format, one is a normal map using BC5. Then the VT system needs to create two caches. If you have more materials with VT properties, each could randomly have 1, 2, 3 or 4 textures (so number of layers), and all of the assigned textures have either a BC7 or BC5 format, then the VT system will not allocate more caches and just use the two existing ones. Once you render with a material that has a texture with another graphics format assigned to the VT property, for example a DXT1 texture, then the system will allocate another GPU cache for DXT1 content. The default size is currently 128MB per cache and you can set the size using the HDRP settings.

    In Unity 2020.2 we added a Virtual Texturing profiler chart so can can see what caches are allocated and how much of the cache is being used to render the current frame. We'll add more documentation to help guide you in setting the optimal sizes to maximize texture quality while minimizing memory usage. VirtualTexturingProfiler.png
     
  33. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    This would have been my follow up question. Profiler view doesn't give you info about allocated formats sizes per item though, only which one are there + total GPU cache size. Of course if you set all the sizes yourself, you'll know what's there.

    I had to test this in practice. So basically this is correct:
    Meaning... this could lead into some wild goose hunts while trying to find those textures that are set to "wrong" compression in case there's been custom settings for texture compression. Would be nice to be able to get some full list somehow on which textures were allocated in which slot.
     
    futurlab_xbox likes this.
  34. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    Additionally, this is probably a bad idea?



    In this graph, I have first VT layer for color + height (on alpha), second for mask map and third for normal map. I can't use the LOD mode for regular maps here because automatic streaming will not work with vertex stage. And if I use it without it, I get lower resolution texturing everywhere unless I deal this with custom code (which is kinda pointless as there is already code to handle this automatically).

    What would be the most efficient way to get feedback/automatic streaming to work and to be able to use the lod sampler?

    Edit: I see docs state this:
    but it really sounds pretty inconvenient to me.
     
    Last edited: Jul 16, 2020
    keeponshading likes this.
  35. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    yes, we'd like to find a good solution for this. With the profiler we intent to give you a clear view on what is going on. In the future we might have more explicit cache management or we have more metrics to clearly indicate what caches could be created as you mentioned.
     
    futurlab_xbox likes this.
  36. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    Having your heightmap as part of a texture property that you also sample with automatic streaming should work if you sample with the same uvs. Performance could be slightly worse than sampling a separate texture property with only the heightmap but that should be limited.
     
  37. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    257
    The example project posted in this thread worked fine up until 2020.2.0a18 but broke on 2020.2.0a19, is this a known issue?

    Edit:
    Hmm, it doesn't always seem to happen, I'll play with this some more.

     
    Last edited: Aug 1, 2020
  38. futurlab_xbox

    futurlab_xbox

    Joined:
    Nov 5, 2018
    Posts:
    22
    Hi @AljoshaD is there a roadmap for VT features and platform support? We're evaluating solutions for an Oculus Quest game concept that'd benefit greatly from this tech, and having a better idea on when it'd support URP + mobile (if ever!) would be great.
     
  39. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Enabling Virtual Textures crashes every project (even a completely new one) with Unity 2020.1 and HDRP 9.0 preview 33.
     
  40. jonaskhalil

    jonaskhalil

    Unity Technologies

    Joined:
    Apr 6, 2019
    Posts:
    33
    Hey @passeridaepc that definitely looks like something serious! Unfortunately I wasn't able to reproduce this yet ... Creating a fresh project in 2020.1.1f1, upgrading HDRP to 9.0 preview 33 and then toggling VT nicely re-opened the editor for me ...

    We'll be looking further into this, but it could be useful if you submit a report for this (https://unity3d.com/unity/qa/bug-reporting) to make sure this issue is tracked properly.
     
  41. jonaskhalil

    jonaskhalil

    Unity Technologies

    Joined:
    Apr 6, 2019
    Posts:
    33
    Upon further discussion (and carefully looking at the crash window header bar to see it's version b16 you've opened it with), it could be that this issue has already been fixed (and seems to be available somewhere in b17). That particular bug seemed to be related to the profiler window being open as well, though ... So actually, before opening a bug report, I would suggest:
    • if the profiler window is open in your layout, you could just close it and then you should be able to enable VT properly
    • otherwise, could you try again with latest 2020.1?
     
  42. Raul_T

    Raul_T

    Joined:
    Jan 10, 2015
    Posts:
    363
    Will there possibly be any way to use this for texture arrays?

    I understand if it might not be the case given the nature of texture arrays, just wondering since we've been working on a terrain shader that makes heavy use of texture arrays to render multiple PBR terrain layers in a single pass (16 at the moment but we might increase to 32 for our future projects) and in a lot of cases this is what takes up most of our project's vram budget. VT streaming would clearly be a benefit for us
     
  43. ChezDoodles

    ChezDoodles

    Joined:
    Sep 13, 2007
    Posts:
    107
    Any updates on this?
    I use the Parallax Occlusion Mapping node in ShaderGraph on almost everything and I also have it packed into a 4channel texture along with AO, Metal and Smoothness.
     
    m0nsky likes this.
  44. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    395
    Yes, that was the case! Thank you very much. The editor has successfully reloaded with the profiler window closed.
     
    jonaskhalil likes this.
  45. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    Hi @futurlab_xbox , we don't have a public roadmap yet.
    Right now we are focused on the quality of the feature and adding support for assetbundles/subscenes/addressables. We definitely will support Streaming Virtual Texturing in URP and we'd like to start working on that in a few months. After that, we can add additional platforms like mobile. I expect that will support mobile in 2021 but this is not locked down yet.
     
  46. hungrybelome

    hungrybelome

    Joined:
    Dec 31, 2014
    Posts:
    336
    I'm in the same situation—developing for the Oculus Quest, though I'm using the legacy RP. The built-in texture streaming doesn't seem to work for the Quest (SystemInfo.supportsMipStreaming returns true, but I can't get it work, even wih LZ4 compression). Really hoping to use this solution in the future!
     
    futurlab_xbox likes this.
  47. Raul_T

    Raul_T

    Joined:
    Jan 10, 2015
    Posts:
    363
    Hi @AljoshaD , any thoughts on this?

     
  48. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    220
    Hi
    Many of the use cases of texture arrays can be replaced with an atlas/virtual texture. We don't have plans to use VT for a texture array at the moment.
    In your example, VT might not be the solution if you need many samples per pass. The VT sampling cost per texture is higher. Also, for tiling detail/terrain materials, these are typically not ideal for memory saving because they are entirely in view most of the time.
     
    Raul_T and Lars-Steenhoff like this.
  49. flyer19

    flyer19

    Joined:
    Aug 26, 2016
    Posts:
    121
    vt.png
    unity2020.2.0a19 can not download vt.
     
  50. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    You do not need to do this on a19. Just enable Virtual Texturing from player settings and have compatible HDRP version installed.