Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Official Feedback Wanted: Streaming Virtual Texturing

Discussion in 'Graphics Experimental Previews' started by AljoshaD, Mar 18, 2020.

  1. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    388
    Hi!

    I want to add two "sample virtual texture" nodes inside my shader graph, but the second of them is intended to be optional. I'm planning to feed a set of custom lightmaps (4 texture maps) into each of these nodes and lerp between them in runtime in some situations. The thing is, I will fill both of the sampler nodes (with 8 textures in total) for some materials and only one node for the others. So one of the samplers may remain empty in some cases. Is it a bad approach? Is it better to create a separate shader with only one VT sampler for such scenarios?

    Also, If you use more than 8 regular texture 2D samplers in shader graph, you'll get an error "Maximum ps_4_0 sampler register index (16) exceeded". And you'll have to reuse sampler states. How does it apply to VT samplers, since they take 4 texture maps each? Do they count as 4 samplers? Or if they don't, does it mean, I can use 8 VT samplers (32 texture maps in total) without running into this error?

    And is it okay to use both VT samplers and regular 2D/3D samplers inside the same graph?

    Thanks!
     
  2. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    144
    Hi,

    currently, all texture slots need to be assign for each VT property or this will not work in the player.
    The number of samplers indeed rapidly increases with each texture sample. You have per VT sample 1 + NumberOfLayers amount of samplers.
    You can mix VT and regular texture samples in one graph.
     
  3. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    388
    So, if one 4-layered vt sampler = 5 samplers, then I can use only three 4-layered vt samplers per graph (12 texture maps in total). What do I do if I need more virtual texture maps? I'm aware of only two solutions for regular samplers in this case: use a texture array or use a sampler state node. VT Sampler, however, doesn't have a sampler state input and can't take texture array as far as I understand.
     
  4. TekHK

    TekHK

    Joined:
    Jul 30, 2014
    Posts:
    37
    I am getting an editor crash on build every time. It is a very simple test project - 32 x 4k textures, all set to VT only on import. One layer in the shader. HDRP, raytracing set to on. Virtual texturing enabled. Works fine in the editor I can see the virtual texturing profiler loading and unloading (although I see in the memory profiler that it is using texture memory which is wierd considering I set the textures to VT and applied). It basically crashes every time I do a build when it gets to "build player". Version is 2020.2.6. Everything is updated to the latest versions via package manager. Is this a known bug?
     

    Attached Files:

  5. ValentinHm

    ValentinHm

    Unity Technologies

    Joined:
    Dec 10, 2020
    Posts:
    9
    Hi,

    could you please file a bug report for this and attach a small repro project to it ? So we can try to reproduce it with the exact same settings as you.

    Here's the link for more informations about bug reporting: https://unity3d.com/unity/qa/bug-reporting
     
  6. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    935
    Hello there, with VT active, LOD crossfade doesn't seem to be working. Is this the intended behavior?
     
  7. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    481
    Ooo Interesting, have you checked the VT debug section in the RP debug menu? It may give you some clues.
     
  8. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    935
    I did and while VT works just fine, the LOD transition (dithered crossfade) is gone. It is a pop in pop out with VT involved.
     
  9. NicoLeyman

    NicoLeyman

    Unity Technologies

    Joined:
    Jun 18, 2019
    Posts:
    33
    Hi,
    You need to enable LOD Cross Fade support in your custom shader through ShaderGraph's Graph Inspector window:
    LODCrossFade.PNG
     
  10. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    935
    Thanks! Wasn't aware that LOD was a checkable list.

    Thanks for the quick rep!
     
  11. Onat-H

    Onat-H

    Joined:
    Mar 11, 2015
    Posts:
    190
    Hi, do you have any news about Adressables support for VT? Ist this still going to happen in the 2021 tech cycle?
     
  12. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    144
    Hi,
    assetbundles/adressables support for VT will not land in 2021.
     
  13. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,357
    keeponshading likes this.
  14. Kmsxkuse

    Kmsxkuse

    Joined:
    Feb 15, 2019
    Posts:
    296
    And transparent VTs. I'm making due with multiple cameras and overlaying them on top of each other but that is a performance cost I would prefer not to incur.
     
  15. ali_mohebali

    ali_mohebali

    Unity Technologies

    Joined:
    Apr 8, 2020
    Posts:
    114
    Hi @rz_0lento, we intend to expose the roadmap for the core graphics area as well and we are working on that. So yes VT and texturing area-related roadmaps will be exposed as well.

    Just to give an update, PVT, Addressables, and support for URP are not part of our roadmap in 2021, but we do have them in our backlogs. The team has put the majority of their effort and focus on quality and stability in texturing area in 2021. As for URP, the focus this year is similarly improving quality and performance next to bringing the pipeline closer to parity with the Built-in renderer. SVT support for URP will be a new feature and will not be the focus area for URP in 2021.
     
    m0nsky, JoNax97 and rz_0lento like this.
  16. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    388
    @AljoshaD and everyone else who knows this.
    1) Are VT compatible with ray-tracing?
    2) What does the lack of support of Asset bundles mean in practice? Can I still ship a game with a lot of high-res textures without asset bundles?
    3) Can regular texture streaming (for lightmaps especially) work along with VT?
    4) If I use VT Samplers in Shader Graph but then choose to disable VT in the project settings, will these samplers continue to work okay as regular 2D Samplers?
    Thanks!
     
    Last edited: Jun 18, 2021
  17. Ruchir

    Ruchir

    Joined:
    May 26, 2015
    Posts:
    925
    Is there a way to make my lightmaps in the scene work with VT?
    It would be really helpful, or at least if someone could point me to some resources for managing lightmaps in big scenes.
     
  18. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    144
    Yes and no. The resolving uses rasterization to figure out which tile to stream. So you'll need to manually request tiles to stream to work with raytracing. In practice, it will be difficult to get this working.

    that is correct. If you don't want to pack your textures/materials in assetbundles but bundle them with the Player then this is not a problem for you.

    yes, you can use both mipmap streaming and Streaming VT in one project

    Yes that should work currently although we are still thinking about how this "fallback" should work exactly.
     
  19. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    144
    not at this moment
     
  20. keeponshading

    keeponshading

    Joined:
    Sep 6, 2018
    Posts:
    929
    VT will really boost the quality of our HDRP scenes but without support of addressables i see no way to use them.
    Is it not possible to add base support for this in 2021. This would really help.
     
    Last edited: Jul 18, 2021
    eggsamurai likes this.
  21. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    388
    BTW, not-loaded VT textures affect reflection probes. They appear as default grey or black in the baked reflection probes. So, at the time of reflection probe baking, you virtually have to position the editor camera in such a way, that it sees all the objects that you want to be correctly present in the reflection. Or you have to look around, to force the loading of all the VT textures around you and then you have some time (half of a minute maybe) to bake your reflection probe.

    @AljoshaD Can we have VT textures always loaded when a reflection probe is being baked?

    Using latest 2021.2 beta 4 and HDRP12 Thanks!
     
    Last edited: Jul 28, 2021
  22. NicoLeyman

    NicoLeyman

    Unity Technologies

    Joined:
    Jun 18, 2019
    Posts:
    33
    Hi,
    These issues (reflection/lightmap baking and assetbundle support) are our top priority for VT. However our team is currently still doing some work to the core texture area which needs to be finalized before we can tackle new VT features. I'm hoping we can get these on the roadmap once the summer vacation season ends but again that will depend on the foundational texture work being complete.
    Regards.
     
  23. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    388
    @NicoLeyman
    @AljoshaD

    Can you please provide a minimalistic example of how to sample a virtual texture in a custom function node?
    It would be very helpful, thanks!
     
    Last edited: Aug 3, 2021
  24. cholleme

    cholleme

    Unity Technologies

    Joined:
    Apr 23, 2019
    Posts:
    24
    Sampling VT from custom function nodes is currently not supported. The shadergraph generation code for the VT node generates a lot of things which are not possible from a custom node.
     
  25. Passeridae

    Passeridae

    Joined:
    Jun 16, 2019
    Posts:
    388
    That's a pity. So there's no way to sample any texture that is not a material property?

    I want to sample some textures that are supplied through a material property block. Any chance I can do this?
     
  26. cholleme

    cholleme

    Unity Technologies

    Joined:
    Apr 23, 2019
    Posts:
    24
    A material's textures for VT are currently statically defined in the editor. When you build the player the VT textures for that material are "baked". This means you can't modify the material's VT textures through property blocks or other means (like material.SetTexture). This page https://docs.unity3d.com/Manual/svt-requirements-compatibility.html lists this and other limitations of the current VT implementation in detail. Removing several of these limitations are high-priority on our roadmap for improving VT.
     
  27. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    695
    What about making HDRP Lit and other built in shaders compatible with VT? Is that even planned?
     
    PutridEx and pbritton like this.
  28. SkySupra

    SkySupra

    Joined:
    Jun 22, 2014
    Posts:
    8
    Hello, all. I'm experimenting with how VT works. I can't find the answer to one question. I need to always preload lowest mips.

    There is an example of a example solution in the quoted post (https://forum.unity.com/threads/fee...-virtual-texturing.849133/page-3#post-6444272). Also, I understand there is a more recent version of "VTManualRequesting.cs" in the project "VirtualTexturingSample-2020.1.b14.zip".

    Can I combine the enabled "Automatic Streaming" in Shader Graph and the enabled "VTManualRequesting.cs" script? I turned them on at the same time, everything seems to work fine. I want to know if this is the right thing to do?

    If anyone knows the answer, please share it. Thanks in advance.
     
  29. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    144
    Yes you can. The automatic streaming and the manual request API is additive, meaning, both inputs are combined to know what tiles are requested/visible in a frame. Typically, you'll want to use both, the auto requesting for the highest resolution mipmaps and manual requesting as a prefetcher for low resolution mipmaps. Unfortunately the manual requesting API is pretty expensive currently.
     
    SkySupra likes this.
  30. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    1,099
    I second this, or a version of it that is. Most big HDRP games have serious VRAM problems. Which can cause performance problems.
     
    Deleted User likes this.
  31. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    695
    If most of HDRP features are used, shaders and render textures take more than half of the available memory on PS4 and Xbox One. That makes VT rather mandatory for shipping on these platforms as: What is the point of HDRP if you have to deactivate all the effects to make it run without busting memory?
     
    PutridEx likes this.
  32. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    481

    Still in tech stream but can save a lot in other areas in HDRP for sure.
    https://portal.productboard.com/uni...746-improve-performance-with-rendererlist-api.
     
  33. Ruchir

    Ruchir

    Joined:
    May 26, 2015
    Posts:
    925
  34. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    695
    Performance, yes, but memory, not so much. On consoles, HDRP Lit shader takes 1.25 gb with most features disabled, and 2.5 gb under normal circumstances. This wouldn't be an issue if texture streaming worked well of VT was built in.
     
  35. PerunCreative_JVasica

    PerunCreative_JVasica

    Joined:
    Aug 28, 2020
    Posts:
    47
    @AljoshaD

    Hi,

    since 2022.1 alpha is already out and there have been quite silence about what is going on under the hood since the initial preview with 2020.1, it would be great to release updated roadmap/time predictions of the upcoming improvements and features.
     
    rz_0lento and PutridEx like this.
  36. TekHK

    TekHK

    Joined:
    Jul 30, 2014
    Posts:
    37
    Hi, I am having another go at this in Unity 2021.2 beta 14. It appears to be working if I check the profiler but I cannot for the life of me find the debug view - can the manual be updated or can somebody give me a pointer here please?

    Thanks!
     
  37. OlofAviron

    OlofAviron

    Unity Technologies

    Joined:
    Feb 18, 2021
    Posts:
    4
    Hey!

    In Unity 2021.2.0b14, setting the debug view is done by going to: Window > Analysis > Rendering Debugger > Rendering, then select "RequestedVirtualTextureTiles" in the drop-down next to "Fullscreen Debug Mode".
    It does seem like the instructions in the manual are a bit out of date, I'll make sure this is corrected.

    Olof
     
    TekHK likes this.
  38. SkySupra

    SkySupra

    Joined:
    Jun 22, 2014
    Posts:
    8
    Hello, all. Continuing to deal with VT and it works fine. But I ran into an oddity.

    I'm capturing a snapshot with renderdoc. I want to see the inderection texture. I found VT_TransTab, I guess it's a translation table (aka inderection texture). But the texture is always black, even in unity frame debug.

    I can not understand why. Maybe I'm looking in the wrong place?
    If anyone knows the answer, please share it. Thanks in advance.
    f1.png f2.png
     
  39. OlofAviron

    OlofAviron

    Unity Technologies

    Joined:
    Feb 18, 2021
    Posts:
    4
    Hey!

    This behaviour is actually expected: the translation table contains complex data that cannot be easily represented visually. (If you pick and inspect the black pixels in RenderDoc, you should see "NaN" values)

    However, you can still view the raw data by clicking the "Open Texture data in Buffer Viewer" button in RenderDoc.
    upload_2021-10-6_15-34-6.png

    It is also possible to create a custom visualisation shader and use it in RenderDoc, you can find additional information here: https://renderdoc.org/docs/window/texture_viewer.html

    Olof
     
    Last edited: Oct 6, 2021
    SkySupra and Kmsxkuse like this.
  40. Sab_Rango

    Sab_Rango

    Joined:
    Aug 30, 2019
    Posts:
    121
    Does it support baked GI lightmap of the editor?
     
  41. Ruchir

    Ruchir

    Joined:
    May 26, 2015
    Posts:
    925
  42. jashan

    jashan

    Joined:
    Mar 9, 2007
    Posts:
    3,304
    We're on Unity 2020.3.18 and VT looks like it could solve a few problems for us ... but we'd need it for URP, and we'd need it for mobile (Quest). Unfortunately, we can't currently go beyond Unity 2020 because we also rely on DOTS / ECS.

    Would it be possible to get VT working for URP on Android (Vulkan) with a reasonable amount of work?
     
  43. OlofAviron

    OlofAviron

    Unity Technologies

    Joined:
    Feb 18, 2021
    Posts:
    4
    Hey!

    Regarding the aliasing issues: you are correct, we currently lack miptails so you might notice some artefacts in certain scenarios.

    At the moment, the Triplanar node is not compatible with VT as you typically would not want that combination due to how expensive it is. Your best option is to perform Triplanar Mapping on your own I'm afraid, though it isn't completely unlikely that you would see a "Triplanar VT" node or VT support added to the regular node in the future, it is more of a UX issue rather than anything else.

    Concerning the settings available in the "Sample Virtual Texture" node, this page should come in handy: https://docs.unity3d.com/Packages/com.unity.shadergraph@13.1/manual/Sample-Virtual-Texture-Node.html
    Different sampling methods for the "Sample Virtual Texture" node require different shader code rather than a different sampler state, which is why you cannot provide a sampler state to the node. Instead, we provide some sampling options on the node itself.

    With regards to the debug view, at a glance I am unsure if the behaviour is incorrect but please feel free to report a bug if you believe there might be one, we can have a closer look that way.

    Hey!

    Baking lightmaps for VT materials can still present issues at the moment. (for example: textures not being loaded during baking as mentioned earlier in the thread)
     
    Last edited: Nov 24, 2021
    Ruchir likes this.
  44. Ruchir

    Ruchir

    Joined:
    May 26, 2015
    Posts:
    925
    Sorry for the amateur question, but which sampling options exactly?


    Also, what is the address mode? This isn't mentioned in the documentation
     
  45. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    It won't be possible because it's not the focus in 2021 according to @ali_mohebali (I could be wrong but I understood URP's focus this year to be parity with built-in, not VT).

    Personally speaking I think it's hopeless for mobile VR. You would need it aggreessively optimised for Vulkan and local tile memory as well, both things that are still WIP in URP (I'm waiting for the local tile stuff myself). Otherwise it'd eat into your perf too much.
     
  46. NicoLeyman

    NicoLeyman

    Unity Technologies

    Joined:
    Jun 18, 2019
    Posts:
    33
    Address (or Texture Sampling) Mode determines how we should handle texture coordinates outside of the [0,1] range.
    Wrap/Repeat/Tile will result in the typical texture repetition. Clamp will use the nearest color on the border of the texture.
     
    Last edited: Nov 25, 2021
  47. NicoLeyman

    NicoLeyman

    Unity Technologies

    Joined:
    Jun 18, 2019
    Posts:
    33
    URP parity with Built-In is indeed the focus. URP VT support would ideally only happen after VT and its HDRP integration have matured enough. We're currently focused on some foundational work in the Texture area so there's no timeline for that yet.

    We had the underlying Granite VT tech running on mobile a few years ago. Granted that was only a simple AR app. A few years of hardware improvements aren't guaranteed to be enough to offset the cost of a more complex project running in VR.
    Regardless of hardware: integrating the current VT system into URP and its shaders isn't really feasible for a user.
     
    hippocoder likes this.
  48. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,357
    Could you elaborate a bit on this, why manual URP VT port isn't feasible for users? Is this mainly because of the complexity of the shader graph integration related to this or are the other technical challenges?
     
  49. Ruchir

    Ruchir

    Joined:
    May 26, 2015
    Posts:
    925
    So, the settings in the texture import settings are overridden? (Because they also have the same option and that's what is used when using normal SampleTexture2D node)
    Also, shouldn't it be named WrapMode just like in the texture import settings? It would avoid confusion like mine :p
    upload_2021-11-25_14-38-51.png
     
  50. NicoLeyman

    NicoLeyman

    Unity Technologies

    Joined:
    Jun 18, 2019
    Posts:
    33
    I think most of it can be reverse engineered based off of the HDRP integration. The shaders themselves are probably the most complex bit. There's also the matter of integrating a visibility resolve pass into the one or more rendering methods. (forward, deferred, forward+). This results from this pass need to sent back to the CPU so we know what texture data to stream. There's a lot of ways to make it slow or not work as desired. (synchronisation, too high a resolution, wrong objects rendered).
    It's a lot of effort to get it on par with HDRP VT at best. So you should be aware of the current limitations of VT before investing the effort to port it.
     
    rz_0lento likes this.