Official Feedback Wanted: Streaming Virtual Texturing

Any updates on asset-bundles?

Planning to release a game on Steam Early Access in the **2nd half of 2022** and Early Access means REGULAR updates. The game currently has about 100GB+ of textures, but this figure grows every month. Would love to split the core game app from the textures, letting players update JUST the core game app. Would also like to let players choose between a high-res texture package and a low-res texture package.
This would all be possible with asset-bundles.
VT is awesome!

 

But which is better/faster? mip map streaming or virtual texturing? Assuming there are some 8k lightmaps around and some 8k texture atlasses in one map. Other materials having average 2k textures, sometimes 1k. For a PC game that pretty much targets as lower end pcs as possible

 

Every time I enter playmode, I'm getting this error:
RTHandle.Initialize should only be called once before allocating any Render Texture. This may be caused by an unreleased RTHandle resource.
Unreleased RTHandles:
VTFeedback lowres

Any idea what the reason for this might be?

Also: Is there a roadmap for VT in 2022? There are still some important things missing like VT behind transparent objects and addressable support.

(Besides of these things it is amazing!)

 

I had a lot of these issues - especially when combined with popular 3rd party assets like Expanse (HDRP atmosphere) and Crest (HDRP water). However, they almost completely vanished when I rolled back Unity from 2021.2, 2021.1 and 2022.1 (beta) to 2020.3 LTS.

ALSO: Would like to see a roadmap for VT, it is nowhere to be found - should it not be on the official Unity roadmap for HDRP, URP or general (for adressable support).

ALSO, would love to have some docs/pros/cons for using Virtual Textures vs Texture Streaming.

 

Hi is this usable on built-in pipeline ? it's still is the most popular version of the engine after all so I'm confused if it isn't.. If not why is it available in the editor ? if yes how can I use it I keep getting an error on build and if it involves shaders are there some compatible standard shader variants ready for use for anyone who isn't a shader programmer?

 

Nope. Only HDRP (for the foreseeable future).

 

Ok thanks for the reply

 

hi, so i'm experimenting with SVT.
it works fine in editor but in player build, nothing got loaded. what can i do to resolve this?
i've marked textures as virtual texture only . attached shadergraph that i'm using that emulates the Lit shader.

you can see the ceiling has texture while the wall/shelf doesnt. the ceiling uses the built-in lit shader while the rest uses the SVT shader.


using 2022.1.0b

 

update: spent a whole afternoon but i think i identified the cause. i use cinemachine, and the main camera 's position in the scene isnt the same as the camera on play. i think this "teleport" of the camera on game start killed SVT.

should i file a ticket for this?

 

both Unity 2021.2f15 and Unity 2022.1b11 constantly crash on mac (didn't test other versions or PC). According to the editor.log it happens because of various VT related things.
I unfortunately can't file a bug report as the project size is over 100gb...

Edit: After disabling VT (which is not an option for our project), Unity is stable again, so I can confirm that VT causes instabilities in current Unity versions.

Here is a part of the editor.log

 

Hello, we have some problem in our project, we built everything based on the virtual textures, and works like a charm in Editor but in build some textures are loaded, some not, this is a huge problem, do You have maybe any advice why in editor everything works fine, but in build textures won't load ?

 

What's the actual limit of samplers when using virtual textures in the shader graph? Is it 16?

Let's say that I want to use 3 layers per every virtual texture. Does it mean that every virtual texture node will use 4 samplers (3 layers + VT sampler) so I can use 4 virtual textures like this in one shader graph before reaching the limit?

Is it possible to use 'Sampler State' node for this by any chance?

Thanks

 

But first of all it's not even production ready for HDRP

 

What's the state of lightmap support?

 

Hi everyone, is there any artist (meshing / level design) focused workflow for SVT described anywhere?

As I understand "default" HDRP shaders does not support SVT, so I need to create my own shader graph, as outlined in the video, and then feed texture maps into it.

So considering above, I'd need to have a custom SVT shader for base material (let's say for enviro), which artists use building level (placing meshes / prefabs), correct?

There are layered materials in HDRP - can I use SVT shader as a layer material there?

One additional question: is there any way I can hack API to allow decals baked into the textures, during build process? Example: prefab w/ a model is placed in the scene, decorated w/ HDRP decals - I want these decals to be baked into the model texture and later streamed by SVT.

Thanks,
S.

 

Anyone have a 1:1 lit shader shadergraph that can be used with VT?

 

Hi,
Some questions before i start reimplementing the wheel / create a custom VT solution for our next project:

1.)
Is it possible to use SVT without shadergraph? (I guess the answer is yes, but the documentation is only talking about shadergraph if a user wants to use VT, so i want to be sure.)

2.)
We do not use URP/HDRP but are writing our own SRP. Is it possible to implement SVT in a custom SRP without Unity source access?

3.)
If yes: Is there a high level overview on how to do that?
i.e. i do know that VT needs a feedback texture but i don't know what Unity expects here. PageID and Miplevel? In which channel(s) and in what format should these be put? I noticed that the feedback texture used in HDRP is of format R16G16 so i guess UVs, pageID and miplevel are combined/packed?

I had a look at the HDRP sources, but the codebase is pretty extensive and unfortunately mostly uncommented, so it's very tedious to find all the places where SVT stuff happens and understand how it all works together.


Please note that I had a look at papers/presentations from Sean Barrett, AMD, Nvidia, id-software, etc. because I had planned to implement VT myself and I didn't know Unity already had this feature. Now that I know that Unity already has VT support, I obviously want to use it and probably save a lot of time

TL;DR;
Is there more info on how SVT is implemented in Unity and how to customize it? At the moment the current documentation is a bit sparse.

 

Lightmaps already enjoy classic mip streaming so they'd probably be OK for quite a while. Were you talking about insane resolution lightmaps? Those would take forever to bake in PLM without a render farm.

 

May be other thir party solutions could handly theese big lightmaps? may Bakery can? just wondering

 

Bakery is cool but PLM also bakes in tiles, neither would be performant. Probably best to think about realtime techniques but projecting to UV map for the ultra sized baked maps. Honestly, at these scales it can't be classic lightmap baking without a farm.

 

bump

Do you need more info from my side to be able to give an answer? If so, please let me know

 

Hey,

Should VT work with translucent opaque alpha cutout material? It only temporally "works", when I disconnect and reconnect the alpha channel or disable/reenable alpha clipping or save a slightly changed scene. Textures then get loaded and stay after reattaching alpha.



For anyone interested: I added the VTManualRequesting script that ensures the 128px mip is always in cache.

 

Same problem exists for Unity 2021.3.5. Sometimes changing the Cut Out variable that we set on the Shader Graph also temporarily solves the problem. However, we are waiting for a permanent solution.

 

There has recently been a change to the way virtual texturing handles transparency/alpha channels. It could be worth trying to reproduce the issue in a sample project and opening it in 2022.2.0b1.



Screenshot taken from the 2022.2.0b1 release notes.

 

I will wait for entities 1.0 and hope that VT will support subscenes soon. Any plans on that?

 

Hello again,

Is there an estimated time for the VT system to support Addressables? We are implementing our own solution for this incompatibility, but we may make changes if there is built-in support before our project is released.

Also, how can we ensure that we have some control over the files that the Virtual Texture system creates in the StreamingAssets folder in the Builds? We do not want the update sizes to increase for a game running on the launcher. Because we see very unstable file sizes in this folder. Actually, we know the logic of packaging, but not being able to intervene and divide the number of packages creates some problems.

Thank you.

 

Could anyone from VT team could KINDLY find a moment to connect with their customers and provide answers to the questions, pending here? Thank you!

 

^ Well, that's just beautiful....

So, correct me if I am wrong, but if someone's looking for a VT technology for Unity we have following situation:

- we don't have Granite - coz Unity bought it
- we don't have Amplify - for whatever reason
- we don't have Unity solution here as well, since: "no VT work scheduled for 2023.1" means it's dead.

Cool... just an idea... you know guys that there's no point in pushing HDRP without any VT solution on the horizon?

I bet there's lot of dev time scheduled for mobile monetization: don't need VT there, for sure!

Cool, now at least we got some clarity where this whole thing is heading.

 

VT technology is unfortunately not ready. While the Mipmap streaming texture system partially provides this, we've seen it corrupt on any Drawmeshinstanced command in a massive open world game. Our team started to write their own virtual texture system. In short, the situation is unfortunate, because the Texture technologies given by unity are mostly incompatible with their own systems. I hope we can now see a faster development process in such matters. Because while we strive to use Unity in large-scale projects, we see that Unity developers are behind.

 

I haven't followed VT much, but asking out of curiosity:
Since VT can stream virtualized textures, if, let's say, 3D meshes can be projected to 2D textures and backprojected to 3D meshes from 2D textures, VT could theoretically be used to stream automated LODs for meshes as well and therefore Unity would have a Nanite counterpart right?
Is something like this in production? Or is my above assumption just wrong?

 

Similar project is already happening here: https://www.indiegogo.com/projects/the-unity-improver-nano-tech#/
It's far from production-ready though.

 

That's right - I know about this project and I hope it absolutely succeeds . Nonetheless it would be interesting to know if my above assumption makes any sense^^. At least roughly .

 

Roughly yes, but there's a little bit more to that: Karis_Nanite_SIGGRAPH_Advances_2021_final.pdf (realtimerendering.com) -- talk by the developer behind Nanite at Epic.

 

is there a solution to these very obvious texture popping? it is pretty unusable

https://drive.google.com/file/d/16GeJl59UuVYmF_WnEh8NMascf-O6xwoD/view?usp=sharing

 

had to manually requestregion the highest mipmap. had wrongly assumed VT keeps the lowest resolution in memory

 

so i made some improvements upon the posted streaming VT requester

https://gist.github.com/NanyiJiang/d11828eb09e3cf00155deceef7f6e986

for some materials in my scene,
VT.Streaming.GetTextureStackSize(mat, stackId, out w, out h);

gives w = 0, h = 0. i dont know why; but i skip over them.

also including skinnedmeshrenderer, as well as disabled renderer in case they become active.

most importantly, i notice the requester is eating a lot of CPU time. in my fairly complicated scene i'm only seeing 100+ textures in the stack. I decide to do a round robin(request one texture at random per frame). it is almost imperceptible, but it cuts down the CPU time down to negligible.

 

Sad I wonder if the Graphine developers are still at Unity.

 

Any updates on SVT?

 

Hi everyone! Could you help with that question? It's about gtp large files in build. We would like to get more gtp files but with smaller size, to simple patch them.
https://forum.unity.com/threads/granite-sdk-tools.1439602/

 

Unity Developers. Why did you bury VT? You bought the Granite SDK, hastily integrated it into unity and just buried it. It is not compatible with some of the internal concepts of the engine, and with large volumes of textures (for which they were invented) in your version becomes completely useless due to frequent glitches, bugs, as well as insanely huge monolithic files.

 

Please let me use virtual textures on Android

 

That is the main issue with acquisitions. Killing the product that has been acquired.

 

So...another feature Unity started but never developed much further and likely never will? Despite the competition having it production ready for years at this point?

Also I just found out on this thread after several hours trying to mess around with it and not understanding why it doesn't work that it doesn't even support the built in renderer...which in itself is baffling to say the least and isn't mentioned anywhere in the documentation.

 

DirectX 12 changed the landscape for Virtual Texturing and this system was designed for pre DX12. Likely Unity is going to start from scratch. Given Unity mostly focuses on mobile games I am not holding my breath. I don’t expect to see any progress for the indefinite future.

 

Bump, this is sorely missing. I was hoping to use this in place of Mipmap Streaming for Texture Arrays, because that was missing support. And yet again the alternative unity directs me towards is also unsupported.

 

As of Unity 2023.2.11 (HDRP) SVT is working but when alpha clipping is enabled on the Virtual Texture shader graph it makes the game object invisible when being rendered in Windows build.

 

I've implemented VT in my SRP for a few months now and it seems to work very smoothly. I've been running around with four 65k textures layered on top of each other with transparency and additional 16k triplanar mapped overlay textures with no problems.

All that is required is 2 files that needs to be copy-pasted from HDRP: DownsampleVTFeedback.compute and VTBufferManager. Make sure you construct the buffer manager with the downsample compute shader for each camera. I think multiple cameras can share but HDRP makes a unique manager per camera so I'm not going to touch it.

At the start of camera rendering, call VTBufferManager.CreateVTFeedbackBuffer() to create the feedback texture that each VT shader needs to write to (alt SV_Target from your color output). Then the per camera VT manager needs to call Resolve() to update the VT cache. That's it, it's about 10 lines in total that is needed in a custom RenderPipeline file.

Reference can be found by looking through the HDRenderPipeline.cs file and control-f all instances of ENABLE_VIRTUALTEXTURES define. The code within the define is what you need to splice into the render loop of all cameras to get working. One before rendering and one after rendering.

Shadergraph obviously does not work with SRP but thankfully shaderlab is far better documented. All information on implementing VT in a shader can be found in: Packages/com.unity.render-pipelines.core/ShaderLibrary/TextureStack.hlsl

You might be able to get this to work with URP and even built-in through some very creative use of render features. I don't see why not, the actual meat of the code is baked into the editor just waiting to be used. URP Shadergraph definitely won't support it though. Although if you're trying to shove in VT into URP, y'all should know your way around shaderlab right?

 

Bump, I get there are other priorites but it'd be nice if the current (almost working) implementation could be polished and finished for Unity 6.