Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Questions Regarding Sparse Quadtree Texturing...

Discussion in 'General Graphics' started by DINKmod, Feb 3, 2017.

  1. DINKmod

    DINKmod

    Joined:
    Jan 25, 2017
    Posts:
    14
    Hey all!

    I've been doing a bit of research into tackling the problem of large terrain within Unity. As I look around, I can't find much information in regards to dealing with texture memory on very large terrain, although I did find a very brief and vague Documentation page in the Unity manual about Sparse Texturing. (Which from what I understand, is the same technique used by id Software for Rage, and EA DICE for their Battlefield games.)

    And according to the Unity Documentation, and the tech demo available for download from its page, Spare Texturing in Unity only works on DX11.2 (which limits its use to Windows 8+ users...).

    This baffles me. DICE's older games that used Sparse Quadtree Texturing ran on systems with DX9, and John Carmack was adamant about continuing to push the boundaries with Rage on DX9.

    So why is Unity's Sparse Texturing solution limited to DX11.2? Clearly this technique has been done in the past on older versions of the API.

    Also, after digging around on the Asset store, I didn't see many third party solutions. Although I did see the talented team at Amplify Creations do have a Virtual Texturing plug-in (Sparse Texturing) available only on their website for FOUR HUNDRED DOLLARS!!!

    I'm not saying what they made isn't worth 400 USD, I'm just saying that's a lot of money to get a proper terrain texturing solution working in Unity. It's also the only solution to this problem that I've been able to find, but perhaps I'm not looking correctly! :)

    Can the wonderful Unity community enlighten me on the most common Terrain Texture Streaming solutions among Unity developers?

    Thank you so much!!
    -Alex
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,197
    Because the "Sparse Texturing" the Unity documentation is referring to is a specific DX11.2 feature, actually called Tiled Resources.

    The tech used in Rage, which they referred to as MegaTexture, is subtly different from the "Sparse Texture" in DX11. It's more accurately a version of Virtual Texturing.
    http://silverspaceship.com/src/svt/

    DX 11's Tiled Resources lets you use a single, very large image (say 32kx32k) as a texture, and the graphics API handles only loading the sections of the texture it needs to render, limiting the memory usage. However it also limits you to a single texture per "tiled resource". This is great for large terrains, or maybe single objects, but that's about it.

    Virtual texturing is much more adaptable tool. It's a single texture resource on the GPU, but you can have data from any number of texture assets within it, as well as generate new texture data on the fly programmatically if needed, neither of which Tiled Resources allow. The difference is subtle, but it means I know of no one in games who use Tiled Resources, and a large number of major AAA game studios use some form of virtual texturing.

    Rage used a single set of virtual textures for the terrain, buildings, vehicles, weapons, characters... everything really... with the terrain and buildings being made from a lot of uniquely textured assets. Many of those unique textures were made with a tool that let artists take common texture assets and layer them together, along with truly unique bits of artwork here and there. It means the possibility of fully unique art assets everywhere, but also a ton of space required to store those baked down unique textures. Carmack has talked about how to get Rage to fit on consoles (specifically 3 Xbox 360 DVDs) they had to cut down texture resolutions and only ship with low resolution versions of textures outside the "main path".
    DICE does something similar for the Battlefield games, but instead of storing the baked down unique final textures, that tool for generating the final layered texture runs in real time and writes them into the virtual texture as the texture is needed. This means far less disk storage required, and you can modify it in real time if needed.

    Anyway, virtual texturing is totally possible on DX9, or even mobile hardware. You can look at how other people have implemented it and try to do it yourself, but it can be a ton of work to get performant and working properly, hence the high price of Amplify. Alternatively you can use something like MegaSplat and / or RTP, or many of the other terrain tools available for Unity that use many tiling texture assets layered together in nicer ways than Unity's built in tools, instead of large unique textures.
     
    razzraziel and DINKmod like this.
  3. DINKmod

    DINKmod

    Joined:
    Jan 25, 2017
    Posts:
    14
    Bgolus, thanks so much for your thorough reply!! Very insightful.

    Yes, Virtual Texturing isn't a technique that I was thinking of utilizing anytime in the near future, but it's certainly something I wanted to learn more about since it seemed like Unity had at least some kind of implementation of it, as noted in the Sparse Texturing documentation.

    Multi-layered tiling through use of a custom shader is without a doubt the route I'll be going when I get to developing my terrain work. However, I prefer to build the shader myself. I think I could do this using Shader Forge or Amplify.

    I've just been doing a bunch of research on various techniques for tackling the problem of terrain, and I have seen some developers talk about applying a 'low-res' texture over the entire terrain model.

    For masking my many tiled texture layers, I'm planning on using a combination of techniques.
    1. Vertex Color Masks (painting RGBA vertex colors)
    2. Procedural Slope Masking
    3. Hand-Painted texture masks
    My concern is how the game may perform with number 3. One mask would be anywhere from a 1024x1024 or a 2048x2048 texture applied to the entire terrain model. (I'm not worried about the low-res nature, I'm going to tile a detail greyscale over the mask.) The problem with this is that there may be a lot of wasted space in areas of black where I don't want a texture to appear. Moreover, I'm concerned about performance drops after an unknown number of masks, but I suppose I can only find out about this by actually building the shader and making the painted texture masks until I start to see some significant performance issues.

    That's originally why I was looking at Unity's Sparse Texturing technique. I was concerned about my painted texture masks that are covering the entire terrain, even when the player can't see all of the terrain.

    And although I'll just have a lot of tiled textures on my terrain shader, I am definitely interested to see how Unity would handle rendering the terrain with my own custom terrain model and a custom terrain shader. Will it render the textures that are close to the player the same as textures that are very far away, or not in the player's view? Are these things I can control at the shader level?

    I'd begin tackling these problems by getting my hands dirty on building a test project, but unfortunately I'm busy on too many projects at the moment.
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,197
    A 1024x1024 or even 4096x4096 control texture isn't anything to be concerned about. Sparse texturing is all about being able to use a texture larger than can fit on the GPU's RAM. Something like an uncompressed RGBA 4k texture with mipmaps is still only going to be ~86 megabytes, and modern GPUs have multiple gigabytes of RAM. A single uncompressed RGBA 32k x 32k texture in contrast is ~5.5 gigabytes!

    Unity's built in terrain stuff currently uses option 3, and it's a pretty common way to go about it. In Unity's terminology they call them splat maps, but I've seen them called control maps, or just masks like you said. There are certainly several assets on the asset store that build on that concept. The downside is every pixel on the terrain has to sample every texture from every layer to blend between them. MegaSplat is interesting because it leverages texture arrays and some smart shader work to allow up to 256 textures, but still be cheaper to render than Unity's solution.
     
    DINKmod likes this.
  5. DINKmod

    DINKmod

    Joined:
    Jan 25, 2017
    Posts:
    14
    Oh I know it. Was some real remarkable stuff to see when Rage was in development and was finally released. At least, that's the first time I saw the use of virtual texturing in video games.

    That all being said, I haven't gone into actually making the terrain, so I wasn't too sure how things would play out with such a low-res texture applied to such a large terrain. Would mipping be a problem? Because technically the player pretty much always sees the model. Does it mip based on the player's location? If so, that'd be pretty cool. And despite its relatively low resolution, it still consumes VRAM for a full texture at all times, and I wasn't sure exactly how much of a problem that is even with most cards having 3-8 gigs of VRAM.

    And even so, all the unused texture space does seem like a complete waste. I was certainly interested in any way to cut down on that waste. But if it really isn't a problem, then I'm not too concerned.

    I've seen MegaSplat, and it's impressive. I just want to have full control over my shader, rather than use someone else's shaders. It's just what I'm going to need on my projects moving forward.
     
  6. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    4,392
    Well, that really depends of what features you turn on, etc. MegaSplat can be turned up to 12 and practically melt a graphics card if you want to really push for quality. It can also scale down to very efficient modes which can be faster than Unity's shaders. I get a lot of people who set the dials for 12 and then wonder why it doesn't perform well on a low end mobile device. I try to provide as much information about these tradeoff's as possible.

    IMO, the most interesting uses of virtual textures are not landscapes, but rather as texture caches for things like GPU driven rendering (See the AC presentation at 2015 siggraph, for instance). I personally think the MegaTexturing techniques used in Rage made the game look blurry and glitchy, mostly for artistic detail and freedom which was never realized, and which would have been accomplished in other ways.

    I ship nodes for Amplify's Shader Editor with MegaSplat so people can play around with the core technique without knowing how it works. That said, if your using a graph, you'll never get to the same shader complexity and optimization levels of hand written shaders. At least not without writing custom nodes for them..
     
    DINKmod likes this.
  7. DINKmod

    DINKmod

    Joined:
    Jan 25, 2017
    Posts:
    14
    Thanks for the comments, JBooth. Very interesting! That's neat that you throw in some Amplify (and I presume Shader Forge) shaders into the package for people to see how it all works.

    When it comes to building my own shaders, it's not necessarily that I want complexity or even necessarily efficiency, it's just that I want full control. I know what I want to do and what I don't need to do. While I don't like coding (but I'm not against learning and doing more of it), I love quickly networking nodes together to get the exact thing I'm after or my colleagues are needing. There's a node or two I may need in the future, and either I'll work with a buddy of mine to make it or I'll figure it out myself. But yeah, when you're using a graph editor and nodes that someone else made, it's certainly not the most efficient - but it's an incredibly effective and quick way to get a custom shader done, and it's fun. I also argue the same thing that if you're using shaders that someone else made, it's not always the most efficient, because they probably made the shader knowing that a lot of different people will be using it for many different effects.

    If you build something from the ground up, you know exactly what it was made for and can slim down the design for a specific purpose. It's also a huge area of interest of mine. I'm enjoying learning this stuff right and applying it to the work I'm doing.

    Really impressive work on Megasplat by the way. When I was digging around the asset store to see how everyone else is approaching terrain shaders, Megasplat was one of the most impressive. Kind of baffled me how you were able to setup 256 unique textures to blend together. I'd definitely keep that tool on a list of considerations if I needed to work on an a really large terrain. And a mere 45 bucks for such a complex tool...

    I don't remember if I did it but I know on the PC version you could modify the config files to force the megatexturing tool to keep the textures on the higher end of the resolution spectrum. According to the community online, this improved some of the problems present with megatexturing in Rage at the time, including slow streaming and blurry textures. Either way, I think the work that they did on that game, for the time, was impressive and I thought it was gorgeous.

    What's the AC presentation? Assassin's Creed? I'd love to take a look at it, so I'll google around.

    But yeah, since I was speaking on the context of terrain, that's all I was looking at. From what I understand, everything in Rage, for example, was streamed through their virtual texturing process - not just terrain. I've heard the same from a couple friends of mine who worked on another project that used the same technique. They ended up pretty much just streaming everything and using only one material.
     
  8. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,197
  9. DINKmod

    DINKmod

    Joined:
    Jan 25, 2017
    Posts:
    14
  10. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    208
  11. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,197
    Nope. This requires an absolute ton of asset preprocessing and introduces a bunch of limitations on your content pipeline. (ie: All textures on all objects have to be the same size and format.) For a big AAA game these kinds of strict rules are fine, but a bit harder for your average indie.
     
    Beloudest likes this.
  12. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    208
    I thought this much, it's almost what I crave the limitations that actually make your game have next level performance. I suppose it's this depth of perfection that the AAA studios have over indies. You need a big workforce to pull off it... I noticed Unity 2018 has texture streaming, is that entirely different from virtual texturing with megatextures? Are they making 3rd party solutions redundant. I am keen to experiment with making my own implementation and why I landed on this thread. I'm not sure where to start yet but I figured it will be a good introduction to GPU code etc.
     
  13. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,197
    Actually, I forgot they were using a virtual texture and not a texture array for that assassin's creed thing. They don't have the fixed texture size issue, though texture format still exists.

    Texture streaming is what most AAA games did before virtual texturing. Unreal Engine games were especially "famous" for this as you might walk around the corner and the game would look like a PS1 game for a few seconds before all the textures streamed in.

    Virtual Texturing is a different beast. Technically it's still streaming textures in, but it's doing it with pages (usually 64x64 single mip texture chunks) rather than having to load the whole texture upto the GPU. The original mega texture implementation as used by id's Rage had the entire world uniquely textured and stored as JPGs on disk, and loading those JPG images and runtime compressing them before uploading to the GPU. No one really does that anymore with uniquely textured worlds, but rather using more traditional texturing techniques with repeating textures, or maybe doing some form of texture synthesis on the GPU (using merging textures together like terrain systems do, but storing the merged textures). Battlefield / Battlefront games do this for their terrain for example.
     
  14. razzraziel

    razzraziel

    Joined:
    Sep 13, 2018
    Posts:
    169
    I guess there is no Virtual Textures solution on asset store anymore since Granite and AT2 were removed. Does Unity has something similar now?
     
  15. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,197
unityunity