Search Unity

MegaSplat - 256 textures in one splat map shader

Discussion in 'Works In Progress' started by jbooth, Oct 22, 2016.

  1. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    Hi,
    I'm working on a new shader which allows for hundreds of textures in one splat map shader while having a consistent performance cost regardless of how many layers you use on a surface. It also supports flow mapping any or all of those 256 textures, as well as macro texturing, detail texturing, etc. I hope to have it up in the Asset Store in the next few weeks. Check out an early preview here:



    It uses my free vertex painting system, which is already available for free, and includes more traditional splat mapping shaders:

    https://github.com/slipster216/VertexPaint

    The new system should include:
    - A custom brush for painting in my vertex painting package, though you can paint the data in any package you prefer
    - Ability to have up to 256 unique textures per material
    - The cost of the shader is similar to a 3 layer splat map shader
    - Currently supports a metallic workflow with options for normal, emissive, metal/smoothness, ambient occlusion maps, detail textures, and macro textures
    - Ability to flow map and or all textures, and paint flow directions on the vertices
    - Per texture control over uv scale, metal/smoothness, and flow map speed
    - A copy of my vertex painter, which is always available free
    - A lot more..

    Let me know what you think, what features you might like to see or variations of shaders would be useful for your project.
     
  2. jason-fisher

    jason-fisher

    Joined:
    Mar 19, 2014
    Posts:
    133
    Sounds amazing. I love your other shaders. I am working on voxel octree planets and need this immediately!
     
  3. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    5,870
    First question is going to be, does it work on mobile.. : )
     
  4. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    @mgear
    Usually I get "Is it optimized for mobile?" - which is a question that's impossible to answer, since mobile can mean anything from super low end android to a last gen console, and the number of different GPU architectures is far wider than in PCs and consoles.

    Anyway, for future reference, it requires shader model 3.5, which means opengles2.0 devices are out, but things on opengles 3.0 and metal will work fine. Performance depends entirely on how many features you turn on - it's basically entirely texture sampling bound, as there's not a ton of ALU involved.
     
    theANMATOR2b likes this.
  5. jason-fisher

    jason-fisher

    Joined:
    Mar 19, 2014
    Posts:
    133
    Is it possible to get this working as a tri-planar-type shader? I don't want the top/bottom/sides concept of tri-planar -- just the local texture scaling/sampling so I can avoid calculating UVs?
     
  6. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    @jason-fisher
    You have to sample the texture once for each projection regardless of if those are three different textures or the same textures used for each projection. So either way it's 3x the cost.

    A standard surface shader with diffuse/normal/specular terms is 3 texture samples.
    A triplanar shader has to do a sample for each projection (3x), so using the above term it would be 9 samples.
    A three layer splat map shader (or this one with 256 textures) requires 3 samples for the blending of layers, so if you do triplanar plus 3-layer splat mapping it's 27 texture samples per pixel.

    This grows even higher if you add other features, such as detail texturing, macro texturing, etc.

    For instance, if you were to do macro texturing (diffuse/normal/spec), parallax mapping, per texture parameters, splats with diffuse/normal/spec, and detail diffuse/normal, that would be 18 samples per pixel, or 56 texture samples per pixel with triplanar texturing. That's brutally expensive, especially since the cache-coherency of those samples is likely to be low (ie: they'll be in different positions in the textures, etc).

    So yeah, it can be done, but if it's going to be too expensive for anyone to effectively use it it's likely not worth the effort.

    Once I'm done with the main shader, I'll do some performance tests and see how crazy it is on a modern PC and consider it's usefulness.
     
  7. jason-fisher

    jason-fisher

    Joined:
    Mar 19, 2014
    Posts:
    133
    Appreciate the detailed response. My current TODO: approach is to generate UVs that map isosurface-generated faces to a cubical sphere where the normal of the tri aligns with the normal of that chunk vs the center of the planetoid and use a curve correction as it deviates into overhangs and caves like you might use for a bezier curve tool. Triplanar is very expensive.
     
  8. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    @jason-fisher
    I added a triplanar option for kicks. You lose macro texturing and flow mapping in this mode, since you don't have UVs and both of those require uvs to make sense. But if you go reasonable on the other features, it might be affordable depending on your platforms..
     
  9. jason-fisher

    jason-fisher

    Joined:
    Mar 19, 2014
    Posts:
    133
    That's great. I think there will be quite a few people that use voxel engines that are trying to implement a triplanar atlas of sorts and this is probably their best option?
     
  10. jason-fisher

    jason-fisher

    Joined:
    Mar 19, 2014
    Posts:
    133
    Since you were able to get triplanar in .. someone's next question will be: Were you able to add the triplanar projection in local vs world space? i.e. if you rotate or move the textured object, the texturing follows the transform correctly?
     
  11. jason-fisher

    jason-fisher

    Joined:
    Mar 19, 2014
    Posts:
    133
    There really are a lot of appealing aspects to this new shader for the planet/voxel/simulation crowd. The flowmapping and streaming combined is very novel for dynamic weather systems and erosion. I am also now thinking of the streaming's application to GPU-generated isosurface meshes--streaming density data updates and either rendering MC directly from the GPU or returning mesh data to stream back. (maybe you can then just re-march the density update area on a modification vs the entire chunk)

    I think streaming stepped flowmap updates that also animate the vertices could produce interesting dirt/sand-piling effects -- basically triggering GPU-driven animations of tweens toward a target determined on the CPU.
     
  12. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Great work again :)
     
  13. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    development log #2 is up:



    Playing with a technique that allows partial blends between textures like a traditional splat map shader, as well as some new approaches to 'smart brushes' for the system.
     
  14. Frednaar

    Frednaar

    Joined:
    Apr 18, 2010
    Posts:
    147
    Very nice... for terrain texturing one nice idea would be to combine this with multicolored splatmaps so instead of 4 channels RGBA, associate one of your new "materials" to a specific color on a splat.
    This worflow would work really well with worldmachine or photoshop splatting..

    and if you need a betatester for this, I am doing a lot of terrain work these days...

    thanks!
    Fred
     
  15. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    That might be useful for starting out- basically, map the four textures they choose to four "texture clusters" and populate the map. You'd still want to paint more textures down by hand, otherwise you're not getting the advantage of having so many textures available.
     
  16. Frednaar

    Frednaar

    Joined:
    Apr 18, 2010
    Posts:
    147
    yes, I understand, what I meant is actually colormapping, not really related to your shader, but similar to the brand new GeNa asset store tool does...
    instead of having just four textures on a splat you define a relation between a color on the map and a "texture cluster" in Unity Editor. This would make it really easy to create very varied terrains with very little effort...
    1. define the texture clusters
    2. paint the colormap
    3. set the relations between the colors and the clusters
     
  17. Captaingerbear

    Captaingerbear

    Joined:
    Mar 6, 2013
    Posts:
    20
    This looks outstanding. I'll be keeping my eye on this one.
     
  18. jason-fisher

    jason-fisher

    Joined:
    Mar 19, 2014
    Posts:
    133

    Attached Files:

  19. one_one

    one_one

    Joined:
    May 20, 2013
    Posts:
    518
    Does this shader perform well enough for a mid- to large-scale terrain on PC? Or should this be reserved to "hero" objects that need some additional detail? I understand that it depends on what other functionality is used, but compared to e. g. RTP, what features would need to be dropped in exchange for this crazy amount of splat textures?
     
  20. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    It's actually quite performant depending on the feature spec, in many cases being much faster than the shaders Unity uses for terrains. Although the shader will not work with Unity terrains, using it as a comparison point is actually a good way to understand it's cost. On most hardware, these types of shaders are going to be bottlenecked by texture sampling and memory bandwidth, so counting texture samples per pixel is a reasonable guesstimate of performance.

    For instance, the standard Unity terrain shader with 4 splats uses 4 texture samples per pixel per texture type (so, if you are doing 4 splats with diffuse + normal, that's 8 texture samples per pixel). If you go to 8 textures, then it draws the terrain twice, making diffuse + normal take 16 texture samples per pixel. If you add a specular component, that would be 24 samples per pixel for an 8 layer terrain with the mesh being drawn twice.

    MegaSplat offers several packing modes for textures, allowing you to optimize the number of samples based on what you really need. For instance, one packing gives you per pixel albedo, normal, roughness, and either ambient occlusion or metal packed into just 2 textures.

    Under this packing it takes just 12 texture samples per pixel, which is the same cost as Unity's shader doing only 4 textures. Compared to Unity's 8 texture mode, it's half the number of samples, and your doing it all in one pass. It doesn't matter if you have 1 textures or 250 painted onto the mesh, the number of samples is the same. The only real cost of painting extra textures is the bandwidth cost of accessing that much texture memory, and how much this hurts will be highly dependent on texture compression formats (DXT is either 8:1 or 6:1, while formats like PVR are 16:1 or 12:1, which is much smaller) and hardware bandwidth.

    Now, if you turn on parallax mapping, triplanar or flow mapping, etc, things can start to add up quickly. At the bottom of the shader is a display which tells you how many texture samples you are performing per pixel, which should give you a pretty decent idea of cost.

    I'm not familiar enough with RTP to use it as a performance comparison.
     
    one_one likes this.
  21. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    Also of note is that all texture samples are not equal. In other words, a 20 sample shader is not usually twice as expensive as a 10 sample one. For instance, sampling the same texture multiple times near previous samples (like, say, in a blur filter) is much faster than sampling the same number of times in multiple textures, or even across a wider area of the same texture.

    As always, performance is a complex topic that ultimately comes down to "Test it". But, as someone who optimizes high end graphics systems for low end mobile devices all day, I do my best to keep things optimal.
     
    zenGarden and one_one like this.
  22. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Good news.
    I would like alpha support for brushes, to allow new variations between textures blending.
     
  23. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    You mean support for surfaces which have an alpha channel? (transparency?)
     
  24. one_one

    one_one

    Joined:
    May 20, 2013
    Posts:
    518
    Speaking of texture blending - it seems to be height based in your video. Is this the default?
     
  25. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    Yeah, it's height based..
     
  26. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Only a possibility to use an alpha for the brush when you paint on terrain.
     
    Last edited: Nov 9, 2016
  27. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    The painting is done on vertices, so unless you have a LOT of vertices, using a texture based brush wouldn't be that useful.
     
    zenGarden likes this.
  28. Frednaar

    Frednaar

    Joined:
    Apr 18, 2010
    Posts:
    147
    So the shader will not work on Unity Terrains ?
     
  29. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    Unfortunately not - Unity Terrains don't give you access to the vertex data, so there's no way to put the necessary data onto the vertices, and Unity Terrains assume a maximum texture count as well..

    [EDIT] MegaSplat now supports Unity Terrains, as well as conversion from many terrain systems, such as Gaia, MapMagic, etc..
     
    Last edited: Dec 23, 2016
  30. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    jason-fisher likes this.
  31. jason-fisher

    jason-fisher

    Joined:
    Mar 19, 2014
    Posts:
    133
    I was watching this thread and missed the release/other thread. The asset is now released. :)
     
  32. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    Thanks! I forgot to update this one. We're like 5 updates in as well..

    Asset Store
     
  33. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    You need Mesh blending to terrain to make it really usefull.





     
    Peter77 likes this.
  34. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    First part of that is already done:



    Normal blending is coming- a user has already added it to the vertex toolset, I just need to clean it up and integrate an official version..
     
    jason-fisher, Peter77 and zenGarden like this.
  35. bac9-flcl

    bac9-flcl

    Joined:
    Dec 5, 2012
    Posts:
    787
    Very impressive work, I really like the new alpha and tesselation features! Can you go into a bit more detail on how your shader works? I vaguely recall seeing a similar terrain splatting approach in CryENGINE (vertex-attached indexes, no hard limit on surface types) and was always interested in the principle behind it. I suppose this was made possible by the Texture Array feature added in Unity 5.4? :)

    P.S.: On the blending question above your post - I think it was about providing a special shader for terrain props (stuff like stones dropped on top of the MegaSplat terrain object), not about adding a feature to the main splat shader itself. I recall a workflow for that supported by Relief Terrain Pack, where you render your terrain props with a special shader which lets the terrain through based on vertex color, that's where the question probably comes from. Not sure how it was accomplished there, though.
     
    Last edited: Dec 29, 2016
  36. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    I believe what most people (UE4, Cry) do is break up the mesh such that there are some fixed number of textures per mesh and use traditional splat mapping techniques from there. That can work well, but requires tooling to process the mesh, and the cost still becomes more expensive based on how many textures you use (more draw calls). This isn't really practical for Unity meshes, and not possible with Unity Terrains, where you have no control over vertex generation. The other common approach (used in Battlefield, for instance) is to have a Virtual Texture Cache, and render out the splats for each area to the cache as you move around. (Or if you're id on Rage, you just render the whole thing out to a massive jpg).

    Mine doesn't work that way and is based on a new approach; essentially each layer stores an index per control point (vertex or pixel in a unity terrain texture). Textures are then height blended across each face. Since each triangle can only have 3 points, each surface takes 3 samples per texture, irregardless of how many textures are used across the entire mesh. Then, when working in multi layer modes, those results are height blended together. In essence, you can think of it as if the mesh was broken into individual triangles and splat mapped with a 3 layer shader for each triangle- but without the performance overhead of actually doing something like that.
     
  37. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
  38. jamesllama

    jamesllama

    Joined:
    Jun 1, 2017
    Posts:
    3
    Im using megasplat to paint my Unity terrain. I was just wondering if there is a way to rotate the magasplat brush while painting my textures. I know you could export the texture back to Photoshop or Substance designer and rotate it from there but it would be easier if there was a rotate function on the megasplat brush tool.
     
  39. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    Hey James,

    The active MegaSplat thread is here:

    https://forum.unity3d.com/threads/r...lat-mapping-system.441329/page-8#post-2900775

    Do you mean rotate the source textures by 90 degree increments or something else?
     
  40. jamesllama

    jamesllama

    Joined:
    Jun 1, 2017
    Posts:
    3
    Hi Jbooth, thanks for the super prompt reply , yea was thinking something like that. like rotating my source textures by a certain degree, doesn't have to be 90 but sure. In Substance painter i can rotate the brush by a combination of some keys I think holding down (Alt+RMB)
     
  41. FiveFingerStudios

    FiveFingerStudios

    Joined:
    Apr 22, 2016
    Posts:
    394
    I'm having an issue with the build of my game. The painted texture shows up when I run it in the Editor, but doesn't show up in the build.

    I'm using the vertex painter and painting on a mesh.

    You can see the difference in the pics.
     

    Attached Files:

  42. FiveFingerStudios

    FiveFingerStudios

    Joined:
    Apr 22, 2016
    Posts:
    394
    No suggestions?
     
  43. Freznosis

    Freznosis

    Joined:
    Jul 16, 2014
    Posts:
    231
    You're better off asking in the main thread
     
  44. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    @sbmhome Yeah, I just happened to catch this via search, but the main thread is way better. Most likely your running a version of Unity before Unity 5.6.1p2 - Unity broke texture arrays in 5.6 and didn't fix it until several months of patches later..
     
  45. FiveFingerStudios

    FiveFingerStudios

    Joined:
    Apr 22, 2016
    Posts:
    394
    I'm running Unity 5.6.1f1. So if I upgrade to Unity 5.6.2f1...I should be good?

    P.S. I found the main thread...thanks.
     
  46. YHS

    YHS

    Joined:
    Aug 10, 2014
    Posts:
    31
    Hi, I use Terrain with 12 textures. I know terrain will use 4 textures as 1 splatmap, so I think I have 3 splatmaps now and it produce a lot of drawcalls. I'm wondering is megasplat's "singlePass" can help to make 3 splatmaps into 1(Main purpose to reduce drawcall)? If it's possible, how can I do it? Thank you.
     
  47. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    Both MicroSplat and MegaSplat will draw the shader in one pass. However, they do it in very different ways.

    MicroSplat uses standard Unity Terrain workflow but with all textures packed into a texture array. At runtime, it samples the control textures unity creates and does all the shading (up to 16 textures) in a single pass. This makes it very easy to get up and running, and you can purchase modules to add more features (MicroSplat's core package is free).

    MegaSplat replaces the whole texturing pipleline. It works on Meshes as well as Unity Terrains. It does not use the Unity Terrain control maps at all- once you convert, you can delete all the textures off your Unity terrain, etc. It uses a single control map with a custom format, and can work with 256 textures on a single terrain, with 2 being blended at any control point and 6 being blended across the face.

    If your mostly concerned with performance, I'd suggest MicroSplat- it's extremely easy to use, and you can download the core module for free, then purchase additional modules when you want more complex features.
     
  48. eastes

    eastes

    Joined:
    Sep 1, 2012
    Posts:
    51
    This is still one of my favourite assets. The improved terrain quality in my VR game has been received very well! Thank you, Jason.

    Our artist and I are very excited to start working on a new game. We are seriously considering using MegaSplat for nearly all meshes in the scene. We may only need a few different materials which will be fantastic for dropping draw calls.

    Do you anticipate any issues we may have with this? Is there a better solution? Our other consideration is to create our own custom shader using Amplifiy Shader Editor which now has texture array support.

    Once again, Thank you.
     
  49. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    3,891
    I know there are a few people doing this already- I think the only limitation is that the shaders all have the parameterization of the MegaSplat shaders. I could imagine modifying the standard shader to work with texture arrays so you get the workflow your used to, for instance..
     
  50. JBT27

    JBT27

    Joined:
    Jul 13, 2016
    Posts:
    197
    I'm generating terrain in World Creator, and this pebble area's distribution in the foreground is controlled by a PNG mask. The first image is purely the WC output, the second is what happens when I add the MicroSplat script to the terrain. It's not terrible as I can retile the textures in MicroSplat, but in the tutorials, it looks like the conversion carries the tiling with it.

    Using Unity 2017.3.0f3 with the latest MicroSplat version on Win10 64bit.

    WorldCreator_01.JPG

    WorldCreator_01WithMS.JPG
     
unityunity