Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[FREE]MicroSplat, a modular terrain shading system for Unity Terrains

Discussion in 'Assets and Asset Store' started by jbooth, Aug 9, 2017.

Thread Status:
Not open for further replies.
  1. mons00n

    mons00n

    Joined:
    Sep 18, 2013
    Posts:
    304
    Ah gotcha. I've been toying with extending the shadow distance quite a ways while decreasing the nearest cascade percentage to achieve this but as you can imagine I'm taking performance hits due to all of the other fun stuff in between. All I really want is long distance shadows for trees->terrain + terrain->terrain because it makes a huge difference in how the scene feels, especially in canyon/mountainous regions.

    It sounds like the trees->terrain is in the words via VS; can I bribe you to put terrain->terrain on the roadmap somehow? =)
     
  2. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    Well, it'll be worth it...but no texture clusters for me, alas.
     
  3. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    I think I can squeeze an extra sampler or two out of the current arrangement based on the MaxTextureCount. It's really annoying that OSX's drivers impose this limit..

    You could likely do the terrain vs terrain shadows by storing an approximation of the angle at which a vertex becomes shadowed in a texture. Something like an SH lobe. Then at runtime each pixel can just look up this angle and shadow accordingly.
     
  4. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    It would of course be ideal to combine both tree and terrain shadows into the same mechanism. You need a new challenge, don't you?

    So what puzzles me about the sampler limit (besides Apple's idiocy) is all the games that are made on Windows with more samplers -- what happens when someone runs those on a Mac? Do they degrade gracefully? (That sure doesn't seem to happen when I overshoot the Mac limit.)
     
  5. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Like I need a hole in my head..

    They stay within the sampler limit, or use a LOD of the shader with less features, or use multiple passes. Most shaders don't use more than a few samplers. Additionally, most games don't have things like dynamic water and lava flows, texture clusters, etc, that MicroSplat has. If you read the papers on rendering terrains in AAA games, most do with a lot less than what MicroSplat supplies, and rely on using custom shaders for custom situations over an Uber shader that "just works" in every situation. I think, in general, Unity users who haven't been in the AAA trench don't realize how many hacks and tricks are used in AAA games, and to what lengths companies go to customize solutions for their particular use case, or even a single camera shot in a game.

    In the Guitar Hero and Rock band games, for instance, dropping a single frame was not allowed- we had to hit 60, all the time, on every platform. A single dropped frame was extremely jarring to the user. So every night the game would run on a number of different platforms, dress characters the most expensive way possible, and run the most demanding songs possible, and file bugs with the artists if a single camera shot dropped a frame. They would then go in and adjust the shot to make it hit frame rate - manually cull a few objects which the user would not notice were missing, push an LOD down, etc - for just that shot. A given level would have upwards of 1200 shots in it, each tuned to stay within the performance budget. The lights which hit each object were actually chosen by hand, such that no object could ever have more than 3 lights affecting it (which were all computed inside the shader, instead of doing multi-pass rendering). The whole renderer was immediate mode to save on latency, and all timing was all driven from the sound card instead of the CPU to keep things in tight sync. Everything was customized to make the experience better, not to make a general engine that could just do anything (though we tried to be reusable and general when possible). Having to cut sampler counts when porting to a new platform would have been the least of our worries..
     
  6. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    Wow, that's a great peek into AAA optimization. (And in Rock Band, who's got time to scrutinize the graphics anyways?)

    But what I was really thinking about were indie games on Steam etc where the developers are unlikely to be thinking much about Mac sampler limits....
     
  7. mons00n

    mons00n

    Joined:
    Sep 18, 2013
    Posts:
    304
    This makes sense in my head but no idea how to would implement it =O

    That's super interesting
     
  8. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Most likely they just happened to stay within those limits and got lucky, or designed explicitly within those limits. DX9 and older OpenGL versions had similar limits, and you can see how older packages like RTP are completely designed around trying to stay within those types of limits..
     
  9. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    Huh, okay.

    So now I'm setting up a dynamic water flow, but confused about the Emitter -- you can see here I've made the emitter (blue) very small, but the water isn't behaving as I'd expect. I thought it would all emit from within the Emitter sphere, but it's emitting from a larger area -- and not just in the immediate area of the emitter, but further downstream, it's emitting from up on the sides of the creek groove, instead of just flowing down from the emitter. Increasing Resistance helps some, but if I crank it up too much, then the water stops flowing down the groove entirely. I think I can finesse that if I can figure out how to shrink the emit area.

    watergroove.jpg wateremitter.jpg
     
  10. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Do you need it to be dynamic? Because you can use the painter to paint fixed flows, which are cheaper because they don't need to do any dynamic calculations, and you won't have any issue with controlling exactly where the water goes..

    I went back and forth between doing something more physically correct and what I have now during development. The problem with a physically based approach was that you have to use HDR buffers for all of the values, which take 4x as much memory, and the calculations are much more expensive, and the result was no easier to control. So what I have now is that resistance and evaporation form a kind of balance against the spread of the water, but the total amount of water in the system is not controlled by the emitter, so to speak, but rather a balance between resistance/evaporation and a spreading factor.
     
  11. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    I do need it to be dynamic (the water has to start flowing after an obstacle is removed). That's not possible with regular streams, right? So okay, I'll keep fussing with it. (It works great for flowing over convex areas, which does make sense as the main use case.)
     
  12. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    It is possible, but would require you to "paint" the control map at runtime, effectively, which would require writing some script.
     
  13. mwituni

    mwituni

    Joined:
    Jan 15, 2015
    Posts:
    345
    I saw an update ... does it fix the circular bug i showed you in streams?

    What is the recommended way to upgrade MicroSplat and components?

    I guess just delete the MicroSplat folder (keeping MicroSplatData) and update MS, then components?
     
  14. magique

    magique

    Joined:
    May 2, 2014
    Posts:
    4,030
    I just noticed a couple of issues with your Microsplat store page. The description states the following:

    (emphasis mine)

    Also, the Support Website link points to the Megasplat forum instead of this forum. Thought you might want to know.
     
  15. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Good catch - Unfortunately Unity only allows us to list a single link for all of our products (it's a publisher link) which is why it ends up linking to the MegaSplat thread. As I prefer the forums over maintaining a website, I'm not sure what to do about that. I'll fix the typo though, thanks.

    No, that's actually part of the distortion which creates the foam - when the foam is too hot it can look a little circular like that. I have two types of upgrades to MicroSplat, minor versions (like 1.41 -> 1.42), in which you can just update the individual assets, and Major versions (1.41->1.5) in which you have to download all of the modules + core, otherwise the core module will disable any modules which don't match it's version number. Anyway, I've never had any trouble with updating assets without removing them first, but I know a lot of devs recommend that.
     
  16. wwg

    wwg

    Joined:
    Apr 2, 2014
    Posts:
    130
    Great news on the texture clusters!

    All things equal, will MicroSplat texture clusters still be faster/cheaper than Megasplat? I own both.
     
  17. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    I haven’t timed this yet. Clusters are a fallout from how MegaSplat blends, but they are additional work in MicroSplat. However, in both they are mostly a cost on texture bandwidth.
     
  18. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    Would it be possible to make your distant tree shadows work on unity trees, not just in Vegetation Studio? (and grass colormap?)
     
  19. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Someone would have to write something to render out the correct data, that’s the only part unique to vegetation studio..
     
  20. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    My talk from Unite is posted:

     
  21. mwituni

    mwituni

    Joined:
    Jan 15, 2015
    Posts:
    345
    Excellent talk, and brilliant work on that game ... unreal what one can squeeze from mediocre hardware!

    Also, nice to see someone using some of Eric Lengyel's work (your fog) ... we used his C4 engine in a project a few years ago. He knows his stuff ... specially with Voxel algorithms too !

    How about some assets spinning off? ... a bit of fog maybe, but I think the big one would be Asset Bundles on Steroids.
    They're a PITA to work with.
     
  22. frosted

    frosted

    Joined:
    Jan 17, 2014
    Posts:
    4,044
    Hi Jbooth,
    I've been thinking about improving the roads on the game's strategy map and I'd like to ask your advice on if this is possible somehow using microsplat.

    chrome_2017-11-12_08-21-25.png

    I'd like to do very fine roads as shown here in total war 2. Almost all good 'strategy map' road rendering appears to indent the terrain under it, while also remaining very small.

    The indent might be possible to represent via tessellation, but the road itself needs to be too high resolution for normal splatmap approach.

    If I render the road out into a mesh, would it be possible to tessellate into terrain using your blending technique? Obviously this is kind of an edge case and not your primary goal for the object blending.

    So if that's not an option, do you have any other suggestions that might work?

    Also, I'm generating all the maps procedurally and I mentioned before - it's useful to adjust heightblending so the results are more consistent. You mentioned looking at your calculation for tessellation, that code looks great, do you have any suggestions on potential changes for a height blend modifier?
     
  23. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    So, tessellation is based on distance from the camera- so if you're goal is to get things depressed like that from a distance, you would have to not only have a global map of displacement, but modify the tessellation functions so that it tessellated in those areas as well even at a distance. If you're goal is only close up displacement, then a global displacement map would potentially work if you could have it high resolution enough to matter (if it was just at terrain resolution, then you might as well just modify the terrain).

    Another option is to entirely fake it in the normals. You're global road map could have a normal that creates the depressed look and modifies the terrain normal, without actually depressing anything. Remember, at that distance, it's only the lighting that lets you see the depression- so you don't actually need to modify the terrain to get it to light that way. You could likely do this with the global normal map, assuming you can afford a high resolution enough texture for it.

    For the road itself, there are a number of tools which will generate road geometry from a spline such that you can get really crisp edges on the road itself. If you were to extend the edge of the road and draw that area like a decal, you could modify the normals around the edges of the road to create the depressed effect.
     
    frosted likes this.
  24. frosted

    frosted

    Joined:
    Jan 17, 2014
    Posts:
    4,044
    The more I use microsplat the more I'm generally impressed.

    There's a guy who sells animation on the asset store, he worked on witcher 3. Working with his stuff was eye opening for me, "ahh, so THIS is what real professional work should look like".

    MicroSplat feels the same, it's just on a different level from most stuff on asset store.
     
    jbooth likes this.
  25. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    So, if you own the terrain blending and snow modules, snow will now appear on terrain blended objects. There's two controls; one to dampen the snow if you'd like to control the amount of it (1== no snow), and another which adjusts the normal based calculation for snow, allowing you to wrap it around more or less of the object like in this video.



    Currently this only works with the system the terrain blending module uses. But I'm thinking of making this a more general system, where any number of shader effects from upcoming modules could be applied to objects in the scene - wetness, snow, new stuff being worked on, etc..

    Anyway, this opens some doors to unifying the scene through the technique used by the terrain blending system, so you don't have to line up various wetness/snow/rain shaders with slightly different algorithms, but rather put this on any object and turn on the bits you want.
     
    Last edited: Nov 14, 2017
    Seneral, one_one, mons00n and 5 others like this.
  26. AndyNeoman

    AndyNeoman

    Joined:
    Sep 28, 2014
    Posts:
    938
    Hi @jbooth, I'm Just starting with Microsplat, it looks really great and fast. With the water module can you control it from an external script? (I.E use my weather system to drive wetness/puddle growth and retreat as it dries)

    Also the blending module would it work with trees in Vegetation studio or just objects?
     
  27. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    I'm setting up World Streamer, and when it tries to unload a terrain tile, there's a console error:

    Can't remove Terrain because MicroSplatTerrain (Script) depends on it

    But then it apparently does remove the terrain (at least, it's not in the scene anymore). Not sure if this is a question for you or NatureManufacture, so I'll post in both places. https://forum.unity.com/threads/rel...-create-big-world.325673/page-23#post-3286948
     
  28. LennartJohansen

    LennartJohansen

    Joined:
    Dec 1, 2014
    Posts:
    2,394
    Hi.

    The blending module adds an extra material to the prefabs. I need to add support to draw this extra material on all other submeshes in the Vegetation Studio render loop. I will add this soon.
    I see no reason why it should not work on tree trunk meshes also.

    I will let @jbooth go in depth on the water module, but as far as I know you can edit the control texture in code and add wetness to any location by changing the value in the right channel.

    Lennart
     
    AndyNeoman likes this.
  29. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    You can adjust any settings you need on the terrain's template material and then call MicroSplatTerrain.SyncAll() to propagate those changes too all terrains that are using the same material template. Puddles and wetness also allow you to paint or render a specific paint job, then ramp it in when it rains. It will even raycast your scene to figure out where things should not get wet. If you wish to edit the control textures at runtime, they are just textures, so you can do whatever you want to them.
     
    AndyNeoman likes this.
  30. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    Do you have any recommendations for creating LOD meshes from MicroSplat terrains that will look as similar to the terrain as possible? There are a couple of tools in the asset store which do this, but the textures lose a lot in the conversion. Is there any way to export a basemap from MicroSplat? (I know, I should've stuck with MegaSplat if I wanted to do mesh terrains....)
     
  31. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Yes, there's a Render Baking section on the MicroSplatTerrain component which will render out the inputs to the lighting equation (normals, albedo, etc).

    As for terrain->Mesh conversion, I haven't used any of those tools.
     
  32. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,240
    Ah, how'd I miss that. (Well, I didn't realize it was a click-to-expand panel). Great to have all those options, thanks much! This will help a lot.
     
  33. StuntFriar

    StuntFriar

    Joined:
    Sep 10, 2015
    Posts:
    8
    Hi Mr Booth,

    We've just started using MicroSplat, running the conversion tool on our existing Unity terrain and we started getting this error message on development builds on consoles:

    GetPixels32 failed: insufficent pixel buffer size (1), must be at least 4 x 4
    (Filename: C:/buildslave/unity/build\Runtime/Graphics/Texture2D.cpp Line: 1229)

    Failed to get pixels of splat texture
    (Filename: C:/buildslave/unity/build/Runtime/Terrain/SplatDatabase.cpp Line: 241)

    We get similar error messages when running in the editor.

    We've tried setting the textures to read/write enabled, as you've mentioned in a previous reply, but it only clears the errors from the editor (they still pop up on console).

    Do we have to worry about these messages, though? They don't seem to be affecting the game but I worry that something sinister might be happening under the hood.
     
  34. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Well, even messages that do not cause an issue are worth looking into. Do you have any textures that are missing from the Unity version of the terrain? Not sure why it would end up with a 1x1 texture in there- perhaps it's really just missing the texture all together and creating a 1x1 texture for it. Unity's base map system is what causes this- it's trying to create a replacement version of the terrain for the distance view (which no decent shader uses) and does this on the CPU, apparently- unfortunately there's no way to shut off their systems, so what I do is make it generate a 16x16 texture so it doesn't take long or waste memory. But still needs splat textures which are read/write for this..
     
  35. superkratos

    superkratos

    Joined:
    Sep 4, 2017
    Posts:
    112
    Hi, Mr Booth! We've met a problem recently using MS in our project and need your suggestion.. Our programmers split the scene into several parts and load them in runtime like MapMagic does. But we got this error when loading objects with BlendableObject scripts:
    upload_2017-11-17_10-48-5.png
     
  36. superkratos

    superkratos

    Joined:
    Sep 4, 2017
    Posts:
    112
    if (msTerrain == null && msTerrain.blendMat != null)
    return;
     
  37. superkratos

    superkratos

    Joined:
    Sep 4, 2017
    Posts:
    112
    They think there could be something wrong with these codes and cause the null reference exception. What's your opinion, Mr Booth?
     
  38. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Can you send me a repro so I can take a look? I've actually recently changed a bunch of that code for the next patch, so it may be an issue I have already fixed.
     
  39. StuntFriar

    StuntFriar

    Joined:
    Sep 10, 2015
    Posts:
    8
    Thanks for the quick reply.

    We have found that this error gets generated if you use the same texture twice in different splat maps entries (in our case, two had the same diffuse texture but different normal maps).

    We still had error messages after reassigning different diffuse textures to the splat map entries - it looks like the only way to get rid of them (the error messages) is to delete the offending entries from the MicroSplatConfig Inspector window and then re-add the textures as new entries.

    Is this something that can be fixed (pretty please)? If not, it's not that big a deal - our artist would just have to suck it up and repaint his terrain :D

    Thanks again~!
     
  40. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    So, the MicroSplatConfig keeps the Unity Terrain in sync- basically, if you swap the textures in the MicroSplatConfig, it will push the changes back to the terrain. However, if you change the ones on the terrain, the MicroSplatConfig would reset them back. So fix them in the config and it should be all set.

    Not sure why Untiy's terrain system has an issue with using the same texture twice- MicroSplat doesn't care about that, as it copies everything uniquely into the texture arrays, but I can't think of a good reason why Unity's system would care..
     
  41. StuntFriar

    StuntFriar

    Joined:
    Sep 10, 2015
    Posts:
    8
    Yeah, I made sure all changes were done via MicroSplatConfig.

    After a little more digging, it looks like this error message only happens when the multiple-used texture also has crunch compression enabled in its import settings. Not sure why that would make a difference.

    Crunched or not, it looks like the vanilla unity terrain editor doesn't like it when you use the same texture multiple times (in this case - again - two splat maps with the same diffuse but different normal maps) - you draw over the 1st splat with the 2nd one, but not the other way round (i.e. if you painted the terrain with the 2nd splat, painting over those sections with the 1st one has no effect).

    When using Microsplat, this inconsistency doesn't happen.
     
  42. superkratos

    superkratos

    Joined:
    Sep 4, 2017
    Posts:
    112
    Thanks for your quick reply, but I'm sorry I couldn't send you the repro because the project is in our restricted LAN. Can we just change the code to the following or something similar as a temp workaround?

    if (msTerrain == null || msTerrain.blendMat == null)
    return;

    And then we call the Sync at some later time. What sets us into this dilemma is probably because we split the scenes and load the objects scene before terrain scenes. And we’ll also double check we have all the MS configured properly after splitting terrains.
     
  43. superkratos

    superkratos

    Joined:
    Sep 4, 2017
    Posts:
    112
    This is the reply from our programmer, Mr Booth:)
     
  44. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Oh, I see, you are loading the terrain and it's objects asynchronously. Yeah, if that works around it and you manually call sync once everything is loaded, then that might work.
     
  45. mwituni

    mwituni

    Joined:
    Jan 15, 2015
    Posts:
    345
    I'm cross posting this here. The original post with more detail is here.

    This is re our recent experience with the new Vegetation Studio asset, and microsplat.

    VS is totally awesome and frees up the ability to create really dense vegetation at extremely low cost. I can't think of any other asset that delivers such an advantage. If interested, read more about it on the original post here.

    I've been trying to get one of our scenes to a point I can post some screenies to support the launch of VS, and while I'm not there yet (real life commitments etc) and they need some more tweaking and post processing etc, I thought I'd put some up before I totally miss the bus. Also may be good they don't have the eye-candy additions yet so readers can see them in raw form.

    Being the beach, this area is probably not too dense, but the 2k x 2k scene has hundreds of thousands of grass and small plants (including underwater), and around ten thousand trees.

    I'd also like to commend (and 100% recommend) Jason Booth for his Free MicroSplat terrrain shader asset - which totally plugs into VS and they have built-in integration like creating colormaps including grass etc for distance viewing. Justin is hands-down the best shader developer I've met, anyone doubting that needs to watch his Unite presentation video (Rendering 10000 objects on low end devices). I really don't understand why everyone doesn't use it - its free and immediately allows you to use up to 16 terrain textures plus loads of enhancement options on the shader, plus its the fastest terrain shader for unity. If you want even more power (like the Streams module I added for these examples, you can add them for a small cost).

    You can't see it in the pics, but the Streams module adds a fully animated sea-level wetness - so you get the wetness of the beach waves animated onto the beach. I have water running back there too. So the water here is Suimono infinite ocean, with its Shorewaves module creating waves that wash up onto the beach based on the terrain shape underwater (so where reefs are you'll have waves breaking etc and waves onto beaches following the beach contour), MicroSplat then paints the animated 'wetness' onto the beach surface, and I added some water streams for wash returning to the ocean. I probably need to get something to record a video to show it.

    We need Justin (Suimono) to get together with Jason (Microsplat) to build in a function to easily synchronise the beach wetness animation with Suimonos shore waves. It should happen soon ...poke :)







    Other assets to mention here are Tenkoku (sky and light system) ... which integrates with Suimono, so light color and weather affects water etc.
     
    Last edited: Nov 17, 2017
    jbooth, treshold and coverpage like this.
  46. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    I don't know who this justin is, but his terrain shader is really nice ;)
     
    Teila, Baldinoboy, coverpage and 2 others like this.
  47. mwituni

    mwituni

    Joined:
    Jan 15, 2015
    Posts:
    345
    Ja, you should pick up a few tips from him!
     
  48. Teila

    Teila

    Joined:
    Jan 13, 2013
    Posts:
    6,932
    OKay, second try...I thought I would post some screenshots of a terrain I am creating that uses Microsplat. It adds so much and I am not even done with it. :) No post processing in these as I forgot to put it in.

    Screen Shot 11-19-17 at 05.12 PM.PNG Screen Shot 11-18-17 at 12.51 PM.PNG
     
  49. superkratos

    superkratos

    Joined:
    Sep 4, 2017
    Posts:
    112
    Hi, Mr Booth! I find that I can't use MicroSplat's Stream&Lava module and MegaSplat at the same time because they always cause some error about 'TerrainJob', something like this:
    upload_2017-11-20_16-37-47.png
    So I can't paint object-terrain blend areas using MegaSplat's Vertex Painter function. Is there any solution about this?
    Because I need Streams&Lava module to paint wetness, too. Something about both the painting scripts seems in a conflict state..
     

    Attached Files:

  50. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Oh, interesting- must be some kind of namespace conflict. I'll fix that. MicroSplat doesn't use the vertex data, so you can't use MegaSplat's painters for it - different formats and all. But if you send me your invoices I'll fix this tonight and send you the latest..
     
Thread Status:
Not open for further replies.