[Released] MegaSplat, a 256 texture splat mapping system..

No, I'm talking about painting the same asset/mesh, and progressively saving/preserving different stages during the painting process and having the ability to roll back if needed. This way you can even paint multiple versions and bake 2 out of 5 versions, maybe, lets say, painting different seasons. I wouldn't mind to have a new tab, with large icons showing a shot of that particular painting stage/prefab automatically indexed when I hit a key to save the painting stage(prefab). It's far less disruptive when you paint.

Would it also be possible to auto-magically map the bakes an save baked meshes, prefabs and the materials with a chosen shader? It'll be a complete solution.

 

I'm not sure if I totally understand your use case, but would a snapshot system fulfill this request? Like, you press a button and it stores off the paint job, then keep painting, store off another. When you want to change it, you just do a "recall snapshot #2" and it restores it to that point? If so, that could be added fairly easily to the vertex painting toolset, though I doubt I'd be able to get to it soon.

So bake out each snapshot to it's own prefab/mesh/material? My long term plan for this was to have some sort of build/lighting preprocessor, which takes all the meshes with paintjobs in the scene, bakes them to new asset files and replaces the ones in the scene with the baked one, and continues on with the build.

 

Actually my original idea is painting textures directly, which need to change the vertex painting tool to a texture painting tool and shader need more texture input. Your baking texture tool is also good, as long as the workflow is convenient

 

I mean texture's color become incorrect if checked Linear.

 

When importing Megasplat0.91 into a new project, I got a Error message
"Assets/MegaSplat/Scripts/Editor/ASENodes/MegaSplatSamplerNode.cs(9,7): error CS0246: The type or namespace name `AmplifyShaderEditor' could not be found. Are you missing a using directive or an assembly reference?"

My Unity Ver. is 5.4.1f1

 

Oh crap, that's bad. Open that file and move the #if AMPLIFYSHADEREDITOR up above the includes. I'll have to see if I can get Unity to quickly patch that.

 

Ok, I've submitted a patch (ironically, version .911) and Unity is going to rush it through for me. I've also fired my QA staff (me at 1am) and rehired a new QA team (me at noon) and put new procedures in place to prevent that from happening again. Sorry for the inconvenience!

 

Hehe, don't worry, the patient survived.

 

Nothing inspires confidence like buying a new asset and getting a compile error. (Unity says the new version should be pushed over the weekend and if not ping them on monday- so not a total disaster). In the meantime, feel free to download the package and move the #if AMPLIFYSHADEREDITOR up a few lines or delete the complaining nodes if you don't care about amplify integration. It really has a ton of nice improvements, so I'm interested to know what you all think.

That aside, I've made quite a lot of progress on Unity Terrain support. I had to write a new painting app for working with terrain, as it wasn't easy to fit into the existing vertex painter, but you can now paint on Unity terrains using the same brushes you can with meshes. It's a little annoying to jump back and forth between Unity's tools for height painting, my vertex painter for painting on meshes, and the new window for terrain painting, but I might be able to smooth that out somehow. Additionally, I don't currently support Undo's for the terrain painter since I can use the normal undo system, but will focus on that as an improvement down the road.

Next up, I'll get the color->splat converter working with it, then some needed cleanup to all the shaders so they can share more code and hopefully have less bugs.

 

Wow, they got that through already! Version .911 is now live!

 

My point is that in case you paint multiple variations of the same job it'll be very useful to roll back, or chose multiple versions to have them as final versions. You can also this way, can change only parts you see appropriate in case some gameplay or game scenarios change during time.
As you probably know, you can end up with many many versions and you might forget which one is which. And of course, a few screenrenders/screencaptures will be priceless in this case. A picture is worth a thousand words, isn't it?
Very convenient to choose your final versions this way.

I don't know how big the snapshot system file might be, but it might be very usefull in case you paint in multiple sessions, which most likely will happen. What exactly a snapshot system is in this case?

And if I read your replies again and summ them up, they perfectly describe what my feature request looks like. Add to that a tab to store the icons(screenshots which can be maximized in case you want a close up examination). Not a problem if it's not the first one in the que.

 

Cool!

 

Is it possible using this asset to get which texture is painted at a particular world coordinate? I'm looking to switch from a Unity terrain to a mesh terrain, and thinking of using this asset, but a requirement for me is to know which texture is painted at which points to be able to know things like is the current ground grass, sand, stone, etc.

 

There is no native API for it, but the data is all on the vertices, so you can read it from the mesh. Essentially, it's either an index, or two index's with a blend factor between them for the two layer shader. When the .92 patch is released, I'll have support for Unity terrains as well. When used with Unity terrains, the data is stored in a texture, but fundamentally the same. The nice thing about looking things up via a texture is it's much easier that searching through the vertices (just GetPixel(x, y)).

 

Where is this data stored at on the vertices? In a mesh I see vertices is just a Vector3. And what is the index? Is that the order that you define textures or something when painting? If so, from there could I then get the texture name?

I can't use Unity terrains anymore because I need holes in the terrain for caves, which is why I am switching to a mesh terrain. I already have this functional using a Unity terrain because Unity offers an API way to get the texture name at a point.

I don't necessarily have to search through the vertices because raycasts can give you the triangle index hit, and from there can get the vertex index.

 

To map back to texture name, you can get the vertex color.a value, multiply it by 255, and this will give you the index into the texture array. The texture array config, which you use to setup texture arrays, has a list of textures in it, so that index will give you the source texture in that list. If you are using the two layer shader, then a second texture index is stored in UV3.w, and a blend factor between the two layers is stored in UV3.x.

So yeah, it should be pretty easy to do, and I should likely add an API for it at some point because I'm sure others will eventually want this as well..

 

Okay thanks!

 

@jbooth hi! Its very intresting system, but I need to know, before buy. Do you have some plans for include substances into your pipeline? I know that you cant do it directly, but maybe copy textures on GPU will be fine?

 

I haven't looked into it yet (someone in this thread was), so I'm not entirely sure what's possible (been focused on getting Unity Terrain working). Are you basically looking to avoid the download cost of the textures?

 

@jbooth, sorry... English not my main language and I'm not sure that I understand what you mean in this phrase: "Are you basically looking to avoid the download cost of the textures?"
If you talking about compression - no, I'm not try yet, but substance graphs can generate compressed outputs (generaly not Texture2D), and I think that copy by Graphics class should not break compression (if compression algorythm can be copied ofcourse). I want try it on this weak, if you dont have information from somebody who try.

 

@jbooth and another question, your plugin have some cginc files for use it in custom shaders?

 

I added nodes for Amplify's Shader Editor in the last patch that allow you to sample texture arrays and use the mega splat technique. I have a refactor of the current shaders that I'm working on which will allow you to easily work with MegaSplat's with a surface shader type structure, but these are not coming in the next patch (well, sorta- the new shaders for Unity Terrain use them, but the old ones have not been refactored yet).

That said, you can walk through the current shaders and figure it all out if you're shader savvy.

No, I was wondering why baking out textures from substance was an issue for you. Usually, the main reason to ship the substances is if you want to reduce your total application size, or animate parameters. In either case, it should be possible to do a Graphics.CopyTexture from the substance into the Texture Array..

 

@jbooth our game its procedurally generated dungeons like levels. Substance - its very cheap and usefull method get some variations for all assets (make the same room older, stouns darker or maybe different colors). We need this features )
Yes, we can copy textures to array after generation of all outputs, and recopy if some material will be changed (rarely). But its make your libraries useless, because I need generate arrays by self, thats why I ask about plans)

 

If all you need is something to build texture arrays at runtime from substances, it's likely about 50 lines of code to build that yourself. My package provides a lot more than just a texture array manager, and substance integration is not very high on the list right now, though certainly something I want to look into in the future.

 

@jbooth, ok, thx for answers, I need some time to thinking about it.

 

Video for version .92. The big features for this version include:
- A new shader that works on Unity Terrains
- A terrain painting system, similar to the Vertex Painter, but which works on Unity Terrains
- The color to splat utility has been enhanced, both in functionality and usability, and now works with Unity Terrains as well.
- Misc bug fixes

 

Hi~~
Is it possible to do terrain hole like this asset "Terrain Hole System" ?

 

No, and that's a really bad way to do terrain holes, since the entire terrain gets drawn as alpha, which is much slower than drawing an opaque shader. You'll also end up with sorting issues, shadowing issues, etc. You're much better off converting the terrain to a mesh and cutting a whole in it, or using a voxel based system if you really need dynamic terrain.

 

I'm not a big fan of the Unity terrains but I have to say this is quite an accomplishment. Very impressive.
I was asking myself if I paint a mesh having as a displacement source the same heightmap as the terrain, would the Megasplat painting be identical?

 

yeah, should look exactly the same.

 

Hi, I try the change mesh workflow with some other vertex painter plugins not store vertex info in vertex stream.

I use Export2Maya to directly export mesh from unity scene to a maya file with new painted vertex color and position, then I can open the maya file to add edges, also can change vertex color. After that, I export a fbx replace original fbx file. Then the mesh in unity scene will update automatically. This workflow is really convenient.

I also try the vertex painter pro with Export2Maya, but I can't export painted mesh with vertex stream. I have to bake mesh first. And Export2Maya can export only 2 UV from unity.
For dynamic props, vertex stream is good, no need to duplicate a mesh.
But for a mesh terrain, most terrain has different shape and won't be duplicated. I think an option to store the vertex info in mesh will be more convenient than bake mesh later.

 

You'll have to contact the author of Export2Maya if you want it to export more than 2 UVs from Unity. Also, Unity's UVs can be Vector2, Vector3, or Vector4, and I have a feeling Maya will only treat them as Vector2's, so a full export from Unity's format might not be possible without splitting the data into even more UV sets and recombining it, or exporting it as userData in maya and recombining it on import.

Storing the vertex info in the mesh itself would not only be a large workflow change to the vertex painter, but remove the ability to paint on multiple instances of the same mesh, and make all edits final once the mesh is saved (where as with the current model, you can remove edits at any time). You could, however, modify the baking process to not save out a separate mesh but bake the data directly into the vertices of the current model instead. You'd still have to hit bake, and if you had multiple instances of the same mesh they would all get updated to be whichever one was baked last, but it might make your workflow with Export2Maya easier.

 

Wow, just saw you last video and realy like it

I have some questions or wishes for future updates.
Will this work together with RTP?
1. I would like to use some of the render features of RTP for this.
2. I would like to use the blending feature from RTP with this to blend objects to the terrain or use some extra features that let me paint a way for example and blend that over the terrain.
3. Is the RTP hole system working then too?

About the second feature i read ove but want to say that i also would like to use this with substances that are not baked but can be changed in the game for example to animate something or, to let my surfaces and especial my terrains get wet from dry and back I know i asked about it but thought that way would be good.

Will then later come a tool to save a splatmap from my colored terrain i made with megasplat?

You say in the video we can use it on more terrains at the same time.
How can i use then a splatmap from outside (maybe from worldbuilder) for all the same terrains at once?

 

RTP is it's own shader system, so it's more of a competitor with MegaSplat that something that could be made to work with MegaSplat.

1. Which features of RTP are you interested in?
2. You can paint megasplat data onto objects as well as terrains using the vertex painting workflow and the various blend mode settings.
3. No hole system and no plans for one. Doing holes in Unity terrains in the way most people do them is a very bad way to make a hole, since it makes the cost of drawing all of the terrain much higher. You're much better off using a mesh (and actually, that's true for almost all terrain cases. Unity Terrain is really slow). While I would likely gain some sales from putting this feature in, I'm not going to make shaders which perform poorly and are, IMO, not viable in production environments. You can, however, use the ASE nodes to do whatever you wish.

I have plans to look at substance integration in the future.

Yes, one of my upcoming tasks is to make a tool that allows you to render out an albdeo, normal, roughness, metallic, ao, emission, height, and weights maps from a MegaSplat mesh or terrain. The main use for this is to generate a distance map from the painted data.

[/QUOTE]

Well, the painter lets you select multiple terrains and paint across them all as if they are one terrain. Each terrain has it's own material and control texture.

 

The problem before your asset was that meshes as terrains could not use the way how terrains get their textures to save texture space and drawcalls. But now, with megasplat we can do that and even witch much more textures on one mesh as on terrains.

The problem for me is now, that i bought all the nice assets for Unity terrains that give me glittery snow, wetness and some other nice features, that i can not use them anymore but like that features.
So i aske you to put them all in your product
No no, but would be nice if we can use some of them together but you are working on it.

Now i would like to know a good way on how to make nice lod´s for/from terrains with megasplat?
Will such a feature that help me here come to the asset or do i have to export my terrain to my 3D program and make my lod´s there (what will be the most controlled)?
That was a nice feature of Unity terrain that i liked a lot.
On normal meshes i will do the lod´s anyway by hand but on terrain it is a little more complicated becouse they have to match from every seeable perspective to the other terrains. Could be nice if there where some automation.

But i want to say, even in this state, your asset is one of my favorite now

 

You can certainly emulate most of them by using more textures, but I realize that a lot of these are done dynamically. None of these effects are particularly hard to add either; however, the number of shader variants I'm already creating is causing Unity's shader compiler to take 7+ minutes to compile the shaders on a new install. If I were to produce variants which had these effects, they they would likely be new shaders which remove other features (flow mapping, for instance) so that the shader compiler doesn't blow up (It just gives up at some point if you have too many variants).

I'm planning on a refactor of the shader system which will make it much easier to spit out more variants, once that's done, I'll see how hard it would be to add some of these. For instance, the standard "Snow on up vectors" stuff.

Once I get the baking of MegaSplat data to textures working, you can just load the model into a 3d package or use whatever Unity package you want to decimate the mesh, then apply the baked out textures to a standard shader and use it for a distance model. On most modern hardware, the real reason to do polygonal LODs are when triangles start getting smaller than a few pixels in size (this is because the pixel shader has to compute multiple pixels, even if the triangle is only one pixel big), so the continuous LOD stuff that Unity terrains do is not as important as it once was.

Glad your finding it useful!

 

I figured I should report a little on how getting substances to work, worked out for me.

The short answer is that substances are flakey as hell. They are much better than what they used to be (I remember when unity would crash like clockwork when you changed a substance parameter too fast). But they are still hard to work with and really hard to get data out of effectively.

I was able to switch the TextureArrayConfigs references from Texture2D to just Texture but I was never able to get a texture out of a substance that I added to the array. It would always show up as white. CopyTexture never threw an error but it didn't seem to copy the data from a substance like I was expecting. I am certain you could use GetPixels32 to get the data from a ProceduralTexture and stuff it in a Texture2D that way but it requires the substance output to be set to Raw (vs compressed) and that wasn't a valid path for me for runtime support.

Ultimately I ended up writing a fairly custom editor in unity that allows me to select a bunch of substance archives and creates a list of procedural materials in those archives. Then on that list of procedural materials I can then check which materials and which textures on those materials I want exported and to where. I also went into all of my substances and added a MSEO output and created a new substance (for non substance textures) that will output a meso from the appropriate inputs (just to speed up my workflow). So it was a bit of work on the editor side but I did get things working in an acceptable manner.

 

I finally did get megasplat integrated with my voxel engine but I ran into a Triplanar bug that I figured I would report:
in TextureArraySplat.shader around line 310

Code (csharp):

1.  
2. #if _TRIPLANAR
3.           float3 tpuv0_x = float3(i.coords.yz, i0);
4.           float3 tpuv0_y = float3(i.coords.xz, i0);
5.           float3 tpuv0_z = float3(i.coords.xy, i0);
6.           float3 tpuv1_x = float3(i.coords.yz, i1);
7.           float3 tpuv1_y = float3(i.coords.xz, i1);
8.           float3 tpuv1_z = float3(i.coords.xy, i1);
9.           float3 tpuv2_x = float3(i.coords.yz, i2);
10.           float3 tpuv2_y = float3(i.coords.xz, i2);
11.           float3 tpuv2_z = float3(i.coords.xy, i2);
12.  

The .yz coords are rotated incorrectly and this causes textures in the x direction to be rotated 90 degrees. You won't really notice this on most textures like grass or rock because this orientation isn't as important, but on textures like bricks it is very noticeable (they run vertical as opposed to horizontal when viewed from the x direction). You simply need to make the .yz into .zy.

Code (csharp):

1.  
2. #if _TRIPLANAR
3.           float3 tpuv0_x = float3(i.coords.zy, i0);
4.           float3 tpuv0_y = float3(i.coords.xz, i0);
5.           float3 tpuv0_z = float3(i.coords.xy, i0);
6.           float3 tpuv1_x = float3(i.coords.zy, i1);
7.           float3 tpuv1_y = float3(i.coords.xz, i1);
8.           float3 tpuv1_z = float3(i.coords.xy, i1);
9.           float3 tpuv2_x = float3(i.coords.zy, i2);
10.           float3 tpuv2_y = float3(i.coords.xz, i2);
11.           float3 tpuv2_z = float3(i.coords.xy, i2);
12.  

This fixed the issue for me.

 

First, thanks for reporting back and blazing the trail on this!

You might try using CopyTexture to copy substance data to a regular texture which is of the same format, if this works, then it's likely a bug in CopyTexture that Unity should know about. If it doesn't, then CopyTexture just doesn't work for substance outputs, which can also likely be fixed by Unity. CopyTexture is also not available on some API's at runtime (Metal, for instance, which Aras just mentioned fixing in a tweet), so it's likely it's not a fully reliable API outside of the editor.

I'm assuming you discarded GetPixels as a possibility for the runtime cost and memory allocation? I really wish they would add a version that tool a presized array; it might almost be useable if it had that API.

 

I'll include this fix with the .93 update, thanks for finding the issue!

 

OK, I think I should wait for your update if I need to change mesh with megasplat.

 

Hello, I just have a question and I'm sorry if it was already asked, I skimmed the forum but might have missed something. Can this be used on a programmatically generated mesh? I.e., a procedural terrain (regular mesh) generated and edited at runtime? Would I just be able to assign UVs/Color-mappings and get something working? Or does it need to be an actual file so it can be "converted"?

Thanks,
Lucas

 

Yes, it can work on a procedurally generated meshes, though you will need to mark each vertex of each face with either red, green, or blue in the vertex color channel, such that each triangle has exactly one red, green, and blue vertex. This is what the mesh converter does. Several people with voxel engines have already integrated MegaSplat in this manner.

Once you've assigned the values in the vertex colors, you can put an index (index/255) for first layers texture choice in the vertex color's .a channel, and if your using a two layer shader, a second texture index goes in UV3.w and the blend between the two layers in UV3.x.

Note also that the new Terrain shader doesn't need these markings. Instead, it constructs virtual triangles in the shader (2 for each pixel in the lookup texture). You could use this technique instead, and write your texture choices into a texture that's mapped to your object.

Finally, a third technique is to mark the triangles in the geometry shader instead of preprocessing them into the color channels. It's all of about 5 lines of code, since you have access to all three vertices and can mark them uniquely then. I didn't go with this technique on landscape because of the continuous LODs (changing topology causes textures blends to change), but can post the code here or add a "procedural" version of the shader to the next update if this is useful. I thought about removing the preprocessor all together and only using the geometry shader, but invoking the geometry shader is slower than marking the vertices, and some platforms that can support MegaSplat's don't support geometry shaders. Come to think of it, I should likely just add these variants in the next update, because they're a great choice for people generating procedural geometry on high end platforms.

 

Great, that's very helpful. I'm still struggling to achieve multiple textures at all, for some reason. I'm using the MagaSplat_MarcoDetailFlow shader (Triplaner), and for each vertex in my code I'm assigning colors as such:

colors.Add(new Color(1, 0, 0, a));
colors.Add(new Color(0, 1, 0, a));
colors.Add(new Color(0, 0, 1, a));

with "a" depending the on the location (as index#/255), and it's outputting a single texture for the entire mesh (I'm using an Albedo that came with the package) even if I deliberately make each alpha different within a triangle.

If I use floating point numbers for the alphas like .1f, .2f, etc, it outputs a few textures overlapping each-other with extreme z-fighting (I'm assuming because those numbers don't exactly map to a single texture). Any idea what I'm doing wrong? And those variants as features would be great!

Edit: Just to clarify, the colors are definitely being assigned correctly as I have checked that (just to eliminate that as a variable)

 

So, this is assuming that your not sharing any vertices, which I'm betting is a wrong assumption. If you end up with a triangle with 2 red and one green vertex, for instance, then you'll get a smear of textures across the triangle. To make sure this is true, you might have to duplicate some vertices. (This won't be necessary with a geometry shader variant)

Multiple textures are only available with the two layer shader (Which currently doesn't support triplanar, unfortunately). However, I'm currently unifying the shader code so that all features are available in all shaders, so you can have 2 layers of triplanar if you want it. Keep in mind that triplanar is very expensive, as it requires 3x the samples of a regular mapping, so two layers of triplanar is a ton of texture samples.

if you want index 7 in the array, then you should set .a to (7.0f/255.0f). Values between that are rounded, so it will only smear through a bunch of textures if your vertex colors are marked wrong, though it may round to something you didn't expect, so it's always best to set the data explicitly in increments of 1/255.

 

AH! It was the "(7.0f/255.0f)" format that did it (I had no shared vertices by the way). Thank you so much for the help, this is a truly amazing product and is exactly what I had hoped for. And I haven't even begun tinkering yet!

 

Awesome! I'm hoping to have the new unified shaders finished this week, which will make it a lot easier to write customized shaders as well as giving us feature parity across all the shaders. I'll also add geometry shader versions there, which will make integration with procedural geometry instant regardless of mesh topology..

 

So I wound up getting this asset, how do I access the texture array config at runtime so I can lookup the index?

 

The TextureArrayConfig's are editor only; this is so you don't accidentally duplicate all the textures by shipping them both in the array and as separate textures because they're referenced in the config.

If you want the list of textures at runtime, I suggest making a ScritpableObject that holds a string[], and having the TextureArrayConfig output this object in it's compile to keep it always in sync. I can add this as a todo on my list for the next patch as well.

 

I'm not quite sure I understand what you're saying to do.

Also, I'm trying to lookup the color like you said, but mesh.colors is returning an empty array. Is that the right way to be doing it? If it is, it might be something wrong on my end. I'm not the one that painted the terrain, I'm just implementing the functionality for it.