Help me extend my Unity Metaball shader effect?

Hi everyone! I need some Unity Shader help

In my game players need the ability to throw goop. So right now, I've got a system where separate game cameras see feathered, transparent blobs of pixels, passes that to a RenderTexture, renders them if they surpass a color Threshold, and finally check adjacent pixel positions in order to add shadows, highlights and outlines.


Looks pretty good! (What the camera sees on top, what is actually rendered on bottom)


The metaball effect is great cause it really makes everything goop together.

But I need to be able to render four different colors of these at once. My first thought was to just have four cameras and four render textures, but that quadruples the effect's overhead, and it starts to lag on my laptop. But I can consolidate some of them! If I assign different colors to the RenderTexture's R,G,and B values, I can check for 3 different color thresholds in one pass. That pesky fourth color still requires its own camera though.



This works, but there's a few problems still:
1. Colors are rendered in order, so Red will always be on top, then Green, then Blue, etc. Throwing new goop should land on top of old goop, and I can't change push one color of goop to the top without snapping all of that goop's color to the top.
2. The 2 added camera system is cumbersome, and I feel it should be able to be done with just 1?
3. I don't have a way to rasterize all the goop on the screen to a texture, which I'll need to do eventually so that zillions of particles don't eventually collect.

Does anyone have any ideas as to how to make the system better? I would be eternally grateful!

 

Great effect! Can you not write to Alpha? I haven't used much renderTexture so it is just a guess.

 

As far as I know there's no way to write R,G, and B without Alpha, if the camera's picking it up, it's only picking it up because it's Alpha is > 0, so that would effectively mask the other three colors onto the area being rendered by the fourth? Even if it did work it also wouldn't help with the overlap issue. I'm a shader novice though, so someone let me know if I'm wrong!

 

Depends on the blend mode. Alpha blend (Blend SrcAlpha OneMinusSrcAlpha) uses the alpha to blend, so alpha is required. You could use an additive (Blend One One) or soft additive (Blend OneMinusDstColor One) which doesn't need alpha at all or can use alpha as just another channel. There's also ColorMask which you can use to only write to one or select color channels, though this can be slow on some platforms.

So, what you could do is a soft additive blend and use all 4 color channels as your four goo colors. Note: the alpha from the texture you're using doesn't need to be alpha on the output. For an additive shader you could simply select the color channel(s) you want to be adding to and multiply it by the alpha from the texture before returning the value.

Sorting is however a harder issue. The additive shader method side steps it by having no sorting, all 4 goo colors can be in the same place, you just have to choose which one to draw on top or you could blend the colors together and have the outline just be around all of the goo rather than separated by color.

Ultimately goo sorting is something you need to handle prior to rendering rather than as a product of rendering. I have no idea how you're rendering your goo right now, but you could do it with a single shuriken particle system and set the individual particle colors via script and sort the particles by age. Then the sorting is kind of handled for you to some extent, if you're using the alpha blend shader you're using now.

 

Ahhhhh additive blending!!!
i.color * tex2D(_MainTex, i.texcoord).a
did the trick, thank you so much for that!

Sorting is indeed a harder issue. I tried blending the colors together, but I don't like how muddy that turned out.


I render the goo with a metaball shader applied to a Quad with the goo camera's output being the material's input. It checks whether each channel in each texcol is over a threshold, and if it it, it renders it the appropriate color as per the channel being used.

I suppose I could wait until no goo on the screen is moving, and then copy a texture of all the goo onto a static texture. I don't know how to make something like Graphics.Blit copy the shader's output though, as opposed to just the RenderTexture full of metaballs that the camera spits out. But even then that'd be problematic when there's lots of action and theoretically some amount of goo being rendered never stops moving.

I could also delete the particles as I go but never clear depth, but then I'd need another camera again for the blobs of goop that travel through the air, so they don't leave streaks behind them perpetually.

Maybe there's a way to take dead particles, duplicate their sprites on a large, map-sized canvas, delete them, and then send that canvas to the shader, along with any moving, alive particles?

I feel like I'm so close! Thanks for the help so far, everyone!

 

I'm more curious about the rendering of the goo sprites into the off screen camera texture rather than the rendering of the metaball shader to screen. That's when you need to do the sorting if you want it to be consistent. If you're waiting until the final shader there's only so much you can do. This is why I was suggesting a particle system; you basically don't use anything from the built in particle system features, you just manually pass a giant list of particle positions and colors to a single particle renderer and let it handle rendering and sorting. That way you can mix the different colors, have it be red, green, red with the red both below and on top of the green, still only be one draw call.

If you're not opposed to the single outline, go back to alpha blend, but use "Blend SrcAlpha OneMinusSrcAlpha, OneMinusDstColor One", yes there's a comma in there. That'll have the color values be treated like alpha blend which will let you sort them before you draw, but alpha channel itself will be soft additive blended (that's what the comma is for). The color values you should use should be the actual final goo colors and not the keyed values, and the threshold, outline and lighting should be calculated from the alpha and not the RGB color values. Now you can have an infinite number of colors and it'll just work. You might notice in this case the colors aren't flat, but that can be solved with color / alpha in the final metaball shader, and since you're already thresholding based on the alpha you shouldn't ever get a divide by zero.


Oh, and for caching, just cache the off screen camera output and not the final goo. You can do it either way, but you're still paying the cost of the metaball shader either way what and just complicating the system. At some point you'll need to take all of the sprites that have stopped moving (or are within a threshold of speed) and render them separately, storing off the texture. Since these are older sprites, presumably these should be sorted behind the moving sprites, so just render it as the background for the off screen camera. This comes back to proper pre-sorting as well as you'll only want to split of sprites that are behind ones that are still moving; it's plausible you'll have sprites that have stopped moving above sprites that are still moving and you'll want to just let those continue to be. You could do it every time a goo comes to a stop, but that might be a little too frequent and the texture copy is not free. You could do it every X seconds, but then it might happen when nothing has changed. You'll want a mix of the two; start timer to do a pass every x seconds. If the timer is already running, just let it continue until nothing is moving. Depending on perf you may have to do some checks for x number of splats forcing a pass if rendering a lot of the sprites gets slow.

 

So I'm using that blending mode now, and I divide by the alpha channel which looks great!



Unfortunately, we're still blending colors together though, so I posterize the splats into one of the four applicable colors. I check which color each pixel is closest to in color, and then snap it to that color.


Looking good!

But now how do I shade, highlight, and outline my splats? Beforehand, my sampler2D's were separated into 4 distinguishable channels. In order to check surrounding pixel positions to determine whether it should be shaded and how, I now need to calculate and posterize the color of those surrounding pixels first for stuff like

if(pixelColorAbove != myPixelColor) {
myPixelColor = highlightPixelColor
}

Which means that I'm checking 20 neighboring pixels...for every pixel... Computers are fast so it keeps up on my desktop, but it lags a bit on my laptop.


...this...is awful...

So obviously there's gotta be a better way to do this. It'd be great if I could do a first pass where I posterize everything into those four initial colors, and a second pass where I shade, highlight, and outline, cause then for every pixel I'm not calculating 20 surrounding pixel values, I'm just calculating one. I need to setup a second Tex2D that's written with the values of that posterization, and then use that to calculate all that other junk, but Google hasn't been cooperative in helping me figure that out, so let me know if you know how to do this! For each pixel I'm calculating I'm also doing tex2D (AliveSplats, UV ) + tex2D (DeadSplats, UV), and it'd be more efficient if I just combined the textures first and called tex2D() just once, right? Another thing I'm not sure on how to do :/

As for Cache-ing, someone on TIGSource suggested using a camera that never clears just for the "dead" splats.

So then I had two materials being rendered, one for "alive" splats (those that are airborne and shouldn't streak across the screen), and one for "dead" splats, that are written to a camera that's never being cleared. This is great! I don't have to do any memory management too because all the particles die within a second or two after throwing the goop, and their resulting splats live on!


But there's another problem, alive splats are all laid on top of dead splats, which works, but it sort of breaks the metaball magic I have going on when moving particles are overlaid on top of still splats. So instead of having two shaders/materials, I now combine the color values from both the alive and dead RenderTextures.

AND IT LOOKS FANTASTIC!!! Both behaviors combine seamlessly in a single pass!


This looks even better at 60fps but LICECAP only gives me .gif's at like 15 or so


Thank you SO MUCH for helping me, bgolus!!!! Let me know if you know how to do those two optimizations I mentioned earlier or anything else like that!

 

This is why I had suggested the single outline around all colors using only the alpha. With that only doing shading around the whole mass as well. You can reduce the color bleed by reducing the falloff in your alpha, but you do loose the interior outlines (which I agree look great so it's sad to loose them).

As for merging the two textures, the dead splat render target should be rendered in the background of the living splats. Just have a large quad with the dead splat texture on it that gets rendered before the moving splats.

If you want to keep something closer to the current implementation, but make it faster, you could do the posterization as a camera post process on the live splat render to texture camera; could even do the thresholding so the texture you get is already hard edged flat colors. Though honestly halving the number of texture lookups you do by combining the live and dead splats in the camera will make things a lot faster.

Other things to think about: if conditions are relatively slow, especially if they change very pixel and aren't preventing a lot of extra math, so use them sparingly. Often times it's faster to just do a some extra math and a lerp instead of an if, though that might take a bit more thought and liberal usage of the saturate and sign functions.

if (xA > xB) color = colorA; else color = colorB;
vs
color = lerp(colorB, colorA, saturate(sign(xA - xB)));
​

Those two should produce the same result, but the later will likely be faster. Modern computers are super fast though, as you said, so it might not be that noticeable.