Performance - need help figuring out some things

Hi there.

We're working on Candle, a 2D sprite-based sidescrolling game, and we are currently running into optimization tasks. I would love to have some basic knowledge on this. Our general setup is this: 3D physics working with a full 3D environment and perspective cameras (despite using 2D sprites, the game makes use of the Z axis).

Our scenes have very simple colliders (cube primitives, mostly, with some simple mesh colliders for the grounds), no 3D lights, no shadows, a basic use of physics (character gravity and some special objects here and there, such as moving boxes or falling rocks. No more than 5 per scene), batching for almost every object in scene (full static gameObjects), no "Instantiate" at all (no bullets and stuff. The only multi-prefab environment is our Inventory UI, and we use basic pooling for all the icons, buttons and descriptions)... Right now, we are running on rather low specs, here's a quick glimpse (simple VSync enabled):



Left is our smallest scene in the game - Right is the biggest. With final assets in both of them (Right scene has way more verts & tris because there's a big ocean in there, a very large mesh that we deform in realtime to simulate waves). Overall, we are not experiencing performance issues except for very old (decade-old) PCs, but we do have some issues with the Wii U. We can't talk publicly about the console specs and such, but we are really having trouble with stability in that console. Either we go for 1080p@30fps or 720p@60fps, but no more, and even then we experience some stuttering sometimes.

I was wondering how could we gain a some extra frames by solving things. We barely use intensive shaders (we have stuff like vignetting, grain and contrast, all home-made, and some simple water reflection shader), but we could maybe tweak those. Our few post-processing effects we use are fullscreen, but something as simple as applying a PNG texture to simulate vignetting shouldn't be too expensive, right (albeit being fullscreen effects)?

Also, I was wondering how to deal with some Quality settings. We've disabled almost everything (no lights, no shadows, no anisotropic filters, no anti-aliasing, our textures are compressed...), but our main issue is VSync.

If we disable VSync (and do not set any Application.targetFrameRate) our FPS count skyrockets and reaches 450-500 FPS, but this is not quite optimal beacuse then physics go nuts. I can either set VSync to "Every VBlank", OR disable it but set Application.targetFrameRate to 60, but then the framerate issues arise (again, not in PC), since WaitForTargetFPS consumes a lot, as seen in the Profiler (which is obvious since we are limiting the framerate). And if we disable VSync but limit targetFrameRate, then the screen tearing is brutal.


Anyway, that's our scenario, any tips on how could we proceed with some more optimizations (our game is only targeted to home systems: PC and next-gen consoles)?

 

You could use the new GUI system for the vignette. Just make a 256x256 texture in photoshop like this then stretch it across the screen. But I don't think the vignette effect has much of a performance cost.

I can't help with the rest.

Also, final build performance is higher than when reading from the editor, especially if you are spamming debug messages in the editor, etc. Try removing them when testing performance, and remove them from the final build.

 

Yep, we have hundreds of debug messages, but we will remove them from the final build.

As for the vignette, we switched to a solution similar to yours Thanks!

 

I'd imagine that if you are deforming a big mesh, there is some significant bottleneck involved there. Does it also deform the collider? That can be quite slow. Your draw calls are also much higher in the big scene, any way to combine things more?

 

There is no collider for the water mesh, it is just a procedural plane with a texture (and the deformation script). Nothing interacts with the water plane, luckily.

Unfortunately, that water plane is a sea, so it is quite big since it has to be drawn to a considerable distance. We've tried reducing the number of polygons we generate for it, but then the low mesh resolution really affects how the wave deformation looks (generates lots of artifacts instead of looking smoother).

 

Understandable. But I imagine pushing that new mesh data up to the gpu every frame is the main performance hog. what happens when you totally cut it out?

 

Are you saying that with VSync disabled, it runs with a smooth frame rate on the Wii U, well in the hundreds, but once you enable it, you get fps drops below the refresh rate and stuttering? That would sound more like a bug of some sort rather than a performance issue with your game and you should contact Unity.

Anyway, assuming the problem is not specific to the console, but rather in the overall performance, I'd suggest you to track down the bottlenecks through Unity's built-in profiler. It appears you're not GPU bound, but rather the problem lies in the CPU code or in the communication between the two. As mentioned above, you should focus on trimming down on the draw calls. Using a texture atlas for the sprites might be a great way to enable otherwise incompatible objects to batch together. Same thing with the static scene geometry.

From what I gathered from your post, the frame drops are not limited to just that one scene that uses dynamic water, otherwise the culprit would be obvious Another reason is that I don't think it should be that much of a problem on consoles. The CPU to GPU pipeline is usually much shorter there and moving data through it generally doesn't take as long as on PC. Either way, you should consider moving your vertex displacement code to the vertex shader and get rid of this altogether.

 

@imaginaryhuman If we remove the water meshes we gain some extra frames, yeah, but it's not a radical difference. I guess the problem lies in what @Dolkar says.

GPU calculations take 0.1-0.2ms (extremely good). With VSync disabled all the CPU calculations take an average of 1.2ms - even in the biggest scene. But if we enable VSync, then our main issue in the profiler is the WaitForTargetFPS callback, it takes 90% of the CPU time and generates the biggest performance peaks. It is fine in the smaller scenes (in those we're around 80-100fps at 1080p even with VSync), but in the biggest scenes it gives us quite a headache. Our textures are very high resolution, and the water, shaders and particle effects really play against us. Again, if we disable VSync the issue is completely gone, but I'm not comfortable with the possible impact a skyhigh FPScount can have in physics.

We have, like, 95% of the mesh gameObjects in the scenes marked as full static (batched meshes and stuff) since they are mainly backgrounds, and we do use large sprite collections for our graphics so we can save as much draw calls as possible. Let me show you some screenshots:



What's the first thing you notice? The huge resolution of our textures. This is totally intentional and something that is really affecting the development. Everything you see is not digital: they're watercolours (you can see our development process here in case you are interested), and since we spend weeks painting physically every character, item or background our main commitment is to image quality. If we compress further our sprites (or reduce their resolution) we lose a lot of quality, and that's something we don't want since it would be pointless to have such a demanding hand-made process but pixelated, low-res in-game textures.

We try to include as much of them as possible into sprite collections, but we don't have any single scene whose graphics fit in a single atlas. For the biggest one we've needed eight 4K atlases for all the backgrounds to fit. We can't save much more draw calls, I'm afraid (but I don't think that's a problem since our average draw calls per scene is 25-50, nothing to worry about).

Should we focus our attention in VSync? I'm still unsure on how to deal with that, FPS count and physics.

 

WaitForTargetFPS should take significant amount of CPU time.. it means that the engine is done rendering the frame sooner than the target FPS is, so it just waits. In your heaviest scene, you should see less time is spent waiting, but unless it disappears completely, you should still have stable 60 fps. In theory at least. If you see it spiking or generally slowing down your game more than it should, you have a problem. If that's the case, would you mind taking a screenshot of the profiler in that heavy scene?

I'm loving the visual style, by the way. Your artistic pipeline sure helps a lot and I definitely wouldn't suggest downscaling the textures just to reduce the draw calls a bit.

 

The thing is that the final compiled version always plays better performance-wise. The Editor version is always slow as hell (maybe a decent 64-bit editor would have solved many things, but...), and enabling the Profiler tab only makes thing worse (not to speak about "Deep profiling"). Anyway, here's a comparison:

• VSync enabled:

• No VSync:

It may seem like a little difference, but there's a significant drop on the frame count. This scenario is obviously only happening in the large sea scene (the AdvancedSpringWaterMovement script is the procedural water mesh one, and it is obviously our main bottleneck there), so maybe we shouldn't worry? Since every other scene works OK at 60fps with VSync enabled, maybe we are just worrying too much.

But since we can't know the end user setup for each one who buys our game, we can't know for sure how good will it perform in old PCs (we have one here for testing purposes, but there are so much possible hardware setups that we're a little concerned).

 

Your screenshot without VSync also does not have the the water script, so it's hard to compare, but from what I'm seeing here, VSync is probably not the issue at all... Besides the actual water script taking considerable amount of time (almost 30% every frame), it's also heavy on memory allocation. That means every once in a while you get a spike caused by the garbage collector freeing up all that memory. There are plenty of guides around the forums and the web on how to reduce memory allocations.
Still, the best option here would be to move the water movement to the vertex shader, if possible. Because you are definitely not vertex bound, doing the displacement on the GPU most likely wouldn't even make a dent in your performance, saving you the entire 30% for that scene.

 

I missed water in that one? I'm sorry, I didn't notice. I disabled it just a moment to compare and totally forgot to re-enable it :S Will definitely rework the water displacement thing to a shader.

Oh, and by the way, we tried some kind of custom culling solution, but we couldn't achieve anything due to how camera rendering works in Unity.

We tried attaching a script to every sprite and particle effect that would disable those gameObjects not only if they're not inside the camera canvas, but if they are obscured by another sprite. But we couldn't make it work, because in a situation like this:



The selected area inside the gizmo is a background located far beyond in Zpos, yet

Code (CSharp):

1. Renderer.isVisible

returns TRUE, which is such a pain. I know that the Editor view counts as a camera, but we've tried compiling the game and

Code (CSharp):

1. Debug.Log(Renderer.isVisible)

also returns TRUE. It is only FALSE when the whole sprite canvas is out of the camera view (which is also bad since a PNG transparecy with alpha = 0 shouldn't count as an image since there's actually nothing in those pixels). 2DToolkit's sprites are just textures, so even if a sprite image is completely obscuring another sprite (or particle effect), it will still be marked as visible by Unity. That prevents us from saving even more performance in the Rendering area.

 

Renderer.IsVisible only handles cases where your objects are completely outside of the view and are therefore frustum culled. It does not return false for objects occluded by another object, because that would be quite an expensive operation.

From looking once more at the profiler... I noticed most of your gpu time is spent rendering transparent objects.. those seem to be the background layers... but to me it seems like you do not use actual transparency very often, but you rather use the alpha channel as a mask where a layer is either completely visible or not. Consider rendering the background layers with an alpha test shader instead of true transparency as that should be much faster. Even if you need the transparency, it might still be a good idea to clip pixels with zero and near zero alpha with something like clip(color.a < 0.01 ? -1 : 1) so they are not rendered at all, otherwise even though these pixels don't actually affect the final image, they are still quite expensive to render due to blending.

 

Oh, that's quite interesting. The Transparent shader is the default 2DToolkit one and we didn't really think on changing that, but I will try a solution like yours and see if that helps us. The background images are always still images, we never apply any kind of alpha change to them, so I see your point, it really would be a good idea to modify them.

Unfortunately, the main zones, items and characters do use transparency often (with smooth alpha transitions), so we can't solve that with them. But fixing it for the backgrounds seems like an obvious solution.

Thanks for that!

 

Your graphics are very nice indeed. I notice the visual beauty first, although I dont put it down to texture resolution as much as first noticing the objects and colors. Anyway.. I agree I would not want to sacrifice quality by going with compressed textures. Texture compression is 'lossy' and has a lot of artefacts and lowering of the color quality. If you're really into quality then you have to avoid it. BUT... that only applies to the build-in hardware driven texture compression. There is other compression you can do. For example, any of the individual objects in your scene appear to have a fairly focussed color palette. Brown rocks, a beige tree, green clumps of grass, etc... this makes them a good candidate for compression. I would not go with 16bit color either. You still want 24-bit, but I would go with a custom palette shader, either 8-bit palette or 16-bit palette. You can with 8-bit use an alpha8 1-byte-per-pixel texture which will save a tonne of texture memory AND render fast. Provided you can manually create your palettes to a quality you like, via quantizing algorithm, you could probably get very nice looking 256-color objects. If that's not enough, then use 2 alpha8 textures per sprite, with a 16-bit color palette, ie the two bytes index a row and column lookup in a palette texture. 65536 unique colors (with 24-bit quality) will be practically indistinguishable.

 

We do compress our images. We use some tweaked compression in Photoshop as well as Posterizing, which reduces color information on the final image.

But I will delve a bit deeper following your advice in case we still can compress them a bit more.

 

Need detailed expanded profiler shots, sorted by Time ms, only need to see the data, not the silly graph up top.

In any case, the other brothers had 6-8x 2048x2048 unique textures per level. We simply tiled everything into large 512x512 chunks (for easy culling), turned off static batching and occlusion and took the overdraw hit. It's not a problem. Any device should be able to draw the entire screen at least 4 times at 60fps.

What I see here so far is an abnormally high CPU usage. How much garbage are you allocating per frame? Also if your stuff is too optimised then it will hurt your performance. Static batching in 2D is actually a bad idea on 4.x but a better idea on 5.0.

10ms / frame gives you very little wiggle room for 60fps. Your limit is 16, so a few issues there. For 2D, I imagine 10ms to be too high.

 

We do have static batching for the scenes and we are working on 4.x, should we avoid using that?

 

Profiler can go through the roof with culling cost with too much static batching in 4. You really need to show cpu profiler, sorted by time ms and expanded in each rollout.

 

I'm looking at this:

Do you mean that we should avoid making 4096x4096 collections and go for, say, 1024x1024 ones with sprite dicing? We thought it was better to have one sprite collection per scene rather than several collections for a few sprites each.

 

No, I mean post your profiler stats. It's the 3rd time I've asked, and possibly the best way to narrow down what is actually happening.

Concerning the collections, we found that just having a 'megatexture' approach where you just have massive square lumps to be kinder on culling for huge levels, where all the art is part of a massive painting. That was our approach: every pixel was unique. That's fine, but in your case we still don't know if what you're doing is actually a problem there without profile data.

At this point I'm running blind and I can't assist.

 

Hey, it's OK, no need to feel offended, and I'm sorry if I made you upset somehow. I'm just very busy right now and I'm not going to stop everything I'm doing for the Profiler thing, but please give me a couple of hours and will post that info.

 

Sure, I'm not offended. I'm just saying, you keep asking for info but that's the most valuable thing to use in unity for framerate.

 

Ok, here you guys go. Please let me know in case you need something more specific or more expanded shots. Deep Profiling on the biggest scene in the game:

• Rendering

• Water script & NGUI (asset for UIs)

Now some clarifications. This scene, as I said, has a big sea which is a very large procedural plane drawn to a considerable distance and deformed at runtime to simulate waves (no collider attached to it since it does not interact with physics), with a material whose shader has a reflection effect so it drawns all the graphics & particles mirrored. Also, it's raining in that scene, so (among other efects) we spawn lots of raindrops randomly, like this:



The waterdrop sprite is very, very low resolution, but we are already aware of its potential performance impact. Right now, the raindrops are spawned, drawn, and destroyed when they leave the camera frustum. But we are working on an effect (we are still unsure if we'll go for a particle or for an animated sprite) that will replace this by a much more performance-friendly one (having all the rain as one unique graphic instead of creating tons of them every sec).

As for the 'UISprite' calls in the Profiler, that's a UI asset we use called NGUI. Unfortunately, the guy who works on it dramatically changed how things work in the lastest versions. It now performs way better but we are facing our last 3-4 months of development and NGUI's code is so different now that we don't want to change everything on that side (since our UI involves a quite complex inventory management system and reworking that would be nuts). We are working with Unity v4.3.4f1 since the Wii U editor version is stuck there, so no 4.6 UIs for us (although we've been really tempted to jump to 5.0 and use its UI and Audio tools, but we can't know when will Nintendo jump to the 5.x cycle).

 

Some thoughts:

1. GC Alloc is quite high 1.4K per frame will cause occasional stutters when things are this tight & a collect happens. Perhaps see why the culling here is reporting allocations - that should never happen. Perhaps something's static that shouldn't be? regardless, investigate to remove that...

2. the 70 (!) K from multiple generate planes isn't something you ever want to see - in short, for any hardware, you shouldn't be allocating more than a few hundred bytes in a frame. Once it gets to kilobytes, you start feeding the GC monster and it repays you back with hiccups as it collects. So you might be coming in say 14ms / frame and then bang, a collect takes 3ms, and it'll drop from 60 to 30, causing a hiccup. This hiccup might not be noticed if the scene isn't consuming much time, but will be felt when you are close to the wire.

In short you need to really clear out this massive amount of garbage allocation you've got going on. If you modify meshes, *do not read from the mesh* (except in Start()) and instead store this data locally, modify it and only write to the mesh.

Other thoughts are it's odd culling is taking over 3ms, and allocating.

I'm assuming this is a sample from when it's running ie a current sample, and not during setup.

 

That's because it's not the culling itself that causes the allocation, its the OnWillRenderObject() of the WaterReflection script that gets called in there. So yes, as I was saying, its the water script that's causing you most of the problems, both in terms of huge memory allocations and cpu usage in general (9.66ms!). An ideal solution would be to move this code to the vertex shader, if possible. Another option is to simply optimize.

 

Ok, some things I'm ashamed I didn't notice.

1. The water mesh IS marked as static (my bad). I don't know when or how did this happen, but it definitely wasn't intentional. That mesh has thousands of verts, so... that's fixed now. We barely have static gameObjects (definitely not particles or dynamic sprites), but how do you think we should deal with that in a 2D game (not orthographic, we do use Zpos for gameplay)? Are the 'static' options in 4.x really necessary in a game like ours?

2. That's definitely an issue with our water deformation script (only exists in this room). Mind if I post here our code?

Code (CSharp):

1. public class AdvancedSpringWaterMovementDepthConstant : MonoBehaviour
2. {
3.     private GeneratePlane m_planeScript;
4.     public float A = 1.0f;
5.     public float angularFrequency = 1.0f;
6.     public float offset = 1.0f;
7.  
8.     public float currentHeight = 0.0f;
9.  
10.     void Awake ()
11.     {
12.         m_planeScript = GetComponent<GeneratePlane>() as GeneratePlane;
13.     }
14.  
15.     void Update ()
16.     {
17.         ArmonicMovementSimple();
18.  
19.         m_planeScript.AssignValues();
20.     }
21.  
22.     private void ArmonicMovementSimple()
23.     {
24.         float height = 0.0f;
25.  
26.         int j = 0;
27.         int counter = 0;
28.         for(int i = 0; i < m_planeScript.m_vertex.Length; i++)
29.         {
30.             //for(int i = 0; i < (int)fXSections; i++)
31.             //{
32.             if(j == (int) m_planeScript.fZSections+1)
33.             {
34.                 counter++;
35.  
36.                 j = 0;
37.             }
38.             height = A* Mathf.Sin ((angularFrequency * Time.time) + offset*(float)(counter));
39.             m_planeScript.m_vertex[i] = new Vector3(m_planeScript.m_vertex[i].x, m_planeScript.yOffset + height,m_planeScript.m_vertex[i].z);
40.  
41.             currentHeight = height;
42.  
43.             //m_vertex[i] = m_myMeshFilter.sharedMesh.vertices[i];
44.             j++;
45.         }
46.  
47.         m_planeScript.AssignValues ();
48.  
49.     }
50. }

And from the GeneratePlane.cs script:

Code (CSharp):

1. public void AssignValues()
2. {
3.     reposePositionY = yOffset; //Init position
4.  
5.     m_myMesh.vertices = m_vertex;
6.     m_myMesh.uv = m_uvs;
7.     m_myMesh.triangles = m_tri;
8.     //m_myMesh.normals = m_normals;
9.     m_myMesh.RecalculateNormals();
10.  
11.     m_normals = m_myMesh.normals;
12.  
13.     m_myMesh.RecalculateBounds();
14. }

Would it be better to rework this in the vertex shader, as @Dolkar pointed? Or should we handle this behaviour differently?

The reflection script is quite intensive, we know, but is there a nice solution to recreate water reflection without impacting performance too much?

Anyways, thanks a lot for your help and advice guys.

 

I forgot to mention that I have some important family issues to deal with in the coming days, so please excuse my absence if I do not answer immediately.

 

Here's a fast edit you can do... you're calling m_planeScript.AssignValues() twice, once in the ArmonicMovementSimple() function and once in Update(), so get rid of one of those.
The allocations are caused by reading back the mesh normals after you recalculate them... If you don't actually need them for anything, you should remove that as well. That should get you rid of all those unnecessary allocations.

 

That's certainly useful.

We changed the AssignValues() call from Update() to inside the ArmonicMovementSimple() function since it felt more appropriate and readable, but I didn't notice we left the original line untouched. I guess we wanted to comment it but forgot.

As for the normals, we do need them but only for one specific thing, so it will be better to read them in that moment instead of updating the values constantly.

I'm leaving the studio right now, but I will try to find some time tonight to make those changes and I'll get back to you with the result ASAP.

 

All water stuff is best off done in vertex shader. From the looks of it, it doesn't seem a problem to port it to that, and it'll then just calculate for essentially free (vertex shaders are incredibly fast). This, above all should be your optimisation goal, then have another round of profiler action.

 

Having looked at the game, I'm 100% sure you should be getting 1080p @ 60fps on mobile, let alone Wii U. Please don't be shy and hassle the forum for optimisation tips. With art like this, I want to see it 1080p on every platform - so people will help (it just takes time to notice these threads).

Reading up on the thread, I couldn't help but notice you were having physics problems with vsync off. This indicates you're not performing forces in FixedUpdate, which is kind of vital.

 

I'm not sure if it would be that easy in this case. If you want to support reflection, you need normals. Computing smooth normals of deformed vertices in the vertex shader is not trivial, as you don't know anything about other vertices in the triangle. It could be solved by passing a bunch of extra data per vertex, but I think some math could help too...

If you start with a flat plane and displace the vertices only along the Y axis... then you're basically "plotting" the height function.. that means that at any point, the normal should be the cross product of the tangent vectors you get from derivatives of that function at the x and z axis. I guess that's what many other gpu based water shaders already do, so that could save you some work...

I still see plenty of room for optimizing your script there.. so it might not even be necessary.

 

Oh, no no no, you got me wrong there (sorry if I didn't explain myself correctly, English is not my primary language ). We haven't had any physics issue in the entire development. Every physics calculation applied to the rigidBodies is made in the FixedUpdate() and using things like Time.fixedDeltaTime when necessary. Candle is the very first thing I've ever coded, but even I can deal with such basic stuff. Thanks for point it out, though!

What I was trying to say is that, since our main issue is framerate on Wii U (and some extra optimization for PC can't harm anybody), I was wondering if disabling VSync could be a viable tradeoff, but that makes our FPScount skyrocket up to an average 300fps, and I was worried on the possible implications that could have in the game's physics calculations. Yeah, I could just clamp the framerate with Application.TargetFrameRate = 60, but with VSync disabled the screen tearing is atrocious.

We don't have crazy physics, just basic pulling/pushing crates, killer indianajones-esque rolling rocks, some (oh my, I HATE those in 4.x) hanging ropes with joint physics... and that's it, mostly. Well, all the characters movement behaviours are basically forces applied to their rigidBodies, but again that's very simple code.

Our main performance headache is framerate, which comes from CPU bottleneck (on the largest room mainly caused by the water thing, but in the others it's just a Rendering issue, for what I'm observing in the Profiler. We've oversaturated the scenes with particle effects, and our camera post-processing effects are GrabPass shaders which have proven to be quite intensive as well. The water reflection also takes a lot).

 

Application.TargetFrameRate is a horrendous piece of bad Unity code. It's never worked properly or consistently, so it's probably best to avoid using it.

Regarding normals in the water shader, I'm not understanding why this would be a complicated thing if it's pointing up. I've done water shaders and I didn't actually need to fiddle with the normals since I was merely raising or lowering height. Is this acceptable in your case?

 

No worries, we're not using it at all. As for the water, we have 2 codes that work together: a shader and a C# script.

Here's the full shader code:

Code (CSharp):

1. Shader "Teku/SeaLayer" {
2.     Properties
3.     {
4.         _MainTex ("Main Texture", 2D) = "white" {}
5.    
6.         _LayerTex ("Layer Texture", 2D) = "white" {}
7.  
8.         _NoiseTex ("Noise Texture (RG)", 2D) = "white" {}
9.      
10.         _ColorTint("Color tint", Color) = (1,1,1,1)
11.    
12.         _ReflectionTex ("Internal Reflection", 2D) = "" {}
13.  
14.         _GeneralDistortion ("Distortion", float) = 0.06
15.    
16.         _TimeScaleDistortion ("Time Scale Distortion", float) = 1.0
17.    
18.         _MainTextureAlpha ("Main Texture alpha override", Range(0.01, 1.0)) = 1.0
19.    
20.         _ReflectionMixGrade ("Reflection mix grade" , Range(0, 1.0)) = 1.0
21.    
22.         _OverallAlpha ("Overall alpha", Range(0, 1.0)) = 1.0
23.    
24.         _SectionsWide ("Water sections WIDE", float) = 1.0
25.    
26.         _SectionWidth ("Water section width in Unity units", float) = 1.0
27.    
28.         _SectionsHeight ("Water sections HEIGHT", float) = 1.0
29.    
30.         _SectionHeight ("Water section height in Unity units", float) = 1.0
31.    
32.         _ActiveDepthFar("Active smooth depth FAR", float) = 1.0
33.    
34.         _ActiveDepthNear("Active smooth depth NEAR", float) = 1.0
35.    
36.         _DisplacementUV ("Displacement UV Main tex", float) = 1.0
37.    
38.         _DisplacementUV2 ("Displacement UV Layer tex", float) = 1.0
39.    
40.          _AlphaHeightFactor ("Alpha Height border", float) = -1.0
41.      
42.         _AlphaWidthFactor ("Alpha Width border", float) = -1.0
43.    
44.     }
45.  
46.     Category
47.     {
48.         Tags { "Queue" = "Transparent+1" "RenderType"="Transparent" }
49.  
50.         SubShader
51.         {
52.             Pass
53.             {
54.                 Name "NEXT"
55.  
56.                 Tags { "LightMode" = "Always" }
57.            
58.                 Fog { Color (0,0,0,0) }
59.  
60.                 Lighting Off
61.  
62.                 Cull Off
63.  
64.                 Blend SrcAlpha OneMinusSrcAlpha
65.          
66.                 CGPROGRAM
67.            
68.                 #pragma glsl
69.                 // Upgrade NOTE: excluded shader from Xbox360; has structs without semantics (struct v2f members distortion)
70.                 #pragma exclude_renderers xbox360
71.              
72.                 #pragma vertex vert
73.              
74.                 #pragma fragment frag
75.              
76.                 #pragma fragmentoption ARB_precision_hint_fastest
77.              
78.                 #pragma fragmentoption ARB_fog_exp2
79.            
80.                 #pragma target 3.0
81.              
82.                 #include "UnityCG.cginc"
83.            
84.                 sampler2D _GrabTexture : register(s0);
85.  
86.                 sampler2D _NoiseTex;
87.            
88.                 uniform float4 _NoiseTex_ST;
89.            
90.                 sampler2D _MainTex;
91.            
92.                 uniform float4 _MainTex_ST;
93.            
94.                 sampler2D _LayerTex;
95.            
96.                 float4 _ColorTint;
97.            
98.                 sampler2D _ReflectionTex;
99.            
100.                 uniform float _GeneralDistortion;
101.            
102.                 uniform float _TimeScaleDistortion;
103.            
104.                 uniform float _MainTextureAlpha;
105.            
106.                 uniform float _OverallAlpha;
107.            
108.                 uniform float _DisplacementUV;
109.            
110.                 uniform float _DisplacementUV2;
111.            
112.                 uniform float _ReflectionMixGrade;
113.            
114.                 uniform float _ActiveDepthFar;
115.            
116.                 uniform float _ActiveDepthNear;
117.            
118.                 uniform float _AlphaHeightFactor;
119.            
120.                 uniform float _AlphaWidthFactor;
121.            
122.                 float _SectionsWide;
123.    
124.                 float _SectionWidth;
125.            
126.                    float _SectionsHeight;
127.    
128.                 float _SectionHeight;        
129.    
130.                 struct data
131.                 {
132.                     float4 vertex : POSITION;
133.              
134.                     float3 normal : NORMAL;
135.              
136.                     float4 texcoord : TEXCOORD0;
137.                 };
138.            
139.                 struct v2f
140.                 {
141.                     float4 position : POSITION;
142.              
143.                     float4 screenPos : TEXCOORD0;
144.              
145.                     float2 uvmain : TEXCOORD2;
146.                
147.                     float2 uvTex : TEXCOORD1;
148.                
149.                     float4 ref : TEXCOORD3;
150.                 };
151.  
152.                 v2f vert(data i)
153.                 {
154.                     v2f o;
155.              
156.                     o.position = mul(UNITY_MATRIX_MVP, i.vertex);      // compute transformed vertex position
157.              
158.                     o.uvmain = TRANSFORM_TEX(i.texcoord, _NoiseTex);   // compute the texcoords of the noise
159.                
160.                     o.uvTex = i.texcoord.xy;
161.              
162.                     o.screenPos = o.position;   // pass the position to the pixel shader
163.                
164.                     o.ref = ComputeScreenPos(o.position);
165.              
166.                     return o;
167.                 }
168.  
169.                 float4 frag( v2f i ) : COLOR
170.                 {
171.                     // compute the texture coordinates
172.              
173.                     float2 screenPos = i.screenPos.xy / i.screenPos.w;   // screenpos ranges from -1 to 1
174.  
175.                     screenPos.x = (screenPos.x + 1) * 0.5;   // I need 0 to 1
176.              
177.                     screenPos.y = (screenPos.y + 1) * 0.5;   // I need 0 to 1
178.  
179.                     // check if anti aliasing is used
180.              
181.                     //if (_ProjectionParams.x < 0)
182.              
183.                     screenPos.y = 1 - screenPos.y;
184.  
185.                     // get two offset values by looking up the noise texture shifted in different directions
186.              
187.                     half4 offsetColor1 = tex2D(_NoiseTex, i.uvmain + (_Time.xz * _TimeScaleDistortion));
188.              
189.                     half4 offsetColor2 = tex2D(_NoiseTex, i.uvmain - (_Time.yx * _TimeScaleDistortion));
190.              
191.                     // use the r values from the noise texture lookups and combine them for x offset
192.              
193.                     // use the g values from the noise texture lookups and combine them for y offset
194.              
195.                     // use minus one to shift the texture back to the center
196.              
197.                     // scale with distortion amount
198.              
199.                      float xDistort = ((offsetColor1.r + offsetColor2.r) - 1) * _GeneralDistortion ;
200.                      float yDistort = ((offsetColor1.g + offsetColor2.g) - 1) * _GeneralDistortion ;
201.              
202.                      //BACKGROUND distortion
203.                     screenPos.x += xDistort;
204.                     screenPos.y += yDistort;
205.                     float4 col = tex2D( _GrabTexture, screenPos );
206.                
207.                     //MAIN texture mixing with LAYER
208.                
209.                      // Main texture mixing
210.                      float2 newUV = TRANSFORM_TEX(i.uvTex, _MainTex);
211.                      float2 newUV2 = newUV;
212.                  
213.                      //Distortion and displacement MAIN
214.                      newUV.y += yDistort;
215.                      newUV.x += (_Time.x * _DisplacementUV) +  xDistort;
216.                      //Distortion and displacement LAYER
217.                      newUV2.x += (_Time.x * _DisplacementUV2) + xDistort;
218.                      newUV2.y += yDistort;
219.                  
220.                      //MIXING MAIN and LAYER
221.                      float4 mTexCol = tex2D(_MainTex, newUV) * _ColorTint;
222.                      //mTexCol.a = _MainTextureAlpha;
223.  
224.                      float4 mTexCol2 = tex2D(_LayerTex, newUV2);
225.                      mTexCol.rgb = (mTexCol.rgb * _MainTextureAlpha) + (mTexCol2.rgb * (1.0f - _MainTextureAlpha));
226.                      //mTexCol has the mixed color NOW :)
227.                
228.                     //REFLECTIONS
229.                     float4 uv1 = i.ref;
230.                     //Distortion reflection TEX
231.                     uv1.x += xDistort;
232.                     uv1.y += yDistort;
233.                     half4 refl = tex2Dproj( _ReflectionTex, UNITY_PROJ_COORD(uv1) );
234.                
235.                     //MIX MAIN/LAYER with REFLECTION
236.                     col.rgb = (refl.rgb * _ReflectionMixGrade) + (mTexCol.rgb * (1.0f - _ReflectionMixGrade));
237.                
238.                     //col.rgb = (mTexCol.rgb) + (refl.rgb * refl.a * (1.0f - mTexCol.a));
239.                
240.                     float objectWide = _SectionsWide * _SectionWidth * 100.0;
241.                     float objectHeight = _SectionsHeight * _SectionHeight * 10.0;
242.                
243.                     float midWidth = objectWide * 0.5f;
244.                     float midHeight = objectHeight * 0.5f;
245.                
246.                     float widthFactorLeft;
247.                     if(_AlphaWidthFactor == -1.0)
248.                         widthFactorLeft = midWidth *0.5f;
249.                     else
250.                         widthFactorLeft = _AlphaWidthFactor;
251.                
252.                     float widthFactorRight = midWidth + (midWidth - (widthFactorLeft));
253.                
254.                     float heightFactorLeft;
255.                     if(_AlphaHeightFactor == -1.0)
256.                         heightFactorLeft = midHeight *0.5;
257.                     else
258.                         heightFactorLeft = _AlphaHeightFactor;
259.                     float heightFactorRight = midHeight + (midHeight - (heightFactorLeft));
260.                
261.                     float widthAlpha = 1.0f;
262.                     float heightAlphaNear = 0.0f;
263.                     float heightAlphaFar = 0.0f;
264.                
265.                     //TRANSPARENCY in all edges
266.                     if(i.uvTex.x <= widthFactorLeft)
267.                     {
268.                         widthAlpha = 1.0f -((widthFactorLeft  - i.uvTex.x) /widthFactorLeft);
269.                    
270.                         if(i.uvTex.y <= heightFactorLeft)
271.                         {
272.                             if(_ActiveDepthNear == 1.0)
273.                             {
274.                                 heightAlphaNear =(1.0f -((heightFactorLeft - i.uvTex.y) /heightFactorLeft));
275.                                 col.a = (widthAlpha * heightAlphaNear);
276.                             }
277.                             else
278.                                 col.a = widthAlpha;
279.  
280.                         }else if(i.uvTex.y >= heightFactorRight)
281.                         {
282.                             if(_ActiveDepthFar == 1.0)
283.                             {
284.                                 heightAlphaFar =(1.0f -((i.uvTex.y - heightFactorRight) /heightFactorLeft));
285.                                 col.a = widthAlpha * heightAlphaFar;
286.                             }else
287.                                 col.a = widthAlpha;
288.  
289.                        
290.                         }else
291.                             col.a = widthAlpha;
292.                
293.                        
294.                     }
295.                     else if(i.uvTex.x >= widthFactorRight)
296.                     {
297.                             widthAlpha = 1.0f -((i.uvTex.x - widthFactorRight) /widthFactorLeft);
298.                        
299.                             if(i.uvTex.y <= heightFactorLeft)
300.                             {
301.                                 if(_ActiveDepthNear == 1.0)
302.                                 {
303.                                     heightAlphaNear = (1.0f -((heightFactorLeft - i.uvTex.y) /heightFactorLeft));
304.                                     col.a = heightAlphaNear * widthAlpha;
305.                                 }else
306.                                 col.a = widthAlpha;
307.  
308.  
309.                             }else if(i.uvTex.y >= heightFactorRight)
310.                             {
311.                                 if(_ActiveDepthFar == 1.0)
312.                                 {
313.                                     heightAlphaFar =(1.0f -((i.uvTex.y - heightFactorRight) /heightFactorLeft));
314.                                     col.a = heightAlphaFar * widthAlpha;
315.                                 }else
316.                                 col.a = widthAlpha;
317.  
318.                            
319.                             }else
320.                                 col.a = widthAlpha;
321.                            
322.                     }
323.                     else
324.                     {
325.                         if(i.uvTex.y <= heightFactorLeft)
326.                         {
327.                             if(_ActiveDepthNear == 1.0)
328.                             {
329.                                 heightAlphaNear = (1.0f -((heightFactorLeft - i.uvTex.y) /heightFactorLeft));
330.                                 col.a = heightAlphaNear;
331.                             }else
332.                                 col.a = widthAlpha;
333.  
334.  
335.                         }else if(i.uvTex.y >= heightFactorRight)
336.                         {
337.                             if(_ActiveDepthFar == 1.0)
338.                             {
339.                                 heightAlphaFar =(1.0f -((i.uvTex.y - heightFactorRight) /heightFactorLeft));
340.                                 col.a = heightAlphaFar;
341.                             }else
342.                                 col.a = widthAlpha;
343.  
344.                            
345.                         }else
346.                             col.a = widthAlpha;
347.                    
348.                
349.                     }
350.  
351.                      //GENERAL TRANSPARENCY
352.                      col.a = col.a * _OverallAlpha;
353.  
354.                     return col;
355.                 }
356.             ENDCG
357.                }
358.           }
359.      
360.     }
361.     FallBack "Diffuse"
362. }

And here's the WaterReflection script:

Code (CSharp):

1. public class WaterReflection : MonoBehaviour
2. {
3.     //Reflection layers
4.     public LayerMask m_ReflectLayers = -1;
5.     public float m_ClipPlaneOffset = 0.07f;
6.  
7.     //Render texture size
8.     public int m_TextureSize = 256;
9.     public Camera m_reflectionCamera;
10.  
11.  
12.     public RenderTexture m_ReflectionTexture = null;
13.  
14.     // Cleanup all the objects we possibly have created
15.     void OnDisable()
16.     {
17.         if( m_ReflectionTexture ) {
18.             DestroyImmediate( m_ReflectionTexture );
19.             m_ReflectionTexture = null;
20.         }
21.  
22.         if(m_reflectionCamera)
23.         {
24.             DestroyImmediate (m_reflectionCamera.gameObject);
25.         }
26.     }
27.  
28.     private void UpdateCameraModes( Camera src, Camera dest )
29.     {
30.         if( dest == null )
31.             return;
32.         // set water camera to clear the same way as current camera
33.         dest.clearFlags = src.clearFlags;
34.         dest.backgroundColor = src.backgroundColor;    
35.         if( src.clearFlags == CameraClearFlags.Skybox )
36.         {
37.             Skybox sky = src.GetComponent(typeof(Skybox)) as Skybox;
38.             Skybox mysky = dest.GetComponent(typeof(Skybox)) as Skybox;
39.             if( !sky || !sky.material )
40.             {
41.                 mysky.enabled = false;
42.             }
43.             else
44.             {
45.                 mysky.enabled = true;
46.                 mysky.material = sky.material;
47.             }
48.         }
49.         // update other values to match current camera.
50.         // even if we are supplying custom camera&projection matrices,
51.         // some of values are used elsewhere (e.g. skybox uses far plane)
52.         dest.farClipPlane = src.farClipPlane;
53.         dest.nearClipPlane = src.nearClipPlane;
54.         dest.orthographic = src.orthographic;
55.         dest.fieldOfView = src.fieldOfView;
56.         dest.aspect = src.aspect;
57.         dest.orthographicSize = src.orthographicSize;
58.     }
59.  
60.     // On-demand create any objects we need for water
61.     private void CreateWaterObjects( Camera currentCamera)
62.     {
63.         //if(m_reflectionCamera != null)
64.             //DestroyImmediate(m_reflectionCamera.gameObject);
65.         //m_reflectionCamera = null;
66.         // Reflection render texture
67.         //if( !m_ReflectionTexture )
68.         //{
69.             if( m_ReflectionTexture )
70.                 RenderTexture.ReleaseTemporary(m_ReflectionTexture);
71.                 //DestroyImmediate( m_ReflectionTexture );
72.             m_ReflectionTexture = RenderTexture.GetTemporary (m_TextureSize, m_TextureSize, 16);//new RenderTexture( m_TextureSize, m_TextureSize, 16 );
73.             m_ReflectionTexture.name = "__WaterReflection" + GetInstanceID();
74.             m_ReflectionTexture.isPowerOfTwo = true;
75.             m_ReflectionTexture.hideFlags = HideFlags.DontSave;
76.         //}
77.    
78.         // Camera for reflection
79.         //m_ReflectionCameras.TryGetValue(currentCamera, out reflectionCamera);
80.         if (m_reflectionCamera == null) // catch both not-in-dictionary and in-dictionary-but-deleted-GO
81.         {
82.             GameObject go = new GameObject( "Water Refl Camera id" + GetInstanceID(), typeof(Camera));
83.             go.tag = "Untagged";
84.             m_reflectionCamera = go.camera;
85.  
86.             //m_reflectionCamera.enabled = false;
87.             m_reflectionCamera.transform.position = transform.position;
88.             m_reflectionCamera.transform.rotation = transform.rotation;
89.             //m_reflectionCamera.gameObject.AddComponent("FlareLayer");
90.             go.hideFlags = HideFlags.DontSave;
91.             //m_ReflectionCameras[currentCamera] = reflectionCamera;
92.         }
93.     }
94.  
95.     public void OnWillRenderObject()
96.     {
97.         Camera cam = Camera.main;
98.         if( !cam )
99.             return;
100.  
101.         if(!renderer)
102.             return;
103.  
104.         if(!renderer.sharedMaterial)
105.             return;
106.  
107.         //CREATE OBJECT NEEDED
108.         CreateWaterObjects(cam);
109.  
110.         if(!m_reflectionCamera)
111.             return;
112.  
113.         //Set reflection camera render as the current one!
114.         UpdateCameraModes( cam, m_reflectionCamera );
115.  
116.         //SET REFLECTION TEXTURE
117.  
118.         // find out the reflection plane: position and normal in world space
119.         Vector3 pos = transform.position;
120.         Vector3 normal = transform.up;
121.  
122.         // Reflect camera around reflection plane
123.         float d = -Vector3.Dot (normal, pos) - m_ClipPlaneOffset;
124.         Vector4 reflectionPlane = new Vector4 (normal.x, normal.y, normal.z, d);
125.    
126.         Matrix4x4 reflection = Matrix4x4.zero;
127.         CalculateReflectionMatrix (ref reflection, reflectionPlane);
128.         Vector3 oldpos = cam.transform.position;
129.         Vector3 newpos = reflection.MultiplyPoint( oldpos );
130.         m_reflectionCamera.worldToCameraMatrix = cam.worldToCameraMatrix * reflection;
131.    
132.         // Setup oblique projection matrix so that near plane is our reflection
133.         // plane. This way we clip everything below/above it for free.
134.         Vector4 clipPlane = CameraSpacePlane( m_reflectionCamera, pos, normal, 1.0f );
135.         Matrix4x4 projection = cam.projectionMatrix;
136.         CalculateObliqueMatrix (ref projection, clipPlane);
137.         m_reflectionCamera.projectionMatrix = projection;
138.    
139.         m_reflectionCamera.cullingMask = Camera.main.cullingMask;//~(1<<4) & Camera.main.cullingMask; // never render water layer
140.         m_reflectionCamera.targetTexture = m_ReflectionTexture;
141.         GL.SetRevertBackfacing (true);
142.         m_reflectionCamera.transform.position = newpos;
143.         Vector3 euler = cam.transform.eulerAngles;
144.         m_reflectionCamera.transform.eulerAngles = new Vector3(-euler.x, euler.y, euler.z);
145.         //m_reflectionCamera.Render();
146.         m_reflectionCamera.transform.position = oldpos;
147.         GL.SetRevertBackfacing (false);
148.         renderer.sharedMaterial.SetTexture( "_ReflectionTex", m_ReflectionTexture );
149.  
150.         GeneratePlane gp_script = GetComponent<GeneratePlane>();
151.  
152.         renderer.sharedMaterial.SetFloat( "alphaWide", gp_script.fXSections * gp_script.fWidth );
153.  
154.     }
155.  
156.  
157.  
158.     #region Reflection methods
159.  
160.     // Extended sign: returns -1, 0 or 1 based on sign of a
161.     private static float sgn(float a)
162.     {
163.         if (a > 0.0f) return 1.0f;
164.         if (a < 0.0f) return -1.0f;
165.         return 0.0f;
166.     }
167.  
168.     // Given position/normal of the plane, calculates plane in camera space.
169.     private Vector4 CameraSpacePlane (Camera cam, Vector3 pos, Vector3 normal, float sideSign)
170.     {
171.         Vector3 offsetPos = pos + normal * m_ClipPlaneOffset;
172.         Matrix4x4 m = cam.worldToCameraMatrix;
173.         Vector3 cpos = m.MultiplyPoint( offsetPos );
174.         Vector3 cnormal = m.MultiplyVector( normal ).normalized * sideSign;
175.         return new Vector4( cnormal.x, cnormal.y, cnormal.z, -Vector3.Dot(cpos,cnormal) );
176.     }
177.  
178.     // Adjusts the given projection matrix so that near plane is the given clipPlane
179.     // clipPlane is given in camera space. See article in Game Programming Gems 5 and
180.     // http://aras-p.info/texts/obliqueortho.html
181.     private static void CalculateObliqueMatrix (ref Matrix4x4 projection, Vector4 clipPlane)
182.     {
183.         Vector4 q = projection.inverse * new Vector4(
184.             sgn(clipPlane.x),
185.             sgn(clipPlane.y),
186.             1.0f,
187.             1.0f
188.             );
189.         Vector4 c = clipPlane * (2.0F / (Vector4.Dot (clipPlane, q)));
190.         // third row = clip plane - fourth row
191.         projection[2] = c.x - projection[3];
192.         projection[6] = c.y - projection[7];
193.         projection[10] = c.z - projection[11];
194.         projection[14] = c.w - projection[15];
195.     }
196.  
197.     // Calculates reflection matrix around the given plane
198.     private static void CalculateReflectionMatrix (ref Matrix4x4 reflectionMat, Vector4 plane)
199.     {
200.         reflectionMat.m00 = (1F - 2F*plane[0]*plane[0]);
201.         reflectionMat.m01 = (   - 2F*plane[0]*plane[1]);
202.         reflectionMat.m02 = (   - 2F*plane[0]*plane[2]);
203.         reflectionMat.m03 = (   - 2F*plane[3]*plane[0]);
204.    
205.         reflectionMat.m10 = (   - 2F*plane[1]*plane[0]);
206.         reflectionMat.m11 = (1F - 2F*plane[1]*plane[1]);
207.         reflectionMat.m12 = (   - 2F*plane[1]*plane[2]);
208.         reflectionMat.m13 = (   - 2F*plane[3]*plane[1]);
209.    
210.         reflectionMat.m20 = (   - 2F*plane[2]*plane[0]);
211.         reflectionMat.m21 = (   - 2F*plane[2]*plane[1]);
212.         reflectionMat.m22 = (1F - 2F*plane[2]*plane[2]);
213.         reflectionMat.m23 = (   - 2F*plane[3]*plane[2]);
214.    
215.         reflectionMat.m30 = 0F;
216.         reflectionMat.m31 = 0F;
217.         reflectionMat.m32 = 0F;
218.         reflectionMat.m33 = 1F;
219.     }
220.  
221.     #endregion
222. }

Our main artist designed the water, so we had to work from his idea and maybe we did some things wrong. He wanted the water to have not 1, but 2 textures that could blend to each other. Those textures must be displaced to match the waves' movement and give the player the feeling of looking at a moving sea (UV displacement). Then we add a Noise texture that acts as a distortion, and finally there's the reflection effect. And also he wanted the edges of the water plane to fade out instead of just ending in an ugly digital cut.

For the reflection, we use special secondary cameras to take the reflected image texture from their view, then apply it to the water.

Yes, it is quite heavy, but blame the lead artist on this one!

 

Oh sorry the whole scene of your game is transparency, use opaque for water is not a good idea.