New relief mapping techniques (fast)

Hi there,

This is a demo using new relief technique. It's based on assumption of extrusion strictly in vertical direction. When we also extrude along uv axis in tangent space shader can do calculations quite fast (2-3 simple steps to resolve ray hit point) with self-shadowing, too.



It's something I'd like to put into asset store, so every feedback (how it works on different platforms) would be appreciated.

Demo web player:
http://www.stobierski.pl/unity/spaceShip_WebPlayer/WebPlayer.html

and standalones for PC and Mac (to better test performance on different resolutions and quality settings):

http://www.stobierski.pl/unity/spaceShip_PC.zip
http://www.stobierski.pl/unity/spaceShip_MAC.zip

I made this demo as a proof of concept. Stay tuned as I'll put more demos soon (brickwalls, pavements, etc.). I'm also working on procedural relief shaders (no heightmap use at all to get paralax occlusion effect), so I intend to put it all in one package on the store.

 

Very nice...the web player works on my Mac always at 60fps with all options, although the DOF effect goes completely mental if shadows are enabled. Disabling shadows makes it work correctly.

--Eric

 

Wonderful. Your shader extrusion is a work of art.

 

very cool!

 

very nice!

 

looking superb

 

I love it!
Show us more, oh great one.

 

Any chance this would work on IOS, iPad 2/iPhone 4S + ?

 

Eric, have you tried standalone versions ? Please, could you be a bit more specific with DOF+shadows issue (screenshot or anything) ? I used HDR camera rendering with this DoF (Bloom and Lens flares post FX), maybe that's the problem. Unless you turn on DoF, do the rest of postFXes work fine with shadows on your Mac ? The only thing I can do testing OpenGL is forcing this mode on my PC in editor.

 

I'm not much mobile oriented Unity developer, I don't have any iOS hardware to check it. As far as I know the shader compiles for GLES, even for ARB / OpenGL with some options turned off. I don't exclude this target platform in my surface shader code, so it could work I guess. The problematic part could be using tex2Dgrad / tex2Dlod functions inside for better apperance on extruded edges (uv coords becomes uncontinuous when offseting them), but this can be turned off by commenting one line in shader code. This case shader won't look that fine on inner edges, but the same time works much faster (ddx/ddy/tex2Dgrad are pretty heavy against regular tex2D).

I can't say what's the GPU power of iPad2 comparing to desktop machines. What I can say is that the simplest version of my shader with most options turned off needs about 30%-40% more power comparing to ParallaxBumpedSpecular shader on my PC (nVidia GT240). With the every option turned on it is up to 2-2,5 slower than mentioned reference shader. There are plenty of options adjustable, i.e. normal smoothing, ambient occlusion (for procedural relief only), detail maps, displacing UVs by texture, etc. I've decided to name my shader "fast", because classic relief mapping is considerably slower in most cases. It's, of course not that fast like, let's say, bumped specular. No way. We need to compute exact ray hit position in tangent space, whether analytical or using texture look-ups.

As said most of my shaders will be faster than other relief mapping shaders. For example relaxed cone mapping takes a few initial steps to cross the surface (texture lookups + calculations needed), next it uses binary/secant search to finetune exact ray hit position which is at least another couple of texture operations on heightmap. Worse, if we like self shadowing, we have to do the same process back again, tracing ray from ray hit point to light source. It's heavy, so I'm not surprised that only a very few AAA games (in UDK, Cry Engine, etc.) uses relief mapping extensively. It just costs much. When you have GPU like GTX580 it's fine, but when you got old good nVidia 7600 it becomes impractical. That's why GPU developers went into tessellation in DX11 for example.

Reassuming in comparation to ParallaxBumpedSpecular. If you can consider spending twice a GPU power on shaders and it's fine for your target audience platforms, my shaders are definitely something for you .

One more thing. To be precise, don't expect that my shaders are replacement for generic relief algorithms. My extrusion works much faster but it's special case shader. It can handle a subset of situations fast enough to compute more bells-and-whistles (self shadowing), but it's not general solution like relief mapping.

 

On my Mac it works fine with all post FX enabled. And it runs at smooth 60 fps.

Good work tomaszek!

 

OK,

Below you've got demo with more materials. Pavements, bricks relief are realised by texture info (which defines where are holes, aligned to uv tangent space axis). Balls at the center are extruded proceduraly. User can adjust every aspect of its look. Tiling (not necesarilly the same in u nad v axis), dimensions, extrude height, ceil/floor levels, ambient occlusion, self-shadow. Another option for all of my shaders is that they're _properly shaded_ (although from one light only) even when lightmapped by single / dual lightmap - shader doesn't need directional lightmapping to get normal dependent shading (like bump mapping for example - they don't work when lightmapped unless unsing directional lightmapping).



http://www.stobierski.pl/unity/pavements_WebPlayer/WebPlayer.html

 

Świetna robota

Looking great! I love it. What will be the price?

 

Price This is the most difficult job always. I intend to make it just "relief for masses". I count on people to buy it on large scale (should I ?), so I'll probably set the price in low-mid range ($20-$35). Package will consist of several procedural shaders, and a few versions of "extrude map" shaders (extrusion from bitmap like pavements/brick boxes here and space station in previous example). User will have tool to make distance maps from height bitmaps (extrude map shaders needs them to work). I have to sort things up first and check it good before I put it on asset store. I think I'll also put this kind of shader (exact extrusion - walls dont have to be uv-axis aligned) like in this thread:

http://forum.unity3d.com/threads/147498-Extrude-Shader

I think I'll put some teaser shaders today - extrude map shader and one procedural shader (compiled for D3D9, ARB) free of charge for people to play around with this stuff. Thus potential users will have better idea how does it work.

 

It's very cool, and fast too. It seems like it would be really easy to use since it's based on a texture. Makes `modelling` seem much simpler to do when you just need to change the image to paint the surface shape, instead of going into a 3d modelling program. I think this will offer easy editing benefits to a lot of people who might not be adept at 3d modelling but who can mess with textures. I think you could go in the $30-$50 price range if you include a number of useful/useable example textures. This is what I like to see in a tool - easy to use, takes a huge burden of pain off of development time and complexity, opens up possibility to a larger audience that they didn't have before, and does something ingenious at the same time.

 

The standalone version does work fine.

Actually it's DOF+AA+glow+shadows. If one of those is disabled, then it works. Here's a series of screenshots taken over a second or two:



--Eric

 

Thanks Eric,

I'm afraid the origin of issue is somewhere outside my shader. It's rather matter of mysterious postFX effect interaction in Unity web player (bug ?). I confess I pushed fragment shader a bit (to the limits), but I definitely didn't hacked surface shader output, nor shadow caster/collector. I use clip() (discard) for silhouettes cut on uv borders, but not in this case. I output alpha, combining it from color and main texture (as alpha is used by some postFXes). So can't help, but good to hear it works in standalone version.

 

I'm running Windows 7 home premium - Intel Core 2 duo - 3gb ram - Nvidia GeForce GTX 560 Ti

Both webplayers ran perfectly with 60 fps with everything turned all the way up.
Didn't have the same issues as eric.

Standalone:

640 fps at 640x400 Fantastic fullscreen
330 fps at 1440 x 900 Fantastic fullscreen
380 fps at 1440 x 900 Fastest fullscreen

Again, no issues at all. All my test had everything turned up and moving the sliders around saw no different in performance. Looked amazing fullscreen with max resolution.

 

Looks awesome. I would love to see a video or something of this being setup. Is it as simple as throw the shader on a material and play with the settings?

 

So,

Here is little teaser as a unity package. You can play a bit with the shaders (compiled versions, w/o full options)

http://www.stobierski.pl/unity/ReliefDemo.unitypackage

You've got 2 examples:
- extrude map materials (demo version - no texture color/bump/detail mapping nor uv displacement) - bricks / hello world distance maps
- procedural relief shader with its all options available except for texturing

Check it out and tell me what do you think about it (performance in different configurations, etc). Especially you can texture different objects (different uvs) with procedural shader and play with its settings. I can imagine we're able to use it for many interesting applications, even with no texturing.

 

That seems pretty fair. So far I think you've done a really great job, I've been messing around with the web player and stand alone clients. It all looks quite nice.

 

As I'm coming to release of the package, I'd like little help from other users. Could anybody tell me if it works (esp. on Macs) on GPUs with SM3.0 ?

It's not that cheap shader as previous examples, but it will be fine addition to the package. On SM3.0 cards (true dynamic conditional branching needed for good performance) it should work quite decent. This shader will be issued in various configurations and this is 2 sided version with soft shadows turned on so that adjanced geometry casts shadows each other. On my GT240 it's ca. 300fps in 1280x1024 with shadows.

http://www.stobierski.pl/unity/ExtrudeMap_WebPlayer/WebPlayer.html

I'm also curious if anybody checked demo version of some of previous shaders here:

http://www.stobierski.pl/unity/ReliefDemo.unitypackage
(final versions will have a few more fetures like uv space rotation and more)

 

hi tomaszek,

i had a look at almost all your demos: they work perfectly on my mac book pro (nvidia geforce 9400m) and pretty fast.
so i am wondering whether it would be possible to built a terrain shader using your algorithm.
fading out the extrusion over distance might speed up rendering so it might be fast enough even for large object covering a lot of screenspace like terrains.

i am asking as there have been 2 terrain shaders discussed lately supporting extrusion which either do not work on mac or might not be as fast as yours.

lars

 

Works on my Radeon 5870.

--Eric

 

Beautiful!

 

Hi, as you're experienced shader programmer it should not be difficult to adopt algorithm for terrain shaders you're specialized with. My package will come in form of surface shaders which everybody can easily tweak (a lot of options as defines to adjust if you'd like to disable some unused features to gain performance). I can't say if my cheaper shaders (like pavements, brickwalls) will be good choice for terrains unless you'd like one of the splat layers to be "pavement". This kind of shaders doesn't work for irregular extruded shapes like "bumpy rocks". For this kind of stuff you'd need "more classic" approach like steep parallax or relaxed cone (which is probably the fastest way to raymarch thru empty space, but needs additional conemap texture).

To make my package "all-in-one" relief solution I plan to put steep parallax too with every optimization I can use. This way it's possible that extrusion will work on large terrains well enough on PC/Mac. You'll be able to adjust settings the way you need (for example turn off self shadowing completely, adjust raymarch step length if you can accept small artifacts on the edges for sake of performance).

 

That's it, I'm trying to find out the fastes algorithm for terrain relief mapping, but actually it's heavy on the gpu, because it uses high res textures for albedo, normalmap and relief. However I introduced a lod setting, but made my shader run 2x-3x times faster and it actually runs really well

So your shader doesn't support for example bumpy rocks on a flat plane? I can only imagine that u round the value of the relief map and then calculate the extrusion from a value of 0 or 1?

 

Impressive and also so fast!

 

New extrusion algorithm (like pavement, spaceship) doesn't rely on height map at all. It uses distance map to travel fast thru empty space. As extrusion is realised on "rectangles aligned to uv-axis" I can compute ray hitpoint in 2 steps (for pavement) no matter the viewing angle - it's always 99.99% precise location of hit point in tangent space. What I do for pavements is disturbing a bit uv entry point so that extruded sews are not that straight (it's fake but works very fine unless we'd make extreme close-up and view extruded part in different directions). For terrains I think that well optimized (distance fade out) steep parallax mapping will be fine. I have to work on them a bit to check the performance. Meantime I use the same idea for extrusion strictly downward (like in cylindrycal cage example) and it seems to work quite fast as we break loop early depending on viewing angle.

 

Here we are - after digging intensively (Lars - thanks for inspiring me !) we can use these cool POM stuff over terrain. I've also added some extra features here. The only limitation is that we have to use uniform (I mean the same) splat tiling for every 4 splat group. In reward we've got reasonably working terrain that looks so cool...



http://www.stobierski.pl/unity/Terrain_WebPlayer/WebPlayer.html

I'll include it in the relief package. So it will consists of:

1. Procedural relief mapping (fast !)
2. Distance map extrusion (fast - refer to space ship at page 1 of the thread)
3. Vertical extrusion (cage example above)
4. Parallax occlusion mapping. Adopted steep algorithm, optimized for distance with self-shadowing of course and REAL correct silhouettes on arbitrary mesh). We've got treebark mapped with this shader. One can complain that silhouette of the tree bark isn't convincing, but it's only issue caused by low-poly tree bark model. When you'll use better cylinder (higher poly for tree bark) you'll have correct silhouette.
5. Terrain shaders like in example above
6. Tools for managing stuff (computing distance maps, curvature of objects for silhouettes, etc.)
7. Shaders are able to handle real volumes so that we can look at extruded items from the side (they won't disappear). It's not "true impostor" solution (1 height map only), but will be very suitable (and can be fast if we build tight mesh of volume) for fake hi-poly 3D elements. You can think - this "lion head" of steep parallax mapping example - google it on the net - will be at last real 3D entity and could be viewed from sides.

Comments, thoughts ?

 

Looks cool!

 

wow, i am more or less speechless. this looks really awesome.
and is pretty fast – even on my lowend machine.

so i would say: this is definitely one of the most advanced terrain shaders – if not THE most advanced – i have ever seen in unity.
congratulations!

unfortunately i get some rendering artifacts on my computer (mac book pro, os x 10.8.2, nvidia geforce 9400m) – as shown in the picture below – but i am pretty sure that you can get rid of those.

so just let me add some questions: how many detail textures does the shader support? and do you see any possibility to support color maps on the terrain?

again: i am totally impressed and would like to see this shader being the standard one for AAA productions made with unity.
so go for the support of multiple terrains (at least in unity 4) and supply a proper artist work flow.
then this one should really push the boundaries of everything we have seen so far.

very well done tomaszek.


lars

 

I get the same artifacts on Windows with GForce 7600 GT, but runs very smooth at 60 fps with all on except for the self shadowing!
To bad the self shadowing breaks the fps totally. Will test when i get home as well.

 

Here it runs perfectly smooth at 60+ fps, no settings breaks anything. Impressive!
Radeon 5770.

 

I saw them yesterday when running in GL mode on windows so I guess I'll be able to know what's going on and I hope it can be resolved.

You may use any splat count you need (to the extent of what Unity can handle by default). I worked on how to improve performance for more than one pass (additional splats 4-8, 9-12, and so on). As we have to run it in different passes the problem is that GPU power needed should be multiplied by passes count. But there are good news - for areas where only one splat set is used we don't have to pay the price of rendering expensive fragment program. The general rule is - DON'T mix splat groups together on larger areas. Of couse we need to blend them in some places (and there they have to be rendered both) - in my example it's border where sand (belonging to splat group 1-4) touches "the tree isle" (dry cracked ground - splats 5-8). In this border area we have to compute full processed passes, but in other areas (for example the isle with 4 splats - dry ground, rocks, yellow stones, snow) we compute only what we need and the inactive splats are early skipped in fragment program.

Color maps - yes, we can just blend it at start. Now I add detail normal map at start for far distance - just perlin noise as normal map blended with normal output from surface shader.

Thank you, this is kind and uplifting after weeks of very intensive work. Your terrain shaders are very inspiring. Wouldn't you mind if I add some features to make my shader as useful as yours (color map, fancy color blending on splats) ? I believe our products won't be that competitive as yours is blazing fast, while mine isn't (but looks cool like these fancy POM shaders in AAA titles).

 

I don't experience artifacts on windows (XP, nVidia GT240). Which version do you use (win 7 ?). Yes, self shadowing can kick GPU's ass but we don't have to be that pesimistic. Quality settings for color/shadows rendering are all adjustable. You can turn off smooth shadows (they can be expensive), decrease its quality and push your light to zenith so that it won't produce that "long" shadow silhouettes . What I did here in my example were kind of "stress test" to show how it behaves in difficult conditions (shadows, water reflections, multiple splats, etc.). It seems that it can work reasonable. That's what I was trying to prove.

 

I use windows xp here at work, but i will try when i get home as well. I had artifacts on this computer here before were most ppl didn't : p

 

Could you test it again ? Should be no artifacts (was float point precision math fail in some tiny cases )

ATB, Tom

 

thanks.
i do not mind at all if you add features like color map and color blending at all, but i would ask you to do so!

as far as performance is concerned: i am wondering if it would make sense to mix regular bump mapped detail textures (let’s say: the first 4 detail maps – as i don’t have to have relief mapping on grass or sand with just little stones) and may be 2 relief mapped details on top of them (as i really love your rocks…)?
and is it possible to do this in one shader pass?

lars

 

This is probably one of the more exciting threads I've seen in such a long time. The terrain engine and its capabilities (or lack thereof) have been of interest to me for such a long time.

I'm especially interested to see how many textures overall can be supported.
One question:
What is the difference in difficult between providing this functionality for the terrain engine, and it also working with procedural meshes?

 

Works perfectly at my home computer with all settings on, and no bugs here : )

 

Regular bumpmaps mixed with POM shading - it's already done in splats 5-8 (we can turn on/off full parallax occlusion for add pass). Do it using "flat" splats 5-8 with simple materials, sweeten them using splats 1-4 with "bumpy rocks and such cool stuff".

First pass (splats 1-4) is always POM and I don't see any point to optionaly turn it off (this case just use your shaders - they're everything we need without POM).

Lets talk about limitations:

1. the same tiling for splats we need to be POM - without this limitation we'll kill almost every GPU (think 4 independent POM calculations per pass, performance horror at least in all areas where we want splats to be blended). When we've got uniform tiling we can get POM calculations in time comparable to regular POM shaders (+ tex fetches for splat colors/normals, etc.)

2. hard borders (heightmap blended) only inside one splat group, so we can't mix this way borders of splat 1 and 5 (different groups, different pass)

3. making more in one pass - possible, but with limitation no. 1. We just quickly ran out of texture interpolators. We have also think about how to get control textures/splats for this (I guess it's possible to dig them from some undocumented terrain variables)

4. 8 splats at this moment (4 POM + 4 POM, or simplier 4 POM + 4 bumpmapped). More - I don't know how to difference additional passes (in unity default it's just "AddPass" and it works for every splat >=5), but we need to provide this additional passes with more global variables (bump/height textures, params).

5. only one terrain this moment (limitation of providing params to shaders from above point) as more terrains would have interfering global params/textures.

 

Doing the trick on arbitrary (procedural mesh), I mean blending 4 POM splats "at once" is quite possible. One could just adopt my terrain shader and provide it with textures/params needed. This moment 8 textures are supported (look at my previous post for explanation). I don't know the way to provide the shader with data needed and how to dig it from unity. Maybe somebody already resolved it (Lars ?). The problem is how to get Unity's control and detail texture references for any splat we need. If I can get reference to texture, let's say - splatN detail and control texture for any splat group we could talk about removing our 8 textures constraint. As far as I know Lars in his ats product also use 6+2 detail textures, but in one pass, so maybe he knows how to manage it.

 

i thought about some "if" conditions within the shader or things like this…
but i can imagine that this would not make the shader faster – but even slower.
what is about moving the POM to the second pass?

i think that is absolutely ok: in fact that is the way all terrain textures should be handled anyway.

we have 13 texture samplers in shader model 3 (in case you want support light mapping and light cookies i think – and in case it makes any sense…)
in the color map Ultra shader i combine as many textures as possible into one (like normal maps) in order to safe texture samplers – and i even think about using single blank color channels (like the alpha in the color map) to get most out of it (for holes in the terrain, some kind snow distribution map or destruction map…)
doing so gives me the chance to store 6 detail maps including specularity (rgba) + 6 detail normal maps(3 textures rgba) + 2 splat maps (rgba) + 1 color map (rgb, one channel left, splat channel?) + 1 terrain normal map (rgb, which could be compressed to ra, so 2 channels are left, splat map?).
in case this is not enough you could completely drop the the 2nd splatmap and add some textures procedurally (like rock according to the slope value).
and i roughly take the blue channel of the diffuse texture as height map saving one channel per detail texture…
so may be it is possible to render 6 detail textures within one pass: 4 bump mapped and to height blended POM ones? all color mapped? that would be just great.

4 POM texture on the terrain should be really enough! that is "rocks", "little rocks or cobblestones", "twigs and dry branches" and "rippled sand"!

you are right about unity 3.x. unity 4 will give you the possibility to assign materials to the terrain so no more need to use global textures or global vars.

lars

 

How do you control additional splat covering ? Can you access 2nd control texture from Unity terrain ? When using such demanding shaders like POM on terrains we have to consider sacrificing vertex shader load, I mean - second pass for terrain (even if it's hi-poly) isn't bottleneck here, but rather POM calculations. For bumpmapped terrain like yours it makes sense to put everything in one pass, this way shader gets really fast for low-end machines while POM shading is so demanding that we don't have to worry that much about poly count rendered. If user don't have GPU to push easily 200000+ polys for terrain, then he can't afford POM neither. Anyway idea of pushing as many things as possible in one pass is great itself so I'll work a bit further on the issue.

Sure, but you have to take into account that all these POM elements can pop-up above each other only between POM elements. In different words, you just can't put simple flat grass (not POM) and make POMs pop-up over it. Thats why my first layer is grass. In fact it is POM, but height map for this is just flat white so that we don't have to trace ray to find hitpoint/texture offset - such splats renders as fast as regular bumpmapped splats.

 

Your relief mapped shadr works great on my machine, good work, im not sure entirely what i'd use it for as it stands but i'm sure i could think of something

 

yes and yes: simply send a 2nd splat map as global texture.
you can take it from practically anywhere: an imported splatmap or from any other terrain object…

fine.

if "flat" POM textures nearly render as fast as simply bumped mapped ones then everything is perfect already! i just thought about mixing cheap and expensive to render textures within one pass and as far as i understand this is already possible.

lars

 

amazing, I would love to see the actual code of this shader...

not that I expect you to show it to me, just saying I would love to see the code...

If we purchase it, can we see the code? Or will you only give out so compiled version?

 

Compiling shader is an idea but would break OpenGL usage - I use GLSL compiling for Macs (shader is quite complex and uses tex2Dlod like stuff). This situation it does not make sense not giving users source code for shaders. I'm not that jealous, but hope to have not that naive faith that my work would be appreciated not only in terms of warm community feedback but also further AssetStore activity...

So - as far as you don't go, buy my package, change syntax and sell it as different AssetStore product, I've got no problem with it. I also use many people ideas myself combining it new creative way (I believe).

 

Hi, when will the package be available on the asset store? I'm very eager to see it.