Realtime cloud as 3D model with SH

I just finished a little experiment with realtime cloud rendering, which I wanted to share. Most approaches are either very 2D or very expensive due to raymarching, so I figured I'd try something else. And I'm quite happy with the result. It does look a bit artistic and not 100% photorealistic, but it is fairly cheap and fully 3D.

The approach is to make an actual 3D cloud model and then bake the lighting into spherical harmonics. (First 9 components similar to Unity does.) The transparency of the cloud is determined based on the (distance through the) actual 3D model. The lighting is quickly calculated using 3 textures with spherical harmonics mapped on the cloud model.

And this is the result (with some other sky layers in the background):

Note: Only the front cloud is done this way. The background cloudiness is just a typical 2D technique. Render time for the whole scene is 0.4 ms.

 

This is very very interesting. Any way I could test this in VR?

 

I have added some decals since then to add some detail to the borders. Still working on a complete scene where the clouds are also automatically placed.


Above the version without decals and below the version with.

Earlier tests with VR confirm that at least without the decals the solution is VR proof.

 

Hey jvo3dc!
I ran into your work and it's very similar to a look I'm trying to achieve for a cloud shader!

I'm just starting on writing my own shaders, and I'm trying to work through the process you described. Do you have a bit of time to help me understand some of this?

My two main questions are how do you bake the lighting, and how are you calculating thickness of the mesh for transparency?

Spherical harmonics:
So you're starting with a model of a cloud, add detail via sculpting / noise and then bake out the spherical harmonics to three textures (I'm assuming one order per channel)? Can you talk a little bit more about how you're doing that bake? I read a bit into spherical harmonics, and know they're used in light probes, but can't quite figure out how to calculate and bake them into a texture and then how to use that info onto the model. Is this done in Unity, or in your 3d package? I use Blender currently.

Transparency:
I'm new to the hand coded shaders. How are you getting the thickness of the object? I played around with a few things and couldn't figure it out. One method recommended turning on additive blending, rendering the front faces, then negating z and rendering backfaces, but I couldn't get that to duplicate the thickness effect you show here.


The best I've been able to get so far is this (ignore the blue tint... that's there for something else), using a fresnel approach for edge transparency, and a noise-based vertex offset with a bit of tessellation, but yours is MUCH nicer, and I'm betting faster.


Thanks!

 

Hi evanvlane,

The baking is done in 3DS Max with VRay. The spherical harmonics are up to level 2, greyscale, so 1+3+5 = 9 in total, which fits nicely into three RGB textures.

The thickness is done in a similar way as you describe. To a separate RenderTexture using both the front and back faces.

There is a lot that needs to come together here, so it might no be the best testcase when you're just starting with shaders.

 

Thanks for the response!
I've definitely started to notice that, but I am learning a lot by unpacking the errors I'm running into.

I may have to consider a different lighting model, anyway, due to the complexity and dynamic translation limitations of SH, but the thickness approach will be really helpful!

 

Hi, did you keep working on this? We need a solution to render Our 3d cloud models with this kind of appearance. Thanks!