Marmoset Skyshop - Image-Based Lighting Tools [RELEASED]

Every time I see a screenshot or webplayer with Skyshop in it, I'm amazed. It's absolutely breath-taking.

Unfortunately, most of my stuff is indoors. I haven't kept up with this thread lately... Has indoor stuff been implemented? Or is it still on the radar? Is there any way to get results something like the outdoor ones, even if they aren't quite as sparkling?

 

What a massive thread. Took me half a day to read through it :-D

MonkeyScience, a couple of questions:

- Did I understand it correctly that the Cube map generator does not yet create true 32 Bit HDR cubemaps with exposure data? But this is planned for v1.05?

- What exactely is preprocessed/convoluted when creating the diff and spec cubemaps? Any publications about this technique? What is different as compared to a lookup into a cubemap by one of Unity's reflective materials? I would like dive a bit under the hood to get a deeper understanding about what I am working with.

- What does the "Lightmapped Specular" multiplier exactely do? Since there is no specular component in a lightmap I assume this globally controls the intensity of the specularity of lightmapped game objects which use Marmoset shaders? So that you can separately control the specularity of dynamic and static objects that use marmoset shaders?

- Does the dedicated AO channel multiply with the diffuse map only or also with the specular map? Or anything else?


Thanks a lot!
Sean

 

About Augmented reality with vuforia 2.6 and last updated skyshop plus your great matte shadows shading we realised we had an app crash after 4-5 minutes of user experience, if we remove it, no problem at all, monkeyscience would you have any idea why it could happen, we see nothing special in the profiler...
THX a lot!

 

Oh man! I've been pulling my hair out for hours trying to work out what I was doing wrong lol. I open up terraindemo and it has exactly the same issues, decided to check on here, sure enough .

Looks like I will be working on something else until this gets fixed.

 

I forgot to ask something.

- Assume you have a normal IBL setup with diffuse and specular sky cubemaps and various objects with Skyshop shaders. If I now add light probes, will they see the diffuse dynamic IBL that hits the objects with Skyshop shaders and bake that light accordingly? Or are they blind when it comes to this non-Unity light?
I'm not talking about the sky light, which by the way is not set to "Use for GI".

 

I'm glad the problems with 4.3 and Skyshop have been picked up. I thought it wasn't working properly...

 

In regards to the breaking Sky Prefabs:

We're seeing the same thing, and its causing us some grief, as we want to swap amongst multiple sky's in a real-time fashion, but don't want to have them ALL in the scene at load time (this is on Mobile).

The prefab breaks completely for us here, with a missing mono script being the only thing left on the sky prefab.

Did this work prior to this and its just a new bug?

Nathaniel Hunter
SuperTales Inc.

 

Ok so update on the terrain shaders:
I'm not finding anything wrong with the actual terrain shaders, at least not on PC/Web. Is anyone else?

The grass shaders however have a color correction bug. Unity 4.3 introduced proper gamma correction of grass detail mesh textures, so I'm removing this hack from MarmosetGrass.cginc:

Code (csharp):

1.  
2. #ifdef MARMO_LINEAR
3.     //HACK: The terrain system is buggy and does not sRGB sample grass textures. We're on it though!
4.     diff.rgb = toLinearApprox3(diff.rgb);
5. #endif
6.  

Doing so will break the grass shader for Unity 4.2 and earlier :-(, but there is not much I can do to help it.

 

Yeah I'm only seeing issues with grass.

I will remove the hack soon and see if it solves the issues I was having.

edit: I'll report back in an hour when Unity has finished recompiling all the shaders lol

 

OK, so I removed the hack from MarmosetGrass.cginc as a test.

Here are the results using unchanged terrainDemo example scene.

GAMMA Colour Space with Linear UNCHECKED in Sky object



LINEAR Colour Space with Linear CHECKED in Sky object



Gamma space seems to work fine but not linear. Note I changed the settings around but couldn't get the grass anywhere close to as good as what gamma space looked. If you want me to try anything else let me know. The grass just always seems way too dark compared to the rest of the objects, and terrain, in linear space.

Thanks.

Edit: I just realised that the hack was only relevant to Linear space so Gamma was most likely working all along, but I am using Linear for my project.

 

I need different tiling for the normal&specular maps and the diffuse map

 

I just saw this too but with gamma space looking all dark and wrong. After switching the project back and forth between linear and gamma a couple times, it went away . I'm going to dig deeper but I think there is a strange project settings and terrain refresh bug at work here.

 

You're asset looks fantastic! After watching the videos I noticed that in order to use imaged based lighting you'll have to use your custom shaders. With that said, are you planning on releasing a skin shader as well as any tessellation shaders to use with your system?

 

I can't get it to refresh no matter what I try.

So gamma is working fine for me, when I switch from gamma to linear in player settings the grass lighting slightly changes but looks correct. When I click the sky object and the editor code refreshes, the grass becomes dark. I have tried both the example terraindemo and my own project with exactly the same results. I am just waiting for shaders to compile in a fresh project with no assets apart from SkyShop just to eliminate any outside factors (I will update this post when complete).

It seems very weird that you are getting the opposite issue. I am on windows too FYI, in case this is a factor.

Edit: Fresh project has exactly the same issue.

 

Finally found a method to solve the zfighting and the hair ... unfortunately had to use true dbl sided geometry to get it work, but it works... is there a way to add shadows to the transparency shaders?

 

Neptune is skin Marmoset shader too or any other solution. Just wondering.

 

this is the regular shader...

 

Which shader are you using for skin to work with Skyshop?

 

Just the regular Bumped Diffuse ...

 

I was hoping he/they would integrate a skin and tessellation shader, but I haven't heard back from the developer yet.

I know the developer of the 'Pre-Integrated Skin Shader' has developed a skin shader that works with Skyshop, but he's waiting for approval before he releases it. Just an FYI since you're probably looking for something a bit more robust, and that asset is fantastic.

 

I already have that Pre Integrated Skin Shader... I used it on my Blade Character... which you could see in my thread... (need to update it)

 

You have some skills! Nice work!

 

But now I have to render depths maps for the skin shaders and that will take a LONG time to do (since the textures are 8k). Also, they will probably end up on Unity 4 anyway (I downloaded 4.3 recently in prep to upgrade)

 

We're experiencing the same prefab issues. Bit of a major problem! Subscribing so I can keep up to date with any updates on it.

 

I am working on a skin shader yes, I hope to roll out at least a beta version of it for people to play with soon. I haven't looked at tessellation yet.

 

I have been digging into the prefab issue and it's a tough one. There seems to be some conflict in how Unity handles namespaced classes, and I've filed a bug report on it. That or I'm just failing at Serialize :-/.

 

I haven't got a reply from you yet, MonkeyScience...

 

MonkeyScience.

For a pure performance reason in mobile platform, I would like to test a scene with non hdr cube maps.
However, when I looked at the shader code and sky script, there isn't any optimization for this case.

What I mean is that the shader still calls fromRGBM with is another multiply per pixel. But I assume that for non hdr cubemaps, all the data is within c.rgb and it doesn't need to be multiplied by the c.a ?

I don't mind if there are no optimizations for encoding the RGBM but for mobile platform with no HDR cubemaps, I think every instructions counts, so I want to know if it is ok if I def another set say "HDR" and then just return c.rgb from "fromRGBM" function.

I am also bit unease about all half and float precision variables when it comes to mobile platform so I wonder if using fixed precision where it can work in this special case can help the mobile performance.

(special case : mobile = gamma + no hdr cubemaps)

thanks.

 

Oops, sorry SeanM3D!

Correct, hopefully HDR render-to-cubemap will be fully functional come 1.05.

Diffuse and specular convolution is a special, physically accurate way of blurring reflection cubemaps. The lookup is the same (normal vector for diffuse, reflection vector for specular) but the reflecting image itself is blurred to match phong lighting and the gloss term. Here is a good GPU Gems article that describes the algorithms involved: http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter10.html

They are multipliers on the light contributed to a surface by the sky. The multipliers only affect surfaces with baked lightmaps and affect both specular and diffuse IBL. Even though a surface has baked diffuse light in a lightmap, image-based ambient light can still be added on top. This added light reflects both diffusely and specularly.

Use these multipliers to tone down the IBL contribution on indoor scenes and in cases where the desired ambient light is already baked into the lightmap. There are a couple of detailed posts on the subject in this thread as well, worth a read.

The AO map occludes both specular and diffuse IBL. The red channel occludes diffuse light, the green channel the specular light. A standard, grayscale AO map will occlude both equally but some people like to have a more contrasted black-or-white occlusion map for reflection. This way you can easily do both in photoshop.

 

Correct, no HDR means you can use .rgb directly.

It certainly could, I haven't tried squeezing anything to the point of fixed precision. Our shaders err on the side of quality, Unity ships with plenty of high-performing, low fidelity stuff already .

 

@Monkeyscience and all... happy thanksgiving...

And I am looking forward to seeing the new tools... I've been trying to get an answer in regards to getting transparency mapped objects to cast shadow... is there a way to add that functionality to it...? I was able to solve the flickering on my hair...

 

Hi MonkeyScience.

I am wondering if there is any progress made into using SH as diffuse IBL lighting?
This would get rid of per pixel calculation of worldnormal + texture look up + no cubemap (nore mem) so I would definitely want this..

I know you have mentioned this before so I am just checking.

I think this will help greatly on mobile as well.

cheers

 

Thanks for the explanation and the link. A couple more questions:

- How much of the convolution is GPU and how much is CPU?

- In general, is Skyhop more GPU or CPU?

- Do you use Spherical harmonics as described in the article that you linked?

I try to understand whats happenig under the hood a little better, because I need to explain some of the technical details in a university project. It would be great if you could give me some insights into Skyshop
I understand that there is a preprocessing step in which you probably ray trace the environment map for every normal direction N (in case of diffuse map) in order to integrate the irradiance for every N.

Assume, the irradiance for a particular normal direction N results in the RGB value 233, 45, 67. Where and how is this value stored? Is it directly stored in a pixel/texel of the environment map so that each pixel/texel of the blurred environment map directly represents an irradiance value? Or is it stored in a lookup table? Which format does this lookup table have?

I guess what I don't really understand is the purpose of the blurred diffuse map. If a lookup table is created that stores an irradiance value for every normal direction N, then what do we need the blurred environment map (as a visual entity) for? Isn't a textual lookup table all that is needed?

Thanks a lot, MonkeyScience!

 

Hi Marmoset,

I contact you by email with our new iOS app using Skyshop framework shader.

I hope you have time to take a look.

Thanks

 

It seems that with the latest Unity the shaders are all pink. The errors showing in the console are like "Material doesn't have a property '_Color', as well as '_SpecColor', '_SpecInt', '_Shininess', '_Fresnel', etc.

 

Thanks! I've been on a turkey-themed hiatus for the past week. But now that I am back and full of gravy, I have excellent news! Skyshop 105 is ready and about to be launched.

A lot of features and fixes coming your way. I will post a full change-log soon, but meanwhile here are some highlights:

Skyshop 1.05

Beta Shaders - I've decided to try something new and have added a "Beta" category to the Marmoset shaders. It will contain shaders and tech that is aaalmost production-ready. This way I can get you guys new features to play with sooner, in an isolated environment, and without having to worry about many platforms or huge future changes.

**Developers working on existing, large-scale projects would be wise to steer clear of beta features as they may change in the next few releases. But if you're feeling adventurous and want to see what's coming down the line from Marmoset, try them out.

Box Projected Cubemaps (Beta) - The beta shader category now has the four basic diffuse, specular, and bump shaders with box projection enabled. Skies can be designated as having dimensions (controlled by the transform scale parameter), and box-projected objects will use these dimensions to do their thing.

Per-Object Sky Binding - It is now also possible to call Sky.Apply on a specific Renderer, meaning you can detect when a GameObject changes rooms or environments and bind a different Sky to it. Great in tandem with box projection or even just indoor/outdoor environment switching.

Per-Object Exposure - Through scripting, you can now also set IBL intensity and exposure per Renderer. This allows you to dynamically tune the contribution of image-based light per object and implement your own, macro-scale occlusion. An example would be a character going deeper into a cave would be lit by less and less IBL based on some scripted algorithm or shadow sensor.

Spherical Harmonics Diffuse IBL (Beta) - All skies now also compute spherical harmonics coefficients to represent the diffuse light term (think 27 magic numbers that represent the entire diffuse cubemap). This happens in the Sky Inspector and may cause a slight hitch in performance but nothing like a full compute. Currently SH is beta, and included everywhere but only used by the skin shader.

Skin Shader! (Beta) - Spherical harmonics allows for fancier lighting and IBL scattering simulations so I have included the beginnings of a skin shader in the Marmoset/Beta category. The skin shader features:
- Diffuse Scattering Subdermis texture
- Translucency Translucency texture
- Diffuse Fresnel/Peach-Fuzz
- Anisotropic highlights (for hair)

HDR Render to Cubemap - Sky probes can now be rendered in HDR, through the magic of RGBM and replacement shaders! There are a few short-comings: currently it is limited to opaque or cutout shaders and will render any non-Marmoset shaders as Marmoset shaders. It works out better than I would have hoped. True shader-agnostic HDR rendering may be possible down the road.

Fixed Sky Prefabs! - New skies created with Skyshop will make proper prefabs now. Old skies can be turned into prefabs with a handy script in the Edit/Skyshop menu.

The prefab bug, for those interested, is from the moon and caused by namespacing + serialization = sad compiler.

 

All of this sounds SOOOO good... especially the skin shader (which I have been getting WELL prepared for) and HDR Render to cubemap... now I won't have to purchase any special HDR tools and stick with photoshop... (Still using 3.5.7)

 

Hi Monkeyscience,

about the Skyshop 1.05 THIS ALL sounds soooo fantastic !!! You are really a GENIOUS !! I really much appreciate all kinds of your hard work on those updates. especially the BPCEM,Skinshader fixed Sky Prefabs!!

In my last production with your Skyshop ( which i can't show now, but later i will do ) It became just an almost photorealistic production...especially with your AO shading support, which saved me such a LOT of time. it is just an Enjoy to work with your system you made for Unity.

Again, huge thanks for what you have done !!!

Best wishes
OHNO

 

I have been drilling Skyshop shaders to work better with mobile platfoms and got some decent result.
It "almost" has the same features as standard bumped specular shader except using tex2Dbias instead of lod.
Performance is.. okish but I found out that if I use SH lighting instead of cubemap for IBL diffuse term , things run a lot faster.
So I am looking for version 1.5 which supports SH natively, instead of having to use unity light probes.
It was extreamly challanging to make it work under shader model 2.0. Hardest part is making per pixel IBL world normal and world reflection to work without running out of number of interpolators.

I do really hope opengl es 3.0 becomes a standard soon. Working with shader model 2.0 is just way too painful.

 

Hi castor,

this really sounds exciting and very useful. Is it possible to share those things so we all can have advantage for optimising our scenes ?
This would be much appreciated it!

 

yea seriously. Mobile is my biggest priority. I can care less how good the shaders look if mobile isn't a possibility.

 

I think i've been spoilt, ive been making stuff for es 2.0 since i started doing mobile things so i was still at 'phones can do this?' for a long time, and now i have an ES 3 phone and no financial motivation to serve any lower specification so i still get all the fun stuff! I love it when you get to do everything you want

 

Skyshop 1.05 is now live!

Here is the complete change-log:

=====================
Major Features
- Box Projected Cubemaps (Beta)
- Spherical Harmonics Diffuse IBL (Beta)
- Skin Shader (Beta)
- Anisotropic Highlights (Beta, skin shader only)
- HDR Render to Cubemap
- Fixed Prefabs

Script
- Skyshop no longer clears inputs and settings when entering play mode
- Added Apply(Renderer target) function to Sky script for per-object Sky binding
- Added diffuse spherical harmonic computation to happen automatically in the SkyInspector
- Added undo helpers into Util to conform to both Unity 4.2 and Unity 4.3 undo API
- Cleaned up Undo and made it more specific in a few cases
- Fixed non-mipmapped specular preview bug in SkyInspector
- Fixed mset_BeastConfig unreferenced leak
- Added UniformOcclusion shader parameter that can be set per object
- Added Sky.Apply to drawGizmosSelected for sky changes without an open inspector tab
- Cleaned up and refactored Sky.cs
- Added proper serialization support to a whole list of classes

Edit/Skyshop
- "Create Prefabs from Selected" will create prefabs from all selected skies (required for old skies).

Shaders
- Added "Beta" shader category for new shaders that are not quite production-ready
- Added Box Projection Cube-Mapping (Beta)
- Added diffuse spherical harmonics (Beta)
- Added SH-based skin shader (Beta)
- Added anisotropic direct light specular (Beta, skin shader only)
- Removed manual sRGB sampling code from grass detail shaders (Unity 4.3 fixed a bug)
- Fixed invalid fallback paths in mobile terrain shaders
- Made non-blended shaders output alpha of 1.0
- Disabled lightmapping in mobile bumped-spec shaders that use vertex colors/occlusion (too many interpolators)

New shader feature #defines
- #define MARMO_BOX_PROJECTION 1
- #define MARMO_SPHERICAL_HARMONICS 1
- #define MARMO_ALPHA

Alpha
- Shaders will write 1.0 into the output alpha unless this flag is defined

Box Projection
- Shaders with box projection enabled treat the Sky as being a box with finite dimensions
- Sky object's transform scale defines the bounds of the projection box
- Does not yet work with vertex-lit shaders
- Enabled with #define MARMO_BOX_PROJECTION 1 (Note: the 1 is new and important)
- Can also be toggled as a multi-compile keyword
- This feature is considered beta and only enabled in the beta category of shaders

Spherical Harmonics
- All skies now compute spherical harmonics coefficients to be used in place of the diffuse IBL cubemap
- SH is computed from the specular, skybox, or diffuse cube depending on which is present
- SH is computed automatically in the Sky Inspector, no recomputation necessary for older content.
- SH will be used in place of the diffuse IBL cubemap in shaders that enable the feature
- Enabled with #define MARMO_SPHERICAL_HARMONICS 1
- This feature is considered beta and only enabled in the beta category of shaders

Known Issues
- Unity 4.2 grass detail shaders are now rendered incorrectly, they still need manual sRGB sampling which was fixed in Unity 4.3.0.
- Old Sky objects cannot be turned into prefabs (they lose script references), use Edit->Skyshop->Create Prefabs from Selected
- RGBM probe does not capture transparent objects
- RGBM probe does not capture tree or grass billboards properly
- RGBM probe renders non-Marmoset shaders as Marmoset ones (by-product of replacement shaders)

 

I gotta play with this one... and you sir launched this at a perfect time since I finished up a character recently... Looks like I will be waiting to use this one... I am not on Unity 4.3 yet

 

Well then good news everyone! Minimum requirements: Unity 4.2.1 and full compatibility with 4.3 .
No 3.57 though, it gave me too much trouble :-(

 

Well, if these features do work in the free version, I will use that for now... I can always look at the preview videos... curious about the HDR render to cubemap feature

 

I've been having a problem with Skyshop. Whenever I try to take a high resolution screenshot with the capture screenshot command, or for example, if I set my resolution to 640x480, all the specular values are set to 1.

Objects that had almost no reflection, like a simple box, are now super reflected like a mirror.

Is there any fix for this?

 

Could someone post previews of Skin shader?

 

+1

 

MonkeyScience for PRESIDENT!

i really much appreciate it that you was able to do it that quick. It perfectly comes out at a perfect timing.
Really really THANK YOU !