Marmoset Skyshop - Image-Based Lighting Tools [RELEASED]

I've used several point lights on a IBL model with a directional light

 

Do they all cast shadows?

 

ah, yea only directional lights are casting shadows in deferred rendering.

 

Crap :/

 

Here is a little test I was doing on the iPhone of a very low poly desert / tropical scene with image based lighting. Diffuse only, not spec etc... 32 draw calls

 

Looks Good

 

I have a possible fix for deferred shadows.
Don't know if it's the best, but hope it will give monkeyscience direction for a correct fix.

Option 1 (tested):
http://docs.unity3d.com/Documentation/Components/SL-SurfaceShaders.html

edit each shader code and change 1 line
FROM:
#pragma surface MarmosetSurf MarmosetDirect
TO:
#pragma surface MarmosetSurf MarmosetDirect fullforwardshadows

Screenshot shows shadows now working!
Not sure if this is the fastest, as it may be better to compile different shaders for Forward and Deferred just as he does for Gamma and Linear color modes.


Option 2 (untested):
http://docs.unity3d.com/Documentation/Components/SL-SurfaceShaderLighting.html

write the function in MarmosetDirect.cginc
half4 LightingMarmosetDirect_PrePass (MarmosetOutput s, half4 light)

*I believe this is required for Deferred shaders?
*Since option 1 works for me, I'll leave it to him to decide

 

From what I've seen and read in bug reports, it seems Skyshop's shaders are always rendered in Forward mode, along with transparency and every other shader that deferred rendering cannot handle. The above fix does not make deferred Skyshop work but rather enables all types of shadows in forward rendering.

http://docs.unity3d.com/Documentation/Components/RenderTech-DeferredLighting.html

The Skyshop shaders define a custom lighting shader, mainly for the RGB specular map and fresnel, this is probably what breaks it. I can try coming up with a shader that uses Unity's Blinn-Phong maybe, but this means no fancy spec maps for deferred lighting folks :-/.

Deferred lighting is a PC-only feature, right? Is it a must-have for PC platforms and performance?

 

Defered lighting is a unity pro feature... I have been using Deferred lighting so I will be able to have shadows from every light... point lights don't cast shadows in forward rendering

 

After working a bit more with your wonderful Skyshop I have a point I'd like to bring up.
At my company, we often work with photorealistic materials, and our customers often want to view materials from very up close. We also often target mobile devices with limited memory. This means I have some problems with skyshop.

Before, I used to work with my own shaders, that most often were lightmapped reflective-specular shaders, that had 3 texture inputs; one for diffuse texture, one for the lightmap and one cubemap for reflection. The major difference was that the lightmap used UV2, while diffuse used UV1; and here's the kicker: the lightmap affected the reflected color. Much like your ambient occlusion texture does today. The important thing was that diffuse used UV1, which meant we could have a tiling texture; for example a scratched metal-texture, or a grainy paint texture.

In skyshop, the UV2 is used for Unitys beast bakings, but from my tests they don't affect the reflected color. I would like another category within Skyshop that is "Lightmapped", and the only difference is that these shaders use UV2 for the ambient occlusion; so UV1 can be used for tiling textures. When trying to bake tiling textures in to unwraps (such as the ones used for lightmapping), huge textures are required to allow for any closeups at all. And 4096-s or even 2048-s are a no-no on mobile, pretty much.

 

Looks awesome, and moreover nice interactive demo

 

Replying to my own post here.

I have modified the Skyshop shaders to include a lightmapped variant. Below is a link to web player where I have illustrated what this shader does.
There are 2 objects in this scene, one of them uses "Marmoset Bumped Specular IBL", the other uses my "Marmoset Lightmapped Bump Specular IBL".

http://www.animechtechnologies.com/skyshoplightmapped/

The objects have 2 UVsets, one of them (uv1) is used to tile the scratch-texture, which is as small as 512/512. The ambient occlusion (or lightmap) is using the second uv-set. To create this kind of object without my approach, you would have to have the scratching in the same texture set as the ambient occlusion, which would make is impossible to tile.

This approach works for desktop, but not for mobile, since they are compiled.

What do you say, monkeyscience, is this something you would want to include in an upcoming version?

 

For me this just displayed 2 black cubes I could orbit around... I assume i'm using a duff graphics card.

 

Yea that does look nice.

What was your process here (i.e. baked/realtime? material count?) and is there room to optimize the drawcalls?

What kind of performance are you getting?

Cheers

 

Yea there are many texture lookups and stuff, so yeah. If you see black boxes, just ignore and move on

 

I am more of a mobile developer, hence how useful would it be for me.
I am really looking to improvise my lighting system in my game and it is a 3D game.

I just read your introduction which state that it is also compatible with iOS.

Could you pl elaborate a little bit on that.

Thanks.

 

Hi, do you mean me, or Marmoset in general?
If you mean me, the performance is great on desktop (I wrote mobile first for some reason, I meant desktop. stupid typo). It's one more texture lookup than the regular Skyshop shader, which of course affects performance, but nothing huge. The process in this case was baking ambient occlusion with Maya.

 

Sorry I thought I replied to cl-apps most recent screenshot.

I suspected it was baked in part, but was curious if there were dynamic elements too...which would make the matched lighting even more impressive.

Not told it was 32 drawcalls, I would guess 4, considering the dupe objects, terrain texture resolution (including lightmap).

As for your performance on Mobile, Horp, do you have a practical example (stats/screens)?

Thanks guys!

 

Edited my post. It was a stupid typo there. Was "great performance on desktop". Haven't tried mobile but I would suspect it wouldn't be very good performance with that bumped/lightmapped/specular, but perhaps with just lightmapped/spec for example.

 

can i request you to add an interior demo of some hall or something

 

Hey Monkey,

Really love your tool, it's a great addition to the community. There are only a few bits that keep this from being a total solution for my projects that other people may appreciate too.

Cook-Torrance versions of the specular shaders.
Rim-lit variations of the shaders.
A normal map intensity slider on the shaders would also be really useful.

Keep it up!

 

I've been meaning to answer this question:

please do NOT include Skyshop shaders or scripts with models and assets you are distributing through the asset store. You would be giving away the fruits of our hard work and labor, source-code and all, to anyone who downloads your asset.

This is also a no-no in the Asset Store license agreement:

 

Great demo, Horp1. Yes, it would be cool to have an alternate mode of the shader where baked lighting is considered occlusion rather than bounced light. You could bake lightmaps and probes with just a white sky + black objects and get an occlusion-only term that you can multiply with your IBL. Ideally Unity would have both in some dual-lightmapping mode, one lightmap is added bounced light, the other multiplied occlusion of ambience.

 

If you mean the interior supplying the skybox and lighting, and have IBL lit objects inside that scene, then thats definitely possible, but if you mean an interior scene in 3D being lit using IBL, then I think it's worth considering that IBL as a concept is definitely best suited for showcasing "single objects". It doesn't really make sense to light an interior with an outer image. Also, when the size of an object grows, or when there are more objects, the IBL lighting approximation becomes increasingly bad. Mind you, IBL is just light sampled at a single point in the scene.

Maybe I'm misunderstanding you though!

 

Skyshop currently is perhaps best suited for outdoor games, this much is true. For indoor scenes where environment lighting and reflections vary greatly from room to room, you really need cubemap probes of some sort. These kinds of tools are slated for future updates.

Plenty of games have shipped with IBL though, and I would like to dispel the myth that Skyshop is only for showcasing single objects. If you can make your Unity project look good with Skyshop, you too have made a game with IBL .

 

Is it possible to programmatically and incrementally fade between multiple cubemaps? Say, to emulate a day/night cycle?

 

Not with the included shaders unfortunately, though you could write shaders that sample two spec and two diffuse cubemaps and blend between them. More texture samples but it might look really nice.

 

By the way, wouldn't it be nice if you moved away from surface shaders for skyshop? Surface shaders are badly optimized and always bloated. Would be nice to have pure vertex/fragment-shaders. Beast (especially the old version that is in Unity), is pretty bad compared to for example VRay, so a version of Skyshop as I suggested before; that uses custom bakes and is lit by those, and then also can ignore the built in lighting pipeline that comes with the clunky and bloated surface shaders.

 

Maybe I'm misunderstanding you, but I'm not really getting the idea of using IBL for lighting an interior scene; and I mean the scene itself. Would you bake an HDR from that interior environment, and then use Skyshop shaders for all the objects in the environment? Where i have seen IBL in games in interior scenes is to create the scene with a normal static bake, then create an IBL cubemap from that very scene, and then light characters or certain objects inside the interior scene with IBL (those that aren't prebaked, most likely because they are moving). Are you suggesting an approach using IBL to light the scene itself (essentially, the walls around)? How would that work?

 

My scene was light mapped for the shadows but the brightness of the objects and the warmth of the scene was all Skyshop shaders working with my panorama I made from Lumion. I ran the above scene on my iPhone 5 and it ran smooth as silk ( max fps ) with skyshop mobile diffuse ibl shaders. Ran fine on iPhone 4 too but not quite as silky smooth. I compared to CORE framework shaders and there is not much difference in speed on iPhone 4 but the scene is very flat looking then.

I think the overall look is definitely dependant on the quality of your panorama you input and takes a few before you get a feel for what images work best for what scenes.

 

Ps. I had a little mecanim character running around my scene with touch controls and there is no problem with speed on mobile.

 

Ah yes, I maybe misunderstood what you meant by "standalone objects". That is a fair point: in an interior setting you would typically light dynamic and discrete, stand-alone objects with IBL. Big sprawling objects like the floor, ceiling, and walls are going to be tough to light with a cubemap without projection tricks; you will have much better luck with traditional lightmaps.

This is true for diffuse light anyway, you cannot bake specular reflections and an IBL/cubemap approach is still desirable. It is also difficult to spot inaccuracies in specular reflections on most surfaces, which lets you get away with much cheating.

Speaking of projection tricks and lighting walls with IBL, box-projected cubemaps are very extremely awesome (if your rooms are mostly made of boxes):

 

hopping that's an answer to my question XD.

Speaking of which is it confirmed that skyshop only works with forward rendering? Because if that's so then that is one very serious draw back, these days unless you are targeting mobiles (and even then it's a matter of time) there is barely any reason to go forward rendering instead of deferred.

 

Any chance you have given any thought to parallax corrected cubemaps for the lighting?

 

WOW!!! I am really liking what you are bring into Unity!!!

 

Definitely want this...

 

Impressive stuff. Is there a documentation or tutorial for the mobile shaders somewhere? Can't get specular to work...

 

That's good to know. I can easily see your tools taking over lighting/rendering for most games in the future. I'll supply PDF's on how they can add your shaders and set up the models.

 

That's a video from a few years ago, taken in the Blender Game Engine. Has nothing to do with Skyshop or Unity.

Anyway...

Box-projected cubemaps can be fine in concave rooms if you split the areas up into convex shapes and interpolate. Also Sébastien Lagarde talks, in the Part A section here, a lot about cubemaps and what he calls "image proxies" and what I usually see referred to as billboard impostors for reflections. Just follow the links to the FXGuide article, and also he was a white paper / presentation up on how they implement the cubemaps.

----

I had a few questions about the shaders. Do you do any kind specular filtering / AA, such as Toksvig or LEAN? (Link 1, Link 2). It drastically improves the appearance of specular highlights at a distance, and reduces specular aliasing, which is the bane of my existence.

Do you handle metals in a physically based way? Such as black diffuse, colored specular, and super high specular values? (Specular Chart link). I assume you do not support anisotropic highlights. Any possibility of those too?

 

another simple material setup, SkyShop is amazing! Made in UnityFREE


Web Player

 

Dude, that is BEAUTIFUL!! Man I really want this tool... an as for solving lighting IBLs for interiors... you could just use Beast and GI with Light Probes...almost like an Unbiased Renderer. I used IBL in Unity long before Skyshop, and using HDR for interiors is not hard to do, just a lot more work...mainly with baking cubemaps on flat surfaces (which is why I used either real time reflections or matte surfaces). I also need to figure out what to do with deferred rendering for now... (Skyshop is too good to pass up)

 

It looks like I will be buying it soon as well. But I am wondering is there any parallax relief mapping shaders in Skyshop???

 

Whoa! Very cool.

 

Currently, Marmoset shaders always render in "Forward" mode, even if your project is configured as "Deferred Lighting". This is a behind-the-scenes technical detail, your models will still show up in a "Deferred Lighting" project.

Adding 'fullforwardshadows' to the shaders, as mentioned by Bocs, will enable multiple shadow-casting light sources, including point and spotlight sources. I'm going to include this fix in the upcoming patch because shadows are awesome .

I've been playing around with deferred Marmoset shaders a bit and they will likely be included in a larger material system update.

There are some drawbacks to deferred lighting and trade-offs to consider:
- It is not mobile-friendly so basically a PC-only feature.
- Blending does not work and is still rendered in "Forward" mode.
- Data is bounced between RGBA8 buffers, causing quality and dynamic range issues.

Here are some illustrations of the quality and clamping problems I ran into, exhibited by both Unity and Skyshop's deferred shaders.

 

I see the clamping issues... and I don't think they will be much of an issue... when i render my scenes with GI, I make sure that my lights are not set to an insane amount of intensity... in Linear mode, the light intensity doesn't wash out my textures... And cubemaps look way better when in deferred.

 

Can I dynamically switch between cubemaps? I know fading or lerping was asked for earlier, but I'd like to simply switch the diffuse and spec cubes referenced in a material through code.

 

Great!

Also maybe consider adding the source for ALL the shaders in another .zip file incase we need to modify one of the pre-complied ones.
*I do understand they are just variations of different defines, but we shouldn't have to guess what you used.

 

Hey guys, for those using skyshop now, I was just sitting here thinking of a nice way to baking local cubemaps and work it into a full level. So, here is something you can do

Exterior Levels

1. The first thing you should have a cubemap of is the sky, which you will use for baking with Beast. These will be panoramas of just the sky, no objects (you can find one on CGTextures or generate one, I found that panoramas work best with Beast)! Process this as a Skybox for Skyshop and use it as GI. Then in Lightmapping Extended, (I think Skyshop will set this automatically for you), just check to make sure that the environment to set to IBL, you can leave the defaults alone. Works best if it's a HDR/EXR panorama. Then BAKE! You can use FG for initial tests. I will usually use FG and Monte Carlo as a secondary integrator.

2. If you have a cubemapper (there is a GREAT one on the asset store for ten dollars, or you can wait until Marmoset adds in that feature.), you can capture cubemaps from different positions with cubemap probes all at once or one at a time to prevent tracing errors which will be barely noticeable anyway (just name the probes appropriately). And yes, you do this after you bake in the lightmaps for the full level.

You will do the same for characters, though for my game, since it takes place in a static location, I can just compute one cubemap from the origin point in every level, which in this case the spot before the fight starts. I'm making a 2.5D fighting game. And the characters models will share this specific cubemap at runtime. So if I have 12 levels, you will have to do this process for 12 levels). Keep in mind that you will have to add Light probes since skeletal meshes CANNOT accept lightmapping.

3. And after the probes are processed into Cubemaps, you can apply them to your models and texture as normal. And the best part is, if you don't like where the cube maps are, just recompute them. It will take into account the GI in the scene. And if you don't like the GI, just redo the GI bake, and recompute the cubemaps It is overkill, but you will thank me later when your games looks SUPER SLICK.

Side note, for flat things like the ground, I am still trying to solve this... it looks REALLY off if you don't use a probe in local space.

Interior Levels

It is the same process as above, except, in Lightmapping Extended, you turn off the environment, by setting it to NONE. Then bake the lightmaps and probes using FG (play around with the ray setting, leave the rest at default) or if you have the patience of a cat waiting for a mouse, then use Monte Carlo... you will have to wait a LONG time for it to bake. But it will be worth it.

Then capture the cube maps and process them for skyshop... and apply them to your models. Texture as normal.

For you guys rendering car metal, your specular map should be a nice mid range dark grey, and your gloss alpha should be near white... about 65% white. and for rubber tyres, high spec map, low gloss... and definitely make sure you increase your normal map intensity. Treat Skyshop as if you were texturing for a PBL engine... it is virtually there... sorry for the long post

 

I believe you can just add multible sky objects, enable the one you want active, and disable the others.

 

Yep, each material has a custom diffuse and specular cubemap slot you can change out in a script.