Shader blending sky color with two lightmaps

I have been working on a shader for a day/night cycle for an old style game. It needs to support sunlight coming and going (no shadows) and other instanced lights in the map.

I have come up with a solution where I bake two lightmaps; one for the sunlight (area light) over the whole world and then another lightmap for the other instances of light (torches, campfires, etc). I originally started with two lightmaps where I had combined these types of lights (one for night and one for day) but I didn't realize that I also needed to add color tinting from the sky (blue at night, red during sunset). This made it impossible to correctly used the combined light maps because the sky color should only affect areas the sun light can reach and not say a torch light in a cave.

The current way I am doing it works well and I bake the "sun light" and the "instanced lights" separately. The problem is that I am unclear about the best way to shade the sun map while keeping the zone lights looking realistic. If we think about an outdoor campground, during the night with a blue tint, everything around the camp without light should be tinted blue. But at the campfire, we shouldn't tint it blue at all or maybe just a little bit the further it fades away from the center.

Here are the relevant areas of my shader code:

Code (CSharp):

1.     // Decode the second lightmap (light from sun)
2.     float3 sunLight = DecodeLightmap(UNITY_SAMPLE_TEX2D(_Lightmap, i.uv1));
3.  
4.     // Let's calculate the color of the sun at this pixel using black as a night blend color
5.     float3 sunLightBlend = lerp(sunLight, float3(0.0f, 0.0f, 0.0f), 1.0f - _BlendFactor);
6.  
7.     // Get the luminance so we can blend the lightmaps accurately
8.     float luminance = sunLightBlend.r * 0.3 + sunLightBlend.g * 0.59 + sunLightBlend.b * 0.11;
9.  
10.     // Lerp between the lightmaps to get the combined lightmap contribution (gi.indirect.diffuse is the zone lights map
11.     float3 combinedLightmaps = lerp(gi.indirect.diffuse, sunLight, brightness);

The results with this are great. In fact, it looks exactly the same as baking the maps together. But here is where I want to tint just the darkness in the sun map with the sky color (blue for now to test).

Should I try to find the luminance of the combined lightmaps and apply it that way? Should I apply the color before combining the lightmaps? How do I ensure that the light from a torch outdoors doesn't get too muddied with blue? A torch light inside a cave should have no isses as the sun map would be dark in there and therefore have no sky color contribution.

Interesting problem. I hope some of you guys have suggestions

Thanks!