GPU Instancing and Shadow Ranges

I'm having a wonderful time doing GPU instancing, but I've run into a major issue. I can easily GPU instance a few thousand objects, it works great! While I'm outside shadow distance, it runs at 60+ FPS. Inside shadow distance, it runs at 60+ FPS.

But CROSSING into shadow distance drops it to <10 FPS. Investigating shows me that rather than batching into a few GPU instances, it batches into literally hundreds of batches, each with the frame debug phrase "Some objects are within the shadow distance, while some other objects are not."

This is strange, you think it'd have twice as many batches... but instead, it has hundreds of batches.

I'm open to any possible solution here. As it stands, I may have to do very silly things like manually detecting whether objects are approaching shadow range and switch them from an unlit shader to a lit one or something. Any advice would be welcome.

 

Do you do this with the DrawInstanced API or just letting Unity instance?

 

I'm just letting Unity instance them, since otherwise I have to do the z-sorting myself. I could do it as a particle system, but their max vertex count is a factor.

 

I gave up and did it manually, since I can tell .DrawMeshInstanced to use or not use shadows.

 

If I set shadowdistance to 500 in my mixed lighting setup test scene from exactly same viewpoint I got about 20 batches and after setting shadowdistance to about 30-40 the batch breaked to about 40.
More batches increase render times?

 

I have the same problem with unity 2017.2 and 2018.2