Directional light shadows and LODs

We are looking at performance and we are noticing that despite having LOD's on objects, the geometry drawn in the shadow maps are always LOD0. Is there a way to control this behavior?

For example it is not uncommon for us to have a small object with a very high poly count, and have many of these in the scene. You would normally only see this if you are right on top of it but when the shadow map is draw it draws all off these at their highest poly count. It ends up being the majority of the polys being drawn in the scene. For example a camera may be drawing a scene with 200k polys, but when it goes to draw the shadow maps it adds another 1.5m.