Turning off lights that you can't see

In my top down game I'd like to stay away from a full on line of sight fog, so I am trying to just get it done with lighting. The problem is you can see "through" walls if lights are on. I'm trying to find a clever way to handle this without having to shoot out 20491024 raycasts to check what can be seen and what can't be. Here is an example image:

https://imgur.com/a/qsAnwbM

If our player is in spot A, he shouldn't be able to see lights from the other room. A simple 1 time raycast. But if player is in spot B, he should be able to see the lights from that room, though the raycast can't reach the light.

Anyone know of a lightweight way to handle something like this? I was thinking of putting a couple activator nodes throughout the rooms that attach to all the rooms lights and raycasting to those, but just wondering if there's a better way.

 

If you're on the current version of Unity, you may be able to, rather than contorting yourself to avoid tons of raycasts, use the new Jobs system to do those raycasts efficiently (and on their own thread, so they can be offloaded onto a separate processor core and not impact your game's performance).

 

You could use something like https://docs.unity3d.com/Manual/CullingGroupAPI.html. However, I'm not sure if this works as intent in 2D.

 

I think red blob can help.

 

I tried both this and the node based approach, and the nodes actually came out a bit quicker, (nothing noticable) because less casts overall. Thanks for the suggestion though