Third person camera stutters when facing a large number of objects.

I'm working on this third-person shooter game. I made the controller using cinemachine and everything seems to finally work fine. On the side, I'm working on a city for the game. It still isn't that big but there are a few big low poly buildings (maybe about 50 objects in total). So when I face towards the city and move my mouse, it stutters a lot and overall is not smooth at all. If I look in the other direction it's perfectly smooth. I checked the FPS as well and it does not really fluctuate. The movement while facing the city doesn't stutter, it's only the camera. I tried disabling the city part and there's no stutter then. I don't think I have nearly enough assets yet for it to be a performance issue. If it is a performance issue then how would I even continue building the city if it stutters just for a few buildings?

I've seen a lot of stuttering threads but most of them are linked to the update methods and their stutter is consistent in all directions. Since in my case it happens only when a decent number of objects are present, could someone please help me out?

 

Try measuring fps in the game view using Graphy from the Asset Store. Pretty sure you‘ll see a drop in fps, I can‘t think of anything besides lowered fps causing a stutter even if you only drop a single frame every couple frames.

Have you checked the profiler? It could also be due to editor overhead.