Maintain dead zones for the Player when using a TargetGroup as the target

It's a 2D game and I'm using a perspective camera with body set to framing transposer.

I have a cinemachine vcam with 'follow' set to a target group. At start the TargetGroup has Player as the target. Once Player is close to the Portal within certain distance, I add the Portal to the TargetGroup with weight of 1. So the camera position at the center of the group. And that's intended

With the default settings, the camera is too close to the Player, so I increase the min FOV. And decrease max FOV to constraint the camera from zooming out too much. I also adjust dead zones, so to keep minimum distance between the Player and the screen edges.

When Player gets further away from the portal, the camera zooms out and position at the center of the group. So the Player and the Portal ends up in the left/right edges of the screen, which is not what I want. Adjusting the dead zones didn't seem to help as the camera follows the TargetGroup instead of a single target.

 

But that depends on the 'Maximum FOV' property right? I don't want the camera to zoom out too much.

As for now I attached a script to the TargetGroup to adjust the weights of the each target by its distance to the player. That helped to keep the player within the screen. This is a turn based game, so I make it to adjust the weights after each move.

But the camera looks overdone though, as the weights change a lot. Can I damp the zooming BTW?