Search Unity

Question Unrelaible results using cinemachine confiner with group framing transposer

Discussion in 'Cinemachine' started by SudoCat, Sep 9, 2023.

  1. SudoCat

    SudoCat

    Joined:
    Feb 19, 2013
    Posts:
    64
    I'm working on a 2D project with local multiplayer. We need to keep all players on screen at the same time, and keep the camera confined to the boundaries of the arena. When players are close together, zoom the camera in, when they're further apart, zoom farther out.

    This seems like it should be a fairly straight forward requirement to solve with Cinemachine; Just use a Target Group, a Framing Transposer, and a Confiner extension on the vcam.

    Except I've never been able to make this work. I've mostly ended up ditching the confiner part of this setup, because as far as I can tell, it does not work with the Confiner. I've tried using both the Confiner in Confine 2D mode, and the Confiner 2D extension. We have a simple Camera Bounds polygon, set to a 16:9 aspect ratio.
    The framing transposer is configured to never let the camera frustum grow larger than this camera bounds size. I've attempted using the oversize window property of the Confiner 2D script, with a broad range of settings.

    Unfortunately, even with all of this, the results are unreliable, and often worse than having no confiner at all.

    From what I can gather, the confiner only works with cameras that have a fixed zoom level. We have a perspective camera setup that uses both dolly and zoom, although in the past I've tried with either only dolly or zoom, and using orthographic camera too.

    The issues we observe is that the camera confiner only seems to work at the size the cache was baked for. If I test in editor by manually baking the cache and moving the player around, it will work perfectly, until the camera zoom adjusts, and then the results become unreliable again; either extending the viewable area beyond the camera bounds, or constraining the camera too tightly, causing players to be off screen.

    Obviously it would be a terrible idea to keep calling invalidate cache every time the zoom changes, which is the only answer I've found to make things work. It was better to simply remove the confiner.

    We even have a custom target group script, that prevents out target group bounding box from exceeding the size of the CameraBounds, and an offscreen camera solution, which again, helps the problem, but doesn't always prevent the camera escaping the bounds in extreme scenarios, revealing the seams beyond the level.

    I've read just about every thread in this forum looking for answers, but always come up short. I've tried implementing custom confiner scripts, but usually found myself sorely out of my depth.

    So I'm finally at my wits end, and I'm here to ask for help. Am I misunderstanding how this feature is supposed to be used, or is this just not possible with Cinemachine? Is there something on the asset store that may save me? (In previous projects I've used ProCamera2D, which did not seem to encounter these issues, although it's been years since I used it)

    How should I proceed?

    Thanks in advance!
     
  2. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,086
    Here to say I've had similar experiences and ended up making preset cameras for each possible situation and then blending between them with complex yet judicious use of the Cinemachine Mixer's functionality. Despite great care it's not ideal, and gets very messy with 3 or more possible camera situations.

    Here's hoping discussions in this thread reveal a way to reliably use confiners and group framing.
     
    SudoCat likes this.
  3. SudoCat

    SudoCat

    Joined:
    Feb 19, 2013
    Posts:
    64
    Glad to know I'm not the only one battling with this at least.

    The part that confuses me the most is what reads like conflicting information in the documentation

    "When confining the camera, the Cinemachine Confiner 2D considers the camera’s view size at the polygon plane, and its aspect ratio. Based on this information and the input polygon, it computes a second (smaller) polygon, and constrains the camera’s transform to it. Computation of this secondary polygon is resource-intensive, so you should only do this when absolutely necessary."

    This definitely seems to imply that Confiner 2D functions with a fixed camera size, and that changes to the camera view size will require recalculation, yet...

    "set the Max Window Size property to the largest size you expect the camera window to have."​

    this statement seems to imply that the window size could be variable, which would surely mean the camera size could be variable?

    The source code for the CinemachineConfiner2D also seems to check the frustum height each frame, and convert it to the baked space - this made me think that it should be adjusting to frustum size, but then it doesn't seem to actually do any comparisons to previous frustum sizes.

    Code (CSharp):
    1. var currentFrustumHeight = CalculateHalfFrustumHeight(state, cameraPosLocal.z);
    2. // convert frustum height from world to baked space. deltaWorldToBaked.lossyScale is always uniform.
    3. var bakedSpaceFrustumHeight = currentFrustumHeight * m_shapeCache.m_DeltaWorldToBaked.lossyScale.x;
    4.  
    5. // Make sure we have a solution for our current frustum size
    6. var extra = GetExtraState<VcamExtraState>(vcam);
    7. extra.m_vcam = vcam;
    8. if (confinerStateChanged || extra.m_BakedSolution == null || !extra.m_BakedSolution.IsValid())
    9. {
    10.     extra.m_BakedSolution = m_shapeCache.m_confinerOven.GetBakedSolution(bakedSpaceFrustumHeight);
    11. }
    If this is simply a limitation of the confiner, I feel like this should be clearly stated on the documentation, at the very least. Ideally, it'd be great to be presented with an alternative solution.
     
  4. Gregoryl

    Gregoryl

    Unity Technologies

    Joined:
    Dec 22, 2016
    Posts:
    7,711
    The confiner does indeed bake for a specific frustum height. If you change that height dynamically, then you must re-bake the confiner (calling InvalidateCache is the way to do that). We don't do it automatically because of the performance implications.
     
  5. SudoCat

    SudoCat

    Joined:
    Feb 19, 2013
    Posts:
    64
    Thanks for the clarification, it would be great if the documentation clearly stated this, and perhaps offered some alternatives - I imagine this must be a fairly common requirement. However, I recognise that probably isn't going to be a priority.

    In the end I created my own custom, much dumber confiner, which is working much better for my use case, without taking the perf hit of baking a bunch of specifics I don't need.
     
    antoinecharton likes this.
  6. antoinecharton

    antoinecharton

    Unity Technologies

    Joined:
    Jul 22, 2020
    Posts:
    189
    SudoCat likes this.
  7. SudoCat

    SudoCat

    Joined:
    Feb 19, 2013
    Posts:
    64
    This is great to see!

    This section in particular clearly describes the issue.

    My only suggestion would be adding a note either below this, or possibly better in the Efficiency section, highlighting that due to the expense of repeatedly invalidating the lens cache, this confiner is not suitable for cameras which change zoom, size or field of view frequently per frame, such as when using the Group Framing Extension, so a custom confining solution is advised.
     
    antoinecharton likes this.
  8. urbannomadnome

    urbannomadnome

    Joined:
    Oct 26, 2023
    Posts:
    9
    hi sudo, im looking for the same answer, and im at the edge of simply not using the confiner when using the target group camera, how did you managed to make your own custom confining solution? cheers!
     
  9. antoinecharton

    antoinecharton

    Unity Technologies

    Joined:
    Jul 22, 2020
    Posts:
    189
    Have you tried calling
    Code (CSharp):
    1. InvalidateLensCache()
    ? Or are you looking for something different?

    This is why it's not done automatically.

    Was it what you were looking for?
     
  10. SudoCat

    SudoCat

    Joined:
    Feb 19, 2013
    Posts:
    64
    If you're looking for something to update in realtime without the perf implications of invalidating the cache, then you'll want to do the custom confiner too. My solution still has some little edge case bugs I need to sort out (it's a bit unreliable right now), so I can't guarantee you'll be satisfied with the following code either. There's definitely something wrong with my maths, but it's been good-enough for prototyping with.

    I basically looked at how the built-in confiner worked, and worked backwards from there.

    My solution doesn't handle orthographic cameras, as I'm not using one so didn't need it.

    Essentially it just does some frustum calculations to figure out the world-space bounds of the current camera state, and then displaces the camera to stay within the constraint area. In my version I read the bounds of the current level, but for the sake of sharing the code, I've just added the bounds as a field on the confiner itself so you can test it easier.

    I also use a custom target group script, which constrains the target group to the same bounds.

    Code (CSharp):
    1.  
    2.     public class CinemachineCustomConfiner : CinemachineExtension
    3.     {
    4.         [Range(0, 5)]
    5.         [SerializeField] private float damping;
    6.  
    7.         [SerializeField] private float bottomScreenOmitZone;
    8.         [SerializeField] private Bounds bounds;
    9.    
    10.         class VcamExtraState
    11.         {
    12.             public Vector3 dampedDisplacement;
    13.             public Vector3 previousDisplacement;
    14.         };
    15.  
    16.         protected override void PostPipelineStageCallback(CinemachineVirtualCameraBase vcam, CinemachineCore.Stage stage, ref CameraState state, float deltaTime)
    17.         {
    18.             if (stage != CinemachineCore.Stage.Finalize) return;
    19.             if (vcam.State.Lens.Orthographic)
    20.             {
    21.                 // currently not needed!
    22.             }
    23.             else
    24.             {
    25.                 var extra = GetExtraState<VcamExtraState>(vcam);
    26.                 var constraint = bounds;
    27.                 var fov = state.Lens.FieldOfView;
    28.                 var position = state.CorrectedPosition;
    29.            
    30.                 var distance = Mathf.Abs(constraint.center.z - position.z);
    31.                 var frustumHeight = 2.0f * distance * Mathf.Tan(fov * 0.5f * Mathf.Deg2Rad);
    32.                 var frustumWidth = frustumHeight * state.Lens.Aspect;
    33.                 var center = new Vector3(position.x, position.y, constraint.center.z);
    34.                 var size = new Vector3(frustumWidth, frustumHeight, 1f);
    35.                 var cameraBounds = new Bounds(center, size);
    36.                 var newBounds = ConstrainBounds(constraint, cameraBounds, state.Lens.Aspect);
    37.            
    38.                 var prev = extra.previousDisplacement;
    39.                 var displacement = newBounds.center - position;
    40.                 displacement.z = -(newBounds.extents.y / Mathf.Tan(fov * 0.5f * Mathf.Deg2Rad) - distance);
    41.                 extra.previousDisplacement = displacement;
    42.            
    43.                 if (!VirtualCamera.PreviousStateIsValid || deltaTime < 0 || damping <= 0)
    44.                     extra.dampedDisplacement = Vector3.zero;
    45.                 else
    46.                 {
    47.                     // If a big change from previous frame's desired displacement is detected,
    48.                     // extract difference for damping
    49.                     if (prev.sqrMagnitude > 0.01f && Vector2.Angle(prev, displacement) > 10)
    50.                         extra.dampedDisplacement += displacement - prev;
    51.                
    52.                     extra.dampedDisplacement -= Damper.Damp(extra.dampedDisplacement, damping, deltaTime);
    53.                     displacement -= extra.dampedDisplacement;
    54.                 }
    55.            
    56.                 state.PositionCorrection += displacement;
    57.             }
    58.         }
    59.  
    60.         private Bounds ConstrainBounds(Bounds constraint, Bounds target, float aspect)
    61.         {
    62.             // first check if too big
    63.             var height = Mathf.Min(constraint.extents.y, Mathf.Max(target.extents.x / aspect, target.extents.y));
    64.             target.extents = new Vector3(height * aspect, height);
    65.        
    66.             // next check if over the bounds
    67.             var xMinOverhang = Mathf.Max(0, constraint.min.x - target.min.x);
    68.             var yMinOverhang = Mathf.Max(0, constraint.min.y - target.min.y);
    69.             var xMaxOverhang = Mathf.Min(0, constraint.max.x - target.max.x);
    70.             var yMaxOverhang = Mathf.Min(0, constraint.max.y - target.max.y);
    71.             target.center += new Vector3(xMinOverhang + xMaxOverhang, yMinOverhang + yMaxOverhang);
    72.  
    73.             return target;
    74.         }
    75.      
    76.         // This function is copied straight out of Cinemachine's built-in confiner class.
    77.         static Bounds GetScreenSpaceGroupBoundingBox(
    78.             ICinemachineTargetGroup group, ref Vector3 pos, Quaternion orientation)
    79.         {
    80.             var observer = Matrix4x4.TRS(pos, orientation, Vector3.one);
    81.             group.GetViewSpaceAngularBounds(observer, out var minAngles, out var maxAngles, out var zRange);
    82.             var shift = (minAngles + maxAngles) / 2;
    83.  
    84.             var q = Quaternion.identity.ApplyCameraRotation(new Vector2(-shift.x, shift.y), Vector3.up);
    85.             pos = q * new Vector3(0, 0, (zRange.y + zRange.x)/2);
    86.             pos.z = 0;
    87.             pos = observer.MultiplyPoint3x4(pos);
    88.             observer = Matrix4x4.TRS(pos, orientation, Vector3.one);
    89.             group.GetViewSpaceAngularBounds(observer, out minAngles, out maxAngles, out zRange);
    90.  
    91.             // For width and height (in camera space) of the bounding box, we use the values at the center of the box.
    92.             // This is an arbitrary choice.  The gizmo drawer will take this into account when displaying
    93.             // the frustum bounds of the group
    94.             var d = zRange.y + zRange.x;
    95.             Vector2 angles = new Vector2(89.5f, 89.5f);
    96.             if (zRange.x > 0)
    97.             {
    98.                 angles = Vector2.Max(maxAngles, UnityVectorExtensions.Abs(minAngles));
    99.                 angles = Vector2.Min(angles, new Vector2(89.5f, 89.5f));
    100.             }
    101.             angles *= Mathf.Deg2Rad;
    102.             return new Bounds(
    103.                 new Vector3(0, 0, d/2),
    104.                 new Vector3(Mathf.Tan(angles.y) * d, Mathf.Tan(angles.x) * d, zRange.y - zRange.x));
    105.         }
    106.     }
    On top of that, I'm also using a CustomTransposer script, which is basically a heavily simplified version of the built-in one, missing a lot of the core features. It may not be suitable for your use case. This is mostly used for better control over the target size and offset, and I don't believe you'll need it for the confiner, but it's been a little while since I did this.

    Code (CSharp):
    1.  
    2.     public class CinemachineCustomTransposer : CinemachineComponentBase
    3.     {
    4.         [Range(0f, 20f)] [SerializeField] private float xDamping;
    5.         [Range(0f, 20f)] [SerializeField] private float yDamping;
    6.         [Range(0f, 20f)] [SerializeField] private float zDamping;
    7.         [SerializeField] private float cameraDistance = 10f;
    8.         [SerializeField] private float minimumDistance;
    9.         [SerializeField] private float maximumDistance;
    10.         [SerializeField] private Vector2 targetSize = Vector2.one;
    11.         [SerializeField] private Vector2 targetOffset = Vector2.zero;
    12.         [SerializeField] private float maxDollyIn;
    13.         [SerializeField] private float maxDollyOut;
    14.         [SerializeField] private bool centerOnActivate;
    15.  
    16.         const float MinimumGroupSize = 0.01f;
    17.    
    18.         public override bool IsValid => enabled && FollowTarget != null;
    19.         public override CinemachineCore.Stage Stage => CinemachineCore.Stage.Body;
    20.    
    21.         public Bounds LastBounds { get; private set; }
    22.         public Matrix4x4 LastBoundsMatrix { get; private set; }
    23.         public Vector3 PreviousCameraPosition { get; private set; }
    24.         public Vector3 TrackedPoint { get; private set; }
    25.         private Quaternion _prevRotation;
    26.         private bool _inheritingPosition;
    27.  
    28.         public override void MutateCameraState(ref CameraState curState, float deltaTime)
    29.         {
    30.             var lens = curState.Lens;
    31.             bool previousStateIsValid = deltaTime >= 0 && VirtualCamera.PreviousStateIsValid;
    32.             if (!previousStateIsValid)
    33.             {
    34.                 PreviousCameraPosition = curState.RawPosition;
    35.                 _prevRotation = curState.RawOrientation;
    36.                 if (!_inheritingPosition && centerOnActivate)
    37.                 {
    38.                     PreviousCameraPosition = FollowTargetPosition
    39.                                              + (curState.RawOrientation * Vector3.back) * cameraDistance;
    40.                 }
    41.             }
    42.             if (lens.Orthographic) return;
    43.        
    44.             var verticalFOV = lens.FieldOfView;
    45.        
    46.             var group = AbstractFollowTargetGroup;
    47.             var isGroupFraming = group != null && !group.IsEmpty;
    48.             if (!isGroupFraming) return;
    49.             var followTargetPosition = ComputeGroupBounds(group, ref curState);
    50.  
    51.             TrackedPoint = followTargetPosition;
    52.  
    53.             var targetHeight = GetTargetHeight(LastBounds.size / targetSize);
    54.             targetHeight = Mathf.Max(targetHeight, MinimumGroupSize);
    55.  
    56.             var boundsDepth = LastBounds.extents.z;
    57.             var z = LastBounds.center.z;
    58.             if (z > boundsDepth)
    59.                 targetHeight = Mathf.Lerp(0, targetHeight, (z - boundsDepth) / z);
    60.  
    61.             // What distance from near edge would be needed to get the adjusted
    62.             // target height, at the current FOV
    63.             var targetDistance = targetHeight / (2f * Mathf.Tan(verticalFOV * Mathf.Deg2Rad / 2f));
    64.  
    65.             // Clamp to respect min/max distance settings to the near surface of the bounds
    66.             targetDistance = Mathf.Clamp(targetDistance, minimumDistance, maximumDistance);
    67.  
    68.             var targetDelta = targetDistance - cameraDistance;
    69.             targetDelta = Mathf.Clamp(targetDelta, -maxDollyIn, maxDollyOut);
    70.             targetDistance = cameraDistance + targetDelta;
    71.        
    72.             // Apply offset
    73.             var frustumHeight = Mathf.Tan(0.5f * verticalFOV * Mathf.Deg2Rad) * (targetDistance);
    74.             var frustumWidth = frustumHeight * lens.Aspect;
    75.  
    76.             if (targetOffset != Vector2.zero)
    77.             {
    78.                 TrackedPoint += new Vector3(targetOffset.x * frustumWidth, targetOffset.y * frustumHeight, 0);
    79.             }
    80.        
    81.             // Optionally allow undamped camera orientation change
    82.             Quaternion localToWorld = curState.RawOrientation;
    83.             if (previousStateIsValid)
    84.             {
    85.                 var q = localToWorld * Quaternion.Inverse(_prevRotation);
    86.                 PreviousCameraPosition = TrackedPoint + q * (PreviousCameraPosition - TrackedPoint);
    87.             }
    88.             _prevRotation = localToWorld;
    89.        
    90.             // Work in camera-local space
    91.             Vector3 camPosWorld = PreviousCameraPosition;
    92.             Quaternion worldToLocal = Quaternion.Inverse(localToWorld);
    93.             Vector3 cameraPos = worldToLocal * camPosWorld;
    94.             Vector3 targetPos = (worldToLocal * TrackedPoint) - cameraPos;
    95.        
    96.             // Move along camera z
    97.             Vector3 cameraOffset = Vector3.zero;
    98.             float cameraMin = Mathf.Max(minimumDistance, targetDistance);
    99.             float cameraMax = Mathf.Max(cameraMin, targetDistance);
    100.             if (targetPos.z < cameraMin)
    101.                 cameraOffset.z = targetPos.z - cameraMin;
    102.             if (targetPos.z > cameraMax)
    103.                 cameraOffset.z = targetPos.z - cameraMax;
    104.  
    105.             cameraOffset += new Vector3(targetPos.x, targetPos.y, 0);
    106.             var damping = new Vector3(xDamping, yDamping, cameraOffset.z > 0 ? zDamping : 1);
    107.             cameraOffset = VirtualCamera.DetachedFollowTargetDamp(cameraOffset, damping, deltaTime);
    108.             curState.RawPosition = localToWorld * (cameraPos + cameraOffset);
    109.             PreviousCameraPosition = curState.RawPosition;
    110.         }
    111.  
    112.         private float GetTargetHeight(Vector2 boundsSize)
    113.         {
    114.             return Mathf.Max(boundsSize.x / VcamState.Lens.Aspect, boundsSize.y);
    115.         }
    116.  
    117.         private Vector3 ComputeGroupBounds(ICinemachineTargetGroup group, ref CameraState curState)
    118.         {
    119.             var cameraPos = curState.RawPosition;
    120.             var fwd = curState.RawOrientation * Vector3.forward;
    121.  
    122.             // Get the bounding box from camera's direction in view space
    123.             LastBoundsMatrix = Matrix4x4.TRS(cameraPos, curState.RawOrientation, Vector3.one);
    124.             var b = group.GetViewSpaceBoundingBox(LastBoundsMatrix);
    125.             var groupCenter = LastBoundsMatrix.MultiplyPoint3x4(b.center);
    126.             var boundsDepth = b.extents.z;
    127.             if (!curState.Lens.Orthographic)
    128.             {
    129.                 // Parallax might change bounds - refine
    130.                 var d = (Quaternion.Inverse(curState.RawOrientation) * (groupCenter - cameraPos)).z;
    131.                 cameraPos = groupCenter - fwd * (Mathf.Max(d, boundsDepth) + boundsDepth);
    132.  
    133.                 // Will adjust cameraPos
    134.                 b = GetScreenSpaceGroupBoundingBox(group, ref cameraPos, curState.RawOrientation);
    135.                 LastBoundsMatrix = Matrix4x4.TRS(cameraPos, curState.RawOrientation, Vector3.one);
    136.                 groupCenter = LastBoundsMatrix.MultiplyPoint3x4(b.center);
    137.             }
    138.             LastBounds = b;
    139.             return groupCenter - fwd * boundsDepth;
    140.         }
    141.  
    142.         private static Bounds GetScreenSpaceGroupBoundingBox(
    143.             ICinemachineTargetGroup group, ref Vector3 pos, Quaternion orientation)
    144.         {
    145.             var observer = Matrix4x4.TRS(pos, orientation, Vector3.one);
    146.             group.GetViewSpaceAngularBounds(observer, out var minAngles, out var maxAngles, out var zRange);
    147.             var shift = (minAngles + maxAngles) / 2;
    148.  
    149.             var q = Quaternion.identity.ApplyCameraRotation(new Vector2(-shift.x, shift.y), Vector3.up);
    150.             pos = q * new Vector3(0, 0, (zRange.y + zRange.x)/2);
    151.             pos.z = 0;
    152.             pos = observer.MultiplyPoint3x4(pos);
    153.             observer = Matrix4x4.TRS(pos, orientation, Vector3.one);
    154.             group.GetViewSpaceAngularBounds(observer, out minAngles, out maxAngles, out zRange);
    155.  
    156.             // For width and height (in camera space) of the bounding box, we use the values at the center of the box.
    157.             // This is an arbitrary choice.  The gizmo drawer will take this into account when displaying
    158.             // the frustum bounds of the group
    159.             var d = zRange.y + zRange.x;
    160.             Vector2 angles = new Vector2(89.5f, 89.5f);
    161.             if (zRange.x > 0)
    162.             {
    163.                 angles = Vector2.Max(maxAngles, UnityVectorExtensions.Abs(minAngles));
    164.                 angles = Vector2.Min(angles, new Vector2(89.5f, 89.5f));
    165.             }
    166.             angles *= Mathf.Deg2Rad;
    167.             return new Bounds(
    168.                 new Vector3(0, 0, d/2),
    169.                 new Vector3(Mathf.Tan(angles.y) * d, Mathf.Tan(angles.x) * d, zRange.y - zRange.x));
    170.         }
    171.     }
    all in all, this mostly works, but I still need to polish it. There's probably some very stupid mistakes in it! Maths is not my strong suit :oops: (Maybe we'll get lucky and one of these far more capable Cinemachine developers will point out what I messed up ;))

    When I get time, I plan to just merge the confiner behaviour into the transposer, which I hope will help resolve some of my issues.
     
    Last edited: Jan 19, 2024
    antoinecharton and Unifikation like this.
  11. urbannomadnome

    urbannomadnome

    Joined:
    Oct 26, 2023
    Posts:
    9
    yeah, thanks for your answer, but only tried clearing the cache while in editor and that solve it, but not shure if doing it every frame via scripting would kill performance for potato pcs. :/
     
  12. urbannomadnome

    urbannomadnome

    Joined:
    Oct 26, 2023
    Posts:
    9
    thanks a lot for sharing, gonna try it :)