Cannot negate mouse input values

I have been working with a freelook camera.
I want it to rotate back to normal when a mouse input is not held, and I do not want the camera to move from mouse inputs unless that same mouse input is held.

Even when setting the mouse input value to 0 each frame it still moves with my mouse, here is the code:

Code (CSharp):

1.     private void HandlePlayerInputs()
2.     {
3.         if (Input.GetKey(KeyCode.Mouse0))
4.         {
5.             cFL.m_RecenterToTargetHeading.m_enabled = false;
6.             cFL.m_YAxisRecentering.m_enabled = false;
7.         }
8.         else
9.         {
10.             cFL.m_YAxis.m_InputAxisValue = 0f;
11.             cFL.m_XAxis.m_InputAxisValue = 0f;
12.         }
13.         if (Input.GetKeyUp(KeyCode.Mouse0))
14.         {
15.             cFL.m_RecenterToTargetHeading.m_enabled = true;
16.             cFL.m_YAxisRecentering.m_enabled = true;
17.         }
18.     }

Not sure what is going on.

 

I have been googling this forever I can't believe that thread didn't pop up. Thanks this looks like exactly what I'm looking for!