Search Unity

  1. The 2022.1 beta is now available for testing. To find out what's new, have a look at our 2022.1 beta blog post.
    Dismiss Notice

Help Wanted Unity Camera Jittering When Using Coroutines Instead of Executing Every Frame

Discussion in 'Barracuda' started by Afavar, Jul 23, 2021.

  1. Afavar

    Afavar

    Joined:
    Jul 17, 2013
    Posts:
    22
    I am detecting faces with BlazeFace using the repository below:

    https://github.com/keijiro/BlazeFaceBarracuda

    It actually works quite well and it makes the inference in the LateUpdate method. I have a third person controller (using the new unity starter asset) and everything is smooth when I am playing with the character and also detecting faces.

    Code (CSharp):
    1. void LateUpdate()
    2. {
    3.     if (_webcam != null) RunDetector(_webcam.Texture);
    4. }

    However, I don't need to detect faces in each frame, lets say every 0.5 seconds is enough. So I used a
    Coroutine and executed it in the Start method.

    Code (CSharp):
    1. IEnumerator Example()
    2. {
    3.     while (true)
    4.     {
    5.         yield return new WaitForSeconds(0.5f);
    6.         if (_webcam != null) RunDetector(_webcam.Texture);
    7.     }
    8. }
    As expected my fps increased dramatically but I have started to have jittering with the Unity camera when I am moving my character around. I have also tried doing it with a timer but ended up with the same jittery result.

    Code (CSharp):
    1. if (Time.time > nextActionTime)
    2. {
    3.     nextActionTime += period;
    4.     // Webcam test: Run the detector every frame.
    5.     if (_webcam != null) RunDetector(_webcam.Texture);
    6. }
    What am I missing here? When I detect faces in every frame, Unity camera is smooth however I am wasting my performance since I don't need to detect faces in every frame. I try to detect faces in every x seconds, my fps increases a lot but I end up with jittery Unity camera.
     
  2. amirebrahimi_unity

    amirebrahimi_unity

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    395
    Hi @Afavar, it's hard to say what could be causing your jitteriness. I'd say it would be best to use remote profiling to see what is happening when the spike occurs. Could you try that and report back?
     
  3. amirebrahimi_unity

    amirebrahimi_unity

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    395
    Also, a colleague of mine mentioned that on it might help to set the following:
    _webcam.requestedFPS = 30;
     
  4. Afavar

    Afavar

    Joined:
    Jul 17, 2013
    Posts:
    22
    Hi @amirebrahimi_unity , thank you for your suggestions. I have tried setting the requestedFPS to 15 and used
    WebCamTexture.didUpdateThisFrame method to process the image. The result was def better. However, I have noticed that I experienced this behaviour while I am in the play mode. I tried to take a build and strangely everything was smooth.
     
  5. amirebrahimi_unity

    amirebrahimi_unity

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    395
    Odd that it stutters in play mode, but glad to hear it is working correctly on the device.
     
    Afavar likes this.
  6. Afavar

    Afavar

    Joined:
    Jul 17, 2013
    Posts:
    22
    Yeah I am glad it works quite well. The bigest issue I had after that was to realize texture colors are messed up when using the Linear color space. (which urp and hdrp uses by default) I haven't realized that until I had a look at the output texture of the tensor. Hopefully I have found a Unity blog post regarding on style transfer and thank god they also used linear color space for that demo. It took some time but I managed to use the shader there to fix the color issue. I hope this would be fixed with the later Barracuda releases by default.
     
unityunity