Search Unity

  1. We are migrating the Unity Forums to Unity Discussions by the end of July. Read our announcement for more information and let us know if you have any questions.
    Dismiss Notice
  2. Dismiss Notice

Question Mlagents: unity not rendering new frames every-time the environment is stepped

Discussion in 'ML-Agents' started by unity_73F42838A916509B8C3B, Apr 9, 2024.

  1. unity_73F42838A916509B8C3B

    unity_73F42838A916509B8C3B

    Joined:
    Jan 28, 2022
    Posts:
    2
    I am currently working on a project were I require a one-to-one relationship between the env.step() function and the frames that are actually rendered on the screen. I need this because I am developing a low latency VR system using mlagents as an interface. To test this I have a c# script in unity that controls a UI Image which on each rendered frame changes between black and white.

    Code (CSharp):
    1. using UnityEngine;
    2. using UnityEngine.UI;
    3.  
    4. public class PhotodiodeChange : MonoBehaviour
    5. {
    6.     private Image squareImage;
    7.     private Color blackColor = Color.black;
    8.     private Color whiteColor = Color.white;
    9.     public float sync_state = 0;
    10.  
    11.     private void Start()
    12.     {
    13.         squareImage = GetComponent<Image>();
    14.     }
    15.  
    16.     private void FixedUpdate()
    17.     {
    18.         ChangeColor();
    19.         if (squareImage.color == Color.black){
    20.             sync_state = 1f;
    21.         } else {
    22.             sync_state = 2f;
    23.         }
    24.     }
    25.  
    26.     private void ChangeColor()
    27.     {
    28.         squareImage.color = squareImage.color == blackColor ? whiteColor : blackColor;
    29.  
    30.     }
    31. }
    32.  

    I then have a mlagent python script which runs in a loop with a time.sleep() call such that i can change the frequency of the env.step() calls.


    When i set this to 1 second - time.sleep(1) - I can see that unity is not rendering at this 1 second delay in fact it appears to be missing frames entirely. I assume this is because the fixed update in unity needs to be called at a fixedtimedelta. How can i ensure that I get a one to one mapping between the env.step() call and the image that is actually rendered?