Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Perception cam frame drop

Discussion in 'Robotics' started by floriegel, Jan 15, 2024.

  1. floriegel

    floriegel

    Joined:
    Feb 16, 2023
    Posts:
    3
    Hi,

    I'm currently working on a multi-robot simulation in unity (RTX4090, Intel i7-13700F, 64 GB RAM).
    The goal is a simulation containing at least 3 mobile robots equipped with 2 LiDARs (published via ROS2) and 1 camera each.
    Since I want to use the camera images later for deep learning, I installed the perception package.
    Problem is: frames are dropping dramatically.
    When using 1 Robot with 2 lidars, the frame rate is around 60 fps.
    When adding the perception cam with 2D bounding box label generation, the frames drop by around 5 fps.
    When adding semantic segmentation drops to 30 fps and with adding depth images, it drops to 15 fps.
    That wouldn't be a problem for me since I want to record all of that data (LiDAR via ROS) and real-time is not needed.
    But unfortunately the sim time in Unity seems to be somehow decoupled from the frame rate.
    I tried to publish sim time to ROS but even after recording as a ROS bag, the data is not steadily published.
    Is there a way to couple sim time and FPS?

    Thanks in advance!
     
  2. ActiveSim

    ActiveSim

    Joined:
    May 10, 2019
    Posts:
    59
    Hi floriegel,
    have a look at the execution order. Here you find which call is made when: https://docs.unity3d.com/Manual/ExecutionOrder.html
    You have the greatest chance by deciding which process you would like to run in the update frame and which in the fixedUpdate frame. You can also have a look which package (ROS, perception, etc) is using what.
     
  3. floriegel

    floriegel

    Joined:
    Feb 16, 2023
    Posts:
    3
    Ok. My first idea was to use Time.CaptureDeltaTime to slow down the overall simulation and to give my PC more time to execute the scripts. Unfortunately, this didn't make any difference. I then tried to move everything into fixedUpdate and increase Time.FixedDeltaTime. But again, this made no difference.