Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Question Simulation Control and Replays in Unity DOTS and Netcode

Discussion in 'NetCode for ECS' started by Opeth001, May 31, 2023.

  1. Opeth001

    Opeth001

    Joined:
    Jan 28, 2017
    Posts:
    1,068
    Hello everyone,

    I recently watched "The Path to Leveraging DOTS in Production | Unity" at GDC 2022 and found some concepts quite interesting and important for production. One key concept is the ability to fix bugs using match replays.

    In DOTS, the behavior is deterministic, meaning that the same inputs will always produce the same outputs (excluding cross-platform simulations). So, a straightforward approach to handle replays is to save the inputs for each game tick. However, due to the stateless nature of the ECS approach, certain features like rewinding or accelerating the simulation by a specific number of ticks are not possible because some states get lost (e.g., destroyed Projectile entities).

    As far as I know, the best approach currently is to save players' inputs every simulation tick and use them to modify the current world state.

    I have a couple of questions:

    1. Is there a better way to implement replays and simulation rewind/acceleration in ECS?
    2. What are the recommended strategies to implement the replay feature in DOTS Netcode?
    Thank you in advance.
     
  2. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    762
    Not really. Depend what you record and actually more about how do you run your deterministic simulation.

    To have a deterministic simulation there are various approach, but indeed having it running at fixed time step is definitively the more straightforward and probably robust.

    So, given the situation runs at fixed step, and given the initial state is identical, all random identical, all source of randomness removed (so don't depend on time, or anything like that) re-running with just input will give the same result.
    Even if the accumulated time may vary (because the delta time may vary) you are just resimulate everything at the same fixed frequency.
    Accelerating or Decreasing the delta time is also fine. You accumulate less and so simulate less often. As well as, you accumulate more, and simulate more ticks per frame if necessary.

    The problem is eventually how to handle the "partial" simulation steps. You can either interpolated the frame results (so you are running speculatively the next frame) or you playback slightly behind, or similar techniques.

    If you want to rewind (go backward), you can store the last lets say second of simulation state (worst scenario is a copy of the world state, can be fast, @tertle has a solution that was backing up millions of entities realtime) or having already in the recording (suggested anyway) snapshot at regular or irregular intervals, and restart from there upward.

    Having full state intermixed is also used to jump forward. You restore the state from X and move forward to the point you need.
    Problem to solve are then usually effects, particles, audio. These stuff must be designed as much as stateless as possible (they only depend upon the absolute time) so they can rewind appropriately.

    There is nothing special about ECS that makes it different in respect of any other solutions, apart the speed at which you may copy data, that open to some options that are less viable otherwise.
     
    toomasio, UniqueCode and Opeth001 like this.
  3. Opeth001

    Opeth001

    Joined:
    Jan 28, 2017
    Posts:
    1,068
    In my case, I am opting for a singleton entity that contains a seed, which is incremented every simulation tick to make randomness predictable for Clients, Server, and Replays.

    Storing checkpoints in the simulation for fast restarts can be a viable solution. However, it is important to consider potential drawbacks such as space requirements and complex entity selection when implementing this approach.

    In my case, Visual Effects are represented as entities and have their own state but are only handled by reactive systems in the PresentationSystemGroup. By designing them to be as stateless as possible, they can be appropriately rewound during the replay.


    I asked about ECS because its efficient memory representation and stateless nature make it a potentially advantageous choice for handling Simulation Replays. It offers better approaches in terms of memory efficiency and data manipulation, allowing for more efficient replay implementations.
     
  4. CMarastoni

    CMarastoni

    Unity Technologies

    Joined:
    Mar 18, 2020
    Posts:
    762
    Attention: this may be not deterministic at all. Dunno how you are doing the simulation on both server and client and what the server replicate (maybe nothing, just forward input) but clients are not guaranteed to run the same ticks exactly as the server: partial ticks and catch up (some ticks may skipped). may mess up. You are guaranteed to run the same ticks though in the prediction loop. However, client may be doing that multiple time if it receive predicted ghost data.
    Not even talking about the fact , the client may update or resimulate only some ghosts (partial snapshots). And that would already cause a divergence in the simulation time 0.
    Even admitting it does not, because the ghost are not interacting/influencing each other, that would still introduce the fact the seed (that looks like is common or shared among entities) may be called a different number of time on Server and Client.

    Again, without known detail is hard to give advice, but a safer solution is that every ghosts should have its own seed, and it should still need to be rollback as necessary. (it may be even replicated, if not for sake of checking what you have locally is consistent). This at least in a generic setting scenario.

    Yes, there is a cost with checkpoint. But depending on the type and replay need it may be a cost that you may be willing to pay to jump at any point of the replay in "realtime".

    I agree, this is one is big advantages. ECS though is technically stateless by itself. It is stateless if you make and programming like that. So it is always a matter of how you write your systems (rule of thumb: no caches, no state, every data it touch must be in the entity storage). And similar results can be achieved without a pure ECS implementation (
     
  5. Opeth001

    Opeth001

    Joined:
    Jan 28, 2017
    Posts:
    1,068
    I understand your perspective, and I agree with your point. In my MOBA project, I have implemented the Character Controller approach, which involves users sending input commands to their respective Player ghost. The ghost then converts these inputs into commands to control one or multiple characters.

    The server's role is to replicate the characters' states, and each user has local prediction capabilities for any ghost they control, including characters, projectiles... However, in cases where clients' local prediction involves running resimulation (multiple ticks per frame), having a singleton seed can result in incorrect predictions. This occurs because old ticks are simulated with the latest tick seed.

    AFAIK, the server always runs one tick per frame, making it reliable for replays even with the singleton seed approach. In the other hand, client side prediction must rollback the seed foreach tick simulation to make randomness more accurate.

    How does resimulation work precisely? If I store a seed per ghost entity, does the netcode package roll back the seed data based on the actual resimulated tick?
    Also, knowing that the seed is incremented every simulation tick wouldn't it be better to simply remove that difference between the server tick and the actual resimulation tick to get the correct seed instead of storing a seed per ghost ?

    thank you!
     
    Last edited: Jun 1, 2023