Search Unity

Audio Issues with audio timing for rhythm game (audio time ahead rather than behind)

Discussion in 'Audio & Video' started by so3arbelnox, Jun 6, 2019.

  1. so3arbelnox

    so3arbelnox

    Joined:
    Apr 17, 2019
    Posts:
    3
    Hello,

    I am working on converting our old iOS game Rhythm Control 2 to Unity. Initially things were going quite well, but when I got to working on the main game logic I started having issues with audio syncing. To explain how the game works: A circular note spawns at certain millisecond values in the song and shrinks until it reaches the size of a smaller static circle object on the screen. At that point you hit the note and are awarded points based on how accurate you are (33ms for an amazing, 66ms for a great etc).

    The game logic works as follows:

    In Start:
    ------------
    - Read from a text file containing all information about the song "notes" and store them in an array of custom objects.
    - Play the song with a small delay music.PlayScheduled(AudioSettings.dspTime + 3.0f);

    In Update:
    ---------------
    - Process keyboard inputs (compare animation time of a note with when the button was pressed)
    - Calculate audio time, float audioTime = music.timeSamples / 44100.0f * 1000.0f;
    - Check array of inactive notes and if audioTime > the hit time of the note - 500ms (the animation time), mark as active and begin animating the note.
    - Add inactive note to the active note array for keyboard processing.

    This was identical logic to what we used for the cocos2d iPhone version of the game and we never had any issues. We also used the same song data files.

    The problem:
    --------------------
    The note spawn times do not seem to line up with the audio time. However, this does not appear to be a latency issue because the notes are actually spawning EARLIER than anticipated, by around 200ms. This strikes me as quite odd. If I manually apply a 200ms delay to the spawn time, everything lines up perfect. I have tested with 2 different songs for our game and the offset is consistent.

    Things I have tried:
    -----------------------------
    - Tried using .mp3, .ogg and .wav files. All exhibit the same issue.
    - Tried various compression options and also disabling all compression for the audio file.
    - Verifying track length is identical in the editor to what I see in Audacity.
    - Quadruple checking my logic and how I animate the circles. Creating a timer with dt and also using stopwatch to verify the animations are taking around 500ms to complete (and not going faster).
    - Using FMOD as an alternative, also exhibiting the same issue.
    - In project settings, setting the system sample rate as well as all sound devices on my computer to 44100.
    - Trying different DSP buffer sizes (Good latency, best latency etc)
    - Trying outside of the editor with a Windows build. Experimenting with various quality settings.
    - Using audio.time, audio.timeSamples, delta time and stopwatch after song play to try and check audio time.

    Conclusion / Questions:
    ------------------------------------
    I am honestly not sure what is going on at this point. I have sunk about 40 hours into this issue and although I could simply apply the 200ms offset, I really want to understand what is going on, or if I have made a really stupid logic mistake.

    1.) When importing a wav file with no compression, I am noticing the file size is still being altered. For example, in the inspector I see "Original Size: 22.2mb, Imported Size: 7.1mb". Why is this happening? Is there any way to disable this?

    2.) Does Unity strip any audio information from files upon import or play that could affect the overall length of the track?

    3.) What other things could I check?

    Thank you for your time. I would greatly appreciate any help.