Question Rhythm game sync issue

I want to make such a game where you play as a cube which bounces of the walls to some rhythm.

Yes, i can easily move my cube based on music playback position. This is essential rhythm game design idea.
But heres my main issue. Since deltatime between frames is not constant I cant precisly predict where my cube is actually going to be after some time. Therefore my cube is not going actually touch the wall.

Visualized what i mean for better understanding:

This is absolutely unacceptable!

So i tried to use FixedUpdate to presicly calculate what is going to happen and play music over this simulation.
But this idea obviously results in music desync over time (any game lag and music is ahead of image)

Help me with ideas please. I am lost

 

I'm not fully sure, as I haven't done anything like this, but I thought FixedUpdate and Time.fixedDeltaTime were "consistent", and just Time.deltaTime mainly had the variance due to CPU usage. I'm gonna have to re-learn the differences between Update and FixedUpdate all over again(as if I ever understood them), lol..

But one question, at what targetFrameRate are you running this logic on?

 

FixedUpdate, despite the name, is not a fixed interval. It instead plays catch up with Update, and can sometimes be called multiple times in a row, potentially giving fixedDeltaTime results of zero or near zero. It's only meant to be use for physics.

Rhythm games will ultimately require some degree of healthy fudging and allowance for error. Games like Guitar Hero are certainly generous with timing.

 

Uhh, yeah. I guess you are right about FixedTimeStep consistency. But who said that audioSource cant lag at all for some reason? I mean, running audio and image fully independently is not great idea in general as far as i understand

What is targetFrameRate? Is this some kind of Unity setting?

 

I guess in most rhythm games several pixel error while drawing something is unnoticeable as long as you taping in beat.
In my case though it will hurt your eyes

 

I am not sure I understood how the game works, does the cube bounce automaticaly to the rythm or I click to bounce the cube and you want to measure if it happened in the rythm?

 

FixedUpdate should work reliable but only if the fixed timestep value (default: 0.02s) matches the beat pattern of the music in question. Personally I would try tweaking that value higher such that it matches the fastest possible beat of any music you support. For example at 120 bpm (beats per minute) you have 2 beats per second thus a fixed timestep of 0.5 should work perfectly since you don‘t need any sub-beat updates, perhaps only to catch late input so a 0.25 timestep may be more responsive.

You simply record the actual time an input was made by the player and the next FixedUpdate calculates how far off that time was to the beat.

This implies that the music is started in a FixedUpdate too. And either the timestep is adjusted for each music‘s bpm or you use the smallest common denominator.

If you want this to be absolutely precise with music playback you‘d actually have to hook into music playback to get regular sync events as the music plays, typically on a background thread.I doubt this is possible with Unity both due to main thread API and no such audio events. A professional audio framework like Fmod may provide such functionality, which I would expect to be used by AAA rhythm games like Rockband.

 

Actually i planed to implement both things.
Like 2 mods:
Mode where you actually tap to the beats
And theatre mode where you just watch and enjoy

 

Unity's audio system works on a separate thread. If you want to perfectly synchronize a GameObject's position to your audio, you'll have to work with AudioSettings.dspTime, which is the current time of the audio system. What spiney said above still applies, the discrete and unpredictable nature of frame times means you'll have to do some fudging, but it should be possible to achieve a result that at least looks accurate.

 

My bad that i didnt say anything about it but I actually use midi files and DryWetMidi lib to parse them

 

From my experience, even on the default time-step you will get two FixedUpdate calls in a row somewhat regularly. These will definitely add up over time.

Don't compare your developer eyes to the eyes of the average player.

Also remember that you can - or should - separate logic and data from visuals. This is very much likely a case where what's happening on screen should just be a reflection of what goes on in the invisible data.

 

Code (CSharp):

1. // main script
2. void Awake()
3. {
4.    Application.targetFrameRate = 60; // set's max fps to 60
5. }

Yup I use it all the time, 60fps for PC applications and 30 or lower for others(mobile, console). So I was just thinking if you found your sweet-spot of your lowest frame drop(complicated computations), you can just that as max FPS, and code and music timing should be on par with each other. Key word.. (thinking)

 

it will desync with time anyway, one lag and everything is off. Also target framerate is just setting and can be ignored by device.

 

I don't see why that would be the case.... You are basing one thing off the position of another thing... How are you going out of sync over time, when the only thing your code does is sync constantly..

 

Assuming you use some audio delta time instead of standard frame rate then it would be in sync, but I assumed this is just standard delta time and audio runs completely separately, so in my understanding lag would delay the whole thing (or miss events or whatever), but the audio itself would still progress further (play). Obviously I am not talking about FixedUpdate.

 

AudioSettings.dspTime indeed the solution.
I guess this is the only one approach to sync music with logic. Works perfect for me.

However, i still dont know how to achieve pixel perfect wall collision without FixedUpdate...

Also some useful links that helped me a lot:
https://www.gamedeveloper.com/audio/coding-to-the-beat---under-the-hood-of-a-rhythm-game-in-unity