Lerp jitter in rhythm game

Hi all,

I've been trying all sorts to get this figured out for my VR game: basically I have some objects moving towards the player in time to the music (smoothly by intention, so reaching the player on a beat).

For my Vector3.Lerp, I have a spawn point, and end point and a "t" value.

Code (CSharp):

1.  
2.  
3. songPosInBeats = Spawner.Instance.songPosInBeats;
4.  
5. t = (BeatsShownInAdvance - (beatOfThisNote - songPosInBeats)) / BeatsShownInAdvance * speedModifier;
6.  
7. transform.position = Vector3.Lerp(spawnPos, removePos, (float)t);
8.  
9. if (transform.position == removePos) { gameObject.SetActive(false);  }

beatOfThisNote simply stores the audiosettings.dsptime value of the note expressed in beats (its position in the song), and similarly songpos is the place where the player is standing. BeatsShownInAdvance is an arbitrary value (in my case 4), which expresses how many beats the note must travel through from spawn to remove position.

So "t" expresses the percentage of how far along those 4 beats our note should be, considering its value in beats and considering the current songposition.

It all seems to work perfectly in time, however I get a mild but distracting choppiness, which I have not been able to get rid of.

I have tried with no success:

1) lowering performance demands
2) changing floats to doubles for increased precision
3) increasing/decreasing timestep and putting the code in Fixed Update
4) implementing Smoothstep
5) putting code in coroutine and looping the coroutine at different looptimes (very high very low, no difference)
6) tried both with my Oculus Quest attached to Unity in PC mode, but also via build in standalone - same thing

So it doesn't seem to be dependent on high/low frame rate, nor on performance, I simply have run out of ideas.

The only thing I can think of is that I would need to somehow interpolate the interpolation to make it run more smoothly or perhaps it's due to the 72 Hz Cap of the Headset, but I don't believe so because Beat Saber works fine.

Any help is greatly appreciated!

 

Broadly speaking, choppiness should be a result of one of the following:

1. The object's position on the screen isn't being updated often enough (e.g. your framerate is too low, or you're running this code less often than Update)
2. Your calculation of the new position is truncating your precision somewhere (e.g. you performed an integer division instead of a floating-point division)
3. The input data into your calculation is, itself, "choppy".

Your posted code doesn't contain enough detail to rule out any of those options, but it sounds like you have already investigated the first two options, so I'd suggest you look into the third. How often is "songPosInBeats" updated, how is it calculated, how much precision does it have?

 

Hey there, thanks for answering!

Yes, I believe you're right about that, but I would have no clue as to how to solve it:

I found a somewhat strange workaround for now: basically I keep these spheres invisible and just record them as transforms, then I create a series of "overspheres" which follow them this time with a lerp referring to time (with t multiplied by deltaTime).

basically as I can't find any way to eliminate the choppiness, I just get a second layer of objects to follow as if they were a set of smooth cameras: works better than I expected, although it's not the prettiest code in the world.

Regarding your question, here's how some of the rest is done in my code:

Code (CSharp):

1.  
2.  
3. void update()
4. { songPosInBeats = ((double)AudioSettings.dspTime - timeAtStartDSP) / secondsPerBeat;
5.  
6.             if (nextIndex < notesBeatPosList.Count && notesBeatPosList[nextIndex]< songPosInBeats + beatsShownInAdvance )
7.             {
8.                 beatOfThisNote = notesBeatPosList[nextIndex];                                                              
9.                 int randomColumn = Random.Range(0, 7);                                                  
10.                 int randomRow = Random.Range(0, 6);                                                      
11.                 position = columns[randomColumn].instances[randomRow].position;                          
12.  
13.                 objectPooler.Instance.SpawnFromPool("sphere", position);
14.              
15.                 nextIndex++;                                  }  

 

If you fixed your problem by calculating position based on time instead of based on songPosInBeats, that pretty conclusively means that your problem choppiness was in songPosInBeats.

If you use songPosInBeats for anything else in your game, it may be worth some effort to track down the root problem so that it doesn't affect anything else.

 

While trying to lookup

dspTime

I stumbled over the following old thread, which suggests that dspTIme does not necessarily increase every frame. If that is still the case today, it could explain the jittering you're observing. The thread also has a solution.

https://forum.unity.com/threads/do-...e-is-can-someone-explain-how-it-works.402308/

 

You're both right, and although I was rather proud of my unorthodox ragtag solution (I've only been coding three months and it's not my main job so even little things excite me ) , that thread you found contained the solution, namely to create a self adjusting timer due to the fact that AudioSettings.dsptime sometimes gives out duplicate values for more than one frame.

To anyone facing the same problem, there is a nice little code for a metronome on that link, but I'll post how I implemented the solution below anyway (not sure if this is performance costly but it seems less expensive than following my transforms with an extra layer of game objects), besides it's pretty much a core aspect of the game:

Code (CSharp):

1.  
2. //set timer to dsp value in startSong method, don't try to update it in Update before applying the following code, obviously:
3.  
4. dspGetTime = AudioSettings.dspTime;
5.  
6. if (dspGetTime == AudioSettings.dspTime) { timer += Time.unscaledDeltaTime; }
7. else { timer = AudioSettings.dspTime; }
8.  
9. songPosInBeats = (timer - timeAtStartDSP) / secondsPerBeat;

I'm still getting a super mild stutter on every beat, which I don't get, but nothing compared to before!

Thank you guys greatly, your help has been really much appreciated!

 

sorry that's complete rubbish what I just wrote... the code works however it's only ever applying unscaled time, however if you put the

dspGetTime = AudioSettings.dspTime;

below as it should be theoretically, it's choppy, so I don't know

 

I wonder if your start time is off, i.e. if you set your start time as

Code (CSharp):

1. timeAtStartDSP = AudioSettings.dspTime;

perhaps that should be

Code (CSharp):

1. timeAtStartDSP = timer;

instead?