Question Help Wanted - Training with an Animation

Hi, I have been working in a project where u have to hit a ball with a bat (sort of like baseball). I have been training with some animations to make to learn the model to when to swing its bat to hit the ball. The way I called this was by using the 'Invoke' function in unity.

It has been working fine until recently, I have got a new animation and I tried to train the agent to learn the timing like before, and then this issue came up.
Issue is that even though that data suggests it train successfully while achieving the rewards, when I run the inference with the trained model it doesn't work as its supposed to, doesn't get the same cumulative rewards as its training.

After a while invigilating I have found that there is a delay in playing the animation in training session, so the invoking delay becomes differ than when the model in inference or heuristic modes.. How to fix this? Please help!!!