Question How to catch a ball thrown by a real person

I'm building an app (initially targeted at Oculus Quest) that will mix real video with user interaction. A simplified version of what I'm trying to build is to have a recorded 360 video of a real person tossing a ball at you. The player will need to move their hands or controller to catch the ball. The key point is that the ball and the person throwing the ball need to be real video (not computer generated).

I am new to VR and Unity. I have a 360 camera, an Oculus Quest 2, and have done some of the tutorials on Unity. I also have a fair amount of programming experience in C++, C#, Python, and Javascript.

I have seen tutorial on how to include 360 video in Unity. I have seen other tutorials about adding hot spots to the 360 video, but I think I will need something more complicated as the ball will be moving during my video.

Can anyone suggest how I can catch the ball?

Thank you.

 

if its 2d video, if I understand it right, then you'd have to create a 3d ball in unity and time it to move with the ball in the video. so fake a 3d ball over the 2d ball, once it leaves the persons hand it would be good if you could delete it from the video and have the 3d ball take over. make it appear in the same location of the video just as you make the 2d ball disappear...

 

I like your suggestions Vanz.

It's a 3d video.

Do you have any ideas on what tools to use to determine the ball location by looking at the 3d video?

Also, what's the process/tool/tutorial to insert a virtual ball into a real 3d video?

Thanks again. I'm still learning.