Question Grabbed interactable is being dropped

I just noticed this. On the Quest 2 (a VR app) I obviously have a left and right controller. There are two "controllers" on each controller. One switches between normal and UI and the other between normal and teleport.

I can grab an interactable object with either hand but I just noticed that when holding that object if I push my teleport button the interactable falls out of my hand. The ability to be grabbed is in the object (e.g. a cup) but I'm not sure if there is a controller associated with that action such that swapping the controller makes the object think it is no longer being held. Does that make sense?

Is there a controller object that gets "possession" of the grabbed object? Do you think I can easily swap that when I change controller "modes"?