VRTK Grab parent object or child object

Hello,

I'm using VRTK (with Unity) for a personal project on the HTC Vive. I'm currently working on adding guns in my game.

Here is my issue:

I have a rifle object in my game which is made out of several parts (children): grip, scope, magazine, etc.

I'd like to be able to grab the whole rifle from any part (child), except the magazine: if I move the rifle, the magazine should stay attached and follow along. Besides, I'd like to be able to grab the magazine independently of the rifle and detach it to simulate a reloading mechanism.

Currently I can grab the rifle, but when I try to grab the magazine, it grabs the rifle instead because the magazine is a child of the rifle.

Here's what I have:

Rifle (gameobject):

Scripts:

- VRTK_InteractableObject (IsGrabbable and IsUsable are both on)
- VRTK_FixedJointGrabAttach​

Children (gameobjects):

- Grip
- Scope
- Magazine
​

Scripts:

- VRTK_InteractableObject (IsGrabble is on)
- VRTK_TrackObjectGrabAttach​

I know that VRTK_InteractableObject will react upon interactions with any colliders of the gameobject or its children's. Since the magazine is a child of the rifle, VRTK_InteractableObject on the rifle will react when I try to grab the magazine, hence preventing the magazine's own VRTK_InteractableObject from triggering.
I've tried using the "Ignored Colliders" property, but the problem is that my magazines are loaded dynamically in the game, from a separate prefab: the colliders that I could provide would not match, so this is not a valid solution.

I'd like to know if there's any way I could make this work without involving too much scripting, as I would like to keep things as clean as possible.

My environment:
- Unity 2017.1.2f1 64-bits
- VRTK 3.2.1
- SteamVR 1.2.3

Thank you all in advance. Cheers!