2 different collisions 1 object?

I have objects that slowly follow to the character, and I need them to not move it if they collide with it. I got that just fine by making the player a kinematic rigidbody, but I also need the player to collide normally with the environment, like not being able to pass through walls.

So I want the objects to not affect the character's rigidbody, and the character to not affect other objects' rigidbodies (environment).

How should I do this?

 

You need a regular non-kinematic rigidbody in order to collide with the environment. As for the other objects not to affect the character, I can think on two solutions:

1. Use Physics.IgnoreCollision to prevent the objects to collide with the character, but manually detect the collisions with Physics.ComputePenetration and use the method's result for keeping the objects apart from the character.
2. Configure a very tiny mass for the objects so when they collide with the character they cause little to no effect.