Box collider position ignored

I have a problem with an enemy gameObject that has several child game objects.

The enemy (parent) gameobject has the rigid body and the box collider.
I detect collisions on my player character.

This works, except for one problem: the size and shape of the box collider on the enemy have no effect!
If I remove the box collider, there are no collisions.
If I add the box collider, the WHOLE parent object causes collisions, regardless of the shape of the box collider. I can make it very small or very high, it doesn't matter.

I just want a small part of the enemy to cause the collision. How can I achieve this? Shaping the box collider seems to have no effect.