Question Ignore physics between 2 objects, but do not ignore their collision

I have 2 gameobject, one is a spinning stick, and another one is a bullet.

the bullet has collider 2D, and the stick has collider 2D and rigidBody 2D

the spinning stick should spin when a bullet hits it.
I want to carefully choose the amout of spin with AddTorque(x). when I detect a colision with OnCollisionEnter2D()

the problem is that when a bullet hits the stick. the physics does its thing, and it moves/spins it. but I don't want it to happen because I want to do it by myself with code to get the exact amount of spin.

I cannot use the collision matrix because if I disable the collision between them, then I cannot use OnCollisionEnter2D().

So how do I detect the collision between them without moving the stick?