VR - collision miss while onCollision works

hi,
I’m fairly new to unity and game development.
I have an issue where my sword in VR doesn’t
Collide(it passes through) object with static collider or any collider for that matter.

I figured it is because of the frames that the collision is being missed, but the oncollision function works and it Is being recognized .

When I use large colliders it collides even if I speed my hands, when it become smaller as hand sword size or column size it starts to miss.

I tried some solutions found on the web such as:
1. Adjusting the settings of physics in project settings
2. Adjusting time step in project settings
3. Adjusting on the rigid body the collision detection to continuous and other options there.
4. When colliding increased the collider through script and when exit made it go back to normal size - made unexpected results


Is there a way to add some code to the oncollision to enforce the physics? Or to manipulate the frames some how?

Tnx in advance.