How make particles run pass an rigibody?

My case is: character A attack character B by flamethrower, and B attack A by pistol

My current setting is:

A, B: many box colliders on trigger,(head, chest, foot, etc), no rigibody

Flamethrower: particle system with collision, script use OnParticleCollision()

bullet from pistol: box collider, rigibody

My current problem that is the bullet can hit the flame(particle) away, it doesn't make sense. I want the flame can ignore the bullet and hit B.