What's the real skinny on colliders without rigidbodies?

I created a quick prototype of my next title and used Unity's CharacterController. This has a collider and no rigidbody. My game will be networked and have some more advanced requirements than the default CharacterController, so I want to have full control of the characters and objects, with motion not physics based, so decided I need to roll my own character controller. I have something working quite nicely with a capsule collider and no rigidbody.

Just reading around and it's unclear to me whether not missing a RB is okay or not. I do not want any Unity physics resolution and am handling that myself - I just need collisions. Plenty of advice is saying use RB's and set them as Kinematic.

So what's actually best practice in this case? Set everything to kinematic RBs and move the RB? Or don't use RBs, just attach colliders, and move the transform, as my current prototype is doing?