Question about Animation Overrides

So here's a rundown of my situation and the issues I've ran into.

I'm going to have a large amount of entities that all use the same basic complex logic for animation, but with differing animations for each of them. Because of this, I thought to use the Animation Override Controller so I could have a simple base animation state machine, and just swap in and out the animations based on the specific entity. I plan to have a single prefab that all the entities are made from, and swap in the override on creation.

I can create overrides and set their base animator controller, and I can create animation assets and link them to that override, but I can't figure out how you're supposed to animate the prefab in these overridden animations. The animation assets by themselves don't seem to be "linked" to the prefab, so opening them by themselves I can't edit the animation. Selecting the prefab and opening the animation window just fills the animation window with the animations from the base animator controller, and I'm unable to edit the override animation.

I could set the animator controller of the prefab over to the override and then edit the animation and change it back afterwards, but that's pretty cumbersome and I can't help but feel there has to be an easier way to just... make an animation for an override.

Is there a way to just create an animation for an override without needing the override to be set on the prefab?

Or is the idea that you have a ton of different prefabs each with their own override controller and own set of animations?