Question Setting a sprite to be behind *and* above other sprite

I am making a 2d platformer game in which the main character always has a sword on his back. The sword is moving accordingly to his body animation and its sprite can be changed to other swords by the player.
The animations of the character consists of whole Sprites from a sprite sheet and not from separate organs.
The problem is, the sword needs to be behind his front arm, but in front of his back arm.
I am straggling to find a simple solution to this. As for now, I can use sorting layers to only put the whole sword completely behind or in front of his whole body.
I could separate his back arm and put it on a lower layer, however, I will have to do so to all of my sprite sheets, and it will turn into a great mess.

I will be very thankful for any other ideas!