Question Isometry, problem with perspective and sorting axis with different floors

So, I have a problem regarding isometry in Unity : I am working on a project in Isometric perspective that looks like this :



Not only there is perspective and depth thanks to the Y camera sorting axis, but there are also floors, so the Z camera sorting axis is also at play. The player character can move from floor to floors (there is more than 1 floor) by changing his z during play via triggers. I realised that there was a problem regarding the second floor however, On this screen shot and the previous one we clearly see that the player goes behind and in front of the bloc Same goes for the bloc of the second floor, the player is displayed correctly






But once the player moves one floor, the problems start in this screenshot we see the player being displayed fine, behind the second floor and above the first one,



in this one however the player is displayed above the second floor when he should be behind it




If I lower the player z, he swims in the first floor like this



If I elevate the z of the second floor, the player is displayed behind it when on the floor like this



I messed with these values for a long time, and everytime it would solve something, another problem would arise. Every of my sprite pivot are at the base of the tile, my player's pivot is at his foots, everything is on the same layer, ground, first floor, second floor are all different tilemaps.
My Transparancy Sort Axis : X 0, Y 0.88, Z -0.39 ( 0.88 because that's the ratio my tiles are in for my grid dimension, -0.39 because that's the point at which my tiles started to stack each other when placed at the same place with Z )
My Grid settings : Cell Size : (2, .88, .88); Cell layout : Iso Z as Y; Cell Swizzle : XYZ
Does anyone have a clue as to how to solve that problem ? Surely there's a hole somewhere in my logic