Applying world rotation by setting a new LocalToWorld (Issues with simple CCD IK solver)

I'm implementing an IK solver and am having an issue setting the rotation of the entities that I am using to represent the bones.

Basically, it is storing the translation, rotation, and localToWorld before the job, calculating the world rotations of the entities that need to be set, and then applying them by setting a new localToWorld component on the entity like so:

new LocalToWorld { Value = float4x4.TRS(bones.Value, boneRotation.Value, new float3(1, 1, 1))

where bones is the world position of the entity, boneRotation is the new calculated rotation of the entity and then the last float3(1) is the scale.

The problem I am experiencing is that the bones are getting moved (translation is changing) when applying these rotations. I assume I am incorrectly applying the LocalToWorld.

Can it be applied in the way that I did, in that the world translation and rotation set in a float4x4 and applied to the entity?