Changing skybox at run time

Is there a way to properly change the scene skybox at run time? It looks like in URP, changing the skybox material (found in the lighting menu) does not update the diffuse or specular GI meaning that the visual environment and the way objects are lit do not match.
For the specular side of things, I suppose I could set the reflection source to Custom and pass it my cubemap, but I don't believe there is a similar solution for the diffuse lighting.


Any ideas?

Many thanks