Addressables slow Android loading in Unity 2021.2.x and 2021.3.x

We run into issues using newer Unity version when trying to upgrade from Unity 2021.1.24f1 into Unity 2021.2.x or 2021.3x since any version of those caused issues.
We encountered strange loading errors but could work around those to some extent but the biggest issue came that loading of bundles was extremely slow. We use Addressables.LoadAssetAsync to load currently local bundles and it started to take maybe 20x times more than previously and this didn't happen using Unity 2021.1.24f1.
We use Addressables version is 0.20.0 and also tried other versions with same results.

We found a fix to work around this issue from here:
https://answers.unity.com/questions/1902940/unity-addressables-loadassetasync-is-too-slow.html

Basically just enabled 'Use UnityWebRequest for Local Asset Bundles' setting. But this does not seem like a correct fix since we have local bundles and this setting should be off and Unity 2021.1.24f1 worked fine without this setting enabled.

We still have some errors in Editor and can't run Simulate Groups.
But does anyone else have some strange issues with loading of bundles using newer Unity versions and are you forced to enable the same setting than had to?