AudioClip.LoadAudioData returns true but clip remains unloaded

Crosspost from /Audio/ https://forum.unity3d.com/threads/a...eturns-true-but-clip-remains-unloaded.455000/

In our project we are having issues with certain AudioClips not being loaded when they should be (I.E when we want to use them).
My first suspect was the "Preload Audio Data" setting on the audio files not working properly as the problem only seemed to occur after we loaded the same scene a second time.
Therefore, I turned that setting off and added my own "Preloading" by calling LoadAudioData manually in objects Start() function. This caused the same symptoms with the exception that it now occured 100% of the time instead of working properly only the first time. The funny thing I noticed though was that LoadAudioData returned true although the LoadState was still set to "Unloaded". This only happens for some specific objects though (our main characters) and there are other objects that reference and use the same audio resources.

Has anyone got an idea at what could cause this behaviour?

TL;DR
What could cause AudioClip.LoadAudioData to return true but still leave the LoadState at "Unloaded"?

Unity Version: 5.4.2f2
Relevant audio clip data:
- Load in Background = false
- Load Type = Compressed in memory
- Preload Audio Data = false (in my test, true otherwise)
- Compression format: Vorbis