Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Loading Streaming Audio from the Resources Folder?

Discussion in 'Editor & General Support' started by NoBullIntentions_P, Dec 16, 2012.

  1. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    I have a lot of audio which I need to load in each of my scenes. Each scene can easily contain a few hundred moderate length audio files, but it only ever needs one or two at a time. Precisely which clips are needed is not possible to predetermine, so I've dismissed AssetBundles as impractical. Resources.Load would be ideal, but it seems to load them into memory even if they're marked as streaming.

    So at the moment, I'm doing:

    Code (csharp):
    1.  
    2. VoiceClip = Resources.Load(Voice) as AudioClip;
    3. Resources.UnloadAsset(VoiceClip);
    4.  
    Amazingly, this actually works. Unloading the asset doesn't make it unusable - it plays correctly when I attempt to play it, and the maximum audio memory usage is never higher than the size of the largest single clip.

    This cannot be the most efficient way of doing this though, surely? There has got to be more efficient way to dynamically load streaming audio. If I referenced the audioclip in the inspector, I imagine that would achieve this precisely, so it must be possible to achieve the same effect. It's just that I can't reference the clips as they're determined at runtime.
     
  2. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
  3. janm_unity3d

    janm_unity3d

    Unity Technologies

    Joined:
    Jun 12, 2012
    Posts:
    36
    Yes, this is currently the way this has to be done. The reason is that AudioClips currently open and start prebuffering the stream as soon as they are created -- that way the clips are able to play immediately without any delay and all relevant properties can be queried. This is actually a behavior we would like to change at some point, but it requires that we store more information in the asset, so that all the properties that can currently be queried still return correct information while the sound is still loading, and we may also need to add a callback or use AudioClip.isReadyToPlay to return true when loading has finished. The hard bit is obviously to make this change without breaking the current behavior wrt the scripting API.
    As far as I know it's not possible to load resources in the background (correct me if I'm wrong), but there are some alternatives to this that you could try out:
    1) Use the WWW object to load the sound from the local asset folder ("file://") then use GetAudioClip(false, true) to open it as a streamed audio clip (this is what the last argument does. You may also experiment with setting the last argument to false, in that case the WWW will load the file completely and can be played when AudioClip.isReadyToPlay is true, but the actual loading will still happen in the background.
    2) Wrap your AudioClip into an asset bundle, as these can be loaded asynchroneously. Requires a bit more work and some scripting, but may be worth it anyway, if you need to process a large quantity of audio files.
     
  4. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    Thanks for the reply. It would definitely be nice if the default behaviour was changed, but I can see that it would be difficult to change without breaking anything. As I know well, API changes which break existing code are incredibly frustrating, so I can see that you guys need to tread lightly with this.

    The two suggestions seem promising. I didn't really think of using asset bundles on a 1-to-1 basis with clips, but that does sound doable, since I can automate things with an editor script. The WWW object and the streaming assets folder sounds even better, since it would probably require no changes to the data I've already created. I'll give them both a go and see whether either one looks as though it suits my needs perfectly.

    Thanks very much for the assistance.
     
  5. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    I've tried implementing suggestion #1 but I can't set streaming to true. If I set streaming to true, the start of the sound is skipped and we begin a fraction of a second into the audio. If I set the streaming flag to false, it's fine. I am, of course, waiting for isReadyToPlay to be true. I've tried both OGG and WAV, but the results are always the same. I tested to see whether isReadyToPlay was returning true too soon, but it's irrelevant. No matter how long I wait, setting the Streaming flag to true always causes the audio to skip the first half a second or so of the clip. This seems to be a bug in Unity 3.5? Is it solved in 4? I see nothing in the release notes to suggest that it is.
     
    Last edited: Feb 3, 2013
  6. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    I've also had time to try your other suggestion now, but I'm stumped. I don't see any way to access the content of the asset bundle before it's fully loaded. If I can't access the content of the bundle before it's fully loaded, I can't see any point in it. The whole point of streaming audio is to begin playing it before it's fully loaded.
     
  7. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    I've submitted a bug report with the streaming audio clip issue. In response, I got an email telling me that the list of bugs is huge and it probably won't be fixed any time soon. If I'd like to pay $1,500 for a three month premium support subscription, they'll take a look at and make sure it's really a bug as a matter of priority. As far as I can see, I wouldn't actually stand any better chance of getting it fixed, despite that $1,500 investment.

    Is this now official Unity policy? Call me odd, but most people would be ashamed to admit their bug list was so large it was unmanageable. Actually turning that into a pitch for why I should pay another $1,500 for you to determine whether one of your major features you've been advertising for age is actually completely broken took me by surprise.

    I emailed back for clarification if I had understood correctly, but have received no reply. So I guess you do have at least a little shame.

    EDIT: In case it matters, the case number is 522051.
     
    Last edited: Feb 16, 2013
  8. Fattie

    Fattie

    Joined:
    Jul 5, 2012
    Posts:
    476
    I totally agree with Phila.

    This is a huge, gargantuan problem in Unity, for any project other than a toy project.

    Consider sound effects. Every game has this. In Resources/SFX you'll have say 40 folders (the different category of sound effects)

    In each of those folders you'll have a random number of SFX for that category (whether 1, 5, 10 etc)

    YES -- you can tediously make an AudioSource for every single one of these in the Editor (!)

    And manually drag every one in the Editor

    THEN - you can correctly use stream-from-disc

    But just as Phila says if you try to simply avoid using the childish drag-in-editor situation ....... you cannot.

    Loading of audio assets is broken because it doesn't understand stream-from-disk if you load them from a Resources folder.

    I wish there was a fix for this.

    it's a catastrophe.
     
  9. Games-Foundry

    Games-Foundry

    Joined:
    May 19, 2011
    Posts:
    632
    Streaming audio from asset bundles is not possible, only loading the entire file out of the bundle into memory and decompressing first. The code for streaming .ogg files from the StreamingAssets folder is included a few posts below.
     
    Last edited: Feb 23, 2013
  10. Rob-Mills-Audio

    Rob-Mills-Audio

    Joined:
    Sep 3, 2012
    Posts:
    46

    +1

    It's a real shame.

    Unfortunately Unity won't work with fmod for integration. They really should. This form of audio engine is primitive and becomes more primitive every day.
     
  11. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    Many thanks for the suggestion, Games Foundry. I've attempted to implement it, and it seems to be working fine, except that there is a huge pause before the first WWW request returns true for isDone when I build to a standalone. From the editor, it works fine. I've timed each section of the process: Loading the data, asynchronously loading the bundle and then streaming the audio from the bundle.

    From the editor, the times are something like

    0.05 seconds
    0.2 seconds
    0.05 seconds

    From the Standalone Windows build, the times are something like:

    2.2 seconds
    0.2 seconds
    0.05 seconds

    I added code to check the progress each frame. Progress is zero every single frame until the frame it loads, when it ticks to one. Which seems to suggest that loading is not really taking place for the first two seconds. Very odd.

    Note that this ONLY seems to happen on the first request. So there is a two second gap before the first line of a conversation begins playing, but all subsequent lines load at normal speed. Am I doing something wrong? Do I have to do something to initialize use of the WWW class or something?
     
    Last edited: Feb 23, 2013
  12. Games-Foundry

    Games-Foundry

    Joined:
    May 19, 2011
    Posts:
    632
    I believe we witnessed this too. As a quick fix I preload all the AssetBundles but I haven't gotten around to looking at the memory impact of doing this. I'm assuming that if all the audio files in the bundle are set to streaming, pre-loading the Asset Bundle should have minimal memory impact. But then you know what they say about assumptions. I've just been working on our AssetBundle code so it's a perfect time for me to run some tests and post the results.

    UPDATE 1:
    So it looks like using this method ends up loading the entire audio file into memory and decompressing, even though it's set as "stream from disc" and in ogg format within the bundle. Audio Memory jumps from a few hundred K to 29MB ( the size of a single uncompressed soundtrack file ) from an ogg file of approx 7MB. This explains the delay while the background thread loads and decompresses and entire file. Next I'll try www.audioClip to see if that works.

    UPDATE 2:
    So I tried not using asset bundles and instead adding a .ogg file marked as "stream from disc" to the StreamingAssets folder and used www.audioClip. Alas, it still loads the entire file into memory.

    UPDATE 3:
    Tried audioSource.clip = www.GetAudioClip ( false, true ); and that does stream, taking 130ms before actually playing and adding about 350K to memory. However, it does mean your .ogg file is available to everyone in the StreamingAssets folder. It doesn't look like it's possible to stream from asset bundles so I've updated the original post.

    I can also confirm the bug in clip.length for streaming clips, and audioSource.isPlaying not reseting to false when the clip has finished playing. The temporary workaround solution is to pass in the lengths manually. Strangely, pausing the editor causes isPlaying to reset.

    Code (csharp):
    1.  
    2.         if ( Application.platform == RuntimePlatform.IPhonePlayer )
    3.             filePath = "file:///" + Application.dataPath + "/Raw/";
    4.         else if ( Application.platform == RuntimePlatform.OSXPlayer )
    5.             filePath = "file:///" + Application.dataPath + "/Data/StreamingAssets/";
    6.         else
    7.             filePath = "file:///" + Application.dataPath + "/StreamingAssets/";
    8.  
    9.         // The above can now be replaced with Application.streamingAssetsPath ( old code )
    10.        
    11.         filePath += oggFilename; // including .ogg file extension
    12.  
    13.         WWW www = new WWW ( filePath );
    14.         yield return www;
    15.            
    16.         if ( www.error != null )
    17.             Debug.Log ( www.error );
    18.            
    19.         audioSource.clip = www.GetAudioClip ( false, true );
    20.        
    21.                
    22.         if ( audioSource.clip != null )
    23.         {
    24.             if ( audioSource.clip.isReadyToPlay )
    25.             {
    26.             ... continuation of original code
    27.  
    28.  
     
    Last edited: Feb 23, 2013
  13. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    I was actually going to ask you whether loading assetbundles was actually streaming or not once I had it working properly, so thanks for answer that. Now that you've gone back to GetAudioClip(false, true) though, isn't that exactly what I posted above? When I use GetAudioClip(false, true) the beginning of the sound gets cut off.
     
  14. Games-Foundry

    Games-Foundry

    Joined:
    May 19, 2011
    Posts:
    632
    We've encountered a streaming sound having the first bit of it being skipped in Unity 4.0 as you described. Happens during cut scene dialogue for us, when streaming the 30th sentence audio file in the dialogue ( we play a sentence, then clean up both the audio clip and the www about 0.5s after; profiler shows everything behaving as expected ). The bug happens every time at exactly the same spot, no idea why. In our case we are able to work around it by combining audio files.
     
    Last edited: Feb 25, 2013
  15. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    I've received an email from support telling me that the bug has been confirmed and telling me that they are now passing it over to be fixed. I don't know if it's going to be a fix for 3.x, 4.x or both, but it's definitely good news.
     
  16. wickedworx

    wickedworx

    Joined:
    Mar 13, 2013
    Posts:
    54
    Resources.Load on an AudioClip currently seems to always load/decompress the whole thing in to memory. The only work around I've found so far is to create a prefab for each AudioClip (which just contains a script which references the audio clip), and do the Resources.Load on this.
    I've described it in this other thread, here !
     
  17. Lad-Ty

    Lad-Ty

    Joined:
    May 7, 2013
    Posts:
    61
    Hi,

    I don't quite understand when is the audio streamed and when not (provided I have selected StreamFromDisc as LoadType on my music).

    It might not be entirely according to the thread title since I'm not using Resources folder now, but I guess you here will have the most info about it so I'll ask here, hope it's ok.

    Let's say I have a folder Music, with some, well, music... Pretty big files, pretty long, StreamFromDisc on them. In scene, I have an 'Ambient' object (simple object with AudioSource on it). If I simply assign my music file to this AudioSource, will it stream the music without loading it whole into memory?

    Second case - I want to change the ambient music. I have some trigger for example, with script attached. To this script, I assign another music from the inspector (to an AudioClip field). When I enter the trigger, I assign this AudioClip to the audio.clip in my 'Ambient' object. In this case, when is the music loaded (StreamFromDisc selected, of course)? When the scene loads, the trigger is entered (-> the clip is assigned to the audioSource), or is it streamed without previous loading as it should be?

    Thanks, I'm really confused about this loading behavior since many people write about bad loading even with StreamFromDisc selected, I'm just not sure if it's always weired or only while using Resources.Load etc? :(

    Edit: Btw I use Unity 3.5 Indie (+Android/iOS) if it matters
     
    Last edited: May 10, 2013
  18. Rob-Mills-Audio

    Rob-Mills-Audio

    Joined:
    Sep 3, 2012
    Posts:
    46
    (i'm not a pro with unity and I don't know this for a fact, this is just my opinion)

    when you load a scene, the objects within the scene that have audio clips attached to them will decide whether or not to load the audio to ram. if the setting is 'stream from disk', no loading will occur until the moment of playback.

    So when you run a script (your trigger) to change the audio.clip field, no audio is loaded, still, until the very moment you wish to PLAY that audio file, then the audio is streamed.

    as long as you work with gameobjects that have the playback setting set to streamfromdisk, you can change that audio.clip as many times as you wish, and nothing will pass through the ram / cpu until you push play on that object.

    this is how it typically works - but again, i'm no Unity guru, and the unity audio engine is quite terrible.
     
  19. meat5000

    meat5000

    Joined:
    Mar 5, 2013
    Posts:
    118
  20. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    It's such a shame that this issue has been ignored for so long. I've contacted support several times but the only responses I've had so far were to confirm that they haven't fixed it and an offer that I could pay $1,500 USD to have them confirm it as a bug. (But not fix it). They have subsequently confirmed it as a bug despite me not paying the ransom. I tried contacting Graham, since he seems to be the main man on these issues, but I've had no reply at all from him.

    All hope is not lost, however. Unity were announcing on Facebook that the IssueTracker votes are used to prioritize which bugs they fix. This seems like a very unwise way of prioritizing your resources, but at least now I know why the issue was being ignored. I didn't even know I *could* vote for a bugfix, so it was on zero votes. A few kind Unity users on FB have already added some of their votes to it, so if you would like to get this issue resolved, please add all the votes you can spare here :

    http://issuetracker.unity3d.com/issues/streaming-audio-is-broken

    Maybe, with luck, we can get something done about this. Creating 10,000 prefabs doesn't seem like a solution to me, as kind as it was of wickedworx to share that solution.
     
  21. Brenden-Frank

    Brenden-Frank

    Joined:
    Aug 5, 2012
    Posts:
    110
    Just ran into this very specific issue. I'm porting to Xbox right now so memory is tight!
     
  22. NoBullIntentions_P

    NoBullIntentions_P

    Joined:
    Jul 2, 2012
    Posts:
    314
    I'm afraid I never found a satisfactory workaround for this problem. You can add each separate audio file to a bundle and stream the bundle into memory, but you can't begin playing it until the entire bundle is loaded. So it would be entirely worthless for long streams. I tried contacting support again and I've tried to contact Graham personally, but I just get ignored. To be fair, if the decision makers have decided that this isn't going to be fixed, there isn't anything they can say which wouldn't just inflame the situation.

    Given that Unity are due to announce that Unity 5.0 will come with a vastly redesigned audio system tomorrow, my guess is that - at some point - we'll be told to pay for an upgrade to 5.0 if we want the features they've been advertising since 3.x.
     
    Last edited: Mar 17, 2014
  23. jerotas

    jerotas

    Joined:
    Sep 4, 2011
    Posts:
    5,574
    I tried the WWW method mentioned on the previous page to load Resource files acynchronously, and I could play them fine. I couldn't figure out how to release the memory after I played them though, which makes it useless. I couldn't use Resources.UnloadAsset, it fails. Any ideas?
     
  24. Fattie

    Fattie

    Joined:
    Jul 5, 2012
    Posts:
    476
    Does anyone know if this has been

    fixed by Unity now that it is 2016?



     
  25. Kagedmonkey40

    Kagedmonkey40

    Joined:
    Feb 2, 2016
    Posts:
    6
    Does anyone know how to get this www method to work for Android and iOS. It seems to work fine on PC but on nothing else. PArticularly how to setup the url for Android to be used with WWW for .ogg files storede in the streamingassets folder.
     
  26. chrismarch

    chrismarch

    Joined:
    Jul 24, 2013
    Posts:
    473
    Each streaming AudioClip seems to allocate around 100KB (a big problem when you have thousands of clips to choose from) even if I leave the preload option unchecked. This seems broken. What am I missing?
     
  27. Develax

    Develax

    Joined:
    Nov 14, 2017
    Posts:
    67
    You don't have to do it tediously, just use OnValidate() method in UNITY_EDITOR mode:

    Code (CSharp):
    1. [SerializeField]
    2. private AudioClip[] _audioClips;
    3.  
    4. #if UNITY_EDITOR
    5.  
    6. private void OnValidate()
    7. {
    8.     _audioClips = Resources.LoadAll<AudioClip>(Path);
    9. }
    10.  
    11. #endif
    since _audioClips are loaded in the EDITOR mode they will be used in runtime the way as if they were created manually.