Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Official Audio system

Discussion in 'Open Projects' started by cirocontinisio, Oct 13, 2020.

  1. cirocontinisio

    cirocontinisio

    Unity Technologies

    Joined:
    Jun 20, 2016
    Posts:
    219
    Thread for discussing the Audio System in the game. From the card:

    A system that is able to play music and SFX on demand.
    Characters, effects, events, can ask the audio system to play a sound once, repeated, and specify requested volume, pitch, location, etc.
    The audio system could also have a few extras to do things like picking a random sound from a list and/or randomise pitch (for footsteps).
    It acts as a centralised system to control volume. The settings modify the volume on the audio system, and the audio system outputs music and sfx with the correct volume each time.

    Extra: start thinking of how this also integrates with the Timeline. Maybe just through a common AudioMixer?

    Card on the roadmap
     
    Last edited: Oct 13, 2020
  2. Fuzzeh

    Fuzzeh

    Joined:
    Feb 7, 2016
    Posts:
    1
    Hi There!

    A while back, I opened a simple PR related to Audio. This was before this card became available.

    It's quite simple right now, but I think it at least provides a foundation in which we can expand on. Right now, it offers the following functionality:
    • Play background music
    • Play sound effects on demand
    • Does not destroy the object on scene load. This allows the background music to continue to play as we transition scenes.
    • Static access to make it easier to use
    • Basic support for using AudioMixerGroup

    Here is the PR in case you are interested:

    https://github.com/UnityTechnologies/open-project-1/pull/54
     
    Last edited: Oct 14, 2020
  3. Neonage

    Neonage

    Joined:
    May 22, 2020
    Posts:
    171
    One question - Adaptive music?
     
    davejrodriguez likes this.
  4. RKS13D

    RKS13D

    Joined:
    May 20, 2019
    Posts:
    4
  5. RKS13D

    RKS13D

    Joined:
    May 20, 2019
    Posts:
    4
    What we can do is have 2 structs one for Theme and one for SFX
    Theme will be decided according to which area or the Scene we are in.
    And the SFX can be controlled according to which the events or functions called by the characters.
     
  6. RKS13D

    RKS13D

    Joined:
    May 20, 2019
    Posts:
    4
    Music design is done by the Unity Team as per Code Deck Roadman.
    But if they decide to use it then it may be great.
     
  7. davejrodriguez

    davejrodriguez

    Joined:
    Feb 5, 2013
    Posts:
    36
    Do you mean like having music in two or more parts, solo and full band (for example) and fading up/down full band/other parts based on current game intensity or some other parameter? Because that would be awesome.
     
    Neonage likes this.
  8. Neonage

    Neonage

    Joined:
    May 22, 2020
    Posts:
    171
    Yeah, we can have unique arrangements for each location or section of the game, and add new layers as the player is making progress. Also we can create seamless transitions between beach, town, forest or cage, all just by changing instruments of the main theme!

    For direct examples, check out this beautiful video by GMTK that covers this topic:


    And let me know if were going for it! I'd love to help with making arrangement tools, and maybe even with some chunks of music (I'm an amateur composer, but it would be hell lot of experience) :D
     
  9. Zantis

    Zantis

    Joined:
    Jun 20, 2016
    Posts:
    6
    Well, the idea is obviously pretty nice. I also don't see anything hindering us there. If i'm not mistaken, we should be able to easily achieve this by some triggers (invisible colliders), which would then change the music once the player enters a specific area (this would also work on any other condition, not just on collisions of course).

    I also don't see too much work from a programming side, as this should be solved by a simple fade-in fade-out of two soundtracks, right?

    So as much as programming goes the AudioManager only needs one method like transitionMusic(newSong), which would then fade-out the old song and fade-in the new song. Though this might need a bit of tweaking on where to start the new song.
     
  10. koirat

    koirat

    Joined:
    Jul 7, 2012
    Posts:
    493
    First of all create ScriptableObject that will hold reference to audio and never use bare AudioClip on your prefab/components. Only drag and drop AudioClipScriptableObject.
    Naturally fields on components should be of type IAudioClipScriptableObject.

    In my project I have something like this.

    Code (csharp):
    1.  
    2. public abstract class IAudioClipScriptableObject : ScriptableObject{
    3.  
    4.     public AudioClip GetAudioClip() {
    5.         return OnGetAudioClip();
    6.     }
    7.  
    8.     protected abstract AudioClip OnGetAudioClip();
    9. }
    10.  
    Code (csharp):
    1.  
    2. using UnityEngine;
    3. using System.Collections;
    4.  
    5. [CreateAssetMenuAttribute(menuName = "Game/AudioClipScriptableObject")]
    6. public class AudioClipScriptableObject : IAudioClipScriptableObject{
    7.     public AudioClip audioClip;
    8.  
    9.     protected override AudioClip OnGetAudioClip() {
    10.         return audioClip;
    11.     }
    12. }
    13.  
     
  11. Zantis

    Zantis

    Joined:
    Jun 20, 2016
    Posts:
    6
    Hey koirat,

    i would be interested in knowing

    a) what you are getting at, as i can not connect this to anything within this discussion
    b) why it is important to wrap the AudioClip in a ScriptableComponent

    ?
     
  12. koirat

    koirat

    Joined:
    Jul 7, 2012
    Posts:
    493
    a) the topic is about audio system, why it is not connected ?

    b) Flexibility.
    You can always swap the content (even runtime).
    You can extend ScriptableObject whenever needed, you could for example chose random AudioClip from a list.
    You can create multiple AudioClipScriptableObject that refer to the same AudioClip, than just change some of them in the future.


    In my project I put this class on a component that is going to spawn the sound. (some things are specific to my project).

    Code (csharp):
    1.  
    2. [Serializable]
    3. public class SoundInfoData{
    4.  
    5.     public IAudioClipScriptableObject audioClipObject;
    6.    public float baseVolume=1;
    7.     public float basePitch = 1;
    8.     public float SoundGeneratorRadius=-1;
    9.    public SoundPriority soundPriority;
    10.    public bool isMusic;
    11.    public bool ignorePause;
    12.    public bool ignoreTimeScale;
    13.    public AudioSourceType audioSourceType;
    14.     public GameObject customAudioSourcePrefab;
    15. }
    16.  
    17. //40 is arbitrary.
    18. public enum SoundPriority{
    19.  
    20.    Mandatory = 0, //music || ambient
    21.    VeryHigh = Mandatory + 40, // GUI || InfoSounds
    22.    High = VeryHigh + 40, //Events
    23.    Medium = High + 40, //Characters
    24.    Low = Medium + 40, //Background
    25.    VeryLow = Low + 40, //World
    26.  
    27. }
    28.  

    I definitely will change if possible
    Code (csharp):
    1.  
    2.    public AudioSourceType audioSourceType; // <--my enum
    3.     public GameObject customAudioSourcePrefab;
    4.  
    for a ScriptableObject in the future.

    And damn it i just realized I have got SoundGeneratorRadius in PascalCase. I'm so angry right now.
     
  13. Viurtun

    Viurtun

    Joined:
    Oct 4, 2014
    Posts:
    1
    I opened a PR with a sound system implementing a sound manager that has functionality to modify, save and load volume of mixer groups, manage a pool of audio sources wrapped in a custom class so its easily extendable while being relatively simple.

    Link to the PR:
    https://github.com/UnityTechnologies/open-project-1/pull/90
     
    davejrodriguez likes this.
  14. davejrodriguez

    davejrodriguez

    Joined:
    Feb 5, 2013
    Posts:
    36
    Hah I was just working on almost that exact implementation last night. First impression is that it looks solid! I will try to leave a detailed review over my lunch break.
     
  15. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,232
    In the video below I present my idea for the audio system. I'm going over how to integrate sounds and how to trigger those sounds through scripting, using the audio system prototype.

    It's a rather lengthy video, but I wanted to explain it in more detail to make sure it's fully understood. I hope my explanation communicates the idea well enough, but you can let me know.

    What I completely missed to mention in the video is that the AudioCue picks a random sound from the list that is then played. That's why the list exists in the first place.

    Please post constructive feedback what you think.



    Tagging Unity staff who seem active in the open projects forum @cirocontinisio @superpig @MileyUnity , hope you don't mind.

    PS: Sorry for bad English/accent :oops:
     
    davejrodriguez and Neonage like this.
  16. Neonage

    Neonage

    Joined:
    May 22, 2020
    Posts:
    171
    Hmm, the idea is solid, but I'm not sure whats the benefit of using prefab for that.
    May we have a link to your branch or PR?
     
  17. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,232
    I recorded a video where I try to explain it:


    There is no branch or PR yet. I wanted to get the overall workflow-design approved first. Then I go and make the code "compliant" and push it.
     
  18. koirat

    koirat

    Joined:
    Jul 7, 2012
    Posts:
    493
    Generally I'm using abstracted ScriptableObject as you can see in my example. (IAudioClipScriptableObject)
    The reason for this is that It gives me flexibility. For example I can create ScriptableObject that will use prefab.

    Additional when you use ScriptableObject you will be able to chose from a list when filing field on a component. With pure prefab you don't have this functionality.

    Now this IAudioClipScriptableObject only holds AudioCip.
    I got different components to play them, there you can have more settings, as AudioSourceType etc.
     
  19. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,232
    It's a great idea to use an interface.

    In order to use the interface to its actual potential, so that it doesn't matter if the implementation is a ScriptableObject or Component, requires to expose the IAudioClipScriptableObject interface to the Inspector, rather than the type that implements that interface.

    How do you make Unity 2019.4 serialize an interface? Using the SerializeReference attribute does expose it, but there is no UI:
    Code (CSharp):
    1.  
    2. public interface IAudioClipScriptableObject { }
    3.  
    4. public class NewBehaviourScript : MonoBehaviour
    5. {
    6.    [SerializeReference]
    7.     public IAudioClipScriptableObject audioThingy;
    Unity does show the field "audioThingy" in the Inspector, but it doesn't allow me to pick anything. Do I need to implement a custom property drawer for it?
    upload_2020-10-18_19-47-7.png

    I would like Unity to display a "Selection window" for ScriptableObjects that implement IAudioClipScriptableObject, as well as Prefabs that have components that implement the IAudioClipScriptableObject. How would I do this?


    Exposing the ScriptableObject that actually implements the interface, rather than using interface, unfortunately defeats the purpose to use an interface in the first place in my opinion.

    Because then it's again bound to a specific type, in this case the "abstract ScriptableObject". Means if I implement the IAudioClipScriptableObject in a Component instead of a ScriptableObject, I would need to expose the component to the Inspector. So we would end up with different types and the user needs to understand when and where to use what. That's not ideal in my opinion.
    Code (CSharp):
    1. public class NewBehaviourScript : MonoBehaviour
    2. {
    3.     public AudioClipScriptableObject audioSO;


    That's true. It reveals another weakness of the Unity editor that Unity Technologies should solve. The Open Projects initiative is a great opportunity for us to show Unity staff where the tech lacks. This is another prime example where functionality is missing.


    Can you provide examples of those different components that implement the IAudioClipScriptableObject interface? What is the AudioSourceType for?
     
    Last edited: Oct 18, 2020
  20. koirat

    koirat

    Joined:
    Jul 7, 2012
    Posts:
    493
    For a little more take a look at few previous posts.
    Since we cannot use interface we have to cheat it out with abstract class.

    Code (csharp):
    1.  
    2. [INDENT][LIST=1]
    3. [*]public abstract class IAudioClipScriptableObject : ScriptableObject{
    4. [*]
    5.  
    6. [*]    public AudioClip GetAudioClip() {
    7. [*]        return OnGetAudioClip();
    8. [*]    }
    9. [*]
    10.  
    11. [*]    protected abstract AudioClip OnGetAudioClip();
    12. [*]}
    13. [/LIST][/INDENT]
    14.  
    The system is more complicated, I got my SoundManager that plays SoundInfo object. Programmer almost never use AudioClip or AudioSource during programming. All this is hidden from him.

    I use for example this component to play sound.


    Sound Source - position of the sound.
    Audio Clip Object - Audio clip that will be played (abstracted)
    Base Volume - Base volume of the sound. This will let you simulate the volume of your sound. It will work like your AudioClip had lower volume if lower than 1.
    Base Pitch - Same as base volume but for pitch.
    Sound Generator Radius - As I said I'm using my own layer around unity3d audio system. In my system only sound inside a FOV of camera are played but sound got radius so if the sphere is intersecting with fov it will be played. -1 = infinite radius.
    Sound Priority - Basically audio source priority but turned into ranges and "enumed" (look my previous post).
    Is Music - is sound a music. Mostly for volume control/muting of music in game.
    Ignore Pause - Sound is played when pause is turned off.
    Ignore Time scale - Sound ignore scaling of time. (no sound distortion)
    Audio Source Type - this were predefined AudioSource configurations (like long range / short range sounds etc.). But I will change it for ScriptableObject in a short future.
    Custom Audio Source Prefab - when Audio Source Type set to custom will use this prefab to create AudioSource, but as i have said this will be changed for scriptable object in the future.
    Loop - loop
    Start At Random Position - sound start at random position, useful for some looping sounds, especially when started simultaneously.
    Enable Disable Aware - When component disabled/enabled should sound stop/start.
    Smooth Enable Disable - on enable disable sound should gradually increase/decrease it's volume (no cutting hard, no starting out of the blue) (useful for sound transitions)
    Smooth Dimnish/Amplify Speed - the speed of this volume change.
     
    Last edited: Oct 18, 2020
    Peter77 likes this.
  21. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,232
    Thanks again for your feedback. I now changed it to use ScriptableObjects instead of Prefabs. If that's what the community wants, then be it. :)

    I actually wished Unity staff would have shared their thoughts on this too, but you can't have everything.

    Please see video below for the update.

     
  22. koirat

    koirat

    Joined:
    Jul 7, 2012
    Posts:
    493
    Don't scare me like that !
    I just wanted to show how I do it and that it worked for me quite well.
    I'm definitely not the will of community.
     
    cirocontinisio and Peter77 like this.
  23. cirocontinisio

    cirocontinisio

    Unity Technologies

    Joined:
    Jun 20, 2016
    Posts:
    219
    That's generally true in life, but never despair :)

    I watched the video, it's great work!! I also agree with the others that SOs are the way to go to encapsulate AudioCues.
    In your last video, at 5:26, you say you can't define a template for the sound settings anymore because of SOs. That's not true, actually: you could have another type of SO just for that! And then link it inside your AudioCue, so that several AudioCue SO could refer to the same AudioSettingsTemplate SO. That would be a really powerful workflow, allowing us to control all settings from one centralised location (in fact I think we would only need 2-3 AudioSettingsTemplate SOs in the whole game).

    As the others suggested, I'd suggest too to open a PR and start exposing what you're working on. People could comment faster, and potentially make contributions to your PR too before it's merged.
     
  24. cirocontinisio

    cirocontinisio

    Unity Technologies

    Joined:
    Jun 20, 2016
    Posts:
    219
unityunity