Will Unity ever make a move to async/await/Task?

Currently I see IAsyncOperation is still heavily dependent on the yield-waiting paradigm.

The core of everything came from the coroutine runner system which utilize enumeration of a function and wait a frame or mutiple of frames on each return. This applies to IEnumerator Start() where there is an invisible hand enumerating the method.

Addressables is a new package that is coming out just when we will get the latest Roslyn update for Unity. Isn't this a good opportunity for transition? Reading the source code seems like Unity is very committed to the old yield waiting pattern already, so I would like to ask if Unity is considering moving away from yielding to more C# natural async/await at some point? Addressables could be the package most benefit from C# async becuase of its asynchronous-only concept.

A lot of changes will be required but I thought it might be better in the (very) long run. For example allowing declaration of `async Task Start()`. Also it will enable us to try catch + return value more reasonably where C# prevents us from using them with the enumeration.

I have tried to hack yield wait to interface with async before, that is yield return null infinite loop until the Task is done, then from that point I can go async/await with full try catch + easy return value support. But it goes against too many of Unity's yield-wait tools it is not worth it and I reverted the hack. (Mainly the test runner API is kinda yield-wait, but what I am testing is the Firebase Unity SDK which utilized Task) If it is in async pattern from the start we might be able to program much leaner code together with all these async asset loading from Addressables transformed into a linear and easy to follow code.

 

If you haven't taken a look at the new ECS + JobSystem, it's worth following. I would tend to doubt a Unity move to (managed-side) Tasks when they already have an integrated native scheduler.

 

Ahaha nice joke @5argon like I, He on ECS from first previews of course He know, talk just about replacing old coroutines

 

I have played with ECS for a while. I think Unity can keep using the JobSystem for worker thread operation but utilize `Task` for main thread operation emulating the yield coroutine system.

From what I understand the class Task is special because it is hardwired to the C# keyword async/await. The `await` can magically get the T of Task<T>, and so the easy async return value is possible.

I imagine something like each `await` in the `async Start()` can await in unit of frame like yield return YieldInstruction before (not sure if this is possible in design) and then we could await for Addressable.LoadAsset<T>, make that return Task<T> instead of IAsyncOperation<T>, and we can get T on the left side immediately for example.

 

await

is not magically linked to

Task<T>

.

you can make any type awaitable by implementing the awaiter pattern. that is what Task does.

therefore, you can (manually) make anything be awaitable. (UT should do this for Addressables / IAsyncOperation<T>).

for the legacy runtime, you can always

#if NET_4_6

the extras away

 

Although you can create async/await pattern by implementing GetAwaiter() pattern, it is very annoying and requires a lot of effort (ex, for minimizing overhead and GC). If Unity's asynchronous methods supports async/await pattern natively, many C# developers will be happy

Of course I know there are very good 3rd-party libraries (like UniRx), but in my opinion, these basic / infrastructural level API features should be implemented by the engine side.

 

Code (CSharp):

1. public static class IAsyncOperationExtensions {
2.  
3.   public static AsyncOperationAwaiter GetAwaiter(this IAsyncOperation operation) {
4.     return new AsyncOperationAwaiter(operation);
5.   }
6.  
7.   public static AsyncOperationAwaiter<T> GetAwaiter<T>(this IAsyncOperation<T> operation) where T : Object {
8.     return new AsyncOperationAwaiter<T>(operation);
9.   }
10.  
11.   public readonly struct AsyncOperationAwaiter : INotifyCompletion {
12.  
13.     readonly IAsyncOperation _operation;
14.  
15.     public AsyncOperationAwaiter(IAsyncOperation operation) {
16.       _operation = operation;
17.     }
18.  
19.     public bool IsCompleted => _operation.Status != AsyncOperationStatus.None;
20.  
21.     public void OnCompleted(Action continuation) => _operation.Completed += (op) => continuation?.Invoke();
22.  
23.     public object GetResult() => _operation.Result;
24.  
25.   }
26.  
27.   public readonly struct AsyncOperationAwaiter<T> : INotifyCompletion where T : Object {
28.  
29.     readonly IAsyncOperation<T> _operation;
30.  
31.     public AsyncOperationAwaiter(IAsyncOperation<T> operation) {
32.       _operation = operation;
33.     }
34.  
35.     public bool IsCompleted => _operation.Status != AsyncOperationStatus.None;
36.  
37.     public void OnCompleted(Action continuation) => _operation.Completed += (op) => continuation?.Invoke();
38.  
39.     public T GetResult() => _operation.Result;
40.  
41.   }
42.  
43. }
44.  
45. // This makes the following code compile without issue:
46. async void TestAsyncAddressables() {
47.   Sprite sprite = await assetReference.LoadAsset<Sprite>();
48. }
49.  

Here's a quick set of extension methods that makes any IAsyncOperation awaitable, and shouldn't allocate any more GC than before. Should also be reapplicable to the engine level AsyncOperation as well. This was adapted from this article about making virtually anything awaitable in C#. https://blogs.msdn.microsoft.com/pfxteam/2011/01/13/await-anything/. That said, I wish this was incorporated into the base library instead of needing to add this as an addendum that users of the code need to add themselves.

The code should compile with the C# 7.2 update in 2018.3b, but should also compile just fine in C# 6 if you remove the readonly struct designations.

It should be noted that Unity does NOT properly terminate running awaitables when exiting play mode and any incomplete operations will have their continuations executed upon exit.

I haven't tested this beyond checking for compilation, so if anyone finds any bugs with the above code, feel free to contribute or fix it up.

 

@Kichang-Kim that's why I also said that UT should do it for Addressables, and generally whatever new feature they are developing that is asyncronous.

making existing stuff awaitable (and async Task Start() for MonoBehaviour) can be lower priority (async Task Start() likely requires engine changes, making an awaitable for IEnumerator/Coroutine can be done in a package, altough it requires polling as they do not have a Completed callback)

@james7132 exactly that, except OnCompleted should call the continuation on SynchronizationContext.Current if set.
also, non-generic IAsyncOperation can return UnityEngine.Object

 

Get the UniRx, use UniRx.Async, done.

Code (CSharp):

1. //with a little bit of extension and sugar
2. async public static UniTask<T> ToUniTask<T>(this IAsyncOperation<T> operation) {
3.     await operation;
4.     return operation.Result;
5. }
6.  
7. //we get
8. async void MyStuff(object someKey) {
9.     var obj = await Addressables.LoadAsset<Object>(someKey).ToUniTask();
10. }
11.  
12. //or something more interesting like
13. async void CombineLevel(
14.     IAsyncOperation<Object>[] assetsToPreload, string someSceneName
15. ) {
16.     await UniTask.WhenAll(
17.         assetsToPreload.Select(operation => operation.ToUniTask())
18.     )
19.     .ContinueWith(async _ =>
20.         await Addressables.LoadScene(someSceneName);
21.     );
22.    
23.     foreach (Object asset in assetsToPreload.Select(operation => operation.Result)) {
24.         ///GameObject.Instantiate(asset); bla-blah
25.     }
26. }
27.  

You get the idea.

 

Hey, thank you for this. I just test a bit by replacing my `IAsyncOperation` callback hell with this one and it works. I was using an extension method to help me creating `ChainOperation` with an extension method `.ChainTo`.

Code (CSharp):

1. public static class ResourceManagementExtension
2. {
3.     /// <summary>
4.     /// Make a `ChainOperation` in a fluent way from any `IAsyncOperation`. Context and key are null.
5.     /// </summary>
6.     public static IAsyncOperation<Obj> ChainTo<Obj, Dep>(this IAsyncOperation<Dep> ops, System.Func<Dep, IAsyncOperation<Obj>> func)
7.     {
8.         return new ChainOperation<Obj, Dep>().Start(null, null, ops, func);
9.     }
10.  
11.     /// <summary>
12.     /// Make a `ChainOperation` in a fluent way from any IAsyncOperation. Context and key are null.
13.     /// This overload returns a `CompletedOperation` since the function does not return `IAsyncOperation`.
14.     /// </summary>
15.     public static IAsyncOperation<Obj> ChainTo<Obj, Dep>(this IAsyncOperation<Dep> ops, System.Func<Dep, Obj> func)
16.     {
17.         return new ChainOperation<Obj, Dep>().Start(null, null, ops, (dep) =>
18.         {
19.             return new CompletedOperation<Obj>().Start(null, null, func(dep));
20.         });
21.     }
22.  
23.     /// <summary>
24.     /// Make a `ChainOperation` in a fluent way from any IAsyncOperation. Context and key are null.
25.     /// This overload allows you to use a lambda that does not return anything, the `IAsyncOperation` will
26.     /// wait on the prior operation. Returns a `CompletedOperation`.
27.     /// </summary>
28.     public static IAsyncOperation<Dep> ChainTo<Dep>(this IAsyncOperation<Dep> ops, System.Action<Dep> func)
29.     {
30.         return new ChainOperation<Dep, Dep>().Start(null, null, ops, (dep) =>
31.         {
32.             func(dep);
33.             return new CompletedOperation<Dep>().Start(null, null, dep);
34.         });
35.     }
36. }
37.  

As an example this code load one addressable, in that one addressable there are multiple addressable addresses to be loaded, then load each one and collect the loaded data to the dictionary. With `ChainTo` we can get one IAsyncOperation that can wait for the whole sequence with one final result.

Code (CSharp):

1. private static IAsyncOperation<IList<SimfileInfo>> LoadAllBuiltInSimfiles()
2. {
3.     LoadProgress = Addressables.InitializationOperation.ChainTo(resourceLocator =>
4.     {
5.         return DatabaseLoader.SimfileList();
6.     }).ChainTo(simfileList =>
7.     {
8.         return Addressables.LoadAssets<SimfileInfo>(simfileList.builtInKeys, sis =>
9.         {
10.             LoadedSimfileInfo.Add(sis.Result.SDD.ReferenceName, sis.Result);
11.         }, Addressables.MergeMode.Union);
12.     }).ChainTo(addedToDict =>
13.     {
14.         UpdateSimfileDatabaseStats(addedToDict);
15.         return addedToDict;
16.     });
17.     return LoadProgress;
18. }

I attempted to linearize the code with the extension method but using async/await is much clearer. The version with extension method + lambda is in the end just a callback hell disguised as linear code. Too many symbols.

Code (CSharp):

1. private static async Task<IList<SimfileInfo>> LoadAllBuiltInSimfilesNEW()
2. {
3.     await Addressables.InitializationOperation;
4.     var simfileList = await DatabaseLoader.SimfileList();
5.     var addedToDict = await Addressables.LoadAssets<SimfileInfo>(simfileList.builtInKeys, sis =>
6.     {
7.         LoadedSimfileInfo.Add(sis.Result.SDD.ReferenceName, sis.Result);
8.     }, Addressables.MergeMode.Union);
9.     UpdateSimfileDatabaseStats(addedToDict);
10.     return addedToDict;
11. }

Similar number of lines but looks more like a human written sentence.

Also this code was activated by `RuntimeInitializeOnLoadMethod`, it works with async method signature nicely. That is if I put a log before and after the 2nd await, the last log will wait for the method first (and so, all of Addressables async data read)

Code (CSharp):

1.     [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.BeforeSceneLoad)]
2.     private static async void LoadAllBuiltInSimfileStartup()
3.     {
4.         await Addressables.Initialize();
5.         await LoadAllBuiltInSimfilesNEW();
6.     }

Great, it is available as a standalone not dependent on UniRx also. I will definitely check this out.

Also new knowledge, not just that we can await anything, with AsyncMethodBuilderAttribute we can actually async anything as well. (Other than Task, Task<T>, void)

Migrating to Addressables requires changing the code to async, and the change ripples throughout the source code (async is "viral", "contagious") . Instead of changing all return value of a once-synchronous to IAsyncOperation<T> I would like it to be more C# standard like async Task<T> so awaiting can get the .Result. I hope UT use this chance to make it a common practice.

 

I checked out UniRx.Async. It seems like just this I would be able to utilize async/await throughout Unity.

Code (CSharp):

1. IEnumerator Start() => StartTask().ToCoroutine();
2. async UniTask StartTask()
3. {
4.     ...

This applies to all my [UnityTest]

And also I can await many Unity things such as new WaitUntil, etc. so migration is not difficult. Combined with awaitable IAsyncOperation extension it should be pretty awesome.

 

I hear everything you are all saying. Consider...

- The awaitable/async patterns, as implemented by Microsoft, are versatile and flexible, but also complicated and not particularly approachable. It's my impression that Unity strives to be accessible to as wide an audience as possible, while providing depth for those with more coding experience. That's one of many things I love about it. An 'async-everywhere' approach is a barrier to entry, and why I feel the current (cool) Accessibles work Unity is doing, needs some work WRT ease-of-use, migration and general approachability.

- It is my hope that Unity will not encourage the use of .NET Threading/ThreadPool/Tasks and awaitable/asyncs. Those patterns/mechanisms are appropriate for applications/containers, which drive their own outer loops/pumps. Game engines drive their own outer loops. I wouldn't want .NET firing up OS threads to support any of that, as well.

- I encourage a mental shift and looking into ECS/Jobs, as patterns appropriate for game engines, where L2-friendly batching and data layouts facilitate significant performance gains, and OS/platform resource balancing is very important. And I actually haven't tinkered with ECS, yet. I'm in that camp on design-merit alone. It's that compelling, to me.

That's how I feel about it this morning. Respect. Toots

 

async/await can be used without relation to spawning a separated thread, it is a generic syntax for marking an asynchronous method which can be async in other sense depending on where the task completion source is. When implemented without firing a thread and make it so that it works like the current coroutine system I think it is fully replacable and a direct upgrade to the old enumerating-based coroutine system + imo easier to understand.

About accessibility of the current way, now I am comfortable but long time ago I remembered got confused by yield return null, yield return new WaitForSeconds, new WaitUntil, etc. and why is that an enumerator? Enumerate what? What is the IEnumerator I must return and why? Why I have to yield break to compile? The `return` in a coroutine function is a hack, you are not actually going to "return from a function" yet but it is how the enumerator works.

Instead, I think newbies will appreciate things like `await Wait.Frame(1)` which is true to the name. They will also don't have to hack return value by other way like passing lambda or assigning to an outer variable from inside the coroutine. The `await` is clear in meaning and now the `return` is real because you are really returning a value at that point.

At least Addressables is still likely far from finalizing, not until the next year. But C# Roslyn will be finalized this year in 2018.3. I think it is a perfect opportunity to release async-enabled Unity along with Addressables in 2019 where Unity will mark Resources & Asset Bundle obsolete.

 

As soon as Unity starts making 4.x by default it should add some new assemblies that add APIs compatible with async/await/Task.

At least for YieldInstructions like:

- WaitForSeconds
- WaitForUpdate
- WaitForEndOfFrame
- WaitForFixedUpdate
- WaitForSecondsRealtime
- WaitUntil
- WaitWhile

Classes like:

- AsyncOperation
- ResourceRequest
- WWW
- AssetBundleCreateRequest
- AssetBundleRequest

I don't want to depend on a MonoBehaviour (StartCoroutine) for tasks like waiting a second or downloading a file...

 

Are you open for this @unity_bill ??

I just read your post:

https://forum.unity.com/threads/await-addressable-instantion.549367/#post-3646243

 

I heard awaiting IAsyncOperation is officially coming, but I have modified @james7132 's code a bit so that it could throw faulted task on get result. Maybe someone could use it while waiting for official ones.

Code (CSharp):

1. public static class IAsyncOperationExtensions
2. {
3.     public static AsyncOperationAwaiter GetAwaiter(this IAsyncOperation operation)
4.     {
5.         return new AsyncOperationAwaiter(operation);
6.     }
7.  
8.     public static AsyncOperationAwaiter<T> GetAwaiter<T>(this IAsyncOperation<T> operation)
9.     {
10.         return new AsyncOperationAwaiter<T>(operation);
11.     }
12.  
13.     public readonly struct AsyncOperationAwaiter : INotifyCompletion
14.     {
15.         readonly IAsyncOperation _operation;
16.  
17.         public AsyncOperationAwaiter(IAsyncOperation operation)
18.         {
19.             _operation = operation;
20.         }
21.  
22.         public bool IsCompleted => _operation.Status != AsyncOperationStatus.None;
23.  
24.         public void OnCompleted(Action continuation) => _operation.Completed += (op) => continuation?.Invoke();
25.  
26.         public object GetResult()
27.         {
28.             if(_operation.Status == AsyncOperationStatus.Failed)
29.             {
30.                 throw _operation.OperationException;
31.             }
32.             return _operation.Result;
33.         }
34.  
35.     }
36.  
37.     public readonly struct AsyncOperationAwaiter<T> : INotifyCompletion
38.     {
39.         readonly IAsyncOperation<T> _operation;
40.  
41.         public AsyncOperationAwaiter(IAsyncOperation<T> operation)
42.         {
43.             _operation = operation;
44.         }
45.  
46.         public bool IsCompleted => _operation.Status != AsyncOperationStatus.None;
47.  
48.         public void OnCompleted(Action continuation) => _operation.Completed += (op) => continuation?.Invoke();
49.  
50.         public T GetResult()
51.         {
52.             if(_operation.Status == AsyncOperationStatus.Failed)
53.             {
54.                 throw _operation.OperationException;
55.             }
56.             return _operation.Result;
57.         }
58.  
59.     }
60. }

 

Just remember that unlike Coroutines, async methods do not cease to execute when you stop the game in the editor. For advanced users this is probably fine and they *need* to work around it one way or another.

https://forum.unity.com/threads/asy...e-in-the-editor-potentially-dangerous.591235/

But for most users, I think the expected result of stopping the game in the editor is for their code to cease executing too. Imagine a simple bit of code that await's for an addressable to load and instantiates it into the scene. If a user stops their game right after triggering the load call, the load will complete sometime in the future, and the prefab instantiation code will execute and spawn the requested prefab into whatever scene is currently open.

Until this fundamental issue is addressed, async/await is way to dangerous (and if you try to work around this issue, too cumbersome) to work with IMO.

 

`async` keyword does not means a thread would be involved. It is just a syntax that says it yields the control to somewhere else. How did you call your `async` method in the first place? (what is the "runner"?) If you used Task.Run to kick it up, then I think that's the expected behaviour.

For example with UniRx.Async, the "runner" of `async` method is the Experimental.PlayerLoop. This make async works like coroutine that they try again a frame later in the specific step. (defaults to Update step, I remembered coroutines are after all Updates) If you do that for example, stopping game in the editor will cease execution of `async` since it turns off the player loop.

 

Some of my methods are fire and forget, Async methods with no return type for example. I start them just by invoking them directly. Those ones would need to be started up with UniRx explicitly to avoid such issues right? Otherwise yeah UniRx sounds sweet.

 

+1 for UniRx. It is an amazing library.

 

the load itself should abort and never complete. your task will then eventually get GC'd

the only things that resume execution after you stop the game are

Task.Delay

,

Task.Yield

and stuff that interact with the

SynchronizationContext

directly without knowledge of Unity lifecycle.

async operations done through Unity stuff should be async-playmode safe, e.g. if they schedule the continuation in

op.onCompleted += ...

, either it doesn't get called if you stopped playing, or it does and you have a bug regardless of being

async

because you can add the callback yourself.

the only "problem" (that also exist with coroutines) is if you are using

try {...} finally {...}

, because your Task/IEnumerator will vanish without being disposed and running the finally blocks

 

I use an asset from the store, just using await on IEnumerator methods or async operation returning methods.

https://assetstore.unity.com/packages/tools/integration/async-await-support-101056

For addressables for example:

var loadPrefab = (GameObject) await Addressables.LoadAsset<GameObject>("Label");

 

I use Await Extensions, you definitely want to check this out https://assetstore.unity.com/packages/tools/integration/await-extensions-127006
it's quite well documented and maintained by author

 

*edit* actually the issue was unrelated to IAsyncOperationExtensions and a change with 0.7.4 runtime keys.

the extensions should still work, just need to use AsyncOperationHandle instead

 

One issue I'm finding with Await Extensions compared to coroutines is they don't cancel if the GameObject or Component is destroyed. So, for example, the following code will crash if the game object is destroyed before the await completes:

Code (CSharp):

1. async void Start()
2. {
3.     await new WaitForSeconds(5);
4.     transform.position = new Vector3(1, 2, 3);  // will crash if object was destroyed
5. }

Accessing the gameObject property will also crash, so the following won't fix it:

Code (CSharp):

1. async void Start()
2. {
3.     await new WaitForSeconds(5);
4.     if (gameObject)  // will crash here if object was destroyed
5.         transform.position = new Vector3(1, 2, 3);
6. }

You can use the following instead:

Code (CSharp):

1. async void Start()
2. {
3.     var self = gameObject;
4.     await new WaitForSeconds(5);
5.     if (self)
6.         transform.position = new Vector3(1, 2, 3);  // won't crash
7. }

Does anyone have a better solution?

 

The *standard* way to handle cancellation is CancellationTokenSource.

Code (CSharp):

1. MyBehaviour
2. {
3.     CancellationTokenSource tokenSource = new CancellationTokenSource();
4.  
5.     void DoSomethingAsync()
6.     {
7.         await sometask();
8.         if (cardTaskTokenSource.IsCancellationRequested)
9.             return;
10.         // Do the job...
11.     }
12.  
13.     void OnDestroy()
14.     {
15.         cardTaskTokenSource.Cancel();
16.     }
17. }

Pretty much as ugly as check gameObject == null after each await.

Maybe it will make the life slight easier to create a parent behavior with default tokenSource used to represent gameobject destroyed.

But I'm looking for a more elegant way to cancel my task within a gameobject lifecycle.

 

So not long after posting here, I discovered this lovely little library: https://github.com/muckSponge/UnityAsync

It lets you write code like this:

Code (CSharp):

1. await Await.Seconds(5).ConfigureAwait(this);
2. transform.position = new Vector3(1, 2, 3);    // won't crash

ConfigureAwait() lets you pass in either an owning UnityEngine.Object or a CancellationToken. If the owning object is destroyed, the code after won't be called.

The source code is also clean and well commented, performs well, and doesn't cause allocations, so I'm liking this library so far.

 

Await Extensions knows nothing about GameObjects, Components and other Unity objects. They work in a separate thread. You should manually cancel them if you destroyed GameObjects or loaded another Scene.

 

Yeah, that's the problem with Await Extensions. UnityAsync's solution is pretty nice.

 

Lol. My comment was more about whether Unity internals would utilize the .NET Scheduler rather than (presumably) remain driven from their main thread. Tasks are a bit heavy and have to be adapted to play nicely with the YieldInstruction constructs, like WaitForEndOfFrame that seem driven from their native layer. That and, with ECS/Jobs we're talking multi-threading and you more/less can't touch main-thread created Objects - different use case.

The Addressables system is interesting in that they've gone all async and utilize Tasks. See Runtime\ResourceManager\AsyncOperations\AsyncOperationBase.cs. @5argon makes a good point below.

But this thread is fairly dated. Now UnityEngine.ResourceManagement.AsyncOperations.AsyncOperationHandle<T>.Task is exposed, sooo yep...
AsyncOperationHandle<SceneInstance> asyncOp = sceneAsset.LoadSceneAsync();
await asyncOp.Task;

 

Nice lib (y)

 

I-m trying to use this, but i don't know which namespace are you using, is all in red lines...

 

I built a script that takes care of this and wrote a blog post about it: https://marcospereira.me/2022/05/06/safe-async-tasks-in-unity/