Asynchronous Texture Upload and DownloadHandlerTexture

Hello. I am attempting to download and display a texture using DownloadHandlerTexture. Once it is downloaded and we display it we get a giant spike in Gfx.UploadTexture on the render thread. At the same time the main thread is waiting for Gfx.WaitForPresent for roughly the same amount of time. (30-60 ms). The images are of sizes around 1400x2000~ish.


So obviously we are stuck synchronously waiting for the image to upload to the GPU. Multithreaded rendering and Graphics Jobs do not help much. (Turned off for this profiler capture for debugging). Turning Vsync off does not help.

We have tried getting the raw bytes through DownloadHandlerBuffer, converting from jpeg to pixels in native code and then feeding the pointer into Texture2D.LoadRawTextureData but we hit the same problem as soon as we call Texture2D.Apply().

I found the documentation about Asynchronous Texture Upload, but it only seems to apply to files already in the project at build time as far as I can tell. Is there a known way of triggering time-sliced texture upload to GPU from downloaded textures or bytes?

EDIT: This is in Unity 2018.1.9f1 for Windows. Upgraded from 2017.4.6f1 where we had the same issue.

 

Bumping this one

 

Have the same Issue.
currently using `texture.LoadImage(bytes);` and `((Texture2D)texture).EncodeToJPG();` on Unity2017.4.14

 

Can you just use sockets and preload textures in another thread before they're needed?

 

I don't know .. we probably could if we had any clue how to do that ^^

 

Your interests are not even worth a google search? Pity.

https://docs.microsoft.com/en-us/dotnet/api/system.threading.thread?view=netframework-4.7.2

https://docs.unity3d.com/ScriptReference/WWW-texture.html

 

Nope, uploading textures to the gpu needs graphics context (think main thread). It‘s not about loading and decoding images, we get hiccups uploading texture data to vram.

 

Ofcourse we googled, man .. a lot ^^

Your first link is about threading .. ofcourse I know how to do that .. but I didn't find any good example of encoding and decoding Texture to PNG or JPG.

I know we somehow would have to decode the received bytes to a Color32 array in the other thread and than use SetPixels which some peolpe claim to be faster (and the other way round) but I still don't know how. You can not use anything from the Unity API since Unity is not thread save.

The second link is what we also tried .. with no difference. It runs in a Coroutine -> also in the main thread and therefore blocks the UI the same way LoadImage does...

 

Combine the two.

 

I said "before they're needed", not "at the exact moment when they're needed". Threading + wwwtexture + intptr.

 

Ah passing the native texture pointer? That tickles my interest. Have you tried that already with success?

 

How?

The problem is as said: your second link is a Coroutine and still runs in the main thread.

The Unity API and EncodeToJpg, texture.LoadImage etc is only available in the main thread so it can't be used in another thread.

So how can we "combine the two"? If it would be that easy believe me I would have done it but I just couldn't find any good example of how to do that.That is what I didntd find and therefore we ask here.

 

mega bump, we are looking at the same issue, from https://answers.unity.com/questions/1582837/multithreaded-rendering-of-large-volumes-using-tex.html

 

That is the same problem I have now, we cannot build VR applications that load images, because loading textures from file still hang the applications for half a second. Thats unacceptable for VR application, and it is yet a problem for games that need to load images that are not build into the app. There is two main problems here, first one: you need to load the file from the disk on a Thread, decode it on a Thread. Unity will not help you with that. Second problem here: You need to upload it to the GPU. You cannot do it in a single frame, you need to split it into several frames, according to the size of the buffer. Unity will not help you with that. Thats it, Unity do not want you to create a VR app, thats unbelievable, how can I create a VR app that cannot load a JPG from my pendrive? WWW async LoadImageIntoTexture also DO NOT DO THAT. In conclusion, they still dont care, Unity3D is the most popular, the main development engine for building VR applications for the Industry, the leading, the awesome, and yet you cannot build an VR application that load a texture from disk. Please, Unity3D, we need it so much, Load Texture from my pendrive without HANGING my application!!!!

 

Found this thread while trying to solve a similar problem. The solution I've found is to use Unity 2018.3 or later. Then don't LoadRawTextureData, instead GetRawTextureData as a native array. From there, you can write into it and then Apply() with a minimal stall.

 

I have to say its pretty amazing that we still cant upload to GPU a texture load over multi frames. Why do unity only care about build time EVERYTHING. I would like to see this mentality at unity broaden to real time data loading. This feature should of been available 4 years ago !!!

 

Unity as your probably aware does not provide out of the box asynchronous texture loading to the GPU for (runtime created textures) but it does however support it for build time textures.

You could write a unity native plugin C++ to achieve this which I have done successfully. It does require getting your hands dirty though. There is a unity native plugin example which will give you a good start on what to do. You can strip out the unnecessary code easily and add your own DirectX code. The solution will depend on what platform you want to support as the implementation changes for example OpenGL, DirectX9 or DirectX11.

The other aspect you will need to consider is what texture format(s) you want to support for this. Examples would be compressed (DXT1/5) style or Uncompressed (ARGB).

My own solution required that I needed a runtime Crunch CRN converted to DXT 1/5 to be natively updated on the GPU to reduce download times. So I ended up using the LoadDDSTextureFromFile/Memory methods from the DirectXTK.

Here is my build documentation for standalone windows (if it helps get you started)

Introduction
Unity bottlenecks at being able to update texture content on the GPU.
Only approach at the moment uses Texture.Apply() which is very slow.

Project Deliverables
DirectXTK-master - Provides C++ LoadDDSTextureFromFile and LoadDDSTextureFromMemory
RenderingPlugin - Provides Native Unity C++ Plugin
Runtime Crunch - Provides C++ to Create and Load CRN / DDS textures
Unity5Project - Provides Unity Test Project

Build Instructions
All projects build and work with Visual Studio Community Edition 2019
All projects natively debug with unity 2019.3.0f3

1. DirectXTK-master
1.1 Build DirectXTK-master/DirectXTK_Desktop_2019.sln either Release / Debug
1.2 Lib location \DirectXTK-master\Bin\Desktop_2019\x64\Release\DirectXTK.lib
1.3 Project/Properties
C/C++->Code Generation
Runtime Library
Multi-threaded DLL (/MD)
2. RenderingPlugin
2.1 Build RenderingPlugin\VisualStudio2013\RenderingPlugin.sln
2.2 RenderingPlugin->Properties
VC++ Directories/Include Directories
C:\Unity Projects\Native Texture Rendering\DirectXTK-master\Inc

VC++ Directories/Library Directories
C:\Unity Projects\Native Texture Rendering\DirectXTK-master\Bin\Desktop_2019\x64\Debug

C/C++->Code Generation
Runtime Library
Multi-threaded Debug DLL (/MDd)

Linker/Input
Additional Dependencies
DirectXTK.lib
2.3 Release DLL
Copy Native Texture Rendering\RenderingPlugin\VisualStudio2013\build\Release\RenderingPlugin.dll ->
\Native Texture Rendering\Unity5Project\Assets\Plugins\x86_64
2.4 Debug DLL / PDB
Copy Native Texture Rendering\RenderingPlugin\VisualStudio2013\build\Debug\RenderingPlugin.dll ->
\Native Texture Rendering\Unity5Project\Assets\Plugins\x86_64
Copy Native Texture Rendering\RenderingPlugin\VisualStudio2013\build\Debug\RenderingPlugin.pdb ->
\Native Texture Rendering\Unity5Project\Assets\Plugins\x86_64

3. Runtime Crunch
3.1 Build Native Texture Rendering\Runtime Crunch\crunch-unity\crn.2010.sln
3.2 crnlib->Properties
C/C++/Code Generation : Multi-threaded DLL (/MD)
3.3 Release DLL
Copy Native Texture Rendering\Runtime Crunch\crunch-unity\lib\VC9\Release_DLL\x64\crnlib.dll ->
\Native Texture Rendering\Unity5Project\Assets\Plugins\x86_64
3.4 Debug DLL / PDB
Same but also copy crnlib.pdb

 

bump

 

Is your C++ part multi-threaded? I have been reading that GL.IssuePluginEvent can only be called from the render thread.

 

Anyone got any update on the matter ?

I also have huge UploadTexture compute times after trying to download and display a texture, and I could not find a way to spread that over multiple frames.
Using a UnityWebRequestTexture:
Code snippet:

Code (CSharp):

1. private IEnumerator GetTextureFromMediaPathWWW(string mediaPath, Action<Texture2D> done)
2. {
3.     using UnityWebRequest uwr = UnityWebRequestTexture.GetTexture(mediaPath);
4.     yield return uwr.SendWebRequest();
5.  
6.     if (uwr.result != UnityWebRequest.Result.Success)
7.     {
8.         Debug.Log(uwr.error);
9.     }
10.     else
11.     {
12.         var texture = DownloadHandlerTexture.GetContent(uwr);
13.         done.Invoke(texture);
14.     }
15. }

Also tried with Texture2D.LoadImage, which instead blocks the main thread:

Code snippet:

Code (CSharp):

1. private async Task<Texture2D> GetTextureFromMediaPath(string mediaPath)
2. {
3.     Texture2D tex = null;
4.     if (File.Exists(mediaPath))
5.     {
6.         var fileData = await File.ReadAllBytesAsync(mediaPath);
7.         tex = new Texture2D(516, 516);
8.         tex.LoadImage(fileData);
9.     }
10.     else
11.     {
12.         Debug.LogError("NO FILE AT " + mediaPath);
13.     }
14.     return tex;
15. }

Edit:

• On Unity v2021.3.16
• The texture targeted is a 516x516 JPG file