ID3D11Device::CreateTexture2D in low-level native plugin freezes Unity

I'm trying to load textures from files (JPG, PNG, etc.) in Unity for a VR application. Texture2D.LoadImage does exactlty what I want, except it blocks the main thread. Since loading seems to take at least 50 ms even for the tiniest of images (and can exceed 500 ms for larger images), this means dropping a significant amount of frames. This is a showstopper for VR.

People on this forum have suggested using the WWW class to do loading (using the WWW.textureNonReadable field) since it supposedly does some of the work in the background, but even that takes too much time on the main thread.

Since DirectX 11 allows texture loading on non-rendering threads, I decided to write a low-level native plugin to handle loading and supply a pointer for use with Texture2D.CreateExternalTexture. This works fine if I create the texture on the main thread; it even works when I do it on a separate thread and make the main thread wait until the other thread is finished using WaitForSingleObject. But as soon as I remove WaitForSingleObject, Unity (or at least the Unity Editor) freezes completely. I can see in the Editor.log file that my loading code finishes executing (both on the native side and on the C# side), but Unity freezes before it can even print these logs in the editor console. No error messages anywhere. I've been debugging this mess by commenting out pieces of code to see if it stops Unity from freezing. That's how I've traced it back to a call to ID3D11Device::CreateTexture2D.

Code: RenderingPlugin.cpp and UseRenderingPlugin.cs on as a gist on GitHub(adapted from the low-level native plugin sample). Alternatively, here's a link to a DL of the complete project (VS2017).

I tested the same code in a Direct3D11 sample project and it ran without issues, so I'm pretty sure this has to be something specific to Unity.

I have limited experience with C++ on this level, so I'm kind of lost at this point. Can someone more experienced spot a mistake somewhere? Can I get an actual error message somewhere? Has anyone else attempted asynchronously loading textures via native plugins?

 

Minor update: I attached the Visual Studio debugger to Unity and tried breaking execution after Unity freezes, but this simply results in Visual Studio freezing as well, probably waiting for Unity to respond. After killing Unity, VS unfreezes displays an error dialog with: "Unable to break execution. General Exception".

I also tested if the issue exists in builds, and it does. Only difference is it also freezes on the loading thread right after calling ID3D11Device::CreateTexture2D (instead of only freezing the main thread).

Note that the first call to CreateTexture2D to validate the D3D11_TEXTURE2D_DESC and
D3D11_SUBRESOURCE_DATA inputs (as documented here on MSDN) doesn't cause any issues:

Code (csharp):

1.  
2. // doesn't freeze
3. if (m_Device->CreateTexture2D(&tdesc, &tbsd, NULL) != S_FALSE)
4. {
5.     return false;
6. }
7. // still fine
8.  
9. // Unity freezes after this call (tested by removing everything after this)
10. if (FAILED(m_Device->CreateTexture2D(&tdesc, &tbsd, tex)))
11. {
12.     return false;
13. }
14.  

PS: I'm using Unity 5.5.2f1
PPS: Just tested in 2017.1.0.b4, same issue.

 

Activate D3D Debug Layer and try again(check this link). I had similar problems at one point and the debug layer provided the info I needed about what I was doing wrong.

 

Oooh nice! It took me a while to figure out how to enable the debug layer for Unity applications[1]. Unfortunately I don't see any unexpected calls or error messages, but I'll definitely be using this for future projects.

I think I figured out a workaround, though. If I create the resource on another ID3D11Device as a shared resource and open it on Unity's device with ID3D11Device::OpenSharedResource everything seems to work. I'm still getting random crashes, but now I at least get a call stack and a debug dump. Will investigate further.

[1] For those wondering: if you make a build of your project, you can force enable the debug layer for the executable in the DirectX Control Panel (dxcpl.exe); you then open your executable as a Visual Studio project, enable native debugging in the project properties, and attach the debugger.

 

Nearly how you also go about debugging shaders for Unity applications using VS instead of RenderDoc.