Integrate Qt into Unity

Dear All,
I'm not sure this is the right place to ask this question. Anyway ...
We have a unity application and we need to add some GUI on it. Since we have much more experience in Qt than Unity UI (and our idea is to unify the UI with other Qt applications) we would like to embed Qt in Unity.
Unity must not run windowed for performance reasons. I tried simply running a Qt application from C# script with:

Code (CSharp):

1. System.Diagnostics.Process.Start(applicationFullPath);

But Unity application freezes when the focus is on my Qt application and this is not acceptable. We would like the GUI part/application to remains on top while Unity runs as usual.

So, I tried to run a separate Qt application but we can also consider a Qt GUI embedded in Unity (i.e. Qt application loaded from a dll).

Is there anything you can suggest to implement such a GUI in QT and make it working with Unity not windowed?

Best regards,
Gianni

 

Where I work (I'm an intern, so take everything I say with a grain of salt) we use Qt, and I have actually been working to build a much more efficient 3D rendering framework in Qt that works with QML.

First off, you picked the worst combination to pair Qt with. I've seen several threads of successful Qt integration in other engines that use open C++ source. But it sounds like you already have this Unity application built. In that case, I would think that you are already familiar enough with GameObjects and MonoBehaviours that the confusion of adding and mixing a second system would undo the familiarity benefits of said system.

However, if you still insist on continuing, here are the conflicts and possible solutions worth trying. I'm assuming you are using Qt5, QML, and Qt Quick 2.

1. Both Qt and Unity are designed to be standalone applications (have some sort of window or reserving of pixel space on your display). You'll want one of them to run without window-making functionality. I would check out threads regarding Qt integration with other engines just to see how to build up a QML engine without a window.

2. Both Qt and Unity control GPU context state. You are going to want to hit up native plugin rendering on the forums and in documentation so you know how to keep them from clashing with each other. You'll also likely need to tap into the QQuickRenderControl API to link Qt rendering to Unity callbacks. I would suggest rendering Qt GUI into a render texture that Unity can load. Heck, I think Unity might even have some API specifically for native plugin rendering to render textures. Then you would just blit the GUI over the scene as a post-effect.

3. Input is going to be a real pain. I've tried using Qt's input system for game-like controls and it did not work well the moment I wanted to use the number keys and modifier keys to do different things. Both are going to consume input events from the OS. You'll have to quiet one of them, and then use virtual inputs to stimulate behavior.

4. IPC of a screen-size RGBA texture could be expensive, so I recommend not doing multi-process and if possible keep that GUI texture on the GPU.

Hopefully this helps, and good luck!

 

Hi DreamingImLatios,
I"m not familiar with unity, the application we have has been developed by someone else and I have to struggle with it now!
Anyway, it seems it would be easier to learn Unity UI than integrate Qt into it.

Thanks for the explanations,
Gianni

 

I don't have a real successful story for you. But let Qt and Unity communicate with RPC still looks promising.

 

With maybe the exception of input (I still don't understand order of event consumption for foreground and background applications) this is totally doable. But whether or not it is worth doing rather than keeping everything in Unity is the real question I propose evaluating.

Just out of curiosity, what platform are you targeting?

 

Hi, did you find a solution to show web page inside unity?