What is the best way to display 3D models as UI elements?

I am trying to display 3D models on the canvas layer (for an inventory system) and I can't figure out the best way to do this. So far I've read that the best solution is to change the canvas to World Space but then I can't use anchors and the UI moves around as the camera follows the player. Changing it to world space just to display 3D models on a UI level feels clunky. Is there a more efficient way?

 

A render texture is certainly one way to go, however remember that UI is rendered on a 3D scene, usually best not to do it the other way round unless you are displaying additional 3D elements on top of a canvas which is already displayed over another 3D scene (aka an overlay)
A simply answer is to have a camera rendering on screen where you want the 3D item as normal, then position your UI on or around it, don't be afraid to use multiple UI Canvases for depth.

Many ways to cut the cheese as it were, RenderTexture works well for static 3D elements (simplest way), blending the UI in a 3D scene works better for more active elements. As always comes down to performance as a Rendertexture generally causes the UI Canvas to be redrawn (in my view)

 

I think the issue is that I want the 3D objects to be positioned the same way UI elements are so that I can easily place them on a UI object (they are being used as icons). I also want the UI elements to react to mouse clicks and what not. This means I can't just use a render texture right? Or can I use a render texture just for the 3D objects and regular UI elements for the background that will be clickable?

 

So render each model individually as a texture or render the model and the button to click as a texture?

 

Great, thanks @Tim C ! now I got even more stuff to dig back in to

Thanks for the update, missed that in the release notes.

 

I'm also interested in showing a 3D model in the UI.
I went through the 5.2 feature list ( https://unity3d.com/unity/whats-new/unity-5.2 )

and found the entry Tim C was talking about:

"UI: CanvasRenderer now takes a Mesh instead of List this allows for the use of imported meshs as part of the UI"

Beside from the weird description "now takes a Mesh instead of List", it is unclear how to make this work. The CanvasRenderer component has no fields in the inspector you can assign to. Also from code there's seems to be no way to make this work (no applicable functions).

Any pointers on how this works would be appreciated

 

"UI: CanvasRenderer now takes a Mesh instead of List this allows for the use of imported meshs as part of the UI"

I cant find this also. Any help appreciated!

 

Ok so using canvas set to screen space camera works

 

Did anyone ever figure this out?

 

set canvas set to screen space camera. put obj as child of in canvas. Only problem is you have to deal with crazy scales fro objs.

Does anyone know if rect transform for changing screen res on devices will work?

 

I went ahead and used the screen space camera and have one problem. I have my 3D objects attached to UI elements as children that are part of a scrollview.

The UI elements respect the viewport mask, but the 3D objects do not. This makes the 3D objects fly over the top of header elements that are above the scrollview.

Is there any way you can think of to make the 3D objects also respect the viewport mask?

 

OK guys, so when I want to have a plenty of rendered objects on UI, do I have to create such a big number of render textures and render cameras on each object that will be on UI element? I think there could be a problem with fps

 

Hi, I know this an old thread, but this is not strictly true. You can add a Rect Transform to the 3d objects. You will get a warning and it will overwrite the default transform but the anchoring works fine if you do. You still have to use Screen Space - Camera render mode for the canvas though. Peace

 

I need the exact same. Could you find any solution?
(working on a character selection menu like crossy road with rotating modals)

 

I am curious about how games like Raven Field deal with this problem. If you check the weapon selection menu or the killing popups, you can see the models are realtime rendered. Can't imagine hundreds of render textures in a scene, even with low res.

 

Viewport mask problem might be caused by the shader. Only UI specified shader can apply the mask effect I think.

 

Just finished searching Google Images and YouTube to see the examples you mentioned and frankly I don't see how you would come to the conclusion that they're realtime rendered at all. One of the newest videos I watched is only nine hours old and the inventory is basically just static images.

 

Maybe my mistake. But from my experience of making gun mods, I don't find a place to put the static preview pictures. I think those are rendered later in game using customized shader. However, one thing you are correct, because all those are static, render textures are ok to help. You don't have to create too many cameras at the same time.

 

I've found a solution that doesn't involve render textures nor World Space Canvas!
I had the same problem and came up with a 'creative' solution. I use a dual camera setup: One camera for my Canvas (which is set to Screen Space: Camera) and one camera for the rest of the game. Now follow these steps:

• Make the 3D object a child of the canvas.
• To deal with the anchoring, use a parent RectTransform above the 3D object.
• The 3D object will look strange on the canvas, stretched near the edges of the screen. This is because of the perspective of the camera. I don't want to use an orthographic camera, so I just reduced the FieldOfView attribute of my UI camera to 1.

Here's a screenshot of the scene view of my inventory with a 3D cube in it:

 

Can the 3d objects be arranged correctly in a scroll view? Is this method "Mask-Friendly"?

 

Unfortunately, no. I want to do that too -- but I have a backup plan!
Use another camera that will render all the content of the scroll view into a single render texture. A render texture is a flat image, and images are "Mask-Friendly"!
If you need to interact with the 3D objects (buttons), you can make all 3D objects invisible (by disabling their renderer) and draw the masked render texture on top of them.

 

Perfect, thanks a lot, just what I was Looking for

 

Thanks for that, works perfectly

 

Hey guys,
I want to make something like Asset Forge object select menu as you see at left side of the screen.
There are too many objects and they are rotated around themself.
I want to use multiple camera and create RenderTexture and put it in RawImage but I think there is performance problem.
So I want to know what is your suggestion ? And it is really unoptimized to use multiple camera and RenderTexture ?

 

Well, unoptimized depends on how many objects you want to display and how big/crisp the render textures need to be. It can definitely perform quite badly when having multiple, larger render textures displayed at once.

 

So - what if you do want animating 3D objects in the UI? It's 2020 after all

Edit: I've got a UI camera, and a 3D object as a child of a canvas set to Screen Space-Camera.

It renders, however - for the object to be visible, the scale of the 3D object needs to be x1000 or something - is there any way to avoid the scale issue? It seems messy as hell.

 

I set up 3D objects in UIs that respect masking here. I have a 'portal' quad with a shader that sets a bit in the stencil buffer, then the 3D objects have another shader that only renders where that stencil bit is set.

IINM, this is what the UI shaders do already.. I think they use the first bit of the buffer, but I haven't had much luck making it work directly with UI mask objects. What I did instead was have my portal quad set the 2nd bit of the stencil buffer, and any 3D objects I want to mask to the portal use a portal-object shader that tests against the 2nd bit.

You also have to tinker around with the shader render queues a bit. It's a bit fiddly so I don't think I can be much more specific than this.


The advantage of this method is that it doesn't use separate cameras or render textures. Back in KSP the parts menu used separate cameras and RTs, but this time I needed the menus to also be VR friendly, so the stencil buffer solution was a much better choice.

The rendering setup is largely unaffected, and the menu can also work in a worldspace canvas (the 3D elements are actually 3D, which looks neat in VR).


Cheers

 

Upon scan target Image, I have to show the 3D Animation Model along with Information in Text format.
In this case I am trying to use the UI-->Panel to display 3D Animation along with Text information but 3D Animation model is not displaying and not working as expected n run time.

Could anyone guide me the right approach.

Cheers

 

Love the 2015 Karl Solution with the render texture, works nice -- but still struggling when having multiple 3D objects (in a menu, for example). Do I have to create cameras and render textures for each 3D object? Or is there an more efficient way? (I tried to change the target texture of the camera by script and activating and deactivating the 3D object, but, oh well... I would not post here if this would have worked out)

 

Thanks for your super fast reply! I did a setup with a camera for each object then. Works. Performance is also not much of an issue (it's a menu scene anway, without gameplay). Thanks again!

 

Hiya, is it possible to have a character animate in a pause menu? Getting the character in the UI isn't the tough part in 2021 but how to you have it animated with timescale=0? Unscaled time? Think of recent titles like Surge 2 here, where you pause the game to bring up the Gear menu, with the animated 3d character.

 

I don't know how much rewrite this would be, but I use a bool isPaused. Then only execute code based on this. Most of the update() don't run and objects go static based on isPaused. I know dark souls keeps the game running in the menus.

 

I agree, timescale = 0 seems like an easy solution for a pause but can lead to a of problems. That aside and if you just want some animations: the unity Animator works with timescale = 0, just set the animation to "Unscaled Time".

 

a few years late but i didnt seem to see anyone suggest how to go about this using render textures. So basically you will need to make a function that outputs a texture2d and to do this use the following code.

Code (CSharp):

1.     public Texture2D IconGen(RenderTexture renderTexture)
2.     {
3.         Texture2D newTexture = new Texture2D(renderTexture.width, renderTexture.height, TextureFormat.RGBAFloat, false, true);
4.         RenderTexture.active = renderTexture;
5.         newTexture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
6.         newTexture.Apply();
7.  
8.         return newTexture;
9.     }

What this will do is read the pixels from your render texture, you will need to instantiate your object in front of the camera and then it will make a new texture 2d from that render texture. Finally just return the new texture 2d and use the new texture for whatever your using it for. I suggest having a Ienumerator that yields for a frame in between instantiating the object and creating the texture otherwise you may just get a blank texture. Also you may want to implement a list of items to make textures for so that your not instantiating all your objects at one time otherwise youll be left with an icon thats just a mixture of gameobjects. I suggest at some point in your game running Resources.UnloadUnusedAssets() because even though you will be destroying your game objects for some reason they still are stored in memory and your ram usage will slowly begin to grow