Do you need to load FBX at runtime? (Any comments about fbx are welcomed.)

​

RuntimeFbx handles FBX at runtime without dependency on FBX SDK in 2022.
https://assetstore.unity.com/packages/slug/137338

---------------- old post content below ------------------

Some VR app developers said they need it desperately because they don't want to convert fbx files to Unity3D format at runtime any more. They just want to load and display them directly without conversion.

Do you need a runtime FBX loader? Why? And for what platform?

I'm thinking of writing a plugin for this if it is really needed by many people. (I know there exist some runtime FBX importer. But it seems that they are not maintained, undocumented or buggy.)

 

For VR? That seems like a ridiculous choice. I can understand the point on a PC.

The main reason to load models at runtime without routing them through Unity is to allow mod support. I suppose it was only a matter of time before people start modding VR games.

 

No. I could write one if I wanted, but I'd try to use some other format. Any other format, pretty much. FBX is quite messy, undocumented, and is tied to proprietary sdk (which is why runtime loader is buggy and unmaintained). Anything would be better than it, even Collada.

I'd probably go either with a custom format (based on json) or something like gltf. gltf had some interesting ideas in it, even though it is quite obviously heavily tied to OpenGL API.

 

Sometimes you cannot control the formats of 3D model files from your clients. They just insist on using FBX.

 

In that case I'd provide a separate conversion tool for the one and only format app supports.

Skyrim and fallout doesn't use fbx for its models. That didn't stop people from modding the game and creating custom meshes for it (aaand they reverse engineered the mesh format).

----
Also, keep in mind that to properly support fbx, you'll need to use fbx sdk which will require native plugin written in C++. One fun thing about native plugins in unity is that while the game usually can survive C# exception being thrown, when native plugin goes down, it often takes the application with it.

So, definitely does not look like a good idea.

 

Sure that's fine, but why the runtime loading? Its a very specific edge case where runtime loading makes any sense.

 

Thank you, @neginfinity and @BoredMormon . Very helpful comments!

I'm just very familar with FBX SDK and want to make something useful based on it in case some people need that. Would you please give me some advice? I'm also thinking of making an editor plugin that can modify or create fbx files in Unity3D editor.

 

I believe for editing meshes from within the editor there's already "Pro Core" suite of plugins (they are not my kind of thing, though). There are probably couple of FBX exporters on asset store as well. If you're looking for money, you could try undercutting their prices by large factor. If you're looking for helping people you could try releasing an mit/zlib licensed plugin on github/bitbucket, although since FBX sdk is not opensource, there will be some difficulties.

Personally, I'd be much more interested in a tool that can export files from blender into unity without using fbx (there are issues with coordinate space conversion). I could write that myself, if I had the time, that is.

Another issue you could look at is that standard unity fbx importer handles coordinate conversion by introducing rotations on some bones. That occasionally causes trouble.

That's all I can think of without being asked a specific question.

 

I can see reasons for other than VR to do runtime loading outside of pre-processing with Unity.

The difficulties though IMO wouldn't be with mesh itself, but handling all the other useful things you can do with it pre-processed such as animators, material re-assignments, etc...

how would you handle that part of the workflow?

 

In all 3D software I know of there is a very useful feature that allows the skin to deform based on the angle of two bones. In 3D Max this feature is called joint angle deformers - and as the name implies when 2 bones bend from a set angle - to another set angle - the bones deform the mesh. This is great for things like bulging muscles and also great for correcting problem areas in the mesh (mostly arm pits and hips/pelvis area).
And to top this feature off - this handy and useful feature is built right into the skin modifier in 3D Max. There is no additional hooks, no additional meshes needed (like a morph/blend shape), no additional game objects or additional extra stuff needed for these joint angle deformers to work as expected. And lastly these awesome angle deformers have been around since Max version 5 or 6 = 2003 or 2004.

For some STUPID unknown illogical reason fbx rips this data out of (used by all engines) the skin data upon export from all 3D packages.

This would be a super awesome addition to fix in the fbx export process. I don't know how this could be done.
I don't know if you could create this or not but it is an extremely VERY helpful feature that would be great to have if we could get the support for it in fbx format.

A distant second request would be if Unity supported twist bones, especially for the forearms. It's not really needed for any other bones in a humanoid setup, but could be useful for other rigs, like tentacles. I don't think this is a fbx limitation - but is a limitation in Unitys humanoid rig setup.

 

What should be done when loading fbx at runtime is just contructing Unity3D object directly from data read from the FBX file.

As for the skin-mesh animation, to construct a SkinnedMeshRenderer and its animation (AnimationClip and AnimationCurve), we need:

1. the skinning info(weights of bones and related vertices) per skin-mesh
2. the transformation info(translation, rotation and even scaling) every frame per animation.

As for the material, yes, the work flow should be adjusted. The developer should use some standardized shaders that make the mesh has the same looking as they are in 3D modeling software like 3DSMax. It's very easy to extract material info from FBX files. So construct a Material is easy as long as we think of a way to use properly standardized shaders.

 

Do you mean Morph Angle deformer by deforming based on the angle of two bones? It seems to be a rendering technology that not supported by FBX. I'm not familiar with that, but I can imagine the usage somehow.

Spoiler: Morph Angle deformer

Bending the arm without the Morph Angle deformer causes the sleeve to crumple.


Using the Morph Angle deformer creates a smooth bend in the sleeve.

I also found some posts about this:
https://feedback.unity3d.com/suggestions/bone-driven-morphs
http://answers.unity3d.com/questions/874461/does-morph-angle-deformer-in-skin-modifier-of-3ds.html
https://forum.unity3d.com/threads/3dsmax-fbx-import-morph-angle-deformer.130963/

Unity3D just doesn't support this feature. But I think it can be implemented in Unity3D with some special shader and runtime scripts just like how Unity3D implements their SkinnedMeshRenderer. As of FBX, it is just a file with data created by FBX exporters of 3D modeling softwares.

To achieve that, we have to write some tools first. First, write a exporter for 3D modeling softwares that can export customized data file (or customized FBX or the Unity3D asset file) contatining the morph angle data. Then write a importer for Unity3D to read the morph angle data file (or customized FBX using a customized FBX importer-not the offical one-or the asset file directly). Finally load the data into the runtime scripts, send them to GPU and render them and the mesh with some deforming shaders.

The big problem is the runtime script. I really worry about whether one can have enough access to GPU efficiently via Unity3D APIs.

Twist bones? Did you mean making the vertices between two joints deform along a curve (e.g. nurbs) from joint A to joint B, and so the bone from A to B was not straight but twisted?

 

Similar concept, but no - I'm talking about joint angle deformers, which is basically the same, but they are different. The joint angle deformer is (imo) simpler and could be included in the fbx file without any extra file size.
The info you linked to is built into the skin modifier as well, but that info is directly related to another modifier which morphs the skin based on bone angle. This is essentially the same concept, but the only two modifiers that are export/import supported through fbx is skin and morphs (blend shapes) so that process would not work.
Anyway - just a suggestion.

No - twist bones are bones between two joints (wrist and elbow) that increment there rotation based on the lower bones full rotation. So if 2 twist bones were used and the wrist bone rotated 60 degrees, the upper wrist bone would rotate 20 degrees and the second bone would rotate 40 degrees. This gives great falloff of the skin to deform properly around the axis point down the main bone, without the skin collapsing upon itself - if no twist bones are used.
An experienced rigger/skinner/animator can make the skin behave as it is supposed to - but the process is iterative, more time consuming and isn't as easy as skinning verts to the twist bones and good to go.

Since you are discussing more run-time concepts these enhancements to the fbx export/import process really isn't relevant. Though improving the base fbx export/import support I would argue is more important and would be more beneficial to the community.

 

Oh.. that seems too much for me because as a programmer I don't have any experience in creating skin-mesh animation.

I'll make some small FBX editor plugin first. For example, a node renamer of FBX hierchary or node transformation adjuster inside the Unity3D editor.

 

Bumping an old thread, but I wanted to comment that as an AR developer having users be able to display their self made or other custom 3D models with their phones would be very useful. Unity is useful and easy for AR, but not being able to download and display .fbx files in runtime restricts use cases.

Current Asset Store solutions seem buggy, undocumented and not updated often which is a shame because, if there was a proper runtime importer, we would surely buy it and I believe many other AR and VR developers would find it very useful.

 

@TeroKuparinen The main obstacle of runtime fbx loading is that FBX is a proprietary file format and can only correctly read by FBX SDK.

Let's have a look at what kind of platforms does FBK SDK support:

1. Windows
2. Windows Store and Windows Universal App Platform
3. Mac
4. iOS
5. Linux

Do them contain the AR platform you are using?

 

Some use glTF now for streaming (is designed for it), open spec, and there's a FBX to glTF converter that has worked well for us so far with some pretty simple buildings in city models. Is a commandline tool so you could maybe run it as a part of your AR solution somehow.

https://github.com/cyrillef/FBX-glTF

Apparently there is active work on a Unity glTF runtime loader now too,
https://github.com/KhronosGroup/UnityGLTF

 

@zwcloud i notice this thread has been going on for a while... so have you written this asset yet? where can i grab a beta?

p.s. fbx importer for Unity WebGL would be great!

especially if it supports blend shapes (SkinnedMeshRenderer)!

 

Thanks for your attention. But as what I have replied before, that's impossible because FBX SDK doesn't support WebGL. Fbx is not designed for runtime loading. It is just an interchange format between 3d modeling softwares.

 

Port fbx parser to WebGL?

Also, skinned mesh renderer? (particularly blend shapes)

 

The biggest problem is that fbx is a proprietary format owned by Autodesk. There does exist some fbx parser that works for certain fbx files but I don't think it's possible to make those parsers work for Unity-supported fbx especially complex data like blend weight, blending bone index and animation curves. (Note: Unity itself is using the latest FBX SDK by Autodesk.)

 

Okay, what about fbx import that supports blend shapes in skinned mesh renderer in general for the other platforms - iOS etc

 

That's possible because FBX SDK supports iOS, but not Android. The price of a FBX importer specially for iOS platform will be a little high since there was a windows-only one which is already $250. Could you describe the case why you need a runtime iOS importer with more details?

 

Blend Shapes for playing with iPhone X face tracking

Also your Windows-only asset is priced a bit high and people have left reviews of stuff you might want to fix (last review is from 3 years ago). Have you added support for importing blend shapes in skinned mesh renderer?

 

Thanks.

That's not mine.

------

I just need to investigate a bit more to determine whether to write a runtime importer for ios. I will reply here if there are any progress.

 

Maybe try another route other than FBX SDK as other WebGL frameworks can definitely handle fbx https://threejs.org/examples/webgl_loader_fbx.html

 

What progress have you made since you started this thread?

 

Nothing. Because there are alternative solutions to fbx runtime loading/importing and there are not enough requests.

 

there does not appear to be a good 4/5 star asset for runtime fbx loading

 

Yes we have use for it ... ideally what we would like from a 3D model loading tool kit is as follows

1) Runtime ... this is to support modding so runtime is key, The ability for it to stand alone is a nice to have/extra e.g. for standalone modding tools to hand out to our players
2) Freindly licensing ... its for modding thus we would want to plaster the tool anywhere we could
3) Support as wide of an import format as possible e.g. FBX, OBJ, etc. ... OBJ we already have; I bother to post because FBX is a commonly used format that we do not have the ability to work with at the moment.
The key point here is that this tool handle importing of these various formats in a similar way e.g. resulting in similar output, limitations, etc. Consistancy of the resulting models is more important supporting every feature one can define in FBX or anything else for that matter.
4) Supports PC, Mac and Linux

I suppose what we are asking for then is a tool that would support runtime use in a Unity game for at least Win, Mac and Nix that can consume FBX, OBJ and as many other formats as you can manage and outputs some unified (as in regardless of input type output has same set of features, limitations, etc.) mesh ... i.e. basicly what Unity does now only at runtime

Very importantly this tool would need to be very robust, e.g. it must fail gracefuly in such a manner that it does not crash the game and preferably in such a manner that it could be reset and reloaded during the same runtime session. Things that come to mind are resource monitoring ... never let this tool hang on disk IO or flood RAM, etc. again its for modding ... will be used in strange ways by strange people to do strange things
If I was going to approch such a thing I would create a tool that was actually seperate to the Unity game that was using it that the game could pass a path which targeted a given file to and would report back to unity its progress and eventually the path to a resulting 'cleaned and redy for use in game' file this way even if it did outright faceplant during the heavy lifting it wouldn't have had any impact on Unity at all. Unity side would need tooling to pick up the resulting file and use it and some quality of life toolign in this area would be nice e.g. components to handle loading and unload, setting up of materials,etc. but that is all nice to have stuff

 

I've seen this one on the asset store:

https://assetstore.unity.com/packages/tools/modeling/trilib-unity-model-loader-package-91777

would that work?

 

Hey you are good with that stuff, you say, and I have a challenge. There is something that has been going on for years that just bothers me to the core. It makes absolutely no sense to me. If its money I would pay more than 2x to cover the difference just for the ease on the dev end.

Look UMA, Morph3D, and theres a couple other minor ones out there. You have blend shapes. You can turn a friggin human into a dog. WHY ON EARTH DO I NEED A MALE AND A FEMALE. WHY CANT I HAVE ONE THAT CAN GROW BOOBS AND ONLY NEED 1 GEAR FOR EACH THING INTEAD OF A MALE AND FEMALE ARMOR.

That has stumped me and annoyed me for so long I almost tried to learn 3d art, but I can't draw.

 

Well, generally it's because males, females and dogs are all shaped differently, and they all need polygons in different places in order to be able to deform better. Plus the humans/dogs skeletons behave differently (they way the bones move). So if you had a much higher polygon budget, and a more complex rig, this would all be doable, but at the cost of performance.
There's nothing stopping you from doing this in UMA right now, actually. You could use the "female" UMA, and create your own DNA for it that allowed you to make it more masculine (and of course swap in more male textures). I believe that the original UMA was both male/female actually. Its just that you can make it look nicer (and cheaper to render) with two different base models.

But this is off topic for this thread. If you would like to discuss it further, I recommend using the UMA thread, or going to the UMA slack.

 

On the why do you need a 'male' and 'female' re-morph and similar you dont strictly technically need them. The blender esk Humans tool which does similar to the others you noted just has a 'human' mesh. it can grow more than just boobs as well e.g. its anatomically correct. The way it does this though isn't very efficent re poly count and will cause stretching of textures ... though I think that could be solved by simply re mapping the UV with the morph.

Basicly all humans have the same mesh its simply inflating/deflating them to give the different geometry. Now in our typical use cases male and female have different armatures as well ... they share a common base e.g. human of course but we use bones to introduce psedo softbody to hair, belly, breast, etc. point being male and female while roughly the same are not similar enough to share the same mesh, UVs or armature with out some wasted bits in both cases.

What we do to get the best of both worlds is use these morph tools to get something close to what we want then pull that into a more traditional tool to optimize the resulting mesh as our 'base' for each 'race' as they would be called in UMA. What would be nice is a tool that could understand that or perhaps a better workflow if someone has one to help streemline that e.g. right now its do the morph thing, export it to a format your modeling tool likes, do the clean up, export that to a tool your texturing/skinning tool likes, do the skinning/baking/etc and export to that to a format Unity likes ... wouldn't it be grand if there was a middle layer that plugged into Morph, Human, Maya, 3DS, Substance, Unity, etc. that could at least aleveate the file management pain in the butt part of it all.

 

We need simple fbx run-time importer for AR (iOS, later android too).
It is for furniture shop/showcase app.
This will load - 3d fbx models from "database" hosted on web.
Then inside Unity, based on user selections for materials, variations etc. dynamically 3d GameObject will be constructed and placed on the scene.

 

Any News about this ?

 

The company i´m working for provides a set of tools to designers to right visualize and to homologate models of its clients.
We need to load at runtime this models on WEBGL, and int the future, Windows 10 apps.
Today we use GTLF format file load models at runtime.

 

Yeah we also need a runtime importer for WebGL. I am trying various different OBJ runtime file loaders one for GLTF but fbx would also be good.

 

Why unity does not load fbx at runtime yet? This is done at editor time already.

 

I dont know it would be alot better if we could just import files at runtime rather than having to pack each thing in asset bundle (packing it different for each platform).

 

this not support blendshape and doesn't work by large texture on android

 

So, 5 years later, I finally decided to write a runtime FBX plugin without FBX SDK. It's still at an early stage of development.

Here are two preview videos of basic save and load function of static model:

Now I'm adding data compression support.

 

Updated video:

The lite version of RuntimeFbx is about to publish on asset store!

 

Skeleton hierarchy loading has been implemented:

This video displays a skeleton hierarchy loaded with RuntimeFbx Pro. The one with mesh is created by Unity. The one without mesh is created by RuntimeFbx skeleton loading function.

 

SkinnedMeshRenderer (rig, namely bone weight/index and bindposes) loading has been implemented:

 

A difficult problem when handling vertex attributes has been solved. Here is a mesh with color attribute correctly imported by RuntimeFbx Pro (WIP):

 

A SkinnedMeshRenderer created by RuntimeFbx Pro from a fbx file. Triangles(indices), vertex position, color, bone-index, bone-weight, bones and bindposes are loaded by RuntimeFbx. The animation is loaded by Unity.

 

hello
This is the feature I need.

Is it possible to load FBX in WebGL?

 

Yes. You can try RuntimeFbx Lite for loading a static mesh. It should support any platform since it's written in pure C#.

 

hello,

can you expose an API for exporting materials and textures?