MegaCache - Object & Particle Sequence Playback [Released]

Introduction
MegaCache is an editor extension of for the Unity game engine, it allows you to import cached animated mesh geometry regardless of the topology, vertex count, material use etc of each mesh in the cached sequence as well as importing Particle systems exported from 3d packages. The system can accept a sequence of OBJ files where each file can have a number of objects all with different materials, MegaCache will import the sequence of files merging multiple objects and material sub meshes into a single mesh object for you, you can then play back the imported sequence either by asking the system to directly change the mesh using its stored list of generated meshes, or the sequence can be read from a generated cache file, or finally be read from a memory based image file. Play back of the animation is blazingly fast no matter which option you choose with the system making use of multiple threads where possible to further increase performance. You can also tell MegaCache to optimize the data that is imported reducing the memory use to under a 1/3rd of storing raw mesh data.

MegaCache is perfect for exporting complex animated meshes that are not possible to recreate using skin and bones, such as fluid simulations, fracturing objects, cloth simulations or anything where the vertex count, face count, materials etc change during the course of the animation. It is also perfect for playing back complex animations made using modifiers or deformers in 3d packages, all the animation can be baked down to a sequence of frames using standard OBJ sequence exporters which are available for all the 3d packages.

Included in the system are components to allow the import of a sequence of OBJ files and play those back as well as an OBJ reference component where multiple objects can playback a sequence from an already loaded source so dozens of objects can all be sharing the same data but playing back at different speeds or positions. Also included are components to playback imported particle simulations either on the Unity Shuriken particle system or the Legacy system.

Particle Simulations
As well as importing sequences of Mesh geometry MegaCache can also import and playback particle systems such as Particle Flow simulations from 3ds max. Again the system will optimize the memory use for imported simulation and gives you complete control over the playback of the particles, selecting emit rate, scales, speeds etc and works alongside the existing Unity Shuriken and Legacy particle systems allowing you to add even more detail to the finished particle animation.

Particle Exporters
At release we have available an exporter for 3ds max to exporter Particle Flow simulations, we also have a beta Maya exporter for Maya particle system exporting, please note the Maya plugin is still a work in progress and does not yet support all the features. If you require the exporters please get in touch with the invoice number for your purchase and we will send the exporter of to you. MegaCache now also supports the Maya .pda particle cache file format for the direct importing of Maya Particle systems.

Features

• Import OBJ file sequences
• Import Particle systems
• Point Cloud Import
• Support Maya .pda particle files
• 3DS Max particle exporter
• Blender particle exporter
• Blazingly fast playback
• Optimized memory use
• Merge multiple objects
• Playback from file or memory
• Multi threaded code
• Full Source Code Included
• Works on all Platforms
• Works in Unity Free

Example Videos
This video shows the Object cache system playing back a fluids simulation as well as a fracturing object animation.

This video shows the worfklow of exporting a sequence of OBJ files from a 3d package, in this case 3ds Max, and then importing them into Unity using the MegaCache OBJ system.

The next video shows a Particle Flow system being exported from Max and Used in Unity.

Available Now

MegaCache is available now on the AssetStore or on our Website for the early adopter price of $50

 

Another great looking system Chris, so can I check that the OBJ cache will allow me to play back an animation of a bunch of objects with multi materials where the vertex and poly counts and even material count change from frame to frame, so for example I had a cube with 6 materials that I blow up in max so that the next frame has a dozen fragments where a new material has been introduced for the internal fracture faces, that would work fine? That is what I really need to make use of the RayFire plugin I have for Max to play back my destruction animations.

 

That would work fine, it doesn't matter how the vertex or poly count changes for each frame in the sequence, also it doesn't matter if every frame has different materials the system will import and combine those with no problems, it will even merge multiple objects into a single object for you as well, so it should easily handle anything you can throw at it.

 

How does this version differ from the included version in megafiers?

 

Any chance MegaCache can be used to load point cloud data? I recently picked up a Project Tango tablet and am exploring ways to visualize XYZij files. Thanks.

 

@HeadTrip - The basic version in MegaFiers will just import OBJ sequences, it will not import particle animations and any further options added to the system will be for MegaCache only, so as we get feature requests or support other file types etc they will only appear in MegaCache.

@contempt - Should be easy enough to add support for point clouds, will take a look at the file format and see what can be done. Most likely could be linked into the Particle system for the best display options for each point. Do you have an example XYZij file?

 

Just submitted a small update to the Asset Store, this has added the option to update a mesh collider, if present, for the Object Cache and Object Cache ref objects, please note that to update Mesh Colliders in Unity is a slow process so it may be too slow for complex meshes. Also added a runtime folder option for cache files, so if you need to use cache files loaded at runtime you specify the folder in the inspector and after building place the cache files you used in that folder for the system to be able to read them in at runtime.

Changes in v1.02

• Added option to update mesh collider for Object Cache.
• Added option to update mesh collider for Object Cache Ref.
• Added a runtime folder option for OBJ Cache files, cache files for standalone builds need to be copied the data folder for the build after the Unity build.

 

Here is some info about the format from Google's Project Tango docs:
https://developers.google.com/project-tango/overview/depth-perception

I'll post a sample file when I get in to the office today. Thanks!

 

Thanks for the link, I don't have access to any device that would generate the data so would need a file or two to test with.

 

Sorry apparently there is no export option in the current builds provided by Google, I have to modify the source to generate the .xyzij file. I've almost got it working...

 

Just a quite note to say there was a small problem in the last version in the demo scene the data source for the Cache Object was set to File by mistake so the demo scene was looking for a cache file so the scene would not play correctly, submitted a new version where the data source is correctly set to 'Image' so the cache data is stored on the object so it will play back correctly. Sorry about that.

 

Just submitted a very small update to the Asset Store to fix a bug with the particle importer expecting files with a .txt extension whereas the Particle exporter uses the .prt extension.

 

Another small update has been submitted, this fixes a couple of issues int he Particle Inspectors and also we have added a beta exporter for Blender allowing the export of particle systems from Blender to be used with MegaCache.

Changes since v1.02

• Fixed bug in Particle Inspector
• Fixed bug in Legacy Particle Inspector
• Added a beta exporter for Blender to export Blender particle systems to MegaCache prt format.
• Fixed bug with particle importers wanting .txt extension instead of .prt generated by the Max exporter.
• Demo scene was saved using a file as a data source which would not work without the cache file. Demo scene now uses an 'Image' data source.

 

Just submitted an update to the Asset Store, this adds a beta of a new component that can import CSV point cloud data to be displayed by a particle system, so can display Lidar style data. The csv format is the basic x,y,z,intensity values, you can load a sequence of csv files and have them playback via a particle system. The component is in beta so let us know what features need to be added and possibly other point cloud file formats we can support. To use the new component create a Point Cloud object from the GameObject/MegaCache menu, add a Particle system component. Then import your files using the Import Data section of the Point Cloud Inspector by setting your first frame and last frame and naming, then click the Import button and select any file in your sequence, the system will then import all the data. In the inspector you will see the value for the Max number of particles in a frame, set the Max Particles value in the Particle system to that value or higher and that's it, When you press play you will see the point cloud data. You may need to adjust the import scaling values depending on your scene settings, there are scale options to adjust the cloud size at playback as well.

Also MegaCache is fully compatible with Unity 5.

Example Point Cloud
The new Point Cloud playing back a Point Cloud animation with over 12,000 points per frame.


Changes in v1.06

• Smoke in demo scene working correctly.
• Added a beta of a new Point Cloud import and playback system.
• Works with Unity 5

 

Oh man you rock!!! I'm going to test this right now!!!

 

Just submitted a Unity 5 version to the store so that 5 users no longer get a warning when downloading the asset.

 

Another quick update submitted to the Asset Store, this sees the addition of a beta of a new particle playback component that is designed for playing back discreet one shot particle simulations for example explosions, the existing particle system is designed for playing back continuous emissions such as smoke and dust simulations.

Changes since v1.06

• Added option to particle import to remove particles alive in last frame from import.
• Improved particle importer to remove particles that only have one frame of data.
• Added a new Particle Playback component for direct particle playback as opposed to the emitter version, means one shot effects like explosions etc can be used as opposed to constant effects like smoke.
• Added playback component for legacy particles as well.
• Removed the unused function LoadTexture which was causing a compile error when building for webplayer.
• Fully Unity 5 compatible.

 

I know you don't have to use the same number of vertices for this, but if you were to use models with the same number of vertices is there any difference in performance between using Mega Cache and using Mega Fires point cache vertex animation?

 

If you use the raw mesh data source for the playback then MegaCache will be a lot faster than the Point Cache in MegaFiers since all it is doing is changing a mesh pointer each update instead of updating an array of vertices. Using the other data sources ie cache file or image will be similar performance as that will be updating vertices.

 

The new particle playback system is great, was wondering how to get my explosion effects working with the system but the latest update added just what I needed, thanks so much again for your great assets and listening to the needs of the users, can't wait to see what you release next MegaMesh? MegaCar? MegaSpring?

 

No problem, if you need any other features just post or email. Didn't get much interest in MegaSpring so not sure a the moment what the next system will be, getting on with our own game.

 

It is maybe not what you are planning for. Do you consider to have MegaCache for Shuriken or Particle Playground support? I recently try to figure out how to have big or more Particles, but with cheap cost, especially to avoid Fillrate issues as much as possible. I wonder if your system could do anything about it.

 

@ikemen_blueD - MegaCache does already have support for importing Particles from Max and Maya and using the Shuriken or Legacy particle system to play it back, it gives you control over the size and number of particles you want to play back from the saved sequence and you can scale the whole system and other effects. If though you have a more complex particle system in your 3d package then most 3d packages allow you to turn the particles to meshes and you could then export those as an OBJ sequence. The video below shows the particle playback system, is that what you mean or did I misunderstand the question?

 

hi there, from what I understand Mega Cache can import cached animated mesh geometry, made by 3D Software, like 3DS Max or Maya, turn them into a single mesh object, and being played nicely by Shuriken or Legacy particle system.

What I wonder do you have a similar system or any plans to create one. A system, that can boost Shuriken performance, especially focusing on Mobile device. For example, I have a quite big, lots of Particles, made by Shuriken (not by 3D Software). I wish I have a system, similiar like Mega Cache, can pre-catch Shuriken movements or turn Shuriken particles to one Opaque Mesh or something awesome, which I don't know about, but can boost the overall particle performance greatly.

Just a suggestion, right the time I saw Mega Cache, I wish I could have Shuriken Cache or something similar.

 

I just bough mega cache to import a sequence of objs. They are basically 2 cubes flying through space. When they meet a boolean union happens. However exactly at the frame of them touching for the first time the import stops. Is mega cache not able to handle this, or am i just missing an import option?

I have a attached a few of the objs:
https://www.sendspace.com/file/asvml0

Update:
So I found out, that it has something to do with the way maya exports obj sequences. It seems to work fine with 3dsmax.

 

Hello SpookyCat,

I'm willing to buy your product MegaCache for an animation i did.
I would like to ask, currently does your plugin supports Blender 3d particle simulations -and how is working? I 'mean I should export the particles as obj.sequences or there is a Blender add-on for this job. In one of your previews posts you've said:

"and also we have added a beta exporter for Blender allowing the export of particle systems from Blender to be used with MegaCache."

How does it happen? Thank you in advance!

 

Sorry for missing the messages here, Unity does not send out repeat alerts it seem.

@ikemen_blueD - That is any interesting idea to cache Unity particles for quicker playback, I would have to do a test to see what performance boost one could expect, I imagine the backend of the Unity particle system is well optimized.

@1dealist - There should be no problem with Maya OBJ sequence export, we have used the system with Maya OBJ files, can you let me know which OBJ exporter you are using so I can do a test? I will try the files you linked to.

@artsunward - MegaCache works with any OBJ file sequence so Blender would be no problem, as for particles we do have a beta exporter for Blender particles, if you could maybe send me an example file I can do a test for you to make sure it works. But if your particles are also meshes then the OBJ sequence would also work for that, but the particle playback has more features.

 

@SpookyCat I'm using C4D to generate all my content for this unity project my OBJ sequences. When I load them into the Mega Cache game object I get the following error. "Can't load file" see attached screen shot.

I have also attached a C4D file that shows what Im trying to export. I have baked all the animations and tried to make this a simple as possible. Any assistance would be greatly appreciated.

I've tried the native OBJ exporter, the Riptide OBJ exporter and the Plexus OBJ exporter. Nothing seems to be working. I am wondering if there is something with the coordinate systems that each software uses. I have heard that C4D uses a slightly different system than Unity. There is of coarse user error that may have occurred when baking things. But the 12 keyframes I need are in the .zip attached.

I've also tried the Mega Fiers with a MDD file which was working at one point but now is not either.

Please help. As in most cases, I'm up against a deadline that is fast approaching.

-TOMbot

 

Just purchased this great tool! Thank you for creating it. Question though, where is it that the Blender particle exporter is available to download?

 

@3Duaun - Thanks for the purchase, hope its proving useful to you. To get the beta exporter for Blender you just need to submit a support ticket at our website or email us, don't forget to include your invoice number.

 

Just submitted a small update to the Asset Store, this makes MegaCache fully Unity 5.3 compatible, it also fixes a couple of bugs and adds the missing method GteNextFrame which was missing when being used on target platforms that do not support multi threading.

Changes in v1.10

• Added missing GetNextFrame() method when target platform is one that does nto support multi threading.
• Fixed error in the OBJ Ref editor script with a missing GUI call.
• Fixed mesh not updating when multiple OBJ ref objects used the same frame data one after the other.
• Made Unity 5.3 compatible

 

Another small update to MegaCache in the Assetstore, this fixes a roblem with the inspector for the Cache Objects when you did not have the Data Import section open an error would appear in the console. This update also brings a beta new feature, we have added support for the importing of Maya .pda particle files, this is a much better solution than using the custom exporter for Maya as we can now more easily import more of the particle data from Maya. To use you just need to use the Mel command line to say what you want to export such as:

Code (csharp):

1. dynExport -mnf start -mxf end -os 2 -atr position -atr velocity -atr particleId  -atr lifespanPP -p "path" -f "pda" objname;

Where start and end are the first and last frame of your particle sequence, path is where to save the files and objname is the name of your particle object in Maya. You can add and remove attributes as required but currently MegaCache only uses, velocity, position, particleid, lifespanPP, radiusPP. You can find more details on the export options at http://download.autodesk.com/us/maya/2011help/Commands/dynExport.html

When you have your sequence of .pda files you can import them into MegaCache by using the Import Maya Particles button in the inspector, above that you will see Start and End values so you can pick your own bit out of any sequence.

Changes in v1.11

• Fixed Layout errors in console when Data Import section in inspector was closed.
• Added new beta feature of importing Maya .pda particle files. Use the MEL command
• dynExport -mnf start -mxf end -os 2 -atr position -atr velocity -atr particleId -atr lifespanPP -p "path" -f "pda" objname;
• Where start and end are the first and last frame of your particle sequence, path is where to save the files and objname is the name of your particle object in Maya

 

I'm working on a project that imports over 5000 OBJ frames but some of them have different models and materials assigned which are not always present in all the frames. In the Mesh Renderer portion of the Inspector for the MegaOBJ game object I get a warning that "This Renderer has more materials than the Mesh has submeshes. Multiple materials will be applied to the same sub mesh, which costs performance, consider using multiple shader passes." Is there a way to access the sub meshes for a frame and assign the appropriate material to each? Rebuilding all 5000+ OBJ files to contain all the submeshes required will take a long time. All my other frames can use the same single material assigned to any meshes/submeshes so I've been based on frame swapping the material for the OBJ game object and it works except in the circumstance where each of 4 submeshes have different materials. Any advice would be greatly appreciated. Thanks.

 

@contempt - When MegaCache imports all the objects and materials it checks for duplicate materials by name, is it possible that some of your obj files are using materials with the same name when they are different? Is it possible you could submit a support ticket with a small set of OBJ files that show the problem so we can do some testing, not had any other reports of materials not being applied correctly so would like to see if we can track down what is causing it in your project.

 

Just submitted a small update to the asset store, this fixes a bug with the ping pong animation of the Point Cloud system and also adds a new Point Cloud XYZ component which will read xyz files that contain the position and color of points. A little example below.



Changes in v1.12

• Fixed the pingpong not working in the Point Cloud animation.
• Added a new Point Cloud Object, this one will read xyzrgb files so you can have colored point cloud particles.

 

Just submitted a small update to the Asset Store, this fixes a bug where a mesh would not display if multiple materials were found in the OBJ sequence but load materials is not selected. We also removed some Debug.Log messages that were left in in the last update and MegaCache is now fully 5.4 compatible.

Changes in v1.13

• Fixed bug where mesh would not display if multiple materials were found in the OBJ sequence but load materials is not selected.
• Removed Debug.Logs from the import code.
• MegaCache is fully 5.4 compatible.

 

Another small update submitted to the Asset store, this adds a fix for any obj files where the material names contain the character ':' this would stop the materials being imported as you can not use the ':' in any Unity asset name. We also fixed the material importer setting the alpha values in any color to 0 instead of 1.

Changes in v1.14

• Import filename format now displays the split character in the inspector.
• Material importer now removes any ':' found in the material names, ':' can not be used in asset names in Unity.
• Material importer now defaults alpha value to 1 instead of 0 is alpha component missing, this stops meshes from being invisible after import.

 

New version submitted to the Asset Store, this update will now detect OBJ files with no normal data and will now compute the normals for each frame. Also added an option to force the normals to be calculated even if they are present.

Changes in v1.15

• Option added to force calculation of normals on import, this will override any normals in the OBJ files.
• OBJ files which contain no normals will now have normals calculated on import.

 

Hi.

I'm looking into using MegaCache for an exhibition involving Augmented Reality and I was wondering if there is a way to randomise the playback of the OBJ sequence?

I am trying to create a "glitch" effect on objects, so the animation will play normally for x frames and then jump to a random frame (OBJ) which is distorted, then go back to the beginning. I was thinking I could structure the imported OBJ sequence so that frames 0 - 49 are normal, then when the current frame equals to 50 the script tells the playback to display to a random frame between 51 - 60 etc and then back to frame 0.

I hope this makes sense and would be open to any ideas? Loving MegaCache and its potential.

many thanks

Darius

 

Can possible thinking particles (3dsmax) import in MegaCache particle system?

 

bump

 

@3d_Artist1987 - I don't own Thinking Particles so can't test but if they derive their particles from the Max particle class then in theory the exporter should be able to get the values needed for export, I can't do a test though.

 

Thanks for reply.

 

Just submitted an update to the Asset Store, this removes obsolete warnings for Unity 5.4 and 5.3, note you will still get warnings regarding the legacy particle system, we will be removing support for the legacy particle system when Unity 5.5 is released if you don't use the Legacy particle system you can just delete the source files with 'legacy' in the name. We also changed the way materials are imported so now if you happen to use an OBJ sequence exporter that changes the name of the material files to be different from the OBJ file names you will now see the materials import correctly, before the importer relied on the mtl files having the same name as the obj files, now it will use the mtllib attribute in the OBJ files to get the correct material file name, this will fix the problem Cinema4D users were having with materials not being imported.

Changes in v1.16

• Improved the material importer so will use names in mtllib instead of expecting same filenames as obj files.
• Updates for obsolete particle values in the Unity 5.4.
• Note in Unity 5.4 you will get warnings about the Legacy particle system, support for Legacy particles will be removed when Unity 5.5 is released

 

Another quick update, fixed a couple of small bugs as well as adding an option to change the Y Up when importing particles and point cloud data, so if your data imports on its side you can turn on the Y Up Change option. We also added support for PLY files to the Point Cloud system.

Example of a PLY point cloud imported via MegaCache.


Changes in v1.17

• Particle size now takes the startsize from the Particle system into account for more control over the particle size;
• Removed redundant FrameNum slider from Particle Pro component.
• Added option to Particle Pro components to change the Y Up on import, turn on if your particle files are importing on their side.
• Added support for PLY files in the PointCloudXYZ component, just single files supported at this time.
• Added option to Point Cloud components to change the Y Up on import, turn on if your point cloud files are importing on their side.
• Fixed bug that stopped frame number slider working it Show Paths was selected.

 

Hi @SpookyCat

I was about to purchase Mega Cache, just a quick question really.

I will import low resolution Ink drop simulation from Blender to Unity for mobile devices. Will it work for mobile devices?

Thanks

 

@sagarkar - I am not a Blender expert but that video seems to use something other than a particle system, if it just uses a particle system then it should be able to be exported but it does seem that the file size could be very very large with the number of particles making up the scene and the length of the animation.

 

Thanks @SpookyCat

How about the same effect in Maya, according to this video?

 

@sagarkar - Sorry I missed your post, I will see if I can do a test on a similar file.

 

Just submitted an update to the Asset store and our webstore. This brings some improvements to the PLY importer, namely it will now check that the PLY file is in ascii format. The PLY importer as also been greatly sped up and now also streams files during import instead of loading the whole file into memory first, this means less memory is used during import which might help with large files.

Changes in v1.18

• Added check for PLY files to error if not Ascii format.
• PLY point cloud import now uses a stream the PLY data during import instead of reading whole file into memory, reduced memory required for very very large PLY point clouds.
• General Point Cloud import now also streams during import instead of reading whole file.
• Added Point Cloud and Point Cloud XYZ to the component menus for easier adding to objects.
• Fixed some exceptions with Point Cloud XYZ with empty image files.
• Sped up import of PLY files.
• Particle Skip value now taken into account for PLY data import.