Feedbacks for experimental feature: GameObjectRecorder (2017.1)

Sounds interesting, this is for recording cut scenes or similar videos?

 

I can see this being really useful for recording complex physics simulations and then playing them back. Say having a house collapse - you can set it up on a powerful computer and add a ton of forces and other complex stuff to make the thing fall apart in a natural looking way, record the details, and then play them back cheaply, so you can get as good looking stuff on cheap hardware.

Like how DK Country got away with looking way too good for the snes.

For things like that, it would be nice if there was an easy way to specify a scaling of the recording. Like:

Code (csharp):

1. Time.timeScale = .1f;
2.  
3. //record something that only runs smoothly at 60fps if we're recording at .1 times the normal speed
4.  
5. m_Recorder.SaveToClip(clip, .1f); //this would generate the clip at 10 times the recorded clip

This can be solved by just playing the clip at 10x speed, or by manually multiplying all of the keyframe's time by .1, but having this directly in the API would probably make that smoother (and more obvious to users!)

 

No, I was thinking purely from a performance standpoint.

Like, I'm doing something that's going to make the game run at 5 fps, so I want to record at 1/12th of the normal speed and then speed that recording up by 12 times so it looks like everything's happening smoothly.

I'm not sure if fiddling with the timescale fixes that. I know that some AAA companies (not to be named) record teaser trailers by stepping through frame-by-frame and then playing those frames back at 60fps to make it look like their game runs smoothly.

That's the obvious use of a recorder for me. I mean, record something that's way too expensive and then play it back cheaply, not lying to my customers.



Out of curiosity, how much is it viable to store? I know that eg. Braid did a ton of compression to make it possible to rewind time a lot, so if you're doing something like that, just naively making keyframes every second is not going to give good results. Thoughts on that?

The Super Meat Boy scenario looks like it'd probably just work straight out of the box, which is nice. Ghosts like that are also nice to have in racing games, but that will require the system to handle very long animations. Is there any upper limit to AnimationClips in Unity?

 

Hey this sounds interesting, could it potentially be used to save gameplay replays?

 

This looks like a great feature! I wish it'd been available last year, would have saved me heaps of time.

Am I right in thinking this can save a hierarchy of transforms, i.e. a skeleton, easily?

 

I mean as in "recording", sorry that wasn't very clear. I like the word "yet" in your response

Out of curiosity, why would a humanoid rig need to be handled any differently than any other hierarchy of transforms?

Great job again!

 

Great stuff. Thanks for the response and keep up the good work!

 

Can it be used to bake Cloth physics? example: Record a flag waving for a few seconds and save it as a clip so we could remove the Cloth component and save performance.

 

For me this would be really useful if it worked in the editor-less Player Because today I got the task of recording the motion (and spawning) of everything in our project, to be saved to disk. Then later we want to play back exactly what happened, fast forward to a specific part, and then let the project continue running.

It’s a research project, so the goal is to be able to play back how an event went down, change one part, and then see how that affects the outcome.

 

This looks very promising. Can't wait to get my hands on this.

Question, would this be able to record blendshape values on a skinned mesh renderer component?

 

I implemented a very similar system that stores the movement and arbitrary values on components animation files. Yes, this could be made to record replays as long as you record the time that gameobjects are created and place the animations on them.

However, there is a problem. These replays get really big, really fast. I had a rather simple Asteroids game the replays would just be enormous. I even wrote an algorithm that simplifies the curves when saved but still, even then, they get very large. I wanted to record a number of 5-minute for an attract mode or something, but it was just generating too much data. But Asteroids is kind of a worst case scenario with this type of replay. There are so many little objects being created and tracked with many bullets, small asteroids, etc.

 

Is there any possibility you could use such a system as this to also record / bake particles? I seem to remember an article about the making of Zombieville USA and how they created cheap particle effects for mobile by baking animations in Maya or some such program and exporting that to Unity instead of using a real-time particle system. So it would be handy if we could also bake particle simulations, just like physics simulations, directly in the Unity editor. However, I'm assuming doing this might be difficult and somewhat outside the scope of such a feature because you would probably not be simply recording properties, but instead converting special particle system mesh and transform data into something else entirely in order to "bake it"?

 

Just tested it. Works great. Now we can record real-time animation. This is fantastic. Great work guys.

 

Any chance you know when this will move out of experimental? We would love to use this in our production plugin.

 

Not in this case, at least not for me anyway . I did actually try binding a ParticleSystem to a GameObjectRecorder already, but it just seems to record the initial ParticleSystem settings which are useless. I'm now guessing you can probably create some kind of advanced script that records each particle's state using ParticleSystem.GetParticles() or something, so I guess it's kind of possible, but it's not exactly simple .

 

Hello @RomainFailliot ,
Do you think that this feature will also work for not-editor scripts as well soon? Or is it possible to achieve something similar thing like creating animation in runtime? I was planning to record game sessions with this feature.

Thanks,
Emre

 

Hello,

I have some feedback for the recording system. One such application of mine is to record all the 3D transform data of moving full body avatars as well as the 3D transform data of any virtual part of an assembly model that is grabbed and moved by the user in the game. I also want to be able to dynamically load those recordings directly back into the game after finishing recording.

My problem is that I have to use the Animator Controller Mecanim stuff, which doesnt allow me to access the specific key frames of the recorded animation. I also am having a hard time dynamically creating states with necessary transitions in the state machine via scripting. I've searched and found that only in Legacy could you just attach an animation to an object and play it, but now you have to do the Animator Controller stuff which is causing a real headache when it comes to coding for it. I'm seeing you can script entry and exit transitions, but not a simple transition between states? Maybe this exists but I've looked far and wide. I've only found the ability to script a transition between state machines, not just states though. Help?

Also, as for the recording animations in the 2017 beta, might I suggest that it would be incredibly useful to allow a childed object to be recorded in world space, not just local space relative to its parent. Because as you might see, if the child is recorded but not "moving" in respect to the parent, then it's never getting any keyframe data. I can't unchild the object due to my specific uses, and I can't put the GameObjectRecorder script on the parent (because I know it gets the root and all its children). It must specifically be on that one child and that's it. When I grab objects, they become childed to my remote, and it would be nice if I could then do the "add property" function in the animation to start recording the transform data of the object, but the moment you let go of the object, that added property then goes away and you lose it. A fix with this would be incredibly useful.

I would just find it incredibly useful if there was a simpler way to access the keyframe data in the animation itself during the game via script. The ability to rewind, pause, play, and fast-forward these animations at different speeds while in game is incredibly interesting and useful to me.

 

Yeah, I would also be most interested in a run-time implementation of this, since it would open up a lot of possible game-play features, like instant replays. I immediately thought of something like the demo recording tools for the Source Engine when I first heard of this. In fact, I just realized you could probably already use this in a similar manner to the Source Filmmaker to record game action in the editor and then tweak and edit the results in Timeline or something.

 

Hey thanks for the heads up UziMonkey I was thinking it would be cool to have a replay of you could change the camera in after the fact but sounds like it will be a huge pain to build and the file size might be prohibitive.
I did see one cool replay system in a unity game http://turbodismount.com/
My app has a crazy mechanim network though so that would probably be another reason not to try making one

P.

 

@petey @UziMonkey @DreamPower There was a very interesting presentation for the Entitas framework where they created a replay system that required very little data storage, even for pretty complex games. I wont pretend to completely understand how it works, but the key seems to be having a deterministic simulation and only recording the actual user input or events. Then the replay system can use this input/event history to recreate everything at any point in time without any additional data needed. This is however quit different from recording animations and not at all useful for a non-deterministic simulation, like physX.

 

well i tried this for recording animations and using it with new timeline tool. it works quite well but still need some improvements. workflow is crippled right now.

also, i guess we can use this for cheap motion capture animations in engine with Kinect SDK I haven't tested yet but it's promising, I will share results once I test and build a stable and fluent way to record motion capture animations in engine.

 

I'm interested in the fact that it saves properties in the hierarchy. If this can run in play mode, then we can use this as a way to save game state. We would only need to save one frame.

 

So we have finally gotten back to our Unity plugin and started building a feature around this new API. Works well in regards to recording real-time blendshape animation values. However if you try to modify the resulting .anim file or just reload the editor, the transform animation still looks smooth but the blendshape animation no longer is smooth. Rather it only animates on each keyframe, displaying the animation value that was recorded on that frame. All movement between frames is lost for some reason and the resulting animation is extremely choppy. Is there anywhere to alleviate this issue?

 

UPDATE:

After looking at it for the day I noticed that the keyframes created for transforms vs blendshapes are very different. Transforms seem to set keyframes whenever a change to that property occurs while blendshape values, regardless of change at runtime will always keyframe every 50 frames. Please note that I am using the same sample code from the original post, just binding a skin mesh renderer as well as the object transform. It is also worth noting that blendshape keyframes are keyed as "Constant Broken" when the transform keyframes are keyed as "Free Smooth". This could explain why when I reopen the animation the blendshapes values are choppy.

But what I am wondering is why blendshape values are being sampled every 50 frames rather than on every frame their values change, similar to the positional and rotational values.

 

This is how most replay systems work. It's not difficult to set up at all, the problem though is when games get non-deterministic. Frames don't always happen at the same exact time in a game due to different framerates and inconsistent framerates, this makes it very difficult to simply record input and get the same result every time. And if your game uses physics as you mentioned... good luck.

 

i am very interrested by this feature and will try it!

i already used a script found somewhere to record anim but it is better if it is part of unity ! (unityAnimationRecorder) that work great in editor, but without keyframereduction.

it's easy to reduce keyframes on a animation deleting no-change keyframes, or small changes. but i am not a math genious to create smooth curves from linear keyframes!
(i wished the keyframe reducer used in modelImporter could work with a simple generated animation.)

I used it to record VR movement to animate things in cutscenes easily.

 

Is it supposed to record Active property? From my tests, it doesn't.

 

The same happens for values on custom scripts.

 

Hey I was just thinking, pairing this add on up with this > http://docs.unity3d.com/ScriptReference/Animator.StartRecording.html
Could make an awesome replay system.
What do you think?

 

Very interesting, I'm trying to create an iOS app, and I need to animate a character at runtime, so the user can move every part of the character and record the movements and store or save on a ".anim" file, pretty similar you are talking about but, it is possible edit those animations on runtime and save or store again?

 

FrameRecorder with animation recorder (alpha) is out: https://github.com/Unity-Technologies/GenericFrameRecorder/releases/tag/0.2.0003

 

Did someone manage to bind and record Mesh.vertices?
I've been trying for days but I just can't get it to work.

 

Has anyone found a workaround for non-transform values only recording sporadic frames? Is it fixed in newer versions of unity? I'm on 2017.1.0p4

I'll try storing the data while recording and stuffing it into the clip after.