Feedbacks for experimental feature: GameObjectRecorder (2017.1)

I haven't received a bug for it yet, and I must admit I don't really understand this 50 frames bug. There is nothing forcing the recording at 50 frames per second (AFAIK). If you're having this bug, please report a bug with a project in it so that it's easy to reproduce. The easier it is, the faster it'll be fixed.

The only related bug I do have though is that the GameObjectRecorder is sampling at 60 frames per second, and it would be nice to let the user set its own FPS.

 

Unfortunately I can't submit the project due to NDA. 50 frames is just what I recognised above, my team's first interpretation of it was that it was just sampling once every second. It only seems noticeable when we are turning objects on and off. I

I guess that doesn't help much but I thought I'd mention it!

 

@Velcrohead I completely understand the NDA problem. I'll try to find this bug considering your description and @Crazy-Minnow-Studio's.

That being said I don't have much information about it. Re-reading the previous posts, it seems to be recording at 50 fps when recording blendshapes. Whether you use Update() or LateUpdate(), the result is the same. Is that it? Do you have more information to share maybe?

 

I don't know if it's related, but 1-2 months ago when I recorded blendshapes with this, it appeared to be fine until reopening the Unity project, then the recorded animation went to 1 fps.

 

Yeah absolutely, thank you for the help.

We have a hierarchy of a lot of objects, and I am binding the transform of each child, along with the isActive property. The objects are all mechanical so the animations are usually pretty straight forward, just moving and rotating.

The actual movement is being run by coroutines in play mode, and the recorder is saving the transforms. The transforms themselves seem to be running fine and accurately. When we turn change a gameobjects activity though, it seems to sometimes lag behind.

For example, an arm reaches down to pick something up. Something is supposed to turn off as it lifts it up immediately, but there is about a second delay before it toggles.

Looking through the animation, it seems like the animation is only taking key frames of the isActive property every second (or 50 frames as described above), whereas we were expecting it would do it frame by frame. We are calling TakeSnapshot every frame on the LateUpdate as it does on the documentation page.

Hopefully this is of some use!

 

Thank you @Velcrohead

There are a bunch of passes that are done before saving the animation. One of those is the key reducing pass.
That's why even if you take a snapshot at every frame, it's possible that some keys are "reduced" in the end.

That being said, this key filtering has a low threshold, so it doesn't lose much data (i.e. it does test the keys with an epsilon, but the epsilon is very small). It may be that there is a bug in the key reducer though. I'll look into that too.

 

Thank you for your swift responses, Romain. Really appreciate your help

I assume any fixes you might potentially make would have to wait for a new Unity update?

 

Sadly, yes. I'd like to make a package out of the GameObjectRecorder, but it still needs internal functions that can't be exposed as is.

 

Okay so I think I have a workaround.

We have seriously long animations that needed to be recorded - with a complex hierarchy (one of the animations is an hour long). I'd split the recording into clips of 180 seconds which is when the accuracy seemed to be lost (taking a keyframe every 50 seconds).

I've reduced the clip times to 15 seconds and it's pretty much perfect now.

TLDR ; reduce the clip lengths

 

@Velcrohead: On my side, I've been digging in our code and found something as well. The "one key every 50 frames" is actually a limitation on our side. In order to prevent an n^2 algorithm to take too long, we decided (a loooong time ago) to arbitrarily limit the number of keys reduced in the same batch, and this limitation is at 50 frames.

The good news is that, although the key reducer doesn't reduce all the keys, it doesn't destroy the data either. In other words, the key reducer could be more effective, but it doesn't change what's been recorded.

Now that the "one key every 50 frames" behaviour has been eluded, I'd like to know more about your bug. Why did you need to reduce the clip time? if the clip time is greater than 15 seconds, you start losing data?

 

15 was just an arbitrary number I was using to see if it would make a difference (as its a signifcant change from 180). It was only a fleeting thought which I decided to follow up on, but seemed to make the difference!

There are a lot of objects being animated in the scene, I guess I thought it might be trying to optimise it behind the scenes if it was too much data.

 

It's ok, 15 seconds or 25 doesn't really matter at this point

I'd really like to know what's happening at 180 seconds and above though. Could you take a screenshot of your bug without breaking the NDA, like the curves in the animation window?

Thank you

 

Yeah i absolutely can - ill get something over soon.

My original issue however seems to be repeating itself and it looks like im about to lose a weekend to it! Im hoping you might have small idea if anything is going on behind the scenes.

I have a series of 8 long animations that need to be recorded. For each one i create 40 or so smaller animation clips. I wrote a script to track the time and reset the recording after 15 seconds and change the clip it saves to. When the recordig is finished i place all of the animations into a timeline to play it back.

When i record the first animation and play it back afterwards, it is entirely accurate and the objects turn off at the correct time.

I record one or two more, and they all play back fine.

When i seem to record a fourth (and potentially build the application), some of the animations lose their ‘accuracy’. There are ansolutely no changes to the oroginal animations, but they stop working. (This is verified through source control, zero changes to any previously made files).

All of the timeline assets are in one main scene and get selected and applied to the playable director on the object. I am wondering if its a memory issue and should have each timeline and animated object in its own scene to be loaded in dynamically.

Do you have any idea what could be going on?

 

There seems to be a lot of parameters involved actually, so it's difficult to figure out what could be the source of the bug from here.

I see two solutions: either you can reproduce your bug with cubes and share that here, or you can share your assets directly with me (romainf@unity3d.com), if your NDA authorizes it. Be aware that we can't share anything the users share with us, so your safe with us, but I guess you'll have to talk to your legal department).

 

I am seeing this reduction to 1 sample per 50 frames too. I am capturing a cube's MeshRenderer.enabled property. This issue is really causing me a problem and it would be nice if some solution was available. Is there a workaround or will a fix be released in the semi-near future? Thanks all!. PS. Is there an existing defect that I can attach this to so?

 

Hi @MikeChr,

Why is it a problem exactly?
The additional keys are extraneous, but the generated clip should still play correctly in the end.
It would be nice to have a key reducer algorithm that would reduce even more keys, but it's not intended to fix this issue right now, especially since it doesn't break anything (on the other hand, changing the compression algorithm could have a huge impact so it would need a lot of tests to be sure no regressions have been injected).

 

Sorry if I'm missing something in the conversation, but I'd assume most people interested in recording animations would be expecting 60 fps rather than one sample per 50 frames.

 

Oh, ok

Well, the additional key every 50 frames is hardcoded, but it has no link with the actual FPS of the clip.
Basically, it's completely benign. It's a by product of the compression algorithm; because there can be tangents on keys, we have to sample each frame to verify if the result is the same with and without the candidate key. This leads to a n^2 algorithm and 50 frame is a subjective value that sounded reasonable, because a maximum of 2500 samples (50^2) between two keys seemed like an acceptable value.

Now that the mystery of the key every 50 frames is solved, some people still have a problem with the resulting clip, they have a result that is apparently different from what they expect. But I need more information in order to fix this problem one day, or maybe one day I'll stumble upon the problem and will be able to fix it.

 

Is this still for editor only? Is timeline for when it will be able to create instant replays in games?

 

Hi Romain. Thanks for the reply.
The problem I am seeing goes like this. I am recording a simultaneous change to both an object’s position (translation) over the duration of a curve of ~100 frames and an immediate transition from hidden to visible (enabled) at the start of the curve. In play mode, both movement start and enabled occur on the same frame. However, the recorded playback is no longer in sync. The enabled transition lags the start of the translation by about 50 frames and the object appears about half way though the curve. I will try and better characterize the situation today and get back.
Mike

 

@eobet Yep, it is still editor only. there's no timeline for a player version at the moment. I know it's a very frequently asked feature, but for now it's impossible to create animation clip at runtime in Unity. Fixing this first part implies quite a huge refactorization that is not yet planned. Also, I'm not sure the GameObjectRecorder, as it is today, is the solution for realtime recording. For instance, it will consume a lot of memory while recording, which is ok on a machine running the editor, but could be problematic for players that can run on different, smaller form-factors.

@MikeChr Thanks, I think I start to see a potential cause of that. I'll need to do some tests of boolean curves over a long period. Don't hesitate to contact me if you have more information.

 

@MikeChr Thanks, I think I start to see a potential cause of that. I'll need to do some tests of boolean curves over a long period. Don't hesitate to contact me if you have more information.[/QUOTE]

Is it possible to record the render enable state as an integer (0,1) curve and would that solve the problem short term?
Thanks!

 

I don't think it would solve the problem, because both boolean and int curves are treated as float internally. I think it could be a problem with the constant tangents, since the FBX importer can't import such curves, the GameObjectRecorder is the first one feeding the key reducing algorithm with constant keys. (that is actually my only lead on this possible bug)

 

IMPORTANT POST

Hello!

We're thinking that it's time to un-experimentalize the GameObjectRecorder.
Do you see a problem with the API? Do you have a bug? Now is the time to express yourselves!

Here are the bugs that I know of:

• Potential bug with boolean curves (but no repro yet)

Here are the features that I know of:

• Recording humanoid characters (don't know yet if it's going to make it for v1.0, I think it will depend if it will need to change the API or not)
  
• Recording with the player (not happening for v1.0, but just to let you know that we're aware of the request)

Thank you!

 

IMPORTANT POST (2)

Hey all!

We figured out a way to record humanoid characters and it will have a small impact on the GameObjectRecorder API. Thanks to that, we will be able to push the GameObjectRecorder out of experimental and implement the recording of the humanoid in the next version.

@MikeChr Do you have more information about the boolean bug you've hit? I'll start working on that before officially get the GameObjectRecorder out of experimental.

Cheers!

 

Good news everyone! (and especially @theotrian, @Crazy-Minnow-Studio, @Velcrohead, and @MikeChr)

I found the boolean bug!
I have a fix in progress right now.
As soon as it's done, I'll put the GameObjectRecorder outside of Experimental.

Beware that it fixes the bad key reduction that was happening on the boolean curve, but not the keys every 50 frames. The rational behind is that the boolean bug was producing bad animations, while the 50 frame bug is "as designed" and don't alter the quality of the animation.

 

Hi can this be used at runtime.
For example I want to record user action and than use it to replay same action to other. Similar to macro.
Rahu

 

Hi @P_Chufal,

Nope, the GameObjectRecorder doesn't work at runtime because it's not possible yet to create an animation asset at runtime in Unity.

I know this feature is often requested, but I'm not sure the GameObjectRecorder, as it is, would be the right solution because it uses a lot of memory while recording and it can hang the engine while saving. Also, replay might need game specific tweaks since you won't have the animation events.

 

Hi Romain,
I encounter a bug when I record an audiosource, it must be the same with every components having float values.
Everything works great when I record / play my animation but when I close / reopen Unity then there is no float anymore, but only 1 or 0 values.

There was a bug report about it, I voted for it and even let a comment saying it was solved in Unity 2018 but I am testing it again on the 2018 release and I still have the bug.
I can't find the bug report anymore, do you see what I am talking about?

If not I'll make a new bug report, it is easy to reproduce.

 

I think you (Unity) should keep this as editor tool, and focus to it as it is. Sure there is demand for runtime recorder, but that kind of thing is more game specific. This tool can be used for any games, no matter if the game needs runtime recording or not.

Perhaps this can be used to make (runtime) game recorder, but that should be another plugin.

 

Could you file a new bug? Because I think I see what you're saying, but I still think it's been fixed in 2018. A bug report with all the Unity information + repo steps would be absolutely marvelous

I think you're right, sir

 

Hi Romain,

I made a bug report from the editor called :
(Case 1041964) The GameObjectRecorder transforms float values into int values once the editor is closed for non animation components

There are easy repo steps, you should find something to work with ;-)

I am not english native so I hope everything is well understandable.

Take care

 

OMG! How cool! It's July 2018 already! any news about the recorder getting out of experimental?

 

@TrueZerg It has landed in our main branch a month ago, it will happen for 2018.3!

 

I'm so excited, Ibeen experimenting with unity for animation(although its a game engine) since the introduction of CINEMACHINE and Timeline, plus this! it's a complete package! only question I have now is.... will it record blendshape changes? cause I've tried the original code(from page 1 in this thread) and it seems to record TRANSFORMS only, the blendshape changes I've made (ones under skinnedmeshrenderer) doesn seem to get recorded .

 

So that include the recording of humanoid animations? Was really impressed with my tests of it so far

 

Nope, sorry, the need for a stable release was more important than this feature. But, on the bright side, we have a plan on how to do it now, and without breaking the API, so it would arrive in a future v1.1

 

so it willn ot record blendshapes yet? :/

 

@TrueZerg I've tested with 2018.2.0b11 and I've managed to record blenshapes.

Here's the code:

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEditor.Animations;
4. using UnityEditor.Experimental.Animations;
5. using UnityEngine;
6.  
7. public class Recorder : MonoBehaviour
8. {
9.     public AnimationClip clip;
10.  
11.     private GameObjectRecorder m_Recorder;
12.  
13.     void Start()
14.     {
15.         m_Recorder = new GameObjectRecorder(gameObject);
16.         m_Recorder.BindAll(gameObject, true);
17.     }
18.    
19.     void Update()
20.     {
21.         m_Recorder.TakeSnapshot(Time.deltaTime);
22.     }
23.  
24.     void OnDisable()
25.     {
26.         m_Recorder.SaveToClip(clip);
27.     }
28. }
29.  

I did a video for proof

 

Wow ...! thank you! This is awesome cant wait it's official release! what a grea tool Unity is... I recently started with Unity
As a amateur animator is just a bless! thank you and all the people behind this improvements!

 

Btw is the version of unity you used? (2018.2.0b11) the same as the beta wich is available for download at the unity website?, Can I download it and try the code in that?

 

Yep, that's it, I used a publicly available Unity beta to make this video. Since then 2018.2 has officially been released, so I advise you to use this one instead.

FYI I used BindAll (recursively) which might be a bit intense for your use case since it will record EVERYTHING. You might want to bind just the SkinnedMeshRenderer component using BindComponent, or even just the blenshapes in it using Bind (but it's a bit trickier since you need to know the internal properties name, as you can see in the example in the documentation).

 

Thanks!!

 

I recently found about the RECORDER saset... does this asset records keyframes too? on it's description it says it records animation :/ or maybe just video?

 

If you're talking about the Recorder package from the Asset Store, yes, it can record both animation and video.
For the animation it uses the GameObjectRecorder as a backend.

 

Hi, I have some problems with recording custom components.

1. Recording does not always work
2. Custom component order on the target gameobject influence which component is recorded

All this is visible on the video. First I try to record the component which I could record before but this time it doesn't work. Then I change component order on the gameobject to show that other component is recorded than the one binded. At the end I revert the component order and the component which wasn't recorded previously, now works.

Test project: https://1drv.ms/u/s!ApyPqduXdyo-kqRN3ebkAyO2N_hbgg

I'm using Unity 2018.2.0f2.

 

Thanks @metroidsnes !

Your bug description is very good and giving the project as well is excellent, though could you use the Unity bug reporter? you'll find it in the menu, under Help -> Report a Bug

It has several advantages like being able to track it (both for you and me), it gives a bit more information on your Unity release, and you can send your project as well with it. You can post the bug link here afterwards.

In the forum, your bug might get lost at some point since it's very difficult for us to track anything here.

I hope you understand
Cheers!

 

I have a problem with the bug reporter. Each time I try, it gets stuck at around 50% upload. Is there any other way to submit the bug?

 

It should have worked :/
Don't worry, I'll enter the bug myself. Thanks for trying though

Edit: it should appear here in a couple of minutes: https://issuetracker.unity3d.com/product/unity/issues/guid/1063505/

 

Good news @metroidsnes, I fixed your bug
It will be in 2019.1 (because 2018.3 is being branched off right now).