shape deforming

Hello every one,
I want to ask please for the best way to make a swapping eyes for a robot model,
so i have a robot game, the robot has a face screen, which displays different modes, normal, happiness, sad, mad etc.
i can make images and just swapping between them depending on the mode of the robot, but the problem is that the swapping proccess will be just an image disappears and another one appears,
How can i make somthing smoother than this ?
Like mesh deforming depending on an image, or shader that reforms smoothly to match the new targeted image
i mean somthing like the eyes here in this video :

thanks in advance

 

Mesh deforming based on images sounds quite unorthodox to me. Although also a possibly complex subject, you could look into 2d facial rigs and make something like this:

 

Definitely not an easy task. I have spent quite some time thinking about this too since I have a similar robot usecase.

Thing is, does it have to look.. humanly? Because an easy cop-out that works for characters that accentuate that they are robots is a glitch effect. Blend in a form of flickering or zapping of that screen as a transition between the frames.
Inspiration for this and why this effect is part of pop culture is the channel switching of CRT screens.

Personally I've just ended up doing an alpha fade since my robot is a companion and not main focus.

One thing you could try however in the age of AI: Frame interpolator of videos. On an animation I got good results. It was actually able to meaningfully transpose the sharp edge of objects in the interpolated frames.
The software I used was thisnone as it has a .png mode: https://nmkd.itch.io/flowframes
For something like a hand it already failed though (hands will forever be the bane of AI, won't they? )
It could work well for abstract faces like robot ones though.
Then you'd have transition animation between all variants and can use some statemachine to move between them.

Last but not least, with the Animation2D package you could set up something similar to blend poses. That's what I ended up doing with the hands of a 2D character.
That means have single-keypoint animation clips of the different poses and use the animator statemachine to interpolate between them.
Note that you can not only rotate bones but also move and scale them - that'll be useful for such stretching.

 

@Paul-van-der-Laan yes mesh deforming based on images is really complex, im looking for another solution,
The eye swapping process in the video (at second 24) is like the one i already made, it's just swapping a sprite with another one, but the customer wants it to be smooth switching, like if the eye is smiling eye, and will be the normal one, so it will (deform) to fit the intended new shape of the normal eye

 

@DragonCoder nice idea, i will give it a try,
could you share an example video of the result please ?

 

There are examples on that linked page!

 

This is problaby more relevant to your use case:

Although it seems he uses multiple constraints made in Blender so you would need to figure out a way to do the same in Unity which might not be easy.

 

You could just make a few frames of glitch animation and put that in between each facial expression.

 

That is pre-rendered movie where one frame can take hours on render time.

In this particular case, there are probably splines attached to eve's head which determine shape of her eyes, and they work as a mask for the "screen" below.

The least painful way to approach this is how stray did it - they have flipbook animation for robot faces. 3-4 frames are enough to portray movement. And yes, "glitch" effect can work too.

You could replicate the full effect, but I wouldn't bother. For example, you could render robot face off-screen on a large square using polygons, while animating it with blendshapes. You would render it onto a render target, then feed it into a shader that paints dots. There's no warranty whatsoever that it will look good, though.