Volumize 2D Video

Hello!

I am trying to take a MP4 video (2D) and move it around in 3D space but, as the video is moving, it leaves behind a 'snapshot' of the previous frame. I like to think of it as similar to particle trailing, but the trail is a video.

Is there a good way to do this? I'm pretty new to Unity so any solutions/advice would be greatly appreciated!!! Thank you!

Currently, I have a Quad with a RenderTexture of the video attached to it and it can move in 3D space.

I haven't found any documentation on how to use a 'Trail' but replace the trail with a video, but that was one of my ideas for this.

 

parameter on camera clearing
'depth only' and a black background (IIRC) will need to be assigned to overwrite color buffer onFrame for the overlaying cam

this is why images leave a print on the Background using renderToTextures

 

Hello,
To clarify, I WANT to figure out how to make the image trail. Currently it does not, but I would like to figure out a way to make the video trail behind the 2D quad as it moves.

 

oh a GOOD way to do it i see. The depth residual is frame based so it lacks smoothness

Yes simply speaking, but easy no it is not 'easy' like you hope. At least nothing I have seen

if you want the trail to be a residual of the pixels it 'was' you may want to instead leave particles (1 particle for every ~2-4 pixels) that adopts the color of the video and holds it over X seconds so that the video actaully seems to leave a ghost like wisp of itself. Getting the emitter to only emit on the side that is the 'back' needs a bit of a script to figure that out IIRC

you say '3d" space so I am imagining you want the video to be capable of tumbling full 3D not locked to screen like a 2D quad? (note even the Trail renderer will break visual parity - it will not (out of the box) have a parameter to scale it's width to correspond to the video when it move diagonally.)

It should do it as a base test to see if the concept could work like a comet + its tail, however you need the trail renderer to apply the video smeared from the last 10% of the video edge which is going to be a complex shader. And no procedural geo system currently can trail an objects 'edge' because there is no easy-off-the-shelf way to grab the 'trailing' end of an object

1) Ribbon is the term for 3D procedural geo so that the trail 'tumbles' and exists in the same plane if the video is actually moving in 3D. (Trails are strictly 2D to orient to cam) so you need this concept and it needs to be capable of trailing from the RTT object silhouette which requires your own procedural geo construction.

2) the next hurtle will be to sample portions of the UV's (based on velocity) and apply a distortion to the UV's to get the remaining pixels to drag across the tail; - It needs a falloff between image & tail ribbon otherwise it looks like a bad streak to the eye if it moves too fast.

none of this seems easy imho; advanced tech art

it might be easier to emit 4 trails on each edge of the RTT quad, but make the shader Stencil buffer on it so that it looks like 1 objects, but still applying the textures 'correclty' that will be a complicated UV/screenSpace corretion calc i think.

frankly, this is outside my skillset at this point but perhaps the RTT can be it's own post-processing shader that takes the non-clear depth residual and can smooth it more directly in the direction of the objects motion on screen,,, all that without extra geo systems in place that use CPU

 

Thank you for the advice!

Is it possible to leave a ‘snapshot’ of the video at a certain location rather than have it trail? Like as the quad moves it leaves a picture of the video at the Quad’s location every 5 frames? Currently I am keeping the video in a set orientation and just changing its position (rotation along the XYZ axis will come later).

also if possible could you please elaborate on the abbreviations, I am still quite new to Unity

thank you again!!!

 

RTT - Render To texture
placing clones of the video on screen may be a good use of your RendertoTexture as-is

this is outside my skillset but just from looking at documentation

1. Having a number of pooled objects knowing FrameCount / perFrame = clones ready
2. The pooled instances have the video initialized and synch but with the flag to hide
3. toggle on and pause their playback when you need a clone and stop moving it.
4. leave that version alone so it stays in the RTT screen space and you can fade them out with a simple shader.

i am not familiar enough with the instancing of that feature to know if it batches or if transparency is not vertex bound so that it could be.. if not you will need to keep draw calls in mind

if there is a more flat process by pre-computing the frames you want and ripping them off the video into a textureArray that might be more optimized during playback