Video Transcode VP8 video with transparency at runtime on Android

"Keep Alpha" is a property of the Video Clip Importer, and the recommended way of enabling transparent backgrounds on Android, but I'm looking to stream a .webm video directly from a remote URL using the Video Player API. This works perfectly on iOS, but obviously not on Android.

How do I overcome this issue? Is there a solution?

 

Is there a third part alternative that supports streaming/downloaded transparent video? I'm hoping to find a solution ASAP.

 

Hey did you find a solution?

 

We really need an answer for this, can any of the developers tell us if it is possible please ?

 

Hi, Currently there is no support on android. You can use a chroma key shader to render transparent video. Or you can have alpha channel in video and write a shader to get the alpha for the video.
Android needs transcode in unity for transparent video. IDK if there will be any runtime transcoder for transparent video for android.

 

Best way would be to put RGB and A separately "side-by-side" like in a stereo-video and use a shader to re-combine them I guess. Depending on the agressiveness of the video codec you might introduce some new artifacts which might be a dealbreaker.

 

Hi all,

At this point I've tried chroma-keying, side by side video, and a custom video transparency solution.

Hilariously, for my use case, all were less performant and more memory intensive than just using a sprite sheet.

I would recommend image swapping or sprite sheets if possible.

Best,
Aidan

 

I've implemented a side-by-side video solution in the meantime, works great. Guess we have different use cases as I would not convert our ~1 minute videos to sprite sheets

 

What kind of video is it. if it is mp4. you witll have to export the video from its source program as transparent. then when its in unity u have to click the video in the assets folder and transcode the video specifically for the device u want to deploy on i think the codec is ETC somthing but try those settings. but if it is a .mov file then the transparency will also happen when you transcode if you dont it will either have a white or black background. try it and let me kno please

 

could you provide some code example on how you did this? any other solution on how to get transparent videos working on Android would also be appreciated

 

I couldn't find my original code but this looks similar to the side-by-side alpha I did: https://github.com/keijiro/AotsMovieTexture

You can also buy this asset for advanced chroma keying
https://assetstore.unity.com/packages/tools/particles-effects/chroma-key-kit-71684

 

Thank you very much for the links! I tried with the AotsMovieTexture and indeed it seems to work on Android with a transparent webm movie loaded from url. I'll run some more tests and will post an update with adjustments that might still be needed.

 

so here’s my solution in case anyone needs this as well. there are two steps:

1. convert the webm to a side-by-side video. I do this on a node server. might be a bit off topic but here’s the basic code anyway:

Code (JavaScript):

1.  
2. import path from "path";
3. import fs from "fs";
4. import ffmpegInstaller from "@ffmpeg-installer/ffmpeg";
5. const ffmpegPath = ffmpegInstaller.path;
6. import ffprobeInstaller from "@ffprobe-installer/ffprobe";
7. const ffprobePath = ffprobeInstaller.path;
8. import ffmpeg from "fluent-ffmpeg";
9.  
10. ffmpeg.setFfmpegPath(ffmpegPath);
11. ffmpeg.setFfprobePath(ffprobePath);
12.  
13. // note: data received from upload with multer
14. async function createVideoWithAlpha(data, dataPath, outputPath, height) {
15.   try {
16.     height = Math.round(height);
17.     if (height % 2 == 1) height++;
18.   } catch (e) {
19.     console.log("error " + e);
20.   }
21.   const newFileName =
22.     data.filename.substr(0, data.filename.lastIndexOf(".")) + ".mp4";
23.   const rgbPath = path.join(data.destination, "/rgb_" + newFileName);
24.   const alphaPath = path.join(data.destination, "/alpha_" + newFileName);
25.   const alpha2Path = path.join(data.destination, "/alpha2_" + newFileName);
26.   await createRbgVideo(dataPath, rgbPath, height);
27.   await createAlphaVideo(dataPath, alphaPath, height);
28.   await resizeAlpha(alphaPath, alpha2Path, height);
29.  
30.   return new Promise((resolve, reject) => {
31.     ffmpeg()
32.       .input(rgbPath)
33.       .input(alpha2Path)
34.       .inputOptions(["-filter_complex hstack"])
35.       .withOutputFormat("mp4")
36.       .on("error", async (err, stdout, stderr) => {
37.         // handle error
38.         return reject(err);
39.       })
40.       .on("end", async (stdout, stderr) => {
41.         // handle success
42.         return resolve();
43.       })
44.       .save(outputPath);
45.   });
46. }
47.  
48. function createRbgVideo(dataPath, outputPath, height) {
49.   return new Promise((resolve, reject) => {
50.     ffmpeg()
51.       .input(dataPath)
52.       .inputOptions(["-vcodec libvpx"])
53.       .size("?x" + height)
54.       .withOutputFormat("mp4")
55.       .autopad()
56.       .on("error", async (err, stdout, stderr) => {
57.         // handle error
58.         return reject(err);
59.       })
60.       .on("end", async (stdout, stderr) => {
61.         // handle success
62.         return resolve();
63.       })
64.       .save(outputPath);
65.   });
66. }
67.  
68. function createAlphaVideo(dataPath, outputPath, height) {
69.   return new Promise((resolve, reject) => {
70.     ffmpeg()
71.       .input(dataPath)
72.       .inputOptions(["-vcodec libvpx"])
73.       .outputOptions(["-vf alphaextract"])
74.       .size("?x" + height)
75.       .autopad()
76.       .withOutputFormat("mp4")
77.       .on("error", async (err, stdout, stderr) => {
78.         // handle error
79.         return reject(err);
80.       })
81.       .on("end", async (stdout, stderr) => {
82.         // handle success
83.         return resolve();
84.       })
85.       .save(outputPath);
86.   });
87. }
88.  
89. // need to make sure both videos have the same height otherwise it will throw an error
90. function resizeAlpha(dataPath, outputPath, height) {
91.   return new Promise((resolve, reject) => {
92.     ffmpeg()
93.       .input(dataPath)
94.       .size("?x" + height)
95.       .withOutputFormat("mp4")
96.       .on("error", async (err, stdout, stderr) => {
97.         // handle error
98.         return reject(err);
99.       })
100.       .on("end", async (stdout, stderr) => {
101.         // handle success
102.         return resolve();
103.       })
104.       .save(outputPath);
105.   });
106. }

2. in the app and if on android, I load the converted fallback video instead of the webm. The video prefab is set up pretty much as in the Aots Repo, just use VideoPlayer component with MaterialOverride and put the TransparentMovie material on it. If you set aspect ratio make sure to divide it by 2 (as the video has double width with RGB + Alpha channel)