[Released] AVPro Video - complete video playback solution

Decided to abandon 360 audio in the first iteration of our app hence my late reply. However I'm now trying to implement it again and I still have some problems Could you elaborate a little bit more on how to set this up on Android? Do you have any example projects? Or maybe an example MKV file? Also, I've read that HLS streaming of MKV files isn't supported in ExoPlayer (https://google.github.io/ExoPlayer/supported-formats.html) but is it possible to stream it using MPEG-DASH?

Thank you,
Johan

 

Ok thanks for the reply. As I was picturing it working, we'd ask AVPro to open a file, when it opens, it dispatches an event, with some basic metadata, including codec, bit rate, dimensions and any other info that we might not have readily available.

 

Hi thanks for your reply. It did help but we did run into another problem. When switching the ExoPlayer the switching of the audio tracks worked perfectly but the video playback wasn't as smooth. So what we did in the end is have the video playing on the media player and the sound playing back on another file, blank video and sync it then have that play back using the ExoPlayer. This worked for our purpose

I noticed that someone else mentioned image support. This is something I would be very interested in. The system used to load png or jpg files in Unity3d is pretty poor. Firstly it has a maximum resolution size but doesn't have any error report relating to this so there is no way of telling what the largest size is and secondly if runs on the main freed. So whenever you load in an external image it freezes unity3d for a few seconds. Incredibly annoying for VR.

So I very much would like to see image support in some way

Apart from that AVPro is the best plugin by far and good to see you are still working on it.

All the best

Phil

 

Hello,

I seem to be experiencing an issue on GearVR w/ Samsung Galaxy Note 8 (SM-N950U). The application crashes at the beginning of playing a video. Has anyone experienced this? Logs below to show error message(s).

-I am running Unity 2017.2.2p4
-Android version 8.0.0
-Have the latest version of AVPro
-Does not occur on other samsung devices

Code (JavaScript):

1. runtime.cc:506] Pending exception java.lang.RuntimeException: Error during updateTexImage (see logcat for details)
2.     runtime.cc:506]   at void android.graphics.SurfaceTexture.nativeUpdateTexImage() (SurfaceTexture.java:-2)
3.     runtime.cc:506]   at void android.graphics.SurfaceTexture.updateTexImage() (SurfaceTexture.java:243)
4.     runtime.cc:506]   at long com.RenderHeads.AVProVideo.AVProMobileVideo_GlRender.Blit(android.graphics.SurfaceTexture, float[]) (AVProMobileVideo_GlRender.java:327)
5.     runtime.cc:506]   at boolean com.RenderHeads.AVProVideo.AVProVideoPlayer.Render() (AVProVideoPlayer.java:943)
6.     runtime.cc:506]   at void com.RenderHeads.AVProVideo.AVProMobileVideo.RenderPlayer(int) (AVProMobileVideo.java:166)
7.     runtime.cc:506] Dumping all threads without appropriate locks held: thread list lock mutator lock

 

Hey there.. I was just checking out the AVPro Video trial and I must say you did a great job on it!

I do have a Question however...

I am trying to display a video file from a local REST server on a simple mesh and it does display just fine running in the Editor. Once published to the device (Hololens) however all I see is a placeholder animation where the video should be!? Is that normal?

I did check already the documentation and setup everything as described but no success. The Hololens does have a proper and stable network connection and trying this with a different video player solution does work as well, that's why I am a bit clueless here.

Regards

ps: I should mention that our REST server will not output a filename with extension and rather give back generic names like Media1 and Media2.. i.e: http://192.168.128.18:5010/Media2

 

Hi all, I want to offline my current build, the files I am playing are hefty 1.5gb. My offline device is oculus go but Im having trouble finding where I can place the videos and then locate via the build. I want to avoid Resources and Streaming Assets as I have 5/6 videos and this would kill the apk and install. Ideally if I could throw them into the video folder of the device that would be perfect. I should add that this isnt distributable so I am not concerned about different platforms and file will always be standard location (if that makes sense). Anyone worked with local videos etc?

 

I'm getting a 2.4+ second hitch when loading a video from a URL. Is there a way that this can be performed asynchronously and not stop the main Unity thread? This is obviously unacceptable for VR use.

 

That shouldn't be happening.. Calls to open the file should be asynchronous. Perhaps you could email us (www.renderheads.com/contact/) at the unitysupport address with more details such as:

1) Unity version
2) AVPro Video version
3) The platform you're seeing this on
4) A copy of the video

Thanks,

 

Thanks for the feedback! Image support isn't high on our list I'm afraid.. I do see how it would be useful though

It's strange that you had worse experience with ExoPlayer...I'd be interested to see that...

Cheers!

 

Thanks for posting this. Please could you email us (www.renderheads.com/contact) at the unitysupport address. It would be useful if you could send us a reproducible case as this isn't something we've seen before.

Thanks,

 

Hi,

It's a little difficult to know what is going on without more details. What format is your video file? What does this 'placeholder animation' look like? Do any of our demo scenes work for you on your Hololens?

Thanks,

 

We are currently handling this via email. Suggested using an absolute path (eg /Storate/emulated/0/MyVideos/MyFile.mp4) and also persistantDataPath

 

Yes it does look like MKV isn't supported in a DASH stream...
An example MKV file can be found in the Facebook Audio 360 SDK here:
https://facebook360.fb.com/downloads/rendering-sdk-v1-5-0/

You should just need to point to that file, set the video API on Android to ExoPlayer (in MediaPlayer > Platform Specific > Android) and set the Audio device to Facebook Audio 360. Then in the MediaPlayer > Audio settings you can set the head transform. We're just about to release a new version of the AVPro Video plugin which includes the latest ExoPlayer and FB Audio 360 libraries included.

Cheers,

 

Hi All,

AVPro Video version 1.8.8 has just been released!

You can find the updated version on the Asset Store and the free trial version on our website.

Changes include:

Android

• Upgraded ExoPlayer to version 2.8.1 from 2.6.0
• Upgraded Facebook Audio 360 to version 1.5.0 from 1.4.0
• Fixed audio track 1 not being set initially on ExoPlayer
• Removed unneeded debugging and logging code on ExoPlayer

Windows Desktop

• Upgraded Facebook Audio 360 to version 1.5.0 from 1.4.0
• Fixed bug where videos using FB Audio 360 wouldn’t stop at the end of the video
• Fixed a bug where using D3D9 and DirectShow and resizing the window or switching fullscreen mode would cause a crash

macOS / iOS / tvOS

• Fixed an issue in Unity 2018.1 and above that caused the projects not to compile

WebGL

• Fixed issue where Buffering and Stall events weren’t triggered correctly

General

• Fixed garbage generation in DisplayUGUI component
• Added support for Stall event on platforms that don’t natively support it
• Added documentation notes on seeking and playback rate

Please report any issues here on the forum, or to our customer support email.

Thanks,

 

The video files are mp4 and they are working in the unity editor. The "placeholder" animation looks like AVPro.png and AVPro1.png from your Resources folder.

Your demo scenes are working fine and like I said, our video files are playing fine in our scene when I try it in the unity editor.

Regards

ps: I tried the new version you just published but the same problem.

Here is a screenshot of what I am seeing instead of the video when using the hololens...

 

It looks like it isn't properly detecting the platform....

You're building for UWP 10 right?

I know people have used the plugin with Hololens before..so it should work.

If you look in Assets/AVProVideo/Scripts/Components/MediaPlayer.cs
Check that at the top of the file, that the line defining AVPROVIDEO_FORCE_NULL_MEDIAPLAYER is commented out

also check line 1438, it should be the method CreatePlatformMediaPlayer()
in there line 1461 is where it should be entering for the UWP platform, so you could add some logging there to make sure it is. It could be that the call to WindowsMediaPlayer.InitialisePlatform() is failing for some reason.

Also please check your log file/console as it should log an error if the platform isn't found, and it will log information about which platform it thinks it is, which could be useful.

Let me know!

Thanks,

 

Hey,

yes I am building for UWP 10.0.14393 (to be exact) and the line with AVPROVIDEO_FORCE_NULL_MEDIAPLAYER is commented out as you said.

Ok.. I am getting this output now:

Exception thrown: 'System.DllNotFoundException' in Assembly-CSharp.dll

[AVProVideo] Failed to load DLL. Unable to load DLL 'AVProVideo': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

[AVProVideo] Not supported on this platform MetroPlayerX86 HoloLens (Microsoft Corporation) Intel(R) Atom(TM) x5-Z8100P CPU @ 1.04GHz Windows 10 (10.0.14393). Using null media player!​

Regards

 

Hi,

Thanks for the extra information. I've just done a Desktop UWP test here and it all seems to work fine. I don't have a Hololens so unfortunately I can't test that

You say that the other demo scenes work fine on the Hololens? If that is true, what is the difference between the demo scenes and your scene?

The error log indicates that it can't load the DLL. I wonder if it's loading the correct version as there are three: ARM, 32-bit and 64-bit. Perhaps you could run it via Visual Studio and confirm that you're building to the X86 (32-bit) platform?

Let me know how it goes - I'm keen to help you get to the bottom of this.

Thanks,

 

Andrew,

yes, it looks like the DLL can not be loaded.
I am building for X86 however. As far as the demo scenes go, I can only see one difference...
We are passing a URL during runtime to your AV Player, your demo scenes only load files locally if I remember it correctly.

 

Interesting... Did you enable the capabilities to allow for network access?



Thanks,

 

Hello.

Can you help me with advice?
I'm just trying to play transparent .MOV file with latest version of plugin, but it can't be loaded and next error appears:

File is OK, playing with QuickTime Player, path is correct.
Which codec I should install?

Windows 10, QuickTime installed, Unity 5.1.0f3 x32 and Unity 2018.1.1f1 x64 tested.

 

AVPro Video doesn't support QuickTime codecs I'm afraid. It does support playing MOV files, but only with known codecs such as H.264. The animation codec and ProRes aren't known. There are a few options for Windows:

1) Install LAV Filters, and then in the MediaPlayer component in Platform Specific > Windows set the Video API to DirectShow. Then it should be able to play your file, but obviously will require any other PC to also install LAV Filters.

2) Convert your video to use packed-transparency, which uses standard formats like H.264 and a double width/height video with the alpha data embedded next to the colour data. See our PDF documentation for how to do this. We also include 2 sample videos and a demo scene that shows you how this works.

Let me know if you have any other questions about this.

Thanks,

 

Thanks a lot, the first answer ideally suit. The second one doesn't solve our problem.

 

Yes, the first thing I checked upfront.
And like I wrote before, it does work using another 3rd party video player. That one is just not as good as yours

 

Hi,
When I try to run
ffmpeg -i 1.mov -vf "split [a], pad=iw*2:ih , [a] alphaextract,
overlay=w" -y 3.mp4

I get error

ffmpeg -i 1.mov -vf "split [a], pad=iw*2:ih , [a] alphaextract,
ffmpeg version N-91396-g27662ed681 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.3.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
libavutil 56. 18.102 / 56. 18.102
libavcodec 58. 20.104 / 58. 20.104
libavformat 58. 17.101 / 58. 17.101
libavdevice 58. 4.101 / 58. 4.101
libavfilter 7. 25.100 / 7. 25.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100
Trailing options were found on the commandline.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mov':
Metadata:
major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2018-06-29T10:53:05.000000Z
Duration: 00:00:07.50, start: 0.000000, bitrate: 44925 kb/s
Stream #0:0(eng): Video: qtrle (rle / 0x20656C72), bgra(progressive), 1920x1080, 43806 kb/s, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 24 tbn, 24 tbc (default)
Metadata:
creation_time : 2018-06-29T10:53:05.000000Z
handler_name : Apple Alias Data Handler
encoder : Animation
timecode : 00:00:00:00
Stream #0:1(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:
creation_time : 2018-06-29T10:53:05.000000Z
handler_name : Apple Alias Data Handler
timecode : 00:00:00:00
At least one output file must be specified




I dont know how to fix this. Do you any idea?
Any help would really be appreciated

 

Hi,
I am using your trial version and willing to buy full version. But I am stuck at one place. I read documentation and read about

Here we show two examples using ffmpeg to convert a source video containing a
transparency/alpha channel into an alpha packed format:
Left-right alpha packing:
ffmpeg -i %1 -vf "split [a], pad=iw*2:ih , [a] alphaextract,
overlay=w" -y %1.mp4


Now I installed ffmpeg, hap codes.
And I typed this in cmd
ffmpeg -i 1.mov -vf "split [a], pad=iw*2:ih , [a] alphaextract,
overlay=w" -y 3.mp4 (I copied file name 1 in bin folder of ffmpeg)


After that, I get an error. below are details. Please help me with this..


ffmpeg -i 1.mov -vf "split [a], pad=iw*2:ih , [a] alphaextract,
ffmpeg version N-91396-g27662ed681 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.3.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
libavutil 56. 18.102 / 56. 18.102
libavcodec 58. 20.104 / 58. 20.104
libavformat 58. 17.101 / 58. 17.101
libavdevice 58. 4.101 / 58. 4.101
libavfilter 7. 25.100 / 7. 25.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100
Trailing options were found on the commandline.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mov':
Metadata:
major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2018-06-29T10:53:05.000000Z
Duration: 00:00:07.50, start: 0.000000, bitrate: 44925 kb/s
Stream #0:0(eng): Video: qtrle (rle / 0x20656C72), bgra(progressive), 1920x1080, 43806 kb/s, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 24 tbn, 24 tbc (default)
Metadata:
creation_time : 2018-06-29T10:53:05.000000Z
handler_name : Apple Alias Data Handler
encoder : Animation
timecode : 00:00:00:00
Stream #0:1(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:
creation_time : 2018-06-29T10:53:05.000000Z
handler_name : Apple Alias Data Handler
timecode : 00:00:00:00
At least one output file must be specified

 

Try putting the whole command on a single line:
ffmpeg -i 1.mov -vf "split [a], pad=iw*2:ih , [a] alphaextract, overlay=w" -y 3.mp4

Thanks,

 

Hmm...

It seems strange that it would have issues loading the DLL when you specify a stream URL instead of a local file. The loading of the DLL happens right at the start before it even knows what URL type is specified. Perhaps you even get that error message when you play a local file? Perhaps it's a red herring...

If you build it as a PC UWP app and just run it Windows, are you able to test it connected to that (or another) stream?

Thanks,

 

Hey there!

Just wondering if there is any planned support for OpenVideoFromBuffer for OSX. It'd be insanely helpful for our project.

 

I'm sorry to say that we don't have any plans for this at the moment...

 

Hi,
Thanks for reply. But now I am getting this below error.


Press [q] to stop, [?] for help
[AVFilterGraph @ 000001b8ae367dc0] Too many inputs specified for the "alphaextract" filter.
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0
Conversion failed!

 

Your command must be missing some elements, I think because the forum here uses some of those elements for formating. Here is the command derived from our PDF documentation:

Code (CSharp):

1. ffmpeg -i 1.mov -vf "split [a], pad=iw*2:ih [b], [a] alphaextract, [b] overlay=w" -y 3.mp4

Thanks,

 

Hey guys! I'm back!
I have to update some old stuff to 2018.

Can AVPro play TBE files? I saw some settings :


But I'm not sure how this works. Where do I tell it what file to play and how do I tell it when to play it.

THanks <3

 

Thanks a lot! It worked now.

 

Is there anyway to increase/decrease the stereo disparity for 3D playback?

Something like this option in Adobe Premiere would be excellent to adjust on the fly : https://helpx.adobe.com/after-effects/using/vr-effects.html#ApplyVRPlanetoSphereeffect

 

Are videos shot on portrait mode supported yet? I see old posts that it would be supported in the next update, but that was awhile ago. I'm trying it now but it doesn't seem to be doing it, the video is rotated 90 degrees. The video is shot in portrait mode on an iOS device (iPhoneX).

Thanks, I've really been enjoying your product

 

I've had a good look around and a dig through the plugin code, but I can't find any information on manual control of HLS Streams.

Is it possible to get a list of the available streams from a .m3u8 file and select a certain stream? It always seems to get the first one in the stream list.

 

TBE files are not supported. We do support the Facebook Audio 360 format though on Windows 10 and Android, but it needs to be inside a MKV or WebM video file, with the audio encoded as Opus. Their tools generate these files. If you download their SDK it includes a sample MKV file which you can use to test this feature. We have a section in the PDF documentation that explains what settings need to be enabled, but it's quite simple really.

 

I'm not 100% sure what you mean... Stereo disparity is a setting of the Unity Camera when you enable VR mode... I think it also might be overridden when you select some VR headsets, but I'm not sure. Anyway it's a feature of the rendering engine (Unity) and not really applicable to our video player plugin I think...

Thanks,

 

Yes, portrait mode videos should be supported now. It depends on which component you use to display the video.. For example our DisplayIMGUI and DisplaGUI components should support it - but the ApplyToMesh / ApplyToMaterial don't support rotating the video.... Perhaps you could tell us which component you're trying to use it on? If it's not working then feel free to contact us at the unity support email here: www.renderheads.com/contact with a copy of your video so we can help to debug it.

 

We include a utility class called HLSStream which can be used to parse the m3u8 manifest and extract all of the different quality levels and substreams. You could then manually tell the AVPro Video MediaPlayer component to play a specific substream URL instead of the URL of the main manifest. This would give you a specific quality / resolution level, but it would lose the ability for it to dynamically select the quality level.

Thanks,

 

Hello, I have a problem with some of my projects using AVpro, when I play a 360 video there is one thin vertical line (black color) always on the video, Im using the sphere prefab to render the video, even with the last version of AVpro I have the same problem, how can I resolve this ?

thanks.

 

Try changing the texture wrap mode to "Repeat" in the MediaPlayer > Visual section

Thanks,

 

Of course I made sure all capabilities are in place. I am loading images from the same server btw. and that is working as expected.

 

How can I make a unity GUI button trigger a new media clip. The dream would be to only have one media player and have that change its source path to the video. Option two would be to have multiple MediaPlayers each with the different clips and have the Unity GUI button change what MediaPlayer the apply to material script is referencing.

 

Hello, I have been trying to play HEVC video, 4096x4096 on an Oculus Go (Android) and am encountering dreadful seek accuracy, to the point where it's off by over 3 seconds. It's not a keyframe issue, as far as I can tell, and works fine on other devices not using AVPro. A 3054x3054 h264 version of the video also works fine.

 

When you load the rtsp file (rtsp: //184.72.239.149/vod/mp4: BigBuckBunny_175k.mov) or (rtsp://85.255.175.241/h264)
all rtsp

[AVProVideo] Error: Loading failed. File not found, codec not supported, video resolution too high or insufficient system resources.

Dose not rtsp support?


Target Platform: Standalone

AVPro Video: plugin v1.8.2 scripts v1.8.7

 

Hmm, I'm sorry then I don't know what the issue could be.. We do have a new version of the plugin coming out today (version 1.8.9) which has a recompiled UWP plugin...it may help
Otherwise it may just be that it is a limitation of that platform. Without a Hololens we're not able to test this.

 

Please have a look at our demo scenes

03_Demo_VideoControls.unity = has a GUI button that loads a different video. It also uses two MediaPlayers and switches which one is displayed when the loading one has completed loading.

Thanks,

 

Hi,

That sounds very strange. If the key-frame interval is not the issue then I don't know what it could be besides a bug in the decoder in the operating system.

Feel free to send us your video so that we can investigate. (email us at the Unity Support address at http://www.renderheads.com/contact/)

Also see if selecting the MediaPlayer or ExoPlayer video API makes a difference (in MediaPlayer > Platform Specific > Android)

Thanks,