[Released] AVPro Video - complete video playback solution

Hi,

Perhaps you could email us (http://www.renderheads.com/contact/) a copy of your video? Also, is this issue on macOS platform?

I'm not sure that macOS support displaying the initial frame without calling Play() we will investigate this - thanks for bringing it to our attention.

Currently you have to load the whole video to get access to the frames.

Thanks,

 

Yes there is. Perhaps you have an older version of the plugin. The method looks like this, but it is currently only available for Windows and when using the legacy DirectShow video API:

public bool OpenVideoFromBuffer(byte[] buffer, bool autoPlay = true)

Thanks,

 

Alas this is out of our hands and is an issue with changes in Unity. See our support thread here:
https://issuetracker.unity3d.com/is...er-compilation-failure-on-some-mobile-devices

I think they only fixed it in 2017.2... I haven't yet tried the latest 5.6.X builds though....

Thanks,

 

This is because Android seek currently seeks to the nearest keyframe. We have plans to improve seeking support on Android (now that they've added support for this in Oreo). So this will be available in a version very soon. Until then you can fix this by adding more key-frames to your video encoding.

Thanks,

 

Was running H.265 videos fine last week with Unity 2017.1.1f1(64 bit) on Windows 10 PC. Got the Windows 10 Fall update over the weekend and now I get
[AVProVideo] Error: Loading failed. Codec not supported or video resolution too high or insufficient system resources.
UnityEngine.Debug:LogError(Object)
RenderHeads.Media.AVProVideo.MediaPlayer:UpdateErrors() (at Assets/AVProVideo/Scripts/Components/MediaPlayer.cs:1512)
RenderHeads.Media.AVProVideo.MediaPlayer:Update() (at Assets/AVProVideo/Scripts/Components/MediaPlayer.cs:912)

I tried downloading the latest trial version of AV Pro Video and got the same error message. Any idea what changed?


EDIT: For anyone with the same problem, I fixed it by getting the (free) HEVC Video Extension app from Microsoft via the Windows Store.

 

Is there any news on Ambisonics support? Thanks

 

Hi, Let me answer your questions:

1) the "faststart" encoding flag just makes the video suitable for streaming by moving one of the chunks in the file that contains useful information (such as resolution, frame-rate etc) from the end of the file (the default position) to the beginning. This just allows plays to start playing the video without having to download the WHOLE first first.

Fine-grain control over how much of the video buffers before playback is something we can't control currently - it is handled by the video decoder in the operating system. In the future we may be able to expose some of this but I think this feature is a long way off.

For now you could experiment with keeping the video paused and waiting until 4-5 seconds has been buffered (by using the methods like GetBufferingProgress() and then playing the video. The buffering behaviour can differ between platforms though.

2) This sounds very strange. It sounds like you have something else in your scene that is affecting rotation. I don't see how a seek bar could affect rotation otherwise..... Sorry I don't have any ideas here.

Thanks,

 

Hmm, I wonder what your bit-rate is. Maybe the video bit-rate is too high.... Can you check how much of the video it is buffering? (see our 02_Demo_imGui.unity example scene that shows you how much of the video is buffering). Perhaps you can use this information to resume playback. If you're still running into issues then please share the URL with us so that we can investigate (you can contact us at www.renderheads.com/contact).

Thanks,

 

This is very curious. Can I ask: which version of Unity and AVPro Video are you using?
Also, does it make a difference if you tick the "Use YCbCr" option in the iOS section of "Platform Specific" panel of MediaPlayer?

Thanks,

 

Ahh yes, I think the Fall Update actually removes the native H.265 support from Windows 10 and requires users to download this extension We'll make sure to add this to the documentation in case other users are running into this issue.

Thanks,

 

We are about to release a version that includes support for Facebook Audio 360 (Windows and Android) which is a spatial audio system similar to ambisonics.

Thanks,

 

Also, does it work if you remove one of the slashes?
Like try setting it to load AbsolutePathOrURL and use:
file://var/mobile/Containers/Data/Application/1F6E75BF-4066-42C4-A991-52CA66832F10/Documents/Videos/leSuquet.mp4

And if you could supply the answers to the questions in my previous post that would be most useful

Thanks,

 

I have sent an e-mail to unitysupport@renderheads.com with a link to download the 2 videos (the version that was re-encoded with only I-Frames and the original). I was unable to reproduce this issue on macOS.
Thank you.

 

I'm targeting windows, android and in a later stage ios. The buffering progress is not exactly what I want, it is 0 before the first frame is ready and 1 when the first frame is ready to play, but I want to see how far a video has been downloaded, for example 0:30/1:47 minutes. I don't know how to use
GetBufferedTimeRange(int index, ref float startTimeMs, ref float endTimeMs);(no idea what to use as refs and index)
and I don't entirely know how to use GetBufferedTimeRangeCount().
I've tried this: float downloadProgress = currentMediaPlayer.Control.GetBufferedTimeRangeCount() / currentMediaPlayer.Info.GetDurationMs(); And used it as if 0 was 'not downloaded' and 1 'completely downloaded' but the "download progress" stays: 1.871258E-05 all the time. About 0.000001871258

Kind regards,

Yorick

 

Hi,
Just wonder if there are any plans to create capture solution for macOS/iOS/Android. If yes, are there any ETAs?
Windows plugin is very handy and functional, it would be nice to have one for other platforms.

 

So this morning I got the Load Failure again. I had to run one of my HEVC videos in "Movies & Television" first and then it would run in AV Pro without a problem. I suspect there is some sort of setting or reading of the HEVC extension that is needed. Clearly this will be a problem when releasing to customers - could you have a look into what is happening please?

 

We are a multi-seat AV Pro license holder, and we have a client that has requested video downloads from the web through our app to be saved to android device for playback. We are considering using Unity asset bundles to download the content. It appears that asset bundles download in one location on the device and AV Pro requires that video assets reside at another location. Is there anyway that we can point AV Pro to the correct location of the newly downloaded asset bundles without copying the entire file to the AV Pro directory?

 

Hi

I read above that you were going to be releasing an update very soon that will support facebook 360 audio. I'm very interested in this feature and was wondering if it was referring to just the accompanying tbe file or 360 videos with muxed 3d audio?

Great plugin and I very much look forward to this update.

Phil

 

Hi,

I'm trying to load movies created with FFMPEG, the movies load in AfterEffects correctly but not in the AVPro Video.

Can you tell me what I am doing wrong? The cmd line i'm using right now is:

ffmpeg -i rgb.mp4 -filter_complex "movie=alphamask.mp4 [alpha]; [0:0][alpha] alphamerge [out]" -map "[out]" -c:v hap -format hap_alpha -y -an converted_uid.avi

Thank you in advance,
j

 

Sounds cool, what would be great is a way to pass the 4 channel audio track to Oculus or GVR Audio SDK's. I currnetly have to have a separate audio file from the video which is just a little extra hassle for all involved in projects.

 

Thanks, so it's finally the time to upgrade to Unity 2017

Tested and appears to work great in Unity 2017.2

 

Unity 5.6.2p1 and Avpro 1.6.14, yes Use YCbCr is selected

 

Definitivly not working with any previous IOS 9 version on iphone 5S

Thanks

 

Did you set the video API to "DirectShow" in the Windows Platform Specific section of MediaPlayer? Or if you're using macOS, make sure it's a MOV file. Thanks,

 

Hi AndrewRH,

Just updated to AVPro 1.6.15, I have some trouble too... first when video start screen is green or purple, and video not play. Event stay on MetaDataReady and nothing happen, all video (vimeo streaming) are playing on editor correctly, But on iphone 6 IOS 9.2 and iphone 5S IOS 10.3.3 gives error.

Sometimes video playing but stop 5 -15 seconds later and never restart...

Setup: https://screencast.com/t/D1QeRBBpJq and https://screencast.com/t/JwB8uPsg

Vimeo link:
https://player.vimeo.com/external/241648325.m3u8?s=18ba638067ef3bbe373a101176108c92c0665694

Part of code used to load&listen videos

Code (CSharp):

1. // Play video localy or streamed
2.     public void InitiateLoad()
3.     {
4.         // Get name of current video from url
5.         videoUrlName = Manager.GetCurrentVideo360Url().Substring(Manager.GetCurrentVideo360Url().LastIndexOf("/") + 1);
6.         videoName = Manager.GetCurrentVideo360DLUrl().Substring(Manager.GetCurrentVideo360DLUrl().LastIndexOf("/") + 1);
7.  
8.         if(Constants.DEBUG)Debug.Log("videoUrl = "+videoUrlName);
9.         if(Constants.DEBUG)Debug.Log("videoName = "+videoName);
10.  
11.         // If video folder in persistant datapath not exist
12.         bool exists = UnifiedIO.Directory.Exists(Constants.VIDEOS_FOLDER);
13.         if(!exists)
14.         {
15.             // Create folder
16.             UnifiedIO.Directory.Create(Constants.VIDEOS_FOLDER);
17.         }
18.  
19.         // Search if file exist in persistentdatapath by name  
20.         string file = Application.persistentDataPath + "/" + Constants.VIDEOS_FOLDER + "/" + videoName;
21.         if(System.IO.File.Exists (file))
22.         {
23.             // File exist and entierly downloaded
24.             if (!PlayerPrefs.HasKey("DownloadLength_"+videoName) && !PlayerPrefs.HasKey("DownloadProgress_"+videoName))
25.             {
26.                 // Get persistentdatapath file by device
27.                 _location = MediaPlayer.FileLocation.RelativeToPeristentDataFolder;
28.          
29.                 _mediaPlayer.m_VideoPath = Constants.VIDEOS_FOLDER + "/" + videoName;
30.          
31.                 downloadIcon.gameObject.SetActive(true);
32.                 downloadbackground.color = Color.green;
33.                 donwloadButtonText.gameObject.SetActive(false);
34.          
35.                 // Deactivate download, file already downloaded
36.                 downloadEnable = false;
37.          
38.                 if(Constants.DEBUG)Debug.Log("Play video from persistentdatapath: "+_mediaPlayer.m_VideoPath);
39.             }
40.             else // download not yet finished
41.             {
42.                 SwitchFileLocationForIntroVideo();
43.          
44.                 // Set downlaod and activate it
45.                 downloadIcon.gameObject.SetActive(false);
46.                 downloadbackground.color = initialColor;
47.                 donwloadButtonText.gameObject.SetActive(true);
48.          
49.                 downloadEnable = true;
50.                 // Autostart download
51.                 largeFileDownload.enabled = true;
52.             }
53.         }
54.         else // Video is not downloaded
55.         {
56.             SwitchFileLocationForIntroVideo();
57.  
58.             // Reset download icon
59.             downloadIcon.gameObject.SetActive(true);
60.             downloadbackground.color = initialColor;
61.             donwloadButtonText.gameObject.SetActive(false);
62.      
63.             downloadEnable = true;
64.         }
65.  
66.         _mediaPlayer.m_AutoStart = true;
67.         _mediaPlayer.OpenVideoFromFile(_location, _mediaPlayer.m_VideoPath);
68.  
69.         // Set pause image sprite when player begin
70.         _cannesVideoPlayerControls.playPauseButtonImage.sprite = _cannesVideoPlayerControls.pauseButtonSprite;
71.     }
72.  
73.     private void SwitchFileLocationForIntroVideo()
74.     {
75.         // use current url from JSON
76.         videoUrlName = Manager.GetCurrentVideo360Url();
77.  
78.         switch (videoUrlName)
79.         {
80.         case"introAND.mp4":
81.             _location = MediaPlayer.FileLocation.RelativeToStreamingAssetsFolder;
82.             break;
83.         case"introIOS.mp4":
84.             _location = MediaPlayer.FileLocation.RelativeToStreamingAssetsFolder;
85.             break;
86.         default:
87.             _location = MediaPlayer.FileLocation.AbsolutePathOrURL;
88.             break;
89.         }
90.  
91.         _mediaPlayer.m_VideoPath = videoUrlName;
92.         if(Constants.DEBUG)Debug.Log("Play video from url: "+_mediaPlayer.m_VideoPath);
93.     }
94.  
95.     public void OnBtnDownloadClicked()
96.     {
97.         if(downloadEnable)
98.         {
99.             largeFileDownload.enabled = true;
100.         }
101.     }
102.  
103.     // Callback function to handle events
104.     public void OnVideoEvent(MediaPlayer mp, MediaPlayerEvent.EventType et, ErrorCode errorCode)
105.     {
106.         switch (et)
107.         {
108.             case MediaPlayerEvent.EventType.Error:
109.                 if(Constants.DEBUG)Debug.Log ("Video Player Error: " + errorCode.ToString());
110.                 _mediaPlayer.CloseVideo();
111.      
112.                 // back to usermode
113.                 if(Manager.GetUserModeSelected() == "vr")
114.                 {
115.                     // Set camera VR and input event              
116.                     Manager.SetUserModeSelected("mobile");
117.              
118.                     VRSettings.enabled = false;
119.                     gameObject.GetComponent<Player360Controller>().ApplicationController.cameraMobile.GetComponent<Camera>().ResetAspect();
120.                     gameObject.GetComponent<Player360Controller>().ApplicationController.gvrReticlePointer.enabled = false;
121.                     gameObject.GetComponent<Player360Controller>().ApplicationController.gvrPointerInputModule.enabled = false;
122.                     foreach (var item in gameObject.GetComponent<Player360Controller>().ApplicationController.gvrPointerGraphicRaycaster)
123.                     {
124.                         item.enabled = false;
125.                     }
126.              
127.                     gameObject.GetComponent<Player360Controller>().ApplicationController.standaloneInputModule.enabled = true;
128.                 }
129.          
130.                 gameObject.GetComponent<Player360Controller>().ApplicationController.FadeIn();
131.                 gameObject.GetComponent<Player360Controller>().ApplicationController.ShowScreen(FluxMVC.Ravr.Cannes360.ApplicationController.Screens.UserMode);
132.                 break;
133.          
134.             case MediaPlayerEvent.EventType.ReadyToPlay:
135.                 if(Constants.DEBUG)Debug.Log("ReadyToPlay");
136.                 if(!_mediaPlayer.Control.IsPlaying())
137.                     _mediaPlayer.Play();
138.                 break;
139.          
140.             case MediaPlayerEvent.EventType.Started:
141.                 if(Constants.DEBUG)Debug.Log("Started");
142.                 break;
143.      
144.             case MediaPlayerEvent.EventType.MetaDataReady:
145.                 if(Constants.DEBUG)Debug.Log("MetaDataReady");
146.                 break;
147.          
148.             case MediaPlayerEvent.EventType.FirstFrameReady:
149.                 if(Constants.DEBUG)Debug.Log("FirstFrameReady");
150.                 // Reveal video and stop loading effect
151.                 gameObject.GetComponent<Player360Controller>().ApplicationController.FadeIn();
152.                 gameObject.GetComponent<Player360Controller>().ApplicationController.Loading.SetActive(false);
153.                 break;
154.          
155.             case MediaPlayerEvent.EventType.FinishedPlaying:
156.                 if(Constants.DEBUG)Debug.Log("FinishedPlaying");
157.  
158.                 // If it's video bestof espace pro, launch image360 espace pro (O hotspot, 1 espacepro)
159.                 if(Manager.GetIsBestofHotspotOrEspacePro() == 1)
160.                 {
161.                     CannesSignalManager.Instance.EmitSignal(CannesSignalManager.Open360Image);
162.                 }
163.          
164.                 // If it's intro video
165.                 if(Manager.GetMenuType() == 0)
166.                 {
167.                     CannesSignalManager.Instance.EmitSignal(CannesSignalManager.OnVideoIntroFinnished);
168.                 }
169.          
170.                 // It's video hotspot
171.                 CannesSignalManager.Instance.EmitSignal(CannesSignalManager.OnVideoFinnished, Manager.GetMenuType());
172.                 _mediaPlayer.Pause();
173.                 //_mediaPlayer.CloseVideo();
174.                 break;
175.         }
176.         if(Constants.DEBUG)Debug.Log("Event: " + et.ToString());
177.     }

 

EDIT 1 : After some test on Iphone 5S IOS 10.3.3, if I deselect ypCbcr420 video playing fine and screen is not green same on Iphone 7 IOS 10.3.2. But on Iphone 6 IOS 9.2, screen is black and video are not played

EDIT 2 : Same error with unity version 5.6.4p1

 

I just purchased the plugin, I'm satisfied with its performance.

Is any integrated method exist to take average color of current frame? (Or I need to take current frame and scale it down)
Is spacial audio format supported?

 

Hi,

We've tried with your video and still aren't able to replicate any problems - it works fine for us! I have a few questions that will help us resolve this:

1) If you use our demo scenes (eg 01_Demo_BackgroundShader.unity) do you still have these problems?
2) Could you perhaps send us your whole Unity project so we can try to replicate what you're seeing? It will be secure - you can send it via www.renderheads.com/contact/ where there is a unitysupport email address.

Thanks,

 

There isn't any method for getting the average color, so you would have to do this yourself. Yes, I guess scaling the texture down and then grabbing the pixels would work.

No spatial audio formats are supported yet, but the next version will support Facebook Audio 360 spatial audio.

Thanks,

 

Hey Phil,

It will support video files with the Facebook 360 Audio muxed in. Currently requires MKV files with the audio encoded with Opus. Windows 10 (1607) and Android are supported.

Should be out soon!

Thanks,

 

Thats really good news. I very much look forward to the update. So good to see you're still developing and updating the AVpro, in comparison to Unity's built in video support it's a revolution.

 

We want to use the buffering system on android and ios, those are our main platforms. Is it possible to get that functionality? It's probably not vissible for me because I'm targeting Android for the moment.

Kind regards,

Yorick

 

Hi Andrew.
I sent you an email yesturday about a black screen on Android device while vuforia support is enable in XRsetting. Had you reproduce the bug?
- Empty project with only AVpro in and vuforia support enable in XRsetting
- unity 2017.2 with last version of Avpro
- appears on galaxy S8 on android 7.0
- using your video and your demoscene 04_Demo_ugui
I have the sound, the app is running well but the media player is black.
thanks for your replay
best regards

 

hi, @AndrewRH is support network sync like vlc? thanks.

 

I don't know why but suddently my videos appear like this:



I havn't touched this project in about 1 month... no idea what is wrong

EDIT: Using Unity 2017.2 but the same problem seems to be in 2017.1 too.
EDIT: THe problem seems to be with the Top-Bottom stereo. If I put it to "none" it seems to work. It's a bit strange because my videos are 100% stereo.


See... it's stereo

EDIT: I'm trying this with unity 2017.1.2 + Latest version of AVPro + A total new Top-Bottom video. Same problem. I also tried to setup the top/bottom on the sphere shader directly.

 

Andrew - can you confirm you have someone looking into this please? We have demonstrations coming up next month.

 

I need change the video's rate at any time.On windows10 was fine,but on windows7 SP1 is very lag(only the video looks bad,others running fine),especially while changing the rate.
The AVPro's version is 1.6.6.
The same hardware environment,only the different system operation.
I have no idea.Could you help me?

 

We're looking into it still. Thanks,

 

GetBufferedTimeRangeCount() is used when there are multiple time ranges that have already been downloaded and cached. For example it could have cached 0...5 seconds and 10..15 seconds, but it is still missing 5..10 seconds. In this case GetBufferedTimeRangeCount() would return 2. You can see how to use this function in the SimpleController.cs script, which is used in the 02_Demo_imGui.unity scene and draws a green line on the timeline to indicate which parts of the video have been downloaded.

 

This is the wrong forum for this question - you should have posted it to the AVPro Movie Capture forum. We currently only have any timeline for supporting other platforms, but this is something we're investigating.

Thanks,

 

Hi,
Did you install the HEVC Video Extension?
We don't yet have the Fall Update from Microsoft, but as soon as we do we'll investigate this change in Windows.

Thanks,

 

We do not support opening videos from AssetBundles. You would have to extract them from the AssetBundle and copy them to the persistantDataPath, or you would have to download them as original files to this location.

Thanks,

 

It'll be muxed into the video. Currently MKV file with Opus audio is supported. It'll be released today.

Thanks,

 

It uses the audio track embedded into the video. It has to be authored specially though using the Facebook Audio 360 tools. I'm not sure if there is any way to transcode other ambisonic audio into this format.

Thanks,

 

Hi, I just tried your ffmpeg script and it worked fine for me. Make sure that in MedPlayer > Platform Specific > Windows that you select DirectShow for the Video API, otherwise Hap Alpha will not play.

Thanks,

 

That functionality isn't currently supported on Android or iOS. Can I ask why you need to play from a buffer?

Thanks,

 

Thanks, we're not sure why this happens, but it seems to be some conflict with Vuforia. To fix it you can disabled multi-threaded rendering. Hopefully we'll be able to fix it for the next update (1.7.2).

Thanks,

 

Not currently, but it would be possible to add support for this yourself using Unity scripts.

Thanks,

 

Hmm strange..
A few ideas pop to mind:

1) You changed something in a script or shader and forgot about it and that's broken things
2) You did a Windows update and somehow that changed things. Use "winver" to see which version you're running. Microsoft recently released the new Fall Update which includes some video changes, especially the removal of HEVC codec.
3) The videos are encoded with stereo metadata that is confusing the encoding (see the FAQ about removing this).

Something for you to try - try putting in our "AlphaTopBottom.mp4" sample video file. See if you get the split correct (obviously the 360 mapping will be wrong, but hopefully you'll get the correct image in left and right eye). If this works then it could be a video encoding problem.

Thanks,

 

Please use the latest version of the plugin for best results (either 1.6.15 or 1.7.0).

Support for playback rate changing is very different per platform and also depends on the codec used and how the video is encoded. Trying to get a concistent behaviour for this between platforms is difficult. Windows 7 uses the DirectShow API while Windows 8 and above use the Media Foundation API, so already there is a difference. For DirectShow it can also depend on which codecs you have installed. For video encoding generally you want to have a video that is simplified and doesn't contain B frames.

Thanks,