[Released] AVPro Video - complete video playback solution

Hey, looks like latest version has some incompatibility with Unity 2017.2:

AVProVideo/Scripts/Editor/MediaPlayerEditor.cs(293,50): error CS0117: `UnityEditor.PlayerSettings' does not contain a definition for `mobileMTRendering'

Not sure for 100% but fix could look like

Code (CSharp):

1. //showWarningMT = !UnityEditor.PlayerSettings.mobileMTRendering;
2. showWarningMT = !UnityEditor.PlayerSettings.GetMobileMTRendering(BuildTargetGroup.Android);

Also it fires up the Obsolete API dialogue which fixes these APIs:

 

Hello

I'm currently using AVPro for a VR-setup and everything works fine until I've installed Unity2017.2 yesterday.

Now I get the following error message:

Assets/AVProVideo/Scripts/Editor/MediaPlayerEditor.cs(293,50): error CS0117: `UnityEditor.PlayerSettings' does not contain a definition for `mobileMTRendering'

Can anyone help me? Thanks!

 

@Michael_Me just check out my previous message here.

 

Hi.

I'd like to know how to redirect audio to Oculus Rift headphones.
AvProVideo 1.6.15 document mentions how to redirect Media Foundation audio to Oculus Rift headphones in section 4.4.1 Audio Redirection. I'm trying this way for usability.
But the sounds output only from Windows default audio device.

Media Player's Windows Platform Specific setting is below.
Video API = Media Foundation
Hardware Decoding = checked
Use Unity Audio = checked
Stereo = checked

And I placed Audio Output component on the same scene as below.
Audio Output Mode = Multiple
Media player = Media player above
Channels = Everything

1. Are there something else to do?
2. I am not sure about “channels” in Audio Output inspector. I'd like to know how does they work.

 

hello, when i switch to video scene, this comes.

10-14 10:05:20.624 E/Unity (11919): AndroidJavaException: java.lang.ClassNotFoundException: com.RenderHeads.AVProVideo.AVProMobileVideo
10-14 10:05:20.624 E/Unity (11919): java.lang.ClassNotFoundException: com.RenderHeads.AVProVideo.AVProMobileVideo
10-14 10:05:20.624 E/Unity (11919): at java.lang.Class.classForName(Native Method)
10-14 10:05:20.624 E/Unity (11919): at java.lang.Class.forName(Class.java:324)
10-14 10:05:20.624 E/Unity (11919): at java.lang.Class.forName(Class.java:285)
10-14 10:05:20.624 E/Unity (11919): at com.unity3d.player.UnityPlayer.nativeRender(Native Method)
10-14 10:05:20.624 E/Unity (11919): at com.unity3d.player.UnityPlayer.c(Unknown Source)
10-14 10:05:20.624 E/Unity (11919): at com.unity3d.player.UnityPlayer$c$1.handleMessage(Unknown Source)
10-14 10:05:20.624 E/Unity (11919): at android.os.Handler.dispatchMessage(Handler.java:98)
10-14 10:05:20.624 E/Unity (11919): at android.os.Looper.loop(Looper.java:148)
10-14 10:05:20.624 E/Unity (11919): at com.unity3d.player.UnityPlayer$c.run(Unknown Source)
10-14 10:05:20.624 E/Unity (11919): Caused by: java.lang.ClassNotFoundException: Didn't find class "com.RenderHeads.AVProVideo.AVProMobileVideo" on path: DexPathList[[zip file "/data/app/mob.fish.mpndebug-2/base.apk"],nativeLibraryDirectories=[/data/app/mob.fish.mpndebug-2/lib/arm, /data/app/mob.fish.mpndebug-2/base.apk!/lib/armeabi-v7a, /vendor/lib, /system/lib]]
10-14 10:05:20.624 E/Unity (11919): at dalvik.system.BaseDexClassLoader

android 6.0.1, nexus 5. unity 2017.1.1.

what should i do?

 

Hello.
i using Avpro Video.

Could it be play automatically all video in the folder respectively?
it means when a video is finished immediately next one play automatically.
thanks.

 

Does this let you play back a video that the user selects from the photo album on iOS or android

 

Anyone got issues with playback on Safari on a Mac through WebGL?
Doesn't seem to work with the 03_demo_videocontrols.unity scene.
Same build works in Chrome on a Windows 10 PC though.

 

Hi,
I would like to play the NASA TV video stream using AVPro but I get an error saying wrong codec or not enough resources. Any tips or thoughts?

 

Hello,

I want to play a video with a absolute path. It can be played in Unity Editor, but failed in UWP program.
Can AVPro play a video with absolute path when it built as UWP (Universal Windows Platform)?

Thansk.

 

Hello there,
I have a couple of questions:
1. There is point about 1.7.0 version in user manual (Version 1.7.0 High quality glitch free multi video player with optional transitions and nice user interface). Does it means, that seamless playing of multiple videos would be possible? If so, what is ETA of 1.7.0, otherwise, any suggestions on how to implement this?
2. Using CloseVideo() method doesn't free any memory, at least, not instantly. Also, seems like reloading scene, using GC.Collect and Resources.UnloadUnusedAssets not affect on this behavior (iOS 10, AVPro 1.6.12, Metal/OpenGLES3). Is there any way to fastly free resources?
3. Since multiple audio tracks not supported in WebGL builds by browsers, I've implemented this using default unity Audio Source, all is ok, but when/if video stuck for some seconds (after Started event), sound continues to play, because I haven't found any event/field that notifies about it. How could I work around this issue?

Thanks.

 

Hey, @Wanderer13

I've recently implemented seamless playback on Android and Windows.
The idea is pretty simple - just use 2 instances of the MediaPlayer.

Load video into 1st instance and play, then on demand (or on 1st video end), load another video into 2nd instance and wait for the FirstFrameReady event. Then switch reference at the display component(s) to the 2nd instance of the MediaPlayer and so on.

It's relatively simple to make a factory-like behaviour for the infinite seamless playback of any amount of videos.

The only one serious issue I've faced - bug on Windows causing few black frames to appear at the start of the video. Renderheads are aware of it and hopefully will fix it somehow in future.

It can be workaround though with one of this:
- set video api to direct show
- disable hw decoding
- use -force-d3d11-no-singlethreaded command line argument

Hope that helps.

Did you checked if MediaPlayer emits Stalled \ Unstalled events while video is stuck?
Also you could check mediaPlayer.Control.GetCurrentTimeMs() to sync time with audio or check mediaPlayer.TextureProducer.GetTextureFrameCount() to figure out if video texture was updated more that 1 time to make sure actual playback was started.

 

I want to play mpg dash videos on Android device but I can´t. How do I use exoplayer API and AVpro in Android?

 

Unless you recompile the .jar file in your project: impossible. Doing so turned out to be the solution to my problem here as well. I get that this might be in some violation of the TOS but I was under a lot of time pressure and didn't want to rely on Andrew fixing the problem for me.

(Btw Andrew if you're reading this, I'm more than willing to share my version with Exoplayer implemented. It fixes a bug where low-end devices cannot stream video's, and allows mid-high end devices to play 4k by 4k video's.)

 

Hey everyone,

Is there a way to see how far a video has been downloaded? I cant find anything in MediaPlayer.Control or MediaPlayer.Info

Kind regards,
Yorick

 

Hi Team,

Pretty irritating bug.
I'm starting a clean project for Oculus and GearVR, using AVPro 1.6.15 and Unity 2017.2.0f3 (running Windows)
(I've reported the same bug with earlier version of both Unity and AVPro before)

This setup (Platform Android)
GameObject with:

• MeshRenderer (With sphere from demo assets)
• Apply to mesh (with a top bottom stereo image as default texture)
• Update Stereo Material
• Material with "AVProVideo/VR/InsideSphere Unlit (stereo+fog)" shader

So no actual Mediaplayer running a video, just the still stereo image.

When you run your scene in the IDE previewing it with an Oculus the stereo is inverted. The left eye sees the what the right eye should be seeing.

When you build that exact scene to android and look at it in a Gear VR the stereo is correct.

The moment you also start playing a movie on that same setup the stereo is correct, so its just the (placeholder) image that is flipped and only on the Rift.

The bug only occurs when the image is Top/Bottom, Left Right works fine.
When trying to reproduce, please look at the actual stereo in the image, don't rely on the debug color hinting to verify its working.

Thanks!

 

You'll find what is actually occurring is the video or still is being flipped vertically at runtime - ran into same issue with Editor vs Android.

I don't fully understand why the content flips in the first place.

There is built in handling for flipY in the ApplyToMaterial and ApplyToMesh scripts, but because I was using own shader, it was failing.

My content was only inverting on Windows (MediaFoundation) - I had to add checking for -1 scaling on Y in my shader.

If it is flipping, this will likely be why LR is working and TB is reversing eyes. Is your content looking upside down / inside out?

Not sure if this helps track down issue?

 

Hi Pleribus77,

This is indeed what is happening, the content is not upside down or inside out but the flipping is indeed the reason it gets inverted.

Fixed it by changing these lines in ApplyToMesh.cs (line 221) (only tested on windows and android) Your texture needs wrapmode mirror for this to work.

Code (csharp):

1.  
2. // 1 means TOP_BOTTOM
3.                                         float alphaMode = -1;
4.                                         float stereoMode = -1;
5.                                         if (mat.HasProperty("Stereo"))
6.                                         {
7.                                             stereoMode = mat.GetFloat("Stereo");
8.                                         }
9.  
10.                                         if(mat.HasProperty("AlphaPack"))
11.                                         {
12.                                             alphaMode = mat.GetFloat("AlphaPack");
13.                                         }
14.  
15.                                         //Debug.Log("STEREOMODE: " + stereoMode + " : " + SystemInfo.deviceType);
16.                                         //Debug.Log("ALPHAMODE: " + alphaMode + " : " + SystemInfo.deviceType);
17.  
18.                                         if ((stereoMode == 1 || alphaMode == 1) && SystemInfo.deviceType == DeviceType.Desktop)
19.                                         {
20.                                             mat.SetTextureScale(_propTexture, new Vector2(_scale.x, -_scale.y));
21.                                             mat.SetTextureOffset(_propTexture, Vector2.down + _offset);
22.                                         }
23.                                         else
24.                                         {
25.                                             mat.SetTextureScale(_propTexture, _scale);
26.                                             mat.SetTextureOffset(_propTexture, _offset);
27.                                         }
28.  

 

Hi, AndrewRH.

Is there a way to amplify the sound embedded in my video in AVProVideo? (..Normalize?)
Some videos are too small for a sound recording.

Thank you.

 

Hello, if I want to build a UWP app with AVPro, can AVPro play the video located in any folder?
I found I can't do that while building a UWP program.

 

Hello, My System is Window10. Recently I have got a problem about the plugins of AVPro video. I want to open a video by stream. So I convert the video file to buffer.
And then，I use the function of OpenVideoFromBuffer to load the buffer.Finally, I failed.Flowing is the error information:
[AVProVideo] Failed to open buffer
UnityEngine.Debug:LogError(Object, Object)
RenderHeads.Media.AVProVideo.MediaPlayer:OpenVideoFromBufferInternal(Byte[]) (at Assets/AVProVideo/Scripts/Components/MediaPlayer.cs:669)
RenderHeads.Media.AVProVideo.MediaPlayer:OpenVideoFromBuffer(Byte[], Boolean) (at Assets/AVProVideo/Scripts/Components/MediaPlayer.cs:484)
RenderHeads.Media.AVProVideo.Demos.LoadFromBuffer:Start() (at Assets/AVProVideo/Demos/Scripts/Scriptlets/LoadFromBuffer.cs:39)

Here is my system Information:
AVPro Video: plugin v1.6.14 scripts v1.6.14
Target Platform: Standalone
Unity: v2017.1.1f1 WindowsEditor
OS: Desktop - B85-D3V (Gigabyte Technology Co., Ltd.) - Windows 10 (10.0.0) 64bit - Chinese
CPU: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz - 8 threads - 16284KB
GPU: NVIDIA GeForce GTX 960 - NVIDIA - Direct3D 11.0 [level 11.1] - 2016KB - 16384

Strangely to say, There are no errors when I chose the system of windows 7 to do the same work.I don't konw how to slove the problem in Window 10.
I wish you can help me an give me hand.Thanks!

 

Hi, I have two 360 videos that I use to test our app with (one of them is the sphere video you have in the sample video). My problem is that when I use a playback speed of around 1.8x or higher, there is an offset on the video's slider cursor and current time value. The higher the playback speed is, the bigger the offset. I have only seen the bug using my video, the sphere one is fine. For example at 1x speed, my video lasts 30 seconds. When using 3x speed, the slider correctly start and finishes in 10 seconds, but the video keeps going for another 5-6 seconds (so the visualised playback speed is more like 2x). I have re-encoded both the sphere video and mine with the same settings using Avidemux (to have videos with only I-Frame). The re-encoding did not change anything aside from being able to seek faster. Both versions of the sphere video works fine, and both versions of my video have the issue. It feels as if my video's playback rate was capped at around 1.8x even if the current time speed has the correct rate. If you have any idea what I could do to fix this, i'm all ears. I am assuming it might be a problem with how the video was encoded, but the resulting issue is still wierd.

Specs:
Windows 7
Unity 5.6.1f1
AVPro Video v1.6.14
08_Demo_360SphereVideo Scene

On another unrelated note, FirstFrameReady event is never fired in Mac OS.
m_texture != null is true, m_Control.CanPlay() is true, but m_Texture.GetTextureFrameCount() > 0 is always false. m_Texture.GetTexture() also always returns null.
To divert this I start the video in the ready to play event, return to the first frame and pause the video in the FirstFrameReady when it now correctly gets fired. Not a very elegant solution, if you have a better one i'm also interested.

Also on another note, is it possible to get the first frame of a video quickly or do you have to load the whole video just to have access to it. (to have access to an image in a similar fashion as windows browser when you're in "large icon" viewing mode)?

Thanks.

 

Is there a function to open the video from a buffer? Where can I find that? Can't find it in the MediaPlayer. Can't find it in the documentation either.

 

Dear Andrew,

How is it going with the OES support for Unity versions after 5.5.3? We would really like to switch to the newer Unity to speed up our production but we need to use the OES path to witheld the app from overheating. Is there anything we could do do help with the OES support?

Kind regards,

Yorick

 

Seek working in Unity 5.6.3 Mac, but working wrong in Android device (7.1.2)

void Update () {
float seek = mp.Control.GetCurrentTimeMs ();
if (seek > 4960) {
mp.Control.Seek (2800); // or 2800.0f
}
}

video starts from 0 when seek > 4960

 

Hi,

I am making a VR app for Oculus and have used your DEMO version for testing purpose, and I will purchase the full version on Monday as integration is complete.

I need some minor help, as I am unable to find things similar to it in the document. Please answer each point separately.

1. I am streaming a video from my own server. Video format is .mp4.
The problem is: I want to implement progressive streaming, so that after 4-5 seconds when video starts playing once it plays continuously, whne the internet is good.
Please advice how can I do it ?

ffmpeg -i %1 -acodec copy -vcodec copy -movflags faststart %1-streaming.mp4
I got this reference, so I need to add this to the video header at encoding or in our player code?


2. SEEK BAR: When I operate the seek bar to forward the video it rotates the complete scene, so I need only video to move forward without rotating the scene.

I will really appreciate the help. These are the last 2 points I am struggling with, if done, the project can go live.

 

The problem is, video plays on streaming too, but is very slow... like it plays for 3 seconds then halts for 2-3 minutes and so. I want to improve that using progressive streaming.

Secondly, our videos are 4K and are made for VR. App being developed is for Android, iOS targetting Samsung S7 for Oculus.

 

Thanks for this explaination, that makes sense and your solution is a good one! But I think this is quite an edge case.. If you think this is something we should add support for please can you email unitysupport@renderheads.com to create a ticket for this and we can then get our Apple developer to investigate further.

Thanks,

 

Hi Nick,

I'm not sure cropping a 360 equirectangular gives you the same as a 180 degree render? Did you try that sample 180 degree video that ships in the StreamingAssets folder?
Perhaps you could share some of your sample footage links so we can investigate the issue further. Thanks,

 

Hi GamerPET

We didn't design AVPro Video to play images. Your idea to convert the still images into little videos is probably the best workaround. We can look into having still image support for the "default" images so that they can be useful seamlessly.. I'll add this to the list.

Probably the easier way to do this would be to have a second sphere that renders on top, and you adjust the alpha value of this sphere so it fades down to reveal the other sphere. Then you hide that sphere. Each time you load a new media you would have to swap which sphere has the video applied. A bit tricky for sure but that's the simplest method I can think of without playing with shaders.

 

Hmm..the performance should be good. Please check that you have multi-threaded rendering enabled in the Unity "Player Settings". You may also benefit from enabling OES rendering mode (in Android settings for the MediaPlayer) as this results in more direct rendering.

Thanks,

 

In a few days we're releasing a new version with ExoPlayer support for Android. After that we'll be releasing a version with improved HLS and DASH stream switching support for all platforms.

Thanks,

 

Oh really?? Hmmm..thanks, have added to the list to investigate ASAP.

 

Hi, this is the wrong forum for this post This Forum is for AVPro Video, and you wanted AVPro Movie Capture forum. I'll answer your question anyway.

Currently it is not possible to capture multiple audio sources. This is something we have on our radar but currently have no timeline for yet.

Thanks,

 

Thanks for reporting the fix for this! We'll be releasing the new version (1.7.0) which will include this fix in a few days. Cheers,

 

Hi,

I'have some trouble to play movie on Iphone 5S & iOS 8.1.2 (12B440) , I use BestHTTPPro to download and save video on PersitentDataPath, video are downloaded and played fine on Android device but on IOS:

Code (CSharp):

1. Initialising for graphics device type Metal
2. Player 54: Failed to play movie: file:///var/mobile/Containers/Data/Application/1F6E75BF-4066-42C4-A991-52CA66832F10/Documents/Videos/leSuquet.mp4
3. [AVProVideo] Error: Loading failed.  Codec not supported or video resolution too high or insufficient system resources.
4. RenderHeads.Media.AVProVideo.MediaPlayer:UpdateErrors()
5. RenderHeads.Media.AVProVideo.MediaPlayer:Update()

code used to play video:

Code (CSharp):

1. _location = MediaPlayer.FileLocation.RelativeToPeristentDataFolder;
2. mediaPlayer.m_VideoPath = Constants.VIDEOS_FOLDER + "/" + videoName;
3. _mediaPlayer.m_AutoStart = true;
4. _mediaPlayer.OpenVideoFromFile(_location, _mediaPlayer.m_VideoPath);

The file exist on mobile folder and I can play on my mac when I open the .xcappdata

Video setting: https://screencast.com/t/k2N6up61d

The strange way is that the same video works well with AbsolutePathOrURL.

Any suggestion

Thanks

EDIT: Works fine on Iphone 5S & IOS 10.3.3, Iphone 6 with IOS 9.2 (13C75)

 

I believe currently you have to use the "DirectShow" video API if you want to redirect audio to a non default audio playback device. The AudioOutput component in theory should work though with Media Foundation API - I'll add a task to the bug testing list to check this.

Thanks,

 

That's strange..

Does it only do this when you're switching scene? Does it work fine normally?

If it only happens when you switch scene perhaps you could send us a sample scene with your setup so we can debug it, or at least describe in details how you're using the plugin.

Thanks,

 

Hi,

This is custom functionality that you would need to add yourself. I would:

1) Scan the file system for the list of media in a folder (using sometihng like System.IO.Directory.GetFiles())
2) Set up an event for FinishedPlaying event
3) call OpenVideoFromSource with the next URL

I hope that helps.

Thanks,

 

Yes that is possible, however we don't include any demo for this case. For iOS you would have to use a 3rd party plugin to "pick" the file - which would return a URL you can pass into your plugin for playback. See the FAQ in our PDF documentation (number 15 and 16) for more information about iOS. For Android I'm not sure how it works, but I think it should definitely be possible.

Thanks,

 

Yes, we recently found this issue. It seems that newer versions of Safari block videos from auto-playing. The only way around this is to go to the Safari Preferences and disable this. We're trying to see if we can at least prompt the user that they need to manually play the video in this case. Hopefully this "fix" should be in an upcoming version.

Thanks,

 

Hi,

It should work if you can find the URL of the HLS stream. The URL of the websites containing the stream isn't enough.

Thanks,

 

I think UWP has all sorts of security restrictions about accessing files...The file has to probably be within the application folder, or in some related application folder....This isn't something I know about specifically but it should be in the UWP documentation. You might want to try using the Application.persistentDataPath

Thanks,

 

Hi, let me answer your questions:

1) 1.7.0 is coming this week. Unfortunately that feature has had to be pushed back. Seamless playing is currently possible, and we have it in the demo scenes - we just wanted to improve upon this and make it something of a more reusable component. Currently there isn't a timeline for this.

2) CloseVideo() should free memory. On iOS though a lot of the memory management is out of our hands and is controlled by the operating system. It may like to keep some memory around in case the video is played again... The memory is unmanaged so this is why it is not affected by GC.Collect() or Resources.UnloadUsedAssets(). The only thing I can suggest is to enable the YCbCr mode (in the iOS settings of MediaPlayer) as this option will use a lot less memory during playback.

3) This is tricky. The WebGL still has a lot of missing features, so it is even difficult to know how many video frames have been delivered. Normally I would suggest using the timestamp of the video frame to sync to the audio playback, but this isn't available in browsers...Perhaps you just have to check the current video playback time value and use that to compare against the audio track time value..

Thanks,

 

This feature is being added to the new version released later this week.

Thanks,

 

On which platform?

You should be able to use either:
GetBufferingProgress() which will return a value between 0 and 1

or GetBufferedTimeRangeCount() and GetBufferedTimeRange() which can be used if various time ranges have been downloaded

These functions aren't available on all platforms though, so let me know if its missing our your platform so I can see if its possible to add it.

Thanks,

 

Ahhh, that does sound annoying. Thanks for reporting this - have added it to the list to test and fix.

 

We don't have any feature like this I'm afraid. One solution would be to just fix the source video

What you could do perhaps is use the AudioOutput component (only supported in Windows currently) to send the audio to Unity, and then if Unity has a method to boost the audio volume then you could use that?

Thanks,

 

UWP has a lot of security restrictions. I don't think it lets apps access files in any folders.

You need to use a special file selection system to access other folders. We have a demo of this called NativeMediaOpen.cs

Thanks,

 

The OpenFileFromBuffer currently only supports the DirectShow API in Windows. In Windows 7 this is the default API, but in Windows 10 it is not and you would need to use it from the Windows Platform Specific options.

Thanks,