[Released] AVPro Video - complete video playback solution

Hello,
Question, is there any methods/events to detect if a video is 10bit or cannot be played so we can show relevant message or just open the video's folder ourself so user can use external player ?

 

And any way to cast the video to a chromecast/appletv or likewise ?

 

Hey @nir_unity1

Can you provide some more information about your problem? Which version of AVPro Video are you using? Are your videos playing in the editor before making a WebGL build? Can you tell us a bit more about the videos (resolution, codecs)?

Thanks.

 

Hi @shubhank008

Please see the updated documentation on 10-Bit video here: https://www.renderheads.com/content/docs/AVProVideo/articles/feature-10bit.html.

In terms events, please see this page for a list of events you could hook into: https://www.renderheads.com/content...ia.AVProVideo.MediaPlayerEvent.EventType.html. There's no specific way to detect 10-Bit video but you could check if there's an error playing the video.

Does this help?

 

Hi @trojant

I've made an issue on our Github for you so that we can look into this properly. Please follow progress here: https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/754. Please add any relevant details as well.

Thanks!

 

You could cast to Apple TV - you'd need to make sure 'allow external playback' is checked in platform options and we also don't provide any UI for triggering it, you'd need to add that in. Chromecast is not supported.

 

Hiya I'm attempting to get packed stereo and alpha running on Quest, I have heard tell that i need to enable multiview, I added a #define based on the .cginc and another post referencing display indexing. I modified the Android OES Inside Sphere shader, and it runs on Quest, but it seems like its making the whole thing semi-transparent. almost like its clipping off anythign below a certain threshold of black. It looks fine on the desktop, but i realize thats single pass vs multi-view. Any advice would be greatly appreciated!

 

Hi RenderHeads,

Can you advise on how to connect the Editor preview to the timeline playhead in AVPro 2?

We're big fans of AVPro here at Scatter. Our volumetric video tool Depthkit integrates directly with it, and we recommend it to all of our users above the Unity Video Player. Thanks for your amazing product!!

One key area where the Unity Video Player still wins out is that the Unity Video Playable timeline component supports scrubbing videos in Editor on the timeline.

We are in the process of upgrading to AVPro 2 and trying to get that functionality working without success. We feel like it should be possible because the AVPro Component has a timeline based scrubber/slider in the Inspector view that does work in the Editor. Can this be easily connected to the Unity timeline playhead through the Playable component?

You can see an example of what I mean in the beginning of this tutorial from us where we set up the timeline for scrubbing.

Thanks so much!

 

Hi @harold_unity365

We've had trouble getting multiview and OES to work in the past - it's a Unity/OculusXR issue.

If possible, would you be able to send us a sample project with a reproducible case over to unitysupport@renderheads.com? We could then look into it further.

Thanks!

 

Hi @obviousjim

You can expect a response on the Github issue you made: https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/755

Thanks!

 

Already checked EventsPage but there no specific way to handle crash or failure to handle 10bit video, OpenMedia returns true even for 10bit and the Error event or anything like that is never called, the player simply never plays after Play and nothing happens.
FirstFrameReady is the only event which is called for proper 8bit videos while its never called for 10bit videos so I was trying to handle closing the video player based on that but cannot figure out a proper way.

Also, if the above scenario happens (player doesn't play anything and all) and you try to exit play mode or call _mediaplayer.stop() it will cause Unity or the Standalone player to crash.

 

Basically I want to know if a 10bit is opened or failed so I can handle it by showing my user relevant message and stop the app from crashing.
--Update--

So I tried two things:

1. If using MediaFoundation (Win10), there is absolutely no way to detect or tell if a video is 10bit and there is no way to handle the crash which happens as soon as you _mediaplayer.stop or exit play mode.
Furthermore, in MetaDataReady event, doing _mediaPlayer.VideoTracks.GetActiveVideoTrack().DisplayName only gets you something like _track(en) or something generic.
This is frustrating.

2. If using DirectShow (LAVFilters installed), 10-bit video still does not play even though am on Windows 10 and have LAVFilters installed and DirectShow API selected (no other option modified).
However, trying to get video's name in DirectShow actually provides more detail including the part "yuv**" which I think can be used to detect 10bit video and close video player right there and then and maybe show a error message.
Below is the video track name returned:
V: English [eng] (h264 high 10 L5.0, yuv420p10le, 1280x720) [default] (en-US)

However, not only the video still cannot be played, even then the crash problem is there even with DirectShow.

Somehow I have a feeling either _mediaplayer.Stop() or the CloseMedia method is making the crash. Unfortunately no crash logs or anything usefull in either Editor or Standalone Player logs, last entry is always MediaPlayerEvent.EventType.Closing

 

how to add video path through script?

 

Hey @sudhishkumark6

Please have a look at this link for some examples: https://www.renderheads.com/content/docs/AVProVideo/articles/usage-loading-media.html#scripting.

Thanks.

 

Hello, sir. I have a compatibility question. My mp4 file couldn't be loaded in my Android app. I tried to look at its metadata. It said it was a mp4 file type but was Apple QuickTime (.MOV/QT) and video/quicktime as Major Brand and mime type. I read that avpro didn't support mov in Android. Does this mean Avpro saw the video as mov format instead so it wasn't compatible? Or it was another problem?

 

I am using the AVProVideo 1x. 1.11.7 (IOS) plugin with Unity 2020.2.0a9, but , while I can successfully build to device, I cannot pass validation for upload to appstore. There is a codesign failure of theAVProVideo.bundle.
It was suggested by techSupport to use
codesign -f -s "" --timestamp --options=runtime "".

While I have signed packages previously with terminal for Mac builds, I have no idea how to go about this for an IOS build. Is it possible to sign it through xcode?

 

Hi @ysleungrockman

It sounds like a compatibility issue, yes. MP4 is not a video codec, it's a container for video and audio (and other data). That means that not all mp4's will be compatible. So to ensure your video will play you have to check both the container and the video codec, as well as the audio codec.

For a list of supported media on Android please see the following link if using ExoPlayer API: https://exoplayer.dev/supported-formats.html, and this link if using the Android MediaPlayer API: https://developer.android.com/guide/appendix/media-formats.html.

 

This was resolved over email but adding this link for anyone else who has this problem: https://docs.unity3d.com/Manual/HOWTO-PortToAppleMacStore.html.

 

Linking the solution you found posted on the Github issue in case anyone else is looking into this: https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/755.

 

Hi, @TerryRH I used playlistmediaplayer(BETA) script, and after running there are two error,What can I do please?Thank you!
AVPro Version 2.1.0 Window10-64

(1)Transition shader not assigned to script
UnityEngine.Debug:LogError(Object)
RenderHeads.Media.AVProVideo.PlaylistMediaPlayer:Awake() (at Assets/AVProVideo/Runtime/Scripts/Components/PlaylistMediaPlayer.cs:299)
UnityEngine.Object:Instantiate(GameObject, Transform)
MediaManager:CreateMediaPlayer() (at Assets/Scripts/MediaManager.cs:200)
MediaManager:Start() (at Assets/Scripts/MediaManager.cs:28)

(2)ArgumentNullException: Value cannot be null.
Parameter name: shader
UnityEngine.Material..ctor (UnityEngine.Shader shader) (at <bc7613b106634015aa64439b702416be>:0)
RenderHeads.Media.AVProVideo.PlaylistMediaPlayer.Awake () (at Assets/AVProVideo/Runtime/Scripts/Components/PlaylistMediaPlayer.cs:301)
UnityEngine.Object:Instantiate(GameObject, Transform)
MediaManager:CreateMediaPlayer() (at Assets/Scripts/MediaManager.cs:200)
MediaManager:Start() (at Assets/Scripts/MediaManager.cs:28)

 

Hi all!

I get following error using AVpro Core, whenever I try to launch the sample Demo_360Stereo scene



I am trying to run the "Cones-360Stereo-2K30-HEVC" file from you project examples.

 

I Use Unity 2019.4.17f1 and try to run it on a Oculus Quest 2 headset via link cable (should run the same as Oculus Rift thethered PC headset).

 

Hi,

First of all, great asset ! Has good performance and is very easy to use.
I'm posting here because I have a question regarding performance optimization for VR, I'm not sure this is the right place to post so tell me if it's not.

My Setup :
Unity 2020.3.4f1
Oculus Integration 27.0

Hardware :
Nvidia GTX 1070 with 8Go VRAM
32Go of RAM
Intel I7-8700
Oculus Rift S

My goal is to have a smooth playback of 5.2K (5120*2560) at a frame rate of 80FPS encoded in HEVC at 200Mo/s. So far I managed to have really smooth playback at 60FPS and 150Mo/s bitrate but for some video with high motion there are some flickering appearing on the side. Going to 80FPS solve that issue because it match the headset refresh rate, but some performance issue start appearing. It's working pretty decently at 80FPS and 180Mo/s except for the first 10-20 seconds that aren't smooth at all. Maybe a buffer issue ? I've try to used the FinishedBuffering and ReadyToPlay event before starting the playback but it did no difference.

Here are some metric about the usage on the GPU (I'm using the windows task manager for this, might not be very reliable...) :
Playing 5.2K@80FPS encoded HEVC 180Mo/s : GPU peak at 100% on video decode and then stabilize around 60%.
So it feel like the GPU could keep up with the decoding but struggle a bit on the beginning.
CPU is fine and never exceed 20-25%.

I know I'm not providing very good metrics or detail (right know I don't have the time to make better analysis), but are they some common tips that I can use to improve those performance ? Can I for example increase the buffer size ?
I know obviously that I could reduce the bitrate and framerate but ideally I would like to not change those parameter.
When encoding to HEVC I'm using the fast decode profile that already helped me to have those performance.
I'm also aware of other codec like HAP but from the test I did it would require overs 2Go/s of disk access and file would be very huge.

Thank you for anyways for this great asset !

 

Hi All,

AVPro Video version 2.1.1 has just been released!

Changes include:

General

• Fixed
  • Fixed editor issue where certain layouts or having multiple inspectors visible would cause the MediaPlayer inspector to display collapsed and corrupted and SerializedProperty errors in the console window

Android

• Improvements
  • Limiting video streams cached by resolution and/or bitrate now supported
  • [#626] Added support for 'preferred max bitrate'

macOS/iOS/tvOS

• Fixed
  • Fixed issue with seek events not being triggered when a seek request completes within a single frame
  • Fixed issue with current time being incorrect when the max playback rate is set to greater than 2
  • Fixed issue with seek time not being correct when the max playback rate is set to greater than 2
  • Fixed issue where seeking in a finished video and then calling play would start from the beginning
  • [#750] Fixed issue where a local mp4 providing a visual media selection group would not configure the player correctly for playing back the video track
  • [#751] Fixed buffer over-release when playing video with multiple audio tracks, using audio capture and switching tracks before closing
    
  • [#753] Fixed issue with progressive downloads (mp4, mov, etc) attempting to be cached when that shouldn't be permitted

iOS

• Changed
  • Lowered minimum required iOS version to 10.0

Download Links

Editions
On the Unity Asset Store we've made several editions of AVPro Video available so you can pick the one that's best for your project:

• Core Edition

• Core Windows Edition
• Core Android Edition
• Core macOS/iOS/tvOS Edition​

• Ultra Edition
• Enterprise Edition (contact us for multi-site licensing / support options)​

You can read this page to learn more about the features, pricing and upgrades for the different editions.

DOWNLOAD THE FREE TRIAL VERSION HERE

Thanks to everyone who reported bugs or feature requests! We're looking forward continuing to support this product into the future and adding more features.

Thanks,

 

AVPro Video version 2.1.2 has just been released!

Changes include:

General

• Fixed
  • Fixed Texture leak in PlaylistMediaPlayer

Android

• Fixed
  • Bug regression

Download Links

Editions
On the Unity Asset Store we've made several editions of AVPro Video available so you can pick the one that's best for your project:

• Core Edition

• Core Windows Edition
• Core Android Edition
• Core macOS/iOS/tvOS Edition​

• Ultra Edition
• Enterprise Edition (contact us for multi-site licensing / support options)​

You can read this page to learn more about the features, pricing and upgrades for the different editions.

DOWNLOAD THE FREE TRIAL VERSION HERE

Thanks to everyone who reported bugs or feature requests! We're looking forward continuing to support this product into the future and adding more features.

Thanks,

 

Working on a project using AVProVideo version 1.11.7

Works great in a windows platform build, but when hoping to publish it on the Windows Store, switched the platform to UWP (Universal windows Platform) and the build fails with this:



I'm wondering if this is an incompatibility with .net or ? I've tried both .net 2 & 4, but with the same results.
Any idea how I might get this to build for the Windows Store/UWP platform? It's actually only meant to run on PC's. (have tried PC as the "Target Device" in the UWP platform build settings.)

Thanks,

Justin

 

Hi @RazT

This is mostly likely because you don't have the HEVC extension installed. This used to come free with Windows but they removed it a while ago. You can buy it for $0.99 on the Microsoft store: https://www.microsoft.com/en-us/p/hevc-video-extensions/9nmzlz57r3t7#activetab=pivot:overviewtab.

 

Hi @jolanbar

It might be a good idea to try encoding with a framerate of 40 so it can sync up better to the 80fps of the device. In terms of the high motion areas of the video, it could be if you're encoding with a variable refresh rate, that in areas of high motion it will consume more bitrate, so you could try using a constant bitrate instead. It might also just be that your bitrate is too high, in which case you could try two-pass encoding, which will give you optimal encoding with a smaller file size, in other words, smaller bitrate for the same quality https://trac.ffmpeg.org/wiki/Encode/H.265.

 

Hi @jtok4j

Not getting any errors our side. Could you send screenshots of the code that those errors point to please?

 

Hi, just upgraded to Ultra version because I need the 360 Audio support, but I'm having some issues to have it working.

I need it to play 360 video/audio on Android, specifically on Oculus Quest and PicoVR devices, and I'm following the doc hints about creating the right format to have spatial audio support, namely where the docs say:

"The best way to encode the video is to use the FB360 Encoder tool which comes as part of the FB360 Spatial Workstation. Set Output Format to “FB360 Matroska (Spatial Workstaton 8 channel)” and then set your video and audio source files and encode your video.This should create a MKV file with 10 channels of Opus audio."

Despite having followed the guidelines, when i try to load a mkv file with those specifications, I have this error:

[AVProVideo] Error: Loading failed.  File not found, codec not supported, video resolution too high or insufficient system resources.

That happens even in Unity editor, and moreover:

- If I open the same mkv file in Oculus TV app, I've it working, with spatial audio too, so the file shouldn't be corrupted or having any trouble.

- Every Ambisonic mp4 versions (1st and 2nd order) of the same file are regularly opened and played by AVPro, except the audio is not 360. The same files are played with spatial audio when I open them in Oculus TV app.

So far, I haven't been able to play a video with spatial audio, even in editor, and it seems every mkv file is creating issues to AVPro. Unfortunately, the docs about audio 360 are rather concise, so I haven't many more ideas to try.

Could you give me some help and more detailed infos to have spatial audio working?

Thanks!

Riccardo

 

2021/5/12
@TerryRH
I haven't found a solution yet.

 

Sorry, had some problems with materials and adjusted scripts being overwritten there, but have recovered.
I have updated to 1.11.7 but unfortunately it doesnt seem to make any difference . I still get the intialization exception


Everything works fine when I do a direct build on MacOS standalone. The exceptions occur when I build an XCode project for Mac Standalone.
I get the following exception:
DllNotFoundException: AVProVideo

at (wrapper managed-to-native) RenderHeads.Media.AVProVideSXMediaPlayer.AVPPluginRegister()

at RenderHeads.Media.AVProVideSXMediaPlayer.Initialise () [0x0000d] in <32f2f394f8a54acebf57e1d6d27b17eb>:0

at RenderHeads.Media.AVProVideSXMediaPlayer..cctor () [0x00016] in <32f2f394f8a54acebf57e1d6d27b17eb>:0

Rethrow as TypeInitializationException: The type initializer for 'RenderHeads.Media.AVProVideSXMediaPlayer' threw an exception.

at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_generic_class_init(intptr)

at RenderHeads.Media.AVProVideo.MediaPlayer.CreatePlatformMediaPlayer () [0x00002] in <32f2f394f8a54acebf57e1d6d27b17eb>:0

at RenderHeads.Media.AVProVideo.MediaPlayer.Initialise () [0x00000] in <32f2f394f8a54acebf57e1d6d27b17eb>:0

at RenderHeads.Media.AVProVideo.MediaPlayer.Start () [0x00008] in <32f2f394f8a54acebf57e1d6d27b17eb>:0

 

How to detect if a video is actually decoded to 10bit?

We are currently testing 10bit support with the Ultra Edition on Windows10 x64 and are able to play 10bit NotchLC and h265 (with installed extension) using Media Foundation and 10bit h264 and h265 using DirectShow and LAV filters. We have 'use10BitTextures' and 'useCustomMovParser' enabled as well as 10bit swap chain in the Unity project settings. The format of the created texture is always RGBA8 UNORM. So I would think that the 10bit videos are still decoded to 8bit internally. How do I verify that the video is decoded to a 10bit texture? Which format should the texture have? What might we be missing?
Tested on a MacBook Pro with dedicated AMD graphics and a Workstation with Quadro M6000, Unity 2020.2.5f1 and AVPro Ultra 2.1.1

 

Hi, the fixed script is attached - this will be included in the next release as well.

Thanks for reporting it.

 

Could you tell us the specs of the video?

Did you set the AudioOutput mode to Facebook Audio 360?

 

Is there a way to stop frames from skipping. Like option in builtin Unity VideoPlayer: SkipOnDrop = false?
My application is running on 30 FPS with Time.CaptureFramerate = 30, video is also at 30 FPS. And I want to render every frame even if some frames are rendered more than 1/30 sec. I need this because there are rare spikes in deltatime and other content moves out of sync with video because of CaptureFramerate.

I tried settingTimeScaleSupport, but it is not working. Application starts working at less than 15 FPS, and video switch frames at about 2 FPS, Video Decode in task manager shows 100%. It seems that the video constantly rewinds. Without this option video rendering is smooth at 30 FPS.

PS
Platform: Windows 10
Codec H.265, 8k

 

Hello,
I had some issues to get AVPRO Video v.2.1.2 work on Mac OS High Sierra 10.13.6: the plug-in cannot load
because of this library: @rpath/libswiftCore.dylib (cause : image not found).

I've found the solution: to get it work, I've installed the "Swift 5 Runtime Support for Command-Line Tools Package" (see
https://developer.apple.com/documentation/xcode-release-notes/swift-5-release-notes-for-xcode-10_2). This package is included by default starting with macOS Mojave 10.14.4. In macOS Mojave 10.14.3 and earlier, there’s an optional package to provide these runtime support libraries for Swift command-line tools that you can download from More Downloads for Apple Developers.

Could You update the documentation or the FAQ with this tip?
Thanks!

 

Hi, on my clients project using AVpro 1.10.0 seems that video volume on IOS Iphone is very low but on IOS Ipad is ok, this is normal? How to fix it?

 

Is the phone volume set okay? Our code for iOS is the same - no difference for iPhone or iPad.
If you're still having a problem please post it a new issue to GitHub - this is where our dev team monitor issues

Thanks,

 

Hi,

Please post this as a new issue to GitHub - this is where our dev team monitor issues.

Thanks,

 

Thanks for reporting this tip! Very interesting.... We will have a think about whether to add it to the documentation or adjust the plugin so this isn't an option....

Thanks again!

 

Hi! @TerryRH
There are two playlistmediaplayer(beta) script problem:
(1)It's not easy to control the player list with the function code of mute control, volume control and transition volume. And they seem to have conflicts. For example, after I modified the audio volume, the sound returns to the original value through the next video transition.
(2)Another test is that there is only a very short 4s video in the play list. When I play the list on a high frequency, the video sometimes turns white, and then I can't play it again. It's always white.is this a transition bug at openvideo?

Finally, We hope playlistmediaplayer script can be more perfect, thank you.

 

Hi,

Please can you post your issue to our GitHub as our technical support team monitor issues there more than on the Forums and you're more likely to get a reply.

Thanks,

 

Hi All,

AVPro Video version 2.1.3 has just been released!

Changes include:

General

• Improved
  • Preprocess build script now warns if an unsupported graphics API is used
  • Plugin preprocessor build script now handles case where plugin files have been moved outside of the AVProVideo folder
  • Added new SeekToLiveTime method to MediaPlayer component
• Changed
  • Some HEVC sample videos now encoded with H.264 mostly for better Windows compatibility
  • Unlit transparent shader now supports stereo
• Fixed
  • Fixed missing transition shader in PlaylistMediaPlayer during Awake()

Windows

• Improved
  • Added new useLowLiveLatency option for WinRt which allows live streams to have less latency
• Fixed
  • #772 Fixed fallback stereo packing mode not correctly handled by DirectShow

Download Links

Editions
On the Unity Asset Store we've made several editions of AVPro Video available so you can pick the one that's best for your project:

• Core Edition

• Core Windows Edition
• Core Android Edition
• Core macOS/iOS/tvOS Edition​

• Ultra Edition
• Enterprise Edition (contact us for multi-site licensing / support options)​

You can read this page to learn more about the features, pricing and upgrades for the different editions.

DOWNLOAD THE FREE TRIAL VERSION HERE

Thanks to everyone who reported bugs or feature requests! We're looking forward continuing to support this product into the future and adding more features.

Thanks,

 

Hi,

Thanks for reporting these issues.

If you could post details of this via a new issue on our GitHub - this is where our dev team monitor issues and is the best chance to get it resolved quickly.

Thanks,

 

After a deep analysis, seems that the problem is this code:

Code (CSharp):

1.   microphoneSource = gameObject.AddComponent<AudioSource>();
2.             microphoneSource.clip = Microphone.Start(null, true, 10, AudioSettings.outputSampleRate);
3.             microphoneSource.Play();
4.            yield return null;
5.  
6.  

The problem is that if i start a microphone on IOS the audio volume will drop a LOT, how to solve this?

I also create a new project with the above script, and added to the default scene
03_Demo_VideoControls


This is a HUGE problem for me because i need to record the screen...
Any solution?

EDIT:
Also with the trial last version (UnityPlugin-AVProVideo-v2.1.3-Trial.unitypackage) the problem still occurs.....

 

Have you tried enabling the "Prepare iOS for Recording" option in the iOS specific player settings? (see here)
You may also want to try enabling "Force iOS Speakers when Recording".

 

It seems that worked! thank you so much

 

Sorry for the late replay, I was in a big rush with my project.

I finally managed to have the spatial audio working, following the hints in your doc. I had many issues to have it working (one of them was I had Facebook360 audio set in the Android tab but not in the Windows tab, so it was not working in the Unity editor) but in the end I made it.

Do you think in one of the next releases AVPro could manage mp4 with Ambisonic audio (1st and 2nd order)? That would be great, because of the better support and standardization for the mp4 container.

 

Hi @RiccardoAxed

Could you please log this as a feature request on our Github with as much detail as possible.

Thanks