[Released] AVPro Video - complete video playback solution

Thanks to your instructions the problem might be easier to spot than expected: I checked the plugin folder and there is no UWP folder. So in the end it only seems to be that my company did not purchase a package fit for UWP development.
I'll refer this to the person in charge and come back to tell you if I still have any problem exporting on hololens with the full edition.
Thank you for your kind answer

EDIT: obviously I made a fool of myself by not realizing that the DLLs are in plugin/WSA/UWP and not just in plugin/UWP. Anyway, the DLLs are here and rightfuly assigned to the ARM64 platform. The 360spherevideo scene also works (very pixelated but maybe it's how it's meant to be).
So in the end I still don't understand why your demo video works but not my own.

 

That does help slightly, but I still get a white flash in the left eye in VR while the frame is extracting. Also, what is this timeout value exactly? Not specified in the documentation.

 

Oops sorry that I told you the wrong folder.

So if you use one of our sample videos in your application, then it works?

If there's true then there must be some problem with your video encoding...
If you can give information about the codec and video format you're using then we might be able to suggest something.

Thanks,

 

The timeout value is just how long it waits for a new frame to arrive. I think though that in most cases because Unity has to wait for the render thread, this isn't timeout isn't actually useful. So I think you could perhaps try set the timeout to 1 or even 0.

As for the white flash - that's strange... Are you seeking to another part of the video to extract while also displaying it?

Are you able to reproduce this in a simple scene/project? Then perhaps you could send this to us so we can investigate.

Thanks,

 

There is also a TimeThreshold para meter:

public Texture2D ExtractFrame(Texture2D target, float timeSeconds = -1f, bool accurateSeek = true, int timeoutMs = 1000, int timeThresholdMs = 100)

I don' t know what that is. Good documentation on this would be very useful.

Will try to send you something that you can reproduce. How do I send it?

The other problem is that half of the calls to ExtractFrame return the first frame of the video, evenif "timeSeconds" parameter is 20. I can see in the MediaPlayer that the video actually loads and seeks to the proper place, but the returned texture is the first frame. Why??? I tried to play with the Timeout values or timeThreshold values to see if that made any difference, but nothing...

And yes, normal seeking in the video works like a charm, no problem with this. Only with frame extraction do I get a problem.

 

I'm starting to have something that works, but I have to seek manually and catch EvenType.FinishedSeeking and then extract the frame. However, isn't this the point of the "timeSeconds" parameter of ExtractFrame? To seek within the video without having to do this manually?

 

Yes you're right, it should work like that.

We have a demo scene of this feature called 06_Demo_FrameExtract which uses the example script FrameExtract.cs - does that work for you?

 

However, isn't this the point of the "timeSeconds" parameter of ExtractFrame? To seek within the video without having to do this manually?

 

hello
thank you for this smashing component .

i looking for a Playmaker help to make a download stream indicator ( an simple icon in waiting video be have enough buffering to play . ). more pricesely when video is not playing .
all day i've tried many combinations and i don't success . I don't understand why "Info Is Playback Stalled" action doesn't have "Every Frame" tick .
When I am launching the process , this action stay grey or mute . should "Every Frame" be in this action ?

help please
thank you

 

To give closure to this case and the whole story : At one point I noticed that the package I had was outdated, so I decided to test with the latest trial version. That's the version I used to test on the 360video scene, thereby confirming that the problem was how old my version was (I did not explain that in my previous post, which is why it seamed strange).
It appeard that the package my coworker gave me was an illegal one downloaded on a russian forum (that's why it was so outdated). So I asked the manager to buy the official asset, removed the old one from my project, reimported everything and re-added the components to the scene, and now it works perfectly.

So in the end the problem was entirely on my (and my company's) end. Nethertheless thank you for your fast and efficient response !

 

The issue here is that the StreamingAssets folder is exported as a separate folder and not within unity files (im talking about WebGL), and we had issues with black screen/video not starting from time to time and we think it might be related to server or something like that, do you have any suggestions ?

 

I'm afraid I dont' know what to suggest. Ideally we need a reproducible case so that it can be debugged. Does it do this when you run the WebGL build locally? Which platform and browsers have you tested? WebGL is tricky for videos because some browsers will not auto-play videos, especially if they contain audio. I can't see any reason why having the video in StreamingAssets woudl make a difference to whether the video plays or not though.

 

Ahh funny story, well I'm glad that you solved it and that using a legitimate copy triumphed over the pirated version

 

I'm afraid I really don't know. I'm not familiar with the Playmaker actions - actually they were written for us by the author of Playmaker so perhaps you could ask him?

The code for InfoIsPlaybackStalled seems correct though - and that is the right thing to use.

Do you use need to use Playmaker? If not , then in C# you would just need to call:

bool isStalled = mediaPlayer.Info.IsPlaybackStalle();

 

Can you please tell me whats the difference between Stop() and CloseVideo() functions ?
In other words, how to properly start and stop a video please ?

 

thank you. is he Jean fabre ? who is if possible ?
thank you

 

Yes Jean Fabre kindly created those Playmaker actions

 

Hello again,

I'm successfully using the Media Foundation -> Unity -> Audio Output -> Audio Source -> visualizers path. But it only seems to work if I set it before running, if I switch between unity and system direct during runtime, it won't show on the visualizer. Is that expected? (My use case is that I want to use system-direct almost all of the time unless they load a scene that has a visualizer, system-direct does things like play smoothly during scene loads, etc)

Thanks!

 

Yes, currently you would need to create the MediaPlayer component again if you wish to change the audio rendering mode. This is something we should have fixed in AVPro Video 2.0 so that it can be toggled directly. I guess a workaround for now would be to have 2 MediaPlayer instances - one for direct audio and one for AudioSource audio. Just make sure that you CloseVideo() on the one that you're not using so that it free's up memory and video decoding resources.

Thanks,

 

Hi All,

AVPro Video version 1.11.5 has just been released!

You can find the updated version on the Asset Store and the free trial version on our website.

Changes include:

General

• Fixed issue where Seek() sometimes didn’t trigger the seek events as it would complete the seek too quickly. Now the seek events are always triggered.

iOS / tvOS

• Fixed issue where YCbCr texture output would be black due to unrecognised colour transform type.

Windows

• Fixed Facebook Audio 360 mode handling for non FB360 audio content, for example where it would play back a stereo 22050hz track at the wrong speed
• Removed the Low Latency option as it created issues and didn’t really bring any advantages

Thanks to everyone that reported bugs that were fixed in this release

Please report any issues here on the forum, or preferably to our Github Issue tracker

Thanks,

 

Describe the bug:
The video play, but, not show image, only sound.

Your Setup (please complete the following information):

Unity version: 2019.4.x
AVPro Video version: 1.11.5
Operating system version: iOS 13.5
Device model: iphone SE 1 gen
Video specs: https://d22npu57smr6wp.cloudfront.n...ca_video_03_saopaulo_15fps_comsuavizacao.m3u8 (HLS format)

On log of iOS, not show any error

 

Is it normal that playing video by URL caused stalling a lot in Android?
It also caused stalling when I'm rewinding the already played video.

 

We'll reply to your issue over on GitHub where you posted it too.

Thanks,

 

It really depends on the video specs and the network connection. Perhaps you could give more details? WE have template on GitHub which would help us to offer you a fix/solution:
https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/new/choose

Thanks,

 

Hello, would it be conceivably possible for music to create a fading effect if I have two players and trigger one player to lower it's volume as it gets to the end of the track and then start the second player on the next track towards the end of the fade, raising it's volume as it takes over?

 

Yes that should work exactly as you expect.
We do have a component called PlaylistMediaPlayer (see the demo scene for this) which actually does this for you. It allows you to specify a playlist and then it can fade the audio (and video too) while transitioning.

 

Hi, can you say step by step how to play video from local android folder? I manually download it, set write permission of player settings. Url path was /storage/emulated/0/Download/snow.mp4 but video on phone not play. Please help.

 

Hello guys does your plugin support Hardware Decoding on Windows 8.1 Embedded ?

 

I'm not sure whether it is supported or not...I've not tried that OS before. In theory it should work, but some editions of Windows don't include media playback support. For that Microsoft added an optional download called Media Pack:
https://www.microsoft.com/en-gb/download/details.aspx?id=40744

So with that installed (if needed) I would say there's a good chance that it would be supported.

Hope that helps,

 

Hi,

Hmm it should work. Which version of Android are you using? I think that some versions of Android may block access to those OS folders by default without adding special app permissions to the manifest, see here:
https://stackoverflow.com/questions/57977190/scoped-storage-download-folder-access

You may have to uninstall your app and reinstall or, or add the 'external' folder install option in Player Settings, or add some of the above permissions for legacy access.

Otherwise, have a look at the Android log (adb logcat) as that should give the reason why access wasn't granted.

Hope that helps.

Thanks,

 

Hello guys,

I'm really interested in decording 10-bit videos to 10-bit textures.
When can we expect to try "Version 2.0.0 RC-1"?

With many thanks,

 

It's difficult to say - perhaps in 2-4 weeks?

 

Great!
I'm looking forward to it.

 

Thanks for answer, it was really helpful for me, but now I can see videos work from local storage only around 100mb. Its important for me play large videos around 3gb, it is possible to do it with Avpro and how to do it? Best regards.

 

Hello,

We have an issue where the last frame displayed from a video will be displayed before the next video to be played starts. Video 1 gets played, user stops the video half way through - our video player setup now gets disabled. When the user chooses Video 2 to play, before this video starts, the frame where Video 1 was stopped is displayed first. We made sure that the video is stopped, closed and load an empty file after it to make sure. Changing the render textures on the mesh used to display didn't seem to work, neither from ApplyToMesh and ApplyToMaterial. Is this something to do with a frame buffer? Is there a way to clear it? If not is there something else we could try? We're using Unity 2019.3.15, the issue happens on Android.

Thanks in advance!

 

Hi,

I am testing with hls stream from vimeo. Trying to simulate network problems with disconnecting from the internet and then connecting back. I am getting "Stalled" event, but it never unstalls after the connection is restored. Calling Play() also does nothing in that state. Is this a bug? Or if it is by design, what is the standard procedure to continue video when a connection was restored?

Thanks!

 

I was excited about this plugin when I saw "Facebook Audio 360" support, but looking at the documentation it only works on Windows and Android, and my target is both Android and iOS (you should probably make that limitation a bit more obvious).

Is there any way or any workaround to have spatial audio on iOS for a streaming video?

 

Hi, I'm evaluating AVPro for use in VR 360 stereo video.

I have a source top bottom mp4 video 4096x4096x30fps. I can load it with unity video player and see it in a 4096x4096 texture with the proper top bottom layout. Of course it runs pretty slow, which is why I am considering AVPro.

I'm testing in Unity 2019.1/win10/GTX1070

However, in all the demo scenes provided with the trial, the video seems to load only one eye (4096x2048), or at least this is what is displayed in the inspector. The video runs smoothly. I looked at the documentation but couldn't figure out why it's not displaying as 4096x4096 video in the preview.

Ideally I would like to have access to that texture with both eyes as I'm handling the proper remapping to sphere geometries in my code.

Am I missing something ?

 

IT's difficutl so say - we dont' have much information to go on. Your best bet is to submit details of the issue to our support teach on our Github Issues page here:
https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues

Thanks,

 

Please submit details on our Github Issues pages so that our support team can investigate:
https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues

The more details you give they higher the chance they'll be able to assist you.

Thanks,

 

Hi,

Our support team can assist you if report the details of your issues on Github Issues here:
https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues

Thanks,

 

Ahh sorry about that. There are so many details to convey it's difficult to highlight everything.

The only way to sort of have spatial audio would be to position the AudioSource elements in your scene yourself, and set the iOS audio output mode to Unity. You can then direct some of the audio channels to Unity AudioSources.

This won't support something like an ambisonic soundtrack though encoded within the media...

That's currently the only solution on iOS. I believe the latest version of iOS has added some support for ambisonic media, so this is something we'll be looking into soon.

I guess another option would be to use the Facebook Audio 360 Unity plugin itself (which has iOS support) and just use that to play back your audio instead of embedded it into the video.

Thanks,

 

Hi,

If you're seeing this behaviour in Windows, then this is a known issue. On Windows if the video contains stereo metadata, then only a single eye is decoded. One solution is to strip the stereo metadata from the video, or not embed it in the first place. This is only a problem on Windows.

We have also fixed this issue in the new upcoming AVPro Video 2.0 which should be released soon.

Thanks,

 

Yeah, that's my working theory now: streaming the video from some online service, but having the audio track inside the Unity project.

If I do have an audio track embedded in the video, will I be able to (a) mute it (or send it to a muted bus) and (b) still read the audio sample position from it in order to sync it with a "local" audio track?

Or is there another recommended way to ensure sync?

 

Yes you can mute it by checking the Mute option in the MediaPlayer component, or via script:

// muting after instantiation
mediaPlayer.m_muted = true;

// muting while it's loaded
if (mediaPlayer.Control != null)
{
mediaPlayer.Control.MuteAudio(true);
}

As for sync, if you start them both at the same time, they should be pretty much in sync. You might want to load both, pause them, rewind to 0 and then call Play on them to get the best chance of sync.

Thanks,

 

@NathanRH as I mentioned, video will not be shipping with the app, but instead it will be streaming from the web someplace (potentially Vimeo). So starting at the same time might not work, especially if the video stalls or buffers.
Are you saying there is no way to get the current timecode of the video?
Does GetCurrentTimeMs() return accurate information for streaming videos?

 

Ahh yes sorry I misread your post.
I think that streaming a video and keeping it in sync with local audio will be quite tricky. In most cases it'll work fine, but in cases where there isn't enough bandwidth to stream smoothly, it will be difficult.

Yes you can get the time from the video via GetCurrentTimeMs() - that is accurate. If you look in the Demos/Scripts/Scriptlets folder we have an example called PlaybackSync.cs which shows how to sync multiple MediaPlayer components - sync a bunch of slave components to a master. This may be a useful reference for how to keep things in sync, even though obviously you woudlnt' be using multiple MediaPlayers.

I think that perhaps your audio should be the 'master', and you should drive the position of the video based off time. You could even pause the video and seek through it instead of letting it Play() - but you would have to make sure there was enough data buffered (via Control.GetBufferedTimeRange). It's certainly tricky and not a feature we had planned to have native support for.

If the video time drifts too far ahead of the audio time, you can do 'micropauses' each frame, where you call Pause(), Sleep the thread for 1ms and then call Play() on the video. This will slowly make the video playback slow down, hopefully without any visible glitches. You could also just seek the audio to the video time, or perhaps even fractionally alter the playback rate of the audio if the API allows it, until it has a very similar time.

If the video time drifts too far behind the audio time, then you might need to pause the audio and wait until the video has caught up (and also that it has enough buffered time ahead of it, just to be safe).

Ideally you want to allow the video to buffer a bit before you start playback, so you should open the video in a paused state. This should allow buffering so that when you start the audio and video playback, they will stay roughly in sync. This would apply when seeking too. Then the above methods can be used to mitigate any drift that may occur.

I hope this is at least a bit helpful. It's definitely an edge case for us. We tried to add FB360 support on macOS and iOS but couldn't find a way to achieve it with the Apple API's.

Thanks,

 

Hello RenderHeads,

I maybe have a problem with the decoding. When I play a video I get artifacts in random occurrence. They are not in the video, but they seem to appear when the video has lots of changing information in it. (For example: in slow moving footage they almost never occur).

I captured a video of it, in a video thats heavily changing content:

(Note this is multiple projectors and multiple computers+players, that's why you see the artifacts only in one area)

For me it looks like the decoder misses a keyframe and the proceeds with the delta frames and make this mess.

I'm not 100% certain but i think this wasn't the case a few versions of AVPro Video ago. Can you see what causes the issue? Maybe its a bug? Can you recommend me encoding/format settings (ffmpeg h.264) that make it easier for the decoder (or less likely to produce these errors)?

 

Hi,
I want to trigger an event at some specified frame like Animation event in Unity. Is there any way I can associate it with specified frame in video? for example if I am playing VideoA.mp4 then while playing at say frame 00:00:25 I want to trigger an event. How can I associate it ? Is there any way I can create list of frames and events data associate with a video? Please suggest the best possible method to create it.

 

Thanks for reporting this - your use case looks very cool

It looks like the decoder can't keep up - which may be due to high encoding bitrate or driver issue.

Could you give specs of the video please (codec, frame-rate, bitrate, encoder settings), and also the playback machine specs (mostly GPU)?

Which version of AVPro Video are you using?

Thanks,