Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

[Released] AVPro Video - complete video playback solution

Discussion in 'Assets and Asset Store' started by AndrewRH, Feb 12, 2016.

  1. sandman3

    sandman3

    Joined:
    Feb 21, 2017
    Posts:
    12
    Hey AndrewRH!

    I am having a serious issue on our Android app using Exoplayer. I checked the logs, and it would appear to be a threading issue:

    Relevant code:

    while (!pendingPlaybackInfoUpdates.isEmpty()) {
    pendingPlaybackInfoUpdates.peekFirst().notifyListeners();
    pendingPlaybackInfoUpdates.removeFirst();
    }


    It looks sound, but still crashes. I've read that this is caused by using the player on different threads. When manually upgrading ExoPlayer to 1.9.0 (because they mentioned it giving warnings for when you use the wrong thread) I get a "wrong thread" warning for each rendered frame.

    See: https://github.com/google/ExoPlayer/issues/4738

    Do you have an ETA for when this is solved? It's a hard crash that appears to occur to about 3% of all sessions, at full random (because it's a race condition).
     
    Last edited: Nov 13, 2018
  2. dorukeker

    dorukeker

    Joined:
    Dec 6, 2016
    Posts:
    37
    Thanks for the reply. I will change our implementation accordinlgy. We are building an AR experience. It tells a story about a certain exhibition piece. The story is told via video; playing in multiple 2D planes in the 3D space; creating a paralax efect.

    I'll make sure to send some links when the exhibition opens ;)
     
    AndrewRH likes this.
  3. coldstar

    coldstar

    Joined:
    Sep 20, 2014
    Posts:
    11
    Do I have the opportunity to get the wav law data before outputting them to Unity or Sound Device?
    Than I can modify them to be louder... :)
     
  4. shoo

    shoo

    Joined:
    Nov 19, 2012
    Posts:
    62
    Is it possible to reload MediaPlayer options like "Android API" and "use TBE audio" on runtime?
     
  5. jacknorrisuk

    jacknorrisuk

    Joined:
    Jun 17, 2014
    Posts:
    38
    Hey guys, thanks for your pointers for this. Unfortunately your suggestions 2) 3) and 4) don't help with the problem, so we are now upgrading the project to the latest AVPro in case that solves it.

    Worryingly, we have also found that the Lenovo Mirage headsets still update their Daydream version even when automatic updates are disabled - only 3 out of our 11 headsets are still usable until we fix this bug! :O

    I would be interested to know if anyone else can reproduce this on the current version of Daydream.
     
  6. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Yeah we're still on ExoPlayer 2.8.4....2.9.0 will come soonish.

    Which version of AVPro Video are you using? We recently released 1.9.5 which has some related Android fixes.

    Thanks,
     
  7. sandman3

    sandman3

    Joined:
    Feb 21, 2017
    Posts:
    12
    The current live version is 1.8.9, so I'll see what happens when I update to 1.9.5. Thanks!
     
    AndrewRH likes this.
  8. jacknorrisuk

    jacknorrisuk

    Joined:
    Jun 17, 2014
    Posts:
    38
    Okay, we now have the latest AVPro in our project and can confirm that the problem still exists.
    Will continue testing and report back - any ideas in the meantime would be appreciated.
     
  9. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    :( that's a shame....I don't suppose Lenovo / Google have a list of what changed in that version?
     
  10. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    I'm sure that streaming 5 videos at once is beyond the intended design of most phones :)
    Phone hardware only has limited GPU video decoding capabilities, and memory is very low....iOS devices in particular suffer from memory issues when playing videos - they tend to allocate a lot of memory for playing a single video.

    Firstly I would highly recommend making sure that you're using the latest version of AVPro Video (version 1.9.5) as many improvements have been made. Also on iOS make sure that you're using the YCbCr option as this helps to save memory for video decoding. Then on Android, if you're still having issues, try switching the API to MediaPlayer from ExoPlayer. All of these options are in MediaPlayer > Platform Specific > Android/iOS.
    In general you should also try to make your Unity app use less memory. There are various documents available to tips on how to do this. For one thing you should go into Graphics settings and remove all built-in shader types that you don't need. Also in auto-graphics API options, if there are two APIs listed, try only keeping one of them.

    I see that you're using a very old version of the plugin (1.6.6). You should upgrade to the latest version (1.9.5) as we've made many improvements to memory usage and other things.

    Thanks,
     
  11. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Yes that's true...having a single display is best. Hope you're winning there...

    Thanks,
     
  12. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Kind of..On Windows desktop (only) we support playing back the audio via Unity (in MediaPlayer > Platform Specific > Windows > Audio Output). This uses a script called AudioOutput. You could modify this script to boost the volume.

    Thanks,
     
  13. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Currently no...you have to remove the MediaPlayer component and recreate it if you want to change those options. Those options change a lot of fundamentals of how the player is created, so we can't switch them dynamically. In a future version when we improve the API we will try to make all of these options changable.

    Thanks,
     
  14. jacknorrisuk

    jacknorrisuk

    Joined:
    Jun 17, 2014
    Posts:
    38
    Documentation for the Daydream platform let alone the Lenovo Mirage itself is horrendously thin...we have tried to contact their support about it but get no response. We are going back through all of our code and project settings to try and fix this by stabbing in the dark - but thanks for your help!
     
  15. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    First thing I would try: disable multi-threaded rendering
     
  16. Monarc

    Monarc

    Joined:
    Jun 20, 2016
    Posts:
    7
    Hi,

    I found following exception occuring on the android while opening the video clip. My devices are Galaxy S7(OS 8) and Pixel 2(OS 9) and Pixel 3(OS 9). Most of exeptions occurred on Pixel 2,3.
    The version number of avpro is 1.9.5

    I want to fix this issue as soon as possible because it makes crashes.
    Let me know what should I do or if you need more detailed information.

    Thanks in advance.

    이미지 406.png
     
  17. coldstar

    coldstar

    Joined:
    Sep 20, 2014
    Posts:
    11
    Thanks, I will try it.
     
  18. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    How often does the crash happen?
    Are you able to replicate it with one of the demo scenes?
    Are you using the FB360 Audio feature?

    Thanks,
     
  19. remy_rm

    remy_rm

    Joined:
    Jan 16, 2017
    Posts:
    17
    Hi Andrew,

    We recently updated our AVPro video to 1.9.5, and since the update we have been unable to build for mobile because of a shader error.

    Code (CSharp):
    1. Shader error in 'VR Composers': 'Convert420YpCbCr8ToRGB' : no matching overloaded function found; 'constructor' : not enough data provided for construction at line 168 (on gles)
    2.  
    3. Compiling Vertex program with MONOSCOPIC STEREO_DEBUG_OFF HIGH_QUALITY_OFF APPLY_GAMMA_OFF USE_YPCBCR
    4. Platform defines: UNITY_NO_DXT5nm UNITY_NO_RGBM UNITY_ENABLE_REFLECTION_BUFFERS UNITY_FRAMEBUFFER_FETCH_AVAILABLE UNITY_NO_CUBEMAP_ARRAY UNITY_NO_SCREENSPACE_SHADOWS UNITY_PBS_USE_BRDF3 UNITY_NO_FULL_STANDARD_SHADER SHADER_API_MOBILE UNITY_HARDWARE_TIER1 UNITY_COLORSPACE_GAMMA
    And the same on Gles3 and Vulkan

    Shader error in 'VR Composers': undeclared identifier 'Convert420YpCbCr8ToRGB' at line 168 (on gles3)

    The faulty line of code:
    fixed4 col = fixed4(Convert420YpCbCr8ToRGB(ypcbcr), 1.0);

    which is in the
    Code (CSharp):
    1. #if USE_YPCBCR
    2.     #if SHADER_API_METAL || SHADER_API_GLES || SHADER_API_GLES3
    3.                 float3 ypcbcr = float3(tex2D(_MainTex, uv).r, tex2D(_ChromaTex, uv).rg);
    4.     #else
    5.                 float3 ypcbcr = float3(tex2D(_MainTex, uv).r, tex2D(_ChromaTex, uv).ra);
    6.     #endif
    7.                 fixed4 col = fixed4(Convert420YpCbCr8ToRGB(ypcbcr), 1.0);
    8. #else
    9.                 fixed4 col = tex2D(_MainTex, uv);
    10. #endif
    block.

    Playing in the Unity editor works fine without any errors.

    Build device is a samsung galaxy S8 edge running android 8.0

    Do you know how to fix this?

    The VR Composers shader is basically the AVProVideo/Unlit/Opaque android OES ONLY shader, but with a small edit to it. But the same error occurs using the standard Unlit/opaque shader
     
    Last edited: Nov 15, 2018
  20. dorukeker

    dorukeker

    Joined:
    Dec 6, 2016
    Posts:
    37
    Hello Again,

    I am implementing the option 2 as suggested.
    There is single video file playing in one MediaPlayer.

    We try to display different parts of the video on different planes using offset and scale of ApplyToMesh component.
    Please see the drawing below. (Main video with dimensions on the right and planes on the left)

    I cannot get the offset and scale values for each plane correct.
    Probably I don't have the correct understanding.
    Can you help me understanding how those work?
    For example what would be the offset and scale values of the ApplyToMesh components of the planes in the drawing?

    Thanks in advanced.
    Cheers,
    Doruk

    IMG_0021.jpg
     
  21. jacknorrisuk

    jacknorrisuk

    Joined:
    Jun 17, 2014
    Posts:
    38
    Hey, thanks for your help. I can confirm this does not make a difference I'm afraid.
     
  22. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Hi,

    Yes in 1.9.5 we removed Convert420YpCbCr8ToRGB and replaced it with ConvertYpCbCrToRGB..which now has an extra matrix parameter for the transform. This was to improve the accuracy of the conversion. You will need to update any of your customised shaders to use this new method, or copy the old method into your shader. Here is the method for reference:

    Code (CSharp):
    1. INLINE FLOAT3 Convert420YpCbCr8ToRGB(FLOAT3 ypcbcr)
    2. {
    3. #if 1
    4.     // Full range [0...255]
    5.     FLOAT3X3 m = FLOAT3X3(
    6.         1.0,  0.0,      1.402,
    7.         1.0, -0.34414, -0.71414,
    8.         1.0,  1.77200,  0.0
    9.     );
    10.     FLOAT3 o = FLOAT3(0.0, -0.5, -0.5);
    11. #else
    12.     // Video range [16...235]
    13.     FLOAT3X3 m = FLOAT3X3(
    14.         1.1643,  0.0,      1.5958,
    15.         1.1643, -0.39173, -0.81290,
    16.         1.1643,  2.017,    0.0
    17.     );
    18.     FLOAT3 o = FLOAT3(-0.0625, -0.5, -0.5);
    19. #endif
    20.  
    21. #if defined(SHADERLAB_GLSL)
    22.     return m * (ypcbcr + o);
    23. #else
    24.     return mul(m, ypcbcr + o);
    25. #endif
    26. }
    Also in 1.9.5 we actually introduced a bug into the OES shader code..so please upgrade to 1.9.6 which will be released shortly.

    Thanks,
     
  23. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Hi All,

    AVPro Video version 1.9.6 has just been released!

    You can find the updated version on the Asset Store and the free trial version on our website.

    Changes include:

    macOS
    • Fixed a Metal rendering issue where the watermark didn’t display in the editor
    • Fixed issue with Hap and Metal rendering in the editor
    • Upgraded to latest Hap version (1.6.9)
    iOS / tvOS
    • Fixed a crash in iOS 9 and lower that could happen when closing a player
    • Fixed packed transparent shaders to expand range correctly when using YpCbCr mode
    Android
    • Fixed bug where using the OES shader would cause video to display pink/black, this was a regression introduced in 1.9.5
    General
    • Shaders refactored

    Thanks to everyone that reported bugs that were fixed in this release :)

    Please report any issues here on the forum, or to our customer support email

    Thanks,
     
  24. Polygoat

    Polygoat

    Joined:
    Aug 29, 2015
    Posts:
    7
    I'm currently evaluating AVProVideo for a client project, but when using the latest trial version in Unity 2018.2.11f1 (on MacOS 10.12.6) every video in every sample scene just loops the first few frames. The audio plays fine.

    Any idea what is wrong here?



    >> Read the same problem on page 56. This a a known problem apparently. I switched to OpenGL, and everything is working ok now.

    Still, would't it be a good idea to list known bugs or something. If I'm going to base a long-term project on this asset, I have to be sure about long-term support.
     
    Last edited: Nov 17, 2018
  25. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    You're right - this is a known limitation of our plugin. We will add a section to the documentation that list these. This particular issue is only on macOS and only when using the Metal graphics API with a specific GPU (so far we've only heard about it on Macbook Pros). We haven't managed to fix it as we don't have the hardware to replicate it. For now the only option is to switch to OpenGL. Hopefully we'll be able to get this issue happening in our studio so we can work out the problem.

    Out of interest, which version of AVPro Video are you using (it should print the version to the console window)? We just released 1.9.6 which we were hoping would fix this issue..

    Thanks,
     
  26. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    You should be able to set the Scale to 0.33333, 0.5 for all ApplyToMesh instances
    Then you can adjust the offset value for each video, so:

    TopLeft = 0.0 , 0.5
    TopMiddle = 0.3333 , 0.5
    TopRight = 0.6666, 0.5
    BotLeft = 0.0 , 0.0
    BotMiddle = 0.3333 , 0.0
    BotRight = 0.6666, 0.0

    Hope that helps,
     
  27. Polygoat

    Polygoat

    Joined:
    Aug 29, 2015
    Posts:
    7
    Thanks for your answer. This test was with v1.9.6.

    One more question:
    The application I'm developing with AVPro Video will download video's from a backend and store them locally. The first version should support Windows and MacOS, later versions will also be released on Android and iOS. I'm searching for a way to store these video's while protecting them from usage in other players. I understand DRM is currently not supported, but the Android version supports a file offset feature. Is it possible to use this on Windows/MacOS/iOS as well, as a fast/easy protection? It would be better than nothing, and prevent playback for most users.
     
  28. remy_rm

    remy_rm

    Joined:
    Jan 16, 2017
    Posts:
    17
    Thanks for the quick update with the OES shader fix, solved my issues :)

    Just in case others run into this: After updating to 1.9.5(/6) from an earlier version the following dex error might occur (as it did in my case) when building for Android:
    CommandInvokationFailure: Unable to convert classes into dex format. 
    resulting in a build failure.

    This is caused by the new "audio360-exo28.jar" being added in the Assets/Plugins/Android/ directory. You can remove the old "audio360-exo26.jar" to resolve this error.
     
  29. jeromeWork

    jeromeWork

    Joined:
    Sep 1, 2015
    Posts:
    357
    @AndrewRH The Unity asset store doesn't appear to explicitly mention this, but how is the licensing handled for this asset? Is it one license per seat?
    I'm working as lead developer on a project but will have to share the project with a colleague who may do some edits to the code base. Do we both need a license?

    Great to have AVPro Video in the sale btw :)
     
  30. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Thanks for checking in with us.

    On the Unity Asset store, I believe that all assets are site licensed (any number of developers at a single company site/location), except for editor-extensions, which are per-seat.

    The terms are here:
    https://unity3d.com/legal/as_terms

    I'm no lawyer, so I don't know how it applies to you and your colleague in different locations.

    Thanks,
     
    jeromeWork likes this.
  31. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Yes that's right, the old exo26 JAR should be deleted. Thanks for mentioning this. We will add it to the FAQ.
     
  32. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    I'm afraid the file-offset feature is currently only on Android. We don't have support for this on other platforms and we don't have plans to add it in the near future.
     
  33. VRPE

    VRPE

    Joined:
    Feb 21, 2015
    Posts:
    2
    Hello,
    i got huge problems with tearing on an Intel Iris Plus 655 and Windows 10 while loading/switching Video. Resolution is 1920x1080.

    It really looks terrible. I tried everything to force Vsync, but no success.

    Have you got any tips for me?

    Best regards
     
  34. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Could you give more details? Have a screenshot? Are you able to reproduce it using our demo scenes + media?

    Thanks,
     
  35. VRPE

    VRPE

    Joined:
    Feb 21, 2015
    Posts:
    2
    Hi, thank you for your quick response.

    I attached an short video, only had to black out the corner ;) In the video you can see the artefacts.

    We tried with the demo scenes, same problem. We also have the problem with the demo media.

    There seems to be a problem with the graphic card and unity/ AV Pro.
     

    Attached Files:

  36. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    I've seen this sort of triangle artifact before when using Intel chipset for decoding. I always assumed it was some sort of optimisation in the Intel decoder...
     
  37. jfawcett

    jfawcett

    Joined:
    Jan 13, 2018
    Posts:
    4
    I don't know if you're aware of it, but playing anything other than .NET 3.5 on Android has been broken since September.

    https://issuetracker.unity3d.com/is...-when-executing-certain-sslstream-constructor

    There are only 12 votes for this bug, but it's completely shut down the player on Android for me. Checking the log, you see:

    Unable to lookup library path for 'libc', native render plugin support disabled.
    Unable to find libc​

    Is there another workaround I'm not aware of? Do you have a fix of your own? Going back to .NET 3.5 would be quite painful for me.
     
  38. idunnuhow

    idunnuhow

    Joined:
    Apr 16, 2014
    Posts:
    23
    Is there a way to increase playbackrate to more than x4?
     
  39. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    I wasn't able to replicate this.

    I tried in Unity 2018.2.11f1, using AVPro Video 1.9.6, with IL2CPP enabled using .Net 4.x. It worked fine.

    The only issue that I had was when enabling ARM64 architecture, I had to manually go to Plugins/Android/libs/arm64-v8a and set the platform of all of those files to Android, and their architecture to ARM64.

    If you're using 1.9.6 and the above settings, and still having problems, please send us a copy of your project or a cut-down reproducible scene (www.renderheads.com/contact/) thanks,
     
  40. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    It depends on the platform, video encoding, and video resolution, bitrate, frame-rate, etc.
    In the UI we have limited it to 4x, but via the API you should be able to increase it more.
     
  41. Monarc

    Monarc

    Joined:
    Jun 20, 2016
    Posts:
    7
    The crash occurs once in 5 times.
    I can't reproduce it with demo scenes. (Some of demo scenes causes crashes but it caused by another issue)
    I'm using 'facebook audience network' which require exoplayer. Could this be the cause? Is there anything I especially need to know to use these 'facebook audience network' and 'avpro video' together?

    Thanks.
     
  42. jfawcett

    jfawcett

    Joined:
    Jan 13, 2018
    Posts:
    4
    I made sure I am using the same settings you describe above, and I am getting the same error. Then I tried loading one of your demo scenes. That worked. What was the difference, then?

    I narrowed it down to 3 things. You can take it from here...

    1. Start with Demo 4
    2. I use a stream source. Try https://videos3.earthcam.com/fecnetwork/tokyo1.flv/chunklist_w480902807.m3u8
    3. Check the Persistent box
    4. Change the Force File Format to Smooth Streaming
    5. Build for Android
    6. Run and watch nothing show up.

    Expected: A live stream from Tokyo Harbor where the rabbit usually is

    Actual: A blank video stream and 404 errors in logcat. Turn off Persistence and change the Force Type back to Unknown and you'll see Tokyo.
     
  43. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    605
    Hi Andrew

    Thanks for the wonderful plugin. I am getting a serious hickups when opening HAP videos, even with relative small (~5mb) files. What is the best practise for reducing load duration?

    All the best!
     
  44. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Please could you let me know what is the model of your machine? Ideally a screenshot of the About screen would really help. We're trying to narrow down which machines this happens on, as it isn't happening on any of our mac machines here. Thanks,
     
  45. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Do you mean when you load the file? Our DirectShow video player isn't asynchronous, so the main thread is blocked during loading. If you're using MOV, try swapping the container to AVI..or vice-verse, as the container type may affect loading time. Or if possible try using multiple MediaPlayer components, and pre-load all the Hap videos upfront.
     
  46. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    I meant, how frequently can you get it to happen - every single time, every second time, once in a hundred? :)

    If you can't reproduce it in the demo scenes, only in your own scenes, then yes it is likely that sometimes else is causing the problem. That facebook exoplayer extension does sound like a likely candidate. I have never used it before though. I wonder what version of ExoPlayer they're using. We use 2.8.4, so if they're using a different version then there could be a conflict. You could also try making AVPro Video use the MediaPlayer API instead of ExoPlayer. You can toggle this in MediaPlayer > Platform Specific > Android > Video API.
     
  47. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Ahh, I see - thanks for explaining. So the link you gave (m3u8) is a "HLS" stream, not "Smooth Streaming" which is a Microsoft stream and not very popular. When you force the stream type and it isn't correct, then it won't be able to load it. We will improve the Force File Format list so they are more descriptive in the next update. Thanks,
     
  48. Tom_3D

    Tom_3D

    Joined:
    Oct 1, 2014
    Posts:
    1
    Does AVPRO android support using the video texture as a vertex shader (displacement) in ES3.1 (gearvr/oculus go)? We are having to abandon UE4 since they are commited now to a huge feature regression with their media textures.
     
  49. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,655
    Hi Tom,

    Yes, the texture is just a normal texture...so you can use it for anything, including a displacement map, and access it in the vertex shader.

    I didn't know UE4 was reducing media texture support - do you have a link?

    Thanks,
     
  50. LostPanda

    LostPanda

    Joined:
    Apr 5, 2013
    Posts:
    151
    @AndrewRH Function request: support external clock or external frame sequence number for synchronization.Thanks
     
    ECNUnity likes this.
unityunity