Search Unity

  1. Unity 2020.2 has been released.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

[Released] Demolition Media Hap — multi-platform 8k@60fps GPU video playback

Discussion in 'Assets and Asset Store' started by leavittx, Feb 12, 2017.

  1. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111


    Hap is a video codec which allows to decode high-resolution videos using mainly GPU power and gives ability to play videos with alpha channel transparency.

    It proved to be the only one solution which managed to preserve glitch-free high-framerate video playback in our mixed reality playscape with Unity running on 8 FullHD projectors. We tried other solutions, but they just didn't work. Finally, we made this plugin.

    Main features:
    — Native hardware Hap video playback support without any codec installation needed.
    — Low CPU/memory usage. Frames are decompressed on the GPU
    — Works with all the render interfaces (DX9, DX11, OpenGL, Metal) on the desktop platforms.
    — Both 32 bit and 64 bit supported! Even on OS X!
    Best playback performance achievement: 8k@60fps Hap, 4k@60fps Hap Q
    — Play multiple videos at once with minimal CPU overhead
    — Transparent videos with Hap Alpha codec!
    — Audio output through the Unity Native Audio API
    — Can preload video to memory if your SSD isn't fast enough
    — Suits for both programmers and artists: C# API, IMGUI/uGUI wrappers provided
    — Demonstration scenes for typical use scenarios
    — Demo version provided (see the link on the asset store)

    If you work with video in Unity, which includes, but not limited to
    • VR projects
    • multi-display/projector walls and playscapes
    • playing alpha channel-enabled content
    • VJing
    • other cool stuff
    get it on the Asset Store: https://www.assetstore.unity3d.com/en/#!/content/78908
     
    Last edited: Feb 19, 2017
    m4d and TechnicalArtist like this.
  2. izm1252

    izm1252

    Joined:
    Dec 24, 2015
    Posts:
    7
    Thanks for Great Assets!

    Can I use this plugin with playback speed rate change?
    I want to change playback speed in scene.
     
  3. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    Hey! Thanks for your request. I will add playback speed control possibility to the upcoming new version of the plugin. Do you need the audio to be played as well with the varying speed, or what you need is only the video (audio could be a bit more tricky to add) ?
     
  4. izm1252

    izm1252

    Joined:
    Dec 24, 2015
    Posts:
    7
    thanks for your quick reply!

    I want only Video playback speed change (currently).
     
  5. adriaanwormgoor

    adriaanwormgoor

    Joined:
    Nov 14, 2013
    Posts:
    7
    Hi, thanks for the asset. My video is now FLYING! I wonder if you have the API documented somewhere. I can snoop around in the C# files but I'd rather look at some documentation.
    I fully concur with izm1252 about manipulating playback speed. Yes please! If it helps you crank out an update sooner to focus on just the video playback speed and not the audio playback speed too then please do that, at the moment my primary application would be just video.
     
  6. pojoih

    pojoih

    Joined:
    Mar 30, 2013
    Posts:
    188
    does it do any benefit for mobile VR devices like GearVR? Playing 8k on there would be awesome :)
     
  7. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @adriaanwormgoor, @izm1252

    I have a workable version with playback speed controls here. Please directly contact me via lev.panov@gmail.com, I will send you the needed asset version, and publish on asset store if it works for you!

    And I will definitely think of how it's possible to make some nice-looking docs out of the C# files! Thanks for the idea!
     
  8. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    I think it will work in VR environment, since you can render the video onto any material you want! There is an example scene which shows how to do it in the package. You can check it without buying the asset, using the demo version here: https://github.com/DemolitionStudios/DemolitionMediaHap. Important: for 8k you will need a really fast SSD!

    Please let me know how it goes! I'm highly interested whether it works, unfortunately have no VR equipment nearby
     
  9. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    I've just submitted a new version 0.9.3 with playback speed controls and some other changes.
    Let's wait couple of days, so it gets approved by the Asset Store admin!
     
    Last edited: Apr 8, 2017
  10. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    Updated demo version available for download here!
     
  11. Pode

    Pode

    Joined:
    Nov 13, 2013
    Posts:
    125
    @leavittx : is it possible to use on mobile (high end Samsung phones, for example) ?
    Real 4K or 8k video would be of a great help when doing 360 mono/stereo videos !
     
  12. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @Pode Hi there! No plans to support mobile targets right now! Our main aim is to play videos of killer resolutions, but only on desktop platforms. However things may change in the future, since there is a technical possibility, but as far as I know plenty of other nice-working video assets exist for mobiles
     
  13. Pode

    Pode

    Joined:
    Nov 13, 2013
    Posts:
    125
    @leavittx yes they are other assets, but nothing for HAP.
     
  14. Carpet_Head

    Carpet_Head

    Joined:
    Nov 27, 2014
    Posts:
    151
    How might I got about recording or capturing video with transparency? in particular lets say from a unity camera or rendertexture
     
  15. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    Muckel likes this.
  16. LostPanda

    LostPanda

    Joined:
    Apr 5, 2013
    Posts:
    157
    @leavittx If the purchase of resources can provide all the source code? I see there are a lot of dll files.
     
  17. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    Hey! I've replied via e-mail!
     
  18. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    New version 0.9.3 is on the asset store!

    Changelog:
    — Playback speed can be controlled
    — Framedrop can be enabled or disabled using the C# API
    — Overall stability improvements
    — Fix resource leaks (GPU and CPU)
    — Updated the IMGUI example scene: set active segment, playback speed, etc
    — New Hap Alpha example video

    Get it here
     
    LostPanda likes this.
  19. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
  20. LostPanda

    LostPanda

    Joined:
    Apr 5, 2013
    Posts:
    157
    Thanks very much!I have already responded to your mail.
     
  21. massivebacon

    massivebacon

    Joined:
    Apr 24, 2014
    Posts:
    19
    I'd actually like to know this well. We're working on a project where we need to play back as many high-res (2K+) videos as possible, and ran into a wall with other HAP solutions because we couldn't see what they were doing outside of DLLs.
     
  22. mwaller_RM

    mwaller_RM

    Joined:
    Jul 1, 2014
    Posts:
    7
    Hi I would need to use your plugin to stream RTMP, RTSP and potentially ASX, I tried RTMP in the demo and got an error for FindCodedError, where do i need to install the codecs for the the plugin to find them?
     
  23. sunilnayak

    sunilnayak

    Joined:
    Apr 14, 2017
    Posts:
    3
    @leavittx
    Hi! Thanks for the plug-in, we bought the full version and our 4k 360 videos play on the Oculus rift now at 90+ FPS.
    We did have one issue, however, the texture quality appears to be downgraded.
    Is there any easy fix to this? Are we facing a known problem?
    We're using the render to material script and our video appears to be pixelated. We've checked the texture after pausing the scene and the size appears to be correct - 3840 x 2160.
    We've also tried using a higher resolution sphere to render the video on, but that's not helped either. It'd be amazing if you could help us out here.
     
  24. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    Are you wondering about the source code? If yes, the native dll's C++ source code isn't included in the asset store package! And I don't think you really need it for anything! Shouldn't be a problem to run a lot 2k+ videos! Some people managed to run 120 HD videos at once. And if you have questions about the plugin architecture, please ask.
     
  25. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    This version of my plugin doesn't have anything to do with network streams by any means! Although it has the possibility to open them internally (for other video codecs like h264), for Hap codec it doesn't really makes sense due to high data throughput! Why would you want to do that with Hap?
     
  26. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    Hey! Thanks for your message! Now I can finally confirm that the plugin works in VR.
    Regarding your problem: the video could appear slightly downgraded due to DXT1/5 compression artifacts, but not that much really (gradient areas could look not as smooth as in the source video). Pixelization shouldn't be caused by the Hap codec!

    Could you please check on the test scenes included into the asset (render a 1/4 part of the texture if you don't have a 4K display) if the pixelization takes place there as well? If that doesn't helps, you can also play the video in an external player like QtAV. Let me know!
     
  27. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    Last edited: Apr 19, 2017
  28. sunilnayak

    sunilnayak

    Joined:
    Apr 14, 2017
    Posts:
    3
    @leavittx I can confirm that the pixelation is not happening inside QtAV for our footage.
    We also have a bunch of Media scripts running at the same time, but I made a test scene with just two media scripts running (one to render the video and one for an alpha mask because we need transparency and also shadows to work) and the footage looks pixelated there as well. Any advice?
     
  29. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @sunilnayak first of all, please check whether the pixelization takes place in the 02_Hap_ControlsIMGUI example scene included into the Asset Store package! I can do it for you if you send me the videos.

    I also wonder whether you're using Hap or Hap Q? Try the latter if you haven't yet.

    And how do you encode your videos? I suggest to do it using FFmpeg. AE-like programs could produce broken videos. See the documentation on that included into asset for the instructions!

    You can also write me directly via e-mail lev.panov@gmail.com
     
    Last edited: Apr 21, 2017
  30. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    Just noticed that it's 30% off for the full version of my plugin on the Asset Store these days! Be fast before 30.04!
     
  31. sunilnayak

    sunilnayak

    Joined:
    Apr 14, 2017
    Posts:
    3
    @leavittx Sorry about the late reply, we were testing out a lot of different things and found out that it was the source footage itself that was causing the issue. Unfortunately, 4K in 360 (inside a headset) doesn't look as good as we thought it would. But your plugin works perfectly! Thanks again!

    Details: We were using HapQ and ffmpeg to encode the footage to HapQ but we needed AE in the pipeline as we had some keying to do. However, we tested the footage we obtained directly from the camera (after converting it to HapQ using ffmpeg) to see if AE was breaking it and the footage looked just as pixelated. We used KodakSP360 cameras to record said footage.

    Thanks for your prompt replies!
     
  32. jeroenvdv

    jeroenvdv

    Joined:
    Oct 25, 2010
    Posts:
    49
    Hi! Testing this asset right now, looks good. It would be great to also have audio following playback speed... Would that be possible?
     
  33. cerrec

    cerrec

    Joined:
    Jan 19, 2017
    Posts:
    26
    Hi, I’m using the trial version and it’s great, but I have a somewhat unusual problem – playback is too fast! Here’s what I mean…

    I’m using AV Movie Capture to record the game output in non-realtime. Frame rates can drop down to around 15 fps during recording. Now the HAP video clip I’m using in the scene has a 60 fps rate and it plays back at that speed even when the rest of the scene is down to 15 fps. As a result, AV Movie Capture records only 1 out of every 4 frames of the HAP video clip. So, upon playback, the HAP clip is sped up by 4 times.

    My request is, can you make playback speed of the HAP movie actually match the playback speed of the scene, when the scene is going slower than 60 fps ? Or another way of saying it is, can there be a 1 to 1 correspondence between the frames of the video clip and the frames of the scene?

    Do this and you’ve got another customer!
     
  34. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @jeroenvdv I will try to do it in the next version of the plugin! Would be a nice feature indeed! Thank you
     
  35. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @cerrec I think it can be easily achieved by manually calling Media.StepForward() method every time you need a new video frame to be displayed! Not sure how smooth the audio playback will be in this case, but probably you don't need it in 15 fps case anyway, do you? Let me know if it helps!
     
  36. cerrec

    cerrec

    Joined:
    Jan 19, 2017
    Posts:
    26
    I like the phrase 'easily achieved' but I have absolutely no idea what Media.StepForward() is. Could you please elaborate?
     
  37. cerrec

    cerrec

    Joined:
    Jan 19, 2017
    Posts:
    26
    Here's another problem - The Color Space of my project is set to Linear. But the Hap video clip is washed out on playback. If I set the Color Space to Gamma, the video looks correct, but the rest of the scene is too dark. Since I want to continue working in Linear Color Space, is there any way for your plug-in to set the video clip to Gamma so it will blend in correctly with the scene?
     
  38. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @cerrec are you familiar with C#? For what you want you need to write some C# code!
    I would do something like that in your situation:
    1. Attach a new script to your Movie object (it should already has Media script attached!) with a name similar to MovieFrameUpdate
    2. In the Update() method of your script, you need to keep track of whether the current frame has changed. I assume you can use Time.frameCount for that.
    3. When the new video frame is needed, get the Media component of the current object and execute StepForward() on it.
    The code is roughly the following:
    Code (CSharp):
    1. using UnityEngine;
    2. using DemolitionStudios.DemolitionMedia;
    3.  
    4. RequireComponent(typeof (Media))]
    5. public class MovieFrameUpdate : MonoBehaviour
    6. {
    7.     private int lastFrame = -1;
    8.  
    9.     void Update()
    10.     {
    11.         int currentFrame = Time.frameCount;
    12.         if (currentFrame == lastFrame)
    13.             return;
    14.  
    15.         lastFrame = currentFrame;
    16.  
    17.         var media = GetComponent<Media>();
    18.         media.StepForward();
    19.     }
    20. }
    21.  
    Don't forget to set the Play On Open flag to false!

    Regarding your second question, I need more time to get an idea of how to do that.
     
    Last edited: Jun 13, 2017
  39. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @cerrec okay, I've just tried setting Color Space to Linear in the Player Settings and building one of the example scenes. And it works correctly, I think! I can see both the video and the reset of the scene (i.e. the IMGUI elements).
    Here is what exactly I get. Color space set to Gamma:

    Color space set to Linear:


    The Unity tutorial on the Gamma/Linear workflows approves that both linear and gamma textures can be used in linear modes, so I see no problem here. Maybe you can give me some more details of the problem you're facing?
     
    Last edited: Jun 13, 2017
  40. cerrec

    cerrec

    Joined:
    Jan 19, 2017
    Posts:
    26
    Thanks for the responses.

    I tried your script – it entered in with no errors (after I added ‘ [ ’ in front of RequiredComponent).
    Unfortunately it had no effect. I know a little about C#, but not enough to do any real programming.
    BTW, I’m using the excellent FFmpegOut to record the play, if you want to try it yourself.

    As for the gamma issue, here’s a more detailed explanation of my setup. I have an ordinary 3D scene with a ground, 3D objects, skybox, lights, etc. The project setting for Player is set to Linear. The scene is lit so everything looks good with this setting. The Hap video clip is mapped onto a plane which is an object in the scene. The shader for the plane is set to Unlit. With this setup, the video appears washed out and over exposed. However, if I set the Player to Gamma, the video is rendered correctly, but the rest of the scene is now under exposed and very contrasty. So the Linear and Gamma settings appear to affect everything in the scene globally. What would be great is if you could isolate the video from that and give the user the ability to set Linear and Gamma for it independently.

    Hope that helps.
     
  41. marcusestes

    marcusestes

    Joined:
    Feb 2, 2017
    Posts:
    17
    Does this support stereoscopic playback for VR HMDs?
     
  42. ibompuis

    ibompuis

    Joined:
    Sep 13, 2012
    Posts:
    71
    Hi,

    Purshased Demolition Media HAP for OSX and PC, tried all scene example on unity 5.6.2p1 on MAC OSX 10.12.5, HAP codec installed.
    In play editor mode all works fine, video are playing, but if I build for OSX, video don't play, no error message !

    Editor:
    https://screencast.com/t/1KlFiTiIr2

    build:
    https://screencast.com/t/oh2O33HrL4

    Thanks for help
     
  43. ibompuis

    ibompuis

    Joined:
    Sep 13, 2012
    Posts:
    71
    Hi, any news ? sended mail and post on forum, no response... Tested project with 5.4.2p3 too, same problem

     
  44. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @marcusestes Hey! It's totally possible to play top-bottom or left-right stereoscopic videos with my asset! But with some hand work at the moment. You need to keep track of which eye is currently being rendered by the VR system and correspondingly change the texture coordinates of the video material. I think I can include the needed scripts to the package, but I'd like someone to help me with testing them first (since I don't own any VR gear). If you (or anybody else) are interested with giving it a try, please write me an e-mail to lev.panov@gmail.com. Thanks!
     
  45. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @stephane-cottin Hey! I think I've found what the problem is. Please check your mail!
     
  46. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @cerrec Hey! At that point, this is slightly out of the forum thread scope. If you could send me a simple version of a Unity project which shows your problem, then I may be able to help you. You can find my e-mail two posts above! Cheers
     
  47. ibompuis

    ibompuis

    Joined:
    Sep 13, 2012
    Posts:
    71
  48. vishal_changrani

    vishal_changrani

    Joined:
    Jul 11, 2017
    Posts:
    1
    Hi,
    I keep getting this error when I try to run a 4K HEVC clip.

    [DemolitionMedia] OpenFailed: FindCodecError
    UnityEngine.Debug:LogError(Object)
    DemolitionStudios.DemolitionMedia.Media:populateErrors() (at Assets/DemolitionMedia/Scripts/Components/Media.cs:434)
    DemolitionStudios.DemolitionMedia.Media:Update() (at Assets/DemolitionMedia/Scripts/Components/Media_Unity.cs:134)​

    Please can you tell me what I might be doing wrong? Should I be downloading the Codec seperately?
    This is using the demo version.
    thanks,
    V
     
  49. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    111
    @vishal_changrani Hey! No codec installation is needed! But you need to re-encode your source HEVC videos to the Hap format. It allows to play videos with very low CPU usage. Please see the .pdf file included in the unity package for details on how to do it. I think ffmpeg command-line tool will work nicely for you!
     
  50. breezer

    breezer

    Joined:
    Aug 16, 2017
    Posts:
    2
    @leavittx I've recently purchased and I'm running into an issue while using your plugin, after a while of usage the unity player sporadically crashes. The application has two media components and is swapping a RenderToUGUI sourceMedia as well as swapping the Media sources. I've tried turning off Enable Audio, Use Native Audio Plugin, Open On Start, Play On Open and all combinations of which and still hasn't helped and in some instances made it worse.

    error.log
    Unity Player [version: Unity 5.5.2f1_3829d7f588f3]
    nvwgf2umx.dll caused an Access Violation (0xc0000005)
    in module nvwgf2umx.dll at 0033:7eda0e8c.
    Error occurred at 2017-08-16_151814.

    Read from location 000000cc caused an access violation.

    It seems likes it's crashing the gpu though i'm not sure, I've updated to the latest GPU drivers.

    I've made sure that I've downloaded the latest Asset from the Asset Store, Calling Open and then Play doesn't seem to always fire if Open On Start, Play On Open are not both set to true.

    Any advice or direction would be greatly appreciated.

    EDIT: Only crashing the final outputted build of the project and the crash doesn't get caught by the debugger.
     
    Last edited: Aug 16, 2017
unityunity