[Released] Demolition Media Hap — multi-platform 8k@60fps GPU video playback

Hi -- I just bought the asset.

I'm finding the playback performance much better than UMP Pro, AVProVideo, but it has some issues.

I'm using the 2019.1 release of Unity; the biggest issue (after compilation errors due to all the defines being out of date) is that calling various query methods causes huge performance issues. I'm playing back an 8k x 8k video for example, and it looks really smooth. As soon as I call "VideoDecodeFramerate," the performance drops.

First of all, it seems that calling this method (or even the video framerate call) pauses the video player. If I unpause it, then the performance is terrible -- around 3 fps.

Do you have plans to update this soon? Have you seen issues like this?

Thanks

 

this is no update assets, dev say update but fews month nerver see any update! i sugget using avpro!

 

Looks like I need to... I really like the frame-accurate nature of this API, but it's partially broken.

Won't run on mac with metal, in fact crashes the metal editor.

Sound quality is sometimes noisy, sometimes audio plays maybe at 2x speed (on 60 fps files), 8k x 8k will load any particular frame but won't play the video, even if frame drops enabled.

 

I suggest you use the 2017lts version if possible.

 

That's how the demo version "watermark" effect works.
The full Asset Store version doesn't have any stripes (make sure you properly replace the DLLs)

 

Hi
Can this asset be used to play back video within Unity's Timeline?

Thanks

 

@klvo Not at the moment, but that's on our roadmap. You can just tell the plugin to Play/Pause the playback at the moment (Seek is also available of course). Btw do you need the Timeline support for both audio/video streams or just for the video?

 

Hello.



Our company make 360 stereoscopic movie.

We want to play back 360 videos using Delmolition within Unity.

Our video resolustion is 6144x3160 and 60.0 frame/s.

Does Delmolition support our video?

Thank you for reading.

I look forward to your reply

 

@EliotVR Yeah, we support that resolution/framerate for sure. But the SSD read speed should be quite high for that kind of video.

Today I've made a Hap Data Rate Calculator, which estimates the data rate to 308Mb/s or more for Hap (standart quality, may have some artifacts), or 618Mb/s or more for Hap Q (better quality) for your kind of video.

Overall fast machine (but might be not extermely fast) in terms of CPU/GPU is also required for smooth playback.

Please find 360 stereoscopic sample scene for the plugin here (full or demo plugin version should be already installed)

 

The data rate calculator is avaliable for everybody: http://demolition-studios.net/hap-calculator/

 

Thanks -- I'm struggling to play back 8k x 8k @ 60 -- I can only achieve about 15 fps. DirectShow / LAVFilters are able to do only 20 or so fps. TouchDesigner can play back at 60 fps.
We have 4 nvme drives in raid 0, so the hd speed is fine.

It looks like the GPU is a bottleneck, not file i/o or CPU. Any ideas?

 

That sounds like a challenging task.

I recommend the following encoding parameters
1. Use hap, not hap q, as it is faster to decode
2. Make sure you have at least 8 chunks
3. Disable 2nd stage compression
Check out the documentation for the needed ffmpeg arguments.

I’m currently developing a new plugin version where the performance might be better. If the above tips don’t help, maybe we should work closer together to make some tests

I would also use a tool like CrystalDiskMark to measure the read speed just to be sure (also I’m wondering which number is achievable with raid)

 

I am looking to display different 4K clips synchronized with other GUI and interactive elements, so everything is choreographed.

The clips will have audio and video streams.
Are there any issues with HAP and audio?

Thanks

 

Well, playing audio and video simultaneously is no problem.
You also able to seek from time to time to change position in the audio/video streams.

But if you want frame-accurate video playback (and I guess that’s what you need in the Timeline case), having audio also synced to it (and even most important smooth) is quite tricky, and I haven’t seen any solutions which allow to do that yet.


I would assume that one possible solution would be to have audio playback being done separately from the video - you just play audio using built-in Unity player for example, and correct the video stream position from time to time - when the difference in time is more than some threshold value (e.g. 1 or 2 frames)

 

Thanks for your reply!
I am looking for frame accurate playback if possible, so I will be testing the general implementation first and then figure out how much flexibility there will be with the video player.

 

Which general implementation are you talking about?

 

I mean the implementation approach for the whole project, in which the video playback is only a part of.
Thanks for your help so far!

 

Dear Demolition Studios,

How do you do?
I am a user about your Demolition Media Hap for Unity. Now, I would like to play videos as followings:
- 8K(7680 x 4320)
- 60 fps
- 5.1ch sound
- Hap Q (Rendered by premiere[HAP Video], premiere[after codecs] and ffmpeg)
- filesize: about 1.4GB per 1 second
- test with ControlsIMGUI.unity
and my PC spec is as follows:
- Thinkpad P1 Gen1
- Windows 10 pro for Workstations
- Intel Xeon E-2176M CPU @ 2.70GHz
- 64.0GB RAM
- 64 bit
- data disk: internal NVMe SSD (read 3GB/s available)
- Unity ver: 2018.4.9f1.

However I CAN NOT play the videos fluently. The fps is about 30 (attached image).

How can I play those videos?
I would like to know the way.

 

Hi. Please make sure you use Hap chunks while encoding. I recommend 24-32 chunks. Your case is pretty extreme but it might be possible using your setup.

 

Thank you so much for your reply.
Thanks for your suggestion about chunk setting, I tested videos as followings:
- 32, 64 chunks
- encoded by ffmpeg with command like:
ffmpeg -r 60 -i [path]\[input-filename]_%05d.tif -vcodec hap -format hap_q -r 60 -chunks 64 [path]\[output-filename].mov
and,
I changed the test PC to a higher spec one as followings:
- HP Z8G4 (Desktop PC)
- Xeon Gold 6134 3.2GHz * 2
- 16 cores
- 32 threads
- 128GB RAM
- 64 bit
- Windows 10 Pro
- NVIDIA QUADRO RTX 6000
- data disk : RAM Disk by Softperfect (over 7GB/s read)
- Unity ver.: 2018.4.23f1

However, a decoding fps wasn't improved so it's about 35 fps.

The result is an attached file when I tried a 64 chunks hap-Q video.
Especially, a strange point is the number of using CPU threads and looks like to be used only 16 threads.
This result is almost the same as test for a 32 chunks video.

Moreover, I tried some Hap encoding settings until now.
The followings are the video spec:
- Premiere pro + Hap Video:
- Hap-Q, chunk 6
- Hap-Q, chunk 12
- Hap-Q, chunk 24
- Premiere pro + After codecs:
- Hap-Q, chunk 8
- Hap-Q, chunk 8, disable snappy compression
- Hap-Q, chunk 16
- Hap-Q, chunk 16, disable snappy compression
- ffmpeg + After codecs:
- Hap-Q, compression none
- Hap-Q, chunk 6
- Hap-Q, chunk 12
- Hap-Q, chunk 24
- Hap-Q, chunk 32 <- new
- Hap-Q, chunk 64 <- new
I am happy if the above video spec information is useful for you to point out some problem roots.

How about my situation and How can I test to solve this problem?

Best regards,

 

@ksumiya
I see that you also tested the avpro plugin. The current version of this plugin is supposed to be working on a new version internally by the author, but no word on that. I tried to reach out on zgoogle, but no reply. It would be really good if there is a new version available. The author said there will be a big improvement in performance. If there is an alpha version, it should be tested by more people. Hopefully the author will see the news. Your desktop pc is already very highly configured, and if you can't play hap smoothly with such a configuration, chances are that plugins and The problem with unity. If you don't care much about the sound, I suggest you try this: https://github.com/ keijiro/KlakHap

@leavittx I really hope the new version will be released soon, waiting for you to release the new version on the store.

 

@LostPanda Working on a new version indeed, it might be better there.

@ksumiya May I recommend you try using a commercial Hap encoder Jokyo. It's not free, but should give really superior results compared to others. You may replace Hap Q with just Hap with it and have almost the same quality level:

8k@60fps Hap should be fast enough.

Meanwhile I will try to investigate why the CPU utilization is low in your case.
About KlakHap, I had really weird playback especially with high-resolution videos. But you might try it also of course if you want.

 

@leavittx Thanks for the reply. In fact, I've been waiting almost a year. I wish you could invest more time in this, I know you are a very busy person. What I can do for now is to use avpro instead of hap playback for now. As soon as you release the new version, I will buy it.I think it would be better if there was a ROADMAP and a release date. Thanks again.

 

Yeah, I encourage everybody to try the new Jokyo Hap Encoder.
It is faster compared to the community exporter and able to preserve steep gradients within the standard Hap / Hap Q containers.

Encode speed

Quality (SSIM)


You can see side by side quality comparison here.

There is only After Effects version now, but they will release soon a version for Media Encoder/Premiere Pro (they are in the last step of beta test)

Comparison of Jokyo with another paid AE exporter called AfterCodecs:

As you can see it's ~2x faster and also better quality.

 

@ksumiya ok, I've just checked that the SSIM is almost the same for community / Jokyo Hap in the slow preset, but I expect there might be some quality improvement anyway. I will ask devs if there actually some improvement expected.

 

@ksumiya Just checked and there seem to be a limitation for the number of decode threads in the current version indeed. I might be able to roll out the new version without such limitation soon in that case. Do you have any deadline for your project?

 

@ksumiya I saw your video file spec on AVpro github, and I totally not recommend using sowt audio codec (it's a proprietary Apple codec). We've seen some problems with it in the past. Also the bitrate seems to be a bit too high, which may also be a problem.

What I recommend doing (given that your primary video source is AfterEffects) is reencode the audio stream using ffmpeg later on - without touching the video stream in order to maintain source quality.

The command is as follows:

ffmpeg -i hap_in.mov -vcodec copy -acodec aac -b:a 960k hap_out.mov

I've chosen bitrate as 320kbps (2 channels) * 3, but it's up to you which value you set (I don't recommend using more than that though)

The 5.1 audio support is also not in the v0.9.6 (it will convert it to 2 channels), but it's 90% ready for next one.

p.s. I couldn't find a screenshot with the video footage you've posted. I wanted to send it to Jokyo Hap Encoder author so he could give some content-specific tips (or maybe you can upload a short h264/h265 video)

 

@ksumiya as a last resort until I finish a new version (which may or may not help), here is a quote from documentation for you:

The data-rate will grow but since you've mentioned 7Gb/s disk, I'm posting that as well (though note that 10 seconds long video is likely to be fully cached to the RAM with 128Gb of it)

Also note that reencoding Hap Q video to Hap Q with ffmpeg can degrade the quality, and I recommend reencoding high-bitrate H264 -> Hap Q instead

 

@leavittx Can you tell us about release date of the new version? Thank you.

 

At the moment I can't unfortunately. But it's in my priority list right now. Just need to make sure everything works before the release. I will let you know

 

@leavittx One more question, is there a new solution for multiple computer hosts hap playback sync?thanks

 

@ksumiya Here are the recommendations based on discussion of Jokyo Hap Encoder.

The information below might be interesting for other people as well

Based on that I'd try to use slightly smaller resolution and/or framerate with Hap Q / Jokyo Hap Q until the stable playback is reached and see if there is any noticable difference compared to original 8k@60fps. Or try Hap with moving anti-banding, where 8k@60fps is possible.

Might still be possible to play full 8k@60fps HAP Q (with or without 5.1 audio) in next version of Demolition Media on your machine (without the threads limitation of v0.9.6)

 

Yes working on that as well. The results are quite promising already

 

Does it support ios?

 

@leavittx
I am so sorry for my late reply and thank you so much for such a lot of comments.
I am trying your suggestion step by step.

First of all, I purchased and tried the Jokyo HAP Encoder but playing fps is still about 30 on Thinkpad P1 Gen1 (laptop).
Moreover, the Jokyo HAP Encoder look like not to be available for a 5.1 ch sound. I need the 5.1 ch sound for my project...

 

Hello,

It's plan to add 5.1 support inside Mov (in Jokyo HAP Me encoder plugin).

There is several way to store 5.1 audio inside a Quicktime file like :
- One 5.1 track in Left, Right, Center, Lfe, Ls, Rs order
- One 5.1 track in Left, Right, Ls, Rs, Center, Lfe order
- 6 mono tracks (for each order)

What is your need regarding 5.1 ?
What is your deadline (to try to send you a beta version with 5.1 output) ?

Best,

Martin

 

@JokyoHAPEncoder Any plans to add a standalone unity plugin? unity is widely used! A lot of times it doesn't use ae or cc.It would be nice if you could call the command like ffmpeg.thanks

 

Hi there! Thanks for your question. Hap can't be used on iOS due to the hardware limitations and also its datarate is too high for mobile devices

 

Hello,

Do you mean a plugin to render from unity to hap, or a standalone app to encode HAP file without Adobe software ?

Martin

 

thanks. encode HAP file without Adobe software .
It would be better if you could use unity directly.

 

this new version be released in the next few days? Thanks

 

@ksumiya May I ask about how are you going to play 5.1 audio from Unity? Are you having 5.1 sound system or just 2 speakers? It might be that Unity only outputs 2 channels, at least I'm unable to get it working for 6 channels right now (other people also had such problem). I'm almost sure my sound card doesn't support 5.1 channel output in WASAPI (default) mode, only in ASIO mode (which is not working in Unity out of the box!). There seems to be a better 3rdparty plugin with ASIO support, but it would require more time to integrate: https://assetstore.unity.com/packages/audio/sound-fx/low-latency-multichannel-audio-147091

What about your sound card?
Maybe you should try playing 5.1 wav file with Unity first (using Audio Source's Audio Clip) to confirm it's working at all.

 

That's another article that I've just found: https://thomasmountainborn.com/2016/06/10/sending-audio-directly-to-a-speaker-in-unity/

Some info here as well: https://forum.unity.com/threads/5-1...ixed-to-stereo-on-mac-os.671086/#post-4542643

 

@leavittx
Unity itself plays 5.1 videos or just 5.1 wav files with no problem; I tested the Asus 7.1 built-in sound card, German Terratec X fire 8.0 HD usb external sound card. If your plugin can add asio support that would be great. Plenty of devices support ASIO, and it's easy to control each channel. low-latency the multichannel Audio plug-in is based on NAudio and I hope this information is useful to you.

 

I use HDSPe AES (a PCI sound card made by RME) as my current project's sound system.
Therefore, 5.1 ch sound is necessary...

 

@LostPanda Cool, thanks for the info

I've just got it working: thanks to https://spatialsoundcard.com/ - a virtual 5.1 sound card.
Now I can finally make some tests with multichannel sound!

@ksumiya I think it will be working in new plugin version

 

Seems to be working: https://www.dropbox.com/sh/9bhytjt1x1fgmvv/AADwCqmeCfPcMs0e7nHauao-a?dl=0

 

@leavittx
thanks for work.
I want to know the new version has been submitted to assetstore？thanks again

 

@leavittx

Understand that you are very busy, but a week later, there is no news for the audio multi-channel version release?

 

There are some news actually - today I've submitted a new version to the Asset Store (a review is pending).
Will post more info on it in a bit. Thanks for being patient.