Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

[Release] FM ExhibitionToolPack - [New in v1.08] Cross Platform WebSocket WebGL Streaming Demo

Discussion in 'Assets and Asset Store' started by thelghome, Apr 30, 2019.

  1. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    Artboard 1-100.jpg

    battery drain? overheat? live/game view streaming? gigapixel panorama? custom tcp/udp network action solution?

    Webpage | Asset Store
    FM Exhibition Tool Pack
    provides you a simple solution for popular troubles in public events or exhibitions. All source codes are optimised for exhibition purpose, and used in many exhibition over past few years.

    Some new features are in Beta version. If you found any bug, please report to us via email and we would like to provide support.
    Support::thelghome@gmail.com

    supported platform:
    iOS, Android, Mac, PC

    [v1.08] submitted, will be available soon
    [New!] WebSocket Demo (cross platform)
    -added WebGL support, tested WebGL(mobile)
    -tested on native build with iOS/Android/Mac/PC/Oculus Go


    In-Game Audio+Video Streaming Demo
    - Capture & Streaming In-Game Audio and Video
    - Tested: Streaming Game View between Mac/PC/iOS/Android/Oculus Go

    Network Action (Beta)
    - Custom RPC networking solution with TCP and UDP
    - Sync / Remote trigger for multiple clients
    - Simple Example Scene provided

    Battery Saving Mode
    provide a sleep mode for your app, when there is no one playing. it's extremely useful with AR app when there is no power supply in your exhibition gallery.

    Live Streaming
    Simple local streaming solution, used for live streaming game view, live webcam, live reflection map for AR

    Panorama 360 Viewer
    Gyro Demo & Editor Mode Simulation

    Gigapixel Example
    Custom Shader for high resolution panorama
    Mobile Devices: 4 x 4 x 4096 (W) x 2 x 4 x 4096 (H)
    on 8k supported platforms: 4 x 4 x 8192 (W) x 2 x 4 x 8192 (H)

    *All Source Code in C#
    *All Core parts are used in many exhibitions in past few years
    *Will Keep adding new features
    Tested devices in our public projects:
    iPhone 5
    iPad Air 2
    iPad pro(2017)
    iPad pro(2018)
    iPhone 6s
    iPhone XS
    Macbook(2017)
    Macbook Pro(2018)
    Windows 10
    Oculus Go
    Samsung S9+

    Artboard 2-100.jpg Artboard 3-100.jpg Artboard 4-100.jpg Artboard 5-100.jpg
     
    Last edited: Aug 9, 2019
  2. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    We just submitted an update to Asset Store and the new feature will be available in v1.02 very soon.
    [NEW!] Network Action (Beta)
    - Custom RPC networking solution with TCP and UDP
    - Sync / Remote trigger for multiple clients
    - Simple Example Scene provided

    Artboard 6-100.jpg
     
  3. hrjunker

    hrjunker

    Joined:
    Nov 15, 2011
    Posts:
    24
    Hi, do you have any experience streaming from a mobile VR headset. We are looking for a solution that can do this without compromising the framerate on the headset, currently using a Oculus Quest.
     
  4. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    Thanks for reaching us!
    I assumed that you want to mirror the headset view to other screens(pc/tablet). We mainly do testing between iOS/Android/Mac/Win10, and even iphone5 can stream the webcam/game view without problem.

    for the performance, it depends on few things:
    1. hardware side: cpu / network card / wifi router
    2. image quality: resolution / compression(file size)

    We tried GearVR with Samsung S9+ and it works fine. Unfortunately, we don't have Oculus Quest in our lab now, but we will order it soon.
    For your concern, we will do some test with Oculus Go in coming few days.
     
  5. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    We did a game view streaming on Oculus GO with resolution of 1024x576(streaming fps: 20).
    It costs 5ms in headset performance. I believe Oculus Quest should have a better processor than Oculus Go, thus, the performance will be better too.

    The headset view is wirelessly streaming to iphone XS / Macbook Pro almost in real time without noticeable delay.
    the latency with my plugin is much much lower than "Oculus Go Wired mirroring 2~4 second delay" said on their official website.
    https://developer.oculus.com/blog/oculus-go-wired-mirroring-how-to/

    I will update a GameViewStreaming demo very soon, which are tested with Oculus Go.
     
  6. Tibor0991

    Tibor0991

    Joined:
    Dec 11, 2016
    Posts:
    18
    I have a similar question regarding VR streaming: in my case, I want to stream the content of a desktop Unity app to an instance of another Unity app running on a mobile VR headset (Go or Quest); will your plugin satisfy this use case? More specifically, does your plugin provide a way to render the stream on an arbitrary surface?
     
  7. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    We just submitted version1.03 update to Asset Store, hopefully it will be available soon.
    -Added Game View Streaming, Tested VR Headset: mirror Oculus Go Game View to Macbook Pro & iPhone XS
    -Optimised Connection(Server)
    -Available to Send Raw Byte[] in Network Action(custom RPC networking solution)

    Any questions from you guys could help us understood your needs! We would like to test and take action immediately if possible.
     
  8. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    It should work. for your use case, I need to test the performance on Oculus Go in office next week.
    what is your target streaming resolution & streaming fps, which will satisfy you?

    PS: Once your server & clients are on Mac/PC/iOS/Android, they should work. but the image quality and performance rely on your hardware. Usually, decoding streaming on clients side demands more resources.
     
  9. Tibor0991

    Tibor0991

    Joined:
    Dec 11, 2016
    Posts:
    18
    I am looking to stream content on an Oculus Go/Quest at 640x720, 60 fps at least; I have written a contrived setup that allows me to stream still frames over the net and I get around 20-ish frames per second, but it wastes a lot of bandwith.
     
  10. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    I personally don't believe that Oculus Go can achieve 60fps game view streaming, as it's a very cheap VR headset. Maybe Quest can do it, but I didn't own one at the moment.

    Would you mind to share your current bandwidth/data size for each frame?
     
  11. agokoglu

    agokoglu

    Joined:
    Sep 23, 2014
    Posts:
    3
    Hi, I wonder if the network solution can also be used for AR applications. Is the RTMP protocol supported during Live Stream? For example, can I use a mobile broadcast to transfer to Wowza or Dacast servers? Or what else do I have to do for this? I would be very happy if you could help me with this subject. Will you also prepare a video sample for the new network feature?
     
  12. Tibor0991

    Tibor0991

    Joined:
    Dec 11, 2016
    Posts:
    18
    I own them both, and as far as I can say there's no huge improvement, it's also true that I have to test my system on the Quest yet.


    Each frame is 160000 bytes long tops, packed as JPEGs with a 80% compression rate at 640x720, theoretically speaking a second should be around 9 MB (60 frames per sec) of data and a full 30 minutes session should be around 15 GB; of course this is a ridiculous amount of data sent that should (could?) be compressed further with a proper codec. The biggest bottleneck so far has been the Texture2D.Apply() function, which is obviously needed to switch from CPU space to GPU space.
     
  13. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    we have two separated demos for "live webcam/game view streaming" and "custom rpc networking(experiment feature)".
    However, those features are aiming for local network during our development.
    Technically speaking, they are all developed based on UDP+TCP. As all our codes are writing in C# and can be modified freely, they might be a good starting point for your subject.
    We would like to do some research about RTMP protocol and It's kind of interesting topic.

    PS: please feel free to discuss and hopefully we both can learn something new.
     
  14. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    I agreed that it needs a proper codec for compression. In general, encoding and decoding heavily will affect the performance, which it's another challenge for all-in-one headsets.

    I suggested that you should target 30fps for oculus quest, with balanced image quality.
    In order to maximise your device's hardware limit, your streaming fps should be same as your game fps. It means, your streaming fps should not be higher than your game fps.

    In your case, I can imagine your biggest bottleneck should be VR headset, as you can still upgrade your computer with insane cpu for "Texture2D.Apply()" if budget allows.
     
  15. Tibor0991

    Tibor0991

    Joined:
    Dec 11, 2016
    Posts:
    18
    (I've bought your plugin in the meanwhile, if it doesn' fit this project I have still another use planned.)

    Yes, my biggest bottleneck is the VR headset, because Texture2D.Apply() takes too much time to update the screen; it should be possible to stream a hi-def video though, since the guys from Cubicle Ninjas (Guided Meditation VR) are able to play a 360 video in 5k/Strips format in Unity without any stuttering on Oculus Go (okay, the source is different since they stream from persistentDataPath, but still the texture has to be updated fast enough somehow).
    I will take a look at your code in the meanwhile, maybe there's something I'm missing.
    Thank you for your replies!
     
  16. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    loading video from persistentDataPath is totally another story, as it doesn't need to render image and transfer any data via wifi in real time.

    share a little bit of my experience:
    I had a project using AVPro Video with Oculus Go for normal 5k video playback in 60~72fps in last year. if you do not need to render texture in real time wirelessly, this might be your option.

    I am just curious: Does Oculus Quest have sleep mode after 5mins? I gave up Oculus Go as its annoying sleep mode.

    Recently, there is a project for an immersive 360 cylindrical projection system, which requires a video player solution to sync 6~11 computers. I was using UNet for networking solution but it will be depreciated soon. Thus, I started creating a custom RPC networking method to upgrade the player. I am not sure if someone will be interested in a multi-video sync player solution demo?
     
  17. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    and please be reminded that the latest version 1.03 is just submitted in few hours ago, it will take few days to get approval on Asset Store. In case you are urgent, you could send me an email with invoice number.
     
  18. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    [Good News!]
    As requested by some customers, we planned to add Audio Streaming demo in future updates, which allows you to capture in-Game Audio, and stream it between any platforms (Mac/PC/iOS/Android).

    PS: now we are in debugging stage & trying to optimise it, hopefully it can be included in coming updates asap.
     
  19. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    [v1.04] updated:
    basic audio streaming demo is available on Asset Store

    [v1.05] submitted & will be available soon:
    optimised audio streaming demo for different Sample Rate & Channels Count between server & clients
     
  20. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    [v1.061] submitted
    -optimise for Windows 7
    -optimise for old cpu, which are less than 4 threads
    -testing async solution on devices, in order to reduce thread usage
    -streaming experiment on VR headsets
     
    Last edited: Jul 23, 2019
  21. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    [NEW: work in progress]
    Web Streaming demos
    -support: Web Browser, Mobile browser, Html, WebGL

    current status:
    tested on devices & doing documentation & organising files.
    we will release in coming updates when ready.
     
    Last edited: Jul 28, 2019
  22. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    [v1.07] New feature: Streaming Game Camera to web browser
    Supported: iOS/Android/Mac/PC/Oculus Go
     
    Last edited: Jul 29, 2019
  23. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    32
    [v1.08] submitted today, will be available soon
    [New!] Native Build + WebGL support for WebSocket Streaming Demo (cross platform)