Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[Release] FMETP STREAM: All-in-One GameView + Audio Stream + Remote Desktop Control(UDP/TCP/Web)

Discussion in 'Assets and Asset Store' started by thelghome, Apr 30, 2019.

  1. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    I suggested that you may try migrating to FMWebSocketManager system, it will replace FMSocketIOManager for long term development. The setup is similar, but it doesn't require socket.io anymore and it's good for performance and bandwidth.

    Meanwhile, this new system allows you to send data to your target client via wsid.

    test.jpg
     
  2. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    It should be possible with FM WebSocket 2.0 system, via FMWebSocketManager. You can send data to your target client via wsid. Client may request for access, and server will response for request and stream via wsid.
    It's also included in FMETP STREAM 3.0.
     
    Last edited: Aug 24, 2022
    JudahMantell likes this.
  3. Tatyana_st

    Tatyana_st

    Joined:
    May 19, 2022
    Posts:
    1
    Hi! I try to stream from multiple devices (Clients) to a PC (Server). I set the Label IDs run time to pair encoders-decoders to separate the different streams. The connection starts and the stream works. But there is an exception:
    "ArgumentException: Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection."
    I can't get rid of it. Could you give me some suggestions please?
     
  4. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Do you have screenshots of the error?
    If possible, please write us an email for technical support.

    technical support: thelghome@gmail.com
     
  5. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    We just released the fix in V3.120, which should solve the issue.
    In our testing, we discovered that there is a hardware specific issue related to the CPU, which returns the wrong byte[] on mobile devices occasionally.
     
  6. DreamcatcherProduciton

    DreamcatcherProduciton

    Joined:
    Sep 29, 2016
    Posts:
    32
    Will there at some point be support for web GL audio encoding?
     
  7. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    It's in our To-Do List, we will investigate this feature but not in high priority at the moment.
     
  8. miracletechteam1

    miracletechteam1

    Joined:
    Sep 18, 2019
    Posts:
    24
    Can we livestream to youtube using RTMP?
     
  9. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    In our latest experiment, we stream it to ffmpeg first, then route to Youtube.
    Please refer to below commands and reference link:
    https://frozenmist.com/docs/apis/fmetp-stream/unity-to-ffmpeg/

    Code (CSharp):
    1. // + ffmpeg to Youtube Commands(Example):
    2. // + Playback source: Unity GameViewEncoder(Output Format: MJPEG)
    3. // + with FMNetworkManager(DataStream, Sender mode, port 3001)
    4. ffmpeg -re -f mjpeg -i "udp://127.0.0.1:3001" -f lavfi -i anullsrc -c:v libx264 -g 60 -c:a aac -ar 44100 -ac 2 -f flv "rtmp://a.rtmp.youtube.com/live2/<key>"
    PS: In theory, you may embed ffmpeg into Unity, and call the command in the backend.
    Screenshot 2022-08-31 at 3.48.05 PM.png
     
    Last edited: Aug 31, 2022
  10. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    476
    I had another quick question, that may be pretty simple:
    If I'm sending UDP messages from my Unity app using FM Stream, am I able to read these messages with another UDP enabled app and vice versa?

    Thanks!
     
  11. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Yes, The "FM Stream" mode in FMNetworkManager is compatible with other apps with UDP enabled.
    For example, data stream between gstreamer, ffmpeg...etc
     
  12. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [Notes] Streaming Quest 2 to Website is much easy now :)
    Our VR Stream template has been updated recently, with FM WebSocket 2.0 demos (local IP detection is also included).
    The performance is hugely improved, and has almost no performance impact in comparison to FM WebSocket 1.0(old version).
    https://github.com/frozenmistadventure/fmetp_tutorial_questvr_stream

    You may try streaming the Quest 2 view to web or the Internet with ease now :)
     
  13. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [FM WebSocket 2.010]
    -fixed event invoke bugs and add "FMWebSocketManager.instance.ConnectedClients" option, for the ease of sending data to specific client's wsid.
     
  14. VirgiliuPurple

    VirgiliuPurple

    Joined:
    Jan 6, 2021
    Posts:
    1
    Hello, is SocketIO 4 supported in FMETP 2 or 3 ? I tried changing the connection string EIO argument from 3 to 4 and it didn't seem to work.
     
  15. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    The new features in SocketIO4 isn't fully compatible, for example, EIO argument should stay as older version.
    Since the performance with socket.io isn't very good for streaming purpose, the "FMSocketIOManager" will be replaced by "FMWebSocketManager". Our new system is re-built based on native WebSocket, with the same function of Server-Clients methods, it's much light-weight and recommended for mobile devices.
     
    VirgiliuPurple likes this.
  16. EdisonTan_66

    EdisonTan_66

    Joined:
    Jul 24, 2022
    Posts:
    5
    Hello author! I recently encountered some problems while testing. Before, I could see the shared screen on the computer on the mobile webpage, but now the mobile phone can't even open the localhost:3000 page. Moreover, another computer under the same WiFi cannot open this webpage. Only on this computer with unity and node.js open can the webpage be opened to see the picture.These functions were all good about a month ago, at least I can see the picture on the mobile phone through the website
     
  17. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    You have to replace the "localhost" with your node.js IP address, otherwise, only local computer can access "localhost:3000". For example: you should type "192.168.1.123:3000" on other computer/mobile
     
  18. EdisonTan_66

    EdisonTan_66

    Joined:
    Jul 24, 2022
    Posts:
    5
    Wow! Thank you so much! It's that I haven't used this plugin for so long that I forgot these details.
     
  19. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    We've added a new video tutorial for FM WebSocket 2.0 setup, wish it may help others in the future.
     
  20. FyndNorway

    FyndNorway

    Joined:
    Sep 4, 2018
    Posts:
    18
    Hi @thelghome! We just updated from version 1 to 3, for streaming from Hololens 2. While that's in progress, I also need desktop streaming to work, through our own UI. Before I used this code:
    Code (CSharp):
    1. FMDesktop.Manager.instance.Reinitialize();
    2. int monitorCount = FMDesktop.Manager.monitors;
    And when I upgraded, I tried to do the same setup with the new structure, but there is no initialize method now. When I try to go straight to this:
    Code (CSharp):
    1. int monitorCount = FMDesktopManager.instance.MonitorCount;
    I get a null reference error. What do I need to to for this to work?
     
  21. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    There are some major changes in this part. Thanks for pointing out this issue, and we will release an official example in next update asap.

    Meanwhile, we could share the temporary example codes to you via email.
    Could you please write us an email for technical support?
    email: thelghome@gmail.com
     
  22. FyndNorway

    FyndNorway

    Joined:
    Sep 4, 2018
    Posts:
    18
    Absolutely, thanks a lot!
     
  23. blanx

    blanx

    Joined:
    Feb 26, 2016
    Posts:
    73
    Hi @thelghome , when using the desktop streaming, can the receiving client somehow send back mouse input to the sending server to execute the inputs there? So we have some real remote desktop connection?

    Thanks
     
  24. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    For PC users, Windows has its own APIs for simulating mouse keyboard input, which can be accessible in Unity by default.

    Meanwhile, we are working in progress for a complete remote desktop control template, which should be released soon in coming weeks by this year.
     
    Last edited: Oct 11, 2022
  25. blanx

    blanx

    Joined:
    Feb 26, 2016
    Posts:
    73
    @thelghome do you have a hint where to find that api? I have no clue what to look for.

    Would you provide beta access to the template?
     
  26. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    For technical requests, please reach us via email: thelghome@gmail.com

    Let’s connect and see what’s the best solution for your need.
     
  27. darkmax

    darkmax

    Joined:
    Feb 21, 2012
    Posts:
    83
    I'm not sure if this is the tool that I need I hope you can clarify me.
    I have an application running on Hololens 2, and I want to stream video of the hololens screen, to a webpage for other to see. But in your examlpes I see that you run unity3d editor, but in my case I want the app of hololens made in unity start like a video transmision and on a webpage see this transmision live. Is this possible with your plugin?
     
  28. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    YES, this is compatible. HoloLens view to Web is fully supported.
    There are many many use cases and our demo videos are just showing part of interesting features.
     
  29. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    For everyone, who are interested in remote desktop feature.
    We've made a successful demo in our internal test, this will be a private work-in-progress feature at the moment.

    This interesting feature is similar to Teamviewer, Anydesk, Meta Quest 2 Remote Desktop..etc, but you can achieve it and embed it into your own project in Unity3D.
    Meanwhile, there are still some work before final release, we plan on adding VR, Web, Mobile demos.

     
  30. RiccardoAxed

    RiccardoAxed

    Joined:
    Aug 29, 2017
    Posts:
    119
    Hi, I'm starting a project where I need to stream video/audio via Websocket/WebRTC. Namely, I should receive a live streaming 360 video with audio in my mobile app and send only live audio from device's microphone to another mobile app.

    Just to give you some further reference, to stream video we should use Flashphoner services (https://flashphoner.com/) and the flow described in this doc page: https://docs.flashphoner.com/display/WCS52EN/In+an+Android+mobile+application+via+WebRTC

    So I would just ask if FMETP STREAM - or any other of your plugins - can be the right solution for me.
     
  31. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    You have to search their supported formats first, as we are not familiar with their service.
    For further technical support, please reach us via email: thelghome@gmail.com
     
  32. GaboFacil3D

    GaboFacil3D

    Joined:
    Jan 15, 2020
    Posts:
    19
    Hi! i got your asset and it's great! But I'm trying to use it with Photon Pun 2 and I'm having a problem with the client buffers limit wich is 500Kb/s. I know thats a lot to reach but i need to stream my pc desktop to an oculus quest 2, and I dont know how to get a decent video quality to be able to read everything in the screen as you can do in Horizon Workrooms (quest 2 app). If i increase the resolution or the quality, as you might expect the buffer size grow.

    in the GameViewEncoder there is an "Encoded Size(byte)" field and I dont get what is telling me because the maximum value it gets is 50000 but im not sure if its bytes per second or maybe the size of the last frame sended.
     
  33. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    It’s the size of last encoded frame in byte[].

    Could you please send us an email for technical support?(with your encoder screenshot settings)

    technical support: thelghome@gmail.com

    PS: We are working on some new features, including remote desktop control template. It might take us some time to finish, but once it’s ready, it should reduce bandwidth hugely. Meanwhile, welcome for any donations if you’d like to boost the development on this part, which can secure our manpower in research team. “Buy Me a coffee” link is available at the bottom of our home page. https://frozenmist.com/
     
  34. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [V3.180] minor updates
    -added different Audio Read method for AudioEncoder: "OnAudioFilterRead", "AudioListenerGetOutputData"
     
  35. FyndNorway

    FyndNorway

    Joined:
    Sep 4, 2018
    Posts:
    18
    I have found an issue in IL2CPP with using GameViewEncoder set to desktop.

    Steps to reproduse:
    1. New unity project Unity 2020.3.11
    2. Import FMETP 3.170
    3. Open SampleScene and add new GameObject with GameViewEncoder component.
    4. Set GameViewEncoder Capture Mode to Desktop
    5. In Project Settings -> Player, Set Scripting Backend to IL2CPP
    6. Open Build Settings, Add SampleScene to build and Build project for windows.
    7. Start .exe for project.
    8. An error is written to Player.log. (C:\%userprofile%\AppData\LocalLow\DefaultCompany\UnityProjectName\Player.log)
    Error starts in GameViewEncoder.cs ".AddComponent<FMDesktopManager>()
    NotSupportedException: To marshal a managed method, please add an attribute named 'MonoPInvokeCallback' to the method definition. The method we're attempting to marshal is: SCL.MonitorCaptureConfiguration::OnCapture
    at SCL.MonitorCaptureConfiguration..ctor (SCL.MonitorCallback callback) [0x00000] in <00000000000000000000000000000000>:0
    at FMDesktopManager.StartAll () [0x00000] in <00000000000000000000000000000000>:0
    at UnityEngine.GameObject.AddComponent[T] () [0x00000] in <00000000000000000000000000000000>:0
    at FMETP.GameViewEncoder+<RenderTextureRefresh>d__122.MoveNext () [0x00000] in <00000000000000000000000000000000>:0
    at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00000] in <00000000000000000000000000000000>:0
    UnityEngine.GameObject:AddComponent()
    FMETP.<RenderTextureRefresh>d__122:MoveNext()
    UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)


    I can work with Mono builds for now, but it would be great if IL2CPP was supported for Desktop streaming.
     
  36. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Thanks for reporting this issue, we will investigate it further.
    You may also reach us via email, and we can update you asap if it's been solved.
    Technical support: thelghome@gmail.com
     
  37. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [V3.190] Fixed Windows IL2PP compile bug related to FMDesktop feature

    PS: We've released the quick fix for your issue, it should be available for download via Asset Store now.
     
    FyndNorway likes this.
  38. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [News & Progress]
    This is one of recent working in progress features, Youtube Live Stream(Audio + Video) and related...etc
    It's still in private test, any donation or fund can help us to boost the development.
    Screenshot 2022-10-26 at 11.02.34 PM.png
     
    Jom000 and artpologabriel like this.
  39. fish-rp

    fish-rp

    Joined:
    May 6, 2021
    Posts:
    9
    Having trouble getting this to work in WebGL, the connection is closed whenever I try to connect with warning:
    Code (CSharp):
    1. WebSocket is closed before the connection is established
    I've sent you an email, please follow up when you can.
     
  40. dawnr23

    dawnr23

    Joined:
    Dec 5, 2016
    Posts:
    41
    FMETP STREAM 2.0
    FMETP STREAM 3.0

    Hello, I am planning to purchase only one of the two assets.
    The functions below are required, please check which one is better to purchase.

    1. Platform and device status
    '1 Android tablet' acts as a server,
    '20 Oculus Quest2s' will serve as clients.

    2. First Required Function
    The server needs to send and receive data to 20 clients.
    ex) Functions such as checking the connection status and moving on to the next step

    3. Second Required Function
    The server chooses 1 out of 20 clients to stream the screen to. We will only stream one screen at a time.

    If you select a different client, the previous screen streaming ends and only the selected one needs screen streaming to work again.


    *For reference, the current project is 2019.4.34, so please check if FMETP STREAM 3.0 can be used.
     
  41. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    For mobile devices, FMETP STREAM 3.0 is recommended for better performance. There is no limit on connection count, it’s depending on your device and router specs and performance.

    If you are only streaming to one client at a time, it should be working well.
     
  42. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Below is the suggested solution, though we are not expert in docker setup.
    According to previous customer store review & solution from ChrisWalsh3:
    Code (CSharp):
    1.  
    2. //If you are running in a docker container and are using subdomains with an NGINX reverse proxy
    3. //(browser will block any non-secure socket access and any access to sockets away from the root domain),
    4. //make sure to add the lines:
    5.  
    6. # enable WebSockets
    7.              proxy_http_version 1.1;
    8.              proxy_set_header Upgrade $http_upgrade;
    9.              proxy_set_header Connection "upgrade";
    10. //to your proxy config to get WSS working,
    11. //spent a while fixing that and it's an obscure thing so hopefully will help someone out.
    12.  

    ref: https://frozenmist.com/docs/apis/fmetp-stream/troubleshooting/
     
  43. dawnr23

    dawnr23

    Joined:
    Dec 5, 2016
    Posts:
    41

    The current Unity project version is 2019.4.34.
    Will I be able to use FMETP STREAM 3.0 with that version?
     
  44. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    It should be compatible down to Unity 2018 LTS, though is lower than our Asset Store labelled version.
     
  45. dawnr23

    dawnr23

    Joined:
    Dec 5, 2016
    Posts:
    41
    I am planning to purchase FMETP STREAM 3.0.

    Unity version wants to use 2021.3.12f1 .

    1. Please answer whether all of the following functions are available in 2021.3.12f1 version.
    - Updated Quest 2 templates in Unity2021.2.13f using Oculus SDK v37 (stable 90 FPS in-game)
    - UDP? added reliable send option(experiment)

    2. Please reply whether the TCP communication specified in this asset refers to WebSocket communication or Socket.IO communication.

    3. Is there only UDP communication that can communicate from Android (server) to Android (client) in this asset? (I want reliable TCP communication with the above device method.)
     
  46. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Our latest testing project is in Unity 2022.1.16f1, it shouldn't be a problem with older version, otherwise please report to us.

    #1 YES.
    #2 Both WebSocket, and Socket.IO, and a legacy TCP+UDP system. For Quest 2, WebSocket will be recommended if you are looking for reliable solution.

    #3 Yes, it's very easy to switch between FM Network UDP, or FM WebSocket system, by changing the network manager component.

    PS: the reliable UDP system is still in experiment. If you want 100% reliable solution, please go ahead with FM WebSocket or Legacy TCP(not recommended) system instead.
     
  47. FyndNorway

    FyndNorway

    Joined:
    Sep 4, 2018
    Posts:
    18
    I can confirm that version 3.0 works fine for us on Unity 2019.4.11
     
  48. Airmouse

    Airmouse

    Joined:
    Jan 12, 2019
    Posts:
    107
    Hello I purchased this plugin around a year ago, I have been very happy with the quality of the video stream. But I am starting a new AR project and was wondering if there have been any recent updates that may have improved the encoder/decoder performance by any amount? My AR project requires slightly higher resolution than I currently am able to achieve, I have to greatly turn down the image quality for the video to appear very realtime - but this which tends to make the video appear slightly pixelated.

    I am encoding video on a Gaming PC
    and decoding the video stream on a Meta Quest Pro HMD,

    Any advice on how to achieve realtime HD video using FMETP (streaming from PC to Quest Android)?

    Thanks!!
     
    Last edited: Nov 13, 2022
  49. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Thanks for your trust in our product.
    The overall performance in FMETP STREAM 3.0 is better than 2.0, with optimisation in encoder/decoder, and networking, and major improvement on HL2.

    We haven't tested on the latest Quest Pro yet, is it significantly faster than Quest 2 in terms of hardware?
    Meanwhile, wifi is another concern related to latency.
     
    Airmouse likes this.
  50. CodePoKE

    CodePoKE

    Joined:
    May 6, 2015
    Posts:
    23
    @thelghome
    We've bought FMETP Stream 3.0 and are looking into upgrading to FMWebSocketManager as you suggest.
    However, we've already built a backend service now around socket.io. We've also built up bi-directional communication on socket.io. Is there an document that outlines the upgrade path?

    Is there a way to easily port over communication using FMSocketIOManager's `On()` to `FMWebSocketManager`?