Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[Release] FMETP STREAM: All-in-One GameView + Audio Stream + Remote Desktop Control(UDP/TCP/Web)

Discussion in 'Assets and Asset Store' started by thelghome, Apr 30, 2019.

  1. maxuntoldgarden

    maxuntoldgarden

    Joined:
    Dec 9, 2019
    Posts:
    7
    Hi there, Thanks for a great asset!

    The audio stream sounds very choppy using WebSockets regardless the StreamFPS value.
    The stereo output often fails to play at all.
    How can I stream dual channel Game Sound so it plays in the browser as smoothly as possible?

    I'm using Unity 2020.2.1f1.

    Thank you!
     
  2. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    For better browser audio playback quality, we suggested using WebGL build instead of pure html5 demo.

    Our html audio playback demo is very basic without using 3rd party libraries, which didn't handle the buffer well. You may consider some javascript libraries for better audio playback, as long as they support PCM data input.

    Hope this can give you some directions.
     
  3. lokey357951

    lokey357951

    Joined:
    Jan 1, 2019
    Posts:
    1
    Does it support large data streaming, I am trying to stream around 60Mb/s for point cloud streaming, any suggestion how to achieve this if possible? thanks
     
  4. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    It might be heavy for websocket streaming, we didn't do much test.
    If you are already working in progress, please feel free to reach us for discussion. surely we can share some idea of how to optimise for some extreme use case.
    email: thelghome@gmail.com
     
  5. citron8000

    citron8000

    Joined:
    Nov 25, 2014
    Posts:
    22
    We are trying this plugin with the Quest 2 and are having performance issues. We are going from > 72fps to 50fps with 256px resolution, 15 fps and 20% quality (with Fast Mode and Async activated).

    Looking at previous messages, it looks like the RenderCam is the fastest way? But it looks like this method always needs to have two cameras in the scene? The camera that is used by the GameViewEncoder seems to be "stolen" from the script and does not render anymore. So an additional camera needs to be added to the camera under this one. And of course, an additional camera is a big hit on performances. Am I missing anything?
     
  6. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    The reason why "MainCam" mode is slower, because it blocks main thread a lot especially on mobile devices, as it has to wait for the rendered frame from main cam.
    Of course, it will still have some impact with 2 cameras setup in the scene. But overall the extra impact is on GPU instead of CPU. That's why it's faster than "MainCam" mode in general.
     
    Last edited: Jan 17, 2021
  7. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    PC (Windwos10) to HoloLens2 (UWP)

    I want to transfer the .mp3 file in the Application.persistentDataPath path in a UDP environment.

    Can you get example code?
     
  8. danielesuppo

    danielesuppo

    Joined:
    Oct 20, 2015
    Posts:
    331
    Hello thelghome!
    We should develop a virtual showroom, and we have 2 options:

    A) the users (more that 1 at the same time) would be able to visit the showroom 24h a day (so the unity app I think would be hosted on Furioos, and distributed in a web page), and the showroom owner would be able to drive the user (1 at time) across the room (something like remote assistance).

    B) the users (1 at time) would be able to visit the showroom (through a web page) only when he's "invited" (so the unity app I think woud be hosted on the showroom owner PC), and the showroom owner would be able to drive this user across the room (something like remote assistance).

    I'm very newby to streaming/networking technologies, and I admit that I'm a bit confused, so maybe my questions can be naive, I'd like to know:

    1) would it be possible with your asset to let the showroom owner to drive the user (so take control of the app), with the Unity app hosted on Furioos (option A) ?
    (I think that the owner should have its own version of the Unity app on its PC, that would connect to the one on Furioos...)

    2) would it be possible with your asset to let the showroom owner to drive the user (so take control of the app), but also let the user move on its own in the room if he like (option B) ?

    I know that these questions are confused, but the project is in a really early stage, and I'm trying to understand what can be realized.
    Many thanks!
     
    Last edited: Jan 19, 2021
  9. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    For mp3, you can consider as just a file in bytes. You can convert your mp3 into bytes and stream the byte[].
    FMNetworkManager.instance.Send() supports bytes streaming, if you have large file, you can pass data into LargeFileEncoder instead.
     
  10. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    basically, the server can send commands, and client can also send commands to each other.
    I guess what you are asking is the possibility of remote control. If that's the case, you can surely do it.
     
  11. asdferasv

    asdferasv

    Joined:
    Dec 24, 2020
    Posts:
    8
    I tried to share the Window desktop to HTML according to your method, but when I clicked the connect button on the HTML page, Unity got stuck and the HTML page didn't receive any images.The version of Unity I'm using is 2019.4.7f1, what's going on?
     
  12. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Did you follow this tutorial?


    It may be related socket.io version too.
    Please make sure you have correct version installed via this command.
    npm install socket.io@2.3
     
  13. tkbala

    tkbala

    Joined:
    Nov 17, 2017
    Posts:
    1
    Hi,

    We are trying to stream views from multiple virtual cameras that are present within a Unity environment. Would this asset help us achieve that?

    Thanks,
     
  14. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Fyi, you can have multiple render cams with different IDs for streaming.
     
  15. taktylstudios

    taktylstudios

    Joined:
    Aug 3, 2018
    Posts:
    1
    Hi how do you setup Amazon Web Services with this?
     
  16. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    You have to follow other's node.js hosting guide on Amazon Web Services, as we don't involve much about this topic.
     
  17. jean42loire

    jean42loire

    Joined:
    Jun 8, 2019
    Posts:
    2
    Hello,
    Currently we are streaming the view of the oculus quest 2(Server) on Android phone(Client) but we have performance problems(Low FPS on Server), how can we setup Server-Client communication properly?(We are using FM Network UDP) Thank you
     
  18. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Please ensure that "Fast Mode, Async Mode, and GZip Mode" are all enabled in your encoder/decoder.
    This will give your fast performance.

    And of course, reducing streaming resolution and streamFPS may improve too.
     
  19. jean42loire

    jean42loire

    Joined:
    Jun 8, 2019
    Posts:
    2
    Thank you very much ! We are testing it now :)
     
  20. danielesuppo

    danielesuppo

    Joined:
    Oct 20, 2015
    Posts:
    331
    Hello thelghome,
    I'm trying to make you demo node.js server to work on HTTP.
    I've modified your "index.js" this way
    Code (JavaScript):
    1. var express = require('express');
    2. var app = express();
    3. app.use(express.static(__dirname + '/public'));
    4.  
    5. var https = require('https');
    6. var fs = require('fs');
    7.  
    8. var httpsOptions = {
    9.     key: fs.readFileSync('/etc/letsencrypt/live/issimissimo.com/privkey.pem'),
    10.     cert: fs.readFileSync('/etc/letsencrypt/live/issimissimo.com/fullchain.pem')
    11. };
    12. var https1 = https.createServer(httpsOptions, app);
    13.  
    14. var io = require('socket.io')(https1);
    15. https1.listen(3000, function(){ console.log('listening on *:3000');});
    and I've enabled SSL on "FM Socket IO Manager" in Unity,
    but the Unity app can't connect to the server, I get this error:

    InvalidOperationException: A series of reconnecting has failed.
    WebSocketSharp.WebSocket.Connect () (at <986d058667b4442995cd7127d93d0835>:0)

    Please could you give me some help to make it work on HTTPS ?
    (P.s. with HTTP I had no problem at all, but unfortunately I must use HTTPS)

    Many thanks!
     
  21. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    In FMSocketIOManager, there are different Ssl options, like Ssl2,Ssl3,TLS..etc.

    did you try different options for Ssl connection?
     
  22. danielesuppo

    danielesuppo

    Joined:
    Oct 20, 2015
    Posts:
    331
    Thank-you for your super fast reply!
    Yes, you are right!
    I've been able to connect with Tls12
    Thanks a lot!
     
    thelghome likes this.
  23. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [v1.333] minor updates, submitted to the asset store.
    -FM WebSocket is compatible with socket.io v3.1 and v2.xx now.
    -Fixed minor bug on URP editor.
     
  24. danielesuppo

    danielesuppo

    Joined:
    Oct 20, 2015
    Posts:
    331
    Hello thelghome,
    actually I have my Unity app (as server) that send the video to a JS client (user1).
    I'd like to send only strings to another JS client (user2).
    But I have some doubt:

    1 - If I connect to web socket both user1 and user2, do both will receive the dataByte of the video from the node.js server, regardless of whether they use it or not (just because them both are connected to the web socket)?
    If it's so, there would be double outgoing web traffic from node.js server, and a lot of unused incoming web traffic for User2, and this is not what I'd like...

    2 - if I send from my Unity app (as server) video and string to user1 and user2 (==> send to others), how can I understand form JS client side if I'm receiving a video or a string?

    Many thanks
     
    Last edited: Feb 3, 2021
  25. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    There are few ways to optimise the server logic, for better bandwidth control.
    1) modify our node.js server, with room feature. and you can use your own query string instead of our default:


    2) add some meta data in your strings, and check those meta data in node.js server.

    Those solutions are recommended by customers, which they modify our server for their own use.
     
  26. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Technically, our default FMWebsocket logic:
    Unity App(Server/Client) => node.js server => check the received data from unity, with some meta data => send data to "All/Server/Client" => Unity App(Server/Client)

    and you can modify the node.js server script, added some room logic with custom query string.
    and you will have sth like below:
    Unity App(Server/Client) => node.js server with specific room name ("RoomTom") => check the received data from unity, with some meta data => send data to "All/Server/Client" => Unity App(Server/Client)

    Theoretically, all data will go to the node.js server. But you can filter out the data by room name, and only the users in the same room can communicate.
     
  27. danielesuppo

    danielesuppo

    Joined:
    Oct 20, 2015
    Posts:
    331
    Yes, I perfectly understand.
    I had some misunderstanding on socket.io, that's why I was a bit confused.
    Moreover my flow is
    Unity App(Server/Client) => node.js server => check the received data from unity, with some meta data => send data to "All/Server/Client"
    so I don't have any UnityApp on the end, just the html.

    On the end I've just checked on the node.js what kind of data is coming, and send specific json to clients.
    Many thanks for your help!
     
  28. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    our html demo is also acted as same as unity app, and node.js with socket.io is the "real" server for the whole communication.
     
  29. danielesuppo

    danielesuppo

    Joined:
    Oct 20, 2015
    Posts:
    331
    I also understand this very well.
    Because of my really poor Socket.IO knowledge, and web server in general, I have a question:
    how many clients (for example with your index.html) can receive the stream at the same time?
    Is there a software server limit (due to Socket.IO), or is it just a matter of bandwidth?

    P.S. with last release of your asset can we use Socket 3.xx or do we have to stay with 2.3 ?
     
    Last edited: Feb 8, 2021
  30. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    I think the major limit is bandwidth, which we are still working on different ways for optimisation at the moment.
    socket.io v3.1 and v2.xx are supported, but socket.io v3.0 doesn't.

    There is a newer version FMETP STREAM v1.338 coming soon today/tmr, which fix some known bugs on WebGL build with socket.io v3.1.
     
  31. Deusald

    Deusald

    Joined:
    Feb 11, 2014
    Posts:
    31
  32. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    FMETP STREAM is targeting streaming between Unity platforms. It's not a wrapper for other online streaming service, unfortunately.
     
  33. danielesuppo

    danielesuppo

    Joined:
    Oct 20, 2015
    Posts:
    331
    I'm getting crazy...
    I'm using "OnReceivedStringDataEvent (string)" of "FM Socket IO Manager".
    When I put a function to execute on this event, sometimes it expose an empty string value to fill (and so the string value is NOT received), sometimes not (and so the string value is received).
    This happen also with the very same function cloned...!

    Please look at this:

    Furthermore, now when I change the function It always expose this empty string value (and so I can't receive any string value with both functions anymore).
    What's going on?

    Many thanks
     
    Last edited: Feb 11, 2021
  34. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Please use the first dynamic option without textfield. The difference is dynamic value vs static value.

    This is a very common use of Unity event system.
     
  35. danielesuppo

    danielesuppo

    Joined:
    Oct 20, 2015
    Posts:
    331
    Yes, sorry my fault, too tired o_O
     
  36. dragadaga

    dragadaga

    Joined:
    Oct 2, 2016
    Posts:
    1
    Can you add mirror integration?
     
  37. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    We don't have mirror example scene, but it should work according to customers' feedbacks.
    The setup logic is very straightforward:
    GameViewEncoder -> bytes -> Network Manager(your mirror byte streaming script) -> bytes -> GameViewDecoder

    Hope this can help.
     
  38. chlagou

    chlagou

    Joined:
    Jan 15, 2014
    Posts:
    13
    Hello,
    when I test in unity editor everything works fine but when I upload my build to the server I get this

    error gunzip.jpg

    has anyone encountered this? how I can solve this?
     
  39. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    According to your "404" error, did you include "gunzip.min.js", which should be in our latest node.js demo server?
     
  40. varan941

    varan941

    Joined:
    Jul 10, 2019
    Posts:
    7
    Please, tell me how to properly configure FMNetwork Stream for remote work, if I use my computer as a server. I have a white IP and an open port for UDP (8888), but it doesn't work. I need the clients to see what the host shows.
    How can I fix this? Or I must use not FMNetwork Stream?
     

    Attached Files:

  41. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    For Internet streaming, you have to switch to FMWebSocket instead of FMNetworkUDP solution.
    Another alternative is via Photon.

    FMNetworkUDP is for local network unfortunately.
     
    varan941 likes this.
  42. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [v1.341] updates (submitted & soon)
    added "Ignore similar Texture" check in GameViewEncoder, in order to save bandwidth for similar images.
     
  43. GabsKapellmann

    GabsKapellmann

    Joined:
    Jun 13, 2018
    Posts:
    10
    Hello,
    I was wondering if this package could help me stream a webcam that is captured in a .Net application (in C#) and stream it into a Unity app through the network (WAN)?
     
  44. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    If it's a mjpeg based stream, we have fast decoder for it.
    It's not a direct solution, you may need to create a small wrapper for receiving your raw data.

    However, we cannot guarantee anything for your use case.
     
  45. yuricabrita

    yuricabrita

    Joined:
    Nov 10, 2020
    Posts:
    1
    Hello, can anyone point me to a good hosting place of the node.js server used in the websocket demo? I wanna be able to communicate not only to the local host but some website on the web. But I am not sure which deployment sites would allow for the best integration.
     
  46. hyperbaton

    hyperbaton

    Joined:
    Dec 16, 2018
    Posts:
    7
    Hi, are you still planning to add RTMP to the asset? If so, is there an ATA?
     
  47. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    If budget allows, AWS is always a good option.
     
  48. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    still in To-Do list, but we don't have schedule yet unfortunately.
     
  49. longvann1912

    longvann1912

    Joined:
    Aug 22, 2020
    Posts:
    3
    Hi, I want to clients stream video to server. Websocket can do that? Thank you
     
  50. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Yes, it supports.