Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

[Release] FMETP STREAM: All-in-One GameView + Audio Stream + Remote Desktop Control(UDP/TCP/Web)

Discussion in 'Assets and Asset Store' started by thelghome, Apr 30, 2019.

  1. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    It seems like Unity3D misconfiguration after upgrading some project files.
    Solution: Please try to delete your Library folder and manually refresh and reimport the packages.

    the importing FMETP Stream package requests me to update package manager, try to press skip.
    or, ignore our project settings when import, as it might be out-dated between Unity2018LTS & Unity2019. But those settings are not important.

    In case that your problem exists, please feel free to email us: thelghome@gmail.com
     
  2. SkyWalkerMK

    SkyWalkerMK

    Joined:
    Jul 27, 2016
    Posts:
    8
    I tried it and it gave me some new errors. I am also using Socket.IO for Unity, It might be the reason maybe? What can be done?
     

    Attached Files:

  3. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    It seems some duplicated library issue between your Socket.IO plugin and FMETP STREAM.
    We've also included socket.io support, we recommended that you should remove either your Socket.IO plugin, or remove FMWebSocket folder in this case.
     
  4. blevok

    blevok

    Joined:
    Feb 16, 2017
    Posts:
    75
    Hi, i'm trying to stream windows 10 desktop (with monitor selection, and including taskbar and mouse pointer) to android in real-time over the local network and display at 30fps on an object in world space. I don't care about audio.

    I came up with my own solution, two actually, but they're not very good. The first has over 2000ms of lag and streams at 30fps. The second has less than 50ms of lag, but only streams at 10fps.

    I'm trying to get 30fps with less than 50ms of lag, so that the user can use the PC mouse and keyboard, with the 3d object on android acting as the monitor. I also want to send commands from android to the PC to change FPS and compression.

    After skimming this thread, it looks like i can do that. Do you agree that this should work? Also, i'm not sure which version i need. Since i only need local network, can i use FM Network UDP, or is the desktop capture only available in FMETP STREAM?

    Thanks
     
  5. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    In order to solve your question, you may refer to our iPhone 5 x Win10 Monitors demo.
    Things will also work on Android Devices.
    PS: iPhone 5 is a very low-end devices, any new phones nowadays can work much better.
     
  6. blevok

    blevok

    Joined:
    Feb 16, 2017
    Posts:
    75
    Okay thanks, that looks good. But can i do this with FM Network UDP, or is this only available in FMETP STREAM?
     
  7. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    It requires FMETP STREAM for making this. Hope this can solve your concerns.
     
  8. blevok

    blevok

    Joined:
    Feb 16, 2017
    Posts:
    75
    Okay thanks.
     
  9. blevok

    blevok

    Joined:
    Feb 16, 2017
    Posts:
    75
    This is working well with my main monitor, but i'm having an issue with my second monitor.
    FMDesktop.Manager.monitorCount returns the correct number of monitors, but FMDesktop.Manager.GetMonitor(1).state returns "unsupported".
    Any idea why that would happen? I tried to investigate, but i don't see any source for the FMDesktop dll, so i can't see what's it's checking to decide if a monitor is supported.

    edit:
    I just discovered that it reads the monitor info about the monitor that the player is running on.
    When i drag the window to the other monitor, i detect the resolution change and set FMDesktopMonitorID = 1; then it just shows a grey texture.
    So now i just need to figure out how to get it to properly change the monitor that's being captured. I tried Action_ReloadScene(); and i tried disabling and re-enabling the FMNetworkManager game object, but no luck yet.
     
    Last edited: Aug 24, 2020
  10. SkyWalkerMK

    SkyWalkerMK

    Joined:
    Jul 27, 2016
    Posts:
    8
    Last edited: Aug 25, 2020
  11. Green-VR

    Green-VR

    Joined:
    Apr 15, 2013
    Posts:
    12
    Hi, I'm looking to buy your asset. Quick question, Can I use it to stream my webcam footage and audio on a PC to a unity Webgl texture in-game? hosted on my website.
     
  12. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    Would you please write us an email for technical support? we can follow your case easily.
    This feature is using Windows Desktop Duplication API (DDA) in the backend and doesn't allows it to run app on discrete GPU.

    There are some potential solutions and alternatives for this, which I could share more info to you via email: thelghome@gmail.com
     
  13. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    There might be some duplicated libraries between FMETP and your tool.
    We recommended that you should switch to FMWebSocket, as it has better performance and using newer version of websocketsharp.

    Or, you should remove our FMWebSocket Folder completely as alternative.
     
  14. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    FMETP STREAM allows you to stream any in-game video & audio between Unity3D builds.
    For example, if you have a video source, you can setup a standard in-game camera to capture your video result for the streaming.

    You can host on website, but you will need to host your node.js server, or cloud server, or Photon..etc as the middle part of communication. We do have step by step tutorial on your youtube channel for node.js local hosting. You may check them out and see if it fits your need.
     
  15. SkyWalkerMK

    SkyWalkerMK

    Joined:
    Jul 27, 2016
    Posts:
    8
    I already deleted FMWebSocket. It was giving errors. but if it is easy to use I would like to try it out. Is there a guide for FMWebSocket so I can understand it.
     
  16. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    We have few tutorials on youtube.
    Functions are very straight forward, with FMSocketIOManager.instance.SendToAll/Server/Others()

    # setup for live streaming part:


    # or for customised socket.io event..etc
     
  17. blevok

    blevok

    Joined:
    Feb 16, 2017
    Posts:
    75
    Okay thanks. I sent you an email about that, and a couple other questions about streaming desktop audio, and possibly connecting via USB cable instead of wifi.

    Edit: I did a little research about DDA and found that it does support discrete graphics, but it cannot switch between discrete and dedicated gpu while the program is runnning. The program has to be run on the hardware you want to stream the output from. So if you have a PC with 2 monitors, and one is plugged into the onboard gpu, you can't switch between outputs while the program is running, but if you run it on the discrete gpu then it will work.
     
    Last edited: Aug 26, 2020
    thelghome likes this.
  18. Green-VR

    Green-VR

    Joined:
    Apr 15, 2013
    Posts:
    12
    Hi, just got your asset from the asset store and was trying to find some doc to help my use photon as the network.
    are there any demos or documents I can look at to try this? also all the demos i can find show just local networking so an not sure.
     
  19. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    Thanks for trying our asset, you may follow below logic for your Photon setup.
    GameViewEncoder -> bytes -> Photon -> bytes -> GameViewDecoder

    Sorry for the lack of document in Photon part, as we didn't involve much tutorial for 3rd party networking solution.
    But it won't be difficult to set it up yourself.

    In case that you need further help, please write us an email for technical support.
    thelghome@gmail.com
     
  20. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    [ver1.319] Stream In-Game 360 Panorama View
    Submitted and waiting for approval.
    BTW, 3 mins step-by-step guide is ready now! you may check it out.
    https://youtu.be/OLsrNnoRMJ0

     
  21. SkyWalkerMK

    SkyWalkerMK

    Joined:
    Jul 27, 2016
    Posts:
    8
    Hi. I checked the url and it is correct but I have still getting this error. FMLog: Connection Execption: A series of reconnecting has failed.
     
  22. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    Could you please write us an email for technical support? to: thelghome@gmail.com
    It seems like that you have missing/wrong IP & Port for WebSocket.
    You can always follow this tutorial guide:


    and please don't forget to setup your node.js server for web streaming.
     
  23. SkyWalkerMK

    SkyWalkerMK

    Joined:
    Jul 27, 2016
    Posts:
    8
    I sent an email.
     
  24. Barritico

    Barritico

    Joined:
    Jun 9, 2017
    Posts:
    365
    I apologize in advance for my ignorance on the subject.

    Whenever I see tools like this, I wonder if the authors recommend some kind of server. I mean, more than a specific brand, some characteristics.

    I guess you can hire a dedicated server with a fixed IP. And I guess that server should allow node installation, among other things.

    Is that so?

    A server with a bandwidth of 500 would be enough ... how many concurrent users would it accept?
    These things (the same thing happens to me with multiplayer tools) are the ones that I do not understand.

    Sorry and thank you.
     
  25. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    It’s a bit hard to answer you with exact number of users for some server specs.

    But you can always estimate the limit in terms of zipped jpeg file size x fps x streams(count).

    It’s a very dynamic & magical number as always. Resolution and compression quality, and content are all the key factors for bandwidth estimation. Some cheap servers may have performance issue, while good servers are costly.

    We cannot guarantee for any server suggestion, as we are mainly working on functionality for C# Live Stream solution.

    The network system can always be replaced and there are many alternative.

    That’s also good for development with flexibility.
     
    Barritico likes this.
  26. Barritico

    Barritico

    Joined:
    Jun 9, 2017
    Posts:
    365

    Thank you very much for your reply.
     
  27. lincode

    lincode

    Joined:
    May 2, 2019
    Posts:
    5
    I's a great asset. Thanks for your work. I bought it, and using it for my project.
    Do you have any plan for MacOS's Desktop Capture?
     
  28. lincode

    lincode

    Joined:
    May 2, 2019
    Posts:
    5
    When I run Windows Capture demo, I got the warning like blow:
    and I can't see the Preview in the Encoded area of GameViewEncode, we don't see the result on Client of couse.
    Do you have any idea?
    My OS is Window 10.

    [FMD] 0:\\.\DISPLAY1 => Unsupported.

    UnityEngine.Debug:LogWarningFormat(String, Object[])

    FMDesktop.Monitor:.ctor(Int32)

    FMDesktop.Manager:CreateMonitors()

    FMDesktop.Manager:Awake()

    UnityEngine.GameObject:AddComponent()

    FMDesktop.Manager:CreateInstance()

    FMDesktop.Manager:get_instance()

    FMDesktop.Manager:get_monitors()

    FMDesktop.Manager:GetMonitor(Int32)

    <RenderTextureRefresh>d__50:MoveNext() (at Assets/FM_ExhibitionToolPack/FMCore/Scripts/Mapper/GameViewEncoder.cs:337)

    UnityEngine.GUIUtility:processEvent(Int32, IntPtr)
     
  29. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    I think this is about GPU issue, which I will provide more info for you and reply your email soon.

    This feature is using Windows Desktop Duplication API (DDA) in the backend and doesn't allows it to run app on discrete GPU.

    PS: Mac OS Desktop support is in our To-Do list, it needs some time.
     
    Last edited: Sep 5, 2020
  30. blevok

    blevok

    Joined:
    Feb 16, 2017
    Posts:
    75
    I'm a little confused about this. Have you done enough testing to be able to clarify what situations won't work?
    I've done the following tests:

    Desktop PC (windows 10 pro)
    Radeon RX580 - 1 monitor (primary)
    Intel HD Graphics 4000 - 1 monitor (secondary)
    Primary monitor (dedicated GPU) works, secondary monitor (integrated GPU) returns "unsupported"

    Surface Pro 2 (tablet - windows 10 pro)
    Intel HD Graphics 4400 - built-in monitor
    Works

    Lenovo ThinkCentre M92p (mini PC - windows 10 pro)
    Intel HD Graphics 2500 - 2 monitors
    Both monitors work

    Toshiba C855D (laptop - windows 8.1)
    Radeon HD 6310 - built-in monitor
    Does not work

    Toshiba C855D (laptop - windows 10) - same machine as above, after OS upgrade
    Radeon HD 6310 - built-in monitor
    Does not work

    So it seems that it requires an integrated GPU, but will work on dedicated GPU, as long as integrated GPU is also present. But it only works with the GPU that the primary monitor is currently connected to, even if it's not the integrated GPU.


    Lincode can you report on your PC situation? I'm guessing you have a laptop with an nvidia/ati GPU and no integrated GPU. As far as i know, desktops without integrated graphics are pretty rare.
     
  31. Juruhn

    Juruhn

    Joined:
    Jun 25, 2013
    Posts:
    13
    Hi again,

    Small question:
    How to capture a single snapshot with the 'GameViewEncoder' from code?
    Reason: We stepped away from updating the stream in realtime but I want to send updates from the Client of the 'Render cam' at certain moments in my application to the server.
    Right now I'm enabling / disabling the whole 'GameViewEncoder' script in 0.1 seconds in a coroutine, but there must be a nicer way :)

    Thanks for any assistance!
    Cheers
     
  32. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    Yes, there is a manual way to do it.
    First, you have to set the StreamFPS as 0 FPS.
    And you can call "Action_UpdateTexture()" function manually for capture once only.
     
    Juruhn likes this.
  33. vietman00

    vietman00

    Joined:
    Dec 27, 2018
    Posts:
    3
    Has anyone been able to use this asset to connect a Unity WebGL game to a node.js express app from the command line with socket.io? I can only get it working with the Unity Editor and Windows App..
     
  34. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    We have a demo WebGL Echo test connecting to echo.websocket.org
    https://frozenmistadventure.github.io/demofmwebsocket_echo_webgl_ssl/

    You may check it out. However, we don't have tutorial on node.js cloud hosting, as there are too many platforms & services providers.
     
  35. vietman00

    vietman00

    Joined:
    Dec 27, 2018
    Posts:
    3
    How can we get WebGL working with regular websockets (ws://)? The demo included in the assets folder specifies ws:// and works in Unity Editor but not when building WebGL..

    FM_ExhibitionToolPack\FMWebSocket\Scenes
     
  36. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    In my case, we simply build the WebGL demo without other tricks.
    In order to host the WebGL demo on github as webpage, I enabled the "Ssl" option in demo scene. As github page is https, which requires "wss://".

    In local WebGL hosting test, I use MAMP to init the web server environment. but without "Ssl", which is also working well.

    According to feedbacks from customers, it's most likely the problem on web server or node.js server settings, and there might be some trick and modification needed on cloud hosting.
     
  37. Zarkow

    Zarkow

    Joined:
    Jul 27, 2015
    Posts:
    91
    What is the difference between:
    SendToAll()
    SendToOthers()
    I am reading the documentation but it remains unclear.
     
  38. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    SendToAll() will stream the data to everyone including the sender.
    SendToOthers() will stream the data to everyone except the sender.

    For example, PlayerA,B,C,D:
    PlayerA uses SendToAll(), PlayerA,B,C,D will receive the data.
    PlayerA uses SendToOthers(), PlayerB,C,D will receive the data.

    Hope this can give you more idea.
     
  39. Zarkow

    Zarkow

    Joined:
    Jul 27, 2015
    Posts:
    91
    Alright, I suppose that make sense for some usecases. But unless I misunderstand then, any and all of these functions must use an intermediate hosted server, which runs into the realm of forcing security to prevent data-leakage and to have a stable server in some service-park that cost money? I.e. does any of these or related functions work peer-to-peer?
     
  40. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    We provided few networking solutions: UDP, TCP, and WebSocket.

    For the FM Network UDP example, you can also use SendToTarget() with specific local IP.
    For FM WebSocket, it's a WebSocket solution with socket.io, which requires an intermediate server(node.js server).
    You can host the node.js server yourself, or pay for some cloud services. It's up to you.
    There is a step-by-step tutorial of how to setup your own node.js server.


    Beside our default networking solutions, you can also apply your own networking system which supports byte streaming, by following below logic:
    GameViewEncoder -> bytes -> Network Manager -> bytes -> GameViewDecoder

    According to feedback from many customers, it's compatible with Photon, Mirror, UNet..etc and some customised server on Linux.
     
  41. iMagesBlues

    iMagesBlues

    Joined:
    Mar 18, 2013
    Posts:
    41
    Hello guys. I have purchased this plugin and it works really great.
    Though I was wondering if I can also stream my game view directly into a virtual camera (eg. for Zoom and Skype).
    Thanks!
     
  42. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    In the current version, I'm afraid that it's not possible to simulate as a virtual cam on Zoom/Skype.
    Your idea is great, we might put into our To-Do list and seek for solution in future.
     
  43. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    Hello! Please answer my questions.

    Can I transfer large mp4 files shot on mobile (Android/iOS) to PC (Windows 10) on the same network via UDP network system without server?
     
  44. Zarkow

    Zarkow

    Joined:
    Jul 27, 2015
    Posts:
    91
    I am looking into using Mirror and Complete Steamworks, to use Peer2Peer over Steams gaming network, as I will release it on Steam anyway. In that case, I could still use the feature-pack here, right?
     
  45. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    If you only need to stream files, you may use "LargeFileEncoder/Decoder".
    It will help you split the big data into chunks for FMNetworkUDP.
     
  46. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    FMETP STREAM should be compatible with any networking system with byte streaming.
    You may need to write some simple code for Mirror, in order to send bytes from GameViewEncoder, and decode the data on the other side.
     
    Last edited: Oct 6, 2020
  47. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    [v1.325] MJPEG Decoder (Experiment feature)
    We added MJPEG decoder support, which you can pass any MJPEG stream data to GameViewDecoder via Action_ProcessMJPEGData(byte[])
     
  48. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    [Known Issue] Unity2020 WebGL Build

    FMWebSocket may not work properly in WebGL Build from Unity2020, due to missing "gameInstance" or "unityInstance" is not defined.

    Solution: After building WebGL from Unity, please open index.html, and add below line.

    Code (JavaScript):
    1. window.gameInstance = unityInstance;
    After adding this line:
    Code (JavaScript):
    1. var script = document.createElement("script");
    2.       script.src = loaderUrl;
    3.       script.onload = () => {
    4.         createUnityInstance(canvas, config, (progress) => {
    5.           progressBarFull.style.width = 100 * progress + "%";
    6.         }).then((unityInstance) => {
    7.           window.gameInstance = unityInstance;
    8.           loadingBar.style.display = "none";
    9.           fullscreenButton.onclick = () => {
    10.             unityInstance.SetFullscreen(1);
    11.           };
    12.         }).catch((message) => {
    13.           alert(message);
    14.         });
    15.       };
     
  49. Flamacore

    Flamacore

    Joined:
    Dec 17, 2013
    Posts:
    133
    Hey there :) Great work on the asset, really eases a lot of pain on the websocket part.

    Although, when I run the demo in WebGL as a receiving client, I immediately encounter an error even without being able to see anything loaded. Unity 2020.1.2f, standart renderer.

    Error is below;

    Code (CSharp):
    1. Uncaught ReferenceError: io is not defined
    2.     at _WebSocketConnect (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
    3.     at /rcv/<anonymous>:wasm-function[44264]:0xd7b13b
    4.     at /rcv/<anonymous>:wasm-function[44278]:0xd7b506
    5.     at /rcv/<anonymous>:wasm-function[28078]:0xa4c803
    6.     at /rcv/<anonymous>:wasm-function[28077]:0xa4c778
    7.     at /rcv/<anonymous>:wasm-function[23180]:0x982e9d
    8.     at dynCall_iiiii (/rcv/<anonymous>:wasm-function[49029]:0xe52c2d)
    9.     at Object.dynCall_iiiii (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
    10.     at invoke_iiiii (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
    11.     at /rcv/<anonymous>:wasm-function[47108]:0xdfbb9d
    12.     at /rcv/<anonymous>:wasm-function[46401]:0xddb86f
    13.     at /rcv/<anonymous>:wasm-function[4297]:0x19ad56
    14.     at /rcv/<anonymous>:wasm-function[4296]:0x19ac7f
    15.     at /rcv/<anonymous>:wasm-function[10109]:0x3ad8b0
    16.     at /rcv/<anonymous>:wasm-function[10093]:0x3ac6aa
    17.     at /rcv/<anonymous>:wasm-function[12291]:0x4d811e
    18.     at /rcv/<anonymous>:wasm-function[12290]:0x4d7e37
    19.     at /rcv/<anonymous>:wasm-function[10501]:0x3da58a
    20.     at /rcv/<anonymous>:wasm-function[10199]:0x3b62a7
    21.     at /rcv/<anonymous>:wasm-function[10199]:0x3b62bc
    22.     at /rcv/<anonymous>:wasm-function[10194]:0x3b5e36
    23.     at /rcv/<anonymous>:wasm-function[10187]:0x3b40bd
    24.     at dynCall_v (/rcv/<anonymous>:wasm-function[49046]:0xe52f0a)
    25.     at Object.dynCall_v (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
    26.     at browserIterationFunc (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
    27.     at Object.runIter (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
    28.     at Browser_mainLoop_runner (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
    Got any idea about this?
     
  50. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    631
    Thanks for trying our assets.
    It's a common error of missing socket.io.
    You have to make sure that you type correct IP & Port, where you host your node.js server.

    + More tips: And please don't forget my tips on Unity2020 WebGL Build, in previous post.