Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[Release] FMETP STREAM: All-in-One GameView + Audio Stream + Remote Desktop Control(UDP/TCP/Web)

Discussion in 'Assets and Asset Store' started by thelghome, Apr 30, 2019.

  1. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    476
    Thank you for the reply. I considered that as well, but I use the server to control the camera's properties (focal length, focus, etc) and display it on UI on the client application, so using a secondary camera won't exactly work.

    Is there no way to adjust the GameViewEncoder script to work with my use-case?
    Like I said, I need the RenderCam to go to a targetTexture for use in UI and stream it to the server.

    Thanks so much for the amazing support! :)
     
  2. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    We released V2.210, which can reserve your existing RenderTexture without much impact on performance.
    PS: Hope it may help your development, and wish you may also consider writing us a review on store in the future if it helps. :)
     
  3. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    476
    Wow, that fixed it and works great! I left a glowing review, because this is incredible :)
    I'll share that I did use V1 for a little bit but the latency was too much so I switched to an overcomplicated WebRTC setup on my local network. It worked, but was a nightmare. I saw that V2 was out and after giving it a try for a day, I'm working on switching over to it 100% :D
     
    thelghome likes this.
  4. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    476
    Sorry for the bunch of questions, but I'm actually running into one more issue.
    I want to force disconnect the client from the server.
    I tried this line below, but the client still shows a connection running.
    networkManager.Server.Action_CloseClientConnection(networkManager.Server.ConnectedClients[0].IP);

    Action_Close() does work, but I don't want to close the entire connection, just force disconnect from the client. There's only ever one connected, so I believe the line above should work...?
    Thank you again!
     
  5. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Only the server has permission to call below command, and we also verified on our side.
    Code (CSharp):
    1. //Server disconnect Client remotely, with target client's IP for example "192.168.1.123"
    2. FMNetworkManager.instance.Action_CloseClientConnection(FMNetworkManager.instance.Server.ConnectedClients[0].IP);
     
  6. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    476
    Sorry, maybe I was unclear. From the server, I am trying to close client connections without completely closing the Server's connection, so if desired, a client can connect again. The line above is being called on the server, but for some reason isn't working. The server acts as if it's disconnected, but the the client shows that it's still connected to the Server.
    Strange, maybe I'm doing something wrong. My code is exactly as yours is.
    In any case, it's not a big deal. Thanks again!
     
  7. halilzeybekx

    halilzeybekx

    Joined:
    May 11, 2020
    Posts:
    4
    @thelghome Hi, I'm trying to implement cross-platform screen sharing by using Photon's PUN2 and thinking about purchasing this asset for screen capturing part. I want users to be able to share their desktops screens with others If they are on the Windows or If they are on the android/IOS they should be able to share their phone's screen with others (something like zoom). Can I achieve that goal with this asset?
     
  8. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    We reviewed the script again, and noticed that's a bug of syncing connection status on client side.
    V2.211 should solve the remote client disconnection bug.
     
    JudahMantell likes this.
  9. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    In the current version(V2.xxx), only Windows Desktop stream is supported. You can share webcam on mobile too.
    Desktop streaming on Mac OS and Linux will be coming soon in next major update, estimated before or between the summer this year.(but not 100% confirmed yet)
     
  10. JudahMantell

    JudahMantell

    Joined:
    Feb 28, 2017
    Posts:
    476
    Hm... that issue looks like it was fixed, but now the client isn't even registering as being connected. The server shows connected, but client doesn't and therefore nothing is being sent over from the gameViewEncoder.

    I have checked all settings to make sure they are all correct, and this issue still persists when using the same PC for both client and server, or two different devices. Any help would be appreciated.
    Thanks so much for all of your assistance so far!
     
    Last edited: Mar 3, 2022
  11. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Could you write us an email, which we may send you a beta version with the latest version of FMNetworkUDP?

    We are testing different use case at the moment, and probably will release a major update for the networking system.

    technical support: thelghome@gmail.com
     
    Last edited: Mar 3, 2022
    JudahMantell likes this.
  12. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    Before purchasing, I'm wondering if I can stream the game view in standalone (build) to other build or web browser. The hololense 2 remoting works awefully and this would be the replacement.

    Thanks in advance
     
  13. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Thanks for considering our solution.
    Yes, it's cross-platform streaming solution for almost all Unity supported platforms. It's very easy to stream the GameView to other devices, including WebGL and HTML5 javascript demo.
     
  14. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    Perfect! I will purchase it asap so i can test it out!
     
    thelghome likes this.
  15. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [V2.300] Release notes
    - FMNetwork Optimisation & Connection Bug fixes
     
    BreynartStudios likes this.
  16. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
  17. MagicK47

    MagicK47

    Joined:
    Sep 5, 2017
    Posts:
    30
    Is there a way to remove the camera's background and send it? I want to render only the game objects, the background color of the camera and sky box not rendered or transparent, and the receiver only sees the game objects and content
     
  18. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    You did similar experiment before, with masking solution. you may refer to this post and get some idea.
    https://twitter.com/LeosonCheong/status/1266787769948368897
    Screenshot 2022-03-14 at 9.05.38 PM.png
     
  19. halilzeybekx

    halilzeybekx

    Joined:
    May 11, 2020
    Posts:
    4
    I'm already using Photon with another desktop streaming solution however It's really hard transfer large amount of bytes(for exp;1920x1080 texture) with PUN2 at once. If I use this asset is it going to solve that problem?
    I mean will I be able to stream my desktop to others smoothly via PUN2?
     
  20. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Our encoder can split the data into small chunks for the stream.
    There are few variables like compression quality, and downscale resolution and stream fps may help a little bit.

    As far as I know, it also depends on your PUN2 server location and bandwidth.
     
    Last edited: Mar 15, 2022
  21. halilzeybekx

    halilzeybekx

    Joined:
    May 11, 2020
    Posts:
    4
    I don't necesserly need to stream desktop view to the web right(this is what I saw in the your video)? Can I stream it to Unity and use it as a texture by using this method in the link? https://frozenmist.com/docs/apis/fmetp-stream/pun2-example/
     
  22. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Yes, the whole setup will be like this.
    GameViewEncoder(DesktopCapture) -> byte[] -> PUN2(SerialStream, RaiseEvent, or RPC) -> byte[] -> GameViewDecoder -> Texture
     
  23. halilzeybekx

    halilzeybekx

    Joined:
    May 11, 2020
    Posts:
    4
    Thank you for your replies. One more thing, If streamer in the Windows with Encoder, even tho desktop sharing is not supported yet in Android/IOS, those people in Android/IOS still can still Decode the things that person streaming from his Windows sends right?
     
  24. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Correct, there is no limitation on the viewer app.
     
  25. Abdul_Malik_Badshah

    Abdul_Malik_Badshah

    Joined:
    Oct 28, 2016
    Posts:
    13
    @thelghome
    just purchase the asset. I want to do a Video Chat between client and server but instead of the Server sending the Webcam video I want the Sever side to send the Game scene view and the client side as normal sending the webcam video.
    In Simple, a video chat between server and client where the server sends the Gameview.
    Please if there is any Demo plz let me know or just guide me in the right direction.
     
  26. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Yes, it's possible. You may test our "Demo_FMNetworkStreaming" scene first. By default, you may choose either GameView or Webcam View for the stream, it also includes In-Game sound or mic stream.

    The only difference between FMNetwork demo and "Demo_WebSocketStreaming", is the network manager.
     
    Last edited: Mar 17, 2022
  27. HajiyevEl

    HajiyevEl

    Joined:
    Feb 19, 2020
    Posts:
    46
    Also just purchased the asset. Can i make video chat with it without server? Like P2P? One client just streaming to another client and vice versa?
    And please, i need video tutorial or at least some preferable steps for setting mobile video chat.
    Thanks!
     
  28. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Our example network systems are designed for server-clients.
    The encoder is a separate module from networking part. If you have P2P networking system, you may replace the networking demo.

    The general idea of the whole setup:
    GameViewEncoder -> byte[] -> Networking System -> byte[] -> GameViewDecoder
     
    HajiyevEl likes this.
  29. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    @thelghome since last update I'm getting

    Assembly 'Assets/FMETP_STREAM/FMCore/Plugins/FMTurboJpeg/FMTurboJpegWrapper/FMTurbojpegWrapper_Editor.dll' will not be loaded due to errors:
    Assembly name 'FMTurbojpegWrapper' does not match file name 'FMTurbojpegWrapper_Editor'

    Assembly 'Library/ScriptAssemblies/Assembly-CSharp.dll' will not be loaded due to errors:
    Reference has errors 'FMTurbojpegWrapper'.

    I'm using 2021.2.16f1
     
    Last edited: Mar 22, 2022
  30. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Hi Everyone, we just released a quick fix v2.400 for a critical bug if you imported v2.380 & v2.382.
    Sorry for any inconvenience caused.
     
  31. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    No problem! Thanks for the quick fix. Confirmed now working flawlessly with Hololense 2 and Meta Quest 2. Good job!
     
    thelghome likes this.
  32. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    Small thing. What settings do you recommend for smooth display but maintaining good quality? 2048x2048 RT looks good enough but sometimes there are small framedrops. In addition, what should I do if my computer is connected by ethernet and the device by wifi? The Network manager seems not able to grab the connection
     
  33. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Could you please send us some screenshots of your current settings/setup via email? We may give you further advise, or beta version for performance testing.

    The auto network discovery is via UDP broadcast, ethernet may block or ignore the broadcast signal if your ethernet isn't connected to the router directly. In this case, you have to input the Server IP manually via FMNetworkManager component.
     
  34. paulioms

    paulioms

    Joined:
    Jul 29, 2021
    Posts:
    15
    Hi, I'm trying to follow the example for streaming from a Hololens 2. I've followed the video but I get the following error from the HL. Any ideas?
    Code (CSharp):
    1. 'B8G8R8A8_UNorm' doesn't support ReadPixels usage on this platform. Async GPU readback failed.
    2. UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
    3. UnityEngine.Logger:Log(LogType, Object)
    4. UnityEngine.Rendering.AsyncGPUReadback:ValidateFormat(Texture, GraphicsFormat)
    5. UnityEngine.Rendering.AsyncGPUReadback:Request(Texture, Int32, TextureFormat, Action`1)
    6. <ProcessCapturedTextureGPUReadbackCOR>d__72:MoveNext()
    7. UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
    8. UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
    9. GameViewEncoder:OnRenderImage(RenderTexture, RenderTexture)
     
    Last edited: Mar 24, 2022
  35. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    That's not happening for me but when streaming from the HL2 as server and encoder to the PC as client + decoder, the image is not updated. What Unity are you using?
     
  36. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    @thelghome some documentation should be added regarding this (yes it's in the video but it's not really intuitive):

    - Who should be the server (encoder or decoder)?
    - If the Network manager is client-> call with OnReceiveData to the decoder-> Process Image data
    - If the Network Manager is server-> the encoder calls the manager to SendToOthers

    Regarding to my issue, it is connected but no image is displayed:

    upload_2022-3-25_0-13-19.png

    Note:
    using the pc as server and HL2 as client works as long as it's fullscreen and not "main cam" as the tutorial states
     
    Last edited: Mar 25, 2022
  37. paulioms

    paulioms

    Joined:
    Jul 29, 2021
    Posts:
    15
    V2020.3.30f1
     
  38. paulioms

    paulioms

    Joined:
    Jul 29, 2021
    Posts:
    15
    A couple of additional points; If I run 2 instances of the unity editor then the "hololens" app works fine. If I turn off Async GPU Readback then the error goes away. However, when running on the hololen my PC server app does not receive a connection either via auto discovery or a fixed IP address.
     
  39. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    [New Template for HoloLens Users]
    We uploaded our previous testing template on github, for the ease of testing on HoloLens.
    https://github.com/frozenmistadventure/fmetp_tutorial_hololens2_stream

    @paulioms @manurocker95 As far as I remember, there is no limitation of either server or client, and the encoded stream should be bidirectional. It's most likely firewall issue on PC of blocking UDP packet if you have connection issue.
    When you have multiple stream, please check your label ID and make sure it's a unique pair for encoder/decoder.
     
  40. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    Thanks for grabbing this log info, it's very helpful for us to troubleshoot.
    It's obviously texture format issue "B8G8R8A8_UNorm", we will investigate this error on UWP asap.
     
  41. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    That's not the case as it was because of using "MainCam" instead of "FullScreen". Even if it's connected by cable and the IP set as the ethernet connection, there's no way to connect the HL2 (WiFi) to the PC (Ethernet)
     
  42. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    btw @thelghome it's stupid but you could add TMP_Text now that Text is set as "legacy" for the UIStatus
     
  43. paulioms

    paulioms

    Joined:
    Jul 29, 2021
    Posts:
    15
    FYI I also tested by connecting Hololens to Ethernet (with adapter). Update: The connection problems seem to be between HL and Win11, I'm getting a connection between HL and Win10...although on the PC client it seems to show the connection and then in less than a second it loses the connection. But on the HL server it's showing as still connected and sending bytes.
     
  44. manurocker95

    manurocker95

    Joined:
    Jun 14, 2016
    Posts:
    209
    Hololense to ethernet as well or HL2 to wifi and PC to ethernet? I'm also using W11 btw (in case that matters)
     
  45. paulioms

    paulioms

    Joined:
    Jul 29, 2021
    Posts:
    15
    HL to ethernet, WiFi off. [Edit] At least I have tested that. But I am normally using WiFi
     
    Last edited: Mar 25, 2022
  46. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    We just released an update (v2.420), which should solve the ethernet detection issue.
    We noticed that there was #if-else filter for standalone and editor version, now we include "WINDOWS_UWP" in FMETP STREAM v2.420.
     
  47. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    We can reproduce this error on our side, and it's now been fixed in FMETP STREAM v2.420.
    The HL2 performance improves too without this error.
     
  48. paulioms

    paulioms

    Joined:
    Jul 29, 2021
    Posts:
    15
    Getting broken script references from that project, looks like there are some OVR dependencies? [Edit] Unfortunately, it's a bit of a chicken and egg, without the package installed the references break. Which scripts are referenced by the FM managers in the scene?
     
    Last edited: Mar 25, 2022
  49. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    737
    This template is a public project on github, which we shouldn't include the paid asset.
    You may import the latest FMETP STREAM on your side.
     
  50. paulioms

    paulioms

    Joined:
    Jul 29, 2021
    Posts:
    15
    I appreciate that. Adding the package in does resolve the missing scripts. But I'm not sure how to use the demo. I assume you run the PC app as the server and either one of the MainCam or RenderCam apps on the Hololens? If so they do not seem to be connecting, the server isn't registering any connections.