Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice

[Release] FMETP STREAM: All-in-One GameView+Audio Stream (UDP/TCP/WebSockets/HTML)

Discussion in 'Assets and Asset Store' started by thelghome, Apr 30, 2019.

  1. virtualjay

    virtualjay

    Joined:
    Aug 4, 2020
    Posts:
    66
    We had a couple of interns for a while that used Macs. We're still cleaning up their line endings. :D
     
  2. Cloudless-Rain

    Cloudless-Rain

    Joined:
    Jun 24, 2015
    Posts:
    12
    I think I'm going to buy this asset but it's borderline, I'm a little nervous.

    My main use is for video chat. Agora isn't quite right, users need to enter a session, I'd rather just have a webcam texture with audio, not sessions. Plus Agora is potentially expensive; you have to use their server service in addition to your regular server for other uses.

    I looked at just doing it by hand with Mirror and webcam textures + audio, but there's tutorials or anything for that and I'm not a strong enough programmer to go into that with confidence.

    FM Stream looks perfect but it could be better. I'd be more confident if there was tutorial for integrating the encoders with Mirror. Also, when I buy an asset I always look for a strong community on Discourse or similar user forums, which FM doesn't seem to have.

    Anyway, I'm not writing this to criticize the asset or company, I just though it might be useful to see what a perspective customer is watching for.
     
  3. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    Thanks for sharing your perspective to us, and we totally agreed your points.
    We'll surely find a better way to gather all discussion and questions from all users, which can help each others to troubleshoot immediately, instead of emails.

    At the moment, we have a small facebook group for this purpose.
    https://www.facebook.com/groups/275354490106676

    We sometimes post plugin related topic, including coming features, testing, and bugs discussions. But it's still in a small scale.
     
    Cloudless-Rain likes this.
  4. joan4d

    joan4d

    Joined:
    Aug 8, 2019
    Posts:
    3
    Hi, i need help for this project. I need use OBS player for streaming live (video and audio) on the unity, for android and WebGL.With FMETP STREAM is posible? thanks
     
  5. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    I think it's a duplicated question from your email, I tried to re-cap my email's answer here:
    FMETP STREAM doesn't support OBS player directly, though you can still capture Win10's desktop into Unity without system sound.

    Edited: there are some softwares can redirecting system sound into Unity as Mic Input, then it may work as well.
     
    Last edited: Nov 13, 2020
  6. pranvokua9

    pranvokua9

    Joined:
    Feb 5, 2018
    Posts:
    5
    Hi, I want to stream UDP from OBS to Unity for a project to be deployed on the Hololens 2, is this package capable of that?
     
  7. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    Please be reminded that it’s not a wrapper for OBS.

    but if you are streaming motion jpeg in udp without sound, then it might work. As we don’t have h264 stream decoder for this.
     
  8. adiashIntu

    adiashIntu

    Joined:
    Aug 13, 2018
    Posts:
    4
    Hi with this plugin is it possible to stream to a unity app on another PC with IP? Also in addition to stream is it possible to stream other data?
    Thank you
     
  9. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    For Internet Streaming, please refer to this old demo. and you can send byte[] or string in bidirectional.
     
  10. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    [v1.329] submitted to store
    -bug fixed: Panorama Capture mode bug on build.
     
  11. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    I want to stream the screen of HoloLens2 to Windows10.

    https://www.dropbox.com/s/ub54pomjwi6zklv/Test BG Debug.mp4?dl=0
    => test video Link

    Windows10(Server)-[Demo_FMNetworkStreaming] Scene
    HoloLens2(Client)-[QRCode Demo] Scene

    I checked that streaming works well, but there was a big error.

    The background, other than UI and controller rendering, remains black.

    What settings do I need to set up for real-world cameras to be streamed in the background?

    I have confirmed that both MainCamera mode and Full mode are not available.

    The streaming background on HoloLens 2 should be displayed as a webcam (real world) background, not black.

    Among the things I tested, I used the "[ Enable for Test] WebcamDemo" object in the Demo Scene, but the symptoms were the same.

    Is there a way to display the webcam as the background in HoloLens 2?
     
    Last edited: Nov 24, 2020
  12. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    it's a known issue since Microsoft updates.
    There are some webcam stream/access issues between Unity3D and UWP.

    Some customers implement their own solution to get the webcam back in HL2 with FMETP STREAM.

    There was a solution replied by Microsoft, but I didn't have a chance to try it yet.
    https://docs.microsoft.com/en-us/wi...and-apis/mixed-reality-capture-for-developers

    Another solution is via Vuforia class, which should work, but no guarantee since so many months.
    https://stackoverflow.com/questions/58499314/hololens-how-to-get-webcam-texture-2d-from-vuforia

    Sorry for any inconvenience caused, and hope this can be fixed in newer version of Unity in future.

    Edit: thanks for one of customers(Chris), who shares below solution on HL2:
    "unity web texture loaded with a mesh plane. It works as well with latest unity 2019.4.14f"
    and you will need to set a separated rendercam with GameViewEncoder: RenderCam Mode.
     
    Last edited: Nov 24, 2020
  13. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62

    Thank you for the quick reply. This guide gives me hope.

    When setting this mode after proceeding with the guide [GameViewEncoder: RenderCam Mode]

    https://www.dropbox.com/s/ub54pomjwi6zklv/Test BG Debug.mp4? dl = 0
    Basically, the general objects featured in the link above should appear well.

    At the same time, it should be streamed using the HoloLens 2's webcam as the background.

    Is it possible if I set it according to the guide?
     
  14. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    You may follow below setup, but we haven't tested further on HL2 webcam yet.
    1) create mesh and assign webcam texture manually yourself, and fill it in the far clipping plane distance.
    2) create a layer and assign it to your webcam texture mesh, let's say "BackgroundCam"
    3) in your HL main cam, please ignore "BackgroundCam" layer
    4) create a render cam, and parent it to your main cam, and render everything including "BackgroundCam" layer.
    5) add a script "TargetProjectionMatrix" on render cam, and refer to your main cam. This will help you match HL2's projection matrix.

    If you succeed, please let us know too :)
     
  15. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    Unfortunately, one of the default settings, HoloLens2's Webcam failed to read.

    <Demo_FMNetworkStreaming> Demo scene <[ Enable for Test] WebcamDemo> I copied and set the game object configuration as it is, but the following error occurs on HoloLens2.

    [Error Log]: Error: Operation has failed with error 0xc00d36e6: The requested attribute was not found.
    (C:\buildslave\unity\build\PlatformDependent\MetroPlayer\MetroWebCamTexture.cpp:WebCamTexture::Update at 613)

    What could be the reason?
     
  16. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    For troubleshooting, would you please try Unity's default example for getting webcam texture in unity 2019.4.14f?
    https://docs.unity3d.com/ScriptReference/WebCamTexture.Play.html

    Our FMETP STREAM webcam example script is checking all available webcams, not sure if this is allowed on HL2.
     
    Last edited: Nov 26, 2020
  17. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
  18. jyp3505

    jyp3505

    Joined:
    Nov 30, 2020
    Posts:
    4
    Is it possible to stream game view to mobile in non local network?
     
    D3m0n likes this.
  19. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    We have FMWebSocket demo for Internet Streaming, which should also work for public network without internet.

    Other alternative networking systems like Photon, Mirror and UNet..etc should still be compatible via their bytes streaming feature.
     
  20. jyp3505

    jyp3505

    Joined:
    Nov 30, 2020
    Posts:
    4
    I mean i wanted to stream game view implemented from my PC to Mobile of external network without port forward.
    it is possible?
     
  21. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    In this case, you will have to host your node.js server via other cloud service provider. like Amazon..etc.
     
  22. jyp3505

    jyp3505

    Joined:
    Nov 30, 2020
    Posts:
    4
    um...
    If I use port forward, is it the same?
    Should I host a server like Amazon?
     
  23. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    In general, you have to connect to a node.js server for communication between any network.
    The node.js server can be hosted locally with port forwarding, or a cloud server with Internet.

    It depends on your project scale and budget, either of above solution will work for your case.
     
  24. Silverhook

    Silverhook

    Joined:
    Jul 15, 2020
    Posts:
    1
    I'm thinking about buying this asset, but first I need to know if this asset has the capability I'm looking for.
    I need to display in my app a live camera feed (live highway, street, and shopping mall cameras).
    My setup consists of an Android/iOS Unity app, NodeJS server, and a Raspberry Pi hooked to a camera, where the app should display the live video stream captured by the RPi camera.

    1. Can this be done with this asset?
    2. Do you have any code examples for achieving this goal?
    3. Can UI elements can be displayed on top of the live video stream?
     
  25. jyp3505

    jyp3505

    Joined:
    Nov 30, 2020
    Posts:
    4
    Thanks for reply.
    You were a big help to me.:)
     
  26. nsmith1024

    nsmith1024

    Joined:
    Mar 18, 2014
    Posts:
    866
    Hi, i just saw your product on Unity Asset store, and I want to buy, but first have question...

    While in my Unity Game, can i stream live from the phones (Android/IOS) CAMERA AND MICROPHONE to a server, and then others (WebBrowser/Android/ios app) can connect to the server to view the live video and audio?

    Also I need it to work with multiple streams, so many different people can be streaming to different set of people at the same time. Meaning person A can be streaming to person B,C,D,E,F while at the same time person X is streaming to persons J K L M. So people should be able to select which stream to watch based on the URL or something like that.

    Basically a webinar type setup where one person streams their phone camera and mic, and others can listen. But many streams from different people coming in at the same time, going out to different people as well as described above.

    Does it use HLS, or DASH, RTMP, or some other streaming LIVE streaming protocol?

    Thanks
     
    Last edited: Dec 9, 2020
  27. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    FMETP STREAM allows you to stream Game Views between Unity apps.
    For your case, it seems likely you are looking for some IP cam solution instead.

    We did some experiment for StereoPi for VR streaming, but I have to wait for their official announcement before telling you script details.

    The good news is, we have MJPEG fast decoder in the package. As long as you have mjpeg stream from RPi to Unity, our decoder can help you do the rest.

    Once you receive live video within Unity, we have a "Full Screen" capture mode, which will capture with UI canvas. Then, you can stream the captured result to your node.js server via socket.io with ease.
     
  28. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    You can stream your webcam & mic to others with our asset.

    But, according to your large scale streaming need, our local hosting FMWebSocket solution might have performance impact.

    Photon with FMETP STREAM might be an alternative, as they have GameRoom system with UDP protocol option, which should fit your use case. It's been tested by other customers, though we didn't include Photon demo scene.

    We are still working in progress for standard live streaming platform support in future, but still need more time and some hard work.
     
  29. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    Hello I am using this asset well.

    In the current demo scene, we succeeded in receiving a string value using <Receiver> in FMNetworkManager.cs.

    However, I want to call the same function from the server to the client, not to receive the string value when the UI button is pressed.

    How can I implement it?
     
  30. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    We provide option of SendToAll, Server, Others.
    Could you please specify your issue?

    I assumed that "SentToOthers()" would be your choice, if you don't want to receive the string sent from yourself.
     
  31. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    Hello I am trying to use microphone streaming.

    The volume of the mic is very low. Is there a way to increase the volume of the mic on the cord?

    (This is a situation where you can no longer turn up the volume on the device.)
     
  32. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    We don't have this option in current version.
    But since our captured data is in PCM raw data, you may hack our MicEncoder a bit, by multiplying the value of raw data.
     
  33. asdferasv

    asdferasv

    Joined:
    Dec 24, 2020
    Posts:
    8
    Can I sync the Win10 Monitor to HoloLens2?
    I need to share my desktop on Hololens2 in real time. All the methods I have found are about synchronizing the HoloLens2 screen to the PC side instead of the PC side to HoloLens2.
     
    Last edited: Dec 24, 2020
  34. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    Yes, it should be compatible.
     
  35. asdferasv

    asdferasv

    Joined:
    Dec 24, 2020
    Posts:
    8
    Thank you,I will try it
     
  36. alexisholgado

    alexisholgado

    Joined:
    May 15, 2019
    Posts:
    3
    Hello! I am trying to stream a video from a webcam into Oculus Quest 2. I've bought the package but I'm not sure from where to start now. I am guessing: create a UDP client in Unity (which will be deployed to Oculus) and a UDP server from the PC with the webcam. Is this correct? If so, is there any example code for the UDP server that I need to implement in the PC?
    Thanks!
     
  37. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    For beginning, you can first try our demo scene "Demo_FMNetworkStreaming".
    The core components are FMNetworkManager, and GameViewEncoder/Decoder.
    GameViewEncoder -> bytes -> FMNetworkManager -> bytes -> GameViewDecoder

    you may also refer to this AR tutorial, similar setup should also work on VR headset.
     
  38. alexisholgado

    alexisholgado

    Joined:
    May 15, 2019
    Posts:
    3
    I will try, thank you! And thank you very much for the super fast reply!! Happy New Year!
     
  39. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    Happy New Year! We just want to make sure customers saved their time, by rapid reply :)
     
  40. onecrane

    onecrane

    Joined:
    Mar 19, 2013
    Posts:
    5
    Just bought this today, and can't get the demos to work consistently. The echo test works fine. But the Demo_WebSocketNetwork scene claims that it can connect, whether to my localhost running the NodeJS example, or to my Amazon EC2 instance running the same - but neither socket server shows any console confirmation of a connection, and even with multiple applications claiming to be connected, there is no evidence that any information is being transmitted. Did I miss something obvious?
     
  41. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    I guess it's a known issue of socket.io version, and here is a temporary fix.
    Please type this command in your terminal, for reinstalling the correct version.
    npm install socket.io@2.3

    hope this can solve your problem.
     
  42. iMagesBlues

    iMagesBlues

    Joined:
    Mar 18, 2013
    Posts:
    17
    Hi, I just purchased this asset and I can happily say it works really well. I'm just currently having trouble with Websockets on WebGL mobile and getting the following error:
    ReferenceError: io is not defined

    It runs perfectly on desktop browsers and standalone though. Did I possibly miss a step or something?
    Thank you in advance.
     
    Last edited: Jan 4, 2021
  43. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    It seems likely the wrong IP input, please check if you are testing with localhost on your desktop browser? If yes, you should change it to your node.js server IP.

    If your mobile is on public network, please try forwarding your port to your node.js server IP.
     
  44. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    Please tell me how to reconnect if connection is lost during UDP connection.

    The configuration is as follows:
    Server: UWP (HoloLens2)
    Client 1: UWP (HoloLens2)
    Client 2: PC (Windows 10)
     
  45. alexisholgado

    alexisholgado

    Joined:
    May 15, 2019
    Posts:
    3
    Hello Again!

    I've made progress based on your previous explanation. I have 2 unity projects, one handling the webcam (sending into a RawImage) and the other one on Oculus. When I run both on my computer, the image from the webcam is transferred between projects, however, after deploying the app into the Oculus (Oculus Quest 2), the image is not shown. Is there any permissions or flags I should set in the manifest to allow the connection of the FMNetwork package? If that is not it, what else could be the cause of this problem? Thanks in advance!
     
  46. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    In general, you can simply disable and re-enable the GameObject with FMNetworkManager.cs.
    It will automatically stop and clear all previous connection, and try a new connection.

    If you have some unknown errors related to connection, please check if you have duplicated Label ID for Encoder/Decoder between those devices. Each Label ID should work as a pair, if there are more than one pair, the received data will be interfere though it's still connected.
     
  47. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    If you are working on PC, please don't forget to disable the firewall as possible. and please try enabling the network discovery feature in your Win10 OS, which should be under networking settings in OS.

    This is the most common situation, and solution.
     
  48. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    Thank you for quick response!

    I want to connect two platforms of HoloLens2/PC (Windows 10) in the same Wi-Fi environment. How do I do it?
    (When I tested it on Android and PC, it was well connected.)
     
  49. thelghome

    thelghome

    Joined:
    Jul 23, 2016
    Posts:
    588
    below is my suggested setup:
    FMNetwork Server: PC
    FMNetwork Client: HL2

    some tricks, it might help:
    -Do not connect your USB cable to your PC when running the app
    -Try switching on/off "UseMainThreadSender" on all platform
    -Try remove all "AudioEncoder/MicEncoder", see if it's related to network traffic..etc

    For technical support, I recommended that you can write us an email: thelghome@gmail.com
     
  50. iMagesBlues

    iMagesBlues

    Joined:
    Mar 18, 2013
    Posts:
    17
    Solved it, thank you for the suggestion. My google app engine server was unreachable due to a DNS issue with my ISP provider. (Still very weird it only happens on mobile). The error was:
    DNS_PROBE_FINISHED_NXDOMAIN

    Got it to work by manually setting a private DNS (dns.google) in Android connection settings, and (8.8.8.8) in iOS
    Now everything works in all platforms - WebGL (Desktop/Mobile), iOS and Android

    Thank you for such an amazing asset!
     
    thelghome likes this.