[Release] FMETP STREAM: All-in-One GameView + Audio Stream + Remote Desktop Control(UDP/TCP/Web)

I've added the diagnostics text to the HL app and it's saying 0 connections.

 

Next up I've hard coded the IP address of the server and the HL app now continuously flip flops between IsConnected True and false. When it's true it increments the last send numbers. The server doesn't show any signs of a connection.

 

I'd love for this to work, but I'm I just can't seem to. I think it would be really helpful to setting on the required settings and to explain why. E.g. do you need that specific target version of Windows and why? Does it needs to be at that bit depth or is this just what you're using here? Etc. Ahh, it's a shame I can't get it to work.

 

No problem, we updated the ReadMe file on the github page too.

//testing Unity Version: Unity 2021.2.7f1
//testing environment: local network via wifi
//system: windows 11 (firewall off), hololens 2

There are 3 demo scenes inside "_Scene" folder:
Demo_FMNetworkStreaming_PC -> This is the Viewer app(as Server)
TestScene_UDP_Client_MainCam -> This is for HoloLens 2(as Client), streaming Live View with webcam texture via Main Camera
TestScene_UDP_Client_RenderCam -> This is for HoloLens 2(as Client), streaming Live View with webcam texture via Render Camera

Step 1: download the template from github
Step 2: import the latest FMETP STREAM from package manager( which you purchased on Unity Asset Store)
Step 3: Switch Build Platform to Universal Windows Platform
Step 4: Select either "TestScene_UDP_Client_MainCam" or "TestScene_UDP_Client_RenderCam" for HoloLens UWP Build
Step 5: Depoly the UWP Build via Visual Studio
Step 6: Run the "Demo_FMNetworkStreaming_PC" on Unity Editor, which is the viewer app

 

Latest version still has the "FullScreen" issue on my side. MainCam and RenderCam don't give image on the HL2 as client sending to the PC (as server), but FullScreen mode does. MacOS is still not working and the firewall should not be blocking anything. Localhost on macos works, it is blocking the IP bc yes?

 

Registration window still blocks my unity editor after authentication.

 

It works for me on W11 and W10. Check you are adding the "IN". It didn't work for me otherwise. You can use the asset without the registration anyway x)

 

manurocker95 thx
The invoice number has been added but the registration window still pops up when the Unity editor plays.

 

+1.
I have the same problem.

 

@iiiVSLAM @Kikki1024 Would you mind sending us your invoice number via email? We could help solving this issue on our side. our support email: thelghome@gmail.com

PS: And please keep your invoice number private, and don't post or share it to others.

 

That's not a normal/expected behaviour, if you are running our template with same or above Unity version.
If you have stream of "Full screen" mode, that means the connection of wifi should be fine.
-> You may check your HL2 UWP version, and try to update it to the latest. This way automatically fixed many unknown weird UWP bugs, like webcam issue..etc.

In local network, you may test between Mac and PC, or mobile. If they couldn't find each other, it's 99% related to firewall on your computer, or router. For example, some business/school router may have some security options for blocking UDP broadcast data. Normally, it won't be a concern for home wifi with default settings.
-> But anyway, please check if there is any HL2 UWP OS updates, we noticed that some weird behaviour if you didn't update the HL2 UWP OS for more than 1 year. It could be some hidden permission compatibility with the latest Unity version.

 

I'm using my custom scene and not the template in 2021.2.17f1. and latest update in HL2. Both encoder and decoder have the same label.

In addition, since version 2.421 the server keeps connecting and disconnecting all the time. This happens in HL2 and Oculus Quest 2.

 

For the ease of troubleshooting, would you mind to give a try on our template?
Or if you have a demo scene for us to reproduce the bug on our side, we will help investigate.

 

the weird thing is it works just fine with Android APK so it may be e

I made a fresh project and it worked. Sorry for the inconveniences going and coming xD

 

Hi, I have a quick question : I have 3 devices which are connected together using UDP.
Device n°1 sends video and audio stream to device n°2 & n°3, while device n°3 sends string information to device n°1.
For performance reasons, I'm wondering if it would be better for device n°1 to act as a server for video and audio stream, and as a client for string messages received from device n°3.
In which case, I would have 2 FM Network Managers (one server and one client) in my Unity scene on the app running on device n°1. Would it be a problem? Would I have to change the Listen ports in the Server & Client settings to avoid any conflict?

 

You could have multiple FMNetwork with different listen ports.
In one server-clients setup, client's message will always redirect to server first, for the safety of the data.

 

Awesome and good to know that it's been solved!

 

For the next version could you add namespace and/or assembly definition? <3

 

We may consider your suggestion for next major updates, which would be during the early summer as possible.

 

Btw, I may have found a memory leak. Opening the app (with the manager set as client + encoder and connecting to the computer by wifi) on quest and hololense 2, then closing the app and repeating this a few times makes both devices to get stuck and require a complete force restart. This does not happen without FMETP so... any idea? Should we manually disconnect the manager services by code on quit?

 

Do you have any log or warning message?
This seems likely an unexpected behaviour when the app pause and resume many times.

We are working on major updates for FM Network.
You may write us an email, we could provide the latest beta for your testing purpose.

technical support: thelghome@gmail.com

 

Haven't displayed logs on standalone. Gonna send you an email to test it!

 

Hmm, tried to connect the oculus to the steam deck (running linux), the manager says it’s connected but there’s no image on the deck. Checking logs, no error log is displayed. Any idea?

 

We don't have a chance to test on Steam Deck. Is it possible to install Unity Editor on it, for testing?
If it's connected, the networking part should be fine.

You may try different settings in decoder first, for the ease of troubleshooting.

 

hm, maybe I'm doing something wrong because the default scene works crossplatform between Windows, Mac and Linux (even on the Steam Deck), but if I add:

- Windows:

GameObject1:

- FMNetwork->


- Camera (Decoder):



and on SteamDeck:

- NetworkManager



- Camera (Encoder):



This doesn't work for no reason.

 

Hi. I also encountered this problem, the audio delay keep increasing on Android, up to 3~5 seconds.

I set the channel as mono, sample rate as 11025, and set DSP buffer size as Best Latency.

Is there any other advice? And do you have plan to support native audio playback solution? Thanks.

 

game view encoder and decoder not working even on demo scene

 

the encoded preview works but it's not decoded

 

Extra note: game viewer doesn't work if the receiver is not the first client to connect. If the server or the clients disconnect, the first one doesn't send/receive the game view either after reconnecting.


1001 and 1002 as label seem to not work but 8001 as the demo worked. No idea why.

 

That's a strange behaviour. Sometimes the networking issue could be fixed by restarting the router. We had similar issue before, when we are doing heavy reliable data stream testing.
(We sent tones of byte[] in very short period of time, and our router buffer stuck.. and can't release until restart the hardware).

For the label ID, probably 1001 or 1002 was occupied by other encoders in your scene, which may feed the wrong data into your GameViewDecoder.

You may also compare our beta & store version of FM Network UDP. As our beta version is still in development, it might have bugs as expected. But overall, it's an improved version with more features.

 

Are you using FMNetworkUDP, or FMWebsocket?
Since FMWebsocket is TCP, which will accumulate the latency if have unstable connection between node.js server and your device.

We will investigate the possibility of native audio playback, but it will take some time for cross-platform support.

 

We are using websocket.

OK. Thanks for your response.

 

@thegholmes
I've hosted TestServer_v2.0.0.zip to the Herokuapp server. Now I am trying Demo_WebsocketStreaming with Heroku app server HTTPS URL in FMSocketIOManager. When I am trying to run the app, the socket is unable to connect. In localhost, it's working.

Is there any modification required in TestServer_v2.0.0.zip before hosting at the Heroku app server?

My HerokuApp hosted URL is looks like : https://***********.herokuapp.com/ with port : 50273

 

I want to display my multiple oculus camera views on my website dashboard admin panel.
My web dashboard server is hosted at AWS as well as the Heroku app.

In localhost, the environment works properly but does not work on a real server.
Does FMETP STREAM V2 work in this scenario? Can this work on the real server like aws? not on localhost.

 

Hi team.

About "FM Network UDP".
Is AutoNetworkDiscorvery not supported on Windows (server) <-> Android (client)?
Or are there any precautions or settings?

The Android (client) Server IP is always specified as "127.0.0.1" and I cannot connect.
All Windows firewalls are disabled.
The FMNetworkUDP settings remain the default except for server / client switching.


In addition, Windows (server) <-> Windows (client) has succeeded in connecting.
Also, Ping communication is OK on Windows (server) <-> Android (client).
The Windows (server) Connection Count counts up.
On Android (client), you can connect by turning off AutoNetworkDiscorvery and setting a fixed IP address for Server IP.

Version:FMETP STREAM V2 ver 2.422 April 07,2022
OS1:Windows10 Pro
OS2:Android
Unity:2021.3.1f1

 

As our default node.js example is the minimal script for http test, you have to add your own ssl certification in script for ssl support.
If it works properly, the html demo should be reachable too.

 

Yes, we tested on AWS Windows Server.
The setup is almost the same as our local hosting tutorial, the only trick is enabling the in-out bound firewall rules, and make sure the public port is reachable.

 

This is not an expected behaviour, it might be hardware specific issue as there are too many Android specs.
If you purchased recently, please reach us via email: thelghome@gmail.com
It would be easier for us to follow your case.

*We are working on some major upgrade about FMNetwork, we could share the beta version for your quick test.

 

Thank you for your reply.
I accepted the inquiry by email.
I will send you an email.

In addition, there are corrections regarding the environment, so I will describe them here.

<×>
OS1: Windows10 Pro
OS2: Android

<○>
OS1: Windows10 Pro (Unity Setting: Windows 64bit Build)
OS2: Windows10 Pro (Unity Setting: Android 64bit build)
or
Blackview Tab6 Android 11 (actual machine)
Kernel Version: 4.14.193

 

Awesome, we received your email.

 

I want to end an existing stream and then restart it later in my game. Will FMETP Stream continue to Broadcast/Receive data if the GameObject it lives on is disabled, or no? I'm wondering if I need to call a method to stop the server/client or if this will happen automatically when disabled.

 

The Encoder will stop by default, when it's disabled.
Or, you can set the StreamFPS to zero, and call below function manually whenever you need an update.

Code (CSharp):

1. GameViewEncoder.Action_UpdateTexture();

 

Oh, so in the solution above you would sort've manually transmit every frame? Is that right?

In this case does the receiver need to be stopped? Or is it passively receiving data and only working when the encoder is transmitting?

 

The decoder will only process the data when receive the stream with paired label ID. And yes, it’s passively,

 

Hello,

I am trying to add FMETP Stream 2 to a hololens2 unity project. However, After I add it and change the build settings to match those suggested in your Youtube Video (

), I just get a blank screen after deploying to hololens 2. The debugger has a number of exceptions in the MRTK files.

I am only hoping to use the synchronized game objects and microphone audio stream, not the live view video stream. Is there a scaled down version of your product that supports those things?

Thank you in advance for your help.

 

For your first HL2 test, which you may try our github template:
https://github.com/frozenmistadventure/fmetp_tutorial_hololens2_stream

When you import the plugin, please untick the project settings and don't override the github template's build settings.
The target minimal visual studio version should be within 10.0.10240.0 ~ 10.0.18362.0.

Once you get the live view stream, you can add "MicEncoder" on Sender, and "AudioDecoder" on receiver side.
The overall setup will be like this:
MicEncoder(Sender) -> byte[] -> FMNetworkManager -> byte[] -> AudioDecoder(Receiver)

You may please feel free to reach us via email for technical support: thelghome@gmail.com

 

Thank you very much for the fast response. I will try it.

 

It is working for me now. Thanks very much!

 

Awesome, glad to know it works!

 

Hi... is it possible to have description of which scene is doing what in demo project. I am exploring the demo and its difficult to find what scene is used for what.