[Release] FMETP STREAM: All-in-One GameView + Audio Stream + Remote Desktop Control(UDP/TCP/Web)

It seems like Unity3D misconfiguration after upgrading some project files.
Solution: Please try to delete your Library folder and manually refresh and reimport the packages.

the importing FMETP Stream package requests me to update package manager, try to press skip.
or, ignore our project settings when import, as it might be out-dated between Unity2018LTS & Unity2019. But those settings are not important.

In case that your problem exists, please feel free to email us: thelghome@gmail.com

 

I tried it and it gave me some new errors. I am also using Socket.IO for Unity, It might be the reason maybe? What can be done?

 

It seems some duplicated library issue between your Socket.IO plugin and FMETP STREAM.
We've also included socket.io support, we recommended that you should remove either your Socket.IO plugin, or remove FMWebSocket folder in this case.

 

Hi, i'm trying to stream windows 10 desktop (with monitor selection, and including taskbar and mouse pointer) to android in real-time over the local network and display at 30fps on an object in world space. I don't care about audio.

I came up with my own solution, two actually, but they're not very good. The first has over 2000ms of lag and streams at 30fps. The second has less than 50ms of lag, but only streams at 10fps.

I'm trying to get 30fps with less than 50ms of lag, so that the user can use the PC mouse and keyboard, with the 3d object on android acting as the monitor. I also want to send commands from android to the PC to change FPS and compression.

After skimming this thread, it looks like i can do that. Do you agree that this should work? Also, i'm not sure which version i need. Since i only need local network, can i use FM Network UDP, or is the desktop capture only available in FMETP STREAM?

Thanks

 

In order to solve your question, you may refer to our iPhone 5 x Win10 Monitors demo.
Things will also work on Android Devices.
PS: iPhone 5 is a very low-end devices, any new phones nowadays can work much better.

 

Okay thanks, that looks good. But can i do this with FM Network UDP, or is this only available in FMETP STREAM?

 

It requires FMETP STREAM for making this. Hope this can solve your concerns.

 

Okay thanks.

 

This is working well with my main monitor, but i'm having an issue with my second monitor.
FMDesktop.Manager.monitorCount returns the correct number of monitors, but FMDesktop.Manager.GetMonitor(1).state returns "unsupported".
Any idea why that would happen? I tried to investigate, but i don't see any source for the FMDesktop dll, so i can't see what's it's checking to decide if a monitor is supported.

edit:
I just discovered that it reads the monitor info about the monitor that the player is running on.
When i drag the window to the other monitor, i detect the resolution change and set FMDesktopMonitorID = 1; then it just shows a grey texture.
So now i just need to figure out how to get it to properly change the monitor that's being captured. I tried Action_ReloadScene(); and i tried disabling and re-enabling the FMNetworkManager game object, but no luck yet.

 

Hi. I am using socket.io for sending events. It used to work but after implementing video stream, it is not working anymore. What might be the reason, and solution? I am using this one for the socket.io https://assetstore.unity.com/packages/tools/network/socket-io-for-unity-21721

 

Hi, I'm looking to buy your asset. Quick question, Can I use it to stream my webcam footage and audio on a PC to a unity Webgl texture in-game? hosted on my website.

 

Would you please write us an email for technical support? we can follow your case easily.
This feature is using Windows Desktop Duplication API (DDA) in the backend and doesn't allows it to run app on discrete GPU.

There are some potential solutions and alternatives for this, which I could share more info to you via email: thelghome@gmail.com

 

There might be some duplicated libraries between FMETP and your tool.
We recommended that you should switch to FMWebSocket, as it has better performance and using newer version of websocketsharp.

Or, you should remove our FMWebSocket Folder completely as alternative.

 

FMETP STREAM allows you to stream any in-game video & audio between Unity3D builds.
For example, if you have a video source, you can setup a standard in-game camera to capture your video result for the streaming.

You can host on website, but you will need to host your node.js server, or cloud server, or Photon..etc as the middle part of communication. We do have step by step tutorial on your youtube channel for node.js local hosting. You may check them out and see if it fits your need.

 

I already deleted FMWebSocket. It was giving errors. but if it is easy to use I would like to try it out. Is there a guide for FMWebSocket so I can understand it.

 

We have few tutorials on youtube.
Functions are very straight forward, with FMSocketIOManager.instance.SendToAll/Server/Others()

# setup for live streaming part:

# or for customised socket.io event..etc

 

Okay thanks. I sent you an email about that, and a couple other questions about streaming desktop audio, and possibly connecting via USB cable instead of wifi.

Edit: I did a little research about DDA and found that it does support discrete graphics, but it cannot switch between discrete and dedicated gpu while the program is runnning. The program has to be run on the hardware you want to stream the output from. So if you have a PC with 2 monitors, and one is plugged into the onboard gpu, you can't switch between outputs while the program is running, but if you run it on the discrete gpu then it will work.

 

Hi, just got your asset from the asset store and was trying to find some doc to help my use photon as the network.
are there any demos or documents I can look at to try this? also all the demos i can find show just local networking so an not sure.

 

Thanks for trying our asset, you may follow below logic for your Photon setup.
GameViewEncoder -> bytes -> Photon -> bytes -> GameViewDecoder

Sorry for the lack of document in Photon part, as we didn't involve much tutorial for 3rd party networking solution.
But it won't be difficult to set it up yourself.

In case that you need further help, please write us an email for technical support.
thelghome@gmail.com

 

[ver1.319] Stream In-Game 360 Panorama View
Submitted and waiting for approval.
BTW, 3 mins step-by-step guide is ready now! you may check it out.
https://youtu.be/OLsrNnoRMJ0

 

Hi. I checked the url and it is correct but I have still getting this error. FMLog: Connection Execption: A series of reconnecting has failed.

 

Could you please write us an email for technical support? to: thelghome@gmail.com
It seems like that you have missing/wrong IP & Port for WebSocket.
You can always follow this tutorial guide:

and please don't forget to setup your node.js server for web streaming.

 

I sent an email.

 

I apologize in advance for my ignorance on the subject.

Whenever I see tools like this, I wonder if the authors recommend some kind of server. I mean, more than a specific brand, some characteristics.

I guess you can hire a dedicated server with a fixed IP. And I guess that server should allow node installation, among other things.

Is that so?

A server with a bandwidth of 500 would be enough ... how many concurrent users would it accept?
These things (the same thing happens to me with multiplayer tools) are the ones that I do not understand.

Sorry and thank you.

 

It’s a bit hard to answer you with exact number of users for some server specs.

But you can always estimate the limit in terms of zipped jpeg file size x fps x streams(count).

It’s a very dynamic & magical number as always. Resolution and compression quality, and content are all the key factors for bandwidth estimation. Some cheap servers may have performance issue, while good servers are costly.

We cannot guarantee for any server suggestion, as we are mainly working on functionality for C# Live Stream solution.

The network system can always be replaced and there are many alternative.

That’s also good for development with flexibility.

 

Thank you very much for your reply.

 

I's a great asset. Thanks for your work. I bought it, and using it for my project.
Do you have any plan for MacOS's Desktop Capture?

 

When I run Windows Capture demo, I got the warning like blow:
and I can't see the Preview in the Encoded area of GameViewEncode, we don't see the result on Client of couse.
Do you have any idea?
My OS is Window 10.

[FMD] 0:\\.\DISPLAY1 => Unsupported.

UnityEngine.Debug:LogWarningFormat(String, Object[])

FMDesktop.Monitor:.ctor(Int32)

FMDesktop.Manager:CreateMonitors()

FMDesktop.Manager:Awake()

UnityEngine.GameObject:AddComponent()

FMDesktop.Manager:CreateInstance()

FMDesktop.Manager:get_instance()

FMDesktop.Manager:get_monitors()

FMDesktop.Manager:GetMonitor(Int32)

<RenderTextureRefresh>d__50:MoveNext() (at Assets/FM_ExhibitionToolPack/FMCore/Scripts/Mapper/GameViewEncoder.cs:337)

UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr)

 

I think this is about GPU issue, which I will provide more info for you and reply your email soon.

This feature is using Windows Desktop Duplication API (DDA) in the backend and doesn't allows it to run app on discrete GPU.

PS: Mac OS Desktop support is in our To-Do list, it needs some time.

 

I'm a little confused about this. Have you done enough testing to be able to clarify what situations won't work?
I've done the following tests:

Desktop PC (windows 10 pro)
Radeon RX580 - 1 monitor (primary)
Intel HD Graphics 4000 - 1 monitor (secondary)
Primary monitor (dedicated GPU) works, secondary monitor (integrated GPU) returns "unsupported"

Surface Pro 2 (tablet - windows 10 pro)
Intel HD Graphics 4400 - built-in monitor
Works

Lenovo ThinkCentre M92p (mini PC - windows 10 pro)
Intel HD Graphics 2500 - 2 monitors
Both monitors work

Toshiba C855D (laptop - windows 8.1)
Radeon HD 6310 - built-in monitor
Does not work

Toshiba C855D (laptop - windows 10) - same machine as above, after OS upgrade
Radeon HD 6310 - built-in monitor
Does not work

So it seems that it requires an integrated GPU, but will work on dedicated GPU, as long as integrated GPU is also present. But it only works with the GPU that the primary monitor is currently connected to, even if it's not the integrated GPU.

Lincode can you report on your PC situation? I'm guessing you have a laptop with an nvidia/ati GPU and no integrated GPU. As far as i know, desktops without integrated graphics are pretty rare.

 

Hi again,

Small question:
How to capture a single snapshot with the 'GameViewEncoder' from code?
Reason: We stepped away from updating the stream in realtime but I want to send updates from the Client of the 'Render cam' at certain moments in my application to the server.
Right now I'm enabling / disabling the whole 'GameViewEncoder' script in 0.1 seconds in a coroutine, but there must be a nicer way

Thanks for any assistance!
Cheers

 

Yes, there is a manual way to do it.
First, you have to set the StreamFPS as 0 FPS.
And you can call "Action_UpdateTexture()" function manually for capture once only.

 

Has anyone been able to use this asset to connect a Unity WebGL game to a node.js express app from the command line with socket.io? I can only get it working with the Unity Editor and Windows App..

 

We have a demo WebGL Echo test connecting to echo.websocket.org
https://frozenmistadventure.github.io/demofmwebsocket_echo_webgl_ssl/

You may check it out. However, we don't have tutorial on node.js cloud hosting, as there are too many platforms & services providers.

 

How can we get WebGL working with regular websockets (ws://)? The demo included in the assets folder specifies ws:// and works in Unity Editor but not when building WebGL..

FM_ExhibitionToolPack\FMWebSocket\Scenes

 

In my case, we simply build the WebGL demo without other tricks.
In order to host the WebGL demo on github as webpage, I enabled the "Ssl" option in demo scene. As github page is https, which requires "wss://".

In local WebGL hosting test, I use MAMP to init the web server environment. but without "Ssl", which is also working well.

According to feedbacks from customers, it's most likely the problem on web server or node.js server settings, and there might be some trick and modification needed on cloud hosting.

 

What is the difference between:
SendToAll()
SendToOthers()
I am reading the documentation but it remains unclear.

 

SendToAll() will stream the data to everyone including the sender.
SendToOthers() will stream the data to everyone except the sender.

For example, PlayerA,B,C,D:
PlayerA uses SendToAll(), PlayerA,B,C,D will receive the data.
PlayerA uses SendToOthers(), PlayerB,C,D will receive the data.

Hope this can give you more idea.

 

Alright, I suppose that make sense for some usecases. But unless I misunderstand then, any and all of these functions must use an intermediate hosted server, which runs into the realm of forcing security to prevent data-leakage and to have a stable server in some service-park that cost money? I.e. does any of these or related functions work peer-to-peer?

 

We provided few networking solutions: UDP, TCP, and WebSocket.

For the FM Network UDP example, you can also use SendToTarget() with specific local IP.
For FM WebSocket, it's a WebSocket solution with socket.io, which requires an intermediate server(node.js server).
You can host the node.js server yourself, or pay for some cloud services. It's up to you.
There is a step-by-step tutorial of how to setup your own node.js server.

Beside our default networking solutions, you can also apply your own networking system which supports byte streaming, by following below logic:
GameViewEncoder -> bytes -> Network Manager -> bytes -> GameViewDecoder

According to feedback from many customers, it's compatible with Photon, Mirror, UNet..etc and some customised server on Linux.

 

Hello guys. I have purchased this plugin and it works really great.
Though I was wondering if I can also stream my game view directly into a virtual camera (eg. for Zoom and Skype).
Thanks!

 

In the current version, I'm afraid that it's not possible to simulate as a virtual cam on Zoom/Skype.
Your idea is great, we might put into our To-Do list and seek for solution in future.

 

Hello! Please answer my questions.

Can I transfer large mp4 files shot on mobile (Android/iOS) to PC (Windows 10) on the same network via UDP network system without server?

 

I am looking into using Mirror and Complete Steamworks, to use Peer2Peer over Steams gaming network, as I will release it on Steam anyway. In that case, I could still use the feature-pack here, right?

 

If you only need to stream files, you may use "LargeFileEncoder/Decoder".
It will help you split the big data into chunks for FMNetworkUDP.

 

FMETP STREAM should be compatible with any networking system with byte streaming.
You may need to write some simple code for Mirror, in order to send bytes from GameViewEncoder, and decode the data on the other side.

 

[v1.325] MJPEG Decoder (Experiment feature)
We added MJPEG decoder support, which you can pass any MJPEG stream data to GameViewDecoder via Action_ProcessMJPEGData(byte[])

 

[Known Issue] Unity2020 WebGL Build

FMWebSocket may not work properly in WebGL Build from Unity2020, due to missing "gameInstance" or "unityInstance" is not defined.

Solution: After building WebGL from Unity, please open index.html, and add below line.

Code (JavaScript):

1. window.gameInstance = unityInstance;

After adding this line:

Code (JavaScript):

1. var script = document.createElement("script");
2.       script.src = loaderUrl;
3.       script.onload = () => {
4.         createUnityInstance(canvas, config, (progress) => {
5.           progressBarFull.style.width = 100 * progress + "%";
6.         }).then((unityInstance) => {
7.           window.gameInstance = unityInstance;
8.           loadingBar.style.display = "none";
9.           fullscreenButton.onclick = () => {
10.             unityInstance.SetFullscreen(1);
11.           };
12.         }).catch((message) => {
13.           alert(message);
14.         });
15.       };

 

Hey there Great work on the asset, really eases a lot of pain on the websocket part.

Although, when I run the demo in WebGL as a receiving client, I immediately encounter an error even without being able to see anything loaded. Unity 2020.1.2f, standart renderer.

Error is below;

Code (CSharp):

1. Uncaught ReferenceError: io is not defined
2.     at _WebSocketConnect (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
3.     at /rcv/<anonymous>:wasm-function[44264]:0xd7b13b
4.     at /rcv/<anonymous>:wasm-function[44278]:0xd7b506
5.     at /rcv/<anonymous>:wasm-function[28078]:0xa4c803
6.     at /rcv/<anonymous>:wasm-function[28077]:0xa4c778
7.     at /rcv/<anonymous>:wasm-function[23180]:0x982e9d
8.     at dynCall_iiiii (/rcv/<anonymous>:wasm-function[49029]:0xe52c2d)
9.     at Object.dynCall_iiiii (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
10.     at invoke_iiiii (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
11.     at /rcv/<anonymous>:wasm-function[47108]:0xdfbb9d
12.     at /rcv/<anonymous>:wasm-function[46401]:0xddb86f
13.     at /rcv/<anonymous>:wasm-function[4297]:0x19ad56
14.     at /rcv/<anonymous>:wasm-function[4296]:0x19ac7f
15.     at /rcv/<anonymous>:wasm-function[10109]:0x3ad8b0
16.     at /rcv/<anonymous>:wasm-function[10093]:0x3ac6aa
17.     at /rcv/<anonymous>:wasm-function[12291]:0x4d811e
18.     at /rcv/<anonymous>:wasm-function[12290]:0x4d7e37
19.     at /rcv/<anonymous>:wasm-function[10501]:0x3da58a
20.     at /rcv/<anonymous>:wasm-function[10199]:0x3b62a7
21.     at /rcv/<anonymous>:wasm-function[10199]:0x3b62bc
22.     at /rcv/<anonymous>:wasm-function[10194]:0x3b5e36
23.     at /rcv/<anonymous>:wasm-function[10187]:0x3b40bd
24.     at dynCall_v (/rcv/<anonymous>:wasm-function[49046]:0xe52f0a)
25.     at Object.dynCall_v (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
26.     at browserIterationFunc (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
27.     at Object.runIter (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)
28.     at Browser_mainLoop_runner (39449390-8f1e-4b6a-871e-d58cbdd553a7:2)

Got any idea about this?

 

Thanks for trying our assets.
It's a common error of missing socket.io.
You have to make sure that you type correct IP & Port, where you host your node.js server.

+ More tips: And please don't forget my tips on Unity2020 WebGL Build, in previous post.