[Release] FMETP STREAM: All-in-One GameView + Audio Stream + Remote Desktop Control(UDP/TCP/Web)

When I talk, if I say "Ah~" and make a long sound, the other person will say "Ah...Ah...Ah...The sound is not smooth in this way
I can't understand what the other person is saying

 

Usually you may reduce the sample rate of MicEncoder.
For the quick test, we suggest that you may simply build and test our example scene "Demo_FMNetworkStreamingMainCam" and enabled the Mic toggle.

 

Thank you very much for your reply
I'm still testing it, but thanks to your kind response, I think it'll go smoothly
Thank you so much

 

We want to control the remote computer desktop and realize data interaction through the VR device. Is this tool open source and can it be developed again?

 

This is a paid asset, not open source.
You can develop your own apps based on our SDK, but you are not allowed to expose, or resell our source codes/modified version.

 

thank you for your reply.I have researched your tool, we are interested in remote desktop control, but you have very little display of this. Please, do you have a detailed example about it

 

Thanks for your interest too, it's a new add-ons feature which we released this year.
The package itself includes all the demo scenes, very straight forward solution to achieve remote desktop control result, like teamviewer, anydesk...etc.

The remote client can be cross-platforms, including VR like Quest 2. (template on github)
PS: What you see is what you get

 

Is this working already on version 3? looking forward on this

 

Not fully compatible yet, still working in progress.
*The old method is routing the FMETP Stream as mjpeg(udp) through ffmpeg on computer, then redirect it to Youtube.
https://frozenmist.com/docs/apis/fmetp-stream/unity-to-ffmpeg/

 

They are running in different node.js server scripts, the data structure is completely different too in the backend.
Please review your use case, and see if you can migrate the node.js server too, which you can get the best benefit in performance improvement.

 

Hello, i have a strange issue with the plugin, seems to not work on a Windows UWP build? The Websocket classes can't be found

 

Do you have screenshot of the error?
Websocket has been tested and verified on UWP, as it's a very common protocol.

 

Yeah, i get about 20 errors, most related to WebSocket or WebSocketSharp. They only appear if i switch to UWP and try to build.

 

Hello, can I do video compression when transmitting video, such as ffmpeg and h264, etc. How to add video compression function to FMETP STREAM? Thank you very much!

 

Could you please check if "websocket-sharp" is existed in your folder?
And make sure UWP platform is enabled for this common library.

 

This is a work-in-progress feature, not available in the current version.

 

There's no option for UWP, i just enabled "Any platform" and it's still not letting me build.

 

Seems likely you are using Photon's websocket-sharp, you may consider removing our FMWebSocket folder completely if not in use.

 

Oh, weird. Deleting that folder fixed it. Thanks

 

Hi, I recently set up my application to stream from Quest (client) to another mobile/android application (Server) on a local connection, and everything was working all right.

Today, doing some tests, the server app stoped receiving any textures from the client, even though the server app recognizes the client connection.

Just to be sure, I tried your's quest tutorial version and the same thing happen, server says I'm connected, but I just don't get any texture from the stream.

Any ideas about what is happening?

 

Hi there, I've bought your asset but I couldn't add the components (i.e. FMWebSocketManager) because the script class is not found. Were the scripts completely updated? I did not expect to encounter such errors.

EDIT: Even in the demo scenes all my scripts are missing the reference, what is going on?

 

Do you have screenshots of the errors?
The most common situation is due to duplicated libraries like websocket-sharp from PUN2. If that’s the case, you can simply remove the duplication one.

 

If everything working in local machine but not in other devices. It’s 99% due to firewall on PC or router.

The quest 2 demo scenes are verified by many customers, it should be very stable.

Are you testing in public wifi, like guest or university wifi?(those will block unknown udp stream)

 

No, i'm using my home local wifi... The strange thing is that it seen to connect, but the stream wont work

 

Still not working, but it seens tha the client is failling to find my server ip. I checked and the "Auto Network Discovery" is marked

 

For troubleshooting on connection issue, please reach us for technical support via email: thelghome@gmail.com
It will be easier for us to follow your case.

 

I want to stream an android camera to an android vr headset like Pico4 / Quest1/2. Can I do that with your plugin? So I already have a unity app running on the android phone that displays the front camera on a rawimage. I just want that to be streamed to another unity app running on the headset displaying the camera of the phone on the rawimage in the headset. That is all I want to do, so can this be done or do you know of another solution? Thanks in advance.

 

Hello.
I have a question about the video call feature, I purchased the plugin and I want to use it to make a video call feature on Hololens, is this possible?2 devices in different networks.

 

Yes, in fact, you can stream Android webcam to any platforms(including VR) with FMETP STREAM 3.0.

 

It's possible. If you've already tested with FMNetwork system, you should simply following the FMWebSocket guide and setup for Internet stream. The only different is the network manager, with an extra node.js server hosting.
https://frozenmist.com/docs/apis/fmetp-stream/fm-websocket-stream/

 

Thank you for your reply, I am a non-native English speaker and a beginner Unity student, so I may not be expressing the meaning or phrasing accurately.
Does this mean I need 2 different servers and 4 different ports? Or is it one server running 2 server programs?

 

Technically, the structure is very straightforward like this:
Unity Server -> node.js server -> Other Unity Clients

Our example node.js server script is designed as Server-Clients structure, which supports only one Unity Server in your case.

Depending on your project structure:
1) you can setup multiple node.js servers with different ports, for multiple Unity servers.
2) Or, some users modify our example script to support multiple Servers or Rooms with single node.js server hosting only.
3) Or, you can collect all the Websocket IDs and build your own Room system.

The first option is easier, while the option 2 & 3 will be a bit advanced.

 

Thank you very much for your explanation and help.

 

so @thelghome, I'm thinking of going for the second option you mentioned, but I don't know how to go about it. Do you have any advice/code starts/roadmap to give me?

ps: the tool is incredible

 

Okay thanks. I couldn’t find any example of this in your documentation. Is there an example in the asset itself? What kind of resolution would I be able to stream in a local 5Ghz network? I am looking to stream the camera of a galaxy s8 to a Pico4 to the best quality possible with as little latency as possible.

 

Have you also thought of adding USB camera support for Android? Or can you do that already with FMETP?

 

For option 2, you can firstly remove the checking of "require Unity Server" in node.js index.js file.
In the node.js server part, each connected user has unique websocket id(wsid). Then, you can create your own routing logic from there, based on wsid.
(since there are too many use cases with websocket, we only provide this simple Unity Server-Clients minimal setup as an example)

 

You can simply follow this guide and build our example scenes:
https://frozenmist.com/docs/apis/fmetp-stream/

There is a toggle to enable webcam in our example scenes.

 

FMETP STREAM is accessing the webcam via Unity's Webcam class, it depends on Unity3D.
In other words, whatever items visible in Unity 3D space, they can be all streamed.

 

I got the lastest version and run scene with "Demo_FMWebSocketNetwork", when click server or client button, ws tried to connect and will triggerd "onClose" at the same time, could you tell me what happened?

 

Is your node.js server running and reachable?
ref: https://frozenmist.com/docs/apis/fmetp-stream/fm-websocket-stream/

 

Hi! I've been using this for a long time now to send camera image from one Unity app to another, but I want to expand my platform and am considering rebuilding the receiving app (currently the Server) as a web application that can be accessed via a browser. Rather than using Unity WebGL, I would build it using regular web tech.

Is there a way to send picture, byte, and string data back and forth between a Unity app, and a website over the local network?
I don't need any type of fancy internet networking, only over local network, I basically want to mimic the existing Unity-to-Unity functionality, but with one of them being accessed via a browser, which is displaying the sent camera view and sending back a byte[].

Any help would be appreciated, as well as any resources on how to best achieve this.
Thank you!

 

There is a demo html when you are running node.js script, for FMWebSocket.
There are some example javascript for sending byte[] and string in that html file, which you may refer to.

 

Thank you, I will look into those!
Is there a way of doing this without an additional node.js server?
Ideally, the web app would just be a website the user can go to and connect over UDP/local wifi to the Unity app.
Thank you!

 

UDP is not allowed on web browser, you can only choose either WebSocket or WebRTC, both require server hosting.

 

what if i only want to use audio, on the same network, UDP, over wifi, what is expected the delay?

 

In local network, the expected delay of audio would be between 0.5~1.0 seconds.
The data arrive in realtime, but the Unity audio playback causes delay.

*If you playback via HTML demo via javascript, it's almost realtime then.

 

Could you please teach me how to send only a specific area image from the device to the PC, instead of sending everything from the MainCamera of Quest 2 to the PC?

 

You may consider using RenderCam mode.
1) duplicate your CenterEyeCamera, remove the "MainCam" tag and set the target eyes to none.
2) set the GameViewEncoder mode as RenderCam
Then you should have freedom to stream whatever area you wants.

 

Thank you very much.
I was able to specify the display area using RenderCam.