Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

VNC Client Code for Unity

Discussion in 'Scripting' started by iPedro, Apr 1, 2015.

  1. iPedro

    iPedro

    Joined:
    Dec 28, 2011
    Posts:
    14
    I would like to build a VNC client using Unity. I am wondering if there are any plugins, libraries, packages, tutorials, etc that I can use to help me along the way?

    Any input or guidance on how I might achieve this will be greatly appreciated!!

    Thanks!
     
  2. lordofduct

    lordofduct

    Joined:
    Oct 3, 2011
    Posts:
    7,352
    Find a .net vnc library whose api has some way to get at the stream of image data.

    Draw that image data to a Texture2D.

    Attach that Texture2D to some surface in unity.


    Really, you're asking for something really weird. You're not really going to find many unity users that have implemented this in itself.

    Thing is, you'll definitely find lots of blogs/tuts/info about implementing VNC in .Net/Mono. All you need to do make the connection from unity to that... which is rendering the stream to a Texture2D, and forwarding the mouse/keyboard events to the server.
     
  3. iPedro

    iPedro

    Joined:
    Dec 28, 2011
    Posts:
    14
    Thanks for the info! I figured this isn't being done or hasn't been done by very many Unity users but I do have a need for it all of a sudden :-/

    You wouldn't happen to have any links to any of those blogs/tuts/info you mentioned that might work within Unity?
     
  4. lordofduct

    lordofduct

    Joined:
    Oct 3, 2011
    Posts:
    7,352
    Not that I know of, especially not one that would work off the bat with unity.

    Here's what I'd do...

    Here is a vnc implementation in .Net:
    https://cdot.senecacollege.ca/projects/vncsharp/doc.html

    The source code is available for download. This means we can rip it apart and take a look at how stuff is done.

    Now, the directions on the site tell you to use the WinForms control called 'RemoteDesktop' for including it in your project... but that we can't do, because we're in unity. So what we need to do is mimic this winforms control.

    So if we load up the project, and open that file, we can see how they connect the vnc, and how they draw it to winforms control. You'll be connecting to vnc in the same exact way, but instead of drawing to the winform control, you'll write it to a texture2d.

    Digging through it, it appears to store a System.Drawing.Bitmap, and on the VncUpdate event of the vnc client, it draws the image data to this Bitmap. Which then in the OnPaint event (this is a winforms event used to draw the graphics of the control) it draws that bitmap to the control.

    This, is the part we want... thing is System.Drawing isn't supported in Unity. So we need to rip this dependency out of it. How this is getting drawn is the VncUpdate hands us an IDesktopUpdater that accepts a Bitmap to draw to. All we have to do is convert this IDesktopUpdater to support a Texture2D call, implement said interface, and then go into the factory that creates the IDesktopUpdater and make it create our custom version again.

    It'd probably be a good measure to purge any and all code that references System.Drawing and System.Windows.Forms. As just having it can cause issues in unity, even if you don't access it.
     
    ThermalFusion likes this.
  5. lordofduct

    lordofduct

    Joined:
    Oct 3, 2011
    Posts:
    7,352
    I did some extra digging. To purge the System.Windows.Forms and System.Drawing dependencies...

    there's 6 classes to remove:
    RemoteDesktop
    PasswordDialog
    #these are just the display controls which we want to mimic

    VncClippedDesktopPolicy
    VncScaledDesktopPolicy
    VncDesktopTransformPolicy
    VncDesignModeDesktopPolicy
    #these are behaviour policies for how the controls above should act depending the state of the winform.

    Also, in VNCClient, there's a bit of code at line 390 that attempts to sync with the WinForms UI thread... this should be removed, just call the VncUpdate event. You will have to sync to the unity thread when handling the event... but honestly, that syncing should be handled by the event receiver, not the event sender (not sure why the put such a dependency into the VncClient class... that's bad design).

    see:
    Code (csharp):
    1.  
    2. // In order to play nicely with WinForms controls, we do a check here to
    3. // see if it is necessary to synchronize this event with the UI thread.
    4. if (VncUpdate.Target is System.Windows.Forms.Control) {
    5.     Control target = VncUpdate.Target as Control;
    6.     if (target != null)
    7.         target.Invoke(VncUpdate, new object[] { this, e });
    8. } else {
    9.     // Target is not a WinForms control, so do it on this thread...
    10.     VncUpdate(this, new VncEventArgs(er));
    11. }
    12.  
    And lastly, several classes have minor references to types like 'Point', 'Rectangle' and 'Bitmap' from the System.Drawing namespace. You'd just change these to use 'Vector2', 'Rect', and 'Texture2D' instead.
     
  6. iPedro

    iPedro

    Joined:
    Dec 28, 2011
    Posts:
    14
    OMG you're so awesome. I'm going to start digging into this asap (tonight hopefully) will keep updating this thread as I go. Thank you so much!
     
  7. grobm

    grobm

    Joined:
    Aug 15, 2005
    Posts:
    217
    This was a helpful article for my pet project... thanks!
     
  8. See_Sharp

    See_Sharp

    Joined:
    Mar 3, 2016
    Posts:
    47
  9. iPedro

    iPedro

    Joined:
    Dec 28, 2011
    Posts:
    14
    No luck - the project never got off the ground although I still think it's an interesting one.
     
  10. kinodax

    kinodax

    Joined:
    Dec 3, 2014
    Posts:
    22
  11. barab

    barab

    Joined:
    Sep 29, 2016
    Posts:
    1
    I'm really interested in this, was any progress made on this?

    I'm a Java dev that's new to C#/.Net and Windows APIs in general (and totally new to Unity or anything like it). So my progress will be slow, but for now I'm going to poke with lordofduct's suggestions.
     
    See_Sharp likes this.
  12. grantsrb

    grantsrb

    Joined:
    Jan 10, 2017
    Posts:
    4
    @barab, I'm in a similar boat. Did you make any progress?

    Right now I'm converting all of the Rectangles and Bitmaps to their respective Rects and Texture2Ds, but I'm running into troubles converting BitmapData into a Texture2D equivalent.
     
  13. YohanB

    YohanB

    Joined:
    Jun 9, 2015
    Posts:
    10
    Hi, looking for the same to create a virtual desktop concept in VR. If you guys are able to share code or discuss id like to help with getting that last issue with texture solved
     
  14. cfloutier

    cfloutier

    Joined:
    Jul 30, 2009
    Posts:
    34
    Hello all,

    I've made an open source project that is based on a csharp integration of a vnc client.
    It is available here : https://github.com/cfloutier/Unity-VNC-Client

    The csharp decoding is quite slow but it is a good stating point.
     
    jeanrouq and JUSTINMKAUFMAN like this.
unityunity