Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

A desktop inside unity

Discussion in 'Getting Started' started by UnityZelos, Apr 4, 2019.

  1. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
    Is there a way to show my desktop when I activate play in unity. Basically having my desktop inside my scene on a object. I'm new to unity and hope someone can help me.

    love,
    Zelos
     
  2. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
  3. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    9,833
    Sounds very difficult to me. But if you want to pursue it, be sure to mention which platform you're thinking of, as the answer to this will certainly be platform-specific.
     
  4. JeffDUnity3D

    JeffDUnity3D

    Unity Technologies

    Joined:
    May 2, 2017
    Posts:
    14,446
    Agreed, difficult but interesting! I believe some screen sharing apps provide a streaming video of the desktop. You might want to look into support for streaming video, perhaps in the Asset Store.
     
  5. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
    First off; thank you JoeStrout en JeffD for spending your time and trying to help me out.

    What I want to achieve is the following;

    Step 1;

    -- Having an object appear using Merge cube or Vuforia. (
    )
    -- This is pretty easily achievable, even for a beginner like me.

    -- Just having it appear 2 meters in front of my face is also fine. ( Like this this guys did at 8:14 (
    ))

    -- I'm not sure which one I prefer.

    -- Now the tricky part;

    -- I want my desktop to appear (on my phone or my AR headset).
    -- For this to work I need code that opens my desktop in-scene.
    -- Or maybe a streaming video like Jeff suggested!
    -- I'm currently brainstorming.
    -- Both IOS or Windows Desktop. But preferably Windows.

    \\ Solutions I'm currently brainstorming about;

    -- Using Rainity - Dynamic Wallpapers

    -- I found this piece of code about viewing a desktop in-scene.
    -- https://answers.unity.com/questions/869378/viewing-desktop-in-scene.html
    -- Using vr desktop
    -- A desktop GUI

    I'm basically trying to set up an Augment Reality Desktop!

    // Step 2;
    // Make it track my camera, so I have a head tracking Augmented Reality Desktop. (Which I need for my main project)
     
  6. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
  7. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
    There are still a lot out there trying to get Windows or other programs (Browser, Notepad, etc) to run inside an object in Unity during runtime. There are many approaches, although most seem to involve long, processor-intensive methods. For example, many use the screenshot method where the app takes constant screen shots of the web-browser and 'feeds' it to a texture in Unity. I have gotten emulators to work in Unity, but a Windows 7/10 emulator would be super-slow and complicated (however Windows 10 Mobile Edition might work). You could make a simple C# or java Web Browser, I did one for a hand-held system before, basically it 'reads' the *.html page at the web address and looks for HREF links, IMG pictures and text and displays them. Then use www in Unity to download the pics, and put the text into a GUI object. (This is basically making an app that runs inside Unity) I've seen a few people trying to get NotePad or WinCalc to run on a Unity object, but it might be easier to write your own text editing app/calc that runs inside Unity. There's currently(2017) an asset called Awesomium, although it is dependant on the C++ Awesomium SDK, really complicated to set up, and it's newest version is not free. However, it is supposed to render a real-time browser window onto an render-texture object. I haven't tried it yet so I don't know what browser it uses but...

    https://medium.com/google-developers/real-time-image-capture-in-unity-458de1364a4c

    Texture2D tex = new Texture2D (200, 300, TextureFormat.RGB24, false);
    Rectangle screenSize = System.Windows.Forms.Screen.PrimaryScreen.Bounds;
    Bitmap target = new Bitmap (screenSize.Width, screenSize.Height);
    using (System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(target)) {
    g.CopyFromScreen (0, 0, 0, 0, new Size (screenSize.Width, screenSize.Height));
    }
    MemoryStream ms = new MemoryStream ();
    target.Save (ms, ImageFormat.Png);
    ms.Seek (0, SeekOrigin.Begin);

    tex.LoadImage (ms.ToArray ());

    renderer.material.mainTexture = tex;

    -- UPDATE IM GOING TO TRY AND USE AWESOMIUM + https://rainway.com/

    But any experienced coder, what is faster the real time rendering of images or the real time rendering od the browser :D

    The more complicated it is the more excited I get if it works, -- il update tomorrow
     
  8. JeffDUnity3D

    JeffDUnity3D

    Unity Technologies

    Joined:
    May 2, 2017
    Posts:
    14,446
    Where do you plan to run this game, on a mobile device or on the Desktop? You'll need a streaming solution, screenshots would never be able to capture mouse movement, etc.
     
  9. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
    I want to play in on A mixed reality headset, so windows oculus. I am thinking of buying a GUI in the asset store. It's also not really a game. More an app to just make my desktop show in AR.
     
  10. JeffDUnity3D

    JeffDUnity3D

    Unity Technologies

    Joined:
    May 2, 2017
    Posts:
    14,446
  11. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
    I am going to buy a GUI, I can try and do the same thing, essentially recreating what these GUI systems do. Set up an orthographic camera that renders a specific layer. Outside of your regular scene, add quads, colliders, and the like, within this camera's view area.
     
  12. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
  13. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
  14. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
  15. UnityZelos

    UnityZelos

    Joined:
    Apr 3, 2019
    Posts:
    14
    Ok I got everything working. Thank you all. *opens 24th energy drink*
     
  16. JeffDUnity3D

    JeffDUnity3D

    Unity Technologies

    Joined:
    May 2, 2017
    Posts:
    14,446
    Oculus does not make an AR device, only VR devices. So you look to be placing a desktop inside a VR environment, as there is no view of the outside world in the headset. Unless you are adding an external camera like the Zed (which I have done!)
     
  17. SilkyZielke

    SilkyZielke

    Joined:
    Jul 20, 2021
    Posts:
    1
    So im Developing an office application for work, and im trying to figure out how to share my desktop screen inside the unity scene for like presentations inside VR. Is this like what you were doing?
     
  18. fadycolllllll

    fadycolllllll

    Joined:
    May 21, 2022
    Posts:
    3
    Have you found a solution yet ?
     
  19. unity_yVLiKCix-iITug

    unity_yVLiKCix-iITug

    Joined:
    Aug 22, 2019
    Posts:
    2
    To anyone that has achieved this successfully - would you mind explaining how it can be done? Or posting a link to a project on GitHub or something? I'm looking to copy my desktop to a render texture in Unity on Windows.