Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Is it possible to capture the Android home screen to a texture ?

Discussion in 'Android' started by nasos_333, Jul 16, 2018.

  1. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,288
    Hi,

    I want to capture the Android home screen (default launcher for example) to a texture2D. Is this possible ?
    Like the On button + Low volume button combo

    I can capture the in game image, but i want to capture the image of the Android desktop (to mirror it for example to a PC)

    Thanks in advance for any insight on this
     
  2. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    Hi!
    If you are using OpenGL, you can enable "Preserve Framebuffer Alpha" in the player settings. If you render with a camera that has clear color set to 0 on alpha, it will show the Android desktop.
    This is an experimental thing, idk if it works across all devices/OS versions. But you can try :)
     
  3. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,288
    Thanks a lot for the information, i will test it asap.
     
  4. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,288
    I tested using a Samsung S8 and does not seem to work, i get the white background only. Is there a way to get the desktop in code ? e.g. in a Render Texture so can use it to blend it in our program. e.g. grab the Framebuffer directly.

    Other useful would be to be able to get the launcher icons positions, which would be a bit harder though as i guess the launcher itself should provide that information as a first step.

    I also found that thread that mentions the manifest file (https://docs.unity3d.com/Manual/android-manifest.html)

    "The Manifest.xml in the Android build has to have UnityThemeSelector.Translucent theme set for the application."

    https://forum.unity.com/threads/2017-3-0b4-transparency-on-android.499673/

    Is this also a requirement ?
     
  5. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    This setting is set automatically when you check "Preserve framebuffer alpha".
    What is your camera cleared to?
     
  6. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,288
    I clear to solid color, choosing white and zero alpha

    I also tried with not clearing the buffer, but got a black screen overall
     
  7. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    Are you using a single camera and are you on OpenGL ES?
    Also, please try setting the clear color to all 0.
     
    nasos_333 likes this.
  8. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,288
    It is single camera and OpenGLES3 (i see it first when i uncheck the auto selector for graphics API)

    I tried all black color and zero alpha (all zeros) and still not see the game above the desktop icons for example

    EDIT: Ok, seems to work now, i was exporting the project to Android Studio first and probably some option is not set there, when i did the export directly from unity worked fine !!!

    So i could grab this to a render texture i assume, by rendering the background first without any scene items and then composite.

    Thank a lot for the guidance :)
     
    Last edited: Jul 17, 2018
  9. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    I actually don't think it will work like this. But you can see the Android desktop through it :)
    The thing is that we're just writing 0 to alpha in the framebuffer, and then, when all the windows are rendered by the OS, it becomes transparent, and you can see through it. But the contents of the framebuffer (and the result of a grab) would only be the things that you've rendered.
    I don't think there is a way to capture the Android desktop contents programmatically.
     
  10. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,288
    I see, so the camera wont actually see the backdrop, interesting. I am trying to find a way to interact with the desktop icons in Android, this would open many new options for effects. I have found some ways to get the image from commands and a C++ program that can do it and will try to adapt that one as a solution, though this will be rather hard i guess, if possible.

    Thanks for all the help.
     
  11. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,288
    I have found this code that reads directly the frame buffer in C

    https://github.com/oNaiPs/droidVncServer/tree/master/jni/vnc/screenMethods

    It has 3 different ways and uses adb to launch them is my guess (which is also implemented in the code)

    I will try to create this as a plugin for Unity, though it does not seem trivial and have not done similar before, do you gather it would be possible ?.
     
  12. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    I guess it's possible then, but it might be quite slow :)