Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Applying filters to the passthrough camera feed (meta quest, pico, vive, etc).

Discussion in 'VR' started by Avedis, Aug 23, 2023.

  1. Avedis

    Avedis

    Joined:
    Jan 12, 2013
    Posts:
    24
    In most modern HMD/SDKs you set the camera's clear flags to a solid color (like black) and alpha to 0 to view the passthrough feed after enabling it in the SDK's api. However we would like to apply overlay filters to the camera feed to produce interesting effects in a live setting.

    So far I have been unsuccessful, I have tried to use a render texture and a plane to get the feed on something the renderer sees but this doesn't work as the camera only "sees" black. We are currently using the built-in pipeline, has anyone been able to apply a filter of any kind to passthrough feed?

    I'm sure it has something to do with how the unity camera is rendering objects but I don't have enough knowledge to be able to write a shader or something that will allow us to manipulate a passthrough feed.

    Any help would be appreciated!
     
    Ali_V_Quest likes this.
  2. TomGoethals

    TomGoethals

    Joined:
    Jan 29, 2022
    Posts:
    45
    at least for Meta this is very much a feature rather than a bug. developers should not get, by default, access to the camera's. with the current integration there is no way to access the feed from your app. However, I'm wondering why they do not allow it after asking the android system for camera access, just like with the microphone. Then the user is aware he's potentially vulnerable to being recorded.
     
    Ali_V_Quest likes this.
  3. Avedis

    Avedis

    Joined:
    Jan 12, 2013
    Posts:
    24
    I suspect the limitation might have been due to either performance boosts and/or privacy concerns. You don't want images or audio of a users surroundings stored or uploaded somewhere without consent, I totally get it. On the other hand a webcam texture can be used on androids and allow developers to do the same nefarious activity.

    We simply want to apply effects to the camera feed to make it more interesting, but the more I look into it the more I'm thinking it's not going to be possible. This seems to also be the case with HTC and Pico headsets. PS, love your VR mirrors & reflections asset! :)
     
    Last edited: Aug 24, 2023
    TomGoethals likes this.
  4. TomGoethals

    TomGoethals

    Joined:
    Jan 29, 2022
    Posts:
    45
    Was looking for doing the exact same with the "Portals for VR" asset, making it AR compatible.. being able to do interesting effects to the real world texture would be awesome.. but so far, no luck.