Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct here to familiarize yourself with the rules and how to post constructively.

  2. Unity 2022.1 is now available as the latest Tech release.
    Dismiss Notice
  3. Improve your project's performance with our new guide on profiling in Unity.
    Dismiss Notice

DOF on IOS, How we did it (mini post mortem)

Discussion in 'iOS and tvOS' started by brn, Jul 16, 2012.

  1. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    Depth of field and Bloom on Unity IOS ( Run Fatty Run mini post Mort-em)

    Recently I've had the chance to venture into IOS. In many ways its like the year 2000 all over again in terms of some hardware specs, but in many others its defiantly 2012 with HD resolutions and massive amounts of memory. With a few wins and losses, I thought I'd take the time to share some of the things we have uncovered, firstly as an overview, then in later posts with detailed descriptions of each stage.



    A friend of mine at Disparity Games, Jason Stark asked me if I could get DOF and Bloom happening on their latest project Run Fatty Run. After looking at the project and its frame rate at the time, it looked like a real possibility and to cut a long story short, we did get it working on ARM 7 devices Iphone4, IPod 4, IPad2 and later. Before I describe how, let's cover some basics first.

    What are the general steps needed to do a standard DOF?

    1) Create a depth buffer.
    2) Write the scene to that depth buffer frame buffer.
    3) Read from that depth buffer frame buffer writing to X number of down sampled textures
    4) Blit the final result to the frame buffer based on depth and DOF parameters.

    Sounds pretty straight forward right, so why haven’t we seen more games with DOF implemented on IOS?

    Step three is not possible because we have no read access to the depth buffer using Unity IOS 3 ( It will be available in Unity 4, Fantastic!) . Step three seems broken and doubly so when you discover a single grab of the frame buffer on an Ipod4 in HD, dropped the frame rate to near 20fps with only a single cube on screen.

    Not interested in the cant dos? Well neither am I and this is how we addressed the main bottle necks as an overview.

    1) We created a camera render texture at full screen resolution and used that instead of the frame buffer.
    2) We wrote the depth as a grey scale value to the alpha channel of the render texture and used the RGB channels as per normal.
    3) By reading from the render texture we avoided reading from the frame buffer and were able to write to various smaller textures, down sampling each time.
    4) We then combine the down sampled textures based on depth and blitted the result to the frame buffer. In our case we integrated it into the GUI camera pass, using it to clear the camera to save on fill.




    This covers the main bottle necks and their work arounds and for those familiar with the shader side of Unity its probably all you will need to start your own implementation. However over the next week I'll be posting detailed information on how we achieved each of the steps and the decisions behind them.

    To see the results you can view this video or even better purchase the game here.
    If you have any specific questions you can contact me via the details on my website.

    You can also find a temporarily available unity package with a very simple demo scene here

    Kind regards,
    Brn
     
    Last edited: Sep 28, 2012
  2. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,165
    In unity 4 you can access the depth buffer on mobile :) That being said the custom solution here with custom shaders is STILL probably faster and nicer, especially on texture cache. Awesome work, it looks good.
     
  3. opsive

    opsive

    Joined:
    Mar 15, 2010
    Posts:
    4,995
    This effect looks awesome! Nicely done. I will definitely be watching this thread when you post more details
     
  4. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,189
    Really nice , please put this technique on the asset store
     
  5. coolpowers

    coolpowers

    Joined:
    Mar 23, 2010
    Posts:
    125
    Purchased the game just because you proved it could be done and then posted this. I don't have iOS Pro so I can't test it out yet, but if I get some time I'll try doing it outside of Unity.
     
  6. I am da bawss

    I am da bawss

    Joined:
    Jun 2, 2011
    Posts:
    2,574
    Teriffic! Thanks for sharing!
     
  7. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    @ Stramit, Thanks Stramit. I'm really looking forward to Unity 4 with its new functionality. I've edited the first post to reflect the the Info about the depth buffer.

    @ opsive , I am da bawss coolpowers , thanks for checking out the thread and the game. I'll be able to do the first detailed posting in the morning.

    @ Lars Steenhoff, At this stage I'm not sure if I will put it on the asset store. The system doesn't package well, because it requires custom shaders for every material you want to be effected by DOF and some tampering to be done to the GUI, cameras ect. The best I could perhaps offer is a free sample scene package with the option to donate. That way if its not suitable for your project, no one is put out. Its because of the messy nature of it that I decided to do the mini post mortem. Besides sharing this sort of info is a nice way to give a little back to the Unity community.

    Cheers,
    Brn
     
    Last edited: Jul 17, 2012
  8. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    28,862
    This is different considerably to the DOF I did for an old project for polycount. I think that the "custom shader" thing is intriguing, is there a reason it has to be like that?
     
  9. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    Hi Hippo,

    It was the best way I could think of to write the depth data into the camera render texture alpha channel. It produces a little extra overhead on the vertex shader but next to nothing on the fragment shader. I guess the other method I could have used was a replacement shader technique, but that would require extra passes. Also by cramming the depth data into the alpha channel, I could read/write depth and color values with a single a texture fetch or fixed4.

    The other bonus was that I could write the desired blur factor into the alpha channel instead of the depth, which is the method I went with. This made the final composite much cheaper because I didn't have to convert the depth value into a blur factor for every screen pixel. It also gave me the option to do a near blur almost for free because it was calculated at the vertex stage of the objects shader. In fact by writing the blur factor it allowed us to do quite a few other tricks such as motion blur at a per vertex fidelity ( we ended up using another type of trail blur instead but it did work ).

    I'm sorry its reading like a brain dump, I'll cover this all a bit better in the morning with some screen buffer captures.

    Cheers,
    Bruno
     
  10. actuallystarky

    actuallystarky

    Joined:
    Jul 12, 2010
    Posts:
    187
    Although the Asset Store suggestion is an obvious one, I can't stress enough the added advantage of personally working with Bruno to develop a custom solution for my particular game. Instead of relying on an "out of the box" package we could work together to produce effects that took advantage of the particular camera angles and visual style of Run Fatty Run.

    And he could look at my particle effects and make me cut down on the horrible fill rate abuse. :)

    If you're looking for a magic, "Make my game look awesome button," talk to this man. He'll make your game look great and probably make it run faster while doing it.
     
  11. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    28,862
    Glowing praise from a respected developer indeed. I can echo that sentiment, its worth every penny to find a decent dev that knows his or her stuff.
     
  12. macdude2

    macdude2

    Joined:
    Sep 22, 2010
    Posts:
    685
    How is the depth buffer access in Unity4 going to be different than in Unity3? It is possible right now to write to the depth buffer on mobile devices. In fact I am using this ability in my game. What is going to change?
     
  13. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    Hi macdude2,

    I could have described that much better. Its reading from the depth buffer that's the issue in unity 3. I believe in unity 4 it is no longer a problem. I'll edit the post to make that clearer, thanks macdude2.
     
    Last edited: Jul 18, 2012
  14. Muckel

    Muckel

    Joined:
    Mar 26, 2009
    Posts:
    446
    ...many thx for sharing your knowledge !!!
    a sample scene would be fantastic !
    anyway great post !
     
  15. DanTreble

    DanTreble

    Joined:
    Aug 31, 2010
    Posts:
    590
    Looks awesome! Nice work Bruno and Jason, checking out the game now.
     
  16. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    @ Jason, Hippo Muckel, Thanks for the kind words.

    I apologise for not getting the second post done. As I was writing it I found a few places where I could improve things, and tidy up the system generally. By the time I'd changed a bit here and there I was half way to creating a usable package. So I'll do the free package up first then follow it up with some documentation.

    Kind regards,
    Bruno
     
  17. BrUnO-XaVIeR

    BrUnO-XaVIeR

    Joined:
    Dec 6, 2010
    Posts:
    1,687
  18. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    Thanks Dan,

    Fantastic work on Ski Safari by the way. I regularly use it to bribe my kids into good behaviour, Its particularly good because even my youngest can play it. There's allot to be said about a single input mechanic done well.
     
  19. goodhustle

    goodhustle

    Joined:
    Jun 4, 2009
    Posts:
    310
    Awesome job, and thanks for posting the how-to. This is how intermediate game coders like me learn how things are done! :)
     
  20. cupsster

    cupsster

    Joined:
    Apr 14, 2009
    Posts:
    363
    @actuallystarky: I can second that working with Bruno is pleasure!
     
  21. AdsySingle

    AdsySingle

    Joined:
    Jan 5, 2011
    Posts:
    116
    This is exactly what I'm am trying to find a solution for now. I don't have any experience writing shaders, and understand very little about them. I understood what you were saying when you broke down the process, but converting that into practice is unfortunately beyond me. I've been trying to find a good way to learn and completely understand shaders and shaderlab specifically, but so far not much luck. Is there anywhere you might be making this available. We'll quite happily pay for it.

    Took me a little while to tie together that you were the same man that wrote the xray shader on youtube. That's a fantastic one too, thanks for that.
     
  22. toto2003

    toto2003

    Joined:
    Sep 22, 2010
    Posts:
    520
    any chance to got a DOF friendly solution on the asset store out of the box! i m willing to pay to get this!
     
  23. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    Hi,

    I've been a little overrun with things of late so I haven't had a chance to do any more on the package to get it to the Asset Store. However I've put the package together on my site:

    Link.

    Feel free to test it out on your project and please credit my work if you use it.
    Package contains a sample scene with an example shader or two.

    Kind regards,
    Brn
     
    Last edited: Sep 26, 2012
  24. AdsySingle

    AdsySingle

    Joined:
    Jan 5, 2011
    Posts:
    116
    I don't know what to say. That's incredibly generous of you!

    I know you're really busy, but I'm looking for someone to teach me shaders from the ground up via email correspondence if you're interested. You'd set the pace so it'd just be whenever you got back to me. Mainly I'd like someone to bounce questions off. I've started going through video tutes, and that's great, but I like to verify that I'm understanding things as I learn. If you're interested let me know, I'd be massively appreciative.
     
  25. ddeaco

    ddeaco

    Joined:
    Jan 17, 2012
    Posts:
    44
    Cool thanks, ive been following this thread in the hope you would release it. :)

    Have you played around with unity 4 and the access to the depth buffer?

    I would pay good money for a set of iOS post effects that run faster than a slide show.
     
    Last edited: Sep 26, 2012
  26. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    @ ddeaco , I think you will find these effects slightly faster than a slide show ;) . The demo scene runs in the high 50's on an IPod 4 in HD. Also the way the system is set up its very easy to extend to create extra effects. For example in RFR it also is used to do a simple real-time reflection. Personally I've had motion blur and screen warps working with almost no additional cost. I've been advising an Indie team that is doing very well use the system to achieve some really awesome stuff. I think they will release some captures in this thread soon.

    I'm going to have a look at using the depth buffer in unity 4. Depending on how quick it is to read from it might be a great step forward. However if porting to all mobile devices is ( Android variants) important avoiding the depth buffer might still be a good alternative.

    @ AdsySingle , Feel free to send me an email now and then, I cant promise anything like full tuition, but Id be happy to give you a pointer or two and guide you to the correct documentation ( which is half the battle IMO).

    Kind regards
    Bruno
     
  27. opsive

    opsive

    Joined:
    Mar 15, 2010
    Posts:
    4,995
    Thanks for releasing this - it looks awesome.

    I'm working on getting it integrated into my project, and I have a quick question - if I don't want a objectto be affected by DOF, is there a way to exclude it? It looks like you are using the alpha layer to determine how blurry something is (or I could be completely wrong, I'm still learning). If I have a shader that uses the alpha color to for transparency then setting the alpha layer doesn't work if I don't want it to be blurry. Is there a good solution for this? You could render the object to a different camera with a different culling mask, but then you run into a sorting problem.

    Again, thanks for posting this, it is really appreciated.

    -----

    Edit: Solved! Reversing the alpha value worked for my case. 1 means not blurry, 0 means blurry. Although it would be nice to know if there is a way to get access to a fifth fixed variable, but that is a general shader question.
     
    Last edited: Sep 29, 2012
  28. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    2,137
    Wow, thanks for this!
     
  29. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    320
    Hi opsive,

    Have you tried adding colormask RGB to your particle shader. Although the unity docs say that its not a good option for mobile hardware, I cant find anything in the PowerVR docs that suggest it would be. Perhaps Its an armv6 issue only (fingers crossed). Also if you want to blur your particles based on depth, tex2Dlod() might be a nice way to go about it. I believe its supported by GLES 2.0. I'm unsure on how it will affect pre fetching of the particle texture.

    anyway nice adaptation on inverting the the Alpha channel. Did this result in an extra calculation though? I vaguely remember setting it up the way to save on a cycle or two. As for the fifth fixed variable, if you mean a fifth channel for the rendertexture? no this is not possible without an additional rendertexture and a whole heap of overhead.

    Cheers
    Brn
     
    Last edited: Oct 1, 2012
  30. Crazy Robot

    Crazy Robot

    Joined:
    Apr 18, 2009
    Posts:
    921
    Hello,

    Is there a way to turn off blur and just use bloom?

    Thanks,
     
  31. gjfjfg

    gjfjfg

    Joined:
    May 2, 2012
    Posts:
    18
    Great Post .

    how to use with my own shader instead of NeverBlur Shader ?
     
  32. G3al

    G3al

    Joined:
    Nov 28, 2014
    Posts:
    4
    Hey opsive,

    Can you please elaborate how you implemented the problem with the transparent objects not being affected by the blur? I am trying to make a particle system that should not receive any blur.

    and great job brn !
     
  33. ilya_ca

    ilya_ca

    Joined:
    Nov 19, 2011
    Posts:
    274
    I took some inspiration from the approach listed in this post. I developed it further into a "progressive blur blending" algorithm that provides much better visual results without decreasing the performance.

    Here are the two major improvements I made to the original algorithm:
    1. I made an optimization by downsampling the RenderTexture (which looks as good when blurred, but requires x4 times less pixels to process).
    2. I improved the Gaussian blur algorithm, by using self-modification based on the required blur amount:

    blur = dX * blurKernel[dX] * blurBias;
    The first two terms of the equation is the usual blur equation, and the blurBias term depends on Circle of Confusion (amount of blur). By using this modified equation I was able to "scale" the blurring algorithm, and achieve a smooth realistic transition without any drop in performance.​

    Another limitation I've encountered is that iOS doesn't by default render to alpha channel. The only two options are HDR rendering and enabling depth texture. I find the depth texture option to be faster and more precise.

    This improved version of DOF is available on the asset store in my FxPro asset. I hope people will find it useful: https://www.assetstore.unity3d.com/en/#!/content/22189
     
  34. Steamc0re

    Steamc0re

    Joined:
    Nov 24, 2014
    Posts:
    144
    I just got a popup when I booted unity this morning.

    ABSOLUTELY NOT COOL.

    I've paid for dozens of plugins. Can you imagine what would happen if they all started using the update window to serve advertisements? Having to close 40 popups when I boot unity? I reported this to support and I suggest you remove it.
     
    meejabych likes this.
  35. alsharefeeee

    alsharefeeee

    Joined:
    Jul 6, 2013
    Posts:
    60
    Amazing asset and works with android too.

    Thanks @brn
     
  36. Pourya-MDP

    Pourya-MDP

    Joined:
    May 18, 2017
    Posts:
    135
    Hello
    It seems it doesnt work in unity 2017
     
unityunity