Search Unity

Question Unity's love of internal classes and closing doors to extensions.

Discussion in 'SRP Dev Blitz Day 2022 - Q&A' started by jbooth, Sep 28, 2022.

  1. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    I've noticed what I consider a disturbing pattern in modern Unity code, which is also prevalent in the SRPs. Code which you cannot easily extend, modify, or otherwise use without extensive reflection, asmref injection, or forking a million lines of code into your own version that has to be maintained. The Shader Graph and VFX graph, for instance, are prime examples of this - yet they are for the most part editor time tools which generate code, not critical paths at runtime. So I'd love some insight into why Unity has begun to code systems such that other developers cannot easily use or extend them.
     
  2. arttu_p

    arttu_p

    Unity Technologies

    Joined:
    Jan 15, 2021
    Posts:
    22
    Hi, thanks for the question. I understand the frustration. For us, as much as we'd like to expose public APIs to users, there are unfortunately clear drawbacks to doing so as well. Once we expose a public API, we are effectively committed to not breaking that API without a planned deprecation cycle. In practice this means for example if I start the process of removing an API function 23.1, it will first just throw a deprecation warning. Then in 23.2 I can make that an error, and then finally in 24.1 I can remove it. We follow this procedure to make the process of upgrading from one Unity version to the next as easy as possible for users. However, this has the drawback that it adds a lot of overhead to the development process. If I instead have kept my API internal, I can iterate on it and make changes as often as I'd like. So for features where we don't expect users to need a public API, it is better to keep it internal. Naturally sometimes we don't get this right and some parts are internal when they should be public. In these situations we rely a lot on our users to let us know what they'd like to access.
     
    joshcamas likes this.
  3. ManueleB

    ManueleB

    Unity Technologies

    Joined:
    Jul 6, 2020
    Posts:
    110
  4. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,362
    You could avoid a roundtrip for you and much headache for us by asking @jbooth and a few others before making those decisions.
     
    timmehhhhhhh likes this.
  5. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    @arttu_p

    I mean I get the desire to not deal with the overhead of a public API. But the shader graph API was closed without going through any of this process - I had a product in the asset store which added nodes to the shader graph, and one release the API was simply removed. I chose to remove that product from the store rather than reflection or asmref my code into it.

    But it seems once an API goes internal, it never comes back. The Audio Mixer was released in Unity 5 (like 8 years ago), and we were told its API would be opened up eventually - but nothing ever happened, and the mixer system is nearly useless for real productions because you can't script it. A recent project I was on needed to script mixers, but instead we ended up writing out 7600 yaml files for every mixer combination we needed, with all the internal GUIDs remapped, etc. (The Audio Mixer API is really hard to hack into since it uses wrapper classes in the editor to do everything)

    We were told the same thing about the shader graph API, that it would open up, and it's been 3 years since then and nothing has ever happened. And about VFX graph.

    So effectively for any real production that needs access to these things, your API is just behind a big barrier that makes our lives hell, and when we can we'll hack into it anyway, so all that 'safety' you feel about changing your API is just an illusion which only you enjoy. We end up with the worst of both worlds- all the hacks to work around things and get in anyway, and all the breaking changes as well. Or forking a million lines of code and maintaining that.
     
    LooperVFX, n3b, cxode and 6 others like this.
  6. joshcamas

    joshcamas

    Joined:
    Jun 16, 2017
    Posts:
    1,277
    Big thumbs up on this - we need the API's to make our game, so we will get them somehow, since... well, we need it. Being forced to hack it isn't a great feeling and is a lot worse than having it be public then it changing, because if we're using reflection we're forced to change our code anyways if the api changes, it's just way harder.

    An example for me was hooking into playing sounds in editor. It's hidden, I had to use reflection, and when the API changed (which was fine), I had to dig into the sourcecode to find what changed. If the API was public this process to update would have been much faster.

    I definitely understand the assumption that a public API's meaning you can't change it, but I wonder if there could be a way to mark it as "this could change at any moment", like a disclaimer.
     
    cxode likes this.
  7. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,336
    The timeline on public things getting deprecated/changed is also a lot worse for us than the timeline on internal things becoming public.

    If you decide to deprecate something when we're on 23.1, then we have until 23.3 to change it - that's a year. We can deal. If you decide to take something internal and make it public when we're on 23.1, then that lands in... 25.1 I believe? So that means that there's two years without being able to do anything.

    The approach where you put something in a
    .experimental
    namespace is vastly superior to the approach where you have things internal. The only practical difference is that our code is easier to read and performs better.
     
    LooperVFX, cxode, goncalo09 and 14 others like this.
  8. joshcamas

    joshcamas

    Joined:
    Jun 16, 2017
    Posts:
    1,277
    This!! ^^^^^ It communicates that stuff will break!
     
  9. Mike-Geig

    Mike-Geig

    Unity Technologies

    Joined:
    Aug 16, 2013
    Posts:
    254
    Just so you don't think you're being ignored, this has kicked off a very healthy debate internally! This is the type of stuff I love to see :D
     
  10. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    868
    I wasted endless hours trying to access some of the APIs of official and experimental Unity packages. Often I ended up with copying everything into the Asset folder and starting changing things like asmdefs. This often seems unnecessary protective for no good reason
     
  11. guoxx_

    guoxx_

    Joined:
    Mar 16, 2021
    Posts:
    55
    That happened quite often. It becomes even worse that some packages are not hosted in GitHub, so we have no way to send a pull request to the developer, and we have to integrate our custom modification whenever we upgrade a package.
     
    LooperVFX likes this.