Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice

Official IMPORTANT: Upcoming changes to the Graphics repository

Discussion in 'General Graphics' started by ali_mohebali, Feb 14, 2022.

  1. ali_mohebali

    ali_mohebali

    Unity Technologies

    Joined:
    Apr 8, 2020
    Posts:
    97
    Hi everyone!

    We would like to update you on changes coming to the Graphics public repository at https://github.com/Unity-Technologies/Graphics.

    We are moving the development of the Graphics packages from the public GitHub to an internal repository while maintaining the public repository as the mirror of our internal development. We are committed to keeping the SRP source open. You can continue browsing this public repository to track the latest changes per landed PR.

    Why are we doing this?
    We are making this change to improve our developer velocity, decrease the complexity of the development process, and improve our automated testability. This change will allow us to develop features and improve the packages faster, with higher quality and stability. We will also be able to test SRP packages integration better with the other Unity components.

    We understand from your previous feedback that keeping the Graphics public repository active and regularly updated is vital for you. We hear that it is essential for you to follow the development progress and have the option to cherry-pick individual changes for your custom forks. Therefore, we are investing in a mirroring service to continuously mirror changes from the internal repository to the existing public Graphics repository. We have looked at different solutions to retain most of the benefits of the current Graphics repo. However, to make sure we can significantly improve the development of the Graphics packages, we have to make some changes.

    What will be changed?
    • Active development: Development activity will move from the public repository to our internal one. The public repository will remain active, and the latest changes will be mirrored to this repository frequently. However, we will mirror the release branches only (2021.2/staging, 2022.1/staging at the moment, and master). We will not be mirroring the development branches.

    • Change in folder structure: We will move the packages to a new /Packages folder and the content in TestProjects to Tests/SRPTests/Projects. The mirroring service requires aligning the folder structure with the internal unity repository folder structure. We aim to ensure that these changes will automatically merge with your existing local changes. However, you might need to rebase your changes. We are prototyping this now and will provide more info in this thread once we have updates to share.

    • Changes availability and frequency: The availability of the latest changes will not be immediate. However, the latest changes will be mirrored frequently. Our goal is to ensure that the delay is not longer than three weeks.

    • Changelogs: The graphics repository packages changelogs will be integrated and synced with Unity Editor changelogs releases. This improvement has already been started with the Unity 2021 releases and ensures that the packages' changelogs are automatically available as part of each Unity release note.

    • Versioning: Going forward, we will increase the minor and patch versions in sync with Unity Editor releases.

    • The granularity of mirroring: Lowest level of granularity will be a Pull Request (PR). A PR will be squashed and mirrored as a single commit. You will be able to cherry-pick individual PRs for your custom branches.
    What will not change?
    • The SRP and Graphics repository source code remains open, maintained, and updated with the latest changes so you can continue to track the development progress and update your custom forks.

    • We aim to keep the current package changelogs in the public repository. We are now investigating how to update them from the core Unity release notes automatically.

    • You can continue working as you are with these public branches. If you have custom branches of SRPs, you can continue to merge and cherry-pick changes as before.
    When will the change happen?
    We will stop developing in the public repository by February 18. We will then migrate the content over to the internal development repository as the next step. Once all content is in the internal repo, we will run final tests with the new mirroring service. We will then start the mirroring service around March 11.

    Thanks for reading. As ever, we appreciate your feedback. You can provide your feedback or post questions in this post's comment thread, and our team will be on-hand to answer.

    Unity Graphics Team

     
    daneobyrd, Ruchir, Tanner555 and 5 others like this.
  2. Onigiri

    Onigiri

    Joined:
    Aug 10, 2014
    Posts:
    329
  3. SugoiDev

    SugoiDev

    Joined:
    Mar 27, 2013
    Posts:
    395
    I would like to voice my dissatisfaction with this decision.
    As many have said before, being able to see the commits is absurdly important, especially with Unity's track record and even moreso with the Graphics package!

    We are all developers here, so we can reason a lot about the development, struggles, goals, etc, just from being able to follow the commits and see the actual progression.

    Having access to the code itself is of minimal importance, since we can just take a look at the local package, and decompile any DLLs (as many of us have been doing for years to make sense of Unity's internals).
    Also, let's not forget that Unity killed the previous GitHub repository that hosted Unity's decompiled source and then proceeded to ignore the replacement they created themselves for years.


    I implore you to reconsider.
     
  4. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    Hi SugoiDev,

    we definitely want our development to be in the open. Every individual PR will be mirrored (as a single commit) so you have more information than just from the source in a released version. This way you can cherry-pick PRs and also get more context about changes in the code. The PR description will be mirrored as well in the commit message.

    And although WIP branches will not automatically be visible, our developers will also still be able to share ongoing work in a branch in the public repo to solicit feedback before we land a PR. In the end we want to build tooling around this to easily share WIP branches and integrate this into our process. We are definitely looking into keeping as much of what made the public repo useful for you. We need to start somewhere though and the solution mentioned above is what we know we can do in the short term.

    We are definitely committed to keeping the public github and mirroring service going. It's likely that other packages that are now developed privately will become public using this approach of developing in the monorepo where everything can be tested together and mirroring out to a public repo to expose a view on the development.
     
    t_tutiya likes this.
  5. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,047
    Can you please explain why this is the case? Specifically, how does this move enable you to "develop features and improve packages faster"?

    Without that piece of the explanation, this sort of sounds like a bunch of meaningless marketing speak. I.e. why was the public repository holding your team back in the first place? As a software developer, it is difficult to understand why retargeting your git remote from one address to another would impact development velocity.

    This statement is simply incorrect, though, isn't it? Aren't your feature branches being squashed? Cherry-picking changes, then, ends up with a completely different granularity than before. Depending on the number of changes in a given feature branch, it may actually be faster to simply find the relevant lines and copy them around. What's more, individual commits can frequently reveal lots of useful information, especially when the code itself lacks any useful comments. This also requires your teams to have excellent commit discipline, of course...

    Can you please explain why this delay exists in the first place? One would presume that such a system would be 100% automatable. Having a "goal" of "not longer than three weeks" is far, far less helpful than having a rule. We cannot trust a goal as goals are frequently missed.

    Additionally, is that three weeks delay the delay between feature branch merging or actual Editor release? If the latter, then have you considered how this may affect Asset Store Publishers who have to integrate lots of stuff with the SRPs? This would potentially imply that publishers wouldn't have access to changes until "up to three weeks post-new-Editor-release if the goal is met". That opens a window where Unity's and Publishers' mutual customers will get really upset at Publishers due to assets not being fully compatible on day one (or possibly even month one).
     
  6. SugoiDev

    SugoiDev

    Joined:
    Mar 27, 2013
    Posts:
    395
    Sadly, that's far from enough, especially with the implication that this is not automated.

    I'm sure we'll have many developers voicing similar opinions in this thread, and I hope you'll take those into consideration to, at the very least, keep a full (every commit), automated (and I can't stress this enough) mirror of the monorepo visible to us developers.
     
  7. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    Everything is automated indeed but in the first months we are introducing a review stage to verify that the mirroring service does not copy other pieces of the code that could be under NDA. That's why we mentioned the conservative estimate of three weeks. This is the delay between when a PR lands in the private repo and when it is visible in the public repo. This should be visible once the unity editor is released with the change. Once we are confident that the mirroring service is solid and we have the necessary guardrails in place then we can remove the manual verification step to be fully automatic. At that point it should be a matter of days at most.
     
  8. vis2k

    vis2k

    Joined:
    Sep 4, 2015
    Posts:
    4,259
    What ever happened to "Democratizing game development" :)
     
  9. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    With this change we are not merely switching the repo from public to internal. We are bringing together the whole graphics stack into one monorepo. The graphics packages and core unity (that contains the platform backends, the graphics abstraction and the core graphics features) will be in the same repo versioned together and tested together automatically. There is a lot of research about the benefits of a monorepo versus a polyrepo. This paper covers some of the pros and cons in large organizations. Every organization is unique though and in Unity developing in one monorepo means we'll be able to have much better automated test coverage of the entire graphics stack, have less complexity in our development process when evolving our graphics stack across the software layers, have faster bisections when investigating bugs on specific platforms, simplify manual QA work, etc. A lot of the reasoning behind why this will help us to accelerate our development is very specific to our current workflows.

    Unfortunately, the core unity source code contains confidential platform information so bringing everything together in a monorepo means we need to develop on the internal repo. We definitely did not want to close things up. That's why we are investing in the mirroring service. There are some constrains now currently to make sure we don't mirror other code by accident but once our confidence increases we want to improve on what we presented in our post here.
     
    Last edited: Feb 14, 2022
  10. Zentropy

    Zentropy

    Joined:
    Sep 30, 2016
    Posts:
    3
    Anyone who's spent longer than 3 months in Unity ecosystem who is remotely shocked by this is deliberately burying their head in the sand.
     
    Prodigga and jdtec like this.
  11. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I always felt Unity should be higher performing, more reliable, cohesive and be somewhere ahead of where it is right now. If this change helps that, then what more could I want?
     
    daneobyrd, NotaNaN, Oxeren and 3 others like this.
  12. LordDarkon76

    LordDarkon76

    Joined:
    Apr 1, 2015
    Posts:
    22
    Git has the option for Submodules it isn't a technical problem to have the module public.

    It gives the feeling that are trying to hide bad development.
     
    funkyCoty and equalsequals like this.
  13. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    558
    I support faster development! Whatever gets me display HDR in URP faster! XD
     
    daneobyrd, mariandev and phobos2077 like this.
  14. equalsequals

    equalsequals

    Joined:
    Sep 27, 2010
    Posts:
    154
    Provided that the mirroring solution actually works as intended, it seems like we will (mostly) not lose anything aside from increased latency with the ~3 week delay and decreased granularity with the commit squashing. Is my understanding correct?

    I maintain our in-house custom SRP and I rely heavily on the GitHub repo's staging branches for inferences to otherwise undocumented APIs in lieu of any actual documentation (inline or other). If this suffers from this change at all it is taking an already very inconvenient workflow and making it painful.
     
    daneobyrd and florianBrn like this.
  15. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,962
    Could you explain why making these changes does any of these things? What about the current workflow is preventing say automated testing? What workflow issues does using Github have that internal repos don't?
     
    florianBrn likes this.
  16. tatoforever

    tatoforever

    Joined:
    Apr 16, 2009
    Posts:
    4,268
    Im smelling a new built-in renderer...
     
  17. merpheus

    merpheus

    Joined:
    Mar 5, 2013
    Posts:
    190
    So you cant manage your own development environment without making it private, but expect us to nitpick stuff out of squashed commits? Plus, it is obvious that this "somewhat open code base" idea is not well maintained as it took ages for unity editor csharp code to be updated to the latest editor releases.

    The year is 2022. Instead of making the whole engine code public, it's getting more and more internal and hidden. The granularity of gfx repo was very useful to track individual WIP features and test them on our environment before it lands. So that it is easier to adopt new features and come up with fixes and PRs possibly.

    Honestly, I am not happy with this. Why do we have to be constantly disappointed in using Unity instead of Unreal?
     
  18. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,559
    We actually don't use Git for the core Unity source code today (though we're working on transitioning to it this year), but submodules also do not eliminate the need for multiple pull requests in separate repositories. By moving to a more monorepo-like setup, graphics developers will be able to land core-side and SRP-side changes atomically.

    As @AljoshaD mentioned, a major concern is NDA materials - e.g. we cannot store CI job definitions for NDA platforms in the public repository like we do for other platforms, because it would disclose things like the names of executables in the SDKs (and we could debate whether it's really necessary for NDAs to protect such things, but nevertheless, they do protect such things, so we honour that). We've done our best to work around the problem from early in the SRP project, but the result has been a huge amount of friction for the developers. Moving the 'source of truth' to be internal allows us to incorporate all of these NDA pieces directly into the repository, just like any other platform, and then just strip them out in our mirroring process, much like we already do for customers who license the core Unity source code.

    We know that being able to see and understand the history of development is valuable information, which is why as @ali_mohebali said, we're investing in a mirror service - we want to mirror as much as we can, while still protecting our partners' IP, and without creating lots of friction for developers. As @AljoshaD said, 'three weeks' delay is a conservative estimate while we get confident that the process is working correctly; we absolutely want to get to a place where we are much faster than that.
     
  19. slime73

    slime73

    Joined:
    May 14, 2017
    Posts:
    101
    Many of the existing PRs/commits targeting stable versions of HDRP are themselves already a big group of sometimes unrelated changes - for example this one https://github.com/Unity-Technologies/Graphics/commit/3179cc5e90c0bc23f00d0489f1f48f664f13614a . With the current setup it's already kind of annoying to trace actual changes through those monolithic commits (doing so has been necessary many times for me as part of making my large Unity project shippable as well as for submitting bug reports that are actionable by Unity), but since those monolithic commits link back to individual smaller PRs at least I'm able to do that tracing myself.

    It sounds like the new setup might lose useful information like that (e.g. PR links or commit hash information for the original changes a monolithic backport PR is comprised of) – is that an accurate guess, or do you have a plan to make those at least as usable as they are currently?

    Also, will existing/old PR and branch information be preserved once the mirroring begins? Losing the old ScriptableRenderPipeline repository a couple years ago meant a bunch of history links got broken and it got harder to debug issues, I'm hoping something similar doesn't happen again.

    Unfortunately it sounds like the development of the Unity projects I work on will be hurt by this change because at least some of the workflow points I brought up the last time around will still be impacted: https://forum.unity.com/threads/cha...github-repository.1029502/page-3#post-6699961
    For example I've been keeping a close eye on the HDRP water system pull request content and comments (not just commits) recently and it's helped guide where I spend time, and now I'll be losing some of that.
     
    Last edited: Feb 15, 2022
    Elringus and Jeakim like this.
  20. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    In the new setup all these will land in separate PRs instead of a batched PR. This will make it much easier to cherry-pick at that granularity.


    yes it will be preserved. We will link up the history. The aim is that you can just continue pulling in changes from the release branches. However, we need to change the folder layout at the same time. We'll be running experiments in the next weeks to see how well git can handle our mirror setup and what steps are needed to rebase an existing fork.
    Also, all existing PRs, branches and history will remain available in the Graphics repo.
     
    DragonCoder, t_tutiya and slime73 like this.
  21. slime73

    slime73

    Joined:
    May 14, 2017
    Posts:
    101
    Thanks for the clarification! At the very least I feel much better about this than the previous attempt a year ago.
     
    daneobyrd, t_tutiya and ali_mohebali like this.
  22. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,047
    You realize that a "conservative estimate" is "one in which you are cautious and estimate or guess a low amount which is probably less than the real amount", yeah? Did you perhaps mean a "liberal estimate" here? One where you would estimate "plenty of extra time over what you actually anticipated"?

    This is, of course, part of the issue from our end - that your "three weeks" will actually be something like 3-8 weeks.

    I'm still a little confused/concerned here. Did you perhaps mean to say that "By the time a new Editor version is released, all Graphics Repo-related content will be available in the repository"?
     
  23. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    I assume the use of "conservative" is a matter of perspective. In this case, less time is better so a conservative estimate is more time than what we expect. However, I'm not a language expert nor a native English speaker so I might be wrong.

    In any case, I indeed meant that three weeks is the maximum and we assume that it will be less.


    Yes, that is exactly what I meant.
     
    SonicBloomEric and fherbst like this.
  24. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I have to wonder what it is it people are making that requires such cherry picking from source, that would cause this to be a big concern so I am very curious about what it is people actually need from the live repo...

    Last thing we tried (and failed to do) was merge the HDRP water shader. It was such a failure we wondered exactly what the point of this all was vs making local modifications?
     
    valarus likes this.
  25. slime73

    slime73

    Joined:
    May 14, 2017
    Posts:
    101
    Speaking for myself and my project, it's a larger game that runs on multiple platforms and doesn't have the exact same release schedule as major Unity/HDRP versions. Most of what I've cherry picked has been important fixes to specific bugs in HDRP rather than features (waiting for months for the backport -> release process isn't always feasible and has a lot more risk than pulling just the change we need). There are also features introduced in beta HDRP versions which aren't too massive like HDR tonemapping support, that can be important for a project targeting HDRP 12 for example.

    Aside from those, figuring out where I should spend my time implementing something myself versus waiting for Unity features versus backporting something from Unity has been invaluable. Being able to track development as it happens is a huge resource for me.
     
    Gekigengar, sacb0y, fherbst and 3 others like this.
  26. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    478
    In my opinion the git repository need a bit of a cleanup :)
     
    Ruchir likes this.
  27. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    864
    Speaking from experience in non-game-related industry: Git with submodules is a pain. You are adding lots of dependencies that way when you want to merge things and it makes having good continuous integration harder as well because you can end up with cyclic dependencies with the automated tests.
    For a newer, equally massive project as before, my employer has decided to go with singular monorepo.
     
    ali_mohebali likes this.
  28. OneManEscapePlan

    OneManEscapePlan

    Joined:
    Oct 14, 2015
    Posts:
    162
    I haven't yet had to do this with Graphics, but I did have a similar problem with the Render Streaming package. Render Streaming is still in a very rough alpha state. We developed some code to extend and work with Render Streaming - and then in the next version update of Render Streaming, the Unity team did some refactoring that completely broke all of our code. Because Render Streaming is a public repo with full commit history on Github, I was able to look through the individual commits, determine what had changed that was and wasn't important to us, and cherry-pick specific changes we wanted while forgoing those that would break our custom code.

    As frustrating as closed source and lack of transparency can be for us that are using Unity's products, we should keep in mind that open source and transparency do create their own headaches for Unity's team. It's easier to experiment and iterate in a private repo where you don't have to worry about the rest of the world scrutinizing and criticizing your temporary WIP code.
     
    BOXOPHOBIC, NotaNaN, SugoiDev and 4 others like this.
  29. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,047
    This (specifically) is also trivial to do while maintaining the open source repo. You simply have a separate downstream private fork where you do all of your experiments and, when ready, merge your branch upstream. It's pretty straightforward.

    I personally don't mind keeping experiments and early proof-of-concept code private. That's better private. What I find disconcerting is commit-squashing and lag time between when a release occurs and when the relevant code makes its way into the public repo.
     
  30. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    Hi everyone,

    I'd like to update you on our progress.

    Today we've completed the migration of the Graphics packages to our internal repo. All the latest changes from the release branches master, 2022.1/staging and 2021.2/staging are now in the internal repo. Our developers can now directly work on a monorepo that contains both the Graphics packages and core Unity (that includes the lower level graphics stack). The immediate benefit you will see is that once a PR lands in the monorepo then it will automatically be part of the next Unity patch release. Up until now, there was a minimum delay of 4 weeks between something landing in the public Graphics repo and that code shipping with Unity. From now on, changes to the Graphics packages will be available faster in a Unity release.

    Now that this migration is complete, we can start testing the mirroring service end-to-end. This service will mirror the PRs back from the internal monorepo to the public Graphics repo. It's likely that it will take a few weeks of testing until we have validated that everything works correctly. Therefore, we’ll have a short period in which the Graphics packages in the released Unity patch releases are ahead of what is in the Graphics repo.

    Once we have validated that everything works correctly then we can start the regular mirroring of the PRs. Now that we can experiment with the mirroring service we'll quickly get more clarity on our mirroring frequency and latency. We do expect to be able to do better than the 3 weeks delay we mentioned earlier. Just to be clear, this still means that the code will be in the public Graphics repo by the time a Unity patch version is released with this code. The end goal is to completely automate the mirroring service such that the delay is a few days at most. We temporarily have introduced a manual verification step to confirm that no confidential information is mirrored by accident.
     
    Luxxuor, EagleG, hippocoder and 4 others like this.
  31. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,047
    @AljoshaD Two things:
    1. That doesn't actually change the amount of time between fix-and-release, correct? I.e. what you're saying here is that the public (us) simply no longer have any insight into the changes that are coming in the next release of Unity until they land, yes?
    2. You understand how it is actually preferable to some users to be able to see changes coming before a release of Unity is publicly available, yeah? Asset Publishers, for instance, will now have zero chance of identifying breaking issues for their assets until the same exact day as end users who update and find that things broke.
     
  32. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,559
    No, it significantly shortens it. Previously, the graphics team had to land their PR in the Graphics repository, and then also land a PR in the main Unity repository to make the Editor adopt the new versions of the packages, a step which took weeks. That second step has now been eliminated.
     
  33. DEEnvironment

    DEEnvironment

    Joined:
    Dec 30, 2018
    Posts:
    380
    wow will the new LTS come two weeks early ;)
     
  34. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,047
    Got it, thanks for the clarification.

    That does make even tighter for Asset Publishers who track changes in the repo to understand the effects on their assets.

    Out of curiosity, do you happen to know what the new minimum delay between landing in the repo and shipping in Unity will be?

    This is all somewhat academic until the public mirroring solution gets worked out. If public mirroring has any meaningful delays then it would simply be faster to go look at diffs over on Needle Mirror and start figuring out changes manually. (This also assumes that PRs are kept at Feature-scale and not at ReleaseVersion-Scale because, you know, squashing...)
     
  35. vlery

    vlery

    Joined:
    Jan 23, 2017
    Posts:
    12
    Hi, I just have a question about the github version tag. I have seen recent changes while the latest tag is still 10.8.1. Is that possible for you to still maintain the version tag and make it consistent with the pkg version from package manager. I think it will be so helpful for us to compare, upgrade or migrate features.
     
    funkyCoty likes this.
  36. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    963
    First, I do appreciate the level of communication in this thread.
    That being said, three weeks is too long. I know you guys are aiming for something faster than that, but I just wanted to pitch in and say it really needs to be way more often than 3 weeks. I realize it takes time, just worried with this thread going inactive things might slow down and it ends up losing steam, never to happen.
     
  37. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    We are definitely working towards that. That worst case 3 week number means that code will be available in the public repo around the same time (or earlier) that the code ships with a core unity patch version.
    In the end, the change proposed here is to optimize our development process so we can get features and bugfixes out faster. As an example, let's say, with the old process a fix would land in the public repo on May 1st (done by dev) and shipped with Unity on June 1st. With the new process, assuming a dev can work more efficiently, the fix would land in the internal repo a bit earlier on April 20th, ship in a Unity patch version around May 5 (25 days earlier) and be available in the public repo May 5 or earlier. This mirroring is just a small piece of what impacts the time that something lands in the public repo. Even though there is a delay with the internal repo there will in the end be an improvement compared with the old situation. And for users that use the packages from the package manager the speed to get improvements is very significantly increased.
     
    Last edited: Mar 26, 2022
    mariandev and PutridEx like this.
  38. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,357
    The amount of bugfixes I could have done without full PR history would have been dramatically different. In fact I don't think I could have solved most of them without the access or it would have been super tedious as you can't really find the moment things break without full history and only alternative then is to reverse engineer the root cause which is always way more difficult.
     
  39. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,357
    I'm also saddened by this decision because it really leaves out all the WIP branches that Unity never merged that still had useful work put into them. I don't expect people who don't follow the development as closely to understand this but there's a real vault of information on these WIP branches that help you better understand how SRPs are used. If you only ever use whatever is given to you as is, of course it doesn't matter for you but people who have to figure the inner workings of these systems themselves, now one valuable resource is way less valuable.

    Couple this to the fact that none of the internal structures of SRPs are documented in any way. In fact there's often a clear aim to keep as much code as possible internal to reduce support and documentation amount.

    While getting full PR's as single commits is better than nothing, having PR's individual commits around would still tell a whole lot more of the code and what really happened there.
     
  40. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    558
    I think what you're missing here is these bug fixes instead of taking a month to fix could come out in a week without you having to fix them. In the time it takes for you to look over the repo to try to figure out and fix the problem someone who knows the software can read a bug report and likely have it out by next release, which won't take a minimum of a month. To me that's a much broader benefit.

    Sure the previous process made making fixes yourself easier, but also meant that unity releases were much slower.

    And at least for me the reason people use an engine like unity is to not deal with issues, and have issues that may occur get fixed by unity as soon as possible.

    Just about everyone can agree unity's development has been slow, and if there's was a legit month wait period as described anything removing that is a good thing even with the draw backs.

    People complain the SRP keeps changing but at the same time development is so slow no standard can really be formed. And by the time the next major release comes out everything changes. I would hope if the goal is better standardizing the way SRP's work the issues related to your understanding of SRP's from what you're talking about could be less nessecary. And those standards can be formed and built on faster so there's less to relearn.

    I'm just saying man look at the long term.
     
    Saniell and DragonCoder like this.
  41. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,357
    If it worked like that, yes that would be fine. But on my own part, I used to be able to fix some of the issues myself in few hours instead of waiting for weeks for Unity to maybe fix the issue. In my own experience the chances of Unity actually fixing some issue is way more likely if I can pinpoint the source of the issue to them and recently even that hasn't always worked. And now I can't even do that.

    Also in past when I've reported issues I have often seen dev branches for the fixes. Therefore I have been able to test the fixes early while they've been still being developed and have often given Unity devs feedback at this stage. This is not an option anymore either.

    I'm really not buying the "private repo leads to faster delivery". They could do that even with public repo if they wanted to and I don't even think Unity truly tried to communicate they needed private repo for this. It's just additional thing they are doing now. I do welcome faster delivery though.

    As a bonus this new approach means that they will not mirror things immediately, so getting hands to the source code history of the URP / HDRP version you are currently using is going to get even less likely if you like to keep your engine up-to-date.
     
    Last edited: Apr 6, 2022
    m0nsky and AydinDeveloper like this.
  42. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,357
    One thing I'm seriously concerned here is that we will only get final PR's mirrored as commits and Unity has actually moved to UE style setup where they pick a whole bunch of PR's into single commit when they merge them to main, this has been actually a standard practice for HDRP PR's for a long time already so being able to separate the changes for individual PRs is pretty much impossible already (if the change you want to examine falls under such commit).
     
    Last edited: Apr 6, 2022
  43. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    558
    I think we have a few years of evidence this simply isn't the case, I doubt this is exclusively the problem but likely a big part of it. I doubt the painfully slow development for the last few years was just to excuse this change. Just about everyone can tell faster development is needed to keep up, not just in graphics, new technology and formats, still no native USD support, etc.

    The problem seems so prolific Unity has been releasing what might be considered new features as packages entirely outside of Unity itself (Like the LookDev tool), to me it looks like they've been trying to deal with this for a while but it's too messy.

    I mean I don't exactly think you should need to do this in the first place. I understand why you do it, and if I considered myself a programmer I would do it too. But to me this is a service issue that shouldn't be considered necessary, just a bonus at best.

    I would imagine this faster process also means faster iteration of those fixes so personally I'm not concerned here.
     
  44. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    233
    How does this affect the process for contributions? I'd like to contribute some code; namely, I'd like to make dependencies on certain built-in modules optional. (For example, my project doesn't use the 3D physics module, but it *does* use post-processing.)
     
    Last edited: Apr 10, 2022
  45. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    It doesn't affect this.
     
    JesseSTG likes this.
  46. AljoshaD

    AljoshaD

    Unity Technologies

    Joined:
    May 27, 2019
    Posts:
    123
    Hi everyone, another update.

    We are getting close to starting the mirroring from the internal repo to the public repo. We first need to align the folder structures of both repos. We'll be landing a PR shortly in the public repo that moves a number of folders in the release branches master, 2022.1/staging and 2021.3/staging.

    If you have changes in your forks, then there are a number of steps to follow to easily merge the folder changes with your custom branch. You can find more detailed guidelines here.
     
    bac9-flcl and DEEnvironment like this.
  47. sabojako

    sabojako

    Joined:
    Jul 7, 2017
    Posts:
    48
    The new structure that landed no more has History of previous changes. That is quite cumbersome. Some of my saved PR web links still work tho. Is there some way we can see the old structure to navigate the previous history in the github webpage? I haven't forked the repo.

    Is there a tag or branch that corresponds to pre-structure change? There's one branch that looks like it and it's recent, but its named 10.x.x/release, which is not the latest number at all.
     
    Last edited: Apr 14, 2022
  48. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,357
    All previous commits are there on my end, which branch are you looking at specifically?

    Nobody is going to maintain two branches for each version from now on. Like mentioned on post above yours, this change only applies to master, 2021.3 + 2022.1 staging branches and whatever staging branches come after this. If you are a branch for older version it does not apply to you. If you are on newer affected version, just read those detailed guidelines on how to work with this.

    If you really have to get to last version prior this change, just checkout the branch with commit prior the restructure PR.
     
  49. sabojako

    sabojako

    Joined:
    Jul 7, 2017
    Posts:
    48
    I'm checking on master, on webpage: https://github.com/Unity-Technologies/Graphics
    The history is gone.

    upload_2022-4-14_16-16-37.png

    Like I said, I don't have a fork.

    I just want to be like to be able to navigate the History on the webpage like I have been doing for the past 3 years. To inspect PRs and do stuff like Blame on files.

    They surely must have saved a backup in a branch or a tag so that we can inspect the previous PRs. Nobody needs to maintain that branch.
     
  50. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,559
    https://github.com/Unity-Technologies/Graphics/commits/master

    Works for me?

    upload_2022-4-14_22-15-22.png

    EDIT: Ah, I think it's because things moved from the root in the `Packages/` subfolder, and Github isn't doing 'follow moves/renames'. If you clone the repository locally I think Git tools should be able to follow the history properly.