Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Feedback Package Manager behavior change in Unity 2023.2.0

Discussion in 'Package Manager' started by manuelgoellnitz, Nov 20, 2023.

  1. manuelgoellnitz

    manuelgoellnitz

    Joined:
    Feb 15, 2017
    Posts:
    398
    From the Release Notes:
    "Package Manager: Fixed an issue that caused Package Manager to immediately invoke a "Resolve" operation to revert unexpected changes to assets in immutable packages. Instead, Package Manager now logs a console warning."

    That means you now can change the content of a package and Unity will compile it as if the code was in the project.

    Code in the Project normally ist in git or some other system.
    But the packages will not.
    So when you accidentally change a script in a package, the changes will only affect the local editor.
    The mentioned warning only is shown once, when you change the file, which can be overlooked very easy.

    I would suggest:
    1. Show the warning on every compile
    2. Flag the package in the package manager as "modified" or something

    For me, the previous behavior was not a bug, but a feature.
    So when I wanted to modify a pacakge, I had to move it out of the package cache into the project. Then it you automatically be commited in the version control system.
     
    KyryloKuzyk likes this.
  2. APSchmidt

    APSchmidt

    Joined:
    Oct 31, 2023
    Posts:
    262
    I may be wrong but I don't think this is possible.
     
  3. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    6,507
    Wrong! ;)

    Like I tried to say above: That's the wrong way to do it. ;)


    You do NOT want to have your package under Assets. It is then no longer a package.
    If it's not your own package, there may be valid reasons to modify that package locally but I would try to avoid that as much as possible.


    The package workflow with regards to version control and the ability to modify a version-controlled package is as follows:

    - Clone or create a package repository somewhere on your machine
    - Import this package in Package Manager via "Add from disk"

    The package is then writable in every project you import it in. Any changes to the package will be made in the package's repository and can be committed upstream.

    Only if you work with a team do you need to employ more changes. Specifically you want to make sure every user has the package folder in the same relative location to the project. Every time you add such a source controlled package to a project you have to edit manifest.json afterwards to change the absolute package path to a relative path eg "../Packages/ThePackageIJustAdded".
     
  4. manuelgoellnitz

    manuelgoellnitz

    Joined:
    Feb 15, 2017
    Posts:
    398
    It seams what I wrote can be misinterpreted:

    What I wanted to say was that you don't have the packages in the same repository as your project.

    I don't have any package under "Assets", what I meant was: You can move the package folder from the library into the folder where the manifest.json file is. Then the package is flagged "Custom" and you can edit it and commit that local changes. I only meant that as an example on how modified packages can be in the project.

    The thing what happens in Unity 2023.2.0 is that you now can modify packages that are in the Library/PackageCache and only get a Warning. And you normally don't add the Library Folder to your projects repository.
     
    rdjadu likes this.
  5. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    571
    Hi @manuelgoellnitz,

    Thanks for the suggestions.

    For context, this change was done for two reasons:
    1) Allowing temporary changes to files in a package without first having to embed it was a frequent request. This means iterating is now faster for those cases, for instance if you need to debug the code in a package (add logs, force specific code paths, etc.)
    2) This will enable additional performance improvements. The forced synchronization with the package source was more of a side-effect of the underlying mechanism, which induced unnecessary overhead anytime the packages were resolved, even when everything was already perfectly in sync. Without this forced synchronization tied to package asset import, we'll be able to optimize the package resolution mechanism, which will also help with iteration speed.

    In any case, the recommended way to alter packages durably is still to either embed them (copy/move them to the Packages directory in your project) or use a "file:" dependency referencing a copy of the package in its own folder.
     
  6. dKleinTriCAT

    dKleinTriCAT

    Joined:
    Jul 2, 2019
    Posts:
    21
    Hi @maximeb_unity
    I'd like to disagree strongly with the first reason. While we also fell into the "I just edited this, where has my change gone" trap a couple of times when we started working with custom packages, this was more of a learning problem and if we had wished for a change, it would have been better readability of what is readonly and what is not (in unity and visual studio). What we certainly did not wish for was the ability to accidentally change a script in a package residing in the packagecache and have that change linger there on the local machine with no way of knowing that it even happened.

    Also I'm linking this thread from the accelerator, as I believe this to actually be a problem related to the package manager and its general changes in behaviour for unity 2022

    https://forum.unity.com/threads/cac...s-which-then-causes-asset-corruption.1516361/
     
    timmehhhhhhh likes this.
  7. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    571
    Hi @dKleinTriCAT,

    I took at look at that thread, but problems described there seem to be a disconnect between what's imported in the Editor and what's on disk. I'm not sure how that would be related to being able to locally modify package files (or rather, to not have the Package Manager immediately lock up the Editor while it's potentially reverting changes). You can still force the changes to be reverted by using the UnityEditor.PackageManager.Client.Resolve() method or the "Manual resolve" button in the Package Manager UI. Still, I'll make sure to follow-up with the person in charge of the ticket that was created to see if there's any link.

    That's a bit unfair, considering we print warnings which also end up in the Editor.log - though I can understand that warnings are easily ignored or forgotten about. For what it's worth, I totally agree that it would be better for these changes to alter a visible state on the package ("dirtied") so Future You can remember about a change you might have made if that change remains a while.

    Do you have any example of parts in the Editor where you would suggest adding this kind of information? I'm thinking the Package Manager window being an obvious place. It already indicates which package is "Local" ("file:" dependencies) or "Custom" (packages embedded in the project's Packages folder). These packages are mutable packages, i.e. packages you can modify "safely", but that's probably not obvious from the labels. However you cannot modify files directly from the Package Manager, so displaying more explicit "read-only-ness" there might not be the best place. Are there specific places in the Editor where you've been modifying assets in immutable packages?

    As for how package immutability is conveyed in Visual Studio (or any IDE in general, in fact), the Package Manager team's influence is pretty limited. The development, features and UX of IDEs is in the hands of the owners of those IDEs: for instance, Microsoft owns Visual Studio and Jetbrains owns Rider. Any usability issue in those IDEs arising from how they present or interact with immutable packages is something that they should be made aware of. I think you can reach out to them via the Code Editors & IDEs forum.
     
  8. dKleinTriCAT

    dKleinTriCAT

    Joined:
    Jul 2, 2019
    Posts:
    21
    Hi and thank you for the quick answer,

    Regarding the asset corruption:
    I am not sure if it is directly related to the ability to locally modify packages in the cache. But the bug is definitely linked to changes in how the package manager updates packages since unity 2022. The breakage happens when packages were updated and seems to be about the editor being unable to consistently import/update scripts from packages. The fact that assets in a package might have lingering changes that could impact that process struck me as a possibility. You're right they might be unrelated, but it's probably worth investigating (and a package manager problem anyway).

    Regarding knowing that packages in the cache are changed:
    As far as I have understood the op there is an editor log message when a change to a package is made, but that is it. If that is the case, there are probably dozens of situations where changes to a package might occur (manual or not) without one noticing or ever discovering. This is especially problematic as the package cache is not under version control (for good reason), further reducing the visibility of these changes. I really don't want to spend my time hunting ghosts in the readonly caches of me and my colleagues. Sorry if that sounded harsh, but it is a very definite fear of mine.

    Regarding readonly visibility:
    The package manager already does that quite well with the "custom" label, as does the inspector with not making assets editable. I would for example love a separation in the addressables group window between addressable groups in project/in writeable packages/readonly packages. In visualstudio I can see if the package is readonly by checking the path in the header of the filetab, but it would be nice if it showed a "readonly" label or something similar, so I wouldn't need to hover the tab to find out.
     
  9. I am just here to provide a vote:
    - in PackMan (you can add a label, or different color or whatever, or even move into a different section like you do with the custom packages)
    - and/or you can extend the "experimental packages are in use" label on the main toolbar with the ability to show "Modified Unity packages" and the mix of both experimental AND modified packages. If the user dismisses it, that's on them.
    Capture.PNG
     
  10. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    571
    Thanks for sharing your understanding and concerns, @dKleinTriCAT.

    Regarding the asset corruption, the package manager has not really changed how it updates packages. It still just fetches and deploys files to a folder under Library/PackageCache, then tells the Editor "here be packages, please import their files". It does not itself import anything, "load" anything, trigger any compilation, etc. These are decoupled steps executed by other subsystems of the Editor, which are triggered later during the Asset Database refresh process. Therefore, any problem with asset state mismatch between what's on disk and what's been actually imported really lies elsewhere. That said, this change (i.e. the Editor no longer aggressively reverting changes by making the Package Manager re-resolve packages) is probably correlated with other changes in asset importers, import/compilation/scripting/build pipelines, etc. which could make it seem like it's involved: when the warning is logged, it means the Asset Database finished importing the changed file, therefore the modified file was fully processed by its importer. That said, if you are ever able to create a repro project, or know about specific versions where you didn't have the issue and now do, you should definitely report a bug with this project and/or information and we'll definitely look at it.

    Regarding the change in how we handle immutable package modification, your understanding is correct. We now log warnings instead of immediately triggering a Resolve Packages operation. Just to clarify, this was introduced during the 2023.2 Alpha and not backported to previous releases, so it doesn't affect 2023.1 or any 2022 LTS version. If you encountered the corruption issue in any of those versions, then it's a proof that it's not related to this change in the handling of immutable package modification.

    I understand - "But it works on my machine!" issues are definitely very annoying to troubleshoot. I'm still not sure this change will lead to massively more occurrences of such issues, at least if your users pay attention to warnings (which will pop-up every time a modification occurs) and automated tasks flag them as problematic.

    To be clear, I fully support the idea of somehow labeling or flagging packages as altered, with an easy, efficient way to revert individual packages. I'll definitely pass this on to the product team.

    Thanks for your suggestions, I'll include them with the information I pass on to the product team.
     
  11. dKleinTriCAT

    dKleinTriCAT

    Joined:
    Jul 2, 2019
    Posts:
    21
    @maximeb_unity Thank you for your time and the clarification. So I'm still not a fan of the behaviour changes in 2023, but they are at least unrelated to the asset corruption. I gotta admit I am still sceptical about entirely dismissing the package manager as a culprit, as the asset corruption only seems to happen during package updates, which seems like a pretty strong link to me.

    Regarding bug reports, there are two already. @Tim2021 created one while investigating the accelerator, while I created another one after realizing that the accelerator wasn't the guilty party (as the issue als pops up without it). I can't speak for Tim2021's ticket, but mine (IN-60092) has been on the incoming status for close to a month now, with no acknowledgement or reaction from the unity support side whatsoever. As this is heavily disrupting our release schedule I am currently exploring other options. After your explanations I created a new thread in the asset import section and have linked the previous discussions there, hoping for this issue to gain some traction of any kind.
    https://forum.unity.com/threads/inconsistent-script-imports-when-updating-packages.1523956/
     
    Last edited: Dec 4, 2023
  12. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    571
    Hi @dKleinTriCAT,

    I'm not sure what's holding back your incident report. Thanks for sharing its number, I'll chase it up and see what's going on.
     
    dKleinTriCAT likes this.
  13. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    571
    Hi @dKleinTriCAT,

    I found the Incident ticket. I have no idea why it wasn't processed sooner but I'm nudging the right people to move it forward.

    When you upgraded to 2022.3.10, which version did you upgrade from? You noticed that the issues seem correlated with updating packages, but you said it was not directly/easily reproducible. I think you could probably reproduce the issue with a folder under Assets that has an .asmdef with scripts where you do this renaming, then switch between Git branches/commits to emulate a package "upgrade"/"downgrade". The notes you added later in that ticket really seem to point to a bug in the script importer/compilation pipeline.
     
  14. dKleinTriCAT

    dKleinTriCAT

    Joined:
    Jul 2, 2019
    Posts:
    21
    @maximeb_unity
    Thank you for staying on this, it really means a lot that we finally might be able to make some ground here :)

    We upgraded from 2021.3.28f1. We didn't notice the issue right away so it's hard to say when the first corrupted imports occured. Took us a while to put the reports about more and more failing/broken builds and weird issues in editor&playmode together. But we do have a couple of projects still on older unity versions (2021 and even 2019) and those teams haven't had any issues yet, so we believe it is a unity 2022 and (likely) upwards problem. I'll try your suggestion and see if that triggers it reliably. I'm guessing the rename just makes it more likely to break and doesn't trigger the breakage itself, but it has been hard to pin down without additional knowledge about the import/update/compilation process. We'll see.

    In the interest of not derailing this feedback thread any longer, I'm of course open to continuing this conversation in the ticket or the new thread I linked above.
     
    maximeb_unity likes this.