Search Unity

Package PerProject Assets

Discussion in 'Package Manager' started by Jes28, Mar 10, 2019.

  1. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    352
    What UT have for packages that create per project assets?
    - PerProjectSettings - I think this is must be default assets for each package and show it in inspector when we select package in project browser.
    - PackageSpecificAssets - TextMeshPro has many assets for functioning correctly that must not bloat assets folder but must be kept inside .comtextmeshpro/PerProjectAssets folder.
    - Generated Content - Pro Builder generate meshes on the fly and by default all the need to be ketp inside .com.unity.probuilder/PerProjectAssets/... folder and not bload user project too

    and so on.

    This issue will be far bigger when Packages start arrive from AssetsStore and each will bloat project with some assets in random directories.

    Is there some solution for such assets from packages?
     
  2. ethan_jl_unity

    ethan_jl_unity

    Unity Technologies

    Joined:
    Sep 27, 2018
    Posts:
    25
    Hi,

    Packages are not designed to contain any project specific content - and under `Assets` seems to be the right place put those files.

    Also, I don't think `TextMeshPro` and `Pro Builder` is the norm of how packages are. Do you see a lot of other packages that generate assets per project?

    Best,
    Ethan
     
    Last edited: Mar 11, 2019
  3. N3uRo

    N3uRo

    Joined:
    Dec 10, 2011
    Posts:
    640
    I'm with @Jes28 you need to analyze this situation.

    There should be an isolated folder or something like that so when we uninstall a package we can know what contents are from what package. To do this there should be a specific API where asset developers can do IO operations on this sandbox.
     
  4. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    352
    @ethan_jl_unity

    Many packages have something to store in Project folder and each do it in special custom way.

    com.unity.entities has settings - stored somewhere
    burst - has settings - stored somewhere
    lwrp - stores settings
    post processing - stores settings
    pro builder and pro grids - stores settings
    poly brush - store pallete, settings and something
    shader graph - store shaders (it most user friendly but also do it in custom way)
    ...

    For now authors of packages decide do I need their asmdef reference by each and every my code or not. What is not correct. This must be settings of project.

    For now author of package decide where to store additional assets inside project, how to name it and to refernece it. Pro Builder do reference it settings by string path so we even cannot move these somewhere.

    com.unity.entities have their build settings that we just need to know about and Add special defines to player settings
    Burst just add "Jobs" menu item and modify it settings from there.


    For now Package User after installing package must dig into manual to find all places where some settings was stored and how to tuneup package for project this is too bad.

    We need some Exact way from Unity where everything ProjectWise from package will be stored and UI on package itself that will show all package settings.

    No need to store PerProject data inside package dir (which is most clear way I think) you can standardize path like Assets/PackageAssets/[packagename]/ and add api to easy find it from package code. And just deny any package that store some settings or generated assets outside of this folder without permission from project.

    Things like compilation defines in Burst and Entities must be part of package settings that must be easily seeable and modifiable from package UI (when package selected in project browser)

    Assets created by any editor tool must use prompt to ask path where to store assets or just store inside PackageAssets/.../ directory
     
  5. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    Hi @Jes28, @N3uRo

    One main goal of the *Unity Package Manager* is to isolate or (sandbox as you have said) package assets. But, some of these assets cannot be sandboxed. Three examples that I'm aware of:
    • Samples
    • Editor settings
    • Runtime settings

    The main reason why these are not stored inside the package itself is that packages are immutable (read-only). Any assets requiring to be modified by the package consumer needs to be located under the
    Assets/
    folder. For samples, we standardized the location. They are always located under the
    Assets/<package name>
    folder. Once they are copied there, you can do anything you want with them.

    For editor settings, we have a defined flow to isolate the settings under the
    ProjectSettings/<package name>
    folder. It is possible that some package maintainers did not get the memo and store the editor settings under the
    Assets/
    folder. I'll validate with them.

    For *runtime setting* (and possibly other features), there are no guidelines as far as I know. This is up to the package maintainers to decide where they store them. From what I get from your post, we have collectively generated a mess for our users! We need to fix this! I will bring this issue with our internal package maintainers community.

    Thank you for raising this issue!

    Regards,

    Pascal
     
    Last edited: Mar 13, 2019
    Jes28 likes this.
  6. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    425
    is this a public API? I am currently writing an internal package that requires some settings, for now I create an SO asset, save it where the user wants with
    EditorUtility.SaveFilePanelInProject
    , then use
    EditorBuildSettings.AddConfigObject
    , but the file is still under
    Assets/*
    .

    afaik, I can't use
    AssetDatabase
    to save stuff in
    ProjectSettings
     
  7. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    Hi @M_R,

    Yes, there is a new public AP for that available since 2018.3. But, you need to manually set the config file location under the
    ProjectSettings
    folder yourself.

    See the SettingsProvider class documentation. If you look at the example, they still use an asset. But you can change that with your own custom file if you want. The Probuilder package uses a JSON file under the
    ProjectSettings/
    folder. I strongly recommend that you scope your setting files under your package name (i.e.
    /ProjectSettings/<com. my-company.my-package-name>/
    ). I don't know why the Probuilder guys did not do this!

    You can take a look at Probuilder package source code for an implementation example.

    I hope that we can, someday, provide an API that you just need to set the package name or something like that and all the magic is done for you.

    Regards,

    Pascal
     
  8. kaarrrllll

    kaarrrllll

    Unity Technologies

    Joined:
    Aug 24, 2017
    Posts:
    147
    Hi @Jes28

    I think you make some very valid points about the need for packages to be good citizens with regards to where generated assets and other per-project data is stored. As has already been mentioned, there currently are no guidelines for package owners to follow, hence the wide range of approaches you see today.

    I think internally we are all in agreement that there should be at the least recommendations to follow, if not rules enforced by the package verification process.

    That said, I'd like to address some of your points specifically relating to ProBuilder.

    I'm not sure if this was intended to be hypothetical, but just to clarify, we do not store user created meshes as unique assets.

    Currently they are serialized as part of the scene in which they were created.* This may change in the future, however, and I think at that point it will be important to be careful about where these assets are stored. I absolutely agree we should not be bloating a user's project with any kind of generated data.

    * A caveat being if you have enabled the experimental "Meshes are Assets" setting.

    ProBuilder will find it's settings files anywhere in the `Assets` directory, they do not have any special requirements as to where they are placed.
     
  9. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    425
    Hi,
    so there is still no way to have an asset saved under ProjectSettings...
    the SettingsProvider is not relevant for my package, as I have a custom window for it (my package generates code on-demand, the "settings" are parameters for that, I only want them persisted. it's not actual/global settings)

    I was hoping for something close to
    Code (CSharp):
    1. T AssetDatabaseOnSteroids.LoadOrCreateProjectSettings<T>() where T : ScriptableObject;
    that:
    - gets the package id from
    typeof(T).Assembly

    - loads or create the asset under
    ProjectSettings/{packageId}/{typeof(T).Name}.asset

    - is nice with
    SerializedObject
    (i.e. handles it being dirty and saves it with the project)
     
  10. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    @M_R, I'll share your proposal with the team supporting the Settings. API. But, I have a question. Why do you need the setting file to be an asset? What is the problem with a YAML or JSON file (or any file type you would like)?

    Pascal
     
  11. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    352

    May be because asset is actually YAML and it is most Unity and easy to use way to store something in Unity.
    Also most of Assets in ProjectSettings is Assets written in YAML format
    Additionally T can be TextAsset so we can write it in any other way if we want.

    Any way I think spread package info around many directories of actual project is bad Idea.
    Ideal will be to store all package related data (Shared and PerProject) in single root directory.
    I think Packages/[packagename]/ProjectLocal/.. will be best solution.

    And move package cashe from Library to Packages/[packagename]/ will be more understandable. just make it readonly to indicate it shared state and add button to "unpack" package (like unpack prefab) to make it project local package which just clean readonly state. May be even allow to mark project local not entire package but only one or few files, so updating package will update all files but local ones will try to merge using Merge tool setted in editor preferences.
     
  12. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    425
    @okcompute_unity because I want the additional features that
    AssetDatabase
    offers over
    System.IO.File
    :
    • I can load the asset without having to figure out the absolute path
    • it is automatically deserialized into a c# object, as a singleton
    • if I create a
      SerializedObject
      out of it and then modify its properties, AD will automatically:
      • track if it's dirty
      • track modifications with
        Undo
      • saves the asset to disk with all the other assets (and settings) when the user saves the project, if dirty
    • format on disk is coherent with other assets. if the user really want to use binary, they can. if you eventually decide to change from YAML to another format, it follows that
    • I could write a custom editor (/ importer ?) that will also handle SerializedObject management
    • etc...
     
    Jes28 likes this.
  13. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    @Jes28, @M_R ,

    Having a dedicated folder for packages related data is something that was discussed on our end too. We will continue to explore the idea.

    Thank you for your feedback! It gives me a better perspective on this topic :)

    Regards,

    Pascal
     
  14. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    352
    My best thoughts about user friendly package structure with regard to Source Control systems:

    Packages/ - root for all packages
    Packages/[package name]/ - root for specific package
    Packages/[package name]/@1.0.0-preview.0~ - main package content cache (~ is readonly sign)
    Packages/[package name]/@1.0.0-preview.0 - main package content local
    Packages/[package name]/samples/X~ - sample that is not imported to project
    Packages/[package name]/samples/Y - sample that is imported to project
    Packages/[package name]/projectAssets - assets that is local to project
    Packages/[package name]/settings - all package settings and per user editor preferences cache

    Source Control can use this ignore filter to ignore cached package data: [project path]/Packages/*/*~/

    Thus all package content will be concentrated inside single root directory of package and every user can easily find all package data, source code, settings, project assets, samples, etc.

    Additional show all package info and settings in inspector of package folder i.e. Packages/[package name]
     
  15. N3uRo

    N3uRo

    Joined:
    Dec 10, 2011
    Posts:
    640
    I'm with you, we need an API to load and save assets from project settings folder.
     
  16. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    @N3uRo,

    We have listened and we are looking at providing this kind of API for runtime and editor settings. Don't hold your breath though. We are in the brainstorming/analysis stage.

    Regards,

    Pascal
     
  17. N3uRo

    N3uRo

    Joined:
    Dec 10, 2011
    Posts:
    640
    How is this going @okcompute_unity ?? Can we expect this in 2019.3 alpha???

    Thanks.
     
  18. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    Hello @N3uRo , expect this to land in the 2020.x releases cycle or later.

    Regards,

    Pascal
     
  19. N3uRo

    N3uRo

    Joined:
    Dec 10, 2011
    Posts:
    640
    2020.x or later? Really? o_O
     
  20. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    352
    What about Asset Store packages?
    When they arrive?
    What shape they will be?
    How many trash they create in Assets folder?
    How we need to deal with all that?
     
  21. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    Yes. I'm deeply sorry. We are on the finishing touch for 19.3 right now. Work for 20.1 will start in a couple of weeks. This how Unity release cycle works (And a good reason why the Package Manager helps a lot by shortening this release cycle). One day, we will make the Package Manager a package itself and you'll get access to new feature faster :)

    Regards,

    Pascal
     
  22. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    That's one of our main focus right now. I can't give you a precise ETA for when *you* will have access to this new format. It will be a progressive transition. As you expressed, there are a lot of questions to be answered and we want to be cautious. Expect this to happen also during the 2020.x release cycle.

    Regards,

    Pascal
     
    Jes28 likes this.
  23. N3uRo

    N3uRo

    Joined:
    Dec 10, 2011
    Posts:
    640
    I'm surprised because the request we made about project settings has nothing to do with package manager.

    We ask a way to be able to load and save and asset from project settings folder because AssetDatabase class only allows to operate from Assets folder.

    There needs to be a way to load and save project settings without our code being a package...
     
  24. okcompute_unity

    okcompute_unity

    Unity Technologies

    Joined:
    Jan 16, 2017
    Posts:
    526
    Sorry @N3uRo , I clearly did not get your question :(. If you feel this feature request is unrelated to the Package Manager, could you please ask your question again in the *Editor* forum?
     
    Last edited: Jun 4, 2019
  25. N3uRo

    N3uRo

    Joined:
    Dec 10, 2011
    Posts:
    640
    I could but I'm afraid it's not going to be read by the right people and it's going to get lost between pages.

    That's why you're my hope.