Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

Plugins and Resources inside package

Discussion in 'Package Manager' started by iPickle, Aug 19, 2019.

  1. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    39
    Hi everyone,
    I've been working on moving most of my reusable code to packages, and I encounter a weird problem...

    When I create the package (meaning, the package is located in UnityProject/Packages/com.domain.package-name), everything works as I expect it, even when building, everything works

    No the problem starts when I upload my package to a registry, and then download it from a different project (so now the package is located in UnityProject/Library/PackageCache/com.domain.package-name@0.0.1, and is read only)
    The plugins inside the Plugins folder are not loaded (or at least not accessible from other scripts, no even inside the same package folder)
    And Resources cannot be loaded (both from Resources folder, or for example, editor icons, etc)

    Anyone knows how to set it so it will work?
    Thanks!
     
  2. ethan_jl_unity

    ethan_jl_unity

    Unity Technologies

    Joined:
    Sep 27, 2018
    Posts:
    99
    Hi there,

    Just to clarify, the plugins & resources folder worked when you place them under `UnityProject/Packages/com.domain.package-name` but not when you have it published and downloaded elsewhere?

    Also could you provide a bit more info on which system and what version of Unity you are using?
     
  3. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    39
    @ethan_jl_unity Hi! thanks for the reply

    I'm using unity 2019.3 (but also test on 2019.2) working on windows machine
    Testing this for android platform, but it reacts the same when I tried for windows

    My structure is this:
    UnityProject/Packages/com.domain.package-name
    -- Resources
    -- Runtime
    -- -- Plugins
    -- Editor
    -- -- Plugins

    Every thing works this way.

    So I publish my package to an npm repository, and I use a new project to download the package.
    Now, in the new project, the package is downloaded fine, but the plugins are not loaded, the resources cant be accessed

    I mentioned the package path because that's the only difference, when I download the package from the registry, unity saves it in the Library folder, and the files are marked as "Read Only"

    Edit:
    A quick question.... does Plugins work the same with assembly files?
    Do I also have to reference the assembly file in order to work with them?
    Maybe this is whats wrong with my plugins....
    But if so, why is it working when the package is in a different location?
    Im confused......
     
  4. JPFerreiraVB

    JPFerreiraVB

    Joined:
    Sep 18, 2017
    Posts:
    35
    I had the same problem.

    Do you have assembly definition for Runtime and Editor folder?
     
  5. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    39
    I do, I actually have a few packages, so I have a few cases
    One is a full package with plugins and cs code inside, so i have the assembly files in the runtime and editor
    I have another package that I use as a dependency, this one have ONLY plugins, so I dont have an assembly file since I get a warring that the assembly will not be compiled because it have no other scripts there

    Both cases behave the same as I described in the previews post
     
  6. JPFerreiraVB

    JPFerreiraVB

    Joined:
    Sep 18, 2017
    Posts:
    35
    Not shure if it helps.

    But in order to fix my problem i did the following IN THIS SPECIFIC ORDER.

    1 - Create the folder for package
    2 - Create Runtime and Editor Folders
    3 - Create Assembly Files for each folder
    4 - Add Reference in the Editor Assembly file, to the Runtime file.
    5 - Mode Editor Scripts and Runtime scripts to their folders

    The final structure of my package is this:
    -MyPackage
    --Editor
    ---EditorPluginA
    ---EditorPluginB
    ---unity.mypackage.Editor.asmdef
    --Runtime
    ---MyScriptFolderA
    ---MyScriptFolderB
    ---ThirdPartyScriptsFolder
    -----PluginA
    -----PluginB
    ---unity.mypackage.Runtime.asmdef

    With this i was able to use the package in the project where i created it, and to upload to a private repo, and use it in another project
     
  7. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    39
    Thanks for the replay
    This is working as you described if you only have your own c# scripts in the folders, when I used plugins (dll), Its not getting loaded
     
  8. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    39
    Maybe this is a hint, the dlls that normally load as managed plugins, now marked as native plugins... maybe this is why its not loaded in the editor? why is this happening?
     
  9. ethan_jl_unity

    ethan_jl_unity

    Unity Technologies

    Joined:
    Sep 27, 2018
    Posts:
    99
    Hi there,

    Would you mind filling a bug report for this issue so we can investigate this further on our end? (https://unity3d.com/unity/qa/bug-reporting)

    Please include the information about the managed plugin being recognized as a native one, which could be a key to this issue.
     
    Last edited: Aug 21, 2019
  10. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    I just bumped into this. In 2019.2.0f1. I had Newtonsoft's json dll in a Plugins folder under Runtime folder and also defined as a precompiled assembly in the Runtime's asmdef. In the development project, it all works. As soon as I downloaded the project from a Git URL, the Runtime asmdef is the same if opened with a text editor from inside PackageCache, but if viewed on the Inspector, it shows None where it should have shown the Newtonsoft assembly name. Specifically it shows a message bubble that the greyed out assembly is missing and will not be referenced during compilation.
    Also if you copy the Plugins folder(can't move it as the folder is read only) inside the Assets folder, you get duplicate assembly errors.
     
  11. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    39
    Yea, this is annoying, I solved this by having the plugins inside a unitypackage, and import it from an editor script automatically after the package is imported

    No optimal at all, but it solved everything for me, including sharing dlls between packages (because unitypackage imports the meta file with the object GUID, you shouldn't have duplicate dlls)


    There is another bug, you cant export unitypackage from any assets inside a package, not even in development
     
  12. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    I asked for this in another thread. Since packages might go through custom deploy scripts(see TextMeshPro) to import stuff that aren't accessible directly from packages folder(e.g. Materials), at least make the deploy script integrated with the whole package pipeline so you don't need unitypackages nor metas. Also asmdefs have a dll name in them when it comes to precompiled assemblies. Doesn't need a GUID. It'll get whatever GUID is attached to the assembly name in the project. If it's the same assembly it should only exist once anyway. Otherwise the compilation will probably complain of duplicate/ambiguous entities(I actually tried it out and unless I missed a combination, nearly all times the compiler complained). If I'm not mistaken, unitypackages disable entries they can also find in the project they are imported into.
     
  13. iPickle

    iPickle

    Joined:
    Oct 21, 2013
    Posts:
    39
    Well, about the assembly GUID, you can just uncheck use UGUID on the assembly file, and it will use names in stand

    Materials, as far as I tested, can actually be used from packages, so does models, what is the case that was not working for you?

    I do support your idea of having an "integrated" way to have "OnPackageImport" or something, because right now I am using an editor scripts that runs every compile, and potentially, it can be slow if you have too many of them
     
  14. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    I opened the asmdef in a text editor and it reveals that the precompiled dll dependency , from within the package, is linked to by name not by guid.

    Indeed, I can't repro it( at some point, I had an issue referencing a material so I assumed that deploy scripts are used for this purpose). On the other hand, if assets are accessible, why does TextMeshPro need to copy over essentials(shaders, materials etc) inside the project? I don't imagine it's for write privileges(you can't mess with package stuff, only read them).

    Yeah, I've made sth like this for a UnityPackage some time ago. The deploy script I made for unitypackages were using InitializeOnLoad, in which case, you comment that InitializeOnLoad attribute out until you actually need it( or until you actually package it).
    Btw I think you can use conditional compilation for this and use script build were you can enable the flag prior to building. That way, even on the dev machine, you don't incur the deployment while developing the package.
     
  15. g__b

    g__b

    Joined:
    Sep 17, 2014
    Posts:
    29
    Hi all, thanks for your posts, they are really helpful.
    I have created a custom package to wrap an opensource library, it has the same structure defined above in this thread.
    I have problems including native code in my package, I always get DllNotFoundException.
    Should I create a package with just native plugins and set this as a dependency for the other package?
    BTW, my plugins (.dll, .so, .framework) are recognized as native on the editor, is this a problem?
    On macOS, I have the library in the form of a .framework. Does it need to be a .bundle?
     
  16. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    Make sure the asmdef referencing any dlls. Look it up in the documentation. Otherwise when the package is uploaded, it ignores the unreferenced stuff(I think).
    Also are you by any chance using any LFS? If so, expect that it might have problems for users. For some reason things tracked by LFS get fetched as corrupt file pointers locally when the package is retrieved in the Package Manager. Not sure if it's fixed lately, but at the very least, look out for such an issue.
     
  17. g__b

    g__b

    Joined:
    Sep 17, 2014
    Posts:
    29
    Thanks for your answer.
    I tried referencing the libs from the Assembly definition inspector (setting override references), but I can only reference nunit.framework.dll. What should I do to reference them? editing the json manually?
    I also tried adding the plugins in a custom package with no assembly definitions, and added that package as a dependency in another custom package where I have the code that uses the plugins, with no luck.
    I'm not using any LFS, so that cannot be the problem.
    [EDIT]
    They are NATIVE plugin, not managed ones:
    so, how to reference native plugins in unity package manager?
    I tried editing manually the json but it doesn't work.
    I tried creating an assembly definition reference (asmref) but it's useless since I can only reference existing Assembly definitions.
     
    Last edited: Apr 13, 2020
  18. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    As far as the custom package docs go, they do mention native assemblies. But there's no other mention, which I assume means that there should be no special treatment in order for the dll to be packaged, also means there's no reason to create a separate package just for the dlls. I found another thread where someone else said that native plugins don't have to be inside a Plugins folder(e.g. Runtime/Plugins), it could be anywhere inside Runtime and it should work. Aligns with my assumption from the documentation. But out of curiosity, what's your case? Where is the native plugin , folder wise, inside the custom package directory?
     
  19. g__b

    g__b

    Joined:
    Sep 17, 2014
    Posts:
    29
    I tried everything I could.
    Let's focus on Windows: if I place the .dll inside Assets/Plugins it works, my C# wrapper (that resides in the package) sees the dll and I can use it.
    If I place the .dll inside the package, it doesn't work, I have a DllNotFoundException.
    The structure of my package is:

    PackageName
    - Scripts
    -- Editor folder
    ---- myLib.Editor.asmdef
    ---- myLibEditor.cs
    -- Runtime folder
    ---- myLib.asmdef
    ---- myLibWrapper.cs
    ---- Win64 folder
    ------ myLib.dll
    ---- Android folder
    ------ myLib.so
    ---- macOS folder
    ------ myLib.framework (I read that this should be a .bundle, I'll try to convert/export the library as a bundle)

    I tried putting the libs inside a Plugin folder inside the package, but I always get a DllNotFoundException.
    I also tried not using a "Scripts" folder to contain everything, but since in the future I will add resources and example scenes, I'd like to separate things at this level.
     
    Last edited: Apr 11, 2020
  20. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    Sounds like a bug. I can't think of anything else. File a report for good measure.

    PS: Have you tested it with different unity versions? Does it happen on others as well?

    PS2: Never asked, but better make sure, the dll itself is not present in the host project where you download the package right?
     
  21. g__b

    g__b

    Joined:
    Sep 17, 2014
    Posts:
    29
    I did some other tests, sounds like a bug of the editor, the build is working!
    I'm on the latest UnityEditor (2019.3.9f1).

    I tried creating the Visual Studio projects with 3 cases:

    - dll placed in Assets/Plugins:
    a Plugins folder is created inside vsproject/build/bin/AppName_Data, the dll is there and it works
    - dll placed in Package/Scripts/Plugins:
    a Plugins folder is created inside vsproject/build/bin/AppName_Data, the dll is there and it works, doesn't work on Editor (DllNotFoundException)
    - dll placed in Package/Scripts:
    a Plugins folder is created inside vsproject/build/bin/AppName_Data, the dll is there and it works, doesn't work on Editor (DllNotFoundException)

    Same behaviour on UnityEditor 2019.2.18f1,
    except that I can see 6 lines like these before the DllNotFoundException:

    Plugins: Failed to load 'C:/Path/To/Project/Packages/PackageName/Scripts/Runtime/Win64/myLib.dll' because one or more of its dependencies could not be loaded.

    On UnityEditor 2018.2.6f1 is not working neither on the Editor nor in build (DllNotFoundException), wherever I place the dll (Assets/Plugins, Package/Scripts/Plugins, Package/Scripts) and that sounds really strange (nobody ever noticed that??), I'll look for some bug report about Plugins on this version.

    Could the problem reside in the dll in some way?
     
    Last edited: Apr 13, 2020
  22. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    Could be. Are there any dependencies in your lib? Does it link to any other dll internally which you are not providing in the package? Because it shows that it clearly tries to load it but cannot do so fully.
     
  23. g__b

    g__b

    Joined:
    Sep 17, 2014
    Posts:
    29
    I'm almost sure the .dll has no dependencies, or how could it work in the build?
    I'll investigate further to be 100% sure.
    If you're curious, this is the opensource library I'm trying to wrap inside a Unity Package:
    https://csound.com/
    https://github.com/csound/csound
    I'm using the csound64.dll found in the 64bit Binaries v6.14.0 archive.
    This is the existing project that wraps this powerful library for Unity https://rorywalsh.github.io/CsoundUnity/.
    Here is the code that wraps the native API.
    It's working but it uses the StreamingAssets folder to hold the libs for the various platforms. And has some problems when building for macOS (assumes the Csound framework is installed in the system), so I thought that creating a package could help with the distribution and use of this library. Harder than expected o_O
    BTW thanks again for your help :)
     
  24. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    Question, why did you use the Streaming Assets for the libraries? Didn't they get built with the built player when they were in the Plugins folder? Or are you downloading them?

    PS: Do you have a link to the Package Project repository?
    PS2: Not sure, been a while since I last used a native dll, but I see the kernel32 dll referenced up top. Not sure if Unity links that automatically. You could try putting the kernel32.dll next to your dll in the PackageCache of the host project and see how it goes. I could be wrong about this though but it's the only reference outside the dll you have.
     
    Last edited: Apr 15, 2020
  25. g__b

    g__b

    Joined:
    Sep 17, 2014
    Posts:
    29
    I didn't started the project, the author told me it was the easiest way he found to make it work for Windows and macOS.
    It works indeed (they are not built by Unity), but in fact it's not so clean to have things from a library inside the StreamingAssets folder. That's one of the main reasons I started to move everything in a Package.

    I have everything in local for now, I am going to create a branch of the repo soon, so that you can have a look!

    Great I'll give it a try! But if the kernel32.dll is not linked, how could it be that the library works when built?

    Again thank you for your suggestions! I'm glad you took some time to have a look :)
     
  26. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    152
    Technically the correct way would be to move the native dll(whatever tool compiles them) to a Plugins folder and provided you properly set up the import settings for the Windows player, it would be part of Unity's build. I've done it before, which is why it seems weird to see the StreamingAssets used. I've used StreamingAssets for config files(they remain text and are part of the build), but never for libraries.

    Sure, let me know when, either by DM or a reply here.

    Could be that Unity links to some stuff by default. But that one is only for the wrapper. You still have to add any dependencies the C++ lib itself is using. Otherwise the host machine that downloads the package might not have the dlls needed. I saw in csound that it uses libsndfile and boost and several others. You probably need to add all those to the build as well, or require them installed by whoever downloads the package and their paths added to PATH or sth so they are accessible while just csound.dll is part of the package. I see python and java requirements as well but not sure what they are. I hope they are not all used, otherwise you might have a problem. Are you in contact with the dev for such questions?(My cmake knowledge is limited, nor do I know how exactly that csound64.dll was built).

    Btw you should take a look at the built exe's output log. Not sure but I think it would mention the dll it's missing. If you're adventurous I think you can figure out the name of the library missing if you get a crash dump from unity's application and go through it in WinDbg(see here). You probably have to build as development build and make sure the debug symbols are copied over.

     
    g__b likes this.
  27. metaphysician

    metaphysician

    Joined:
    May 29, 2012
    Posts:
    128
    this sounds very interesting @g__b. i'm interested to see whether the integration is still working in Unity 2019.3 or even in 2020 beta. not being updated in 3 years does not give me a lot of confidence. i did work on it back in 2017 but not since.
     
  28. g__b

    g__b

    Joined:
    Sep 17, 2014
    Posts:
    29
    We are working on it and we are very close to an alpha release.
    You can try the dev2 branch of CsoundUnity and tell us what you think! And also contributions are welcome!
    We still have some problems building for macOS since the .framework isn't copied in the build, so we have to copy it manually or with a post build script.
    But apart from this there are lots of improvements!
     
unityunity