Search Unity

[Solved] Force reload package

Discussion in 'Package Manager' started by dzamani, Feb 13, 2019.

  1. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    122
    Hi,

    I'm looking for a solution to this specific issue: let's say I have a local package which handle some basic code generation.
    Now that part is working, my issue is that if the project is in a state of compilation error I'm not able to update the source code of this package and then calling the code generation in order to fix the compilation error.

    A clearer example would be a typo in the code gen process which would create files that would not compile. So I would update the code gen (the .dll of this package is actually correctly refreshed from what I see with ILSpy after the typo fix) and I expect that if I trigger the code gen now it would work since the .dll is up to date but it's not (I trigger the code gen from a MenuItem).

    Now in 2018.1, Unity team made a change like this:
    If I had access to this, I would be able to reload my package or at least change the order of compilation so that I would compile the package and then try to compile Assembly-CSharp.

    Anyway, is there any known solution to that issue ? Alternatives ?

    The only solution I see is to make the code gen in something else like Python and just execute the script with Unity, that way it would work. Could also try Mono.Cecil but not at this point of my project.

    Thanks!
     
  2. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    Hi @dzamani,

    All Assembly Definition files are built before the project assemblies. Their compilation order is determined by the References they declare. This cannot be changed. The snippet you quoted is special startup logic that is not used after startup. After startup, Unity ensures your compiled assemblies are always consistent, so all compilation errors must be resolved before it reloads the active assemblies. Not doing that would lead to all sort of problems with inconsistent states that would be really hard to clear up. E.g. Assembly A got recompiled after being modified, but there's an error in Assembly B's code so we're using the old version of Assembly B... which might be incompatible with the modified Assembly A!

    Anyway, the issue you're running into seem fairly clear - basically, any change you make to your package cannot be used by Unity until all compilation errors are resolved, but your changes are meant to fix those errors by fixing the generated code. I might have a couple suggestions:

    - Write your Code Gen outside Unity (in a regular .NET solution) and just execute with Unity. You only need some hooks to call you code generator, e.g. in a custom EditorWindow or using your own MenuItems. No need to change to some other language (Python etc.), by the way; you may need to invoke your code generator using Mono if you need cross-platform compatibility.
    - Make your code generator data-driven (see Microsoft's T4 templates for an example), so that you can change your code generator output without changing the code generation logic. That's easier said than done, but I've seen it done before.
     
  3. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    122
    I understand this kind of logic but since the .dll for this specific package is updated, I'm missing the reason why if we can't swap the last used .dll with the new one. I guess that dependency hell is the reason behind this.

    Having the code gen outside of Unity would mean that it won't be inside the package, right ? This is not something I want in this case.

    I could think of something similar to Microsoft's T4 templates since I've already used them in another project, that would be the next best thing to do but even then, let's say I have a template, there will be things that need to be the inputs of the template. If the reason making the project unable to compile is coming from the inputs of this template and I need to update these inputs then it's the same problem again because I won't be able to recompile and relaunch the code gen in order to fix it.
     
  4. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    In a way, yes. Nothing guarantees the new DLL is compatible with its previous version, so you might be painting yourself in a corner further.

    Say you have assemblies A and B, with B depending on A. You make a change to B's source code, and it no longer compiles; Unity treats this by keeping the last known version of A and B loaded. If you were to change A as well, allow it to compile and be reloaded, then maybe you could have removed some functionality needed by B, so there's no guarantee we could also reload B - that could put you in even more trouble.

    Note that this is not specifically related to the Package Manager, but how scripting is handled in Unity.

    The code gen can be shipped as a DLL in your package, with the DLL generated from an external VS solution. You can even include its source in your package, by putting it in a folder that's '.'-prefixed (e.g.
    .CodeGen
    ) or '~'-suffixed (e.g.
    CodeGen~
    ). Note that regenerating the DLL would only work while you're working on the package in a way that it's editable; if someone were to add the package after fetching it from a registry, it will be treated as read-only, and any changes made to the package in the Library/PackageCache folder would easily be lost as this is a volatile location.

    Well, these inputs don't necessarily require recompiling the code generator, which would break the cycle of "Generator code can't be fixed because Unity fails to compile code generated by Generator etc." Of course, whenever you need to change the Code Generator code itself, you'd be in the same situation - hence my suggestion to move its compilation lifecycle outside of Unity's control.
     
  5. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    122
    I will look for a comfortable workflow for this CodeGen, while I know an external VS Solution would work it's not as easy as just opening the Unity project VS solution (used for testing and updating the package before it's published, it's where I have that issue since once the package is published, it doesn't really matter anymore).

    The only caveat I can think of is that, currently, I'm using scriptable objects to store data which is then used by the CodeGen. The external project will need to have a reference to UnityEditor.dll and UnityEngine.dll (I think I will need both).

    Just to be sure: let's say I have this .dll with my code gen code. I run the CodeGen, the output is preventing the compilation. Now with the external VS Project, I fix the issue and rebuild the .dll. I swap the previous .dll with the new one. Will Unity be able to access the new .dll or, like you said, it will keep a previous working version of that .dll in memory ?
     
  6. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    Yes, that might make things a bit more complicated. You could also save it as JSON instead. Since you're already using ScriptableObjects, you can even read/write that directly using JsonUtility (Unity API) in Unity, then you can then read it in your favorite JSON parser in your external project. No need for UnityEditor.dll/UnityEngine.dll in that project.

    It still won't be able to use the new one, for the same reason, but if you make it an .exe instead of a DLL, you can launch it using System.Diagnostics.Process. (On Mac/Linux, you'd need to call `mono` with your .NET executable as the first argument instead.).
     
  7. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    122
    But if it's an .exe, is it right to put it inside a package?
    It kind of doesn't feel like a clean solution.

    JSON is also a solution for the data.
     
  8. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    556
    Sure, why not? Conceptually, a (.NET) .exe and a .dll are essentially the same thing, except that one has an entry point for launching it as a separate process, rather than importing the symbols it exports into an existing process.
     
    dzamani likes this.
  9. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    122
    Ok then, I will switch to an external project, JSON and compiled .exe for the codegen.

    Thanks a lot for all of your answers!
     
    maximeb_unity likes this.