Search Unity

Dealing with new style .aar's that have ${applicationId} in their AndroidManifest.xml

Discussion in 'Android' started by seaders, Jan 19, 2016.

  1. seaders

    seaders

    Joined:
    Oct 28, 2014
    Posts:
    29
    Obviously this is more in regard the problem of *not* building with gradle, while Google are pushing everything Android that direction, but currently it's starting to be quite problematic with some of the newest .aar files coming out, like play-services-measurement-8.4.0.aar, which in their AndroidManifest.xml is,

    Code (csharp):
    1.  
    2. <provider
    3.   android:authorities="${applicationId}.google_measurement_service"
    4.   android:name="com.google.android.gms.measurement.AppMeasurementContentProvider"
    5.   android:exported="false"/>
    If you build with a reference to this in Android studio, this manifest is merged by replacing ${applicationId} with your proper application id, so you end up with something like

    Code (csharp):
    1.  
    2. <provider
    3.   android:authorities="com.mycompany.myapp.google_measurement_service"
    4.   android:name="com.google.android.gms.measurement.AppMeasurementContentProvider"
    5.   android:exported="false"/>
    But at the moment, if you drop any of these newer .aar files into Plugins/Android, you get an error message on building attempt,

    Code (csharp):
    1.  
    2. stderr[
    3. AndroidManifest.xml:19: Tag <provider> attribute authorities has invalid character '$'.
    4. ]
    Now, I know that in general, you can just unzip .aar files in the Android plugins folder, make those edits manually, and away you go, but realistically these .aar files *are* valid .aar files, and Unity, by enabling you to drop .aar files into the Android plugins folder should be able to deal with them.
     
    Hansaem Kim likes this.
  2. SidarVasco

    SidarVasco

    Joined:
    Feb 9, 2015
    Posts:
    163
    Any update on this?

    I'm facing the problem and manually unpacking the AAR files shouldn't be an option imo. It should be automated.
    Using unity 5.2.3
     
  3. dillon_yeti

    dillon_yeti

    Joined:
    May 11, 2015
    Posts:
    9
  4. dillon_yeti

    dillon_yeti

    Joined:
    May 11, 2015
    Posts:
    9
    Double Bump?
     
  5. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    443
    Triple Bump.
     
  6. paradizIsCool

    paradizIsCool

    Joined:
    Jul 10, 2014
    Posts:
    171
    Quadruple Bump
     
  7. tp_jer

    tp_jer

    Joined:
    Aug 31, 2015
    Posts:
    1
    quintuple bump
     
    Qbit86 likes this.
  8. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    Sorry guys for the very late answer.
    We don't currently support ${variable} substitutions in .aar manifests in Unity. Actually, we are treating .aars as Android library projects, so some gradle functionality is missing.

    Good news: gradle build is coming (already available in 5.5 alpha version). It will support these things.
     
  9. JKasten

    JKasten

    Joined:
    Apr 11, 2013
    Posts:
    23
    @Yury-Habets Awesome!

    Gradle support in Unity 5.5 only a build target? Or can `build.gradle` files now be added to the Unity project where it will pull from Maven Central? Such as the following gradle line.

    Code (csharp):
    1. compile "com.google.android.gms:play-services-gcm:+"


    This is in context to plugin developers looking to make it easier to add their decencies. This way .aar files don't have to be bundled with the plugin or have to include Google's unity-jar-resolver.

    The only documentation I found on gradle with Unity is as follows:
    https://docs.unity3d.com/550/Documentation/Manual/android-gradle-overview.html
     
  10. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    @JKasten yes Gradle is a new build target in Unity Android.

    You can build a project in Unity using Gradle (we generate .gradle file and invoke gradle from the Editor), or you can export a Gradle project and use it in Android Studio directly.

    Unfortunately we don't support custom build.gradle files or templates yet. Everything to come - please stay tuned and report bugs if you meet them!

    An unofficial and unsupported way to modify the build.gradle is to modify it in the Unity installation.
     
  11. paradizIsCool

    paradizIsCool

    Joined:
    Jul 10, 2014
    Posts:
    171
    @Yury-Habets
    Is there any news about custom build.gradle, especially for cloud build.

    And for the main subject, use gradle build do not fix this issue as applicationId is not defined in the build.gradle file (in
    android {
    defaultConfig {
    applicationId "com.xxx"
    ...
    )
     
    Last edited: Jan 16, 2017
  12. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    On the custom build.gradle: not yet, but is under development.
     
  13. paradizIsCool

    paradizIsCool

    Joined:
    Jul 10, 2014
    Posts:
    171
    It seems to be released with 5.5.1p1, but without any documentation, need some help
     
  14. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    Put your build.gradle at the following path:
    Assets/Plugins/Android/mainTemplate.gradle
    Documentation update is on its way.
     
    paradizIsCool likes this.
  15. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    I don't understand.

    In android there is a mess with this "Assets/Plugins/Android" magic folder that all asset store publishers drop assets into converting it into a mess replacing manifests files from other assets, having two assets the same jar duplicated (support library)... making impossible to know what file it's from each asset and making really difficult to remove or update an asset without breaking other assets...

    I thought that with Gradle would disappear this but I see that we are running into the same wall again...

    Really don't you see the problem? Are you doing nothing to stop this?
     
    paradizIsCool and Qbit86 like this.
  16. paradizIsCool

    paradizIsCool

    Joined:
    Jul 10, 2014
    Posts:
    171
    Can we add the applicationId value with a keyword like **APPID** or **BUNDLEID** ?
     
  17. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    @paradizIsCool
    Done.
    It will take some time to get into our code mainline, and then get backported to 5.5 and 5.6. Will appear in some patch releases of 5.5 and 5.6 - check the release notes!
     
    paradizIsCool likes this.
  18. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    @N3uRo I understand your concerns. We don't have a nice dependency management system (yet).

    However, most of the conflicts with the "magic" happen because plugin authors don't think of the other plugin authors. For example, most manifest changes can be added to a library, which will then get merged into the main manifest. Instead, the plugin authors put their manifest in Assets/Plugins/Android/AndroidManifest.xml, trying to override everything.
     
    JohnTube and paradizIsCool like this.
  19. Anjin_nz

    Anjin_nz

    Joined:
    Jun 24, 2014
    Posts:
    74
    Hi Yury,

    Thanks for your comments. Two suggestions-

    * Could gradle build support be backported to 5.4? 5.5 is more (b)leading edge and I'm afraid to go there if its not stable.
    * Perhaps a writeup in the official documentation with some tips on the Android build system dealing with the dependencies would help users until something more concrete arrives. It would really help to have something official in one place.

    Keep up the good work. Android Unity3d is getting better all the time.
     
    Last edited: Feb 10, 2017
    Yury-Habets likes this.
  20. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    @Anjin_nz thanks for the warm words!

    - 5.6 is coming out really soon - it's worth checking 5.5 as it has dozens of patches and should be stable. We consider backporting gradle into 5.4, but no decision has been made yet, so no promises.
    - A huge update to documentation is on its way (also includes several updates on gradle). It has more info on the Manifest and how to deal with it.

    On the dependencies - I don't have a solution that will work for everyone at the moment... Other than removing the duplicate google play services or support libraries manually. We're working on that.
     
    Qbit86 and Anjin_nz like this.
  21. Anjin_nz

    Anjin_nz

    Joined:
    Jun 24, 2014
    Posts:
    74
    Thats cool. Thanks Yury. I will take a look at 5.5 for our next Unity upgrade.
     
  22. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    I know that but you have coworkers that are verifying that each asset have a minimum quality. You should check for that... New or updated assets should be in aar packages (not jars) that can have each manifest "slice" and then merge it. And also because including dependencies can be done with build.gradle and not copying android support jars directly to the project.

    Can you share with us if you are discussing internally that matter? It's a real problem...
     
  23. Anjin_nz

    Anjin_nz

    Joined:
    Jun 24, 2014
    Posts:
    74
    Hi N3uRo,

    In the Ad integration work we have just done, the plugin jars/aars had to come directly from the Admob/Firebase website or the ad company itself. There was no way Unity could have vetted them, it probably should have been done by Admob.

    I agree it would be a good idea to ask the plugin guys to run them through the asset store.

    In some cases the documentation is really sparse. I googled around for ages before realising that an project.properties file was a legacy from the old eclipse build system.

    Yury is absolutely right about plugins not playing well with each other. I thought I had a full on core war going on in Unity for a while.

    I think some sort of discussion is needed but its a very broad group who need to engage. Not just within Unity.
     
  24. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    @Yury-Habets I have a problem with gradle and aar package dependencies, I think that you could help and add it to docs if it's worth:

    I have an android project library that generates an aar package that has dependencies like:

    compile 'com.android.support:appcompat-v7:25.1.1'
    compile 'org.apache.commons:commons-io:1.3.2'

    I don't know why but appcompat gets included automatically: "{PROJECT_PATH}\Temp\gradleOut\settings.gradle" has a single line "include 'appcompat'" but apache commons-io doesn't get resolved and I have a java.lang.NoClassDefFoundError: Failed resolution of: Lorg/apache/commons/io/IOUtils

    So, is there no automatic discovery to resolve this with a build.gradle in the same folder as the aar package? or is there something automatic that it's not enabled? There must be a way because when you do a native application you don't need to include in your build.gradle the dependencies of your included library (example: the dependencies that apache commons-io has on other libraries).

    For now I had to include it in Unity manually from: https://archive.apache.org/dist/commons/io/binaries/
     
    Last edited: Feb 12, 2017
  25. Anjin_nz

    Anjin_nz

    Joined:
    Jun 24, 2014
    Posts:
    74
    Hi N3uRo,

    Perhaps a good idea to show the full console trace message and tell us the exact version of Unity you're using. I'm guessing 5.5.1? Nice to see someone doing work in this area as I find it quite difficult to know how the tools work at this level.
     
  26. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    Are you Unity staff? Because you don't have the black badge of "Unity Technologies".

    If you really need it I can give you the full stacktrace but I think that with the bold text in my previous message says it all, it doesn't include the library so that class is not available.

    And I'm using 5.6b8.
     
  27. Anjin_nz

    Anjin_nz

    Joined:
    Jun 24, 2014
    Posts:
    74
    No I'm not Unity staff. Just an interested bystander who would like to see this made easier for the Android community. I'm a developer on the DeadBotz game series.
     
  28. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    @N3uRo there's no point to be rude to the other forum users. Unity forums are for everyone to help and exchange their opinion, not only a means of getting the answer from someone with a "Unity" badge.

    As @Anjin_nz has said, the info you provided is not full. You didn't even say whether you are building in Unity or exporting to Android Studio.

    As I said, Unity is not doing any dependency resolving at this point. The only thing we do is:
    - adding references to any AAR plugins we find in the project
    - adding references to any android library plugins we find in the project

    If you are using a custom build.gradle - then it might be related. If the project builds for you with "Legacy" build system and does not with "Gradle" - then more details are greatly appreciated, we will try to fix it.

    As for the docs - they will not include any mention of specific plugins or something that might change or we don't have control of. Resolving plugin conflicts is kinda tricky and requires some insight into Android. However the docs will include sections on most popular errors, such as dex failed, aapt failed, manifest merging failed etc.
     
    Anjin_nz likes this.
  29. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    I'm sorry that you misunderstood my answer to Anjin, I mean that sometimes I've seen new Unity staff member without the badge and as Anjin asked me some questions I thought that could be one. It was a question.

    - I'm building to a device directly, not an Android Studio project.
    - I don't have a custom build.gradle.
    - I'm building with Gradle because with the old system it doesn't parse AAR's AndroidManifest.xml with ${applicationId}.

    One thing I don't understand is how it works the dependencies resolving system with Gradle. If I have:

    1. aar package that dependens on a Library (like I said apache commons).
    2. In my project I include only the aar package.
    3. Unity includes aar package but doesn't check for dependencies? Gradle doesn't handle that?

    In a normal Android project I include a library that references 3 other libraries and I don't need to include these 3 libraries.

    I said these because Android Support library it's included by many asset store publishers and you can get 2 or 3 copies in the same project.
     
  30. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    Thanks for the details.

    > 1. aar package that dependens on a Library (like I said apache commons).

    How is this set up?
     
  31. paradizIsCool

    paradizIsCool

    Joined:
    Jul 10, 2014
    Posts:
    171
    @N3uRo

    3. Unity includes aar package but doesn't check for dependencies? Gradle doesn't handle that?

    If you add the aar of your directly in unity, unity add the aar in local folder, gradle can't resolve dependencies as the dependencies is in the pom and you cannot add it.

    So yes if you put the aar directly in unity, you have to add all the dependencies by yourself. This is not a unity limitation but a gradle one.

    You have 3 soutions :
    _ remove the aar from unity, upload your project to a maven repository, add this repository in the gradle template and add your project dependency from this repository, the dependcencies are added by gradle with the maven pom file
    _ add the dependencies in the gradle template
    _ add the dependencies aar in your unity project.
     
    Last edited: Feb 24, 2017
  32. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    I have an Android Studio project that generates my AAR package that then I copy to my Unity project, in that Android Studio project I have this in build.gradle:

    My problem is that in the generated AAR package doesn't get included "appcompat" or "apache commons" so when I build the APK I get classnotfound error and I need to know what dependencies I have and include an "appcompat" or "apache commons" aar/jar files in Unity.

    Is there no way to get that resolved automatically in my AAR package generation step or by Unity gradle project generation?

    I say this because in this case I know my dependencies but when you use a third party AAR package you don't know the dependencies that a package can have.
     
  33. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    @N3uRo well, your built AAR probably has no knowledge about its own dependencies. It's the Android Studio which knows that it need appcompat and commons when building the resulting APK.
     
  34. hutch_jl

    hutch_jl

    Joined:
    Jan 8, 2013
    Posts:
    40
    Hi,

    I've been trying to get a plugin we use (Helpshift, https://developers.helpshift.com/unity/getting-started-android/) to build with Gradle in 5.5.2. Building directly in Unity with Gradle selected fails with a bunch of AAPT errors e.g

    /HSGradle/Temp/gradleOut/helpshift/build/intermediates/bundles/release/res/values/values.xml:299: AAPT: Error retrieving parent for item: No resource found that matches the given name 'Base.Widget.AppCompat.Button.Borderless'.

    However, if I export project I can build the exported project with Android Studio with only a simple change to the build.gradle inside the helpshift plugin directory (not the root gradle file), I just need to add 3 dependencies and it builds fine:

    compile 'com.android.support:design:23.4.0'
    compile 'com.android.support:recyclerview-v7:23.4.0'
    compile 'com.android.support:cardview-v7:23.4.0'

    I've tried putting mainTemplate.gradle into Plugins/Android and adding the same dependencies, but the errors persist. Feels like I'm very close - is there a way to customise the template so it builds equivalently to the exported project with the added dependencies? I have a sample project if that helps.

    Thanks!
     
  35. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    Can we have multiple Gradle templates and have it in the same folder as the AAR package? Not in "Assets/Plugins/Android".There is no docs on this yet.

    So we can include our dependencies for our package and don't get overriden the main template by other assets like it happens with AndroidManifest.
     
    paradizIsCool likes this.
  36. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    @d00fus this "simple change" is adding dependencies. You may want to add the dependent packages to your Unity project.
     
  37. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    @N3uRo this is not supported yet, but we will consider it.
     
  38. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    Do it please so Asset Store developers can have it's own template on their asset folder and don't override or collide with anything.
     
  39. paradizIsCool

    paradizIsCool

    Joined:
    Jul 10, 2014
    Posts:
    171
    Totally agree with @N3uRo. Forcing developers to share a file and a folder causes so many trouble in mobile development.
     
  40. masugata_kensuke

    masugata_kensuke

    Joined:
    Mar 8, 2017
    Posts:
    1
    What about specifying applicationId in build.gradle? (see #11)

    It's necessary for ${applicationId} to be expanded correctly.
    Without applicationId specification in build.gradle, ${applicationId} is expanded to the value of "package" attribute of .aar file's AndroidManifest.xml.
    Authors of .aar files expect ${applicationId} to be expanded to apps' own package names. (see #1)

    It's a common trick for .aar developers to include ${applicationId} in "android:authorities" attributes in <provider> elements, which are required to be unique.
    e.g. https://firebase.googleblog.com/2016/12/how-does-firebase-initialize-on-android.html

    I found Unity 5.5, 5.5.1, 5.5.2 and 5.6.0b10 lacking applicationId specification.
    Also I checked that it solves the problem to add applicationId specification into /Applications/Unity/PlaybackEngines/AndroidPlayer/Tools/GradleTemplates/mainTemplate.gradle.
    (Though I don't know how to reference bundle ID. (see #16))
     
  41. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    894
    I totally agree but it should be something provided by Unity in it's own template without us doing anything.

    It's more common to want that than not wanting it.

    I asked for that in another thread.
     
  42. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
    As I said, we are adding support for applicationId soon. It will be assigned to the application's bundleid (package name).
     
  43. pfleetwood22

    pfleetwood22

    Joined:
    Jun 6, 2016
    Posts:
    28
    @d00fus I also am dealing with the helpshift plugin while trying to switch to a gradle build. I had the same problem of not being able to control the generated sub-project build.gradle files (in order to specify the dependencies that were needed to resolve resources).

    However, I realized that I could replace the entire Plugins/Android/helpshift folder with the following lines in my main gradle.

    Add a repository to pull packages from:

    allprojects {
    repositories {
    jcenter() <------
    flatDir {
    dirs 'libs'
    }
    }
    }​

    Then, add a reference to the aar you need:
    compile(group: 'com.helpshift', name: 'android-helpshift-aar', version: '4.7.0', ext:'aar')​

    I'm still testing, but at least I'm able to build now. (I've done similar tricks with google play services, Facebook SDK and bolts libs, android support libs, etc.)

    EDIT: It was not enough to use the Android AAR library. The dex file they ship is almost the same as the Android aar, but it also includes some special unity classes. I needed to preserve the helpshiftloader.jar and helpshiftnotificationhandler.jar in the Plugins/Android/helpshift folder by moving them up a level. Then, I zipped the remaining helpshift folder as an aar (to use in place of the added dependency in the guild.gradle snippet above), and placed it in Plugins/Android (removing the helpshift folder). I needed to add its listed dependencies to my build.gradle file (recycler, etc), but I also needed to depend on coreui.
     
    Last edited: May 4, 2017
    JohnTube likes this.
  44. hutch_jl

    hutch_jl

    Joined:
    Jan 8, 2013
    Posts:
    40
    Are you all up and running @pfleetwood22 ? I got it to work by putting Helpshift in an empty project, generating an Android Studio (AS) project from Unity and modifying the exported gradle config for Helpshift to set its dependencies, then building it in AS. I then took the generated release AAR for Helpshift (that AS built), put it in Plugins/Android along with its dependency AARs and kept the dex file that it loads in Plugins/Android/assets/helpshift. In my case I also had to add a modified mainTemplate.gradle to enable multidex and add the multidex support lib dependency as well as Helpshift pushed us over the dex reference limit. But after all that a build using Gradle in Unity succeeds and Helpshift seems to work fine :)
     
  45. evanngo

    evanngo

    Joined:
    Jun 16, 2017
    Posts:
    1
    I have same problem when I tried to compile
    compile 'com.android.support:appcompat-v7:23.4.0'
    compile 'com.android.support:mediarouter-v7:23.4.0'
    in my Gradle file, can call it in Java file but when build APK file by Unity and install app, app was crashed because Caused by: java.lang.ClassNotFoundException: Didn't find class "android.support.v7.app.MediaRouteButton".
    I still can't fix it for 2 weeks. Using unity 5.4.4
     
  46. pfleetwood22

    pfleetwood22

    Joined:
    Jun 6, 2016
    Posts:
    28
    Yes - we got running. It definitely doesn't just "drop right in" yet.
     
  47. zero_null

    zero_null

    Joined:
    Mar 11, 2014
    Posts:
    159
    Hello, I am developing a plugin and I need to reference the Library by prefixing with the package name. The exact same issue with the title post.

    I am just curios how Facebook SDK for Unity, Admob and Play Game Services for Unity are handling this ?
    I don't want my users to specifically change the build system from Internal to gradle only for my small plugin. Is there a way I can also use my included library and not avoid using Gradle build system ???


     
  48. paradizIsCool

    paradizIsCool

    Joined:
    Jul 10, 2014
    Posts:
    171
    I guess they all use JarResolver which doing much more stuff that resolve jar dependencies.

    BTW everybody should avoid Internal and use gradle

    (And JarResolver should disappear because it does a lot of strange/magic workaround to fix unity -and cloud build- limitations)
     
    Last edited: Nov 6, 2017
    zero_null and Yury-Habets like this.
  49. AbandonedCart

    AbandonedCart

    Joined:
    Mar 4, 2014
    Posts:
    72
    Unity 5.6.5p4 (Released 12 April 2018) is 1 year, 1 month, and 8 days after your post.

    Now correct me if I am wrong, but "soon" seems like it should have been significantly less than that, but somehow I am still getting
    attribute name has invalid character '$'
    for
    ${applicationId}


    It seems like a simple task to integrate identifying a specific string that should be replaced with Unity's "Package Name" value, so is there some reason it has taken over a year with 0 progress?
     
  50. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,148
unityunity