Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Custom Hosting Service

Discussion in 'Addressables' started by aurelien-morel-ubiant, Nov 21, 2018.

  1. aurelien-morel-ubiant

    aurelien-morel-ubiant

    Joined:
    Sep 27, 2017
    Posts:
    236
    Hello guys,
    we want to create our Addressables server but we want to have something securised and not only a simple HttpService.
    Does IHostingService allow us to create our own service with something securised for example with a login and a password ?

    The current documentation is quite light and it's hard to know if we can create really everything we want in the current Addressable hosting service state.
    Is it (will it be) possible to have some custom field in the Hosting Service window like Auth / Password which are for me something quite usual in this domain ?
    I think we can create our own (if your services allowed us to do it) but given that this is something commonly used by developer it could be something native.
    Maybe others entries that I forget.


    Any information could be nice before we start trying this.

    thanks !
     
    AustynPember likes this.
  2. Reichert

    Reichert

    Unity Technologies

    Joined:
    Jun 20, 2014
    Posts:
    63
    The current implementation of HttpHostingService is designed to be as simple as possible for the primary use case as a development service only - it was not intended to be deployed as a production hosting service.

    I think the requirements you've listed can be expressed by implementing your own IHostingService. The caveat would be that you'll need to implement a matching IResourceProvider that understand how to talk to your custom service.

    Over the coming weeks/months we'll be providing demo projects that you can use as starting points for these kind of scenarios.
     
  3. aurelien-morel-ubiant

    aurelien-morel-ubiant

    Joined:
    Sep 27, 2017
    Posts:
    236
    Thanks for your answer !
    It was what I was thinking but I prefered to ask before starting.
     
  4. AustynPember

    AustynPember

    Joined:
    Mar 14, 2018
    Posts:
    17
    Would a host like AWS or Azure fit what you want to do? Trying to understand why the alternative.
     
  5. aurelien-morel-ubiant

    aurelien-morel-ubiant

    Joined:
    Sep 27, 2017
    Posts:
    236
    Yes we probaly hosted our asset on an AWS why ?
    I downloaded the AWS unity SDK to figure out what I will need to do to manage all that AWS stuff in Unity and to make it works with Addressables assets..
     
    Last edited: Dec 5, 2018
  6. Cleverlie

    Cleverlie

    Joined:
    Dec 23, 2013
    Posts:
    204
    I'm really lost here, I just downloaded the package from the Package Manager, the version I'm using is 0.5.3-preview

    I've watched these two videos





    and in both of them you have a editor hosting working, in the videos there is no explanation on how to setup this stuff, even more the package as it is downloaded from the package manager, only contains a "Default" profile, while in the Unite videos you have multiple profiles and everything seems to automagically work, I'm I doing something wrong? Do I need to download a set of profiles from somewhere else? how do I configure properly a editor hosting to start testing this thing for remote groups of addressables? where and how are the magic link paths generated such as:

    {UnityEngine.AddressableAssets.Addressables.RuntimePath}
    [BuildTarget]
    etc..

    is there any manual for these?

    thanks for any clarification, I can't find anything useful to start testing the hosting and remote features.
     
    MNNoxMortem likes this.
  7. unity_bill

    unity_bill

    Unity Technologies

    Joined:
    Apr 11, 2017
    Posts:
    965
    javierfed and MNNoxMortem like this.
  8. Cleverlie

    Cleverlie

    Joined:
    Dec 23, 2013
    Posts:
    204
    Hi Bill, thanks for your answer, actually I'm blind or an idiot, because I checked that same manual multiple times but never found that page about hosting services before, unless it is recently added.

    So I just finished reading it, and it clarifies a lot, I will try and do a test project following those steps and see what I get. What I would really love is to have a hands on tutorial (maybe a youtube video) where I can see someone implementing from a fresh new project an http service and making it work locally, that is a first step, secondly a tutorial of someone creating a very basic custom http service or IHostingService, and making it work from the ground up, those kind of tutorials are a really good kickstarter for people that are completely new to some module or feature of unity, like in my case and many others where the addressable system is a completely new thing. If I have to vote for something I think a 20 minute video doing this stuff is more self explanatory that many manuals, because a lot is still to be told and be explained in that addressable assets manual, anyways I know you are working hard on this feature and is in a very initial state to waste time making tutorials where the functionality and steps may change completely in a few months.

    If I get more doubts about the procedure I'll follow this forum for help, thanks again.
     
  9. Cleverlie

    Cleverlie

    Joined:
    Dec 23, 2013
    Posts:
    204
    UPDATE: I made it work, a test project wich has a floating cube that replaces its texture at the application start with an AssetReference texture, which is remotely hosted, even I tried updating the texture and then doing a Build For Content Update and it worked perfectly, even when the hosting service is off, the built player still loads a cached version of the most recently downloaded asset.

    some questions arised during this testing, the ones that come to my mind now are:

    1. the addressables_content_state.bin file, this one is created under "Assets/AddressableAssetsData", I had to figure out that when you either do a Prepare for content update or a Build for content update the Build File Dialog asks for this bin file but the manual says that asks for a player build folder with the file inside, so I just copy pasted it inside my already build player. So... is this file really needed to be copied to the built player folder? the dialog is not explicit about that. Also at which point this file is generated and when it gets deprecated? I mean is it overwritten every time you do a "Build Player Content"? and if so, should I copy paste this updated bin to the built player again?

    2. seems that after a couple of content updates, the player cache starts to pile up old versions of the same content (in my case since I'm building for windows, the " %AppData%\LocalLow\Unity\DefaultCompany_TestAddressablesProject " folder is starting to have a lot of subfolders such as "remotegroup_assets_all_00c2f034c1...." which I guess are the different versions of the AssetBundles containing the texture, cached over time.

    How do I manage these downloaded content? Can I ask the system to keep the latest version cached and delete the unneeded ones? this might be a question related to some AssetBundle API but since I'm new to those too, I'm not sure.

    btw once I have all this beginner level stuff sorted out I plan to do a tutorial or a guide so newcomer people have another source of knowledge, hopefully that will help reducing the number of devs asking for the basic things.
     
    MNNoxMortem likes this.
  10. aurelien-morel-ubiant

    aurelien-morel-ubiant

    Joined:
    Sep 27, 2017
    Posts:
    236
    @Cleverlie does your asset bundle host on a distant server (like AWS or stuff like this) or a local one like unity does in their presentation ?
     
  11. Cleverlie

    Cleverlie

    Joined:
    Dec 23, 2013
    Posts:
    204
    no, I just did a test following the steps in the "Hosting Services" part of the manual, people at my job are asking me to implement this for a dedicated server that can be run in a dedicated PC or even an amazon server, so I hope more info or tutorials appear in the next few weeks, I'm sure they made it easy to extend and make modules o custom services through their API and interfaces, but until we have a more complete documentation I'm pretty much depending on IntelliSense and guesses to understand the machinery.
     
    aurelien-morel-ubiant likes this.
  12. roshaantariq

    roshaantariq

    Joined:
    Dec 6, 2017
    Posts:
    17
    Hi, Any sample project available about setting custom hosting service? The document is not much explanatory
    I want to know if I can host the addressables on Amazon EC2 server?
     
  13. roshaantariq

    roshaantariq

    Joined:
    Dec 6, 2017
    Posts:
    17
    Hey, I'm trying to achieve the same goal "hosting addressables on distant/dedicated server like AWS"
    Please let me know if you have completed this already. Thanks!
     
  14. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    431
    @roshaantariq You can host the addressables on any indexable public space. It's just a folder with a bunch of files and a URI pointing to this folder. As long as the folder is accessible it should work - also you might need to configure CORS (e.g. WebGL).
     
  15. aurelien-morel-ubiant

    aurelien-morel-ubiant

    Joined:
    Sep 27, 2017
    Posts:
    236
    I think he means hosting and accessing to them. Which currently not smooth as it could be.
     
  16. Cleverlie

    Cleverlie

    Joined:
    Dec 23, 2013
    Posts:
    204
    I think there is a big difference between 1) hosting the generated files in a public space (FTP, dropbox or whatever) and 2) hosting the headless running Unity editor which is running itself the server who provides the specific service that the Unity Player installed in whatever platform needs to get the actual correct asset bundles etc.

    @unity_bill can you bring some light here?
     
    MNNoxMortem likes this.
  17. unity_bill

    unity_bill

    Unity Technologies

    Joined:
    Apr 11, 2017
    Posts:
    965
    From reading this thread, I think there's some general confusion about our hosting service, and the ability to create custom hosting services. The hosting service is a service that creates an HTTP host running from the editor for testing & emulation purposes. The reason to create custom hosting services is if you need this emulated server to do things like require authentication or do some other thing your real server happens to do.

    Hosting your content on a real server is completely disconnected from that, and unfortunately not something we can help you with right now. We build the content, and emulate a server for testing, but setting up, uploading, and configuring a real-world server on AWS or other is currently up to you.
     
    Cleverlie and MNNoxMortem like this.
  18. Cleverlie

    Cleverlie

    Joined:
    Dec 23, 2013
    Posts:
    204
    update: we figured out that we can host the files generated by the addressables system (the catalogs, hash, bundle etc files) in an amazon S3 server, if you configure the LocalLoadPath and RemoteLoadPath to check the url of the server it will work perfectly, so I was confused because I thought that the http hosting server that is there by default was doing something else than just serving files, that's not the case, you can replicate the same functionality in any file hosting service/server I guess. but yeah if you need some complex stuff like authentication in the server side, I guess then you will have to mess with the API and interfaces to create a custom provider in the client that knows how to deal with your custom server API. btw I have almost zero experience related to networking and servers, so probably I'm saying obvious stuff, sorry :p
     
  19. roshaantariq

    roshaantariq

    Joined:
    Dec 6, 2017
    Posts:
    17
    hey @Cleverlie did you wrote any custom provider class or did you just changed the load/remote path to server path? Can you please write up the steps you followed to achieve this? Thanks much!
     
  20. dri_richard

    dri_richard

    Joined:
    Mar 10, 2017
    Posts:
    57
    I'm still confused. The documentation states: "Hosting Services provide an integrated facility for serving packed content, based on the Addressable Assets configuration data, to local or network connected Unity Player builds from within the Unity Editor."

    This gives the impression that Hosting Services - in the general sense - are for testing only. If that's not the case, and Hosting Services are for use in production too, then please update the documentation.
     
  21. roshaantariq

    roshaantariq

    Joined:
    Dec 6, 2017
    Posts:
    17
    @unity_bill any update on hosting addressables on servers like MS Azure?
    I have bought the Azure server to host my addressables but still no luck. I can use addressables on mobile if I'm connected to the same network (local network & local machine) but if I go to Azure or change my network it does not work anymore. Please help

    THanks
     
  22. unity_bill

    unity_bill

    Unity Technologies

    Joined:
    Apr 11, 2017
    Posts:
    965
    As mentioned earlier in this thread, we build you the content, but setting up a server is not something we can solve for you. There are a lot of different hosting solutions out there, all with various complexities. This might be something we can provide some guidance/examples/help on late this year, but in the near term, the actual hosting setup is unfortunately disconnected from what we can do.
     
  23. tencnivel

    tencnivel

    Joined:
    Sep 26, 2017
    Posts:
    13
    @unity_bill, I have to say I also got mislead.

    That is probably because we all look for the same thing (what @aurelien-morel-ubiant is asking in his question): how can we tweak the HTTP GET request so that we can perform authentication on the server side? in my case it would be to add a Json Web Token in the header of the request, the JWT being retrieved from the result of a POST request using Networking.UnityWebRequest.

    In the inspector of AddressableAssetSettings (attached) we can specify the LocalLoadPath and RemoteLoadPath ; in the inspector of an asset group (in AssetsGroups) we can see the resulting URL (in 'Load Path').

    How can we tweak the HTTP GET request to this URL ?
     

    Attached Files:

    Last edited: May 31, 2019
    aurelien-morel-ubiant likes this.
  24. unity_bill

    unity_bill

    Unity Technologies

    Joined:
    Apr 11, 2017
    Posts:
    965
    For now, you'd need a custom provider to do this. In the group inspector, you can specify what bundle provider should be used for all asset bundles created by that group. We are working on extending the built in provider, but for now, you need your own.
     
  25. alexmorco

    alexmorco

    Joined:
    Aug 10, 2017
    Posts:
    1
    Yes, I am hosting 10 WordPress websites with all different login details with 1 server, we can also choose different server as per our choice, Currently using Cloudways WordPress website hosting. You can create different roles for different users with a staging environment.
     
  26. Leoniciux

    Leoniciux

    Joined:
    Aug 14, 2013
    Posts:
    44
    Hi Guys, after waiting for a while, I decided to give it another try to your package, which I'm sure has been long awaited by the community. I have to say that I was very excited when I attended to your presentation last year at Unite Berlin but I got a lot of issues with the first preview packages, so I decide to wait for a while, right now I find my self at a point where I have to implement DLCs again and I decide to give it another try and to be honest I'm very disappointed with the progress, this feature should make our life easier not harder, I don't know who in your team had the brilliant idea that everything should go into scriptableobjects that you can directly access and modify. I'm very fan of having CI/CD and I have everything scripted specially when it comes to the resources organisation and asset bundles creation, I have been working with asset bundles since Unity 3.0 and right now seems that your brand new stuff that is supposed to help us handling the downloadable resources is going only to bring a brand new hell and not help us at all. As many point in the forum you examples are not self explanatory and the documentation is very far away to what unity usually use to be great at, I don't know if this is still a preview package... at least in the package manager is not labelled as such but after giving it a try this last two days, I don't think this is a production ready solution. I think I'll stay with the "legacy" way of working for now, I'm not in the mood of fighting a brand new beast.
     
    chanon81 likes this.
  27. emran_17

    emran_17

    Joined:
    Dec 10, 2014
    Posts:
    1
    At first impression I said WOW ! what a clean way to control every thing ! now I see that , It's just a handling of relations between the name and paths :D ! Like the work that we do by a Dictionary<String,String> .
     
    Leoniciux likes this.
  28. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    587
    That's reasonable but we need some guidance on which files to upload and by what URL scheme they need to be reachable by clients and how to configure the clients to use those URLs. If we had all of that clearly specified, then it would be reasonable for "exposing the right files via those URLs" to be "our problem". Is there a guide like that?

    Thanks!
     
  29. unity_bill

    unity_bill

    Unity Technologies

    Joined:
    Apr 11, 2017
    Posts:
    965
    no, but it does seem reasonable to make one.

    the short version is this:
    1. Set your remote build path to some folder. say "ServerData/[BuildTarget]"
    2. Set your remote load path to the URL root of where your stuff will be. say "mycdn.com/stuff/android"
    3. upload everything in "ServerData/android" to "mycdn.com/stuff/android."
    4. run game.

    Assuming things like "ServerData/android/something.bundle" is accessible at "mycdn.com/stuff/android/something.bundle" it should work.

    If your provider requries signed URLs, that's a little more complex. We're working on an out-of-the-box solution for that. For now it may require a custom provider.
     
    javierfed, leobilck and Elmstrom like this.
  30. javierfed

    javierfed

    Joined:
    Apr 10, 2015
    Posts:
    24
    I have successfully set this solution up myself, simple and easy. I am now aiming for something a little more complex, as my next project will have users connecting, creating annon identities that will be given permission to download from our s3 bucket (using AWS)

    I read through this thread and thought it would be the best place to ask if anyone got a solution for the access. The AWS sample for cognito and S3 involve doing async calls through an AmazonS3Client object. Since this is a custom approach (not a generic http file download request I assume is written in addressables) how would I approach writting it, if no one else has written anything like it.

    note that I want to upload everything to an s3 bucket, including the catalog, which I think is downloaded on a different thread as the content to check before doing the expensive download of bundles...
     
  31. javierfed

    javierfed

    Joined:
    Apr 10, 2015
    Posts:
    24
    how would we begin making the custom provider. assume we know how to make the request, to get an object, how do we connect that to the addressables system? I can do that authentication calls and request calls, but I don't know where the resulting file data should be assigned, and if this will be used for all artifacts, including catalogs.
     
    MNNoxMortem likes this.
  32. ctapp1

    ctapp1

    Joined:
    Jan 11, 2016
    Posts:
    4
    Hey Bill, I have set everything up correctly to connect to an S3 server, but I am getting while using LoadAssetsAsync. Do I need to DownloadDependenciesAsync first?

    Exception encountered in operation UnityEngine.AddressableAssets.Initialization.InitializationOperation, result='', status='Succeeded' - Chain<GameObject,IResourceLocator>: ChainOperation of Type: UnityEngine.GameObject failed because dependent operation failed
    Dependency Exception

    any ideas?

    Edit: It seems when I created a new group and moved all my assets into that one it caused all the errors. I reverted back to the default group and updated my settings through that and it worked
     
    Last edited: Oct 12, 2019
  33. EdgarSantos

    EdgarSantos

    Joined:
    Nov 11, 2013
    Posts:
    20
    Hi!

    I think I understand what is being discussed here but I have a couple of related questions:
    1. Can we customize (e.g. authorization header) the request to the "Remote Load Path?" We need a scenario with no server-side logic (e.g. S3, Azure)
    2. I understand that we don't have to re-build the app to be able to use new asset bundles BUT do we have to rebuild all asset bundles (and upload them) if we build a new version of the app/player? If not, in general terms, how does that work?
    Are there any news about this?


    Thank you.
     
  34. robinryf

    robinryf

    Joined:
    May 22, 2014
    Posts:
    14
    I think I can at least provide some insights to this. I have published a library how to download AssetBundles via the Addressables ResourceManager system using custom providers for the Google FirebaseStorage backend. I am not familiar with Azure so I am not sure if the same use case applies. But basically in this solution there is a custom provider which gets called when a certain URL scheme is configured as remote path ( e.g. gs:// for Firebase ) and this custom provider uses Firebase APIs to handle authentication of the client and then gets the "pure" download link which is then passed on to the "normal" AssetBundle downloader so caching/CRC checks, etc. are still done by Unity.

    You can find the plugin here: https://forum.unity.com/threads/git-plugin-to-load-addressables-from-firebase-storage.752789/


    This would not directly work if you just want to change the Headers on the HTTP request sent out to get the bundle. For that I think you would either have to copy or inherit the com.unity.addressables@1.1.7/Runtime/ResourceManager/ResourceProviders/AssetBundleProvider.cs class (which is responsible for creating the UnityWebRequests which load the bundle ) and rename it to use for your own purpose.


    Cheers
     
  35. roshaantariq

    roshaantariq

    Joined:
    Dec 6, 2017
    Posts:
    17
    @unity_bill Hey, so I was wondering if the addressables are final yet? And what about the hosting part? Is there any documentation about hosting the addressables on Amazon S3 or Azure and then access the data from clients?
     
  36. unity_bill

    unity_bill

    Unity Technologies

    Joined:
    Apr 11, 2017
    Posts:
    965
    Documentation on S3, etc.? no, we are not writing up instructions for third party hosting services. But as to customizing the call, we're likely adding an API in the next-ish release that will allow you to have a callback before any "internal id" is evaluated. So can dynamically change any path at runtime. if you need to.
     
  37. JGrierson

    JGrierson

    Joined:
    Sep 12, 2016
    Posts:
    9
    This question is for Unity_Bill
    We have Addressables somewhat working but need a good use cases for reducing the load time of instantiating assets from asset bundles to the minimum time necessary. Right now to instantiate a single asset from a loaded asset bundle is taking 4-7 seconds. Not OK.
    Compression is turned off in all cases. These asset bundles reside on the Hard drive not on the Web. Our asset bundles consist of FBX's which contain meshes and textures. Eash FBX is roughly 20MB in size.
    Also we tried Application.backgroundLoadingPriority = ThreadPriority.High; to speed upo this process and this appears to do absolutely nothing! Is Unity really setting thread priority?
     
    Last edited: Nov 3, 2019