Search Unity

  1. Unity 2019.1 beta is now available.
    Dismiss Notice
  2. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  3. We're looking for insight from anyone who has experience with game testing to help us better Unity. Take our survey here. If chosen to participate you'll be entered into a sweepstake to win an Amazon gift card.
    Dismiss Notice
  4. Want to provide direct feedback to the Unity team? Join the Unity Advisory Panel.
    Dismiss Notice
  5. Unity 2018.3 is now released.
    Dismiss Notice
  6. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice

AB Testing & release/development switch on remote config

Discussion in 'Unity Analytics' started by chrisendicott, Oct 11, 2018.

  1. chrisendicott

    chrisendicott

    Joined:
    Apr 19, 2017
    Posts:
    7
    Could someone explain how AB testing's use of remote config variables interacts with the release/development switch for the variables in remote config?

    If I create a variable in remote config in development mode, it doesn't show up as available as a treatment in AB testing which makes me assume you can only use AB testing on release variables? How does this interact with being able to test the AB test in the editor?

    Is there a reason the AB test page doesn't have the same release/development switch as the remote config page?
     
  2. ap-unity

    ap-unity

    Unity Technologies

    Joined:
    Aug 3, 2016
    Posts:
    1,082
    AB Testing will only use the Release configuration of Remote Settings. That means it won't affect the values sent to the Editor and Development builds (those platforms use the Development config.)

    You still can and should test your Remote Settings in the editor/dev builds first. How you do this is up to you, but since the Development config is completely separate from the Release config, you could give the variants different names (e.g. button_color_variant_1 and button_color_variant_2) and then have some way to switch between them in development. Or you could just keep the key the same and just make sure to test all values during development.

    The reason for only using the Release config is that AB Testing is only really useful if the sample size is large enough to deliver statistically significant results. This is more typical of a production environment than development.
     
  3. chrisendicott

    chrisendicott

    Joined:
    Apr 19, 2017
    Posts:
    7
    Thanks - I appreciate the explanation.