Search Unity

  1. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  2. Improved Prefab workflow (includes Nested Prefabs!), 2D isometric Tilemap and more! Get the 2018.3 Beta now.
    Dismiss Notice
  3. Want more efficiency in your development work? Sign up to receive weekly tech and creative know-how from Unity experts.
    Dismiss Notice
  4. Participate with students all over the world and build projects to teach people. Join now!
    Dismiss Notice
  5. Build games and experiences that can load instantly and without install. Explore the Project Tiny Preview today!
    Dismiss Notice
  6. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

AB Testing & release/development switch on remote config

Discussion in 'Unity Analytics' started by chrisendicott, Oct 11, 2018.

  1. chrisendicott

    chrisendicott

    Joined:
    Apr 19, 2017
    Posts:
    7
    Could someone explain how AB testing's use of remote config variables interacts with the release/development switch for the variables in remote config?

    If I create a variable in remote config in development mode, it doesn't show up as available as a treatment in AB testing which makes me assume you can only use AB testing on release variables? How does this interact with being able to test the AB test in the editor?

    Is there a reason the AB test page doesn't have the same release/development switch as the remote config page?
     
  2. ap-unity

    ap-unity

    Unity Technologies

    Joined:
    Aug 3, 2016
    Posts:
    1,049
    AB Testing will only use the Release configuration of Remote Settings. That means it won't affect the values sent to the Editor and Development builds (those platforms use the Development config.)

    You still can and should test your Remote Settings in the editor/dev builds first. How you do this is up to you, but since the Development config is completely separate from the Release config, you could give the variants different names (e.g. button_color_variant_1 and button_color_variant_2) and then have some way to switch between them in development. Or you could just keep the key the same and just make sure to test all values during development.

    The reason for only using the Release config is that AB Testing is only really useful if the sample size is large enough to deliver statistically significant results. This is more typical of a production environment than development.
     
  3. chrisendicott

    chrisendicott

    Joined:
    Apr 19, 2017
    Posts:
    7
    Thanks - I appreciate the explanation.