A/B Testing Suggestion

Hey guys, this might be a little out of scope for Analytics, so thought I'd suggest it here first to get some feedback before posting about it on the suggestions section.

It'd be cool if we had the abillity to set up A/B testing via the dashboard, and view analytics from users who are assigned to group A or B. Here is how I imagine it might work:

Say we want to test which configuration sells more in app purchases: IAP Dialogue #1 or IAP Dialogue #2

• We set up an A/B test via the Dashboard 'IAPDialogue'
• When we provide the users ID to the analytics framework using Analytics.SetUserId(userID), the user is assigned a group (A or B) for all the active A/B tests on the dashboard. The user will be bound to this group, and this should persist between session.
• We can check the active users group placement using Analytics.GetGroupForABTest.
• When the user opens the IAP dialogue in our game, we show the appropriate one for the user by first checking which group they belong to
  
  Code (CSharp):

  1. var group = Analytics.GetGroupForABTest('IAPDialogue');
  2. if(group == ABGroup.A)
  3.     ShowIAPDialogueA();
  4. else
  5.     ShowIAPDialogueB();

On the dashboard, we would be able to analyse the data for an A/B test. The data would be displayed in exactly the same way it does Data Explorer, except that we would see 2 copies of it - data from group A and group B

A crude mockup, showing the number of purchases made by users in different groups.



It doesn't need to be limited to 2 groups. Analytics.GetGroupForABTest could just return an int or a string for infinite possibilties.

 

+1 for A/B testing. I would love to see A/B testing natively supported by Unity because from my experience, the more API's we use, the more our game crashes, the poorer it reviews, the less money we make, the less we have to spend on Unity licenses =D