With Firebase, you can perform A/B testing in applications with minimal effort and better results. It’s a mighty easy platform.
This is the second part of the series to guide you through set and Testing! Part 1- Ways to implement A/B tests in Firebase.
How to configure the A/B test?
We are starting with Firebase A/B testing using the Remote Config feature. Set up the A/B test in the Firebase console.
1. Click the A/B testing item from the left side menu and click the Create Experiment button.
2. Popup box will ask to choose from- Remote Config, Notifications, or In-App Messaging to configure an A/B test. Since we will be configuring the test via config parameters select Remote Config option.
3. Create an experiment via Remote config:
In the A/B test configuration screen, allow users to define and set up the A/B test with a couple of conditions and parameters to be satisfied.
Here we’ll use the same parameters that were created in the “Set up Remote Config in Firebase console” section.
Below are the steps listed to show you how to configure the A/B test in the Firebase console using Remote Config.
Step 1: Provide the Experiment name and details.
Step 2: Condition to target users
In this section, we need to select the app in which we need to activate this experiment.
Here, for targeting, take All users to be eligible for an experiment based on conditions we use.
An event that activates the experiment for a user. All targeted experiment users will receive an experiment variant treatment from the start of the experiment. Only those users who trigger the activation event will be included in the experiment measurement.
Here, select Continue_Shopping(custom event) as an activation event to track how many users are clicking the shopping button, then click Next.
Step 3: Define Goals
By utilizing goal metrics (such as Crash-free users, User retention) plus maximum occurrences of events (such as daily user engagement, app_remove, first_open, custom events).
Goal will be deemed a metric that can be chosen for the variant that best achieves the experiment leader as the primary goal. Click Next.
- Types of metrics to track
Same as goals, there is a list of metrics available in the above screencap, at max 5 metrics can be used per A/B test. Distinct metrics as additional metrics (secondary goals) can be configured as compared to goals as secondary goals.
For e.g. for the test experiment, we have set Purchase revenue as the primary metric and in additional metrics, we’ve selected other metrics
Step 4: Select variants
By using the parameters which are defined in Set up Remote Config in Firebase console section, multiple parameters are applied for variant setup with values for Control Group and Variants. Also, the A/B test has been configured as an alternative text and color shade for the shopping button.
- Types of variants
Baseline indicates App’s original view, whereas variants define the different variations to accomplish the goal. At max 7 variants are allowed for configuration.
Multivariate A/B tests can be achievable by adding parameters/clicking Choose or create new.
- Adjust variant weights
Variant weight refers to the distribution of variants to users. It will be in the form of a ratio among all variants.
Step 5: Click the Review button.
4. Save the draft and review it and test in the testing environment
Test A/B test in Testing Environment
1. It’s a cherry on a cake.
Firebase leverages users to verify A/B tests in the device before making it live. Click AB test tab -> Select your AB test from the drafted section -> Open settings using 3 dots -> Click
Manage test devices.
2. Provide device instance ID and select the variant which you want to test. To get Instance IDs for Android and iOS:
a. Android snippet to fetch Instance ID as below:
b. iOS snippet to fetch Instance ID as below:
3. Add ID in the token box, select the variant which you want to verify. Click on Save.
4. Below is our example screens with changes visualized:
Deploy Experiment in Live Environment
Yeah! It’s time to make experiments live across all users to provide easy access and engagement. It’ll help us in deciding whether to roll out the variation in the live environment or not.
- In the Firebase console, on the left side menu clicking the A/B Testing option will show a list of running experiments, drafted versions that are in review/testing environment, and completed A/B test in case any.
- Our experiment will be visible in the Draft section as it is not yet published. Click the experiment title and open it. Experiments can start in two ways directly: one is from the Drafted section. Then, click the start menu.
- With this, we come to an end of our PART-2 of Firebase: A/B testing series. In the next part we will be covering Notification set-up, In-App messaging and exploring Multivariate A/B testing. Meanwhile, you can start implementing and reach out to us in case of additional queries at firstname.lastname@example.org. And we would be, as always, happy to help.