Firebase: Learn A/B Testing In Applications – How to configure

With Firebase, you can perform A/B testing in applications with minimal effort and better results. It’s a mighty easy platform.  

This is the second part of the series to guide you through set and Testing! Part 1- Ways to implement A/B tests in Firebase.

How to configure the A/B test?

We are starting with Firebase A/B testing using the Remote Config feature. Set up the A/B test in the Firebase console

1.  Click the A/B testing item from the left side menu and click the Create Experiment button.

Firebase A/B test console

Firebase A/B test console

2.   Popup box will ask to choose from- Remote Config, Notifications, or In-App Messaging to configure an A/B test. Since we will be configuring the test via config parameters select Remote Config option.

Firebase A/B test experiment types

Firebase A/B test experiment types

3. Create an experiment via Remote config:  

In the A/B test configuration screen, allow users to define and set up the A/B test with a couple of conditions and parameters to be satisfied. 

Here we’ll use the same parameters that were created in the “Set up Remote Config in Firebase console” section.

Below are the steps listed to show you how to configure the A/B test in the Firebase console using Remote Config.

Step 1: Provide the Experiment name and details.

Create firebase remote config experiment

Create firebase remote config experiment

Step 2: Condition to target users

In this section, we need to select the app in which we need to activate this experiment.

Target Audience Apply for firebase A/B test

Target Audience Apply for firebase A/B test

 

Here, for targeting, take All users to be eligible for an experiment based on conditions we use.

Activation Event(Optional): 

An event that activates the experiment for a user. All targeted experiment users will receive an experiment variant treatment from the start of the experiment. Only those users who trigger the activation event will be included in the experiment measurement.

Firebase A/B test Activation event

Firebase A/B test Activation event

Here, select Continue_Shopping(custom event) as an activation event to track how many users are clicking the shopping button, then click Next.

Step 3: Define Goals

Firebase A/B Test Primary Metric

Firebase A/B Test Primary Metric

By utilizing goal metrics (such as Crash-free users, User retention) plus maximum occurrences of events (such as daily user engagement, app_remove, first_open, custom events). 

Goal will be deemed a metric that can be chosen for the variant that best achieves the experiment leader as the primary goal. Click Next.

Firebase A/B test Additional Metrics

Firebase A/B test Additional Metrics

  • Types of metrics to track 

Same as goals, there is a list of metrics available in the above screencap, at max 5 metrics can be used per A/B test. Distinct metrics as additional metrics (secondary goals) can be configured as compared to goals as secondary goals.

For e.g. for the test experiment, we have set Purchase revenue as the primary metric and in additional metrics, we’ve selected other metrics

Firebase A/B Test Goal setup

Firebase A/B Test Goal setup

Step 4: Select variants

By using the parameters which are defined in Set up Remote Config in Firebase console section, multiple parameters are applied for variant setup with values for Control Group and Variants. Also, the A/B test has been configured as an alternative text and color shade for the shopping button.

  • Types of variants

Baseline indicates App’s original view, whereas variants define the different variations to accomplish the goal. At max 7 variants are allowed for configuration.
Multivariate A/B tests can be achievable by adding parameters/clicking Choose or create new.

Firebase A/B Test variants

Firebase A/B Test variants

  • Adjust variant weights

Variant weight refers to the distribution of variants to users. It will be in the form of a ratio among all variants.

Firebase A/B Test variants traffic allocation

Firebase A/B Test variants traffic allocation

Step 5: Click the Review button.

4.  Save the draft and review it and test in the testing environment

Test A/B test in Testing Environment

1. It’s a cherry on a cake.

Firebase leverages users to verify A/B tests in the device before making it live. Click AB test tab -> Select your AB test from the drafted section -> Open settings using 3 dots  -> Click

Manage test devices.

Run Firebase A/B testing on test device

Run Firebase A/B testing on test device

2. Provide device instance ID and select the variant which you want to test. To get Instance IDs for Android and iOS:

     a. Android snippet to fetch Instance ID as below:

    b. iOS snippet to fetch Instance ID as below:

3.  Add ID in the token box, select the variant which you want to verify. Click on Save.

Run Firebase variant A/B test on test device

Run Firebase variant A/B test on test device

4.  Below is our example screens with changes visualized:

Firebase A/B Test Variant vs Baseline

Firebase A/B Test Variant vs Baseline

Deploy Experiment in Live Environment

Yeah! It’s time to make experiments live across all users to provide easy access and engagement. It’ll help us in deciding whether to roll out the variation in the live environment or not.

  1. In the Firebase console, on the left side menu clicking the A/B Testing option will show a list of running experiments, drafted versions that are in review/testing environment, and completed A/B test in case any.
  2. Our experiment will be visible in the Draft section as it is not yet published. Click the experiment title and open it. Experiments can start in two ways directly: one is from the Drafted section. Then, click the start menu.
  3. Firebase A/b Test starting to experiment

    Firebase A/b Test starting to experiment

  4. With this, we come to an end of our PART-2 of Firebase: A/B testing series. In the next part we will be covering Notification set-up, In-App messaging and exploring Multivariate A/B testing. Meanwhile, you can start implementing and reach out to us in case of additional queries at hello@tatvic.com. And we would be, as always, happy to help. 
×

Please enter your email id to download Ebook.

    We hate SPAM and promise to keep your email address safe

    ×

    /p>

    Thank you for writing to us.
    We've received your request for free
    Our team will get back to you within 1 business day.
    maulik

    maulik

    Maulik Shah is a GAIQ Ceritified professional & working as Sr Technical Analyst at Tatvic Analytics.
    he is responsible for helping clients to improve their business performance by building their analytics capabilities.
    He manages engagements with key clients & driving end to end analytics practices.When not working with clients, he loves to Play Sports. He loves to explore new places.
    Previous Post
    Leverage GA4 for Ecommerce Reporting & Tracking: Make the Best Decisions!
    Next Post
    Change the dimensions of your digital marketing funnel with DV360

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    Fill out this field
    Fill out this field
    Please enter a valid email address.
    You need to agree with the terms to proceed

    Menu