Firebase: Learn A/B Testing In Mobile Apps – Notifications and In-App Messaging Experiments

Welcome back to the Firebase quadrilogy. We have been learning a lot about Firebase through our 4-part series! Firebase enables you to perform A/B testing in mobile apps with minimal effort and better results.

So far we have learned about Ways to Implement A/B Testing in Mobile Apps and How to Configure it. In case you missed the first two parts, take a quick look at it from here.

A/B Testing In Mobile Apps – Notifications

The app analyzes notifications to increase user engagement. Its primary focus is to accomplish the app’s goals. The notifications act as a bridge between the users and the app to determine its effectiveness towards the end goals.

They allow app users to spend more time inside the app and make purchases. This leads to a higher conversion rate. The only requirement is:

“Send a message to the right user at the right moment.”

Personalizations via notifications are possible to ensure users constantly engage with mobile applications.

Below is the step by step guide to implementing Personalization via Notifications:

Setup Notification in Android Application

1. Add Cloud messaging dependency to app-level build.gradle file. Use the latest sdk version.

2. To implement Firebase notification, import FirebaseMessaging library in Java file

Firebase Notification: A/B testing in Mobile Apps

Firebase Notification: A/B testing in Mobile Apps

3. After creating the class, this step is mandatory to allow the notification service to be active when the app is in the background. Add service class which is created in the above step.

4. This step is Optional. It allows users to set a default notification icon and color.

 

Firebase A/B test experiment manifest meta-code snippet

Firebase A/B test experiment manifest meta-code snippet

Setup Notification in Firebase Console

A simple step-by-step directive to set up notification in Firebase Console:

1. Sign in to the Firebase console: In the Engage section, select AB testing. Click on Create Experiment. Select In-App Messaging from pop up.

Firebase A/B test experiment types

Firebase A/B test experiment types

2. Add experiment: Enter name, description, target users and other requisite details. Click Next.

Firebase A/B test Notification details & experiment targeting conditions

Firebase A/B test Notification details & experiment targeting conditions

Condition to Target Users: In Notification A/B test, there are various conditions to isolate users whom you want to send notification for app engagement. Such as App ID, User audiences (It allows to choose all users, purchasers, or any custom that are created), user properties, etc.

Firebase A/B test experiment targeting user types

Firebase A/B test experiment targeting user types

3. Define Variants: Add messages that should be displayed to target users through multiple variants or you can prefer original text. Click Next.

Firebase A/B test experiment variant details

Firebase A/B test experiment variant details

4. Define Goal: Do the users fall under a variant or original, which goal should you consider for accomplishment?

Goal types and metrics are the same as set in remote config. One new event Notification opens and a metric is added to better measurement.

Firebase A/B test experiment Goal condition selection

Firebase A/B test experiment Goal condition selection

5. Send Now or Schedule: Selecting Message option allows you to select either Send Now or to be Scheduled for the specific time to send notification.

Firebase A/B test experiment message options

Firebase A/B test experiment message options

Let’s explore Advanced Message Options:

a. Advanced options: Some advanced configurations can be set to send notifications with dynamic data.

b. Title: The notification title that is shown on Android devices and Apple watchOS devices.

c. Custom data: A set of key-value pairs that is delivered to the app along with the notification.

d. Android Notification Channel: It provides the ability to group the notifications that our application sends into manageable groups. By default, it will be blank

e.  Sound: Disabled or Enabled

f.  Expires: An interval that determines how long the message is kept for redelivery

v

Firebase A/B test experiment Notification configuration advance options

6. Click on Review.

Firebase A/B test experiment overview

Firebase A/B test experiment overview

7. Test the Experiment: Access the device registration token. FCM SDK generates a registration token for the client app instance using that we are allowed to target single or multiple devices at once.

Firebase A/B test experiment testing on device

Firebase A/B test experiment testing on device

8. Post Testing Completion: Review the experiment and start it. Analyze the results for a few days. Here is a sample notification of our A/B test that Firebase has sent to all users who fall into the variant.

Firebase A/B test experiment sample screenshot

Firebase A/B test experiment sample screenshot

In-App Messaging

Firebase A/B Testing in Mobile Apps lets you evaluate multiple variants of a In-App Messaging.

It helps you engage your app’s active users by sending them targeted, contextual messages that encourage them to use key app features.

For Example, you could send an in-app message to get users to subscribe, watch a video, complete a level, or buy an item. You can customize messages as cards,

banners, modals, or images, and set up triggers so that they appear exactly when they’d benefit your users most.

Setup In-App Messaging in Android Application

1. Add Firebase SDK & In App Messaging Library in android app as shown below.

In App Messaging Library in android app

In App Messaging Library in android app

2. Sign in to the Firebase console

In the Engage section, select In-App Messaging. Click Create Experiment. Select Notifications from pop up.

Firebase A/B test experiment types

Firebase A/B test experiment types

3. Add Experiment name & its description as shown below

Firebase A/B test experiment details

Firebase A/B test experiment details

4. Add banner details, button text, image url, etc in baseline & other variant models. 

Firebase A/B test experiment variant details

Firebase A/B test experiment variant details

Create a card-like structure which will be visible to the users inside the app.

5. Add details for targeting users in apps.

Firebase A/B test experiment targeting conditions

Firebase A/B test experiment targeting conditions

6. Add Goals to experiment. 

Select Primary metric to decide a leader / winner for variants shown to users. Add additional secondary metrics to compare the numbers for multiple variants in A/B Test experiment.

Firebase A/B test experiment Goal condition selection

Firebase A/B test experiment Goal condition selection

7. Schedule In-App Messaging. 

You can schedule In-App Messages or send it on the go as well. Additionally, you can mention the end date on the console to automatically stop the campaign for all users. 

Add activation conditions for In-App Messages for users who have triggered that event. 

For example, below we have shown a button_click as an activation event for In-App Messages to be displayed to users.

Firebase A/B test experiment in-app messaging scheduling

Firebase A/B test experiment in-app messaging scheduling

8. Click on Review

Firebase AB test experiment review

Firebase AB test experiment review

9. Test In-App Messages

Now we can test the In-App Messages as well in testing devices with the help of Instance ID token or FIS auth token. We have shown below the same for reference. Add testing device token & save that.

Firebase A/B test experiment testing on device

Firebase A/B test experiment testing on device

For sample, we have shown below two variants of configured In-App Messaging Banners.

Firebase A/B test baseline vs variant

Firebase A/B test baseline vs variant

10. Post verification in the testing device. Click on Start Experiment to make this experiment live.

Firebase A/B test experiment Overview

Firebase A/B test experiment Overview

On a Closing Note

Firebase has proved to be a crucial addition to enhance In-App messaging capabilities. You can perform A/B testing in notifications with quick and accurate results. 

So, how has your experience been so far? Drop your feedback and experiences below and we will reach out to you!

This is the third part in the series to guide you through Firebase A/B Testing in Mobile Apps! Part 4 will help you with –  How to Set Configure A/B testing.

 

The following two tabs change content below.
Previous Post
BigQuery Integration with GA4. You heard it right, it’s FREE!
Next Post
Simply, know more with GA4 Advertising Reporting Features

No results found.

Leave a Reply

Your email address will not be published.

Fill out this field
Fill out this field
Please enter a valid email address.
You need to agree with the terms to proceed

Menu