Feature Enhancement: Compare Your Journey’s Performance With a Control Group

We have made a few updates on how conversions are reported on the journey page.

Before we explain how these conversion numbers are calculated and how you can compare it with a control group, let’s first delve into a core concept of journeys called Trips. Conversions numbers are reported on a trip basis so its important to cover the concept of Trips first before we discuss journey conversion in detail.

A user can enter a journey more than once. Let’s say that you are running a journey that gets triggered each time a user adds something to his cart (in other words, user does the event Add to cart). You would like to wait for 30 minutes in this case and check whether the user has done a purchase. If the user has made a purchase, then you end the journey. If the user has not made a purchase then you have a complex workflow defined where you send a series of email and push notifications to the user to get him to purchase. In this case, each time the user adds something to his cart, the journey gets triggered. If the user is adding to his cart 5 times a week, the journey would get triggered 5 times. Let’s also define Purchase as the conversion event so that as soon as the user purchases something after receiving your campaigns, that instance of the journey of the user ends. Please note that the journey itself does not end but only an instance of that journey ends. We’ll call each such instance of the journey as a Trip. Each time the user adds something to cart, he enters the journey again and a trip starts.

Let’s elaborate on this further.

Let’s say that the user adds to his cart for the first time on Monday and immediately purchases the item within 10 minutes of adding to cart. We’ll call this Trip 1. Trip 1 ends immediately as the user has performed the purchase immediately. On Tuesday, the user adds something to his cart but forgets to make a purchase that day. Let’s call this Trip 2. In this case, the workflow involving email and push notifications would now get activated. The user clicks on the CTA in the email and makes the purchase. Trip 2 now ends because of the conversion. On Wednesday, the user again adds something to his cart. As soon as that happens, Trip 3 gets activated. Increase your time horizon to a month or even to a year and you realize that you have the same journey running for the same user in so many different ways. Isn’t this such a beautiful concept? Just abstract all your complex scenarios and let WebEngage do its magic by tracking the user in real-time along each of these trips.

Let us also go through the concept of Engaged Users Through a Journey before we define conversions. You would agree with me when I say that it wouldn’t be fair to attribute journey conversion numbers to WebEngage unless a user has received a campaign because of the journey. If a user doesn’t receive a campaign through a WebEngage journey but still ends up doing the conversion event, WebEngage should not be given the credit for this conversion. For a particular journey, let us refer to all the users who have received a campaign because of this journey as Engaged Users. If you want to calculate the Unique Engaged Users, these uniques would be on the basis of trips. If a user has received 2 campaigns as part of Trip 1 and 3 campaigns as part of Trip 2, the Unique Engaged User count for that journey is 2. Remember, we are talking about unique users in a trip. In this case unique user in each of the trips is 1. And there are 2 such trips which brings the user count to 2.

We’re now ready to talk about how the conversion numbers for journeys are calculated and how you can compare the journey conversion numbers with a control group.  

WebEngage reports unique conversion numbers for journeys, by default. These unique conversion numbers are based on engaged users in trips. If an engaged user does a conversion event more than once for a particular trip, the unique conversion number for this trip (Trip 1) would still be 1 (since the same engaged user has performed the conversion event). For another trip (Trip 2), the same engaged user again does the conversion event after receiving a campaign through Trip 2. In Trip 3, the same engaged user does not do any conversion event in spite of receiving a campaign through Trip 3. Therefore, the unique conversion number for this journey is 2 (since the user has performed the conversion event 2 times in the 3 trips he has undertaken in this journey).

Always remember this – the unique conversion numbers in journeys are based on unique conversions in trips.

Here’s the formula we use to calculate conversions.

  • Conversion = Number of unique engaged users (uniques in trips) who converted / Total of all unique engaged users (uniques in trips)
  • Control Group (CG) Conversion = Number of unique users (uniques in trips) in CG who converted / Total of all unique users (uniques in trips) who were in CG. Remember that CG users who are part of the journey do not receive any campaigns. Therefore, we don’t use the Engaged User definition for the Control Group. Also, please note that the users who are part of CG will always remain a part of CG irrespective of the number of trips they undertake. New users can enter the CG since WebEngage ensures that the total users in CG is always a certain percentage (5% by default, and can be changed by you) of the total users who enter the journey.

In the screenshot above, you can see that the conversion of the journey is 5.86% whereas the conversion number for the Control Group is 3.78%. This means that the journey has a great positive impact on your overall conversion numbers. In this case, the uplift is 55%. This means that through this journey, you have been able to increase your purchases by 55% for the group of users who entered this journey. Imagine if you were running many different journeys for all sorts of different scenarios. Beautiful, isn’t it? 🙂

Lastly, please note that the conversion numbers in journeys get refreshed every 2 minutes. All other numbers you see in journeys such as entries, exits, numbers on different blocks of journeys etc. get refreshed every 5 seconds.

We hope this clarifies how the conversion numbers in journeys are calculated and how they compare against control groups. If you still have any questions around this, please get in touch with us at support@webengage.com.

Comments are closed.