Have you ever wondered how you can accurately attribute user activity or even increased revenue to a Pulsate campaign? Maybe you sent a campaign and saw an increase in user activity or transactions but were wondering if that would have happened anyway without the campaign being sent. If any of that sounds familiar then control groups are the perfect tool for you.
What is a Control Group?
A control group is typically a small group of users that you do not send a campaign to so that you can compare the attribution rate of the campaign between the control group and the larger group of users who did receive the campaign.
One of the problems that mobile marketers often face is that it is difficult to accurately attribute a user action to that user receiving a campaign. A user may have just received a campaign for one of your products and purchased that product but how do you know that they wouldn't have purchased that product anyway?
If you use a control group, you can compare the behaviour of users who received the campaign with those in the control group who did not receive it and accurately determine the level of attribution for the campaign.
Control groups work best when they are used on campaigns that are sent to a large group of users. In Pulsate, we recommend that campaigns are sent to at least 10,000 users to ensure that the control group results remain statistically relevant.
To send any campaign with Pulsate you have to set up a conversion goal. Control groups work really well with conversion goals as Pulsate will track how many users in the control group carried out the conversion goals so that you can track the uplift in conversion goals between users who received the campaign and those who were in the control group and did not receive it. For more information on conversion goals, please click here
How do Control Groups work?
For most campaigns, we do not know how many users will receive the campaign when it is sent because new users are being added to the segment for that campaign or if it is a beacon or geofence campaign, new eligible users will enter the specific locations where the beacon or geofence are located during the period of the campaign.
For that reason, when a control group is enabled for a campaign, Pulsate will choose a random sample of users to be added to that control group and not receive the campaign. If a control group is set to 10% of users that are eligible to receive a campaign, this means that every campaign recipient has a one in ten chance of being added to the control group. This also means that the control group may not be exactly 10% of the recipients because we did not know how many users would receive the campaign until it was finished.
Using Control Groups
When you send a Pulsate campaign you can select whether or not to add a control group in the set-up page when you are naming your campaign and setting the conversion goal. To enable a control group, toggle on the slider beside Enable Control Group and choose what percentage of eligible users for that campaign should be added to the control group. When you are finished setting the name, conversion goals, and control group press Save.
Enabling a Control Group
Before you send the campaign, you will have an opportunity to check that the control group has been accurately set on the Campaign Review page.
Campaign Review Screen
When a campaign is finished, you can export a campaign report to find out what the difference in behavior was between the users in the control group and the users who did receive the campaign. To export a campaign report, go to the campaign list screen and click on the download synbol to the right of the campaign you sent.
Export Campaign Users
An email will be sent to the email address of the Admin that is currently signed into Pulsate. The email will contain links to CSV files of both the users who received the campaign and those who did not so that you can accurately compare the attribution rate between both groups.
You can also click on the stats page for that campaign to check what the difference in conversions is between users who received the campaign and those in the control group who did not receive it.