Digital Marketers have been using the same metrics to evaluate campaign success for more than 20 years. Well, a lot has changed in that time; from an ever-changing privacy landscape and technology advancements to marketing-automation software and consumer behaviour.

With all these changes, Vigorate has been working to evolve email campaign measurement. Version 1 of our Asset Efficiency Model aims to answer these common questions we’ve seen surface:

1. Does an ‘open’ really reflect engagement?
2. How can I identify the success of one campaign over the other, regardless of audience size?
3. How can I take into account the open-rate bias that Apple's Mail Privacy Protection has introduced?

Sometimes, evaluating campaign success is simple, but more often than not we need a way to account for the variables at play to ensure we’re comparing apples-to-apples.

Let me start by using two examples,

Looking at the metrics above, it’s clear that campaign B outperformed campaign A. The open rate % and click rate % are better in comparison to campaign B.

Now, consider this scenario; at first glance, it may look like campaign C outperformed campaign D. 

However, in this example, it may not be an easy task to determine which campaign is more efficient than the other, for a few reasons,

1. The volume of emails is different; campaign C was delivered to only ten people, while campaign D was delivered to 1,200 people. This is where the % bias comes into play; just because campaign C had a nominal send volume, the results can look to be inflated. In reality, however, campaign C is unlikely to perform as well when sent to 1,200 people.
2. If the campaign were to be judged solely based on open rate, campaign C would be declared the winner over campaign D; however, with Apple’s Mail Privacy Protection, open rates are heavily inflated with some clients seeing as much as a 100% increase in open rates.
3. Evaluating a campaign solely based on these two metrics may not take into account other considerations, such as how well the subject line and pre-header copy were aligned with the body of the email, how impactful the email’s layout and CTAs were, etc.

This is precisely where our Asset Efficiency Scoring model comes into play.

The Asset Efficiency Score aims to compare the performance of different campaigns by assuming a volume of 100, then ranking them based on their click rate and open rate in a weighted fashion to paint a more accurate picture. This model considers metrics like Emails Sent, Emails Delivered, Unique Opens, Unique Clicks, Open Rate, Unique Unsubscribe Rate and Click Rate.

To better illustrate the model, we have used a weighting spectrum, highlighted below. The range considers the combinations with open rate and click rate and what they can collectively signify.

A desirable place for any business to be in would be the top left side of the table marked in dark green, which speaks to the fact that the audience is highly interested through the subject line and is compelled to open the email. When that happens, they are equally engaged, to such an extent that they go ahead and click through; a winning combination. On the other end of the spectrum, is the area marked in red towards the bottom right of the table, highlighting low click rates and moderate to low open rates (a poor combination).

The weighting spectrum helps us understand what makes a campaign successful over other campaigns. In short, it all boils down to a more effective weighting of the open rate and the click rate. Considering the above spectrum and Apple's MPP, our weighting is x2 points for click rate (representing actual engagement rate) and x0.5 points for open rate (representing initial engagement rate).

When this weighting is put together into the final equation, our Asset Efficiency Model looks like this:

Asset Efficiency Score = (((Unique Opens per 100/Emails Delivered) *0.5) + (((Unique Clicks - Unique Unsubscribes) per 100/Emails Delivered) *2))/2.5) 

You can download our open-source Asset Efficiency Scoring model here.

The output of which is on a scale of 100; the higher the score better the campaign performance, the lower the score, the poorer the campaign performance.

Let's look at this model in action:

These metrics are adopted from actual data; the campaigns have different send volumes. They range from as low as 51 recipients to as high as 20,832.

From this example, we are able to identify the following outliers that help us eliminate any bias,

1. Campaign 3 and campaign 4 are ranked 7 by the asset efficiency scoring model. Note that the send volume for campaign 3 is 51, while for campaign 4 it is 622.
2. Campaign 5 is ranked 10, whereas campaign 1 is ranked 20. Again, note the difference in send volumes. Campaign 5 has a sent volume of only 20,832, as compared to 5,559 for campaign 1.

The Asset Efficiency Scoring model provides a common denominator to the send volume for different campaigns and then evaluates them objectively in a weighted fashion. At Vigorate, we use this model to identify the top 10 campaigns and the bottom 10 campaigns to gain insights into what worked well and what did not work well with the given audience. This helps us extract more valuable learnings from campaigns!

You can download our open-source Asset Efficiency Model here.

A few things to remember before adopting this model:

1. This model does not consider email send volumes less than 10, as the data may be skewed and provide inaccurate/biased results.
2. This model is only suited for campaigns with a CTA, which can yield a click rate metric.

A few commonly asked questions that may come to your mind have been addressed below:

Why is the delivery metric used over sends?

How an email campaign is structured, in terms of its design and content, has no relation to the accuracy of contacts the email is sent to. The number of successful deliveries is to be used as a base to get a clear picture of the asset efficiency model.  

Why are unique opens used over the total number of opens?  

The impact of a campaign influencing the same person more than once versus a campaign influencing multiple people once would have a more significant effect from a business lens.  

Why are unique clicks used over the total number of clicks?  

The impact of a campaign influencing the same person more than once versus a campaign influencing multiple people once would have a more significant effect.  

Why is click rate used over click-through rate?  

The Click-through rate is dependent on open rates. For an unbiased asset efficiency score, the parameters should not be interrelated and, as such, should provide an independent score.  

What about Apple MPP?  

For this reason, only the unique open rates are being considered. And in the AEI calculation, unique open rates have a small weighting of 0.5. As such, the impact of MPP is minimized in this model.  

What about clicks used to unsubscribe?

For this reason, we are subtracting all unique unsubscribes from the unique clicks to account for unsubscribes and make the metric more accurate.

For effectively reporting on email campaigns and eliminating the % bias, the Asset Efficiency Model can provide a new lens for establishing patterns for understanding and replicating top-performing campaign characteristics.

Use this model to make the most of your marketing cloud, journey building and email marketing.