Understanding KPIs Through Email Marketing Analytics Tutorial

4.2 Introduction

Hi, everyone. My name is Avinash. I'm the author of the books Web Analytics 2.0 and Web Analytics In Our Day. And this is a video about email marketing analysis. And specifically how to choose the optimal set of metrics that you would use in order to analyze your email marketing campaigns from an end-to-end perspective. As well as how do you go about defining these metrics? While this video is all about email marketing, my hope is to teach you how to be very careful and thoughtful and strategic in how you define your key performance indicators. What goes in the numerator, what goes in the denominator, and, and how, whether you substitute one thing or the other in your definition, can drive a completely different kind of behavior in the, in your company's decision makers. So, pay attention to this video, because you're going to learn tons about how to do great email marketing analysis. But also pay attention to the choices being made in the metrics. Because those lessons you'll be able to apply to all other things that you're doing, all other kinds of math campaigns that you have, all other kinds of metrics that you've created for your company.

4.3 E-mail Matters

Now you perhaps are thinking in this age of social media, and all of the other things that are so much cooler than spam and emails and Viagra emails and Nigerian princes, should anybody even invest in email marketing? And let me emphatically say yes. I've worked with numerous companies and I continue to work with numerous companies that span the B2B, B2C, nonprofit, for-profit spaces. And email marketing done right continues to be one of the most profitable acquisition channel and a great retention channel for most companies. In fact, if the portfolio of any company does not contain a great email marketing program, I almost immediately think that they're leaving a lot of value on the table. The only caveat, the little asterisk that you see at the corner of my slide, is, is to emphasize that I'm a very big fan of permission-based emails. And, and so giving people clear opt-outs, only emailing people when they've opted in, and only sending relevant messages to people, not bombarding them constantly, and not becoming a unwanted stalker for your consumers. As long as you can meet these few criterias that I all roll up into permission-based marketing, you will discover that email is going to be one of your BFFs when it comes to long-term success. So I highly encourage you to experiment with it if you don't do it. And if you do do it, then to strongly embrace the principles of permission marketing and success will be yours.

4.4 Measurement Challenges

With that little preamble, let's get going. The first thing that I want to cover is the unique challenge that measurement of email posses before you jump into google analytics or other things it's important to understand why email marketing is so challenging. Regardless of the company that you are using, typically, a lot of stuff actually happen in email platform providers. Here are two of the ones that are in the marketplace, but really doesn't matter which one you use. And typically, you'll upload your lists into these programs. And you might even use them as some kind of CRM system where all kinds of attributes of your consumers are easily available to you. And it's essentially a database where you do all things related to the beginning and execution of your email marketing campaigns. Once, you actually release your emails into the wild, hopefully permission based, they end up in people's inboxes. And there you have all kinds of email programs. You see Gmail on the screen here, but you have got Outlook, you've got Thunderbird, you've got Yahoo, you've got all kinds of email programs all around the world. And your emails will have to pass through lots of hop, skip, and jumps, and spam filters, and JavaScript being turned off in email programs, etc etc, try to make it there. And if you write attractive subject copy and are clear on where the email comes from, so the from field and the subject field is perhaps the most important two things that will ensure the email actually gets opened, then people might read it. And if it is compelling and relevant they might click on a link in the email and come to your website, where you're selling whatever you're selling. CrashPlan is a backup provider that I, personally, use in order to backup multiple amounts of my data from many different computers into the cloud. It's a great service. And so, this is a end to end sort of email experience when you do marketing, and hopefully, when people land on the website, they'll go sign up for the service or the product that you're selling. Now, when this process happens, let's flip the model to thinking about where all the data exists. So, there's a lot of data that exists actually in your email service providers. In fact, very critical crucial data that most web analytics forget exists in this source. The second of core source of data will be get created within the email programs that your consumer's using. And the final bit of data, of course, is the website analytics tool that's going to collect the behavioral data of what actually happens on the website. Though, the interesting thing about this three step process, is that data actually ends up in two completely different places. The data that is collected by the email program, or by the number of emails that you had created, the number of people to whom it was sent out, and perhaps if you have segmented the attributes, this email is going to all mend who are bald and have purchased sexy underwear. So, that's one group of people. Another one is women who like to fly over mountains. That's another. So, all of that actually critical demographic, psychographic data, all just attributes of your list are all stored in your e-mail service providers database. And, that of course can be stored in the first source of data. The execution of you campaign data actually is also going back into this database. And so, the number of emails that you send and what happens at the ISP, how many of them bounced to back, etc, will be collected and sent into this same database. And a lot of people don't realize that most of the data related to the email itself, let's say email open rates, as an example, is collected and sent back also into this database. So, these three valuable sources of data actually never reach your web analytics tool, unless you've done some special extreme hacks and integrations, etc etc. All of this data actually will be sitting in data source number one. And most likely, this will be your e-mail service providers database, so that's the first source of data we're going to deal with. Once people click on the email and there is action undertaken. That source of data actually ends up in your web analytics too. And so, that's one first source of data. And the second source of data is everything that happens within the website itself. So, all of that stuff around the number of people who have seen various pages, and etc etc. And if they ended up purchasing something, all the data actually ends up in these second source, which is your web analytics vendor. And so, actually, a lot of people don't realize that if you actually want to do end-to-end analysis of your email marketing programs, you're going to have to tap into these two databases. And after your BFF is actually Excel. Right, again, we live in a very advanced ages, with many different amazing tools available to us, but it is most likely that you're going to have to leverage your BFF Excel in order to bring these two sources of data together, and then be able to do some very effective and efficient analysis of your email marketing program. There are some tools that allow you to integrate superficial data both Constant Contact and MailTube had some integration with Google Analytics, Armature, Webtrends, and other tools. But what a lot of people don't realize is that integration often is this red arrow and this orange data. So, what they do is they'll campaign tag these links so that when people click and come to website, this particular orange arrow carries with it the identifiers of the email marketing programs. There are other vendors that have integrated with tools like Omniture where some shreds of this data, usually just this particular arrow is, comes through to your web analytics tool, but not these two pieces. And so, it's important to realize that even if your web analytics window tells you that all of the data for email marketing campaigns is available in the tool, it's most not likely not the complete picture you would like to see. So, whether you use an Access database or an Oracle database, or Microsoft Excel, God bless Excel. You're going to have to bring these two data stores these two blue boxes that you see into excel. Extract data from those two sources in order to do the end to end analysis that is required. So, it's very important that you appreciate the complexity of measurement and prepare your company and yourself to be able to ensure that all the orange arrows are available to you because without it you won't be able to compute many of the metrics that I'm going to recommend to you in the rest of this video. Three step process for doing email marketing campaigns Excel BFF, that's your summary.

4.5 Categories of Data

There are three categories of data that you will be analyzing when it comes to your email marketing program. The first category of data is Acquisition. And that essentially is all your campaign data. How many emails were sent, to whom, what happens when the email went out, etc, etc. So, all of the things you did in order to acquire the traffic, whether they're current customers, prospective customers, or just well, let's hope you're not sending emails to people who are not expecting to be contacted by Nigerian princes. So let's say, let's leave that aside. These are essentially the kinds of things you'll do in order to acquire traffic, and so the first bucket of data you have to think about is the acquisition bucket. The second bucket of data that you'll think about is your behavior data. In our scenario, that is all of the data that you'll get from the website. What happened after the click left the email program? How many people bounced? What was the engagement on the website once people came, etc., etc., etc.? So everything that happens after the person lands on your website. And the third category of data that you'll have, you'll be needing is the outcomes data. So it's the goals, it's the conversions, the revenue, it's the multi-channel impact of all the people that you drove to your website from email marketing programs. How many of them ended up making a purchase in the store? How many of them ended up calling your call center? So, tracking all of those conversions is super important. And credit it, crediting it back to an email program is super important. And in our curriculum, we've got videos about multi-channel analytics that can teach you exactly how to do this. But be aware that you're also going to compute multi-channel impact. And of course customer lifetime value and other key things to look beyond the single session conversion that happens because of email. And look at the customer life cycle value for the consumer you acquired through the emailed marketing programs. So there are three important categories of data. They should call metrics key performance indicators they should be thinking about. Acquisition, behavior, outcomes.

4.6 Optimal Metrics: The Dirty Dozen

So let's move to the meat and the potatoes and the apple pie part of our video, the Optimal Metrics, the key performance indicators if you will, that you would be tracking through your email marketing programs. And again as I said, just keep at the back of your mind that we're not only going to learn the specific key performance indicators for email marketing, we're also going to learn about nuances and creating any kind of key metrics and computation of these metrics in order to be much smarter about doing analysis. Before we go on, I want to do a quick Sidebar, just a very quick Side bar before we jump in to Key Performance Indicators. And it's this, it's this, I really want to implore you that never, you should never, ever send out an email marketing campaign without insuring that every single link that is in that email marketing program is tagged with campaign tracking parameters. So with that, instead of sending out an email like email, a link in the email that looks like this, yoursite.com/page or whatever it is, always ensure that the rest of this chain is attached to this key engine. Essentially the campaign tracking parameters. And these are the campaign tracking parameters. For Google Analytics. The UTM source, UTM medium, and the UTM campaign. If you look at Omniture it might be SID. If you look at Coremetrics\g, it might be something else. If you look at Webtrance, it might be something else. What's important to understand is that these campaign tracking parameters must be attached to every single link that you include in your email marketing. If you do not do this one simple step, you can not compute 50% of what I'm going to share with you in this video. So it's very, very important that you do this. With that, let's get going.

4.7 1. Delivery Rate

So, the first metric that we are going to analyze in our quest to understand email marketing programs is called Delivery Rate, and this is the very first metric in most of the car data that you need will come from your email service provider. And the definition of delivery rate is the number of emails sent minus the number of bounce backs, the email that never even made it through the ISP. In fact it bounced back to your company because you had a wrong email address or because of some other problem. So the number of emails sent minus the number of bounce backs divided by the number of emails sent. And what this tells you is how many emails even had a fighting chance of getting to the person to whom they were intended rather than dying on the first step that was available. It's very important to compute delivery rate and this helps you ensure that you keep your email list as clean as you possibly can to ensure a 100% delivery rate. And in the three buckets that we had, the three categories of metrics, acquisition, behavior and outcomes, delivery rate is an acquisition metric. So we're going to cover a set of acquisition metrics initially.

4.8 2. Open Rate

The second metric that you will analyze is called open rate. And open rate is the number of emails opened divided by the number of emails delivered. So the number of emails that actually got to the inbox and the number that were opened. Now it's important to understand that we could have used the number of emails opened divided by the number of emails sent. But that wouldn't really represent the complete story, because a bunch of these emails bounced back. And so in order to be much better about measuring more accurately the number of emails that even had a chance of being opened, they had to be delivered to the person at the other end. And that's why the metric that we've defined is not using the number of emails sent, but by the number of emails delivered. So be very careful about how this choice can show the correct picture or an imprecise picture. There's one other very important caveat that you should be aware of for open rate. And that is, most modern email reading programs have a preview pane. And when an email is shown in the preview pane, the JavaScript is normally turned off, images are normally turned off. And that means that the, that your email service provider will not be able to capture the fact that the email was previewed. And in most email marketing programs, for me, both Thunderbird and Gmail as an example. You would not be able to know that I had opened the email, because while the email was quote unquote opened, it was opened in the preview pane. It would not be reported back to you as an opened email. So it's important to understand that the opened rate that you are looking at from your email marketing campaign represents a very tiny percent, usually. Of the people who actually opened the email because most people previewed the email. The sad thing is there's no way for us to know that they previewed the email. So take this particular metric with a tiny, if not a bigger than tiny grain of salt, but it is a standard metric. And you should know the caveat, and you should measure it if you do with the emails delivered. And make sure that your management team is aware of the caveat. This also of course is an acquisition metric.

4.9 3. Subscriber Retention Rate

The sec, third metric that we're going to analyze is Subscriber Retention Rate. Subscriber in this case simply refers to the person to received your email. And the way that this metric is defined is the number of subscribers minus the bounce backs, minus the unsubscribed from that email program, divided by the number of subscribers. And what you are trying to understand here is how clean your list is by measuring the bounce backs. Because if your list is not clean, well you've not going to have a very good retention rate especially if the problem was at your end. Secondly, the number of unsubscribes that happen. So, if you actually send me a unsolicited email, I will instantly unsubscribe you without so much as reading the subject line of your email. I just scroll up, scroll down, find the unsubcribe link, instantly unsubcribe you, and you'll lose me as a subscriber because you did not use permission marketing principles when it came to email marketing. So, subscriber retention rate is a quality metric. It helps you understand how clean your list is but also helps you fine tune the number of times you send emails, to whom you send emails, and are you upsetting them or delighting them? Because if you upset them in an instant people unsubscribe. And so this is a fabulous metric to hit people on the head with to ensure that you have a high subscriber retention rate and in order to have a high subscriber retention rate, you're going to have to do a lot of things correctly. So subscriber retention rate is a very valuable acquisition metric.

4.10 4. Click-to-Delivery Rate

The fourth metric that we'll analyze is one of my favorites, it's called the click-to-delivery rate. And essentially what it measures is the number of clicks in the email divided by the number of emails delivered. Again, not emails sent but emails delivered. The first thing that, this metric helps you understand is the mailing list quality. Because if you didn't, if you didn't actually even manage to deliver the email, then your mailing list quality is pretty bad. And the second thing it helps you measure, is the email content relevant? And as I said from metric number three, this is so critical and of course because it measures the clicks that actually helped on the email. It helps you understand if the content you're delivering to them is relevant or it is not relevant. So I really loved this metric and what it measures and it is important to understand that this acquisition metric could have a higher than 100% rate because if your emails were highly relevant and provided many stories, then perhaps every person who received the email might have clicked on individual links multiple times or in multiple links inside one email. And so, don't be surprised if the number of clicks are higher than the number of emails delivered. Of course for your business you want higher number of clicks, so that is awesome.

4.11 5. Bounce Rate

Metric number five is bounce rate. And we know that bounce rate is a wonderful metric, but in this case it really helps us identify one very important thing. Essentially what it measures is the number of email campaign visits with a single page view and no other action like an out bound click, or click play on a video, or any other click on your website. Divided by the number of email campaign visits. So bounce rate essentially is a b, is, helps you meet this wonderful quote that I keep repeating all the time. Never let your campaigns write checks that your website cannot cash. Essentially what we're saying is that the persons responsible for sending the email campaigns is also responsible for ensuring the landing page in that email campaign is really relevant. And creates a continuity between what was said in the email and what is then said on your website to ensure that people don't land on a page that is completely irrelevant and leave right away. So bounce rate is a wonderful metric to ensure the effectiveness of your landing pages for your email marketing campaigns. In the three categories of acquisition behavior in outcomes, bounce rate is a behavior metric because we have stepped onto the website and we're trying to ensure that we measure the effectiveness of our landing pages on the website.

4.12 6. Depth of Visit

Metric number six is depth of visit. All right, so people did not bounce. Well, not bouncing and creating a single click by itself doesn't mean that everything will be successful on your website. So depth of visit measures the percent of email campaign visits that last longer than xx pages. And the xx pages is a metric that you get to define. What is after all of the cost and effort that went into email marketing campaign. How deep should the person, at least, visit the website, even if they don't buy something? At least, if they see three pages, get to the product information page. It increases the chances that they'll come back again or buy the product in the offline world. So you get to determine what this number looks like. In fact this number, this metric is especially important for non e-commerce websites, where content is essentially the thing that you're selling. And the ad space next to the content is the thing that you're selling. So it's very important to measure depth of visit, especially if you have a non e-commerce website segmented for your email marketing campaigns, but before you go into computing this metric, work with your management team to try and figure out what depth of engagement on your website represents some modicum of success for the business? Depth of visit is also behavior metric.

4.13 7. Actions Completed

Back to number seven, pardon me, are the actions completed. And essentially it's the number of campaign visits that watched a video or did x, y, or z. And x, y, or z could be that they downloaded something. Or that they've looked at the demo of the product, or that they did something, took an action on your website that is of value. If I am a content website, let's say a newspaper, then the number of actions completed could be the percent of people who actually submit a comment on the website or the number of people who subscribe to your RSS Feed or things like that. So, so viewing pages is important, but actions completed start to get into this idea of micro-conversions, and we're going to talk about this a little bit later in this video. But it's important to analyze the behavior of people beyond the fact that they just compute a viewed pages, but actually did something on the website that was a deeper engagement, took an action that was valuable to your business, and especially for ecommerce website. All of these actions are super important, because at the end of the day, your conversion rate will just be a few percent. Then it's important to understand if they didn't convert, what are the other actions that it took that are value to the business and we're going to measure the percent of email visits where people are taking actions because that is adding value to our business. So, number seven is a very valuable behavioral metric.

4.14 8. Macro Conversion Rate

Metric number 8 of course is the reason that we exist to do business in the world. It is our macro conversion rate, and it is a very simple metric. The number of one big outcome on your website, and one big outcome for any commerce is a conversion rate. One big outcome for a car company is a lead submission. One big outcome for a content site is that you subscribe, make an off-line subscription for a print version of the magazine or the novel, or anything. One big outcome from a church is that you downloaded this sermon. One big outcome, well, you get the point. What is the single biggest most important thing to your company? And we're going to measure, we're going to count the number of times that happens, and we're going to divide it by the number of email campaign visits. And we're going to compute our macro conversion rate. And number 8 is our first outcomes metric. Outcomes are so important, and I am so disappointed that frequently we don't even bother to measure outcomes for our marketing campaigns. Well, we're going to start that by measuring a macro conversion rate.

4.15 9. Average Revenue Per E-mail Sent

Metric number nine for e-commerce websites only is the average revenue per email sent. This is simply the total revenue divided by the number of emails sent. Now you might be wondering why is it that I had emphasized emails delivered and suddenly I've switched to the number of emails sent? Perhaps you would have expected me to use the number of emails delivered. But I'm trying to be sly here, sly as a fox, perhaps. Because I have also discarded the number of emails opened. So I have now discarded the number of emails opened because we knew there was an asterisk next to that metric and this doesn't, and this metric does not measure correctly what's happening in the email program. So rather than using these two metrics as denominators, were going to be sly like fox and use the number of emails sent. The number of emails sent is actually going to make the total, the average revenue look lower than it should be. Because not all emails were delivered and the reason we're going to sent using the number of emails sent and not emails delivered or open, is because we want to put a lot of pressure on our email marketers to ensure that the emails are clean, I mean to say the email addresses are clean. That mode, that they're fewer bounce backs, that the emails are relevant, the context is good and if we do all of those things, this metric will become better and better over time. In fact the number of emails sent might be the number of emails delivered if you ensure 100% delivery rate. So we're going to deliberately make the average revenue per email sent, the average revenue look lower. And use email sent to put pressure on our email marketers to ensure that they are working very hard to make sure every email gets delivered. And if they do their job well, well the average revenue will be reflected correctly. So you see how we're being sly like foxes here in order to drive the right kind of behavior. Average revenue of course is a outcomes metric.

4.16 10. Micro Conversion Rate

Number 10 is the micro conversion rate, and micro conversion rate essentially is the number of many small outcomes of value to the business divided by the number of email campaign visits. And in this case, we are using the number of visits to the website for my web analytics tool. We're measuring all our microconversions, the number of leads submitted, the number of wish lists created, the number of videos watched, the number of demos watched, the number of downloads the number of. You get my point? All of the small things that are a value to your business, compute the total of that, divided by the number of email campaign visits and you have your micro-conversion rate. If you put the micro plus macro conversion it together you are showing your business the complete picture of success and not just the single short-term conversion. It's very important to measure both of these metrics. Micro conversion rate is an outcomes metric.

4.17 11. Average Economic Value Per E-mail

Number eleven, the average economic value per email. And this is simply the total economic value divided by the number of emails sent. Notice again the number of emails sent. And the economic value is their revenue from the macro conversion, plus the economic value from the micro conversions. You put that together and you have your total economic value. And this metric, again, is to show the holistic Impact of email marketing. If you just show revenue, you're showing partial success. You show economic value, you're showing complete success. And again, if you would like in depth knowledge and information on specific strategies to compute, to identify macro plus micro conversions and compute economic value, please look at the videos we have on this topic. They are fantastic videos, and I guarantee it they're going to get you to think radically differently about web data analysis. But that said, metric number 11, our outcomes metric focuses on emails sent again and is an outcomes metric of great importance.

4.18 12. Profitability

Our last metric but not the least metric, perhaps our most important metric is Profitability of our email marketing campaigns. Measure the revenue generated, minus the campaign cost, minus the cost of goods sold, divided by the number of emails sent. And what we're measuring here is the revenue that we acquired from our campaign. We subtract the campaign cost, what it cost to said do the campaign, you're going to have to work with your finance team to get this number, but this includes the fees of your email service providers, you're, the input into your list management process, the cost of sending all the emails out, the cost of managing the unsubscribes. And if you're really, really good, it will include your salary if you're an email marketer. And all of those roll into the cost of running email marketing campaigns. And the last part of this computation is the cost of goods sold. So if you provide this service or a product it costs you something to provide that and so you would subtract that and simple example is. You're selling an iPod for $400, it cost $350 to sell the i, to create the iPod, so that is your cost of goods sold. So the two, the numerator of this particular formula, in assuming costs of $10, revenue $400, cost of $10, cost of goods sold of 350. This number will come up to be $40, $40 went to the bottom line of your company that was the profitability that you acquired from this particular conversion. And that you will divide by the number of emails sent and you know how much profit you're making for every email that you send, and this of course is an outcomes metric. So 12 simple metrics that help you understand the end to end performance of your email marketing program. You'll have to go to two different sources to get this data. You'll most likely create these metrics which are a mashup of things that are happening in all of those orange arrows that you saw at the very beginning of this video. But this will give you a very good picture, a comprehensive picture of how your email marketing program is performing for your company.

4.19 The Critical Few

If you don't have a lot of time, you know how much I love the Critical Few concept. And in this case if I had to choose the Critical Few metrics and recommend them to you, my first recommendation would be to use number four, Click-To-Delivery Rate. I believe it is one of the best acquisition metrics that is available for email marketing programs and it drives a lot of right kinds of behavior both on the click side because of the email content relevance and on the delivery side everything happens in your ESP your email service provider. Metric number two, I will recommend would be bounce rate. I believe most email programs have a very poor conversion rate because people are being sent to horrible landing pages. And that's why I love using bounce rates. Now if you were a non-ecommerce website, you could perhaps use the depth of visit as a metric. If you believe your landing pages are optimized you can swap this one out, and use the number of actions taken. Whichever behavior metric you choose is okay, but I would initially at least choose Bounce Rate as the behavior metric. And the third critical view metric I would use is Profitability. For all of the reasons that I just covered a few minutes ago. At the end of the day, revenue does not represent what you added to the bottom line of the company. You have to take out the cost of the campaigns. You have to take out the cost to get sold. And then you know that for every email you sent out, you added X or XX, or XXXXX Dollars, pesos, rubles worth of value to your company. So if I had to choose Critical Few metrics, I would just choose these three metrics out of the 12 from the email marketing programs. These would be my key performance indicators. Now, in order to analyze them optimally, you might have to go back and look at the other metrics, and drill down on them. And that is okay. But these are the three Key Performance Indicators I would chose and you notice that very conveniently they've, there is one from each of the critical categories that we should be focusing on.

4.20 Conclusion

So, that's email marketing and email campaign analysis. I hope that you got a lot of specific ideas about what metrics you should measure, and you have a much better appreciation for how to overcome the measurement challenge that email poses us. Do permission marketing emails, and I guarantee it, that it will end up being one of the best acquisition channels for you company. I wish you all the very best and happy analytics.

  • Disclaimer
  • PMP, PMI, PMBOK, CAPM, PgMP, PfMP, ACP, PBA, RMP, SP, and OPM3 are registered marks of the Project Management Institute, Inc.

Request more information

For individuals
For business
Phone Number*
Your Message (Optional)
We are looking into your query.
Our consultants will get in touch with you soon.

A Simplilearn representative will get back to you in one business day.

First Name*
Last Name*
Work Email*
Phone Number*
Job Title*