Bronto Summit 2017: Let Your Data Be Your Guide

2017 marked my fifth presentation at Bronto Summit, and this year, I had all kinds of interesting new data to share.  Hopefully, I'm able to challenge some assumptions, and help you get a clearer vision of your own shoppers!

Full Transcript:

Roy Steves:

[00:00:30]

Thanks for coming, everybody. This is actually my fifth year presenting at Bronto Summit, and this year is going to be a little bit different. I need to sort of give you some background context so that you can understand what I mean by that. If you run across anything in this presentation that you can prove me wrong on, I'll buy you a drink, so that's the addition to the normal Oracle legalese. The presentation was difficult to title. I appreciate you taking a risk on it, but I'm a little bit all over the map with what I was putting together, so I think you'll see why that was a challenge. Back story, in 2010 I started working with Pool Supply World. I was an engineer. They were doing quite a bit of business but it was mostly organic and comparison shopping engines, and so I told them, I'm like, "I'm pretty sure AdWords is just applied algebra. Why don't you give me a shot at that?"

[00:01:00]

[00:01:30]

We added 30% to the business in the first year, and then I'd teach myself another marketing channel, find somebody better at it than me, which usually wasn't very hard, and then hire them and add them to the team, so over the course of the first three years I went from an engineer to CMO. That was quite the roller coaster. Then in 2013 was my first Bronto presentation, and I did a talk on multi-touch attribution because I was collecting all of this browsing behavior directly off of my site into a database and I was showing off what I could do with that. 2014, I did data visualization and storytelling, so taking all this data that was in my warehouse and trying to make it actionable and useful, something that the rest of my team could understand and use. Then the next year was after we had been acquired by Leslie's, so in 2015 I had access to more data than God.

[00:02:00]

[00:02:30]

I mean, it was just ridiculous. I had billions of rows of data in a bunch of different slices, and so I presented on some segmentation strategies that I'd been using with that larger brand. For those of you that don't live in the 35 states they're in, they have over 900 stores, and do just a colossal amount of business, the only national brick and mortar chain in pool supplies, so I had huge amount of data to play with, which was great. Then, I left that in September of 2015 to start my own thing. Then I was thinking last year, presentation-wise, I'm no longer a retailer. What can I tell retailers that they're going to find interesting? What I did was I took basically the greatest hits of the first two chapters of my career, and did a presentation on A/B testing and site optimization, so some of you I know were there, and I appreciate that.

[00:03:00]

The big difference was that I had an incredible depth of data, so obviously I had platform data like Google Analytics and Pronto and AdWords and everything like that, but then I also had a ton of data layers ... Oh, yes, SkyNet was our order management system, so it had a ton of data. Grinder was the session tracker and things like that, so I had just an absolutely colossal amount of data to work with. The depth of data was unsurpassed. Now I'm out here on my own doing something else and I don't have that depth of data, but now what I have is access to an incredible breadth of data. The things I'll be sharing from here are an aggregate from a set of sites that total almost a billion dollars in annual revenue. There are people with more data than that, but it's enough that I have quite a bit of fun, and we can find some interesting things out from sifting through it, so what I traded for depth, I now make up in breadth.

[00:03:30]

[00:04:00]

What we're going to do is we're going to take a look at some of these types of things. When I talk about best practices and industry standard kind of common knowledge type things, I'm talking about stuff like this. StatCounter's not a bad source of information, but there's a lot of information that you don't have in terms of how you need to be able to interpret this. From a statistics standpoint, you don't know what the sample size is, you don't know what the error bars should be, you don't want the standard deviation is. There's just a lot of missing pieces, and that ends up being sort of like just sort of feeling around the outside of the data. I mean, it's analytical phrenology basically. You're not really getting down into the guts of it. In order to understand the data, we're going to have to peel it back, so instead of phrenology we're going to go with X-rays and MRI style.

[00:04:30]

[00:05:00]

We're going to look at aggregate data kind of like that StatCounter one, but then we're going to peel it back and I'm going to show you how different anonymized sites make up those totals, so you can get an idea of whether or not you should take anything you see on the Internet seriously when it comes to e-commerce. We're going to be looking at a bunch of dimensions. These are just some examples. Sometimes we'll use more than one, just so you can get a flavor for what we're getting into. The first thing I'm going to be doing, and most of this is from Google Analytics with a little bit of flavor text pulled in from AdWords data as well, but we're going to start with the channels and we're going to focus on three channels that I selected both for their size, the audience, and then how much influence you can have over those. We'll start with organic PPC and email, obviously, and this is in Google Analytics under conversions, multi-channel funnels, and overviews.

[00:05:30]

This Venn Diagram does a pretty good job of giving you a rough idea of the overlap of your channels. Overlap is a different conversation, but we're just looking at the relevant sizes. The reason that I wanted to do that was when you are a retailer you're in your own echo chamber, and that's all you hear for most of the year is how you're doing, and you don't really know what is normal. You don't know if you're weak or strong in any given channel. That's part of the reason we all love events like this, is because it's one of the few opportunities where we get to compare notes with people. Well, imagine this as being able to compare notes to a couple of dozen people all at once. We're going to start with typical. This is fairly normal. This is a relatively large site, it's fairly mature, and it has organic is the largest, followed by paid, followed by email. I'm not saying that this is necessarily virtuous. I'm just saying it's normal.

[00:06:00]

There are other variations on this. This one is also very typical. This one is actually a little bit different. You'll notice email and paid are the same size. Paid is a little bit smaller relative to organic, so they might be under-investing in paid relative to the size of the organic audience, but that might be a function of operating on a narrow margin, for example where the organic ends up being more critical to them because they can't afford to participate in PPC in the same way, and then email relative to paid is doing pretty well, and relative to organic is fairly typical. You'll see this pattern again and again, so these are three examples of fairly normal looking channel contributions. What does it look like when it's not normal?

[00:06:30]

[00:07:00]

This is the first example. It looks similar to the ones in the previous slide with one notable difference. Paid and organic have switched spots. This is actually a bit more expensive. This happens sometimes when a site does a re-platforming or something like that, or has been penalized by Google and is otherwise trying to fill in a gap using paid. It's a perfectly reasonable strategy but it does take some monitoring of your overall profitability to make it work. That's a little bit concerning even for somebody who's in the paid search space now. That makes me a little bit uncomfortable. Then there's this one. Not only is paid much smaller than organic, which is a problem, email is practically non-existent, and anyone here can recognize how that doesn't quite make sense. In this case, this is very likely a one-person shop. They're responsible for all of the channels and they've resorted to just batch and blast emails out of necessity, right? Some of us have been there before.

[00:07:30]

[00:08:00]

That's something that would be at least an opportunity in email, if not an opportunity in PPC as well, but that's not even the worst we've seen. This is pretty ugly. Now, there's two possibilities here, and these are ... I made them smaller. They're smaller sites, so not to scale, but you get an idea. This is one of two possibly catastrophic situations. Either they're not doing email at all, which is disastrously bad, or they're not tracking email, which is also disastrously bad. Nothing about that makes me happy with how they're doing on email. It's not the only one. I've seen that again and again, and then this is actually the worst one yet. This one makes me cringe down to my bones because not only are they not doing or tracking email, then paid is also larger than organic. That is a very expensive way to scale a business.

[00:08:30]

[00:09:00]

If we line up all of the different channel types with direct on the far left for those at the back, all the way down to social on the right, this is the kind of data that I could have generated if I were trying to do the same kind of sort of white paper nonsense that you see out on the Internet everywhere. It's very easy to produce these kinds of charts, but they're not that insightful. The reason they're not that insightful is if we then kick on the X-ray machine, we see that there's a huge amount of variety of distribution between the sites. Each of these sub-bars is then a different site that's adding up to that total, and as you can see the blue, the largest site at the bottom, makes up almost all of that display. If you had been on this slide and you saw that your display channel is smaller relatively to this chart, then you'd overestimate the value of that channel if you weren't that blue site. I'm going to refactor this data a little bit. We're going to show you another way of visualizing the same thing.

[00:09:30]

[00:10:00]

Now, each of the channels are broken into three types of sites: brands, which sell something with their own name on the label, so they are manufacturing and direct to consumer sales. There are retailers who sell things manufactured by other people, and then there's a few sites in my sample that do enough of both that they defied that categorization, but each of these where you add up the three columns responsible for those site types would be 100%, so that gives you a relative revenue share for each of those channels by that site type. The reason I was doing that is I wanted to validate some assumptions about how brands versus retailers work in this space, and so just so you know how you're reading this channel, remember when I pointed out display is almost all blue? Well, that blue site over there is a brand site over here, and you can see that it then dominates the relative charts, so that's how you interpret this.

[00:10:30]

The first thing I wanted to look at was the brands. I thought that brands would have a more dominant direct than organic because if you know Levis for example, then you seem to me to be more likely to type in Levis in Google or go to levis.com, but if you look at it, the retailers are actually holding their own on direct and actually have a little bit of a lead on organic, so I was completely wrong. Then that actually ends up flowing into the paid section as well, because I assumed that retailers were going to be trying to make up for that brand identification gap by investing more in PPC. I'm wrong. The retailers have a lower average share than the direct consumer brands in my sample. That was interesting to me. The next thing I looked at was email, and this did follow that a pattern a little bit.

[00:11:00]

[00:11:30]

If you know the brand that's selling you the product directly and it's a brand that that name is on the item that you have, that physical object, then I figured you're going to have more loyalty because you're going to be more familiar with the brand of the manufacturer than the brand of whoever sold it to you. If it happens to be the same brand, then you're naturally more likely to repeat purchase from them. Retailers on the other hand, don't have that advantage, so when I was with Pool Supply World, we'd send you a pump. It said Hayward on it. It didn't say Pool Supply World on it, and we would have to make up that gap in customer loyalty through the email program. You can see that we weren't the only ones using that pattern by how the retail section was dominant in email. Then the other three are much, much smaller samples, and pretty noisy, although it's interesting with the amount of bluster in the social space, that nobody in this sample was really generating a ton of revenue off of that.

[00:12:00]

[00:12:30]

The summary is basically if your Venn Diagram when you go into Google Analytics, doesn't look vaguely like this, either you're handling a special use case and that might be, or it might mean you have an opportunity. I'll be sharing all the slides to you so you can use these benchmarks however you like, and then hit me up with any questions thereafter, but if it doesn't look like this maybe it should. Then Google Analytics obviously gives us a ton of data on device types and there's this fetish for mobile first, mobile optimization, mobile site speed, and all those kind of factors, and it has been around since 2008, and it's not wrong, but some of the ways that we've been brainwashed into thinking about mobile didn't quite line up, so I'll use my own assumptions as examples in this. We're going back to the brands versus retailers first, and my assumption was that brands would have a stronger performance on mobile because that trust is there.

[00:13:00]

The product is the brand, then there's not a barrier to get over to say, "I need a Hayward pump, but I don't know about this Pool Supply World company." If it's a Hayward pump and you're buying it from hayward.com, then that trust is already sort of implicit if you're already going to buy the product. To sort of go through the rest of these slides, I do want to point out that I'll be using this metric quite a lot, value procession. The reason I'm using this is it takes into account both average order value and conversion rate, and both of those will vary depending on which segment we're looking at, so this ... You may not sit in front of this every day, but it's a very powerful metric and in the AdWords space, obviously value per click is sort of an analog somewhat. This is where we'll start and then I'll break out differences in AOV and conversion rate where appropriate.

[00:13:30]

[00:14:00]

I'll be doing a lot of scatter plots like this. What I'm plotting here is mobile value procession on the vertical axis and desktop value per session on the horizontal axis, and then I've got brand and retailers and both split out once again. The size of the dot is roughly descriptive of the size of the site, so you can get an idea of which ones are big, which ones are small, and for most of these ... The small ones are also younger sites. They're not one million dollar a year that had been one million dollar a year for 10 years, at least in my sample, so those might behave differently than this group, but you can see there's not really a correlation. It's just sort of all over the map, which I didn't really get. What I did was I broke this down into conversion rate. If I can't make sense of the conglomerate, then I'm going to go down to the atomic level and this is what I saw.

[00:14:30]

[00:15:00]

This obviously does tend to have some sort of pattern to it. You can see it goes along this diagonal, which is three to one, so a lot of these sites down here in the lower corner are converting on mobile at about a third to about a half of whatever they are on desktop, so this is sort of where I'm getting these metrics is saying that if your mobile conversion rate is a tenth that of desktop, then you're out of the band of normal behavior and that might be a problem, but there are some exceptions. This guy way up here, that mobile conversion rate, and this is over 12 months of data, is higher than the desktop conversion rate. It's not a huge site, but that is a very unusual behavior indeed. Then obviously these two down here have a very rudimentary mobile experience and a miserable mobile checkout, and you can see that reflected there.

[00:15:30]

[00:16:00]

This is another way to visualize this. I broke this one up because I wanted to get a look at tablets, but obviously three dimensions in a two-dimensional graph are kind of tricky, so I couldn't quite do it in that last graph, but what I've done here is I've just flat-out plotted revenue versus sessions. The reason I've done that is that depending on where you're getting your data, tablets will get lumped in with desktop sometimes because they're a large form factor screen. The browsing behavior is very similar to desktop in a lot of ways, but they also get lumped in with mobile a lot because it's a touch screen and the conversion rates are lower than desktop. Why is it that there's no consistency in our industry about what to do with this tablet segment? It's smaller than the other two, sure, but it seems to vary day by day whether it's treated as a mobile device or a desktop equivalent, and that's because those yellow dots literally are just sort of mashed in between. The reason that it's not consistent is because it ends up being muddled right in there.

[00:16:30]

[00:17:00]

It's a complicated topic, so we're going to dive into some more tablet stuff, but here's an example of what I'm talking about. This is a mobile retail commerce sales, percentage of retail, et cetera, and this is not telling us whether tablets are included, so with or without that segment that's relatively high converting compared to mobile phones, I can't really use this data. This is that phrenology problem all over again. Also, while it's difficult to read at this size, 2016 through 2020, all our asterisks, I think this is a single year's worth of data in 2015 that they just extrapolated with a suspiciously linear curve, so there's a few reasons I don't really buy this chart. What we can do is find other examples like that. I already showed you this once. This one at least did us the favor of telling us that it lumped mobile and tablet together. Now, if you came across this as I did, and were looking for this to inform how important your mobile site optimization projects might be, you'd be forgiven for thinking that it's about as important as your desktop optimization projects.

[00:17:30]

However, there's a couple of reasons that that would be an incorrect interpretation of this chart. One, this is global data, so outside of the US market there is a higher preponderance of using mobile as a primary device, so that exaggerates the effect if you're a domestic company. If you're global, that's fine. The other thing is I don't know where traffic is on your PNL, but it's not on mine, so you can't buy a beer with traffic. Therefore it doesn't actually count for anything, and this is guilty of that. It's looking at traffic only. The other thing that we're going to do is this kind of breakdown, and this is also from StatCounter. The odd thing is that it doesn't add up to the same thing as what the other chart was showing us. There's a difference in their methodology and they don't really tell you what the difference is.

[00:18:00]

[00:18:30]

This is once again global. It's still traffic share, but once again, mobile looks like it's about as important as desktop and then tablet is almost nothing. Now, global data, just like I said, it emphasizes mobile, it deemphasizes tablet, and that's why you see such a small slice here. If I take my data and try and reproduce this chart, what does it look like? It actually looks pretty similar. This is still on traffic, but you can see the tablets are obviously a much larger slice and then the relative performance of desktop and mobile is still there, but this is still the phrenology version. Let's kick on the X-ray. I've put each of the contributing sites that fit this type of research in here as a horizontal bar, but if you just sort of blur your eyes a little bit, you can see that that's about the same kind of average spread, but there's a huge amount of variety within there. In fact, I'd like to highlight a few of these.

[00:19:00]

[00:19:30]

The blue line and then the purple line above it as well, to a different degree, are fairly typical on desktop, but they have a relatively small mobile share of traffic, and a relatively high tablet share of traffic. That's an interesting thing. We'll dive into some possible explanations for that, but the first thing I want to do is once again point out that these ... The larger sites are on the top and the smaller sites are on the bottom. These bottom three are all kind of interesting. The orange one is almost exclusively a mobile site. I almost want to go back and look at that account, and see what are they doing inbound wise that's leading to such a preponderance of mobile users on that site, and then almost no tablet users. There's something almost suspicious about that one, and then the other two, despite being even smaller still, have something that's more typical in terms of performance, so even the smallest sites can have a fairly normal distribution and then sometimes you get these weird outliers.

[00:20:00]

Once again, traffic isn't revenue, so when we kick it over to revenue, all of a sudden mobile doesn't seem nearly as dominant in terms of where we should be putting our attention. The interesting thing is still that that orange one continues to have a huge amount of revenue, but that might not be because they're exceptionally good at mobile. It's just because they don't have that much other traffic to measure. There's different ways to read that same data there, and we're looking at the smallest sites definitely have the most mobile revenue and the most mobile traffic, so that's interesting that in the middle section over here, we have more share to the mobile side. My hypothesis is that because those are smaller sites, and they're newer sites, they're probably running templates that were responsive from the get-go, but there's an alternative interpretation.

[00:20:30]

[00:21:00]

Larger sites with more resources might have a more developed desktop experience, so even that conclusion might not be sound because the majority of the money is still coming from desktop, so even though you see all of these trends that say mobile first, mobile first, mobile first, yes, mobile's growing. Mobile's critical, especially multi-device shoppers and such, but right now don't forget that desktop is still paying the bills, so as you're working on your responsive templates, be sure to spend a good chunk of your time on the full-size view and not just myopically focus on the mobile version. I'm not saying don't focus on mobile, just remember where the money is coming from that keeps the lights on. Another kind of behavior that I've seen across a lot of site owners is an excessive level of attention paid to new to file customers.

[00:21:30]

[00:22:00]

A lot of sites are overly optimistic about their retention rates, and so they think that they need to spend more money where there are more new customers versus returning because in their minds that customer's going to come back anyway. They're more likely to come back, but by no means guaranteed to, so I was always sort of skeptical of this. The other thing that happens is using the new versus returning stats in Google Analytics, as if you were comparing new shoppers to people who have purchased before, and that is actually wildly incorrect. What it's actually telling you is whether or not that person bought in their first session, if you're looking at the new customer segment in Google Analytics, or whether they are a multi-session shopper. That could be multiple ... You know, just two days back to back. They might have placed two orders. They might have placed one order. They might have not placed any orders, so just be sure that when you're looking at this report, you're not thinking in terms of repeat customers. These are repeat visitors.

[00:22:30]

[00:23:00]

But there still is a difference in the behavior of those two segments, so this is fairly typical. It actually is a little bit more biased toward returning than the sites that I personally managed, but most of the time I see a lot of this 50/50 split. Once again, just like we saw with the mobile versus desktop, one of these has a much larger revenue contribution and that's either higher average order value or higher conversion rate coming from those multi-session shoppers. What this tells me is that we as a group are probably not spending enough time thinking about multi-touch attribution, but the bottom line is don't think that the new customers are ultimately more important than the returning customers, just because they're new to file. Realistically, most of those returning customers could just as easily have gone to Amazon this time around even if they've been to your site before. Yes, you need new customers, but the money that's keeping the lights on is mostly coming from multi-session shoppers and returning customers.

[00:23:30]

[00:24:00]

Even that 50/50 split is not really a great rule of thumb. Here are all the sites that I had split up, and you can see, and this is obviously I didn't sort it from largest to smallest because it was way too hard to read, but you can see that there's a lot of variety in there. If you were one of the top several sites or one of the bottom several sites, you'd be forgiven for looking at that 50/50 split and thinking, "Well, that's what I should have too." Well, not necessarily. The other thing is that while I didn't sort this by site size, there was a correlation between sites' age and this, and that's because older sites simply have more returning customers to possibly draw in. Not only that, smaller sites are adding fewer new members to their returning customer segment per day, so there's a couple of factors where larger, older sites end up biased more toward returning than the smaller sites, which tend to bias far toward new.

[00:24:30]

[00:25:00]

The way you can think about it is if you spin up a Shopify Plus site tomorrow, all of your customers will be new visitors, right? This is the same kind of pattern, just taken out over years instead of minutes, but once again, what about revenue? Now, this is far noisier, so even though the bottom ... This is, you know, if it's bottom here, it's bottom here, so these are the same order of sites. If you're the smaller sites, even though you have a ton of new shoppers, most of the revenue's still coming from multi-session and returning shoppers, so that ends up being sort of thrown back in the face of that assumption that new to file is somehow more valuable out of the gates. Now, if you want to do some LTV calculations, that's a different story, but just using the Google Analytics data like this, you can see that maybe that's not the best approach. Maybe there should be some sort of hybrid approach that takes into account the fact that those multi-session and returning shoppers are more likely to convert.

[00:25:30]

[00:26:00]

Like I said, this is tied to the value of multi-touch attribution because they came from somewhere if they're a multi-session shopper, so if they came to my site yesterday and then they came to my site today, I should know where they came from yesterday if I want to get a full picture of the story. There are other ways to approach this data. Here I've plotted the percentage of new visitors of revenue, so if all of your revenue came from new visitors, it would be very high on the chart, versus the percentage of sessions. If you have a lot of new sessions coming to your site, single session shoppers, then it would be far to the right. The brand versus retail split is a little bit interesting here. The big brands tend to be clustered down into one corner, but the smaller dots up and out to the side show that that preponderance of more new traffic, but they also have a higher share of revenue coming from those new, relative to their numbers.

[00:26:30]

[00:27:00]

What I found interesting about this actually is those big dots, especially the gold dot and the big blue dots down in the bottom, you'll notice that the larger, more established sites do have a smaller percentage of new sessions in traffic. That's because they have a lot of returning sessions. They were the ones at the top of that session stack slide, but you'll notice they're not much farther down on the chart, so they're not getting a larger share of their revenue necessarily coming from that larger percentage of returning customers. Oh, I had two of the same slide. No problem. All right, so here's another way to look at this. I'm now plotting returning value procession versus new value procession, and we can see a pattern here. Everything is pretty much above this one to two line, so returning shoppers tend to be worth about twice as much compared to new shoppers.

[00:27:30]

If you look at the big dot in the middle, that's four dollars per session of a returning visitor, versus two dollars per session of that new visitor, so remember when I was showing you the pie chart with new versus returning on revenue? This is why, and this is the distribution of that. Here's another approach. Now we're just looking at the conversion rate itself, and this also tends to have an interesting trend here. We're looking at a two to one here as well, so we know that that two to one pattern is producing that pattern on the value procession, but I want to highlight one of these sites specifically, and that's this one. While the value procession is pretty much on that two to one trend, it's under the average trend on the conversion rate. Let me say that again. The value procession is on trend, but it has a lower than average conversion rate on returning customers.

[00:28:00]

[00:28:30]

If that's the case, if the conversion rate is lower but the value procession is the same, then the AOV must be higher to compensate for that, and if we swap out this other chart, we can see that that's the case. Here is returning average order value versus new average order value, and the new average order value is $150 for that dot, but the returning average order value is only $100. Now this was very interesting to me, and maybe this was just me and my company, we thought that if somebody was going to buy a $1500 pool pump, that maybe they'll buy a $50 bucket of chlorine first to get a sense of whether or not they could trust the site, and then they'll come back and place the large order. We don't see any behavior that suggests that that's the case, because if it would be then you'd see a much higher average order value on the chart on the left relative to the first purchase. The first purchase generally ends up being equal to or higher than the second purchases.

[00:29:00]

[00:29:30]

Bottom line here is yes, new to file customers are important, but multi-session and returning customers are producing two to three times as much revenue per shopper and per session. Once again, think about multi-touch attribution, do a little bit of research if you're not familiar with it. Google Analytics makes it easy to sort of get a sense of it without having to be as technical as you needed to be five years or so ago. That's the new versus returning. The next rant is demographics. This is the most abused and misused thing, type of segmentation I've seen in our entire industry. It's effectively, in my opinion, stereotypes gone rampant, and it is ... I couldn't believe that the data would support it. Now let's see if I'm right. There's a couple of different stereotypes I'm going to go into and attempt to blow up. We'll see what my hit rate is.

[00:30:00]

[00:30:30]

This is the first one. We were definitely guilty of this. Millennials are the ones on their phones more than any, Gen X tend to be on their laptops and desktops usually shopping from work, and then the Boomer segment tends to be using tablets for a variety of reasons that were mostly driven by stereotypes. This is something we can measure. We have enough data. We can use the demographics data and analytics compared to the device types, and see if there's anything to this. Here's traffic, and there's definitely a difference between the three of them. The mobile and desktop have the same scale and then obviously tablet is a smaller sample, so I used a different Y scale there, but relatively speaking, tablet does skew to 55-plus. Desktop, it goes down to the 25-plus, but it's definitely in that sort of 25 to 45-plus range, and then mobile has the largest percentage of traffic coming from 25 to 45.

[00:31:00]

[00:31:30]

The mobile segment ends up being a little broader than we expected, but this is an interesting pattern. What happens when we switch from traffic once again to revenue? It exaggerates, and that's because the oldest segment and the youngest segment potentially are producing the least revenue per share per device type, so the possibility here is this might not be a behavioral pattern so much as an economic one. It might not be because they're old or young that they're producing a difference in the revenue per device type. It might just be an economic one. If we then go back to the sites, so we start to see the variety here. This is ... Each of the colors represents one of my age segments, and then I have younger to the far left and older to the far right, and then now I have switched back to having the sites stacked from largest site to smallest site.

[00:32:00]

[00:32:30]

This is noisy. There's not really much of a pattern here. There are a few sites in here that are particularly interesting. The top one of these two has 75-plus percent of its revenue coming from users 55 and older. That's unusual, and it's not a small site. At the bottom, which is smaller by traffic but not necessarily by revenue, has almost 75% of their revenue coming from ... What's that? 35 and under. If either of these two sites saw general, broad spectrum, here's national averages, their site wouldn't fit that model at all. They wouldn't be able to use that data and if they did, they'd be led astray by it, but if we switch from revenue to value procession, then the pattern starts to normalize pretty nicely. Even if you have a random set of people in terms of ages coming to your site, most of them are producing about the same value procession.

[00:33:00]

[00:33:30]

There's a difference between quality and quantity in your shoppers, and the standard assumption is that if 75% of my shoppers are from this demographic, then I should focus on that, but it might be the case that all of the shoppers are roughly the same value. I wanted to point out one of these. This one does have a higher value targeting the 25 to 35 range, so they're the exception to my rule, but as you can see, this is way more predictable than it was with the by revenue, obviously. The VPS is more consistent. The next one is ... That was true by volume but not by quality, so I'll give myself half credit for that one. This one's been a part of online shopping since Amazon first came on board, gender stereotypes, and I intentionally chose the most deplorably stereotypical female shopper image. I mean, she's got a couch. This poor woman's laying on the ground with a credit card with no numbers. She's obviously disturbed and fictional.

[00:34:00]

My assertion is that this person doesn't exist, and that we all need to stop thinking in terms of this shopper that's been crammed down our throats for the past decade. Am I right? Well, here's the revenue share by gender. I'm going to skip traffic entirely and just go to the brass tacks here, and as you can see, there is a wide variety of gender distribution on this, so if I'm selling product X and I'm assuming that my shoppers are all female, not only am I potentially wildly incorrect, what am I going to do with that? Am I going to be one of those schmucks that just makes it pink and calls it targeted for females? It's just ... It drives me crazy. The next thing we do is check out the value procession, and this is very interesting for a couple of reasons. One, the value procession is not only fairly evenly distributed with that one exception of the site at the bottom, second from the bottom, the correlation is actually inverse.

[00:34:30]

[00:35:00]

What you can learn from this is if for example I am on a skin care site, I might be the smaller demographic on it, but I might be really into skin care, and if a woman is on a after-market Harley parts site, she might convert more highly because she's really into after-market Harley parts. Once again, quality and quantity are not the same, and I hear too many instances where people try to use one to explain the other, and it's just not the case. Differences in volume are not the same as differences in value. Then I gave you some examples that I see all the time in AdWords where people are like, "Well, there's more business during the hours of 10:00 to 2:00, so I'm going to increase my bids." That's not reasonable. It's not what that tool is for. If this all sounds a little vague, here's an analogy. I'd rather have one good craft beer than an entire case of Natty Ice, and that's what we're talking about here. The volume and the quality are completely different variables.

[00:35:30]

[00:36:00]

The next thing I'm going to go after is page speed. Especially for the past three to five years, this has become dogmatic, like you have to work on making your site faster. It seems more reasonable. This one, I was going, "This is probably correct," right? Google's even using it as a ranking factor. That alone means that you should probably put some weight into it, but let's see how it correlates with other metrics. You'll see this summarized in a bunch of blog articles and infographics and things, and I love Kissmetrics blog. They're excellent. They're one of the best out there in my opinion, but they're guilty of this kind of nonsense, where what falls in the graph of what users thought. What people say and what people do are not at all connected. If you look at any of the research done on the correlation between survey responses and actual behavior, this is not going to tell you anything useful. Love Kissmetrics though I do, that's not really great.

[00:36:30]

[00:37:00]

Here's the average page load versus conversion rate for the sample of sites that I was working from. There is a correlation, in fact the R-squared is 0.41, which in very rough terms means that I can explain about 40% of the differences in conversion rate across the sites using page speed alone. That's a massively powerful correlation, so you've got the slow and low converting sites on the left, and a fast and very high converting site on the right. Page speed confirmed, does matter, but there's a sub-assumption here that people have been harping, especially in the past 12 months to two years maybe, that this is even more true for mobile. It's just not. It's just not. There's no correlation here. It only explains 7% of the variation in conversion rate when I narrow it just to the mobile segment. What's going on here might be that the expectation is lower. People know that the phone experience is slower, so they're more patient. I'm not saying they'll be patient tomorrow, but today they still are.

[00:37:30]

[00:38:00]

You should definitely work on your page speed. You should definitely work on mobile, because it is a growing segment, but just because you see it on the Internet doesn't make it true. That concludes my sort of survey of all of this data. It's sort of stream of consciousness because that's how I do analysis, but I find these little gems along the way. Check your relative channel contributions. If your Venn Diagram doesn't look vaguely like the standard ones I showed you, understand why. It might not be a bad thing, but you should think about it and understand why it's different. New visitors are important, but multi-session shoppers pay the bills. Regardless of age, device, or gender, quantity does not equate quality, so volume and value are not necessarily correlated, and mobile shoppers are more patient than you think, caveat, for now. Remember, when you're reading things, including on my beloved Kissmetrics blog, that there are three kinds of lies: lies, damned lies, and statistics. Thank you.