Decision-making with data

Wow – a web analytics story in the New York Times!  And one that probably didn't spark mass hysteria by saying that looking at web traffic data is the end of journalism as we know it.  

Jeremy Peters' September 5 story, "Some Newspapers, Tracking Readers Online, Shift Coverage," nicely framed the issues about using web analytics to make decisions about coverage.  At one end you have journalists who completely ignore audiences and report whatever they want.  On the other there are those who "pander to the most base reader interests." 

This was one of the few mainstream media stories I've seen that recognized the new role that audiences have in informing – not dictating – news decisions. 

Actually what was the most interesting to me was that Raju Narisetti, the Washington Post's managing editor of online operations, said that he used "reader metrics as a tool to help him better determine how to use online resources….the data has proved highly useful in today's world of shrinking newsroom budgets." 

Picture 1 In other words, Narisetti didn't just look at overall site traffic numbers like total visits or unique visitors and say "Oh, that's nice," or "Whatever! I've got a paper to put out."  He had to cut his budget, he saw that long-form videos had little traffic compared to other types of content, and then he decided to cut "a couple of people" from that department.

Narisetti made the decision – the data didn't.  He used the data.  In my rather narrowly focused web analytics world that is a really, really big deal.  Data just wants to be useful.  Seriously.  If it could talk, it would say that it doesn't want to be any part of a daily e-mail "about 120 people in The Post's newsroom" get that lays "out how the Web site performed in the closely watched metrics – 46 in all."

Forty-six! 

For data to be useful you have to ask it specific questions that you know it can answer.  A metrics report won't tell you what department to cut, and by how much.  However, it could tell you what types of stories get the most and least audiences, and which story types appear to have growing or declining audiences.  You can also get indicators – such as the percent of long-form videos people watched to the end, rated and/or commented on – of whether you're engaging audiences.

Then the hard part – the decision – is up to you.

 

Number wars

What does the war in Iraq have in common with the war news organizations are fighting for their survival?

Nothing at all, unequivocally.

BBC However, I was struck by Owen Bennett-Jones’ closing of his August 27 BBC story about the disparities in the reports of the numbers of civilians killed in Iraq since the war started in 2003:

“It remains true that people tend to cite a number that reflects not their view of the quality of the research but rather their view of the war.”

In other words, getting someone to use your numbers – your definition of success – is just as hard as getting them to change their thinking.

In the world of web analytics, people “tend to present the metric that’s most likely to work in their favor.  They’re tracking the wrong way or they don’t want to look at a particular set of data. They are wrongly using analytics as what [Robert Rose of Big Blue Moose] calls a Weapon of Mass Delusion….Worst of all, they are not learning to apply insight to action.”

— from “A Web Analytics Trap,” by Jim Ericson, Information Management, August 13, 2010

 

 

Page views: bad metric #3

Why do news organizations persist in using total page views as a measure of success?  Perhaps because if you're afraid of numbers then you're even more afraid of bad numbers, or numbers that tell you that your site isn't as successful as you want.

As with unique visitors and time on site, page views is a deeply flawed metric for understanding how a news organization is growing and retaining audiences.  

If the number of page views goes up, it could be a good thing.  Or, it could be bad.

If page views go down, it could be a bad thing.  Or – you guessed it – it could be good.

We would all like to think that a soaring number of page views means lots of people are eagerly pawing through our sites reading everything that's written.  However, how many times have you gone to a site and clicked on, say, 12 pages, fruitlessly looking for something? 

This is counted as:

1.    One unique visitor
2.    One visit
3.    12 page views

And one dissatisfied person who may not come back.

The page views metric rewards the bad design and navigation that many news sites have (sorry).  Most news sites persist in using section titles that are the same as their legacy media product (e.g., "Local News," "Life"), leaving audiences – if they're so inclined – to have to click numerous times before landing on a story about a particular city or activity like gardening.   

Or, a site breaks up a story into multiple pages, which can be annoying to a reader and reduces the possibility the reader will read the entire story and rate it, e-mail it or leave a comment.  What could be counted as one page view with a comment is counted as, say, five page views. 

If a site is redesigned and readers can find what they want with fewer clicks, total page views will – should – go down. 

And, dynamic content, or content that changes on the page without the reader clicking anything – isn't counted at all.  Streaming stories, videos, widgets, Flash applications, podcasts?  One page view.

To truly grow, a news org must understand every action its audiences are taking on its sites.  These are challenging times that require news sites to experiment and try many different things.  Not all things will work, which means sometimes the numbers will be bad.  But – you guessed it – that's a good thing.  We have to know if something's not working so we can fix it. 

 

“A trend is a friend”

Someday I’ll give up web analytics and move on to something real like pottery or something, but until then I’ll keep fighting the good fight to get news orgs to stop using monthly unique visitors as an indicator of success.

It’s tempting, I know, to count UVs because the number of monthly subscribers is the standard for print, and the number of people Nielsen says watched a program is the standard for TV.

But technology has made everything different.  Strategy is more important than ever, and understanding audiences, not just counting them, is essential.

UV overcount

UVs are counted by counting the number of cookies, or computers,
that go to a site.  This means UVs are always significantly overcounted
or undercounted.

If one person uses three computers, it’s counted as
three unique visitors.

UV undercount Conversely, a number of people going to one computer – for example, at a school or library – means that UVs will be undercounted.

Neil Mason, a web analytics guru who does some pretty thorough research for his clients, noted in a recent ClickZ column that UVs are usually overcounted.

(So that’s the reason why news orgs use total UVs – would they use this number if it were consistently undercounted?  Don’t think so…)

Mason notes that while the UV metric is “particularly important for those sites that are dependent on advertising revenues as a major source of
income,”  it “must always be treated with caution and never taken at face
value.”

Believe me, it’s rare to hear a web analytics expert use the word “never.”

Gingerbread man The advice from the ever-pragmatic Mason:  “A trend is a friend.”   Analyzing significant increases or decreases over time will give news orgs the information needed to build audiences.

(Trends are indeed friends, but don’t even think about using counts of Facebook friends and Twitter followers!)

 

Mobile muddle

How do you measure mobile?  It’s a mess, even for web analytics gurus like Judah Phillips at Monster.com, who said as much in an interview with IQWorkforce:

“The mobile space is interesting to me too, but it’s very much like traditional web analytics on smaller screens with some absolutely crazy data collection, sessionization, and visitorization challenges.”

Huh?  Let’s start at the beginning.

Just as millions of other people, I have a mobile phone.  However, I just discovered that I don’t have a “phone.”  I have a “device,” or just simply, “a mobile.”  That’s right – “mobile” is now a noun.

I guess the definition of “phone” is now limited to something on which you only make or take calls.  And, it turns out, even the lowest end mobiles – or lean mobiles – at least have texting capabilities.  (If you want to sound like a techie, use SMS, or short message service.  Sheesh.)

Samsung V1000 Up until last year I had a lean mobile with a camera.  I loved using the camera but didn’t send photos to anyone because it would have cost me $15, just like that.  I didn’t send any texts because each one made or received cost $0.20 each, which meant every time I got an unsolicited text from a company or an unknowing friend I was a little annoyed. It wasn’t the cost.  It was just the principle of it all.  I neither wanted nor needed these texts, and I had no control over receiving them.

I’m on a smartphone now, an iPhone 3GS, for no particular reason other than it sounded fun.  I’m paying $5/month extra to make or get 200 texts, and I’ve found texting pretty useful – so useful that if I find myself doing more than 200 texts/month I’ll probably pay the AT&T rip-off unlimited-text fee of $20/month.  I’ve also been doing everything else everyone else does – reading news stories, tweeting, updating my Facebook status, checking in for flights, buying things.

IPhone apps And I have no idea whether I’ve been using mobile apps or the mobile web.

It turns out “There are ‘three worlds’ in mobile: apps, mobile Web and SMS. In the
case of smartphone owners, they will use all three to varying degrees.”  (From Internet2Go.net, March 2009.)

You know what “three worlds” means.  Three different sets of metrics.

And, guess what? Apps are device-specific, which means there are different sources of metrics for iPhone apps (which, contrary to popular belief, hasn’t taken over the world), BlackBerry, Palm Pre, etc.

And….

Mobile web browsers (e.g., Safari on an iPhone) are also device-specific.

And….

All of those mobile usage numbers from comScore, Ground Truth (a mobile measurement firm) and the like only measure one mobile world, the world of mobile web browsers.  They don’t measure usage from apps. And how many people use mobile apps rather than the mobile web, especially for Facebook and Twitter?  I dunno – a lot?

The mobile usage numbers may all be flawed, but they all do point to mobile’s continuing rapid growth.  So, unfortunately, that means we’ve got to understand what new nouns like “sessionization” and “visitorization” mean.

 

 

 

 

Mobile app-mania

IPad app tag Is it just too mundane to talk about mobile application metrics when the iPad – touted as the savior of news media – has only been out a day?  Yeah, probably, but before I too get swept up in the frenzy of newness and coolness, I want to go on record noting that the total number of iPhone apps downloaded or purchased doesn’t tell you anything about engaging audiences.  Ditto for iPad apps.

App icon It must be true because I saw a story about how people don’t use smartphone apps in the New York Times.  No, not in David Pogue’s technology blog.  It was in the softy Sunday Styles section, right under a huge article about middle school boys using smelly body washes to get “masculinity in a spray can.”

The key takeaway (of the smartphone story, that is): Most people have more shoes than iPhone apps.  They download a bunch of apps and even pay for some of them, but they don’t actually use a lot of them.

App store “Since Apple launched its App Store in 2008, media, industry observers
and Apple customers have bent over backwards to heap encomium after
encomium on applications,” Mickey Alam Khan, editor-in-chief of Mobile Marketer and Mobile Commerce Daily, wrote last week.  “Yet, for
all the hoopla – 150,000 applications in the Apple App Store, 30,000 in
Google’s Android Market, 50,000 in GetJar – not enough questions are
asked of the efficacy of this mobile channel, either for content,
marketing, retail or entertainment.”

What questions should be asked?  The usual indicators of engagement – number of visits to an application per week or month, visits per unique user and the like.  Khan also wants to know “How many deletions each month and after how many visits?” and “Do users respond to images on applications the way they do on mobile sites?  Do they have the same patience for page load times, application versus mobile Web sites?”

Droid live Unfortunately, Khan notes that the data just isn’t there yet.  “Apple, Google and others of its ilk are asking retailers, publishers
and brands to spend tens of millions of dollars on products created
uniquely for their proprietary application stores.

“The least that
application store owners can do to reward this marketer devotion is to
offer data on an aggregate level….

“With each new mobile device’s launch, more hysteria is created around
the content possibilities.  The Apple iPad’s debut has launched
another round of application development, this time giving hope to
publishers worried over the future of the printed book….

Get jar logo “Applications as the glue that creates
stickiness to mobile device will be the name of the game.

“But is
this simply an Apps Bubble or another viable mobile channel that can
hold its own with the mobile Web? Only time and data will tell.”

The right data, that is.

 

 

 

 

An ounce of Prevention

I exercise regularly and don’t smoke, but I avoid reading Prevention.com. It’s just too annoying to be reminded of all of the other “smart ways to live well.”  When I see “by the editors of Prevention.com,” I have this picture in my mind of a bunch of really healthy people popping out cheerful stories like “5 Vitamins Your Bones Love” and “10 Reasons You’re Always Exhausted.”

Prevention Now I have another (annoying) vision, thanks to MinOnline‘s story about how well Prevention is using web analytics.  Of course a staff that is so pragmatic and probably always mentally alert would resist “going for the cheap link grab and traffic spike” – the junk food of web analytics.

I haven’t been sleeping well because I think too much (the top reason people don’t get enough sleep and are therefore exhausted), so I’ll just plop in these two paragraphs verbatim from Steve Smith’s MinOnline story, “At the Building Prevention.Com, Only The Abs Are Flat.”

Page views rpt Prevention stays “on its own brand message and [courts] the kinds of audiences that it and
its advertisers really want. ‘We got back to engaging with our customer
in the ways we knew they wanted us to engage with them,’ says [vp/digital Bill] Stump.
Fishing for any and all eyeballs and courting simple traffic spikes in
the search-driven universe doesn’t pay off in the end. ‘You get waves of
traffic, but the tide goes back out and what are you left with?’
Instead, by keeping to the needs of the ‘core customer’ in everything
that goes out to syndication or into the e-mail newsletters, prevention.com is courting the
people who tend to stay.

Page views per visit “Now, each big wave raises the sea level for all of prevention.com’s metrics, says
Stump. In the last two years, overall page views climbed 60%. In the
last year, the number of visits per user went up 12%. But it is the
engagement metrics of which Stump is proudest. ‘The number that warms my
heart,’ he says, ‘is page views per visitor that are up 49%.’ That
means the new visitors are sticking with the site and drilling much
deeper than they ever have before. ‘In general, advertisers want an
engaged audience. They want the metrics that show that people value your
brand and come to you for something that is unique. We own natural
health and fitness and beauty. We are the authentic voice.'”