This snippet from the Economist’s review of Brian Christian’s book about artificial intelligence – The Most Human Human: What Talking With Computers Teaches Us About What It Means To Be Alive – reminds me that web analytics is about people, not data:
“People produce timely answers, correctly if possible, whereas computers produce correct answers, quickly if possible. Chatbots are also extraordinarily tenacious: such a machine has nothing better to do and it never gets bored.”
This spring the nine hardy students who took my USC Annenberg web analytics class came up with wonderful insights that could never have come just from reading a report straight from Google Analytics, Omniture or any other chatbot. 
Part of their grade was based on whether the (equally hardy) participating news and nonprofit organizations were actually going to use their analyses for decision-making. This meant each student had to really understand the organization’s strategies, goals and personalities before he/she dug into the data. Here are some of the things we learned.
Content is indeed king, but only if it’s coded
None of the organizations coded site content with enough detail to make decisions about what to do with their sites. Data coming straight out of Google Analytics or Omniture was coded only by date published and by broad categories such as “News.” This is the equivalent of marking a box of books “MISC” – or putting in “stuff” in any search engine.
Larissa Puro
For example, let’s say an organization believes it can build and engage audiences by adding “more local politics and government coverage.” To track whether it did indeed produce “more,” and what coverage did result in increased visits and engagement, the org needs to track how many politics stories it has, by topic and local geographic area, and how much traffic each topic and/or area gets.
Rebecca Schoenkopf
Each student developed a taxonomy of codes the organization could use to classify its content, and then manually (I told you they were hardy!) coded sample data pulled out from chatbots, er, Google Analytics or Omniture.
Track traffic by day, week and by topic, to determine if the site is getting the traffic it should
Courteney Kay
Many organizations look at monthly data, and celebrate traffic spikes. Hidden in monthly data, however, are clues to where to build targeted audiences and advertisers. Health/fitness section traffic, for example, should increase the second week in January, perhaps fall off after Valentine’s Day (!), and increase before swimsuit season.
Kevin Grant
More content = more traffic
Looking at visits by day of week, we saw radical drops in visits on the weekends. This seemed to be due to little unique local content being posted on Saturdays or Sundays. In this age of the 24/7 newsroom and increased Internet access through mobile, can news orgs afford to make resource decisions based on non-audience-based, chicken-and-egg logic (“We don’t get much traffic on the weekends so we can’t justify adding weekend staff.”)?
Josh Podell
Sometimes you should have separate sites for each audience segment….
Josh Podell, an MBA student, focused on analyzing the e-commerce donation functions on the nonprofit sites. He observed that it’s hard to understand what works and what doesn’t when donors are coming to the site to find out more about the organization but residents are coming to find out about programs and services. Josh’s suggestion: Have a completely separate site – and Google Analytics account – for donors. An org could have much more focused content for each audience, and metrics such as visits per unique visitor, page views per visit and the percent of people who left the site after looking at just the home page (home page bounce rate) would give much more clear indicators for both sites.
Dan Lee
….but sometimes you shouldn’t.
One of the organizations had its main site on one Google Analytics account, and its blog on another. Dan Lee, a graduate Strategic Public Relations student, noticed extremely high home page bounce rates from returning visitors compared to new visitors.
With the question of why burning in his head, Dan looked in detail at the site content and structure, and hypothesized that returning visitors were most likely to go to the home page, see the teaser about the latest blog entry, and immediately “leave” the site to go to the blog. The Google Analytics account for the blog did indeed show that its top referring site was the main site.
Meredith Deane
Google Analytics was “correct,” but only a human could have produced the right answer for the organization.
Alexander Davies
So here’s to Brian Christian for explaining how to be “the most convincingly human human,” and how the competition for the Loebner prizes continues to result in what the Economist calls “a resounding victory for humanity.” And here’s to the most adventurous and tenacious students at USC Annenberg – the journalism, public relations and business majors who took an experimental elective class with “web analytics” in the title!
Todd Benshoof
But then Schmidt explained that everything Google does is “dependent on users opting in and finding value in the experience.” With so much information available on so many different devices (see my New York Times Room for Debate forum commentary on paying for device-based convenience), Google’s success is based on people making a conscious choice to use Google products to find and link them to what they want.
Video metrics for everyone!
YouTube‘s become a verb and a household name, but I’ll always see it as an organization that’s brought metrics into the lives of the common people (those who have broadband Internet, anyway). The “Most Popular” and “Featured Videos” are seen worldwide,
sometimes garnering millions of views. “Hey, did you see….” is usually accompanied by something like “…and it has x million views on YouTube!”
Number of views is great for little else other than bragging rights. It’s one of the “famous” metrics (web analytics guru Avinash Kaushik‘s term) that “are staring you in the face when you crack open any analytics tool” but “barely contain any insight.”
Yep, for anyone in the content business, number of views is right up there with hall of famers number of page views and monthly unique visitors.
YouTube has pushed all of its account holders – no matter how amateur – to use meaningful metrics. In March 2008 it launched Insight, its “video analytics tool for all users,”
along with some almost-preachy instructions on how to use metrics to get more people to watch your videos and, of course, come more often to YouTube.
The Insight tool allows you to track “community engagements” (there’s that word again) in terms of ratings, comments, and favorites. YouTube doesn’t want you to settle for people just watching your video. People have to show, in a measurable way, that they not only watched it but also reacted to it.
At the very least people should give a star rating (one is bad, five is good). Rating is easy, quick and anonymous. Tagging a video as a favorite is the next rung. And if they’re really engaged, they’ll leave comments.
But, as anyone who’s ever spent any time at all on YouTube knows, many comments are spam, obscene and irrelevant – just noise. But the value of social media metrics is in looking beyond what James Kobelius in Information Management points out is an “often low and laughable” signal-to-noise ratio.
Kobelius notes that “if you crawl, correlate, categorize, mine, and explore it with the
right tools….[this unstructured information] can yield unexpected insights….The intelligence value of any individual tweet [or comment] in isolation is
negligible….Intelligence emerges from the aggregate.”
If you can stomach a few obscenities, look at this thought in Encyclopaedia Dramatica about YouTube view fraud and how the ratio of VPC, or views per comment, “is the most accurate way to determine if anyone” cares. “A high VPC usually means view fraud has been committed.”
The example in ED shows that a video with 136,097 views and 3,529 comments has a VPC of 38.7, a low number that indicates this is a video “that people actually find funny.” The video with 296,413 views, 541 comments and thus a VPC of 547.9 is probably something nobody really cares about.
I calculated some VPCs from this week’s “Most Popular”
videos and came up with some numbers that I don’t know what to do with yet. To see if VPC can be used as a key performance indicator, I’ll need to calculate VPCs and crawl through the cacophony of a variety of news videos. VPC may never be “famous,” but it might be insightful.
Posted in Comments, Misused metrics, Ratings, Social media, YouTube | Leave a Comment »