From Tim Russert to Poppy Harlow

Tim Russert died last week of a heart attack.  He was 58.  You can read a great deal about him here.  There is more available here about the man, his work, and his life.

The “official” announcement.

I didn’t know Tim Russert.  Didn’t hang with him.  If you want to read those types of eulogies, please feel free.  The only thing I share in common (outside the big, white, male thing) is a passion for the news of the world.  Timmy did it much better than I have, but when you share a passion, you share an outlook, and an order to the world.  People who share a passion value things the same way, and Tim Russert is probably turning over in his just dug grave when he looks at the media he left for the rest of us.

Maybe it just the passing of one of the better ones that leaves the place so desolate.  As I look at across the media universe (and I do, far more often than is healthy), I see a void when I’m looking for someone relevant who tries to be impartial.  Someone who holds the feet to the fire, to quote Jon Stewart.  Jon Stewart…he’s probably the next best at it.

It’s that bad.

And not to pick on Poppy Harlow (daytime “24-news” is on par with daytime “TV”), but gimme a break.  Since when did news need a bass line and background music?  When did the commentators, speaking live, become more important than the speaker, speaking live?  When did the analysis become more important than the event?  When did the taste replace the essence?

In the year 2000, I think.    It’s been long enough now that we can look back and see the changes.  We can see when and how it happened.   You could also say 1996, and give Clinton some credit.  That’s when he axed the ownership rules.  They were rules, to be honest, weakened by Reagan in the 1980’s, when it was decided that even trying to be “fair” wasn’t needed any more.

That’s why Russert stood out, because even though he no longer had a legal need to be fair and attempt to appear to be impartial, he still thought it would be professional to do so.  And he pulled it off, for the most part.

But there is no doubt that he is one of the very youngest of a dying breed.  There are a few more out there, scattered on public television, cable, and maybe the web.  Aaaah, the web.  There’s a wildcard in news if there ever was one.  That’s why it was nice to have a Russert about to keep things grounded.

Now, with Mr. Russert himself becoming the ground, who is left to tell us where it is?

What say you, Poppy?

Every Military Needs Cannon Fodder

Analytics According to Captain Kirk

In my seminars, I enjoy teaching analytics because the fun is in finding effective and memorable methods to help people understand the concepts. One of my favorites is an analysis of the Red-Shirt Phenomenon in Star Trek.

What? You don’t know about the Red Shirt Phenomenon? Well, as any die-hard Trekkie knows, if you are wearing a red shirt and beam to the planet with Captain Kirk, you’re gonna die. That’s the common thinking, but I decided to put this to the test. After all, I hadn’t seen any definitive proof; it’s just what people said. (Remind you of your current web analytics strategy?) So, let’s set our phasers on ‘stun’ and see what we find…

The basic stats:
The Enterprise has a crew of 430 (startrek.com) in its five-year mission. (Now, I know that the show was only on the air for 3 years, but bear with me. 80 episodes were produced, which gives us the data to build from.) 59 crewmembers were killed during the mission, which comes out to 13.7% of the crew. So, that will be our overall conversion rate, 13.7%.

More charts are graphs are available on the other end of that link.  Lesson: It’s pays to be an officer.

How to Understand Better

Datawocky: More data usually beats better algorithms

More data usually beats better algorithmsI teach a class on Data Mining at Stanford. Students in my class are expected to do a project that does some non-trivial data mining. Many students opted to try their hand at the Netflix Challenge: to design a movie recommendations algorithm that does better than the one developed by Netflix.

Here’s how the competition works. Netflix has provided a large data set that tells you how nearly half a million people have rated about 18,000 movies. Based on these ratings, you are asked to predict the ratings of these users for movies in the set that they have not rated. The first team to beat the accuracy of Netflix’s proprietary algorithm by a certain margin wins a prize of $1 million!

Different student teams in my class adopted different approaches to the problem, using both published algorithms and novel ideas. Of these, the results from two of the teams illustrate a broader point. Team A came up with a very sophisticated algorithm using the Netflix data. Team B used a very simple algorithm, but they added in additional data beyond the Netflix set: information about movie genres from the Internet Movie Database (IMDB). Guess which team did better?

I think you can figure out that question if you read the subject heading…or the article itself.   This is why my experience as a robot, pirate, and ninja make my observations so damn prescient.