Watson wins. Jeopardy wins. IBM wins. Next up: the rest of us?

Posted on February 17, 2011 by jyarmis

If you know me, you’re not surprised to learn that I’ve been practically a life-long fan of Jeopardy.  That’s why when I got an invitation from IBM to watch last night’s final episode of the Watson vs. Ken Jennings vs. Brad Rutter match along with IBM’s Dr. David Ferrucci, the Principal Investigator behind Watson, I jumped at the opportunity.  I was not disappointed.  It was an exciting evening and the opportunity to chat with Ferrucci was memorable.

I’m always hesitant to write things in the afterglow of a moment like this because certainly emotion gets the better of me.  I can’t help but think, however, that we will look back on this moment as a defining moment.  In a brief conversation with eWeek‘s outstanding enterprise reporter, Darryl Taft, I talked about this accomplishment in lofty terms.  I likened it to President Kennedy’s establishing a landing on the moon by the end of the decade of the 60’s as a national priority.  I was about to write “this one isn’t quite that significant”…but maybe it is.

What did I mean by that comparison?  Well, setting a moon landing as the target accomplished two things:

  1. It set a target, unifying disparate research and development efforts towards a single common goal.
  2. It set a deadline, giving some urgency to what might otherwise have been “leisurely” scientific endeavors.

So too did winning Jeopardy create those two conditions for the team of IBM researchers who spent four years preparing for this moment.  Without Jeopardy, these advances may never have come together, at least not in this time frame.  For those who think this a frivolous activity, I’d note two things:

  1. Competing at Jeopardy is certainly a tremendous challenge for natural language processing.  If you can win at Jeopardy, many other commercial applications are feasible.
  2. IBM, sensitive to this charge, today announced an initiative whereby they’ll explore options to apply the Watson technology in the healthcare space.

So what do we make of this?

Certainly IBM gets a massive PR boost.  Winning as it did a decade ago with Deep Blue at a chess championship is interesting.  However, the last time Americans cared about chess was when Bobby Fischer was world champion.  (I was a young kid.  A long time ago.)  Jeopardy, however, is a cultural icon and also a great fit for IBM’s target audience.  It would have been one thing to win at Wheel of Fortune (easy challenge, easy competition), another thing to win at Jeopardy (harder challenge, Ken Jennings; UPDATE interview with Jennings here).

More important, perhaps, for IBM, this validates some decisions it made years ago.  Remember when they sold the PC division to Lenovo?  “How could you get out of the PC business, IBM?  It’s the future,” people cried.  Instead, IBM quietly doubled down on cloud computing and big data (although we didn’t call it those things back then).  Looking pretty prescient today, aren’t they?

Now, a lot of this stuff isn’t really commercially viable just yet.  IBM threw a lot of hardware at this problem.  10 racks of servers, 15 terabytes of RAM, 2,880 processor cores operating at 80 teraflops.  But you’ve got to love Moore’s Law.  This will be mainstream computing in 5-10 years.  It will be on your phone in 15.  Pretty exciting stuff.

A few other random observations after my conversation with Ferrucci:

  • Certainly Watson had some advantage in terms of access to information, but the humans had an incredible breadth as well.  One of Watson’s big advantages was not in the information but actually in the time to assess whether it should hazard a guess based on a significant statistical analysis involving competitive position, confidence of answer, how much time was left, etc.  Watson was just a better game player.  That’s why Ken Jennings was so visibly frustrated.  Watson made decisions faster.
  • Watson was a shrewd wagerer.  Based on their analysis of Jeopardy games, the IBM researchers concluded that most players didn’t wager enough on daily doubles.  Watson was an aggressive wagerer early in games when presented the opportunity.
  • Category names were for the most part not very helpful to Watson.  Tying those things back to answers was so nuanced as to be often not valuable.
  • As IBM added some information sources to Watson, results didn’t markedly improve and so they were removed to conserve capacity.  Wikipedia, despite widespread concern about its unreliability, was a highly regarded source, comparable to “traditional” encyclopedias.

Personally, it was an exciting evening.  I was actually rooting for the humans to win.  The fact that they did not, though, is hopefully exciting for our ability to apply computing power to important, valuable tasks that will benefit us all.  Maybe that will be the lasting legacy of the three nights:  it caused us to focus on new vistas for computing benefits and ask some interesting questions about how we might take that next quantum leap.

Filed under: Cloud Computing, Hardware | Tagged: , , , , , , , | 2 Comments »