Watson wins. Jeopardy wins. IBM wins. Next up: the rest of us?

If you know me, you’re not surprised to learn that I’ve been practically a life-long fan of Jeopardy.  That’s why when I got an invitation from IBM to watch last night’s final episode of the Watson vs. Ken Jennings vs. Brad Rutter match along with IBM’s Dr. David Ferrucci, the Principal Investigator behind Watson, I jumped at the opportunity.  I was not disappointed.  It was an exciting evening and the opportunity to chat with Ferrucci was memorable.

I’m always hesitant to write things in the afterglow of a moment like this because certainly emotion gets the better of me.  I can’t help but think, however, that we will look back on this moment as a defining moment.  In a brief conversation with eWeek‘s outstanding enterprise reporter, Darryl Taft, I talked about this accomplishment in lofty terms.  I likened it to President Kennedy’s establishing a landing on the moon by the end of the decade of the 60’s as a national priority.  I was about to write “this one isn’t quite that significant”…but maybe it is.

What did I mean by that comparison?  Well, setting a moon landing as the target accomplished two things:

  1. It set a target, unifying disparate research and development efforts towards a single common goal.
  2. It set a deadline, giving some urgency to what might otherwise have been “leisurely” scientific endeavors.

So too did winning Jeopardy create those two conditions for the team of IBM researchers who spent four years preparing for this moment.  Without Jeopardy, these advances may never have come together, at least not in this time frame.  For those who think this a frivolous activity, I’d note two things:

  1. Competing at Jeopardy is certainly a tremendous challenge for natural language processing.  If you can win at Jeopardy, many other commercial applications are feasible.
  2. IBM, sensitive to this charge, today announced an initiative whereby they’ll explore options to apply the Watson technology in the healthcare space.

So what do we make of this?

Certainly IBM gets a massive PR boost.  Winning as it did a decade ago with Deep Blue at a chess championship is interesting.  However, the last time Americans cared about chess was when Bobby Fischer was world champion.  (I was a young kid.  A long time ago.)  Jeopardy, however, is a cultural icon and also a great fit for IBM’s target audience.  It would have been one thing to win at Wheel of Fortune (easy challenge, easy competition), another thing to win at Jeopardy (harder challenge, Ken Jennings; UPDATE interview with Jennings here).

More important, perhaps, for IBM, this validates some decisions it made years ago.  Remember when they sold the PC division to Lenovo?  “How could you get out of the PC business, IBM?  It’s the future,” people cried.  Instead, IBM quietly doubled down on cloud computing and big data (although we didn’t call it those things back then).  Looking pretty prescient today, aren’t they?

Now, a lot of this stuff isn’t really commercially viable just yet.  IBM threw a lot of hardware at this problem.  10 racks of servers, 15 terabytes of RAM, 2,880 processor cores operating at 80 teraflops.  But you’ve got to love Moore’s Law.  This will be mainstream computing in 5-10 years.  It will be on your phone in 15.  Pretty exciting stuff.

A few other random observations after my conversation with Ferrucci:

  • Certainly Watson had some advantage in terms of access to information, but the humans had an incredible breadth as well.  One of Watson’s big advantages was not in the information but actually in the time to assess whether it should hazard a guess based on a significant statistical analysis involving competitive position, confidence of answer, how much time was left, etc.  Watson was just a better game player.  That’s why Ken Jennings was so visibly frustrated.  Watson made decisions faster.
  • Watson was a shrewd wagerer.  Based on their analysis of Jeopardy games, the IBM researchers concluded that most players didn’t wager enough on daily doubles.  Watson was an aggressive wagerer early in games when presented the opportunity.
  • Category names were for the most part not very helpful to Watson.  Tying those things back to answers was so nuanced as to be often not valuable.
  • As IBM added some information sources to Watson, results didn’t markedly improve and so they were removed to conserve capacity.  Wikipedia, despite widespread concern about its unreliability, was a highly regarded source, comparable to “traditional” encyclopedias.

Personally, it was an exciting evening.  I was actually rooting for the humans to win.  The fact that they did not, though, is hopefully exciting for our ability to apply computing power to important, valuable tasks that will benefit us all.  Maybe that will be the lasting legacy of the three nights:  it caused us to focus on new vistas for computing benefits and ask some interesting questions about how we might take that next quantum leap.

Advertisements

2 Responses

  1. Jonathan hits on some of the key points – it;s the logic for when to “buzz in”, how to wager, etc that will help make the technology viable elsewhere.

    Having spent 10+ years in the semantic technology space, the biggest challenge remains in making solutions work across multiple domains. That’s why rather than “the Semantic Web”, what we’ve seen are many smaller semantic webs.
    Semantic engines need to be tuned for content, but also grammar and writing style. I guarantee that Watson, having been tuned to Jeopardy’s odd “start with the answer” approach, would fail miserably trying to respond to Quora questions.

    I’ve built semantic engines that work very well with news content, but can’t handle chat or tweets.

    The good news is that the technology today is leaps and bounds ahead of where it was when I started in the space in late 2000. And even then it was ahead of the mid-90s systems which touted artificial intelligence, but failed to deliver even basic entity extraction.

    So, I’m impressed with Watson. But at the same time, I’ve yet to see a sentiment engine that exceeds 80% precision or event detection that can shift from one domain to another.

    That tells us that domain-specific solutions will continue to evolve, but we are far away from a general semantic web.

  2. […] This post was mentioned on Twitter by Jonathan L. Yarmis and Adrienne Sabilia, Gabriel Gheorghiu. Gabriel Gheorghiu said: RT @jyarmis: watson wins jeopardy..maybe a big deal..my thoughts http://bit.ly/erExF1 @ajbowles @asabiliaAR @glfceo […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: