File and Object Storage

 View Only

Watson: What is the Smartest Machine on Earth?

By Tony Pearson posted Wed February 09, 2011 12:00 PM

  

Originally posted by: TonyPearson


Tonight PBS plans to air Season 38, Episode 6 of NOVA, titled [Smartest Machine On Earth]. Here is an excerpt from the station listing:

"What's so special about human intelligence and will scientists ever build a computer that rivals the flexibility and power of a human brain? In "Artificial Intelligence," NOVA takes viewers inside an IBM lab where a crack team has been working for nearly three years to perfect a machine that can answer any question. The scientists hope their machine will be able to beat expert contestants in one of the USA's most challenging TV quiz shows -- Jeopardy, which has entertained viewers for over four decades. "Artificial Intelligence" presents the exclusive inside story of how the IBM team developed the world's smartest computer from scratch. Now they're racing to finish it for a special Jeopardy airdate in February 2011. They've built an exact replica of the studio at its research lab near New York and invited past champions to compete against the machine, a big black box code -- named Watson after IBM's founder, Thomas J. Watson. But will Watson be able to beat out its human competition?"

Craig Rhinehart offers [10 Things You Need to Know About the Technology Behind Watson].

An artist has come up with this clever [unofficial poster].

Dr. Jon Lenchner from IBM Research has a series of posts on [How Watson "sees", "hears", and "speaks"] and [Selected Nuances].

Like most supercomputers, Watson runs the Linux operating system. The system runs 2,880 cores (90 IBM Power 750 servers, four sockets each, eight cores per socket) to achieve 80 [TeraFlops]. TeraFlops is the unit of measure for supercomputers, representing a trillion floating point operations. By comparison, Hans Morvec, principal research scientist at the Robotics Institute of Carnegie Mellon University (CMU) estimates that the [human brain is about 100 TeraFlops]. So, in the three seconds that Watson gets to calculate its response, it would have processed 240 trillion operations.

Several readers of my blog have asked for details on the storage aspects of Watson. Basically, it is a modified version of IBM Scale-Out NAS [SONAS] that IBM offers commercially, but running Linux on POWER instead of Linux-x86. System p expansion drawers of SAS 15K RPM 450GB drives, 12 drives each, are dual-connected to two storage nodes, for a total of 21.6TB of raw disk capacity. The storage nodes use IBM's General Parallel File System (GPFS) to provide clustered NFS access to the rest of the system. Each Power 750 has minimal internal storage mostly to hold the Linux operating system and programs.

When Watson is booted up, the 15TB of total RAM are loaded up, and thereafter the DeepQA processing is all done from memory. According to IBM Research, "The actual size of the data (analyzed and indexed text, knowledge bases, etc.) used for candidate answer generation and evidence evaluation is under 1TB." For performance reasons, various subsets of the data are replicated in RAM on different functional groups of cluster nodes. The entire system is self-contained, Watson is NOT going to the internet searching for answers.

On ZDnet, Steven J. Vaughan-Nichols welcomes our new [Linux Penguin Jeopardy overlords]. I have to say I share his enthusiasm!

technorati tags: , , , , , , , , , , , , , , , , ,

5 comments
8 views

Permalink

Comments

Archive User
Tue February 15, 2011 05:20 PM

Originally posted by: TonyPearson


Bryan, There are three minor differences.

First, Watson uses POWER7 versus GA product uses x86 CPU processors. IBM's General Parallel File System (GPFS) runs on AIX, Linux on POWER, as well as Linux-x86, so functionally identical across the board. It was easier for the folks in Austin to put together an all-POWER7 Watson than mixing in a few x86 machines inside. This allows all the frames to be identical.

Second, GA version of SONAS has separate NFS interface nodes from the storage pods so that all NFS interface nodes can talk to all storage pods. A storage pod is two storage nodes connected to a common set of disk. However, for Watson there is only one storage pod, so it was simpler to just run the NFS interface software directly on the storage nodes themselves.

Third, GA version of SONAS uses high-density 60-drive 4U-high drawers, or connects to an XIV disk for capacity. The Watson uses 12-drive drawers, each server connected to two drawers for a total of 21.6 TB raw capacity. The 3.5-inch 450GB 15K RPM SAS drives are the same in both cases.

--- Tony

Archive User
Tue February 15, 2011 01:41 PM

Originally posted by: BSamson


Tony- As a storage seller, how close is the Watson SoNAS implementation to our GA product?

Archive User
Sat February 12, 2011 04:48 PM

Originally posted by: TonyPearson


The full set of moves for each game in the Deep Blue match against Garry Kasparov in 1997 can be found here:

http://www.research.ibm.com/deepblue/watch/html/c.shtml

---Tony

Archive User
Sat February 12, 2011 04:45 PM

Originally posted by: TonyPearson


Valerie, I don't think it is wrong to feel bad for the humans.

I remember back when IBM had DeepBlue compete against Garry Kasparov. In 1989, Garry won easily all games. In 1996, Garry had three wins, two draws and one loss, so he won again. In 1997, Garry had only 1 win, 3 draws and two losses, losing the match. Garry was notably upset.

Now, in this case, we have two champions of Jeopardy! They have already played practice rounds against WATSON, so they know what they are getting in to. Remember the first place gets $1 million, second place gets $300K and the third place gets $200K. The humans will be donating half their earnings to charity, and then party with the rest, so I won't feel bad for either of them. I suspect that they might be more worried about how each human plays against the other human. Brad beat Ken in a tournament before, so will Brad feel bad if Ken does better than him? They are both smart guys, but maybe their egos might take a hit whomever comes in third place.

-- Tony

Archive User
Thu February 10, 2011 11:23 AM

Originally posted by: vskinner


I watched it last night. Very interesting. Is it wrong to say I felt a little bit sorry for the humans playing against him? I also found the host interacting with the computer to be hilarious. More humor than I expected!