In retrospect, there was much more at stake than a mere $1 million when IBM’s Watson computer faced off against two Jeopardy! champions back in 2011. The bot’s victory gave Big Blue a shot at conjuring up a new line of business at the perfect possible moment. A series of advances in image and speech recognition was about to trigger a frenzy of investment and excitement about the money-making potential of artificial intelligence.
Six years later, it’s fair to ask whether that plan could have been better executed. IBM today is even more urgently in need of new business, with quarterly results released earlier this week showing that revenues have declined for 21 consecutive quarters. CEO Ginny Rommety has made a habit of talking about Watson as a kind of savior, and the company declared this week that this part of the business is growing. But IBM won’t release details on Watson’s financial performance, and skeptics abound. Last week, investment bank Jefferies released a report warning shareholders not to expect IBM’s investments in AI to repay themselves; Watson, it said, risks being eclipsed by competing AI platforms from Google, Amazon, and Microsoft.
Talking about Watson is a good way to trigger eye rolls from people in the machine learning and AI community. There’s widespread agreement that its triumph on the specific backwards-question problem of Jeopardy! was notable. Making sense of language remains one of the biggest challenges in artificial intelligence. But IBM quickly turned Watson into an umbrella brand promising a bewildering variety of bold new applications, from understanding the emotional tone of Tweets to scouring genomes for mutations. It bought startups and rebranded their wares as Watson and touted cute but hardly lucrative projects like Watson-designed recipes and dresses. In one TV commercial Watson chatted with Bob Dylan, confessing “I have never known love.”
Critics say IBM executives overshot badly by allowing marketing messages to suggest that Watson’s Jeopardy! breakthrough meant it could break through on just about anything else. “The original system was a terrific achievement there’s no question about that,” says Oren Etzioni, CEO of the Allen Institute for AI. “But they’ve really over-claimed what they can deliver in a big way; the only intelligent thing about Watson is their PR department.”
It’s often said that there’s no such thing as bad publicity, but buzz out of proportion with your product risks setting up customers for disappointment. Sabri Sansoy, an independent machine-learning consultant who has worked on projects for clients including ad-agency giant Ogilvy, says he routinely has to deflate dreams built on Watson marketing messages. “Everyone knows Watson,” says Sansoy, who sometimes uses IBM’s services. “But a lot of people get confused and think it’s one massive artificial general intelligence service that can do everything.”
In fact, like all the AI systems in use today, Watson needs to be carefully trained with example data to take on a new kind of problem. The work needed to curate and label the necessary data has been a drag on some projects using IBM’s system. Ashok Goel, a computer science professor at Georgia Institute of Technology, got written up in The Wall Street Journal and Backchannel after building a Watson bot to answer questions from students to his online course on artificial intelligence. But its performance was limited by the amount of manual labelling of data needed. “It had fairly high precision, but it did not answer a very large number of questions,” Goel says. “We have gradually moved away from IBM Watson for this reason.” (He continues to work with Watson on other projects, for example building a research assistant bot for scientists at the Smithsonian.)
In February we learned that cancer center MD Anderson had walked away from more than $62 million and four years spent on contracts promising a Watson system to help oncologists treat patients. An internal audit reserved judgment on Watson’s intelligence, but said the center had struggled to connect it with an upgraded medical records system. IBM maintains the system could have been deployed if MD Anderson had kept going; the center is now seeking a new partner to work with on applying AI to cancer care.
IBM still boasts plenty of partners building with Watson, including other leading health care centers such as Memorial Sloan Kettering and the Mayo Clinic. It likes to point to startups built around Watson-branded services for tasks such as translation or searching legal documents. And Ruchir Puri, chief architect for IBM Watson, says that the company has begun rolling out technology that reduces the work needed to adapt Watson to a business’s particular problem. “Some of this impression that has existed is antiquated,” he says. The company is bringing in “starter packs” that give Watson an instant shot of knowledge tuned to specific areas of business like banking or retail. Puri says the company will also make Watson a quicker study using a technique called transfer learning, which allows a system trained on a given problem to use some of its knowledge on a different one.
Watson certainly still has a big opportunity laying at its feet. “AI has permeated a lot of consumer companies, like Google search, Netflix, and Uber, but the majority big corporations in the Fortune 2000 are still just experimenting,” says Stephen Pratt, CEO of Noodle.ai, and previously an executive in IBM’s Watson group. “Companies are starting to realize there’s this AI divide.”
That should mean plenty of dollars spent on AI tools in coming years, but IBM isn’t the only one chasing them. Countless startups like Pratt’s are peddling AI tools to businesses, and other tech giants are building up competing platforms. Microsoft CEO Satya Nadella has made so-called “cognitive services” a central part of his effort to build up Microsoft’s cloud business. Google has put AI at the heart of its own cloud strategy (although it has so far refrained from branding the effort after its own human-beating game player, AlphaGo). The Mountain View juggernaut has even set up a unit of engineers that work with cloud customers to build up machine learning and AI projects, a model with echoes of IBM’s own services business. The Jefferies report credits Watson as one of the most comprehensive offerings in the race to sell AI tools to businesses, but says that being costly compared to other options puts it at a disadvantage. It’s still too early for the Jeopardy! clue with the answer “What is IBM Watson?” to read “The AI brand that saved the company’s business.”