In all of the contests of the last 25 years pitting supercomputer against human, one thing has remained constant – there’s always a finite number of possible answers available to flesh and silicon-based player alike.
In 2011, the IBM Watson machine had to trawl its database to find the right answers to cryptic questions asked of it during the quiz show Jeopardy. Watson trounced its human rivals.
But what about a live debate, where facts matter less than human intuition and the power of persuasion?
Last night’s debate in San Francisco between Project Debater, an artificial intelligence engine based on Watson, and Harish Natarajan, the 2016 World Debating Championship grand finalist, suggests this is where humans still have the edge – for now.
Project Debater sat on stage, a monolithic black tablet emanating an even, American-accented woman’s voice. Over 25 minutes she traded statements and rebuttals with Natarajan on the topic of whether pre-school should be subsidised.
Miss Debater, as she has been dubbed, was arguing for the resolution. She and her human rival had just 15 minutes to prepare for the debate. But able to trawl 10 billion sentences of reference material, mainly newspaper and magazine articles, it didn’t take long for the computer to formulate a strong argument for funding preschool.
The computer says yes
Laying out her case logically, she peppered her talk with OECD and US Centers for Disease Control report statistics and even quoted former British Prime Minister Benjamin Disraeli. Most impressive was Project Debater’s ability to listen to Natarajan’s arguments in real time, her display blinking away all the time, and counter them with a reasonable, fact-based rebuttal. She even managed some emotive flourishes at one point saying, “to be clear, my intention is not to leave a suitcase full of money for everyone to grab at will.”
Natarajan, without any research materials to draw on, had to rely on his best rhetorical skills. And his ability to do so skillfully is what separated man and machine. His rebuttals were stronger than Project Debater who largely just continued on with her narrative in favour of the resolution.
Sitting among the 700-strong audience at San Francisco's Yerba Buena Center, which held debaters from around the San Francisco Bay Area and a cohort of journalists, it was my job to help decide who won the debate.
The human edge
After the electronic votes were tallied, a humble and unflappable Natarajan was declared the winner. His victory hinged on the key argument that funding pre-school would largely just subsidise middle-class families who are likely to send their children to preschool anyway.
Prior to the debate, 79 per cent of the audience agreed that preschool should be subsidised. Afterwards however, that had dropped to 62 per cent. The debate rules awarded victory to the debater who gained the most points over the course of the debate and Natarajan subsequently emerged victorious.
But a second question asked of us revealed Project Debater’s true strength – the majority of the audience found Project Debater more informative.
“In terms of rhetorical skills, the system is still not at the level of a debater like Harish,” admitted Noam Slonim, one of the Israeli IBM scientists who led the development of Project Debater.
“That said, the system is capable of pinpointing relevant evidence within a massive collection.”
Indeed, with no pre-programming specific to the debate topic and talking fluently and coherently for up to four minutes at a time, Project Debater is a world away from the short and sometimes garbled snippets emitted from the Siri or Alexa digital assistants.
That’s down to the sophisticated algorithms that underpin Project Debater, but also the huge computing power behind it – a dedicated computer server with dozens of processors and as much memory as 50 laptops.
But the potential real world applications of Project Debater’s intelligence are vast. In the area of government policy development it could canvas peer-reviewed literature to formulate arguments in support of evidence-based policy making, reducing the more subjective approach of human researchers.
Does Shane Jones’ regional growth fund proposal stack up? Should we create a national strategy for cancer treatment? Imagine the questions you could get smart, rational answers on.
The results, of course, will always depend on the quality of the information that is fed into Project Debater and most of it after all, will have been written by humans anyway.
“If you take some of those skills and add a human being who can use it in slightly more subtle ways, I think that could be incredibly powerful,” concluded Natarajan following the debate.
It is a convenient conclusion for the AI companies forging ahead with development amidst fears that automation will destroy millions of jobs. The clear implication is that AI will become smarter at augmenting human skills, rather than taking the human out of the decision-making process entirely.
Taking the tedious and time-consuming effort out of research, giving human beings a better factual basis on which to make decisions, seems to be where Project Debater’s future lies.
Imagine the power of such an AI-powered service on your smartphone or sitting in the smart speaker in your lounge.
Given the exponential growth in computing power, we can expect it to be within our grasp within years. But by that stage, the machine may well be better able to outwit the smartest human debaters too.
Peter Griffin visited San Francisco as a guest of IBM.