Human edges out machine in latest artificial intelligence showdown

by Peter Griffin / 13 February, 2019
ibm project debater  vs Harish Natarajan

IBM's Project Debater AI Machine debated whether pre-school should be subsidised with debating champ Harish Natarajan.

In all of the contests of the last 25 years pitting supercomputer against human, one thing has remained constant – there’s always a finite number of possible answers available to flesh and silicon-based player alike.

In 1996’s match-up between IBM’s Deep Blue supercomputer and Russian chess grandmaster Gary Kasparov, the one constant was the total number of possible moves on a chess board. The result was a draw – one series won by Kasparov, the other by Deep Blue.

In 2011, the IBM Watson machine had to trawl its database to find the right answers to cryptic questions asked of it during the quiz show Jeopardy. Watson trounced its human rivals.

But what about a live debate, where facts matter less than human intuition and the power of persuasion?

Last night’s debate in San Francisco between Project Debater, an artificial intelligence engine based on Watson, and Harish Natarajan, the 2016 World Debating Championship grand finalist, suggests this is where humans still have the edge – for now.

Project Debater sat on stage, a monolithic black tablet emanating an even, American-accented woman’s voice. Over 25 minutes she traded statements and rebuttals with Natarajan on the topic of whether pre-school should be subsidised.

Miss Debater, as she has been dubbed, was arguing for the resolution. She and her human rival had just 15 minutes to prepare for the debate. But able to trawl 10 billion sentences of reference material, mainly newspaper and magazine articles, it didn’t take long for the computer to formulate a strong argument for funding preschool.

The computer says yes

Laying out her case logically, she peppered her talk with OECD and US Centers for Disease Control report statistics and even quoted former British Prime Minister Benjamin Disraeli. Most impressive was Project Debater’s ability to listen to Natarajan’s arguments in real time, her display blinking away all the time, and counter them with a reasonable, fact-based rebuttal. She even managed some emotive flourishes at one point saying, “to be clear, my intention is not to leave a suitcase full of money for everyone to grab at will.”

Natarajan, without any research materials to draw on, had to rely on his best rhetorical skills. And his ability to do so skillfully is what separated man and machine. His rebuttals were stronger than Project Debater who largely just continued on with her narrative in favour of the resolution.

Sitting among the 700-strong audience at San Francisco's Yerba Buena Center, which held debaters from around the San Francisco Bay Area and a cohort of journalists, it was my job to help decide who won the debate.

The human edge

After the electronic votes were tallied, a humble and unflappable Natarajan was declared the winner. His victory hinged on the key argument that funding pre-school would largely just subsidise middle-class families who are likely to send their children to preschool anyway.

Prior to the debate, 79 per cent of the audience agreed that preschool should be subsidised. Afterwards however, that had dropped to 62 per cent. The debate rules awarded victory to the debater who gained the most points over the course of the debate and Natarajan subsequently emerged victorious.

But a second question asked of us revealed Project Debater’s true strength – the majority of the audience found Project Debater more informative.

“In terms of rhetorical skills, the system is still not at the level of a debater like Harish,” admitted Noam Slonim, one of the Israeli IBM scientists who led the development of Project Debater.

“That said, the system is capable of pinpointing relevant evidence within a massive collection.”

Indeed, with no pre-programming specific to the debate topic and talking fluently and coherently for up to four minutes at a time, Project Debater is a world away from the short and sometimes garbled snippets emitted from the Siri or Alexa digital assistants.

Read more: Why Facebook’s mega messaging merger will make you rethink social media

That’s down to the sophisticated algorithms that underpin Project Debater, but also the huge computing power behind it – a dedicated computer server with dozens of processors and as much memory as 50 laptops.

But the potential real world applications of Project Debater’s intelligence are vast. In the area of government policy development it could canvas peer-reviewed literature to formulate arguments in support of evidence-based policy making, reducing the more subjective approach of human researchers.

Does Shane Jones’ regional growth fund proposal stack up? Should we create a national strategy for cancer treatment? Imagine the questions you could get smart, rational answers on.

ibm project debater  vs Harish Natarajan

Natarajan came out on top.

The results, of course, will always depend on the quality of the information that is fed into Project Debater and most of it after all, will have been written by humans anyway.

“If you take some of those skills and add a human being who can use it in slightly more subtle ways, I think that could be incredibly powerful,” concluded Natarajan following the debate.

It is a convenient conclusion for the AI companies forging ahead with development amidst fears that automation will destroy millions of jobs. The clear implication is that AI will become smarter at augmenting human skills, rather than taking the human out of the decision-making process entirely.

Taking the tedious and time-consuming effort out of research, giving human beings a better factual basis on which to make decisions, seems to be where Project Debater’s future lies.

Imagine the power of such an AI-powered service on your smartphone or sitting in the smart speaker in your lounge.

Given the exponential growth in computing power, we can expect it to be within our grasp within years. But by that stage, the machine may well be better able to outwit the smartest human debaters too.

Peter Griffin visited San Francisco as a guest of IBM.

Follow NOTED on Twitter, Facebook, Instagram and sign up to our email newsletter for more tech news.


China could be using Taiwan as a testing ground for disinformation campaigns
102550 2019-02-20 00:00:00Z World

China could be using Taiwan as a testing ground fo…

by Gavin Ellis

A Taiwanese diplomat’s death in Japan has become a symbol of the consequences and dangers of disinformation.

Read more
The best way to beat food cravings? Fill up on the objects of your desire
102087 2019-02-20 00:00:00Z Nutrition

The best way to beat food cravings? Fill up on the…

by Jennifer Bowden

Research has shown that dieters’ attempts to resist eating certain foods appear to lead to cravings for those foods.

Read more
Deepfake: How disinformation fools our brains and damages democracy
102545 2019-02-20 00:00:00Z Tech

Deepfake: How disinformation fools our brains and…

by Gavin Ellis

Message manipulation using bots, algorithms and, now, AI software is making it harder to know what’s real – and threatening democracy itself.

Read more
Move to introduce digital tax for foreign companies profiting online
102519 2019-02-19 09:23:12Z Economy

Move to introduce digital tax for foreign companie…

by RNZ

New Zealand is lining up to introduce a new tax on multinational companies that make money out of online goods and services in this country.

Read more
National's high-risk gamble on marijuana and euthanasia
102484 2019-02-19 00:00:00Z Politics

National's high-risk gamble on marijuana and eutha…

by Graham Adams

Having polarising MPs like Paula Bennett and Maggie Barry leading the opposition to popular reforms could be kryptonite to the National Party.

Read more
Reflections on my encounter with the charming Dan Mallory
102482 2019-02-19 00:00:00Z Profiles

Reflections on my encounter with the charming Dan…

by Michele Hewitson

He penned a bestselling thriller, but as Michele Hewitson discovered, author Dan Mallory also proved himself to be a charmingly adept bullshit artist.

Read more
Sounds of summer: Notable Kiwis name their favourite summer songs
102500 2019-02-19 00:00:00Z Music

Sounds of summer: Notable Kiwis name their favouri…

by Phil Gifford

How music can transport you back to your most memorable summer.

Read more
Inside the close-knit community that lives along the Cromwell-Tarras Rd
102505 2019-02-19 00:00:00Z Travel

Inside the close-knit community that lives along t…

by Mike White

Mike White heads up the Cromwell-Tarras road to merino and wine country.

Read more