Talk With Me: The app getting young people with autism talking

by Donna-Marie Lever / 03 February, 2018
Dr Swati Gupta and Oliver Knight use the Talk With Me app (pictured below), to encourage the kind  of conversations children with autism often find difficult.

Dr Swati Gupta and Oliver Knight use the Talk With Me app (pictured below), to encourage the kind of conversations children with autism often find difficult.

Christchurch computer scientist Dr Swati Gupta prefers people to programming, but it’s her experience in human-computer interaction that’s led to what may be a breakthrough for youngsters with autism.

The 35-year-old has developed a tool called Talk With Me, an app for mobile phones and tablets that teaches children how to interact socially – something many on the autistic spectrum struggle with.

A senior research scientist at Callaghan Innovation, Gupta moved to New Zealand three years ago from Singapore, where she was involved with a project for dementia patients. She admits she was “clueless” about autism until she began working with speech and language teachers at Hillmorton High School’s Upland Unit, a specialist facility in Christchurch for students with very high needs.

“I spent time in class and I saw a lot of repetition,” she says. “It was, ‘Johnny, do you want to ask Mary a question? Johnny, do you want to ask Mary?’ – over and over and over. And Johnny didn’t want to ask Mary, but eventually he did, and then Mary didn’t want to answer. So it was a real challenge.”

The app is aimed at five- to 20-year-olds and uses symbols to prompt language and conversation. It’s not a game you play solo but rather acts like a mirror to help two people engage. Early signs are positive, with parents reporting increases in eye contact, attention span and social interaction.

Her work has been awarded two grants totalling $50,000 from the KiwiNet Emerging Innovator Fund, a philanthropic initiative backed by the Norman F. B. Barry Foundation to help scientists with “disruptive new ideas” work alongside business to take their discoveries to market.

Read more: The curious incident of the boy in the headlines | The Autistic Gardener: How Alan Gardner uses his unique brain to his advantageLee Ingleby opens up about British autism drama The A Word

Another recipient, Otago University PhD student Sean Mackay, has been given $25,000 to help develop a treatment for strawberry birthmarks, a disfiguring and occasionally life-threatening type of vascular tumour that affects one in 10 infants. He’s working with a team to develop a targeted treatment that uses nanotechnology to literally squish drugs directly into the tumour through the skin. Other projects supported by the fund include research into video fingerprinting for border-control scanning, and biodegradable “nano-webs” that can be spread over crops to help with pest control.

Gupta’s grants have enabled her to run several trials with Talk With Me, both here and in India, where she was born. “It’s a very simple idea; I didn’t know how good it was, so we wanted to see if it works,” she says. “I went back to the Upland Unit with my tablet and two girls came in, one with quite severely restricted conversation skills. They sat down and started playing. Ten minutes later, they were still playing. I didn’t realise how huge that was then.”

It was only when the teacher told her the girls “don’t speak” that she realised the app’s true potential. “It was the first time the girl had asked the other one what her favourite animal was – and they’d been studying together for five years.”

Gupta’s now trying to secure commercial funding of at least $200,000 to get an initial “alpha version” of Talk With Me into the New Zealand market. The app, which could prove useful for people with speech and language disorders, cerebral palsy or ADHD, might also help those with autism as they grow into adults.

“It could work for dating scenarios,” she says. “They are often not sure what to ask, and the platform lends itself to being used by adults, but we haven’t trialled it yet.” 

This was published in the January 2018 issue of North & South.

 

Latest

Vital evidence in Pike River mine disaster missing, say families
102465 2019-02-18 09:22:49Z Planet

Vital evidence in Pike River mine disaster missing…

by RNZ

Some families of Pike River mine victims suspect a piece of vital evidence may have been spirited away by the mining company and lost.

Read more
It's time to empower the mayor and make Auckland liveable again
102432 2019-02-17 00:00:00Z Politics

It's time to empower the mayor and make Auckland l…

by Bill Ralston

Making Auckland a liveable city is an unenviable task, writes Bill Ralston, but it's clear the mayor needs more power.

Read more
Knight star: Sir Hec Busby on his extraordinary life
102328 2019-02-17 00:00:00Z Profiles

Knight star: Sir Hec Busby on his extraordinary li…

by Clare de Lore

Northland kaumātua, master carver, navigator and bridge builder Hec Busby was hoping for “no fuss” when he accepted a knighthood.

Read more
Keira Knightley shines in bodice-ripping period drama Colette
102397 2019-02-16 00:00:00Z Movies

Keira Knightley shines in bodice-ripping period dr…

by James Robins

The story of Sidonie-Gabrielle Colette, a heroine of French literature, focuses on her early struggles.

Read more
Is barbecued meat bad for your health?
102255 2019-02-16 00:00:00Z Nutrition

Is barbecued meat bad for your health?

by Jennifer Bowden

Sizzling meat on the barbecue is the sound and smell of summer, but proceed with caution.

Read more
March of the Algorithms: Who’s at the wheel in the age of the machine?
102434 2019-02-16 00:00:00Z Tech

March of the Algorithms: Who’s at the wheel in the…

by Jenny Nicholls

Complacently relying on algorithms can lead us over a cliff – literally, in the case of car navigation systems.

Read more
IBM’s new quantum computer: The future of computing
102458 2019-02-16 00:00:00Z Tech

IBM’s new quantum computer: The future of computin…

by Peter Griffin

The Q System One, as IBM calls it, doesn’t look like any conventional computer and it certainly doesn’t act like one.

Read more
James Shaw: Capital gains tax key to fixing wealth gap
102456 2019-02-15 14:54:45Z Politics

James Shaw: Capital gains tax key to fixing wealth…

by RNZ

The week before a major tax report is released, Green Party co-leader James Shaw has again challenged his government partners to back the tax.

Read more