New Zealand scientists are extending life expectancy with technology that has the potential to diagnose an illness faster than a doctor.
The technology that gave Stone that life-changing information – he’s since started new drugs and a dietary regime that have shaved several years from his heart’s biological age – was a form of artificial intelligence (AI). It’s not what science fiction would dub AI – a type of spooky thinking robot that can take on a life of its own – but a form of machine learning that has been around for many years and is growing in sophistication and usefulness.
Stone had what’s known as an advanced electrocardiograph (ECG) that provides much more detailed and nuanced information than a regular ECG. It breaks down the electrical tracings into hundreds of 3D measurements, which are then compared with those of thousands of other patients in a highly curated international database. ECG tracings are so highly personalised, and the pattern generally so consistent over time, that they can be used as an identifier in much the same way as fingerprints or iris recognition.
The signals from Stone’s heart indicated early damage to his heart muscle, something that was not apparent from his ultrasound or clinical symptoms of atrial fibrillation (rapid heartbeat) and raised blood pressure, which are both common problems.
“Machines can see things we just can’t possibly see,” says Stone’s cardiologist, Patrick Gladding, who believes his Auckland practice is the first in the country to use the advanced ECG clinically. The machine is also used at the Waitematā District Health Board, but only for research. Artificial intelligence is trained to see patterns rather than single features, says Gladding, who uses the example of a city skyline. “Your eye sees key features such as the Sky Tower, but it’s really the relative height of various buildings that tells you this is Auckland rather than Shanghai. Similarly, in medicine, we are hung up on single biomarkers or diagnostic tools for disease, but what the human mind does is integrate that into a pattern comparing this with the knowledge from experience and theory. Human-guided AI really just externalises that and adds an extra layer of sensitivity so that very subtle patterns can be detected.”
The information the ECG provided on Stone prompted the former American football player to lose about 10kg and enabled Gladding to personalise his drug treatment.
Stone, who spent two years as an adviser to the chief executive and founders of Google in San Francisco and is now general manager of strategy and innovation at Datamine, an Auckland data-analysis company, ordered his own portable ECG recorder (at a cost of NZ$150) through the US Food and Drug Administration-approved KardiaMobile app, and began recording his own tracings several times a day, along with his blood pressure, and forwarded the information to Gladding. “We were looking for small changes to find out what makes me go into atrial fibrillation.” His heart was going out of normal rhythm every three or four days. The condition, combined with the effects of his newly prescribed blood-pressure pills, was leaving him physically and mentally drained. “I would debate in my mind whether I would prefer to die or feel like this. If I didn’t die, I would be hoping every day that I would. Climbing out of bed felt like climbing Mt Everest.”
His self-administered ECG and blood-pressure tests soon revealed a correlation between his blood pressure and his arrhythmia. His blood pressure would spike for no apparent reason, and three or four days later, his heart would bounce into fibrillation. After discussions with Gladding, Stone halved his blood-pressure meds, but doubled the dose temporarily whenever he saw the reading spike. The treatment has transformed the management of his heart condition. In mid-May, he celebrated five months clear of atrial fibrillation, and he’s managed to reduce his statin and beta-blocker drugs at the same time.
“We would not have been able to do this using conventional medicine,” says Gladding. “His heart age has actually regressed by three or four years. We made the pattern of his heart younger.”
But the use of AI in medicine raises important medicolegal questions, and does have drawbacks. “Over-reliance on AI could dumb down clinical skills. And the other problem is bias. A poorly trained AI may miss things, or be biased depending on the data fed into it, which could exacerbate already existing biases, such as ethnic disparity. The term garbage in, garbage out is very fitting.”
Saving money and lives
In a recent case Gladding was involved with, retrospective use of AI to read an X-ray found lung cancer in a woman in her sixties – a lesion that had been missed when the image was taken nine months earlier. “Unfortunately, this particular technology is not approved for clinical use, but hospitals need to seriously consider how to use AI, because it has the potential to significantly streamline our workload.” He says large hospitals in the US are developing AI technologies, validating and using them clinically. “We need to start thinking about doing the same. He says in another case involving an advanced ECG, the test identified the genetic cause for a sudden cardiac arrest in a 32-year-old woman.
Gladding believes AI can play an important role in public health, but needs an influential supporter at government level to drive its use. He’d like to use it as a tool for screening and triaging heart patients waiting for hospital clinic appointments, for example, after advanced ECGs performed well in a pilot study. He says it could save money and lives. “A lot of the focus is on big-ticket items. We see lists blowing out in CT or MRI scanning, so people buy a new machine for $1-2 million. These things get funded almost without much question. People seem to overlook tools they already have, thinking you can’t get any more out of the basics, but that just isn’t true.”
He says the public, and health authorities, need to overcome their scepticism about the technology. “The public should know more so they are kept in the loop about what’s happening with their data, and discover that it’s not all hype and that it is exciting. With the right governance and oversight, it won’t become Big Brother or rogue intelligence that could prevent people getting access to healthcare. If anything, you expect it would do the opposite, by enabling primary care to make much more informed decisions, rather than the ambulance at the bottom of the cliff approach we have now.”
In two studies supervised by Gladding at the Waitematā DHB and published in 2015 and 2017, more than 370 ECG readings were assessed by advanced ECG AI, cardiologists and GPs. The AI-read ECG was better at identifying disease than either doctor, and predicted which patients would be readmitted with heart failure or suffer a sudden cardiac death.
“There is a temptation in medicine to be cautious, and medicolegally defensive, which results in overcalling disease, over-ordering investigations and procedures to make sure not to miss anything. That reduces the ability to see health when no disease is present. The advanced ECG is not only better at seeing disease, but substantially better at seeing health than a human.”
Gladding says some patients have to wait longer than they should with current, time-consuming triaging methods, and the AI could also speed up diagnoses. “When I am triaging, I might spend a minute or two or less reading the GP’s notes, so they have to be brief and to the point. Generally, I won’t go into the rest of that patient’s clinical records to see what’s happened in the past. I just don’t have time. But a machine could do that in an instant and bring other information to the fore.”
At present, machine triaging is being investigated by Precision Driven Health, an initiative managed by Auckland-based software company Orion Health. “It will be a while before we get to the point where we have only machine-assisted triaging, if we do at all. We may discover things you just can’t predict.” However, Gladding says, a retrospective analysis of patients referred for ECGs and ultrasound has found that machines predicted, with about 85% accuracy, what an ultrasound would later find, based on information doctors had already gathered.
The advanced ECG was pioneered by flight surgeon Todd Schlegel when he worked at the National Aeronautics and Space Administration’s (Nasa) Johnson Space Centre and has been used on the International Space Station. Schlegel, now in Switzerland, has been working with Gladding to fine-tune the algorithms for cloud-based analyses of advanced ECGs done in Auckland. More than 1000 have been collected and analysed so far.
Nasa technology is also fundamental to research by a 19-year-old software-engineering wunderkind at the Auckland Bioengineering Institute (ABI), who is training artificial intelligence to interpret cardiac ultrasounds.
Will Hewitt “dropped out” of Nelson College without completing his final year to set up his own electronics consultancy doing small projects before approaching public-health physician and health-technology expert Robyn Whittaker about his passion for AI and cardiac ultrasound imaging. Whittaker put the then 17-year-old in touch with Gladding, who directed him to the University of Auckland ABI, which, under the leadership of internationally renowned professor Peter Hunter, specialises in building computational models of the heart. The work has aimed to better understand the biology of the heart and how it works, in the hope of improving diagnostic technology and potentially developing drugs or therapies to correct abnormalities. Information fed into the diagnostic algorithms is highly personalised, including blood, biomarker and genetic data.
Hunter had no qualms about taking on the undergraduate as a part-time researcher, and describes the machine-learning work as “Will’s baby”. People with the most in-depth knowledge on any subject are those who are passionate about the subject, not those tied to their textbooks, he says.
“He’s incredibly fast at absorbing information, and very focused,” Gladding says of Hewitt. “When people turn up as he did, you wonder how much they can know about something complex, such as cardiac ultrasound. It requires years of training to understand, but he’s like a sponge. He’s soaked it up in six months to a year.”
Hewitt is doing an undergraduate degree in applied maths and physics, while also working to commercialise the software that interprets the ultrasounds. He has submitted a paper to the European Society of Cardiology conference to be held in Paris in August, about one of his studies, which showed that AI assessment is more consistently accurate than that of the human eye. More computer power could make AI readings much faster and far more accurate, he believes. “Patrick and a sonographer can spend anything from 40 to 50 minutes annotating the ultrasound and taking measurements off the scan. The object of the deep-learning project is to take those measurements for him, in 10 minutes rather than an hour. We are trying to see if we can get the same numbers he does, but faster.”
If all the research went perfectly, it could still be five years or more before the technology is applied clinically or commercially, says Hewitt. And there is a risk of doctors being too reliant on an AI technology, which is only as good as the data that’s fed into it, including outliers. “You can’t release a tool that might miss 5% of patients.”
A point of no return?
The international recognition of the world-leading work at the ABI resulted in the institute being brought on board by the Cleveland Clinic in Ohio, as a subcontractor with Nasa, in 2010. The ABI’s role was to develop a computerised model of the heart, personalised to each astronaut, to monitor their heart function in space and predict the outcome of deep-space travel to Mars.
In 2014, Nasa published its study of 12 astronauts that showed how the heart, which is usually the shape of a rugby ball, becomes more spherical – almost soccer-ball-like – when exposed to long periods of almost zero gravity. “Simplistically,” says Hunter, “under gravity, quite a lot of your blood is dragged down your body. In space, where you don’t have that gravity, more of the blood stays higher up in the chest, so you blow the heart up like a balloon.”
He says it’s impossible to know yet whether long periods in that state may shorten life, but “you would certainly expect a significant change in the shape of the heart to have flow-on effects. When the heart is under abnormal stretching conditions, it will release hormones that then modulate the neural control of the heart, so that in itself tells you there must be significant biochemical consequences.”
The issue, says ABI investigator Jagir Hussan, who started working on the Nasa project in 2010, is whether scientists can work out if there is a point of no return for astronauts in space, when the heart can no longer resume its normal shape and function, thereby limiting lengthy missions. “They want to send people to Mars and that would be a three-year trip, but would they actually be able to come back to Earth?”
Doctor and data
Back at Datamine’s Parnell headquarters, Bob Stone is giving himself an ECG on his KardiaMobile device by pressing two fingers on to the small pad on his desk. It’s in perfect sinus rhythm. I take the test as well, and the device smells a rat – it’s detected that my electrical readings are different to his. A message pops up on Stone’s phone, connected to the app. “Did Robert take this test?” it asks, bristling with robotic suspicion. “It’s pretty smart,” Stone says, laughing. “It knew you weren’t me.” The app cleans up the “electrical noise” on the reading before pronouncing my rhythm is normal.
Gladding cautions, however, that the technology can sometimes produce false positives or false negatives. “Apple’s current iWatch in New Zealand, which uses an infrared light to measure pulse, is a good example of where it’s gone straight into the community. We are already getting referrals of twentysomethings with high heart-rate recordings, but in the two cases I’ve been involved with, it turned out to be nothing important.”
Stone’s battery of heart tests, and the information revealed by the AI analysis, showed he has a genotype carried by 1-4% of the population that increases the risk of atrial fibrillation, but responds to personalised treatment. This genotype means cardiac ablation, a $40,000 procedure, is much less likely to work for him. Research done at the ABI suggests that a cheap off-patent drug, dantrolene, will probably be more effective, although that work will require clinical trials before it can be used in practice.
In his line of work, Stone was never in doubt about the value of AI. Now, he says, the doctor and the data have saved his life.
People are ready and willing to substitute AI and robotics for humans in their healthcare.
- More than 35% were happy to have AI monitoring their heart condition or have a robot administer a test that checked on heartbeat and made recommendations based on the results.
- 27% of those in the UK were willing to have major surgery performed by a robot (compared with 69% in Nigeria).
- 30% would accept a robot taking and reporting on their blood tests.
- More than 30% perceived AI/robotics would lead to faster or more accurate diagnoses.
- 47% wouldn’t trust robots to make a decision on their healthcare if something unexpected arose.
How an advanced ECG works
Machine learning helps plot the trajectory of a disease and determine biological age.
This article was first published in the May 18, 2019 issue of the New Zealand Listener.