Excitement (still) abounds for the application of artificial intelligence to medicine, with the latest hype migrating to ChatGPT. I wrote a review on the topic (geared for clinicians) back in 2015 that I believe remains relevant to the field as a whole. But in this post, we will drill down on a particular subtopic: the use of artificial intelligence to make chronic disease management better.

Artificial intelligence – or more specifically machine learning, comes in a variety of flavors. Most of the time when people think about how machine learning and medicine they think about a specific type called supervised learning. Here, the idea is that the doctor knows the correct answer and a machine simply mimics the doctor using the same information. The model is trained by doctors giving true and false examples and the machine figures out how to distinguish them. Simple examples includes teaching a machine to detect breast cancer in a mammogram or COVID in a chest x-ray. I’ve trained some more cardiology-focused models in my academic career (see for example our work on cardiac amyloidosis and hypertrophic cardiomyopathy) and we at Atman Health are building some commercial solutions in this space. Why might this be of value to society? If the machine could outperform physicians or perhaps come close enough in resource poor settings where there is a shortage of physicians, we might see the sort of benefits that automation has brought to other industries.

But implicit in this approach is the assumption that the right answer is known and can be obtained from the input data – in this case X-rays. But when it comes to chronic disease management, I would argue (heretically) that in most cases the right answer is NOT known for individual patients. For example, there are over 15 classes of medications that treat high blood pressure and over 8 classes that treat type 2 diabetes (along with many others for heart failure and atrial fibrillation and gout) – and within each class there are many choices. Other than some broad guidelines, we have no idea which medication will be effective for a given patient. On top of this there are many more choices around starting dose, how quickly one can change something, how to predict and manage side effects efficiently, how to combine medications from different classes – all of these which have no answer. And beyond this what behavioral strategies are likely to work best with particular patients to help them overcome the obstacles that come from enacting change. Given this dizzying array of choices, physicians sometimes bring some “art” to decision making and may have some preferred choices – but more often than not these represent personal comfort and habits rather than any matching of therapies to individual patient physiologic characteristics – which, understandably, no individual practitioner has a realistic hope of being able to learn.

In this situation, where the overall goal for disease management is accepted (e.g., time to attaining target, getting patients on evidence-based medications at studied doses) but the specific methods on how to do it are unknown, supervised learning fails. But there is a type of machine learning that is perfectly suited for this situation – known as reinforcement learning. Reinforcement learning has three main attributes: 1) pursuit of an overall goal, 2) the need for exploration or trial and error, 3) delayed recognition of which actions in a given situation is best.

The most celebrated examples of reinforcement learning in recent years have been in teaching computers to master games such as chess and Go. In each case, computers now easily surpass human grandmasters. The reason game-playing is ideally suited for reinforcement learning is that the overall goal is clear (to win), the rules are known, and the system is closed (no relevant environmental factors). All that is needed to develop these incredible game-playing machines has been to simulate billions of games and have the computer learn what moves in any possible scenario maximize the chance of winning, averaged across all possible opponent strategies. Other more sophisticated examples of reinforcement learning include self-driving cars where again the goal is identifiable (to safely reach a destination) and the rules of driving are known – and all that’s needed is to have progressively more and more autonomous cars drive millions of miles to learn.

Now of course chronic disease management lacks the simplicity of playing a board game and there is no way we can realistically simulate billions of patients going through treatment (if we could do that, our problem would be solved). But as described above, the attributes of the problem are similar: a (usually) accepted overall goal for a given disease and an initial set of parameters that can be altered that are seen as equivalent by conventional guidelines. These parameters are already routinely varied by practitioners trying to improve care for their patients, albeit in an inscrutable, heavily confounded manner that it is impossible to learn from. So what’s missing is a platform that can implement some of these variations more objectively, measure what’s working for patients, learn factors that predict responses for individual patient groups, and iteratively improve matching patients with therapies.

Rahul D.