If black patients cost less to treat than white patients, maybe their illnesses aren’t as bad.
That's the underlying message signaled to clinicians
by a widely used AI algorithm that has steered black patients away from higher-quality care, generating more complex treatment for white patients than for sicker black patients.
But in reality, black patients are sicker when they enter care — because they haven’t purchased as many healthcare services as white patients have — which is mostly because they have less access to equal-quality care.
A recent study published by the American Association for the Advancement of Science examined data on 6,079 black
patients and 43,539 white patients.
Impact Pro, the AI algorithm in question, determined that the pool of sickest patients was 81.8% white and about 18% black. In fact, the
researchers say if the algorithm accurately analyzed the sick patients, 46% of black patients should have been identified.
Why is this a problem for the industry?
The American healthcare system relies on commercially developed algorithms for predicting heath needs and costs, managing risk, and planning care delivery and
health needs — particularly for patients with complex medical needs. The scores they produce are input to decisions about future enrollment in a care coordination program that better manages the
sickest patients.
In the U.S., we spend significantly less on care for diverse racial, ethnic and gender segments than on white male patients. When the cost of care is used as a proxy for a patient’s healthcare needs, services are inefficiently allocated, driving higher risk and more disparities in the quality of care.
Why are
algorithms like Impact Pro biased?
What can marketers do to address this problem? Here are three ideas.
Ultimately, we are the communicators and storytellers who are able to spot and address flawed racial and gender narratives affecting how the organization thinks and talks about the patients we serve. We can leverage those skills to change how the larger promise of AI can bring safe and equitable care to all patients.
This is great insight, and very interesting and useful information. Yet purely telling the story from the angle that "people training these algorithms are mostly white male" is somewhat insufficient and almost naive. Because at a quick glance we will also see how Impact algorithm harms the white patients as well as it will tend to put together more comprehensive treatment plans.
If we are going down the racial rabbit hole here, we would inevitably be neglecting the overall negative impact of an algorithm that would favor opiod painkiller, for example, in some situations and not do so in the other. This is more than hindrance of giving treatment but the inability to accurately and correctly evaluate the need of healthcare treatment, because I think we can all agree on how absurd healthcare cost is, in general.
It seems the failing system calls for an upgrade on infrastructure, administration and research more than on "marketers" at this point, because in order to feed and train an AI properly we are assuming that we are actually able to collect somewhat accurate data prior to a patient, black or white, reach out and seek help, whereas the fact is the complete opposite: How could the data be collected if the patient did not seek care, whether it's shopping from a pharmacy over the counter, or going to a doctor? What can communities do to help collect these data? How should these data be reported, sorted and integrated with other data in general?