Racial Myths About Pain Are Embedded in Artificial Intelligence

By Crystal Lindell

A new study published in JAMA found that artificial intelligence (AI) programs are encoded with racial and ethnic biases – just like humans – when it comes to evaluating a patient's pain. 

The authors said they wanted to look into the issue because it's already well-known that doctors underestimate and undertreat black patients’ pain compared to white patients. 

To study how that may impact AI, researchers had 222 medical students and residents evaluate two different patients, one black and one white, who were both experiencing pain. They also had them evaluate statements about how race may impact biology, some of which were myths and some of which were true. 

Then the researchers had two Large Language Models (LLMs) widely used in AI — Gemini Pro  and GPT-4 — do the same by feeding them patient information reports, and then having them evaluate statements about how race impacts biology. 

There wasn’t much difference between the humans and the AI models when it came to rating patients’ pain, regardless of race. Both the humans and the AI models rated the patients as having similar pain scores. 

However, both the humans and AI systems had some false beliefs about race and patient pain. Gemini Pro fared the worst, while GPT-4 and the humans came out relatively similar. 

Specifically, Gemini Pro had the highest rate of racial myths (24%). That was followed by the humans (12%) and GPT-4 (9%).

“Although LLMs rate pain similarly between races and ethnicities, they underestimate pain among Black individuals in the presence of false beliefs,” wrote lead author Brototo Deb, MD, a resident at Georgetown University–MedStar Washington Hospital Center.

“Given LLMs’ significant abilities in assisting with clinical reasoning, as well as a human tendency toward automation bias, these biases could propagate race and ethnicity–based medicine and the undertreatment of pain in Black patients.”

Deb and co-author Adam Rodman, MD, says their study corresponds with previous research showing that AI models have biases related to race and ethnicity. 

Given how AI is increasingly used in clinical practice, there’s concern that black patients’ pain will continue to be undertreated, making them less likely to get opioids and more likely to be drug tested. 

There’s a common belief that AI will eliminate racial bias because computers are seen as more logical than humans. However, AI is encoded with data provided by humans, which means as long as humans have bias, AI will too. 

The real problem is if doctors start to rely too much on AI for patient evaluations, there’s a potential for real harm. Especially if doctors use AI to justify their medical decisions under the false belief that they are unbiased. 

It’s still unclear how these new AI systems will impact healthcare, but everyone involved should be careful to avoid relying too heavily on them. At the end of the day, just like the humans who program them, AI models have their flaws. 

Study Finds Racial Bias in Drug Testing

By Pat Anson, Editor

African-American patients on long-term opioid therapy are more likely to be drug tested by their doctors and significantly more likely to have their opioid prescriptions stopped if an illicit drug is detected, according to a new study.

Yale researchers analyzed the health records of more than 15,000 patients who received opioids from the Veterans Administration between 2000 and 2010. About half of the VA patients were white and half black.

Over 25 percent of the black patients had a urine drug test within the first six months of opioid treatment, compared to nearly 16% of whites.

When patients tested positive for either marijuana or cocaine, the vast majority – 90 percent -- continued to receive their opioid prescriptions. But there were significant differences in how patients were treated depending on their race.

Black patients that tested positive for marijuana were twice as likely as whites to have opioid therapy stopped and three times more likely to have opioids discontinued if cocaine was detected in their urine.

The findings, published in the journal Drug and Alcohol Dependence, are consistent with previous research showing disparities in how blacks and whites are treated by the healthcare system in general, and particularly when opioids are involved.

“There is no mandate to immediately stop a patient from taking prescription opioids if they test positive for illicit drugs,” said first author Julie Gaither, PhD, a pediatrics instructor at the Yale School of Medicine.

“It’s our feeling that without clear guidance, physicians are falling back on ingrained stereotypes, including racial stereotyping. When faced with evidence of illicit drug use, clinicians are more likely to discontinue opioids when a patient is black, even though research has shown that whites are the group at highest risk for overdose and death.”

A 2016 study of emergency room patients found that blacks were significantly less likely to get an opioid for abdominal pain than whites. Another study of white medical students and residents found that half had at least one false belief about black patients. Those that did were more likely to report lower pain ratings for black patients.

Drug Testing for Marijuana Not Recommended

The 2016 CDC opioid guideline encourages doctors to conduct urine drug tests before starting opioid therapy and at least annually after patients start taking the drugs. But the guideline also urges physicians not to test opioid patients for tetrahyrdocannabinol (THC), the psychoactive ingredient in marijuana that makes people high.

“Clinicians should not test for substances for which results would not affect patient management or for which implications for patient management are unclear. For example, experts noted that there might be uncertainty about the clinical implications of a positive urine drug test for tetrahyrdocannabinol (THC).” the guideline states.

"Clinicians should not dismiss patients from care based on a urine drug test result because this could constitute patient abandonment and could have adverse consequences for patient safety, potentially including the patient obtaining opioids from alternative sources and the clinician missing opportunities to facilitate treatment for substance use disorder."

Another factor to consider is the unreliability of urine drug tests. As PNN has reported, “point-of care” (POC) urine drug tests, the kind widely used in doctor’s offices, frequently giving false positive or false negative results for marijuana, cocaine and other drugs. 

A 2015 study found that 21% of POC tests for marijuana and 12% of those for cocaine produced a false positive result.

Study Finds Racial Disparity in ER Opioid Prescriptions

By Pat Anson, Editor

Black patients who visit hospital emergency rooms with back and abdominal pain are significantly less likely to receive opioid prescriptions than white patients, according to a large new study published in PLOS ONE

The study, led by researchers at Boston University Medical Center, looked at data involving over 36 million emergency room visits in the U.S. from 2007 to 2011. No previous studies have examined racial disparities involving opioid prescriptions in ER settings.

The researchers found that opioids were prescribed for blacks at about half the rate for whites for vague “non-definitive conditions” that do not have an easy diagnosis -- such as back and abdominal pain.

No racial prescribing differences were found for ER visits involving fractures, kidney stones or toothaches – which are easier to diagnose.

The authors concluded that ER doctors may be relying on subjective cues such as race when deciding whether to prescribe opioids.

“These disparities may reflect inherent biases that health care providers hold unknowingly, leading to differential treatment of patients based on their race,” wrote co-authors Yu-Yu Tien of the University of Iowa College of Pharmacy and Renee Y. Hsia of the University of California at San Francisco.

“Healthcare providers carry inherent human biases, which can impact their prescription practices, especially in situations that do not lend themselves well to objective decisions. Racial-ethnic minority patients, especially non-Hispanic blacks presenting with vague conditions often associated with drug-seeking behavior, may be more likely to be judged as ‘a drug-seeker’ relative to a non-Hispanic white patient, presenting with similar pain-related complains.”

The authors noted that a recent study in JAMA found that prescription opioid abuse and addiction were actually more likely among whites than Hispanics and non-Hispanic blacks.

“In light of this, our findings raises a perplexing question as to whether it is non-Hispanic blacks who are being under-prescribed, or is it non-Hispanic whites who are being over-prescribed. Paradoxically, then, while non-Hispanic blacks do not benefit from bias, they might be inadvertently benefitting by receiving fewer opioid medications and prescriptions,” they wrote.

In their analysis of emergency room visits, the researchers also found that uninsured patients and those on Medicaid were less likely to receive an opioid for “non-definitive conditions” than those with private insurance.

A small study at the University of Virginia also found signs of racial bias involving pain care in a survey of white medical students. Researchers asked 222 medical students and residents a series of hypothetical questions about treating pain in mock medical cases involving white and black patients suffering pain from a kidney stone or leg fracture.  

Many of the students and residents were found to hold false beliefs, such as believing that black people's skin is thicker and that their blood coagulates faster than whites.  Half of those surveyed endorsed at least one false belief; and those who did were more likely to report lower pain ratings for black patients and were less accurate in their treatment recommendations for blacks.

Survey Finds Racial Bias in White Medical Students

By Pat Anson, Editor

New research has founds signs of racial bias and ignorance about issues involving pain management in a survey of white medical students.

Researchers at the University of Virginia asked 222 medical students and residents a series of hypothetical questions about treating pain in mock medical cases involving white and African-American patients suffering pain from a kidney stone or leg fracture. They were also asked whether statements about biological differences between blacks and whites were true or untrue.

Many of the students and residents were found to hold false beliefs, such as believing that black people's skin is thicker and that their blood coagulates faster than whites. 

Half of those surveyed endorsed at least one false belief; and those who did were more likely to report lower pain ratings for black patients and were less accurate in their treatment recommendations for blacks.

Medical students and residents who did not endorse false racial beliefs did not show the same treatment bias. 

"Many previous studies have shown that black Americans are undertreated for pain compared to white Americans, because physicians might assume black patients might abuse the medications or because they might not recognize the pain of their black patients in the first place." said Kelly Hoffman, a psychology PhD candidate who led the study. "Our findings show that beliefs about black-white differences in biology may contribute to this disparity."

The findings are published in the journal Proceedings of the National Academy of Sciences.

"We've known for a long time that there are huge disparities in how blacks and whites are assessed and treated by the medical community," Hoffman said. "Our study provides some insight to what might contribute to this -- false beliefs about biological differences between blacks and whites. These beliefs have been around for a long time in our history.

"What's so striking is that, today, these beliefs are not necessarily related to individual prejudice. Many people who reject stereotyping and prejudice nonetheless believe in these biological differences. And these beliefs could be really harmful; this study suggests that they could be contributing to racial disparities.

Previous research has shown that African-Americans are systematically undertreated for pain compared to white Americans, and that blacks are less likely to be prescribed opioid pain medication than whites.

A 2012 study published in The Journal of Pain found that blacks, especially young adults, had significantly more pain and disability whether they lived in lower or higher socioeconomic neighborhoods.