How Racist Is Your Algorithm?

Every time Nathan Chomilo, MD, uses a clinical decision support tool, he tells his patients they have a choice: he can input their race, or keep that field blank.

Until recently, many clinicians didn't question the use of race as a datapoint in tools used to make decisions about diagnosis and care. But that is changing.

"I've almost universally had patients appreciate that someone actually told them that their kidney function was being scored differently because of the color of their skin, or how they were identified in the medical chart along lines of race," Chomilo, an adjunct assistant professor of pediatrics at the University of Minnesota Medical School, in Minneapolis, said.

Chomilo is referring to the estimated glomerular filtration rate (eGFR), which combines results from a blood test with factors such as age, sex, and race to calculate kidney function.

The eGFR weighed an input of "African American" as automatically indicating a higher concentration of serum creatinine than a non–African American patient on the basis of the unsubstantiated idea that Black people have more creatinine in their blood at baseline.

The calculator creates a picture of a Black patient who is not as sick as a White patient with the same levels of kidney failure. But race is based on the color of a patient's skin, not on genetics or other clinical datapoints.

"I often use my own example of being a biracial Black man: My father's family is from Cameroon, my mother's family is from Norway. Are you going to assign my kidneys or my lungs to my mom's side or my dad's side? That's not clear at all in the way we use race in medicine," Chomilo, an executive committee member on the section on minority health equity and inclusion at the American Academy of Pediatrics (AAP), said.

Long before the COVID-19 pandemic so publicly exposed the depths of inequality in morbidity and mortality in the United States, health advocates had been pointing out these disparities in tools used by medical professionals. But efforts to recognize that race is a poor proxy for genetics is in its infancy.

In May, the AAP published a policy statement that kicked off its examination of clinical guidelines and policies that include race as a biological proxy. A committee for the society is combing through each guideline or calculator, evaluating the scientific basis for the use of race, and examining whether a stronger datapoint could be used instead.

eGFR is perhaps the best example of a calculator that's gone through the process: Healthcare stakeholders questioned the use of race, and investigators went back to study whether race was really a good datapoint. It wasn't, and Chamilo's hospital joined many others in retiring the calculator.

But the eGFR is one of countless clinical tools ― from rudimentary algorithms to sophisticated machine-learning instruments ― that change the course of care in part on the basis of race in the same way datapoints such as weight, age, and height are used to inform decisions about patient management. But unlike race, height, weight, and age can be objectively measured. A physician either makes a guess, or a patient enters their race on a form. And while that can be useful on a population level, race does not equal genetics or any other measurable datapoint.

In a study published in June in JAMA Pediatrics, researchers reviewed 414 clinical practice guidelines from sources such as PubMed and MetaLib.gov. Almost 1 in 6 guidelines included race in an inappropriate way, such as by conflating race as a biological risk factor or establishing testing or treatment thresholds using race.

Waiting for Alternatives

The University of Maryland Medical System last year embarked on a project similar to the AAP initiative but within its own system. The first use of race to be eliminated was in the eGFR. The health system also recently removed the variable from a tool for diagnosing urinary tract infections (UTIs) in children younger than 2 years.

Part of that tool includes deciding to perform a catheterized urine test. If a doctor chose "White" as the race, the tool would recommend the test. If the doctor chose "Black," the tool would recommend to not test. Joseph Wright, MD, MPH, chief health equity officer at University of Maryland Medical System, said this step in the tool is based on the unproven assumption that young Black children had a lower likelihood of UTIs than their White peers.

"We simply want folks to not by default lob race in as a decision-making point when we have, with a little bit more scientific diligence, the ability to include better clinical variables," Wright, who is also an adjunct professor of health policy and management at the University of Maryland School of Public Health, College Park, said.

The developers of the UTI tool recently released a revised version that removes race in favor of two new medical datapoints: whether the patient has had a fever for over 48 hours, and whether the patient has previously had a UTI.

The process of reexamining tools, coming up with new datapoints, and implementing changes is not simple, according to Wright.

"This is just the baby step to fix the algorithms, because we're all going to have to examine our own house, where these calculators live, whether it's in a textbook, whether it's in an electronic health record, and that's the heavy lift," he said. "All sources of clinical guidance have to be scrutinized, and it's going to literally take years to unroot."

Electronic medical record vendor Cerner said it generally revises its algorithms after medical societies make changes, then communicates those fixes to providers.

Rebecca C. Winokur, MD, MPH, lead physician executive and health equity service line leader at Cerner, explained that if doctors ordered an eGFR a year ago and then another today, the results might be different because of the new code that eliminates race.

"The numbers are so different, how do you know that the patient may or may not have the same function?" Wikokur said.

Winokur said the company is trying to determine at which point a message should pop up in the records workflow that would inform clinicians that they may be comparing apples to cherries. The company also is reconsidering the use of race in tools that estimate the probability of a successful vaginal birth after prior cesarean delivery, a calculator that predicts the risk of urethral stones in patients with flank pain, and another that measures lung function to help diagnose pulmonary disease.

In addition to managing the logistics of removing race, health institutions also need buy-in from clinicians. At Mass General Brigham in Boston, Thomas Sequist, MD, MPH, chief medical officer, is leading a project to examine how the system uses race in calculators.

"People struggle mainly with, well, if we shouldn't use this calculator, what should we use, because we need a calculator. And that's a legitimate question," Sequist told Medscape. "If we're going to stop using this race-based calculator, I still need to know what dose of medication I give my patient. We're not going to pull any of these calculators until we have a safe and reliable alternative."

For each calculator, relevant specialty chiefs come to the table with Sequist and his team; current projects include examining bone density screenings and cardiac risk scores. A large part of the work is communicating the lack of science behind the inclusion of race as a variable.

"It's hard because these tools have been in existence for decades, and people are used to using them," Sequist said. "So this is a big-change management project."

Some clinicians also have difficulty discerning why their health system may stratify patient outcomes by race while providers are being told that race is being removed from the calculators they use every day. The key difference is that stratifying outcomes by race illuminates systemic problems that can be targeted by a health system.

For instance, if readmission rates are higher for Black patients overall after surgery, the reason might be that nurses are not delivering the same level of care to them as they are to non-Black patients, possibly because of hidden bias. Or, perhaps Black patients at a hospital have less access to transportation for follow-up appointments after surgery. The potential reasons can be investigated, and solutions can be created.

"If you look at a population level, what you're looking for is not for the evidence of race as a biological construct," Chomilo said. "You're looking for the impact of racism on populations, and that's the difference: It's racism, not race."

Lisa Gillespie is an editor at Medscape on the primary care vertical, with previous roles at Modern Healthcare, Side Effects Public Media, and Kaiser Health News.

For more news, follow Medscape on Facebook, Twitter, Instagram, and YouTube.

Credits:

Lead image: Stone/Getty Images

Image 1: Drasonallen Byrd

Image 2: The University of Maryland Medical System

Image 3: Mass General Brigham

Medscape Medical News © 2022 

Cite this: How Racist Is Your Algorithm? - Medscape - Jul 01, 2022.