Generative artificial intelligence models effectively highlight social determinants of health in doctors’ notes

The place we stay and work, our age, and the circumstances we grew up in can affect our well being and result in disparities, however these elements might be troublesome for clinicians and researchers to seize and handle.

A brand new research by investigators from Mass Basic Brigham demonstrates that enormous language fashions (LLMs), a sort of generative artificial intelligence (AI), might be skilled to robotically extract info on social determinants of well being (SDoH) from clinicians’ notes which may increase efforts to determine sufferers who could profit from useful resource help.

Findings printed in npj Digital Medicine present that the finely tuned fashions may determine 93.8 p.c of sufferers with opposed SDoH, whereas official diagnostic codes included this info in solely 2 p.c of circumstances. These specialised fashions had been much less liable to bias than generalist fashions equivalent to GPT-4.

“Our aim is to determine sufferers who may benefit from useful resource and social work help and draw consideration to the under-documented affect of social elements in health outcomes,” stated corresponding writer Danielle Bitterman, MD, a school member within the Synthetic Intelligence in Medication (AIM) Program at Mass Basic Brigham and a doctor within the Division of Radiation Oncology at Brigham and Girls’s Hospital.

“Algorithms that may cross main medical exams have acquired a number of consideration, however this isn’t what docs want within the clinic to assist take higher care of sufferers every day. Algorithms that may discover what docs could miss within the ever-increasing quantity of medical information can be extra clinically related and due to this fact extra highly effective for bettering well being.”

Well being disparities are broadly linked to SDoH, together with employment, housing, and different non-medical circumstances that affect medical care. For instance, the gap a cancer patient lives from a serious medical heart or the help they’ve from a accomplice can considerably affect outcomes. Whereas clinicians could summarize related SDoH of their go to notes, this important info isn’t systematically organized within the digital well being document (EHR).

To create LMs able to extracting info on SDoH, the researchers manually reviewed 800 clinician notes from 770 sufferers with most cancers who acquired radiotherapy on the Division of Radiation Oncology at Brigham and Girls’s Hospital. They tagged sentences that referred to a number of of six pre-determined SDoH: employment standing, housing, transportation, parental standing (if the affected person has a toddler below 18 years previous), relationships, and presence or absence of social help.

Utilizing this “annotated” dataset, the researchers skilled present LMs to determine references to SDoH in clinician notes. They examined their fashions utilizing 400 clinic notes from sufferers handled with immunotherapy at Dana-Farber Most cancers Institute and sufferers admitted to the important care models at Beth Israel Deaconess Medical Heart.

The researchers discovered that fine-tuned LMs, particularly Flan-T5 LMs, may constantly determine uncommon references to SDoH in clinician notes. The “studying capability” of those fashions was restricted by the rarity of SDoH documentation within the coaching set, the place the researchers discovered that solely 3 p.c of sentences in clinician notes contained any point out of SDoH.

To handle this challenge, the researchers used ChatGPT, one other LM, to supply a further 900 artificial examples of SDoH sentences that could possibly be used as an additional coaching dataset.

A serious criticism of generative AI fashions in well being care is that they will probably perpetuate bias and widen well being disparities. The researchers discovered that their fine-tuned LM was much less probably than OpenAI’s GPT-4, a generalist LM, to vary its willpower about an SDoH based mostly on people’ race/ethnicity and gender.

The researchers state that it’s obscure how biases are shaped and deconstructed—each in human and pc fashions. Understanding the origins of algorithmic bias is an ongoing endeavor for the researchers.

“If we do not monitor algorithmic bias once we develop and implement large language models, we may make present well being disparities a lot worse than they at present are,” Bitterman stated. “This research demonstrated that fine-tuning LMs could also be a technique to scale back algorithmic bias, however extra analysis is required on this space.”