Using Natural Language Processing to Identify US Veterans with Binge Eating Disorder

huntcopywriterAI and Robotics

Oct 24, 2013 (3 years and 11 months ago)

60 views



1,000 randomly selected instances were manually reviewed to
assess the tool’s ability to correctly identify patients with BED


BED patients were included if they had ≥1 diagnosis of BED


Inclusion Criteria



Adult (≥18 years) veterans


Index date defined as the first BED or EDNOS diagnosis


≥1 year of activity (e.g., office visit) before and after index date


Data for BMI calculation within 60 days of index date


No ICD-9 diagnosis code for other eating disorders was present


Analysis



Comorbidities were identified using ICD-9 codes and medication
use by NDC from prescription fill data


Pairwise comparisons of baseline characteristics for the BED
+EDNOS, BED-only, and EDNOS-only groups were performed
using t-tests and chi-square tests as appropriate


p-value <0.05 was considered significant


Binge eating disorder (BED) does not have a specific ICD-9
diagnosis code making it difficult to identify and perform studies
in patients with BED using healthcare databases
1,2


BED is one of a number of different eating disorders included
under the diagnosis code "Eating Disorder Not Otherwise
Specified" (EDNOS)
1,2



Electronic heath records (EHRs) have enabled greater research
access, including to narrative clinical notes
3-5



Natural language processing (NLP) includes methods to recognize
word and phrase patterns from text and has been used to identify
diseases with no or poorly documented diagnosis codes
5-9



Identify patients with clinician-diagnosed BED in the Department of
Veterans Affairs (VA) healthcare system using NLP


Compare characteristics of patients with both BED, identified by
NLP, and EDNOS, identified by ICD-9 code (BED+EDNOS), to those
with BED-only and EDNOS-only

Using Natural Language Processing to Identify
US Veterans with Binge Eating Disorder
Bellows BK,
1-2

DuVall
SL,
1-3

Ginter
T,
1,3

Kamauu
AWC,
4

Supina
D,
5

Hodgkins
P,
5

Erder
MH,
5

LaFleur
J
1-2
1. VA Salt Lake City Health Care System, Salt Lake City, Utah 2. University of Utah College of Pharmacy, Salt Lake City, Utah
3. University of Utah School of Medicine, Salt Lake City, Utah 4.
Anolinx
LLC, Salt Lake City, Utah 5. Shire, Wayne, Pennsylvania
OBJECTI VES
BACKGROUND
0%
10%
20%
30%
40%
50%
60%
70%
   * *  
   ^ ^  
   * *  
   ^
 
   ^  
BED+EDNOS
BED-only
EDNOS-only


Data Source



EHR data from a national population of US veterans who received
care in the VA system from 1/1/2000-12/21/2011


VA system primarily provides healthcare for former members of
the armed forces and is largely composed of older males


Patient identification and verification



EDNOS Patients


Identified by ICD-9 code (307.50) on ≥2 separate encounters


BED Patients


NLP tool was developed to identify patients with clinician-
diagnosed BED using narrative clinic notes from EHRs


Customized algorithm, similar to ConText
10
was used


Instances where variants of the term “binge eating disorder”
were mentioned in notes were analyzed


The NLP tool classified each instance as affirming, ruling out,
or considering a diagnosis of BED
Presented at the 29th International Conference on
Pharmacoepidemiology
&
Therapeutic Risk Management; August 25-28, 2013; Montreal, Canada
 
CONFLI CTS OF I NTEREST
Shi r e pr ov i ded f undi ng f or t he s t udy. DS, PH, and MHE ar e al l
empl oy ees of Shi r e and AK i s an owner of Anol i nx LLC.

METHODS
RESULTS


Fr om t he nat i onal VA s y s t em, t he NLP t ool was r un on 193,450
cl i ni cal not es ( f r om appr oxi mat el y 130,000 pat i ent s ) t hat
cont ai ned a ment i on of “ bi nge eat i ng” i n or der t o i dent i f y pat i ent s
wi t h a confir med BED di agnos i s


NLP cl as s i ficat i on accur acy was 91.8% compar ed t o human
r ev i ew


Tabl e 1: BED pat i ent i dent i ficat i on NLP measur ement
pr oper t i es

I nst ances
Revi ewed

Cor r ect l y
I dent i fied BED
Di agnosi s

BED Di agnosi s

Possi bl e
BED* * *

Yes

No

N

N

%*

N

%* *

N

%* *

N

%* *

1000

918

91.8%

731

90.7%

177

96.6%

92

91.3%

  * Wei ght ed av er age of t he s y s t em cl as s i ficat i on acr os s al l cl as s i ficat i ons
* * Per cent t hat human r ev i ewer agr eed wi t h t he s y s t em cl as s i ficat i on
* * * I ncl udes di f f er ent i al di agnos es
Pat i ent s wi t h medi cal encount er i n VA bet ween 1/1/00 and
12/31/11
N = 10.9 mi l l i on ( 1.7 bi l l i on cl i ni cal not es )
Pat i ent s wi t h BED
and EDNOS
N = 68
Appl y NLP
Pat i ent s wi t h BED confir med i n
char t
N = 1,487
Appl y s t udy el i gi bi l i t y cr i t er i a
N = 1,486
Pat i ent s wi t h ≥2 EDNOS di agnos i s
codes
N = 3,480
I dent i f y pat i ent s v i a I CD- 9
code 307.50
N = 3,405
Pat i ent s ≥18 y ear s
Pat i ent s wi t h ≥1 y ear of
pr e- i ndex and pos t -
i ndex dat e act i v i t y
N = 742
N = 2,033
N = 1,742
Pat i ent s wi t h BMI on
i ndex dat e ( ±60 day s )
N = 626
N = 1,422
Pat i ent s wi t h no I CD- 9
f or AN, BN, or ot her
eat i ng di s or der
N = 593
Pat i ent s wi t h onl y
BED
N = 525
Pat i ent s wi t h onl y
EDNOS
N = 1,354


Fi gur e 1: Pat i ent i dent i ficat i on  flowchar t


I ncl uded 525 pat i ent s wi t h BED- onl y, 68 wi t h bot h BED and
EDNOS, and 1354 wi t h EDNOS- onl y


Ther e wer e no s i gni ficant di f f er ences i n demogr aphi c
char act er i s t i cs bet ween t he BED+EDNOS gr oup and ei t her t he
BED- onl y or EDNOS- onl y gr oups


Compar ed t o EDNOS- onl y pat i ent s, BED- onl y pat i ent s wer e
y ounger ( mean: 48.7 v s. 49.8, p=0.04), mor e wer e mal e ( 73.0% v s.
62.8%, p<0.001), f ewer wer e whi t e ( 69.9% v s. 75.6%, p=0.01),
and had a hi gher BMI ( mean: 40.3 v s. 37.0, p<0.001)


NLP s y s t em was abl e t o i dent i f y pat i ent s wi t h BED i n t he VA wi t h
>90% accur acy


BED+EDNOS pat i ent s wer e s i mi l ar t o bot h t he BED- onl y and
EDNOS- onl y pat i ent s, but t her e wer e s i gni ficant di f f er ences
bet ween t he BED- onl y and EDNOS- onl y pat i ent s


Di f f er ences i ndi cat e t he need f or a s peci fic i dent i fier of BED i n
s t r uct ur ed dat a


Gener al i z abi l i t y may be l i mi t ed as t he s t udy r epr es ent s v et er ans wi t h BED
and EDNOS t r eat ed i n t he VA s y s t em, but may not r epr es ent t he gener al
popul at i on of pat i ent s wi t h t hes e di s eas es


Ot her i ns t i t ut i ons may us e phr as es and t er ms t o denot e a BED di agnos i s
t hat wer e not cons i der ed i n t hi s s t udy, but VA phy s i ci ans of t en wor k i n
heal t hcar e cent er s out s i de of t he VA s y s t em


Li ke al l NLP- bas ed ext r act i ons, t hi s i s l i mi t ed by what t he cl i ni ci an
i ncl udes or does not i ncl ude i n t he nar r at i v e medi cal not e. When t he
cl i ni ci an ment i oned di agnos i s of BED i n t he not e, t he NLP was abl e t o find
i t wi t h hi gh l ev el of accur acy, but i f i t was not ment i oned i n t he not e, i t, of
cour s e, coul d not be f ound


Fi nal l exi con onl y i ncl uded “ bi nge eat i ng di s or der ”, whi ch pot ent i al l y
r educed t he number of pat i ent s i dent i fied by NLP, but t hi s t er m i s s peci fic
and s houl d be cons i s t ent acr os s i ns t i t ut i ons
CONCLUSI ONS
1.

Font enel l e
LF, et al.
Br az
J
Med

Bi ol
Res. 2005;38( 11):1663- 7.
2.

v on
Loj ews ki
A, et al.
Eat

Wei ght

Di s or d
. 2012;17( 3):e185- 93.
3.

Saf r an
C. JAMA. 2001;285( 13):1766.
4.

Gunt er TD, et al. J
Med
I nt er net Res. 2005;7( 1):e3.
5.

Pakhomov
S, et al. Am J
Manag

Car e
. 2007;13( 6 Par t 1):281- 8.
6.

Mey s t r e
SM, et al.
Year b

Med

I nf or m
. 2008:128- 44.
7.

Jones M, et al. BMC
Med

I nf or m

Deci s
Mak. 2012;12:34.
8.

Gar v i n
JH, et al. J Am
Med

I nf or m

As s oc
. 2012;19( 5):859- 66.
9.

By r d RJ, et al.
I nt
J
Med

I nf or m
. 2013. doi: 10.1016/j.i j medi nf.2012.12.005.
10.

Har kema
H, et al. J
Bi omed

I nf or m
. 2009;Oct;42( 5):839- 51.
REFERENCES
LI MI TATI ONS


Fi gur e 2: Comor bi di t i es i n 12- mont h Post - i ndex Per i od


Fi gur e 3: Medi cat i on Use i n 12- mont h Post - i ndex Per i od
0%
20%
40%
60%
80%
   *  
   ^ ^  
   ^ ^  
   ^  
BED+EDNOS
BED- onl y
EDNOS- onl y
^Compar ed t o BED- onl y, p<0.05
^^Compar ed t o BED- onl y, p<0.001
* Compar ed t o BED+EDNOS, p<0.05
* * Compar ed t o BED+EDNOS, p<0.001
ABSTRACT
Backgr ound

Bi nge eat i ng di s or der ( BED) does not hav e a s peci fic I CD- 9 di agnos i s code but i s i ncl uded under t he non-
s peci fic code f or “ eat i ng di s or der, not ot her wi s e s peci fied” ( EDNOS).

Obj ect i ve

I dent i f y pat i ent s wi t h cl i ni ci an- di agnos ed BED i n t he Depar t ment of Vet er ans Af f ai r s ( VA) heal t hcar e s y s t em
and compar e t he bas el i ne char act er i s t i cs t o pat i ent s wi t h an EDNOS di agnos i s wi t hout BED ( EDNOS- onl y ).

Met hods

A nat ur al l anguage pr oces s i ng ( NLP) t ool was dev el oped t o i dent i f y pat i ent s wi t h cl i ni ci an- di agnos ed BED
us i ng nar r at i v e cl i ni c not es f r om el ect r oni c medi cal r ecor ds. Res ear cher s manual l y r ev i ewed 1,000 r ecor ds
t o as s es s t he t ool ’ s abi l i t y t o cor r ect l y i dent i f y pat i ent s wi t h BED. Pat i ent s wi t h a di agnos i s of EDNOS wer e
i dent i fied s epar at el y by I CD- 9 code. Adul t v et er ans wi t h BED or EDNOS bet ween 2000 and 2011 wer e
i ncl uded i f t hey had ≥1 y ear of act i v i t y bef or e and af t er i ndex dat e ( defined as t he fir s t ment i on of BED or
di agnos i s of EDNOS), BMI meas ur ement on i ndex dat e ( ±60 day s ), and no di agnos i s f or ot her eat i ng
di s or der s. Bas el i ne char act er i s t i cs wer e compar ed bet ween gr oups us i ng t - t es t s and chi - s quar e t es t s as
appr opr i at e wi t h a p- v al ue <0.05 cons i der ed s i gni ficant.

Resul t s

Fr om t he nat i onal VA s y s t em, t he NLP t ool was r un on 193,450 cl i ni cal not es t hat cont ai ned a ment i on of
bi nge eat i ng i n or der t o i dent i f y pat i ent s wi t h a confir med BED di agnos i s. NLP cl as s i ficat i on accur acy was
91.8% compar ed t o human r ev i ew. The s t udy i ncl uded 593 pat i ent s i dent i fied by NLP t o hav e a confir med
BED di agnos i s and 1354 pat i ent s wi t h EDNOS- onl y t hat met al l el i gi bi l i t y cr i t er i a. For BED and EDNOS- onl y
pat i ent s, mean ( SD) f or age was 48.7 ( 10.3) and 49.8 ( 12.5) y ear s ( p=0.04) and mean f or BMI was 40.2 ( 9.9)
and 37.0 ( 11.2) kg/m^2 ( p<0.001), r es pect i v el y. Mor e BED pat i ent s wer e mal e ( 72.2% v s. 62.8%, p<0.001)
and f ewer wer e whi t e ( 71.2% v s. 75.6%, p=0.06) compar ed t o EDNOS- onl y.
Concl usi on

Though t her e i s no I CD- 9 di agnos i s code f or BED, we wer e abl e t o i dent i f y pat i ent s wi t h BED i n t he VA
s y s t em us i ng NLP wi t h >90% accur acy. Compar ed t o EDNOS- onl y, BED pat i ent s t ended t o be y ounger,
mor e obes e, and wer e mor e l i kel y t o be mal e. Fut ur e wor k wi l l i ncl ude f ur t her anal y s i s of t hes e cohor t s.

To r ecei ve a PDF of t hi s post er on your mobi l e phone:



Downl oad a QR Code r eader t o y our mobi l e dev i ce


Scan t he code and get acces s t o cont ent.