Contributions
Abstract: P885
Type: Poster
Abstract Category: Clinical aspects of MS - 5 Epidemiology
Background: Identifying relapse in databases can be difficult due to different symptoms occurring during the event. Use of natural language processing (NLP) to identify relapses in electronic health record (EHR) would be useful for studies of patient outcomes and drug effectiveness.
Objective: Develop and validate operational EHR-based algorithms for relapse identification among relapsing-remitting multiple sclerosis (RRMS) patients.
Methods: EHR data from an integrated delivery health system (IDN) (2010-2014) were queried for study inclusion criteria: multiple sclerosis (MS) diagnosis, ≥18 years old, ≥1 year baseline history, and no other demyelinating diseases. RRMS patients without progressive MS disease were identified by NLP. The EHR-based algorithm identified relapses from unstructured clinical notes using results from NLP for key words and phrases signifying MS relapse. Option 1 required a mention of multiple sclerosis with variations of words relating to relapse (e.g., relapse, exacerbation, attack). Option 2 required a mention of multiple sclerosis and terms or phrases for clinician-documented indication of MS relapse by MS-related symptoms (e.g., fatigue, spasticity, plasmapheresis) mentioned in a clinical note. Random sample medical chart reviews were used for algorithm validation and positive predictive value (PPV) calculations.
Results: Of the 837 RRMS patients identified, 651 patients were identified by the algorithm as having at least one relapse. Options 1 and 2 identified 478 and 579 patients, respectively. PPV (95% CI) for Options 1 and 2 were 36.8% (24.8%-50.7%) and 35.6% (25.0%-47.8%), respectively. During NLP-based chart review, Option 1 algorithm identified several false positive relapses due to notes indicating terms in a context unrelated to MS (e.g., heart attack, asthma exacerbation). Option 2 identified false positive relapses due to the recorded symptoms referring to presence at baseline and not necessarily new symptoms indicating a relapse (e.g., indication that patient has fatigue and MS is currently stable, noted chronic stable fatigue, description of symptoms from several months prior that have since improved).
Conclusions: Both options of the algorithm performed poorly when identifying relapses. The use of NLP alone was not sufficient for identifying relapses among RRMS patients in this IDN. Supplementing the EHR-based algorithm with a claims-based algorithm may improve the ability of the algorithm to identify relapses.
Disclosure: This study was funded by Merck KGaA.
Aaron W.C. Kamauu received a research grant from Merck KGaA.
Chi Thi Le Truong is an employee of MedCodeWorld.
Hoa Van Le is an employee of PAREXEL and a stockholder of GlaxoSmithKline, and was a Harry Guess-Merck merit scholarship recipient.
Camelia M Graham is an employee of PAREXEL.
John R. Holmen has nothing to disclose.
Christopher L. Fillmore has nothing to disclose.
Meritxell Sabidó-Espin is an employee of Merck KGaA.
Schiffon L. Wong is an employee EMD Serono.
Abstract: P885
Type: Poster
Abstract Category: Clinical aspects of MS - 5 Epidemiology
Background: Identifying relapse in databases can be difficult due to different symptoms occurring during the event. Use of natural language processing (NLP) to identify relapses in electronic health record (EHR) would be useful for studies of patient outcomes and drug effectiveness.
Objective: Develop and validate operational EHR-based algorithms for relapse identification among relapsing-remitting multiple sclerosis (RRMS) patients.
Methods: EHR data from an integrated delivery health system (IDN) (2010-2014) were queried for study inclusion criteria: multiple sclerosis (MS) diagnosis, ≥18 years old, ≥1 year baseline history, and no other demyelinating diseases. RRMS patients without progressive MS disease were identified by NLP. The EHR-based algorithm identified relapses from unstructured clinical notes using results from NLP for key words and phrases signifying MS relapse. Option 1 required a mention of multiple sclerosis with variations of words relating to relapse (e.g., relapse, exacerbation, attack). Option 2 required a mention of multiple sclerosis and terms or phrases for clinician-documented indication of MS relapse by MS-related symptoms (e.g., fatigue, spasticity, plasmapheresis) mentioned in a clinical note. Random sample medical chart reviews were used for algorithm validation and positive predictive value (PPV) calculations.
Results: Of the 837 RRMS patients identified, 651 patients were identified by the algorithm as having at least one relapse. Options 1 and 2 identified 478 and 579 patients, respectively. PPV (95% CI) for Options 1 and 2 were 36.8% (24.8%-50.7%) and 35.6% (25.0%-47.8%), respectively. During NLP-based chart review, Option 1 algorithm identified several false positive relapses due to notes indicating terms in a context unrelated to MS (e.g., heart attack, asthma exacerbation). Option 2 identified false positive relapses due to the recorded symptoms referring to presence at baseline and not necessarily new symptoms indicating a relapse (e.g., indication that patient has fatigue and MS is currently stable, noted chronic stable fatigue, description of symptoms from several months prior that have since improved).
Conclusions: Both options of the algorithm performed poorly when identifying relapses. The use of NLP alone was not sufficient for identifying relapses among RRMS patients in this IDN. Supplementing the EHR-based algorithm with a claims-based algorithm may improve the ability of the algorithm to identify relapses.
Disclosure: This study was funded by Merck KGaA.
Aaron W.C. Kamauu received a research grant from Merck KGaA.
Chi Thi Le Truong is an employee of MedCodeWorld.
Hoa Van Le is an employee of PAREXEL and a stockholder of GlaxoSmithKline, and was a Harry Guess-Merck merit scholarship recipient.
Camelia M Graham is an employee of PAREXEL.
John R. Holmen has nothing to disclose.
Christopher L. Fillmore has nothing to disclose.
Meritxell Sabidó-Espin is an employee of Merck KGaA.
Schiffon L. Wong is an employee EMD Serono.