Article Text
Abstract
Objective Medical billing data are an attractive source of secondary analysis because of their ease of use and potential to answer population-health questions with statistical power. Although these datasets have known susceptibilities to biases, the degree to which they can distort the assessment of quality measures such as colorectal cancer screening rates are not widely appreciated, nor are their causes and possible solutions.
Methods Using a billing code database derived from our institution’s electronic health records, we estimated the colorectal cancer screening rate of average-risk patients aged 50–74 years seen in primary care or gastroenterology clinic in 2016–2017. 200 records (150 unscreened, 50 screened) were sampled to quantify the accuracy against manual review.
Results Out of 4611 patients, an analysis of billing data suggested a 61% screening rate, an estimate that matches the estimate by the Centers for Disease Control. Manual review revealed a positive predictive value of 96% (86%–100%), negative predictive value of 21% (15%–29%) and a corrected screening rate of 85% (81%–90%). Most false negatives occurred due to examinations performed outside the scope of the database—both within and outside of our institution—but 21% of false negatives fell within the database’s scope. False positives occurred due to incomplete examinations and inadequate bowel preparation. Reasons for screening failure include ordered but incomplete examinations (48%), lack of or incorrect documentation by primary care (29%) including incorrect screening intervals (13%) and patients declining screening (13%).
Conclusions Billing databases are prone to substantial bias that may go undetected even in the presence of confirmatory external estimates. Caution is recommended when performing population-level inference from these data. We propose several solutions to improve the use of these data for the assessment of healthcare quality.
- primary care
- electronic health records
- quality measurement
- quality improvement
- performance measures
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Footnotes
Twitter @vivicality, @BenGlicksberg, @atulbutte
VAR and BSG contributed equally.
Presented at the Amercan College of Gastroenterology Annual Meeting (2018), and Digestive Diseases Week (2019)
Contributors VAR and BSG conceived the project, performed data extraction and analysis and drafted this manuscript. VAR, PA, EH-T and CW performed the chart review. AJB supervised the project and critically edited this manuscript.
Funding UCSF Bakar Computational Health Sciences Institute and the National Centre for Advancing Translational Sciences of the National Institutes of Health under award number UL1 TR001872. VAR was supported by the National Institute of Diabetes and Digestive and Kidney Disease of the National Institutes of Health grant under award number T32 DK007007-42.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.
Patient consent for publication Not required.
Ethics approval This study was approved by the University of California, San Francisco Institutional Review Board (#18–25166).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available. No data from this study are available for reuse.