Association between physicians’ experience after training and maternal obstetrical outcomes: cohort study
BMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f1596 (Published 28 March 2013) Cite this as: BMJ 2013;346:f1596
- Andrew J Epstein, research associate professor123,
- Sindhu K Srinivas, assistant professor34,
- Sean Nicholson, professor5,
- Jeph Herrin, assistant professor67,
- David A Asch, professor123
- 1Department of Veterans Affairs’ Center for Health Equity Research and Promotion, Philadelphia Veterans Affairs Medical Center, 3900 Woodland Avenue, Philadelphia, PA 19104, USA
- 2Division of General Internal Medicine, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, 423 Guardian Drive, Philadelphia, PA 19104, USA
- 3Leonard Davis Institute of Health Economics, University of Pennsylvania, 3641 Locust Walk, Philadelphia, PA 19104, USA
- 4Department of Obstetrics and Gynecology, Perelman School of Medicine, University of Pennsylvania, 421 Curie Boulevard, Philadelphia, PA 19104, USA
- 5Department of Policy Analysis and Management, College of Human Ecology, Cornell University, 102 MVR Hall, Ithaca, NY 14853, USA
- 6Section of Cardiovascular Medicine, Department of Medicine, Yale University School of Medicine, 333 Cedar Street, New Haven, CT 06510, USA
- 7Health Research and Educational Trust, 155 North Wacker, Suite 400, Chicago, IL 60606. USA
- Correspondence to: A J Epstein eandrew{at}mail.med.upenn.edu
- Accepted 27 February 2013
Abstract
Objective To assess the association between obstetricians’ years of experience after training and the maternal complications of their patients during their first 40 years of post-residency practice.
Design Retrospective cohort analysis.
Setting Obstetrical discharges from acute care hospitals in Florida and New York between academic years 1992 and 2009.
Population 6 704 311 deliveries performed by 5175 obstetricians.
Main outcome measure Three composite measures of maternal complication rates per physician year from vaginal and cesarean births separately and combined, adjusted for secular trends.
Results Obstetricians’ maternal complication rates declined during the first three decades after completion of residency. The improvement was largest in the first decade and diminished thereafter. For all deliveries, the change was −0.21 (95% confidence interval −0.23 to −0.19) percentage points per year in the first decade, −0.11 (−0.13 to −0.09) percentage points per year in the second decade, and −0.05 (−0.08 to −0.01) percentage points in the third decade (P<0.001 for second to first decade comparison; P=0.001 for third to second decade comparison). The patterns were comparable for cesarean deliveries and vaginal deliveries and across several sensitivity checks.
Conclusions Among obstetricians practicing in Florida and New York, those with more years of experience had fewer maternal complications. This association persisted over the first three decades of practice but diminished in magnitude.
Introduction
How does a physician’s clinical performance, as measured by patients’ outcomes, change over the course of a career? Intuitively, one might expect additional experience to improve performance early on, as physicians ascend a learning curve. In practice, patients, referring physicians, and health systems routinely seek out physicians with more experience because they are believed to perform better.
The experience-performance relation for physicians is not well understood, however, and the evidence is not well developed, partly because the potential underlying causal mechanisms are complex and difficult to observe. Physicians may improve by treating additional cases (“learning by doing”) or by observing other clinicians’ actions (“learning by watching”).1 Experience can be conceptualized and measured in multiple ways, such as by cumulative clinical volume over a career or by years of practice. Quantifying performance can likewise be problematic, as standards of care change over time and outcomes are affected by changes in patient mix as well as physicians’ experience. Moreover, the experience-performance relation might differ across settings and over the course of a career; as physicians near the end of their careers, performance might decline. That the results of existing studies vary greatly is thus not surprising.2
In this study, we examined the association between obstetricians’ years of experience after training and their performance, measured as a rate of maternal complications per physician year, during their first 40 years of post-residency practice.
Methods
Patients and physicians
We used data from Florida and New York all-payer hospital discharge databases for calendar years 1992 through 2010, covering all deliveries at all non-federal acute care hospitals. We selected these states because their data contain identifiers of physicians in addition to information typically found in discharge databases, such as patients’ demographics and codes for diagnosis and procedure. We identified cesarean deliveries by an ICD-9 (international classification of diseases, 9th revision, clinical modification) procedure code of 74 in any procedure field. We identified vaginal deliveries by diagnosis codes of 650 or 640.0x through 676.9x (where x is 1 or 2) in the principal diagnosis field and no indication of a cesarean delivery. We augmented the discharge data with information on each physician’s residency program, including specialty and graduation year, from the American Medical Association’s Physician Masterfile.
Because physicians typically graduate from residency programs around 1 July, we measured time according to an academic year calendar (starting 1 July and ending 30 June); we included deliveries in the 18 academic years between 1 July 1992 and 30 June 2010. We excluded deliveries if the delivering clinician did not have a valid state license number, was not identified in the Physician Masterfile as having completed an obstetrics-gynecology residency (typically reflecting delivery by a family practitioner or midwife), completed residency training before 1970 (to ensure that all deliveries were performed by a physician with no more than 40 years of experience), or was observed in only a single academic year.
Outcomes
We assessed physicians’ performance by using two measures of maternal complication. For cesarean deliveries, the measure included infection, hemorrhage, and other major operative and thrombotic complications; for vaginal deliveries, the measure included hemorrhage, severe laceration, infection, and thrombotic complications (appendix table A). We also examined maternal complications for all deliveries together by using a combination of these two measures. To better understand the potential role of changes in patterns of practice over the course of a physician’s career on performance, we examined secondary outcomes of physicians’ cesarean delivery rate, mean number of maternal comorbidities per patient, and total delivery volume. The unit of analysis was the physician year, and we measured all outcomes as rates per physician year.
Statistical analysis
We did a longitudinal analysis of the association between physicians’ experience, measured as years since completing an obstetrics and gynecology residency, and performance, measured by the number of maternal complications per year of practice, during the first 40 years of post-residency practice.
We explored two alternative specifications for years of experience. Firstly, to provide a fine grained picture, we specified years of experience as a set of 39 indicator variables and then plotted adjusted predictions and associated confidence intervals of the outcome for each year of experience. We used Sidak’s method to adjust the confidence intervals for multiple comparisons.3 Alternatively, to assess parsimoniously how the relation changed over years of experience, we specified experience as a set of four linear splines with distinct slopes for each decade of experience and then calculated average marginal effects of the impact of a one year increase in experience on outcomes in their native scale. We calculated confidence intervals for average marginal effects by the delta method.4
In unadjusted analyses, we estimated negative binomial regression models with annual delivery volume as the exposure to ensure that the model accounted for differences in delivery volume across physician years. Standard errors were robust to heteroskedasticity and also accounted for repeated observations per physician.
In adjusted analyses, we estimated negative binomial regression models that controlled for patient level and physician level characteristics and included a random effect for physician. Patient covariates, which were collapsed and measured as means per physician year, included age, race/ethnicity, having Medicaid or no insurance, weekend admission, and 31 maternal comorbidities (including previous cesarean delivery, fetal malpresentation, severe hypertension, multiple gestation, antepartum bleeding, herpes, macrosomia, unengaged head, maternal soft tissue disorder, preterm labor, congenital anomalies, oligohydramnios, and polyhydramnios (table 1⇓).5 6 7 Physicians’ characteristics included state (Florida versus New York), secular trend (modeled as a set of indicator variables for calendar year)5, annual cesarean delivery rate, and total annual number of deliveries (as the exposure).
Patients’ characteristics. Values are numbers (percentages)
For the practice pattern outcomes, we modeled the annual cesarean rate by using negative binomial regression with delivery volume as the exposure. We modeled total annual delivery volume and mean number of maternal comorbidities per patient with linear regression. These models controlled for state and secular trend and used robust standard errors that accounted for clustering.
An important analytic concern is “survivor bias,” which could inflate the association between experience and performance. Poorly performing physicians might be more likely to stop performing deliveries as they practiced longer, making it seem that more experience leads to better outcomes when in fact worse outcomes lead to less experience. We therefore repeated all analyses on the subset of physicians who performed any deliveries in academic year 2009.
We used two tailed tests with P<0.05 to establish statistical significance. We used Stata version 12.1 for analyses.
Results
During 1992-2010, 8 500 303 deliveries took place in Florida and New York, of which 7 337 250 (86%) were performed by physicians with valid state license numbers who completed obstetrics and gynecology residencies and 6 968 506 (82%) were performed in the 18 academic years between July 1 1992 and June 30 2010. After exclusion of physicians who finished their residency before 1970 or who were observed in only a single year, 6 704 311 (79%) deliveries were performed by 5175 physicians. The analytic dataset consisted of 54 736 physician year observations (table 2⇓). We repeated all analyses on the subset of 37 354 physician year observations from 3044 (59%) physicians who performed deliveries through academic year 2009 as a robustness check of survivor bias.
Sample descriptive statistics at physician year and physician levels. Values are numbers (percentages)
Figure 1⇓ depicts trends in unadjusted, annual, physician level maternal complication rates for all deliveries, cesarean deliveries, and vaginal deliveries based on the full sample of all physicians. As physicians gained experience, performance improved for all three outcomes. The largest changes occurred soon after completion of residency, and improvement tapered off later in physicians’ careers. The observed trends in complication rates were similar when we adjusted for patients’ and physicians’ characteristics and a physician random effect (fig 2⇓), as well as when we limited the sample to physicians who performed deliveries through academic year 2009 (appendix figures A and B).
Fig 1 Unadjusted annual maternal complication rates (with 95% confidence areas) by physician years of experience for all, cesarean, and vaginal deliveries. Estimates were generated from negative binomial regression. Heteroskedasticity robust standard errors also accounted for repeated observations per physician. Confidence areas were further adjusted for multiple comparisons with Sidak’s method
Fig 2 Adjusted annual maternal complication rates (with 95% confidence areas) by physicians’ years of experience for all, cesarean, and vaginal deliveries. Estimates were generated from negative binomial regression models that controlled for state, secular trend, physicians’ annual cesarean delivery rate and total number of deliveries, patients’ comorbidities, demographics, weekend admission, and a physician level random effect. Confidence areas were further adjusted for multiple comparisons with Sidak’s method
Results from regressions in which years of experience were specified as linear splines also indicate that physicians’ performance improved over the three decades after training. Based on the sample of all physicians, the adjusted maternal complication rate in the first year after residency was 15.0%. The adjusted maternal complication rate changed during a physician’s first decade of practice by −0.21 (95% confidence interval −0.23 to −0.19) percentage points per year, during the second decade by −0.11 (−0.13 to −0.09) percentage points per year, and during the third decade by −0.05 (−0.08 to −0.01) percentage points per year (P<0.001 for second to first decade comparison; P=0.001 for third to second decade comparison).
The improvements were similar in the cesarean delivery sample and the vaginal delivery sample. Among cesarean deliveries, maternal complications started at 13.7%; each year, they changed by an average of −0.19 (−0.23 to −0.16) percentage points during the first decade, by −0.11 (−0.14 to −0.08) percentage points during the second decade, and by −0.04 (−0.09 to 0.01) percentage points during the third decade (P<0.001 for second to first decade comparison; P=0.02 for third to second decade comparison). Among vaginal deliveries, starting at a complication rate of 15.6%, the average annual change was −0.23 (−0.25 to −0.20) percentage points during the first decade, −0.11 (−0.14 to −0.09) percentage points during the second decade, and −0.04 (−0.07 to −0.01) percentage points during the third decade (P<0.001 for second to first decade comparison; P=0.001 for third to second decade comparison).
The results were consistent across various specifications (appendix table B). Across all three samples, the unadjusted regressions yielded similar patterns of results, although the estimated reductions in maternal comorbidities were somewhat larger. Moreover, the results from both sets of models were similar when we limited the sample to physicians who performed deliveries through academic year 2009.
In contrast to the continuous improvement in outcomes, physicians’ delivery practice patterns seem to have changed little over their careers (fig 3⇓). Physician specific cesarean delivery rates began at around 30% after residency and increased slightly. Physicians’ average number of comorbidities per patient started at 0.83 and stayed flat. Total volume started at 112 deliveries per year, increased by 1.2 (95% confidence interval 0.1 to 1.8) deliveries per year in the first decade, and then decreased by 1.2 (0.6 to 1.7), 2.3 (1.5 to 3.1), and 3.3 (0.6 to 6.1) deliveries per year in the subsequent decades.
Fig 3 Physicians’ practice patterns by years of experience: adjusted outcomes with 95% confidence areas. Estimates were generated from negative binomial (cesarean delivery rate) or linear (comorbidity and delivery volume) regression with adjustment for state and secular trend. Cesarean delivery rate model also controlled for total number of deliveries per year. Heteroskedasticity robust standard errors accounted for repeated observations per physician. Confidence areas were further adjusted for multiple comparisons with Sidak’s method
Discussion
The results of this study show that the clinical performance of obstetricians, as measured by maternal complications, improved with years of experience. This improvement was steepest in a physician’s first decade of practice after completion of residency; it tapered off but remained significant through the second and third decades of practice. These effects are large given the duration of the cumulative exposure. After a decade of experience, for example, an obstetrician is expected to have a maternal complication rate that is roughly 2 percentage points lower than a newly trained obstetrician, which is sizeable compared with the predicted 15% complication rate in the first year. A second decade of experience is expected to confer an additional 1 percentage point reduction, and a third decade further lowers complications by about 0.5 percentage points. These results were consistent whether we examined all deliveries together or vaginal and cesarean deliveries separately, whether we included all obstetricians or only those who did not leave practice before the end of the study period, and whether we controlled for patients’ and physicians’ characteristics or not. Intuition suggests that performance ought to improve with experience, and greater experience is often cited as a reason to select one physician over another. However, past research has shown that performance improves with experience primarily when physicians first perform a procedure,8 9 10 11 if at all,12 and that performance degrades as physicians near the end of their careers.2 13 14 Here we have shown that, in the context of obstetrics, the observed effect of experience is substantial and persistent but diminishes over time.
We chose to examine obstetric care for several reasons: complications are more directly attributable to a single physician than in many other clinical areas; maternal patients are typically young and healthy, with fewer risk factors to account for when comparing outcomes across physicians; the practice of obstetrics, although constantly evolving, has focused over several decades on a very limited set of procedures, suggesting that the effects of experience would be more visible than in fields in which new technologies have changed the treatment mix; and we are aware of no other studies of learning curves in obstetrical care, one of the most common reasons for admission to hospital.
Possible explanations
Why such sustained improvement in performance occurs is not clear from our results. What does an obstetrician with, say, 15 years of experience know or do differently from a peer with five, 10, or even 14 years of experience? Although maternal complications were decreasing steadily during our study period,5 we accounted for this secular trend in our analysis. Better performing physicians probably stay in practice longer, but our results were robust when we limited analyses to physicians still performing deliveries in the last year of data. More experienced physicians might attract patients who are healthier in important ways, but we found no evidence of substantive changes in physicians’ mean number of maternal comorbidities per patient over the first 40 years of their careers—nor did we find any substantive changes in physicians’ cesarean section delivery rates or annual delivery volume over the same time period. Of course, we cannot rule out the possibility of unmeasured confounders correlated with both physicians’ experience and patients’ outcomes, including the possibility that maternal complications are coded more accurately (and thus more frequently) for less experienced physicians.
We think that these results more likely reflect a continued increase in obstetricians’ technical skill in performing cesarean and vaginal deliveries over three decades. This is a much longer timeframe for sustained improvement than others have found, even for relatively complex surgeries such as prostatectomy and heart surgery.8 9 10 11 12 Subsequent decline in technical skill might also explain the degradation in outcomes starting around 30 years after residency, as has been found in some other settings.2 13 14 Maternal complications may decline with experience for so long because obstetricians continue to get better at determining which women would benefit more from cesarean rather than vaginal delivery, such that outcomes in both settings improve. We are not aware of any empirical evidence on the association between experience and clinical decision making skills among early career physicians, however.
Implications for policy and practice
Our findings that new obstetricians continue to improve for three decades have potentially important implications. Future efforts should be devoted to exploring these relations more deeply, with the aim of identifying features that could be exploited for quality improvement. These results suggest that delivery outcomes might be improved by shifting cases from less experienced to more experienced obstetricians. Although we saw a diminishing marginal effect of experience on outcomes, obstetricians with three decades of experience still have fewer maternal complications than their peers near the start of their careers. In contrast, obstetrician-gynecologists commonly reduce their obstetrical practice in later years, shifting more to gynecology, so that deliveries are preferentially performed by obstetricians with less experience. In our data, about 43% of deliveries were performed by physicians with no more than 10 years of experience, whereas only 2% were performed by those with 31-40 years of experience.
However, even if more seasoned physicians could be motivated to delay or reduce their shift to gynecology, the wisdom of doing so is questionable. Echoing concerns about regionalizing cardiovascular care in high volume hospitals,15 a shift of deliveries to more experienced obstetricians might limit the acquisition of experience for later cohorts of physicians, disrupting whatever processes underlie the experience-performance associations we found. A better approach might be to understand how experience promotes quality, with the hope of accelerating it, and to identify which aspects of delivery care vary between more and less experienced obstetricians. Further attention should be devoted to approaches that might help to advance physicians along the learning curve after residency training, such as coaching.16
Similarly, these results might be interpreted as guidance for patients to select their obstetrics providers on the basis of years of experience. Our results do show that average outcomes are better among more experienced physicians, at least for maternal complications. Years of experience may thus be one of potentially many factors patients should consider when selecting an obstetrician. At the same time, experience is only a proxy for quality of provider. Heterogeneity surely exists among physicians in the experience-performance relation, which we have not explored here, such that experience is no guarantor of quality. Accordingly, patients would be served better by having access to direct measures of physicians’ performance.
Strengths and limitations of study
This study has several limitations. Firstly, we relied on administrative data, which do not capture all differences in severity between patients or a complete range of patients’ outcomes. Our data include considerable clinical information reflecting observable comorbidities, however, and the possibility of unobserved comorbidity is low in the setting of obstetrics, where the vast majority of patients are healthy young women. Importantly, we could not observe child related delivery outcomes or assess the potential trade-off between maternal and neonatal outcomes. We also could not follow mothers across multiple deliveries or adjust for parity. Secondly, we studied years of experience, not cumulative volume. Experience and cumulative volume are highly correlated and thus difficult to separate. Although cumulative volume may be more conceptually appealing because it better captures the notion that “practice makes perfect,” better outcomes might drive more volume.1 17 This concern may be particularly justified in obstetrics, where patients have time to search for obstetricians. Because of this, and because years of experience is measured more reliably in our data and in actual practice settings, we focused on experience. Thirdly, owing to limited availability of data on physicians, we studied only obstetricians from Florida and New York, so the generalizability of our findings may be limited.
This study also has several strengths. In contrast to other studies of physicians’ learning curves,9 10 11 12 we used detailed data from two large states comprising more than 6 million deliveries performed by more than 5000 obstetricians over an 18 year period. Moreover, our findings persist across delivery mode, in models that take account of survivor bias, and regardless of adjustment for patients’ characteristics.
Conclusions
We found a large and persistent effect of physicians’ years of experience on maternal outcomes in obstetrics, which decreased as physicians progressed in their careers. If these effects are seen in other fields, they have potentially broad implications for the structure and improvement of medical education and continuing medical education and for projections of the effect of changing physician workforce demographics on healthcare delivery and quality.
What is already known on this topic
-
The notion that physicians’ performance improves with experience has intuitive appeal
-
Limited empirical evidence suggests that performance improves with experience only when physicians first perform a procedure, if at all, and that performance degrades as physicians near the end of their careers
What this study adds
-
Among obstetricians practicing in Florida and New York, greater number of years of experience was associated with a lower rate of maternal complications over the first 30 years of practice
Notes
Cite this as: BMJ 2013;346:f1596
Footnotes
-
Contributors: All authors were responsible for conception and design. AJE acquired the data, which all authors analyzed and interpreted. AJE drafted the manuscript, which all authors critically revised. AJE and JH were responsible for statistical analyses. AJE and DAA supervised the study. DAA provided administrative, technical, and material support. All authors had full access to the data, including statistical results and tables, for this study and can take responsibility for the integrity of the data and the accuracy of the data analyses. AJE is the guarantor.
-
Funding: This research was performed without external financial or material support. The authors maintained full control over the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, and approval of the manuscript.
-
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work.
-
Ethical approval: The study (protocol No 814219) was deemed HIPAA-compliant and exempted from review by the University of Pennsylvania Institutional Review Board, Philadelphia, PA.
-
Data sharing: The statistical code is available from the corresponding author at eandrew{at}mail.med.upenn.edu.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.