THE SHORTAGE OF PRIMARY CARE

ORIGINAL CONTRIBUTION Critical Factors for Designing Programs to Increase the Supply and Retention of Rural Primary Care Physicians Howard K. Rabinowitz, MD James J. Diamond, PhD Fred W. Markham, MD Nina P. Paynter, BS Context The Physician Shortage Area Program (PSAP) of Jefferson Medical College (Philadelphia, Pa) is one of a small number of medical school programs that addresses the shortage of rural primary care physicians. However, little is known regarding why these programs work. Objectives To identify factors independently predictive of rural primary care supply and retention and to determine which components of the PSAP lead to its outcomes. Design Retrospective cohort study. Setting and Participants A total of 3414 Jefferson Medical College graduates from the classes of 1978-1993, including 220 PSAP graduates. Main Outcome Measures Rural primary care practice and retention in 1999 as predicted by 19 previously collected variables. Twelve variables were available for all classes; 7 variables were collected only for 1978-1982 graduates. Results Freshman-year plan for family practice, being in the PSAP, having a National Health Service Corps scholarship, male sex, and taking an elective senior family practice rural preceptorship (the only factor not available at entrance to medical school) were independently predictive of physicians practicing rural primary care. For 1978-1982 graduates, growing up in a rural area was the only additionally collected independent predictor of rural primary care (odds ratio [OR], 4.0; 95% CI, 2.1-7.6; P.001). Participation in the PSAP was the only independent predictive factor of retention for all classes (OR, 4.7; 95% CI, 2.0-11.2; P.001). Among PSAP graduates, taking a senior rural preceptorship was independently predictive of rural primary care (OR, 2.5; 95% CI, 1.3-4.7; P=.004). However, non-psap graduates with 2 key selection characteristics of PSAP students (having grown up in a rural area and freshman-year plans for family practice) were 78% as likely as PSAP graduates to be rural primary care physicians, and 75% as likely to remain, suggesting that the admissions component of the PSAP is the most important reason for its success. In fact, few graduates without either of these factors were rural primary care physicians (1.8%). Conclusions Medical educators and policy makers can have the greatest impact on the supply and retention of rural primary care physicians by developing programs to increase the number of medical school matriculants with background and career plans that make them most likely to pursue these career goals. Curricular experiences and other factors can further increase these outcomes, especially by supporting those already likely to become rural primary care physicians. JAMA. 2001;286:1041-1048 www.jama.com THE SHORTAGE OF PRIMARY CARE physicians in rural areas has been one of the most intractable US health policy problems of the past century. 1-5 With 20% of the US population residing in rural areas, but only 9% of physicians practicing there, 2 people living in rural areas constitute one of the largest underserved US populations. Rural residents are older, sicker, poorer, have less education, and are more likely to be uninsured than are urban residents. In addition, most of the federally designated physician shortage areas are in rural areas. 6 Despite the dramatic increase in the overall national supply of physicians during the past few decades, rural areas continue to be underserved. 3 Unfortunately, fewer than 3% of recent medical school graduates express plans to practice in rural areas or small towns. 7 Within this context, policy makers and educators continue to face the serious challenge of finding the most effective, and least costly, ways to increase the supply and retention of rural primary care physicians. Although the role of medical schools in addressing this problem has been controversial for several decades, 4,8-10 the outcomes of 7 US medical school programs developed to increase the supply of rural primary care physicians have been encouraging. 11-18 One of these programs, the Physician Shortage Area Program (PSAP) of Jefferson Medical College (JMC), a special admissions Author Affiliation: Jefferson Medical College, Thomas Jefferson University, Philadelphia, Pa. Corresponding Author and Reprints: Howard K. Rabinowitz, MD, Jefferson Medical College, Thomas Jefferson University, Suite 401, 1015 Walnut St, Philadelphia, PA 19107 (e-mail: howard.rabinowitz@mail.tju.edu). 2001 American Medical Association. All rights reserved. (Reprinted) JAMA, September 5, 2001 Vol 286, No. 9 1041

and educational program designed to increase the supply of rural family physicians, has been in existence since 1974. The PSAP recruits and selectively admits academically qualified students who grew up or lived in a rural area or small town, and who also have a firm commitment to practice the specialty of family practice in a similar area. The PSAP matriculants (averaging 14 per year, 6.4% of JMC students) are provided with faculty advisors in the Department of Family Medicine throughout their medical school career, receive a small amount of additional financial aid (predominantly as repayable loans), and meet regularly with a family medicine faculty advisor to discuss issues related to rural family practice. During their third year, PSAP students are expected to take their required family medicine clerkship at a rural or small town location. Senior PSAP students are required to take their outpatient subinternship in family medicine, frequently electing a rural private office preceptorship. After graduation, PSAP graduates are expected to complete a family practice residency program and practice rural family medicine, although there is no formal mechanism to enforce compliance. Outcome studies have shown that the PSAP has been successful in (1) increasing the percentage of rural family physicians ( 8 times that of their peers), (2) retaining rural family physicians (87% retention rate over 5-10 years in practice), and (3) having a major impact on the rural physician workforce, despite its small size (accounting for 21% of rural family physicians in Pennsylvania who graduated from 1 of the 7 allopathic medical schools in the state, even though PSAP students represent only 1% of graduates from those schools). 18-20 Despite these positive outcomes of the PSAP and other models, each of these comprehensive programs differs substantially in its admissions, curricular, financial, and other components. However, there are no data available regarding which components are most responsible for their success. Similarly, although prior research has identified background, educational, financial, and career factors that are individually correlated with rural primary care, 2,5,16,21-24 this decision is multifactorial and complex. Few studies have evaluated more than 1 variable at a time, their relative importance and interdependence, or their relationship to the critically important issue of retention. The extent to which these factors influence the supply and retention of rural primary care physicians is critical to the design of interventions to address this problem. Our previous work in this area included a study to determine the independent predictors of JMC graduates practicing in rural Pennsylvania (any specialty), 24 and a retrospective survey of US medical school graduates from 1983-1984 to identify independent predictors of primary care physicians practicing in underserved and rural areas. 21,25 To our knowledge, however, there is no previous study that identifies the independent predictors for rural primary care using prospectively collected data. In addition, no study has determined these predictors for PSAP graduates, or has compared their outcomes with their peers with similar background characteristics. Therefore, we undertook this study to identify those factors that are independently predictive of rural primary care supply and retention and to determine which components of the PSAP are responsible for its positive outcomes. METHODS The study population consisted of the 3414 physicians who graduated from JMC beween 1978 and 1993. Graduates were considered to be practicing in a rural area if their current address, obtained from the JMC Alumni Association in 1999, was in a county designated as a non Standard Metropolitan Statistical Area (non-smsa) in either 1978 or 1993. 18-20 Since physicians graduated from JMC throughout this period, the SMSA designations were used spanning those years. To determine whether inclusion of those counties that changed designation during this period appeared to affect our results, review of JMC graduates indicated that almost all (95.7%) were practicing in counties that had not changed their SMSA designation during that time. Data on physician specialty were obtained from the Jefferson Longitudinal Study of Medical Education, an ongoing longitudinal cohort study tracking JMC graduates for more than 30 years. 26,27 The Jefferson Longitudinal Study contains certification information from the American Board of Medical Specialties and self-reported specialty data from the American Medical Association Physician Masterfile (1999). As in our prior studies, 18-20 practice specialty was considered to be that in which board certification was obtained. For graduates who were board certified in 2 or more specialties, or who were not board certified in any specialty, primary self-reported specialty data from the American Medical Association Physician Masterfile were used. Primary care physicians included those practicing family medicine, general internal medicine, and general pediatrics. For each JMC graduate, 19 predictor variables that had prospectively been collected during medical school and included in the Jefferson Longitudinal Study database were selected for this study, based on prior literature. 2,18-20,22-24 These variables were related to demographic background (sex, growing up in a rural area or small town, father s education, mother s education, and age entered medical school); premedical background (attended college in a rural area or small town, undergraduate science grade point average, Medical College Admission Test Biology and Reading scores); career plans as reported on a matriculation questionnaire (plans for family practice, plans to practice in a rural area or small town, expected length of work week after training [in hours], anticipated percentage of low-income patients in own practice); medical school programs and curricula (PSAP, National Health Service Corps [NHSC] scholarship program, rural or small town location of 1042 JAMA, September 5, 2001 Vol 286, No. 9 (Reprinted) 2001 American Medical Association. All rights reserved.

required third-year family medicine clerkship, elective senior-year rural family medicine preceptorship); and economic issues (freshman-year plan of expected income at the peak of their professional career, total medical school debt). For 7 of these variables (growing up in a rural area, father s education, mother s education, attending college in a rural area, freshman-year plan to work in a rural area, expected length of work week after training, and anticipated percentage of low-income patients in own practice), data had been collected only for graduates from the classes of 1978-1982. All other variables were available for graduates from all classes (1978-1993). Three continuous variables were dichotomized based on prior research of expected peak income, 28 medical school debt, 29 and the anticipated percentage of lowincome patients in one s practice. 25 When no such guidance was available, continuous variables were dichotomized at the median. Analyses were done using SAS software (Version 8.0, SAS Institute Inc, Cary, NC). Univariate analyses were performed to assess the relationship of these 19 predictive variables to the outcome of rural primary care practice, using relative risks with 95% confidence intervals (CIs). Independent predictors of rural primary care were then determined, using a multiple logistic regression model (backward selection method) for those variables that were univariately significant, and which were available for graduates from all 16 classes (1978-1993). To determine the importance of the 7 additional variables collected only for 1978-1982 graduates, a second multivariate logistic model that also included those variables was similarly developed for graduates from these 5 classes. A similar method was used to identify those factors predictive of retention of rural primary care physicians. For this analysis, the study population included 1978-1986 graduates who were previously determined to be practicing rural primary care in 1986 (for 1978-1981 graduates) or in 1991 (for 1982-1986 graduates). 19,20 Those graduates still practicing rural primary care in 1999 (ie, 8-13 years after initially located in practice) were compared with graduates no longer in rural primary care. To determine which components of the PSAP were responsible for its positive outcomes, univariate and multivariate analyses were similarly calculated for the subset of PSAP graduates. The relative importance of the admissions component of the PSAP was specifically analyzed, since prior experience and research suggested the potential importance of factors available at the time of matriculation to medical school, 18,24,25 and because these could be temporally separated from other factors. This was assessed by comparing rural primary care outcomes of PSAP graduates with non-psap graduates who had 2 previously reported key characteristics used to select PSAP students (ie, having grown up in a rural area and freshman-year plan for family practice). 24 Because data on growing up in a rural area were only collected for 1978-1982 graduates, these analyses were limited to those classes. RESULTS Data regarding both practice location and specialty were available for 3365 (98.6%) of the 3414 JMC graduates in the study, including 218 (99.1%) of 220 PSAP graduates. Overall, 5.6% (187/ 3365) of these graduates were practicing rural primary care in 1999, with a slightly lower percentage of rural primary care physicians in the 5 most recent classes (5.2%). Of all rural primary care physicians, 68.4% (128/ 187) were family physicians, 23.0% (n=43) were general internists and 8.6% (n=16) were general pediatricians. Predictors of Rural Primary Care Practice TABLE 1 summarizes the results of the univariate analyses for the 19 predictor variables. Ten factors were univariately related to rural primary care, 7 of which had been collected for graduates from all 16 classes. The logistic model using these 7 factors for 2457 graduates showed that 5 were independently predictive of practicing rural primary care: freshman-year plan for family practice, being in the PSAP, having an NHSC scholarship, male sex, and selecting a senior-year rural family practice preceptorship (the only factor not available at entrance to medical school) (TABLE 2). For the 67 graduates with 4 or more of these factors, 34.3% were rural primary care physicians compared with 3.0% of the 495 graduates with no factors (relative risk, 11.3, 95% CI, 6.2-20.6; P.001). For non-psap graduates, the pattern of results was similar. To understand the importance of the 3 additional variables that were univariately significant but had been collected only from 1978-1982, a separate logistic model was calculated for the 745 graduates from these 5 classes who had data for all 10 univariately significant factors. In this model, 1 of these 3 additional variables, growing up in a rural area, was independently predictive of rural primary care (odds ratio [OR], 4.0; 95% CI, 2.1-7.6; P.001), while the other 2, freshman-year plan to work in a rural area (OR, 1.7; 95% CI, 0.8-3.6; P=.15) and plan to care for low-income patients (OR, 1.4; 95% CI, 0.8-2.7; P=.28), were not. Predictors of Rural Primary Care Retention Of the 144 JMC graduates from the classes of 1978-1986 who were in rural primary care practice when initially studied, 86 (59.7%) were still rural primary care physicians in 1999. Being in the PSAP, selecting a rural preceptorship, growing up in a rural area, and attending college in a rural area were univariately related to retention (TABLE 3). Data on PSAP participation and on having a rural preceptorship were available for the 144 graduates from all 9 classes. In the multivariate model, participation in the PSAP was independently predictive of retention (OR, 4.7; 95% CI, 2.0-11.2; P.001), while taking a rural preceptorship was 2001 American Medical Association. All rights reserved. (Reprinted) JAMA, September 5, 2001 Vol 286, No. 9 1043

not (OR, 1.2; 95% CI, 0.5-2.8; P=.74). To determine the importance of the other 2 variables that had been collected only for the subset of 1978-1982 graduates, a second multivariate analysis was done for the 76 graduates from these 5 classes who had data for all 4 factors. In this model, 1 of these 2 additional variables, attending a rural college, was independently predictive of retention (OR, 7.2; 95% CI, 1.8-28.2; P=.005), while the other, growing up in a rural area, was not statistically significant (OR, 2.8; 95% CI, 1.0-8.4; P=.06). PSAP Graduates For PSAP graduates, taking a rural preceptorship and having an NHSC scholarship were univariately predictive of rural primary care practice. When these 2 factors were included in a logistic model (218 graduates), selecting a rural preceptorship was independently predictive of practicing primary care in a rural area (OR, 2.5; 95% CI, 1.3-4.7; P=.004). Having an NHSC scholarship barely missed our cutoff for significance (OR, 2.5; 95% CI, 1.0-6.2; P=.0503). Unlike the results for all JMC graduates, male sex was not related to rural primary care for PSAP graduates (25.6% male, 22.0% female; P=.61). Non-PSAP graduates who had grown up in a rural area and had a freshmanyear plan for family practice were 78% as likely to practice rural primary care (16/80 [20.0%]) as PSAP graduates (18/70 [25.7%]) (FIGURE 1). In contrast, few non-psap graduates without these 2 factors were rural primary care physicians (7/395 [1.8%]). Similarly, non-psap graduates with these 2 characteristics were 75% as likely to remain in rural primary care (12/17 [70.6%]) as PSAP graduates (17/18 [94.4%]), while non-psap graduates with neither of these characteristics were much less likely to remain (5/13 [38.5%]) (FIGURE 2). COMMENT The most consistent finding from this study was the powerful impact of background and career plans at the time of admission to medical school on future Table 1. Results of Univariate Analysis of Predictors of Rural Primary Care for Jefferson Medical College Graduates, 1978-1993* No. of Graduates No. (%) Practicing Rural Primary Care Relative Risk (95% Confidence Interval) P Value Variable Background/Demographic Sex Male 2515 152 (6.0) 1.5 (1.0-2.1).04 Female 850 35 (4.1) Grew up in a rural area Yes 314 39 (12.4) 3.5 (2.1-5.7).001 No 640 23 (3.6) Father s education No college 248 21 (8.5) 1.5 (0.9-2.5).13 Some college 697 40 (5.7) Mother s education No college 462 30 (6.5) 1.0 (0.6-1.7).96 Some college 468 30 (6.4) Age entered medical school, y 22 1193 76 (6.4) 1.2 (0.9-1.7).13 22 2172 111 (5.1) Premedical College location Rural 341 27 (7.9) 1.4 (0.9-2.2).19 Not rural 612 35 (5.7) Undergraduate science grade point average 3.51 1693 95 (5.6) 1.0 (0.8-1.3).93 3.51 1643 91 (5.5) Biology MCAT score 10 1491 76 (5.1) 0.9 (0.6-1.1).29 10 1870 111 (5.9) Reading MCAT score 9 1502 71 (4.7) 0.8 (0.6-1.0).06 9 1857 116 (6.2) Career Plans as a Freshman Family practice Yes 615 88 (14.3) 3.9 (2.9-5.2).001 No 2058 76 (3.7) Practice location Rural 325 42 (12.9) 4.0 (2.4-6.6).001 Not rural 614 20 (3.3) Hours plan to work per week after training 60 280 17 (6.1) 0.9 (0.5-1.5).65 60 654 45 (6.9) Low-income patients in practice, % 40 232 22 (9.5) 1.6 (1.0-2.7).05 40 695 40 (5.8) Medical School Programs/Curricula PSAP participant Yes 218 54 (24.8) 5.9 (4.4-7.8).001 No 3144 133 (4.2) NHSC scholarship Yes 97 23 (23.7) 4.7 (3.2-7.0).001 No 3268 164 (5.0) Family practice clerkship location Rural 905 86 (9.5) 2.3 (1.8-3.1).001 Not rural 2460 101 (4.1) (continued) 1044 JAMA, September 5, 2001 Vol 286, No. 9 (Reprinted) 2001 American Medical Association. All rights reserved.

Table 1. Results of Univariate Analysis of Predictors of Rural Primary Care for Jefferson Medical College Graduates, 1978-1993 (cont)* No. of Graduates No. (%) Practicing Rural Primary Care Relative Risk (95% Confidence Interval) P Value Variable Medical School Programs/Curricula (cont) Rural preceptorship Yes 345 61 (17.7) 4.2 (3.2-5.6).001 No 3020 126 (4.2) Economic Expected peak income# $131 450 1744 88 (5.0) 0.7 (0.5-1.0).03 $131 450 1059 75 (7.1) Medical school debt** $84 600 383 24 (6.3) 1.1 (0.8-1.7).52 $84 600 2982 163 (5.5) *Data for the 12 predictor variables collected from all classes were available for between 2673 (78.3%) and 3365 (98.6%) of the graduates; the 7 additional variables that were only collected from the classes of 1978-1982 were available for between 927 (84.8%) and 954 (87.3%) of the 1093 graduates from those classes. MCAT indicates Medical College Admissions Test; PSAP, Physician Shortage Area Program; and NHSC, National Health Service Corps. Available only for the classes of 1978-1982. Dichotomized at the median. MCAT Science subtest score for tests taken in 1977 and before, or the MCAT Biology score for tests taken in 1978 and after, normalized to the latter MCAT score range by mean and SD, and using the average of the first 2 test scores if 2 or more tests were taken. MCAT Verbal score for tests taken in 1977 and before, or the MCAT Reading score for tests taken later, normalized to the latter MCAT score range by mean and SD, and using the average of the first 2 test scores if 2 or more tests were taken. DIchotomized based on prior research. 25 #Dichotomized based on prior research at $100 000 (1989 dollars) 28 ; adjusted to 1998 dollars using the consumer price index. **Dichotomized based on prior research at $75 000 (1993 dollars) 29 ; adjusted to 1998 dollars using the consumer price index. Table 2. Results of Multivariate Analysis of Predictors of Rural Primary Care for Jefferson Medical College Graduates, 1978-1993* Odds Ratio Variable (95% Confidence Interval) P Value Freshman-year plans for family practice 2.3 (1.6-3.5).001 PSAP participant 2.5 (1.5-4.1).001 NHSC scholarship 2.6 (1.3-5.1).006 Rural preceptorship 2.4 (1.6-3.7).001 Male sex 1.8 (1.1-2.7).01 Expected peak income $131 450 0.8 (0.5-1.1).18 Rural family practice clerkship location 1.2 (0.8-1.8).35 *PSAP indicates Physician Shortage Area Program; NHSC, National Health Service Corps. The number of subjects is 2457. Dichotomized based on prior research at $100 000 (1989 dollars) 28 ; adjusted to 1998 dollars using the consumer price index. rural primary care practice and retention. Most of the factors independently predictive of rural primary care practice (growing up in a rural area, a freshman-year plan for family practice, participation in the PSAP, having an NHSC scholarship, and male sex), and both of the factors independently predictive of retention (participation in the PSAP, and attending college in a rural area) were available at the time of entrance to medical school. In addition, non-psap graduates who had grown up in a rural area and had freshman-year plans for family practice (2 key selection criteria for the PSAP and independent predictors of rural primary care), were approximately 75% as likely to become rural primary care physicians and to remain so as PSAP graduates, suggesting that the admissions component of the PSAP is by far the most important reason for its positive outcomes. In fact, few graduates without either of these factors became rural primary care physicians. On the other hand, PSAP graduates were approximately 25% more likely than their peers with these 2 background factors to practice and remain in rural primary care practice, suggesting that some of the success of the PSAP was due to factors other than those available at the time of admission. What portion of this was due to curricular, economic, or other programmatic factors related to the PSAP, or to selfselection, is unclear. Taking a senioryear rural family medicine preceptorship was the only independent predictor of rural primary care practice unknown at matriculation, and was also the only independent predictor for PSAP graduates. However, since this experience is selected during the final year of medical school, it is unknown whether those already planning rural primary care chose to take a preceptorship, or whether the preceptorship experience actually increased the likelihood for this career. The fact that few students who did not grow up in rural areas nor had freshman-year plans for family practice selected the preceptorship (n=9) suggests self-selection as an important reason. We plan to address this issue in a future survey of JMC graduates. Participation in the required third-year family practice clerkship at a rural location was not independently related to the study outcome. Although other medical school programs with more extensive curricular components have been shown to be similarly effective, 16 the independent effect of curriculum has never been measured. 22,30 Regarding economic factors, neither freshman expectations of peak income nor medical school debt were predictive of rural primary care for all JMC graduates or for PSAP graduates. Relatively few graduates in this study had high levels of debt, but the recent rapidly increasing level of student debt raises the question of whether this might become a more important factor in the future. 31 Similar to previous reports, 2,5,23 we found that family physicians comprise the majority of rural primary care physicians. However, most family physicians (76%) did not practice in rural 2001 American Medical Association. All rights reserved. (Reprinted) JAMA, September 5, 2001 Vol 286, No. 9 1045

Table 3. Results of Univariate Analysis of Significant Predictors of Rural Primary Care Retention for Jefferson Medical College Graduates, 1978-1986* Figure 1. Percentages of Physicians Practicing Rural Primary Care Percentage of Physicians Practicing Rural Primary Care 30 20 10 Variable 0 PSAP No. of Graduates Rural Origin Plans FP Non-PSAP Nonrural Origin No FP Plans PSAP indicates Physician Shortage Area Program; FP, family practice. areas, and of those who did, almost two thirds had grown up in a rural area. In this study, family practice was the only primary care specialty choice at matriculation that was predictive of rural primary care practice. In fact, a freshman-year plan for general internal medicine was inversely related to practicing rural primary care. Six (2.4%) of the 252 students with a stated intent to pursue general internal medicine became rural primary care physicians compared with 158 (6.5%) of 2421 without such plans. A freshman-year plan for general pediatrics was unrelated to rural primary care. Twelve (5.9%) of the 202 students who planned to pursue general pediatrics entered rural primary care compared with 152 (6.2%) of the 2471 without such plans. No. (%) Still Practicing Rural Primary Care Figure 2. Retention Percentages for Rural Primary Care Physicians Retention Percentage for Rural Primary Care Physicians 100 80 60 40 20 0 Relative Risk (95% Confidence Interval) PSAP Rural Origin Plans FP Non-PSAP P Value PSAP participant Yes 45 37 (82.2) 1.7 (1.3-2.1).001 No 99 49 (49.5) Rural preceptorship Yes 46 33 (71.7) 1.3 (1.0-1.7).03 No 98 53 (54.1) College location Rural 25 22 (88.0) 2.0 (1.4-2.7).001 Not rural 51 23 (45.1) Grew up in a rural area Yes 42 30 (71.4) 1.6 (1.1-2.5).03 No 34 15 (44.1) *PSAP indicates Physician Shortage Area Program. Available only for the classes of 1978-1982. Nonrural Origin No FP Plans PSAP indicates Physician Shortage Area Program; FP, family practice. Growing up in a rural area was also found to be an important predictor of rural primary care practice. However, JMC graduates who combined rural backgrounds with freshman-year plans for family practice were more than twice as likely to become rural primary care physicians as those with only 1 of these factors. This is consistent with prior studies showing the cumulative effect of these 2 important characteristics. 18,24,32 The PSAP, which is itself a combination of these and other factors, was the only independent predictor of both rural primary care practice and retention. And, although not all PSAP graduates became rural primary care physicians, the vast majority (84%) have been shown to be either practicing in rural areas, or in the smallest metropolitan areas, or in one of the primary care specialties. 18 Having an NHSC scholarship was also a predictor of rural primary care, although we could not determine whether this was due to self-selection by individuals applying to the NHSC, who were already likely to practice rural primary care, or whether it was related to the financial or experiential components of the program. In addition, many of the selection criteria for NHSC scholarship recipients are similar to those for the PSAP. 33 As in other studies, participation in the NHSC scholarship program was not related to retention. 34 Also similar to other studies, we found male sex to be predictive of rural primary care. 2,5 However, with 6.0% of men and 4.1% of women practicing rural primary care, this was the least important of the independent predictors. More importantly, there was no significant difference in outcome based on sex for PSAP graduates, suggesting that for individuals already likely to become rural primary care physicians, this is not an important factor. A limitation of this study is that it includes graduates from a single medical school. However, we believe that these results have national applicability for several reasons. First, graduates from JMC are similar to those from other medical schools in the percentage entering family medicine (15.6% vs the national average of 11.7% of all US graduates [who entered family medicine residencies for 1981-1993]), 35,36 and practicing rural primary care (5.0% vs national average of 6.0% for 1976-1985). 37 Second, background and career plans of entering students (the factors found to be most critical in this study) also represent core components of most of the other similar medical school programs 11 and are consistent with prior literature. 22 Third, the overall outcomes of the PSAP are similar to those from other programs with more extensive curricular and financial components, 11 even though JMC is a large private medical school in the northeastern United States (character- 1046 JAMA, September 5, 2001 Vol 286, No. 9 (Reprinted) 2001 American Medical Association. All rights reserved.

istics related to low outcomes of rural and primary care physicians). 23 Fourth, while the physicians in our study all graduated from JMC, they took their residency training at more than 390 hospitals in 43 states, and are currently practicing in all 50 states with most (62%) located outside of Pennsylvania. Although it is important for other medical schools and programs to identify independent predictors of rural primary care supply and retention, JMC is unique in having more than 2 decades of prospectively collected variables from the Jefferson Longitudinal Study database. Another limitation of this study is that it included a limited number of potential predictors. Prior research has shown other important factors not included in this study (eg, spouse background and preference, residency program, loan repayment). 2,5,23,38 However, results from this study showed that JMC graduates, who lacked 2 important predictive factors at the time of medical school matriculation, irrespective of myriad other factors not analyzed, were highly unlikely to practice rural primary care, and of the few who did, most failed to remain. Considering the large amount of money and effort spent to address the rural physician shortage in the last few decades, there has been little focus on identifying what works. For those designing such programs, our findings suggest that the greatest impact by far will be achieved by developing strategies that increase the selection of medical school matriculants who grew up in rural areas, plan to practice family medicine, and have other premedical predictors. In fact, any program that does not do this may have limited success. This is also likely to represent the least costly policy option. To accomplish this, however, medical school admission criteria must be broadened to include these factors, thereby increasing the selection of academically qualified applicants who are most likely to practice rural primary care. At JMC, for instance, more than two thirds of PSAP students were not admitted to another school nor would likely have been admitted to JMC without this program, even though their premedical academic credentials were similar to their peers, as were their academic performances during medical school and postgraduate training. 18-20 Curricular experiences, mentoring, and financial support should also be provided to support these students in their career goals. However, to be successful, medical schools must make eliminating the rural primary care physician shortage one of their priorities. Unfortunately, few incentives exist for most schools to do so, since the major beneficiaries of these programs are rural populations. Without state or federal incentives or regulations, therefore, it seems unlikely that many medical schools will be able to accomplish this. In conclusion, despite widespread acceptance that a physician s background characteristics are related to practicing rural primary care, medical educators have primarily focused on what happens during and after medical school to affect these career choices. Similarly, it is commonly assumed that the curricular components of successful medical school programs are primarily responsible for their outcomes, although such programs preselect for students likely to achieve these career goals, and the independent effect of these curricula has never been studied. An important lesson from this study is the need to reframe the key policy question from what can be done during medical school to what can medical schools do to address the rural primary care physician shortage. The data from this study are clear medical educators and policy makers can have the greatest impact on the supply and retention of rural primary care physicians by designing programs that increase the number of qualified medical school matriculants with background and career plans that are independently related to these career goals. Author Contributions: Study concept and design: Rabinowitz, Diamond, Markham, Paynter. Acquisition of data: Rabinowitz. Analysis and interpretation of data: Rabinowitz, Diamond, Markham, Paynter. Drafting of the manuscript: Rabinowitz. Critical revision of the manuscript for important intellectual content: Rabinowitz, Diamond, Markham, Paynter. Statistical expertise: Diamond. Administrative, technical, or material support: Paynter. Study supervision: Rabinowitz. Acknowledgement: We are indebted to Joseph S. Gonnella, MD, J. Jon Veloski, MS, Mohammadreza Hojat, PhD, and Carol A. Rabinowitz, BA, from the Center for Research in Medical Education and Health Care for the data from the Jefferson Longitudinal Study; to Raelynn Cooter, PhD, from the University Office of Financial Aid for the data regarding student debt and NHSC scholarships; to the Alumni Association for the data regarding JMC alumni practice addresses; to Clara A. Callahan, MD, and Grace M. Hershman, MEd, from the Admissions Office; and to Paul C. Brucker, MD, Thomas J. Nasca, MD, Edward Christian, PhD, and John L. Randall, MD, all from JMC, Thomas Jefferson University; and thank each of the above individuals for their support of the PSAP and their helpful comments on an earlier draft of this article. We also express our admiration of the graduates of JMC and of the PSAP who are practicing primary care in rural areas and small towns. REFERENCES 1. Pusey WA. Medical education and medical service, I: the situation. JAMA. 1925;84:281-285. 2. Council on Graduate Medical Education. Physician Distribution and Health Care Challenges in Rural and Inner City Areas: Tenth Report to Congress and the Department of Health and Human Services Secretary. Rockville, Md: Health Resources and Services Administration, US Dept of Health and Human Services; 1998. 3. Donaldson MS, Yordy KD, Lohr KN, Vanselow NA, eds, for the Institute of Medicine. Primary Care: America s Health in a New Era. Washington, DC: National Academy Press; 1996. 4. Ludmerer KM. Time to Heal: American Medical Education From the Turn of the Century to the Era of Managed Care. New York, NY: Oxford Press; 1999. 5. Geyman JP, Hart GL, Norris TE, Coombs JB, Lishner DM. Educating generalist physicians for rural practice: how are we doing? J Rural Health. 2000;16: 56-80. 6. Phillips DM, Dunlap PG, eds. Physician Recruitment and Retention. Washington, DC: National Rural Health Association; 1998. 7. Jolly P, Hudley DM, eds. Association of American Medical Colleges Data Book: Statistical Information Related to Medical Education. Washington, DC: Association of American Medical Colleges; 1998. 8. Mayers L, Harrison LV. The Distribution of Physicians in the United States. New York, NY: General Education Board; 1924. 9. Kassebaum DG, Szenas PL. Rural sources of medical students, and graduates choice of rural practice. Acad Med. 1993;68:232-236. 10. Cohen JJ. Why doctors don t always go where they re needed. Acad Med. 1999;73:1277. 11. Rabinowitz HK, Paynter NP. The role of the medical school in rural graduate medical education: pipeline or control valve? J Rural Health. 2000;16:249-253. 12. Adkins RJ, Anderson GR, Cullen TJ, Myers WW, Newman FS, Schwarz MR. Geographic and specialty distribution of WAMI program participants and nonparticipants. J Med Educ. 1987;62:810-817. 13. Brazeau NK, Potts MJ, Hickner JM. The Upper Peninsula Program: a successful model for increasing primary care physicians in rural areas. Fam Med. 1990; 22:350-355. 14. Stearns JA, Stearns MA, Glasser M, Londo RA. Illinois RMED: a comprehensive program to improve 2001 American Medical Association. All rights reserved. (Reprinted) JAMA, September 5, 2001 Vol 286, No. 9 1047

the supply of rural family physicians. Fam Med. 2000; 32:17-21. 15. Boulger JG. Family medicine education and rural health: a response to present and future needs. J Rural Health. 1991;7:105-115. 16. Verby JE, Newell JP, Andresen SA, Swentko WM. Changing the medical school curriculum to improve patient access to primary care. JAMA. 1991;266:110-113. 17. Ackermann RJ, Comeau RW. Mercer University School of Medicine: a successful approach to primary care medical education. Fam Med. 1996;28:395-402. 18. Rabinowitz HK, Diamond JJ, Markham FW, Hazelwood CE. A program to increase the number of family physicians in rural and underserved areas: impact after 22 years. JAMA. 1999;281:255-260. 19. Rabinowitz HK. Recruitment, retention, and follow-up of graduates of a program to increase the number of family physicians in rural and underserved areas. N Engl J Med. 1993;328:934-939. 20. Rabinowitz HK. Evaluation of a selective medical school admissions policy to increase the number of family physicians in rural and underserved areas. N Engl J Med. 1988;319:480-486. 21. Xu G, Veloski JJ, Hojat M, Politzer RM, Rabinowitz HK, Rattner S. Factors influencing physicians choices to practice in inner-city or rural areas. Acad Med. 1997;72:1026. 22. Ernst RL, Yett DE. Physician Location and Specialty Choice. Ann Arbor, Mich: Health Administration Press; 1985. 23. Rosenblatt, RA, Whitcomb ME, Cullen TJ, Lishner DM, Hart LG. Which medical schools produce rural physicians? JAMA. 1992;268:1559-1565. 24. Rabinowitz HK, Diamond JJ, Hojat M, Hazelwood CE. Demographic, educational, and economic factors related to recruitment and retention of physicians in rural Pennsylvania. J Rural Health. 1999;15: 216-220. 25. Rabinowitz HK, Diamond JJ, Veloski JJ, Gayle JA. The impact of multiple predictors on generalist physicians care to underserved populations. Am J Public Health. 2000;90:1225-1228. 26. Gonnella JS, Hojat M, Erdmann JB, Veloski JJ. Assessment measures in medical school, residency, and practice: the connections. Acad Med. 1993;68(suppl): 3-106. 27. Hojat M, Gonnella JS, Veloski JJ, Erdmann JB. Jefferson Medical College longitudinal study: a prototype for evaluation of changes. Educ Health. 1996; 9:99-113. 28. Rosenthal MP, Turner T, Diamond JJ, Rabinowitz HK. Income expectations of first-year students at Jefferson Medical College as a predictor of family practice specialty choice. Acad Med. 1992;67:328-331. 29. Rosenthal MP, Marquette PA, Diamond JJ. Trends along the debt-income axis: implications for medical students selection of family practice careers. Acad Med. 1996;71:675-677. 30. Pathman DE. Medical education and physicians career choices: are we taking credit beyond our due? Acad Med. 1996;71:963-968. 31. Procter J. Medical school debt: making sense of life in the red. AAMC Reporter. 2000;9:10-11. 32. Cullison S, Reid C, Colwill JM. Medial school admissions, specialty selection, and distribution of physicians. JAMA. 1976;235:502-505. 33. Division of Scholarships and Loan Repayments, Bureau of Primary Health Care. Applicant Information Bulletin: 2000-2001. Rockville, Md: Health Resources and Services Administration, US Dept of Health and Human Services; 1999. 34. Pathman DE, Konrad TR, Ricketts TC. The comparative retention of the National Health Service Corps and other rural physicians: results of a 9-year follow-up study. JAMA. 1992;268:1552-1558. 35. American Academy of Family Physicians. Table 86. In: Facts About Family Practice: 1990. Kansas City, Mo: American Academy of Family Physicians; 1990: 114-127. 36. American Academy of Family Physicians. Table 113. In: Facts About Family Practice: 2000. Kansas City, Mo: American Academy of Family Physicians; 2000: 158-171. 37. Whitcomb ME, Cullen TJ, Hart GL, Lishner DM, Rosenblatt RA. Impact of Federal Funding for Primary Care Medical Education on Medical Student Specialty Choices and Practice Locations (1976-1985). Seattle, Wash: WAMI Rural Health Research Center; 1991. 38. Rosenthal TC, McGuigan MH, Osborne J, Holden DM, Parsons MA. One-two rural residency tracks in family practice: are they getting the job done? Fam Med. 1998;30:90-93. To be possessed of a vigorous mind is not enough; the prime requisite is rightly to apply it. René Descartes (1596-1650) 1048 JAMA, September 5, 2001 Vol 286, No. 9 (Reprinted) 2001 American Medical Association. All rights reserved.