Awards per 100 FTE: SUNY Data Analysis and Trends May 2014 Performance Measures for Degree Production There are three basic ways of measuring degree production: (1) the numbers of certificates and degrees produced in any given year; (2) the graduation rates; and (3) awards per 100 full-time equivalent students (FTE) ratio. Simple counts of awards produced in any given year are straightforward numbers. They are useful for keeping track of degree production in a state or at a campus over time. Degree counts are not very useful for showing performance relative to inputs. Graduation rates and awards per 100 FTE (a completion ratio) provide a context for counting degrees by answering two different questions: Graduation rates: What percentage of students graduate? Completion ratio: What is the ratio of graduates to overall enrollment? Graduation rates are generally known as an effectiveness measure whereas awards per 100 FTE is an efficiency measure. Graduation rates are longitudinal: the measure begins with an entering group of first-time, full-time freshmen in their fall semester. Then they are tracked over the course of 150 percent of their degree time. For associate programs, this means 3 years; for baccalaureate programs, this means 6 years. The optimal value of a graduation rate is 100 percent. If everything goes smoothly, all students who begin their degree will finish in a reasonable amount of time. Federally defined graduation rates present some well-documented limitations: they include only first-time, full-time undergraduate students, meaning they do not count students who transfer in or students who do not attend full-time their first semester. These measures are campus-bound. They are not easily calculated at the system or state level. By contrast, awards per 100 FTE is a single-year snapshot measure, taken at one point in time. This measure is arguably a more inclusive measure of degree productivity than graduation rates because it covers more students. Awards include all undergraduate degrees and certificates awarded and FTE accounts for all enrolled undergraduate students, adjusted for full-time equivalency. Awards per 100 FTE can easily be calculated at various levels: the campus, the system, the state. Completion ratios are different than graduation rates in terms of interpretation. The optimal value for a campus with predominantly baccalaureate degree programs is 25 percent whereas the optimal value for a campus with predominantly associate degree programs is 50 percent. The same can be applied to campuses that offer upper-division programs or take in a lot of transfer students. These optimal values are derived from the amount of students who could complete their degrees in any given year. In a twoyear campus, with most students enrolled in two-year degrees, about half of the student body has the potential to graduate every year. Page 1 of 10
Definition Unlike federally defined graduation rates, awards per 100 FTE can be calculated in slightly different ways. NCHEMS calculation: Undergraduate credentials (certificates of at least one year or above, associate degrees, bachelor degrees) in one given academic year, divided by the adjusted annual FTE. This resulting proportion is multiplied by 100. See Table 1. Table 1. New York State Awards per 100 FTE, all sectors, AY 2008-2009 2008-2009 Certificates Associates Bachelor Undergrad FTE Awards per 100 FTE New York 4,616 59,048 122,186 875,045 21.2 Awards per 100 FTE = (4,616+59,048+122,186) / 875,045 =.212*100 = 21.2 SUNY calculation: All undergraduate credentials in one academic year, divided by the adjusted annual FTE. This resulting proportion is multiplied by 100. SUNY data do not distinguish between certificate program lengths, so all certificates are included. See Table 2. Table 2. SUNY Awards per 100 FTE, all campuses, AY 2008-2008 2008-2009 Certificates Associates Bachelor Undergrad FTE Awards per 100 FTE SUNY 1,961 30,481 34,276 318,697 20.9 Weighting. Given the difference in optimal value based on prevalence of award type, one of the most debated aspects of this measure is whether or not to weight the different awards for ease of comparison. Patrick Kelly, the NCHEMS analyst who has provided a substantial amount of background research on this measure, discussed weights in his 2009 paper on productivity measurements (see Table 3). Table 3. Weighted values for different undergraduate award types Award Type Weighted Value Certificates 0.68 Associates 0.80 Bachelors 1.00 Source: Kelly, 2009 While weights may be appropriate at the state-level, looking across sectors, systems and campuses, weights are not very useful at the campus level. They tend to disguise institutional mission, placing all campuses on a baccalaureate granting scale. As seen in Table 2 above, SUNY campuses produce almost Page 2 of 10
as many associate degrees as bachelor degrees. As such, an unweighted measure, similar to the NCHEMS calculation, is most appropriate for SUNY data. Sensitivities. Because of the two different optimal values, and SUNY preference to not weight credentials, this measure is sensitive to institutional mission. Specifically, awards per 100 FTE will vary according to two important contextual factors: Percent bachelors to undergraduate awards Percent first-time to total incoming students The first contextual factor identifies the proportion of bachelor awards compared to the total. The lower this factor is, the more one would expect to see an optimal value closer to 50 percent. The second contextual factor highlights the proportion of first-time students to transfer students. The lower this factor is, the more one would expect to see an optimal value closer to 50 percent. This second factor bears more weight on the metric, all other factors equal. For example, if a campus only offers upper division courses, the percent of first-time students will be zero. Even if this campus only offers 100 percent bachelor degrees, the optimal value is still closer to 50 percent. Table 4. SUNY Contextual Factors, average 2007-2008 through 2011-2012 Campus Category % Bachelors to Total % First-time to Total Undergrad Awards Incoming Students SUNY Total 51.1% 66.6% Doctoral 100.0% 56.6% Comprehensive 96.6% 49.1% Technology 41.1% 63.1% Community Colleges 4.0% 74.2% As seen above, Table 4 describes how institutional mission and campus categories can influence awards per 100 FTE calculations at different SUNY campuses. As such, this metric is best for looking at campus performance over time, rather than campus-to-campus comparison. To get a full picture, this measure is best tracked over time alongside changes in other important student data, like enrollments, student demographic characteristics, and transfer and retention rates. National Context NCHEMS has provided baseline, comparative data for this metric using academic year 2008-2009. Including all sectors, New York state performs just ahead of the national average at 21.2 (see Chart 5). On average, national performance for all undergraduate degrees at all campuses is 21.1. Completion ratios range from 15.7 through 28, a 12 point spread. For more information about this study see Attachment 1: National and State Awards per FTE, 2008-09 or http://www.higheredinfo.org/dbrowser/?level=nation&mode=map&state=0&submeasure=379 Page 3 of 10
Chart 5. Awards per 100 FTE all Sectors, by state, 2008-09 Kentucky Louisiana Wisconsin Arizona Washington Florida Georgia Colorado Illinois Kansas Minnesota New Hampshire Wyoming Rhode Island Utah Arkansas Idaho Missouri Iowa Vermont North Dakota Pennsylvania Oklahoma New York Nation Massachusetts Maine Hawaii Nebraska Michigan Delaware Maryland Virginia Texas Connecticut Ohio South Dakota Indiana Tennessee South Carolina North Carolina Mississippi Montana Oregon New Jersey California New Mexico West Virginia Alabama Nevada Alaska 0 5 10 15 20 25 30 Page 4 of 10
Chart 6. Awards per 100 FTE, by campus category, 2008-09 through 2011-12 30 25 20 15 10 5 0 2008-09 2009-10 2010-11 2011-12 SUNY Total State Ops Community Colleges National Public National 4 year National 2 year Undergraduate Awards Per FTE 2008-09 2009-10 2010-11 2011-12 SUNY 20.8 21.0 21.7 22.6 State Operated Campuses 22.8 23.8 24.6 25.3 Research University Centers 21.7 24.1 25.0 25.9 Other Research/Doctoral 31.4 28.5 31.7 32.6 Comprehensive Colleges 23.5 24.0 24.1 24.3 Technology Colleges 22.9 22.2 24.5 26.4 Community Colleges 18.4 18.1 18.7 20.0 Public Total National 17.8 17.5 18.2 19.4 New York State 19.1 18.9 20.0 20.6 Public 4-Year National 20.8 20.9 21.2 21.7 New York State 21.2 21.3 22.4 22.5 Public 2-Year National 14.5 14.1 15.2 17.0 New York State 16.4 15.9 17.1 18.2 Page 5 of 10
SUNY Context In 2008-2009, SUNY completion ratios (20.8) come in just under the national (21.1) and NYS (21.2) data points. However, trend data show that between 2008-2009 and 2011-2012, this metric increased to 22.6, a growth rate of 2 percent. The doctoral campuses grew at nearly double this rate, an annual growth rate of 4 percent during these four years (see Table 6). Technology and community colleges also grew throughout this period, while comprehensive colleges grew at the slowest rate. Table 7. SUNY Awards per 100 FTE, by campus category, 2008-09 through 2011-12 Campus Category 2008-09 2009-10 2010-11 2011-12 Growth Rate SUNY Total 20.8 21.0 21.7 22.6 2.1% Doctoral 22.9 25.3 26.1 26.8 4.0% Comprehensive 23.5 24.0 24.1 24.3 0.8% Technology 22.9 22.2 24.5 26.4 3.7% Community Colleges 18.4 18.1 18.7 20.0 2.1% Growth in the awards per 100 FTE metric can reflect several underlying trends: more students are completing as compared to enrollments, more students are completing shorter cycle degrees and certificates, more students are transferring in with credits already accumulated, or more students are enrolling full-time. In the case of SUNY, the increase in awards per 100 FTE over this time period is reflective of most of these trends. In these years, enrollments, transfers, retention rates and graduation rates all increased. Since the award profile at SUNY campuses do not shift much throughout the years, growth is more likely related to increased retention, persistence, and perhaps increased transfer student enrollments. Attachment 2: SUNY Awards per FTE, 2008-09 through 2011-12, shows more detail about this measure. Gender. Female students tend to complete at higher ratios than men, regardless of campus sector (see Chart 7). That being said, the disparity between female and male students is most notable at the technology centers. Attachment 3: SUNY Awards per FTE by Gender, 2008-09 through 2011-12, shows more detail about this measure, Chart 7. SUNY Awards per 100 FTE, by sector by gender, 2011-12 Page 6 of 10
35 30 25 20 15 10 5 0 Total Female Male Female Male Female Male Female Male SUNY Doctoral Comprehensive Technology Community College Page 7 of 10
Chart 7b. SUNY Awards per 100 FTE, by gender, 5-year trend 30 Female Male Total 25 20 15 10 5 0 2008-09 2009-10 2010-11 2011-12 Race and ethnicity. Underrepresented minority (URM) students tend to underperform their counterparts in all sectors as well (see Chart 8). The disparity is most notable at community colleges, with a 7 point difference. Attachment 4: SUNY Awards per FTE, UnderRepresented Minority Students (URM) Compared to Non- URM, 2008-09 through 2011-12, shows more detail about this measure. Chart 8. SUNY Awards per 100 FTE, by sector, by race/ethnicity, 2011-12 30 25 20 15 10 5 0 Total URM Non-URM URM Non-URM URM Non-URM URM Non-URM SUNY Doctoral Comprehensive Technology Community College Page 8 of 10
Chart 8b. SUNY Awards per 100 FTE, by race/ethnicity, 5-year trend 30 Non-URM URM Total 25 20 15 10 5 0 2008-09 2009-10 2010-11 2011-12 Conclusion Page 9 of 10
References Kelly, P.J. & Jones, D.P. (2005). A new look at the institutional component of higher education finance: A guide for evaluating performance relative to financial resources. Boulder, CO: NCHEMS. Kelly, P.J. (2009). The dreaded P word: An examination of productivity in public postsecondary education. Washington, DC: Delta Cost Project. National Governors Association (NGA). (2013). Beyond completion: Getting answers to the questions governors ask to improve postsecondary outcomes. Washington DC: Author. Page 10 of 10