Evaluation of Michigan State University Advancing Diversity through the Alignment of Policies and Procedures (ADAPP) Annual Report

Transcription

1 Evaluation of Michigan State University Advancing Diversity through the Alignment of Policies and Procedures (ADAPP) Annual Report Miami University 408 McGuffey Hall Oxford, OH Phone: Fax: Website:

2 Please cite as follows: Woodruff, S. B., Morio, K. L., & Li, Y. (2012). Evaluation of Michigan State University Advancing Diversity through the Alignment of Policies and Procedures (ADAPP): Annual report Oxford, OH: Miami University, Ohio s Evaluation & Assessment Center for Mathematics and Science Education. Distributed by Ohio s Evaluation & Assessment Center for Mathematics and Science Education Sarah B. Woodruff, Director 408 McGuffey Hall Miami University Oxford, Ohio Evaluation of MSU ADAPP ADVANCE ii

3 Michigan State University Advancing Diversity through the Alignment of Policies and Procedures (ADAPP) Ohio s Evaluation & Assessment Center for Mathematics and Science Education Miami University Oxford, Ohio Sarah B. Woodruff Yue Li Kristen Morio Principal Investigator Senior Researcher and Statistician Research Associate Evaluation of MSU ADAPP ADVANCE iii

4 Table of Contents Table of Contents... iv List of Tables... ix List of Figures... xiii Executive Summary... 1 Introduction... 3 Project Description... 3 Evaluation... 4 Participants... 4 Instruments... 4 Data Collection... 5 Data Analysis... 5 Indicator Data... 5 Faculty Retention Data... 6 Inventory and Policy Analysis Data... 6 Faculty Composition and Retention... 7 Indicator Data... 7 Baseline, Year 1, and Year 2 MSU Indicator Comparison... 7 STEM Faculty... 7 Hiring... 7 Promotion... 9 Attrition SBS Faculty Hiring Promotion Attrition Leadership Change in Faculty Composition by NSF Category STEM Categories Biological and Agricultural Sciences Earth and Atmospheric Sciences Engineering Evaluation of MSU ADAPP ADVANCE iv

5 Mathematical and Computer Sciences Physical Sciences SBS Categories Professional/Other Psychology Social Science Match IHE Comparison Data Michigan State University Purdue University University of Iowa Match IHE Similarities Match IHE Comparison Faculty Retention Study Comparisons of Faculty Responses on the MSU Work Environment Survey by Retention Status Summary of Faculty Retention Study Findings Human Resources Processes and Policies Background Study of Human Resources Policies and Procedures - Phase Study of Human Resources Policies and Procedures - Phase Study Methods and Data Sources Human Resources Inventory Data Collection Policy Analysis Phase I Data Collection Policy Analysis Phase I Data Analysis Comparison Among Colleges MSU Website Transparency Accessibility Alignment Relationship among university values, university strategic objectives, and university HR principles Congruence of college-level policy with university HR principles Comparisons in the College of Engineering MSU College of Engineering Website Transparency Accessibility Access to Information on the Annual Review Process Evaluation of MSU ADAPP ADVANCE v

6 Access to Information on the Reappointment, Promotion, and Tenure Process Access to Information on Review Committees for Annual Review Access to Information on Review Committees for Reappointment, Promotion, and Tenure Access to Information Regarding Stopping the Tenure Clock Communication and Clarity Communication of Expectations Clarity of Expectations Consistency Consistency of Annual Review Procedures Consistency of Reappointment, Promotion, and Tenure Procedures Consistency in the Use of Materials for Annual Review Consistency in the Use of Materials for Reappointment, Promotion, and Tenure Consistency in Modes of Input for Joint Appointments for Annual Review Consistency in Modes of Input for Joint Appointments for Reappointment, Promotion, and Tenure Consistency in Modes of Mentor Input for Annual Review and Reappointment, Promotion, and Tenure Consistency in Providing Feedback for Annual Review Consistency in Providing Feedback for Reappointment, Promotion, and Tenure Congruence Between Inventory Data and Policy Analysis Alignment Alignment of Annual Review Policies with University Principles Alignment of Reappointment, Promotion, and Tenure Policies with University Principles Comparisons in the College of Natural Science MSU College of Natural Science Website MSU College of Natural Science Inventory and Policies Transparency Accessibility Access to Information on the Annual Review Process Access to Information on the Reappointment, Promotion, and Tenure Process Access to Information on Review Committees for Annual Review Access to Information on Review Committees for Reappointment, Promotion, and Tenure Access to Information Regarding Stopping the Tenure Clock Communication and Clarity Communication of Expectations Clarity of Expectations Evaluation of MSU ADAPP ADVANCE vi

7 Consistency Consistency of Annual Review Procedures Consistency of Reappointment, Promotion, and Tenure Procedures Consistency in the Use of Materials for Annual Review Consistency in the Use of Materials for Reappointment, Promotion, and Tenure Consistency in Modes of Input for Joint Appointments for Annual Review Consistency in Modes of Input for Joint Appointments for Reappointment, Promotion, and Tenure Consistency in Modes of Mentor Input for Annual Review and Reappointment, Promotion, and Tenure Consistency in Providing Feedback for Annual Review Consistency in Providing Feedback for Reappointment, Promotion, and Tenure Congruence Between Inventory Data and Policy Analysis Alignment Alignment of Annual Review Policies Alignment of Reappointment, Promotion, and Tenure Policies Comparisons in the College of Social Science MSU College of Social Science Website MSU College of Social Science Inventory and Policies Transparency Accessibility Access to Information on the Annual Review Process Access to Information on the Reappointment, Promotion, and Tenure Process Access to Information on Review Committees for Annual Review Access to Information on Review Committees for Reappointment, Promotion, and Tenure Access to Information Regarding Stopping the Tenure Clock Communication and Clarity Communication of Expectations Clarity of Expectations Consistency Consistency of Annual Review Procedures Consistency of Reappointment, Promotion, and Tenure Procedures Consistency in the Use of Materials for Annual Review Consistency in the Use of Materials for Reappointment, Promotion, and Tenure Consistency in Modes of Input for Joint Appointments for Annual Review Consistency in Modes of Input for Joint Appointments for Reappointment, Promotion, and Tenure Evaluation of MSU ADAPP ADVANCE vii

8 Consistency in Providing Feedback for Annual Review Consistency in Providing Feedback for Reappointment, Promotion, and Tenure Congruence Between Inventory Data and Policy Analysis Alignment Alignment of Annual Review Policies Alignment of Reappointment, Promotion, and Tenure Policies New and Continuing Evaluation Activities Summary and Recommendations Summary Recommendations Appendices Appendix A: 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Appendix B: 2009 MSU Annual Review Inventory Appendix C: 2009 Reappointment, Promotion, and Tenure Inventory Appendix D: Indicator Analysis Table Appendix E: MSU ADAPP ADVANCE Faculty Excellence Advocate Study Evaluation of MSU ADAPP ADVANCE viii

9 List of Tables Table 1. Tenured and Tenure-Track Faculty New Hires by Gender, Rank and by STEM NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data...8 Table 2. Tenured and Tenure-Track Faculty Promotions (with and without tenure status change) by Gender, Rank, and by STEM NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data...10 Table 3. Tenured and Tenure-Track Faculty Attrition by Gender, Rank and STEM NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data...12 Table 4. Tenured and Tenure-Track Faculty New Hires by Gender, Rank and by SBS NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data...14 Table 5. Tenured and Tenure-Track Faculty Promotions (with and without tenure status change) by Gender, Rank, and by SBS NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data...16 Table 6. Tenured and Tenure-Track Faculty Attrition by Gender, Rank and SBS NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data...18 Table 7. Distribution of STEM Leadership Positions by Gender for Michigan State University, October 2011 Data...19 Table 8. Change in Percentage of Faculty Positions Held by Women in Baseline, Year 1, Year 2, and Year 3 Data by Rank and NSF Category...23 Table 9. Change in Percentage of Faculty Positions Held by Women in Baseline, Year 1, Year 2, and Year 3 Data by Tenure Status and NSF Category...25 Table 10. Distribution of STEM Faculty by Gender and Rank for Michigan State University, October 2011 Data...27 Table 11. Distribution of STEM Leadership Positions by Gender for Michigan State University, October 2011 Data...28 Table 12. Distribution of STEM Faculty by Gender and Rank for Purdue University, October 2011 Data...29 Table 13. Distribution of STEM Leadership Positions by Gender for Purdue University, October 2011 Data...30 Table 14. Distribution of STEM Faculty by Gender and Rank for University of Iowa, October 2011 Data...31 Table 15. Distribution of STEM Leadership Positions by Gender for University of Iowa, October 2011 Data...32 Table 16. Comparative Distribution of STEM Faculty by Gender and Rank for Michigan State University and Match Institutions, October 2011 Data...32 Table 17. Comparative Distribution of STEM Leadership Positions by Gender for Michigan State University and Match Institutions, October 2011 Data...33 Table 18. Comparative Change in STEM Faculty Composition by Gender and Rank for Michigan State University and Match Institutions, Year 3 to Year Table 19. Comparative Change in STEM Leadership Positions by Gender for Michigan State University and Match Institutions, Year 3 to Year Evaluation of MSU ADAPP ADVANCE ix

10 Table 20. Sample Size by Hiring Cohort and Gender, Faculty Retention Study, Table 21. Sample Size by Hiring Cohort, Rank, Gender, and Retention Status, Faculty Retention Study, Table 22. Comparisons of Time to Departure for Assistant Professors Hired between 1991 and 2011 by Gender, Faculty Retention Study, Table 23. Comparisons of Time to Departure by Gender for Each Rank and Hiring Cohort, Faculty Retention Study, Table 24. Comparisons of Rasch Mean Subscale Scores by Retention Status for the Spring 2009 MSU Work Environment Survey, Faculty Retention Study, Table 25. Comparisons of Rasch Mean Subscale Scores by Retention Status for the Spring 2009 MSU Work Environment Survey, Matched Faculty Only, Faculty Retention Study, Table and 2011 AR and RPT Inventory Completion, MSU ADAPP ADVANCE Policy Analysis...50 Table and 2011 Annual Review Inventory Crosswalk, MSU ADAPP ADVANCE Policy Analysis...51 Table and 2011 Reappointment, Promotion, and Tenure Inventory Crosswalk, MSU ADAPP ADVANCE Policy Analysis...52 Table 29. MSU Annual Performance Review Policy University Principles by Category, MSU ADAPP ADVANCE Policy Analysis...54 Table 30. MSU Reappointment, Promotion, and Tenure Policy University Principles by Category, MSU ADAPP ADVANCE Policy Analysis...54 Table 31. Best Practices and Inventory Item Crosswalk...55 Table 32. Access to College Bylaws, MSU ADAPP-ADVANCE Policy Analysis...58 Table 33. Comparison of Term Usage for Task Force Reports, MSU ADAPP-ADVANCE Policy Analysis...59 Table 34. Comparison of Term Usage for Selected College Strategic Plans, MSU ADAPP-ADVANCE Policy Analysis...60 Table 35. Alignment of College-Level Policies with University Principles on Annual Review, MSU ADAPP-ADVANCE Policy Analysis...61 Table 36. Alignment of College-Level Policies with University Principles on Reappointment, Promotion, and Tenure, MSU ADAPP-ADVANCE Policy Analysis...61 Table 37. Access to Department Bylaws, MSU ADAPP-ADVANCE Policy Analysis, College of Engineering...64 Table 38. Review Participants for Annual Review, 2011 MSU Annual Review Inventory, College of Engineering...65 Table 39. Materials Required for Annual Review, 2011 MSU Annual Review Inventory, College of Engineering...68 Table 40. Materials Required for Reappointment (R), Promotion with Tenure (T), and Promotion after Tenure Award (P), 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Engineering...69 Table 41. Selection of Pool and Final External Letter Writers for RPT, 2011 MSU Reappointment Promotion, and Tenure Inventory, College of Engineering...70 Evaluation of MSU ADAPP ADVANCE x

11 Table 42. Types of Annual Review Feedback, 2011 MSU Annual Review Inventory, College of Engineering...73 Table 43. Congruence of Annual Review Inventory Information with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Engineering...74 Table 44. Congruence of Reappointment, Promotion, and Tenure Inventory Information with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Engineering...75 Table 45. Annual Review Best Practices Comparison with 2009 MSU Annual Review Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Engineering...78 Table 46. Reappointment, Promotion, and Tenure Best Practices Comparison with 2009 MSU Reappointment, Promotion, and Tenure Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Engineering...80 Table 47. Access to Department Bylaws, MSU ADAPP-ADVANCE Policy Analysis, College of Natural Science...81 Table 48. Review Participants for Annual Review, 2011 MSU Annual Review Inventory, College of Natural Science...83 Table 49. Materials Required for Annual Review, 2011 MSU Annual Review Inventory, College of Natural Science...86 Table 50. Materials Required for Reappointment (R), Promotion with Tenure (T), and Promotion after Tenure Award (P), 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Natural Science...87 Table 51. Selection of Pool and Final External Letter Writers for RPT, 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Natural Science...88 Table 52. Types of Annual Review Feedback, 2011 MSU Annual Review Inventory, College of Natural Resources...91 Table 53. Congruence of Annual Review Inventory Information with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Natural Science...92 Table 54. Congruence of Reappointment, Promotion, and Tenure Inventory with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Natural Science...93 Table 55. Annual Review Best Practices Comparison with 2009 MSU Annual Review Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Natural Science...99 Table 56. Reappointment, Promotion, and Tenure Best Practices Comparison with 2009 MSU Reappointment, Promotion, and Tenure Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Natural Science Table 57. Access to Department Bylaws, MSU ADAPP-ADVANCE Policy Analysis, College of Social Science Table 58. Review Participants for Annual Review, 2011 MSU Annual Review Inventory, College of Social Science Table 59. Materials Required for Annual Review, 2011 MSU Annual Review Inventory, College of Social Science Evaluation of MSU ADAPP ADVANCE xi

12 Table 60. Materials Required for Reappointment (R), Promotion with Tenure (T), and Promotion after Tenure Award (P), 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Social Science Table 61. Selection of Pool and Final External Letter Writers for RPT, 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Social Science Table 62. Types of Annual Review Feedback, 2011 MSU Annual Review Inventory, College of Social Resources Evaluation of MSU ADAPP ADVANCE xii

13 List of Figures Figure 1. Survival and hazard functions for assistant professors hired between 1991 and 2011 by gender (women in green, men in yellow) Figure 2. Survival and hazard functions for assistant professors hired between 1991 and 2000 by gender (women in green, men in yellow) Figure 3. Survival and hazard functions for assistant professors hired between 2001 and 2011 by gender (women in green, men in yellow) Figure 4. Comparisons of Rasch mean subscale scores by retention status and by gender for the Spring 2009 MSU Work Environment Survey, matched faculty only Figure 5. University-level policy document accessibility map, AR and RPT, MSU ADAPP ADVANCE Policy Analysis Figure 6. University-level policy document accessibility map, annual review and RPT, MSU ADAPP ADVANCE Policy Analysis, College of Engineering Figure 7. Mean score for clarity of expectations for all six types of activities for each department in the College of Engineering Figure 8. Mean score for clarity of expectations for the College of Engineering for each type of activity Figure 9. How input is collected for joint reviews, College of Engineering Figure 10. How input is collected for joint reviews, College of Engineering Figure 11. Alignment of AR policies for the College of Engineering, MSU policy analysis, Phase I...77 Figure 12. Alignment of RPT policies for the College of Engineering, MSU policy analysis, Phase I...79 Figure 13. Mean score for clarity of expectations for all six types of activities for each department in the College of Natural Science Figure 14. Mean score for clarity of expectations for the College of Natural Science for each type of activity Figure 15. How input is collected for joint reviews, College of Natural Science Figure 16. How input is collected for joint reviews, College of Natural Science Figure 17. Alignment of AR policies on expectations for the College of Natural Science, MSU policy analysis, Phase I Figure 18. Alignment of AR policies on procedures for the College of Natural Science, MSU policy analysis, Phase I Figure 19. Alignment of AR policies on follow-up for the College of Natural Science, MSU policy analysis, Phase I Figure 20. Alignment of RPT policies on expectations for the College of Natural Science, MSU policy analysis, Phase I Figure 21 Alignment of RPT policies on procedures for the College of Natural Science, MSU policy analysis, Phase I Evaluation of MSU ADAPP ADVANCE xiii

14 Figure 22. Alignment of RPT policies on follow-up for the College of Natural Science, MSU policy analysis, Phase I Figure 23. University-level policy document accessibility map, AR and RPT, College of Social Science Figure 24. Mean score for clarity of expectations for all six types of activities for each department in the College of Social Science Figure 25. Mean score for clarity of expectations for the College of Social Science for each type of activity Figure 26. How input is collected for joint reviews, College of Social Science Figure 27. How input is collected for joint reviews, College of Social Science.Consistency in Modes of Mentor Input for Annual Review and Reappointment, Promotion, and Tenure Figure 28. Alignment of AR policies for the College of Social Science, MSU policy analysis, Phase I Figure 29. Alignment of RPT policies for the College of Social Science, MSU policy analysis, Phase I Figure 30. MSU ADAPP ADVANCE evaluation proposed timeline Evaluation of MSU ADAPP ADVANCE xiv

15 Executive Summary Michigan State University (MSU) was awarded a National Science Foundation ADVANCE Institutional Transformation grant in Since the award, the ADAPP Project Team has laid the foundation for attainment of this institutional transformation initiative s goals which include increasing the number of women recruited in the Colleges of Natural Science, Social Science, and Engineering (Goal 1), improving their retention and advancement (Goals 2 & 3), and improving the climate for women in these three Colleges (Goal 4). These goals address issues that have been identified as barriers for women in STEM disciplines both nationally and by women at MSU. Major findings from Year 4 data include: While hiring in STEM and SBS units has decreased significantly between 2007 and 2011, the proportions of women and men faculty new hires have fluctuated very little, remaining around 40% for women new hires. Overall, hiring differences in the STEM NSF categories during the first 3 years of the ADAPP-ADVANCE project favored males (n = 29) over females (n = 12). Differences in hiring in SBS units during the first 3 years favored women, with slightly more female professors (n = 18) than male professors (n = 17) hired between 2008 and 2011; however, 95% of female hires were assistant professors as compared to 65% of male hires. Between October 2008 and October 2011, the composition of faculty shifted toward slightly higher percentages of women in Biological and Agricultural Sciences, Earth and Atmospheric Sciences, Engineering, Mathematical and Computer Sciences, Professional/Other, and Psychology. Overall, women gained 0.5% of the available positions in STEM NSF departments and 1.0% of available positions in SBS NSF departments. During this same time period, the composition of non-tenure track faculty changed more significantly with a loss of 4.1% of women non-tenure track faculty in STEM and a loss of 0.7% of SBS non-tenure track women. There was a notable decrease (0.8%) in senior women STEM faculty between 2008 and 2011, primarily due to retirements. Attrition due to retirement during the first 3 years of ADAPP most impacted the number of senior women faculty in the Biological and Agricultural Sciences, Physical Sciences, and Social Sciences. As a percentage of each gender and rank group, rates of voluntary non-retirement attrition were relatively equal for all faculty in the NSF STEM categories, at about 5% for both female and male professors at each rank. In the NSF SBS categories, the rate of voluntary non-retirement attrition was higher for female assistant professors (3%) than for male assistant professors (1%). Voluntary non-retirement rates of attrition were relatively equal for women and men at the rank of associate and full professor. Findings of the STEM faculty retention study suggest that STEM women faculty, historically have left, and continue to leave the University at higher rates than do STEM men faculty of the same rank. Retention rates for STEM women assistant professors are significantly lower than for men assistant professors hired between 1991 and High risk of losing STEM women faculty, particularly assistant professors, extends over longer periods of time than does the high risk of losing STEM men faculty. The attrition rate of STEM women faculty increases and remains high from Year 4 through Year 11. Attrition rates peaked at Years 4, 9, and 10 for women. The average time to departure was 14.6 years for women, which is significantly shorter than the average of 15.9 years for men. Evaluation of MSU ADAPP ADVANCE 1

16 There were a number of statistically significant differences between the Work Environment Survey responses of STEM faculty who departed MSU and those who stayed. In all cases, departed faculty had less positive perceptions of the work environment than did retained faculty. During the first 3 years of the ADAPP-ADVANCE project, in all STEM NSF categories, promotion favored males with 9 females and 26 males promoted from assistant to associate professor, and 2 females and 13 males promoted from associate to full professor. However, female faculty were promoted from assistant to associate at a slightly higher rate than were male faculty (16% and 13%, respectively) and at a slightly lower rate (2% and 4%, respectively) from associate to full professor during the years of the project. For the SBS NSF categories, the same number of female and male faculty were promoted from assistant to associate professor (n = 11) but fewer women than men were promoted from associate to full professor (9 and 12, respectively). As a percentage of each gender group, promotion in both categories (assistant to associate and associate to full professor) were equal at 7%. In , women held 7% of the titled leadership positions available in the College of Engineering, 14% in the College of Natural Science, and 38% in the College of Social Science. There were no women department heads in the Colleges of Engineering and Natural Science. This lack of female leadership representation at the department level accounts for MSU s much lower percentage of females in titled leadership positions (6%) than in peer institutions. Findings of the Phase 1 Policy Analysis indicated that department-level AR and RPT policies and procedures had become more aligned with best practices between 2009 and Common remaining issues are primarily associated with communication of expectations, consistency in the application of procedures within and across departments, and extracting policy and procedure for annual review from the processes of reappointment, promotion and tenure award. Evaluation of MSU ADAPP ADVANCE 2

17 Introduction Ohio s Evaluation & Assessment Center for Mathematics and Science Education (E & A Center) is the project evaluator for the Michigan State University (MSU) Advancing Diversity through the Alignment of Policies and Procedures (ADAPP) project. The ADAPP project is funded through an Institutional Transformation (IT) grant from the National Science Foundation (NSF). Dr. Sarah Woodruff, Miami University, is the Principal Investigator for the evaluation and Kristen Morio, Research Associate, is the Project Director. Yue Li, is the Senior Researcher and Statistician for the evaluation. This report is divided into six sections. Section one provides background information about the ADAPP project and goals. Section two provides information about the Year 4 evaluation. Section three provides data and findings about faculty composition and retention. Section four provides data and findings of the human resources policy study. Section five provides information on Year 5 evaluation plans. The final section provides a summary of findings and recommendations based on findings. Project Description The Advancing Diversity through the Alignment of Policies and Practices (ADAPP) project is a National Science Foundation (NSF) ADVANCE institutional transformation project focusing on the recruitment, retention, and advancement of women Science, Technology, Mathematics, and Engineering (STEM) faculty at Michigan State University in East Lansing, Michigan. The minimal representation of women, and especially women of color, in STEM faculty and leadership positions at MSU is a situation similar to that found at other large institutions of higher education. The ADAPP project focuses on the implementation of policies and procedures developed as part of a Strategic Human Resources Management (SHRM) methodology to align the strategic goals of units and colleges with objective criteria for recruitment, advancement, and retention. The ADAPP project has four major goals: Goal 1: Increase the number of women faculty recruited and appointed into the Colleges of Natural Science, Social Science, and Engineering; Goal 2: Increase the retention of women faculty in Natural Science, Social Science, and Engineering; Goal 3: Increase advancement of women faculty in Natural Science, Social Science, and Engineering; and Goal 4: Improve the work environment (climate) for women faculty in Natural Science, Social Science, and Engineering. Evaluation of MSU ADAPP ADVANCE 3

18 Evaluation Ohio s Evaluation & Assessment Center for Mathematics and Science Education was contracted in December 2009 to conduct internal and external evaluation activities for the remaining 4 years of the MSU ADAPP-ADVANCE project. Year 1 evaluation data from previous evaluation activities were provided to E & A Center evaluators to be used for year-to-year comparisons and to inform subsequent findings and recommendations. Overarching evaluation efforts focus on assessing the progress towards project goals and monitoring project implementation at the department, college, and institution levels. The E & A Center provides both external and internal evaluation services for the ADAPP project. As internal evaluators, the E & A Center: (a) provides formative feedback to inform project efforts, (b) develops instruments as needed to collect data, (c) guides analyses performed by project staff, (d) provides reports and findings of mid-project activities, (e) facilitates on-site meetings and provides post-meeting reports, and (f) works with the ADAPP Project Team to assess project progress and improve project structures and processes. During Year 4 of the project, the E & A Center and Project Team have communicated via , conference calls, and face-to-face meetings to discuss the progress of the evaluation and project. External evaluation activities included: (a) analyzing Year 4 indicator data, (b) developing models for faculty retention, (c) undertaking a comprehensive policy analysis, (d) developing and implementing an FEA case study plan, (e) participating in Project Team meetings, (f) conducting interviews of university administrators and project personnel, (g) advising project research studies, (h) facilitating strategic planning meetings, and (i) administering and analyzing a follow-up human resources department/unit inventory. Participants Participants for the evaluation of the ADAPP-ADVANCE project were tenure-track faculty whose home departments were within the eight STEM categories described by NSF: (a) Biological and Agricultural Sciences; (b) Earth, Atmospheric, and Ocean Sciences; (c) Engineering; (d) Mathematical and Computer Sciences; (e) Physical Sciences; (f) Professional/Other; (g) Psychology; and (h) Social Science. An institutional transformation grant aims to change the climate of a university as a whole. Following implementation of the project in the three target colleges, the initiative is being infused across the university. While other colleges and their faculty may participate in ADAPP-ADVANCE activities, they will not be considered participants in the evaluation. Instruments The initial inventory surveys were administered to all unit heads in Spring 2009 to collect data on unitlevel HR policies and practices related to annual review (AR), reappointment, promotion, and tenure (RPT), and mentoring. These data were collected by three separate inventory instruments. In 2010, the evaluation team recommended that HR data collection be streamlined by the revision and merging of the AR and RPT inventories into a single instrument. The revised instrument, the MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory, was developed in Fall 2011 and administered to all unit heads in the three ADAPP participating colleges between September 2011 and February The purpose of the revised instrument was to measure change in HR practices at the mid-point of the ADAPP initiative by reframing inventory items using data collected by the original inventory instruments. Several modifications were made to the original instruments to facilitate this goal. Based on respondents Evaluation of MSU ADAPP ADVANCE 4

19 data, revisions were made that provided response choices aligned with collected data and what was learned about emerging best practices. Combining the AR and RPT inventories into a single instrument reduced respondent survey load and made it possible to assess these inseparable HR processes more holistically. Items that were found not to provide useful and/or reliable data were removed but with an eye toward maintaining items that can be cross-walked to other data sources, primarily the Work Environment Survey which will be readministered in Spring Further, revisions used operational terms that are common across units and disciplines, and respondents were provided more opportunities to explicate responses when they chose to do so. The revised instrument was vetted with a sample of unit chairs from each college as well as reviewed by the ADAPP team and personnel from the office of Academic Human Resources. The Annual Review section of the combined inventory was composed of 27 multiple-choice items and 2 open-response items. The Reappointment, Promotion, and Tenure section was composed of 25 multiple-choice items and 1 open-response item. Both sections collected information regarding the guidelines, materials, input, and feedback for these HR practices. The MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory can be found in Appendix A. Data Collection Indicator data were collected for Michigan State University through the MSU Enterprise Data Warehouse in April Additional data regarding reappointment, promotion, and tenure and attrition also were provided through this source for purposes of aggregate reporting. Comparison indicators for University of Iowa were provided to the E & A Evaluation Team by The Office of the Executive Vice President and Provost in February Purdue University comparison indicator data were provided by the Purdue Center for Faculty Success (PCFS) ADVANCE Project Team in April Retention study data were collected on 1,415 tenured or tenure-system STEM faculty who were employed in 2001 or newly hired since 2001 in nine colleges through the MSU Enterprise Data Warehouse in February The voluntary attrition of these faculty members was tracked from January 1, 2001 to October 1, Faculty who left MSU before October 1, 2011 for non-retirement reasons were counted as not retained, and those employed October 1, 2011 were counted as retained. The online version of the MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory was administered to 32 department/unit heads in participating colleges using Prezza Checkbox by Ohio s Evaluation and Assessment Center for Mathematics and Science Education between September 2011 and February All units from the College of Engineering and the College of Social Science, and all but two units from the College of Natural Science completed this instrument. Policy documents were collected from the Michigan State University website between February 2011 and March 2012 for university-, college-, and unit-level policy analysis. Requests for documents not found through website search were made and additional documents were collected between March and April Data Analysis Indicator Data Descriptive data were analyzed to determine practical significance of change in the composition of MSU s faculty across project years and to compare MSU data to Purdue University and the University of Iowa data on parallel indicators related to the numbers and leadership of female faculty. Evaluation of MSU ADAPP ADVANCE 5

20 Faculty Retention Data Faculty were categorized into three hiring cohorts: Cohort A hired between 1981 and 1990; Cohort B hired between 1991 and 2000; and Cohort C hired between 2001 and Non-parametric survival analysis was conducted to estimate the retention rate for each rank within each hiring cohort and the results were compared by gender. Retention data also were merged with the 2009 MSU Work Environment Survey Data. Analyses were performed to determine if there were any measureable differences between the responses on the MSU Work Environment Survey of those faculty who stayed and those who departed MSU. Inventory and Policy Analysis Data For the Year 4 evaluation, three sources of data were used to evaluate the access, transparency, and clarity of human resources policies and processes in the three STEM colleges at MSU. The sources of data included (a) the 2009 MSU Annual Review Inventory and Reappointment, Promotion, and Tenure Inventory (Appendices B and C), (b) the 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment Promotion and Tenure Combined Inventory (Appendix A), and (c) the Phase 1 policy analysis for each college. Some data collected by the 2009 MSU Annual Review Inventory and the Reappointment, Promotion, and Tenure Inventory served as pre-intervention data to analyze the extent to which units had modified HR processes, policies, or practices over the last two years. Items that could be matched to the 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment Promotion and Tenure Combined Inventory were analyzed for change. Data from the 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment Promotion and Tenure Combined Inventory were compared to policy documents that were obtained from department websites and electronically upon request. Inventory responses that pertained to policy documents obtained were compared for congruence. The alignment of university principles, college-level policies, and unit-level policies were studied through a rigorous policy analysis. Policies on AR and RPT expectations, procedures, and follow-up were located and analyzed for clarity. Evaluation of MSU ADAPP ADVANCE 6

21 Faculty Composition and Retention Indicator Data A primary goal of the ADAPP project is to increase the number of women in STEM faculty positions and to improve their success as measured by outcomes related to retention and promotion. In order to assess the practical significance of gains for this goal at Michigan State University, indicator data has been monitored across project years. Indicator data were used in two ways for the Year 4 evaluation. First, baseline, Year 1, Year 2, and Year 3 MSU data were analyzed to determine if any significant change had occurred during the first 3 years of the project. Also, two similar institutions of higher education (IHE) were chosen for comparative purposes to determine the relative significance of change at MSU. Data from these two institutions are compared with MSU data annually. Baseline, Year 1, and Year 2 MSU Indicator Comparison The required indicator data tables (submitted with the ADAPP project annual report) provide an overview of progress of the MSU ADAPP-ADVANCE initiative on required NSF indicator measures. While these data represent some aspects regarding the status of women STEM and SBS faculty, they are not particularly informative when reviewed individually. It has been noted by the ADAPP project and E & A Center Evaluation Team that while the indicator data tables are very useful for observing change across years, they may obscure movement of women faculty between ranks and do not provide information on the overall retention of women STEM and SBS faculty. For this reason, the Evaluation Team synthesized independent sets of indicator data to provide a more detailed picture of faculty movement during the first 3 years of the project. These syntheses are presented in this report. Data used for the analyses reported here were compiled by MSU Academic Human Resources in October 2008 for the academic year (baseline data), in October 2009 for the academic year (ADAPP project Year 1), in October 2010 for the academic year (ADAPP project Year 2), and in October 2011 for the academic year (ADAPP project Year 3). Promotions with and without change in tenure status, new hires, and retirement and non-retirement attrition were included in this analysis of STEM and SBS departments in the eight NSF categories. Changes in the numbers of female faculty were compared to changes in the numbers of male faculty for each category. A full year-to-year change analysis table, in which hiring, promotion, and attrition were tracked from before the project (baseline) to October 2011 (Year 3), can be found in Appendix D. Hiring STEM Faculty During the first 3 years of the ADAPP-ADVANCE project, 7 female professors were hired while 7 male professors were hired into units of the Biological and Agricultural Sciences category (Table 1). Engineering hired 3 female faculty members and 12 male faculty. Physical Sciences increased female faculty by hiring 1 female assistant professor and increased male faculty by hiring 7 assistant professors. Mathematical and Computer Sciences hired 1 female assistant professor between 2008 and Earth and Atmospheric Sciences did not hire any new faculty members. Overall, hiring differences in the STEM NSF categories during the first 3 years of the ADAPP-ADVANCE project favored males (n = 29) over females (n = 12). Between 2008 and 2011, no women full professors were hired in STEM units and the majority (83%) of all STEM hires were at the rank of assistant professor for both men and women. Evaluation of MSU ADAPP ADVANCE 7

22 Engineering Physical Sciences Total Evaluation of MSU ADAPP ADVANCE 8 Table 1. Tenured and Tenure-Track Faculty New Hires by Gender, Rank and by STEM NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data NSF Category Project Baseline to Year 1 Year 1 to Year 2 Year 2 to Year 3 Female Male Female Male Female Male Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Biological & Agricultural Sciences Earth & Atmospheric Sciences Mathematical & Computer Sciences

23 Promotion Between 2008 and 2011, in Biological and Agricultural Sciences, 5 female assistant professors were promoted to associate professor with tenure and 2 tenured female associate professors were promoted to full professor, while 6 male assistant professors were promoted with tenure and 2 tenured male associate professors were promoted to full professor (Table 2). The 1 female assistant professor in Earth and Atmospheric Sciences was promoted to associate professor with tenure in In Engineering, during the years of the project, 2 female assistant professors were promoted to associate professor with tenure between 2008 and 2009 and there were no promotions of females after Between 2008 and 2011, 18 male professors received promotions, with 10 assistants promoted to associate with tenure and 8 associate professors receiving a promotion to full professor in Engineering units. Only male faculty received promotions (with and without tenure status change) in the Mathematical and Computer Sciences and Physical Sciences units during the years of the ADAPP-ADVANCE project. During the first 3 years of the ADAPP-ADVANCE project, in all STEM NSF categories, promotion favored males with 9 females and 26 males promoted from assistant to associate professor, and 2 females and 13 males promoted from associate to full professor. However, female faculty were promoted from assistant to associate at a slightly higher rate than male faculty (16% and 13%, respectively) and at a slightly lower rate (2% and 4%, respectively) from associate to full professor during the years of the project. Evaluation of MSU ADAPP ADVANCE 9

24 Table 2. Tenured and Tenure-Track Faculty Promotions (with and without tenure status change) by Gender, Rank, and by STEM NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data Engineering Physical Sciences Total Evaluation of MSU ADAPP ADVANCE 10 NSF Category Biological & Agricultural Sciences Asst. -> Assoc. Project Baseline to Year 1 Year 1 to Year 2 Year 2 to Year 3 Female Male Female Male Female Male Assoc. - > Full Asst. -> Assoc. Assoc. - > Full Asst. -> Assoc. Assoc. - > Full Asst. -> Assoc. Assoc. - > Full Asst. -> Assoc. Assoc. - > Full Asst. -> Assoc Assoc. - > Full Earth & Atmospheric Sciences Mathematical & Computer Sciences

25 Attrition Table 3 displays attrition due to retirement and voluntary, non-retirement (VNR) reasons for NSF STEM categories between October 2008 and October Biological and Agricultural Sciences lost 3 female faculty to voluntary, non-retirement attrition while at the same time 7 male faculty left for voluntary, nonretirement reasons. In Engineering, 1 female tenured faculty member left for voluntary, non-retirement reasons and 6 tenured male Engineering faculty left for voluntary, non-retirement reasons. Physical Sciences lost 1 female faculty member and 3 male faculty members to voluntary, non-retirement reasons. Only male faculty left MSU for voluntary non-retirement reasons in the Mathematical and Computer Sciences (n = 2) and no faculty member left the Earth and Atmospheric Sciences for voluntary nonretirement reasons. Biological and Agricultural Sciences had the largest loss of faculty members to retirement during the years of the project with 5 female full professors, 10 male full professors, 1 female associate professor, and 2 male associate professors retiring between 2008 and In total, in the NSF STEM categories, 5 female tenured and tenure-track faculty and 18 male tenured and tenure-track faculty left MSU for voluntary, non-retirement reasons, while 7 females and 22 males retired during the first 3 years of the ADAPP-ADVANCE project. As a percentage of each gender and rank group, rates of voluntary non-retirement attrition were relatively equal for all faculty in the NSF STEM categories. For example, 5% of both female and male assistant professors left MSU during the years of the ADAPP ADVANCE project for voluntary non-retirement reasons. Evaluation of MSU ADAPP ADVANCE 11

26 Table 3. Tenured and Tenure-Track Faculty Attrition by Gender, Rank and STEM NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data Project Baseline to Year 1 Year 1 to Year 2 Year 2 to Year 3 Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Evaluation of MSU ADAPP ADVANCE 12 NSF Category Female Male Female Male Female Male Biological & Agricultural Sciences Earth & Atmospheric Sciences Engineering VNR Retired VNR Retired VNR Retired Mathematical & Computer Sciences Physical Sciences Total VNR Retired VNR Retired VNR Retired

27 Hiring SBS Faculty Table 4 shows the number of new hires by rank for each SBS NSF category. Between 2008 and 2011, 5 female and 2 male assistant professors were hired in units of the Professional/Other category. In Psychology, 3 female assistant professors and 1 female associate professor were hired between project Year 1 and Year 3. The Social Science hired many new faculty during the years of the ADAPP-ADVANCE project; 9 female assistant professors, 8 male assistant professors, 2 male associate professors, and 3 male full professors. In total, the STEM departments of the SBS categories hired slightly more female professors (18) than male professors (17) between 2008 and 2011; however, 95% of female hires were assistant professors as compared to 65% of male hires. Evaluation of MSU ADAPP ADVANCE 13

28 Professional/Other Psychology Social Science Total Evaluation of MSU ADAPP ADVANCE 14 Table 4. Tenured and Tenure-Track Faculty New Hires by Gender, Rank and by SBS NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data NSF Category Project Baseline to Year 1 Year 1 to Year 2 Year 2 to Year 3 Female Male Female Male Female Male Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full

29 Promotion Promotions with and without tenure for faculty in the SBS NSF categories are displayed in Table 5. In the Professional/Other category, between 2008 and 2011, 4 female assistant professors were promoted to associate professor with tenure, and 2 female associate professors were promoted to full professor. Similar trends were found for male faculty in the Professional/Other category with 3 male assistant professors promoted to associate professor with tenure, and 2 male associate professors promoted to full professor. In Psychology, 3 female assistant professors were promoted to associate professor with tenure and 2 female associate professors were promoted to full. During the same time period, only 1 male professor was promoted from assistant to associate with tenure. In the Social Science category, males received 17 promotions while females received 9 promotions. For the SBS NSF categories, the same number of female and male faculty were promoted from assistant to associate professor (11) but fewer females than males were promoted from associate to full professor (9 and 12, respectively). As a percentage for each gender group, promotion in both categories (assistant to associate and associate to full professor) were equal (7%). Evaluation of MSU ADAPP ADVANCE 15

30 Professional/Other Psychology Social Science Total Evaluation of MSU ADAPP ADVANCE 16 Table 5. Tenured and Tenure-Track Faculty Promotions (with and without tenure status change) by Gender, Rank, and by SBS NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data NSF Category Asst. -> Assoc. Project Baseline to Year 1 Year 1 to Year 2 Year 2 to Year 3 Female Male Female Male Female Male Assoc. - > Full Asst. -> Assoc. Assoc. - > Full Asst. -> Assoc. Assoc. - > Full Asst. -> Assoc. Assoc. - > Full Asst. -> Assoc. Assoc. - > Full Asst. -> Assoc. Assoc. - > Full

31 Attrition Table 6 displays attrition due to retirement and voluntary, non-retirement (VNR) reasons for NSF SBS categories. In the SBS categories, more female tenured and tenure-track faculty (n = 8) left MSU for voluntary, non-retirement reasons than did male faculty (n = 6). The Social Science department experienced the most attrition with 4 female faculty (2 assistants and 2 associates) leaving for voluntary non-retirement reasons and 5 more (1 associate and 4 full professors) retiring between 2008 and At the same time, 4 male faculty (1 assistant, 1 associate, and 2 full professors) from Social Science left for voluntary, non-retirement reasons while 15 (2 associate and 13 full professors) retired. The Professional/Other category lost 3 female assistant professors to voluntary, non-retirement attrition and 1 female associate professor, 2 male associate professors, and 2 male full professors to retirement. Psychology lost 1 associate professor of each gender and 1 male full professor to voluntary, nonretirement attrition, and also 2 male full professors to retirement. In the NSF SBS categories, the rate of voluntary non-retirement attrition was higher for female assistant professors (3%) than for male assistant professors (1%). Voluntary non-retirement rates of attrition were relatively equal for women and men at the rank of associate and full professor. Evaluation of MSU ADAPP ADVANCE 17

32 Evaluation of MSU ADAPP ADVANCE 18 Table 6. Tenured and Tenure-Track Faculty Attrition by Gender, Rank and SBS NSF Category, October 2008, 2009, 2010, and 2011 Human Resources Data NSF Category Project Baseline to Year 1 Year 1 to Year 2 Year 2 to Year 3 Female Male Female Male Female Male Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Asst. Assoc. Full Professional/ VNR Other Retired Psychology Social Science Total VNR Retired VNR Retired VNR Retired

33 Leadership Table 7 displays the distribution of leadership positions by gender for the Colleges of Engineering, Natural Science, and Social Science at Michigan State University. Females held 7% of the titled leadership positions available in the College of Engineering, 14% in the College of Natural Science, and 38% in the College of Social Science. There were no female department heads in the Colleges of Engineering and Natural Science. Table 7. Distribution of STEM Leadership Positions by Gender for Michigan State University, October 2011 Data Dean Assoc. Dean Female Asst. Dean Dept. Head Dean College of Natural Science Assoc. Dean Male Asst. Dean Dept. Head Total College of Engineering Total College of Social Science Total Total Director Female Asst. Director Director College of Natural Science Male Asst. Director Total Total College of Engineering Total College of Social Science Total STEM Categories Biological and Agricultural Sciences Change in Faculty Composition by NSF Category In units of Biological and Agricultural Sciences, female faculty represented 27.8% of the tenured and tenure-track faculty in 2007 prior to the ADAPP initiative (Table 9). Five new assistant professors and 2 new associate professors were added to the female faculty. Five new assistant professors and 2 new full professors were added to the male faculty. During the same time period, 2 female assistant professors, 1 female associate professor, and 7 male faculty (4 assistant professors, 2 associate professors, and 1 full professor) left MSU for voluntary, non-retirement reasons. Six female and 12 male professors retired from Biological and Agricultural Sciences. As a result of hiring and attrition the percentage of females in tenured and tenure-track positions increased by 1.2% in Biological and Agricultural Sciences (Table 9). Five female assistant professors were promoted to associate and received tenure; 6 male assistant professors were promoted to associate and received tenure; and 2 associate professors of each gender were promoted to full professor. The net result of these changes was an overall slight decrease in the percentage of full professor (-0.1%) faculty positions held by females from project Year 1 to Year 3, but Evaluation of MSU ADAPP ADVANCE 19

34 a 2.7% increase in associate professor positions and a 3.4% increase in assistant professor positions held by females (Table 8). Earth and Atmospheric Sciences The number of Earth and Atmospheric faculty is low, thereby resulting in large percentage changes for small numerical changes. No hiring occurred in this category, but one male full professor retired in Loss of this one faculty member increased the percentage of the female tenured and tenure-track faculty by 1.1% during the first 3 years of the ADAPP-ADVANCE project (Table 9). The only faculty movement through promotion in the Earth and Atmospheric Sciences between 2008 and 2011 was a single promotion of a female faculty member from assistant to associate with tenure. This explains the large percentage decrease in female assistant professors and the large percentage increase in female associate professors for this category (Table 8). Engineering There are significantly fewer female faculty than male faculty in units of the Engineering category at MSU. In October 2008, women represented 9.8% of the tenured and tenure-track faculty, which is the second smallest percentage in the STEM categories (women represent 7.7% of Physical Sciences faculty, Table 9). Therefore, small changes in the number of female Engineering faculty appear as a large percentage change, whereas large changes in the number of male faculty are necessary to change their percentages significantly. Between 2008 and 2011, 3 female assistant professors, 9 male assistant professors, and 3 male full professors were hired in Engineering. During the same time period, 1 female associate professor, 5 male assistant professors, and 1 male full professor left for voluntary, nonretirement reasons and 2 male full professors retired. As a result of hiring and attrition, the percentage of women in tenured and tenure-track positions increased 1.1% to 10.9% (Table 9). Promotions favored male faculty in Engineering. Between 2008 and 2011, 10 male assistant professors received a promotion to associate with tenure and 8 tenured associate professors were promoted to full professor. Two female assistant professors were promoted to associate with tenure. The resulting change in the gender composition of faculty in the Engineering category between 2008 and 2011 appears as a loss of female full professors (although there was actually no change in the number of female full professors), a small gain in the percentage of female associate professors, and a moderate gain in the percentage of female assistant professors (Table 8). Mathematical and Computer Sciences The number of female faculty in the Mathematical and Computer Sciences category only increased by 1 female assistant professor hired in Male faculty, however, gained 3 assistant professors, lost 2 associate professors to voluntary, non-retirement attrition, and lost 1 full professor to retirement. Therefore, the composition of female faculty in the tenured and tenure-track group increased 1.2% from 17.3% to 18.5% (Table 9). Male faculty also received promotions in Mathematical and Computer Sciences, while females did not. Between 2008 and 2011, 6 male assistant professors were promoted to associate professor with tenure and 1 male associate professor was promoted to full professor. The result of these changes is a loss for female faculty at the associate professor rank (1.7%), a small gain for female full professors (1.1%), and a large percentage gain in female assistant professors (9.0%) (Table 8). Physical Sciences The percentage of female faculty holding available tenured and tenure-track positions in Physical Sciences is the lowest among the STEM categories at MSU (7.7%, Table 9). A single new hire (assistant) for Year 1, a loss due to retirement (full professor) between Year 1 and Year 2, and a loss for voluntary, non-retirement attrition reasons between Year 2 and Year 3 were the only changes that occurred in the number of female tenured and tenure track faculty in this category. Seven male assistant professors were hired; one assistant and 1 full professor left MSU for voluntary, non-retirement reasons; and 4 full Evaluation of MSU ADAPP ADVANCE 20

35 professors retired. Mainly due to the large amount of male faculty hiring, the change in percentage of tenured and tenure-track faculty positions held by women decreased by 1.0% to 6.7%. Only male faculty were promoted in Physical Sciences between 2008 and 2010 and only female faculty were promoted during Four male and 1 female assistant professors were promoted to associate professor with tenure, and 2 male, tenured associate professors were promoted to full professor. The net result of these changes was an overall decrease in the percentage of assistant professor (-9.9%) and full professor (-1.3%) faculty positions held by females from project Year 1 to Year 3, but a 3.4% increase in associate professor positions held by females (Table 8). SBS Categories Professional/Other In all three SBS categories, females comprise more than 30% of the faculty (Table 9). The Professional/Other category is the only NSF category in which there are more female faculty (58.2%) than male. However, during the academic year, female faculty made up a much larger portion of the assistant professor rank (82.6%) than did their male peers and a much smaller portion of the full professor rank (34.8%). During the first 3 years of the ADAPP-ADVANCE project, 5 female assistant professors were hired and 2 male assistant professors were hired. However, 3 female assistant professors left MSU between 2008 and 2011 due to voluntary, non-retirement reasons, and 1 female associate professor retired. Two male associate professors and 2 male full professors retired from MSU during that time. Due to hiring and attrition, the percentage of tenured and tenure-track positions held by females in the Professional/Other category increased by 2.4%. Females received more promotions than males between the baseline year and Year 1 (3:1) and between Year 1 and Year 2 (2:1), but fewer promotions than males between Year 2 and Year 3 (1:3). Because of net loss of female assistant professors (5 hires, but 4 promotions and 3 losses due to VNR) and a slightly smaller net loss for males at this rank, the overall result for female assistant professors during the years of the ADVANCE project appears as a loss (Table 8). An equal number of promotions from associate to full professor for females and males along with a slightly larger amount of attrition for male professors at this rank appeared as an overall percentage gain for female faculty at the associate rank (4.8%). An equal amount of change due to promotions and a large number of retirements result in a percentage gain (6.9%) for females at the rank of full professor. Psychology Of the eight NSF categories, Psychology is the most equal in terms of faculty gender composition. In October 2008, females represented 46.3% of the tenured and tenure-track faculty (Table 9) and 48.3% of full professors (Table 8). Over the duration of the project, more females were promoted than males; more females were hired than males; and fewer females were lost to attrition. Three female assistant professors and 1 female associate professor were hired while 1 male of each rank was hired between 2008 and One associate professor of each gender left MSU between the baseline year and Year 1 for voluntary, non-retirement reasons and three male full professors left between Year 1 and Year 3. Attrition and hiring resulted in the largest increase (3.7%) in female faculty in tenured and tenure-track positions of all eight NSF categories (Table 9). When promotion data were added, with 3 female assistant professors promoted to associate professor, two female associates promoted to full, and only 1 male promoted from assistant to associate professor, the overall change in faculty composition appears as a gain of female professors at all ranks (Table 8). Social Science A large amount of change occurred in the tenured and tenure-track faculty during the years of the ADAPP-ADVANCE project for units of the Social Science category. According to October 2008 indicator data, assistant and associate faculty positions were relatively evenly distributed between males and females as opposed to a large difference in the composition of full professors (only 18.5% female). Over 3 years, females lost 2 assistant professors and 2 associate professors to voluntary, non-retirement Evaluation of MSU ADAPP ADVANCE 21

36 attrition, and 1 associate and 4 full professors to retirement. During the same time period, 9 female associate professors were hired. Males lost 1 professor from the assistant and associate ranks, and 2 full professors to voluntary, non-retirement attrition, 13 full professors and 2 associates to retirement, and gained 8 assistant professors, 2 associate professors and 3 full professors as new hires. The net effect on the percentage of female faculty in the tenured and tenure-track positions was a slight decrease of 0.1% over the 3 years of the ADAPP-ADVANCE project (Table 9). Between 2008 and 2011, 4 female assistant professors were promoted to associate professor with tenure, while 7 male assistant professors also were promoted. Five female associate professors were promoted to full professor, while 10 male associate professors were promoted. This significant amount of movement in the Social Science faculty translated into a slight increase (0.7%) in the percentage of females at the rank of full professor, a decrease (- 2.2%) in the percentage of females at the rank of associate professor, and no change in the percentage of females at the rank of assistant professor (Table 8). Evaluation of MSU ADAPP ADVANCE 22

37 Table 8. Change in Percentage of Faculty Positions Held by Women in Baseline, Year 1, Year 2, and Year 3 Data by Rank and NSF Category Evaluation of MSU ADAPP ADAPP 23 NSF Category Prof. a Assoc. Prof. Biological and Agricultural Sciences Earth, Atmospheric and Ocean Sciences Baseline: Year 1: Year 2: Year 3: Asst. Prof. Prof. a Assoc. Prof. Asst. Prof. Prof. a Assoc. Prof. Asst. Prof. Prof. a Assoc. Prof. Asst. Prof. Difference between Baseline and Year 3 * Prof. a Assoc. Prof. 24.1% 34.0% 33.3% 24.4% 37.7% 30.3% 23.0% 40.4% 32.1% 24.0% 36.7% 36.7% -0.1% 2.7% 3.4% 0.0% 50.0% 25.0% 0.0% 66.7% 0.0% 0.0% 66.7% 0.0% 0.0% 66.7% 0.0% 0.0% 16.7% -25.0% Engineering 5.5% 12.5% 14.8% 4.8% 15.4% 16.0% 4.8% 14.6% 19.0% 4.8% 14.3% 21.7% -0.7% 1.8% 6.9% Mathematical and Computer Sciences 20.9% 16.0% 7.7% 20.5% 16.0% 7.1% 20.5% 13.8% 9.1% 22.0% 14.3% 16.7% 1.1% -1.7% 9.0% Physical Sciences 4.5% 15.4% 18.2% 4.8% 15.4% 23.1% 3.2% 13.3% 30.0% 3.2% 18.8% 8.3% -1.3% 3.4% -9.9% STEM Total 15.4% 22.1% 21.7% 15.1% 25.6% 20.5% 14.2% 25.0% 23.3% 14.6% 23.9% 23.8% -0.8% 1.8% 2.1% Professional/Other 34.8% 57.1% 82.6% 37.5% 60.0% 81.8% 38.5% 66.7% 76.0% 41.7% 61.9% 81.0% 6.9% 4.8% -1.6% Psychology 48.3% 37.5% 47.1% 46.7% 40.0% 43.8% 51.7% 45.5% 43.8% 53.3% 40.0% 50.0% 5.0% 2.5% 2.9% Social Science 18.5% 40.3% 50.0% 18.5% 40.5% 46.6% 17.8% 38.8% 48.6% 19.2% 38.1% 50.0% 0.7% -2.2% 0.0% SBS Total 25.3% 43.6% 56.7% 25.5% 44.2% 53.2% 26.1% 44.8% 54.0% 27.7% 43.6% 56.1% 2.4% 0.0% -0.6% *Year 3 minus Baseline a Professor category included University Distinguished Professors and Endowed Professors. Asst. Prof.

38 As shown in Table 9, the percentage of women in tenured and tenure-track positions increased from baseline data to Year 3 in Biological and Agricultural Sciences, Engineering, Earth and Atmospheric Sciences, Mathematical and Computer Sciences, Professional/Other, and Psychology. The percentage of faculty positions held by women decreased in the category with the smallest percentage of female faculty in tenured and tenure-track positions (Physical Sciences) where there was only one less female faculty member but no change in the number of male faculty. The percentage of tenured and tenure-track female faculty also decreased in the Social Science category where the composition of the faculty changed dramatically during the years of the project. Overall, the percentage of female faculty in the five STEM NSF categories and the three SBS NSF categories increased in the tenured and tenure-track group and decreased in the non-tenure-track group between 2007 and Evaluation of MSU ADAPP ADVANCE 24

39 Table 9. Change in Percentage of Faculty Positions Held by Women in Baseline, Year 1, Year 2, and Year 3 Data by Tenure Status and NSF Category Evaluation of MSU ADAPP ADVANCE 25 Baseline: Year 1: Year 2: Year 3: Difference between Baseline and Year 3* NSF Category % of Women Tenured & in Tenure- Track % of Women in Non- Tenure- Track % of Women Tenured & in Tenure- Track % of Women in Non- Tenure- Track % of Women Tenured & in Tenure- Track % of Women in Non- Tenure- Track % of Women Tenured & in Tenure- Track % of Women in Non- Tenure- Track % of Women Tenured & in Tenure- Track Biological & Agricultural Sciences 27.8% 52.6% 28.6% 53.5% 28.6% 46.3% 29.0% 45.8% 1.2% -6.8% Earth & Atmospheric Sciences 14.3% 0.0% 14.3% 0.0% 14.3% 0.0% 15.4% 33.3% 1.1% 33.3% % of Women in Non- Tenure- Track Engineering 9.8% 60.0% 10.3% 0.0% 10.4% 0.0% 10.9% 0.0% 1.1% -60.0% Mathematical and Computer Sciences 17.3% 28.6% 16.9% 21.4% 16.7% 33.3% 18.5% 45.5% 1.2% 16.9% Physical Sciences 7.7% 18.2% 9.0% 16.7% 8.0% 25.0% 6.7% 22.2% -1.0% 4.0% STEM Total 18.1% 40.2% 18.7% 33.3% 18.4% 36.0% 18.6% 36.1% 0.5% -4.1% Professional /Other 58.2% 60.9% 59.1% 67.6% 59.4% 61.3% 60.6% 72.2% 2.4% 11.3% Psychology 46.3% 50.0% 44.6% 37.5% 48.2% 63.6% 50.0% 70.0% 3.7% 20.0% Social Science 32.0% 41.7% 31.8% 41.7% 31.3% 37.1% 31.9% 31.7% -0.1% -10.0% SBS Total 38.5% 49.3% 38.1% 49.5% 38.7% 47.1% 39.5% 48.6% 1.0% -0.7% * Year 3 minus Baseline

40 Match IHE Comparison Data Purdue University and the University of Iowa were selected as demographically comparable institutions based upon location (Midwest) and an initial comparison of faculty and student body composition and characteristics (% female and % minority). Indicator data were collected for these institutions through university contacts. A representative in the Office of the Executive Vice President and Provost from the University of Iowa and a Project Team member of the ADVANCE PCFS project at Purdue University completed the tables used in this report. Purdue University (also an NSF ADVANCE institution) and the University of Iowa have agreed to cooperate with this evaluation and provide data for this comparative study. To the extent possible, E & A Center staff attempted to collect comparable data from each site, so applied the following assumptions to data collection efforts: 1. Indicator data for MSU were collected for both STEM and SBS departments. 1 However, only colleges and departments typically identified as STEM (and recognized by the NSF as such) were included in the match analyses. Faculty exclusively serving professional schools (e.g., medical, dental) were not included. 2. The data do not represent adjunct faculty, instructors not identified as professors, or any staff appointments. It is possible that courtesy appointments are included in these data as it was not possible to distinguish these from the available data. 3. Dually appointed faculty (i.e., those working in two or more colleges) may be double counted but these individuals represent a small proportion of the population. 1 ADAPP Project Team members also collected data from departments in the College of Social Science. Evaluation of MSU ADAPP ADVANCE 26

41 Michigan State University Tables 10 and 11 display Year 3 indicator data collected for Michigan State University. Table 10 displays the faculty of Michigan State University by gender and rank. The College of Social Science at MSU had the highest percentage of female faculty (40.2%) and the highest proportion of female full professors (29.1%). The college with the lowest percentage of female faculty was Engineering with only 12.4% of the faculty being female and holding only 7.0% of the full professor positions. Table 10. Distribution of STEM Faculty by Gender and Rank for Michigan State University, October 2011 Data Female Male Department Asst. Assoc. Full Asst. Assoc. Full Total College of Natural Science Biochemistry & Molecular Biology Entomology Food Science & Human Nutrition Microbiology & Molecular Genetics Neuroscience Program Physiology Plant Biology Plant Pathology Zoology Chemistry Physics & Astronomy Geological Sciences Mathematics Statistics & Probability Total College of Engineering Chemical Engineering & Materials Science Civil & Environmental Electrical & Computer Mechanical Computer Science & Engineering Total Evaluation of MSU ADAPP ADVANCE 27

42 Female Department Asst. Assoc. Full Asst. Assoc. Full Total College of Social Science Family & Child Ecology School of Planning, Design & Construction Male Social Work Psychology Anthropology Criminal Justice Integrative Studies Economics Geography History Labor & Industrial Relations Political Science Sociology Total Table 11 displays the distribution of leadership positions by gender for the Colleges of Engineering, Natural Science, and Social Science at Michigan State University. Females held 7% of the titled leadership positions available in the College of Engineering, 14% in the College of Natural Science, and 38% in the College of Social Science. There were no female department heads in the Colleges of Engineering and Natural Science. Table 11. Distribution of STEM Leadership Positions by Gender for Michigan State University, October 2011 Data Dean Assoc. Dean Female Asst. Dean Dept. Head Dean College of Natural Science Assoc. Dean Male Asst. Dean Dept. Head Total College of Engineering Total College of Social Science Total Total Evaluation of MSU ADAPP ADVANCE 28

43 Director Female Asst. Director Director College of Natural Science Male Asst. Director Total Total College of Engineering Total College of Social Science Total Purdue University Tables 12 and 13 display indicator data collected for Purdue University. All faculty listed for Purdue University STEM departments were counted and are displayed by gender and rank in Table 12. The College of Science and the College of Agriculture at Purdue had the highest percentages of female faculty (18.5% and 19.4%, respectively) and the College of Agriculture had the highest proportion of female full professors (13.0%). The college with the lowest percentage of female faculty was Engineering with 12.6% of the faculty being female and only 7.4% being full professors. Table 12. Distribution of STEM Faculty by Gender and Rank for Purdue University, October 2011 Data Female Male Department Asst. Assoc. Full Asst. Assoc. Full Total College of Engineering Chemical Civil Electrical and Computer Materials Mechanical Total College of Agriculture (STEM departments) Agronomy Animal Sciences Biochemistry Botany & Plant Pathology Entomology Food Science Total Evaluation of MSU ADAPP ADVANCE 29

44 Female Male Department Asst. Assoc. Full Asst. Assoc. Full Total College of Science Biological Sciences Chemistry Computer Science Earth & Atmospheric Sciences Mathematics Physics Statistics Total Table 13 displays the distribution of leadership positions by gender for STEM departments at Purdue University. The College of Engineering employed the only female Dean. Females holding leadership positions comprised between 21% and 33% of the STEM leadership positions. The College of Agriculture had the highest percentage of females (33%) in leadership positions. Table 13. Distribution of STEM Leadership Positions by Gender for Purdue University, October 2011 Data Dean Assoc. Dean Female Asst. Dean Dept. Head Dean College of Engineering Assoc. Dean Male Asst. Dean Dept. Head Total College of Agriculture (STEM departments only) Total College of Science Total University of Iowa Tables 14 and 15 display indicator data for the University of Iowa. All faculty listed for University of Iowa STEM departments were counted and are displayed by gender and rank in Table 14. Only two colleges comprise the STEM departments at the University of Iowa. The Liberal Arts and Sciences faculty was made up of 24.3% females. Females represented 14.0% of full professors in this college. The College of Engineering had fewer women faculty (11.3%) and fewer women full professors (7.3%). Most female faculty were found at the rank of assistant professor across both colleges. Total Evaluation of MSU ADAPP ADVANCE 30

45 Table 14. Distribution of STEM Faculty by Gender and Rank for University of Iowa, October 2011 Data Biology Department Chemistry Department Computer Science Department Geoscience Department Mathematics Department Physics & Astronomy Department Statistics & Actuarial Science Department Female Male Asst. Assoc. Full Asst. Assoc. Full Total College of Liberal Arts and Sciences Total College of Engineering Chemical and Biochemical Engineering Civil and Environmental Engineering Electrical and Computer Engineering Industrial and Mechanical Engineering Total Table 15 displays the distribution of leadership positions by gender for the STEM departments at the University of Iowa. The College of Engineering had a higher percentage of women in leadership positions than did the College of Liberal Arts and Sciences (28.6% and 25%, respectively). The College of Liberal Arts and Sciences also employed a female Dean and a female Associate Dean. Evaluation of MSU ADAPP ADVANCE 31

46 Table 15. Distribution of STEM Leadership Positions by Gender for University of Iowa, October 2011 Data Dean Assoc. Dean Female Asst. Dean Dept. Head Dean College of Liberal Arts and Sciences Assoc. Dean Male Asst. Dean Dept. Head Total College of Engineering Total Total Match IHE Similarities Similarities between the three universities were found on analysis. At all three universities, there were more female faculty members in the sciences departments than there were in engineering. Females comprised between 19% and 24% of faculty in the sciences as compared to 11% to 13 % of faculty in engineering. Females in STEM departments for all three universities were found, primarily at the rank of assistant professor. In engineering at all three universities and in sciences at Purdue and Iowa, the largest percentage of females were found at the assistant professor rank (between 32% and 50%). MSU s College of Natural Science was the only college in which females made up a slightly larger percentage of associate professors (30%) than assistant professors (28%). Match IHE Comparison Indicator data quantifying female and male faculty distributions by rank and by leadership position were compared for Michigan State University (MSU), Purdue University (PU) and University of Iowa (UI). Only data collected from departments typically identified as STEM (and recognized by the NSF as such) were used for comparison. For each category (e.g., female full professor), a percentage of the total was calculated to compare among IHE. Tables 16 and 17 summarize data for the three universities. MSU had a larger percentage of female full professors than did PU and UI. Purdue University had an overall lower percentage of female faculty (17%) although not much lower than MSU (19%) and University of Iowa (20%). The distribution of females in leadership positions of Dean and Associate Dean was also similar among the three universities; however, MSU STEM departments reported no Assistant Deans nor female Department Heads. This lack of female leadership representation accounted for MSU s much lower percentage of females in titled leadership positions (6%). The University of Iowa and Purdue University had the same percentage of females holding leadership positions (27%). Table 16. Comparative Distribution of STEM Faculty by Gender and Rank for Michigan State University and Match Institutions, October 2011 Data MSU PU UI Female Male Asst. Assoc. Full Asst. Assoc. Full Total All STEM Colleges and Departments % 7% 8% 12% 20% 49% % 5% 6% 12% 20% 51% % 4% 6% 11% 21% 48% 248 Evaluation of MSU ADAPP ADVANCE 32

47 Table 17. Comparative Distribution of STEM Leadership Positions by Gender for Michigan State University and Match Institutions, October 2011 Data MSU PU UI Dean Assoc. Dean Female Asst. Dean Dept. Head Dean Assoc. Dean Male Asst. Dean Dept Head All STEM Colleges and Departments % 3% 7% 30% 57% % 6% 4% 15% 4% 17% 52% % 11% 11% 5% 16% 53% Total Match IHE data from the Year 3 analysis were compared to the Year 4 analysis to assess MSU s progress towards project goals relative to similar institutions. The total number of faculty counted for the match IHE comparison was lower for MSU in Year 4 (Table 18). University of Iowa female faculty made the most gains in terms of faculty positions, with a 1.0% increase in faculty positions held. MSU s percentage of female faculty positions increased by 0.2% and Purdue University female faculty positions increased by 0.4%. The total number of available leadership positions increased for Purdue University and decreased for University of Iowa and MSU (Table 19). Even with a decrease in the number of positions available, MSU s percentage of leadership positions held by women increased by 1.2%, whereas Purdue female faculty held 3% fewer of the available leadership positions, and UI female faculty held 6.9% fewer in Table 18. Comparative Change in STEM Faculty Composition by Gender and Rank for Michigan State University and Match Institutions, Year 3 to Year 4 Female Male Asst. Assoc. Full Asst. Assoc. Full All STEM Colleges and Departments Change in # of Positions Available MSU 0.4% -0.3% 0.1% 1.0% 0.2% -1.4% -5 PU -1.6% 0.9% 1.1% -2.8% 1.4% 1.1% 29 UI 0.7% 0.4% -0.1% -0.5% 2.6% -3.0% 3 Table 19. Comparative Change in STEM Leadership Positions by Gender for Michigan State University and Match Institutions, Year 3 to Year 4 Dean Assoc. Dean Female Asst. Dean Dept. Head Dean Assoc. Dean All STEM Colleges and Departments Male Asst. Dean Dept. Head Change in # of Positions Available MSU 0.6% 0.6% NC NC 1.1% 5.0% NC -7.2% -6 PU -0.5% -1.5% -1.0% 8.1% -1.0% -7.1% NC 3.1% 11 UI 0.7% 6.0% -13.6% 1.4% 0.7% 2.2% NC 2.6% -3 Evaluation of MSU ADAPP ADVANCE 33

48 Faculty Retention Study In Spring 2012, Ohio s Evaluation and Assessment Center analyzed human resources data on 1,415 tenure-system STEM faculty employed by the university in 2001 or newly hired since 2001 into nine colleges. The voluntary attrition of these faculty members was tracked between 2001 and Faculty who left MSU before October 1, 2011 for non-retirement reasons were counted as not retained, and those employed October 1, 2011 were counted as retained. Faculty were categorized into three hiring cohorts: Cohort A hired between 1981 and 1990; Cohort B hired between 1991 and 2000; and Cohort C hired between 2001 and Table 20 shows the sample size for each hiring cohort by gender, while Table 21 shows the disaggregation of retention status by hiring cohort, by rank at hiring, and by gender. Table 20. Sample Size by Hiring Cohort and Gender, Faculty Retention Study, Hiring Cohort Gender Female Male Total Cohort A: Hired between Cohort B: Hired between Cohort C: Hired between Total Table 21. Sample Size by Hiring Cohort, Rank, Gender, and Retention Status, Faculty Retention Study, Hiring Cohort Beginning Rank Gender Retention Status Total Left MSU Retained Cohort A: Hired between ASST PROF Female Male Total ASSOC PROF Female Male Total FULL PROF Female Male Total Cohort B: Hired between ASST PROF Female Male Total ASSOC PROF Female Male Total FULL PROF Female Male Total Evaluation of MSU ADAPP ADVANCE 34

49 Hiring Cohort Beginning Rank Gender Retention Status Total Cohort C: Hired between ASST PROF Female Male Total ASSOC PROF Female Male Total FULL PROF Female Male Total Non-parametric survival analysis was conducted for each rank within each hiring cohort and compared by gender. For each hiring cohort, more assistant professors were hired than associate or full professors. In addition, more faculty hired between 1981 and 1990 had retired than those in the other two hiring cohorts. Therefore, statistical estimates for assistant professors hired between 1991 and 2000 and between 2001 and 2011 are more accurate. Figure 1 shows the Kaplan-Meier survival and hazard functions for assistant professors hired between 1991 and 2011 (combined data of Cohorts B and C). The survival curves show the percentages of the original population that remained at a given year. The shaded areas of the survival curves show that the 95% confidence intervals for men and women barely overlap, which suggests that the survival rates for women assistant professors were significantly lower than for men assistant professors. The attrition rates for both men and women were low during the first 3 years, while the attrition increased from Year 4 to Year 11 for women and from Year 4 to Year 8 for men. Men tended to leave at a lower rate in their posttenure stage (Years 8 and on) than in their pre-tenure stage. Women showed a similar trend, except for two sudden increases in attrition after Year 15. The data indicated that these two sudden increases in attrition rate in later years could be attributed to the departure of three senior women. The hazard functions show the instantaneous attrition rates, which peaked at Year 4 (7.13% were likely to leave MSU in the following year), Year 9 (7.83%), and Year 10 (7.22%) for women, and peaked at Year 4 (4.27%), Year 7 (5.60%), Year 8 (5.12%), and Year 12 (5.21%) for men. The mean time to departure was 14.6 years for women and was 15.9 years for men, with Log-Rank p = 0.006, Wilcoxon p = 0.004, and Tarone-Ware p = 0.005, as shown in Table 22. Evaluation of MSU ADAPP ADVANCE 35

50 Figure 1. Survival and hazard functions for assistant professors hired between 1991 and 2011 by gender (women in green, men in yellow). Evaluation of MSU ADAPP ADVANCE 36

51 Table 22. Comparisons of Time to Departure for Assistant Professors Hired between 1991 and 2011 by Gender, Faculty Retention Study, Gender Retention Status Left MSU Retained Total M SE 95% Confidence Interval Lower Bound Upper Bound Log Rank (Mantel- Cox) Breslow (Generalized Wilcoxon) Tarone- Ware Female Male Total Figures 2 and 3 show the Kaplan-Meier survival and hazard functions for assistant professors hired between 1991 and 2000 and for assistant professors hired between 2001 and 2011, respectively. In general, the survival functions showed relatively steeper declines (higher rates of attrition) at early times in faculty careers and more moderate declines at later times for all ranks, in all hiring cohorts. Highest rates of attrition generally occurred between Years 5 and 10. The hazard functions showed the percentage of attrition at a given time for those faculty members who had been retained at MSU for the given period of time. There were no statistically significant differences between the retention rates of men and women. However, analyses for assistant professors hired between 1991 and 2000 showed that women had a steeper decline in their retention rate than did men did during Years 6 to 10 (Figure 2). Similarly, women assistant professors hired between 2001 and 2011 had a steeper decline in their retention rate than did men during Years 3 to 6 (Figure 3). Table 23 shows the comparisons of time to departure by gender for each rank within each hiring cohort. Though most findings are not statistically significant (p <.05), potential gender differences in the survival times for Cohorts B and C assistant professors, and for Cohort C associate professors are noted to be of practical significance. In all three cases, women faculty tended to leave earlier than male faculty. Evaluation of MSU ADAPP ADVANCE 37

52 Figure 2. Survival and hazard functions for assistant professors hired between 1991 and 2000 by gender (women in green, men in yellow). Evaluation of MSU ADAPP ADVANCE 38

53 Figure 3. Survival and hazard functions for assistant professors hired between 2001 and 2011 by gender (women in green, men in yellow). Evaluation of MSU ADAPP ADVANCE 39

54 Table 23. Comparisons of Time to Departure by Gender for Each Rank and Hiring Cohort, Faculty Retention Study, Evaluation of MSU ADAPP ADVANCE 40 Hiring Cohort Cohort A Cohort B Cohort C Beg. Rank ASST PROF ASSOC PROF FULL PROF ASST PROF ASSOC PROF FULL PROF ASST PROF ASSOC PROF FULL PROF Gender Retention Status Left MSU Retained Total M SE 95% Confidence Interval Median Lower Bound Upper Bound Log Rank (Mantel-Cox) Breslow (Generalized Wilcoxon) F M Total F M Total F M Total F M Total F M T F M Total F M Total F M Total F M Total Tarone -Ware

55 Survival analyses also were conducted by gender for each rank and hiring cohort for each of the 9 colleges providing tenure homes to STEM faculty. Analyses suggested possible gender differences for full professors hired between 2001 and 2011 in the College of Agriculture, for full professors hired between 1991 and 2000 in the College of Engineering, for full professors hired between 1991 and 2000 and associate professors hired between 2001 and 2011 in the College of Human Medicine, and for assistant professors hired between 1991 and 2000 and between 2001 and 2011 in the College of Natural Science. With one exception for full professors hired between 1991 and 2000 in the College of Human Medicine, all of these differences showed higher rates of attrition for women. However, interpretation of the results at the college level should be done with caution due to small sample sizes. Since the data used to create the models only tracked faculty attrition since 2001, faculty departures prior to 2001 did not inform survival modeling. The data set was both left censored and right censored, i.e., all faculty already working at MSU in 2001 were included in the data set and the eventual times to departure were unknown for faculty who were still employed at MSU in The ideal data set should include only newly hired faculty starting from a certain year going forward (e.g., starting from 1990 to the present) and track attrition over time. As more data are added to this model over time, comparisons can be made between the retention of faculty hired prior to, during, and following the university s targeted efforts to improve faculty quality and diversity. Currently, the model provides important and timely data that can inform areas for further investigation. Further analyses will provide a clearer picture of issues that may impact the retention of STEM women at Michigan State University. Comparisons of Faculty Responses on the MSU Work Environment Survey by Retention Status Sixteen tenure-system STEM faculty completed the MSU Work Environment Survey in Spring 2009 and departed MSU between Spring 2009 and Fall A total of 573 tenure-system STEM faculty also completed this questionnaire and have stayed at MSU at least until October Analyses were performed to determine if there were any measureable differences between the responses of those faculty who stayed and those who departed MSU. First, responses to the MSU Work Environment Survey were compared between the 16 departed faculty and the 573 retained faculty using Mann- Whitney U-tests. As shown in Table 24, there were a number of statistically significant differences between the responses of STEM faculty who departed and those who stayed. In all cases, departed faculty had less positive perceptions of the work environment than did retained faculty. Departed faculty, compared to retained faculty, agreed less often that the recruiting and hiring processes for their units were fair, consistent, and diversified; that they had a clear understanding of the promotion and tenure processes within their units and that the standards used were objective and consistently applied; that the faculty performance evaluation and review processes within their units were fair and that the criteria were clearly communicated; that the climate within their units was diversified in general; that they were treated with respect by students and colleagues within their units; that the allocation of resources and workload was satisfying; and that they were satisfied at MSU in general. Evaluation of MSU ADAPP ADVANCE 41

56 Table 24. Comparisons of Rasch Mean Subscale Scores by Retention Status for the Spring 2009 MSU Work Environment Survey, Faculty Retention Study, Subscale Retention n M SD Mean Rank Mann- Whitney U Recruiting and Hiring Process Left MSU Retained Promotion and Tenure Left MSU Retained Annual Performance Evaluation/Review Left MSU Retained Diversity Issues at MSU: General Climate Left MSU Retained Diversity Issues at MSU: Women Left MSU Retained Diversity Issues at MSU: Faculty of Color Left MSU Conditions and Relationships in Your Unit: Respect Conditions and Relationships in Your Unit: Openness/Transparency Conditions and Relationships in Your Unit: Work Family Balance Retained Left MSU Retained Left MSU Retained Left MSU Retained Resource and Workload Allocation Left MSU Retained Workplace Incivility Left MSU Retained Sexual Harassment at MSU Left MSU Retained Beliefs and Attitudes about MSU Left MSU Retained Satisfaction at MSU Left MSU Retained p A limitation of this analysis was that a large portion of the retained faculty was hired earlier than most of the departed faculty and the hiring environment of colleges could have changed over time. To refine the analysis, a more representative subset of retained faculty was selected to match the departed faculty for the comparisons. Faculty were matched on year of hiring and hiring college for the 16 departed faculty. All faculty hired in the same year and by the same college were selected into the subsample set. For example, if a faculty member hired in 1998 by the College of Agriculture completed the MSU Work Environment Survey and departed afterwards, then all faculty hired by the College of Agriculture in 1998 and retained until October, 2011 were selected into the matched subsample. Thirty-nine retained faculty hired between 1989 and 2009 by seven colleges were selected into this subsample, matching with the 16 departed faculty meeting the same criteria for year and college of hiring. Mann-Whitney U-tests were used to compare the responses between the departed faculty and the matched retained faculty. As Evaluation of MSU ADAPP ADVANCE 42

57 shown in Table 25, while there were no statistical significant differences between the responses of retained and departed faculty on any of the 14 subscales, in all cases the responses of departed faculty were less positive. Table 25. Comparisons of Rasch Mean Subscale Scores by Retention Status for the Spring 2009 MSU Work Environment Survey, Matched Faculty Only, Faculty Retention Study, Subscale Retention n M SD Mean Rank Mann- Whitney U Recruiting and Hiring Process Left MSU Retained Promotion and Tenure Left MSU Retained Annual Performance Evaluation/Review Left MSU Retained Diversity Issues at MSU: General Climate Left MSU Retained Diversity Issues at MSU: Women Left MSU Retained Diversity Issues at MSU: Faculty of Color Left MSU Retained Conditions and Relationships in Your Unit: Respect Left MSU Retained Conditions and Relationships in Your Unit: Openness/Transparency Left MSU Retained Conditions and Relationships in Your Unit: Work Family Balance Left MSU Retained Resource and Workload Allocation Left MSU Retained Workplace Incivility Left MSU Retained Sexual Harassment at MSU Left MSU Retained Beliefs and Attitudes about MSU Left MSU Retained Satisfaction at MSU Left MSU Retained Comparisons also were made for women and men faculty in the subsample separately. No statistical significant differences were found between the retained and departed faculty s responses on any of the 14 subscales either for female or for male faculty. In all cases, women STEM faculty who left MSU had better perceptions of the work environment than did men faculty who left the university. Further, women faculty who departed generally had more positive responses than women who stayed. Figure 4 summarizes the subscale mean scores for female and male retained and departed faculty members. p Evaluation of MSU ADAPP ADVANCE 43

58 Figure 4. Comparisons of Rasch mean subscale scores by retention status and by gender for the Spring 2009 MSU Work Environment Survey, matched faculty only. Summary of Faculty Retention Study Findings While findings of this study may seem intuitive, their value should not be underestimated. This study has mined the university s own rich data sources in order to provide robust analyses that can support the university in confirming or refuting anecdotal information regarding the retention of STEM women faculty. The models provide a snapshot look at how successfully the university is addressing faculty retention issues. Over time, as more data are added to the models, they have the potential to predict with a high degree of accuracy retention outcomes for subgroups of faculty. Key findings of the analyses suggest: STEM women faculty, historically have left, and continue to leave the university at higher rates than do STEM men faculty of the same rank. Retention rates for STEM women assistant professors are significantly lower than for men assistant professors hired between 1991 and 2011 Cohorts B and C. At the university level, 39% of STEM women assistant professors and 28% of STEM men assistant professors hired between 1991 and 2000 departed MSU voluntarily; 20% of STEM women and 15% of STEM men assistant professors hired between 2001 and 2011 left the university voluntarily. While it may appear that retention is improving for Cohort C, it must be noted that Cohort C has been tracked for a very short period of time. Evaluation of MSU ADAPP ADVANCE 44

59 Attrition for this group may be higher, lower, or the same, as it is for Cohort B as time goes on. High risk of losing STEM women faculty, particularly assistant professors, extend over longer periods of time than does the high risk of losing STEM men faculty. Attrition rates for both men and women are low during the first 3 years. Higher attrition rates generally occurred between Years 5 and 10, during pre- and early-tenure. The attrition rate of STEM women faculty increases and remains high from Year 4 through Year 11, while the rate of STEM men faculty attrition increases slightly in Year 4 and drops significantly by Year 8. Patterns in STEM women faculty leaving the university are less easily explained than are patterns in STEM men faculty attrition. In general, attrition rates are higher at early times in faculty careers and more moderate at later times for all ranks, in all hiring cohorts. Men tended to leave at a lower rate post-tenure (Years 8 and on) than during their pre-tenure stage. Women showed a similar trend, except for dramatic unexplained increases in attrition later in their careers. STEM women assistant professors hired before 2000 tended to leave the university between Years 7 and 10, while women assistant professors hired after 2000 tended to leave between Years 3 and 6. Attrition rates peaked at Years 4, 9, and 10 for women and peaked at Years 4, 7, 8, and 12 for men. The average time to departure was 14.6 years for women, which is significantly shorter than the average of 15.9 years for men. Survival analyses also were conducted for each of the nine colleges providing tenure homes to STEM faculty. Analyses suggested possible gender difference in rates of faculty attrition, primarily at the rank of full professor in some colleges, including College of Agriculture (Cohort C), College of Engineering (Cohort B), and College of Human Medicine (Cohort B and also for associate professors in Cohort C). In the College of Natural Science, attrition rates of women assistant professors (Cohorts B and C) also were significantly higher than for male colleagues. Limitations of the current model include the inability to represent faculty attrition of those who left MSU prior to 2001; and the insufficiency of the sample size to provide a clear picture of faculty attrition at the college level. The ideal data set would include only data for newly hired faculty starting from a selected year going forward and track attrition over a longer period of time. Analyses of these data would indicate if ADAPP-ADVANCE project and other university initiatives aimed at retaining women faculty in STEM fields have resulted in positive outcomes for the university. As more data are added to this model over time, comparisons can be made between the retention of faculty hired prior to, during, and following these targeted efforts to improve faculty quality and diversity. Currently, the model provides important and timely data that can inform areas for further investigation. The evaluation team will synthesize and triangulate findings from these models with other data sources to provide a clearer picture of issues that may impact the retention of STEM women at Michigan State University. Evaluation of MSU ADAPP ADVANCE 45

60 Human Resources Processes and Policies Background The goals of the MSU institutional transformation initiative, Advancing Diversity through Alignment of Policies and Procedures (ADAPP), are to (a) increase the number of women recruited in the Colleges of Natural Science, Social Science, and Engineering, (b) improve their retention and advancement, and (c) improve the climate for women in these three Colleges in order to strengthen the scientific workforce through increased inclusion of women in science, technology, engineering, and mathematics (STEM). ADAPP proposes to accomplish these goals by implementing specific components of a strategic human resource management (SHRM) model. The initiative has focused on aligning strategic goals of units and colleges with the university-wide value of diversity and on implementing objective evaluation criteria for recruitment, advancement, and retention in order to reduce bias related to informal and subjective processes. The SHRM approach is unique as no previous NSF ADVANCE Institutional Transformation grant has adopted this model as its operational framework. Specifically, ADAPP addresses policies and procedures related to a) tenure and promotion, b) tenure system faculty annual review process, c) tenure system faculty recruitment/search processes, d) faculty leadership development, and e) faculty mentoring. In 2010, ADAPP team members and other university administrators were interviewed to determine the value of conducting a policy analysis for the project. Interview data suggested a need to assess the SHRM model as a vehicle for driving institutional change. Approximately half of those interviewed were moderately skeptical that aligning HR policies and practices would result in authentic integration of policies, practices, and decision making, particularly at the unit level. Based upon differential assessments of the potential and power of the SHRM model, interviewees differed regarding their expectations for likely outcomes of the project. While interviewees visions of how the culture and climate of MSU might change as a result of the project were positive and well-aligned with project goals of increasing inclusiveness and promoting a more supportive work environment, some perceived a disconnect between the HR focus of the project and specifically how the approach would result in improved recruitment and retention of the target population. The policy analysis plan was proposed, approved and undertaken. Study of MSU s human resources policies and procedures has focused thus far on policies related to faculty annual review and reappointment, promotion, and tenure at the university, college, and unit levels for the three ADAPP STEM colleges. Several sources of data have been and are being collected and analyzed to provide insight to the transformation of HR policy with particular attention to implementation, communication, accessibility, and alignment of policy with university-level HR principles. The approach of the policy analysis is rooted in a number of theoretical frameworks, including those employed in the analysis of public policy and human resource management policy, as well as frameworks that are responsive to issues of gender and ethnic diversity. The ADAPP values of quality, inclusiveness, transparency, objectivity, consistency, and alignment are clearly represented in the processes of this analysis. The study of MSU s human resources policies and procedures is being conducted in two phases. Phase 1 is a descriptive analysis of university-level HR principles, and college- and unit-level policy documents focused on transparency, consistency, and alignment. Additional data informing Phase 1 were collected from the unit-level human resources policy inventories conducted in 2009 and Phase 2 of the policy study will provide an assessment of current policy at each level based upon evidence of quality, inclusiveness, and objectivity in written policy and its implementation. Additional data informing Phase 2 will include Work Environment Survey data collected in 2009 and 2013 and interviews of faculty and administrators. Evaluation of MSU ADAPP ADVANCE 46

61 Study of Human Resources Policies and Procedures - Phase 1 Phase 1 provides a descriptive analysis of current university-level HR principles, and college- and unitlevel policy documents focused on transparency, consistency, and alignment. The initial review of policy related to faculty performance evaluation and annual review provided the test case for this policy analysis approach. Following analysis of annual review (AR) policy, reappointment, promotion and tenure (RPT) policy was analyzed. The Phase 1 analysis has employed the following indicators in order to operationalize important ADAPP and university principles. Indicators of Transparency include: Access to relevant policy by current faculty and faculty candidates Modes and media of policy communications Specificity of policy communications Timeliness of policy communications Explicitness of policy processes Indicators of Consistency include: Lack of omissions/gaps in college/unit policy Lack of incongruent/conflicting college/unit policy Presentation and dissemination of college/unit policy Specificity of college/unit policy Congruence of documented procedures and practices within college and across units, if appropriate Indicators of Alignment include: Relationship among university values, university strategic objectives, and university HR principles Congruence of procedures and practices with written policy within colleges/units (as reported by administrators) Congruence of college- and unit-level policy with university HR principles Coordination of HR policies and practices across colleges/units Consistency in use of language/terminology across levels The following questions guided Phase 1 of the analysis: 1. To what extent do university HR principles represent the values and strategic objectives related to diversity? 2. To what extent are college- and unit-level policy documents accessible to current and prospective faculty? 3. To what extent are college- and unit-level policy documents written in clear and specific terms? 4. To what extent are college- and unit-level policy communications provided to faculty at relevant times and through appropriate modes? 5. To what extent do college- and unit-level policy documents address all university HR principles? 6. To what extent are documented procedures congruent with college- and unit-level policies? 7. To what extent are college- and university-level policies aligned with university HR principles? Study of Human Resources Policies and Procedures - Phase 2 Completion of Phase 2 of the policy analysis will result in a policy study which integrates findings from the evaluation, specifically Work Environment Survey, Human Resources Policies and Procedures Unit Inventory, and interview data, within the policy analysis process. Outcomes of Phase 2 will include an assessment of policy implementation at the college and unit levels, to the extent that data are available. Phase 2 will apply indicators of quality, inclusiveness, and objectivity to determine the extent to which written HR policy is the primary driver of HR processes and decision making across colleges and units. This Phase also will identify how gaps in policy revealed in Phase 1 may impact policy implementation. Evaluation of MSU ADAPP ADVANCE 47

62 Phase 2 of the policy analysis will begin in Summer 2012 and will be focused on impact, as well as process, and will attempt to probe policy implementation at the unit level. Process evaluation, or implementation assessment, determines whether the policy is delivered as intended to the targeted recipients. Evaluation to measure the impact of policy will consider change in the defined outcome measures or goals of the project. Qualitative and quantitative data will be collected and analyzed to look for evidence of change in policy processes and to inform recommendations at the unit level. Final and actionable recommendations could be delivered in January Phase 2 of the policy analysis (if continued during a no-cost extension year) will provide an opportunity to examine how the university and each of the three colleges make meaning of findings and recommendations that will emerge from Phase 2. Continuing the evaluation into 2014 will permit evaluators to observe how recommendations are assimilated into practice. The Phase 2 analysis will focus on the following indicators in order to operationalize important ADAPP principles with the goal of assessing policy and practice. Indicators of Quality: Use of best practice in HR Evidence in results and positive outcomes Indicators of Inclusiveness: Policy is free of bias in language/terminology Policy makes explicit reference to value of diversity Indicators of Objectivity: Level of review of decisions is consistent and appropriate Evidence of fairness in HR decision making Congruence of procedures and practices with written policy within colleges/units (as reported by faculty) The following questions will guide Phase 2 of the analysis: 1. What are the nature and extent of ADAPP and university impact on college- and unit-level annual review and reappointment, promotion, and tenure policies and procedures with regard to: Transparency Consistency Alignment Quality Inclusiveness Objectivity 2. What is the evidence that such efforts have resulted in improved retention and advancement of STEM women faculty in the Colleges of Engineering, Natural Science, and Social Science? 3. What recommendations can be made regarding developing, monitoring, and evaluating the continuous improvement and consistent implementation of AR and RPT policies across the three colleges? Study Methods and Data Sources For the Year 4 evaluation, three sources of data were used to evaluate the transparency, consistency, and alignment of human resources policies and processes in the three STEM colleges at MSU. The sources of data included (a) the 2009 MSU Annual Review Inventory and Reappointment, Promotion, and Tenure Inventory (Appendices B and C), (b) the 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment Promotion and Tenure Combined Inventory (Appendix A), and (c) the Phase 1 policy analysis for each college. Evaluation of MSU ADAPP ADVANCE 48

63 Human Resources Inventory Data Collection ADAPP-ADVANCE addresses human resources processes used for faculty recruitment, advancement, and mentoring and related to this goal has collected data from each unit in each of the three STEM colleges regarding annual review and reappointment, promotion, and tenure processes. This information will be used to guide long-term program and policy decisions across the university. Initial surveys were administered to all department/unit heads in Spring 2009 to collect data on unit-level HR policies and practices related to annual review, reappointment, promotion, and tenure, and mentoring. These data were collected by three separate inventory instruments. In 2010, the evaluation team recommended that HR data collection be streamlined by the revision and merging of the AR and RPT inventories into a single, concise instrument. The revised instrument was administered to all departments/units in the three ADAPP colleges in Fall and Winter The purpose of the revised instrument was to measure change in HR practices at the mid-point of the ADAPP initiative by reframing inventory items using data collected by the original inventory instruments. Several modifications were made to the original instruments to facilitate this goal. Based on respondents data, revisions were made that provided response choices aligned with collected data and what was learned about emerging best practices. Combining the AR and RPT inventories into a single instrument reduced respondent survey load and made it possible to assess inseparable HR processes more holistically. Items that were found not to provide useful and/or reliable data were removed from the survey but with an eye to maintaining items that could be cross-walked to other data sources, primarily the Work Environment Survey which will be re-administered in Spring Further, revisions used operational terms that are common across departments/units and disciplines, and respondents were provided more opportunities to explicate responses when they chose to do so. The revised instrument was vetted with a sample of unit chairs from each college as well as reviewed by the ADAPP team and personnel from the office of Academic Human Resources. Some data collected by the 2009 AR and RPT inventories served as pre-intervention data to analyze the extent to which units had modified HR processes, policies, or practices over the last two years. Table 26 shows the response rates for the three STEM colleges for each of the inventory instruments. The 2011 combined inventory differed from the 2009 inventories. Changes to the inventory items between the 2009 and 2011 administrations resulted in the inability to make direct comparisons of some 2009 and 2011 data. Tables 27 and 28 show crosswalks between the 2009 and 2011 inventory items that could be compared pre-intervention to mid-intervention. Evaluation of MSU ADAPP ADVANCE 49

64 Table and 2011 AR and RPT Inventory Completion, MSU ADAPP ADVANCE Policy Analysis COE Department Mechanical Engineering Computer Science and Engineering Chemical Engineering and Materials Science Civil and Environmental Engineering Biosystems and Agricultural Engineering Electrical and Computer Engineering CNS Department Physics and Astronomy Annual Review 2009 Inventory 2011 Inventory RPT Annual Review RPT Entomology Zoology Microbiology and Molecular Genetics Chemistry Statistics and Probability Biochemistry and Molecular Biology Physiology Food Science and Human Nutrition Geology Plant Biology Math Evaluation of MSU ADAPP ADVANCE 50

65 CSS Department Human Development and Family Studies Annual Review 2009 Inventory 2011 Inventory RPT Annual Review RPT Geography Sociology Labor and Industrial Relations History Psychology Social Work Political Science Criminal Justice Economics Anthropology Table and 2011 Annual Review Inventory Crosswalk, MSU ADAPP ADVANCE Policy Analysis Guidelines Is there a common format that each faculty member uses to submit information for their faculty annual performance evaluation/review? Are expected levels of faculty productivity clearly stated in writing and available to the faculty? Is faculty performance data collected using a standard format? How are expectations of faculty performance communicated to faculty at the unit level? Is there a departmental committee that collects and Who participates in the evaluation of faculty evaluates faculty effort and performance metrics? performance? Input Is input sought from mentors in the annual performance evaluation/review? During the annual review meetings do the faculty members have input in setting future goals and expectations? Is any input expected from faculty mentors for the annual review process? During the annual review process, does the faculty member have an opportunity to provide input in setting their future goals and expectations of professional accomplishments for future evaluation processes? Evaluation of MSU ADAPP ADVANCE 51

66 Feedback Does the Department provide written feedback to each faculty member summarizing the results of the annual review? Does the chairperson (or others) meet with pretenure faculty to discuss results of their annual review? Does the chairperson (or others) meet with any other faculty group (including mid-career tenured faculty) to discuss the results of the annual review? Does the feedback provided on the annual performance evaluations/review refer to the results of prior annual reviews? Are expectations for future performance that are necessary for advancement included in the written feedback to junior faculty? How is annual review feedback provided to the faculty member? Does the feedback provided on the annual review performance evaluation refer to the results of prior annual reviews? If applicable, are expectations of performance necessary for promotion included in the written feedback to the faculty member? Table and 2011 Reappointment, Promotion, and Tenure Inventory Crosswalk, MSU ADAPP ADVANCE Policy Analysis Guidelines Do these guidelines include policies regarding stopping the tenure clock? Is the group carrying out the RPT review detailed in the unit bylaws? If not, where is it detailed? Are specific and pre-specified written criteria provided to reviewers to analyze the strength of each candidate s application? If you answered Yes question #35, are those criteria the same for reappointment, promotion, tenure awards and promotions to full professor? Are external (to MSU) review letters required for: Reappointment, Promotion, Neither Beyond the required MSU Form D, what materials are required for candidates for: Reappointment, Promotion to associate professor tenure, Reappointment as associate professor with the award of tenure, Promotion from associate professor to professor? Materials Do your unit s RPT guidelines include references to university policy on stopping the tenure clock? Is the committee responsible for carrying out the RPT review of faculty described in the unit bylaws or other documents? Are specific evaluation criteria of professional accomplishments used by the RPT review committee to evaluate the strength of each candidate s specific application? Are there different criteria for faculty at different levels in the RPT process in your unit? Beyond MSU Form D, what materials are used for the first faculty reappointment? Beyond MSU Form D, what materials are used for faculty promotion with tenure? Beyond MSU Form D, what materials are used for faculty promotion after tenure award? How is the candidate advised on what materials to submit? What written criteria, if any, are used by your department to identify external letter writers? Does the candidate have input into the selection of external letter writers? Are the process for submission of and the materials required for RPT communicated to faculty members in writing and/or verbally? By whom are the formation of the pool of external reviewers for the RPT process selected? Evaluation of MSU ADAPP ADVANCE 52

67 Policy Analysis Phase I Data Collection Accessibility of university, college, and department/unit policy on annual performance review (AR) and reappointment, promotion, and tenure (RPT) was analyzed through website review February 2011 through May Policy documents were reviewed to compare institutional, college, and department/unit policies and to monitor change in such policies over the life of the ADAPP-ADVANCE project. In addition to a review of policy documents, accessibility of these documents and transparency in procedures were assessed through website review. The descriptive analysis of current university-level HR principles focused on transparency and was based upon the following indicators: (a) access to relevant policy by candidates and faculty, and (b) modes and media of policy communications. Annual review (AR) and reappointment, promotion and tenure (RPT) documents, relevant to tenured and tenure-track faculty, were the focus of this review. The university website ( was searched for accessible policy documents between February and April 2011 using a protocol developed by the Evaluation Team in consultation with the Center for Public Management and Regional Affairs (CPMRA) at Miami University. Evaluators searched the website for university-level AR and RPT documents starting with the MSU homepage. Homepage hotlinks such as, the About tab, the Faculty and Staff tab, and the Human Resources tab were utilized to navigate the website for these HR principles. In order to locate documents not readily available through the indicated hotlinks, the homepage search function was used by searching terms such as annual review policy or tenure policy. Consistently used key words were recorded. Details regarding how the documents were found also were recorded including number of clicks from the homepage and webpages necessary to obtain the document. If the search engine was utilized, a link map to the final document was created. Documents that were not available due to protected access were noted. Following the university-level access study, the ability to access university-level policies on AR and RPT through college websites was assessed following the same process. College- and department/unit-level policies on AR and RPT that could be located through website review were then identified. Department/unit-level policies that were not located through website review were requested from college and department/unit administrators through the ADAPP-ADVANCE project team. Policy Analysis Phase I Data Analysis College- and unit-level AR and RPT policies were assessed for alignment with university-level human resource principles: transparency (including accessibility and specificity), and consistency (including omissions and incongruence). University human resource principles were divided into three categories: (1) Policies on Expectations, (2) Procedural Policies, and (3) Policies on Follow-Up (Table 29, annual performance review; Table 30, reappointment, promotion, and tenure). Evaluation of MSU ADAPP ADVANCE 53

68 Table 29. MSU Annual Performance Review Policy University Principles by Category, MSU ADAPP ADVANCE Policy Analysis Policy Category University Principles Written performance criteria is clear and provided to the faculty member Policies on Expectations Procedural Policies Policies on Follow-Up Decisions should be clear to the faculty member with clear recommendations for future performance review and clarified future expectations All assigned duties are given weight in the evaluation and are known to the faculty member Faculty shall be evaluated annually Faculty Member should submit a written summary of activities Faculty should receive a written review Faculty has opportunity to respond to the review and add documentation within one month of receipt of written review Full Documentation will be recorded in the Personnel File Faculty has the right to Meet with Unit Administrator Table 30. MSU Reappointment, Promotion, and Tenure Policy University Principles by Category, MSU ADAPP ADVANCE Policy Analysis Policy Category University Principles Chairperson will deliver in writing, bylaws and policies that specify unit expectations regarding RPT Policies on Expectations Units should have explicit criteria and quality evaluations consistent with college and university policies and goals The achievement and performance level required must be competitive with faculties of leading research-intensive, land-grant universities of international scope (Peer Universities) Recommendations from unit Chairperson go through Dean, Provost, then President for approval Associate and Assistant professors will have probationary periods after appointment into the tenure system Procedural Policies Unit should provide a form for recording activities in instruction, research, and service Faculty members should be reviewed annually Faculty member must have a PhD or other qualifying degree to be appointed into the tenure system. College materials on RPT must include a procedure for establishing an RPT Review Committee Evaluation of MSU ADAPP ADVANCE 54

69 Policy Category University Principles Unit bylaws for each unit shall provide for a peer review committee for RPT External Letters of Reference are required for all reviews of tenure system faculty involving the granting of tenure or promotion following principles and procedures in the MSU Faculty Handbook Recommendation for promotion of tenured faculty have the same procedures as nontenured faculty in the tenure system Policies on Follow-Up Procedures shall exist by which faculty members may confer with the unit Advisory Committee on RPT prior to a decision In 2009, ADAPP-ADVANCE Project Team members analyzed the 2009 MSU Annual Review Inventory data and the 2009 MSU Reappointment, Promotion, and Tenure Inventory data as compared to the Annual Review Resource Guide and the Reappointment, Promotion, and Tenure Resource Guide that were created by the Project Team in the same year. Eight sets of matched items from the two iterations of inventory data collection (2009 and 2011) could be compared to the best practices used for this analysis. A crosswalk table (Table 31) was created in order to guide analysis of the change in practices, as described by department chairs in the inventory data, as they compare to ADAPP-ADVANCE best practices. Table 31. Best Practices and Inventory Item Crosswalk Best Practices Inventory Item 2009 Inventory Item 2011 Annual Review Guidelines Establish a set of performance criteria and standards for teaching, scholarly output and service that are clearly communicated to faculty. Communicate the workloads for annual evaluation to faculty prior to submission of materials, and make them readily available to faculty. Develop a written set of unit goals to be used in annual performance evaluation which are shared with and made readily available to all faculty members. Item 11 Are expected levels of faculty productivity clearly stated in writing and available to the faculty? Item 8 How are expectations of faculty performance communicated to faculty at the unit level? Responses d & e: In writing, in AR/RPT document or in new-hire paperwork. Evaluation of MSU ADAPP ADVANCE 55

70 Best Practices Inventory Item 2009 Inventory Item 2011 Annual Review Input Involve faculty members in setting expectations during performance reviews. Encourage faculty self-appraisal during the annual review process. Discuss prior year performance review & goals with the faculty member during the annual review process. Consider scheduling performance feedback meetings for all faculty; all pre-tenured faculty; all Assistant and Associate Professors The unit has written materials governing the appointment, promotion and tenure process and the procedure for establishing a unit-level RPT/merit review committee The process for stopping the tenure clock or for extensions of the probationary appointment is made readily available to the faculty. Item 21 During the annual review meetings do the faculty members have input in setting future goals and expectations? Feedback Item 18 Does the feedback provided on the annual performance evaluation/review refer to the results of prior annual reviews? Item 19 Does the chairperson (or others) meet with pre-tenure faculty to discuss the results of their annual review? Item 20 Does the chairperson (or others) meet with any other faculty groups (including mid-career tenured faculty) to discuss the results of the annual review? [20a] If 'Yes,' what group(s)? Reappointment, Promotion, and Tenure Guidelines Item 7 Is the group carrying out the RPT review detailed in unit bylaws? Item 27 Do these guidelines include policies regarding stopping the tenure clock? Item 19 During the Annual Review process, does the faculty member have an opportunity to provide input in setting their future goals and expectations of professional accomplishments for future evaluation processes? Item 21 Does feedback provided on the Annual Review performance evaluation refer to the results of prior annual reviews? Item 20 How is Annual Review feedback provided to the faculty member? A. Through a meeting initiated by the chair. Item 26 Is the committee responsible for carrying out the RPT process review of faculty described in the unit bylaws or other documents? b. Yes, in unit bylaws. Item 25 Do your unit s RPT guidelines include references (e.g., link to the university website) to university policy on stopping the tenure clock? Evaluation of MSU ADAPP ADVANCE 56

71 Best Practices Inventory Item 2009 Inventory Item 2011 Reappointment, Promotino, and Tenure Materials The unit guidelines regarding the number and type of external evaluation letters to be included in the performance review are clearly defined and readily available to all faculty. Description of the materials that must be submitted for the unitlevel RPT reviews for reappointment and promotion are readily available to the faculty. Item 16 What written criteria, if any, are used by your department to identify external letter writers? Item 21 How is the candidate advised on what materials to submit? Item 36a By whom are the formation of the pool of external reviewers for the RPT process selected? c. Through criteria in the unit bylaws. d. Through guidelines written in the MSU Faculty Handbook. e. Through college guidelines. Item 34 Are the process for submission of and the materials required for RPT communicated to faculty members in writing and/or verbally? Comparison Among Colleges MSU Website Annual review (AR) and reappointment, promotion, and tenure (RPT) documents were found at two access points originating from the university homepage: About and Faculty & Staff. Through the About page, a link to Bylaws through the Academic Governance page provided access to principles regarding the establishment of committees for faculty tenure. Other access to AR and RPT documents was found through the Human Resources page accessed through the Faculty & Staff link on the MSU homepage. The Faculty & Academic Staff link on the Human Resources page provided resource links for information and forms for benefits, hiring, awards and recognition, and grievance as well as links to policies on performance review, reappointment, and promotion. A Handbooks, Policies & Procedures link was also available from the Human Resources page. The Faculty Handbook could be accessed from this page, through which University and Academic Human Resources policies on AR and RPT were found under the Tenure System heading. These specific faculty handbook, tenure system policies also could be accessed directly from the Faculty & Academic Staff page through the Tenure System link located under the Performance Review & Development heading as well as the Tenure System Policies link under the Reappointment & Promotion heading. Additional AR forms and information were found on the Performance Review & Development page and additional RPT forms and information were found on the Reappointment & Promotion page (Figure 5). Evaluation of MSU ADAPP ADVANCE 57

72 Figure 5. University-level policy document accessibility map, AR and RPT, MSU ADAPP ADVANCE Policy Analysis. Transparency Accessibility A search for bylaws resulted in access to all three college bylaws on college websites (Table 32). CSS s bylaws were the oldest (last amended 2007). Other college documents that contained information on annual review (AR) and reappointment, promotion, and tenure (RPT) also were found on the CNS website. Table 32. Access to College Bylaws, MSU ADAPP-ADVANCE Policy Analysis CoE Department Found on Unit Website Delivered Electronically Bylaws last amended Additional AR/RPT Docs ENG 2009 CNS 2009 CSS 2007 Evaluation of MSU ADAPP ADVANCE 58

73 Alignment Relationship among university values, university strategic objectives, and university HR principles Strategic plan documents for Michigan State University were analyzed using theory driven content analysis. Themes identified were: (a) community, (b) support, (c) mentor, (d) research, (e) culture, (f) diversity, (g) STEM, (h) global, (i) women, (j) minority, (k) inclusive, and (l) underrepresented. These themes then were used to compare the six Task Force Reports (Table 33) as well as the strategic plans for the College of Engineering, College of Natural Science, and College of Social Science (Table 34). Based upon the quantification of terms, research was the focus of most of the task force reports. The term research was used more than twice as many times (164) as the next most frequently used term, community (75). Global and support also were used often although the use of support was distributed throughout the six reports while global was used mostly in the International Reach report. STEM, women, minority, inclusive, and underrepresented were rarely referred to in the six Task Force Reports. Table 33. Comparison of Term Usage for Task Force Reports, MSU ADAPP-ADVANCE Policy Analysis Student Experience Community Economic and Family Life International Reach Research Opportunities Stewardship Graduate Education Total THEME Community Support Mentor Research Culture Diversity STEM Global Women Minority Inclusive Underrepresented Strategic documents for two colleges 2 were analyzed using the same theme-based, quantitative process (Table 34). Research was the most referenced term (141) in the two reports, especially in the College of Engineering (ENG, 110) report. The College of Social Science (CSS) report referred to community, diversity, and inclusive more often than did the College of Engineering report. 2 Strategic documents for the College of Natural Science could not be located. Evaluation of MSU ADAPP ADVANCE 59

74 Table 34. Comparison of Term Usage for Selected College Strategic Plans, MSU ADAPP- ADVANCE Policy Analysis THEME College of Engineering College of Social Science Total Community Support Mentor Research Culture Diversity STEM Global Women Inclusive Underrepresented The Boldness by Design strategic positioning website clearly projected MSU s goals for the time frame. The Strategic Commitment of being recognized worldwide as the leading land-grant research university in the United States was reflected in all of the Task Force and Special Reports. Research was the number one theme across the strategic documents analyzed with more than 300 references to the term. Although inclusiveness was listed as one of the indicators for the strategic positioning of the university, strategic documents were lacking in their planning for addressing this indicator. A thorough search did not successfully locate an accessible strategic plan for the College of Natural Science (CNS). However, one of the five strategic imperatives from the Boldness by Design strategic position was referenced at the top of the CNS website homepage. A paragraph and a link titled Enhancing the Student Experience connected to a page that provided resource links for students of the college. One resource link, in particular, directly related to the goals of the ADAPP-ADVANCE project. The Charles Drew Science Scholars Program (DREW) website provided information on a program geared towards transforming the face of science. Congruence of college-level policy with university HR principles University-level AR and RPT policies were identified. University-level principles and guidelines were then compared to college-level policies that were available through website review to assess alignment. Table 35 displays the findings of the alignment analysis of college-level annual review policies. Table 36 displays the results of the analysis of RPT policies. Information regarding AR and RPT was located in the faculty handbook for the College of Engineering (ENG). The College of Natural Science (CNS) website included many accessible documents regarding AR and RPT including the college bylaws. Information regarding AR was not specified in college bylaws for the College of Social Science (CSS) and RPT information was limited. The CSS faculty handbook was not accessible through website review and no other AR or RPT documents were found online. Evaluation of MSU ADAPP ADVANCE 60

75 Table 35. Alignment of College-Level Policies with University Principles on Annual Review, MSU ADAPP-ADVANCE Policy Analysis University-level Principles & Guidelines ENG CNS CSS Faculty shall be evaluated annually X X NS Written performance criteria is clear and provided to the faculty member X X NS Decisions should be clear to the faculty member with clear recommendations for future performance review and clarified future expectations NS V NS All assigned duties are given weight in the evaluation and are known to the faculty member NS X a NS Faculty Member should submit a written summary of activities X X a NS Faculty should receive a written review V X NS Faculty has opportunity to respond to the review and add documentation within one month of receipt of written review NS NS NS Full Documentation will be recorded in the Personnel File NS NS NS Faculty has the right to Meet with Unit Administrator NS X a NS Note. NS Not Stated, V Vague, X Clearly Stated a Policies are only clearly stated in policy documents specifically for fixed-term faculty. Table 36. Alignment of College-Level Policies with University Principles on Reappointment, Promotion, and Tenure, MSU ADAPP-ADVANCE Policy Analysis University-level Principles & Guidelines ENG CNS CSS Chairperson will deliver in writing bylaws and policies that specify unit expectations regarding RPT X V NS Recommendations from unit Chairperson go through Dean, Provost, then President for approval Associate and Assistant professors (new to MSU) will have probationary periods after appointment into the tenure system Units should have explicit criteria and quality evaluations consistent with College and University policies and goals Unit should provide a form for recording activities in instruction, research, and service X NS NS X V NS X X NS X X NS Faculty members should be reviewed annually V X NS Faculty member must have a PhD or other qualifying degree to be appointed into the tenure system. NS NS NS Policies on Stopping the Tenure Clock NS NS NS Principals of Faculty Reassignment NS NS NS Policy on Modified Duties for New Dependent Children NS NS NS The achievement and performance level required must be competitive with faculties of leading research-intensive, land-grant universities of international scope (Peer Universities) College materials on RPT must include a procedure for establishing an RPT Review Committee V V NS X X X Evaluation of MSU ADAPP ADVANCE 61

76 University-level Principles & Guidelines ENG CNS CSS Unit Bylaws for each unit shall provide for a Peer Review Committee for RPT X X NS External Letters of Reference are required for all reviews of tenure system faculty involving the granting of tenure or promotion following principles and procedures in the MSU Faculty Handbook Procedures shall exist by which faculty members may confer with the unit Advisory Committee on RPT prior to a decision Recommendation for promotion of tenured faculty have the same procedures as non-tenured faculty in the tenure system X X NS X NS NS V V NS Note. NS Not Stated, V Vague, X Clearly Stated Comparisons in the College of Engineering MSU College of Engineering Website The College of Engineering (ENG) website provided a link for Faculty & Staff on which Human Resources and MSU links provided access to university-level AR and RPT documents (Figure 6). The Human Resources link provided access to university-level policies as described in the MSU website description. The page provided links to worklife information that included Academic Human Resources policies on Stopping the Tenure Clock and Tenure & Promotion. These two policy links access the Faculty Handbook which also can be accessed through the page through the Employee Handbooks link. Many AR and RPT policies can be accessed through the Faculty Handbook page as described in the MSU Website description. Evaluation of MSU ADAPP ADVANCE 62

77 Figure 6. University-level policy document accessibility map, annual review and RPT, MSU ADAPP ADVANCE Policy Analysis, College of Engineering. Six College of Engineering departments; (a) Mechanical Engineering (ME), (b) Computer Science and Engineering (CSE), (c) Chemical Engineering and Materials Science (CHEMS), (d) Civil and Environmental Engineering (CEE), (e) Biosystems and Agricultural Engineering (BAE), 3 and (f) Electrical and Computer Engineering (ECE) responded to the 2011 combined inventory. All six also responded to the 2009 Annual Review Inventory, but only four responded to the 2009 Reappointment, Promotion, and Tenure Inventory (CEE and BAE did not respond). Therefore, all six departments data could be matched and compared for changes between 2009 and 2011 for AR data but only four could be matched for RPT. All six department bylaws were eventually obtained for policy analysis. Transparency Accessibility A search for bylaws resulted in access to three department bylaws on department websites (Table 37). The remaining three department bylaws were delivered electronically on request. CHEMS bylaws were amended most recently, in BAE bylaws were the oldest (last amended 2005). Other department documents that may contain information on annual review (AR) and reappointment, promotion, and tenure (RPT) were not available online nor provided in hard copy. 3 BAE is a department in the College of Agricultural and Natural Resources with faculty who are jointly appointed in the College of Engineering. Evaluation of MSU ADAPP ADVANCE 63

78 Table 37. Access to Department Bylaws, MSU ADAPP-ADVANCE Policy Analysis, College of Engineering ENG Department Found on Department Website Delivered Electronically Bylaws last amended ME 2008 CSE 2006 CHEMS 2011 CEE 2006 Additional AR/RPT Docs BAE 2005 ECE 2006 Access to Information on the Annual Review Process According to 2011 inventory data, 67% of Engineering department chairs reported that the AR process is described in some detail in the department bylaws. ECE indicated that the process is only mentioned and CHEMS indicated that AR processes are not mentioned in the bylaws. Faculty annual review, as a process separate from reappointment, promotion, and tenure, was difficult to extract from department bylaws. Departments that described annual review as a stand-alone process included ME, CSE, CHEMS, and CEE. Of these departments, the most detail on the AR process was provided by CEE, followed by CSE. Both of these policies provided some detail. ME mentioned the AR process but did not describe it. CHEMS, although reported on the inventory to not mention AR in the bylaws, did mention the chair s responsibility to review faculty annually and provided a very brief description of the process. BAE indicated that the chair shall inform faculty annually of progress towards reappointment/promotion. ECE also described the duty of the chair to discuss faculty progress towards RPT annually and provided basis, weighting, and sources of evaluation. Other possible sources of annual review process information included AR/RPT documents and websites. ME, CEE, and BAE indicated that AR process information could be found in AR/RPT documents. The only AR/RPT document received for review was BAE s in which the expectations for annual faculty performance review as well as an example portfolio of materials were included. CSE and BAE indicated AR process information was available on the department website. The evaluator could not locate this information for either department. All departments indicated that a standard format for collecting faculty performance data was used. Two departments (CHEMS and CEE) mentioned the Annual Report of Faculty Load, Activities, Achievements, and Plans in department bylaws. ECE also mentioned a faculty annual report document in the bylaws. With the exception of CSE, who did not respond to the item, the same data were found in 2009 inventory data. Access to Information on the Reappointment, Promotion, and Tenure Process All Engineering departments, with the exception of CSE, indicated that department bylaws made reference to the RPT process. Policy analysis corroborated these data with the exception of CSE for which reference to the RPT process was found in the bylaws despite the chair s indication that it was not mentioned. CSE bylaws described the functions of the chair as they relate to RPT and the functions of the Tenure and Promotion Committee. Also, criteria for annual faculty merit evaluations and service load weight were included in the appendices of these bylaws. CEE indicated that the RPT process was mentioned in the department bylaws. CEE bylaws described the RPT process in some detail by including the basis for evaluation and a brief description of procedure along with forms to be used, where to find expectations, as well as two committees (Committee of Tenured Faculty and Promotion and Tenure Evaluation of MSU ADAPP ADVANCE 64

79 Committee) involved in the process. ECE and BAE indicated that the RPT processes were described in some detail in the department bylaws. Both department bylaws described the chair s responsibilities of reviewing faculty annually and making the final decision regarding RPT, they also described committees involved in the process, and they described, briefly, the procedure for the process. The same information could be found in the bylaws for ME and CHEMS; however, chair inventory data indicated that these departments described this information in much detail. All departments except CSE indicated that the department had an RPT document in which the RPT process was referenced. Of these five departments all of them except BAE indicated that the process was described in much detail. Only the BAE RPT document was obtained for evaluation. BAE indicated that the RPT document described the RPT process in some detail. The expectations and materials for submission were described in detail, however, the process itself was not described. Only two departments (ECE and BAE) indicated that the RPT process was referenced on the department website. This information could not be located. Access to Information on Review Committees for Annual Review Five of the six Engineering departments indicated that the department chair is involved in the AR process (Table 38). CEE did not indicate on the 2011 inventory that the department chair was involved in faculty review, however the department bylaws clearly stated primary responsibility for evaluating faculty members and informing them annually of their progress shall rest with the department chairperson. Also, CEE indicated in 2009 inventory data that the chair was involved in faculty AR. Only CSE, CEE, and BAE indicated that a standing committee is involved in the AR process. Distinction between AR, as a stand-alone process, and RPT, which includes AR, may explain the lack of consistency between policies and inventory data, but all departments mentioned a committee that is involved in faculty annual review. However, with the exception of ME, all of the committees are labeled as Tenure and/or Promotion committees and were described to have an advisory role in reappointment, promotion, and tenure. ME bylaws indicated a Review Committee but did not describe its role. Table 38. Review Participants for Annual Review, 2011 MSU Annual Review Inventory, College of Engineering ME Department Chair Standing Committee CSE CHEMS CEE BAE ECE Access to Information on Review Committees for Reappointment, Promotion, and Tenure Inventory data revealed where the description of RPT committees could be found. Most (83%) indicated that these committees were described in the department bylaws. Only ECE did not indicate that the bylaws described the committee; however, in the 2009 inventory data, ECE did indicate that the committee was described in the bylaws. Policy analysis indicated that for all six departments at least one committee involved with the RPT process was described in the bylaws. Three departments (ME, CHEMS, Evaluation of MSU ADAPP ADVANCE 65

80 and ECE) indicated that the committees were described in RPT documents as well, and ECE indicated that the committee was described on the department website. This information could not be confirmed. Access to Information Regarding Stopping the Tenure Clock Three departments (ME, CSE, and CHEMS) indicated that references to stopping the tenure clock were not included in the department RPT guidelines. The same information was found in the 2009 inventory data. Two departments (BAE and ECE), neither of which responded to the 2009 inventory, indicated that this information was included in RPT guidelines. Information on stopping the tenure clock was not found for either department in department bylaws nor in the BAE RPT document. Communication and Clarity Data regarding the communication and clarity of expectations were collected only by the annual review subscale of the 2011 combined inventory. Analysis of communication and clarity of expectations therefore was not specific to differences between expectations for annual review, reappointment, promotion with tenure, and promotion after tenure award. Communication of Expectations Most College of Engineering departments (83%) indicated that expectations of faculty performance are communicated verbally through a formal meeting with the department chair. BAE was the only department that did not indicate any form of verbal communication of expectations. Many (67%) indicated that written communication of expectations was provided through AR/RPT documents and half indicated that new hire paperwork included written expectations. The only AR/RPT document obtained for evaluation was from BAE and this document provided a detailed communication of expectations as indicated in inventory data from both 2009 and In 2009, ME and CSE indicated that expectations of faculty performance could be found in the department bylaws. Since the bylaws were not amended for these departments since 2008 or before, they should still refer to expectations. CSE bylaws included a detailed description of AR expectations in the appendices of the bylaws. No mention of expectations could be found in the ME bylaws. Clarity of Expectations A mean score was calculated for clarity of expectations using 2011 inventory data. The item asked to what extent are faculty provided specific descriptions of what constitutes above average, average, or unacceptable levels of performance regarding expectations for faculty performance in the areas listed. Response options ranged from Levels of performance are not mentioned (1) to Performance levels are fully described (4). Mean scores were given for each department by averaging the level of clarity scores for each of the six activities: (1) Research, (2) Teaching, (3) Service, (4) Membership in Disciplinary Societies, (5) Mentoring, and (6) Leadership in Disciplinary Societies. CEE reported the most clear descriptions of expectations with a mean score of 3.83 out of 4.0 (Figure 7). Mean scores also were given for the clarity of expectations for each type of activity for the College of Engineering (Figure 8). Departments indicated that expectations for Research (3.57) were communicated most clearly to faculty and expectations for Mentoring (2.33) were communicated least clearly. Evaluation of MSU ADAPP ADVANCE 66

81 Figure 7. Mean score for clarity of expectations for all six types of activities for each department in the College of Engineering. Figure 8. Mean score for clarity of expectations for the College of Engineering for each type of activity. Evaluation of MSU ADAPP ADVANCE 67

82 Consistency Consistency of Annual Review Procedures All College of Engineering department chairs indicated a systematic and consistent performance evaluation process for all faculty within the department. Also, all departments indicated using a standard format for the collection of faculty performance data. All departments except CHEMS indicated the use of a standard performance appraisal process specific to the department. CHEMS indicated that they evaluate faculty on an individual basis and do not use a standard performance appraisal process. Consistency of Reappointment, Promotion, and Tenure Procedures The 2011 inventory data showed that five 4 of the six departments indicated that there are different criteria for faculty at different levels in the RPT process. All six departments indicated that there are specific evaluation criteria of professional accomplishments used by the RPT review committee to evaluate the strength of each candidate s application. All six departments also indicated that college-level criteria are used by the department to develop the department criteria for RPT evaluations. This RPT criteria is provided to writers of external letters of review by five of the six departments. Consistency in the Use of Materials for Annual Review All departments indicated materials used for annual review included an updated CV and student ratings. None of the Engineering departments indicated the use of teaching portfolios for AR (Table 39). Table 39. Materials Required for Annual Review, 2011 MSU Annual Review Inventory, College of Engineering Updated CV Performance Appraisal Form Copies of Publications Citations Student Ratings Teaching Portfolio Personal Statement ME CSE CHEMS CEE BAE ECE Half of the Engineering departments indicated that submission process for AR materials is communicated to faculty through the department AR/RPT document. These respondents included BAE for which the AR/RPT document contained an example portfolio, but did not specifically address procedures for submitting materials. Only one Engineering department (CEE) indicated that the process for submission of and the materials required for AR are communicated to faculty through the department bylaws. CEE bylaws indicated the necessary documents to be submitted and the deadline for submission; however, nothing indicated that this procedure is the same for every type of evaluation (reappointment, promotion, tenure, or annual review). Candidates for all departments are not limited in the amount of materials that can be submitted for AR. 4 BAE did not respond to this item. Evaluation of MSU ADAPP ADVANCE 68

83 Consistency in the Use of Materials for Reappointment, Promotion, and Tenure According to 2011 inventory data, materials required for different outcomes of the RPT process do not differ significantly for departments in the College of Engineering. The only differences found were in the requirement for external letters. Letters were not required in any of the six departments for reappointment; were required for five of the six departments (not CEE) for promotion with tenure; and were required for all six departments for promotion after tenure award (Table 40). Data regarding external letters were the same in 2009 for all departments that responded to the inventory; external letters were required for promotion but not for reappointment. Data regarding other required materials varied slightly in 2009, possibly due to the difference in item type. In 2011 chairs were asked to choose from a list and in 2009, the item was open-response. For example, in 2009, ECE indicated that all candidates are required to prepare a personal statement and in 2011, this response was not chosen. In 2009, none of the departments indicated that citations were required, but in 2011 four departments chose this response option. Table 40. Materials Required for Reappointment (R), Promotion with Tenure (T), and Promotion after Tenure Award (P), 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Engineering Copies of Publication Citations Teaching Portfolio Personal Statement External Letters R T P R T P R T P R T P R T P ME CSE CHEMS CEE BAE ECE Only one department, CEE, indicated that a protocol for submitting materials for RPT is written in the department bylaws. The bylaws for CEE described the documents necessary for submission. Bylaws also described how the candidate will be notified to submit prior to recommendations. The remaining departments indicated that this information could be found in the department RPT document. The only RPT document obtained for evaluation was for BAE in which a protocol for submitting materials was not found. External letter writers for departments in the College of Engineering are, for the most part, selected through criteria and guidelines provided by university and college documents (Table 41). ME and ECE indicated that the candidate has input into the selection of the pool of reviewers. The 2009 data inventory indicated for all departments that the candidate had input. In the data for both 2009 and 2011, the department of ME indicated that the pool of external reviewers was selected through criteria written in the department bylaws. The ME bylaws described a separate document titled, Reappointment/Promotion Procedure ; however, criteria for the formation of the pool of external reviewers could not be found in the bylaws. From the pool, ME and CHEMS indicated that writers are chosen based upon college guidelines; CSE and ECE indicated that university guidelines are used, and BAE and ECE indicated that the department chair made the final decision. ME and ECE indicated that the RPT committee had input into both the selection of the pool and the final selection of letter writers. Data describing the final selection of external letter writers were not collected by the 2009 inventory. Evaluation of MSU ADAPP ADVANCE 69

84 Table 41. Selection of Pool and Final External Letter Writers for RPT, 2011 MSU Reappointment Promotion, and Tenure Inventory, College of Engineering Candidate Chair Guidelines in Department Bylaws Guidelines in MSU Faculty Handbook College Guidelines Other Pool Writer Pool Writer Pool Writer Pool Writer Pool Writer ME RPT Committee CSE CHEMS CEE BAE ECE RPT Committee Consistency in Modes of Input for Joint Appointments for Annual Review Inventory data indicated that joint reviews were completed in a different way for the six Engineering departments (Figure 9). BAE indicated that both departments were equally involved in the annual review of jointly appointed faculty. CSE indicated that the primary department/unit (Unit 1) was generally responsible but that expectations and input from the second department/unit (Unit 2) were collected. CHEMS and ECE indicated the primary department/unit was completely responsible but expectations and input were collected from Unit 2; and ME and CEE indicated that the primary department/unit evaluated faculty without input from Unit 2. BAE indicated that expectations and input were shared between the departments/units in a formal process that evaluated specific expectations for professional accomplishments. The remaining three departments with a joint review process indicated that input and expectations were shared in an informal manner with general expectations evaluated by department chairs. Evaluation of MSU ADAPP ADVANCE 70

85 Figure 9. How input is collected for joint reviews, College of Engineering. Evaluation of MSU ADAPP ADVANCE 71

86 Consistency in Modes of Input for Joint Appointments for Reappointment, Promotion, and Tenure The majority of departments (67%) indicated that for faculty with joint appointments, although the primary department/unit is responsible for the RPT review, the performance expectations of each department/unit are considered and input from each is collected (Figure 10). CSE, BAE, and ECE indicated that they share performance expectations and input in a formal review process, whereas ME and CHEMS indicated they share the information in an informal joint review process. Figure 10. How input is collected for joint reviews, College of Engineering. Evaluation of MSU ADAPP ADVANCE 72

87 Consistency in Modes of Mentor Input for Annual Review and Reappointment, Promotion, and Tenure As of March 2011, the university required colleges to develop policy for and implement a formal faculty mentoring program. University principles regarding this mentoring policy did not explicitly state that input from the mentor would be requested for faculty annual review inventory data indicated that none of the six Engineering departments expected input from faculty mentors for AR. This differs, however from 2009 data in which CHEMS and BAE indicated that input is sought from faculty mentors for AR. Both indicated in 2009 that input is sought through an informal process. In 2009 department leaders were asked whether or not input is sought from mentors and in 2011 expected from mentors. The language difference may explain the CHEMS and BAE data. Only CHEMS and ECE chairs indicated that input was expected from faculty mentors for RPT. CEE indicated that information was collected informally while CHEMS indicated that the mentor is a member of the RPT committee. Data regarding mentor input were not collected by the 2009 Reappointment, Promotion, and Tenure Inventory. Consistency in Providing Feedback for Annual Review Table 42 shows the different ways in which feedback is provided to the faculty member following the annual review process. Most departments (67%) indicated that a written review and a chair-initiated meeting were used to convey the AR feedback. ME and BAE indicated that they provided only written feedback. These data differ from data collected in 2009 in which both ME and BAE indicated that the chair met with faculty regarding the results of the review. All six departments indicated in both 2009 and 2011 that expectations of performance necessary for promotion were included in written feedback. Table 42. Types of Annual Review Feedback, 2011 MSU Annual Review Inventory, College of Engineering Verbal: Meeting with Chair Written Review Document Department Chair Initiated Requested Standardized Non- Standardized ME CSE CHEMS CEE BAE ECE Consistency in Providing Feedback for Reappointment, Promotion, and Tenure According to 2011 inventory data, department RPT committees provide feedback to the department chairs in similar ways. All departments indicated that they provide the chair with a summary in the form of a report or a letter. All departments except CEE indicated that within this document either a vote, ratings for review categories, or recommendations are included. Five of the six departments indicated that the candidate has an opportunity to discuss the results or recommendations of the RPT review. All departments except BAE (no response given) indicated that a meeting for each candidate is scheduled by the department chair to discuss the results. Congruence Between Inventory Data and Policy Analysis Congruence between inventory data and policy analysis (PA) findings for Annual Review (AR) is displayed in Table 43. In terms of AR references in department bylaws for the College of Engineering, human resources inventory and policy analysis data are somewhat congruent. ME inventory and PA data for AR Evaluation of MSU ADAPP ADVANCE 73

88 references were congruent. Although the department chair for CHEMS indicated there were no references to AR in the department bylaws, policy analysis data indicated AR was mentioned in the bylaws as a duty of the department chair. Congruence between inventory data and policy analysis (PA) findings for reappointment, promotion, and tenure (RPT) is displayed in Table 44. Inventory data and RPT policies in the College of Engineering were more congruent than were inventory data and AR policies. Although the department chair for CSE indicated there were no references to RPT in the department bylaws, policy analysis data indicated that RPT policies were described in much detail. Table 43. Congruence of Annual Review Inventory Information with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Engineering DEPT ME CSE Reference to AR in bylaws? Amount of Detail Where else is AR referenced? Yes Some Detail AR/RPT Document? Yes Some Detail Department Website? Evaluator Assessment Bylaw reference and detail level are congruent with PA data. AR/RPT documents were not obtained. Annual review, as a separate process from RPT, is not mentioned in the bylaws. Much detail is provided as AR pertains to RPT. References to annual review could not be located on the department website. CHEMS No NR NL Annual review is referenced in the department bylaws as a duty of the department chair. CEE BAE ECE Yes Some Detail AR/RPT Document Yes Some Detail AR/RPT Document and Department Website Yes Mention NL Bylaw reference is congruent with PA data. Detail level could be considered as in much detail rather than some detail. AR/RPT documents were not obtained. Annual review, as a separate process from RPT, was not mentioned in the bylaws. Annual review was only mentioned as a duty of the department chair. AR/RPT document provided guidelines and example materials, but references could not be found on the department website. Annual review, as a separate process from RPT, is not mentioned in the bylaws. Some detail is provided as AR pertains to RPT. NR Department chair did not provide a response to this item. NL Department chair indicated that AR or RPT is not referenced in any other document nor on the department website. Inventory and PA data are not congruent. Inventory and PA data are somewhat congruent.? Inventory and PA data are congruent. Document was not obtained for review.? Evaluation of MSU ADAPP ADVANCE 74

89 Table 44. Congruence of Reappointment, Promotion, and Tenure Inventory Information with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Engineering DEPT ME Reference to RPT in bylaws? Yes Amount of Detail Much Detail Where else is RPT referenced? RPT Document? Evaluator Assessment Bylaws described RPT in some detail by referencing documents in which more detail was found. Additional RPT documents were not obtained. No NR CSE NL Bylaws described RPT in much detail. Yes Much Detail RPT Document CHEMS? Bylaw reference and detail level were congruent with PA data. RPT documents were not obtained. Yes Mention RPT Document CEE? References to RPT are not just mentioned but are described in some detail. RPT documents were not obtained. BAE Yes Some Detail RPT Document and Department Website Bylaw reference and detail level were congruent with PA data. RPT document provided much detail but references to RPT could not be located on the department website. ECE Yes Some Detail RPT Document and Department Website? Bylaw reference and detail level are congruent with PA data. RPT documents were not obtained and references to RPT were not found on the department website. NR Department chair did not provide a response to this item. NL Department chair indicated that AR or RPT is not referenced in any other document nor on the department website. Inventory and PA data are not congruent. Inventory and PA data are somewhat congruent.? Inventory and PA data are congruent. Document was not obtained for review. Evaluation of MSU ADAPP ADVANCE 75

90 Alignment Alignment of Annual Review Policies with University Principles College- and department-level policies that addressed university-level principles were analyzed for accessibility. Bylaws could not be accessed from department websites for three engineering departments (BAE, CEE, and CHEMS). These documents were provided electronically upon request. Biosystems and Agricultural Engineering (BAE) also provided an additional document describing detailed expectations for faculty review. In the departments of Computer Science and Engineering (CSE), Chemical Engineering and Materials Science (CHEMS), and Electrical and Computer Engineering (ECE), information regarding annual performance review could be found only in the department bylaws under duties of the department chairperson. Civil and Environmental Engineering (CEE) bylaws described annual review under Faculty rights and responsibilities. The CSE Bylaws included an appendix that provided guidelines for AR. However, circular references were found between these guidelines and the bylaws regarding criteria used for AR. The Department of Mechanical Engineering (ME) included a very brief statement regarding faculty annual performance evaluation. In the departments of BAE and ECE, policies regarding annual review could not be found as a separate process from reappointment, promotion, and tenure. The only university-level principle that aligned with all available department policies and the college-level policy was regarding the frequency of faculty review. College- and department-level policies that addressed university-level principles were analyzed for clarity and specificity. Policies that were found to be clearly stated were scored a 1.00; policies that were found to be vague were scored a 0.50; and policies that were not found were scored a Average scores for each category were calculated for the College of Engineering and for each department where policy documents were found. Color-coding visually represents the transparency of the college and department policies based on the average scores for each category. Red represents a lack of clarity (0-0.25), yellow represents vague policies ( ), and green represents an average to high level of policy transparency ( ). The College of Engineering was found to have accessible and clear annual performance review policies that addressed procedures (Figure 11). Similarly, all six departments (CHEMS, CSE, ECE, CEE, BAE and ME) were found to have accessible and relatively clear policies regarding annual performance review procedures. Polices on annual performance review expectations and follow-up were found to be nonexistent or weak in the college and in most departments. Evaluation of MSU ADAPP ADVANCE 76

91 Figure 11. Alignment of AR policies for the College of Engineering, MSU policy analysis, Phase I. Table 45 shows the results of the evaluation s analysis of Engineering departments use of best practices for faculty annual review. Best practices were identified from department responses to the 2009 human resources inventories and are displayed in the crosswalk in Table 31. As shown in the crosswalk, four items were matched from the 2009 and 2011 inventory data and the best practices recommendations in subsequent human resources toolkits. One item referred to AR guidelines, one referred to AR input, and two referred to AR feedback. All departments in the College of Engineering aligned with best practices in both 2009 and 2011 for items related to guidelines and input for faculty annual review. CSE, CHEMS, CEE Evaluation of MSU ADAPP ADVANCE 77

92 and ECE either improved alignment with best practices in AR feedback or remained at a satisfactory level of alignment in ME and BAE, on the other hand, indicated in 2011 inventory data that feedback procedures for AR no longer were aligned with best practices. Table 45. Annual Review Best Practices Comparison with 2009 MSU Annual Review Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Engineering Guidelines Input Feedback ME CSE CHEMS a b CEE BAE a ECE b - Aligned with best practices; - Does not align with best practices. a Meetings with pre-tenure faculty only. b Department Chair indicated that this occurs in some cases, but not in all. Alignment of Reappointment, Promotion, and Tenure Policies with University Principles In the departments of Computer Science and Engineering (CSE), Chemical Engineering and Material Science (CHEMS), Biosystems and Agricultural Engineering (BAE), and Electrical and Computer Engineering (ECE), information regarding RPT could be found in the department bylaws under duties of the department chairperson, functions of the Advisory Board, and functions of the Tenure and Promotion Committee. The department of Civil and Environmental Engineering (CEE) included information on RPT in the description of two committees (Promotion and Tenure Committee and Committee of Tenured Faculty) that were involved in the process, but also described the process under the Rights and Responsibilities of the faculty. BAE also had an additional section in the bylaws, Appointments, Promotions, Tenure, Reappointments, and Program Reductions that provided further information regarding RPT. CHEMS bylaws included an appendix that addressed RPT procedures. CSE Bylaws also included an appendix that provided guidelines for reappointment and promotion evaluations; however, circular references were found, between these guidelines and the bylaws, regarding where faculty members could find criteria to be used for RPT decisions. The Department of Mechanical Engineering (ME) included a very brief statement regarding faculty evaluation for promotion and tenure, including a statement that faculty should be evaluated on criteria consistent with college and university guidelines which were not further defined. Evaluation of MSU ADAPP ADVANCE 78

93 College- and department-level reappointment, promotion, and tenure policies that addressed universitylevel principles were analyzed for clarity and specificity. At the college level, all policies related to RPT were accessible and clearly stated. At the department level, only the Department of Computer Science and Engineering consistently had RPT policies that were accessible and had some degree of specificity (Figure 12). Spring 2009 Human Resources Inventory data indicated that three department chairs (CHEMS, ME, CEE) were unaware that their department bylaws did not provide specific guidelines or information regarding reappointment, promotion, and tenure. Figure 12. Alignment of RPT policies for the College of Engineering, MSU policy analysis, Phase I. Evaluation of MSU ADAPP ADVANCE 79

94 Table 46 shows the results of the evaluator s analysis of Reappointment, Promotion, and Tenure (RPT) best practices. Best practices were identified from department responses to the 2009 human resources inventories and are displayed in the crosswalk in Table 31. As displayed in the crosswalk, four items were matched from the 2009 and 2011 inventory data and the best practices; two items referred to RPT guidelines and two items referred to RPT materials. CEE and BAE did not respond to the 2009 MSU Reappointment, Promotion, and Tenure Inventory. In both cases, where responses were given, inventory data aligned with best practices for RPT in For ME, CSE, and CHEMS, inventory data for the items referring to RPT guidelines did not change in alignment with best practices from 2009 to For the items referring to RPT materials, inventory data for the three Engineering departments aligned more closely with best practices in 2011 than in Table 46. Reappointment, Promotion, and Tenure Best Practices Comparison with 2009 MSU Reappointment, Promotion, and Tenure Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Engineering Guidelines Materials ME CSE NR CHEMS NR CEE - - NR BAE - - ECE NR - Aligned with best practices; - Does not align with best practices. NR Department chair did not provide a response to this item. (-) departments that did not participate in inventories. Comparisons in the College of Natural Science MSU College of Natural Science Website The MSU College of Natural Science (CNS) website was updated after data were collected for website analysis in New links that directed the user to information regarding university- and college-level human resources policies and documents have been added. The updated website will be reviewed for the final policy analysis report. MSU College of Natural Science Inventory and Policies Eight of the 12 departments analyzed for the College of Natural Science responded to the 2011 Human Resources Combined Inventory: (a) Biochemistry and Molecular Biology (BMB), (b) Chemistry (CEM), (c) Geological Sciences (GLG), (d) Mathematics (MTH), (e) Physics and Astronomy (PA), (f) Physiology (PSL), Evaluation of MSU ADAPP ADVANCE 80

95 (g) Plant Biology (PLB), and (h) Statistics and Probability (STT). Entomology (ENT), 5 Microbiology and Molecular Genetics (MMG), and Food Science and Human Nutrition (FSHN) 6 responded to the 2009 inventories but not to the 2011 combined inventory. MTH responded to the 2009 Annual Review (AR) Inventory but not the 2009 Reappointment, Promotion and Tenure (RPT) Inventory. Zoology (ZOL) did not provide any inventory data. Therefore, eight departments data could be matched and compared for changes between 2009 and 2011 for AR data and seven could be matched for RPT. All 12 department bylaws were eventually obtained for policy analysis. Transparency Accessibility A search for bylaws resulted in access to four department bylaws on department websites (Table 47). The remaining eight department bylaws were delivered electronically on request. The departments of Biochemistry and Microbiology (BMB) and Physics and Astronomy (PA) had the most recently amended bylaws (2011). Department of Geology (GLG) bylaws were the oldest (last amended 2002). Other department documents that may contain information on annual review (AR) and reappointment, promotion, and tenure (RPT) were found accessible online for the department of Entomology (ENT) and the department of Plant Biology (PLB). Zoology (ZOL) and Microbiology and Molecular Genetics (MMG) provided additional documents electronically upon request. Table 47. Access to Department Bylaws, MSU ADAPP-ADVANCE Policy Analysis, College of Natural Science CNS Department Found on Department Website Delivered Electronically Bylaws last amended Additional AR/RPT Docs BMB 2011 CEM 2003 ENT 2007 GLG 2002 MTH 2008 MMG 2007 PA 2011 PSL 2006 PLB? STT 2009 ZOL 2006 FSHN 2008 Access to Information on the Annual Review Process According to 2011 inventory data, 50% of department chairs in the College of Natural Science reported that the annual review process is described in some detail in the department bylaws. STT indicated that AR is only referred to in connection with RPT and BMB stated that the bylaws referred to a separate document for AR information. PA indicated that AR is described in much detail and PSL indicated that AR information is not mentioned in the bylaws. 5 ENT is a department in the College of Agriculture and Natural Resources with faculty jointly appointed in the College of Natural Science. 6 FSHN is a department in the College of Agriculture and Natural Resources with faculty jointly appointed in the College of Natural Science. Evaluation of MSU ADAPP ADVANCE 81

96 Faculty annual review, as a separate policy from reappointment, promotion, and tenure, was difficult to extract from department bylaws. Departments that described annual review as a stand-alone process included BMB, MTH, MMG, ZOL, ENT, and FSHN. Of these departments, the most detail about the AR process was provided by the MMG and ENT bylaws, both of which described the process in some detail. FSHN bylaws described the process briefly and MTH and BMB bylaws mentioned the AR process but did not describe it. The PA bylaws were found to describe the process of faculty review in much detail; however, the policies continuously specify PA regular faculty which are defined as those with tenure home in the Physics and Astronomy department. CEM bylaws provided much detail regarding faculty performance review as part of the RPT process, but also included in the appendices some information regarding criteria for merit salary. GLG and PLB bylaws contained significant amounts of information regarding faculty review but only as part of the RPT process. PSL indicated on the 2011 inventory that the bylaws did not mention annual review. These data are congruent with policy analysis findings. Other possible sources of AR process information included AR/RPT documents and websites. BMB, CEM, PSL, and PLB indicated that AR process information was contained in an AR/RPT document. AR/RPT documents for these departments were not provided and therefore could not be analyzed. MMG, ZOL, and ENT did not respond to the inventory and all have additional documents that pertain to AR/RPT. ENT and MMG deaprtments AR/RPT documents provided detailed information regarding faculty review as it pertains to RPT but did not provide information regarding AR as a stand-alone policy. ZOL, on the other hand, provided an additional document that described workload and annual merit evaluation that included guidelines, criteria, and procedures that apply to all faculty members. BMB indicated that information regarding AR was included on the department website. This information could not be located by the evaluator. Annual review information was found online for PLB on a link on the department website called the Intranet under the description of Committees. All departments indicated that a standard format for collecting faculty performance data was used. The same data were found in 2009 inventory data. Only the PLB bylaws specifically mentioned a form that would be used; however, the bylaws indicated that the form is not required for routine annual review. Access to Information on the Reappointment, Promotion, and Tenure Process All eight responding CNS departments indicated that department bylaws made reference to the RPT process. Policy analysis corroborated these data. PSL did not indicate in inventory data the detail in which the RPT process was described but instead listed the RPT document in which the RPT process was described. The bylaws for the department only mentioned the RPT process by referring to the same document. BMB indicated that the RPT process was mentioned in the bylaws but policy analysis indicated that the bylaws did more than just mention the RPT process by describing in the RPT document the major missions of faculty, and the function of the Faculty Advisory Committee as it pertained to RPT. CEM, GLG, MTH, PLB, and STT each indicated that the bylaws described the RPT process in some detail. Policy analysis data indicated that GLG and STT bylaw information on the RPT process was described in some detail; however, CEM and PLB bylaws described the RPT process in much detail and MTH bylaws only mentioned the RPT process. CEM bylaws described the use of the credential file, the reappointment procedure, and the promotion procedure under the Reappointment and Promotion Committee. PLB bylaws included RPT information in an appendix to the bylaws. Through policy analysis it was found that a very detailed description of the criteria for the different levels of RPT, information needed from the candidate, details for submitting necessary documents, and a summary of the review schedule were all included in the PLB bylaws appendix. Policy analysis also showed that MMG, ZOL, ENT, and FSHN, all departments that did not respond to the RPT inventory, provided much detail about RPT procedures in department bylaws. BMB, CEM, MTH, and PSL each indicated that the RPT process was referenced in a department RPT document. RPT documents were not obtained from any of these departments for review. BMB also indicated that RPT information was available on the department website. This information could not be Evaluation of MSU ADAPP ADVANCE 82

97 located by the evaluator. Detailed RPT documents were found for each level of RPT on the department website for ENT. Access to Information on Review Committees for Annual Review All eight of the CNS departments indicated that the department chair is involved in the annual review process. Policy analysis found that the chair s involvement was described in all department bylaws except STT data showed that five of the eight departments also indicated the chair s involvement in the AR process, however, the item on the 2009 inventory only asked the respondent to indicate how the departmental review committee, if one exists, works with the chair in the process. Therefore, lack of data on this item does not necessarily indicate that the respondent did not know the chair was involved with faculty review. Six of the eight responding CNS departments indicated that a standing committee is involved in the AR process (Table 48). Only CEM and BMB responded that there were no standing committees involved in the AR process. Policy analysis also did not locate data on committees that were involved in the AR process for these departments. The remaining 10 departments mentioned in their bylaws a committee that is involved in faculty review. However, for five of the departments, all of the committees were labeled Tenure and/or Promotion committees or were described as having an advisory role in reappointment, promotion, and tenure. The department of Plant Biology described the functions of the Tenure and Promotion committee to include routine annual review where reappointment or promotion within the next 12 months is not anticipated by either the candidate or the committee The remaining department bylaws described Advisory Committees, Personnel Committees, and a Salary Review Committee. Table 48. Review Participants for Annual Review, 2011 MSU Annual Review Inventory, College of Natural Science BMB CEM Department Chair Standing Committee GLG MTH PA PSL PLB STT Access to Information on Review Committees for Reappointment, Promotion, and Tenure Inventory data revealed where department chairs reported the description of the RPT review committees could be found. Most (88%) indicated that these committees were described in the department bylaws. Only PSL did not indicate that their bylaws described the committee; however, in the 2009 inventory data, PSL did indicate that the committee was described in the bylaws. Policy analysis indicated that for all 12 departments at least one committee was described in the bylaws in reference to the duties of the RPT process. Three departments (PSL, MTH, and BMB) indicated that the committees were described in RPT documents as well, and BMB indicated that the committee was described on the department website. This information could not be confirmed by the evaluator. Evaluation of MSU ADAPP ADVANCE 83

98 Access to Information Regarding Stopping the Tenure Clock Two departments (BMB and STT) indicated that references to stopping the tenure clock were included in the department RPT guidelines. This did not match the data collected in 2009 in which all responding departments indicated that references to stopping the tenure clock were not included in RPT guidelines. Information on stopping the tenure clock was not found for either BMB or STT in department bylaws. On the other hand, ENT, who did not respond to the 2011 inventory, did include information regarding stopping the tenure clock in RPT materials found on the department website. Communication and Clarity Data regarding the communication and clarity of expectations only were collected by the annual review subscale of the 2011 combined inventory. Analysis of communication and clarity of expectations was therefore, not specific to differences between expectations for annual review, reappointment, promotion with tenure, and promotion after tenure award. Communication of Expectations Most College of Natural Science departments (88%) indicated that expectations of faculty performance were communicated verbally through a formal meeting with the department chair. STT was the only department that did not indicate a formal meeting occurred, but did indicate an informal meeting with the chair was held and that expectations also were communicated through faculty mentors. Many (63%) indicated that written communication of expectations was provided through AR/RPT documents and half indicated that new hire paperwork included written expectations. The only AR/RPT documents obtained for evaluation were from ENT, MMG, and ZOL. All three AR/RPT documents included clear explanations of criteria but only ZOL identified criteria for all faculty evaluations, not just for RPT decisions. In 2009, PSL and PLB indicated that expectations of faculty performance could be found in the department bylaws. Since the bylaws were not amended for PSL since , they should still refer to expectations. No mention of expectations could be found in the PSL bylaws. An appendix to the PLB bylaws included a detailed description of RPT expectations. Also in 2009, CEM, GLG, and MTH indicated that expectations were not provided in writing. This did not match 2011 data in which CEM indicated that expectations were provided in new hire paperwork and GLG indicated expectations were included in both new hire paperwork and AR/RPT documents. MTH indicated that expectations were written in department bylaws; however, this information was not found through policy analysis. Clarity of Expectations A mean score was calculated for clarity of expectations using 2011 inventory data. The item asked to what extent are faculty provided specific descriptions of what constitutes above average, average, or unacceptable levels of performance regarding expectations for faculty performance in the areas listed. Response options ranged from levels of performance are not mentioned (1) to performance levels are fully described (4). Mean scores were given for each department by averaging the level of clarity scores for each of the six activities; (1) Research, (2) Teaching, (3) Service, (4) Membership in Disciplinary Societies, (5) Mentoring, and (6) Leadership in Disciplinary Societies (Figure 13). For departments in the College of Natural Science BMB was found to have the most clarity in descriptions of expectations with a mean score of 3.50 out of 4.0. Mean scores also were given for each type of activity for the College of Natural Science (Figure 14). Departments indicated that expectations for research (3.29) were communicated most clearly to faculty and expectations for membership in disciplinary societies (2.13) were communicated least clearly. 7 Date of last amendment could not be located for the department of Plant Biology bylaws. Evaluation of MSU ADAPP ADVANCE 84

99 Figure 13. Mean score for clarity of expectations for all six types of activities for each department in the College of Natural Science. Figure 14. Mean score for clarity of expectations for the College of Natural Science for each type of activity. Evaluation of MSU ADAPP ADVANCE 85

100 Consistency Consistency of Annual Review Procedures All department chairs indicated a systematic and consistent performance evaluation process for all faculty within their departments. Also, all departments indicated using a standard format for the collection of faculty performance data. All departments except GLG indicated the use of a standard performance appraisal process specific to the department. GLG indicated that faculty are evaluated using a general performance appraisal process, not specific to the department. Consistency of Reappointment, Promotion, and Tenure Procedures The 2011 inventory data showed that seven of the eight departments indicated that there are different criteria for faculty at different levels in the RPT process. CEM was the only department that indicated that criteria were not dependent upon faculty status. Six departments indicated that there are specific evaluation criteria of professional accomplishments used by the RPT review committee to evaluate the strength of each candidate s application. CEM responded that there were not specific evaluation criteria and PSL did not respond to the item. Six departments also indicated that college-level criteria are used by the department to develop the department criteria for RPT evaluations. The respondent for the CEM department did not know the answer to this item and PA responded that college-level criteria were not used. This RPT criteria is provided to writers of external letters of review for all departments except CEM. Consistency in the Use of Materials for Annual Review All departments indicated materials used for annual review included an updated CV and student ratings. Only two (CEM and PSL) of the Natural Science departments indicated the use of copies of publications for AR (Table 49). Table 49. Materials Required for Annual Review, 2011 MSU Annual Review Inventory, College of Natural Science Updated CV Performance Appraisal Form Copies of Publications Citations Student Ratings Teaching Portfolio Personal Statement BMB CEM GLG MTH PA PSL PLB STT Half of the Natural Science departments indicated that the process for submission of annual review materials is communicated to faculty verbally. CEM, MTH, and PA all indicated that the process could be found in department bylaws. CEM bylaws indicated the need for each faculty member to keep the Evaluation of MSU ADAPP ADVANCE 86

101 credential file updated and indicated that the chair sends an annual reminder to faculty to update the credential file as needed; however, this is indicated as a preparatory step for RPT. MTH bylaws contained no information regarding annual review. PA bylaws contained a very detailed description of Sources of Information for the Salary Review Committee which included specific documents and procedures for annual review. BMB, CEM, and PSL indicated that the process for submitting materials required for AR is communicated in the AR/RPT document. AR/RPT documents were not obtained for these departments. Only MTH indicated that candidates were limited in the amount of materials that could be submitted for AR. Consistency in the Use of Materials for Reappointment, Promotion, and Tenure According to 2011 inventory data, materials required for the different outcomes of the RPT process do not differ significantly by department. More than half of the departments required the submission of citations for first reappointment (R), promotion with tenure (T), and promotion after tenure award (P) (Table 50). PLB, however, was the only department that indicated citations were required for tenure and promotion but not reappointment. PA was the only of the eight departments that indicated teaching portfolios were required for tenure and promotion review but not for reappointment. All eight departments indicated that they required copies of publications and personal statements for all three levels of RPT. All eight departments also indicated that external letters were required for promotion and tenure. PA was the only department that indicated external letters were required for reappointment as well. Data regarding other required materials varied slightly in 2009, possibly due to difference in item type. In 2011 chairs were asked to choose from a list and in 2009, the item was open-response. For example, in 2009 only GLG, PSL, and PLB indicated that all candidates were required to prepare a personal statement for reappointment, tenure, and promotion and in 2011 all eight departments chose this response. On the other hand, in addition to PA, BMB, GLG, and STT indicated that external letters were required for reappointment; these three departments did not report the same in Table 50. Materials Required for Reappointment (R), Promotion with Tenure (T), and Promotion after Tenure Award (P), 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Natural Science Copies of Publication Citations Teaching Portfolio Personal Statement External Letters R T P R T P R T P R T P R T P BMB CEM GLG MTH PA PSL PLB STT CEM, MTH, PA, and PSL indicated that a protocol for submitting RPT materials is written in the department bylaws. Policy analysis showed that the bylaws for CEM and PA describe the process for submission of materials in detail. MTH and PSL did not describe the process for submitting materials for Evaluation of MSU ADAPP ADVANCE 87

102 RPT. BMB, CEM, MTH and PSL indicated that this information could be found in the department RPT document. RPT documents were not obtained from these four departments. External letter writers for departments in the College of Natural Science are, for the most part, selected by the candidate and the department chair, using criteria and guidelines provided in college documents (Table 51). Five departments indicated that the candidate has input into the selection of the pool of reviewers. The 2009 inventory data indicated for all departments except PSL that the candidate had input; therefore, GLG and PA gave different responses on the two inventories. None of the departments indicated in 2009 that guidelines for forming the pool of external letter writers were located in the department bylaws; however, BMB and PSL indicated in 2011 that this information could be found in the bylaws. BMB also indicated that criteria in the bylaws described the process of selecting the letter writers. Policy analysis indicated that both are true for BMB but nothing could be found in the PSL bylaws regarding external letter writers. CEM, PA, and PLB indicated that the RPT committee had input into both the selection of the pool of candidates and the final selection of letter writers. Data describing the final selection of external letter writers were not collected by the 2009 inventory. Table 51. Selection of Pool and Final External Letter Writers for RPT, 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Natural Science Candidate Chair Guidelines in Department Bylaws Guidelines in MSU Faculty Handbook College Guidelines Other Pool Writer Pool Writer Pool Writer Pool Writer Pool Writer BMB CEM RPT Committee GLG MTH PA RPT Committee PSL PLB RPT Committee STT Consistency in Modes of Input for Joint Appointments for Annual Review Inventory data indicated that departments in the College of Natural Science handle annual review of jointly appointed faculty in two different ways (Figure 15). BMB, PLB, and STT indicated that both departments generally were equally involved in the annual review of jointly appointed faculty. On the other hand, CEM, MTH, PA, and PSL indicated that the primary department/unit (Unit 1) was largely responsible but that expectations and input from the second department/unit (Unit 2) were collected. BMB, PLB, and STT indicated that expectations and input were shared between the departments/units in a formal process that evaluated specific expectations for professional accomplishments. CEM and PSL indicated that input and expectations were shared in an informal manner with general expectations evaluated by chairs. PA indicated review by the tenure home department only. GLG responded not applicable to both items relating to joint review processes. Evaluation of MSU ADAPP ADVANCE 88

103 Figure 15. How input is collected for joint reviews, College of Natural Science. Consistency in Modes of Input for Joint Appointments for Reappointment, Promotion, and Tenure Six departments for CNS were evenly split between the different types of review for RPT of jointly appointed faculty (Figure 16). MTH and PLB indicated equal involvement by both departments/units; PSL and STT indicated that the primary department was responsible but input is collected from both departments/units; and PA and CEM indicated that the primary department is completely responsible but input from the secondary department/unit is considered. MTH and PLB indicated that performance expectations and input are shared in a formal review process, whereas STT and PA indicated information is shared in an informal joint review process. PSL indicated review only by the tenure home department. BMB indicated that the process is dependent upon the individual faculty member. Evaluation of MSU ADAPP ADVANCE 89

104 Figure 16. How input is collected for joint reviews, College of Natural Science. Evaluation of MSU ADAPP ADVANCE 90

105 Consistency in Modes of Mentor Input for Annual Review and Reappointment, Promotion, and Tenure As of March 2011, the university required colleges to develop policy for and implement a formal faculty mentoring program. University principles regarding this mentoring policy did not explicitly state that input from the mentor would be requested for faculty annual review inventory data indicated that BMB, PA, and PSL expected input from faculty mentors for AR. The same was found in the 2009 data. All three indicated that input was sought through an informal process. However, in 2009 GLG indicated that input was sought from mentors and in 2011 indicated that input was not expected from mentors. Three departments (CEM, PLB and STT) indicated that although they do not collect input from mentors now and did not in the past, they will or might collect input from mentors in the future. MTH indicated they do not collect input from mentors (2009 and 2011). In 2009, department chairs were asked whether or not input is sought from mentors and in 2011 expected from mentors. The language difference may explain the GLG data. BMB, CEM, and PA chairs indicated that input was expected from faculty mentors for RPT. BMB indicated that information is collected informally while PA indicated that the mentor is a member of the RPT committee. CEM indicated that gathering input from the mentor is a new process and the manner in which it would be collected had not yet been determined. Data regarding mentor input were not collected by the 2009 Reappointment, Promotion, and Tenure Inventory. Consistency in Providing Feedback for Annual Review Table 52 shows the different ways in which feedback is provided to the faculty member following the annual review process. Most departments (75%) indicated that a written review in addition to a chairinitiated meeting was used to convey the AR feedback. MTH and PLB indicated that they provided only written feedback. This differed from 2009 in which MTH indicated that the department chair met with faculty members only if there was a problem and PLB indicated that the department chair met with every faculty member each year. All responding departments indicated in both 2009 and 2011 that expectations of performance necessary for promotion were included in written feedback. Table 52. Types of Annual Review Feedback, 2011 MSU Annual Review Inventory, College of Natural Resources Department Verbal: Meeting with Chair Written Review Document Chair Initiated Requested Standardized Non-Standardized BMB CEM GLG MTH PA PSL PLB STT Consistency in Providing Feedback for Reappointment, Promotion, and Tenure Six departments responded to the item asking for a brief description of the information provided from the RPT committee to the department chair regarding RPT recommendations. Two departments misread the item and therefore did not provide relevant data. Three of the responding departments indicated that chairs are provided a summary in the form of a report or a letter that included either a vote, ratings for review categories, or recommendations for RPT decision. GLG indicated that verbal input was provided to the chair from the RPT committee and STT stated, the committee simply recommends whether to Evaluation of MSU ADAPP ADVANCE 91

106 reappoint with or without tenure and/or promote. All eight departments indicated that the candidate has an opportunity to discuss the results or recommendations of the RPT review. All departments except MTH (no response given) indicated that a meeting for each candidate is scheduled by the department chair to discuss the results of the review. Congruence Between Inventory Data and Policy Analysis Congruence between inventory data and policy analysis (PA) findings for Annual Review (AR) is displayed in Table 53. In terms of AR references in department bylaws for the College of Natural Science, human resources inventory and policy analysis data are mostly congruent. Possibly due to the fact that only an appendix to the bylaws was received for the department of Plant Biology, no AR references were found. This finding was not congruent with inventory data. Congruence between inventory data and policy analysis (PA) findings for Reappointment, Promotion, and Tenure (RPT) is displayed in Table 54. Inventory data and RPT policies in the College of Natural Science were more congruent than were inventory data and AR policies. For the most part, inventory and policy analysis data were congruent in all departments in the College of Natural Science. Table 53. Congruence of Annual Review Inventory Information with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Natural Science DEPT BMB CEM Reference to AR in bylaws? Yes Amount of Detail Refers to a separate document Where else is AR referenced? AR/RPT Document and Department Website? Yes Some Detail AR/RPT Document? Evaluator Assessment Bylaw reference was congruent with PA data. Detail level could not be compared since department chair did not provide a level. AR/RPT documents were requested but not obtained and references to AR were not found on the department website. Annual review, as a separate process from RPT, was not mentioned in the department bylaws. Much detail is provided as AR pertains to RPT. AR/RPT documents were requested but not obtained. GLG MTH Yes Yes Some Detail Some Detail NL NL Annual review, as a separate process from RPT, was not mentioned in the department bylaws. Much detail is provided as AR pertains to RPT. Bylaw reference is congruent with PA data. AR is only mentioned in the bylaws. PA Yes Much Detail NL Bylaw reference and detail level were congruent with PA data. Evaluation of MSU ADAPP ADVANCE 92

107 DEPT PSL Reference to AR in bylaws? Amount of Detail Where else is AR referenced? No NR AR/RPT Document? Evaluator Assessment Bylaw reference and detail level were congruent with PA data. No mention of Annual Review could be found in the department bylaws. AR/RPT documents were requested but not obtained. PLB * Yes Some Detail AR/RPT Document? No references to Annual Review could be found in the bylaws. AR/RPT documents were requested but not obtained. STT Yes In connection with P&T NL Annual review, as a separate process from RPT, was not mentioned in the department bylaws. * - Only received Appendix I from department bylaws, which included information regarding RPT. NR Department chair did not provide a response to this item. NL Department chair indicated that AR or RPT is not referenced in any other document nor on the department website. Inventory and PA data are not congruent. Inventory and PA data are somewhat congruent.? Inventory and PA data are congruent. Document was not obtained for review. Table 54. Congruence of Reappointment, Promotion, and Tenure Inventory with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Natural Science DEPT BMB CEM Reference to RPT in bylaws? Amount of Detail Where else is RPT referenced? Yes Mention RPT Document and Department Website Yes Some Detail? RPT Document? Evaluator Assessment References to RPT are not just mentioned but are described in some detail. RPT documents were requested but not obtained. References to RPT could not be located on the department website. References to RPT are not just described in some detail but are described in much detail. RPT documents were requested but not obtained. Evaluation of MSU ADAPP ADVANCE 93

108 DEPT GLG Reference to RPT in bylaws? Yes Amount of Detail Some Detail Where else is RPT referenced? NL Evaluator Assessment Bylaw reference and detail level were congruent with PA data. Yes Some Detail RPT Document MTH? Bylaw reference is congruent with PA data. RPT is only mentioned in the bylaws. AR/RPT documents were requested but not obtained. PA Yes Much Detail NL Bylaw reference and detail level were congruent with PA data. PSL Yes Some Detail RPT Document and Department Website? Bylaw reference and detail level are congruent with PA data. RPT documents were not obtained and references to RPT were not found on the department website. PLB * Yes Some Detail NL References to RPT are not just described in some detail but are described in much detail. STT Yes Some Detail NL Bylaw reference and detail level were congruent with PA data. * - Only received Appendix I from department bylaws, which included information regarding RPT. NL Department chair indicated that AR or RPT is not referenced in any other document nor on the department website. Inventory and PA data are not congruent. Inventory and PA data are somewhat congruent.? Inventory and PA data are congruent. Document was not obtained for review. Evaluation of MSU ADAPP ADVANCE 94

109 Alignment Alignment of Annual Review Policies College- and department-level policies that addressed university-level principles were analyzed for accessibility. Bylaws could not be accessed from department websites for eight Natural Science departments (CEM, GLG, MTH, MMG, PSL, PLB, ZOL, and FSHN). These documents were provided electronically upon request. Additional AR/RPT documents were either found or were provided for ENT, MMG, PLB, and ZOL. In general, annual performance review for faculty is mentioned or described in some detail in the definition of the duties of various committees (i.e., Faculty Advisory, Salary Review, or Personnel). In the department of Biochemistry and Molecular Biology (BMB) information regarding annual performance review could only be found mentioned in the department bylaws under duties of the department chairperson. The process is described in some detail under Faculty Appointments for ENT and under Terms of Faculty Employment and Evaluation of Faculty Procedures in FSHN. CEM, STT, and PLB only mention AR as it applies to RPT, and PSL does not mention AR but instead refers to a separate document titled Procedures for Evaluating Faculty for Reappointment, Tenure, and Promotion (document not received). No university-level principles aligned with college-level policy and all available department policies; although the principle stating that all faculty should be evaluated annually was found clearly stated in 8 and vaguely stated in 2 of the 12 CNS department policies. College- and department-level policies that addressed university-level principles were analyzed for clarity and specificity. Policies that were found to be clearly stated were scored a 1.00; policies that were found to be vague were scored a 0.50; and policies that were not found were scored a Average scores for each category were calculated for the College of Natural Science and for each department where policy documents were found. Color-coding visually represents the transparency of the college and department policies based on the average scores for each category. Red represents a lack of clarity (0-0.25), yellow represents vague policies ( ), and green represents an average to high level of policy transparency ( ). The College of Natural Science was found to have accessible and clear annual performance review policies that addressed expectations, and procedures, and vague policy references regarding follow up (Figures 17, 18, and 19). Seven of the 12 departments were found to have accessible and relatively clear policies regarding annual performance review expectations. Department-level policies are less accessible and clear in terms of procedures and are even less clear or non-existent regarding follow-up. Evaluation of MSU ADAPP ADVANCE 95

110 Figure 17. Alignment of AR policies on expectations for the College of Natural Science, MSU policy analysis, Phase I. Evaluation of MSU ADAPP ADVANCE 96

111 Figure 18. Alignment of AR policies on procedures for the College of Natural Science, MSU policy analysis, Phase I. Evaluation of MSU ADAPP ADVANCE 97

112 Figure 19. Alignment of AR policies on follow-up for the College of Natural Science, MSU policy analysis, Phase I. Table 55 shows the results of the evaluator s analysis of Natural Science departments use of best practices for faculty annual review. Best practices were identified from department responses to the 2009 human resources inventories and are displayed in the crosswalk in Table 31. As shown in the crosswalk, four items were matched from the 2009 and 2011 inventory data and the best practices recommended in subsequently-developed ADAPP HR toolkits. One item referred to AR guidelines, one referred to AR input, and two referred to AR feedback. Greater alignment was found between responses to best practices and Evaluation of MSU ADAPP ADVANCE 98

113 inventory items referring to annual review guidelines and input in 2011 than in There was less alignment found, however, for items referring to annual review feedback. Table 55. Annual Review Best Practices Comparison with 2009 MSU Annual Review Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Natural Science BMB Guidelines Input Feedback CEM b GLG MTH a c NR PA PSL d PLB NR STT NR - Aligned with best practices; - Does not align with best practices. a Department Chair indicated department does not have annual reviews. b Department Chair indicated meetings with pre-tenure faculty and associate professors. c Department Chair indicated meetings with pre-tenure faculty only if there is a problem or request. d Department Chair indicated that AR feedback refers to results of previous year s AR sometimes. NR Department chair did not provide a response to this item. Evaluation of MSU ADAPP ADVANCE 99

114 Alignment of Reappointment, Promotion, and Tenure Policies The departments of Math (MTH) and Physiology (PSL) only mentioned RPT policies in their bylaws as they described the functions of the Personnel Committee and Advisory Committee, respectively. Chemistry (CEM) and Statistics and Probability (STT) described the process in some to much detail under duties of the respective reappointment, promotion, and tenure committees. Plant Biology (PLB) and Microbiology and Molecular Genetics (MMG) included appendices with the bylaws that described the RPT process in much detail. All other departments described the process in some to much detail under sections clearly labeled as policies for faculty (i.e., Faculty Appointments and Terms of Faculty Employment and Evaluation of Faculty Procedures). College- and department-level reappointment, promotion, and tenure policies that addressed universitylevel principles were analyzed for clarity and specificity. At the college level, all policies related to RPT were accessible and clearly stated (Figures 20, 21, and 22). At the department level, the departments of STT, MTH, and PSL consistently lacked RPT policies that were accessible. The remaining departments included relatively clear and accessible RPT policies that refer to procedures (Figure 20) and follow-up (Figure 21). Only the departments of ENT and GLG had clear policies regarding expectations in the bylaws; CEM, MMG, ZOL, and PLB bylaws discussed expectations vaguely (Figure 21). Evaluation of MSU ADAPP ADVANCE 100

115 Figure 20. Alignment of RPT policies on expectations for the College of Natural Science, MSU policy analysis, Phase I. Evaluation of MSU ADAPP ADVANCE 101

116 Figure 21 Alignment of RPT policies on procedures for the College of Natural Science, MSU policy analysis, Phase I. Evaluation of MSU ADAPP ADVANCE 102

117 Figure 22. Alignment of RPT policies on follow-up for the College of Natural Science, MSU policy analysis, Phase I. Evaluation of MSU ADAPP ADVANCE 103

118 Table 56 shows the results of the evaluator s analysis of Reappointment, Promotion, and Tenure (RPT) best practices. Best practices were identified from department responses to the 2009 human resources inventories and are displayed in the crosswalk in Table 31. As shown in the crosswalk, four items were matched from the 2009 and 2011 inventory data and the recommended best practices; two items referred to RPT guidelines and two items referred to RPT materials. MTH did not respond to the 2009 MSU Reappointment, Promotion, and Tenure Inventory. Half of the inventory items referring to RPT guidelines and materials for this department aligned with best practices in Data for one department (PSL) showed less alignment in 2001 than in 2009 with best practices for items referring to RPT guidelines, while data for two departments (BMB and STT) showed more alignment for these items. The remaining departments did not change. More alignment between inventory data and best practices was found for items referring to RPT materials for most departments in the College of Natural Science. Table 56. Reappointment, Promotion, and Tenure Best Practices Comparison with 2009 MSU Reappointment, Promotion, and Tenure Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Natural Science Guidelines Materials BMB CEM GLG MTH - - PA NR PSL NR PLB STT - Aligned with best practices; - Does not align with best practices. NR Department chair did not provide a response to this item. (-) departments that did not participate in inventories. Evaluation of MSU ADAPP ADVANCE 104

119 Comparisons in the College of Social Science MSU College of Social Science Website The MSU College of Social Science (CSS) website provided a Faculty & Staff link through which all university-level AR and RPT documents were accessed. These documents were located under two headings included on this webpage, Resources and Human Resources. Under the Resources heading, the Human Resources webpage could be accessed from the Other MSU Resources for Faculty & Staff link. From the Human Resources webpage, university-level policies on AR and RPT were accessed as described in the MSU Website description. Performance review and development forms as well as the Faculty Handbook were accessed through links under the Human Resources heading (Figure 23). Figure 23. University-level policy document accessibility map, AR and RPT, College of Social Science. MSU College of Social Science Inventory and Policies All 11 departments in the College of Social Science responded to the 2011 Human Resources Combined Inventory: (a) Anthropology (ANP), (b) Criminal Justice (CJ), (c) Economics (ECON), (d) Geography (GEO), (e) History (HST), (f) Human Development and Family Studies (HDFS), (g) Labor and Industrial Relations (LIR), (h) Political Science (PLS), (i) Psychology (PSY), (j) Social Work (SW), (k) Sociology (SOC), and (l) Urban and Regional Planning (UP). SOC and ANP did not respond to either of the 2009 inventories, and HST and PSY responded to the 2009 Annual Review (AR) Inventory but not to the 2009 Reappointment, Promotion, and Tenure (RPT) Inventory. Therefore, nine departments data could be Evaluation of MSU ADAPP ADVANCE 105

120 matched and compared for changes between 2009 and 2011 for AR data and seven could be matched for RPT. Only SOC bylaws were obtained for policy analysis. Transparency Accessibility A search for bylaws resulted in access to only the SOC bylaws through the department website (Table 57). No department bylaws were delivered electronically upon request. No other department documents that might contain information on annual review (AR) and reappointment, promotion, and tenure (RPT) were found accessible online nor provided in hard copy for departments in the College of Social Science. Table 57. Access to Department Bylaws, MSU ADAPP-ADVANCE Policy Analysis, College of Social Science CSS Department Found on Department Website Delivered Electronically Bylaws last amended ANP CJ ECON GEO HST HDFS LIR PLS PSY SW SOC 2009 Additional AR/RPT Docs Access to Information on the Annual Review Process According to 2011 inventory data, 56% of the 11 responding department chairs in the College of Social Science reported that the AR process is described in some detail in the department bylaws. The department of Economics (ECON) indicated that the AR process was mentioned but not described; the departments of Criminal Justice (CJ) and Anthropology (ANP) indicated that the AR process was described in much detail, and the departments of Geography (GEO) and Social Work (SW) indicated that the AR process was not described in their bylaws. Department of Sociology (SOC) bylaws referred to the AR process and described the process in much detail. Other possible sources of AR process information included AR/RPT documents and websites. The departments of Social Work (SW), Anthropology (ANP), Psychology (PSY), Human Development and Family Studies (HDFS), Criminal Justice (CJ), Geography (GEO), and the School of Human Resources and Labor Relations (LIR) all indicated that AR process information was contained in an AR/RPT document. AR/RPT documents for these departments were not provided and therefore could not be analyzed. PSY indicated that information regarding the AR process was available on the department website. This information could not be located by the evaluator. ANP indicated that AR information could be found in the mentoring policy and LIR indicated that the Research and Scholarship Plan provided this information. Ten departments indicated that a standard format for collecting faculty performance data was used; SW stated, not a formal template, but similar structure/categories. Only three departments responded to this item in 2009 inventory data and the same data were found for LIR, HST, and SW. Evaluation of MSU ADAPP ADVANCE 106

121 Access to Information on the Reappointment, Promotion, and Tenure Process Ten of the 11 responding CSS departments indicated that department bylaws made reference to the RPT process; only GEO indicated that the process was not mentioned in the bylaws. Policy analysis of the SOC bylaws revealed a significant amount of detail about the RPT process. SW, ANP, PSY, HDFS, CJ, and GEO each indicated that the RPT process was referenced in a department RPT document. RPT documents were not obtained from any of these departments for review. PSY also indicated that RPT information could be found on the department website. This information could not be located by the evaluator. HDFS and LIR both indicated that RPT process information could be found in the Research and Scholarship Plan document. Access to Information on Review Committees for Annual Review Eight of the 11 CSS departments indicated that the department chair was involved in the AR process. SOC was one of these eight and policy analysis found that the chair s involvement was described in the department bylaws. CJ, PLS, and HST all indicated that the department chair was not involved with faculty review. Inventory data from 2009 showed that four of the eight departments indicated the chair s involvement in the AR process; however, the item on the 2009 inventory only asked the respondent to indicate how the departmental review committee, if one exists, works with the chair in the process. Therefore, lack of data on this item does not necessarily indicate that the respondent did not know the chair was involved with faculty review. Two departments (HST and PLS) indicated in 2009 that the chair was involved in the AR process, but did not indicate the same in Ten of the 11 responding CSS departments indicated that a standing committee is involved in the AR process (Table 58). Only PSY responded that there were no standing committees involved in the AR process. Policy analysis found that SOC bylaws described an Advisory Committee that shares responsibility with the chair for faculty review. Table 58. Review Participants for Annual Review, 2011 MSU Annual Review Inventory, College of Social Science Department Chair Standing Committee ANP CJ ECON GEO HST HDFS LIR PLS PSY SW SOC Evaluation of MSU ADAPP ADVANCE 107

122 Access to Information on Review Committees for Reappointment, Promotion, and Tenure Inventory data revealed where the description of committees responsible for carrying out RPT review of faculty could be found. Most chairs (82%) indicated that these committees were described in the department bylaws. Only GEO did not indicate that the bylaws described the committee. Inventory data from 2009 matched the 2011 data for all seven of the RPT inventory respondents. Policy analysis indicated that the SOC bylaws described the Advisory Committee and the Peer Review Committee, both of which had responsibilities in the RPT process. Two departments (SW and HDFS) indicated that the committees were described in RPT documents. These documents were not received for review. PSY indicated that none of the options applied because each faculty member appointed his/her own committee. Access to Information Regarding Stopping the Tenure Clock Two departments (CJ and GEO) indicated that references to stopping the tenure clock were included in the department RPT guidelines. CJ was the only department responding in 2009 that references to stopping the tenure clock were included in RPT guidelines. All other responding departments, including GEO, indicated that RPT guidelines did not refer to stopping the tenure clock. Information on stopping the tenure clock was not found for SOC in department bylaws. Communication and Clarity Data regarding the communication and clarity of expectations only were collected by the annual review subscale of the 2011 combined inventory. Analysis of communication and clarity of expectations was therefore not specific to differences between expectations for annual review, reappointment, promotion with tenure, and promotion after tenure award. Communication of Expectations Just over half of the 11 College of Social Science departments indicated that expectations of faculty performance were communicated verbally through a formal meeting with the department chair. ECON did not indicate a formal meeting occurred but did indicate an informal meeting with the chair was held and that expectations also were communicated through senior faculty. PSY indicated that the only verbal communication of expectations was through senior faculty and faculty mentors. PLS and HST did not indicate that expectations were communicated verbally. Many (82%) indicated that written communication of expectations was provided through AR/RPT documents and almost half (45%) indicated that new hire paperwork included written expectations. In 2009, HDFS, LIR, ECON, and HST were the only departments that indicated expectations were communicated to faculty through written means. HDFS indicated new-hire paperwork and AR/RPT policies; LIR indicated written department memos; and HST indicated department bylaws. Economics reported in 2009 and 2011 that expectation information was provided in the Research and Scholarship Plan document. These documents were not obtained for review. Clarity of Expectations A mean score was calculated for clarity of expectations using 2011 inventory data. The item asked to what extent are faculty provided specific descriptions of what constitutes above average, average, or unacceptable levels of performance regarding expectations for faculty performance in the areas listed. Response options ranged from levels of performance are not mentioned (1) to performance levels are fully described (4). Mean scores were given for each department by averaging the level of clarity scores for each of the six activities: (1) Research, (2) Teaching, (3) Service, (4) Membership in Disciplinary Evaluation of MSU ADAPP ADVANCE 108

123 Societies, (5) Mentoring, and (6) Leadership in Disciplinary Societies (Figure 24). LIR reported the most clarity in descriptions of expectations with a mean score of 3.75 out of 4.0. Mean scores also were given for each type of activity for the College of Social Science (Figure 25). CSS departments indicated that expectations for teaching (3.18) were communicated most clearly to faculty and expectations for mentoring (2.09) were communicated least clearly. Figure 24. Mean score for clarity of expectations for all six types of activities for each department in the College of Social Science. Evaluation of MSU ADAPP ADVANCE 109

124 Figure 25. Mean score for clarity of expectations for the College of Social Science for each type of activity. Consistency Consistency of Annual Review Procedures Ten of the 11 College of Social Science department chairs indicated a systematic and consistent performance evaluation process was employed for annual review of all faculty within their departments. SOC stated, we have a very complex faculty, with many joint appointments, etc., so it would be inappropriate to not have variation. Also, 10 of the 11 departments indicated using a standard format for the collection of faculty performance data. SW indicated that although the department template is not formalized, the structure and categories are similar. All departments except SOC, for reasons stated above, indicated the use of a standard performance appraisal process specific to the department. SOC indicated that faculty are evaluated using a general set of expectations but that each appointment may have additional specific performance expectations that are appraised. Consistency of Reappointment, Promotion, and Tenure Procedures The 2011 inventory data showed that all of the responding CSS departments indicated that there are different criteria for faculty at different levels in the RPT process. Ten departments indicated that there are specific evaluation criteria for professional accomplishments used by the RPT review committee to evaluate the strength of each candidate s application. HDFS responded that there were no specific evaluation criteria. Eight departments indicated that college-level criteria are used by the department to develop the department criteria for RPT evaluations. SOC and PLS indicated that college-level criteria were available but not used and HST responded that college-level criteria were not available. This RPT criteria is provided to writers of external letters of review for all departments except SW, ANP, and CJ. Evaluation of MSU ADAPP ADVANCE 110

125 Consistency in the Use of Materials for Annual Review All departments indicated materials used for annual review included student ratings of teaching. Only two (ECON and GEO) of the Social Science departments indicated the use of teaching portfolios for AR (Table 59). All departments except PSY required completed performance appraisal forms. Table 59. Materials Required for Annual Review, 2011 MSU Annual Review Inventory, College of Social Science Updated CV Performance Appraisal Form Copies of Publications Citations Student Ratings Teaching Portfolio Personal Statement ANP CJ ECON GEO HST HDFS LIR PLS PSY SW SOC Nearly half (45%) of the Social Science departments indicated that the process for the submission of annual review materials is communicated to faculty verbally; and for the most part, the other half indicated the protocol could be found in the AR and/or RPT document. CJ and PLS indicated that the information could be found in the department bylaws and LIR stated, standard protocol is described in required annual summary reporting form. Neither this form nor the department bylaws were obtained for this review. Consistency in the Use of Materials for Reappointment, Promotion, and Tenure According to 2011 inventory data, materials required for different potential outcomes of the RPT process do not differ significantly by department. More than half of the departments required the submission of citations for first reappointment (R), promotion with tenure (T), and promotion after tenure award (P), (Table 60). GEO and LIR, however, were the only departments that indicated citations were required for tenure and promotion but not reappointment. CJ was the only of the 11 departments that indicated the requirement of teaching portfolios for tenure and promotion but not for reappointment. All 11 departments indicated that they require copies of publications and personal statements for all three levels of RPT. All 11 departments also indicated that external letters were required for promotion with tenure; all departments except PLS required external letters for promotion after tenure; none of the CSS departments required external letters for reappointment. Data regarding other required materials varied slightly in 2009, possibly due to the difference in item type. In 2011, chairs were asked to choose from a Evaluation of MSU ADAPP ADVANCE 111

126 list; and in 2009, the item was open-response. For example, in 2009 only GEO, SW, and CJ reported that all candidates were required to prepare a personal statement for reappointment, tenure, and promotion, and in 2011, all 11 departments chose this response. Table 60. Materials Required for Reappointment (R), Promotion with Tenure (T), and Promotion after Tenure Award (P), 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Social Science Copies of Publication Citations Teaching Portfolio Personal Statement External Letters R T P R T P R T P R T P R T P ANP CJ ECON GEO HST HDFS LIR PLS PSY SW SOC ANP, CJ, SOC, HST, LIR, and PLS reported that a protocol for submitting RPT materials was written in the department bylaws. Policy analysis showed that SOC bylaws described the materials submission process in detail. Bylaws were not obtained for the remaining departments. SW, ANP, and HDFS reported that this information could be found in the department RPT document. RPT documents were not obtained from these three departments. The departments of GEO and ECON indicated that the protocol for submitting materials for RPT is only communicated verbally. External letter writers for departments in the College of Social Science are, for the most part, selected by the candidate and the department chair and to a lesser extent through criteria and guidelines provided by the college (Table 61). Ten departments indicated that the candidate has input the selection of the pool of reviewers. Data from the 2009 inventory showed that all of the responding departments except PLS reported that the candidate had input. HDFS and CJ reported in 2009 that guidelines for forming the pool of external letter writers were located in the department bylaws; the same information was found in the 2011 data. SOC also indicated in 2011 data that criteria for the selection of external letter writers could be found in the department bylaws. Although there is no indication of a pool selection prior to final selection, criteria for who selected the letter writers were found in the bylaws. ECON, HDFS, LIR, and PSY reported that either tenured faculty or a standing committee had input in both the selection of the pool of candidates and in the final selection of letter writers. SW added that consultation with senior administrators informed in the selection of the pool of writers and Anthropology indicated that the final selection of writers was vetted through the Dean s office. Data describing the final selection of external letter writers were not collected by the 2009 inventory. Evaluation of MSU ADAPP ADVANCE 112

127 Table 61. Selection of Pool and Final External Letter Writers for RPT, 2011 MSU Reappointment, Promotion, and Tenure Inventory, College of Social Science Candidate Chair Guidelines in Department Bylaws Guidelines in MSU Faculty Handbook College Guidelines Other Pool Writer Pool Writer Pool Writer Pool Writer Pool Writer ANP CJ ECON Reading Committee GEO HST HDFS Advisory Committee LIR Tenured Faculty PLS PSY Review Committee SW SOC Consistency in Modes of Input for Joint Appointments for Annual Review Inventory data indicated that joint reviews were completed in a different way in the seven CNS departments (Figure 26); however, most (64%) indicated that the primary department (Unit 1) was largely responsible but that expectations and input from the second department/unit (Unit 2) were collected. Most of these departments (ECON, PSY, SOC, PLS, and LIR) indicated that input and expectations were shared in an informal manner with general expectations evaluated by department chairs. ANP indicated that it depended upon the second department/unit whether or not the AR process was a formal or informal sharing of expectations and input. Only CJ indicated that both departments of the jointly appointed faculty member were equally involved in annual review; a formal joint review process, which evaluates specific expectations, is performed in this department/unit. HDFS and GEO indicated that the primary department was largely responsible but input was collected from the secondary department. PA indicated review by the tenure home department only. GEO indicated that separate formal reviews were performed. Evaluation of MSU ADAPP ADVANCE 113

128 Figure 26. How input is collected for joint reviews, College of Social Science. Evaluation of MSU ADAPP ADVANCE 114

129 Consistency in Modes of Input for Joint Appointments for Reappointment, Promotion, and Tenure Only HST indicated equal involvement by both departments/units in the RPT process; a formal joint review process is used for the sharing of input and expectations. ECON, ANP, SOC, and LIR indicated that the primary department was responsible but input was collected from both departments, and PSY, HDFS, CJ, and GEO indicated that the primary department is completely responsible but input from the secondary department/unit is considered (Figure 27). Of these departments, ANP, CJ, and LIR indicated that they share performance expectations and input in a formal review process; whereas, PSY, SOC, ECON, and PLS indicated they share the information in an informal joint review process. SW indicated that sometimes the two departments are equally responsible and sometimes one of the two takes the lead; in either case, the expectations and input are shared through a formal RPT review process. GEO indicated that separate formal reviews are performed. Evaluation of MSU ADAPP ADVANCE 115

130 Figure 27. How input is collected for joint reviews, College of Social Science. Evaluation of MSU ADAPP ADVANCE 116

131 Consistency in Modes of Mentor Input for Annual Review and Reappointment, Promotion, and Tenure As of March 2011, the university required colleges to develop policy for and implement a formal faculty mentoring program. University principles regarding this mentoring policy did not explicitly state that input from the mentor would be requested for faculty annual review inventory data indicated that ECON, PSY, and LIR expected input from faculty mentors for annual review. SOC indicated that it was up to the faculty member whether or not input should be collected from the mentor. These data were not found in LIR indicated that input from mentors was collected using a standard form and PSY and ECON indicated that the mentors are included on the review committee. Data regarding mentor input were not collected by the 2009 Reappointment, Promotion, and Tenure Inventory. Consistency in Providing Feedback for Annual Review Table 62 shows the different ways in which feedback is provided to the faculty member following the annual review process. Most departments (64%) indicated that a written review in addition to a meeting with the department chair (either chair initiated or requested by faculty) was used to convey the AR feedback. SW and PLS indicated that they provided only written feedback, whereas ANP indicated that feedback was given only in the form of a meeting with the chair. This differed from 2009 in which SW indicated that the department chair met with every pre-tenure faculty member annually and met at least once every two years with tenured faculty. PSY, PLS, and CJ did not provide responses to items regarding verbal feedback in 2009 and ECON, CJ, HDFS, and GEO did not provide responses to the item regarding written feedback. ANP, HDFS, PLS, GEO, and LIR indicated in 2011 that expectations of performance necessary for promotion were included in written feedback. For those departments that provided responses to these items in 2009 (LIR and PLS) these data matched for the two years. In 2011, HST and SW indicated that AR feedback might provide expectations necessary for promotion depending on the faculty member but in 2009 a firm yes was indicated for both. Table 62. Types of Annual Review Feedback, 2011 MSU Annual Review Inventory, College of Social Resources Department Verbal: Meeting with Chair Written Review Document Chair Initiated Requested Standardized Non-Standardized ANP CJ ECON GEO HST HDFS LIR PLS PSY SW SOC Consistency in Providing Feedback for Reappointment, Promotion, and Tenure Eleven departments responded to the item asking for a brief description of the information provided from the RPT committee to the department chair regarding RPT recommendations. Seven of the responding departments (ECON, SW, ANP, SOC, GEO, LIR, and HST) indicated that chairs are provided a summary in the form of a report, a letter, an evaluation, or a memo that included either a vote, ratings for review categories, or recommendations for RPT decision. PSY, HDFS, and SOC indicated that verbal input was Evaluation of MSU ADAPP ADVANCE 117

132 provided to the chair from the RPT committee mostly through meetings, discussions, and presentations. PLS stated, Advisory Committee makes recommendation to chair. Eight departments indicated that the candidate has an opportunity to discuss the results or recommendations of the RPT review; data indicated that this did not occur in HDFS, CJ, and PLS. SW, GEO, and LIR indicated that a meeting for each candidate is scheduled by the department chair to discuss the results; whereas, ECON, PSY, and HST indicated that the candidate is encouraged to request a meeting, and ANP indicated that meetings are only scheduled for challenges to the RPT result of the review. Congruence Between Inventory Data and Policy Analysis Congruence between inventory data and policy analysis (PA) findings for annual review is displayed in Table 63, and findings of the analysis for RPT are found in Table 64. Policy documents for only one department (SOC) were obtained for review. Inventory and PA data for both AR and RPT were only incongruent with regard to the assessment of the level of detail reportedly found (some) in the documents and the amount of detail found (much) by the evaluator for AR and RPT process descriptions in the SOC bylaws. Table 63. Congruence of Annual Review Inventory Information with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Social Science DEPT SOC Reference to AR in bylaws? Yes Amount of Detail Some Detail Where else is AR referenced? Not listed Evaluator Assessment References to AR are not just described in some detail but are described in much detail. Inventory and PA data are not congruent. Inventory and PA data are congruent. Inventory and PA data are somewhat congruent. Document was not obtained for review. Table 64. Congruence of Reappointment, Promotion, and Tenure Inventory Information with Policy Analysis, MSU ADAPP ADVANCE Policy Analysis, College of Social Science Reference Where else Amount DEPT to RPT in is RPT Evaluator Assessment of Detail bylaws? referenced? SOC Yes Some Detail? Not listed References to RPT are not just described in some detail but are described in much detail. Inventory and PA data are not congruent. Inventory and PA data are congruent. Alignment? Inventory and PA data are somewhat congruent. Document was not obtained for review. Alignment of Annual Review Policies College- and department-level policies that addressed university-level principles were analyzed for accessibility. Bylaws could be accessed only for the department of Sociology (SOC) in the College of Social Science. Documents for all other departments were requested but were not received prior to this Evaluation of MSU ADAPP ADVANCE 118

133 evaluation report. For SOC, information regarding annual review could be found under Review and Evaluation of Faculty Members in the department bylaws. Information included materials to be submitted, keeping a public file, and review and evaluation procedures that included criteria for faculty review. College- and department-level policies that addressed university-level principles were analyzed for clarity and specificity. Policies that were found to be clearly stated were scored a 1.00; policies that were found to be vague were scored a 0.50; and policies that were not found were scored a Average scores for each category were calculated for the College of Social Science and for each department where policy documents were found. Color-coding visually represents the transparency of the college and department policies based on the average scores for each category. Red represents a lack of clarity (0-0.25), yellow represents vague policies ( ), and green represents an average to high level of policy transparency ( ). The College of Social Science bylaws contained no information regarding faculty annual review (Figure 28). The department of Sociology, on the other hand, was found to have accessible and relatively clear policies regarding annual performance review expectations, procedures, and follow-up. Evaluation of MSU ADAPP ADVANCE 119

134 Figure 28. Alignment of AR policies for the College of Social Science, MSU policy analysis, Phase I. Table 65 shows the results of the evaluator s analysis of CSS annual review best practices. Best practices were identified from department responses to the 2009 human resources inventories and are displayed in the crosswalk in Table 31. As shown in the crosswalk, four items were matched from the 2009 and 2011 inventory data and the recommended best practices. One item referred to AR guidelines, one referred to AR input, and two referred to AR feedback. ANP and SOC did not respond to the 2009 MSU Annual Review Inventory. Where responses were given, inventory data aligned with best practices for annual Evaluation of MSU ADAPP ADVANCE 120

135 review in 2011 with the exception of annual review guidelines for SOC. For the remaining departments, greater alignment was found between best practices and inventory responses regarding annual review guidelines and less alignment found between best practices and inventory data referring to input in For items referring to annual review feedback, greater alignment was found between inventory data and best practices. Table 65. Annual Review Best Practices Comparison with 2009 MSU Annual Review Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Social Science Guidelines Input Feedback ANP CJ NR NR ECON NR b b GEO NR NR HST a b HDFS LIR PLS NR NR PSY NR NR SW SOC b b b - Aligned with best practices; - Does not align with best practices. a Meetings with pre-tenure faculty only. b Department Chair indicates that this occurs in some cases, but not in all. NR Department chair did not provide a response to this item. (-) departments that did not participate in inventories. Evaluation of MSU ADAPP ADVANCE 121

136 Alignment of Reappointment, Promotion, and Tenure Policies Information regarding reappointment, promotion and tenure also were included in the Review and Evaluation of Faculty Members in SOC bylaws. College- and department-level reappointment, promotion, and tenure policies that addressed universitylevel principles were analyzed for clarity and specificity. At the college level, it only was found clear that an RPT review committee shall be established for faculty RPT. At the department level, the Department of Sociology consistently had RPT policies that were accessible and had some degree of specificity (Figure 29). Evaluation of MSU ADAPP ADVANCE 122

137 Figure 29. Alignment of RPT policies for the College of Social Science, MSU policy analysis, Phase I. Table 66 shows the results of the evaluator s analysis of CSS reappointment, promotion, and tenure (RPT) best practices. Best practices were identified from human resources inventory data in 2009 and are displayed in the crosswalk in Table 31. As displayed in the crosswalk, four items were matched from the 2009 and 2011 inventory data and the reported best practices; two items referred to RPT guidelines and Evaluation of MSU ADAPP ADVANCE 123

138 two items referred to reappointment, promotion, and tenure materials. ANP, PSY, HST and SOC did not respond to the 2009 MSU Reappointment, Promotion, and Tenure Inventory. In these cases, half of the RPT guidelines inventory data aligned with best practices and all but one (HST) of the RPT materials inventory data aligned with best practices in For the remaining CSS departments, inventory data for RPT guidelines did not change in alignment with best practices from 2009 to 2011 (with the exception of GEO which showed more alignment for one item). For the items referring to RPT materials, inventory data for the Social Science departments aligned less with best practices in 2011 than in Table 66. Reappointment, Promotion, and Tenure Best Practices Comparison with 2009 MSU Reappointment, Promotion, and Tenure Inventory and 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Data, College of Social Science Guidelines Materials ANP - - CJ ECON GEO NR HST - - HDFS NR LIR PLS PSY - NR - SW NR SOC Aligned with best practices; - Does not align with best practices. NR Department chair did not provide a response to this item. (-) departments that did not participate in inventories. Evaluation of MSU ADAPP ADVANCE 124

139 New and Continuing Evaluation Activities The E & A Center has engaged in the evaluation of the ADAPP-ADVANCE initiative since January During this time, Center staff and the ADAPP project team have worked together closely to ensure a thorough and rigorous evaluation of project processes and progress toward goals. Via this close working relationship, the evaluation has been well-positioned to measure a number of project accomplishments. As with any short-term initiative, evidence of impact tends to emerge near the end of the initiative s life cycle, and the ADAPP-ADVANCE project is no different. During the final year of the grant ( ), the project and the university will undertake a number of important tasks designed to sustain and enhance the university s focus on recruiting, retaining and advancing a high quality and diverse faculty well beyond the end of NSF grant funding. MSU has begun the process of institutionalization by engaging with the Deans in a planning process for migrating mission-critical project efforts to the colleges. The ADAPP project team and E & A Center leadership have begun to discuss the value of continuing the evaluation into During a no-cost extension year, the E & A Center would continue to collaborate with the ADAPP transition team and IAG to conduct an evaluation that meets the university s needs. The following have been discussed as rationale for continuing the evaluation through September 2014: Allow the evaluation team to monitor the institutionalization process during its first year and provide data and findings regarding successes of and challenges to the process. The university could use this information to efficiently and judiciously adjust and moderate the process. Administer the Work Environment Survey in late Spring 2013, rather than in Winter Extending the time between the first and second administration of the W/E survey will provide more time a full 4 years and more opportunity to measure change in respondents perceptions of human resource processes and the work environment that may be attributable to project efforts. Given the wealth of data generated by the W/E survey, an additional year of time for analysis would facilitate the integration of W/E survey data with a number of other data sources, including college-level retention models, HR inventory data, and final findings of the policy analysis. The evaluation team believes that a thoughtful and thorough synthesis of data will provide a more comprehensive picture of ADAPP-ADVANCE project impact, as well as a clear look at the university s current status with regard to diversity efforts. Conduct a proposed study of the FEA models in the three grant-funded colleges to inform the university regarding the implementation and impact of these models. University-wide support for the Faculty Excellence Advocate role within each college represents an investment of resources that should be supported by evidence of their respective impacts on faculty quality, diversity, and productivity. This study will begin in 2012 but ideally would continue into 2014 so that the models can be studied within the real context of their operation, embedded in each college, rather than as extensions of the ADAPP-ADVANCE project. The plan for the proposed study of FEA models can be found in Appendix E. Develop for the colleges and the university a common set of tools and metrics to measure the effectiveness of the mentoring programs. In 2011, a university-wide mentoring policy was adopted requiring each college to develop and implement a formal mentoring program for new faculty. University guidelines for the mentoring policy require an evaluation component that would assess each program s effectiveness every 5 years. Without such a set of tools, it is unclear how the university and each college will gauge the impact of mentoring at MSU and communicate emergent best practices in faculty mentoring among colleges and units. Facilitate the collection and analysis of a sufficient body of qualitative data to inform recommendations at the unit level for Phase 2 of the policy analysis and provide an opportunity Evaluation of MSU ADAPP ADVANCE 125

140 for the evaluation to examine how the university and each of the three colleges make meaning of findings and recommendations that will emerge from Phase 2. Phase 2 of the policy analysis will begin in Summer 2012 and will be focused on impact, as well as process, and will attempt to probe policy implementation at the unit level. Final and actionable recommendations could be delivered in January Continuing the evaluation into 2014 will permit evaluators to observe how recommendations are assimilated or not into practice. The MSU ADAPP-ADVANCE project is policy-focused, and therefore, how findings from the policy analysis translate into policy action should be of great interest not only to the university but also to the NSF. Specifically, the following evaluation activities would continue after September 2013 if possible: 1. Analysis of Work Environment Survey data Spring 2009 and Spring 2013 comparisons, crosswalk and comparisons of W/E data (2009, 2013) with HR inventory data (2009, 2011, 2013). 2. Comparative analysis of administration of 1 additional HR inventory (Spring 2013). 3. Collection and analysis of 1 additional year of indicator data ( ). 4. Continuation of FEA Case Study interviews of Deans, FEAs, unit chairs, and faculty, and final report on impact and sustainability of each model. 5. Development of metrics and tools for assessing effectiveness of mentoring programs review of colleges mentoring programs, interviews of Deans, FEAs, and unit chairs, draft and pilot tools, and finalize tools. 6. Integration of W/E survey data, HR inventory data, policy analysis, and retention models transform and synthesize multiple sources of data to develop a comprehensive report of 5-year change in HR policies and practices and resultant impact on recruitment, retention, and advancement of women. 7. Observation and analysis of institutionalization of ADAPP-ADVANCE to measure changes in HR and other relevant policy and procedures related to diversity efforts. An updated timeline for conducting project evaluation activities can be found in Figure 30. The following evaluation activities will be conducted during Year 5 of the ADAPP-ADVANCE project. 1. Re-administer the combined Annual Review and Reappointment, Promotion, and Tenure Inventory to in Spring The purpose is to measure change in HR practices postintervention. Further, parallel survey items will be developed and included in the next administration of the MSU Work Environment Survey in Winter 2012/2013 or Spring Continue analyses of Work Environment Survey data in the context of planned or priority project activities. Further analyses of data have been performed and provided to project personnel to facilitate a study to collect qualitative data regarding the experiences of faculty of color. Planning for other such studies is ongoing and the Evaluation Team will provide support for data analysis and research design and implementation. 3. Make minor modifications based upon analyses of reliability and factor loading to and readminister the Work Environment Survey instrument in Winter 2012/2-13 or Spring Implement Phase 2 of the HR Policy Analysis. This comprehensive review and analysis of university, college, and department/unit policy documents and updates to documents pertinent to faculty performance evaluation, reappointment, promotion, and tenure will result in an assessment of current policy at each level based upon evidence of quality, inclusiveness, and objectivity in written policy and its implementation. Evaluation of MSU ADAPP ADVANCE 126

141 5. Collect data on the national and regional availability of STEM and SBS faculty candidates in order to contextualize MSU hiring data. Conduct comparative analyses of MSU STEM and SBS candidate applicant pool data, finalist pool data, and hiring data for 2007 through Conduct interviews of Deans, department chairs, women faculty, and men faculty. Each Dean will be interviewed each year, while department chairs and faculty members will be randomly sampled (when possible, samples will be stratified by department, rank, gender, ethnicity). Interview protocols have been designed to elaborate on and probe themes from the findings of questionnaire data and investigate interpretation and implementation of policies targeted for ADAPP efforts. 7. Continue to refine and enhance statistical models to identify factors that impact STEM/SBS tenure-stream faculty retention at MSU and to examine to what extent these factors impact faculty retention in different colleges. Models will be integrated with MSU Work Environment Survey data in Summer Disseminate via the ADAPP-ADVANCE Team brief and substantive annual reports (Data Digests) for the Deans of each of the three colleges. These reports provide disaggregated results from the MSU Work Environment Survey as well as data and findings from the HR inventories and Phase 1 of the Policy Analysis. The reports will include recommendations for action steps and will be delivered not later than August 30 annually. 9. Meet with the Grant Management Team (GMT), the Institutionalization Action Group (IAG), and other ADAPP project personnel, on site quarterly, to review the current plan for the evaluation, gather feedback and discuss the Year 4 Evaluation Report, and receive an update on current project activities, objectives, and challenges. 10. Continue to collect and analyze primary and secondary data, as requested. The E & A Center will develop/modify and test instruments/protocols and administer them to collect data that are deemed necessary for the external evaluation and will conduct secondary analyses of data provided by internal project personnel. 11. Continue analysis of ADVANCE toolkit indicator data and match university comparison data. ADVANCE toolkit indicators are vital to assessing progress towards project and NSF goals. Information regarding the hiring, retention, and advancement of women faculty at MSU were provided in the ADAPP-ADVANCE Annual Report toolkit data and comparisons of MSU with other institutions were conducted for the Year 2, 3 and 4 Annual Reports. The E & A Center will continue to analyze year to year changes and collect indicator data from no fewer than two other comparable universities in order to compare Michigan State to similar universities for all years of the project. 12. Attend and observe meetings of the ADAPP team with the External Advisory Board and the NSF (as requested). The E & A Center will be available to observe meetings to provide summary feedback regarding process outcomes and incorporate observations into annual reports. Evaluation of MSU ADAPP ADVANCE 127

142 Figure 30. MSU ADAPP ADVANCE evaluation proposed timeline. Evaluation of MSU ADAPP ADVANCE 128

143 Figure 30. MSU ADAPP ADVANCE evaluation proposed timeline (continued). Evaluation of MSU ADAPP ADVANCE 129

144 Summary and Recommendations Summary The following summary is framed by ADAPP-ADVANCE project goals and synthesized from findings of the NSF toolkit indicator data, findings of the Phase 1 policy analysis, the faculty retention study, the 2009 and 2011 Human Resources department/unit inventory data, and document (including website) review, as well as dialogue with project personnel during the past year. Recommendations are based upon these findings, are subject to verification by project personnel, and are offered for the purpose of project improvement. The following summary is framed by ADAPP-ADVANCE project goals and synthesized from findings of the NSF toolkit indicator data, findings of the Phase 1 policy analysis, the faculty retention study, the 2009 and 2011 Human Resources department/unit inventory data, and document (including website) review, as well as dialogue with project personnel during the past year. Recommendations are based upon these findings, are subject to verification by project personnel, and are offered for the purpose of project improvement. Notable Findings related to Project Goals: GOAL 1 - Increase the number of women faculty recruited and appointed into the CNS, CSS and ENG. MSU continues to experience reduced levels of faculty hiring. Between October 2007 and 2008 prior to the beginning of ADAPP-ADVANCE, MSU hired 65 STEM faculty members. During the first year of the ADAPP-ADVANCE initiative ( ), 38 STEM faculty members were hired, including 14 STEM women. During the second year of the initiative, only 20 STEM faculty members were hired, half of whom were women, and during Year 3 ( ), 18 STEM faculty were hired with 6 being women. During the first 3 years of the ADAPP initiative, no women full professors were hired in STEM or SBS units. While hiring in STEM and SBS units has decreased significantly between 2007 and 2011, the proportions of women and men new faculty hires have fluctuated very little. During the baseline year, 40% of STEM hires were women; 37% were women in ; 50% were women in (primarily due to 7 women hired in SBS units); and 33% of STEM hires were women in (primarily due to 8 men hired in engineering and physical sciences departments). Between October 2010 and October 2011, 4 women and 11 men were hired by units in the Colleges of Engineering and Natural Science. Overall, hiring differences in the STEM NSF categories during the first 3 years of the ADAPP-ADVANCE project favored males (n = 29) over females (n = 12). A majority (83%) of all STEM hires were at the rank of assistant professor for both men and women. During , STEM departments of the Social and Behavioral Sciences categories hired only 3 faculty members, 2 women assistant professors and one male full professor. Hiring differences in SBS units during the first 3 years of the ADAPP-ADVANCE initiative favored women, with slightly more female professors (n = 18) than male professors (n = 17) hired between 2008 and 2011; however, 95% of SBS female hires were assistant professors as compared to 65% of male hires. Between October 2008 and October 2011, the composition of faculty in units of the following NSF categories shifted toward slightly higher percentages of women: Biological and Agricultural Sciences increased female faculty representation by 1.2%; Earth and Atmospheric Sciences by 1.1%; Engineering by 1.1%; Mathematical and Computer Sciences by 1.2%; Professional/Other by 2.4%; and Psychology by 3.7%. The percentage of female professors in the Physical Sciences decreased by 1.0% and the percentage of female professors in Social Sciences decreased by 0.1%. Overall, women gained 0.5% of Evaluation of ADAPP 130

145 the available positions in STEM NSF departments and 1.0% of available positions in SBS NSF departments. GOAL 2 Increase retention of women faculty in CNS, CSS and ENG. In the NSF STEM categories, 5 female tenured and tenure-track faculty and 18 male tenured and tenuretrack faculty left MSU for voluntary, non-retirement reasons, while 7 females and 22 males retired during the first 3 years of the ADAPP-ADVANCE project. During , 2 senior faculty women and 8 male senior faculty retired from departments in the Biological and Agricultural Sciences. As a percentage of each gender and rank group, rates of voluntary non-retirement attrition were relatively equal for all faculty in the NSF STEM categories. For example, 5% of both female and male assistant professors left MSU during the years of the ADAPP ADVANCE project for voluntary non-retirement reasons. In the SBS categories, more female tenured and tenure-track faculty (n = 8) left MSU for voluntary, nonretirement reasons than did male faculty (n = 6) between 2008 and The Social Sciences experienced the most attrition with 4 female faculty (2 assistants and 2 associates) leaving for voluntary non-retirement reasons and 5 more female faculty (1 associate and 4 full professors) retiring between 2008 and In the NSF SBS categories, the rate of voluntary non-retirement attrition was higher for female assistant professors (3%) than for male assistant professors (1%). Voluntary non-retirement rates of attrition were relatively equal for women and men at the rank of associate and full professor. Findings of the STEM faculty retention study suggest that the survival rates for women professors are different than for men professors at MSU. STEM women faculty, historically have left, and continue to leave the University at higher rates than do STEM men faculty of the same rank. Retention rates for STEM women assistant professors are significantly lower than for men assistant professors hired between 1991 and At the university level, 39% of STEM women assistant professors and 28% of STEM men assistant professors hired between 1991 and 2000 departed MSU voluntarily; 20% of STEM women and 15% of STEM men assistant professors hired between 2001 and 2011 left the university voluntarily. High risk of losing STEM women faculty, particularly assistant professors, extends over longer periods of time than does the high risk of losing STEM men faculty. Attrition rates for both men and women are low during the first 3 years. Higher attrition rates generally occurred between Years 5 and 10, during preand early-tenure. The attrition rate of STEM women faculty increases and remains high from Year 4 through Year 11, while the rate of STEM men faculty attrition increases slightly in Year 4 and drops significantly by Year 8. Patterns in STEM women faculty leaving the university are less easily explained than are patterns in STEM men faculty attrition. In general, attrition rates are higher at early times in faculty careers and more moderate at later times for all ranks, in all hiring cohorts. Men tended to leave at a lower rate post-tenure (Years 8 and on) than during their pre-tenure stage. Women showed a similar trend, except for dramatic unexplained increases in attrition later in their careers. STEM women assistant professors hired before 2000 tended to leave the university between Years 7 and 10, while women assistant professors hired after 2000 tended to leave between Years 3 and 6. Attrition rates peaked at Years 4, 9, and 10 for women and peaked at Years 4, 7, 8, and 12 for men. The average time to departure was 14.6 years for women, which is significantly shorter than the average of 15.9 years for men. Analyses further suggested possible gender difference in rates of faculty attrition, primarily at the rank of full professor in some colleges, including College of Agriculture, College of Engineering, and College of Human Medicine. In the College of Natural Science, attrition rates of women assistant professors also were significantly higher than for male colleagues. Evaluation of ADAPP 131

146 GOAL 3 Increase advancement of women faculty in CNS, CSS and ENG. During the first 3 years of the ADAPP-ADVANCE project, in all STEM NSF categories, promotion favored males with 9 females and 26 males promoted from assistant to associate professor, and 2 females and 13 males promoted from associate to full professor. However, female faculty were promoted from assistant to associate at a slightly higher rate than were male faculty (16% and 13%, respectively) and at a slightly lower rate (2% and 4%, respectively) from associate to full professor during the years of the project. For the SBS NSF categories, the same number of female and male faculty were promoted from assistant to associate professor (n = 11) but fewer women than men were promoted from associate to full professor (9 and 12, respectively). As a percentage of each gender group, rates of promotion in both categories (assistant to associate and associate to full professor) were equal (7%). In , women held 7% of the titled leadership positions available in the College of Engineering, 14% in the College of Natural Science, and 38% in the College of Social Science. There were no women department heads in the Colleges of Engineering and Natural Science. This lack of female leadership representation at the department level accounts for MSU s much lower percentage of females in titled leadership positions (6%) than in peer institutions, which average 27% of leadership positions held by women. GOAL 4 Improve the work environment (climate) for women faculty in CNS, CSS, and ENG. The Strategic Human Resources Model assumes that improvement in the transparency, consistency, and alignment of human resources policies at the college- and department/unit-level will have direct and positive impact on the work environment for all faculty. Findings of the Phase 1 Policy Analysis indicated that department-level AR and RPT policies and procedures had become more aligned with best practices between 2009 and Not surprisingly, policy and procedural information related to reappointment, promotion, and tenure are more assessable, clear, consistent, and aligned than are policies and procedures related to annual review. Common remaining issues are primarily associated with communication of expectations, consistency in the application of procedures within and across departments, and extracting policy and procedure for annual review from the processes of reappointment, promotion and tenure award. While no new measures of climate were administered this year, MSU Work Environment Survey data were synthesized with findings of the STEM faculty retention study to determine if there were any measureable differences between the responses of those faculty who stayed and those who departed MSU to determine the impact of climate on faculty retention. A number of statistically significant differences were found. In all cases, departed faculty had less positive perceptions of the work environment than did retained faculty. Departed faculty, compared to retained faculty, agreed less often that the recruiting and hiring processes for their units were fair, consistent, and diversified; that they had a clear understanding of the promotion and tenure processes within their units and that the standards used were objective and consistently applied; that the faculty performance evaluation and review processes within their units were fair and that the criteria were clearly communicated; that the climate within their units was diversified in general; that they were treated with respect by students and colleagues within their units; that the allocation of resources and workload was satisfying; and that they were satisfied at MSU in general. A refined analysis, matching retained and departed faculty on year and college of hiring, found no statistically significant differences between the responses of retained and departed faculty, but in all cases the responses of departed faculty were less positive. Comparisons also were made for women and men faculty in the subsample. In all cases, women STEM faculty who left MSU had better perceptions of the work environment than did men faculty who left the university. Further, women faculty who departed generally had more positive responses than women who stayed at MSU. Evaluation of ADAPP 132

147 Another indirect measure of climate was provided by the synthesis of NSF toolkit hiring, promotion, and attrition data that demonstrates how the composition of faculty is changing over time. Between the baseline year ( ) and ADAPP-ADVANCE Year 3, the percentage of tenure-track positions held by women increased by 0.5% in STEM units and by 1.0% in SBS units. While not a specific goal of ADAPP- ADVANCE, the percentage of women in non-tenure track positions also is monitored by the project and evaluators. During this same time period, the composition of non-tenure track faculty changed more significantly with a loss of 4.1% of women non-tenure track faculty in STEM and a loss of 0.7% of SBS non-tenure track women. It is possible that a portion of these losses may represent movement of women into tenure-track positions but that cannot be confirmed at this time. Also of note is a decrease (0.8%) in senior women STEM faculty between 2008 and 2011, primarily due to retirements. Attrition due to retirement during the first 3 years of ADAPP impacted the number of senior women faculty in the Biological and Agricultural Sciences, where 5 women full professors retired, Physical Sciences where 1 female full professor retired, and Social Sciences which lost 5 senior women to retirement. Two of these senior women were replaced in the Biological and Agricultural Sciences and all were replaced in Social Sciences via promotion. These changes suggest that while the ranks of female professors are increasing slightly at MSU the overall representation of women in STEM and SBS units, particularly at higher ranks, is decreasing. Recommendations As the project enters Year 5, the evaluation has directed more attention to collecting and reporting data that respond to evaluation questions related to project impact. These questions are: 1. What is the nature and extent of the impact of ADAPP project activities on the recruitment, appointment, retention, promotion, and leadership of women faculty within the MSU Colleges of Natural Sciences, Social Sciences, and Engineering? 2. What is the nature and extent of the impact of ADAPP project activities on the work environment for women faculty in the Colleges of Natural Sciences, Social Sciences, and Engineering? 3. What is the nature and extent of the impact of ADAPP project activities on the human resources and institutional policy environments that promote diversity and excellence at MSU, including the extent to which policies are more aligned, transparent, consistent, and objective? As more sources of data are analyzed and synthesized in the evaluation of the ADAPP-ADVANCE project, relationships among these questions, persistent conditions, emerging issues, and findings of data begin to crystalize around themes of interest. These themes include: Persistently low and atypical representation of STEM women in titled leadership positions, including a complete absence of department chairs in the Colleges of Engineering and Natural Science; Decrease in the representation of senior women in STEM departments during the past 3 years; Disparity in gender equity in hiring and promotion in particular departments that may or may not be attributable to availability of faculty candidates in certain disciplines such as engineering and the physical sciences; Similar rates of STEM faculty attrition between women and men, and from year to year, still result in significantly higher rates of and less predictable patterns in women s attrition over time; Differences in perceptions of the work environment between women STEM faculty who left MSU and those who stayed, suggesting that women who left MSU were generally more satisfied with the work environment than those who stayed; Misalignments of and information gaps in department-level human resources policies and practices with recommended or best practice following 4 years of ADAPP focus on HR policy and practice; and Evaluation of ADAPP 133

148 Low association between faculty responses to MSU Work Environment Survey items regarding faculty annual review and advancement and department leadership responses on matched items from the Annual Review Inventory and RPT Department Inventory. Based upon these themes and plans for the Year 5 evaluation, the E & A Center Evaluation Team recommends: 1. Close attention to the findings of the faculty retention study and repetition of this study over time. While some findings of the faculty retention study may seem intuitive, their value should not be underestimated. The models provide a snapshot look at how successfully the University is addressing faculty retention issues, including the relationship of work environment to faculty retention. Over time, as more data are added to the models, they have the potential to predict with a high degree of accuracy retention outcomes for subgroups of faculty. Periodic analyses of these data would indicate if ADAPP-ADVANCE project and other University initiatives aimed at retaining women faculty in STEM fields have resulted in positive outcomes for the University. As more data are added to this model over time, comparisons can be made between the retention of faculty hired prior to, during, and following these targeted efforts to improve faculty quality and diversity. Currently, the model provides important and timely data that can inform areas for further investigation. The evaluation team will synthesize and triangulate findings from these models with other data sources to provide a clearer picture of issues that may impact the retention of STEM women at Michigan State University. 2. Continued compilation, synthesis, and augmentation of college-level data from a range of sources to clearly and precisely represent for each College, and to the extent possible, each unit within the colleges, the status of women faculty and identify priority areas of challenge, as well as success. It is paramount to ADAPP-ADVANCE and University efforts to ensure the hiring, retention, and promotion of a diverse and high-quality faculty that sufficient and reliable data are readily available to MSU decision and policy makers. The E & A Center team will continue to produce data digests that enable the Office of the Provost, Academic Human Resources, and Deans to set ambitious yet realistic benchmarks and indicators for each College and to regularly monitor progress toward goals. These reports will be delivered annually in July. 3. Collection of targeted qualitative data to address incongruence in some findings of the evaluation. Data from the MSU Work Environment Survey demonstrated that issues of climate impact how MSU women faculty perceive their work environment, and that women s perceptions are more negative than those of their same-rank male peers. While the required NSF Toolkit indicator data monitor how the composition of faculty is changing over time, evaluators contend that issues related to climate likely will be more challenging to address than improving the numerical representation of women in STEM. It will be necessary to assess climate more proximally (i.e., at the College or unit level) with careful attention to concerns that previous data suggest are prevalent and persistent in the unique work environments. Previously collected interview and survey data generally did not provide a clear picture of the extent to which issues of gender inequity were pervasive and severe at MSU. Most data suggested that the situation for female faculty is highly dependent upon their respective college and/or department of assignment. Data collection efforts should focus on ADAPP and University goals of improving diversity and excellence, while attempting to understand which issues of climate are related to inequitable treatment and which demonstrate important differences in the preferences of women and men faculty with regard to working environment characteristics. 4. Based upon findings of Phase 1 of the Policy Analysis: a) Deans and department chairs should review their respective data for accuracy and provide additional data as appropriate, including missing documents (i.e., department bylaws and AR/RPT documents) for analysis. Evaluation of ADAPP 134

149 b) Deans and department chairs should determine if noted misalignments or information gaps are reasonable and warranted based upon the unique context of each department and discipline. Misalignments or gaps that are determined to be unwarranted should be addressed and the Faculty Excellence Advocate should be engaged in this work. c) Parallel questions regarding transparency, consistency, and alignment of AR and RPT policy and procedures should be embedded in the next administration of the MSU Work Environment Survey (Spring 2013). d) The Annual Review and Reappointment, Promotion, and Tenure Combined Inventory should be re-administered in Spring 2013 as a post-intervention measure of change in HR policy and procedures. 5. Continued collaboration of the Offices of the Provost, Academic Human Resources, and College Deans to identify clear and measurable expectations related to ADAPP and University goals related to diversity. These efforts should be led by the newly-formed IAG. Such discussions may include clarifying and documenting agreements regarding the role, responsibility and authority of the Faculty Excellence Advocates of each college, as well as agreement on benchmarks and indicators of progress toward goals. Evaluation of ADAPP 135

150 Appendices Appendix A: 2011 MSU ADAPP-ADVANCE Annual Review and Reappointment, Promotion, and Tenure (AR/RPT) Combined Inventory Appendix B: 2009 MSU Annual Review Inventory Appendix C: 2009 Reappointment, Promotion, and Tenure Inventory Appendix D: Indicator Analysis Table Appendix E: MSU ADAPP ADVANCE Faculty Excellence Advocate Study Evaluation of ADAPP 136

151 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 Appendix A MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 1 of 27 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory MSU ADAPP-ADVANCE is continuing to address processes by which faculty are recruited, advance, and are mentored. The purpose of this inventory is to document unit-level AR and RP&T processes at the midpoint of the ADAPP-ADVANCE initiative, including any change in practice that has occurred since the original inventory was completed. This information will be used to guide long-term program and policy decisions across the University. Please provide the requested information regarding the annual review and the reappointment, promotion, and tenure processes for your unit. For each item, an opportunity for you to provide more specific information can be obtained by selecting Other" or "Additional explanation. At the end of this inventory, you will have the opportunity to request a short follow-up interview to share any additional relevant information. *1. Name of individual completing the inventory: *2. College: *3. Department/Unit: Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

152 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 2 of 27 Annual Review Instructions: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines 4. Do your unit bylaws make reference to the Annual Review process? Yes No << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

153 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 3 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 4a. Which of the following are true regarding how the Annual Review process is referenced in the unit bylaws? i. The Annual Review process is mentioned but not described. ii. The Annual Review process is described in some detail. iii. The Annual Review process is described in much detail. iv. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

154 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 4 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 5. Is the Annual Review process referenced in another unit document? (Please select all that apply.) i. Unit AR and/or RP&T document ii. Unit website iii. Other (Please specify) Yes No << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

155 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 1 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 5 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) Which of the following are true regarding how the Annual Review process is referenced in unit AR and/or RP&T document? i. The Annual Review process is mentioned but not described. ii. The Annual Review process is described in some detail. iii. The Annual Review process is described in much detail. iv. Other (Please specify): Which of the following are true regarding how the Annual Review process is described on the unit website? i. The Annual Review process is mentioned but not described. ii. The Annual Review process is described in some detail. iii. The Annual Review process is described in much detail. iv. Other (Please specify): Which of the following are true regarding how the Annual Review process is referenced in iii. Other (Please specify):? i. The Annual Review process is mentioned but not described. ii. The Annual Review process is described in some detail. iii. The Annual Review process is described in much detail. iv. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 141

156 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 1 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 6 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 6. Is faculty performance evaluated in a consistent and systematic way (i.e., each case is handled in the same way) or is there variation in the evaluation of faculty performance within the unit? Yes, our faculty with similar appointments are evaluated in a consistent way. No, the process varies Additional explanation: << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 142

157 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 7 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 6a. Please explain reason for variation in the performance evaluation process: << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

158 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 8 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 7. Is faculty performance data collected using a standard format? Yes No Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

159 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 9 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 7a. Please explain reason for not collecting data using a standard format: << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

160 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 2 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 10 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 8. How are expectations of faculty performance communicated to faculty at the unit level? (Please select all that apply.) a. Verbally, in a formal meeting with the chair b. Verbally, in informal discussion with the chair c. Verbally, in discussion with senior faculty d. Verbally, in discussion with a faculty mentor e. In writing, in the unit AR and/or RP&T document f. In writing, in faculty new-hire paperwork g. Other (Please specify): 9. How is faculty performance evaluated at the unit level? (Please select all that apply.) a. With a standard performance appraisal process specific to my unit. b. With a general performance appraisal process not specific to my unit. c. On an individual basis, no performance appraisal process is used. d. Other (Please specify): 10. Who participates in the evaluation of faculty performance? (Please select all that apply.) a. A standing committee b. An adhoc committee c. An individual designee of the chair d. The chair e. The faculty as a whole f. Other (Please specify): Evaluation of MSU ADAPP ADVANCE 146

161 MSU ADVANCE AR and RP&T Combined Inventory Page 2 of 2 9/23/ a. To what extent are faculty provided specific descriptions of what constitutes above average (e.g., outstanding, superior), average, or unacceptable levels of performance regarding expectations for faculty performance in the areas listed in the table? Levels of performance are not mentioned. Performance levels are referenced but not described. Levels of performance are described in some detail. Performance levels are fully described. Not applicable to my unit Research Teaching Service 11b. To what extent are faculty provided specific descriptions of what constitutes above average (e.g., outstanding, superior), average, or unacceptable levels of performance regarding expectations for faculty performance in the areas listed in the table? Levels of performance are not mentioned. Performance levels are referenced but not described. Levels of performance are described in some detail. Performance levels are fully described. Not applicable to my unit Membership in Disciplinary Societies Mentoring Leadership in Disciplinary Societies Other (Please specify): 12. Are unit-specific productivity measures for evaluating faculty performance at Annual Review communicated to the college Dean s office? Yes No Additional explanation: << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 147

162 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 2 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 11 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Materials 13. What materials are used for faculty Annual Review? (Please select all that apply) a. Updated Curriculum Vitae b. Completed performance appraisal form c. Copies of publications d. Citations to research e. Student ratings of teaching f. Teaching portfolios (in addition to student ratings) g. Personal statement h. Other (Please specify): Evaluation of MSU ADAPP ADVANCE /23/2011

163 MSU ADVANCE AR and RP&T Combined Inventory Page 2 of 2 9/23/ Are the process for submission of and the materials required for Annual Review communicated to faculty members in writing and/or verbally? (Please select all that apply.) a. A protocol for submitting materials for Annual Review is written in the unit bylaws b. A protocol for submitting materials for Annual Review is written in the unit AR and/or RP&T document c. A protocol for submitting materials for Annual Review is written in faculty new-hire paperwork d. Faculty are verbally informed of material requirements for Annual Review e. There is no standard protocol for describing the process or material requirements for Annual Review f. Other (Please specify): 15. Are candidates limited in the amount of materials that can be submitted for use in the Annual Review process? Yes No Additional explanation: << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 149

164 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 2 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 12 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Input 16. For faculty with joint appointments, how are units to which the faculty member is assigned involved in the Annual Review process (not including salary review)? a. All units are involved equally in the Annual Review process. The performance expectations of each unit are considered and input from each unit is collected systematically. b. The tenure home unit is primarily responsible for faculty Annual Review. The performance expectations of each unit are considered and input from each unit is collected. c. The tenure home unit is completely responsible for faculty Annual Review, but the performance expectations of and input from each unit are considered. d. The tenure home unit is completely responsible for faculty Annual Review, and the performance expectations of and input from each unit are not considered. e. Other (Please specify): Evaluation of MSU ADAPP ADVANCE 150

165 MSU ADVANCE AR and RP&T Combined Inventory Page 2 of 2 9/23/ a. How are performance expectations and input from multiple units shared in the Annual Review process of jointly appointed faculty? a. A formal joint review process which evaluates specific expectations for professional accomplishments b. An informal joint review process in which unit chairs evaluate general expectations c. Review only by tenure home unit d. Other (Please specify): 17. Is any input expected from faculty mentors for the Annual Review process? Yes No Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 151

166 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 1 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 13 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Input (Cont'd) 17a. How is input from faculty mentors collected? (Please select all that apply) i. Annual Review input is gathered informally from mentors ii. A protocol for gathering input from mentors is addressed in the unit bylaws iii. Mentors are asked to complete a standard form for faculty Annual Review iv. Mentors are included on the Annual Review committee of mentees v. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 152

167 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 2 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 14 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Input (Cont'd) 18. Is involvement in disciplinary societies/professional organizations (e.g., leadership, membership, and/or presentations) discussed and documented in the Annual Review process? Yes No Additional explanation: Evaluation of MSU ADAPP ADVANCE /23/2011

168 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 2 of During the Annual Review process, does the faculty member have an opportunity to provide input in setting their future goals and expectations of professional accomplishments for future evaluation processes? Yes No Other (Please specify): Q19a. Are opportunities for academic leadership discussed with midand senior-level faculty during the Annual Review process? Yes No Additional explanation: << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

169 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 2 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 15 of 27 Annual Review Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Feedback 20. How is Annual Review feedback provided to the faculty member? (Please select all that apply.) a. Through a meeting initiated by the chair b. Through a meeting with chair upon request c. Through a written document standardized by the unit d. Through a written document that is not standardized e. Other (Please specify): Evaluation of MSU ADAPP ADVANCE /23/2011

170 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 2 of Does feedback provided on the Annual Review performance evaluation refer to the results of prior annual reviews? Yes No Additional explanation: 22. If applicable, are expectations of performance necessary for promotion included in the written feedback to the faculty member? Yes No Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

171 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 16 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines 23. Do your unit bylaws make reference to the Reappointment, Promotion, and Tenure (RP&T) process? Yes No << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

172 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 17 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 23a. Which of the following are true regarding how the RP&T process is referenced in the unit bylaws? i. The RP&T process is mentioned but not described. ii. The RP&T process is described in some detail. iii. The RP&T process is described in much detail. iv. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

173 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 18 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 24. Is the Reappointment, Promotion and Tenure process referenced in another unit document? (Please select all that apply.) i. Unit RP&T document ii. Unit website iii. Other (Please specify) Yes No << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

174 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 1 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 19 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) Which of the following are true regarding how the RP&T process is referenced in unit RP&T document? i. The RP&T process is mentioned but not described. ii. The RP&T process is described in some detail. iii. The RP&T process is described in much detail. iv. Other (Please specify): Which of the following are true regarding how the RP&T process is described on the unit website? i. The RP&T process is mentioned but not described. ii. The RP&T process is described in some detail. iii. The RP&T process is described in much detail. iv. Other (Please specify): Which of the following are true regarding how the RP&T process is referenced in iii. Other (Please specify):? i. The RP&T process is mentioned but not described. ii. The RP&T process is described in some detail. iii. The RP&T process is described in much detail. iv. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 160

175 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 2 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 20 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Guidelines (Cont'd) 25. Do your unit s RP&T guidelines include references (e.g., link to the university website) to university policy on stopping the tenure clock? Yes No Additional explanation: 26. Is the committee responsible for carrying out the RP&T review of faculty described in the unit bylaws or other documents? (Please select all that apply.) a. No, the committee is not described b. Yes, in unit bylaws c. Yes, in the unit RP&T document d. Yes, on the unit website e. Yes, in another document (Please specify): Evaluation of MSU ADAPP ADVANCE 161

176 MSU ADVANCE AR and RP&T Combined Inventory Page 2 of 2 9/23/ Are there different criteria for faculty at different levels in the RP&T process in your unit? (i.e., reappointment, promotion to associate with tenure, reappointment of associate with tenure award, promotion from associate to full)? Yes No Additional explanation: 28. Are specific evaluation criteria of professional accomplishments used by the RP&T review committee to evaluate the strength of each candidate s application? Yes No Additional explanation: 29. Are college level criteria used by the unit to develop the unit criteria for RP&T evaluation? Yes, college level criteria are used No, college level criteria are available but not used No, college level criteria are not available Other (Please specify): 30. Are criteria for RP&T evaluation provided to the writers of external letters of evaluation? Yes No Additional explanation: << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 162

177 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 2 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 21 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Materials 31. Beyond MSU Form D, what materials are used for the first faculty reappointment? (Please select all that apply) a. Copies of publications b. Number of citations to research c. Teaching portfolios d. Personal statement e. External (to MSU) review letters f. Other (Please specify): 32. Beyond MSU Form D, what materials are used for faculty promotion with tenure? (Please select all that apply) a. Copies of publications b. Number of citations to research c. Teaching portfolios d. Personal statement e. External (to MSU) review letters f. Other (Please specify): 33. Beyond MSU Form D, what materials are used for faculty promotion after tenure award? (Please select all that apply) a. Copies of publications b. Number of citations to research c. Teaching portfolios d. Personal statement e. External (to MSU) review letters f. Other (Please specify): Evaluation of MSU ADAPP ADVANCE 163

178 MSU ADVANCE AR and RP&T Combined Inventory Page 2 of 2 9/23/ Are the process for submission of and the materials required for RP&T communicated to faculty members in writing and/or verbally? (Please select all that apply.) a. A protocol for submitting materials for RP&T is written in the unit bylaws. b. A protocol for submitting materials for RP&T is written in the unit RP&T document c. A protocol for submitting materials for RP&T is written in faculty new-hire paperwork d. Faculty are verbally informed of material requirements for RP&T e. There is no standard protocol for describing the process or material requirements for RP&T f. Other (Please specify): 35. Are candidates limited in the amount of materials that can be submitted for use in the RP&T process? Yes No Additional explanation: 36a. By whom are the formation of the pool of external reviewers for the RP&T process selected? (Please select all that apply) a. By the candidate b. By the chair c. Through criteria written in the unit bylaws d. Through guidelines written in the MSU Faculty Handbook e. Through college guidelines f. On an individual basis with no formal guidelines g. Other (Please specify): 36b. By whom are the external letter writers for the RP&T process selected from the pool? (Please select all that apply) a. By the candidate b. By the chair c. Through criteria written in the unit bylaws d. Through guidelines written in the MSU Faculty Handbook e. Through college guidelines f. On an individual basis with no formal guidelines g. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 164

179 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 22 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Input 37. Is any input expected from faculty mentors for the RP&T evaluation of mentees? Yes No << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

180 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 1 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 23 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Input (Cont'd) 37a. Which of the following are true regarding how input is collected from mentors? (Please select all that apply) i. Protocol for gathering input from mentors is addressed in the unit bylaws ii. Input is gathered informally from mentors iii. Mentors are asked to complete a standard form for faculty RP&T evaluation iv. Mentors are included on the RP&T committee v. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 166

181 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 1 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 24 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Input (Cont'd) 38. For faculty with joint appointments, how are units to which the faculty member is assigned involved in the RP&T process? a. All units are involved equally in the RP&T process. The performance expectations of each unit are considered and input from each unit is collected systematically. b. The tenure home unit is primarily responsible for faculty RP&T. The performance expectations of each unit are considered and input from each unit is collected. c. The tenure home unit is completely responsible for faculty RP&T evaluation, but the performance expectations of and input from each unit are considered. d. The tenure home unit is completely responsible for faculty RP&T evaluation, and the performance expectations of and input from each unit are not considered. e. Other (Please specify): 38a. How are performance expectations and input from multiple units shared in the RP&T evaluation of jointly appointed faculty? a. A formal joint review process which evaluates specific expectations for professional accomplishments b. An informal joint review process in which unit chairs evaluate general expectations c. Review only by tenure home unit d. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 167

182 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 1 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 25 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Feedback 39. Briefly describe the information the unit RP&T committee provides to the chair or school director. 40. Is there an opportunity to discuss the results or recommendations of the RP&T review process with the candidate? Yes No << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 168

183 MSU ADVANCE AR and RP&T Combined Inventory Page 1 of 1 9/23/2011 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 26 of 27 Reappointment, Promotion, and Tenure Reminder: Please note the current page won't be saved until you click either "Next" or "Save And Exit" button. You can use the "Save And Exit" button to save the responses and come back to finish the inventory at any time later. If you have any questions about using this online survey system, please contact Yue Li at or by at liy@muohio.edu. Feedback (Cont'd) 40a. If yes, which of the following are true: i. A meeting for each candidate is made by the department chair ii. Candidates are encouraged to request a meeting time to discuss review results or recommendations iii. Meetings with the candidate are only scheduled for challenges to the RP&T result iv. Other (Please specify): << Back Save And Exit Next >> Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE 169

184 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Page 27 of 27 Thank you for completing this inventory! If you want to share any additional relevant information, please provide your and phone number below. The Ohio's Evaluation and Assessment Center will contact you for a short follow-up interview. Your Your phone number: Please review the following response summary and make sure all the information is correct. You can print this page using the print option in your web browser. Please click the "Finish" button to submit the responses once your review is complete. Please note that you CANNOT make any changes to your responses or access this inventory after clicking the "Finish" button. << Back Save And Exit Finish Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

185 MSU ADVANCE AR and RP&T Combined Inventory 2011 Page 1 of 1 MSU ADAPP-ADVANCE: Annual Review and Reappointment, Promotion, and Tenure Combined Inventory Fall, 2011 Thank you for completing this inventory! Now you can safely close this page. If you have any questions or issues about this inventory, please contact Yue Li at , or by at liy@muohio.edu. Miami University, Oxford, OH Evaluation of MSU ADAPP ADVANCE /23/2011

186 Appendix B Kim Wilcox, Ph.D., Principal Investigator Estelle McGroarty, Ph.D., Lead Co-PI Melissa McDaniels, Ph.D., Project Director 524 South Kedzie Hall Michigan State University East Lansing, MI ANNUAL REVIEW INVENTORY When complete, please return to: Mary Jane Robb, 524 South Kedzie Hall, MSU, robbmj@msu.edu The following questions relate to the department s process for collecting information for faculty annual performance evaluation/review 1. Department: 2. Name: 3. College: 4. What period does the annual review cover each year? calendar year, academic year, other 5. Is there a form or common format that each faculty member uses to submit information for their faculty annual performance evaluation/review? If so, please attach a copy of this form or cut and paste text below. yes no 6. Is there a performance appraisal tool used during the annual performance evaluation process? yes no 7. Who determines what information is collected for the annual review? Check all that apply Dean Chair Departmental Faculty Subset of Departmental Faculty (for ex. Advisory committee) Administrator or faculty group from the other unit for jointly appointed faculty Other 1 Evaluation of MSU ADAPP ADVANCE 172

187 8. Who determines the how faculty productivity is measured? Check all that apply. Dean Chair Departmental Faculty Subset of Departmental Faculty (for ex. Advisory committee) Administrator or faculty group from the other unit for jointly appointed faculty Other 9. Are there unit-level goals and expectations for faculty productivity that are incorporated into the department s annual performance evaluation/review? yes no If you responded yes, please describe. 10. Are there college-level expectations for faculty productivity that are incorporated into the department s annual performance evaluation/review? yes no If you responded yes, please describe. 11. Are the expected levels of faculty productivity clearly stated in writing and available to the faculty? yes no If yes, how are new faculty made aware of the expectations (also paste in document, if available)? 12. Is there a departmental committee that collects and evaluates faculty effort and performance metrics? yes no If yes, how does the Chairperson work with this committee in the annual review process? 13. Is input sought from mentors in the annual performance evaluation/review? yes no If yes, what form does this input take (informal meeting with chair or evaluation committee, written report, etc.). 14. How is input sought from the other unit(s) for jointly appointed faculty? 2 Evaluation of MSU ADAPP ADVANCE 173

188 15. Are the results of performance evaluations and goals set from previous years used in the performance evaluation/review? Yes no 16. Does the Department provide written feedback to each faculty member summarizing the results of the annual review? yes no 17. Are expectations for future performance that are necessary for advancement included in the written feedback to junior faculty? yes no 18. Does the feedback provided on the annual performance evaluation/review refer to the results of prior annual reviews? yes no If yes to any of the above 3 questions, please paste in the text from a sample review removing all identifying information 19. Does the chairperson (or others) meet with pre-tenure faculty to discuss the results of their annual review? yes no 20. Does the chairperson (or others) meet with any other faculty group (including mid-career tenured faculty) to discuss the results of the annual review? Yes no If yes, what group(s)? 21. During the annual review meetings do the faculty members have input in setting future goals and expectations? yes no unit does not have annual review meetings 3 Evaluation of MSU ADAPP ADVANCE 174

189 22. Are the productivity measures used for faculty annual performance evaluation/review linked to the productivity measures used for determining merit raises? Yes no If yes, is this connection clearly communicated to the faculty? 23. Are the productivity measures used for faculty annual performance evaluation/review closely linked to the productivity measures used in promotion/tenure decisions for junior faculty? Yes no If yes, is this relationship clearly communicated to the faculty, and how? 24. Are the productivity measures used in your unit for annual review of faculty clearly communicated to the College Dean s Office? Yes No 25. Are the results of each faculty member s annual performance evaluation/review provided to the College Dean s Office? yes no If yes, in what way (check all that apply) summary statistics of groups of faculty oral discussion copies of written statement for each faculty other methods (describe): 26. How would you like to see the Annual Review process, policies and forms clarified or changed? Part 4. The following questions ask for your input on the possible adaptation and use of a Faculty Activity Information System developed in the College of Engineering for the Annual Review Process for your unit. To view this system, please go to: and login using : sample@fais.edu and the password: 900. This system allows faculty in Engineering to submit their effort report information during annual performance review. The system can import proposal information from CGA and publications from one biographical database (i.e. endnotes or bibtex). This system may be used to help construct an electronic human resource management data source as part of the Enterprise Business System. We seek your input as to whether this system, with modification, would be helpful in the collection of data for annual review of faculty in 4 Evaluation of MSU ADAPP ADVANCE 175

190 your unit. You may want to consult faculty in your unit about functionalities of the system that they would find valuable. 27. Does this system capture most of the information that your unit collects during the annual review of faculty? yes no 28. If you answered no to the prior question, what additional information does your unit collect for annual performance review? 29. What changes would you recommend to improve this system for annual review, in addition to adding the information listed in the prior question? Include format changes. 30. What additional functionalities related to faculty annual effort review would you like to see. 31. What additional functionalities would you like to see that may be valuable but are not necessarily connected to faculty annual effort review (e.g., generation of departmental reports, generation of faculty CV, etc)? 32. What information is in the system that you would not use? 33. Should any of the items be removed from the system? Yes No If yes, why? 34. Please provide any other comments about this system as a way of capturing information on faculty effort. 5 Evaluation of MSU ADAPP ADVANCE 176

191 Appendix C Kim Wilcox, Ph.D., Principal Investigator Estelle McGroarty, Ph.D., Lead Co-PI Melissa McDaniels, Ph.D., Project Director 524 South Kedzie Hall Michigan State University East Lansing, MI REAPPOINTMENT, PROMOTION & TENURE INVENTORY DEPARTMENT PROCESS MSU ADAPP/ADVANCE is focused on addressing processes used for faculty recruitment, advancement, and mentoring; the project is directed at developing approaches that increase the quality and inclusiveness of the faculty through processes that are transparent, objective, consistent, and aligned. At this initial stage, we are gathering baseline information about the reappointment, promotion and tenure process. This evaluation information will be used to develop materials that will assist faculty and administrators in future reappointment, promotion and tenure cycles, and to highlight best practices. The survey questions are in two sections: (1) PREPARATION FOR THE REAPPOINTMENT, TENURE AND PROMOTION PROCESS (2) REVIEW OF CANDIDATES AT DEPARTMENT LEVEL Thank you for your assistance with this fact-finding process. When complete, please return to: Mary Jane Robb, 524 South Kedzie Hall, MSU, robbmj@msu.edu 1. DEPARTMENT (please specify): 2. INVENTORY COMPLETED BY (name, title, date): (Can be completed by a group) 3. YOUR ROLE IN THE REAPPOINTMENT, PROMOTION, AND/OR TENURE PROCESS THIS YEAR: 4. Which reappointment, promotion and tenure committee ( ) are you referencing (e.g. which department within your college)? 1 Evaluation of MSU ADAPP ADVANCE 177