External Evaluation Report of Department of Educational Studies Purdue University

External Evaluation Report of Department of Educational Studies Purdue University Introduction The external evaluation team visited the Department of Educational Studies at Purdue University from May 6 th through 8 th 2009. The team consisted of Drs. Eric Anderman (The Ohio State University), Daniel Clay (Auburn University), Gary Crow (Indiana University), and Edward J. Kame enui (University of Oregon). The evaluation team was provided an itinerary and a self-study report prior to the visit. During the three-day visit, the team met with various constituent groups, including department, college, and university administrators, program area conveners, graduate studies committee members, graduate students, center and research project directors, and various faculty and staff. This report is based on information gained through a brief visit to the college and a review of limited documents including the self-study. Consequently, it is impossible to gather the depth and breadth of information necessary to fully inform the external evaluation team s review. Additionally, we did not meet with the new dean to gain a perspective on her priorities. Recommendations in this report are based on the limited information available through this process and should be considered in the context of the historical and cultural milieu of the department and college. We are willing to clarify or further discuss these recommendations with the new dean via a conference call, if desired. For purposes of its report to the Head of the Department of Educational Studies, the Interim Dean of the College of Education, and the Department of Educational Studies Faculty, the external evaluation team elected to use a variation of the SWOT (Strengths, Weaknesses, Opportunities and Threats) framework as its basic reporting format. As such, in the following sections we describe the perceived strengths and weaknesses of the Department. In addition, we conclude the document with a section on the perceived opportunities and recommendations. Strengths The team noted numerous strengths of the Department of Educational Studies. 1. There are many talented faculty who hold appointments in the department. Particularly noteworthy is the fact that faculty members are eager to be successful in areas that are of importance to the university s mission and strategic plan. There are many well established faculty with international reputations at Purdue. The numerous new assistant professors are particularly talented. Given the strong reputations of many of the senior faculty and the talent and motivation of many of the new faculty, the team views the present and immediate future as an excellent opportunity for the college to formally or informally refine, refocus and, perhaps 1

in selected areas, re-establish a culture of scholarship that is consistent with the college s and university s aspirations. 2. The provost clearly supports the department and the college. He made it clear to the team that he would like the college to focus on decreasing undergraduate enrollments, while simultaneously increasing enrollments in professional master s degree programs. He encouraged the department to determine how it will be distinctive, and how it will stand out from peer institutions. The provost seems very willing (if not eager) to support the department s efforts, as long as the department develops a clear vision and plan to enact positive changes that are aligned with the goal of seeking clear disciplinary and programmatic distinction among its peers. 3. The provost was positive about the department s contribution to both the college s and university s strategic mission with respect to both graduate student enrollment and scholarly productivity. He sees the department s potential for increasing both enrollment and scholarship in a manner that positions the college well within Purdue University. 4. The quality of graduate students and climate for graduate study are quite strong. The students that we met were all receiving ample funding and opportunities for research and travel to professional conferences. Students were particularly impressed with the mentoring and advising they received from faculty members. The quality of students appears to have risen in recent years. The team noted that we only met with a small group of students but based on our meeting, student climate and support seem to be excellent. 5. There is tremendous potential for collaboration with other units on campus. In particular, there appear to be several STEM-related projects, grants, and initiatives at Purdue and faculty in other units appear to be very willing to collaborate. In addition, there is an infrastructure in place for cross-disciplinary research (i.e., the Discovery Learning Center) which provides an excellent platform for these projects. 6. High demand, quality graduate programs exist in the department. Some programs have many qualified applicants at the doctoral level that exceed a program s ability to accommodate them. These programs also are instrumental in the department s reputation and, therefore, positively impact the US News and World Report rankings both directly and indirectly. 7. The college and department are housed in a relatively new state-of-the art building which is centrally located on campus. The high quality of the facility contributes to the ability to attract and retain highly qualified faculty and students. Weaknesses The team noted several weaknesses. 1. Several faculty noted that annual evaluations in the department do not provide specific feedback regarding progress toward tenure and/or promotion. It was suggested that evaluations should be more in line with the promotion process, that evaluative criteria need to be clearer, and that specific feedback be provided 2

regarding whether the faculty member is making adequate progress toward tenure in each of the areas of discovery, learning and engagement. However, there may be specific guidelines for faculty evaluations provided by the provost s office which we did not see. Faculty members perceive a similar need for clarity in the merit evaluation process. 2. While many faculty are engaged in funded research, faculty reported the perception that there is little structure/support for collaborative research. For example, faculty indicated that there are no research colloquia, and that many faculty within the department and the college are unaware of their colleagues current research and scholarship. Other than a newsletter, there appears to be little communication about research. 3. There is much concern about support for research within the department and the college, including, for example, a reliable infrastructure and process for identifying external funding sources, developing grant proposals, submitting grants, administering grants, and managing grants at pre- and post-award. Faculty noted that current college-level administrators have had little experience with large-scale federally-funded grants, and as such, there is concern that the administrators do not understand the financial and substantive obligations of large-scale grant development and administration. For example, in the external evaluation team s own data gathering, there appeared to be a lack of understanding of how indirect cost returns are shared among colleges and departments when faculty members from several units on campus collaborate on a grant proposal. In addition, of particular concern is the lack of a transparent, consistent, or clear policy for buying out or gaining release from assigned courses. New faculty members in particular feel that there is little help with acquiring external funds. Moreover, faculty note that grant writing is encouraged, but not recognized in a manner consistent with its apparent importance to the college. 4. A clear vision for the direction of the college, consistent with the provost s priorities, should be articulated and communicated to all faculty and staff as soon as practical in order to support and guide the department s vision, development and implementation of its program priorities and initiatives. 5. In the Educational Leadership program, there appear to be too many students and too few faculty. Some of the faculty reported advising loads that were much too large, especially at the doctoral level. 6. Statistics courses are not offered within the department (even though there is a section that specializes in this area), which means that students may be learning some of their basic statistics in a decontextualized manner; that is, independent of the content and real world context of the problems of educational practice. Several students and faculty identified this as a concern. In general, there is a perception that students do not have a clear, high-quality and systematic program of study in research design and statistics. One consequence of this is that students may not be prepared to provide assistance on the faculty s programs of research. Not surprisingly, this concern varies widely by faculty advisor and program area. 7. To effectively invest in opportunities for growth and improvement in strategic areas, the department will need additional revenues that must come through 3

reallocation of current resources, or the development of new revenue streams. To date, an entrepreneurial culture appears to be lacking to pursue new resources and to invest in ripe and appropriate opportunities. Creative and entrepreneurial approaches are critical in this financial climate and entirely consistent with innovative approaches that peer and aspirant colleges of education are currently using. The department has not yet taken advantage of a number of opportunities to generate revenue (e.g., on-line/distance education courses, the counseling clinic, and collaborations with other units on campus). 8. Faculty perceptions of work load, available resources and acceptable levels of productivity may not be consistent with those of peer departments. It appears as though faculty are working hard, but in some cases the productivity levels in the areas of scholarship (e.g., peer-reviewed publications) and external funding are below the norm for comparable departments at peer institutions. Arguably, with a 2-2 semester teaching load, the publication and external funding productivity should be significantly higher. 9. A strong collaborative culture does not appear to exist in the department. Faculty members seem to identify with program sub-specialties which reinforce narrow disciplinary expertise and practice, but hinder broader interdisciplinary discovery, learning, and engagement. Opportunities/Recommendations 1. Given the conspicuous importance of increasing external grant funding to the college and university, we recommend that the new Dean examine the issue of research support. Based on consistent reports from multiple sources, there appears to be a lack of infrastructure and culture that supports funded research. The research stature of the department and college on campus needs to be improved through networking and organizing strategic collaborative research efforts with other research groups on campus. 2. There is a critical need to develop policies and procedures to establish the processes and guidelines required to support external funding including financial rewards for success. These policies include those related to course buy-outs and work-load reassignments upon receipt of a successful grant, post-award support for faculty including compliance and budget monitoring, and completion of annual reports to funding agencies. In order to support department external funding initiatives, we recommend that the associate dean for research establish a faculty advisory committee to help establish these policies through a shared governance model to ensure faculty ownership. 3. In addition, the need for a full-time grant support specialist appears imperative in providing the nuts-and-bolts assistance that many faculty need to be successful with the full-range of grant development, acquisition, implementation, and maintenance. This person can tailor services to faculty interests and needs including identification of funding sources, interpretation of RFPs, budget development, collection of supporting documents/letters, routing of proposals for approval, responsiveness to RFP/RFA requirements, creation of boiler plate college and university level information necessary for all grant proposals, creation 4

of a library of funded proposals to serve as models, etc. The provost appeared particularly interested in this idea and expressed a willingness to negotiate shared support for this new position with the new dean. 4. In light of the previous recommendations regarding the development of a culture and infrastructure for external grant support, the department should align its research efforts with the mission of the college. 5. Faculty need to be better informed about each other s research areas/interests and scholarship. The department (and perhaps college) should initiate a more purposeful, systematic effort to promote interdisciplinary collaborations for largescale, multi-year grants. 6. The structure of the department should be re-examined, along with a reexamination of the administrative and staff structure of the college to eliminate redundancy. Although we were not tasked with a college review and were, therefore, unable to thoroughly assess the college administrative structure, it appeared to us that there was an unusually high number of support staff that may suggest some administrative or clerical redundancies in offices. Perhaps there could be structural changes to increase the efficiency such as the merging of the student services and graduate services offices. 7. We encourage department faculty to identify marquee programs. The best potential for the department is to identify distinctive programs that are unique to Purdue and consistent with the mission and vision of both the university and the college. To accomplish this, some programs may need to be reduced in size or discontinued altogether to invest in those signature programs with the most potential. The new dean will need to work with the faculty and staff to decide upon criteria to identify these programs. This will be a difficult conversation to have, but clearly an important one in order to acquire additional support from the provost s office. 8. To reach its full potential, the department must gain new resources to invest in faculty, programs and opportunities with the most potential for excellence. Adequate resources to accomplish these goals will only be possible if new revenue streams for the department are realized. We see many opportunities for auxiliary revenue including: (a) development of online courses and programs, (b) partnering with the private sector to provide needed services or consultation, and (c) engaging in entrepreneurial activities consistent with the academic mission of the department and college. For example, the counseling program could partner with Caterpillar Corporation to provide career counseling services for employees who have recently lost their jobs. Contractual arrangements like these provide significant revenue, opportunities for students to get real-life professional experiences that make them more competitive in the workplace upon graduation, and establish potentially longstanding partnerships with industry and statewide stakeholders consistent with Purdue s land grant mission. These relationships often parlay into development opportunities for an endowed professorship or other means of supporting the department. One critical factor, however, is that faculty must understand the need for a new model of academic functioning that does not rely on appropriated money to support all their activities. 5

9. We strongly recommend considering development of on-line master s degree programs where appropriate, as there is a considerable market for quality graduate degrees from institutions like Purdue. These programs can address critical needs within the state (e.g., K-12 principal shortages), grow the graduate enrollments consistent with the university s priorities, and provide significant revenues to support academic programs and scholarship in the department. In our site visit, we sensed some apprehension from faculty about these kinds of programs, but research has clearly shown that high-quality on-line programs have equal or better outcomes than on-campus programs when implemented correctly. Other institutions have managed to influence faculty culture and norms to support these endeavors. The significant number of new faculty in the department, and those who will be hired in the next 5 years, may be more open to alternative forms of program delivery. 10. Engagement and scholarship should be conceptualized as part of the same scholarly enterprise. Especially in education, scholarship of engagement and research are often mutually informing and intertwined. To provide a clearer message and increase efficiency of administrative structure, the assistant dean for engagement should report to the associate dean for research. For example, PELISARS is an example of an interdisciplinary effort that is arguably representative of both engagement and research scholarship that has the potential to be a distinctive program for Purdue, with the Department of Educational Studies taking a leadership role. 11. A more rigorous and transparent system should be developed to assess graduate student and faculty success and productivity. We encourage Department faculty and administrators to identify 3 or 4 measureable outcomes in each of the three areas (learning, discovery, and engagement), set benchmarks against peer and aspirant institutions, ensure a transparent process for evaluating and monitoring progress toward these goals, and establish a merit-based structure that rewards individual faculty members and programs that contribute most based on those markers. 12. We encourage consideration of workload reallocations among faculty members to maximize faculty contributions to strategic goals and outcomes noted previously. Many peer institutions are assigning higher teaching loads (e.g., 3-3 or even 4-4) for tenured faculty who have not been productive in research or engagement. This allows those faculty members to earn merit based on exceptional instruction while freeing up faculty with research productivity or potential to increase their productivity in that area. 13. The department and college should employ a data-driven approach to examining productivity in all areas (discovery, engagement, and learning) to determine where inputs (fiscal resources, space, staff) have their greatest impact on desired outcomes. A copy of such an analysis conducted at Auburn University was provided to the Interim Dean. This examination provides faculty, department (and other unit) productivity and cost profiles that can inform administrative decisionmaking. It is likely that some programs that have been in place for a long time are using resources with little contribution to the core mission and desirable outcomes 6

of the department. These degree programs, offices or units should be significantly restructured or discontinued altogether. 14. The structure, function and use of the in-house counseling clinic should be reexamined. This is a state-of-the-art training and service provision facility that is significantly underutilized. The clinic has very limited hours yet there is a waiting list of over 50 people (according to faculty reports). Faculty with limited scholarly productivity are spending 25+ hours supervising in this clinic, which is not an efficient use of resources. The clinic could also serve as a cost center in which revenue can be generated through service provision (sliding scale fee, billing health insurance) or contracts with outside agencies or organizations (see Caterpillar example above). 15. Due to the inappropriately large number of doctoral advisees in the Educational Leadership program, we encourage the program faculty and department and college administrators to consider either capping admissions for a period of time or moving to an every-other-year admission timeline for this program. 16. In order to address issues regarding the need to develop and support a collaborative culture within the department, we encourage faculty to develop various culture-building strategies, including cross-disciplinary department colloquia, social gatherings, and other cross-program interactions suggested previously in this report. 7