Student research and quantitative training in undergraduate ecology courses: results from a systematic survey of U.S. institutions

Transcription

1 Student research and quantitative training in undergraduate ecology courses: results from a systematic survey of U.S. institutions David M. Marsh, Wendy Gram, and Teresa Mourad ABSTRACT Undergraduate ecology courses provide the foundation for the next generation of professional ecologists. We carried out a systematic survey of undergraduate ecology courses in the United States to examine the extent to which students are exposed to real research, interdisciplinary approaches, and quantitative methods. Overall, we found high rates of student involvement in ecological research and a relatively high degree of exposure to quantitative methods. However, we also found large disparities among different types of institutions student research and quantitative training were very common in ecology courses at public and private undergraduate colleges but were much less common at PhDgranting institutions. Large class sizes and lack of a lab period at PhD granting institutions were substantial challenges for involving students in research. However, decreased emphasis on quantitative methods and active learning approaches at PhD institutions could not be directly ascribed to these limitations. We suggest that research universities need to work creatively with their faculty to infuse their institutional research expertise into their undergraduate science classes. INTRODUCTION For two decades, scientists and educators have stressed the need to reform undergraduate science education. The overarching goal of reform is to move from a system where students memorize collections of facts about the world to an approach where students become active partners in learning about science and generating new knowledge. Influential reports such as HHMI s BIO2010 (CUBE 2003) and AAAS/NSF s Vision and Change in Undergraduate Biology Education (Brewer and Smith 2010) converged on a basic set of recommendations with respect to biology education. These recommendations include: 1) infusing real research into courses so that students can experience the scientific process, 2) promoting interdisciplinary approaches, so that students can see the connections between different branches of science, and 3) exposing students to the quantitative and computational skills that are needed by the next generation of scientists. Most ecologists would probably agree that these are worthwhile goals, and many may feel they are already being achieved at their own institutions. But measuring the progress of ecology education as a whole requires some formal assessment of what is happening in ecology courses. For instance, to what extent are ecologists already incorporating real research into their undergraduate courses, or promoting interdisciplinary approaches, or developing students quantitative and computational skills? To answer these questions, we carried out a systematic survey of introductory/general ecology courses at four year colleges and universities in the United States. We focused on introductory/general ecology courses because these courses are offered at most institutions and provide the first major exposure to ecology for most students. Whether or not these courses succeed in attracting and developing the best students will have a major influence on future advances in ecological science.

2 Survey questions were designed to focus on several aspects of ecology courses. First, we asked about student exposure to real research, either formally or through exercises with experimental design or the primary literature. Second, we asked about course content with respect to different subfields of ecology and connections between ecology and other disciplines. Third, we asked about the quantitative and computational content of ecology courses, including coverage of statistics, simulation modeling, and mathematical/analytical models. Fourth, we asked about the primary challenges for instructors, both in general and with respect to the incorporation of research and quantitative methods. The full set of questions, as well as summary responses, is contained in the Supplementary Materials. We examined overall patterns in responses, and we compared ecology courses based on the type of institution (e.g. undergraduate/ masters granting/ PhD granting), the size of the class, the background of the instructor (i.e. field of specialization) and the year of his/her PhD. These comparisons allowed us to examine the factors that affect the content, methods, and challenges of ecology instruction. The last comparable survey by the Ecological Society of America was carried out in 1996 (Brewer 1998). By comparing our results to those of this previous survey, we asked whether significant aspects of ecology instruction have changed over the past 15 years. METHODS Distribution To identify survey participants, we randomly selected four year colleges and universities from a list of accredited schools maintained by the University of Florida ( For each school, we attempted to use materials on the school s official website (e.g., course registration information or a department website) to find the current instructor for an introductory/ general ecology course. Where we were unable to identify a relevant course or its instructor, we skipped that institution and went to the next one on the list. When multiple instructors were listed for ecology, we chose the first one named. We excluded summer classes, online classes, and classes that were clearly not intended for science majors. From an initial selection of 830 institutions, we identified relevant ecology courses and their instructors at 498 institutions. We then e mailed each of the instructors to explain the purpose of the project and to invite them to fill out the anonymous web survey through SurveyMonkey ( Several instructors replied to let us know that they did not currently teach ecology, but provided us with the contact information for the current instructor. In these cases, we followed up with the instructor to whom we d been referred. In 21 cases, invitation e mails bounced and we were unable to find an alternative contact. Thus, our response rate calculations are based on 477 potential respondents. These response rates can be considered low end estimates, as some e mails likely went to instructors who were no longer at the institution or who did not actually teach ecology. Our survey and invitation letter were approved by the Office of Human Research at the University of California, Santa Barbara (protocol ). Analysis Given the number of possible ways to stratify the survey results, we decided on a few a priori sets of comparisons that were of primary interest. First, we stratified results by class size, teaching load, and

3 institution type. We then carried out a series of univariate analyses to determine whether teaching strategies, materials, and challenges differed across these three variables. Analyses use standard techniques: Chi square tests for binary response variables (i.e. yes/no) and Kruskal Wallis tests for ordinal response variables such as Likert scales (e.g. relative importance or emphasis). Class size, teaching load, and institution type were often associated since PhD granting institutions tended to have large classes and low teaching loads whereas undergraduate colleges had small classes and high teaching loads. Thus, where responses differed across these variables, we carried out followup multivariate analyses (general linear models) to determine whether one or another variable better explained the differences. In some cases, the high covariance of these three variables rendered this analysis uninformative. However, in other cases, differences were better explained by one variable than another. Our second set of stratified analyses were based on the year or decade of the instructor s PhD. Although these analyses are based on a cross section of responses, they potentially bear on the question of how ecology instruction is changing over time. For this analysis, we examined responses related to course content as well as teaching strategies, materials, and challenges. Because year of PhD could be treated as a continuous variable, associations between PhD year and binary responses were examined with logistic regression. Our third stratified analysis was based on the academic specialty of the instructor. At smaller schools, ecology is sometimes taught by instructors who are not themselves ecologists. We sought to determine how often this is the case and how ecology courses differ when taught by non ecologists. We compared courses taught by ecologists to those taught by non ecologists in terms of course content, materials, and teaching strategies. Finally, we carried out an analysis of how prerequisites to ecology affected instructors perception that student preparation was a challenge for teaching effectively, incorporating research, or incorporating quantitative methods. We used spearman s correlation to examine the relationship between the total number of prerequisites for an ecology course and extent to which student preparation was cited as a challenge. We also examined the effects of specific prerequisites including general biology, general chemistry, and statistics or calculus. Type I errors are potentially a concern with our analysis since many of the responses are analyzed question by question and then stratified across multiple variables. Ultimately we are interested in general patterns in the results rather than any one specific comparison. Thus, we reduced the risk of Type I errors by only reporting results that: 1) occurred in clusters of statistical significance, 2) had some logical a priori basis and 3) had levels of explanatory variables that corresponded with levels of the response variable. Based on these restrictions, a handful of isolated findings of statistical significance are not presented, with little overall impact on the major patterns in the results. RESULTS

4 We received 232 surveys from ecology instructors for a 45 50% response rate. Response rate was marginally different across institution types (χ 2 =7.8, df=4, p=0.09). The highest response rate came from PhD granting institutions (62%) and lowest response rates came from private (43%) and public undergraduate institutions (46%). Response rates from private and public masters granting institutions were intermediate (45% and 51% respectively). Ecology courses ranged from 10 to 18 weeks with 15 weeks as the median and the mode. Eighty two percent of the courses included lab periods. Of those not incorporating lab periods, 22% had a discussion section. Course structure was highly variable among institution types. At private undergraduate and masters granting institutions, 94% of classes included a lab, whereas only 47% of classes at PhD institutions had a lab. At public undergraduate and masters colleges, labs were attached to 87% and 82% of the courses respectively. Overall, ecology courses tended to focus most heavily on population and community ecology (3.4 and 3.3 weeks, respectively), followed by ecosystem ecology (2.6 weeks), behavioral or evolutionary ecology (1.9 weeks), and conservation ecology (1.6 weeks). Instructors frequently emphasized connections between ecology and global environmental issues (2.83 ± 0.07 SE on a 0 to 4 Likert scale) and gave moderate emphasis to connections with local environmental issues (2.34 ± 0.07 SE). Connections between ecology and other disciplines and between ecology and human health or disease were less emphasized (1.60 ± 0.07 SE and 1.58 ± 0.07 SE respectively). The most emphasized quantitative skill was interpretation of data and graphs (3.24 ± 0.06 SE on a 0 to 4 Likert scale), followed by statistics for data analysis (2.22 ± 0.09), and mathematical models (1.99 ± 0.08). Classes incorporated GIS/mapping (0.49 ± 0.06) or ecoinformatics (0.67 ± 0.06) only infrequently. With respect to class activities, high percentages of the classes analyzed data they had collected (86%), read papers from the primary literature (85%), and worked on group research projects designed by their instructors (70%). Comparatively low percentages analyzed data from ecological databases (20%) or worked with stakeholders to provide recommendations on an applied ecological issue (12%). The two most frequently reported challenges were heavy teaching load (2.32 ± 0.08) and preparation/interest level of the students (2.30 ± 0.07). Large class size was listed less frequently as a challenge overall (1.33 ± 0.09), but was heavily weighted by a percentage of the instructors who listed it (i.e. rated 3 or 4 by 22% of respondents). Limited course budgets (1.38 ± 0.09) and logistical difficulties (1.54 ± 0.09) were also regularly cited as challenges to teaching effectively. Lack of a good text or other teaching materials was cited only infrequently (0.80 ± 0.06). With respect to incorporating research into classes, results were similar except that limited course budgets were cited as a limitation almost as often as heavy teaching load and student preparation (1.47 ± 0.09 for limited course budgets versus 1.64 ± 0.08 for both teaching load and student preparation). With respect to focus on quantitative techniques, student preparation/interest level was by far the most heavily cited challenge (2.19 ± 0.08), followed by heavy teaching load (1.66 ± 0.08). Other factors were cited less frequently (mean score < 1.2 for all others). Class size, teaching load, and school type

5 Course content, materials, and teaching strategies frequently varied with class size. Incorporation of GIS/Mapping (K = 13.32, df = 3, p = 0.004), use of statistics for data analysis (K = 11.57, df = 3, p = 0.009), and use of ecological databases (K = 12.02, df = 3, p = 0.007) were all more likely in smaller classes. In addition, smaller classes were more likely to read papers for the primary literature (χ 2 = 10.24, df = 3, p = 0.02) work on group research projects (χ 2 = 35.27, df = 3, p < 0.001), work on experiments designed by students (χ 2 = 25.53, df = 3, p < 0.001), and work with stakeholders on an applied issue (χ 2 =6.48, df = 2, p < 0.04). Interestingly, case studies, which are potentially well suited for large classes, were also more likely to be used in smaller classes (χ 2 = 8.21, df = 3, p =0.04). Reported challenges and limitations also depended on class size. Instructors of larger classes were more likely to report that student preparation/interest level (χ 2 = 8.55, df = 3, p =0.04) and limited course budgets (χ 2 = 8.64, df = 3, p =0.04) were challenges for teaching ecology effectively. Not surprisingly, instructors of larger classes were much more likely to list large class size as an important challenge (χ 2 = , df = 3, p < 0.001). With respect to incorporating research into their classes, instructors of larger classes were more likely to report student preparation/interest (χ 2 = 8.62, df = 3, p =0.04), limited course budgets (χ 2 = 24.83, df = 3, p < 0.001), lack of a lab period (χ 2 = 43.96, df = 3, p < 0.001), and large class sizes (χ 2 = , df = 3, p < 0.001) as limitations. This same set of limitations with respect to the use of quantitative approaches was also reported significantly more often by instructors of large classes (p < 0.05 in all cases). Teaching load was inversely related to class size and institution type: large classes were generally taught by instructors at PhD granting institutions who taught at most one additional course (χ 2 = 36.69, df = 6, p < 0.001). That being the case, teaching load was consistently less related to teaching strategies and challenges than was class size. Of the significant differences across class sizes, only use of databases (K = 6.65, df=2 p = 0.04), analyzing data collected by students (χ 2 = 13.09, df = 2, p = 0.01), and working with stakeholders on applied issues (χ 2 = 8.06, df = 2, p = 0.02), were significantly more common for instructors with heavier teaching loads and smaller classes. Multivariate analyses (general linear models) ascribed most of the variation in analyzing student collected data to class size (p = 0.004) rather than teaching load (p = 0.39), whereas variation was not sufficient to partition use of databases or working with stakeholders to one of the two factors. School type was more closely associated with class size than was teaching load (Table 1). PhD granting institutions reported that only 4% of classes had fewer than twenty students and 33% had more than 100. At the other end of the spectrum, private undergraduate institutions reported that 68% of their classes had less than 20 students and none had over 100 students. As a result, findings for school type closely paralleled findings for class size. Incorporation of GIS/Mapping (K = 11.29, df=4, p = 0.02) and statistics for data analysis (K = 13.26, df=4 p = 0.01) were more common at undergraduate institutions than at masters and PhD granting schools. Work on group research projects designed either by instructors (χ 2 = 39.48, df = 4, p < 0.001) or by students (χ 2 = 28.66, df = 4, p < 0.001), analysis of data collected by students (χ 2 = 45.21, df = 4, p < 0.001), and working with stakeholders on applied issues (χ 2 = 12.68, df = 4, p < 0.01) were activities common at undergraduate institutions but much less so at PhDgranting schools. Use of case studies was more common at private undergraduate schools than at other school types (χ 2 = 10.29, df = 4, p = 0.04).

6 Reported challenges based on school type also generally paralleled the results for class size. Private undergraduate institutions were less likely to cite student preparation as a challenge (K = 18.26, df=4, p = 0.001) but were more likely to name teaching load as a major challenge (K = 32.34, df=4, p < 0.001). Instructors at PhD granting institutions were much more likely to list large class size (K = 69.51, df=4, p < 0.001), limited course budgets (K = 11.39, df=4, p = 0.02), and lack of a lab period (K = 29.81, df=4, p < 0.001) as major challenges. Public masters granting institutions tended to cite similar challenges as the PhD granting schools. Multivariate analyses that incorporated both school type and class size were generally unable to disentangle the effects of the two variables for responses related to course content, materials, and teaching strategies. However, reported challenges could frequently be ascribed to one of the two factors. Challenges related to student preparation/interest were related to school type (p = 0.008) but not class size (p = 0.57), as was teaching load (p = for school type versus p= 0.69 for class size) and logistical difficulties (p = 0.04 for school type versus 0.44 for class size). Challenges due to the lack of a lab period were more closely associated with class size (p = 0.006) than with school type (p = 0.36). Limited course budgets as a challenge could not be clearly ascribed to either school type or to class size. Year of PhD and teaching experience We found very few effects of PhD year (or decade) on course content, materials, or teaching strategies. We found a small, though significant, tendency for more recent PhDs to devote less course time to ecosystem ecology (r = 0.17, p=0.01). We also found a small positive relationship between PhD year and likelihood of incorporating group research projects (β = 0.03, p = 0.04). All other aspects of content, materials, and strategies were similar across PhD decades, suggesting that more recent PhDs are not necessarily using different approaches than are older PhDs. The number of times an instructor had previously taught ecology was associated with PhD year (U = 8318, p < 0.001). Less experienced instructors (i.e., more recent PhDs) were more likely to emphasize human health (U = 4096, p = 0.04) and to use case studies (χ 2 = 4.38, df = 1, p = 0.04) and software created by others (χ 2 = 4.40, df = 1, p = 0.04). Reported challenges and limitations were similar across experience categories, suggesting that more experience does not necessarily resolve challenges with respect to issues like large class sizes or limited course budgets. Prerequisites and student preparation Student preparation/interest level was, along with heavy teaching load, the most commonly reported challenge. However, the overall number of prerequisites for ecology was not significantly related to the extent to which student preparation was reported to be challenge (r = 0.05, p > 0.50). When we examined individual prerequisites, neither introductory biology nor introductory chemistry were associated with how often student preparation was cited as a problem (U = 2472, p =0.37 for biology; U = 5541, p = 0.55 for chemistry). However, when statistics or calculus was a prerequisite, instructors were significantly less likely to site student preparation as a concern (U = 3514, p = 0.03). Interestingly, familiarity with statistics and calculus seemed to lead to even more focus on these areas within ecology courses. Instructors were much more likely to stress statistics for data analysis when statistics/calculus

7 was a prerequisite (U=2617, p = 0.006). Although this may seem obvious in retrospect, it s possible that instructors would make up for a lack of statistical background in their students by spending more, rather than less, time with the subject. As noted above, student preparation was less likely to be cited as a challenge at private undergraduate institutions. However, this result does not confound our findings for the statistics/calculus prerequisite as this course requirement was largely independent of school type (χ 2 = 1.176, df=4, p=0.78). DISCUSSION Several aspects of our results appear encouraging with respect to exposing undergraduate ecology students to authentic research. Very high percentages of classes reported collecting and analyzing data (86%) and reading papers from the primary literature (85%). Classes frequently worked of research projects that were designed by instructors (70%) or designed by students (64%). Although we have not found comparable data in other fields, we expect that these numbers may well be among the highest across disciplines. Ecologists have distinct advantages in involving students in research because ecological research projects can be carried out at local field sites with limited equipment and material costs. For those who were unable to involve students in research, no single challenge was cited by the majority of respondents. Rather, challenges were typically class specific instructors with particularly large classes, or no lab periods, or who were located in urban environments felt that these were the predominant limitations. Student interest and preparation was also frequently cited as a challenge. This latter result is somewhat surprising, as many ecological research methods require very little training. Emphasis on quantitative and computational techniques in ecology courses was also relatively common. Almost all classes emphasized reading graphs and interpreting data, and substantial numbers also emphasized statistics for data analysis and mathematical/analytical models. Use of simulation models was less common, and use of GIS or ecological databases was rare. The major challenge cited for incorporation of quantitative techniques was lack of student preparation. Our analysis of course prerequisites found that when calculus or statistics was required for ecology, this challenge was cited substantially less often. Although these results are generally encouraging, there were vast discrepancies in results across institution types. At PhD granting institutions, only 47% of classes even had a lab period, whereas at private undergraduate colleges, a lab period was nearly universal. PhD granting institutions were also faced with much larger class sizes. Classes were almost always larger than 50 students, and were frequently larger than 100. In contrast, at undergraduate colleges, less than 20 students was by far the most common class size. Perhaps, then, it s not surprising that PhD granting institutions are far less likely to involve students in research. However, PhD institutions were also less likely to emphasize a range of quantitative methods, from statistical analysis to simulation modeling, that are not obviously class size limited. Even case studies, which have been specifically promoted as an active learning approach for large classes (Haak et al. 2011), were much more likely to be used in small classes at private institutions. For all of these results, masters granting universities were intermediate between

8 undergraduate colleges and PhD institutions in terms of student involvement with research, quantitative methods, and active learning. Our results suggest that large class sizes and limited budgets are important challenges at PhD institutions. However, this may not be the entire story. Anderson et al. (2011) recently argued that the incentive system at research institutions simply stresses effective teaching too little in comparison to publishing and grant seeking. Although we did not ask about the academic position of instructors, we did note that at many high profile research universities, undergraduate ecology was not being taught by a permanent faculty member. This is not to say that these instructors are necessarily ineffective, but it may be indicative of the lower value placed on undergraduate teaching at these institutions. Our response rate (45 50%) was high for an online survey, but was substantially less than 100 percent. Given this, we would expect our results to include at least some measure of response bias. In particular, we suspect that instructors who are highly invested their ecology courses were probably more likely to respond to the survey. For example, the percent of classes reporting that they collect and analyze data (86%) and read primary literature (85%) were quite high; these findings should probably be taken as upper limits rather than accurate estimates. Nevertheless, response rates were generally similar across institution types, with the highest response rates from PhD institutions. Thus, we don t have any reason to believe that the substantial differences observed across institution types were biased by response rates. We lack the necessary data to analyze response rates across other stratification variables, but in general, we feel that these relative comparisons are more reliable than are the overall response rates. Although our survey was not identical to the ESA course survey carried out in 1996, some items do allow for direct comparisons. The previous survey asked about guided investigations which is similar to our designation of group research projects designed by the instructor. Here, their reporting that 69% did this sometimes or often is nearly identical to the 70% that claimed to do group research in our survey. The previous survey also asked about use of computer models, which 69% of respondents claimed to use sometimes or often. Once again, in our survey, a nearly identical percentage (68%) of respondents claimed to use simulation models. And as with our survey, the previous survey reported discrepancies between graduate and undergraduate institutions, though the magnitude of these differences was not presented. These results, along with the general lack of association between course content and the year of an instructor s PhD, suggest that general aspects of ecology courses may have changed little in the past 15 years. Some might consider this stability to be a positive thing, but it also suggests that there has not been much of a broad scale change in the content or methods of ecology courses over recent years. Our results lead to several recommendations with respect to ecology instruction. 1) Graduate institutions appear to be lagging substantially behind undergraduate colleges in exposing students to authentic research in ecology. This is unfortunate, as in a sense, research universities should be ideally positioned to infuse research into their curricula. Class size and lack of lab periods are obvious limitations here. But research universities need to be more creative in finding ways to involve undergraduate classes in their research efforts and in supportive faculty who try to do so. Indeed, a

9 number of resources exist for faculty at research universities who wish to incorporate more active learning into their classes (citations). 2) Instructors seemed generally pleased with the range of textbooks, software, and teaching aids available. They expressed much greater interest in workshops aimed at updating their own skill sets than in the development of any new course materials. 3) Student preparation was cited as a challenge in a variety of categories. At least with respect to quantitative content, responses indicate that requiring calculus and/or statistics can reduce perceived problems with student preparation. It can also allow instructors to place more emphasis on data analysis in their ecology courses, thereby better preparing them for graduate study.

10

11