From Research into Students Learning Processes to Research- Informed Educational Development A Cross-Level Approach Keynote sponsored by the German Association for Academic Development (Deutsche Gesellschaft für Hochschuldidaktik, dghd) Dr. Antonia Scholkmann, Dipl. Psych. Universität Hamburg Faculty of Education Department of Lifelong Learning Seite 1
Overview I. Research into higher educa2on - a servant to too many masters? II. Research into higher educa2on learning and instruc2on on the micro-, meso- and macro-level III. Transfer of higher educa2on research results research integra2on into educa2onal development 2
I. Research into higher education a servant to too many masters? 3
Levels of higher education (cf. Flechsig 1973, Wildt 2002) VII Systemic connections Between higher education and other parts of the educational system and/or larger society VI Course-systems Connections between courses V Courses Combination of modules IV Modules Building bricks of the curriculum III Instructional arrangements Complex instructional scenarios II Learning situations Students actions, directed by the instructional design I Interactions Direct contact between teacher and student/ student and student (limited by time) 4
Levels of higher education (cf. Flechsig 1973, Wildt 2002) Disciplinarybound theories, epistemological beliefs and methodological approaches VII Systemic connections Between higher education and other parts of the educational system and/ or larger society VI Course-systems Connections between courses V Courses Combination of modules IV Modules Building bricks of the curriculum III Instructional arrangements Complex instructional scenarios II Learning situations Students actions, directed by the instructional design I Interactions Direct contact between teacher and student/student and student (limited by time) 5
II. Research into students learning processes 6
Tree types of research 2.1 Research grounded in discipline-specific theory and methodology 2.2 Research deriving from development questions 2.3 Research on the nature of academic work and educational development 7
Research grounded in discipline-specific theory and methodology at the micro-level Video-based research as example Long-standing tradition of video-based approaches in educational research before (e. g. Hugener et al., 2009; Seidel et al. 2011) Draws on the paradigm of behaviour observation Potential to provide valid information beyond questionnaires Allows for very fine-grained insights into learning processes 8
at Universität Hamburg Analysis of factors that influence the effective use of learning time in open, self-directed learning situations Effective use of learning time (time on task): time spent working actively, successfully and productively on a given assignment (Brodhagen & Gettinger, 2012: 33). Video-material from a day-long game-based assignment, 24 hours of video material 9
Effects of student parameters on use of the learning time (Scholkmann et al., forthcoming) General cognitive abilities Student Learning partner Mastery goal orientation ++ Performance goal orientation ++ Performance avoidance goal orientation Work avoidance goal orientation ++ Dependent variable: Effective use of the learning time (time on task) in open, collaborative learning situations; measured via video-analysis ++ ++ = p <.01 10
Research grounded in discipine-specific theory and methodology Descriptive research (Tsang 1997) Explanation of phenomena Discipline-bound theory and methodology However, practical implications for the development of teaching and learning at first seem hard to infer. 11
Voices from the field Would it be wise to test all students for their learning goal orientations before assigning them to teams working on open, self-directed tasks? And if so, is it wise to allocate students to specific teams instead of letting them choose independently whom they want to work with? 12
Research deriving from development questions The research project PBL Efficacy of Problem-based Learning as an example Funded by the German Ministry for Education and Science (BMBF) in a specific strand for higher education research Overall objective: to shed light on various aspects of how learning, instructing and its organisation should be developed further (cf. Berg et al. 2013) 13
Project PBL: Project outline Agenda: To provide knowledge and expertise with respect to whether Problem-based Learning (PBL) should be recommended as instructional approach in institutions of higher education on a broader basis, in particular in Germany. Several sub-studies, mostly comparative in nature (Samples in Sweden, the Netherlands, Switzerland, Germany). Mixed-methods approaches 14
Findings I: Development of deep understanding through PBL (Scholkmann, Loyens, Roters, Ricken, Koch, & Dahlgren 2014) Χ 2 = 3.91, df 4, p =.418 0 1 2 3 4 5 Level of understanding 15
Findings II: Competence acquisition through PBL (Scholkmann & Küng 2016) Knowledge processing Systema2c competence Communica2on competence Personal competence Proceure Comparison of n=259 students from the PBL-based primary-teacher program under survey with a reference sample from educa2onal science studies drawn from Braun & Hannover( 2011) Sta<s<cs t df d +++ 527 0.60 +++ 362 0.38 +++ p<.001 16
Research deriving from development questions Intervention research and activating research Expected feedback on effects of instructional interventions and organisational change Intended to activate teachers and administrators to adopt new concepts and approaches (cf. Metz-Göckel, Kamphans, & Scholkmann, 2012) Has worked out well for the PBL-project 17
However Lack of tight coupling to disciplinary-grounded bodies of research Questions coming from the development area are expected to be answered through field studies in the respective area Difficult research-designs Incomplete measures or analyses No cookbook-answers possible 18
As developers and decision makes we need evidence-based information; however, we need it to be hands-on, so that we can use it right away. Otherwise, we won t use it. Prof. Dr. Eberhard Becker, rector magnificus of TU Dortmund University 2002-2008 in Scholkmann, Roters, Ricken, & Höcker (2008) 19
VII Systemic connections Between higher education and other parts of the educational system and/or larger society VI Course-systems Connections between courses V Courses Combination of modules IV Modules Building bricks of the curriculum III Instructional Arrangements Complex instructional scenarios II Learning situations Students actions, directed by the instructional design I Interactions Direct contact between teacher and student/ student and student (limited by time) Research perspec<ve Development perspec<ve 20
Research on the nature of academic work and educational development Research that informs about the structures, processes, task, roles, culture etc. Academic/scientific work and educational development as two different types of systems within universities 21
Higher education research vs. development in contrast Research systems. Discipline-oriented (e.g. Pellert 1999) Success criteria in professional networks (not predominantly located at one s own institution) Collegial feedback as main source of information (Scholkmann 2011) Draws it workforce from discipline-specific backgrounds/ networks Development systems Topic-oriented success criteria within a specific institution Customer satisfaction as main source of information Draws its workforce from various disciplinary backgrounds 22
Higher education research vs. development in contrast Research systems as. Discipline-oriented (e.g. Pellert 1999) Success criteria in professional networks (not predominantly located at one s own institution) Collegial feedback as main source of information (Scholkmann 2011) Draws it workforce from discipline-specific backgrounds/ networks Development systems Topic-oriented Success criteria within a specific institution Customer satisfaction as main source of information Draws its workforce from various disciplinary backgrounds 23
Disciplinary backgrounds of educational developers in Germany (Stolz & Scholkmann, in print; Scholkmann & Stolz, in prep.) Educational Sciences Language and Cultural Studies STEM Social Sciences Psychology Law-, Management and Economics Medicine and Nursing Political Sciences 2,3 1,4 5,1 7,4 7,1 9,9 15,3 31 N = 301; all numbers in %, multiple choices possible 24
Educational developers job descriptions self perceived (Stolz & Scholkmann, in print; Scholkmann & Stolz, in prep.) Academic Development Educatinoal Development Consulting Facilitating Networking Activities Organisational Development Program Management Higher Education Research Individual Coaching Leadership of Learning and Teaching other 11,9 21,3 27 33,5 30,7 39,8 36,4 51,1 61,1 N = 301; all numbers in %, multiple choices possible 25
A systemic incompatibility? Research systems. Discipline-oriented (e.g. Pellert 1999) Success criteria in professional networks (not predominantly located at one s own institution) Collegial feedback as main source of information (Scholkmann 2011) Draws it workforce from discipline-specific backgrounds/ networks Development systems Topic-oriented Success criteria within a specific institution Customer satisfaction as main source of information Draws its workforce from various disciplinary backgrounds 26
III From research into students learning processes to researchinformed educational development 27
Two dimesions of research Research Information aspect means for provision of nontrivial answers to complex questions/challenges Development aspect Means for development of personal expertise and professionalism AWer Healey 2005, Reinmann 2016 28
Research integration into the development of teaching competence E.g. integration of research into teaching certificates in various forms Integration and discussion of current theoretical approaches and empirical evidence within TC-courses Schoarship of Teaching and Learning-arrproaches (e.g. Kordts- Freudinger et al. 2016) Buidling network-capacities between teacher-researchers and research-teachers 29
Research integration into the professionalization of educational development Developing the developers through. Integration of current theory and empirical evidence in professional development frameworks (e.g. German Association for Academic Development, in prep.) Scholarship of Academic Development (e.g. Scholkmann et al. 2013) 30
Research integration into the institutionalisation of higher education research and development Academic staff who can serve as interface between the two systems Knowing both systems structures and processes Being able to communicate their specifics and demands Supporting inter- and trans-disciplinary research 31
But what about the contribution of educational development to higher education research? 32
From educational development back to higher education reserach. Infusion of educational development questions into research through research-integrated practices Research on educational and academic development as a valid research-strand as its own (e. g. Stolz & Scholkmann, 2014; Urban & Meister, 2010). Educational development s capacity to integrate the various levels of higher education as an additional resource 33
Thank you! 34
References Brodhagen, E. M., & Gettinger, M. (2012). Academic learning time. In N. M. Seel (Ed.), Encyclopedia of the sciences of learning (p. S. 33 36). [S.l.]: Springer. Flechsig, K.-H. (1973). Die Entwicklung von Studiengängen (Vol. 6). Hamburg: Interdisziplinäres Zentrum für Hochschuldidaktik (IZHD) der Universität Hamburg. Hanft, A. (2000). Sind Hochschulen reform(un)fähig? Eine organisationstheoretische Analyse. In A. Hanft (Ed.), Hochschulen managen? (pp. 3 24). Neuwied and Kriftel: Luchterhand. Healey, M. (2005). Linking research and teaching exploring disciplinary spaces and the role of inquiry-based learning. In R. Barnett (Ed.), Reshaping the university: new relationships between research, scholarship and teaching (pp. 30 42). Maidenhead: McGraw- Hill/Open University Press. Kordts-Freudinger, R., Schulte, R., & Velibyoglu, N. (2016). Lehre und Forschung, Forschung und Lehre Lehrforschung! Vorwort zu: Lehr- und Lernpraxis im Fokus III Reflexions- und Forschungsbei- träge aus der Universität Paderborn. Die Hochschullehre, (2), 1 4. Metz-Göckel, S., Kamphans, M., & Scholkmann, A. (2012). Hochschuldidaktische Forschung zur Lehrqualität und Lernwirksamkeit. Zeitschrift für Erziehungswissenschaft, 15(2), 213 232. http://doi.org/10.1007/s11618-012-0274-z Pellert, A. (1999). Die Universität als Organisation: die Kunst, Experten zu managen. Wien: Böhlau. Reinmann, G. (2016). Forschungsorientierung in der akademischen Lehre. Impact Free, (1). Retrieved from http://gabi-reinmann.de/ Ricken, J., Roters, B., & Scholkmann, A. (2009). Projekt PBL: Wirksamkeit problembasierten Lernens als hochschuldidaktische Methode. Journal Hochschuldidaktik, 20(1), 7 10. Scholkmann, A. (2011). Deans in German Universities: Goal acceptance and task characteristics. Tertiary Education and Management, 17(4), 337 353. http://doi.org/10.1080/13583883.2011.601752 35
References (continued) Scholkmann, A., & Küng, M. (2016). Students Acquisition of Competences through Problem-based Learning. Reflecting Evaluation- Outcomes in the Mirror of Existing Empirical Evidence. Zeitschrift Für Evaluation, 15(1), 60 82. Scholkmann, A., Roters, B., Ricken, Judith, J., & Höcker, Marc. (2008). Hochschulforschung und Hochschulmanagement im Dialog: zur Praxisrelevanz empirischer Forschung über die Hochschule. Münster; New York; München; Berlin: Waxmann. Scholkmann, A., Salden, P., Kenneweg, A. C., & Van den Berk, I. (2013). Scholarship of Teaching and Learning und Scholarship of Academic Development zwei (un-)gleiche Schwestern? Konzepte für eine integrierte Sichtweise auf Selbst-Beforschung als Instrument zur Professionalisierung hochschuldidaktischen Handelns. Discussion paper presented at the dgdh Annual Meeting, Magdeburg. Scholkmann, A., Siemon, J., & Boom, K.-D. (conditionally accepted). Effective use of learning time during game-based learning. Influence of general cognitive abilities, goal orientation and characteristics of the learning partner. Seidel, T., Sturmer, K., Blomberg, G., Kobarg, M., & Schwindt, K. (2011). Teacher Learning from Analysis of Videotaped Classroom Situations: Does It Make a Difference Whether Teachers Observe Their Own Teaching or that of Others? Teaching and Teacher Education: An International Journal of Research and Studies, 27(2), 259 267. Stolz, K., & Scholkmann, A. (2014). Professionalization of Academic Development in Germany: Evidence from an Interactionist Theoretical Perspective. Paper presented at the European Conference on Educational Reserach (ECER), Porto, 1.-5. September 2014. Tsang, E. W. K. (1997). Organizational Learning and the Learning Organization: A Dichotomy Between Descriptive and Prescriptive Research. Human Relations, 50(1), 73 89. http://doi.org/10.1177/001872679705000104 Urban, D., & Meister, D. M. (2010). Strategies of Professionalization of Academic Development & Pathways of Academic Developers. Zeitschrift Für Hochschulentwicklung, 5(4), 104 123. Wildt, J. (2002). Ein hochschuldidaktischer Blick auf Lehren und Lernen. Eine kurze Einführung in die Hochschuldidaktik. In B. Berendt, A. Fleischmann, J. Wildt, N. Schaper, & B. Szczyrba (Eds.), Neues Handbuch Hochschulehre (p. Griffmarke A 1.1). 36