Design Research Michiel Doorman Freudenthal Institute Utrecht University m.doorman@uu.nl 2015-8-18
Short introduction Research experience & questions Research question(s) for tomorrow 2
These sessions Reflect on the program Discuss design research Create a poster This week Design research (today) Research questions (tomorrow) Create topic study groups (two?) Next week Your research (Monday) Poster design (Wednesday) Poster presentations on Thursday 3
Reflection on the program Think - pair - share 4
Today: Design research (DR) Why? What? How? Exemplary case: Tool Use in an Innovative Learning Arrangement for Mathematics Doorman et al. 2013 http://www.fisme.science.uu.nl/tooluse/en/ http://www.fi.uu.nl/dwo/tooluse/en http://international.slo.nl/publications/edr/ 5
Exemplary case Tool use project Starting point: Applets Initial ideas for an instructional sequence Initial ideas about improving the learning of functions http://www.fi.uu.nl/tooluse/en/ 6
Exemplary case Tool use project: research with the aim to design an approach and investigate a learning goal How can applets be integrated in an instructional sequence for the concept of function? http://www.fi.uu.nl/tooluse/en/ 7
Design research Why?
Different types of research functions to describe: e.g., what is the achievement of Dutch grade 8 pupils in mathematics? to compare: e.g., what are the differences and similarities between the Spanish and the Dutch curriculum for primary education? to evaluate: e.g., how well does a program function in terms of competences of graduates? to explain or to predict: e.g., what are the causes of poor performance in mathematics? to design and develop: e.g., what are the characteristics of an effective teaching and learning strategy aimed at acquiring certain learning outcomes?
Different types of research methods Measure Influence Degree of closedness Survey with closed questions Systematic observation Multiple choice test Case study Ethnography Interviews Open observation Observe/ask Experiments Quasi-experiments Degree of intervention Action research, Lesson Study Design experiment Stimulated recall interview Ask/act
Why design research? Different types of questions different research methods Is teaching programme A better than B? comparative method What are characteristics of X? Why? case study, ethnography, survey How can we improve Y? How can students learn Z? something needs to be designed, tried out and mostly revised 11
Design research questions and aims Possible questions: How to design such that? Why? Possible research aims: To describe, compare, explain, evaluate But also to: design and advise From what works to how it works Tomorrow: questions <> methods
Why design research? to design and develop an intervention as a solution to a complex educational problem to advance our knowledge about the characteristics of these interventions and the processes to design and develop them to develop or validate theories (e.g. about the learning of a specific topic under certain conditions) (Plomp, 2013, p. 13-15) 13
What characterizes design research?
Design and experiment Experimental research: Design of the research procedure operationalising variables Experiment (no in-between adjustments to the intervention black box ) Results say something about variables Design research: Design, experiment and interpretation of results are intertwined
Design research characteristics 1. Interventionist 2. Iterative 3. Process oriented (instead of a black box model of input-output measurements) 4. Utility oriented 5. Theory oriented 6. Involvement of practitioners 16
Ad 5: to develop theory Local instruction theory (Cobb et al., 2003): a demonstrated learning route for one or more significant mathematical ideas, and substantial means of supporting and organizing learning along that route Local instruction theory Domain specific theory General instructional theory 17
Exemplary case Tool use project: research question How can applets be integrated in an instructional sequence for the concept of function? http://www.fi.uu.nl/tooluse/en/ 18
Exemplary case The Tool Use project Grade 8 (13-14 year old students) Design needed as no technology-rich teaching sequence existed for our interpretation of the concept of function for this target group Design: Teaching experiment of 8 lessons (a lesson takes 50 minutes) Macro cycles: Pilot (one class) Cycle 1 (three classes) Cycle 2 (five classes) 19
Exemplary case A Local Instruction Theory for function? Building blocks for a local instruction theory for the concept of function: A function is a dependency relation from input to output a dynamical process of co-variation a mathematical object with several representations (Carlson et al., 2002; Meel, 1998; Oehrtman et al., 2008;.) 20
Exemplary case A Local Instruction Theory for function? Some activities to get a flavour of the intended instructional sequence Page 1: area of a quadrilateral Page 10: different representations 21
How?
Cycles DR macro cycles consist of three phases: a. Design phase [preparatory phase, preparation and design phase] b. Teaching experiment phase [design experiment] c. Retrospective analysis phase 23
Design phase Conceptual analysis (RME specific) Design principles Didactical phenomenology Mathematization Emergent Modeling Hypotheses and Hypothetical Learning Trajectory 24
Conceptual analysis What is essential in the targetted concept? What is difficult, what are cognitive obstacles? How does it fit in the students existing knowledge What does it prepare for in further learning? Why would students want to know about it? 25
Conceptual analysis: the function concept Exemplary case 26
Conceptual analysis: elaboration in digital design Exemplary case www.fi.uu.nl/dwo/prootool/en
Conceptual analysis: elaboration in digital design Exemplary case www.fi.uu.nl/dwo/prootool/en
Conceptual analysis: elaboration in digital design Exemplary case www.fi.uu.nl/dwo/prootool/en
From conceptual analysis towards phenomenology How do phenomena lead to concept use/awareness? And how are concepts used to understand new phenomena? We can study this from a - Mathematical, - Historical, or - Didactical perspective
Design principle: didactical phenomenology To find phenomena, contexts, problem situations that beg to be organized by mathematical means invite students to develop the targeted mathematical concepts These phenomena can come from real life or can be experientially real 31
Design principle: Mathematization Mathematical objects, structures, methods Abstract Vertical mathematization Realistic context Translate Mathematical model Horizontal mathematization
Exemplary case Vertical: The development of a method / theory for solving systems of two linear equations in general Example horizontal and vertical Horizontal: Translating a problem on fixed and variable costs (e.g., mobile phone offers) in two linear equations
Hypotheses and Hypothetical Learning Trajectory (HLT) Set up a Hypothetical Learning Trajectory which describes: the targeted learning trajectory hypotheses on the type of mathematical thinking that each phase elicits, and the linkages between these phases 34
Exemplary case Hypotheses and Hypothetical Learning Trajectory (HLT) 35
b. Teaching experiment phase Inform teachers and work with them Inform students and parents Think about your role as observer / participating observer / teacher / interviewer / advisor / Decide on data collection (why and how): If you use video: what to focus on? Audio? Number of cameras? Moving or static recordings? Impact of camera on students and teacher? Written data? Field notes / observation forms? Data on group work or work in pairs? 36
Exemplary case b. Teaching experiment phase Teacher guide 37
Exemplary case b. Teaching experiment phase Group work video and Tool use Screen video 38
Exemplary case b. Teaching experiment phase Group work video and Tool use Screen video 39
c. Retrospective analysis phase Data analysis: Compare with hypotheses Software for qualitative data analysis Triangulation Result: Feed forward for next cycle Methodological issues: Reliability Validity Reporting 40
Software for qualitative data analysis Allows you to organize, code, and comment on your data, and in this way to carry out your data analysis in a trackable way 41
Exemplary case Software for qualitative data analysis 42
Exemplary case Results See Doorman et al., 2013
Reliability Internal: does someone else in the team arrive at the same conclusion? Data collection (audio, video, student work, observations ) Systematic coding, peer examination External: replicable? Can someone follow argumentation? Trackability / transparency of researcher s learning process
Validity Internal: quality of design, data and arguments Underpinning of the design (in relation to aims) Ways of data collection Triangulation Method of analysis External: generalizability Are results and theory useful in other contexts? Ecological validity: in real classrooms
Reporting Design Research Developmental research means: experiencing the cyclic process of development and research so consciously, and reporting on it so candidly that it justifies itself, and that this experience can be transmitted to others to become like their own experience. (Freudenthal, 1991, p. 161)
Reflection on the Why, What and How of Design Research
Cycles revisited DR macro cycles consist of three phases: a. Design phase [preparatory phase, preparation and design phase] b. Teaching experiment phase [design experiment] c. Retrospective analysis phase 48
Reflection Please reflect on the why, what and how of Design Research in small groups and discuss the following questions: Do you think DR is useful? When and why? What do you consider as weak points in DR? What do you see as DR s strong points? Which points need further clarification for you? 49
Thank you! Michiel Doorman With input from: Arthur Bakker & Paul Drijvers Freudenthal Institute Utrecht University m.doorman@uu.nl 2015-8-18