Mathematical learning difficulties Long introduction Part II: Assessment and Interventions Professor, Special Education University of Helsinki, Finland Professor II, Special Education University of Oslo, Norway Pirjo.aunio@helsinki.fi 13.10.2014 1
Pedagogical Assessment For seeing progress in learning/ or when there is no progress Identifying children and adolescence with learning difficulties Provide information for desigining and providing teaching and extra suppport for those struggeling with learning Secure that the educational support for children with learning difficulties is adequate and fit in their needs 13.10.2014 2
Pedagogical assessment Early identification & Follow-up the development Screening for those at risk for learning difficulties in mathematics http://www.lukimat.fi/lukimatbedomning-av-larandet/material/identifiering-avstodbehov. Following the development and learning frequently enough. Multidimensional assessment. Multi-professional assessment work. 13.10.2014 3
Pedagogical assessment related to learning difficulties needs several people to work together teacher special educator student wellfare group parents Assessment of learning difficulties Child or adolescent 13.10.2014 4
Special educational thinking (Aunio 2014) Understanding development and learning (difficulties in it) Intervention to at risk development and learning (maldevelopment, learning difficulties) Assessment of development and learning (identification of those at risk and needing extra support) 5
Who are you looking for? (Aunio 2014) Mathematical learning disability Mathematical learning difficulties Dyscalculia Low achievement in mathematics Mathematics disorder Low performance in ICD-10: 5- mathematics 7% 15-20 % Average performance Intensity, specifity and time needed grows Berch & Mazzocco 2007, Geary 2013; Price & Ansari 2013 6
Ways to collect information in pedagogical assessment Development of skills (Developmental psychology) tests screeners Check-lists, Rating scales Curriculum based measurements Formative assessment tasks Summative assessment tasks Combination of learning, teaching and learning environment Interviews and questionnaires observations portfolio Notebooks/journals See for instance: Layton, C. A. & Lock, R.H. (2008). Assessing students with special needs to produce quality outcomes. Pearson Education, Upper Saddle River, New Jersey. 13.10.2014 7
Tests Set of tasks designed to measure ability, features or suitability Norm based Criteria based Easy to use, as manuals are done according the strict criteria The American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME) (2014). Standards for Educational & Psychological Testing. For example: Early Numeracy Test; Banuca 13.10.2014 8
Early Numeracy Test (Aunio, Heiskari, Hautamæki & Van Luit, 2004) 40 Raw Score s on ENT- Fin 30 20 +3sd +2sd +1sd 10 0 M -1sd -2sd -3sd 4 4½ 5 5½ 6 6½ 7 7½ Age Groups 9 13.10.2014 Pirjo Aunio
Screeners To find those who are potentially at risk in their learning or developement Usually screeners are less precise than tests are Aims to catch all potentially at risk children Found in screener will be directed to further investigations Norm or criteria based Easy and fast to use For example: LukiMat- screeners for K, 1 and 2nd graders http://www.lukimat.fi/lukimat-bedomning-avlarandet/material/identifiering-av-stodbehov 13.10.2014 10
Check-lists and Rating scales Tools to evaluate children s skills according the predetermined standards Check-lists contain sequencing skills or behavioral steps to monitor children s growth Easy and fast to use Needs to be well designed so that all relevant steps are included 13.10.2014 11
Example: Mathematical checklist (relational skills) Mathematical relational skills Is able to use/ knows it well Is able to use/ knows it a little bit Cannot use it /does not know it To classify (the most, least) Seriate (according to size) Comparison (more, less) 13.10.2014 12
Curriculum based formative assessment How well the children have learnt the skill they have been thought Can use it daily Most often teacher made Positive features: Instant feedback about teaching -> mondifications to teaching Can give the teacher needed knowledge of baseline performance, where to relate performance later on Possible negative features: Gives narrow picture of learning Does not tell about strategies the child uses 13.10.2014 13
Curriculum based summative assessment Set of tasks that measures performance in particular combination of skills Curriculum guidelines are important Positive features: Informative to teachers who have long experience Tells how well the teaching, teaching environment and the child matched together Weaknesses: There is no norm data where to compare the results Does not tell about the strategies 13.10.2014 14
Interviews and questionnaires Ways to collect information concerning child s learning in broader context To understand the child s learning environment To understand the issues/persons that can support child s learning To understand Strenghts and weaknesses Interests and motivating factors Challenging factors: How to get people to reply? Confidentaility Interviews and analysing takes time 13.10.2014 15
Observations Systematic observations on child s performance Recorded someway Structured: Decided before observation what kind of behavior will be scored 0 or 1 (can count up to 10 without manipulatives = 1; cannot count up to 10 without manipulatives =0) Non-structured - descriptive data is collected (e.g., what kind of strategies the child uses when she is solving simple addition tasks) Benefits: - Information about the authentic learning situations Challenges: - the observing person can disturb in classroom 13.10.2014 16
Portfolios Collections of child s school work, which reflects the child s performance and development Done together with children or adolescence, allows to see what the child things about his/her own work Motivational aspects are important 13.10.2014 17
Notebooks and Journals Writings about child s learning and behaviour Diary-type of writings daily or weekly The child can write about his/her learning at school and/or home What are the disturbing issues, what supports making homework? Etc. Also parents can write on journals Benefits: Information concerning emotions and motivations Challenges: How to analyse and interpret the data? 13.10.2014 18
Can the measurements find the same children performing low in mathematics? Aunio & Mononen, 2011 Spring 2009 120 children in kindergarten 61 boys, 59 girls. Age in time of measurement 6 y. 11 m. 13.10.2014
Can the measurements find the same children performing low in mathematics? Aunio & Mononen, 2011 (2) Cronbachs Makeko.71 EMT.66 Correlations Makeko X EMT.391**, p <. 001 13.10.2014
Can the measurements find the same children performing low in mathematics? Aunio & Mononen, 2011 (3) Measurements in finding the mathematically low performing children Makeko low performing Cut-off point = M -1.5 SD =15 (max 20 points) 17 children with low performance EMT low performing Cut-off point = M 1.5 SD = 18 (max 24 points) 14 children with low performance Makeko and EMT 4 children with low performance 13.10.2014
140 Can the measurements find the same children performing low in mathematics? Aunio & Mononen, 2011 (4) 120 100 80 60 40 pojat tytöt 20 0 120 17 14 4 Koko aineisto Makeko EMT EMT & Makeko 22 13.10.2014 Pirjo Aunio
Educational interventions An educational intervention is a planned modification of the learning environment made for the purpose of altering behavior or development in a prespecified way (Riley-Tillman & Burns, 2009) Evidence-based practice demands have increased and also the research on interventions Can we find intervention benficial for all learners? Publication bias the studies published with positive effects, what about zero effects or negative effects? Cultural-free-interventions: Can we transport a program having positive effects to other culture? 13.10.2014 23
One example Mononen, R. & Aunio, P. (2014) A mathematics intervention for lowperforming Finnish second graders: findings from a pilot study. European Journal of Special Needs Education, http://dx.doi.org/10.1080/08856257.2014.922794 second graders (M age = 8 years, 2 months) with low performance in mathematics, in a quasi-experimental, intervention-control setting. A group of low-performing second graders (LOWi, n = 11) was taught twice a week for eight weeks on the mathematics topics of number word sequence skills, counting skills and conceptual place value knowledge. The intervention s effectiveness was examined by comparing the mathematics performance of the LOWi group to those of two groups consisting of low-performing (LOWc, n = 13) and typically performing children (TYPc, n = 64), who followed their business-as-usual mathematics instruction. The LOWi group made significant improvements in mathematics but did not show significantly better gains, compared to the LOWc and TYPc groups, immediately and three months after the intervention. 13.10.2014 24
Mononen & Aunio, 2014 Pre-test, post-test and delayed post-test performances by group on the mathematics combined scale and addition and subtraction facts measures 13.10.2014 25
How to approach the research literature on interventions? We could select individual intervention studies and look if they have had positive impact Narrow picture? The age range 5-16 years? We could talk about what have been found effective in education in general Hattie (2009): a synthesis of over 800 metaanalyses relating to achievement (student, home, school, teacher, curricula, teaching approaches) To maximize the use of research literature, what can we learn from meta-analysis, reviews and research synthesis? 13.10.2014 26
Systematic reviews Attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question (Higgins & Green, 2008, p.6) Key charachteristics: a clearly stated set of objectives with pre-defined eligibility criteria for studies An explicit, reproducible methodology A systematic search that attempts to identify all studies that would meet the eligibility criteria An assessment of the validity of the findings of the included studies, for example through the assessment of risk bias A systematic presentation, and synthesis, of the charachteristics and findings of the included studies Many systematic reviews contain meta-analysis. In meta-analysis statistical methods is used to summarize results from separate studies In general, the main aim is summarize results from individual studies so that people will understand the evidence and will meka good practical decisions 13.10.2014 27
Effect size (simple) Effect size = [Mean in intervention group] [Mean in comparison group] Standard division in populaation or in comparison group test Used Effect size Effect size small Efect size medium Effect size large Effect size very large standardized mean difference d, Hedge s g.20.50.80 1.30 correlation r.10.30.50.70
What do we know about mathematics learning in nonselected student population? Slavin, R.E. & Lake, C. (2008). Effective programs in elementary mathematics: A best-evidence synthesis. Review of Educational Research 78 (3) 427-515. Slavin, R.E., Lake, C. & Groff, C. (2009). Effective programs in Middle and High School Mathematics: A bestevidence synthesis. Review of Educational Research, 79 (2) 839-911. Cheung, A.C.K. & Slavin, R.E.(2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A metaanalysis. Educational Research Review, 9, 88-113. 29 Pirjo Aunio
All children in comprehensive school (Slavin & Lake 2008) (1) Examines research on all types of math programs that are available to elementary education To be able to select programs and practices most likely to make a difference with the students Three categories of math approaches Mathematics curricula (main focus of the reform in on introduction of alternative text books) Computer-assisted instruction (CAI) (uses technology to enhance students achievement in mathematics) Instructional process programs (higly diverse, focus on teachers instructional practices and classroom management strategies) 13.10.2014 30
All children in comprehensive school (Slavin & Lake 2008) (2) Mathematics curricula (main focus of the reform in on introduction of alternative text books) no significant effects found (Median ES =+0.10) Computer-assisted instruction (CAI) (uses technology to enhance students achievement in mathematics) Effects found when used as supplementary instruction (ES = +0.19) Light and short practice, n 30 min x 3 times a week Instructional process programs (higly diverse, focus on teachers instructional practices and classroom management strategies) ES=+0.33, co-operative learning (ES=+0.29) 13.10.2014 31
All children in comprehensive school (Slavin & Lake 2008) (3) Recommendations: Focus on developing ways to teach and using CAI as supplementary teaching Using intervention programs in a good manner means Professional development training for teachers Continous interaction between program developers and educators 13.10.2014 32
Middle and high school students (Slavin, Lake & Geoff 2009) (1) To summarize the results from mathematical programs aimed to close tha gap between the performance groups and increase the level of performance in all groups Three categories of math approaches Mathematics curricula (main focus of the reform in on introduction of alternative text books) Computer-assisted instruction (CAI) (uses technology to enhance students achievement in mathematics) Instructional process programs (higly diverse, focus on teachers instructional practices and classroom management strategies) 13.10.2014 33
Middle and high school students (Slavin, Lake & Geoff 2009) (2) Mathematics curricula no difference between materials (ES=+0.03) Computer-assisted instruction (CAI) As supplementary (ES = +0.19) As core instruction (ES= +0.09) As organizer (ES = -0.02) Instructional process programs (higly diverse, focus on teachers instructional practices and classroom management strategies) Cooperative learning (ES =+0.46) Metacognitive skills (ES = +0.36) Individual learning (ES=+0.36) Mastery learning, e.g., 80% correct (ES=-0.05) 13.10.2014 34
Technology applications in K-12 grade (Cheung & Slavin, 2013) Supplementary CAI (ES=+0.18) Computer management learning programs (ES = +0.08) Comprehensive programs (CAI + non-computer activities) (ES= +0.07) In general: (ES = +0.16) More benefits in gardes E-9 than grades 10-11 By intensity: more than 30 min/week (30-75 min/week ES=+0.20) More than 75 min/week (ES= +0.06) 13.10.2014 35
The effects of educational support in mathematics for children and adolescents with mathematical learning difficulties Gersten, R., Chard, D. J., Jayanthi, M., Baker, S.K., Morphy, P. & Flojo, J. (2008) Mathematics Instruction for students with learning disabilities or difficulty learning mathematics. A synthesis of the intervention research. Portsmouth, NH:RMC Research Corporation, Center on Instruction. Gersten, R., Chard, D., Jayanthi. M., Baker, S.K., Morphy, P., & Flojo,J. (2009). Mathematics Instruction for students with learning disabilities: A meta-analysis of instructional components. Review of Educational Research, 79 (3) 1202-1242. Kunsch, C.A., Jitendra, A. K., & Sood, S. (2007) The effects of peermediated instruction in mathematics for students with learning problems: research synthesis. Learning Disabilities Research & Practice, 22(1) 1-12. Kearns, D.M. & Fuchs, D. (2013). Does cognitively focused instruction improve the academic performance of low achieving students? Exceptional Children, 79,3, 263-290. 36 11.12.2013 Pirjo Aunio
Mathematics instruction for students with learning disabilities or difficulty learning mathematics (Gersten, Chard, Jayanthi, Baker, Morphy & Flojo (2008) (+ 2009) Mathematical learning diifficulty = Individual Educational plan in mathematics and learning difficulty status Effective features of instruction: Explicit instruction Student verbalization of their mathematical reasoning Visual representations Range and sequence of examples Multiple and heuristic strategies (not tasks dependent) Giving teachers ongoing formative assessment data and feedback on students mathematics performance Providing data and feedback to LD students on their mathematics performance Peer-assisted mathematics instruction 13.10.2014 37
The effects of peer-mediated instruction in mathematics for students with learning problems (Kunch, Jitendra & Sood (2007) Kindergarten -12 grade Students with disabilities and those at risk for mathematics disabilities Peer-mediated interventions in mathematics were moderately effective in improving the performance (overall mean effect sizr d=0.47) Better effects in grades K-9 (d=0.57) than in secondary level (d=0.18) Better effects with students who had at risk for mathematical disabilities (d=0.66) than those who had learning disabilities (d=0.21) (also Baker et al. 2003) Better effects in general education (d=0.56) than in special education classrooms (d=0.32) Better effects computational skills (d=0.63) than skills + concept programs (d=0.34) 13.10.2014 38
Does cognitively focused instruction improve the academic performance of low achieving students? (Kearns & Fuchs, 2013) Cognitively focused interventions, delivered alone or in combination with academic instruction Students with poor academic achievement, learning disabilities or specific cognitive deficits On academical achievement: arithmetic, reading accuracy, reading comprehension, writing quality Several cognitive interventions accelerated lowachieving students academic process but when checked in more detail it does not support using cognitively based interventions Cognitive + academic skills interventions had better results 13.10.2014 39