Predicting Student Performance in University Introductory Physics: The Role of Physics Concepts and Math Skills. Is there a physics placement test? Ken Heller, Jennifer Docktor, Leo Santiago School of Physics and Astronomy 2 year continuing project to improve undergraduate education with contributions by: Many faculty and graduate students of U of M Physics Department In collaboration with U of M Physics Education Group Details at http://groups.physics.umn.edu/physed/ Supported in part by Department of Education (FIPSE), NSF, and the University of Minnesota
1. Two Populations Both Calculus Based Physics Engineering and Physical Science Majors (77% male) Pre-med and Biological Science Majors (59% female) 2. Measuring Basic Physics Concepts (FCI). Gender Differences? Predict Success? 2. Measuring Math Skills Emphasizing the behavior of single objects Emphasizing the behavior of complex systems Gender Differences? Predict Success? 3. Correlations and Background factors? High School Physics High School Math
U of M Physics Course Structure LECTURES DISCUSSION SECTION LABORATORY TESTS 6% drop out rate, 3% D/F rate Not applied perfectly Three hours each week, sometimes with informal cooperative groups. Model constructing knowledge, model problem solving framework. One hour each Thursday -- groups practice using problem-solving framework to solve context-rich problems in cooperative groups. Peer & instructor coaching. Two hours each week -- same cooperative groups practice using framework to solve concrete experimental problems. Same TA. Peer & instructor coaching. Friday -- problem-solving quiz & conceptual questions (usually multiple choice) every three weeks.
Engineering and Physical Science Students Topics first semester Emphasizing the behavior of single objects Kinematics constant acceleration (2 wks) Circular motion & relative motion (1 wk) Forces and acceleration (3 wks) Energy (2 wks) Momentum (1 wk) Rotations (1 wk) Angular momentum (2 wks) Statics (1 wk) Oscillations (2 wks) Pre-Med and Biological Science Students Topics first semester Emphasizing the behavior of complex systems Forces and Torque in Equilibrium including fluids (3 wks) Energy including fluid flow (4 wks) Forces and acceleration including oscillations (2 wks) Kinematics including non-constant acceleration (2 wks) Thermodynamics (4 weeks)
COURSE GRADES BY GENDER CALCULUS-BASED PHYSICS FOR SCIENTISTS & ENGINEERS, FALL TERMS 1997-27 7% 6% MALES AVERAGE COURSE GRADE: 73.5±.2% MALES (N=4375) FEMALES (N=1261) FREQUENCY (NORMALIZED) 5% 4% 3% 2% 1% % F FEMALES AVERAGE COURSE GRADE: 72.±.3% D: 4-49% C: 5-67% B: 68-82% 36 4 44 48 52 56 6 64 68 72 76 8 84 88 92 96 1 COURSE GRADE (%) Males and females do about as well in the course. Absolute grading scale. A: 83-1%
COURSE GRADES BY GENDER CALCULUS-BASED PHYSICS FOR BIOLOGY & PRE-MEDICINE, FALL TERMS 23-27 8.% 7.% 6.% MALES AVERAGE COURSE GRADE: 74.5±.5% FEMALES AVERAGE COURSE GRADE: 72.4±.4% MALES (N=523) FEMALES (N=743) FREQUENCY (NORMALIZED) 5.% 4.% 3.% 2.% 1.%.% 36 38 4 42 44 46 48 5 52 54 56 58 6 62 64 66 68 7 72 74 76 78 8 82 84 86 88 9 92 94 96 98 1 F D: 4-49% C: 5-67% B: 68-82% A: 83-1% COURSE GRADE (%) Males and females do about as well in the course. Students do about as well as in course for Engineering & PS students
1 9 8 7 6 5 4 3 2 1 AVERAGE FCI PRE-TEST & POST-TEST SCORES CALCULUS-BASED PHYSICS FOR SCIENTISTS & ENGINEERS, FALL TERMS 1993-28 A, F93 B, F93 C, F93 D, F94 E, F94 F, F94 G, F94 H, F94 bc, F94 I, F95 J, F95 K, F95 L, F95 D, F95 M, F96 G, F96 N, F96 G, F96 O, F96 P, F97 O, F97 K, F97 D, F97 N, F97 P, F98 G, F98 O, F98 G, F98 N, F98 M, F99 O, F99 K, F99 Q, F99 M, F R, F Q, F N, F S, F T, F K, F1 U, F1 N, F1 W, F1 Z, F1 aa, F2 U, F2 K, F2 I, F2 X, F2 V, F3 T, F3 Y, F3 X, F4 U, F4 Q, F4 Y, F4 at, F5 au, F5 av, F5 az, F6 au, F6 at, F6 ap, F6 X, F7 au, F7 at, F7 ba, F7 bf, F8 au, F8 bf, F8 ba, F8 FCI AVERAGE SCORE (%) +/- STANDARD ERROR OF MEAN 93 PRE-TEST POST-TEST OLD FCI, 1993-1996 NEWFCI, 1997-28 CHANGE FROM QUARTERS TO SEMESTERS F1999 94 95 96 97 98 99 1 2 3 4 5 6 7 8 15 years of Data INSTRUCTOR, TERM Each letter represents one of 37 different professors Incoming student scores are slowly rising (better high school preparation) Our standard course (CGPS) achieves average FCI ~7% Our best practices course achieves average FCI ~8% Not executing any cooperative group procedures achieves average FCI ~5%
1 AVERAGE FCI PRE-TEST & POST-TEST SCORES CALCULUS-BASED PHYSICS FOR BIOLOGY & PRE-MEDICINE, FALL TERMS 23-28 PRE-TEST POST-TEST AVERAGE FCI SCORE (%) + STANDARD ERROR OF MEA 9 8 7 6 5 4 3 2 1 K, F3 ad, F4 K, F4 ad, F5 K, F5 D, F6 ad, F6 M, F7 aw, F7 ae, F8 ae, F8 INSTRUCTOR, TERM 5 years of Data Each letter represents one of 6 different professors Course is still under development Gains are the same as in course for Engineers and Physical Science Students
AVERAGE FCI PRE-TE 1% 9% 8% 7% 6% 5% 4% 3% 2% 1% FCI PRE-TEST SCORES BY GENDER & YEAR CALCULUS-BASED PHYSICS FOR SCIENTISTS & ENGINEERS, FALL TERMS 1997-27 M ALES (N=4375) y =.84x - 16.4 R 2 =.88 Students are getting better from high school FEMALES (N=1261) y =.76x - 14.8 R 2 =.88 There is a gender gap in conceptual performance from high school % 1997 1998 1999 2 21 22 23 24 25 26 27 1% 9% AVERAGE FCI SCORE 8% 7% 6% 5% 4% 3% 2% 1% % PRE-TEST GENDER GAP 15.3±.5% POST-TEST GENDER GAP 13.4±.6% Gains are the same 71.2±.3% 52.8±.3% 57.8±.5% 37.5±.5% PRE-TEST POST-TEST
F CI P re-test S c FCI SCORE (%) 1 9 8 7 6 5 4 3 2 1 24 21 18 15 12 9 6 3 AVERAGE FCI PRE-TEST SCORES BY PREVIOUS PHYSICS CALCULUS-BASED PHYSICS FOR SCIENTISTS & ENGINEERS, FALL TERMS 1997-27 Gap = 13.±1.2 % 14.2±.7 % 17.1 ±1.4 % 14.5 ±3.1 % 1.1±3.6 % No Previous Physics Regular high school only CALCULUS-BASED PHYSICS FOR Pre-med & Biology, FALL TERMS 23-27 MALES (N=523) Females (N=743) Gap = 8.9 1.7 % 12. 1.2 % 14.2 2.6 % 6.8 ± 5.8 % 13.2 4. % ± MALES (N=4215) ± Advanced placement high school only FEMALES (N=1217) 9% 15% 65% 62% 19% 17% 2% 3% 4% 4% 16% 28% 59% 52% 17% 12% 4% 4% 5% 4% Gap for every type of High School Physics Course ± College only Both college and high school ±
Can the FCI be used as a placement test? 1 COURSE GRADE VS. FCI PRE-TEST SCORE CALCULUS-BASED PHYSICS FOR SCIENTISTS & ENGINEERS, FALL TERMS 1997-27 FEMALES (N=1261) MALES (N=4375) 9 8 COURSE GRADE (%) 7 6 5 4 3 2 1 FEMALES: %varience = 11% Correlation =.33 MALES: %varience = 21% Correlation =.46 1 2 3 4 5 6 7 8 9 1 FCI PRE-TEST SCORE (%) The FCI is not a good predictor of performance.
Can a Math Skills Test be used as a placement test? 3 Questions Powers of ten Triangles Graphs Algebra Simultaneous Equations Derivatives Anti-Derivatives Algebra Solve for a in the equation a 2 x + cy = t (a) ± t cy x (b) ± t cy (d) t cy 2x Solve for y in the equation x (e) ( cy t)(cy + t) ax + b cy + d = f [95-99%] (c) ± 1 a t cy (a) ax + b df cf = y [49-72%] (b) ax + b f + d (c) ax + b d 1 cf (d) ax cf + + b d (e) 1 c f ax + b d [15-34%]
1 9 8 AVERAGE MATH PRE-TEST & POST-TEST SCORES BY GENDER CALCULUS-BASED PHYSICS FOR SCIENTISTS & ENGINEERS, FALL TERMS 25-27 PRE-TEST GENDER GAP -.7±.3 (-2.5±1.3%) MALES (N=845) FEMALES (N=266) POST-TEST GENDER GAP -.2±.3 (-.7±1.1%) 7 MATH DIAGNOSTIC SCORE 6 5 4 3 2 16.3±.2 (6.5±.6%) 17.±.3 (63.3±1.1%) 19.±.1 (7.3±.5%) 19.2±.3 (71.±1.%) 1 PRE-TEST Females do slightly better. POST-TEST
MATH DIAGNOSTIC SCORE (%) 1 9 8 7 6 5 4 3 2 1 Algebra AVERAGE MATH PRE-TEST SCORES BY HIGH SCHOOL MATH CALCULUS-BASED PHYSICS FOR SCIENTISTS & ENGINEERS Geometry or Trigonometry MALES (N=83) FEMALES (N=256) 1% 2% 3% 2% 19% 19% 7% 69% 8% 7% Pre-calculus, Functions, or Analysis Calculus Other, more advanced math Math Diagnostic Sco 1 9 8 7 6 5 4 3 2 1 CALCULUS-BASED PHYSICS FOR BIOLOGISTS & PRE-MEDS MALES (N=176) FEMALES (N=224) 3% 2% 9% 8% 28% 29% 57% 6% 3% 2%
Can a Math Skills Test be used as a placement test? 1 COURSE GRADE VS. MATH PRE-TEST SCORE CALCULUS-BASED PHYSICS FOR SCIENTISTS & ENGINEERS, FALL TERMS, 25-27 FEMALES (N=266) MALES (N=845) 9 8 COURSE GRADE (%) 7 6 5 4 3 2 1 FEMALES: % varience = 3% Correlation =.55 MALES: %varience= 25% Correlation =.5 1 2 3 4 5 6 7 8 9 1 MATH PRE-TEST SCORE (%) The Math Skills Test is not a good predictor of performance. Better than FCI
Predictive Power Stepwise Multiple Regression % variance of course grade predicted Engineering and Physical Science Female Math skills test: 31% of grade FCI add: 6% of grade HS background add: negligible FCI pre Math pre correlate 31% Male Math skills test: 25% of grade FCI add: 7% of grade HS background add: negligible FCI pre Math pre correlate 44% Pre-med and Biological Science Female Math skills test: 18% of grade FCI add: 6% of grade HS background add: negligible FCI pre Math pre correlate 34% Male Math skills test: 2% of grade FCI add: 4% of grade HS background add: negligible FCI pre Math pre correlate 39% No effective placement test for our introductory physics Interactive Engagement Pedagogy Cooperative Group Problem Solving Emphasizing Problem Solving Skills Good Conceptual Gains
The End Please visit our website for more information: http://groups.physics.umn.edu/physed/