! Leander S Hughes!! Investigating the Validity and Reliability of Automatically Generated Reading Comprehension Questions! About!
|
|
- Joshua Ray
- 6 years ago
- Views:
Transcription
1 Investigating the Validity and Reliability of Automatically Generated Reading Comprehension Questions Leander S Hughes About Leander Hughes is an Associate Professor at the Saitama University Center for English Education and Development. He is interested in quantitative language research methods, extensive reading, and computer assisted language learning. Other interests include learner autonomy and applying principles of social psychology to the language-teaching context. Abstract This study seeks to obtain a preliminary evaluation of the validity and reliability of reading questions generated by the program Jist for the purpose of gauging learners online extensive reading progress. The study specifically aims to compare Jist-generated questions to teacher-generated questions in terms of their internal consistency reliability, the relationship of their scores with measures of general English reading proficiency, as well as their inter-correlations after factoring out general proficiency. Findings suggest that Jist questions may actually be more reliable measures of basic comprehension of a text than questions created by teachers. : Jist Jist Jist 6
2 In a previous article published in this journal, Hughes (2012) introduced the program, Jist, an online application currently available at for assisting teachers in checking online extensive reading assignments. Specifically, Jist attempts to automatically gauge a learner s reading of any digital text pasted by that learner into the program s text input box through having students answer automatically generated questions on that text and then ing the results along with the actual text read to both the student and her teacher. Although Jist is not the first attempt by researchers to create a system which can automatically measure reading comprehension (see Gates, 2008, for a thorough review of other attempts at automatic question generation), it is to the author s knowledge, the first attempt at using automatic question generation to encourage and evaluate extensive reading progress among learners of English as a second or foreign language. While Hughes (2012) provided an in-depth rationale for and description of Jist, the question of whether or not Jist can actually be trusted as a valid and reliable basic assessment of learners comprehension of a text was left unanswered. Thus, the aim of the present study is to investigate the reliability and validity of Jist-generated questions through answering the following research questions: 1. Do Jist questions on a given text match or otherwise surpass teachergenerated questions on the same text in terms of their reliability? 2. Is there a significant positive relationship between scores on Jist questions and English reading proficiency test scores? 3. Is there a significant positive relationship between scores on Jist questions and scores on teacher-generated reading comprehension questions made for the same text, after controlling for general English reading proficiency? 7
3 First, it is important to find out whether the Jist-generated questions are internally consistent meaning that they measure the same thing every time they are administered. An affirmative answer to the first research question above would imply that Jist questions demonstrate such measurement consistency. Next, if Jist questions really are able to provide confirmation on whether or not a student has read a text, it entails that the questions must to some degree also measure English reading proficiency. Thus for Jist questions to be valid, the answer to the second research question should be affirmative. Finally, even if Jist questions are found to have a significant positive relationship with general English reading proficiency, there is still a chance that students are answering Jistgenerated questions by relying solely on their general English reading proficiency without actually having read the text for which those questions were generated. To indicate that Jist questions are measuring knowledge and effort specifically dependent on the content of a given text and not general English reading proficiency, Jist question scores should also share a significant positive relationship with scores on teachergenerated questions for the that text after factoring out learners general English reading proficiency. Affirmative answers to all three research questions would support the claim that Jist-questions can provide basic confirmation of whether or not learners have read a given text, which in turn would allow teachers to relegate the work required for such confirmation to Jist, leaving them with more time to devote to class preparation and professional development. Participants and Procedure This study initially involved 1,087 university students at a public university in the Kanto area. Of these initial participants, 740 were dropped from the study because they failed to complete all of the assignments. Thus, the final sample consisted of 347 students, whose Test of English for International Communication (TOEIC) reading section scores ranged from 70 to 385 (out of 495). 8
4 Participants read one news/human-interest article and answered two teachergenerated and two Jist-generated questions on each article online at home, once a week for 20 weeks as part of their homework for Preparation for TOEIC courses in which they were enrolled. These courses were taught by 12 different teachers who each chose one or two of the articles used in the study and wrote the teacher-generated questions for them. Articles ranged in length from 616 to 1,217 words with 871 being the average. Each assignment was worth 1.5 grade points, and students were given this credit as long as they completed the assignment. However, as will be explained further in the next section, the only way participants could complete an assignment was by answering each question correctly, retrying the question if necessary until this was achieved. Measures This section explains each question type and how it was scored as well as how general English reading proficiency was measured. As mentioned above, the two human-generated questions per reading assignment were created by the teachers of the students participating in the study. Both questions were multiple choice with four choices and were modeled after the questions used in the Reading Comprehension section (Part 7) of the TOEIC test. The first of the two questions typically asked about specific information contained in the first half of the reading (see Figure 1 for an example), while the second usually required a more global understanding of the reading (Figure 2). Each question was checked by another teacher, and edited if necessary to help ensure that it matched with other questions in its aims and difficulty level. The first teacher-generated question appeared immediately when participants began the reading assignment, while the second teacher-generated question appeared after the first Jist-generated question was answered. 9
5 Figure 1. Example of a TOEIC-Style Specific Information Question Employed in the Study *The article featured in this and the following examples was taken from /04/25/us-apple-results-idUSBRE83N19Q Figure 2. Example of a TOEIC-Style Global Comprehension Question Employed in the Study 10
6 Two different types of questions were generated by Jist. The first was a missing phrase question (referred to as Progress Check in Hughes, 2012) which displayed four choices, each five-to-eight words in length and one of which was not from the text (Figure 3). Participants had one minute to determine which of the phrases presented was missing from the text. The second question generated by Jist was an ordering question which presented participants with three sections of the text up to 15 words long and one distractor of equal length taken from a different text. Participants were given two minutes to determine which choice was the distractor as well as put the other three choices in the order in which they appeared in the text. As with the human-generated questions, participants had access to the actual text while they were answering the questions. Although this allowed for the possibility that learners could respond correctly simply by scanning the text for the sections presented in the questions, the time limits were meant to make this difficult (see Analysis and Results for further discussion of this possibility). Figure 3. A Jist Phrase Question Generated for an Article Used in the Study 11
7 Figure 4. Jist Ordering Question Generated for an Article Used in the Study To discourage random guessing and also increase measurement sensitivity, participants who answered a question incorrectly had to retry that question after a time delay (10 seconds for human-generated questions, 30 seconds for phrase questions, and one minute for ordering questions) until they were able to successfully answer it. While the human questions remained identical with each retry, the choices for the Jist questions actually changed with each retry, thus preventing participants from answering those items correctly simply by scanning for previously displayed choices while waiting to retry the question. Scores were calculated for each question by dividing one point by the number of tries it took to answer that question correctly (one point for answering on the first try, a half point for doing so on the second retry, a third of a point for a correct response on the third try, and so on). Individual scores for each question were then summed to obtain composite scores for each question type. Finally general English reading proficiency was measured by participants scores on the reading section of a TOEIC pre-test given before the study began as well as their scores on the same section of a TOEIC post-test administered at the end of the study period. Note, that most, but not all participants took the tests (N=345 for the pre-test and N=322 for the post-test). 12
8 Analysis and Results This section explains how each research question was investigated, the results, and their interpretation. Reliability To assess and compare the reliability of the Jist versus teacher-generated questions, a Cronbach s alpha coefficient of internal consistency reliability was obtained for each set of items. The results are presented in Table 1 below. Table 1. Internal Consistency Reliability of Question Sets Teacher-Generated Jist Phrase Jist Ordering All Jist Questions Questions Questions Questions (40 items) (40 items) (20 items) (20 items) Chronbach's alpha Mohsen and Reg (2011, p. 55) state that, depending on the researcher and context, the value considered acceptable for a Cronbach s alpha ranges between.70 and.95. As shown above, the 40 teacher-generated items approached but failed to reach the lower threshold of.70, whereas both the 20 Jist phrase items and the 20 Jist ordering items surpassed it. Also, combining the Jist items raised their internal consistency reliability further such that it surpassed the midpoint between the lower and upper threshold given by Mohsen and Reg. In short, the Jist questions were more reliable than the questions developed by the university teachers. Relationship with English Reading Proficiency To determine and compare the strength of the relationships between the different question sets and general English reading proficiency, a correlation analysis was conducted using students pre- and post-toeic reading section scores (labeled Prof 1 and Prof 2, respectively) and the total scores for each question type. Table 2 below displays the results. 13
9 Table 2. Correlations of Question Scores with English Reading Proficiency Teacher-Generated Questions Jist Phrase Questions Jist Ordering Questions All Jist Questions Prof 1 (N=345).17*.35**.33**.39** Prof 2 (N=322).19**.36**.36**.41** * significant at the.01 level, one-tailed ** significant at the.001 level, one-tailed Though all question types shared a significant positive correlation with proficiency measurements, the Jist questions clearly correlate more highly with the two reading proficiency measurements than do the teacher-generated questions. Additionally, Dörnyei (2007, p. 223) indicates that correlations above.30 can generally be considered meaningful. As shown, only the Jist questions surpass this benchmark. Thus, it appears that Jist questions actually provided a better measurement of learners English reading proficiency than the teacher-generated questions. Relationship with Teacher-Generated Questions While Jist questions appear to be a better measurement of English reading proficiency than the teacher-generated questions, one could still argue that this was because readers relied on their general proficiency to scan for or otherwise guess the correct answers to Jist questions rather than actually reading and comprehending the text upon which those questions were based. On the other hand, perhaps the reason why the teacher-generated questions were less strongly correlated with general proficiency was that readers could not rely solely on their general proficiency to determine the correct answers to these questions and instead had to read and understand the actual content of each text. Depending on that content, individual readers may have achieved higher or lower scores on teacher-generated questions than their TOEIC scores would predict, corresponding to individual differences in background knowledge, interest, and effort devoted to the reading of the text. In order to falsify this hypothesis, a partial correlation analysis was conducted investigating for a relationship between teacher-generated and Jist -generated questions after factoring out general English reading proficiency. Table 3 below presents the results. 14
10 Table 3. Correlations after Controlling for English Reading Proficiency Teacher-Generated Questions (df = 328) Jist Phrase Questions Jist Ordering Questions All Jist Questions 0.32** 0.22** 0.32** ** significant at the.001 level, one-tailed As shown, even after factoring out their relationships with general reading proficiency measurements, Jist questions still shared a significant positive correlation with teacher-generated questions (although the ordering questions failed to surpass Dörnyei s benchmark for meaningfulness). This finding implies that students did not simply rely on their general proficiency to answer Jist questions, but as with the teacher-generated questions, actually had to read and comprehend each individual text to some extent in order to respond correctly. At the same time, the weaker correlation for Jist ordering questions could imply that students relied on scanning and guessing strategies derived from general proficiency more often on those questions than on Jist phrase questions. This in turn may have been because 1) more time was allotted to learners for the ordering question, in which they could have mechanically scanned the text to find the order of the text sections presented and 2) each ordering choice consisted of more words than the choices for the phrase question, which would have increased the relative likelihood that a choice could have been located in the text purely through scanning for it while also making it more likely that the correct response could be guessed based purely on the more complete information provided by each choice. Conclusion The results of this preliminary study indicate that questions automatically generated by the program Jist are more reliable and provide a better indication of reading proficiency than teacher-generated questions on the same content. At the same time, Jist questions share a significant positive relationship with teacher-generated questions after controlling for general reading proficiency, suggesting that responding correctly to Jist questions requires at least some reading of the text for which they were generated and not merely scanning or guessing abilities. 15
11 Future studies on Jist questions should look at how varying different aspects of the questions either increases or decreases their reliability and validity. For example, future studies would do well to investigate whether or not the questions can be further improved by hiding the text when questions appear and thereby eliminating the possibility of answering by scanning (as the current online version of Jist does). Other variables that might usefully be manipulated include question time limits, time delays between retries for both teacher- and Jist-generated questions, number of words per choice, and the number of choices presented. This study represents a first step toward determining the reliability and validity of questions generated by Jist. Though, it would be premature to conclude that Jist is a suitable replacement for teachers when it comes to writing reading comprehension questions, the findings of this study support the claim that Jist can be effectively used as an automatic measure of how much learners have read and understood a text. In conclusion, Jist may indeed represent a quick, yet valid way for teachers to encourage and gauge their learners online extensive reading progress. Acknowledgements The author would like to thank Adriana Edwards Wurzinger, Stacey Vye, Nathan Krug, Jason White, Debjani Ray, Richard Sheehan, Decha Hongthong, Samuel Nfor, Kevin Hawkins, Stewart Fulton, Robert Palka, Naoki Otani, Gabriel Wilkes, and Risa Aoki for their assistance with this study. 16
12 References Dörnyei, Z. (2007). Research methods in applied linguistics: Quantitative, qualitative and mixed methodologies. Oxford: Oxford University Press. Hughes, L. S. (2012). Gauging online extensive reading progress via automatically generated comprehension questions. The Journal of Saitama City Educators, 2(5), Mohsen, T., and Reg, D. (2011). Making sense of Cronbach s alpha. International Journal of Medical Education, 2, Gates D. M. (2008). Automatically generating reading comprehension look-back strategy questions from expository texts. Master s thesis, Carnegie Mellon University. 17
and secondary sources, attending to such features as the date and origin of the information.
RH.9-10.1. Cite specific textual evidence to support analysis of primary and secondary sources, attending to such features as the date and origin of the information. RH.9-10.1. Cite specific textual evidence
More informationTHE PENNSYLVANIA STATE UNIVERSITY SCHREYER HONORS COLLEGE DEPARTMENT OF MATHEMATICS ASSESSING THE EFFECTIVENESS OF MULTIPLE CHOICE MATH TESTS
THE PENNSYLVANIA STATE UNIVERSITY SCHREYER HONORS COLLEGE DEPARTMENT OF MATHEMATICS ASSESSING THE EFFECTIVENESS OF MULTIPLE CHOICE MATH TESTS ELIZABETH ANNE SOMERS Spring 2011 A thesis submitted in partial
More informationAlpha provides an overall measure of the internal reliability of the test. The Coefficient Alphas for the STEP are:
Every individual is unique. From the way we look to how we behave, speak, and act, we all do it differently. We also have our own unique methods of learning. Once those methods are identified, it can make
More informationLinking the Common European Framework of Reference and the Michigan English Language Assessment Battery Technical Report
Linking the Common European Framework of Reference and the Michigan English Language Assessment Battery Technical Report Contact Information All correspondence and mailings should be addressed to: CaMLA
More informationAlgebra 1, Quarter 3, Unit 3.1. Line of Best Fit. Overview
Algebra 1, Quarter 3, Unit 3.1 Line of Best Fit Overview Number of instructional days 6 (1 day assessment) (1 day = 45 minutes) Content to be learned Analyze scatter plots and construct the line of best
More informationCreating Meaningful Assessments for Professional Development Education in Software Architecture
Creating Meaningful Assessments for Professional Development Education in Software Architecture Elspeth Golden Human-Computer Interaction Institute Carnegie Mellon University Pittsburgh, PA egolden@cs.cmu.edu
More informationOVERVIEW OF CURRICULUM-BASED MEASUREMENT AS A GENERAL OUTCOME MEASURE
OVERVIEW OF CURRICULUM-BASED MEASUREMENT AS A GENERAL OUTCOME MEASURE Mark R. Shinn, Ph.D. Michelle M. Shinn, Ph.D. Formative Evaluation to Inform Teaching Summative Assessment: Culmination measure. Mastery
More informationMetacognitive Strategies that Enhance Reading Comprehension in the Foreign Language University Classroom
Andragoške studije, issn 0354 5415, broj 1, jun 2015, str. 145 174 Institut za pedagogiju i andragogiju; Pregledni članak UDK 159.955:028]:[378.147:81 243 Marija Mijušković 1, Saša Simović 2 Faculty of
More informationPROFESSIONAL TREATMENT OF TEACHERS AND STUDENT ACADEMIC ACHIEVEMENT. James B. Chapman. Dissertation submitted to the Faculty of the Virginia
PROFESSIONAL TREATMENT OF TEACHERS AND STUDENT ACADEMIC ACHIEVEMENT by James B. Chapman Dissertation submitted to the Faculty of the Virginia Polytechnic Institute and State University in partial fulfillment
More informationA Pilot Study on Pearson s Interactive Science 2011 Program
Final Report A Pilot Study on Pearson s Interactive Science 2011 Program Prepared by: Danielle DuBose, Research Associate Miriam Resendez, Senior Researcher Dr. Mariam Azin, President Submitted on August
More informationMASTER OF ARTS IN APPLIED SOCIOLOGY. Thesis Option
MASTER OF ARTS IN APPLIED SOCIOLOGY Thesis Option As part of your degree requirements, you will need to complete either an internship or a thesis. In selecting an option, you should evaluate your career
More informationWest s Paralegal Today The Legal Team at Work Third Edition
Study Guide to accompany West s Paralegal Today The Legal Team at Work Third Edition Roger LeRoy Miller Institute for University Studies Mary Meinzinger Urisko Madonna University Prepared by Bradene L.
More informationClassifying combinations: Do students distinguish between different types of combination problems?
Classifying combinations: Do students distinguish between different types of combination problems? Elise Lockwood Oregon State University Nicholas H. Wasserman Teachers College, Columbia University William
More informationDOES RETELLING TECHNIQUE IMPROVE SPEAKING FLUENCY?
DOES RETELLING TECHNIQUE IMPROVE SPEAKING FLUENCY? Noor Rachmawaty (itaw75123@yahoo.com) Istanti Hermagustiana (dulcemaria_81@yahoo.com) Universitas Mulawarman, Indonesia Abstract: This paper is based
More informationEffective practices of peer mentors in an undergraduate writing intensive course
Effective practices of peer mentors in an undergraduate writing intensive course April G. Douglass and Dennie L. Smith * Department of Teaching, Learning, and Culture, Texas A&M University This article
More informationVIEW: An Assessment of Problem Solving Style
1 VIEW: An Assessment of Problem Solving Style Edwin C. Selby, Donald J. Treffinger, Scott G. Isaksen, and Kenneth Lauer This document is a working paper, the purposes of which are to describe the three
More informationCalculators in a Middle School Mathematics Classroom: Helpful or Harmful?
University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Action Research Projects Math in the Middle Institute Partnership 7-2008 Calculators in a Middle School Mathematics Classroom:
More informationAfrican American Male Achievement Update
Report from the Department of Research, Evaluation, and Assessment Number 8 January 16, 2009 African American Male Achievement Update AUTHOR: Hope E. White, Ph.D., Program Evaluation Specialist Department
More informationThe Oregon Literacy Framework of September 2009 as it Applies to grades K-3
The Oregon Literacy Framework of September 2009 as it Applies to grades K-3 The State Board adopted the Oregon K-12 Literacy Framework (December 2009) as guidance for the State, districts, and schools
More informationInside the mind of a learner
Inside the mind of a learner - Sampling experiences to enhance learning process INTRODUCTION Optimal experiences feed optimal performance. Research has demonstrated that engaging students in the learning
More informationLearning By Asking: How Children Ask Questions To Achieve Efficient Search
Learning By Asking: How Children Ask Questions To Achieve Efficient Search Azzurra Ruggeri (a.ruggeri@berkeley.edu) Department of Psychology, University of California, Berkeley, USA Max Planck Institute
More informationRunning head: LISTENING COMPREHENSION OF UNIVERSITY REGISTERS 1
Running head: LISTENING COMPREHENSION OF UNIVERSITY REGISTERS 1 Assessing Students Listening Comprehension of Different University Spoken Registers Tingting Kang Applied Linguistics Program Northern Arizona
More informationNumber of students enrolled in the program in Fall, 2011: 20. Faculty member completing template: Molly Dugan (Date: 1/26/2012)
Program: Journalism Minor Department: Communication Studies Number of students enrolled in the program in Fall, 2011: 20 Faculty member completing template: Molly Dugan (Date: 1/26/2012) Period of reference
More informationUniversity of Waterloo School of Accountancy. AFM 102: Introductory Management Accounting. Fall Term 2004: Section 4
University of Waterloo School of Accountancy AFM 102: Introductory Management Accounting Fall Term 2004: Section 4 Instructor: Alan Webb Office: HH 289A / BFG 2120 B (after October 1) Phone: 888-4567 ext.
More informationSkyward Gradebook Online Assignments
Teachers have the ability to make an online assignment for students. The assignment will be added to the gradebook and be available for the students to complete online in Student Access. Creating an Online
More informationDeveloping a College-level Speed and Accuracy Test
Brigham Young University BYU ScholarsArchive All Faculty Publications 2011-02-18 Developing a College-level Speed and Accuracy Test Jordan Gilbert Marne Isakson See next page for additional authors Follow
More informationAC : DEVELOPMENT OF AN INTRODUCTION TO INFRAS- TRUCTURE COURSE
AC 2011-746: DEVELOPMENT OF AN INTRODUCTION TO INFRAS- TRUCTURE COURSE Matthew W Roberts, University of Wisconsin, Platteville MATTHEW ROBERTS is an Associate Professor in the Department of Civil and Environmental
More informationDeveloping Autonomy in Language Learners: Diagnostic Teaching. LEARN Workshop July 28 and 29, 2015 Ra ed F. Qasem
Developing Autonomy in Language Learners: Diagnostic Teaching LEARN Workshop July 28 and 29, 2015 Ra ed F. Qasem Goals o Define diagnostic teaching. o Explore the process of diagnostic teaching. Plan 1.
More informationTHE EFFECTS OF TEACHING THE 7 KEYS OF COMPREHENSION ON COMPREHENSION DEBRA HENGGELER. Submitted to. The Educational Leadership Faculty
7 Keys to Comprehension 1 RUNNING HEAD: 7 Keys to Comprehension THE EFFECTS OF TEACHING THE 7 KEYS OF COMPREHENSION ON COMPREHENSION By DEBRA HENGGELER Submitted to The Educational Leadership Faculty Northwest
More informationImproving Conceptual Understanding of Physics with Technology
INTRODUCTION Improving Conceptual Understanding of Physics with Technology Heidi Jackman Research Experience for Undergraduates, 1999 Michigan State University Advisors: Edwin Kashy and Michael Thoennessen
More informationNotes on The Sciences of the Artificial Adapted from a shorter document written for course (Deciding What to Design) 1
Notes on The Sciences of the Artificial Adapted from a shorter document written for course 17-652 (Deciding What to Design) 1 Ali Almossawi December 29, 2005 1 Introduction The Sciences of the Artificial
More informationThe Effect of Discourse Markers on the Speaking Production of EFL Students. Iman Moradimanesh
The Effect of Discourse Markers on the Speaking Production of EFL Students Iman Moradimanesh Abstract The research aimed at investigating the relationship between discourse markers (DMs) and a special
More informationASSESSMENT REPORT FOR GENERAL EDUCATION CATEGORY 1C: WRITING INTENSIVE
ASSESSMENT REPORT FOR GENERAL EDUCATION CATEGORY 1C: WRITING INTENSIVE March 28, 2002 Prepared by the Writing Intensive General Education Category Course Instructor Group Table of Contents Section Page
More informationEvaluation of a College Freshman Diversity Research Program
Evaluation of a College Freshman Diversity Research Program Sarah Garner University of Washington, Seattle, Washington 98195 Michael J. Tremmel University of Washington, Seattle, Washington 98195 Sarah
More informationEvidence-based Practice: A Workshop for Training Adult Basic Education, TANF and One Stop Practitioners and Program Administrators
Evidence-based Practice: A Workshop for Training Adult Basic Education, TANF and One Stop Practitioners and Program Administrators May 2007 Developed by Cristine Smith, Beth Bingman, Lennox McLendon and
More informationStudy Abroad Housing and Cultural Intelligence: Does Housing Influence the Gaining of Cultural Intelligence?
University of Portland Pilot Scholars Communication Studies Undergraduate Publications, Presentations and Projects Communication Studies 2016 Study Abroad Housing and Cultural Intelligence: Does Housing
More informationRevision and Assessment Plan for the Neumann University Core Experience
Revision and Assessment Plan for the Neumann University Core Experience Revision of Core Program In 2009 a Core Curriculum Task Force with representatives from every academic division was appointed by
More information5. UPPER INTERMEDIATE
Triolearn General Programmes adapt the standards and the Qualifications of Common European Framework of Reference (CEFR) and Cambridge ESOL. It is designed to be compatible to the local and the regional
More informationSETTING STANDARDS FOR CRITERION- REFERENCED MEASUREMENT
SETTING STANDARDS FOR CRITERION- REFERENCED MEASUREMENT By: Dr. MAHMOUD M. GHANDOUR QATAR UNIVERSITY Improving human resources is the responsibility of the educational system in many societies. The outputs
More informationColloque: Le bilinguisme au sein d un Canada plurilingue: recherches et incidences Ottawa, juin 2008
Inductive and Deductive Approaches to Grammar in Second Language Learning: Process, Product and Students Perceptions Approche inductive et déductive en langues secondes: processus, produit et perceptions
More informationA Game-based Assessment of Children s Choices to Seek Feedback and to Revise
A Game-based Assessment of Children s Choices to Seek Feedback and to Revise Maria Cutumisu, Kristen P. Blair, Daniel L. Schwartz, Doris B. Chin Stanford Graduate School of Education Please address all
More informationSummary results (year 1-3)
Summary results (year 1-3) Evaluation and accountability are key issues in ensuring quality provision for all (Eurydice, 2004). In Europe, the dominant arrangement for educational accountability is school
More informationMathematics Program Assessment Plan
Mathematics Program Assessment Plan Introduction This assessment plan is tentative and will continue to be refined as needed to best fit the requirements of the Board of Regent s and UAS Program Review
More informationDOES OUR EDUCATIONAL SYSTEM ENHANCE CREATIVITY AND INNOVATION AMONG GIFTED STUDENTS?
DOES OUR EDUCATIONAL SYSTEM ENHANCE CREATIVITY AND INNOVATION AMONG GIFTED STUDENTS? M. Aichouni 1*, R. Al-Hamali, A. Al-Ghamdi, A. Al-Ghonamy, E. Al-Badawi, M. Touahmia, and N. Ait-Messaoudene 1 University
More informationGUIDE TO EVALUATING DISTANCE EDUCATION AND CORRESPONDENCE EDUCATION
GUIDE TO EVALUATING DISTANCE EDUCATION AND CORRESPONDENCE EDUCATION A Publication of the Accrediting Commission For Community and Junior Colleges Western Association of Schools and Colleges For use in
More informationEmpowering Students Learning Achievement Through Project-Based Learning As Perceived By Electrical Instructors And Students
Edith Cowan University Research Online EDU-COM International Conference Conferences, Symposia and Campus Events 2006 Empowering Students Learning Achievement Through Project-Based Learning As Perceived
More informationBENCHMARK TREND COMPARISON REPORT:
National Survey of Student Engagement (NSSE) BENCHMARK TREND COMPARISON REPORT: CARNEGIE PEER INSTITUTIONS, 2003-2011 PREPARED BY: ANGEL A. SANCHEZ, DIRECTOR KELLI PAYNE, ADMINISTRATIVE ANALYST/ SPECIALIST
More informationSenior Project Information
BIOLOGY MAJOR PROGRAM Senior Project Information Contents: 1. Checklist for Senior Project.... p.2 2. Timeline for Senior Project. p.2 3. Description of Biology Senior Project p.3 4. Biology Senior Project
More informationIntra-talker Variation: Audience Design Factors Affecting Lexical Selections
Tyler Perrachione LING 451-0 Proseminar in Sound Structure Prof. A. Bradlow 17 March 2006 Intra-talker Variation: Audience Design Factors Affecting Lexical Selections Abstract Although the acoustic and
More informationCHAPTER III RESEARCH METHOD
CHAPTER III RESEARCH METHOD A. Research Method 1. Research Design In this study, the researcher uses an experimental with the form of quasi experimental design, the researcher used because in fact difficult
More informationA STUDY ON THE EFFECTS OF IMPLEMENTING A 1:1 INITIATIVE ON STUDENT ACHEIVMENT BASED ON ACT SCORES JEFF ARMSTRONG. Submitted to
1:1 Initiative 1 Running Head: Effects of Adopting a 1:1 Initiative A STUDY ON THE EFFECTS OF IMPLEMENTING A 1:1 INITIATIVE ON STUDENT ACHEIVMENT BASED ON ACT SCORES By JEFF ARMSTRONG Submitted to The
More informationSuccessfully Flipping a Mathematics Classroom
2014 Hawaii University International Conferences Science, Technology, Engineering, Math & Education June 16, 17, & 18 2014 Ala Moana Hotel, Honolulu, Hawaii Successfully Flipping a Mathematics Classroom
More informationCourse Objectives Upon completion of this course, you will: Have a clear grasp of organic gardening techniques and methods
Organic Gardening Instructor: Fiona Doherty, fcd9@cornell.edu Purpose This 6-week online course is intended to examine the basics of small-scale organic gardening. The topics and depth of information offered
More informationNavigating the PhD Options in CMS
Navigating the PhD Options in CMS This document gives an overview of the typical student path through the four Ph.D. programs in the CMS department ACM, CDS, CS, and CMS. Note that it is not a replacement
More informationHow to Judge the Quality of an Objective Classroom Test
How to Judge the Quality of an Objective Classroom Test Technical Bulletin #6 Evaluation and Examination Service The University of Iowa (319) 335-0356 HOW TO JUDGE THE QUALITY OF AN OBJECTIVE CLASSROOM
More informationThe Effect of Personality Factors on Learners' View about Translation
Copyright 2013 Scienceline Publication International Journal of Applied Linguistic Studies Volume 2, Issue 3: 60-64 (2013) ISSN 2322-5122 The Effect of Personality Factors on Learners' View about Translation
More informationShelters Elementary School
Shelters Elementary School August 2, 24 Dear Parents and Community Members: We are pleased to present you with the (AER) which provides key information on the 23-24 educational progress for the Shelters
More informationThesis-Proposal Outline/Template
Thesis-Proposal Outline/Template Kevin McGee 1 Overview This document provides a description of the parts of a thesis outline and an example of such an outline. It also indicates which parts should be
More informationTASK 2: INSTRUCTION COMMENTARY
TASK 2: INSTRUCTION COMMENTARY Respond to the prompts below (no more than 7 single-spaced pages, including prompts) by typing your responses within the brackets following each prompt. Do not delete or
More informationFocused on Understanding and Fluency
Math Expressions: A Fresh Approach To Standards-Based Instruction Focused on Understanding and Fluency K 1 2 3 4 5 Consumable workbooks K-4 Homework & Remembering K-5 Hardback book gr. 5 Consumable Student
More informationABET Criteria for Accrediting Computer Science Programs
ABET Criteria for Accrediting Computer Science Programs Mapped to 2008 NSSE Survey Questions First Edition, June 2008 Introduction and Rationale for Using NSSE in ABET Accreditation One of the most common
More informationEQuIP Review Feedback
EQuIP Review Feedback Lesson/Unit Name: On the Rainy River and The Red Convertible (Module 4, Unit 1) Content Area: English language arts Grade Level: 11 Dimension I Alignment to the Depth of the CCSS
More informationAnalysis of Students Incorrect Answer on Two- Dimensional Shape Lesson Unit of the Third- Grade of a Primary School
Journal of Physics: Conference Series PAPER OPEN ACCESS Analysis of Students Incorrect Answer on Two- Dimensional Shape Lesson Unit of the Third- Grade of a Primary School To cite this article: Ulfah and
More informationEvaluation of Hybrid Online Instruction in Sport Management
Evaluation of Hybrid Online Instruction in Sport Management Frank Butts University of West Georgia fbutts@westga.edu Abstract The movement toward hybrid, online courses continues to grow in higher education
More informationListening and Speaking Skills of English Language of Adolescents of Government and Private Schools
Listening and Speaking Skills of English Language of Adolescents of Government and Private Schools Dr. Amardeep Kaur Professor, Babe Ke College of Education, Mudki, Ferozepur, Punjab Abstract The present
More informationSchool Size and the Quality of Teaching and Learning
School Size and the Quality of Teaching and Learning An Analysis of Relationships between School Size and Assessments of Factors Related to the Quality of Teaching and Learning in Primary Schools Undertaken
More informationThe Impact of Honors Programs on Undergraduate Academic Performance, Retention, and Graduation
University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Journal of the National Collegiate Honors Council - -Online Archive National Collegiate Honors Council Fall 2004 The Impact
More informationACADEMIC ALIGNMENT. Ongoing - Revised
ACADEMIC ALIGNMENT Sandra Andrews December 2012 Erin Busscher, John Dersch, William Faber, Lorraine Fortuna, Laurie Foster, Wilfred Gooch, Fiona Hert, Diane Patrick, Paula Sullivan and Vince James Part
More informationVOL. 3, NO. 5, May 2012 ISSN Journal of Emerging Trends in Computing and Information Sciences CIS Journal. All rights reserved.
Exploratory Study on Factors that Impact / Influence Success and failure of Students in the Foundation Computer Studies Course at the National University of Samoa 1 2 Elisapeta Mauai, Edna Temese 1 Computing
More informationCONSTRUCTION OF AN ACHIEVEMENT TEST Introduction One of the important duties of a teacher is to observe the student in the classroom, laboratory and
CONSTRUCTION OF AN ACHIEVEMENT TEST Introduction One of the important duties of a teacher is to observe the student in the classroom, laboratory and in other settings. He may also make use of tests in
More informationThe Effect of Written Corrective Feedback on the Accuracy of English Article Usage in L2 Writing
Journal of Applied Linguistics and Language Research Volume 3, Issue 1, 2016, pp. 110-120 Available online at www.jallr.com ISSN: 2376-760X The Effect of Written Corrective Feedback on the Accuracy of
More informationNorms How were TerraNova 3 norms derived? Does the norm sample reflect my diverse school population?
Frequently Asked Questions Today s education environment demands proven tools that promote quality decision making and boost your ability to positively impact student achievement. TerraNova, Third Edition
More informationProcedia - Social and Behavioral Sciences 98 ( 2014 ) International Conference on Current Trends in ELT
Available online at www.sciencedirect.com ScienceDirect Procedia - Social and Behavioral Sciences 98 ( 2014 ) 852 858 International Conference on Current Trends in ELT Analyzing English Language Learning
More informationGROUP COMPOSITION IN THE NAVIGATION SIMULATOR A PILOT STUDY Magnus Boström (Kalmar Maritime Academy, Sweden)
GROUP COMPOSITION IN THE NAVIGATION SIMULATOR A PILOT STUDY Magnus Boström (Kalmar Maritime Academy, Sweden) magnus.bostrom@lnu.se ABSTRACT: At Kalmar Maritime Academy (KMA) the first-year students at
More informationScoring Guide for Candidates For retake candidates who began the Certification process in and earlier.
Adolescence and Young Adulthood SOCIAL STUDIES HISTORY For retake candidates who began the Certification process in 2013-14 and earlier. Part 1 provides you with the tools to understand and interpret your
More informationGrade 2: Using a Number Line to Order and Compare Numbers Place Value Horizontal Content Strand
Grade 2: Using a Number Line to Order and Compare Numbers Place Value Horizontal Content Strand Texas Essential Knowledge and Skills (TEKS): (2.1) Number, operation, and quantitative reasoning. The student
More informationPsychometric Research Brief Office of Shared Accountability
August 2012 Psychometric Research Brief Office of Shared Accountability Linking Measures of Academic Progress in Mathematics and Maryland School Assessment in Mathematics Huafang Zhao, Ph.D. This brief
More informationPredatory Reading, & Some Related Hints on Writing. I. Suggestions for Reading
Predatory Reading, & Some Related Hints on Writing I. Suggestions for Reading Reading scholarly work requires a different set of skills than you might use when reading, say, a novel for pleasure. Most
More informationExtending Place Value with Whole Numbers to 1,000,000
Grade 4 Mathematics, Quarter 1, Unit 1.1 Extending Place Value with Whole Numbers to 1,000,000 Overview Number of Instructional Days: 10 (1 day = 45 minutes) Content to Be Learned Recognize that a digit
More informationAchievement Level Descriptors for American Literature and Composition
Achievement Level Descriptors for American Literature and Composition Georgia Department of Education September 2015 All Rights Reserved Achievement Levels and Achievement Level Descriptors With the implementation
More informationThe Efficacy of PCI s Reading Program - Level One: A Report of a Randomized Experiment in Brevard Public Schools and Miami-Dade County Public Schools
The Efficacy of PCI s Reading Program - Level One: A Report of a Randomized Experiment in Brevard Public Schools and Miami-Dade County Public Schools Megan Toby Boya Ma Andrew Jaciw Jessica Cabalo Empirical
More informationNumber of Items and Test Administration Times IDEA English Language Proficiency Tests/ North Carolina Testing Program.
IDEA English Language Proficiency Tests/ North Carolina Testing Program IPT Kindergarten Subtest Tasks Number of Items Testing Time Answer Questions about Yourself & Follow Directions Give Directions Understand
More informationTEXT FAMILIARITY, READING TASKS, AND ESP TEST PERFORMANCE: A STUDY ON IRANIAN LEP AND NON-LEP UNIVERSITY STUDENTS
The Reading Matrix Vol.3. No.1, April 2003 TEXT FAMILIARITY, READING TASKS, AND ESP TEST PERFORMANCE: A STUDY ON IRANIAN LEP AND NON-LEP UNIVERSITY STUDENTS Muhammad Ali Salmani-Nodoushan Email: nodushan@chamran.ut.ac.ir
More informationThe Condition of College & Career Readiness 2016
The Condition of College and Career Readiness This report looks at the progress of the 16 ACT -tested graduating class relative to college and career readiness. This year s report shows that 64% of students
More informationThe Political Engagement Activity Student Guide
The Political Engagement Activity Student Guide Internal Assessment (SL & HL) IB Global Politics UWC Costa Rica CONTENTS INTRODUCTION TO THE POLITICAL ENGAGEMENT ACTIVITY 3 COMPONENT 1: ENGAGEMENT 4 COMPONENT
More informationKristin Moser. Sherry Woosley, Ph.D. University of Northern Iowa EBI
Kristin Moser University of Northern Iowa Sherry Woosley, Ph.D. EBI "More studies end up filed under "I" for 'Interesting' or gather dust on someone's shelf because we fail to package the results in ways
More informationIntegrating Grammar in Adult TESOL Classrooms
Applied Linguistics 29/3: 456 482 ß Oxford University Press 2008 doi:10.1093/applin/amn020 Integrating Grammar in Adult TESOL Classrooms 1 SIMON BORG and 2 ANNE BURNS 1 University of Leeds, UK, 2 Macquarie
More informationUK Institutional Research Brief: Results of the 2012 National Survey of Student Engagement: A Comparison with Carnegie Peer Institutions
UK Institutional Research Brief: Results of the 2012 National Survey of Student Engagement: A Comparison with Carnegie Peer Institutions November 2012 The National Survey of Student Engagement (NSSE) has
More informationProgram Change Proposal:
Program Change Proposal: Provided to Faculty in the following affected units: Department of Management Department of Marketing School of Allied Health 1 Department of Kinesiology 2 Department of Animal
More informationRubric for Scoring English 1 Unit 1, Rhetorical Analysis
FYE Program at Marquette University Rubric for Scoring English 1 Unit 1, Rhetorical Analysis Writing Conventions INTEGRATING SOURCE MATERIAL 3 Proficient Outcome Effectively expresses purpose in the introduction
More informationScienceDirect. Noorminshah A Iahad a *, Marva Mirabolghasemi a, Noorfa Haszlinna Mustaffa a, Muhammad Shafie Abd. Latif a, Yahya Buntat b
Available online at www.sciencedirect.com ScienceDirect Procedia - Social and Behavioral Scien ce s 93 ( 2013 ) 2200 2204 3rd World Conference on Learning, Teaching and Educational Leadership WCLTA 2012
More informationA Study of Metacognitive Awareness of Non-English Majors in L2 Listening
ISSN 1798-4769 Journal of Language Teaching and Research, Vol. 4, No. 3, pp. 504-510, May 2013 Manufactured in Finland. doi:10.4304/jltr.4.3.504-510 A Study of Metacognitive Awareness of Non-English Majors
More informationEffects of Self-Regulated Strategy Development on EFL Learners Reading Comprehension and Metacognition
31 Effects of Self-Regulated Strategy Development on EFL Learners Reading Comprehension and Metacognition Ali Roohani roohani.ali@gmail.com Shahrekord University, Iran Shiva Asiabani Shahrekord University,
More informationEvaluating Statements About Probability
CONCEPT DEVELOPMENT Mathematics Assessment Project CLASSROOM CHALLENGES A Formative Assessment Lesson Evaluating Statements About Probability Mathematics Assessment Resource Service University of Nottingham
More informationGUIDE TO THE CUNY ASSESSMENT TESTS
GUIDE TO THE CUNY ASSESSMENT TESTS IN MATHEMATICS Rev. 117.016110 Contents Welcome... 1 Contact Information...1 Programs Administered by the Office of Testing and Evaluation... 1 CUNY Skills Assessment:...1
More informationSSIS SEL Edition Overview Fall 2017
Image by Photographer s Name (Credit in black type) or Image by Photographer s Name (Credit in white type) Use of the new SSIS-SEL Edition for Screening, Assessing, Intervention Planning, and Progress
More informationEffect of Word Complexity on L2 Vocabulary Learning
Effect of Word Complexity on L2 Vocabulary Learning Kevin Dela Rosa Language Technologies Institute Carnegie Mellon University 5000 Forbes Ave. Pittsburgh, PA kdelaros@cs.cmu.edu Maxine Eskenazi Language
More informationInterpretive (seeing) Interpersonal (speaking and short phrases)
Subject Spanish Grammar Lesson Length 50 minutes Linguistic Level Beginning Spanish 1 Topic Descriptive personal characteristics using the verb ser Students will be able to identify the appropriate situations
More informationAn Empirical Analysis of the Effects of Mexican American Studies Participation on Student Achievement within Tucson Unified School District
An Empirical Analysis of the Effects of Mexican American Studies Participation on Student Achievement within Tucson Unified School District Report Submitted June 20, 2012, to Willis D. Hawley, Ph.D., Special
More information