An Evaluation of the Item Pools Used for Computerized Adaptive Test. Versions of The Maryland Functional Tests. Steven L. Wise, Ph.D.
|
|
- Carol Carpenter
- 6 years ago
- Views:
Transcription
1 An Evaluation of the Item Pools Used for Computerized Adaptive Test Versions of The Maryland Functional Tests Steven L. Wise, Ph.D. July, 1997 A Report Prepared for the Assessment Branch of the Maryland State Department of Education. Ol O
2 An Evaluation of the Item Pools Used for Computerized Adaptive Test Versions of The Maryland Functional Tests In order to meet state-mandated high school graduation requirements, Maryland students must demonstrate competency in mathematics, writing, reading, and citizenship. The Maryland Functional Tests were developed in the late 1970's to ensure that Maryland's high school graduates were competent in these four core subject areas. Early in the 1990's, the Maryland State Department of Education (MSDE) developed computerized adaptive test (CAT) versions of three core area tests (mathematics, reading, citizenship). The CAT versions have augmented rather than supplanted the paper-andpencil versions; they are typically used with transfer students and students who are retaking one or more test. The purpose of this report is to present the findings of my evaluation of the item pools used in the CAT versions of the Maryland Functional Tests. The item pools were evaluated regarding: (a) pool size, (b) adequacy of the test information provided by the pooi, and (c) balance of content domains and item difficulty levels. I provide recommendations regarding both targeted expansion of the item pools and testing methods that will make more effective use of the existing pools. In addition, I discuss the problem of item exposure and the vulnerability of the MSDE item pools. Item Pool Size and Structure The success of any CAT program is largely dependent on the quality of the item pool (sometimes termed an item bank) from which the administered items are drawn. Quality can be conceptualized according to two basic criteria. First, the total number of items in the pool must be sufficient to supply informative items throughout a testing session. Second,
3 the items in the pool must have characteristics that provide adequate information at the proficiency levels that are of greatest interest to the test developer. This criterion primarily means that at all important levels of proficiency there are sufficient numbers of items whose difficulty parameters provide useful information. Thus, a high-quality item pool will contain sufficient numbers of useful items that permit efficient, informative testing at important levels of proficiency. When a testing program has developed a large number of items that are to be used in the CAT, the first criterion obviously is met. Merely having a large number of items, however, does not ensure that the second criterion will be satisfied. Unless the developed items have a distribution of difficulties that is reasonably matched to the important levels of proficiency, there will likely be areas of proficiency in which the test information provided by the CAT will accumulate at too slow of a rate. In the sense of the pool analogy, the item pool will be too "shallow" in some proficiency region(s). In these regions, the CAT will be less efficient--resulting in either higher standard errors of profidency estimation (for a fixed-length CAT) or a longer test being required (for a variable-length CAT) to reach a desired level of precision. An obvious solution to this problem is for additional pool items to be developed--that provide additional information (depth) where it is most needed. The pool depth issue becomes more complicated when the pool is subdivided into a number of content domains---each of which must be represented to a prespecified degree in the CAT. Ideally, each content domain should exhibit a distribution of item difficulties that resembles that of the entire pool. In practice, however, this is difficult to attain. Items written for domains that represent more elementary knowledge tend to be easier than
4 those written for domains representing more advanced knowledge in a given subject. Having a larger item pool has an additional advantage--items in the pooi tend to be exposed in fewer CAT administrations. The integrity of the CAT is dependent upon the item parameters remaining unchanged. If an item is presented too often, students (and/or teachers) may become familiar with it and prepare for it. This would result in a decrease in the item's actual difficulty--which in turn would positively bias proficiency estimation. Hence, item exposure can become a serious problem, particularly in a highstakes test program. Pvrpqse of the Maryland F~nctiQnal Tests It should be emphasized that the purpose of the Maryland Functional Tests is to ensure that Maryland's high school graduates are minimally competent in the four core subject areas. That is, they are primarily criterionreferenced tests, rather than norm-referenced tests--which is important in assessing the suitability of the subject area item pools for CAT. In minimum competency testing, the primary goal of measurement is whether or not a student has attained a proficiency level that exceeds the minimum passing standard. Differentiation among student performance above (or below) from the passing standard is of secondary importance. This implies that the best measurement needs to occur at the passing score. In an item response theory sense, test information should be maximized at the passing score. My judgments and recommendations regarding the item pools are based on this perspective. Although it is tempting to adopt the dual goals of having the Maryland Functional Tests be both good minimum competency tests and good normreferenced tests, I believe that it would be unwise to do so in the case of the
5 CAT versions. The adaptive procedures are quite different in each type of CAT, and if one tried to meet both goals concurrently, then neither goal would likely be met very well. Assessment of the Maryland Functional Test Item Pools Mathematics The current Mathematics item pool consists of 180 items from 7 content domains. Figure 1 shows the distributions of item difficulties for each content domain. The passing score for the Mathematics test, transformed into Rasch model logits, is Figure 1. Mathematics Item Pool Census, Broken Down by Content Domain and Item Difficulty (Delta). Item Difficulty (Delta) to - to - to - to to to to to Mathematics Domair~ Number Concepts 1 1 3! i Whole Number Operations ~ Mixed Number/Fraction Operations 0 0 5! Decimal Operations ' 25 Measurement, 1 ] 6 5 i Using Data 2 i Problem Solving 0 i , n ~ m Figure 1 reveals several problems with the Mathematics item pool. The most serious problem is that the pool is too easy. For a minimum competency test with passing score of 2.15 logits, an effective item pool would have most of its delta values in the region of The Mathematics pool, however, has only 8 items (4%) in the 2.01 to 3.00 range, and no more than 41 (23%) within 1.15 logits of the passing score. This indicates that the majority
6 of the items provide relatively little information at the passing score--where it is most needed. This means that (a) it is difficult for a CAT to match items to students whose proficiency is in the vicinity of the passing score and (b) the CATs administered to moderate to high proficiency students will be virtually identical--an item exposure problem. The distribution of item difficulties varies substantially across content domains. Three domains exhibited particular problems. Whole Number Operations contains a fairly narrow range of very easy items. The items in the Mixed Number/Fraction Operations domain were generally more difficult, but showed an even narrower range. In addition, there were relatively few items available (13) in Number Concepts. Readine v The current Reading item pool consists of 191 items from 5 content domains. Figure 2 shows the distributions of item difficulties for each content domain. The passing score for the Reading test, transformed into logits, is Fie-ure 2. Reading Item PooI Census, Broken Down by Content Domain and Item Difficulty (Delta). Item Difficulty (Delta) I to - to - to - to to to to to Following Directions Locating Information Main Ideas Using Details Understanding Forms
7 The Reading item pool--in contrast to the Mathematics pool--is well centered over the passing score. There are 141 (74%) of the items between logit values of 0 and 3. The only problematic domain was Locating Information, for which the distribution of items was too easy. Citizenship The current Citizenship item pool consists of 261 items from 3 content domains. Figure 3 shows the distributions of item difficulties for each content domain. The passing score for the Citizenship test, transformed into Iogits, is Fi~-ure 3. v Citizenship Item Pool Census, Broken Down by Content Domain and Item Difficulty (Delta). Item Difficulty (Delta) Citizenshiv Dom0in Constitutional Government Principles, Rights, Responsibilities Politics and Political Behavior to - to - to - to to to to to As with the Reading item pool, the Citizenship pool is well centered over the passing score; 156 (60%) of the item difficulties lie within 1 logit of the passing score. Moreover, there is a good distribution of item difficulties within each of the three content domains. Re omm ndations Re~ardin~ the Item Pools My primary recommendation is that the Mathematics item pool be augmented with more items. The overall pool size is not very large; because of the high-stakes nature of the Maryland Functional Tests I would
8 recommend that it be expanded to at least 250 items. The set of new items should be substantially more difficult than the current ones, in order to increase the information of the entire pool at the passing score. Special attention should also be paid to increasing the relative number of items in the Number Concepts domain. For the Reading item pool, I recommend that the pool size be increased to 250 items, using items of moderate difficulty. In addition, I would suggest that special attention be paid to adding more difficult items to the Locating Information domain. The Citizenship item pool is in very good shape---in terms of both numbers of items and distribution of items within content domains. I have no changes to recommend for this pooi. I have two additional general recommendations regarding the item pools. First, I encourage MSDE staff to review all the items in each of the pools to ensure that (a) the item stem and options are correctly entered and (b) the keyed answer is correct. I have heard concerns expressed regarding correctness of keyed answers; a review would address these concerns. Second, I believe that the item response theory (IRT) parameters used for the CAT item pool were based on calibrations of paper-and-pencil versions of the items. It is unclear whether computer administration affects the parameters of the items. The research on this issue is limited. It would be useful to conduct a study that investigated the robustness of the item parameters across administration media. I would, however, consider this a lower priority than a review of the items for accurate content and correct keying. Recommendations Regarding CAT Administration I believe that the three item pools can be used more effectively through changes in the MicroCAT testing programs. There are two shortcomings in
9 the programs currently being used. First, each CAT comprises a 30-item test that administers a predetermined number of items from each content domain. For example, the Citizenship CAT administers a sequence of 10 items from the first content domain, then 10 from the second, and finally 10 from the third. While this procedure ensures that a third of each test will come from each content area, it precludes the possibility of variable-length CATs--which is a key advantage of adaptive testing. The second shortcoming of the current CATs is that they use basically norm-referenced procedures to make criterion-referenced decisions about student proficiency. There are alternative testing procedures that are better suited to criterion-referenced CATs. One is the adaptive mastery testing (AMT) procedure (Weiss & Kingsbury, 1984). In the AMT procedure, the CAT will terminate if the confidence interval around a student's estimated proficiency does not include the passing score value--which indicates sufficient certainty regarding classification of the student's competency status. This procedure results in shorter tests being needed for many students whose proficiency levels lie well above (or below) the passing score, because fewer items would be needed to reach the termination criterion. For borderline students, longer tests would be administered, because more information would be needed to make the more difficult classification decision. Note that the AMT procedure uses variable length CATs. Both shortcomings can be overcome by adopting a CAT procedure that combines the AMT procedure with the use of testlets (Wainer & Kiely, 1987). A testlet is a set of items that are administered as a group. Testlets can be formed that are of differing average difficulty, and a CAT can adaptively administer testlets instead of items. After a testlet has been administered, the
10 CAT algorithm looks for the most informative remaining testlet to administer. Because the content domains are represented equally in each of the three subject CATs, testlets could readily be used to maintain content balance. The size of the testlet in each subject area would be equal to the number of content domains. Thus, the Mathematics, Reading, and Citizenship CATs would use testlets of size seven, five, and three, respectively. The AMT procedure could then be used to adaptive administer testlets (perhaps with a minimum test length imposed) until the termination criterion was reached (a student's profidency confidence-interval no longer contained the passing score). This procedure would permit efficient, variable length testing while maintaining content balance. To illustrate the testlet-based AMT procedure, consider the Reading test. A group of five-item testlets would be formed; each testiet would contain one item from each content domain. Suppose that a minimum of 15 items was imposed, with a maximum of 30. After a testlet had been administered (beginning with the third), the procedure would check to see if the termination criterion had been reached. If so, then the procedure would end; otherwise, the most informative testlet remaining in the pooi would be selected and administered. This process would continue until either the termination criterion was reached or a sixth testlet had been administered--at which point the CAT would end, and the pass/fail status of the student would be determined by comparing his or her finai profidency estimate to the passing score. Use of the testlet-based AMT procedure would more efficiently use the item pool, and items would consequently be exposed at a lower rate. Moreover, the testlets could be reassigned periodically, to help balance the
11 rates at which items are exposed. The testlet-based AMT procedure could readily be programmed using the current version of the MicroCAT testing software. Testlets are called "cluster items" in the MicroCAT scripting language. In addition, code for the implementing the AMT procedure on MicroCAT has been published (Roos, Wise, Yoes, & Rocklin, 1996). Increasing Test Security Any high-stakes testing program must be concerned about security. The credibility and integrity of the testing program is dependent upon how well the security of the test items are maintained. Because all of the items are accessible in a CAT, particular care must be taken to protect the item pool. MSDE has already experienced the loss of a 240-item pool in Mathematics, because the item contents were potentially revealed to an unauthorized individual. Use of the Mathematics CAT had to be suspended until a new item pool was developed. The current Citizenship item pool, however, contains all of the available items. If this item pool were to be compromised, the results would be disastrous, because there are no calibrated paper-andpencil items from which a new pool could be developed. Hence, unless new Citizenship items are developed, MSDE is in a very vulnerable position. Ideally, MSDE should have the resources to develop a backup item pool for each subject test. Moreover, periodic rotation of these pools could further decrease item exposure. MSDE should review all of their procedures for distributing CAT software and collecting student data. MSDE staff must be very careful to limit who has access to the item pool. Moreover, student CAT administration output files should contain a record of exactly which items each student was administered.
12 At the item level, to the degree to which particular pool items are known to students (or teachers), the validity of the CAT is threatened. This problem can be addressed, in part, by controlling the frequency with which a particular item will be exposed. Increasing the size of the item pools and/or adopting testlet-based AMT procedures will help control item exposure. If testlets are used, periodic reassignment of file items by MSDE staff could be done in a fashion that distributes the exposure fairly uniformly across the pool. Resources Allocation V.ersus CAT Program Longevity MSDE currently has limited resources for monitoring and maintaining the CAT versions of the Maryland Functional Tests. Developing new items and backup item pools will require expenditures of time and resources that may be difficult to allocate. Furthermore, there are plans to eventually phase out the Maryland Functional Tests after the Maryland High School Assessment is implemented. Judgments regarding how much effort should be directed toward changing the present CAT tests must take into account the likelihood that the tests may soon be discontinued. I believe, however, that with relatively small expenditures of time and money the current CAT tests can be markedly improved. References Roos, L. L., Wise, S. L., Yoes, M. E., & Rocklin, T. R. (1996). Conducting self-adapted testing using MicroCAT. Educational and Psychological Measurement, 56, Wainer, H., & Kiely, G. L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, Weiss, D. J., & Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21,
Computerized Adaptive Psychological Testing A Personalisation Perspective
Psychology and the internet: An European Perspective Computerized Adaptive Psychological Testing A Personalisation Perspective Mykola Pechenizkiy mpechen@cc.jyu.fi Introduction Mixed Model of IRT and ES
More informationFurther, Robert W. Lissitz, University of Maryland Huynh Huynh, University of South Carolina ADEQUATE YEARLY PROGRESS
A peer-reviewed electronic journal. Copyright is retained by the first or sole author, who grants right of first publication to Practical Assessment, Research & Evaluation. Permission is granted to distribute
More informationHow to Judge the Quality of an Objective Classroom Test
How to Judge the Quality of an Objective Classroom Test Technical Bulletin #6 Evaluation and Examination Service The University of Iowa (319) 335-0356 HOW TO JUDGE THE QUALITY OF AN OBJECTIVE CLASSROOM
More informationSETTING STANDARDS FOR CRITERION- REFERENCED MEASUREMENT
SETTING STANDARDS FOR CRITERION- REFERENCED MEASUREMENT By: Dr. MAHMOUD M. GHANDOUR QATAR UNIVERSITY Improving human resources is the responsibility of the educational system in many societies. The outputs
More informationPsychometric Research Brief Office of Shared Accountability
August 2012 Psychometric Research Brief Office of Shared Accountability Linking Measures of Academic Progress in Mathematics and Maryland School Assessment in Mathematics Huafang Zhao, Ph.D. This brief
More informationProficiency Illusion
KINGSBURY RESEARCH CENTER Proficiency Illusion Deborah Adkins, MS 1 Partnering to Help All Kids Learn NWEA.org 503.624.1951 121 NW Everett St., Portland, OR 97209 Executive Summary At the heart of the
More informationGuidelines for the Use of the Continuing Education Unit (CEU)
Guidelines for the Use of the Continuing Education Unit (CEU) The UNC Policy Manual The essential educational mission of the University is augmented through a broad range of activities generally categorized
More informationNCEO Technical Report 27
Home About Publications Special Topics Presentations State Policies Accommodations Bibliography Teleconferences Tools Related Sites Interpreting Trends in the Performance of Special Education Students
More informationACADEMIC AFFAIRS GUIDELINES
ACADEMIC AFFAIRS GUIDELINES Section 8: General Education Title: General Education Assessment Guidelines Number (Current Format) Number (Prior Format) Date Last Revised 8.7 XIV 09/2017 Reference: BOR Policy
More informationA GENERIC SPLIT PROCESS MODEL FOR ASSET MANAGEMENT DECISION-MAKING
A GENERIC SPLIT PROCESS MODEL FOR ASSET MANAGEMENT DECISION-MAKING Yong Sun, a * Colin Fidge b and Lin Ma a a CRC for Integrated Engineering Asset Management, School of Engineering Systems, Queensland
More informationAlpha provides an overall measure of the internal reliability of the test. The Coefficient Alphas for the STEP are:
Every individual is unique. From the way we look to how we behave, speak, and act, we all do it differently. We also have our own unique methods of learning. Once those methods are identified, it can make
More informationDesigning a Rubric to Assess the Modelling Phase of Student Design Projects in Upper Year Engineering Courses
Designing a Rubric to Assess the Modelling Phase of Student Design Projects in Upper Year Engineering Courses Thomas F.C. Woodhall Masters Candidate in Civil Engineering Queen s University at Kingston,
More informationNorms How were TerraNova 3 norms derived? Does the norm sample reflect my diverse school population?
Frequently Asked Questions Today s education environment demands proven tools that promote quality decision making and boost your ability to positively impact student achievement. TerraNova, Third Edition
More informationDEPARTMENT OF MOLECULAR AND CELL BIOLOGY
University of Texas at Dallas DEPARTMENT OF MOLECULAR AND CELL BIOLOGY Graduate Student Reference Guide Developed by the Graduate Education Committee Revised October, 2006 Table of Contents 1. Admission
More informationStudent Assessment and Evaluation: The Alberta Teaching Profession s View
Number 4 Fall 2004, Revised 2006 ISBN 978-1-897196-30-4 ISSN 1703-3764 Student Assessment and Evaluation: The Alberta Teaching Profession s View In recent years the focus on high-stakes provincial testing
More informationKarla Brooks Baehr, Ed.D. Senior Advisor and Consultant The District Management Council
Karla Brooks Baehr, Ed.D. Senior Advisor and Consultant The District Management Council This paper aims to inform the debate about how best to incorporate student learning into teacher evaluation systems
More informationThe Oregon Literacy Framework of September 2009 as it Applies to grades K-3
The Oregon Literacy Framework of September 2009 as it Applies to grades K-3 The State Board adopted the Oregon K-12 Literacy Framework (December 2009) as guidance for the State, districts, and schools
More informationProgram Change Proposal:
Program Change Proposal: Provided to Faculty in the following affected units: Department of Management Department of Marketing School of Allied Health 1 Department of Kinesiology 2 Department of Animal
More informationGCSE English Language 2012 An investigation into the outcomes for candidates in Wales
GCSE English Language 2012 An investigation into the outcomes for candidates in Wales Qualifications and Learning Division 10 September 2012 GCSE English Language 2012 An investigation into the outcomes
More informationBENCHMARK TREND COMPARISON REPORT:
National Survey of Student Engagement (NSSE) BENCHMARK TREND COMPARISON REPORT: CARNEGIE PEER INSTITUTIONS, 2003-2011 PREPARED BY: ANGEL A. SANCHEZ, DIRECTOR KELLI PAYNE, ADMINISTRATIVE ANALYST/ SPECIALIST
More informationNATIONAL CENTER FOR EDUCATION STATISTICS RESPONSE TO RECOMMENDATIONS OF THE NATIONAL ASSESSMENT GOVERNING BOARD AD HOC COMMITTEE ON.
NATIONAL CENTER FOR EDUCATION STATISTICS RESPONSE TO RECOMMENDATIONS OF THE NATIONAL ASSESSMENT GOVERNING BOARD AD HOC COMMITTEE ON NAEP TESTING AND REPORTING OF STUDENTS WITH DISABILITIES (SD) AND ENGLISH
More informationLecture 1: Machine Learning Basics
1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3
More informationLast Editorial Change:
POLICY ON SCHOLARLY INTEGRITY (Pursuant to the Framework Agreement) University Policy No.: AC1105 (B) Classification: Academic and Students Approving Authority: Board of Governors Effective Date: December/12
More informationApproval Authority: Approval Date: September Support for Children and Young People
Document Title: Pupil Premium Policy Purpose: To set out the principles of the Pupil Premium Award, how it is received and how it has been spent in the last year and to evaluate the impact Summary: The
More information5. UPPER INTERMEDIATE
Triolearn General Programmes adapt the standards and the Qualifications of Common European Framework of Reference (CEFR) and Cambridge ESOL. It is designed to be compatible to the local and the regional
More informationSouth Carolina English Language Arts
South Carolina English Language Arts A S O F J U N E 2 0, 2 0 1 0, T H I S S TAT E H A D A D O P T E D T H E CO M M O N CO R E S TAT E S TA N DA R D S. DOCUMENTS REVIEWED South Carolina Academic Content
More informationABET Criteria for Accrediting Computer Science Programs
ABET Criteria for Accrediting Computer Science Programs Mapped to 2008 NSSE Survey Questions First Edition, June 2008 Introduction and Rationale for Using NSSE in ABET Accreditation One of the most common
More informationExecutive Summary. Laurel County School District. Dr. Doug Bennett, Superintendent 718 N Main St London, KY
Dr. Doug Bennett, Superintendent 718 N Main St London, KY 40741-1222 Document Generated On January 13, 2014 TABLE OF CONTENTS Introduction 1 Description of the School System 2 System's Purpose 4 Notable
More informationDATE ISSUED: 11/2/ of 12 UPDATE 103 EHBE(LEGAL)-P
TITLE III REQUIREMENTS STATE POLICY DEFINITIONS DISTRICT RESPONSIBILITY IDENTIFICATION OF LEP STUDENTS A district that receives funds under Title III of the No Child Left Behind Act shall comply with the
More informationThe Future of Consortia among Indian Libraries - FORSA Consortium as Forerunner?
Library and Information Services in Astronomy IV July 2-5, 2002, Prague, Czech Republic B. Corbin, E. Bryson, and M. Wolf (eds) The Future of Consortia among Indian Libraries - FORSA Consortium as Forerunner?
More informationSoftware Maintenance
1 What is Software Maintenance? Software Maintenance is a very broad activity that includes error corrections, enhancements of capabilities, deletion of obsolete capabilities, and optimization. 2 Categories
More informationStrategic Practice: Career Practitioner Case Study
Strategic Practice: Career Practitioner Case Study heidi Lund 1 Interpersonal conflict has one of the most negative impacts on today s workplaces. It reduces productivity, increases gossip, and I believe
More informationAssessment System for M.S. in Health Professions Education (rev. 4/2011)
Assessment System for M.S. in Health Professions Education (rev. 4/2011) Health professions education programs - Conceptual framework The University of Rochester interdisciplinary program in Health Professions
More informationOklahoma State University Policy and Procedures
Oklahoma State University Policy and Procedures REAPPOINTMENT, PROMOTION AND TENURE PROCESS FOR RANKED FACULTY 2-0902 ACADEMIC AFFAIRS September 2015 PURPOSE The purpose of this policy and procedures letter
More informationCONNECTICUT GUIDELINES FOR EDUCATOR EVALUATION. Connecticut State Department of Education
CONNECTICUT GUIDELINES FOR EDUCATOR EVALUATION Connecticut State Department of Education October 2017 Preface Connecticut s educators are committed to ensuring that students develop the skills and acquire
More informationWE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT
WE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT PRACTICAL APPLICATIONS OF RANDOM SAMPLING IN ediscovery By Matthew Verga, J.D. INTRODUCTION Anyone who spends ample time working
More informationShort vs. Extended Answer Questions in Computer Science Exams
Short vs. Extended Answer Questions in Computer Science Exams Alejandro Salinger Opportunities and New Directions April 26 th, 2012 ajsalinger@uwaterloo.ca Computer Science Written Exams Many choices of
More informationDocument number: 2013/ Programs Committee 6/2014 (July) Agenda Item 42.0 Bachelor of Engineering with Honours in Software Engineering
Document number: 2013/0006139 Programs Committee 6/2014 (July) Agenda Item 42.0 Bachelor of Engineering with Honours in Software Engineering Program Learning Outcomes Threshold Learning Outcomes for Engineering
More informationSSIS SEL Edition Overview Fall 2017
Image by Photographer s Name (Credit in black type) or Image by Photographer s Name (Credit in white type) Use of the new SSIS-SEL Edition for Screening, Assessing, Intervention Planning, and Progress
More informationCollege of Engineering and Applied Science Department of Computer Science
College of Engineering and Applied Science Department of Computer Science Guidelines for Doctor of Philosophy in Engineering Focus Area: Security Last Updated April 2017 I. INTRODUCTION The College of
More informationA Pilot Study on Pearson s Interactive Science 2011 Program
Final Report A Pilot Study on Pearson s Interactive Science 2011 Program Prepared by: Danielle DuBose, Research Associate Miriam Resendez, Senior Researcher Dr. Mariam Azin, President Submitted on August
More informationA Guide to Adequate Yearly Progress Analyses in Nevada 2007 Nevada Department of Education
A Guide to Adequate Yearly Progress Analyses in Nevada 2007 Nevada Department of Education Note: Additional information regarding AYP Results from 2003 through 2007 including a listing of each individual
More informationActivities, Exercises, Assignments Copyright 2009 Cem Kaner 1
Patterns of activities, iti exercises and assignments Workshop on Teaching Software Testing January 31, 2009 Cem Kaner, J.D., Ph.D. kaner@kaner.com Professor of Software Engineering Florida Institute of
More informationLoyola University Chicago Chicago, Illinois
Loyola University Chicago Chicago, Illinois 2010 GRADUATE SECONDARY Teacher Preparation Program Design D The design of this program does not ensure adequate subject area preparation for secondary teacher
More informationLinguistics Program Outcomes Assessment 2012
Linguistics Program Outcomes Assessment 2012 BA in Linguistics / MA in Applied Linguistics Compiled by Siri Tuttle, Program Head The mission of the UAF Linguistics Program is to promote a broader understanding
More informationEFFECTS OF MATHEMATICS ACCELERATION ON ACHIEVEMENT, PERCEPTION, AND BEHAVIOR IN LOW- PERFORMING SECONDARY STUDENTS
EFFECTS OF MATHEMATICS ACCELERATION ON ACHIEVEMENT, PERCEPTION, AND BEHAVIOR IN LOW- PERFORMING SECONDARY STUDENTS Jennifer Head, Ed.S Math and Least Restrictive Environment Instructional Coach Department
More informationMathematical Misconceptions -- Can We Eliminate Them? Phi lip Swedosh and John Clark The University of Melbourne. Introduction
MERGA 20 -Aotearoa - 1997 Mathematical Misconceptions -- Can We Eliminate Them? Phi lip Swedosh and John Clark The University of Melbourne If students are to successfully tackle tertiary mathematics, one
More informationSTUDENTS' RATINGS ON TEACHER
STUDENTS' RATINGS ON TEACHER Faculty Member: CHEW TECK MENG IVAN Module: Activity Type: DATA STRUCTURES AND ALGORITHMS I CS1020 LABORATORY Class Size/Response Size/Response Rate : 21 / 14 / 66.67% Contact
More informationCurriculum and Assessment Policy
*Note: Much of policy heavily based on Assessment Policy of The International School Paris, an IB World School, with permission. Principles of assessment Why do we assess? How do we assess? Students not
More informationAviation English Training: How long Does it Take?
Aviation English Training: How long Does it Take? Elizabeth Mathews 2008 I am often asked, How long does it take to achieve ICAO Operational Level 4? Unfortunately, there is no quick and easy answer to
More informationDelaware Performance Appraisal System Building greater skills and knowledge for educators
Delaware Performance Appraisal System Building greater skills and knowledge for educators DPAS-II Guide for Administrators (Assistant Principals) Guide for Evaluating Assistant Principals Revised August
More informationEMPIRICAL RESEARCH ON THE ACCOUNTING AND FINANCE STUDENTS OPINION ABOUT THE PERSPECTIVE OF THEIR PROFESSIONAL TRAINING AND CAREER PROSPECTS
Persefoni Polychronidou Department of Accounting and Finance TEI of Central Macedonia, Serres, Greece E-mail: polychr@teicm.gr Stephanos Nikolaidis Department of Accounting and Finance TEI of East Macedonia
More informationDelaware Performance Appraisal System Building greater skills and knowledge for educators
Delaware Performance Appraisal System Building greater skills and knowledge for educators DPAS-II Guide (Revised) for Teachers Updated August 2017 Table of Contents I. Introduction to DPAS II Purpose of
More informationDISTRICT ASSESSMENT, EVALUATION & REPORTING GUIDELINES AND PROCEDURES
SCHOOL DISTRICT NO. 20 (KOOTENAY-COLUMBIA) DISTRICT ASSESSMENT, EVALUATION & REPORTING GUIDELINES AND PROCEDURES The purpose of the District Assessment, Evaluation & Reporting Guidelines and Procedures
More informationInside the mind of a learner
Inside the mind of a learner - Sampling experiences to enhance learning process INTRODUCTION Optimal experiences feed optimal performance. Research has demonstrated that engaging students in the learning
More informationWhat Is The National Survey Of Student Engagement (NSSE)?
National Survey of Student Engagement (NSSE) 2000 Results for Montclair State University What Is The National Survey Of Student Engagement (NSSE)? US News and World Reports Best College Survey is due next
More informationRules of Procedure for Approval of Law Schools
Rules of Procedure for Approval of Law Schools Table of Contents I. Scope and Authority...49 Rule 1: Scope and Purpose... 49 Rule 2: Council Responsibility and Authority with Regard to Accreditation Status...
More informationOrleans Central Supervisory Union
Orleans Central Supervisory Union Vermont Superintendent: Ron Paquette Primary contact: Ron Paquette* 1,142 students, prek-12, rural District Description Orleans Central Supervisory Union (OCSU) is the
More informationPractice Examination IREB
IREB Examination Requirements Engineering Advanced Level Elicitation and Consolidation Practice Examination Questionnaire: Set_EN_2013_Public_1.2 Syllabus: Version 1.0 Passed Failed Total number of points
More informationDevelopment of Multistage Tests based on Teacher Ratings
Development of Multistage Tests based on Teacher Ratings Stéphanie Berger 12, Jeannette Oostlander 1, Angela Verschoor 3, Theo Eggen 23 & Urs Moser 1 1 Institute for Educational Evaluation, 2 Research
More informationRETURNING TEACHER REQUIRED TRAINING MODULE YE TRANSCRIPT
RETURNING TEACHER REQUIRED TRAINING MODULE YE Slide 1. The Dynamic Learning Maps Alternate Assessments are designed to measure what students with significant cognitive disabilities know and can do in relation
More informationBASIC EDUCATION IN GHANA IN THE POST-REFORM PERIOD
BASIC EDUCATION IN GHANA IN THE POST-REFORM PERIOD By Abena D. Oduro Centre for Policy Analysis Accra November, 2000 Please do not Quote, Comments Welcome. ABSTRACT This paper reviews the first stage of
More informationU VA THE CHANGING FACE OF UVA STUDENTS: SSESSMENT. About The Study
About The Study U VA SSESSMENT In 6, the University of Virginia Office of Institutional Assessment and Studies undertook a study to describe how first-year students have changed over the past four decades.
More informationCONSTRUCTION OF AN ACHIEVEMENT TEST Introduction One of the important duties of a teacher is to observe the student in the classroom, laboratory and
CONSTRUCTION OF AN ACHIEVEMENT TEST Introduction One of the important duties of a teacher is to observe the student in the classroom, laboratory and in other settings. He may also make use of tests in
More informationEvidence for Reliability, Validity and Learning Effectiveness
PEARSON EDUCATION Evidence for Reliability, Validity and Learning Effectiveness Introduction Pearson Knowledge Technologies has conducted a large number and wide variety of reliability and validity studies
More informationEssentials of Ability Testing. Joni Lakin Assistant Professor Educational Foundations, Leadership, and Technology
Essentials of Ability Testing Joni Lakin Assistant Professor Educational Foundations, Leadership, and Technology Basic Topics Why do we administer ability tests? What do ability tests measure? How are
More informationDRAFT VERSION 2, 02/24/12
DRAFT VERSION 2, 02/24/12 Incentive-Based Budget Model Pilot Project for Academic Master s Program Tuition (Optional) CURRENT The core of support for the university s instructional mission has historically
More informationCommittee on Academic Policy and Issues (CAPI) Marquette University. Annual Report, Academic Year
Committee Description: Committee on Academic Policy and Issues (CAPI) Marquette University Annual Report, Academic Year 2013-2014 The Committee on Academic Policies and Issues (CAPI) pursues long-range
More informationSpecification and Evaluation of Machine Translation Toy Systems - Criteria for laboratory assignments
Specification and Evaluation of Machine Translation Toy Systems - Criteria for laboratory assignments Cristina Vertan, Walther v. Hahn University of Hamburg, Natural Language Systems Division Hamburg,
More informationCHAPTER 4: REIMBURSEMENT STRATEGIES 24
CHAPTER 4: REIMBURSEMENT STRATEGIES 24 INTRODUCTION Once state level policymakers have decided to implement and pay for CSR, one issue they face is simply how to calculate the reimbursements to districts
More information22/07/10. Last amended. Date: 22 July Preamble
03-1 Please note that this document is a non-binding convenience translation. Only the German version of the document entitled "Studien- und Prüfungsordnung der Juristischen Fakultät der Universität Heidelberg
More informationState Budget Update February 2016
State Budget Update February 2016 2016-17 BUDGET TRAILER BILL SUMMARY The Budget Trailer Bill Language is the implementing statute needed to effectuate the proposals in the annual Budget Bill. The Governor
More informationKelso School District and Kelso Education Association Teacher Evaluation Process (TPEP)
Kelso School District and Kelso Education Association 2015-2017 Teacher Evaluation Process (TPEP) Kelso School District and Kelso Education Association 2015-2017 Teacher Evaluation Process (TPEP) TABLE
More informationVisit us at:
White Paper Integrating Six Sigma and Software Testing Process for Removal of Wastage & Optimizing Resource Utilization 24 October 2013 With resources working for extended hours and in a pressurized environment,
More informationStudents Understanding of Graphical Vector Addition in One and Two Dimensions
Eurasian J. Phys. Chem. Educ., 3(2):102-111, 2011 journal homepage: http://www.eurasianjournals.com/index.php/ejpce Students Understanding of Graphical Vector Addition in One and Two Dimensions Umporn
More informationEarly Warning System Implementation Guide
Linking Research and Resources for Better High Schools betterhighschools.org September 2010 Early Warning System Implementation Guide For use with the National High School Center s Early Warning System
More informationAssessment and Evaluation for Student Performance Improvement. I. Evaluation of Instructional Programs for Performance Improvement
Assessment and Evaluation for Student Performance Improvement I. Evaluation of Instructional Programs for Performance Improvement The ongoing evaluation of educational programs is essential for improvement
More informationThe Timer-Game: A Variable Interval Contingency for the Management of Out-of-Seat Behavior
MONTROSE M. WOLF EDWARD L. HANLEY LOUISE A. KING JOSEPH LACHOWICZ DAVID K. GILES The Timer-Game: A Variable Interval Contingency for the Management of Out-of-Seat Behavior Abstract: The timer-game was
More informationUnit 3. Design Activity. Overview. Purpose. Profile
Unit 3 Design Activity Overview Purpose The purpose of the Design Activity unit is to provide students with experience designing a communications product. Students will develop capability with the design
More informationBEFORE THE ARBITRATOR. In the matter of the arbitration of a dispute between ADMINISTRATORS' AND SUPERVISORS' COUNCIL. And
BEFORE THE ARBITRATOR In the matter of the arbitration of a dispute between ADMINISTRATORS' AND SUPERVISORS' COUNCIL And MILWAUKEE BOARD OF SCHOOL DIRECTORS Case 428 No. 64078 Rosana Mateo-Benishek Demotion
More informationReferencing the Danish Qualifications Framework for Lifelong Learning to the European Qualifications Framework
Referencing the Danish Qualifications for Lifelong Learning to the European Qualifications Referencing the Danish Qualifications for Lifelong Learning to the European Qualifications 2011 Referencing the
More informationData Structures and Algorithms
CS 3114 Data Structures and Algorithms 1 Trinity College Library Univ. of Dublin Instructor and Course Information 2 William D McQuain Email: Office: Office Hours: wmcquain@cs.vt.edu 634 McBryde Hall see
More informationKansas Adequate Yearly Progress (AYP) Revised Guidance
Kansas State Department of Education Kansas Adequate Yearly Progress (AYP) Revised Guidance Based on Elementary & Secondary Education Act, No Child Left Behind (P.L. 107-110) Revised May 2010 Revised May
More informationCritical Thinking in Everyday Life: 9 Strategies
Critical Thinking in Everyday Life: 9 Strategies Most of us are not what we could be. We are less. We have great capacity. But most of it is dormant; most is undeveloped. Improvement in thinking is like
More informationPROGRAM HANDBOOK. for the ACCREDITATION OF INSTRUMENT CALIBRATION LABORATORIES. by the HEALTH PHYSICS SOCIETY
REVISION 1 was approved by the HPS BOD on 7/15/2004 Page 1 of 14 PROGRAM HANDBOOK for the ACCREDITATION OF INSTRUMENT CALIBRATION LABORATORIES by the HEALTH PHYSICS SOCIETY 1 REVISION 1 was approved by
More informationPROVIDING AND COMMUNICATING CLEAR LEARNING GOALS. Celebrating Success THE MARZANO COMPENDIUM OF INSTRUCTIONAL STRATEGIES
PROVIDING AND COMMUNICATING CLEAR LEARNING GOALS Celebrating Success THE MARZANO COMPENDIUM OF INSTRUCTIONAL STRATEGIES Celebrating Success Copyright 2016 by Marzano Research Materials appearing here are
More informationQUESTIONS ABOUT ACCESSING THE HANDOUTS AND THE POWERPOINT
Answers to Questions Posed During Pearson aimsweb Webinar: Special Education Leads: Quality IEPs and Progress Monitoring Using Curriculum-Based Measurement (CBM) Mark R. Shinn, Ph.D. QUESTIONS ABOUT ACCESSING
More informationConceptual Framework: Presentation
Meeting: Meeting Location: International Public Sector Accounting Standards Board New York, USA Meeting Date: December 3 6, 2012 Agenda Item 2B For: Approval Discussion Information Objective(s) of Agenda
More informationAccountability in the Netherlands
Accountability in the Netherlands Anton Béguin Cambridge, 19 October 2009 2 Ideal: Unobtrusive indicators of quality 3 Accountability System level international assessments National assessments School
More informationThe My Class Activities Instrument as Used in Saturday Enrichment Program Evaluation
Running Head: MY CLASS ACTIVITIES My Class Activities 1 The My Class Activities Instrument as Used in Saturday Enrichment Program Evaluation Nielsen Pereira Purdue University Scott J. Peters University
More informationWorkload Policy Department of Art and Art History Revised 5/2/2007
Workload Policy Department of Art and Art History Revised 5/2/2007 Workload expectations for faculty in the Department of Art and Art History, in the areas of teaching, research, and service, must be consistent
More informationLesson M4. page 1 of 2
Lesson M4 page 1 of 2 Miniature Gulf Coast Project Math TEKS Objectives 111.22 6b.1 (A) apply mathematics to problems arising in everyday life, society, and the workplace; 6b.1 (C) select tools, including
More informationVolunteer State Community College Strategic Plan,
Volunteer State Community College Strategic Plan, 2005-2010 Mission: Volunteer State Community College is a public, comprehensive community college offering associate degrees, certificates, continuing
More informationWHAT ARE VIRTUAL MANIPULATIVES?
by SCOTT PIERSON AA, Community College of the Air Force, 1992 BS, Eastern Connecticut State University, 2010 A VIRTUAL MANIPULATIVES PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR TECHNOLOGY
More informationReference to Tenure track faculty in this document includes tenured faculty, unless otherwise noted.
PHILOSOPHY DEPARTMENT FACULTY DEVELOPMENT and EVALUATION MANUAL Approved by Philosophy Department April 14, 2011 Approved by the Office of the Provost June 30, 2011 The Department of Philosophy Faculty
More informationA Note on Structuring Employability Skills for Accounting Students
A Note on Structuring Employability Skills for Accounting Students Jon Warwick and Anna Howard School of Business, London South Bank University Correspondence Address Jon Warwick, School of Business, London
More informationApplying Florida s Planning and Problem-Solving Process (Using RtI Data) in Virtual Settings
Applying Florida s Planning and Problem-Solving Process (Using RtI Data) in Virtual Settings As Florida s educational system continues to engage in systemic reform resulting in integrated efforts toward
More informationCONTINUUM OF SPECIAL EDUCATION SERVICES FOR SCHOOL AGE STUDENTS
CONTINUUM OF SPECIAL EDUCATION SERVICES FOR SCHOOL AGE STUDENTS No. 18 (replaces IB 2008-21) April 2012 In 2008, the State Education Department (SED) issued a guidance document to the field regarding the
More informationUnit 13 Assessment in Language Teaching. Welcome
Unit 13 Assessment in Language Teaching Welcome Teaching Objectives 1. Assessment purposes 2. Assessment methods 3. Assessment criteria 4. Assessment principles 5. Testing in language assessment 2 I. Assessment
More informationA Context-Driven Use Case Creation Process for Specifying Automotive Driver Assistance Systems
A Context-Driven Use Case Creation Process for Specifying Automotive Driver Assistance Systems Hannes Omasreiter, Eduard Metzker DaimlerChrysler AG Research Information and Communication Postfach 23 60
More information