Language Testing. Types of Tests

Transcription

1 Language Testing Types of Tests

2 Why do we want to test? Assess Evaluate Test

3 Assessment of people processes products environments

4 Types of tests (test techniques): What comes to mind first? Multiple choice True-false Fill-in-the-blank Short answer Essay questions Dictation Cloze and C-Test Summary

5 What other types of tests can you think of? Listening comprehension test Composition test Oral interview test Reading comprehension test Translation Guided composition, summary

6 Tests are one form of assessment, but other forms include: Observations of language performance Portfolio assessment Peer assessment Self assessment

7 Qualitative Assessment Traditional - Oral interview - Essay/composition - Translation

8 Portfolio assessment: - Personal context - Learner empowerment - Life-long learning - Individual record of achievement, progress - criticism (creative and repetitive)

9 Flexible assessor: - self-assessment - peer-assessment - other-assessment

10 Testing and assessment fundamentally involve: Collecting learner data Analyzing learner data Using the learner results of the data analysis to make interpretations about learners language abilities

11 Thus, types of tests are really types of learner data

12 Most common types of learner data Observations of spontaneous language behavior, e.g. in the classroom Elicited experimental (or highly structured and controlled) data Elicited clinical (or unstructured) data Elicited metalingual judgments Self report data, e.g. learner diaries

13 What s the best kind of test? = What s the best kind of data? All types of data are useful, but none tell the whole story The more types of data you use, the more confident you can be in your interpretations

14 Test types can also be distinguished according to their purposes Research Diagnosis Placement, including entrance tests Achievement Proficiency Aptitude

15 Classic goal-based test categories Proficiency test: - relative to a given standard, often a qualification test - frequently very large scale

16 Classic goal-based test categories The test itself should be tested in terms of reliability: does it achieve consistent results on various occasions validity: does it test the level and abilities which it purports to test objectivity: do different testers come to the same conclusions face validity is desirable (but not necessary)

17 Achievement test - relative to teaching goals - sometimes very large scale, sometimes very small scale - also tested in terms of reliability and validity

18 Diagnostic test: - aims to give feedback on performance - based on error detection and error analysis, e.g. grammar vs. vocabulary; or: the verb phrase vs. the noun phrase

19 Aptitude Tests Aptitude tests are structured, systematic ways of evaluating how people perform on tasks or react to different situations Attempt to predict possible success, e.g. the America SAT (Scholastic Aptitude Test) which is used as an entrance level qualification for college/university studies

20 Aptitude Tests They are characterized by standardized methods of administration and scoring with the results quantified and compared with how others have done on the same tests

21 Classic form-based test categories Written: - objective test (results automatically gradable) - multiple choice test - cloze (gap-filling) test - guided test - translation - guided essay

22 - free test - open answer - essay

23 Oral: - interview - recording-based - transcription - repetition - response (Q+A, gap-filling,...)

24 Multiple choice Parameters: questions vs. incomplete statements Parameter values: - a subset of possible parameter values (answers) is provided - one is correct; the others are "distractors

25 What causes night and day? A. The earth spins on its axis. B. The earth moves around the sun. C. Clouds block out the sun's light. D. The earth moves into and out of the sun's shadow. E. The sun goes around the earth. (Source: P. M. Sadler, "Psychometric Models of Student Conceptions in Science," Journal of Research in Science Teaching (1998. V. 35, N. 3, pp ).) The correct answer is A ; the other answers are socalled distractors.

26 If the distractors are (in some sense) equally plausible and randomly ordered, then - the probability of obtaining a correct answer by guessing is 1/n, where n is the number of values provided (i.e. the correct answer plus distractors) - thus: if there are 4 equally plausible answers, the probability of guessing correctly is ¼ = 0.25.

27 Multiple choice: example Example: The phonemic transcription of sourdough in Southern Educated British English is 1. /so rdu / 2. /sa do / 3. /sa d / 4. /sa rd f/ The probability of guessing the correct transcription of sourdough is 1/4 = 0.25

28 Multiple choice: pros and cons... Pro: reasonably easy for the teacher to construct easy for the teacher to correct easy to evaluate statistically easy for the student to understand the procedure

29 ...Multiple choice: pros and cons Con: tests written knowledge only (unless acoustic media are used) tests receptive knowledge only tests metalinguistic knowledge rather than performance if the distractors are not carefully selected, the probability of random guessing may be high - tends to test knowledge rather than understanding

30 Cloze Tests... A gap-completion task Written Many different kinds of gap possible: letters parts of words (morphs) words word sequences

31 ...Cloze Tests Gap selection procedure may be random or systematic - e.g. gaps every 4-10 words - hard words maybe filtered out - or maybe specific word types are filtered manual or computerized

32 Modified cloze (UBI Entrance Test) Ten years ago representatives from 178 nations met in Rio to plan how to protect the world's resources. Pledges we given t safeguard ecosy, reduce glo -warming ga, and pro humanity thr sustainable devel. Last ye world lea, scientists a activists m again. The agenda: to check whether Rio had changed the world.

33 What is in a test? Some criteria Technical: - comparison of two performances Personal: - interaction between examiner and examinee Institutional: - qualification: license to participate in certain activities

34 Psychological: - performance under highly constrained conditions Linguistic: - Language understanding and language production - Metalinguistic activity

35 Finally, test types can be distinguished by scoring approach Norm-referenced Criterion-referenced

36 Norm-referenced versus Criterion-referenced Norm-referenced tests are scored according to how well each person does in relation to the mean (or average) score on the test, e.g. national standard reading tests in the U.S. Criterion-referenced tests are scored according to predetermined criteria, so that a person s score is not affected by how well everyone else does on the test.

37 Norm-referenced versus Criterion-referenced Proficiency tests tend to be normreferenced. Most other types of tests tend to be criterion-referenced.

38 Testers do not assume that a test score is a person s true score There is always a margin of error in a test, so the true score (the person s true ability) may be somewhat higher or lower than the test score. The amount that the true score may differ from the test score is calculated in relation to the test s overall reliability (standard deviation).

39 Mastery of Reliability Expert testers and testing companies have mastered the art of test reliability. However, the more important question of test validity remains a challenge for everyone, both novice and expert.

40 Analogies How would you test whether a person knows the capital cities of all 50 states? How would you test a person s ability to play tennis? How would you test whether a person can tie his/her shoe? How would you test whether a person can build a cabinet?