Modified Cloze Procedure: A More Valid Reading Comprehension Test

Transcription

1 Modified Cloze Procedure 5 answers appear, made the preparation less of a chore. Though not strictly speaking, an 'explication de texte', these two methods certainly provoked on the part of the students an investigation of texts which had the wholesome effect of making them analyse the meaning of a piece of prose and not hazard interpretations of it Modified Cloze Procedure: A More Valid Reading Comprehension Test D. PORTER Department of English, University of Warsaw A cloze procedure test consists of a passage from which every nth word has been deleted. No deletions are made in the first two or three lines, to allow the theme of the passage to become established. The student's task is to fill the blanks appropriately. Since its birth just over twenty years ago (Taylor 95), varieties of cloze procedure (CP) have been used to measure reading comprehension, language proficiency, knowledge of vocabulary, the relative difficulty of passages of prose, and even the reader's I.Q.; and aural cloze tests have been proposed as measures of listening comprehension (see Oiler and Conrad 97). Although originally concerned with the mother tongue, more recently the procedure has been frequently considered as a measuring technique in the foreign-language-learning situation (e.g. Anderson 97 and 97; Levine 97; Oiler, Bowen, Dien, and Mason 97; Oiler and Conrad 97; Spolsky 969). In this article I shall consider the use of CP to test reading comprehension in the foreign-language-learning situation, and shall propose a modification which should make for a more valid test Virtues of the CP reading comprehension test A CP test is easily and quickly constructed, and probably involves less effort than any other type of test. It is likewise easily marked: although subtler and more complex scoring schemes have been devised, the simplest to treat only 'verbatim' answers as correct (i.e. those identical with the word originally deleted) and all others as wrong has been found to yield the same results as the Downloaded from on 0 April 08

2 5 D. Porter more complex schemes: students are ranked in the same way for reading comprehension, and passages are ranked in the same way for reading difficulty for a given group (Anderson 97, but see Oiler 97 for reservations). From a more theoretical point of view, insofar as the student is required to predict the missing words, he is being tested on his ability to continuously draw from the context clues to the nature of the language immediately ahead: this ability is a measure of reading fluency, and what we may call the 'creative' aspect of the native speaker's reading process. The fact that the ability to predict what lies ahead depends on the ability fully to comprehend the language being 'processed' at any given moment provides the justification for CP as a test of comprehension. Full comprehension necessarily involves more than the understanding of lexical items alone: comprehension of the grammatical structures used is also implied, together with a grasp of the subtle relationships obtaining between the structure and the lexis. Since efficient clue-gathering and prediction draws on all of this, CP tests tap comprehension at all levels of language, and are consequently an example of an integrated approach. The automatic regularity of the deletion-process means that a variety of types of language-item are tested, unrestricted by the subjective decisions of the test-writer. Moreover, the sample of test-items provides a measure of the level of comprehension spread evenly throughout the passage. Weakness of the CP reading comprehensior test The CP test nevertheless attracts the important criticism that it simply does not measure what its promoters say it does. Although comprehension of the passage is necessary to the successful completion of the CP test, it is not sufficient. The student must in addition be able to produce the language. It is true to say that the native speaker's language-ability involves both aspects, but this is not the point at issue. To be valid, a test should measure what it is intended to measure, and language production is not comprehension; neither are the two necessarily concomitant. All of us, as language-teachers, are familiar with the learner who can understand, but who is not sufficiently master of the language to make a reply. Similarly, although I have never studied Italian, I can get much of the meaning from a newspaper in that language; but I would be unable to predict a word at any point. Thus in my case and in the case of the dumb student it would be misleading to suggest that the CP test was measuring comprehension. It may be argued that only low-grade comprehension is Downloaded from on 0 April 08

3 Modified Cloze Procedure 5 involved here. This may be so, but it is a very important kind of comprehension. For large groups of students (e.g. students of science in many countries) comprehension of this kind is the first and sometimes the only language-priority. Without going into the question of whether such students ought also to be able to produce the language, I wish here to stress both the need for, and the theoretical desirability of, a procedure which will test comprehension pure and simple, without the obtrusion of other language-abilities. The modified cloze procedure (MCP) The question, then, is how to devise a method of testing comprehension which has the advantages of CP, which works in basically the same way, yet which does not necessitate language production. The solution which best meets these requirements is to supply a set of words for each gap in other words, to marry the CP to the multiple-choice test. Certainly, the resulting modified CP is slightly more difficult to construct than the simple CP: the problems it poses in the selection of alternative words are of the same sort as those facing the composer of the multiple-choice test. On the other hand, in addition to the validity of the new technique, the process of selecting alternatives introduces an element of control and flexibility which the original model did not have. (See below.) The construction of the MCP begins with the construction of a normal CP test A passage is chosen not specially written, to avoid the possibility of unnatural 'classroom' language creeping in which is appropriate to the interests and ability of the students. Leaving a short introductory section, e.g. twenty words, every nth word is deleted. Whereas the traditional CP would now be complete, and any further sophistication would depend on the scoring system, the second stage in the construction of the MCP requires that we supply the deleted word together with three (or more) distractors. The student then chooses from these the word which is felt to fit the context best. Only the originally deleted word need be accepted as a correct answer. Drawbacks and virtues of the MCP The MCP comprehension test has the virtues of the original CP, except that it has the drawbacks and virtues of the multiplechoice test in addition. The completely open choice in the CP is replaced in the MCP by the restriction to a set of (e.g.) four alternative choices. There is therefore a possibility that the student may recognise the appropriate word when he sees it, although he might not have been able to give the word if the suggestions Downloaded from on 0 April 08

4 5 D. Porter had not been made. Nevertheless, such recognition still depends on the student having understood the previous text The fact that the student is given a set of words to choose from lends an aspect of flexibility and control to the MCP. The distractors chosen may be varied according to the depth of linguistic attainment and fineness of stylistic discrimination of the student. At an elementary level, the incorrect alternatives may all be quite unsuitable, both in meaning and grammaticality. At a more advanced level they may be grammatically but not semantically appropriate, or vice versa. At a fairly high level, distractors may ah be grammatically suitable, the student being forced to make quite sophisticated choices among vocabulary items, etc. Following this line of thought further, one might offer distractors which are all possible, both grammatically and semantically, but which vary in probability in the particular register of the passage: e.g. in a setting of colloquial speech: This (person/gentieman/ca^/individual) comes up to me and he says...'. The student would in such cases not be asked to choose the correct answer, but the most probable item, given the context. Another factor which might affect the choice of alternatives would be the kinds of mistake which the group of students concerned habitually make. Provision could be made for them to make the mistake in the test through their choice of an incorrect alternative. However, I would recommend the exclusion of this factor in the choosing of distractors, on the grounds that it is concerned with defects in language-productive ability, and as such not directly concerned in comprehension. The number of words between deletions will be governed by the level of difficulty desired, but should not be below five or higher than ten. The passage should be at least two hundred words long, to ensure a reasonable number of test-items and thus fair reliability. A final virtue of MCP is that it can be constructed so that a card with holes punched in appropriate places can be placed over the text to allow very rapid marking of the kind used with normal multiple-choice tests. The following short example of an MCP reading comprehension test is arranged in this way. Example Read the following passage. ONE word in each bracket is correct Mark the number of the word with a cross on theright-handside of the page, like this: Alice looked round. There isn't any,' she (said/talked/believed/idea). lx Downloaded from on 0 April 08

5 Modified Cloze Procedure 55 The tea-party There was a table set out under a tree in front of the house, and the March Hare and the Hatter were having tea at (them/ it/once/her): a Dormouse was sitting between them, fast (runner/quict/eating asleep), and the other two were resting their (selves/comfortably/elbows/hand) on it, and talking over its head. '(Very/Much/Heit) uncomfortable for the Dormouse,' thought Alice; 'only, (although/ as/but/see) it's asleep, I suppose it doesn't mind.' (The/A/On/Although) table was a large one, but the (Hatter/table/three/elbows) were all crowded together at one corner (with/of/in/at) it 'No room! No room!* they cried (out/big/to/at) when they saw Alice coming. There's plenty (the/of/table/ from) room!' said Alice indignantly, and she sat down in a large arm-chair at one end of the table. References Anderson, J. (97) 'A technique for measuring reading comprehension and readability', E.L.T., XXV,. (97) The application of cloze procedure to English learned as a foreign language in Papua and New Guinea', EX.T., XXVII,. Levine, H. F. (97) 'Linguistic and paralinguistic changes in Spanish speakers learning English", EX.T., XXV,. Oiler, J. W., Jr. (97) 'Scoring methods and difficulty levels for cloze tests of proficiency in English as a Second Language', Modern Language Journal, LVI,., J. D. Bowen, T. T. Dien, and V. W. Mason (97) 'Cloze tests in English, Thai, and Vietnamese: native and non-native performance', Language Learning, Vol.,. and C. A. Conrad (97) The cloze technique and ESL proficiency', Language Learning, Vol.,. Spolsky, B. (969) 'Reduced redundancy as a language-testing tool', paper presented at the Second International Congress of Applied linguistics, Cambridge. Taylor, N. L. (95) 'Cloze procedure: a new technique for measuring readability', Journalism Quarterly,. Downloaded from on 0 April 08