SCHOOL ACHIEVEMENT INDICATORS PROGRAM OF THE COUNCIL OF MINISTERS OF EDUCATION, CANADA QUÉBEC RESULTS IN THE 1998 READING AND WRITING ASSESSMENTS

Transcription

1 SCHOOL ACHIEVEMENT INDICATORS PROGRAM OF THE COUNCIL OF MINISTERS OF EDUCATION, CANADA QUÉBEC RESULTS IN THE 1998 READING AND WRITING ASSESSMENTS

2 Gouvernement du Québec Ministère de l'éducation, 1999 ISBN Legal Deposit Bibliothèque nationale du Québec, 1999

3 Table of Contents 1 Introduction Context of the Study Target Groups Sampling Procedure Description of the Samples of Québec Students Description of the Assessments Description of the Performance Levels Presentation of Results Results of Québec Francophone Students Results of Québec Francophone Students in the Reading Assessment Results of Québec Francophone Students in the Writing Assessment Overview of the Results of Québec Francophone Students Comparison of the 1994 and 1998 Results Comparison of the Results of Francophone Students in Québec with Those in the Other Provinces Results of Francophone Students in the Reading Assessment Results of Francophone Students in the Writing Assessment Overview of the Results of Francophone Students across Canada Comparison of the 1994 and 1998 Results of Francophone Students across Canada Results of Québec Anglophone Students Results of Québec Anglophone Students in the Reading Assessment Results of Québec Anglophone Students in the Writing Assessment Overview of the Results of Québec Anglophone Students Comparison of the 1994 and 1998 Results of Québec Anglophone Students... 18

4 5 Comparison of the Results of Anglophone Students in Québec with Those in the Other Provinces Results of Anglophone Students in the Reading Assessment Results of Anglophone Students in the Writing Assessment Overview of the Results of Anglophone Students across Canada Comparison of the 1994 and 1998 Results of Anglophone Students across Canada Québec Results According to Language Group Highlights Appendices Appendix A: Reading Assessment Criteria Appendix B: Writing Assessment Criteria Appendix C: Percentage of 13-Year-Olds and 16-Year-Olds at Each Level in the 1994 Reading and Writing Assessments... 33

5 1 Introduction 1.1 Context of the Study The provinces of Canada have given the Council of Ministers of Education, Canada (CMEC) a mandate to implement the School Achievement Indicators Program (SAIP) to assess students performance in mathematics, reading and writing, and science. In April 1998, the reading and writing achievement of students from all the provinces and territories of Canada was assessed. 1.2 Target Groups The reading and writing assessments were administered to students who were 13 and 16 years old on August 31, The group of 16-year-olds consisted mainly of students in their last year of compulsory school attendance. The group of 13-year-olds consisted of students from most of the provinces education systems. They were in their first year of secondary school, which is the transition year between elementary and secondary school. 1.3 Sampling Procedure The samples of 13- and 16-year-olds were selected using a two-stage procedure. The CMEC was responsible for managing the first stage, the aim of which was to select the schools. The provinces and territories were asked to provide the CMEC with a list of all schools with 13- and 16-year-old students. They were also asked to give the number of 13- year-olds (born between September 1, 1983 and August 31, 1984) and the number of 16-yearolds (born between September 1, 1980 and August 31, 1981) for each school. Schools under federal jurisdiction and those with fewer than five students were excluded. Then the schools were selected using a procedure that took their size into account. Each province and territory was responsible for the second stage, the aim of which was to select students using a uniform procedure that obliged the coordinators for the provinces and territories to accord equal probability of selection to all 13- and 16-year-olds. No allowances for exclusion were made at this stage of the process. 1

6 1.4 Description of the Samples of Québec Students As in the other studies in which Québec has participated, different samples were constituted for anglophone and francophone students in each age group. Thus it is possible to analyze the data for each language group. Table 1 shows the size of the Québec samples in the study. More than and 16-year-olds took part. About 6 percent of the 16-year-olds were studying at the college level. In the case of both the 13- and 16-year-olds, the samples for the two components of the assessment are representative of the student population of Québec because they consist of students from the public and private systems as well from both official-language groups. Table 1 Number of Students Included in the Samples Samples Francophones Anglophones Total 13-year-olds Reading Writing year-olds Reading Writing Total number of students

7 1.5 Description of the Assessments The French and English versions of the assessments were designed by teams of language evaluation specialists from Québec, Alberta and Ontario in conjunction with specialists from the other provinces. The materials needed for the assessments were field-tested and the questions and tasks validated. Development of the reading and writing assessments began in December They were administered in April The results of the first pan-canadian assessment were published in December In 1997, a team of specialists from the Northwest Territories, Saskatchewan, Ontario, Québec, New Brunswick (francophone sector) and Nova Scotia (francophone sector) revised the assessments and prepared a second pan-canadian reading and writing assessment. Using statistics and results obtained in 1994, markers comments, the advice of evaluation specialists and samples of students responses, the team performed an exhaustive analysis of the assessments and evaluation procedures. It identified changes to be made to the assessments that would not affect a comparison between the 1994 and 1998 results. In 1998, as in 1994, reading and writing assessments were administered to students in their first language. The same assessments were administered to both 13- and 16-year-olds, thus making it possible to compare the two age groups. For the reading assessment, the students were presented with a booklet of excerpts from recognized youth literature and essays or newspaper articles. The pieces varied in length up to four pages and represented different genres and degrees of difficulty. After reading the material, students were asked to answer multiple-choice questions and to respond in writing to a specific question or perform a task. The writing assessment was structured so that students had an opportunity to read and make notes on texts on a specific theme (heroes), discuss their ideas with fellow students, and write and revise first drafts using standard reference books like dictionaries and thesauruses. The students had two and a half hours to complete their composition and answer a series of questions. 3

8 1.6 Description of the Performance Levels The overall results in reading and writing were expressed in terms of five performance levels. Level 1 corresponds to the learning expected of students in elementary school, whereas level 5 describes the performance expected of students who have a very good mastery of the language. Thus, it was expected that most students in the target groups for the assessment would easily attain performance level 1, and that very few 13-year-olds and few 16-year-olds would meet the criteria for level 5. The following factors were considered in establishing the five levels of reading achievement: the difficulty of the passages read, the classification and reading level of the multiple-choice questions, the students scores on the multiple-choice questions, the reading level of the various written-response questions and tasks, and the students responses to the written questions and tasks. The markers were asked to determine the highest level at which the student demonstrated consistent success by applying the criteria to all of the work produced. Predetermined criteria were used to establish the five levels of writing achievement. Markers used overall-impression scoring to assess the writing. In other words, they relied on their response to the writing as a whole to assign students compositions to the appropriate assessment level. 1.7 Presentation of Results The study makes it possible to place students in one of the five performance levels for the reading and writing assessments. It is assumed that a student who has been placed in a given level satisfies the requirements of the lower levels. Thus, a student who has attained level 3 is also assumed to have attained the requirements set for the first two performance levels. The description of each of the five performance levels in reading and writing can be found in appendices A and B. Although the same number of performance levels was used for both assessments, care must be taken when comparing the reading assessment results with those obtained in the writing assessment. These two assessments measure different aspects of language. Tables 2 to 9 show the percentage of Québec francophone students who attained each performance level in reading and writing, while tables 10 to 17 show the corresponding data for Québec anglophone students. All of these tables cover results obtained by 13- and 16- year-old girls and boys. The report does not give the overall performance of Québec students. 4

9 2 Results of Québec Francophone Students 2.1 Results of Québec Francophone Students in the Reading Assessment Table 2 Percentage of 13-Year-Old and 16-Year-Old Francophone Students at Each Performance Level Performance Levels Populations year-olds Girls Boys year-olds Girls Boys Results of Québec Francophone Students in the Writing Assessment Table 3 Percentage of 13-Year-Old and 16-year old Francophone Students at Each Performance Level Performance Levels Populations year-olds Girls Boys year-olds Girls Boys

10 2.3 Overview of the Results of Québec Francophone Students Results by Age Group Level 1 was attained by almost all 13- and 16-year-olds in both reading and writing. Level 2 was attained in reading by approximately 84 percent of 13-year-olds and almost 95 percent of 16-year-olds. In writing, level 2 was attained by more than 95 percent of 13-year-olds and by almost all 16-year-olds. There is, therefore, little difference between the results of the two age groups in writing. Level 3 clearly differentiates 13-year-olds from 16-year-olds. In both reading and writing, there was a difference of more than 20 points between the results of the two age groups. Almost six out of ten 13-year-olds attained this level in reading, while nearly eight out of ten 16-year-olds did so. In writing, almost seven out of ten 13- year-olds and almost nine out of ten 16-year-olds attained this level. Level 4 was attained by 15 percent of 13-year-olds in both reading and writing. A difference of approximately 25 points separates 13-year-olds and 16-year-olds in both reading (approximately 15 percent compared with 42 percent) and writing (approximately 15 percent compared with more than 40 percent). Level 5 in both reading and writing was attained by only a few 13-year-olds. Approximately 15 percent of 16-year-olds attained this level in reading, while less than 10 percent did so in writing. 6

11 Attainment of Higher Performance Levels It must be remembered that the same assessments were administered to 13-year-olds and 16- year olds. The performance levels include a broad array of language skills and can be used to describe learning acquired over a number of years of schooling. These assessments could, therefore, have been administered to older students. It can thus be assumed that more of these students would have attained a higher performance level. Moreover, it is understandable that very few younger students, like those 13-year-olds evaluated in this study, meet the requirements of levels 4 and 5. Nevertheless, a significant proportion of students attained level 4. Since the same assessments were administered to both age groups, it is possible to evaluate the learning achieved during the additional years of instruction. Looking at level 4, we see significant improvement in the 16-year-olds over the 13-year-olds. There are almost three times as many 16-year-olds as 13-year-olds at this level in both reading and writing. Results by Sex Girls in both age groups obtained better results than boys in both reading and writing. In reading, girls in both age groups scored significantly higher than boys at levels 2, 3 and 4. In writing, girls in both age groups scored considerably higher at levels 3 and 4. 7

12 2.4 Comparison of 1994 and 1998 Results Comparison of Results in the Reading Assessment Table and 1998 Results of Québec Francophone Students in Reading Populations Performance Levels Years year-olds Girls Boys <1 1 Years year-olds Girls Boys A comparison of results from the 1994 and 1998 reading assessments show no significant differences at any level between 13- and 16-year-old Québec francophones. Among 13-yearolds, the girls lead over the boys decreased at levels 1, 2 and 3 between 1994 and Among 16-year-olds, however, this lead increased at levels 3, 4 and 5 during the same years. 8

13 2.4.2 Comparison of Results in the Writing Assessment Table and 1998 Results of Québec Francophone Students in Writing Populations Performance Levels Years year-olds Girls Boys <1 1 Years year-olds Girls Boys In the 1998 writing assessment, more francophone students in both age groups attained levels 3 and 4 than in Among 13-year-olds, the girls lead over the boys decreased at levels 1 and 2 between 1994 and 1998, but increased at levels 3 and 4. Among 16-year-olds, this lead decreased at levels 2 and 3 between 1994 and 1998, but increased at level 4. 9

14 3 Comparison of the Results of Francophone Students in Québec with Those in the Other Provinces 3.1 Results of Francophone Students in the Reading Assessment Overall, Québec francophone students in both age groups obtained much better results in the reading assessment than francophones in the other provinces. Significant differences between the results of francophone students in Québec and those in the other provinces are found mainly at levels 2 and 3 among 13-year-olds, and 3 and 4 among 16-year-olds. Table 6 Percentage of Francophone Students at Each Level in Reading Populations Performance Levels year-olds % % % % % Manitoba Ontario <1 Québec New Brunswick Nova Scotia <1 Other Francophone Canada (average) year-olds % % % % % Manitoba Ontario Québec New Brunswick Nova Scotia Other Francophone Canada (average)

15 3.2 Results of Francophone Students in the Writing Assessment Overall, Québec francophone students in both age groups obtained much better results in the writing assessment than francophones in the other provinces. Differences between the results of francophone students in Québec and those in the other provinces are even more significant in writing than in reading. These differences are found mainly at levels 2, 3 and 4 among 13- year-olds, and at levels 3, 4 and 5 among 16-year-olds. Table 7 Percentage of Francophone Students at Each Level in Writing Populations Performance Levels year-olds % % % % % Manitoba Ontario Québec New Brunswick <1 Nova Scotia <1 <1 Other Canada (average) year-olds % % % % % Manitoba Ontario Québec New Brunswick Nova Scotia Other <1 Canada (average)

16 3.3 Overview of the Results of Francophone Students across Canada In reading, the greatest difference between the results of francophone students in Québec and those in the other provinces is found at level 3. In Québec, this level was attained by more than half of 13-year-olds and by more than three quarters of 16-year-olds. To attain level 3 in reading, students must demonstrate that they can interpret, evaluate and explore complex meanings in complex texts and some meaning from sophisticated texts (Appendix A). In writing, the greatest difference between the results of francophone students in Québec and those in the other provinces is also found at level 3. In Québec, this level was attained by more than two thirds of 13-year-olds and by almost nine out of ten 16-year-olds. To attain level 3 in writing, students must demonstrate a control of the elements of writing. The writing must generally be integrated, and development must be generalized, functional and usually maintained throughout. The writing must convey a clear perspective (Appendix B). 12

17 3.4 Comparison of the 1994 and 1998 Results of Francophone Students across Canada Comparison of Results in the Reading Assessment Table and 1998 Results in the Reading Assessment Populations Performance Levels year-olds Manitoba Ontario <1 Québec New Brunswick year-olds Manitoba Ontario Québec New Brunswick The average reading results obtained by francophone students across Canada in 1994 and 1998 cannot be compared, since Nova Scotia and the provinces in the other category did not participate in the 1994 reading assessment. If, however, the results of francophone students in provinces that participated in both assessments are compared, we see that New Brunswick students in both age groups showed the most marked improvement. 13

18 3.4.2 Comparison of Results in the Writing Assessment Table and 1998 Results in the Writing Assessment Populations Performance Levels year-olds Manitoba Ontario <1 0 Québec New Brunswick <1 <1 16-year-olds Manitoba Ontario Québec New Brunswick The average writing results obtained by francophone students across Canada in 1994 and 1998 cannot be compared, since Nova Scotia and the provinces in the other category did not participate in the 1994 writing assessment. If, however, the results of francophone students in provinces that participated in both assessments are compared, we see that the results of 13-year-old students in almost all the participating provinces increased at levels 2 and 3, while those of 16-year-olds increased at levels 2, 3 and 4. The results of 13-year-olds in Ontario (13 percent at level 3) and 16-year-olds in Manitoba (14 percent at level 3) show the greatest increase. 14

19 4 Results of Québec Anglophone Students 4.1 Results of Québec Anglophone Students in the Reading Assessment Table 10 Percentage of 13-Year-Old and 16-Year-Old Anglophone Students at Each Performance Level Populations Performance Levels year-olds Girls Boys year-olds Girls Boys Results of Québec Anglophone Students in the Writing Assessment Table 11 Percentage of 13-Year-Old and 16-Year-Old Anglophone Students at Each Performance Level Populations Performance Levels year-olds Girls Boys year-olds Girls Boys

20 4.3 Overview of the Results of Québec Anglophone Students Results by Age Group Level 1 was attained by almost all 13- and 16-year-olds in both reading and writing. Level 2 was attained in reading by approximately eight out of ten 13-year-olds and by almost all 16-year-olds. In writing, level 2 was attained by almost all 13- and 16- year olds. Level 3 clearly differentiates 13-year-olds from 16-year-olds. In reading, there is a difference of almost 30 points between the results of the two age groups; more than four out of ten 13-year-olds attained this level, while more than seven out of ten 16- year-olds did so. The difference in writing is less significant: approximately 74 percent of 13-year-olds and 88 percent of 16-year-olds attained this level. Level 4 was attained in reading by more than one out of ten 13-year-olds, and more than three out of ten 16-year-olds. In writing, a difference of more than 25 points separates the two age groups (approximately 17 percent compared with almost 44 percent). Level 5 in both reading and writing was attained by only a few 13-year-olds. Approximately 10 percent of 16-year-olds attained this level. 16

21 Attainment of Higher Performance Levels It must be remembered that the same assessments were administered to 13-year-olds and 16- year olds. The performance levels include a broad array of language skills and could be used to describe learning acquired over a number of years of schooling. These assessments could, therefore, have been administered to older students. It can thus be assumed that more of these students would have attained a higher performance level. Moreover, it is understandable that very few younger students, like those 13-year-olds evaluated in this study, meet the requirements of levels 4 and 5. Nevertheless, a significant proportion of students attained level 4. Since the same assessments were administered to both age groups, it is possible to evaluate the learning achieved during the additional years of instruction. Looking at level 4, we see significant improvement in the 16-year-olds over the 13-year-olds. There are almost three times as many 16-year-olds as 13-year-olds at this level in both reading and writing. Results by Sex Girls in both age groups obtained better results than boys in both reading and writing. In reading, 13-year-old girls scored significantly higher than boys at levels 2, 3 and 4, and 16- year-old girls scored significantly higher in levels 2 to 5. In writing, 13-year-old girls scored considerably higher than boys at levels 3 and 4, while 16-year-old girls scored significantly higher at levels 3, 4 and 5. These differences are less pronounced in writing than in reading. 17

22 4.4 Comparison of the 1994 and 1998 Results of Québec Anglophone Students Comparison of Results in the Reading Assessment Table and 1998 Results of Québec Anglophone Students in Reading Populations Performance Levels Years year-olds Girls Boys Years year-olds Girls Boys There are significant differences between the results obtained by 13-year-old Québec anglophone students in the 1994 and 1998 reading assessments. However, there are no significant differences at any level among francophone 16-year-olds. The lead enjoyed by sixteen-year-old girls over boys in the same age group increased by almost 10 percent at levels 3 and 4 between 1994 and The gap between 13-year-old girls and boys changed very little. 18

23 4.4.2 Comparison of Results in the Writing Assessment Table and 1998 Results of Québec Anglophone Students in Writing Populations Performance Levels Years year-olds Girls Boys Years year-olds Girls Boys In the writing assessment, more 13-year-old Québec anglophone students attained level 3 in 1998 than in 1994, and more 16-year-olds attained levels 3, 4 and 5. Among 13-year-olds, the difference between girls and boys scores changed very little from 1994 to Among 16-year-olds, however, the girls lead over the boys increased at level 5 but decreased at level 3. 19

24 5 Comparison of the Results of Anglophone Students in Québec with Those in the Other Provinces 5.1 Results of Anglophone Students in the Reading Assessment In the reading assessment, there was no significant difference in either age group between the results of anglophone students in Québec and those in the other provinces. In both age groups and at all reading performance levels, the results of Québec anglophone students were equal to or better than those of anglophone students in the other provinces. Table 14 Percentage of Anglophone Students at Each Level in Reading Populations 13-year-olds Performance Levels 16-year-olds British Columbia Alberta Saskatchewan < Manitoba < Ontario Québec New Brunswick Nova Scotia Prince Edward Island Newfoundland and Labrador < Northwest Territories < Yukon Anglophone Canada

25 5.2 Results of Anglophone Students in the Writing Assessment Comparison of the results of anglophone students in Québec with those in other provinces shows no significant differences in writing performance across age groups. The only lead over Québec anglophone students was registered in Alberta, where 13-year-olds scored higher at level 4. In all other cases, the results of Québec anglophone students in both age groups are equal to or better than those of anglophone students in the other provinces. And this applies to all writing performance levels. Table 15 Percentage of Anglophone Students at Each Level in Writing Populations 13-year-olds Performance Levels 16-year-olds British Columbia Alberta Saskatchewan Manitoba Ontario Québec New Brunswick Nova Scotia Prince Edward Island Newfoundland and Labrador Northwest Territories Yukon Anglophone Canada

26 5.3 Overview of the Results of Anglophone Students across Canada For both age groups and all performance levels, the results of Québec anglophone students in both assessments were, with one exception, equal to or better than the results of anglophone students in the other provinces. The exception was in Alberta, where 13-year-olds scored higher in the writing assessment than Québec anglophones in the same age group. In the reading assessment, a little less than half of 13-year-old anglophones and almost three quarters of 16-year-old anglophones attained level 3. At this level, students must demonstrate that they can interpret, evaluate and explore complex meanings in complex texts and some meaning from sophisticated texts (Appendix A). In the writing assessment, almost three quarters of 13-year-old anglophones and almost nine out of ten 16-year-old anglophones attained level 3. At this level, the students must demonstrate control over the elements of writing. The writing must generally be integrated, and development must be generalized, functional and usually maintained throughout. The writing must convey a clear perspective (Appendix B). 22

27 5.4 Comparison of the 1994 and 1998 Results of Anglophone Students across Canada Comparison of Results in the Reading Assessment Table and 1998 Results in the Reading Assessment Populations Performance Levels year-olds British Columbia Alberta Manitoba <1 Ontario Québec New Brunswick Prince Edward Island <1 1 Newfoundland and Labrador <1 Northwest Territories <1 Yukon year-olds British Columbia Alberta Manitoba Ontario Québec New Brunswick Prince Edward Island Newfoundland and Labrador Northwest Territories Yukon

28 The average results obtained by anglophone students across Canada in 1994 and 1998 cannot be compared, since Saskatchewan and Nova Scotia (anglophone sector) did not participate in the 1994 assessment. If, however, we compare the results of anglophone students in the provinces that participated in both assessments, we see that, in most of the provinces, the results of anglophone students in both age groups decreased in 1998 at reading levels 3 and 4. In both 1994 and 1998, Québec anglophone students results in the reading assessment were equal to or better than those of anglophone students in the other provinces. And this applies to all performance levels. 24

29 5.4.2 Comparison of Results in the Writing Assessment Table and 1998 Results in the Writing Assessment Populations Performance Levels year-olds British Columbia Alberta Manitoba Ontario Québec New Brunswick Prince Edward Island Newfoundland and Labrador Northwest Territories Yukon year-olds British Columbia Alberta Manitoba Ontario Québec New Brunswick Prince Edward Island Newfoundland and Labrador Northwest Territories Yukon

30 The average results obtained by anglophone students across Canada in 1994 and 1998 cannot be compared, since Saskatchewan and Nova Scotia (anglophone sector) did not participate in the 1994 assessment. If, however, we compare the results of anglophone students in the provinces that participated in both assessments, we see that, in most of the provinces, the results of anglophone students in both age groups increased in 1998 at writing levels 3 and 4. The results of 16-year-old New Brunswick anglophones increased dramatically at these levels (35 percent at level 3 and 28 percent at level 4). In both 1994 and 1998, the results of Québec anglophone students in the writing assessment were equal to or better than those of anglophone students in the other provinces. This applies to all performance levels except level 4, since 13-year-old students in Alberta scored higher at this level in

31 6 Québec Results According to Language Group Much care must be taken when comparing the results obtained using instruments prepared in different languages, even if the instruments do appear to be equivalent. There are a number of techniques for determining the equivalence of tests, but none of them is perfect. Although the precise reasons for the relatively weaker performance of some francophone groups in writing are unknown, it is often said in comparative linguistics that French is a more complex language syntactically and grammatically and that there are more opportunities for students especially young students to make mistakes in writing More 13-year-old francophones attained levels 2, 3 and 4 in the reading assessment, while more 16-year-old francophones attained levels 3, 4 and 5 in the same test. In reading, the francophones lead over the anglophones increased between 1994 to Percentage of Québec Students at Each Level in Reading Performance Levels Populations year-old francophones year-old anglophones year-old francophones year-old anglophones In the writing assessment, more 13-year-old anglophones attained level 3, while more 16- year-old anglophones attained level 5. In writing, the anglophone students lead over francophones decreased at level 3 but increased at level 5 between 1994 and Percentage of Québec Students at Each Level in Writing Performance Levels Populations year-old francophones year-old anglophones year-old francophones year-old anglophones

32 7 Highlights The following are some highlights of the study of the results obtained in the reading and writing assessments administered to Québec students as part of the Canadian School Achievement Indicators Program. In general, the results of Québec students in the reading and writing assessments resemble those of students in the other provinces. At reading level 3, Québec francophones in both age groups obtained better results than those in the other provinces. However, 13-year-old Québec francophones at level 3 did not perform as well in writing as students in the other provinces. And 13-year-old Québec anglophones did better in writing than francophones in the same age group. There are no significant differences in writing performance between anglophone students in Québec and those in the other provinces. This holds for both age groups. In Québec as in the other provinces, girls in both age groups obtained better results than boys in both reading and writing. Looking at the results obtained across Canada, we see that the results of anglophones in Québec are better than those of anglophones in the other provinces and that, at levels 2, 3, 4 and 5, the results of francophone students in Québec are much better that those of francophone students in the other provinces. 28

33 8 APPENDICES Appendix A READING ASSESSMENT CRITERIA 1 Level 1 The student reader interprets, evaluates, explores surface meanings from straightforward texts and some meaning from more complex texts by: responding to vocabulary, syntax, concrete details, directly stated ideas or key points; making judgments about purpose, content, or relationships; exploring in the context of personal experience. Level 2 The student reader interprets, evaluates, explores surface and/or directly implied meanings from straightforward texts and some meaning from more complex texts by: responding to concrete details, strongly implied ideas, or key points; making supported judgments about purpose, content, or relationships; exploring the context of personal experience and understanding. Level 3 The student reader interprets, evaluates, explores complex meanings in complex texts and some meaning from sophisticated texts by: responding to more abstract language, details, ideas; making informed judgments about purpose, content, or relationships among elements; exploring and demonstrating personal understanding and appreciation. 1. Council of Ministers of Education, Canada, School Achievement Indicators Program Reading and Writing Assessment 1994 (Toronto: CMEC, 1994). 29

34 Level 4 The student reader interprets, evaluates, explores complex meaning in complex texts and some sophisticated texts by: responding to more subtle and/or implicit language, details, and ideas; making well-supported judgments about purpose, content, or relationships; exploring and integrating a thoughtful understanding and appreciation. Level 5 The student reader interprets, evaluates, explores complex meanings in sophisticated texts and questions by: responding to elements of style, selection of details, matters of organization and characterization, and complex ideas; making insightful and substantiated relationships between content, purpose and style; exploring and integrating insightful and substantial understanding and appreciation. 30

35 Appendix B WRITING ASSESSMENT CRITERIA 1 Level 1 The writer demonstrates an elementary and uncertain grasp of fundamental elements of writing. Integration of these elements is not evident. The writing conveys simplistic and/or partial and fragmented meaning. Among the various characteristics of this level are: the writer s voice/tone/stance are discernible but may be ambiguous, inappropriate, and/or unclear; the writer demonstrates surface interest in the task but little awareness of the reader; ideas and development are elementary and may not be clear; lack of control of syntax and rules of language obscures communication. Level 2 The writer demonstrates an uneven and/or uncertain grasp of the elements of writing. Integration of some of the elements is apparent, but development is sketchy and/or inconsistently maintained. The writing conveys simple and/or uneven meaning. Possible characteristics are: the writer s voice/tone/stance are discernible but may be inconsistent or uneven; the writer demonstrates some evidence of engagement with the subject and superficial awareness of the reader; the controlling idea and its development are limited but discernible; grasp of conventional syntax and rules of language is limited; errors are distracting and interfere with communication. 1. Council of Ministers of Education, Canada, School Achievement Indicators Program Reading and Writing Assessment 1994 (Toronto: CMEC, 1994). 31

36 Level 3 The writer demonstrates a control of the elements of writing. The writing is generally integrated, and development is generalized, functional, and usually maintained throughout. The writing conveys a clear perspective. Some characteristics are that: the writer s voice/tone/stance are clear and appropriate; it is apparent that the writer is interested in the subject and in communicating with the reader; the controlling idea and its development are straightforward, clear, and appropriate, if overgeneralized; control of conventional style, syntax, and rules of language is evident; errors do not unduly affect the reader. Level 4 The writer demonstrates an effective control of the elements of writing. The writing is integrated, clearly and fully developed, and comes together as a secure whole. The writing conveys a thoughtful perspective. Characteristics include that: the writer s tone/voice/stance are secure and appropriate; the writer is clearly interested in the subject and in communicating with the reader; the controlling idea and its development are thoughtful, clear, and purposeful; a solid control of style, syntax, and the rules of language is evident and effective; errors are minimal. Level 5 The writer demonstrates an effective and confident command of the elements of writing. The writing is thoroughly integrated, precisely and fully developed, and the elements enhance one another. The writing conveys an insightful and sophisticated perspective. The following are characteristics of this level of writing: the writer s voice/tone/stance are confident, and enhance the impact of the writing; the writer s strong engagement with the subject sustains the reader s interest; the controlling idea and its development are insightful and well considered; command and control of style, syntax, and rules of language effectively enhance communication; errors are minimal. 32

37 Appendix C Table 18 Percentage of 13-Year-Olds and 16-Year-Olds at Each Level in the 1994 Reading and Writing Assessments Performance Levels Populations Reading Assessment Writing Assessment 13-year-olds British Columbia Alberta Manitoba (francophones) Manitoba (anglophones) Ontario (francophones) <1 Ontario (anglophones) Québec (francophones) Québec (anglophones) New Brunswick (francophones) <1 New Brunswick (anglophones) Nova Scotia Prince Edward Island < Newfoundland Northwest Territories Yukon Canada year-olds British Columbia Alberta Manitoba (francophones) Manitoba (anglophones) Ontario (francophones) Ontario (anglophones) Québec (francophones) Québec (anglophones) New Brunswick (francophones) New Brunswick (anglophones) Nova Scotia Prince Edward Island Newfoundland Northwest Territories Yukon Canada

38 1998 SAIP READING PERCENTAGE OF 13-YEAR-OLDS BY PERFORMANCE LEVEL AND BY POPULATION Level 1 Level 2 Level 3 Level 4 Level 5 British Columbia 95.6 (1.2) 74.9 (2.9) 38.7 (3.3) 9.2 (2.0) 1.1 (0.7) Alberta 96.4 (1.2) 78.2 (2.6) 39.4 (3.1) 7.4 (1.7) 1.0 (0.6) Saskatchewan 97.2 (1.1) 76.1 (2.8) 34.8 (3.2) 6.0 (1.6) 0.2 (0.3) Manitoba (a) 95.8 (1.4) 73.4 (3.1) 34.9 (3.3) 5.0 (1.5) 0.4 (0.4) Manitoba (f) 95.5 (3.1) 70.5 (6.7) 42.4 (7.3) 12.9 (4.9) 1.3 (1.7) Ontario (a) 97.1 (1.1) 77.8 (2.7) 38.4 (3.2) 6.4 (1.6) 1.0 (0.6) Ontario (f) 94.8 (1.6) 72.4 (3.2) 35.5 (3.4) 7.9 (1.9) 0.4 (0.5) Québec (a) 96.8 (1.2) 77.6 (2.9) 42.3 (3.5) 11.6 (2.3) 1.3 (0.8) Québec (f) 97.3 (1.1) 83.7 (2.5) 54.6 (3.3) 14.7 (2.4) 1.5 (0.8) New Brunswick (a) 97.2 (1.1) 76.1 (3.0) 38.8 (3.4) 6.9 (1.8) 0.5 (0.5) New Brunswick (f) 95.6 (1.3) 72.8 (2.9) 36.2 (3.1) 8.9 (1.8) 1.4 (0.8) Nova Scotia (a) 96.7 (1.0) 71.4 (2.5) 34.1 (2.6) 8.1 (1.5) 0.9 (0.5) Nova Scotia (f) Prince Edward Island 96.1 (1.4) 77.3 (2.9) 39.3 (3.4) 5.9 (1.6) 0.7 (0.6) Newfoundland and Labrador 96.2 (1.3) 78.2 (2.9) 41.5 (3.5) 8.7 (2.0) 0.4 (0.4) Northwest Territories 79.1 (1.6) 47.4 (2.0) 19.8 (1.6) 4.6 (0.8) 0.2 (0.2) Yukon 94.3 (3.0) 77.5 (5.3) 31.1 (5.9) 2.6 (2.0) 0.7 (1.0) Other francophones 96.2 (1.9) 78.1 (4.1) 41.8 (4.9) 10.4 (3.0) 2.1 (1.4) Canada (a) 96.7 (0.4) 76.7 (0.9) 38.2 (1.0) 7.2 (0.5) 0.9 (0.2) Canada (f) 97.0 (0.6) 82.3 (1.3) 52.3 (1.8) 14.0 (1.2) 1.4 (0.4) Canada 98.6 (0.3) 78.0 (0.7) 41.5 (0.9) 8.8 (0.5) 1.0 (0.2) N.B.: For each age group, the first column presents the percentages of students who attained level 1, while the second column presents the percentages of students who attained level 1 as well as those who attained level 2, and so on. The confidence intervals (± 1.96 times the standard error) appear in parentheses. The results are weighted to correctly represent each population. 34

39 1998 SAIP READING PERCENTAGE OF 16-YEAR-OLDS BY PERFORMANCE LEVEL AND BY POPULATION Level 1 Level 2 Level 3 Level 4 Level 5 British Columbia 98.6 (0.8) 90.1 (2.1) 67.9 (3.3) 29.0 (3.2) 6.7 (1.8) Alberta 99.4 (0.5) 91.4 (1.8) 67.4 (3.1) 28.6 (2.9) 6.9 (1.7) Saskatchewan 99.0 (0.7) 91.4 (1.9) 64.9 (3.2) 24.9 (2.9) 4.8 (1.4) Manitoba (a) 98.3 (0.9) 88.1 (2.3) 65.5 (3.4) 27.1 (3.1) 6.1 (1.7) Manitoba (f) (0.0) 83.8 (5.7) 59.9 (7.6) 27.5 (6.9) 8.4 (4.3) Ontario (a) 98.7 (0.8) 91.1 (1.9) 71.6 (3.0) 35.5 (3.1) 10.2 (2.0) Ontario (f) 98.7 (0.8) 87.8 (2.3) 65.0 (3.4) 28.0 (3.2) 6.8 (1.8) Québec (a) 99.3 (0.6) 93.1 (1.8) 71.9 (3.2) 31.0 (3.3) 9.5 (2.1) Québec (f) 98.9 (0.7) 95.3 (1.4) 79.4 (2.7) 41.8 (3.4) 15.3 (2.4) New Brunswick (a) 97.6 (1.1) 89.7 (2.2) 65.9 (3.5) 27.0 (3.3) 5.2 (1.6) New Brunswick (f) 98.9 (0.7) 90.2 (2.1) 68.1 (3.2) 31.3 (3.2) 9.0 (2.0) Nova Scotia (a) 99.3 (0.5) 90.7 (1.8) 66.4 (3.0) 27.3 (2.8) 5.6 (1.5) Nova Scotia (f) Prince Edward Island 97.7 (1.1) 85.6 (2.6) 63.9 (3.5) 25.5 (3.2) 5.5 (1.7) Newfoundland and Labrador 98.6 (0.8) 93.6 (1.7) 71.4 (3.2) 30.8 (3.2) 6.8 (1.8) Northwest Territories 89.8 (1.8) 68.8 (2.8) 41.1 (2.9) 10.8 (1.8) 2.9 (1.0) Yukon 96.5 (2.9) 79.8 (6.4) 55.3 (7.9) 22.8 (6.7) 5.2 (3.5) Other francophones 94.9 (2.9) 87.8 (4.3) 62.5 (6.3) 30.7 (6.0) 7.7 (3.5) Canada (a) 98.7 (0.2) 90.8 (0.6) 69.3 (1.0) 31.6 (1.0) 8.3 (0.6) Canada (f) 98.9 (0.4) 94.6 (0.8) 78.0 (1.5) 40.5 (1.8) 14.5 (1.3) Canada 98.7 (0.2) 91.8 (0.5) 71.5 (0.8) 33.9 (0.9) 9.8 (0.6) N.B.: For each age group, the first column presents the percentages of students who attained level 1, while the second column presents the percentages of students who attained level 1 as well as those who attained level 2, and so on. The confidence intervals (± 1.96 times the standard error) appear in parentheses. The results are weighted to correctly represent each population. 35

40 1998 SAIP WRITING PERCENTAGE OF 13-YEAR-OLDS BY PERFORMANCE LEVEL AND BY POPULATION Level 1 Level 2 Level 3 Level 4 Level 5 British Columbia 98.7 (0.8) 94.5 (1.6) 72.0 (3.2) 18.1 (2.7) 2.2 (1.0) Alberta 99.5 (0.4) 95.3 (1.4) 74.7 (2.9) 23.9 (2.8) 4.5 (1.4) Saskatchewan 99.4 (0.5) 95.9 (1.4) 73.6 (3.0) 17.9 (2.6) 3.2 (1.2) Manitoba (a) 99.0 (0.7) 94.3 (1.7) 70.6 (3.3) 16.0 (2.7) 2.5 (1.1) Manitoba (f) (0.0) 80.1 (6.1) 28.0 (6.9) 3.4 (2.8) 0.6 (1.2) Ontario (a) 99.8 (0.3) 96.6 (1.2) 73.9 (2.9) 19.8 (2.6) 1.9 (0.9) Ontario (f) 99.3 (0.6) 80.8 (2.8) 30.8 (3.3) 2.3 (1.1) 0.0 (0.0) Québec (a) 99.5 (0.5) 94.7 (1.6) 73.8 (3.2) 17.1 (2.7) 2.4 (1.1) Québec (f) 99.5 (0.4) 95.3 (1.4) 66.2 (3.1) 15.2 (2.4) 1.0 (0.7) New Brunswick (a) 99.9 (0.3) 95.0 (1.6) 70.1 (3.4) 14.2 (2.6) 2.2 (1.1) New Brunswick (f) 99.2 (0.6) 87.7 (2.1) 40.9 (3.2) 6.0 (1.5) 0.4 (0.4) Nova Scotia (a) 99.6 (0.4) 94.4 (1.3) 69.5 (2.7) 14.7 (2.0) 2.4 (0.9) Nova Scotia (f) Prince Edward Island 99.4 (0.5) 94.9 (1.6) 68.2 (3.3) 13.4 (2.4) 1.1 (0.7) Newfoundland and Labrador 99.3 (0.6) 96.1 (1.4) 70.7 (3.2) 14.7 (2.5) 2.5 (1.1) Northwest Territories 92.6 (2.2) 67.2 (4.0) 33.9 (4.0) 5.2 (1.9) 0.6 (0.6) Yukon 99.3 (1.0) 93.9 (3.0) 66.7 (5.8) 17.3 (4.7) 1.8 (1.6) Other francophones 99.8 (0.4) 81.3 (3.9) 35.4 (4.8) 4.8 (2.1) 0.7 (0.8) Canada (a) 99.4 (0.2) 95.6 (0.4) 73.0 (0.9) 19.2 (0.8) 2.5 (0.3) Canada (f) 99.5 (0.2) 93.8 (0.9) 62.1 (1.7) 13.8 (1.2) 0.9 (0.3) Canada 99.5 (0.1) 95.2 (0.4) 70.5 (0.8) 18.0 (0.7) 2.1 (0.3) N.B.: For each age group, the first column presents the percentages of students who attained level 1, while the second column presents the percentages of students who attained level 1 as well as those who attained level 2, and so on. The confidence intervals (± 1.96 times the standard error) appear in parentheses. The results are weighted to correctly represent each population. 36

41 1998 SAIP WRITING PERCENTAGE OF 16-YEAR-OLDS BY LEVEL OF ACHIEVEMENT AND BY POPULATION Level 1 Level 2 Level 3 Level 4 Level 5 British Columbia 99.0 (0.7) 97.1 (1.3) 83.6 (2.8) 37.5 (3.6) 11.0 (2.3) Alberta 99.5 (0.5) 97.8 (1.0) 83.8 (2.5) 42.7 (3.3) 10.0 (2.0) Saskatchewan 99.9 (0.2) 97.4 (1.1) 84.2 (2.5) 34.7 (3.3) 7.1 (1.8) Manitoba (a) 99.9 (0.3) 98.7 (0.9) 86.4 (2.6) 38.9 (3.7) 9.0 (2.2) Manitoba (f) 99.2 (1.4) 92.9 (4.0) 56.8 (7.7) 7.1 (4.0) 0.0 (0.0) Ontario (a) 99.9 (0.2) 98.6 (0.8) 87.5 (2.3) 42.2 (3.4) 10.9 (2.2) Ontario (f) 99.5 (0.5) 91.6 (1.9) 50.8 (3.5) 13.4 (2.4) 1.9 (1.0) Québec (a) 99.7 (0.4) 98.6 (0.9) 87.6 (2.5) 43.6 (3.8) 13.6 (2.6) Québec (f) 99.5 (0.5) 98.6 (0.8) 87.0 (2.3) 39.8 (3.3) 7.5 (1.8) New Brunswick (a) 99.6 (0.5) 98.4 (1.0) 87.5 (2.5) 36.7 (3.6) 7.1 (1.9) New Brunswick (f) 99.5 (0.5) 92.0 (1.9) 61.2 (3.4) 16.8 (2.6) 2.0 (1.0) Nova Scotia (a) 99.9 (0.2) 99.1 (0.6) 88.5 (2.1) 37.5 (3.2) 8.7 (1.9) Nova Scotia (f) Prince Edward Island 99.4 (0.6) 98.0 (1.1) 85.1 (2.7) 33.6 (3.6) 7.0 (1.9) Newfoundland and Labrador 99.7 (0.4) 98.5 (0.9) 88.8 (2.3) 35.4 (3.4) 6.4 (1.8) Northwest Territories 96.3 (2.2) 83.5 (4.4) 51.7 (5.9) 10.8 (3.9) 2.2 (4.3) Yukon 99.1 (1.7) 98.1 (2.3) 83.3 (6.4) 30.7 (7.9) 6.8 (4.3) Other francophones 98.9 (1.4) 93.0 (3.5) 50.5 (6.8) 13.0 (4.6) 0.3 (0.7) Canada (a) 99.6 (0.1) 98.1 (0.3) 86.0 (0.8) 40.3 (1.1) 10.3 (0.7) Canada (f) 99.5 (0.3) 97.9 (0.5) 83.7 (1.4) 37.2 (1.8) 6.9 (1.0) Canada 99.6 (0.1) 98.1 (0.3) 85.4 (0.7) 39.5 (0.9) 9.4 (0.6) N.B.: For each age group, the first column presents the percentages of students who attained level 1, while the second column presents the percentages of students who attained level 1 as well as those who attained level 2, and so on. The confidence intervals (± 1.96 times the standard error) appear in parentheses. The results are weighted to correctly represent each population. 37