Abstract. Introduction

Transcription

1 Reading Students Perceptions of Satisfaction with Read Right Lisa Putnam Cole Professor of Reading and Reading Program Coordinator Heartland Community College, Normal, IL / Abstract This study sought to establish a preliminary benchmark of the effectiveness of the implementation of a lab component in Heartland Community College s Reading 091 classes, effective Fall Semester This design change was characterized by the use of the Read Right instructional methodology. Students completed the Read Right Student Exit Survey at the conclusion of Fall Semester 2009 and the following spring semester. This survey is a reflective, self-report assessment of students satisfaction with the Read Right instruction received in the Reading Lab and their perceptions of the resulting improvement in their reading skills. Responses overwhelmingly mentioned changes in a positive direction for both the cognitive and affective domains. Improvement in reading skills, specifically improved comprehension and moderated speed, were cognitive changes noted most often, and the majority of responses in the affective domain mention improved attitudes, toward reading and instruction in the Reading Lab, as well as increased confidence. These benchmark findings suggest college students perceptions that the Read Right instructional method is an effective tool for improving their reading ability and their attitude toward reading. Additional study is needed to confirm these preliminary findings. Introduction In order to satisfy the requirements for NADE Certification, a self-study team composed of Heartland Community College (HCC) faculty, staff, and administrators was charged with identifying the Reading (READ) Program s areas of strength and those needing improvement so program administrators could make recommendations for improvement. As a result, the following modifications were implemented in HCC s three-course developmental reading sequence (READ 070, 090, and 091) for the academic year: 1) A lab component was added to READ 091. READ 070 and 090 had previously implemented a lab component to facilitate the use of individualized instruction. READ 070 s lab component was first employed during the spring of 2003, while the READ 090 lab was added for the fall of ) The patented Read Right methodology would be used to the greatest extent possible for lab instruction in all sections of all READ courses, with instruction to be supplied by the Reading Lab s Read Right Facilitators. The Read Right training model restricts the number of individuals who can be trained at one time. In those instances where facilitators were not available to supply lab instruction, labs were staffed by the instructor of the lecture portion of the course, who designed and delivered her own lab activities. Method After slight modification that directed respondents to specify the number of semesters they have been served by the Reading Lab, Read Right s Student Exit Survey (Appendix A, p. 8) was administered to HCC developmental reading students who: anticipated successful completion of the developmental reading sequence (i.e., successful completion of READ 091, the final course in the sequence) that semester, Read Right Student Exit Survey Results - 1

2 enrolled in a section of READ 091 which had an associated Read Right lab, and attended lab on the day the TABE (Test of Adult Basic Education) was administered as a post-test measure of reading improvement. TABE testing occurred during the last three weeks of Fall Semester 2009 and Spring Semester Table One outlines the number of students to whom the exit survey was administered (117 of 173), a significantly-sized subgroup of just over two-thirds of all students enrolled in a section of READ 091 having an associated Read Right lab. Table One: READ 091 Students Completing Exit Survey (Fall 2009 and Spring 2010) # Students 091 Students Completing Survey Enrolled in 091 Semester with Read Right Lab # Students % of Enrollees a Fall Spring Total a Students enrolled as of the census date, the 10 th day of the semester. Results The survey (Appendix A, p. 8) yielded both quantitative and qualitative data. Questions 1-5 directed students to circle the number best describing their response to the question posed, while Questions 6 and 7 required a short answer. Questions 1-5 Students responses to Questions 1 through 5 were analyzed by calculating the average rating and the number and percent of positive and very positive ratings for each question. Average Ratings: Each question and its accompanying rating scale are presented in Table B1 (Appendix B, p. 9), along with the average ratings for Fall Semester 2009, Spring Semester 2010, and the academic year. Average ratings indicated moderate to definite levels of perceived improvement and satisfaction with Read Right, and an across-the-board increase in average scores for all items from fall to spring is also evident. This increase may well be a reflection of Facilitators increased skill and confidence upon successful completion of their prescribed Read Right training during the fall semester. A possible trend for low- and high-scoring items also presented itself. Question 5 ( Would you recommend Read Right to someone else who needs help in reading? ) was the highest-scoring item and only question to receive an average rating of higher than 4 for both fall (4.22) and spring (4.43) semesters. This item could be considered an indicator of students confidence in the Read Right methodology as a resource for those who wish to improve their reading skills (even though many students, as responses to Questions 6 and 7 indicate [see below], were skeptical in the beginning). Further, some institutional researchers consider students recommendations to others more valid than self-reports about ways in which students were helped personally. The item receiving the lowest average score for both fall and spring (3.53 and 3.88, respectively) was Question 2 ( Has reading improved your attitude towards school? ). Even these responses indicate an attitude toward school that has improved at least somewhat. Questions 4, 1, and 3 received the second-, third-, and fourth-highest scores for fall semester, and this scoring hierarchy remained intact for spring when averages were carried to the second decimal point. Responses to each of these questions certainly indicate movement in a Read Right Student Exit Survey Results - 2

3 positive direction. However, it is interesting to note that the scores for Question 4 ( Do you feel better about yourself since your reading has improved? ) and Question 1 ( Do you feel you are a better reader as a result of being in Read Right? ) are consistently higher than Question 3 ( Do you do better in other classes as a result of the improvement in reading that you have made in Read Right? ). When coupled with the fact that Question 2 ( Has reading improved your attitude towards school? ) is the lowest-scoring item, student responses to Question 3 seem to confirm the commonly-held notion that transferring skills from one context to another is a challenging task, in perceived reality as well as in theory. Number and Percent of Ratings: The number and percent of positive ( 3, 4, or 5 ) and very positive ( 4 or 5 ) ratings for each question is outlined in Table B2 (Appendix B, p. 9). The following summary of students responses for the academic year reveals an overwhelmingly positive pattern: 99% indicate they feel they are a better reader as a result of being in Read Right 91% indicate they feel that improved reading has improved their attitude toward school 91% indicate they feel they do better in other classes as a result of the improvement in reading they have made in Read Right 98% indicate they feel better about themselves since their reading has improved In response to Question 5 ( Would you recommend Read Right to someone else who needs help in reading? ), ninety-six percent of the respondents indicated they would recommend Read Right to others, with 82% offering very positive responses. The vote of confidence offered by the most telling question posed by this survey effectively buttresses the sentiment offered by the first four. 1 Questions 6 and 7 2 Students responses to these short-answer questions were quantified by a multi-step process. One individual (i.e., coder) read responses, proposed categories emerging from those responses, and then assigned each response to as many proposed categories as was appropriate in a preliminary manner. A second individual (or coder) rated each item independently using the categories assigned by the first individual. After category refinements were agreed upon by the coders, these two individuals independently rated items a second time. The final step required coders to achieve consensus on the category(-ies) to which each response was assigned. As a result, it was possible for each response to be assigned to more than one response category. Table Two outlines the number of responses yielded by Questions 6 and 7 for the entire academic year, as well as each semester individually. Table Two: Questions 6 and 7 and Number of Responses (Fall 2009 and Spring 2010) Number of Responses Questions 6 and 7 Fall 09 Spring 10 Total 6. How is reading different for you now that you have completed a semester of Read Right? 7. Please comment about your reading or about being a student in Read Right Appendix B (Tables B3 and B4, pp. 10 and 11) provides, for each question, a detailed listing of general response categories, specific responses, and, as appropriate, subcategories enhanced 1 As mentioned previously, from a research perspective, commendations to others are often considered more valid than students reports about how something helped them personally. 2 Jennie Kearney s efforts and insight during the coding process are gratefully acknowledged. Her able assistance greatly advanced this project. Read Right Student Exit Survey Results - 3

4 with value-added labels. Table B3 summarizes responses to Question 6 and Table B4, Question 7. Percentages for the fall and spring semesters are provided in aggregated and disaggregated formats on these tables and are based on the total number of responses reported for each respective question. These tables (B3 and B4) serve as the major data source for the discussion of the responses to Questions 6 and 7, though limited reference to them is made. When deeper analysis proved more helpful, it was completed, and the results are presented in tables embedded within the narrative. Question 6 (How is reading different for you now that you have completed a semester of Read Right?). Five general categories emerged for Question 6, and they are listed from most- to least-frequently mentioned in Table 3. Table Three: Question 6 Response Categories, Definitions, and Number of Responses Fall 2009 and Spring 2010, Combined General Response Category Broad Definition of Category # % b Cognitive Domain Improved Reading Skills Affective Domain Attitude Toward Reading c Metacognitive Domain Increased Self-Awareness General Statements Insightful But Very Vague or Very Brief c Couldn t Code/Other/No Response Unable to Assign a Category 11 6 Totals b Percent calculated relative to number of ALL responses to Question 6. (Refer to Table B3.) c Responses in this category also included value-added subcategories (i.e., positive, neutral, negative). The cognitive and affective domains account for nearly three-quarters of the responses to this question, with the remaining quarter mentioning the metacognitive domain, making general statements, or relating responses that could not be coded. Cognitive Domain: Just over half the responses to Question 6 (51%) mentioned improvement in the cognitive domain as evidenced by improved reading skills. Improved comprehension was mentioned most frequently ( Read Right has helped me with my comprehension. I am able to understand more without pausing or reading twice. ). Other improvements mentioned were moderated pace ( I have a normal pace now and can understand what I m reading. ), general expressions of improved reading skills ( I think my reading has improved a lot. ), and a combination of various other aspects of efficient reading, such as increased vocabulary, enhanced attention and focus, and improved retention of material. Affective Domain: Approximately one-fourth (23%) of the responses to Question 6 mentioned the affective domain, with fall (17%) and spring (30%) percentages varying moderately from the year s average. To describe more fully the nature of Read Right s impact on students attitudes toward reading, each response was assigned a value-added subcategory (i.e., positive, neutral, negative) as outlined by Table 4. Table Four: Question 6 Value-Added Subcategories for Attitude Toward Reading Fall 2009 and Spring 2010, Combined Subcategory Value-Added Label # % d Attitude Toward Reading Positive Neutral 5 12 Negative 0 0 Total d Percentages calculated relative to number of responses assigned to Attitude Toward Reading ONLY. Read Right Student Exit Survey Results - 4

5 Eighty-eight percent of all responses presented a positive attitude or mentioned a change in attitude toward reading that was positive in nature. An improved attitude toward reading was mentioned most often ( Reading at first was a [chore], but now after Read Right reading is for pleasure. ), followed by responses relating increased confidence in a general sense ( I feel more confident about my reading and my comprehension. ) and specifically in reading aloud ( I don t mind reading in class. Reading out loud used to bother me, but I don t mind it any more. ). A few responses mentioned reading is easier and that completing reading assignments is not so much of a hassle. Affectively neutral responses ( My reading improved, but I don t like it any better. ) were minimal (12%), with no attitudinal changes in a negative direction reported. Metacognitive Domain: Responses mentioned the metacognitive domain 11% of the time, with students noting increased knowledge of the reading process ( Well, reading used to be difficult. I didn t know when I had to stop or when not to read fast. I had no idea that there was an excellent way to read. That s something I really appreciate learning. Now I know how to read excellently and with style. ), their personal reading process in particular ( If I take my time when I read I will understand better. ), or reading strategies and tactics and how to apply them when reading does not make sense ( I was never a horrible reader, but now I know how to correct myself when I mess up. ). General Statements: This category was created for the 10% of the responses to Question 6 (12% in fall and 7% in spring) that were insightful, but so general or so truncated it was difficult to assign them to another category. Responses within this category were assigned to a positive, neutral, or negative category as presented by Table 5. Table Five: Question 6 Value-Added Responses for General Statements Fall 2009 and Spring 2010, Combined General Response Category Value-Added Label # % e General Statements Positive Neutral 2 11 Negative 0 0 Total e Percentages calculated relative to number of responses assigned to General Statements ONLY. Eighty-nine percent of the responses assigned to this category were positive in nature ( Reading is more of an inspiration to me because when I read now I get into the book., I am able to read more clear than before., and Easier to understand. ). Relatively few of these responses (11%) were neutral ( No, it is the same. ), and none were negative. Couldn t Code/Other/No Response: Responses were assigned to this category as follows: 1) The response did not make sense to coders, 2) Coders would have had to interpret the response beyond what was reasonable to make it fit into an existing category, or 3) No response was provided. Very few responses fell into this category for fall (9, or 9%) and spring (2, or 2%). Question 7 (Please comment about your reading or about being a student in Read Right.). The general categories presenting themselves for Question 7 were identical to those that emerged for Question 6 with one notable exception. Table Six reveals the appearance of one additional aspect of the affective domain: Attitude Toward Program or Class. The frequency with which categories were mentioned for Question 7 differed from those mentioned for Question 6, sometimes significantly. Categories are listed below, in order from most- to leastoften-mentioned, with additional information available from Table B4 (p. 11). Table Six: Question 7 Response Categories and Number of Responses Fall 2009 and Spring 2010, Combined General Response Category Broad Definition(s) of Category # % f Affective Domain Attitude Toward Reading c 11 7 Read Right Student Exit Survey Results - 5

6 Attitude Toward Program or Class c Cognitive Domain Improved Reading Skills General Statements Insightful But Very Vague or Very Brief c Couldn t Code/Other/No Response Unable to Assign a Category 12 7 Metacognitive Domain Increased Self-Awareness 6 3 Total c Responses in this category also included value-added sub-categories (i.e., positive, neutral, negative). f Percent calculated relative to the number of ALL responses to Question 7. (Refer to Table B4.) Affective Domain: The affective domain was mentioned in over half (56%) the total responses to Question 7. Table 7 presents results classified by positive, neutral, and negative. Table Seven: Question 7 Value-Added Subcategories for Affective Domain Fall 2009 and Spring 2010, Combined By Value- Within Value- Value-Added Subcategories Added Label Added Subcat. Label # % g # % h Positive Attitude Toward Reading 8 10 Attitude Toward Program or Class TOTAL POSITIVE Neutral Attitude Toward Reading 3 60 Attitude Toward Program or Class 2 40 TOTAL NEUTRAL Negative Attitude Toward Reading 0 0 Attitude Toward Program or Class TOTAL NEGATIVE Total g Percentages calculated relative to number of items assigned to Attitude Toward Reading and Attitude Toward Program or Class subcategories ONLY. h Percentages calculated relative to number of items assigned to value-added subcategories (i.e., positive, neutral, negative). By far, most of the responses to this question mentioning the affective domain (79, or 85%) were positive in nature. Ninety percent of those positive responses mentioned the program (Read Right) or the class (lab), with the remaining 10% noting the positive attitude or a positive change in attitude toward reading that was mentioned in Question 6. As demonstrated by Table B4 (p. 11), almost half of the positive-attitude responses related that the program or class was fun, enjoyable, helpful, or a good experience. One student related, My time in Read Right was great. Everybody was so helpful and the teachers made the class fun. Also I know for a fact that my reading level has gone up because of Read Right. Further examination of Table B4 (p. 11) also reveals that a small but significant number of responses (21) related initial dislike for the class that took a turn for the better (i.e., resulted in a positive attitude) once students realized how Read Right helped: When I first started this class I thought it was a joke class and I was going to blow it off. After taking it seriously I realized how great of a reader I became and I continued to improve right before my eyes. These 21 change of heart responses represented 27% of the responses assigned to the Attitude Toward Program or Class category. Five percent of the responses assigned to Affective Domain were neutral and 9% were negative ( It s pretty boring and if you read excellent you felt good, but if you read bad, it makes you feel bad. ). Cognitive Domain: About 25% of all responses to Question 7 (41 of 172) addressed the cognitive domain as manifested by students perceptions that their reading skills had improved. Read Right Student Exit Survey Results - 6

7 Overall, students references to the cognitive domain occurred less often in this question (#7) than in Question 6, though the coding categories were nearly identical in nature. Most of these cognitive-domain responses mentioned improved reading in a very general sense, with the remainder of responses noting improved comprehension, moderated pace, and other improvements, such as increased vocabulary, enhanced attention and focus, and improved retention of material, though, in no particular order. General Statements: Twelve percent of all responses to Question 7 (20 of 172) were assigned to this category. Table 8 reveals that the vast majority (70%) of the responses within this category were positive in nature (... now I m better and I m going to keep reading. and The Read Right lab really helps people improve their reading. ), followed by 20% of the responses being neutral ( It s OK. ), and 10% negative ( Not excellent. ). Table Eight: Question 7 Value-Added Responses for General Statements Fall 2009 and Spring 2010, Combined General Response Category Value-Added Label # % i General Statements Positive Neutral 4 20 Negative 2 10 Total i Percentages calculated relative to number of responses assigned to General Statements ONLY. Couldn t Code/Other/No Response and Metacognitive Domain: Twelve responses to Question 7 (7% of the total responses) were assigned the Couldn t Code / Other/ No Response category, and 6 responses (3% of the total responses) mentioned the metacogntive domain. Summary and Conclusions READ 091 students responses to this survey indicate perceived growth in the cognitive domain with specific mention of general reading improvement, increased comprehension, and moderated pace. The affective domain is also mentioned frequently, as students relate improved attitudes toward reading. Perhaps as a result of their perceptions of advancing reading abilities, students relate increased levels of confidence, both in general and in reading aloud, and also in their self-esteem. Students report their experiences with Read Right and the Reading Lab as overwhelmingly positive, in spite of some initial skepticism. Finally, students relate they would recommend Read Right to someone who needs help in reading. Through their experience in the Reading Lab, many survey respondents have come to understand that improved reading is an achievable but labor-intensive possibility. Perhaps more importantly, for the first time in their educational experiences, they may realize that diligent work and improved reading skills may lead to heightened personal satisfaction and greater academic success. Responses to this survey reveal students perceptions that Read Right instruction has improved their reading skills, as well as their confidence and attitude as related to reading. Respondents confidence in and positive perceptions of the methodology may be a consequence of the program s advancing establishment at the college and the staff s increasing skill and confidence at implementing the methodology as time progresses. Developmental reading students are sometimes reticent (to admit that some reading improvement might be beneficial) and quite often reluctant (to participate in a class which does not render transfer credit) participants in developmental reading coursework. Given these conditions, the endorsements offered by the results of this survey are nothing less than remarkable. Read Right Student Exit Survey Results - 7

8 Limitations Student self-reports are but one of many ways in which a program s effectiveness may be assessed. Another valid, informative, and perhaps more persuasive perspective is offered by standardized tests. Additional testing with alternate forms of assessment is needed to confirm the results of this study. Read Right Student Exit Survey Results - 8

9 APPENDIX A: READ RIGHT STUDENT EXIT SURVEY Name: Date: Sem. in Lab (circle): or An important dimension of evaluating the Read Right program is in measuring each student s perception of his or her own improvement. Please help us by circling the most appropriate response or providing a short answer. 1. Do you feel you are a better reader as a result of being in Read Right? No Somewhat better Much better 2. Has improved reading improved your attitude towards school? No Somewhat Definitely 3. Do you do better in other classes as a result of the improvement in reading that you have made in Read Right? No Somewhat Definitely 4. Do you feel better about yourself since your reading has improved? No Somewhat Definitely 5. Would you recommend Read Right to someone else who needs help in reading? No Maybe Definitely yes 6. How is reading different for you now that you have completed a semester of Read Right? 7. Please comment about your reading or about being a student in Read Right. Sign your name on the line provided below if you are willing to grant us permission to use your comments on promotional materials. Your grade will not be impacted by your choice to sign (or not). Read Right Student Exit Survey Results - 9

10 APPENDIX B: STUDENTS RESPONSES TO EXIT SURVEY Table B1: Responses to Exit Survey Questions 1-5 (Average Ratings) Questions 1-5 and Rating Scale 1. Do you feel you are a better reader as a result of being in Read Right? No Somewhat better Much better 2. Has improved reading improved your attitude towards school? No Somewhat Definitely 3. Do you do better in other classes as a result of the improvement in reading that you have made in Read Right? No Somewhat Definitely 4. Do you feel better about yourself since your reading has improved? No Somewhat Definitely 5. Would you recommend Read Right to someone else who needs help in reading? No Maybe Definitely yes Average Rating Fall 09 Spring 10 Total Table B2: Responses to Exit Survey Questions 1-5 (Frequency of 3, 4, and 5 Ratings) Total Resp 3, 4, or 5 Fall09 Spring 10 Total % 4 or 5 only % Total Resp 3, 4, or 5 Qu_ Qu_ Qu_ Qu_ Qu_ % 4 or 5 only % Total Resp 3, 4, or 5 % 4 or 5 only % Read Right Student Exit Survey Results - 10

11 Table B3: Responses to Exit Survey Question 6 Response Categories for Question 6: How is reading different for you now that you have completed a semester of Read Right? Cognitive Domain (Improved Reading Skills) Fall 09 Spring 10 Total N % e N % e N % e Read better / reading level increased / reading skills improved Comprehension improved Speed / pace moderated Other (vocabulary increased; attention/focus, fluency, tone, pronunciation, memory/retention improved) Total Cognitive Domain (Improved Reading Skills) Affective Domain (Attitude Toward Reading) Positive Attitude or Positive Change Confidence increased: General Confidence increased: Reading out loud Attitude toward / valuing or enjoyment of reading improved Reading is easier not as much hassle less effort Neutral Attitude or Neutral Change Attitude toward / valuing of reading stayed the same Negative Attitude or Negative Change Total Affective Domain (Attitude Toward Reading) Metacognitive Domain (Increased Self-Awareness) Gained knowledge of reading process in general Gained knowledge of personal reading process Gained knowledge of strategies / tactics and how to apply them when reading doesn t make sense Total Metacognitive Domain (Increased Self-Awareness) General Statements Positive Neutral Negative Total General Statements Total Couldn t Code / Other / No Response GRAND TOTALS FOR QUESTION e Percentages calculated relative to the total number of responses to this question. Read Right Student Exit Survey Results - 11

12 Table B4: Responses to Exit Survey Question 7 Response Categories for Question 7: Please comment about your reading or about being a student in Read Right. Cognitive Domain (Improved Reading Skills) Fall 09 Spring 10 Total N % e N % e N % e Read better / reading level increased / reading skills improved Comprehension improved Speed / pace moderated Other (vocabulary increased; attention/focus, fluency, tone, pronunciation, memory/retention improved) Total Cognitive Domain (Improved Reading Skills) Affective Domain Attitude Toward Reading Positive Attitude or Positive Change Confidence increased: General Confidence increased: Reading out loud Attitude toward / valuing or enjoyment of reading improved Reading is easier not as much hassle less effort Neutral Attitude or No Change Attitude toward / valuing of reading stayed the same Negative Attitude or Negative Change Total Attitude Toward Reading Attitude Toward Program or Class Positive Attitude or Positive Change I liked it / It was fun, enjoyable, helpful, a good experience At first I didn t like it, but eventually realized the benefit I would recommend this class to others, tell them about it Facilitators were fun, great, knowledgeable, etc Neutral Attitude or No Change Attitude paralleled feedback received on performance Negative Attitude or Negative Change I didn t like it / It was boring, tedious, frustrating, a bad experience Facilitators/class made me feel stupid, like I couldn t read, etc Total Attitude Toward Program or Class Total Affective Domain Metacognitive Domain (Increased Self-Awareness) Gained knowledge of reading process in general Gained knowledge of personal reading process Gained knowledge of strategies / tactics and how to apply them Total Metacognitive Domain (Increased Self-Awareness) General Statement Positive Neutral Negative Total General Statements Total Couldn t Code / Other / No Response GRAND TOTALS FOR QUESTION e Percentages calculated relative to the total number of responses to this question. Read Right Student Exit Survey Results - 12