Teachers teaching skills in two provinces in Indonesia according to the dynamic model of educational effectiveness

Transcription

1 Teachers teaching skills in two provinces in Indonesia according to the dynamic model of educational effectiveness S. N. Azkiyah* 1, S. Doolaard 2, M.P.C. van der Werf 2 Paper presented at the International Congress on School Effectiveness and Improvement Held in Cyprus, 4 7 January 2010 Contact information: 1 Syarif Hidayatullah State Islamic University of Jakarta, Indonesia and a PhD student at GION Institute for Educational Research, the University of Groningen, The Netherlands. * Corresponding author s.n.a.azkiyah@rug.nl 2 GION Institute for Educational Research, the University of Groningen, Grote Rozenstraat 3, 9712 TG, Groningen The Netherlands. Abstract Improving teachers teaching quality has been widely considered to be the most influential factor in improving education and student outcomes. The question is what and how to improve. The dynamic model especially the classroom level can be used as a framework to understand teaching quality and therefore decision on the focus of an improvement project can be made (Kyriakides, Creemers, & Antoniou, 2009). This paper is part of a PhD study on effective teacher training program in Indonesia. It specifically aims to understand the teaching skills of Indonesian secondary teachers according to the classroom level of the dynamic model. Classroom observation data of secondary teachers in the provinces of DKI Jakarta and Banten were analyzed and the results showed that teachers in general do not really practice the factors of the dynamic model. Keywords: Teaching skills, teacher professional development, the dynamic model of educational effectiveness Introduction It has been widely acknowledged that teacher is one of the most influential factors in improving education (Creemers, 1994, Darling-Hammond, 1997, Fullan, 2001, Harris, 2002, Harris & Muijs, 2005, Marzano, 2007, OECD, 1994, Pilot, 2007, Van Der Werf et al., 2000). Although it might be argued that these school or classroom effects are relatively small compared to other factors like family and individual effects, they are nevertheless large in terms of placement in curricular tracks and further development (Doolaard, 1999). The question is what and how to improve teacher or classroom level. Creemers & Kyriakides (2008) have developed the dynamic model of educational effectiveness. It has four level, i.e. context/national policy level, school level, teacher/classroom level, and student level, in which five dimensions of each level are taken into account to provide the 1

2 measurement of each effectiveness factor. They are frequency, focus, stage, quality, and differentiation. Previous research has shown that the classroom level can be used to establish five stages of teaching skills, from the easiest up to the most difficult ones, and therefore decision on the focus of an improvement project can be made (Kyriakides, Creemers, Antoniou, 2009). This paper is part of a PhD study on effective teacher professional development (TPD) program for secondary school teachers in Indonesia. It specifically aims at understanding teaching skills of Indonesian teachers especially with regards to the teaching of English reading comprehension as the focus of the study. The discussion of the paper will be organized in the following sections: 1) he classroom level of the dynamic model; 2) the establishment of developmental teaching skills; 3) method; 4) discussion of the findings; and finally 5) conclusion The Classroom Level of the Dynamic Model Figure 1 The Dynamic model of Educational Effectiveness As seen in figure 1, the classroom level of the dynamic model (Creemers & Kyriakides, 2008) consists of eight factors. They are 1) orientation, 2) structuring, 3) modeling, 4) application, 5) questioning, 6) assessment, 7) management of time, and 8) building classroom as a learning environment (CLE). 2

3 Orientation is the provision of objectives, which is expected to help students understand the importance of their learning activities. Structuring concerns with teacher explanation on series of activities of the lesson. With respect to questioning, research has found out that the effective teachers raise numerous questions and engage students in class discussion (Muijs & Reynolds, 2000). Question difficulties vary with context and teachers should include both product and process questions. Next, teachers are expected to help students use strategies and/or develop their own strategies through modeling. This activity should be completed with application, which is the provision of immediate exercise of topics taught during the lesson. Furthermore, teachers should identify their students learning need through assessment, which should also enable them to improve their teaching. The seventh is CLE, which includes 1) teacher-student interaction, 2) student-student interaction, 3) students treatment by teachers, 4) competition among students, and 5) classroom disorder. Finally, management of time is important to maximize students engagement and make sure that they are on tasks throughout the lesson. Figure 1 also shows there are five dimensions to measure each factor, i.e. frequency, focus, stage, quality, and differentiation. Frequency refers to the quantity of each of the above classroom factors. Focus is dealing with the specificity of the activity; whether an activity is too specific or too general in relation with the goals of the activity. Stage concerns with the period at which an activity takes place and quality questions whether an activity is clear and understandable for students. Finally differentiation refers to the extent to which activities associated with the above classroom factors are implemented in the same way for all groups of students. It is expected that teachers adapt specific needs of students and provide differentiated instruction according to individual student learning needs. The Establishment of Developmental Teaching Skills A study was conducted in Cyprus to measure teacher behavior in the classroom through observation. The Rasch model was applied using computer program Quest. The reliability of the scale was found to be higher than.93. The Rasch analysis shows that they can be classified into five clusters from the easiest to the most difficult ones. (Kyriakides, Creemers, & Antoniou, 2009). Table 1 the five developmental stages based on teaching skills included in the Dynamic Model STAGES 1: Basic Elements of Direct Teaching 2: Putting TEACHING SKILLS Frequency management time Stage Management of time Frequency structuring Frequency Application Frequency Assessment Frequency Questioning Frequency teacher-student relation 3

4 aspects of quality in direct teaching and touching on active teaching 3: Acquiring quality in active / direct teaching 4: Differentiation of teaching and putting aspect of quality in constructivist teaching 5: Achieving quality and differentiation in teaching using different approaches Stage Structuring Quality Application Stage Questioning Frequency student relations Focus Application Stage Application Quality of questions Stage student relations Stage teacher-student relation Stage Assessment Frequency Teaching Modeling Frequency Orientation Focus student relations Quality: feedback Focus Questioning Focus teacher-student relation Quality structuring Quality Assessment Differentiation Structuring Differentiation time management Differentiation Questioning Differentiation Application Focus Assessment Differentiation Assessment Stage teaching modeling Stage orientation Quality teacher-student relation Quality student relations Differentiation teacher-student relation Differentiation student relations Focus Orientation Quality Orientation Differentiation Orientation Quality of teaching modeling Focus Teaching Modeling The first stage refers to the quantitative characteristics of factors associated with direct teaching. Most of them deal with the frequency dimension and thus implies that the quantity of teaching is a prerequisite for instruction. The second stage is putting aspects of quality in 4

5 direct teaching and touching on active teaching, which concerned with qualitative aspects of three factors associated with direct teaching (structuring, application, and questioning). Three dimensions of application are included and this indicates that application is a basic and relatively simple competence that teachers could develop. The third stage is acquiring quality in active / direct teaching. Not only factors associated with direct teachings, two factors associated with constructivism namely orientation and modeling are also included. The fourth stage is differentiation of teaching and putting aspect of quality in constructivist teaching. The teaching skills in this stage are mainly concerned with the differentiation dimension of factors associated with direct teaching. Teachers at this stage understand different needs of their students and are also able to offer appropriate application, structuring, questions, and assessments for different group of students. The last stage is achieving quality and differentiation in teaching using different approaches, which are the most difficult qualitative characteristics of factors related to both direct/active teaching and the new teaching approach. Method Participants The sample is 58 teachers from 56 private junior secondary madrasah schools (JSS) in the provinces of DKI Jakarta and Banten. Madrasah is a type of schools under the authority of the Ministry of Religious Affairs (MORA). It is reported that more than 80% of madrasahs are private with low quality of human resources and limited funds (CEQDA, 2007). Since this paper is part of a broader research project on TPD, it is logical to consider that teachers in madrasah will benefit more from such improvement program. Instrument, data collection and analysis The data on teacher teaching quality were gathered through classroom observation using low and high inference instruments. In this study, the original observation instruments were modified and simplified for English reading comprehension. They are different from in several aspects. There are two low inference instruments in the original study, whereas this study uses only one, which is to record teacher activities according to the factors in the classroom level of the model. The other low inference instrument in the original study deals with teacher-students and students-students interaction and thus there are two observers. Due to some limitation, this study only has one observer who is required to fill in the low inference instrument and then rate the scale in the high inference instrument. The original high inference instrument includes all factors and dimensions, whereas this study included all factors but time management as this factor is measured together with CLE especially concerning students opportunities to learn. Differentiation dimension of some factors are excluded due to the fact that during the pre-testing of the instrument, none of the observed teachers practiced such activity. In order to make the instrument shorter some items measuring differentiation dimension are deleted. In addition, the items in this study are modified specifically for reading comprehension whereas the original is more general. Furthermore, the high inference instrument requires the observer to rate a Likert response scale to indicate the frequency and quality of observed activities. There are 52 items 5

6 which are divided into two; part A concerns with the frequency of observed activities provided on not at all (1) to a great deal (5) scale and part B deals with mostly the quality of observed activities provided on 1 (minimum) to 5 (maximum) scale. The number of items in most factors varies from 6 8 items, except questioning which has 10 and assessment which has only 2. Questioning has the most items as this factor has more sub-factors. On the other hand, assessment has the least since some items in the original instrument are considered to be very close to questioning. Classroom observation was carried out by independent observers who had very good inter-rater reliability (.85). The reliability test of each factor/scale of the high inference instrument in the present study is in general very good; orientation (.889), structuring (.843), modeling (.837), application (.767), questioning (.701), assessment (.668), and building classroom as a learning environment (.828). The low reliability of the assessment could be due to the fact that this factor has only two items. The data gathered in August and September 2010 will then be analyzed using descriptive statistics to understand teacher teaching quality. Discussion of the Findings Participant Table 2 indicates there are more female participants from both provinces. This resembles the data in the real population in which there are more female teachers. In total there are 33 female and 25 male teachers, which make the total number of participants 58. Table 2 the number of participants according to their sex and province Sex Province Total DKI Jakarta Banten Female Male Total How do teachers teach? The discussion in this section will be based on the mean score of observed activities in each factor or scale. On a 5 Likert scale of 1 (not at all), 2 (little), 3 (enough), 4 (much), and 5 (a great deal) for part A and 1 (minimum) to 5 (maximum) for part B, the minimum and maximum score of each item is also provided. 0 is for those which are missing. Table 3 the mean score of each factor/scale Factor N Minimum Maximum Mean Orientation A Orientation B

7 Structuring A Structuring B Modeling A Modeling B Questioning A Questioning B Assessment A Assessment B CLE A CLE B Except questioning and assessment as well as CLE part B, all mean score of the scales are less than 2. This means that in general teacher nearly did not practice the classroom level factors of the dynamic model at all. However, it is important to note that few teachers scored enough in each scale, which indicates that differences in teaching quality among teachers exist. Another good story is the fact that teachers have started to provide questions and assessment although little. Orientation The mean score of both orientation A and B is less than 2 meaning that in general teachers tend not to practice orientation at all. The frequency of each item in this scale indicates that several teachers (26%) provided enough questions to link the day lesson either with previous lesson or students daily life. However, many of them did not follow the questions with explicit explanation on the purpose of the lesson. This indicates that orientation is not an easy activity and is in line with the study in Cyprus, which places frequency orientation in the third stage. It is quite common to find teachers in Indonesia complaining that their students are not motivated and interested in the lesson. One possible explanation is the result of this study, which is the fact that teachers do not provide enough orientation for their students. Structuring Similar with orientation, the mean score of this factor is less than 2, which indicates that teacher did not really provide structuring activities for their students. Yet, 36% of them managed to give enough signals that they moved to different stages. In addition, although less than one fourth, teachers were also quite successful to arrange the lesson from easier to more complex activities. It should also be noted that few teachers were able to provide structuring activities more than enough (much); for example 3% presented the structure of the lesson and 2% explained the structure of in a way that was clear for the students. 7

8 Modeling Nearly one third (28%) of teachers spent some time to explain the content of the lesson. Yet, more than 50% presented only little the concept or the strategies for their students to accomplish their tasks. Similarly, when students were facing difficulties, they provided them only little procedures or strategies to overcome their students problems. In this stage, students engagement was unlikely to occur since many of them did not challenge their students to explain nor demonstrate strategies of learning they might know. In addition, teachers neither used students ideas to solve nor to present the strategies to overcome problems. Application The mean of all items in this factor shows that teacher did not provide enough application tasks to their students. In the case of reading comprehension class, this could happen when teachers spent long time on reading aloud and translating the text. As a consequence, within 40 minutes of the classroom observation, there was not much application tasks provided for students. One aspect in differentiation in reading comprehension class is seen from the texts provided for students. This research found that nearly no teachers asked their students to read different texts in different group. However, 26% of teachers provided application activities according to their plans, which then could explain the low expectancy of the teachers to their students. Questioning For questioning, almost one third of teachers (34%) were able to pose enough questions to their students. The question is how do teachers react? The data showed that around 50% teachers translated the questions into Bahasa Indonesia, which is rather in contrats with the theory of teaching reading comprehension. In addition, they also tend to move to other questions or answered the questions themselves. Useful hints were not really provided as more than 50% provided only little clues or hints. Luckily, almost 50% were successful to pose clear questions for his students. Assessments In this research, assessment is measured only from two items. The results indicated that teachers provided only with both little question or relevant tasks to examine their students understanding. When the activity was dominated by reading aloud and translation of the text, there was little space to explore the content and therefore questions or tasks to examine what students have understood from the day lesson did not really appear. Building classroom as a learning environment (CLE) Using mostly whole class instruction, quite a lot of teachers gave students opportunity to participate in the lesson. Yet, not many students were on tasks during the 8

9 lesson. This could be due to the fact that teachers provided not enough tasks during the lesson. This, at the same time, means that teachers were not successful to manage their time well in order to maximize students learning. There was also no explicit encouragement from teachers for the students to cooperate and compete to each other. Conclusion Based on the above brief description of teacher behavior according to the dynamic model, it can be concluded that in general the quality of teaching of Indonesian secondary teachers of madrasah in the two provinces, especially on the teaching of English reading comprehension is low. Yet, the more important aspect is the fact that the factors and the dimensions of the classroom level of the dynamic model are useful in providing reference on what specific aspect to be prioritized for improvement. When the result of this study is explained to individual teachers participating in the research, they are expected to understand their strengths and weaknesses, which then should enable them to take decision on teaching skills they should improve. Lastly, when it is possible, it is useful that this study employs the Rasch analysis to investigate whether teaching skills can be classified into developmental stages and thus to confirm previous study conducted in Cyprus. 9

10 References CEQDA. (2007). Laporan Dampak Kebijakan Akreditasi, BOS, dan Sertifikasi terhadap Masa Depan Madrasah di DKI Jakarta. Kerjasama LAPIS dan CEQDA UIN Jakarta. Creemers, B.P.M. (1994). The Effective Classroom. London: Cassel. Creemers, B.P.M. and Kyriakides, L. (2008). The Dynamics of Educational Effectiveness. London: Routledge Darling-Hammond, L. (1997). Doing what metters most: investing in quality teaching. Doolaard, S. (1999). Schools in change or schools in chain?: the development of educational effectiveness in a longitudinal perspective. Enschede: Twente University Press. Fullan, M. (2001). The new meaning of educational change. London: Routledge Falmer. Harris, A. (2002). School Improvement: what s in it for school?. London: Falmer Press. Harris, A. & Muijs, D. (2005). Improving schools through teacher leadership. London: Open University Press. Kyriakides, L., Creemers, B.P.M., Antoniou, P Teacher behavior and student outcomes: suggestion for research on teacher training and professional development. Teaching and Teacher Education 25 (2009) Marzano, R.J. (2007). The art and science of teaching. Alexandria, VA: Association for Supervision and Curriculum Development. Muijs, D. & Reynolds, D. (2000). School Effectiveness and Teacher Effectiveness in Mathematics: Some preliminary Findings from the Evaluation of the Mathematics Enhancement Programme (Primary). School Effectiveness and School Improvement, 11 (3), Organization for Economic Co-operation and Development (OECD). (1994). Quality in teaching. OECD, Paris. Pilot, A. (2007). The Teacher as a Crucial Factor in Curriculum Innovation: the case of Utrecht University. Paper presented at the Conference Teaching and Learning according and after Bologna, at the Swiss Federal Institute of Technology (ETH, Zurich, Switzerland, 9-10 March, 2007 Van der Werf, M.P.C., Creemers, B.P.M., De Jong, R., & Klaver, L. (2000). Evaluation of school improvement through an educational effectiveness model: The case of Indonesia s PEQIP project. Comparative Education Review, 44(3),

11 APPENDIX 1. The frequency and mean of items for orientation ITEM Min Max Frequency Mean PART A The teacher presented explicitly the aims of reading certain topics and text and studying specific reading skills The teacher asked pupils to discover the purpose of reading certain topics and text and studying specific reading skills. The teacher posed questions to link the lesson of the day with previous lessons, today lesson or students daily life. The teacher presented or reviewed the aims when he/she moved to another activity. The teacher spent some time on introductory activities. PART B The orientation activities that were organized during the lesson are clear and help students understand the new topic, text and specific reading skills The aims of the lesson were linked to previous activities or today activities or students daily life = 66%, 2 = 26%, 3 = 9% = 71%, 2 = 24 %, 3 = 3%, 4 = 2% = 34%, 2 = 40%, 3 = 26% = 78%, 2 = 17%, 3 = 5% = 9%, 2 = 60%, 3 = 28%, 4 = 3% = 43%, 2 = 47%, 3 = 10% = 42%, 2 = 48%, 3 = 10% The frequency and mean of items for structuring ITEM Min Max Frequency Mean PART A The teacher explained how different activities were linked to each other = 79%, 2 = 19%, 3 = 2% The teacher presented the structure of the lesson (the topic, the text, and specific reading skills) to the students = 40%, 2 = 43%, 3 = 14%, 4 = 3% The teacher signaled different stages or activities during the day lesson = 29%, 2 = 35%, 3 = 36% The teacher reviewed the main ideas / skills presented during the lesson = 3%, 1 = 40%, 2 = 48%, 3 = 9% The teacher spent time on closing-up activities = 15%, 2 = 76%, 3 = 5%, 4 = 3% PART B 11

12 The teacher explained the structure of the lesson in a way that was clear for the students = 31%, 2 = 55%, 3 = 12%, 4 = 2% The lesson transited from easier to more complex activities = 40%, 2 = 38%, 3 = 22% The frequency and mean of items for modeling ITEM Min Max Frequency Mean PART A The teacher spent time to lecture pupils on the lesson s content = 15%, 2 = 41%, 3 = 28%, 4 = 16% The teacher presented concept or strategies that students can use to accomplish specific reading skills planned to be taught. The teacher challenged students to explain or demonstrate strategies of reading they might know. When students faced certain learning obstacles or were confronted with a problematic exercise during application tasks, the teacher provided them with useful procedures or strategies for overcoming them. PART B The teacher used pupils ideas on how to solve a problem to present the strategy or method of solving it = 36%, 2 = 57%, 3 = 7% = 74%, 2 = 17%, 3 = 9% = 36%, 2 = 54%, 3 = 10% = 62%, 2 = 33%, 3 = 5% Students understood the procedures and strategies that were presented by the teacher = 3%, 1 = 24%, 2 = 55%, 3 = 17% The frequency and mean of items for application ITEM Min Max Frequency Mean PART A The teacher provided students the chance to use concepts, skills or strategies that they have acquired throughout the lesson or during previous lessons = 36%, 2 = 48%, 3 = 16% The teacher distributed different texts to different group of students = 91%, 2 = 4%, 3 = 5% The teacher asked students to deal with more difficult text(s) that were more demanding than the text used during teaching modeling. The teacher provided less able students more time to practice what have been taught during the lesson = 91%, 2 = 9% = 71%, 2 = 24%, 3 = 5%

13 The teacher gave the opportunity to the low ability students to compete the high ability students = 79%, 2 = 17%, 3 = 4% PART B The observed application activities referred (were linked) to all specific reading skills 1.00 planned to be taught = 31%, 2 = 43%, 3 = 26% The teacher provided a chance for students to apply / exercise their previous knowledge = 40%, 2 = 41%, 3 = 19% The teacher used to explain the procedures and strategies to the pupils and then he or she requested using them = 2%, 1 = 34%, 2 = 50%, 3 = 14% The frequency and mean of items for Questioning ITEM Min Max Frequency Mean PART A The teacher posed questions throughout the lesson = 5%, 2 = 40%, 3 = 34%, 4 = 21% The questions posed by the teacher ask students to give answers and not to explain the way they find their answers. When students gave wrong answers or no answer, the teacher translated the questions or some of the words into Bahasa Indonesia. When students gave wrong answers or no answer, the teacher posed simpler questions to help students find the answer. When students gave wrong answers or no answer, the teacher moved to another question or answered it him/herself = 3%, 2 = 45%, 3 = 9%, 4 = 3%, 5 = 3% 1 = 3%, 2 = 22%, 3 = 50%, 4 = 21%, 5 = 3% = 54%, 2 = 36%, 3 = 10% = 15%, 3 = 52%, 4 = 26%, 5 = 7% The teacher differentiated the type of feedback that he or she provided to students = 46%, 2 = 47%, 3 = 7% The teacher provided useful hints when students gave wrong answers = 29%, 2 = 53%, 3 = 16%, 4 = 2% When a student gave a wrong answer or no answer, the teacher readdressed the question until somebody gave the correct answer. PART B = 14%, 2 = 51%, 3 = 33%, 4 = 2% The teacher posed questions that were clear for the pupils in terms of their content = 10%, 2 = 34%, 3 = 48%, 4 = 7% The teacher corrected pupils misconceptions using their wrong answers = 34%, 2 = 47%, 3 = 19%

14 6. The frequency and mean of items for Assessment ITEM Min Max Frequency Mean PART A The teacher posed questions to examine what students have understood from the day lesson = 2%, 1 = 14%, 2 = 58%, 3 = 26% PART B The teacher provided relevant tasks or questions to examine what students have understood from the day lesson = 17%, 2 = 64%, 3 = 19% The frequency and mean of items for CLE ITEM Min Max Frequency Mean PART A The teacher gave students the opportunity to participate in the lesson = 5%, 2 = 38%, 3 = 55%, 4 = 2% The teacher explicitly encouraged students to cooperate with each other = 64%, 2 = 26%, 3 = 10% During the lesson, students cooperated on their own initiative = 48%, 2 = 31%, 3 = 21% Each student was engaged in tasks/work assigned to him/her by the teacher = 14%, 2 = 57%, 3 = 29% The teacher explicitly encouraged competition among students = 58%, 2 = 33%, 3 = 7%, 4 = 2% PART B The majority of pupils were engaged in activities that were provided by their teacher = 5%, 2 = 62%, 3 = 33% The teacher provided enough tasks for students during the lesson = 12%, 2 = 57%, 3 = 24%, 4 = 5%