Assessment 7 Rubric for Internship Portfolio: Field Supervisor Ratings, Academic/Behavioral Case Study, and Inservice Delivery Data

Transcription

1 Assessment 7 Rubric for Internship Portfolio: Field Supervisor Ratings, Academic/Behavioral Case Study, and Inservice Delivery Data [Note to KSDE Reviewers: The Internship Portfolio consists of parts of which three contribute to Assessment 7. The rubrics for those three parts appear below] Field Supervisor Rating Rubric Emporia State University School Psychology Supervised Practicum/Internship Evaluation Internship Candidate District/Coop Self-evaluation Field Supervisor Evaluation Date Field Supervisor Ratings provided on this form are intended to guide the candidate and the program in evaluating the candidate s readiness for provisional certification as a school psychologist. Please use the following rating scale in evaluating the candidate on the listed characteristics: N/A or N/O Not observed or not applicable. Candidate was not observed practicing this skill, or it may not be appropriate for practice in this setting.. Unsatisfactory Performance. Not displayed to the degree necessary to perform even under supervision.. Emergent. Adequately knowledgeable to perform under supervision.. Competent. Experience in applying knowledge and skills independent of supervision.. Exemplary. Consistent in initiating, modeling and influencing others use of knowledge and skills. Directions: For each item, put a check in the column that best represents your evaluation of the candidate s skill. If an item is not applicable, write in the Not Observed or Not Applicable (NO) column. If functioning varies significantly, place checks in several columns indicating the range.

2 Not Observe d (NO) or Not Applicab le (NO) Unsatisfa ctory Emergent Compete nt Exemplar y Data Based Decision Making Standard. The candidate uses varied models and methods of assessment as part of a systematic process to collect data and other information, translate assessment results into empirically-based decisions about service delivery, and evaluate the outcomes of services. P. The school psychologist systematically collects information to identify strengths and needs and uses the information to make decisions, plan services, evaluate the outcomes of services, and facilitate accountability for decisions that have been made. P. The school psychologist collects data on individual students, families (as appropriate), educational and health programs, classroom environments, and other aspects of schools and other agencies to evaluate problems and needs, to assess current status, and to evaluate the effectiveness of the decision-making process. P. The school psychologist collects data about school and home environments and cognitive, social, and behavioral factors that have a significant impact on children's school achievement and personal competence. P. The school psychologist evaluates the components of environments that facilitate or impede learning or behavioral changes for infants and children and identifies how environmental factors and children's characteristics interact to affect academics and social/behavioral outcomes (i.e. functional behavior assessment). P.8 The school psychologist administers, scores, and interprets individual tests of cognition, achievement, and social or emotional development. P.9 The school psychologist translates assessment results to clear, concise psycho-educational reports. P.0 The school psychologist maintains confidential records of evaluations and assessments. Consultation and Collaboration Standard : The school psychologist has knowledge of behavioral, mental health, collaborative, and/or other consultation models and methods and of the application to particular situations. The school psychologist collaborates and consults effectively with others in planning and decision-making processes at the individual, group, and system levels. P. The school psychologist has positive interpersonal skills and listens, adapts, addresses ambiguity, and is patient in difficult situations. P. The school psychologist establishes and maintains rapport throughout the problem-solving process.

3 Not Observe d (NO) or Not Applicab le (NO) Unsatisfa ctory Emergent Compete nt Exemplar y P. The school psychologist clearly presents and disseminates information to diverse audiences. P.6 The school psychologist facilitates team meetings effectively. Effective Instruction and Development of Cognitive/Academic Skills Standard #: The school psychologist, in collaboration with others, develops appropriate cognitive and academic goals for students with different abilities, disabilities, strengths, and needs, implements intervention to achieve those goals, and evaluates the effectiveness of intervention. P. The school psychologist assists children with developing behaviors to support effective learning, such as study skills, self-monitoring, planning/ organization, and time management skills. P.7 The school psychologist links assessment information to the development of instructional strategies to meet the individual learning needs of children. P.8 The school psychologist uses appropriate assessment techniques, including treatment integrity, to assess progress toward academic goals and assist in revising instructional methodology as necessary. P.9 The school psychologist applies techniques to evaluate the extent to which the instructional or intervention strategy contributed to the outcome and identifies factors that constitute a successful outcome. Socialization and Development of Life Skills Standard #: The school psychologist, in collaboration with others, develops appropriate behavioral, affective, adaptive, and social goals for students of varying abilities, disabilities, strengths, and needs, implements interventions to achieve those goals, and evaluates the effectiveness of intervention. P.9 The school psychologist links assessment information to the development of strategies to address individual behavioral, affective, adaptive, and social goals for infants and children. P.0 The school psychologist uses appropriate assessment techniques, including treatment integrity, to assess progress toward goals and assist in revising instructional or intervention strategies as necessary. P. The school psychologist applies techniques to evaluate the extent to which the instructional or intervention strategy contributed to the outcome and identifies the factors that constitute a successful outcome. Student Diversity in Development and Learning Standard #: The school psychologist demonstrates the sensitivity and skills needed to work with individuals of diverse

4 Not Observe d (NO) or Not Applicab le (NO) Unsatisfa ctory Emergent Compete nt Exemplar y characteristics and to implement strategies selected based on indivi dual characteristics, strengths, and needs. P. The school psychologist incorporates knowledge of diversity when designing and implementing academic and social/behavioral interventions to achieve learning and social/behavioral outcomes. P. The school psychologist assists schools in identifying what is needed for students with diverse characteristics to succeed and what instructional or other modifications are required to address children's difficulties. P. The school psychologist promotes practices that help infants, children and families of all backgrounds feel welcome and appreciated in the school and community. School and Systems Organization, Policy Development, and Climate Standard #6: The school psychologist has knowledge of general education, special education, and other educational and related services and understands schools and other settings as systems. The school psychologist works with individuals and groups to facilitate policies and practices that create and maintain safe, supportive, and effective learning environments for children and others. K6. The school psychologist has knowledge of resources and services in general and special education. Prevention, Intervention, and Mental Health Standard #7: The school psychologist provides or contributes to prevention and intervention programs that promote the mental health and physical well-being of students. P7. The school psychologist collaborates with school personnel, parents, and the community in the aftermath of crises (e.g., suicide, death, natural disaster, murder, bombs or bomb threats, extraordinary violence, sexual harassment, etc.). P7.6 The school psychologist routinely accesses resources to address behavioral, learning, mental, and physical problems, including diverse health issues (e.g., diet, eating disorders, teenage pregnancy, AIDS prevention, and stress management). Home/School/Community Collaboration Standard #8: The school psychologist works effectively with families, educators, and others in the community to promote and provide comprehensive services to children and families. P8. The school psychologist provides support and assistance for parents and helps them become comfortable, active, effective participants in school functions or activities (e.g., provides support for parents when participating on special education and IFSP/IEP teams and encourages parent involvement in school

5 Not Observe d (NO) or Not Applicab le (NO) Unsatisfa ctory Emergent Compete nt Exemplar y wide committees and improvement teams). P8. The school psychologist facilitates collaboration between schools and parents in designing school curriculum and interventions for students. P8. The school psychologist educates schools and communities regarding the influence of family involvement on children s development and achievement and advocates for parent involvement in planning for their children and school governance whenever feasible. P8.6 The school psychologist understands the relationship between infants, their families and community agencies and assists in coordinating services when programming for infants. Research and Program Evaluation STANDARD #9 The school psychologist evaluates research, translates research into practice, and understands research design and statistics in sufficient depth to plan and conduct investigations and program evaluations for improvement of services. P9.6 The school psychologist provides information about relevant research findings to school personnel, parents, and the public. School Psychology Practice and Development Standard #0: The school psychologist has knowledge of the history and foundations of the profession, of various service models and methods, of public policy development applicable to services to infants, children and families, and of ethical, professional, and legal standards. The school psychologist practices in ways that are consistent with applicable standards, is involved in the profession, and has the knowledge and skills needed to acquire career-long professional development. P0. The school psychologist applies methods from psychology, education, special education, health care, and related fields, in work with infants, children, parents, and professionals in school and other agencies. P0. The school psychologist adheres to appropriate ethical, professional, and legal standards to enhance the quality of services and to protect the rights of all parties. P0. The school psychologist promotes due process guidelines in all decisions affecting students. P0. The school psychologist maintains accepted standards in assessment, consultation, intervention, and general professional practice. P0. The school psychologist fulfills all legal requirements. P0.6 The school psychologist advocates for the rights and welfare of infants, children and families and promotes new public policies and practices in schools and other

6 Not Observe d (NO) or Not Applicab le (NO) Unsatisfa ctory Emergent Compete nt Exemplar y settings. P0.7 The school psychologist evaluates personal knowledge, professional competencies, and outcomes of services to determine specific needs for continuing professional development. P0.8 The school psychologist recognizes personal limitations, biases, and areas of expertise. P0.9 The school psychologist plans and implements systematic and effective techniques to enhance professional development. Information Technology Standard # The school psychologist accesses, evaluates, and utilizes information sources and technology in ways that safeguard or enhance the quality of services. P. The school psychologist uses word processing, spread sheets, test scoring software and other computer resources to function more effectively and efficiently. P. The school psychologist uses information resources and technology (e.g., CD ROM, the World Wide Web, , interactive television, distance learning technology) to acquire information, current research findings, and continuing professional development. P. The school psychologist uses technology when designing, implementing, and evaluating instructional programs or interventions for infants, children and youth. Professional Behaviors. Presents a professional appearance (e.g., appropriate dress and language). Is punctual for appointments and meetings. Demonstrates fundamental organizational skills (e.g., attention to detail, time management, setting priorities). Demonstrates good judgement and common sense. Manages time realistically and efficiently.6 Maintains visibility and accessibility within school.7 Considers all alternatives and implications before making recommendations.8 Completes written reports and forms in a neat, thorough, and accurate manner and with punctuality Interpersonal Skills. Communicates and listens effectively

7 Not Observe d (NO) or Not Applicab le (NO) Unsatisfa ctory Emergent Compete nt Exemplar y. Is tolerant of others' values and viewpoints. Is assertive in problem solving when needed. Meets difficult situations with self control. Establishes comfortable and productive working relationships with other school staff.6 Relates well to students and parents.7 Is sensitive and empathetic.8 Shows concern, respect, and sensitivity for the needs of students.9 Achieves comfortable interactions with diverse students and parents Participation in Supervision. Displays initiative and resourcefulness. Utilizes constructive criticism. Shows evidence of continued self-assessment. Keeps supervisor informed of unusual events and activities. Demonstrates an awareness of competency level, and doesn't accept responsibilities that exceed this level. Consistently follows through when additional action is needed Given these ratings of the candidate s current professional skills, list the three most important goals for his/her continued professional development:. Most important goal:..

8 Letter Grade Recommended for the Reporting Period: Practicum Supervisor Practicum Candidate University Supervisor Date Date Date *Candidate s signature indicates only that the evaluation has been discussed with the student. Field Supervisors Instructions All items will be assigned a Likert-type score, from (unsatisfactory) to (exemplary). Items given an N/O or N/A rating will be excluded without it impacting overall results by assigning those means the average rating for that candidate for that standard. However, if more than 0% of items within a section are given an N/O or N/A rating, it suggests that the roles assigned at the practicum site may not be well aligned to the program s standards. The total proportion will be reported for each section. Within any given standards section, the sum total of the scores will be obtained. The average mean across all items for that standard will be obtained as well. For ratings to be considered acceptable at practicum completion, the candidate must have all ratings within each standard equal to or higher than. If an item is rated below a, the university supervisor may assign a remedial product which if completed satisfactorily will satisfy the criteria for that item.

9 Academic/Behavioral Intervention Case Study Rubric A. Quality Indices. Behavioral definition Definition is (a) objective- refers to observable and measurable characteristics of behavior; (b) clearso unambiguous that it could be read, repeated, and paraphrased by observers; and (c) completedelineates both examples and non examples of the behavior Definition meets only two of the three criteria (i.e., objective, clear, complete). Definition meets only one of the three criteria (i.e., objective, clear, complete). Problem behavior is stated in general terms (e.g., reading comprehension, aggressive behavior, etc). Behavioral definition is not written.. Baseline data Data collected on the behavior prior to implementing the intervention consisting of repeated measures of the target behavior over several (at least three) sessions, days, or even weeks until a stable range of behavior has been identified. Data collected on the behavior prior to implementing the intervention; however, only two data points are reported. Data collected on the behavior prior to implementing the intervention; however, only one data point is reported. Data collected on the behavior prior to implementing the intervention; however, the dimension(s) addressed are not the most appropriate for the selected target behavior. Baseline data not gathered prior to implementing the intervention.. Problem validation The magnitude of the discrepancy is quantified, based on a comparison between student=s performance and the local educational setting demands. The magnitude of the discrepancy is quantified, based on a comparison between the student=s performance and standards outside the local educational setting. The magnitude of the discrepancy is quantified, but is based on an opinion. The magnitude of the discrepancy is described qualitatively. Problem is not validated; magnitude of the discrepancy is not described.

10 B. Problem Analysis Stage. Problem Analysis Examined relevant and alterable factors from curriculum, instruction, environment, and student domains using variety of procedures (RIOT: review, interview, observe, test) to collect data from a variety of relevant sources and settings. Used this information to develop a specific intervention to change the behavior. Examined relevant and alterable factors from two to three domains only using two to three procedures to gather information. Used this information to develop a specific intervention to change the behavior. Examined relevant and alterable factors from only the student domains using a variety of procedures (RIOT) to collect data from a variety of relevant sources and settings. Used this information to develop a specific intervention to change the behavior. Examined relevant and alterable factors from the domains only using a variety of procedures to gather information from a variety of sources. However, there is no indication this information was used to develop a specific intervention to change the behavior. Problem analysis is not conducted. C. Plan Implementation Stage. Goal Setting Goal stated narratively and represented graphically on performance chart specifying time frame, condition, behavior, and criterion, which is based on a comparison between the student=s baseline data and the expectations. Goal represented graphically on performance chart specifying time frame, behavior, criterion, and condition - not stated narratively. Goal stated narratively specifying time frame, behavior, criterion, and condition - not represented graphically. Goal stated narratively and/or represented graphically on performance chart but does not specify all four components (time frame, condition, behavior, criterion). Goal is not measurable or is not set. 6. Intervention plan Plan stated (a) procedures/ strategies, (b) material, (c) when, Plan stated procedures/ strategies. But one of the following Plan stated procedures/strategie s. But two of the following Generic description of intervention strategy (e.g., behavior Intervention plan not written. Or generic descriptions of intervention (e.g.,

11 (d) where, and (e) persons responsible components is missing: materials, when, where, or persons responsible. components is missing: materials, when, where or persons responsible. management) is stated. Materials, when, where, and persons responsible may be present. behavior management) only. 7. Measurement strategy A measurement strategy is developed answering how? what? where? who? and when? A measurement strategy is developed but only answers four of the five questions: how? what? where? who? and when? A measurement strategy is developed but only answers three of the five questions: how? what? where? who? and when? A measurement strategy is developed but only answers two of the five questions: how? what? where? who? and when? Measurement strategy is not developed. Or the measurement strategy only answers one of the five questions. 8. Decision - Making plan The decisionmaking plan indicates (a) how frequently data will be collected, (b) the strategies to be used to summarize the data for evaluation, (c) how many data points or how much time will occur before the data will be analyzed, and (d) what actions will be taken based on the intervention data. The decisionmaking plan indicated three of the four components. The decisionmaking plan indicates two of the four components. The decisionmaking plan indicates only one of the four components. The decisionmaking plan is not documented. D. Plan Evaluation Stage 9. Progress Monitoring Data are collected and charted/graphed two to three times per week. Appropriate graphing/ charting conventions were used (e.g., descriptive title, Data collected and charted/graphed one a week. Appropriate graphing/charting conventions were used. Data collected and charted/graphed irregularly and infrequently (less than once a week, but more than pre and post). Appropriate graphing/ charting Data collected but not charted or graphed. Or only pre-information and post-information was collected and/or charted/graphed. Progress monitoring data not collected.

12 meaningful scale captions, appropriate scale units, intervention phases labeled). conventions may or may not be used. 0. Formative evaluation There is evidence the decision rule was followed and visual analysis was conducted. These data were used to modify or change the intervention as necessary. There is evidence the decision rule followed and visual analysis was conducted, but the data were not used to modify or change the intervention as necessary. Modifications or changes were made to the intervention based on subjective data. Modifications or changes were made to the intervention, but no indication as to what data were used to make these changes. No formative evaluation was conducted.. Treatment integrity Degree of treatment integrity measured and monitored. Plan is implemented as designed, including decisionmaking rules. Intervention changed/ modified as necessary on the basis of objective data. Degree of treatment integrity addressed. Plan was implemented as designed and modified as necessary on the basis of subjective opinions. Degree of treatment integrity addressed. Plan was implemented with variations from the original design with no basis for change stated. Treatment integrity addresses, but intervention was not implemented as planned. Treatment integrity not considered.. Summative evaluation: Outcome decisions are based on the progress of monitoring data. Outcome decisions are based on minimal data (i.e., pre and post test). Outcome decisions are based on subjective data. Outcome decision stated but no indication of what data were used to make the conclusion No summative evaluation took place.

13 Inservice Delivery Scoring Rubric Part I Introduction Identifies intended audience for the inservice Identifies goals that will be accomplished during the inservice Establishes compelling statement of the topic or problem and why it is relevant to the audience: passes the so what? test. Important terms used are identified Scholarship Shows alignment to and/or utilizes Best Practices Literature Identifies/summarizes key people and ideas in area selected Emphasizes relevant literature in which the populations utilized align with populations relevant to the inservice audience Draws conclusions substantiated by evidence in literature presented Conclusions integrate previous literature and major alternative points of view Describes implications relevant to the audience Statistics are explained as appropriate for the audience Effective Instruction Logically organized Stays focused on content related to identified goals in introduction Inservice clearly utilizes constructivist and/or interactive elements (i.e., not strictly expository) Utilizes appropriate media Engages audience, e.g., with expectation that they will share their experiences or perceptions Gives opportunities to ask questions Provides physical copy of handout or tool which outlines topic and/or provides additional resources Provides contact information and affilitation(s) for the presenter. Advocacy Is aligned to NASP professional principles regarding promotion of effective, data-based decision-making Promotes the profession of school psychology Appropriately defines the role of the school psychologist in general and/or the speaker in particular Overall Topic and delivery was well targeted to stated audience Topic hit the mark: practicing educators, parents, or other intended audience members find the content of practical relevance and importance. The topic is significant, i.e., contributed practical knowledge which impacts children, home, and/or schools

14 Part II Efficacy Study: Measurement of Goal Achievement Requires audience to generate a product which measures the extent to which they met goals stated in introduction Items/questions utilized with the product show content validity Scaling of items will allow adequate variance in results Results of the statistical analysis presented in a clear and understandable form Tables used to summarize data Presenter Study: Measurement of Presenter/Trainer Characteristics Items address extent to which participant believes that new knowledge or attitudes/beliefs were learned Items address extent to which presenter showed characteristics of effective instruction such as being prepared, demonstrating confidence, appearing credible, or otherwise identified as effective; shows content validity Items address extent to which participant perceived quality in the training, i.e., whether they would recommend that others attend the presentation or whether they desire that a follow-up training would occur; shows content validity Items also reflect other stated professional development goals of the candidate Scaling will allow adequate variance in results Results of the statistical analysis presented in a clear and understandable form Tables used to summarize data Reflection Clarity: The language is clear and expressive. Abstract concepts are explained accurately. Relevance: The inservice experience being reflected upon is relevant and meaningful to candidate and professional learning goals. Analysis: The reflection moves beyond simple description of the experience to an analysis of how the experience contributed to understanding of self, others, and/or relevant school psychological concepts. Interconnections: The reflection demonstrates connections between the experience and material from other courses; past experience; and/or personal goals. Self-criticism: The reflection demonstrates ability of the student to question their own biases, stereotypes, preconceptions, and/or assumptions and define new modes of thinking as a result.

15 Inservice Data Summary Candidate: Rater:. Introduction Scholarship Effective Instruction Advocacy Overall Efficacy Study Presenter Study Reflection

16 0 = Clear and consistent evidence that section meets standards = Adequate or somewhat inconsistent evidence that section meets standards 0 = Inconsistent or absent evidence that section meets standards