Core-Selective Evaluation Process: A PSW Method to Efficiently and Comprehensively Diagnose Specific Learning Disabilities Edward K. Schultz, Ph.D. Midwestern State University Edward.schultz@mwsu.edu Participant Handout Agenda/Topic 1. Current Status of SLD Evaluation in Texas 2. Core-Selective 3. Basic Steps of the Core-Selective Evaluation Process (C-SEP) 4. Types of Scoring and viewing results through multiple lenses 5. Integrated Data Analysis/Pattern of Strengths and Weaknesses 1
Before Anticipation Guide Texas Law states that academic achievement MUST be assessed using a normreferenced test such as the KABC, WIAT, WJIV, etc. Tess that measure one trait (e.g., memory span) are superior to tests that measure two or more trait. Policy Decisions (such as RTI) are based on Research and Practice A Review of Existing Evaluation Data (REED) can be conducted for initial evaluations. Despite changes in policy and procedures in SLD identification, SLD rates have stayed relatively stayed the same in Texas. Measures that have an increase in cognitive complexity have a decrease in ecological validity CHC Theory and PASS Theory have more similarities than differences. We must ensure that evaluation materials are administered in accordance with any instructions provided by the producer of the assessments. After /During Comparison Model Comprehensive Efficient Precise RTI Only Discrepancy Methods Cross-Battery Approaches Core Selective 2
WJIV-COG Ability Scores General Intellectual Ability (GIA) Tests 1-7: Oral Vocabulary, Number Series, Verbal Attention, Letter-Pattern Matching, Phonological Processing, Story Recall, and Visualization. Gf-Gc Composite Test 1: Oral Vocabulary, Test 2 Number Series, Test 8: General Information, and Test 9: Concept Formation Brief Intellectual Ability Test 1: Oral vocabulary, Test 2: Number Series, and Test 3: Verbal Attention Scholastic Aptitudes Reading Aptitude for the Reading, Broad Test 1: Oral Vocabulary, Test 5: Phonological Reading, Reading Comprehension, Reading Processing, Test 9: Concept Formation, and Fluency, and Reading Rate achievement Test 11: Number-Pattern Matching clusters. Reading Aptitude for the Basic Reading achievement cluster. Math Aptitude for the Mathematics, Broad Mathematics, and Math Calculations Skills achievement clusters Math Aptitude for the Math Problem Solving achievement cluster Writing Aptitude for the Written Language, Broad Written Language, and Written Expression achievement clusters. Writing Aptitude for the Basic Writing Skills achievement cluster. Comprehension-Knowledge (Gc) Fluid Reasoning (Gf) Long-Term Retrieval (Glr) Visual Processing (Gv) Test 1: Oral Vocabulary, Test 3: Verbal Attention, Test 5: Phonological Processing, and Test 11: Number-Pattern Matching Test 1: Oral Vocabulary, Test 2: Number Series, Test 7: Visualization, and Test 17: Pair Cancellation Test 1: Oral Vocabulary, Test 7: Visualization, Test 10: Numbers Reversed, and Test 15: Analysis-Synthesis Test 1: Oral Vocabulary, Test 5: Phonological Processing, Test 6: Story Recall, and Test 11: Number Pattern-Pattern Matching Test 1: Oral Vocabulary, Test 3: Verbal Attention, Test 5: Phonological Processing, and test 11: Number-Pattern Matching. CHC Clusters Test 1: Oral Vocabulary and Test 8: General Information Test 2: Number Series and Test 9: Concept Formation Test 6: Story Recall and Test 13: Visual- Auditory Learning. Test 7: Visualization and Test 14: Picture Recognition. Auditory Processing (Ga) Test 5: Phonological Processing and Test 12: Nonword Repetition Cognitive Processing Speed (Gs) Test 4: Letter-Pattern Matching and Test 17: pair Cancellation. Short-Term Working Memory (Gwm) Test 3: Verbal Attention and Test 10: Numbers Reversed 3
Short-Term Working Memory-Extended Test 3: Verbal Attention, Test 10: Numbers (Gwm) Reversed, and Test 16: Object-Number Sequencing. CHC Narrow Ability and Clinical Clusters (precise) Perceptual Speed Test 4: Letter-Pattern Matching and Test 11: Quantitative Reasoning Number Pattern Matching Test 2: Number Series and Test 15: Analysis- Synthesis Auditory Memory Span Test 5: Sentence Repetition and *Test 18: Memory for Words from the WJIV-OL. Number Facility Test 10: Numbers Reversed and Test 11: Number-Pattern matching Vocabulary Test 1: Oral Vocabulary and *Test 1: Picture Vocabulary from the WJIV-OL. Cognitive Efficiency Test 4: Letter-Pattern Matching and Test 10: Numbers Reversed *Must supplement with the WJIV-OL to obtain cluster score. Table below displays the test, CHC domains, and ranges of g-weights for ages 2-19. Test CHC Domain Ranges of g-weights for ages 2-19. Oral Vocabulary Comprehension-Knowledge.16-.18 (Gc) Number Series Fluid Reasoning (Gf).17-.18 Verbal Attention Short-Term Working Memory.13-.14 (Gwm) Letter-Pattern Matching Cognitive Processing Speed (Gs).10-.17 Phonological Processing Auditory Processing (Ga).17-.19 Story Recall Long-Term Retrieval (Glr).11-.12 Visualization Visual Processing (Gv).07-.12 4
Process for Analyzing a Patterns of Strengths and Weaknesses (PSW) A pattern is defined as a combination of qualities, acts, tendencies, etc., forming a consistent or characteristic arrangement. In the case of SLD identification, patterns need to have predictive utility and converge to answer the following questions: a) what does this student s PSW say about current learning and specific referral question, (present) b) what does it say about future learning (predict), and c) how and where can we help this student (Special or General Education?). Instructions: Summarize the data collected during the FIE and utilize an integrative data analysis approach to assist in identifying a PSW. Integrated Data Analysis Chain of Evidence The first link in the chain includes examination of informal assessment data such as attendance records, home language survey, developmental history, school health files, previous test scores, grades and grade history, and records from previous schools, since these sources often provide a wealth of data that are relevant to the referral concern (Salvia, Ysseldyke, & Bolt, 2010). Other informal sources include class-room observations, writing portfolios, classroom work samples, and parent and teacher interviews which often take place informally. Understanding how the teacher and parent view the problem is critical since these individuals know the child best and spend the most time with him or her on a daily basis. The next link in the chain involves examining the results of non-standardized testing, which holds key information on the student s academic functioning. Most school records provide a plethora of data: results of benchmark testing on grade-level curriculum, curriculum-based testing from end-of-unit tests, progress monitoring data from curriculum-based measures for RTI, classroom running records, reading miscue analyses, and criterion-referenced tests. Criterion-referenced tests provide information on the skills that the child has mastered as well as those where the teacher needs to target instruction. Finally, the chain of evidence for the child s academic problem is joined with the final link of the chain, the results of the formal evaluation. In summary, the chain of evidence used to analyze a PSW in learners suspected of having a SLD as needs to include all three of the links: archival and extant data, observations, teacher and parent information; information derived from informal assessment such as progress monitoring data and benchmark testing; and results of standardized testing (i.e., cognitive processing) in all areas of suspected disability. Here we look for triangulation of evidence: is the description of the hypothesized academic problem supported by all three levels of evidence? When evaluation teams consider all three levels in students suspected of SLD, they are able to form a more complete picture of the academic problem and then determine more effective intervention strategies (Frijters, et al., 2011). Techniques for Pattern Seeking 5
The method of pattern seeking described in this article is an adaption of a process outlined by McMillan and Schumaker (2010) and is based on mixed-methods research data analysis (see Johnson and Onwuegbuzi 2004 for a full description). (Schultz, Simpson, & Lynch, 2012) 1. The first step in the pattern matching process is to examine the trustworthiness of the data. 2. The next step in the process of pattern analysis is triangulation (Leech & Onwuegbuzie, 2007; McMillan and Schumacher, 2010). 3. The third step in the pattern seeking process is to examine the exclusionary factors that must be ruled out when determining the presence of a learning disability. 4. After testing and observations are completed and extant information is gathered, evaluation personnel will need to analyze the data for a pattern of strengths and weaknesses to determine the presence of SLD. 5. After ordering and sorting data, assessment personnel can then graph the results. This process of sorting, ordering, and creating a visual representation represents sound data analysis principles (Johnson & Onwuegbuzie, 2007; McMillian & Shumacher, 2010) 6. The final step in the process is to conduct a logical cross analysis (McMillan and Schumacher, 2010; Onwuegbuzie & Leech, 2006) of the pattern of strengths and weaknesses based on visual inspection of the graphed data. What distinguishes C-SEP from traditional models and PSW approaches is: 1. Expressive (Oral Expression) and Receptive Language (Listening Comprehension) and the imperfect ability to listen, think, speak are formally tested and considered with every evaluation. This not only provides diagnostic information but also provides insight in to teaching and learning. 6
2. Statistical analysis is conducted using actual norms and software/tables from the publisher. Data collected from other batteries are included in the assessment using integrated data analysis. 3. Statistical analysis informs decision-making and professional judgment instead of being the primary vehicle of the eligibility decision. Integrated data analysis including pattern seeking techniques are used to make eligibility decisions. 4. All tests administered including the core should be administered in a purposeful and deliberate manner. Testing should only occur to provide new or previously unknown information. Examiner time is dedicated to interpretation and integrating data instead of test giving. 5. Academic underachievement is determined using multiple sources and standard scores obtained from norm-referenced testing are used to understand the relationship between cognitive and language constructs. Standard scores are never used the sole determinate of a discrepancy or variance with a cognitive or language measure. 6. It is a PSW model that requires professional judgment, and discrepancy analysis is used to identify and support patterns that emerged from the data. 7. Special education policy and assurances are strictly adhered to in order to provide the most legally defensible evaluations. 7
Case Study Table XX. Data from Core used to Corroborate Verbal Reasoning Deficits and Determine if additional Testing is needed Multiple Measures Processes STARR Assessment System Verbal Reasoning is scored and reported RTI (focused on Comprehension) Progress Monitoring Data WISC Similarities Subtest Formal Measure of crystallized intelligence (Gc), word knowledge, cognitive flexibility (Gf), auditory comprehension, long-term memory, associative and categorical thinking, distinction between nonessential and essential features, and verbal expression. KTEA Reading Comprehension Subtest Formal Measure of Language Comprehension. Acquired knowledge (Gc) and achievement, word recognition and decoding. Reading fluency, simultaneous processing (Gf), verbal working memory, executive functions. KTEA Listening Comprehension Subtest Formal Measure of Language Comprehension, discrimination of essential and nonessential information, acquired knowledge (Gc), sequential processing (Gf), executive functions. 8