Education, Education, Psychology, 2012 2013 Assessment Report College of Education Department of Special Education,, and Education, PhD Education, PhD Psychology, PhD Expected Outcome 1: Professional Knowledge Full Description of Expected Outcome: Upon completion of students programs, they will demonstrate proficiency in their ability to: (a) know their content; (b) apply their content; (c) read critically; (d) information literacy; and (e) using/conducting research. Key Assessments: Inventory of Candidate Proficiencies Assessment Method Description Students work is assessed across five domains including: (a) know their content; (b) apply their content; (c) read critically; (d) information literacy; and (e) using/conducting research. Each domain is assessed across four levels of proficiency: (1) Exemplary, (2) Competent, (3) Approaching, and (4) Not Approaching. Each level is described with program specific indicators (rubric). The assessment is completed faculty in the program. The following link provides access this specific key assessment: http://www.education.auburn.edu/files/assessment/key_assessment_grad/icp_gra duate_student.pdf Inventory of Candidate Proficiencies Please indicate whether the student demonstrated candidate proficiencies throughout the field experience/clinical practice by evaluating each of the following:. Candidate Proficiencies Rating Competent Professionals Poor 1. KNOW THEIR CONTENT Understand the central concepts, tools of inquiry, and structures of the discipline of study. 2. APPLY THEIR CONTENT KNOWLEDGE IN PRACTICE Transfer theories and concepts from a specific discipline into professional environments to improve practices in the field. 3. READ CRITICALLY Know now to extract salient facts and themes from current research and discipline based readings in order to critique understanding in the field. 4. DEMONSTRATE INFORMATION LITERACY Recognize when information is needed and have the ability to locate, evaluate, and use effectively the 1
Education, Education, Psychology, needed information (ALA, 1989) 5. COMMUNICATE EFFECTIVELY Write and speak effectively to present knowledge and theories in a specific discipline. 6. USE (Master s) AND/OR CONDUCT (Doctoral) RESEARCH Select appropriate research methods to contribute to the advancement of knowledge within a discipline. 7. USE TECHNOLOGY IN APPROPRIATE WAYS Understand use of and appreciate methods and issues of technology in specific discipline.. Committed Professionals Findings Summary Findings are based on an individual programs offered in the Department of Special Education,, and. At the completion of their program, faculty rate graduates on specific proficiencies that they should demonstrate in and outside of the classroom. According to the results of the Inventory of Candidate proficiencies, Read Critically and Use/Conduct Research are areas for increased attention. This is due to the percent of candidates performing at exemplary or competent level. For a detailed assessment of the results, please refer to table 1a 1d. Table 1a: Frequency and Percentages of individuals in Special Education,, and Doctoral programs. (By performance category: Content Knowledge) Content Knowledge Category Exemplary Competent Approaching Not Approaching Mean (Aggregate) 8 80.00% 2 20.00% 0 0.00% 0 0.00% 3.80 Education 1 33.33% 2 66.67% 0 0.00% 0 0.00% 3.33 Education 6 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Psychology 1 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 2
Education, Education, Psychology, Table 1b: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Apply the Content) Apply the Content Category Exemplary Competent Approaching Not Approaching Mean (Aggregate) 8 80.00% 2 20.00% 0 0.00% 0 0.00% 3.80 Education 1 33.33% 2 66.67% 0 0.00% 0 0.00% 3.33 Education 6 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Psychology 1 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Table 1c: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Read Critically) Read Critically Category Exemplary Competent Approaching Not Approaching Mean (Aggregate) 7 70.00% 3 30.00% 0 0.00% 0 0.00% 3.70 Education 0 0.00% 3 100.00% 0 0.00% 0 0.00% 3.00 Education 6 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Psychology 1 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 3
Education, Education, Psychology, Table 1d: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Information Literacy) Information Literacy Exemplary Competent Approaching Not Approaching Mean (Aggregate) 8 80.00% 2 20.00% 0 0.00% 0 0.00% 3.80 Education 1 33.33% 2 66.67% 0 0.00% 0 0.00% 3.33 Education 6 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Psychology 1 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Table 1e: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Use/Conduct Research) Use/Conduct Research Exemplary Competent Approaching Not Approaching Mean (Aggregate) 7 70.00% 3 30.00% 0 0.00% 0 0.00% 3.70 Education 0 0.00% 3 100.00% 0 0.00% 0 0.00% 3.00 Education 6 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Psychology 1 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Improvement During the 2012 13 academic year, the findings suggest that attention needs to be given to the areas of Read Critically and Use/Conduct Research. Table 2 provides a description of actionable steps that will be taken by program based on the data presented from the Inventory of Candidate Proficiencies. 4
Education, Education, Psychology, Table 2: Use of Findings for Improvement Special Education,, and Doctoral s The program will work to infuse research components into the coursework. Outcomes of this focus will be assessed through the Education research components of the dissertation requirements. Education Psychology The Education program will work to infuse research components into the COUN 8510 Contemporary Issues course. Outcomes of this focus will be assessed through the research components of the CED Doctoral Portfolio. Ratings for students in counseling psychology were all exemplary. We will continue to teach and require our research course. In addition, we will add new content to an existing course on critical analysis of therapy outcomes research (course is COUN 8400). 5
Education, Education, Psychology, Expected Outcome 2 Expected Outcome Title: Knowledge Full Description of Expected Outcome: For students dissertations, they will demonstrate proficiency in their ability to: (a) Introduction, (b) Review of the Literature, (c) Methods and Analysis, and (d) Conclusion Key Assessments: Dissertation Evaluation Assessment Method Description Students work is assessed across four domains including: (a) Introduction, (b) Review of the Literature, (c) Methods and Analysis, and (d) Conclusion. Each domain is assessed across four levels of proficiency: (1) Exemplary, (2) Adequate, (3) Marginal, and (4) Poor. Each level is described with program specific indicators (rubric). The assessment is completed faculty mentor in the program. The following link provides access this specific key assessment: http://www.education.auburn.edu/files/assessment/key_assessment_grad/dissert ationrubric.pdf Figure B: Dissertation Evaluation * Using the recommended guidelines for dissertations, as defined by the Auburn University Graduate School, please indicate a rating from 1 (Poor) to 4 (Exemplary) for each of the following sections of the required Dissertation: Introduction The introduction includes a clear statement of the student's purpose or hypothesis to be tested, an overview of the problem or subject as it is known from the literature, and a broad statement summarizing the findings of the student's study. Review of the Literature The literature review is composed of a comprehensive review of all background knowledge and circumstances pertinent to the subject of the dissertation. It includes a thorough discussion and full literature review (normally considered too long and unnecessary in journal articles and other academic publications). The review provides a unique and valuable reference resource for other scholars in the field of study. Methods and Analysis The methods portion includes appropriate detail in description of (if applicable): subjects, design/approach, methods/procedures, and statistical analyses. All pertinent results are reported in clear, concise and unbiased manner. 6
Education, Education, Psychology, Conclusions The conclusion provides a brief and concise discussion of major findings/outcomes. This section is noted by its accuracy and thought provoking contributions to the field. It describes the importance of the work in a comprehensive manner. Recommendations are appropriate and clearly based on outcomes. Findings Summary On the Dissertation Evaluation Key Assessment, the College of Education aggregate for Special Education,, and s at the Doctoral level indicates specific strengths related to Literature Review. Ratings for those items indicate 100% performance at the competent or exemplary level. Relative to those levels, Introduction, Methods and Analysis and Conclusion also had 100 percent of the ratings at the competent or exemplary level, but fewer students rated at the exemplary level. Thus, relative to the strengths within the College of Education Introduction, Methods and Analysis and Conclusion needs the most attention with respect to this key assessment at the college level. For individual Special Education,, and s at the Doctoral level, indicators that require more attention will be based on the data tables below. (See Table 3a 3d) Table 3a: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Introduction) Introduction Category Exemplary Adequate Marginal Poor Mean (Aggregate) 7 58.33% 5 41.67% 0 0.00% 0 0.00% 3.58 Education 1 33.33% 2 66.67% 0 0.00% 0 0.00% 3.33 Education 3 60.00% 2 40.00% 0 0.00% 0 0.00% 3.60 Psychology 3 75.00% 1 25.00% 0 0.00% 0 0.00% 3.75 7
Education, Education, Psychology, Table 3b: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Literature Review) Literature Review Category Exemplary Adequate Marginal Poor Mean (Aggregate) 8 66.67% 4 33.33% 0 0.00% 0 0.00% 3.67 Education 2 66.67% 1 33.33% 0 0.00% 0 0.00% 3.67 Education 3 60.00% 2 40.00% 0 0.00% 0 0.00% 3.60 Psychology 3 75.00% 1 25.00% 0 0.00% 0 0.00% 3.75 Table 3c: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Methods and Analysis) Methods and Analysis Category Exemplary Adequate Marginal Poor Mean (Aggregate) 7 58.33% 5 41.67% 0 0.00% 0 0.00% 3.58 Education 1 33.33% 2 66.67% 0 0.00% 0 0.00% 3.33 Education 3 60.00% 2 40.00% 0 0.00% 0 0.00% 3.60 Psychology 3 75.00% 1 25.00% 0 0.00% 0 0.00% 3.75 Table 3d: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Conclusion) Conclusion Category Exemplary Adequate Marginal Poor Mean (Aggregate) 7 58.33% 5 41.67% 0 0.00% 0 0.00% 3.58 Education 0 0.00% 3 100.00% 0 0.00% 0 0.00% 3.00 Education 4 80.00% 1 20.00% 0 0.00% 0 0.00% 3.80 Psychology 3 75.00% 1 25.00% 0 0.00% 0 0.00% 3.75 8
Education, Education, Psychology, Improvement During the 2012 13 academic year, the findings suggest that attention needs to be given to the areas of Introduction, Methods and Analysis and Conclusion. Table 4 provides a description of actionable steps that will be taken by program based on the data presented from the Dissertation Evaluation. Table 4: Use of Findings for Improvement Special Education,, and Doctoral s The Education program will integrate additional content related to the development of research. This Education will include additional work on research development and the Education Psychology dissertation process. The Education program will integrate additional content related to the development of research. This will include additional work on research development and the dissertation process. The outcome will be assessed through the research components of the CED Doctoral Portfolio All students attained competence and majority were exemplary. We will continue to integrate scientific writing into coursework and will retain and continue to teach our required research course and continue to retain the required statistics courses. In addition, we have identified additional statistics courses on campus students can take if they wish to have additional depth in statistics coursework. Expected Outcome 3 Expected Outcome Title: Generalizable Knowledge Full Description of Expected Outcome: For students dissertations, they will demonstrate proficiency in their ability to: (a) Significance of the Problem, (b) Appropriateness in the Field, (c) Methodology, and (d) Grammar, Clarity, Style of Organization 9
Education, Education, Psychology, Key Assessments: Dissertation Evaluation Assessment Method Description Students work is assessed across four domains including: (a) Significance of the Problem, (b) Appropriateness in the Field, (c) Methodology, and (d) Grammar, Clarity, Style of Organization. Each domain is assessed across five or four levels of proficiency. The first two indicators are assessed across five levels (a) Excellent, (b) Good, (c) Fair, (d) Poor, and (e) Undetermined while items (c) and (d) are assessed across four levels (1) Accurate/Excellent, (2) Questionable/Good, (3) Inaccurate/Fair, and (4) Undetermined/Poor. The assessment is completed an outside reader for the dissertation evaluation. The assessment instrument itself is provided and deposited in the document repository for the Graduate School. Findings Summary On the Outside Reader Dissertation Evaluation Key Assessment, the College of Education aggregate for Special Education,, and s at the Doctoral level indicates specific strengths related to Significance of the Problem, Appropriateness in the Field, and Methodology. Based on the variability of scores, Grammar, Clarity, Style of Organization appears to be the indicator that needs attention as it relates to this key assessment. Thus, relative to the strengths within the College of Education Grammar, Clarity, Style of Organization needs the most attention with respect to this key assessment at the college level. For individual Special Education,, and s at the Doctoral level, indicators that require more attention will be based on the data tables below. (See Table 5a 5d) 10
Education, Education, Psychology, Table 5a: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Significance of the Problem) Significance of the Problem Category Excellent Good Fair Poor Undet. Mean N % (Aggregate) 11 73.33% 4 26.67% 0 0.00% 0 0.00% 0 0.00% 4.73 Education 4 80.00% 1 20.00% 0 0.00% 0 0.00% 0 0.00% 4.80 Education 4 66.67% 2 33.33% 0 0.00% 0 0.00% 0 0.00% 4.67 Psychology 3 75.00% 1 25.00% 0 0.00% 0 0.00% 0 0.00% 4.75 Table 5b: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Appropriateness in the Field) Appropriateness in the Field Category Excellent Good Fair Poor Undet. Mean N % (Aggregate) 15 100.00% 0 0.00% 0 0.00% 0 0.00% 0 0.00% 5.00 Education 5 100.00% 0 0.00% 0 0.00% 0 0.00% 0 0.00% 5.00 Education 6 100.00% 0 0.00% 0 0.00% 0 0.00% 0 0.00% 5.00 Psychology 4 100.00% 0 0.00% 0 0.00% 0 0.00% 0 0.00% 5.00 Table 5c: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Methodology) Methodology Category Accurate Questionable Inaccurate Poor Mean (Aggregate) 15 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Education 5 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Education 6 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 Psychology 4 100.00% 0 0.00% 0 0.00% 0 0.00% 4.00 11
Education, Education, Psychology, Table 5d: Frequency and Percentages of individuals in Special Education,, and Doctoral s. (By performance category: Grammar, Clarity, Style of Organization) Grammar, Clarity, Style of Organization Category Exemplary Adequate Marginal Poor Mean (Aggregate) 10 66.67% 4 26.67% 1 6.67% 0 0.00% 3.60 Education 4 80.00% 1 20.00% 0 0.00% 0 0.00% 3.80 Education 4 66.67% 1 16.67% 1 16.67% 0 0.00% 3.50 Psychology 2 50.00% 2 50.00% 0 0.00% 0 0.00% 3.50 Improvement During the 2012 13 academic year, the findings suggest that attention needs to be given to the areas of Grammar, Clarity, Style of Organization. Table 6 provides a description of actionable steps that will be taken by program based on the data presented from the Outside Reader Dissertation Evaluation. Table 6: Use of Findings for Improvement Special Education,, and Doctoral s Given that the clarity and grammar was the area of greatest concern, the director of the program forwards announcements Education about available opportunities for graduate students with the university writing center and encourages students identified as having a weakness in this area to pursue assistance at the writing center. This reporting year was the first year for which the writing center held writing workshops and we are not yet able to evaluate Education Psychology the effectiveness of this intervention and additional instruction. The Education program will include in COUN 8510 content relevant to writing and disseminating research. This will include inclusion of training from the AU Writing Center. Given that the clarity and grammar was the area of greatest concern, the director of the program forwards announcements about available opportunities for graduate students with the university writing center and encourages students identified as having a weakness in this area to pursue assistance at the writing center. This reporting year was the first year for which the writing center held writing workshops and we are not yet able to evaluate the effectiveness of this intervention and additional instruction. 12