Assessing the Effectiveness of a Counseling Training Program in CSD

Transcription

1 Assessing the Effectiveness of a Counseling Training Program in CSD Maggie Hilliard, M.A., SLP-CF Quantum HP USD 207 Janet Gooch, Ph.D., CCC-SLP Truman State University ASHA Convention 2012 Atlanta, Georgia

2 Conflict of Interest Disclosure I adhere to the Conflict of Interest Disclosure Policy of the American Speech- Language-Hearing Association. My thesis is available through UMI ProQuest Dissertations and Theses. As the author, I receive compensation when my thesis is purchased.

3 Overview Introduction Review of Literature Methodology Results Discussion & Recommendations Feedback & Conclusions Questions & Discussion

4 Learner Outcomes 1. Describe the role of counseling in Communication Sciences and Disorders. 2. Define three key elements of an effective counseling training program. 3. Describe two considerations for enhancing the methods of evaluation of students in practicum experiences in CSD.

5 Review of the Literature

6 Review of Literature Counseling in CSD ASHA Governing Documents Past Research Assessment in Higher Education Self-Assessment Counselor Rating Forms

7 ASHA Governing Documents Counseling defined as a fundamental component and guiding principle in the clinical process Included counseling in the Scope of Practice, Preferred Practice Patterns, and Standards for Implementation Procedures

8 Counseling Research in CSD McCarthy, Culpepper, & Lucks (1986) Culpepper, Mendel, & McCarthy (1994) Atkins (2007) Alavez et al. (2009) Millar, Harrow, & Morgan (2010) Kaderavek, Laux, & Mills (2004)

9 Self-Assessment and Peer Evaluation Boud & Falchikov (1989) Falchikov & Boud (1989) Nicol & Macfarlane-Dick (2006) Stefani (1994) Austin & Gregory (2007) Mort & Hansen (2010)

10 Counselor Rating Forms Russell-Chapin & Sherman (2000) Syracuse University (2010)

11 Methodology

12 Research Purpose To examine if CSD students demonstrated improvements in their counseling skills as a result of participation in a counselor training program

13 Participants 15 Student Counselors 17 Counselees 15 Student Peers 8 Clinical Supervisors Others Licensed Professional Counselor Professor/SLP Research Investigator

14 Primary Instruments Clinical Experience Questionnaire Counseling Experience in Communication Disorders Counseling Interaction Evaluation Survey Initial and Second Debriefing Forms Counseling Exit Surveys Refer to handout for examples.

15 Components of Study Initial Interview Initial Debriefing Mid- Semester Conference Second Interview Second Debriefing

16 Protocol Initial Interview Beginning of semester Interview between counselor and counselee Counseling Interaction Evaluation Survey Initial Debriefing Meeting held within two weeks of initial interview Counselor and student investigator Debriefing Record Form Initial

17 Protocol Mid-Semester Conference Typical component of clinical practicum Parent/caregiver, clinical supervisor, student clinician Held around midterm break Reviewed by student investigator using the Mid-Semester Review Form

18 Protocol Second Interview End of semester Interview between counselor and counselee Counseling Interaction Evaluation Survey Second Debriefing Meeting held within two weeks of second interview Counselor and student investigator Debriefing Record Form Second

19 Results & Discussion

20 Counseling Experience of Participants Mean number of counseling hours per week Undergraduates: 0.9 hours (range 0-2; mode 1) Graduates: 1.2 hours (range 0-4; mode 1) Supervisors: 3.6 hours (range 0-13; mode 1) Counseling contexts Undergraduates: coursework and practicum Graduate students: practicum Supervisors: practicum and clinical practice

21 Participants Understanding of Counseling Number of Respondents Undergraduate Graduate Supervisors 0 1: Strongly Disagree 2: Disagree 3: Neutral 4: Agree 5: Strongly Agree Summary of Responses to the Statement: It is within a speechlanguage pathologist s/audiologist s role to provide counseling services to individuals with communication/hearing disorders.

22 Participants Understanding of Counseling Counseling coursework Graduate students anticipated completing 1.2 hours (range 0-6, mode 0) 50% of supervisors did not complete counseling coursework as graduate students

23 Perceptions Toward Counseling Training Undergraduate Students 90% - both undergraduate and graduate program Graduate Students 75% - graduate program only Supervisors 100% - graduate program 63% - graduate practicum 50% undergraduate program

24 Perceptions of the Counselees Hypothesized to be crucial in the evaluation of counseling interactions Results: Counselees primarily rated students clinicians as excellent counselors Counselees ratings did not prove useful in evaluating the counseling skills

25 Improvement in Counseling Skills Wilcoxon Signed Ranks test P-value of 0.05 or less Assessed across clinical clock hour categories Results Collective group: 22/28 items Clinical clock hours Low group: 17 skills; overall improvement Mid group: 5 skills High group: 16 skills; overall improvement

26 Outcomes of Video Review Wilcoxon Signed Ranks Test Alpha value of 0.05 or less Initial and second interviews combined to obtain a comprehensive value for each group and overall Student Clinician Ratings Review Group P-Value Similarity Low Not the same Middle Same High Not the same Overall Same

27 Congruency Between Raters Repeated Measures ANOVA P-value of 0.05 or less Based on the role of the rater (i.e., supervisor, student peer, etc.) Results P-value = Implications Raters were congruent in their ratings on the initial and final interactions

28 Congruency Between Raters One-Way ANOVA P-value of 0.05 or less Compared improvement as indicated by rating scores and roles of the raters Results P-value = Implications Raters were less reliable in their ratings with regard to improvement from the initial and final interviews

29 Congruency Between Raters First Interview One-Way ANOVA P-value of 0.05 or less Compared total score of each rater to other raters using only the first interview Results P-value = Second Interview One-Way ANOVA P-value of 0.05 or less Compared total score of each rater to other raters using only the second interview Results P-value = 0.744

30 Congruency Between Raters Post-hoc test Majority of variation occurred during first interview Results Group 1 Licensed counselor, research investigator, supervisor, student peer Group 4 Student clinician immediate, student clinician video, counselee Follow-up test (ANOVA) Group 1 P-value of 0.408

31 Predictors of Improvement ANOVA P-value of 0.05 or less Results P-value: Selected factors compared to improvement ratings of group 1 of reliable raters Perception Questions Predictor Significance: Model Unstandardized Coefficients Standardized Coefficients B Std. Error Beta t Sig. (Constant) Comfort Clients Comfort Relatives Feel Prepared

32 Outcomes of Goal-Directed Targets Goal-Directed Over half of participants improved both 93% of participants improved at least one Non-Goal-Directed Majority improved at least 2 of the 3 behaviors Only 1 student showed no improvement

33 Feedback & Conclusions

34 Feedback on Counseling Training Program Benefits Student clinicians Use of goals was valuable, meeting with investigator enhanced the process Student Peers Opportunities to observe new skills and ideas for counseling, enhanced their understanding of effective counseling, encouraged reflection

35 Feedback on Counseling Training Program Challenges Student Clinicians Inconvenient timing with the clinic schedule, uncertainty during the initial interview, emotional vulnerability, unclear questions on survey Student Peers Need for more clearly defined terms, difficulty with video logistics (volume, angle, etc.)

36 Feedback on Counseling Training Program Supervisor Feedback 50% felt the program should be a required component of the clinical experience Recommendations More guidance during the initial interview, more discussion and learning opportunities, revisions to the evaluation survey

37 Feedback on Counseling Training Program Students identified many of the anticipated benefits Lack of agreement between who should evaluate counseling skills Training program effectively incorporated the core components recommended by Nicol and Macfarlane-Dick (2006)

38 Conclusions Limitations Counseling skills evaluated Qualitative judgments Time constraints Recommendations Counseling Training Program Modifications to primary survey Time frame

39 Conclusions Future Research Focus on the role of peer assessment and supervisor training Counseling class study Spring 2012

40 Summary Counseling plays an integral role in the clinical dynamic Present study piloted a practical program to effectively train counseling skills Academic programs should explicitly target counseling skills in practicum experiences

41 References Alavez, C., Calvert, E., Foster, L., Heibert, L., Helvig, R., Savage, K., Sedlock, N., King, J., Lindstedt, E., Towle-Harmon, M. (2009). Increasing graduate students counseling competencies with early intervention families. Poster session presented at the American Speech-Language-Hearing Association Convention, New Orleans, LA. American Speech-Language-Hearing Association [ASHA]. (2004). Preferred Practice Patterns for the Profession of Speech-Language Pathology [Preferred Practice Patterns]. Available from American Speech-Language-Hearing Association [ASHA]. (2005). Standards and Implementation Procedures for the Certificate of Clinical Competence in Speech-Language Pathology [Standards and Implementation Procedures]. Available from American Speech-Language-Hearing Association [ASHA]. (2007). Scope of Practice in Speech- Language Pathology [Scope of Practice]. Available from Atkins, C.P. (2007). Graduate SLP/Aud clinicians on counseling: Self-perceptions and awareness of boundaries. Contemporary Issues in Communication Science and Disorders, 34, Austin, Z., & Gregory, P.A. (2007). Evaluating the accuracy of pharmacy students selfassessment skills. American Journal of Pharmaceutical Education, 71(5). Retrieved from Culpepper, B., Mendel, L.L., & McCarthy, P.A. (1994). Counseling experience and training offered by ESB-accredited programs. ASHA, 36(6-7), Falchikov, N., & Boud, D. (1989). Student self-assessment in higher education: A meta-analysis. Review of Educational Research, 59(4),

42 References Kaderavek, J.N., Laux, J.M., & Mills, N.H. (2004). A Counseling training module for students in speech-language pathology training programs. Contemporary Issues in Communication Science and Disorders, 31, McCarthy, P., Culpepper, N.B., & Lucks, L. (1986). Variability in counseling experiences and training among ESB-accredited programs. ASHA, 28(9), Millar, D.C., Harrow, K. L., & Morgan, A.A. (2010). Current issues: Incorporating counseling into graduate education. Perspectives on Administration and Supervision, 20, Mort, J.R., & Hansen, D.J. (2010). First-year pharmacy students self-assessment of communication skills and the impact of video review. American Journal of Pharmaceutical Education, 74(5). Retrieved from Parkinson, K., & Rae, J.P. (1996). The understanding and use of counselling by speech and language therapists at different levels of experience. European Journal of Disorders of Communication, 31, Russell-Chapin, L.A., & Sherman, N.E. (2000). The counselling interview rating form: A teaching and evaluation tool for counselor education. British Journal of Guidance and Counselling, 28(1), Stefani, L. (1994). Peer, self and tutor assessment: Relative reliabilities. Studies in Higher Education, 19(1), Syracuse University: Counseling and Human Services. (2010). Counseling interview rating form. Retrieved from

43 Questions & Discussion

44 Contact Information Maggie Hilliard Janet L. Gooch