Assessing the Effectiveness of a Counseling Training Program in CSD Maggie Hilliard, M.A., SLP-CF Quantum HP USD 207 Janet Gooch, Ph.D., CCC-SLP Truman State University ASHA Convention 2012 Atlanta, Georgia
Conflict of Interest Disclosure I adhere to the Conflict of Interest Disclosure Policy of the American Speech- Language-Hearing Association. My thesis is available through UMI ProQuest Dissertations and Theses. As the author, I receive compensation when my thesis is purchased.
Overview Introduction Review of Literature Methodology Results Discussion & Recommendations Feedback & Conclusions Questions & Discussion
Learner Outcomes 1. Describe the role of counseling in Communication Sciences and Disorders. 2. Define three key elements of an effective counseling training program. 3. Describe two considerations for enhancing the methods of evaluation of students in practicum experiences in CSD.
Review of the Literature
Review of Literature Counseling in CSD ASHA Governing Documents Past Research Assessment in Higher Education Self-Assessment Counselor Rating Forms
ASHA Governing Documents Counseling defined as a fundamental component and guiding principle in the clinical process Included counseling in the Scope of Practice, Preferred Practice Patterns, and Standards for Implementation Procedures
Counseling Research in CSD McCarthy, Culpepper, & Lucks (1986) Culpepper, Mendel, & McCarthy (1994) Atkins (2007) Alavez et al. (2009) Millar, Harrow, & Morgan (2010) Kaderavek, Laux, & Mills (2004)
Self-Assessment and Peer Evaluation Boud & Falchikov (1989) Falchikov & Boud (1989) Nicol & Macfarlane-Dick (2006) Stefani (1994) Austin & Gregory (2007) Mort & Hansen (2010)
Counselor Rating Forms Russell-Chapin & Sherman (2000) Syracuse University (2010)
Methodology
Research Purpose To examine if CSD students demonstrated improvements in their counseling skills as a result of participation in a counselor training program
Participants 15 Student Counselors 17 Counselees 15 Student Peers 8 Clinical Supervisors Others Licensed Professional Counselor Professor/SLP Research Investigator
Primary Instruments Clinical Experience Questionnaire Counseling Experience in Communication Disorders Counseling Interaction Evaluation Survey Initial and Second Debriefing Forms Counseling Exit Surveys Refer to handout for examples.
Components of Study Initial Interview Initial Debriefing Mid- Semester Conference Second Interview Second Debriefing
Protocol Initial Interview Beginning of semester Interview between counselor and counselee Counseling Interaction Evaluation Survey Initial Debriefing Meeting held within two weeks of initial interview Counselor and student investigator Debriefing Record Form Initial
Protocol Mid-Semester Conference Typical component of clinical practicum Parent/caregiver, clinical supervisor, student clinician Held around midterm break Reviewed by student investigator using the Mid-Semester Review Form
Protocol Second Interview End of semester Interview between counselor and counselee Counseling Interaction Evaluation Survey Second Debriefing Meeting held within two weeks of second interview Counselor and student investigator Debriefing Record Form Second
Results & Discussion
Counseling Experience of Participants Mean number of counseling hours per week Undergraduates: 0.9 hours (range 0-2; mode 1) Graduates: 1.2 hours (range 0-4; mode 1) Supervisors: 3.6 hours (range 0-13; mode 1) Counseling contexts Undergraduates: coursework and practicum Graduate students: practicum Supervisors: practicum and clinical practice
Participants Understanding of Counseling 14 12 Number of Respondents 10 8 6 4 2 Undergraduate Graduate Supervisors 0 1: Strongly Disagree 2: Disagree 3: Neutral 4: Agree 5: Strongly Agree Summary of Responses to the Statement: It is within a speechlanguage pathologist s/audiologist s role to provide counseling services to individuals with communication/hearing disorders.
Participants Understanding of Counseling Counseling coursework Graduate students anticipated completing 1.2 hours (range 0-6, mode 0) 50% of supervisors did not complete counseling coursework as graduate students
Perceptions Toward Counseling Training Undergraduate Students 90% - both undergraduate and graduate program Graduate Students 75% - graduate program only Supervisors 100% - graduate program 63% - graduate practicum 50% undergraduate program
Perceptions of the Counselees Hypothesized to be crucial in the evaluation of counseling interactions Results: Counselees primarily rated students clinicians as excellent counselors Counselees ratings did not prove useful in evaluating the counseling skills
Improvement in Counseling Skills Wilcoxon Signed Ranks test P-value of 0.05 or less Assessed across clinical clock hour categories Results Collective group: 22/28 items Clinical clock hours Low group: 17 skills; overall improvement Mid group: 5 skills High group: 16 skills; overall improvement
Outcomes of Video Review Wilcoxon Signed Ranks Test Alpha value of 0.05 or less Initial and second interviews combined to obtain a comprehensive value for each group and overall Student Clinician Ratings Review Group P-Value Similarity Low 0.007 Not the same Middle 0.327 Same High 0.003 Not the same Overall 0.421 Same
Congruency Between Raters Repeated Measures ANOVA P-value of 0.05 or less Based on the role of the rater (i.e., supervisor, student peer, etc.) Results P-value = 0.617 Implications Raters were congruent in their ratings on the initial and final interactions
Congruency Between Raters One-Way ANOVA P-value of 0.05 or less Compared improvement as indicated by rating scores and roles of the raters Results P-value = 0.003 Implications Raters were less reliable in their ratings with regard to improvement from the initial and final interviews
Congruency Between Raters First Interview One-Way ANOVA P-value of 0.05 or less Compared total score of each rater to other raters using only the first interview Results P-value = 0.000 Second Interview One-Way ANOVA P-value of 0.05 or less Compared total score of each rater to other raters using only the second interview Results P-value = 0.744
Congruency Between Raters Post-hoc test Majority of variation occurred during first interview Results Group 1 Licensed counselor, research investigator, supervisor, student peer Group 4 Student clinician immediate, student clinician video, counselee Follow-up test (ANOVA) Group 1 P-value of 0.408
Predictors of Improvement ANOVA P-value of 0.05 or less Results P-value: 0.027 Selected factors compared to improvement ratings of group 1 of reliable raters Perception Questions Predictor Significance: Model Unstandardized Coefficients Standardized Coefficients B Std. Error Beta t Sig. (Constant) 23.721 7.316 3.242.008 1 Comfort Clients -1.221 2.669 -.120 -.457.656 Comfort Relatives -4.219 1.985 -.541-2.125.057 Feel Prepared 7.750 2.416.723 3.207.008
Outcomes of Goal-Directed Targets Goal-Directed Over half of participants improved both 93% of participants improved at least one Non-Goal-Directed Majority improved at least 2 of the 3 behaviors Only 1 student showed no improvement
Feedback & Conclusions
Feedback on Counseling Training Program Benefits Student clinicians Use of goals was valuable, meeting with investigator enhanced the process Student Peers Opportunities to observe new skills and ideas for counseling, enhanced their understanding of effective counseling, encouraged reflection
Feedback on Counseling Training Program Challenges Student Clinicians Inconvenient timing with the clinic schedule, uncertainty during the initial interview, emotional vulnerability, unclear questions on survey Student Peers Need for more clearly defined terms, difficulty with video logistics (volume, angle, etc.)
Feedback on Counseling Training Program Supervisor Feedback 50% felt the program should be a required component of the clinical experience Recommendations More guidance during the initial interview, more discussion and learning opportunities, revisions to the evaluation survey
Feedback on Counseling Training Program Students identified many of the anticipated benefits Lack of agreement between who should evaluate counseling skills Training program effectively incorporated the core components recommended by Nicol and Macfarlane-Dick (2006)
Conclusions Limitations Counseling skills evaluated Qualitative judgments Time constraints Recommendations Counseling Training Program Modifications to primary survey Time frame
Conclusions Future Research Focus on the role of peer assessment and supervisor training Counseling class study Spring 2012
Summary Counseling plays an integral role in the clinical dynamic Present study piloted a practical program to effectively train counseling skills Academic programs should explicitly target counseling skills in practicum experiences
References Alavez, C., Calvert, E., Foster, L., Heibert, L., Helvig, R., Savage, K., Sedlock, N., King, J., Lindstedt, E., Towle-Harmon, M. (2009). Increasing graduate students counseling competencies with early intervention families. Poster session presented at the American Speech-Language-Hearing Association Convention, New Orleans, LA. American Speech-Language-Hearing Association [ASHA]. (2004). Preferred Practice Patterns for the Profession of Speech-Language Pathology [Preferred Practice Patterns]. Available from www.asha.org/policy. American Speech-Language-Hearing Association [ASHA]. (2005). Standards and Implementation Procedures for the Certificate of Clinical Competence in Speech-Language Pathology [Standards and Implementation Procedures]. Available from http://www.asha.org/certification/slp_standards/. American Speech-Language-Hearing Association [ASHA]. (2007). Scope of Practice in Speech- Language Pathology [Scope of Practice]. Available from www.asha.org/policy. Atkins, C.P. (2007). Graduate SLP/Aud clinicians on counseling: Self-perceptions and awareness of boundaries. Contemporary Issues in Communication Science and Disorders, 34, 4-11. Austin, Z., & Gregory, P.A. (2007). Evaluating the accuracy of pharmacy students selfassessment skills. American Journal of Pharmaceutical Education, 71(5). Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/pmc2064887/ Culpepper, B., Mendel, L.L., & McCarthy, P.A. (1994). Counseling experience and training offered by ESB-accredited programs. ASHA, 36(6-7), 55-58. Falchikov, N., & Boud, D. (1989). Student self-assessment in higher education: A meta-analysis. Review of Educational Research, 59(4), 395-430.
References Kaderavek, J.N., Laux, J.M., & Mills, N.H. (2004). A Counseling training module for students in speech-language pathology training programs. Contemporary Issues in Communication Science and Disorders, 31, 153-161. McCarthy, P., Culpepper, N.B., & Lucks, L. (1986). Variability in counseling experiences and training among ESB-accredited programs. ASHA, 28(9), 49-52. Millar, D.C., Harrow, K. L., & Morgan, A.A. (2010). Current issues: Incorporating counseling into graduate education. Perspectives on Administration and Supervision, 20, 54-58. Mort, J.R., & Hansen, D.J. (2010). First-year pharmacy students self-assessment of communication skills and the impact of video review. American Journal of Pharmaceutical Education, 74(5). Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/pmc2907843/ Parkinson, K., & Rae, J.P. (1996). The understanding and use of counselling by speech and language therapists at different levels of experience. European Journal of Disorders of Communication, 31, 140-152. Russell-Chapin, L.A., & Sherman, N.E. (2000). The counselling interview rating form: A teaching and evaluation tool for counselor education. British Journal of Guidance and Counselling, 28(1), 115-124. Stefani, L. (1994). Peer, self and tutor assessment: Relative reliabilities. Studies in Higher Education, 19(1), 69-75. Syracuse University: Counseling and Human Services. (2010). Counseling interview rating form. Retrieved from http://soeweb.syr.edu/media/documents/2009/9/counseling_interview_rating_form.pdf
Questions & Discussion
Contact Information Maggie Hilliard margaret.d.hilliard@gmail.com Janet L. Gooch jquinzer@truman.edu