Evaluating the Effectiveness of a Career Certificate Program for International Students Jessamyn Perlus, B.A., Julia Panke Makela, Ph.D., Daniel Scholes, Ph.D., Un Yeong Park, M.Ed., Gaeun Seo, M.S., & Kevin Hoff, B.A. The Career Center, University of Illinois at Urbana-Champaign
The Career Center Research Team We work at the intersection of research and practice Four team members (1 full time, 3 PhD students) Located within a practice environment Advising and leading a cross-functional Assessment Team Exploring research questions, with opportunities for immediate application to practice 2
International (INTL) Students in the U.S. US: The world s TOP education destination for INTL students Number of International students studying in the U.S. increased continuously from 526,670 during the 2005/06 academic year to 854,639 in the 2014/15 academic year Number of International Students Studying in U.S. Colleges and Universities 526,670 541,324 567,039 605,015 623,119 647,246 679,338 724,725 780,055 854,639 + 62% 2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 2013/14 2014/15 Data Source: Institute of International Education, 2015 3
INTL Students Unique Career Challenges Career challenges to obtaining employment in the U.S. Cultural adjustment and language barriers (Lin & Flores, 2011; Lopez, 2002; Reynolds & Constantine, 2007) Complicated employment regulations (e.g., CPT, OPT, H1BVisa) (Kisch, 2015; Lin & Flores, 2011) Career development responses to challenges Challenges have been shown to lower career aspirations or limit consideration of options (Lopez, 2002; Reynolds & Constantine, 2007) International students express strong needs for assistance with career exploration and the job search process (Shen & Herr, 2004; Spencer- Rodgers, 2000) A growing number of higher education career services offices have added dedicated programs/staff to serve international students (NCDA, 2015) 4
Study Purpose The literature acknowledges: Growing population of international students The unique challenges that these students face The development of programs and services to address their needs Yet, what do we know about the outcomes of those programs? How well do career services programs meet the needs of international students? 5
Career Intervention Outcomes Meta-Analytic Studies Consistently demonstrate moderate effect sizes career interventions are effective in relation to a wide variety of career choice outcomes E.g., career maturity, career decision-making self-efficacy, choice certainty, choice satisfaction, congruence, decidedness, indecision, vocational identity (E.g., Baker & Popowizc, 1983; Brown & Ryan-Krane, 2000; Brown et al., 2003; Oliver & Spokane, 1988; Whiston et al., 1998) Critical Intervention Components Written exercises Individualized interpretation and feedback World of work information Modeling opportunities Attention to building support for choices within one s social network (Brown & Ryan Krane, 2000) 6
Career Certificate International (CCI) Innovative 9-week career certificate program dedicated to helping international students become a more competitive job/internship candidate in the U.S. Includes weekly workshops, peer mentoring, and participation in a variety of career services throughout one academic semester Offered by The Career Center at Illinois (TCC) since Fall 2012 7
Method Data collected first and last weeks of 9-week CCI program 3 data sources Indirect Measure: Career Search Efficacy Scale (CSES) Direct Measures: Resumes Written networking introduction statements 8
Career Search Self-Efficacy Self-efficacy Individuals beliefs concerning his or her capability to successfully perform a certain task or behavior (Bandura, 1977, 1986) Can I do this? Career Search Efficacy Scale (CSES, Solberg et al., 1994) Focus on career/job search related domains 9
Method - Career Search Efficacy Scale (CSES) 4 subscales: Job Search Efficacy, Interviewing Efficacy, Personal Exploration Efficacy, & Networking Efficacy 1 new subscale: International Student Job Search Efficacy 1. Discuss CPT/OPT options with an employer 2. Communicate effectively with an employer in English 3. Identify employers more friendly to international students 4. 5. Utilize social media like LinkedIn to identify and connect with individuals who can support international students' career development Identify and utilize career resources dedicated to international students Analysis: Paired samples t-tests to examine change over time 10
Method Document Analysis Documents were blind-reviewed by trained reviewers Resume uses established rubric (Rooney et al., 2012) Networking script rubric designed for this study Analysis: Paired samples t-tests to examine change over time 11
Participants 49 students completed the Spring 2015 program; 43 (88%) participated in the evaluation study Student Response Rates: 31 (63%) completed self-reported pre- and postassessment of Career Search Efficacy Scale (CSES) 43 (88%) submitted pre- and post-resumes 40 (82%) submitted pre- and post-elevator pitches Countries Represented: Bangladesh, China, Ecuador, Indonesia, Malaysia, Mongolia, and South Korea 12
Demographics of Program Participants Missing 10% GENDER Male 29% Missing 13% SEXUAL IDENTITY Female 61% Prefer not to respond 16% Heterosexual 68% Bisexual 3% RACIAL / ETHNIC IDENTITY Asian Hispanic or Latino Prefer not to respond Missing 10% 3% 3% 84% Missing 10% Graduate 35% CLASS STANDING Freshman 13% Sophomore 16% Junior 26% 13
Results Career Search Efficacy Career Search Efficacy Scale 10 9 8 7 6 5 4 3 2 1 0 5.7 ***Overall Career Search Efficacy Scale 7.7 7.8 7.6 7.7 7.6 7.9 6.0 ***Job Search Efficacy 5.4 5.7 ***Interview Efficacy ***Networking Efficacy 6.1 ***Personal Exploration Efficacy 5.3 ***Int l Student Job Search Efficacy *** p >.001 Pretest Posttest 14
1 = Incomplete, 2 = Developing, 3 = Proficient, 4 = Professional Results Resumes 4 3.5 3.6 3.8 3 2.5 2.5 2.7 2.2 2.4 2.5 2.6 2.9 3.0 2.3 2.6 2.8 2.9 2.9 2.4 2 1.5 1.4 1.5 1 **Overall *30 Second Review Presentation Format Spelling/ Grammar *Heading Education *Described Experience Optional Sections *Positive Professional Language * p >.05, ** p >.01, *** p >.001 Pretest Posttest 15
1 = Incomplete, 2 = Developing, 3 = Proficient, 4 = Professional Results Networking Scripts 4 3.5 3.5 3.6 3 2.5 2.4 2.6 2.6 2.8 2.5 2.9 2.6 2.8 2.3 2.3 2.4 2.8 2 1.5 1.9 1.7 1.5 1.7 1 Overall Hook *Desribed Experiences Goals/ Aspirations Assess Fit with Company Closing Length Grammar & Sentence Structure *Clear Story & Positive Language * p >.05 Pretest Posttest 16
Summary of Findings Significant increases in all domains of Career Search Efficacy Scale (CSES) indicate that the CCI program is making a difference in students beliefs in their ability to successfully engage in career search tasks. Significant increases in quality on written resumes and elevator pitches, particularly on rubric subscales that are targeted by the program (e.g., 30 Second Review, Described Experience, Positive Professional Language, Clear Story). 17
Implications for Practice Share Successes Communicate successful aspects of the program to colleagues and future participants Modifications to CCI Program Strengthen closings on networking scripts Recognize program boundaries Provide list of common grammatical mistakes that international students make and direct participants to resources on campus for assistance New Resources: Networking Script Rubric Created a version to use in staff training and student workshops Updated research version to clarify criteria to improve interrater reliability 18
Implications for Research What about this intervention makes a difference? How can it be improved? Can relationships be observed between programs for international students (the intervention) and: Developmental outcomes, e.g. reduced influence of perceived barriers on career aspirations Process outcomes, e.g. increased knowledge and strategies for completing career development tasks Summative outcomes, e.g. obtained employment Comparison outcomes, e.g. vs. domestic students, country of origin 19
Future Directions Qualitative Project in Progress Grant from Midwest ACE to conduct interviews of CCI alums How students perceive each factor of the program contributed to their experience and learning: Mentoring? Peer relationships built with other participants and volunteers? Skills practiced? Resources or information gained during the workshops? What could have helped students make even greater leaps forward? 20
Resources from The Career Center Makela, J. P., Scholes, D. R., Perlus, J. G., Seo, G., & Park, U. (2016). Outcomes of the Career Certificate- International Program, Report. Available upon request. INTL Connect Website: http://www.intlconnect.ill inois.edu/tccservices 21
References (Page 1 of 2) Baker, S. B., & Popowicz, C. L. (1983). Meta-analysis as a strategy for evaluating effects of career education interventions. Vocational Guidance Quarterly, 31, 178 186. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman. Brown, S. D., & Ryan Krane, N. E. (2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed., pp. 740 766). New York, NY: John Wiley & Sons. Brown, S. D., Ryan Krane, N. E., Brecheisen, J., Castelino, P., Budisin, I., Miller, M., & Edens, L. (2003). Critical ingredients of career choice interventions: More analysis and new hypotheses. Journal of Vocational Behavior, 62, 411-428. doi:10.1016/s0001-8791(02)00052-0. Institute of International Education. (20142015). International student enrollment trends, 1948/49-20134/154. Open Doors Report on International Educational Exchange. Retrieved from http://www.iie.org/opendoors Kisch, M. (2015). Helping international students navigate career options. International Educator, 24(3), 66-69. Retrieved from http://www.nafsa.org/_/file/_/ie_mayjun15_fsa.pdf Lin, Y. & Flores, L. Y. (2011). Job search self-efficacy of East Asian international graduate students. Journal of Career Development, 40(3), 186-202. doi:10.1177/0894845311418655. Lopez, A. E. (2002). Factors influencing career decision-making and post-secondary career choices of a cohort group of female Cuban immigrants. Doctoral dissertation. Retrieved from ProQuest. 304788595. 22
References (Page 2 of 2) NCDA International Student Work Group (2015). Resources for partnering with international students. National Career Development Association (NCDA). Retrieved from http://ncda.org/aws/ncda/asset_manager/get_file/64588/resources_for_partnering_with_international_ students.pdf Oliver, L. W., & Spokane, A. R. (1988). Career-intervention outcome: What contributes to client gain? Journal of Counseling Psychology, 35(4), 447 462. Reynolds, A. L., & Constantine, M. G. (2O07). Cultural adjustment difficulties and career development of international college students. Journal of Career Assessment, 15(3), 338 350. doi: 10.1177/1069072707301218. Rooney, G. S., Pipkins, K., DiVietro, M. & Sun, H. (2012). Fall 2012 resume review evaluation findings. [Technical Report]. Champaign, IL: The Career Center, University of Illinois at Urbana-Champaign. Shen, Y. J., & Herr, E. L. (2004). Career placement concerns of international graduate students: A qualitative study. Journal of Career Development, 31(1), 15-29. doi: 10.1023/B:JOCD.0000036703.83885.5d. Solberg, Good, G. E., Nord, D., Holm, C., Hohner, R., Zima, N., Heffernan, M., & Malen, A. (1994). Assessing career search expectations: Development and validation of the Career Search Efficacy Scale. Journal of Career Assessment, 2(2), 111-123. doi: 10.1177/106907279400200202. Spencer-Rodgers, J., & Cortijo, A. (1998). An assessment of the career development needs of international students. Journal of College Student Development, 39(5), 509 513. Retrieved from http://eric.ed.gov/?id=ej581220. Whiston, S. C., Sexton, T. L., & Lasoff, D. L. (1998). Career-intervention outcome: A replication and extension of Oliver and Spokane. Journal of Counseling Psychology, 45, 150-165. 23
Thank You Un Yeong Park and Tina Yeo for their coordination of the CCI program Fall 2015 Assessment Committee for their feedback (Brian Neighbors, Taisha Mikell, Amanda Cox) Midwest ACE for their generous support of our future research in this area 24
25
26
27
28