Condor-Dolphin Peer Mentor Outreach and Transfer Program: Classroom Presentations and One-on-One Peer Mentoring Services

Transcription

1 Condor-Dolphin Peer Mentor Outreach and Transfer Program: Classroom Presentations and One-on-One Peer Mentoring Services Project ASCENCION Oxnard College s Title V STEM grant Evaluation Report 7/17/2013 Submitted to Dr. Cynthia Herrera Title V STEM Director, Project ASCENCION Prepared by Myley Dang

2 Executive Summary Project ASCENSION Oxnard College, in cooperation with California State University Channel Islands (CSUCI), University of California at Santa Barbara (UCSB), Oxnard School District (OSD), and Oxnard Union High School District (OUHSD) set as one of its goals to continue to establish an early readiness awareness framework for STEM degrees among Hispanic students. Under Project ASCENSION, the Condor-Dolphin Peer Mentor Outreach and Transfer Program has two components. First, peer mentors from CSUCI conduct presentations and provide information on the transfer process and financial aid to different classrooms at Oxnard College. Second, Oxnard College students have the opportunity to follow up with the peer mentors, who hold peer mentoring service hours at the Oxnard College STEM Center. Students are welcomed to schedule an appointment or drop in to see the peer mentors. The goal of this peer mentoring outreach service is to increase students knowledge of the transfer process and financial aid, while also improving students self-esteem, social skills, and career opportunities when they meet one-on-one with the peer mentors. The complete logic model of the Condor- Dolphin Peer Mentor Outreach and Transfer Program is found in Appendix A. Concentrating on the program improvement, the Condor-Dolphin Peer Mentor Outreach and Transfer Program was evaluated using student and faculty surveys from the classroom presentations. Student and faculty responses were assessed in this report and the results indicate that the students and faculty members tended to respond favorably to most components of the classroom presentations. Also, the students and faculty members suggested some comments on how to improve the presentations including providing handouts and making the PowerPoint presentation slides available online. There are also other recommendations provided on how to improve the classroom presentations and one-on-one peer mentoring services. 2

3 Background Under Oxnard College s Title V STEM grant ( Project ASCENCION ), CSUCI peer mentors visited classrooms once during the Fall 2012 semester and twice throughout the Spring 2013 semester. The peer mentors presented information on the transfer process during the first class visit and presented information on financial aid during the second class visit. The student and faculty evaluation surveys were distributed after the second class visit. Since only one workshop was presented during the fall semester, there was no evaluation forms administered after the workshop. Thus, this evaluation report will focus on the evaluation forms from the spring semester and the one-on-one peer mentoring service during the spring semester. PARTICIPANTS During the Spring 2013 semester, the classroom presentations were conducted in three physics and two chemistry classes at Oxnard College. Students and faculty were asked to fill out the presentation evaluation forms after the financial aid presentation. There were two faculty members and 98 students who completed these forms. After the classroom presentations, students had the opportunity to follow up one-on-one with peer mentors in the Oxnard College STEM Center. Students signed in through the peer mentors log sheet (see Appendix D). Based on the log sheet, there were only five students who attended the peer mentoring service during the spring semester, where one student visited twice. Thus, only four students utilized the peer mentoring service, which is relatively low considering that the peer mentors provided presentations to four classes. Recommendations on how to increase the number of students attending the one-on-one peer mentoring service will be discussed in the conclusions and recommendations section. 3

4 Evaluation PROTOCOL This evaluation will provide information to help to improve the presentation and one-on-one peer mentoring services in the future. Thus, this evaluation is a formative and summative evaluation formative in the sense that the evaluation is being conducted for the purpose of program improvement and summative in the sense that the evaluation will examine how valuable the presentations were. This evaluation measures the implementation of the presentations through the administration of surveys to Oxnard College students and faculty. The surveys include both closed-ended (i.e., Likert-scale items, multiple choice) and open-ended response questions and statements. Results from the student and faculty surveys suggest ways in which the presentations could be subsequently improved in the future, and offer evidence of what went well, what did not go well, and reasons behind the perceived successes and shortcomings of the presentations. A descriptive analysis of the Likert-scale items will disclose the following: how valuable students felt the presentations were, the students transfer plans, the utilization of the peer mentoring service, what information students felt will be beneficial during the transfer process, and students preference in advising sessions. Open-ended questions and statements elicit responses that can be used for improvement. All surveys were in paper form and were completed after the second workshop, which was the financial aid presentation. Student and faculty responses are analyzed in this report. The survey instruments administered to the students and faculty are found in Appendices B and C, respectively. ANALYSES AND RESULTS The following protocol was used in conducting the analyses of both the student and faculty surveys: 4

5 1. Blank items were removed from the data set and these observations were coded as missing. 2. Not applicable (N/A) responses were also not analyzed, since the purpose of this evaluation is to determine students perceptions of the presentation in order to propose improvements for the future. STUDENT SURVEYS: DESCRIPTIVE STATISTICS There were 98 students who completed the survey. Students were asked to report how valuable they felt the transfer process and/or financial aid presentations were. The results are displayed in Tables 1 and 2 respectively. As shown in Table 1, over 76% of students felt the transfer process presentation was valuable, where 43.9% of students found the presentation somewhat valuable and 32.7% of students found it to be very valuable. Note that 4.1% of students indicated N/A and 6.1% had missing responses, which may indicate that students were not present for the transfer process presentation. Table 1 How Valuable Was The Transfer Process Presentation to You? Count Percent Not valuable at all 1 1.0% Somewhat not valuable 2 2.0% Neutral % Somewhat valuable % Very valuable % N/A 4 4.1% Missing 6 6.1% Total % As shown in Table 2, over 62% of students felt the financial aid presentation was valuable, where 33.7% of students found the presentation somewhat valuable and 28.6% of students found it to be very valuable. Note that 7.1% of students indicated N/A and 9.2% had missing responses, which may indicate that students were not present for the financial aid presentation. 5

6 Table 2 How Valuable Was the Financial Aid Presentation to You? Count Percent Not valuable at all 1 1.0% Somewhat not valuable 5 5.1% Neutral % Somewhat valuable % Very valuable % N/A 7 7.1% Missing 9 9.2% Total % Students were also asked to report whether or not they plan to transfer to a four-year institution, and if so, when they plan on transferring. These results are displayed in Table 3. Of the 98 students surveyed, 83.7% of the students plan on attending a four-year institution, and 11.2% do not plan on attending. This is important to know since the presentation was about the transfer process; thus it seems appropriate that the presentation was directed to this audience. It is also important to know how to provide resources and support these students during their transfer process. Table 3 Do You Plan to Attend a 4-Year Institution? Count Percent Yes % No % Missing 5 5.1% Total % Table 4 shows the students reported transfer date for those students who plan on transferring. Students were asked the term of transfer, however, many students left this field blank. Therefore only the planned year of transfer is displayed in the results. Of the 82 students who plan on 6

7 transferring, 32.9% plan to transfer in 2013 and 36.6% plan to transfer in There were 3.7% of students who have already transferred and 2.4% of students who did not know when they plan on transferring. There were also 12.2% of students who did not report a transfer date. Table 4 Students Reported Transfer Date to a 4-Year Institution Count Percent Already transferred 3 3.7% Don t know 2 2.4% % % % % Missing % Total % Students were also asked if they have used or plan to use the peer-mentoring services offered at the STEM Center. Of the 98 students surveyed, Table 5 shows that 40.8% of students responded yes, while 57.1% of students responded no, indicating that either the student did not use the peer mentor services or the student did not plan to use these services. Table 5 Have You Used or Plan to Use the Peer Mentor Services at the STEM Center to Inform the Transfer Process? Senior Plans Count Percent Yes % No % Missing 2 2.0% Total % 7

8 Although this survey item elicited favorable response, the results displayed in Table 5 are difficult to interpret. One issue with this survey item is that a yes response does not distinguish if students have used the peer mentoring service or if they plan to use the peer mentoring service in the future. Similarly, a no response does not distinguish if students have not used the peer mentoring service or if they do not plan to use the peer mentoring service in the future. Therefore, this question should be revised for future evaluation forms to be more explicit. Recommendations of how this question can be revised are provided in the conclusion and recommendation section. In addition, based on the log signup sheet, there were only four students who followed up with peer mentors, suggesting that the majority of students who responded yes do plan to use the peer mentoring service in the future. Thus, it is recommended to reach out to these students and inform them the resources available to them at the STEM Center. More recommendations will be discussed in the conclusion and recommendation section. Students were also asked what information will be most beneficial to them through the transfer process including financial aid/fafsa, admission/transfer process/application assistance, internships/job opportunities, and personal statement. Students also had the option to write in their own opinion of what they think will be most beneficial to them under the other option. This question allowed respondents to check all options that applied, thus the total is more than the total number of student respondents. The percentage was calculated out of 98, which is the total number of students who responded to the survey. However, because some students marked multiple responses, the total percentage is above 100%. As displayed in Table 6, 55.1% of students responded that admission/transfer process/application assistance will be most beneficial, followed by 52% of students who responded that financial aid/fafsa, and 44.9% of students responded internships/job opportunities will be most beneficial to them through the transfer process. There were 19.4% of students who reported that information on writing a personal statement will be beneficial. There was one student who responded Other, and wrote getting accepted will be beneficial. Note missing or blank responses were not included because students had the option to choose all options that applied to them. Thus only options that were selected were included in Table 6. All together, these 8

9 percentages suggest that providing information on the entire transfer process from the beginning to the end is important. In other words, it is important to provide information on how to prepare to transfer, going through the transfer application process and applying for financial aid, and what to do once students are accepted all of which is critical information for students to know when they are transferring. Table 6 What Information will be Most Beneficial to You through the Transfer Process? Count Percent Financial Aid/FASFA % Admissions/Transfer Process/Application Assistance % Internships/Job Opportunities % Personal Statement % Other 1 1.0% Total % The last item on the presentation evaluation survey asks students their preferred method of advising sessions, whether it is through personal advising, group workshops or presentations, or both. The results displayed in Table 7 shows that 34.7% of students who preferred personal advising, and 51% of students preferred to be advised both in personal sessions and in group workshops/presentations. These percentages indicate there is student interest in at least the personal advising session, which advocates for continuing the peer mentoring service for next year. There were 9.2% of students who preferred group workshops or presentations, which combined with the both response of 51% of students, suggests that continuing the group presentations will also be helpful for students.. There were 1% of students who preferred neither session, and there were 4.1% of students who did not report a preference. It would be interesting to follow up with these 5.1% of students to see which method he/she prefers the best. 9

10 Table 7 Do You Prefer Personal Advising Sessions or Group Workshops/Presentations? Count Percent Personal advising session % Group workshops or presentations 9 9.2% Both % None 1 1.0% Missing 4 4.1% Total % The following section will describe the students responses to the open-ended fields including their reasons for not attending the peer mentor services at the STEM Center and any additional comments, thoughts, or questions the students had. STUDENT SURVEYS: OPEN-ENDED RESPONSES Students had the opportunity to add comments after answering the question, Have you used or plan to use the peer mentor services at the STEM Center to inform the transfer process? Since this evaluation is focused on the improving the presentation, common and unique responses from students responses are highlighted below. Some common responses from the students include the following (misspellings are preserved): o already transferred / graduated / too late looking to go to grad school Maybe to go to a nursing school I already got accepted and just need to talk to a counselor at Pomona. I'm already transferring but thank you! o Did not need it o Won't be transferring for awhile / not transferring yet o Did not know about it 10

11 Didn't notice this. I will! A little early still, but I definitely will at some point. Haven't gone yet Havent had time because work Not enough time, but I will come by soon. haven't heard of it I was unaware about the services. o just found out about it o Do it on my own Gone thru UCMentor, but thank you! I used Assist.org using Cal Poly for help Tutor Some unique responses from the students include the following (misspellings are preserved): o I thought it was not useful o I'm uncomfortable around people o Transfer presentations received after application deadline. The students open-ended responses to the peer-mentoring services question suggests students did not need the services since they already transferred or that students did not know about the services. Some students found other resources to help them through the transfer process including UCMentor, Assist.org or through other counselors. It is important to note the unique responses from students. One student reported that he/she did not think that the service is useful, while another student mentioned he/she is uncomfortable around people. For future peermentoring services, it might be helpful that students are aware of these peer mentor services as well as providing a warm and inviting atmosphere for students to feel welcome to seek peer mentoring services. Students also noted that the presentations should take place before the transfer application and financial aid deadlines. More specific recommendations on how to improve the peer mentor services will be discussed in the conclusion and recommendations sections. 11

12 Students also had the opportunity to add any comments, thoughts, or questions they had for the peer mentors. Since this evaluation is focused on the improving the presentation, common and unique responses from students responses are highlighted below. Some positive comments from the students include the following (misspellings are preserved): o great presentation, keep it up! o Overall, it was informative and easy to follow. o Interesting presentations. Thank you. o Good presentation, good attitude o Always a good refresher o I think your presentation is awesome and will give a better understanding to students who are not aware of these programs o Overall great presentation for those who are going to transfer o Great job on presentation - would have been helpful if I was transfering =) o thank you for the presentation! o Thank you for being available for questions via . --> Important for those w/ tight schedules. Some recommendations from the students include the following (misspellings are preserved): o You should go into more detail about the process of applying for grants. o Financial aid presentation should be before March 2 nd o Make powerpoints available online o Let us know when you have food. o You should offer help with applications for scholarships. o Looking back, I wish someone had told me about the Pell grant and the BOG waiver sooner maybe touch on this for students over 24 yrs. o Other universities outside of California would be a great thing to mention. o Should give handout of the websites you have in powerpoints The students open-ended responses in the comments section suggest that although the presentations were perceived to be a success for some students, there is room for improvement. 12

13 For instance, many student comments indicated that the presentations were great overall. Students felt the presentations were a good refresher and it was informative and easy to follow. However, there were also specific recommendations on how to improve the presentations. Students strongly suggested having the presentations take place before the transfer and financial aid deadlines. Students also suggested having handouts of the websites or PowerPoint slides available online. Students would have also liked to see more information about scholarship applications, as well as mentioning other universities outside of California. One student suggested informing students when food is available, which might encourage some students to come to the peer-mentoring services. More recommendations on how to improve the event will be discussed in the conclusion and recommendations sections. The following section examines the responses from the faculty surveys. FACULTY SURVEYS: DESCRIPTIVE STATISTICS There were two faculty members who completed the survey. Faculty members were asked to report how valuable they felt the transfer process and financial aid presentations were. The results are displayed in Tables 8 and 9 respectively. As shown in Table 8, one faculty member felt the transfer process presentation was somewhat valuable, while the other faculty member felt it was very valuable. Table 8 How Valuable was the Transfer Process Presentation to You? Count Percent Not valuable at all 0 0.0% Somewhat not valuable 0 0.0% Neutral 0 0.0% Somewhat valuable % Very valuable % N/A 0 0.0% Total % 13

14 As shown in Table 9, both faculty members felt the financial aid presentation was very valuable. Table 9 How Valuable was the Financial Aid Presentation to You? Count Percent Not valuable at all 0 0.0% Somewhat not valuable 0 0.0% Neutral 0 0.0% Somewhat valuable 0 0.0% Very valuable % N/A 0 0.0% Total % Faculty members were also asked to indicate the extent to which they agreed or disagreed about the transfer and financial aid presentations taking too much time away from class and the extent to which the presentations were a positive experience for them. Table 10 shows the results from the statement, The presentation took too much time away from my class for the transfer process presentation. Both faculty members disagreed with this statement, confirming that they found the presentation at least somewhat valuable, as illustrated previously in Table 8. Table 10 The Transfer Process Presentation Took Too Much Time away from My Class Count Percent Strongly disagree 0 0.0% Disagree % Neutral 0 0.0% Agree 0 0.0% Strongly agree 0 0.0% Total % 14

15 Table 11 shows the results from the statement, The presentation took too much time away from my class for the financial aid presentation. Both faculty members disagreed with this statement, confirming that they found the presentation valuable, as illustrated previously in Table 9. Table 11 The Financial Aid Presentation Took Too Much Time away from My Class Count Percent Strongly disagree 0 0.0% Disagree % Neutral 0 0.0% Agree 0 0.0% Strongly agree 0 0.0% Total % Table 12 displays the results from the statement, The presentation was a positive experience for me for the transfer process presentation. Both faculty members agreed with this statement, confirming that they found the presentation valuable, as illustrated previously in Table 8. Table 12 The Transfer Process Presentation Was a Positive Experience for Me Count Percent Strongly disagree 0 0.0% Disagree 0 0.0% Neutral 0 0.0% Agree % Strongly agree 0 0.0% Total % 15

16 Table 13 displays the results from the statement, The presentation was a positive experience for me for the financial aid presentation. Both faculty members agreed with this statement, confirming that they found the presentation valuable, as illustrated previously in Table 9. Table 13 The Financial Aid Presentation Was a Positive Experience for Me Count Percent Strongly disagree 0 0.0% Disagree 0 0.0% Neutral 0 0.0% Agree % Strongly agree 0 0.0% Total % The two faculty members were asked if they were open to having these presentations conducted in their classrooms again. Both faculty members agreed they would like to see the presentations again in their classrooms. One faculty member would like to see both presentations conducted again, while the other faculty member would only like to see the financial aid presentation in his/her classroom again. Faculty members also had the option to participate in a short interview to discuss their experience about having the presentations in their class; however, both faculty members declined to participate in an interview. The following section will describe the faculty members responses to the open-ended fields including what they think would make the presentation(s) more effective and any additional comments, thoughts, or questions they had. FACULTY SURVEYS: OPEN-ENDED RESPONSES Faculty members responded to the question, What do you think would make the presentation(s) more effective? Since this evaluation is focused on the improving the presentation, common and unique responses from students responses is highlighted below: 16

17 It is all good information handouts of the websites may help. Possible more activities The faculty members responses suggest that although the presentations were perceived to be a success, there is room for improvement. One of the faculty members resonated with some of the students recommendations, which is to include handouts of the websites mentioned in the presentations. The other faculty member suggested having more activities included in the presentation. This may be a good idea to engage with the students in the class. Faculty members also had the opportunity to add any comments, thoughts, or questions they had. There was only one faculty member who added a comment, which is displayed below (misspellings are preserved): Good information for students, should of more resources! Thank you! This comment suggests that the presentation was good overall for students, although there should be additional resources available. The findings from both the student and faculty surveys will be further summarized in the following discussion section. DISCUSSION Students and faculty tended to have favorable perceptions of the transfer process and financial aid presentations as measured by the student and faculty surveys. In examining the descriptive statistics of the survey items describing the value of the presentations, both students and faculty members felt that both the transfer process and financial aid presentations were somewhat valuable to very valuable. The majority of students who filled out the survey indicated they plan to transfer within 2013 through When asked about the utilization of the peer-mentoring services, 57.1% of students responded no, which suggests that they did not need the services, they were not aware of the services, and/or they were not ready to transfer. Future peermentoring services might consider advertising their services throughout the Oxnard College campus, as well as making their presence known at the STEM Center. Further recommendations will be discussed in further detail in the recommendations and conclusion section. One question item from the student survey indicated that some students felt the information on financial aid/ FAFSA (52% of students), admission/transfer process/application assistance 17

18 (55.1% of students), and internship/job opportunities (44.9% of students) would be very beneficial for students going through the transfer process. Only 19.4% of students felt the information on writing a personal statement will be helpful during the transfer process. These results suggest that peer mentors should provide more information on financial aid, admission and transfer process, and internship/job opportunities that will benefit the students who plan on or are already going through the transfer process. In terms of students advising preferences, 51% of students would like to be advised both personally and through group workshops or presentations, while 34.7% of students prefer to be advised through a personal advising session. These results suggest that students are interested in peer advising and thus, peer mentors should continue holding both personal sessions as well as having class presentations to accommodate students who prefer just the personal sessions and students who like both. The results from the faculty surveys indicated that the faculty members had a positive experience with both transfer process and financial aid presentations. Both faculty members also felt the two presentations did not take too much time away from their classes. Combining the results from both of these findings suggests that faculty had positive perceptions of the presentations. An examination of the open-ended responses confirmed the results from the descriptive analysis of the survey items for students and faculty. For students, the open-ended responses regarding the students use of the peer mentor services indicate that some students will not need the services yet, did not need the services, and/or were not aware of the services. Some students mentioned they thought it was not useful or did not feel comfortable around people. This suggests that peer mentors should have a more inviting space to make students feel welcome, as well as provide beneficial information on the transfer process and financial aid. One student stated the transfer presentations were delivered after the applications were due. It will be helpful to plan out classroom presentations that are well ahead of the transfer application and financial aid application deadlines. This will provide students with ample time to meet with peer-mentors one-on-one to review their applications and address their questions before the deadline. In terms of ways to improve the event, students and faculty members suggested there could have been handouts of the websites, make the PowerPoint slides available online, have more activities 18

19 during the presentations, discuss other universities outside of California, provide information about scholarship and grant applications, and inform students when food will be available during the peer mentor sessions. Details on how to improve the event will be discussed in the conclusion and recommendation section. LIMITATIONS AND BARRIERS There were some limitations that affected the interpretation of the survey data. First, the evaluation of the presentations only took place during the spring semester. Due to timing constraints and schedule conflicts, there was only one workshop during the fall semester. Typically the evaluations are distributed to students after the second workshop is conducted, but since the second workshop did not take place in the fall semester, there was no evaluation. Second, only two faculty members returned the faculty surveys. The peer mentors conducted four presentations in classrooms, in which two of the classes were taught by one faculty member. Since the peer mentors visited four classes, the two faculty responses may not reflect the sentiments felt by the other one faculty member who did not fill out a survey. The third limitation was the wording on some of the survey items. For example, on the student survey, the item Do you plan to attend a four-year institution and, if so, when? does not specify whether the four-year institution is a college, university, professional school, or graduate school. It also does not take into account of students who already transferred and have been accepted to an institution. It appeared that some of the students were unsure of how to respond to this question since some students provided some clarifications in the blank space next to the no response. Future student surveys should clarify this question and perhaps ask if students plan on transferring, and if so, ask the student to specify which institution they plan to transfer to. In addition, the transfer term and year seemed to confuse some students. Therefore, to make the question more explicit, it is recommended to ask students their planned transfer term and planned transfer year, as well as the institution in which they have already been accepted or plan to transfer to. The fourth limitation was the lack of providing all possible answer choices for students to select from. One of the survey questions did not provide an exhaustive list of responses, thereby forcing students to write in their own responses. For example, the question, Do you prefer 19

20 personal advising sessions or group workshops/presentations? did not have all possible answer choices, which lead one student to respond none. This option should be included as one of the responses for future surveys. Finally, another limitation is the short time frame in developing the surveys for students and faculty. The process of survey design typically takes weeks, perhaps even months, to draft, modify, test, and retest before making the final version of the survey. Given that the evaluators only had a few weeks to design the faculty survey, it was challenging to include all the questions we were interested in. The evaluators also did not have enough time to test out the survey items, and thus did not catch some of the poor phrasing or wording of the responses on some of the survey items. 20

21 Conclusions and Recommendations The peer mentor presentations served Oxnard College students by providing information on the transfer process and financial aid. There were 98 students and two faculty members who completed and returned the evaluation survey forms. The presentations focused on exposing students to the transfer process as well as informing students about college applications and financial aid. These presentations aim to impart important knowledge onto students about navigating the track to four-year colleges and universities. These presentations were designed in part to meet the goal of increasing students interest in transferring and learning more about how to pay for college. Analysis of the data collected suggests that students and faculty members tended to be satisfied with the presentations and many reported positive experiences and future anticipated behaviors (i.e., students transferring to a four-year college or university, faculty members open to having presentations in their classrooms in the future), as measured by certain survey item statements. The open-ended responses also tended to be favorable, although some students and faculty members believed that certain improvements could be made in future events. Therefore, it is reasonable to conclude that the presentations were a worthwhile experience for students as it provided students with opportunities to learn about the transfer process to a four-year college or university, as well as how to apply for financial aid. Participation in such presentations may contribute to more intermediate outcomes, such as increasing student interest in STEM and in higher education, and may be a factor in the achievement of more distal outcomes, such as increasing the rates of students applying for college, transfer to a four-year institution, attending college or university, majoring in STEM fields, and pursuing STEM careers. To improve the implementation of this presentation and one-on-one peer mentoring services in the future, some recommendations are provided. Also, there are suggestions to improve the data collection that will better assist in the evaluation. It should be noted that this is the first year the classroom presentations and one-on-one follow up peer mentoring service are being evaluated. Thus, this is a formative evaluation such that recommendations are provided to improve the presentations and one-on-one peer mentoring service for the next school year. These 21

22 recommendations are presented for each classroom presentation and one-on-one peer mentoring service below. RECOMMENDATIONS FOR PRESENTATIONS AND EVALUATION FORMS Start planning early! Many students suggested having these presentations early on, prior to the transfer application and financial aid deadlines. This will help with the organization and planning of the presentations, as well as the coordination of the faculty members who are interested in having these presentations in their classes. This may also motivate students to seek one-on-one peer mentoring services to get help with the transfer application(s) and/or financial aid application. Students suggested that peer mentors provide more detail on the process of applying for grants and scholarships, as well as giving information on the BOG waiver. This information will be useful for students who are applying for financial aid and seeking additional resources to fund college. Students also suggested provided more information on schools outside of California to expand their options. Make the PowerPoint slides available online. Another suggestion is to print the slides on a handout so students have the information available. This handout should also include important deadlines such as the transfer application and financial aid deadlines. It may also be helpful to include the peer mentors contact information and a map to the STEM Center so students can the peer mentors with questions or come into the STEM Center to speak with the peer mentors one-on-one. To help with the data collection and increase response rates, it is recommended to collect presentation evaluation forms after each workshop, rather than administer them after the second workshop. This might have been helpful during the fall semester, when only one workshop was conducted. Having evaluations after each workshop might also be helpful because students may not be able to recall their feelings and/or attitudes about the first workshop. In addition, some students might not be present for both workshops, and will not be able to fill out their responses to both workshops. Thus collecting evaluation forms after each workshop may increase student response rates. 22

23 If the technology is available, another possibility for data collection is using clickers for the Likert-scale items at the end of the presentations and handing out shorter surveys with the open-ended response questions to capture overall impressions of the event. This might be a quicker way to capture students data as well as eliminate the additional step to enter the data manually. In terms of the survey items, it is recommended to revise the survey item, Have you used or plan to use the peer mentor services at the STEM Center to inform the transfer process? The possible response options are Yes and No. Why not? which includes an open ended response. It is unclear what a Yes response signifies because it could either indicate that the student did use the peer mentoring service or it could also indicate that the student did not use the service, but plans to use the peer mentoring service in the future. This is a similar issue for a No response. This may cause confusion as students may not know how to respond and we may not know how to interpret their responses. Another issue with this question may be that the student might have used the peer mentor services but not to inform the transfer process. They may have used the services for more information on financial aid. Thus a revised item is necessary to clear up any kind of confusion. The revised question may be separated into a skip logic question, which may look as follows: Have you used the peer mentor services at the STEM Center? o Yes (If yes, skip to the next question) o No. Why not? (If no, answer the following question:) Do you plan to use the peer mentor services at the STEM Center? Yes No. Why not? Similarly, as mentioned previously, it is recommended to revise the survey item, Do you plan to attend a four-year institution and, if so, when? since the item does not specify whether the four-year institution is a college, university, professional school, or graduate school. It also does not consider students who already transferred and have been accepted 23

24 to an institution. Thus, it might be helpful to revise the survey item and ask the following item instead: o Planned transfer term (i.e., Spring, Fall): o Planned transfer year: o Planned transfer school: o Accepted transfer school (if applicable): The following is a list of recommendations for the one-on-one peer mentoring service held at the STEM Center. RECOMMENDATIONS FOR ONE-ON-ONE PEER MENTORING SERVICE Advertise the peer mentor services more! Both the closed- and open-ended comments indicate that students have an interest in the peer mentoring service, but based on the log sign in sheet, there have not been many people who have actually used the one-on-one peer mentoring service. Many students mentioned that they did not know these services existed until after the presentation. o Perhaps it will be useful to post flyers around campus that includes a map of where the STEM Center is located. o It might also be helpful to professors to make announcements in class about these services. This might also be a good opportunity to see if faculty members are interested in having these presentations in their classes. o Another recommendation is to make announcements during the STEM Stew, which is a seminar for STEM OC instructors to discuss any issues they are experiencing and/or provide useful professional development to instructors. It is possible that students may not have time to meet with the peer-mentors, thus it is recommended that the peer mentors provide their contact information on the board after they present to the classes. Have one consistent sign in sheet where students seeking one-on-one peer mentoring services can sign in. Currently, there are multiple versions of the sign in sheet, which 24

25 does not provide consistent information across the different versions. In addition, this log in sheet, typically does not have all the fields filled out completely. There have been recommendations in previous discussions with the project director and project coordinator to have students fill out a student satisfaction survey after they finish their peer mentoring session, in which they can also fill in information that is required from the log sheet. This will not only keep the log in sheet more consistent, but students can also answer questions about their experience utilizing the peer mentoring services. The evaluators are planning to administer the peer mentoring student satisfaction survey during the Fall 2013 semester. Follow up with students who utilized the one-on-one peer mentoring services. The log sheet contains a tab for correspondences, but it is currently blank, which suggests that there were no correspondences between the students and mentors. Future peer mentoring services should follow up with students to see if the mentor has addressed all of the student s questions and to see if the student has any further additional questions for the mentor. This may be one step in building a stronger student-mentor relationship. This may also encourage students to come back to the peer mentoring services or share this resource with their friends. A theory of action, or logic model, should be revised to distinguish the classroom presentations from the one-on-one peer mentoring service, which may better assess the implementation of the overall Condor-Dolphin Peer Mentor Transfer Outreach program. This theory of action should be made more explicit within the already existing theory of action (Appendix A). Without a specific theory of action of each of the presentations and peer mentoring service, it is challenging to draw conclusions that could address the strengths and weaknesses of these activities and how the presentations and one-on-one follow up fits within the overall program goals and objectives. Also, a summative, or outcomes evaluation cannot be properly conducted without a theory of action. 25

26 Appendix A: Logic Model for Condor-Dolphin Outreach and Transfer Program Resources (Human, financial, organizational, and community resources to direct the work) Activities (Intentional processes, tools, events, technology, & actions, used to bring change/intended results) Outputs (Direct products of program activities, may includes type, levels, and targets of services to be delivered by the program) Outcomes (Specific changes in program participants behaviors, knowledge, skills, status and level of functioning) Immediate Long-term CSU Channel Islands Peer Mentors Math and science faculty Oxnard College STEM Center and resources CSU Channel Islands Outreach Coordinator STEM Grant money Presentations on the transfer process Presentations on financial aid Mini-presentation evaluations Peer-mentoring services provided at the STEM Center Students attend presentations on the transfer process and financial aid Students receive information on the transfer process and financial aid Students attend the peermentoring services provided by the STEM center (sign-in sheet)* Students develop mentorships* Increased awareness of STEM Center s peer-mentoring services Students receive support from mentors Students collaborate with their mentors Vicarious experience of success through tutors/ role models (If they can do it, I can do it too) Increased knowledge of the transfer process and financial aid Improved self-esteem (role models), social skills (mentorships), and career opportunities (presentations, one-onone follow-up) Form learning communities* Increase retention at OC/ transfer rate to a four-year institution Increase STEM degree completion * We currently do not have a measure for this, but we think it is an important output/ outcome. 26

27 Appendix B: Student Mini-Presentation Evaluation Form Evaluating the STEM Outreach Classroom mini-presentations In the past several of weeks we visited your class two times (including today) to present information relevant to the process of transferring from Oxnard College to a 4-year institution. Specifically, we covered the following topics: The transfer process (e.g., general info, deadlines) Financial Aid We also presented the peer-mentor drop-in services available at the STEM Center. We would appreciate your feedback on the presentations and the service! 1. How valuable was each mini-presentation to you? The transfer process Not valuable at all Somewhat not valuable Neutral Somewhat valuable Very valuable Financial Aid N/A N/A 2. Do you plan to attend a 4-year institution and, if so, when? a. Yes. I plan to attend in the term of 20 (year) b. No 3. Have you used or plan to use the peer-mentor services at the STEM Center to inform the transfer process? a. Yes b. No. Why not? 4. What information will be most beneficial to you through the transfer process? a. Financial Aid/FASFA b. Admissions/Transfer Process/Application Assistance c. Internships/Job Opportunities d. Personal Statement e. Other 5. Do you prefer personal advising sessions or group workshops/ presentations? a. Personal advising session b. Group workshops or presentations c. Both 6. Please add any comments, thoughts, or questions you might have for us 27

28 Appendix C: Faculty Mini-Presentation Evaluation Form Evaluating the STEM Outreach Classroom Mini-Presentations - Faculty Satisfaction Survey In the past several of weeks we visited your class two times (including today) to present information relevant to the process of transferring from Oxnard College to a 4-year institution. Specifically, we covered the following topics: The transfer process (e.g., general info, deadlines, university culture) Financial aid We also presented the peer-mentor drop-in services available at the STEM Center. We would appreciate your feedback on the presentations and the service! 1. How valuable or not valuable do you think each mini-presentation was to your students? The transfer process Not valuable at all Somewhat not valuable Neutral Somewhat valuable Very valuable Financial aid N/A N/A 2. Please indicate the extent to which you agree or disagree with the following statements. The presentation took too much time away from my class. The presentation was a positive experience for me. Strongly disagree Disagree Neutral Agree Strongly agree Transfer Financial Aid Transfer Financial Aid What do you think would make the presentation(s) more effective? 4. Are you open to having these mini-presentations conducted again in your classroom in future semesters? a. Yes. Which one(s)? b. No. Why not? 5. Are you willing to be interviewed about your experience? a. Yes. Please provide your address or phone number. b. No 6. Please add any comments, thoughts, or questions you might have for us. Thank you! 28

29 Appendix D: Peer Mentoring Outreach Log Sheet In Person Mentoring Date Name Student ID # Mentor Major Expected Transfer Date Start Time End Time Referred By Phone Number Topics Discussed New or Returning Student Follow up Meeting Scheduled? ( , in person) 29