ANNUAL REPORT FORMAT

Transcription

1 ANNUAL REPORT FORMAT ANNUAL REPORT (July 1, 2013 June 30, 2014) PART 4 PROGRAM LEVEL INSTITUTIONAL EFFECTIVENESS REPORTS NORTH CAROLINA AGRICULTURAL AND TECHNICAL STATE UNIVERSITY Institutional Effectiveness Assessment and Improvement Report Department of Agribusiness, Applied Economics, & Agriscience Education Agricultural Education-Bachelor of Science Anthony Yeboah, Ph.D.

2 1. Expected Outcomes for the Educational Program and Its Student Learning Outcomes Program outcomes and selected student learning outcomes for the Bachelor of Science in Agricultural Education program are as follows: a. Program Outcomes (POs) 1. The Bachelor of Science degree program in Agricultural Education will achieve excellence and recognition for high quality in teaching and learning, including the achievement of NCATE, CAEP and NCDPI accreditation. (A&T Strategic Goals 2, 3, and 6) 2. The Bachelor of Science degree program in Agricultural Education will meet or exceed the University s goals for research/creative productivity, enrollment, retention, degree, completion, and placement of graduates in jobs or graduate education. (A&T Strategic Goals 6 and vision for making a significant difference I the lives of those we educate.) 3. The Bachelor of Science degree program in Agricultural Education will contribute appropriately to the intellectual climate and creative exchange, professionalism, civic engagement, inclusiveness, cultural awareness, and respect for diversity. (A&T Strategic Goals 1, 2, 4, and 5) b. Program Student Learning Outcomes (SLOs) 1. SLO 1: Communication Skills- Students completing the Bachelor of Science degree program in Agricultural Education will exhibit effective communication skills (written, oral, graphic and interpersonal) appropriate for professionals in the agricultural industry at the bachelor s level. 2. SLO 2: Critical Thinking Skills- Students completing the Bachelor of Science degree program in Agricultural Education will effectively use quantitative and/or qualitative analytical problem-solving skills appropriate for professionals in the agricultural industry at the bachelor s level. 3. SLO 3: Disciplinary Expertise- Students completing the Bachelor of Science degree program in Agricultural Education will demonstrate a level of discipline-specific expertise (knowledge, skills, and professionalism) appropriate for professionals in the agricultural industry at the bachelor s level. Page 2 of 8

3 4. SLO 4: Research/Creative Engagement- Students completing the Bachelor of Science degree program in Agricultural Education will demonstrate the ability to engage productively in the review and conduct of disciplinary research and creative professional activity appropriate for professionals in the agricultural industry at the bachelor s level. 2. Analysis of the Extent to Which each Outcome is Achieved Analyze the extent to which each of the POs and the SLOs in the first section of the report has been achieved over the last academic year, based on assessment data. (Answer these questions: How did I measure the outcome? What did I find?) (Provide only data) a. Program Outcomes 1. The Bachelor of Science degree program in Agricultural Education will achieve excellence and recognition for high quality in teaching and learning, including the achievement of NCATE, CAEP and NCDPI accreditation. (A&T Strategic Goals 2, 3, and 6). Through the continuation of maintaining accreditation expectations and requirements, the BS program in Agricultural Education is in alignment and continues to meet the requirements for national accreditation standards. The program is currently in the progress of collecting evidence for the upcoming CAEP accreditation visit in March The Bachelor of Science degree program in Agricultural Education will meet or exceed the University s goals for research/creative productivity, enrollment, retention, degree, completion, and placement of graduates in jobs or graduate education. (A&T Strategic Goals 6 and vision for making a significant difference in the lives of those we educate.) The undergraduate program in Agricultural Education was very productive for the academic year. Overall, 19 students graduated the program during the fall, spring, and summer sessions. Twelve of the 19 students reported they were offered new positions or maintained employment with a promotion. Students accepted jobs from agribusinesses, cooperative extension, local school systems in North Carolina, and government. The three students who were not placed noted that jobs were available, but did not feel that the offers were aligned with their career aspirations. The program had six new students, 13 transfer students for a total of 19 new students for the academic year. Faculty were actively involved in securing grants and participating in scholarship, which led to four students being employed via work-study and who actively participated in the research process by presenting posters at regional and national conferences and students attended literacy awareness events in Page 3 of 8

4 Washington, DC. Four students who were recipients of the USDA NIFA Multicultural Scholarship all graduated on time (see Tables 1, 2, and 3). Table 1: Enrollment Data Academic Year New Students Total Table 2: Retention Rate Cohort Entering First to Second Year Retention Rate % Table 3: Placement Data Graduating Year Degrees Awarded Job Placement Graduate Education Growth/Decline from Previous Year Decline 3. The Bachelor of Science degree program in Agricultural Education will contribute appropriately to the intellectual climate and creative exchange, professionalism, civic engagement, inclusiveness, cultural awareness, and respect for diversity. (A&T Strategic Goals 1, 2, 4, and 5). Due to the diversity of the program, agricultural educations students are often exposed to environments that are research intensive, diverse in nature, and inclusive. For the academic year, four students traveled abroad to Canada. Students also participated in internships across the country with varying degrees of diverse environments that also required them to adhere to the workplace standards of professionalism. It is through these endeavors that the AGED students grow, blossom, and discover what they are truly comprised of. The Collegiate FFA, the student organization of the program, participated in six service and professional development engagements for the academic year. The agricultural education program also co-hosted the National Teach Ag Day event in September 2014 (see Table 4). Page 4 of 8

5 Table 4: Intellectual and Service Activities Academic Year Professional Workshops/ Seminars Conferences Scholarly Publications Scholarly Presentation Other Scholarly Activities Service to the Profession Community Engagement b. Student Learning Outcomes 1. SLO 1: Communication Skills- Students completing the Bachelor of Science degree program in Agricultural Education will exhibit effective communication skills (written, oral, graphic and interpersonal) appropriate for professionals in the agricultural industry at the bachelor s level. The expectations of communication skills via written, oral, graphic, and interpersonal are interweaved throughout the agricultural education program via enrichment activities, Collegiate FFA activities, class assignments, and a numerous of other engagements, activities, expectations, and opportunities for the students. However, major events that students participated in for the academic year included research and poster presentations at national conferences. Eight students also participated in the School of Agriculture and Environmental Sciences Student Showcase of Excellence. Students participated in a variety of recruitment of activities for the program, department, and school to help possible recruit new students, which also reinforced the expectations of communication skills in the academic program. 2. SLO 2: Critical Thinking Skills- Students completing the Bachelor of Science degree program in Agricultural Education will effectively use quantitative and/or qualitative analytical problem-solving skills appropriate for professionals in the agricultural industry at the bachelor s level. The expectations of critical thinking skills via quantitative and qualitative analytical problem-solving skills are also interweaved throughout the agricultural education program via enrichment activities, class assignments, and a numerous of other engagement, activities, expectations, and opportunities for the students. The School of Agriculture and Environmental Sciences through its Research Division offer students the opportunity to actively participate in undergraduate research opportunities. One student is currently involved in this research program. 3. SLO 3: Disciplinary Expertise- Students completing the Bachelor of Science degree program in Agricultural Education will demonstrate a level of discipline-specific expertise (knowledge, skills, and professionalism) appropriate for professionals in the agricultural industry at the bachelor s level. The capstone courses in agricultural Page 5 of 8

6 education, which includes the student teaching internship and extension internship, are the two courses where the faculty is able to determine the disciplinary expertise and the overall quality of the students. By having students in the field for 10 weeks, along with the professional assessment by the industry or educational partner in the field, students are provided a detailed analysis on their strengths and weakness in regard to the profession. Also, it is through theses experiences where a students need for soft skills or employability skills become apparent. For the academic year, eight students were placed in a student teaching experience or extension internship. Overall, the majority of the students performed at the expected level of competency and completed the program requirements in the spring 2014 semester. 4. SLO 4: Research/Creative Engagement- Students completing the Bachelor of Science degree program in Agricultural Education will demonstrate the ability to engage productively in the review and conduct of disciplinary research and creative professional activity appropriate for professionals in the agricultural industry at the bachelor s level. Six students presented their research at various regional and national conferences for the academic year. In class assignments for both the campus and online classes included research assignments that introduced and reinforced to students the expectations of the research process. After an analysis of the data, majority of the students performed well on these types of assignments. 3. Evidence of PO and SLO Improvements Using the Results of the Assessments (Closing the Loop) Summarize and present evidence showing that achievement of each of the Program Outcomes and Student Learning Outcomes has been continuously improved over the years as a function of acting on specific assessment results. (What specific program and learning improvements were made during academic year? What assessment data led to the improvements? Provide anecdotes and examples to help the reader understand how the improvements are continually reviewed and further enhanced each year.) a. Program Outcomes (1) Key Elements: Program will achieve recognition for high quality in teaching and learning and program will achieve national accreditation if available or satisfactory external review. The undergraduate agricultural education program is currently accredited and is preparing for the next upcoming cycle of accreditation for Page 6 of 8

7 (2) Key Elements: Program s annual number of degrees conferred, program s enrollment of declared majors, and program s involvement of students in research/creative projects. For the academic year, a total of 15 students were conferred degrees in Agricultural Education. The program s enrollment was 67 students. The program has also partnered with industry reps to create the SAES Soft Skills Certificate Program, a creative project that will provide students the opportunity to develop their soft skills or employability skills. The feedback from the program s industry partners over the last few years have been the same, they want students who are good team players, open to diversity, and numerous of other soft skills that are necessary to be successful at work. The implementation of this certificate program is an attempt to help alleviate the disconnection that is being noted by faculty and employers in regard to students. The course will be offered as a special problems course for the spring 2015 academic semester. (3) Key Elements: Program will contribute appropriately to intellectual climate and creative exchange, professionalism, civic engagement, diversity and inclusiveness, and cultural awareness. The program will offer more study abroad opportunities for its students via grants and partnerships with international universities. Currently efforts are being made to secure a relationship with a university in Costa Rica and Belize to provide students with an opportunity to travel and study abroad. The 21 st Century grant was funded in September The grant was implemented and has been is successful operation. Outcomes thus far include faculty being trained on how to incorporate teaching online best practices for online and face-to-face courses. Partnerships were also established for the Collegiate FFA Chapter to encourage more interaction among industry personnel to help develop students professionalism, civic engagement endeavors, diversity and inclusiveness appreciation. b. Student Learning Outcomes (1) Key Elements: The students written communication skills, oral communication skills, and interpersonal communication skills are appropriate for the field of study by the end of the program. The agricultural education students participated in national conferences by presenting research and posters this academic year, A total of nine students attended various conferences and events. Other students were provided classroom activities accompanied with faculty-developed and rubricguided assignments that focused on communication skills development. Four students also participated in North Carolina Farm Bureau Discussion Meet, which targets students being able to discuss effectively in a Round Robin Discussion. Eight students also participated in internships that reinforced the need for effective communication skills. (2) Key Elements: The students critical thinking skills as defined as the effective use of quantitative and qualitative analytical problem-solving skills appropriate for Page 7 of 8

8 agricultural education at the BS degree. Students actively engaged in critical thinking assignments that were interweaved with various engagements and activities outside of the classroom for the academic year. Class activities were designed with faculty-developed rubrics to ensure that critical thinking was taught and reinforced throughout the AGED courses, AGED 101, 401, and 403. Faculty will continue to use collaborative learning activities and provide students with field experience assignments that will continue to develop and improve students critical thinking skills. This continuation of improvement is also a recommendation from industry advisors who stated that students often perform better in their work environments after being exposed to the work expectations first. (3) Key Elements: The students possess appropriate level of discipline-specific knowledge and professionalism by the end of the degree program. The agricultural education, undergraduate program successful had 15 students to graduate this academic year. The feedback from their internships and student teaching experiences were extremely positive. One weakness noted were the soft skills development and the program will implement the soft skills certificate program for the 2015 calendar year. (4) Key Elements: The students ability to engage productively in the review of disciplinary research and creative activity at an appropriate level for their degree program. The feedback received from students, employers, and faculty was indeed positive regarding the participation of students in research activity. The program will actively recruit more students to become involved in faculty research endeavors with the expected outcome of students writing journal articles of publication and attending national conferences to present their research. Page 8 of 8

9 ANNUAL REPORT FORMAT ANNUAL REPORT (July 1, 2013 June 30, 2014) PART 4 PROGRAM LEVEL INSTITUTIONAL EFFECTIVENESS REPORTS NORTH CAROLINA AGRICULTURAL AND TECHNICAL STATE UNIVERSITY Institutional Effectiveness Assessment and Improvement Report Department of Business Education Bachelor of Science in Business Education Lisa Gueldenzoph Snyder, Ph.D., Chairperson 1. Expected Outcomes for the Educational Program and Its Student Learning Outcomes Program outcomes and selected student learning outcomes for the BS in Business Education program are as follows: a. Program Outcomes (POs) (1) Excellence and Recognition. The BS in Business Education degree program in the Department of Business Education will achieve excellence and recognition for high quality in teaching and learning, including the achievement of national accreditation by the AACSB International. (2) Enrollment, Retention, and Graduate Placement. The BS in Business Education degree program in the Department of Business Education will meet or exceed the University s goals for enrollment, retention, and degree completion, and will achieve a placement rate of at 75% of its graduates in jobs or graduate education. (3) Program Contributions to Research and Community Engagement. The BS in Business Education degree program in the Department of Business Education will contribute appropriately to intellectual climate and research/creative productivity, community engagement, and respect for diversity.

10 b. Program Student Learning Outcomes (SLOs) (1) Communication Skills. Students completing the BS degree program in Business Education will exhibit effective communication skills (written, oral, graphic, and interpersonal) appropriate for professionals in this field of study at the bachelor s level. (2) Critical Thinking Skills. Students complete the BS degree program in Business Education will effectively use quantitative and qualitative analytical problem-solving skills appropriate for professionals in the discipline at the bachelor s level. (3) Disciplinary Expertise. Students completing the BS degree program in Business Education will demonstrate a level of discipline-specific expertise (knowledge, skills, and professionalism) appropriate for professionals in careers related to business education at the undergraduate level. (4) Research/Creative Engagement. Students completing the BS degree program in business education will demonstrate ability to review and conduct business-related research at the undergraduate level. 2. Analysis of the Extent to Which each Outcome is Achieved a. Program Outcomes (1) Excellence and Recognition. The BS in Business Education degree program in the Department of Business Education will achieve excellence and recognition for high quality in teaching and learning, including the achievement of national accreditation by the AACSB International. The program was last accredited by AACSB International in 2012, following its site visit in November The accreditation reaffirmation report included no recommendations that required a response. The next scheduled accreditation visit will be in Fall In preparation for our continuous improvement review, the Department will continue to gather data on the faculty s intellectual contributions and students performance to document the high quality of the program and faculty as well as assurance of learning. (2) Enrollment, Retention, and Graduate Placement. The BS in Business Education degree program in the Department of Business Education will meet or exceed the University s goals for enrollment, retention, and degree completion, and will achieve a placement rate of at 75% of its graduates in jobs or graduate education. The information in this section was taken from the University Fact Book to document enrollment (Table 1), retention (Table 2), and degree completion (Table 3). Although incomplete at this time, information about placement rates (Table 4) was gathered from the Department s alumni database. Page 2 of 9

11 Enrollment Enrollment in the Department of Business Education for the last three academic years is shown in Table 1. The data indicates a significant increase in AY The faculty does not have an explanation for this one-year growth. Recruitment efforts were consistently maintained throughout the three-year period. Without the surge, we are confident that our numbers would show steady growth (and not the decline) each year. Table 1: Enrollment Data Academic Year Enrollment Growth/Decline from New Freshmen Transfers Total Previous Year % % Retention Rate Retention rates for majors in the Department of Business Education for the last three academic years are outlined in Table 2. The faculty found it interesting that the year with the most growth ( ) was also the year with the lowest first-to-second year retention rate. Several change of major forms were processed that year, leading the faculty to believe that students may have been misinformed of their choice of major during the admission process (rather than a reflection of the Department s true retention efforts). Table 2: Retention Data Cohort Entering First to Second Year Retention Rate (n=06) 83.0% (n=06) 67.0% (n=23) 78.3% Degree Completion Rate Graduation rates for students in the Department of Business Education are presented in Table 3. Although the Department did not reach the goal (18.7%) four-year graduate rate, the six-year goal (42.8%) was surpassed for the Fall 2006 and the Fall 2008 cohorts. Table 3: Graduation Rate Data Cohort Entering Graduation Rates 4 Years 6 Years Fall 2006 (n=06) 00.0% (n=06) 50.0% Fall 2007 (n=10) 00.0% (n=10) 30.0% Fall 2008 (n=05) 00.0% (n=04) 75.0% Fall 2009 (n=18) 16.7% -- Fall 2010 (n=07) 00.0% -- Page 3 of 9

12 Placement Rate Although the placement data are incomplete, Table 4 presents the information currently on file in the Department. Each semester, the Department Chair requests placement information from graduating seniors; however, most students have not finalized their employment plans prior to commencement. Additionally, although s are sent to all Department alumni every semester to request updated information, the response rates have traditionally been only 28.5%. Table 4: Placement Data Graduating Year Placement Rates Degrees Awarded Job Placement Graduate Education Growth/Decline from Previous Year (n=3) 42.0 % (n=2) 28.5 % (n=1) 16.6 % (n=1) 16.6 % % (n=1) 12.5 % (n=0) 00.0% 33.0 % (1) Program Contributions to Research and Community Engagement. The BS in Business Education degree program in the Department of Business Education will contribute appropriately to intellectual climate and research/creative productivity community engagement, and respect for diversity. Faculty Intellectual Activities and Community Service Table 5: Intellectual and Service Activities Academic Years Professional Workshops/ Seminars/ Conferences Scholarly Publications Scholarly Presentations Other Scholarly Activities Service to the Profession Community Engagement Respect for Diversity Topics related to diversity issues are integrated into several courses in the Department of Business Education (BUED). However, at this time, student performance is not measured in this regard. BUED 210, Microcomputer Usage in Business, and BUED 342, Business Programing, address content related to diversity and multicultural issues in lab assignments (BUED 210) and case studies and test questions (BUED 342) that focus on the use of computer applications in the broader context of ethical and social implications. Page 4 of 9

13 BUED 360, Business Communication, integrates appropriate communication techniques for global economics and diverse cultures. Class discussions and written assignments are used to explain concepts and practice techniques. BUED 670/671, Directed Work Experience, addresses students development of interpersonal skills needed to work effectively with coworkers, supervisors, subordinates, and clients. One of the weekly log assignments is to summarize a resource about globalization and/or diversity in the workplace. b. Student Learning Outcomes (1) Communication Skills. Students completing the BS degree program in Business Education will exhibit effective communication skills (written, oral, graphic, and interpersonal) appropriate for professionals in this field of study at the bachelor s level. To assess students communication skills, data is collected every Fall semester in every section of BUED 360, Business Communication, which is a required core course for all students in the School of Business and Economics. Separate rubrics are used to assess students written and oral skills based on course-wide embedded assignments. The data are then disaggregated by program areas and presented to the respective department chairs. Results indicate that students do well, with scores consistently meeting the goal for both oral communication and written communication, as shown in Table 6. Table 6: Oral and Written Communication Data Results for Business Education Academic Year Percentage of Students Earning Passing at 70% or Better Oral Communication Written Communication % 100 % % 75 % % 66 % % 73 % % 81 % % 75 % During the academic year, the SOBE Writing Assistance Center transitioned to become the SOBE Business Communication Center that provides online resources, such as modules and sample documents, and support online writing submissions as well as oncampus peer tutors. (2) Critical Thinking Skills. Students complete the BS degree program in Business Education will effectively use quantitative and qualitative analytical problem-solving skills appropriate for professionals in the discipline at the bachelor s level. Case studies are integrated into BUED courses and are assessed using a common rubric to ensure consistency in data documentation. Data has been gathered for the last five years, with finding ranging from 53% to 86% as shown in Table X. Page 5 of 9

14 Table 7: Critical Thinking Data Results for Business Education Academic Year Number of Students (n) % Earning Passing at 70% or Better Number Not Identified 53 % % % % Not Reported Not Reported NOTE: Data were not collected during the academic year. The faculty discussed the surprising finding that no students were included in the Watson Glazer test that was assessed in the School s capstone course during the academic year. The faculty will continue to incorporate critical thinking activities in their courses and report their results each semester. (3) Disciplinary Expertise. Students completing the BS degree program in Business Education will demonstrate a level of discipline-specific expertise (knowledge, skills, and professionalism) appropriate for professionals in careers related to business education at the undergraduate level. This outcome is measured by students performance on the ETS Major Field Test (MFT) in business. The assessment is administered in the School of Business and Economics in MGMT 520, Strategic Management, which serves as a capstone course for all majors. Each spring semester, the data are then disaggregated by program areas and sent to the department chairs. The test covers a wide range of business content areas, including accounting, economics, management, and marketing. However, the various departmental results are not reported by the separate content areas. Table 8 shows the results for the last several years; the percentages reflect the students who performed above the standard (>=150). Although the goal was met, results range from 0% to 67% and increased for three years before dropping from 67% to 50% in Spring No students in the Department of Business Education were reported to take the MFT in AY Table 8: Major Field Test (MFT) Data Results for Business Education Majors Academic Year % Above Standard No data available % % % % No BUED students reported NOTE: Benchmark MFT scores is >=150 points. Due to the small number of majors, no data were collected for Business Education majors during the academic year. These results suggest that although the BS in Business Education program is preparing its majors to be employed as business educators or IT professionals, additional work must be done to ensure consistent results. To sustain the performance improvements in the MFT, the Department faculty plan to integrate additional business content into its BUED 210, Microcomputer Usage in Business, course and agreed to use it as a basis for teaching Page 6 of 9

15 students to create charts and graphs. Faculty also plan to incorporate more business case studies into all sections of BUED 360, Business Communication. The impact of these improvements will be assessed during the Spring 2015 semester. In addition to the MFT, students majoring in the Business Teacher Education concentration are evaluated on their disciplinary expertise through their student teaching evaluations, which are based on rubrics developed by the School of Education and are used to assess students who will be pursuing careers as business teachers. The students successes in their field placements are also consistently high, as shown in Table X. Table 9: Student Teaching Data Results Academic Year Number of Students (n) % Passing % % % % n/a % During the academic year, the student teaching evaluations were processed as part of the e-portfolio system (Taskstream) to document student performance for NCATE accreditation. (4) Research/Creative Engagement. Students completing the BS degree program in business education will demonstrate ability to review and conduct business-related research at the undergraduate level. As a new student learning objective, research/creative engagement by students has not yet been effectively measured by the Department. We recognize this weakness and are working quickly to ensure that a system is in place this semester to track effective data collection of student research and creative engagement. Research and creative engagement is embedded in BUED 360 (Business Communication) and BUED 400 (Business Reports and Presentations). 3. Evidence of PO and SLO Improvements Using the Results of the Assessments (Closing the Loop) a. Program Outcomes (1) Excellence and Recognition. The BS in Business Education degree program in the Department of Business Education will achieve excellence and recognition for high quality in teaching and learning, including the achievement of national accreditation by the AACSB International. To ensure quality in teaching and learning, the Department of Business Education plans to continue to support the faculty s professional development in workshops and seminars that focus on pedagogical research. Data collection to support AACSB accreditation is also Page 7 of 9

16 ongoing and will continue as part of the Department s standard operating procedures even after the next accreditation visit in Fall (2) Enrollment, Retention, and Graduate Placement. The BS in Business Education degree program in the Department of Business Education will meet or exceed the University s goals for enrollment, retention, and degree completion, and will achieve a placement rate of at 75% of its graduates in jobs or graduate education. Although the Department has not received a good response rate from graduating seniors and alumni about their placement information, the Department Chair will continue to find effective communication tools to gather this information, including social media outlets. Additionally, the faculty will continue to address efforts to increase enrollment, retention, and graduate placement. It is important to note that as an approved online degree program, Business Education includes several non-traditional students who take classes part-time. The faculty will be reviewing our enrollments and graduation rates to determine the extent that part-time students impact our data. (3) Program Contributions to Research and Community Engagement. The BS in Business Education degree program in the Department of Business Education will contribute appropriately to intellectual climate and research/creative productivity, community engagement, and respect for diversity. This outcome requires the most amount of work by the Department and will be an agenda item at each faculty meeting throughout AY Faculty will determine the best opportunities for integrating and assessing students research and community engagement. b. Student Learning Outcomes (1) Communication Skills. Students completing the BS degree program in Business Education will exhibit effective communication skills (written, oral, graphic, and interpersonal) appropriate for professionals in this field of study at the bachelor s level. Although the percentage of students meeting the communication goal decreased during AY , the small number of students in the same (n=12) could impact the results. The faculty is confident that the increase in peer tutors in the Business Communication Center and the additional online resources will continue to support student performance in both oral and written communication. (2) Critical Thinking Skills. Students complete the BS degree program in Business Education will effectively use quantitative and qualitative analytical problem-solving skills appropriate for professionals in the discipline at the bachelor s level. Based on the small number of students (and no students in AY ) reported in the Watson-Glazer data assessment, the Department faculty agreed to use case studies in courses in which critical thinking and analysis are included in the learning objectives to collect critical thinking data using a rubric. These data will be collected each Fall and Spring semester. Page 8 of 9

17 (3) Disciplinary Expertise. Students completing the BS degree program in Business Education will demonstrate a level of discipline-specific expertise (knowledge, skills, and professionalism) appropriate for professionals in careers related to business education at the undergraduate level. Although no BUED students were reported in the MFT data for , the data collected during the academic year indicate a decline in students knowledge, the faculty discussed initiatives to further enhance students business knowledge. To improve Business Education students comprehension of business core knowledge, the Business Education faculty requested and reviewed the MFT sub-category results to better understand the content and specific areas in which students need improvement. Based on this review, the faculty agreed to: Incorporate economic data as a basis for creating charts and graphs in BUED 210, Microcomputer Applications in Business; Integrate advanced business cases in BUED 360, Business Communication. Require communication as an activity to reinforce and assess students ability to effectively communicate business content knowledge. The impact of these improvements will be measured in the Spring semester. (4) Research/Creative Engagement. Students completing the BS degree program in business education will demonstrate ability to review and conduct business-related research at the undergraduate level. As stated previously, students research/creative engagement performance has not yet been effectively measured by the Department. This weakness will be addressed immediately to ensure and effective system of embedding assessments and tracking effective data collection of student research and creative engagement, specifically in the BUED 360 (Business Communication) and BUED 400 (Business Reports and Presentations) courses. Page 9 of 9

18 ANNUAL REPORT FORMAT ANNUAL REPORT (July 1, 2013 June 30, 2014) PART 4 PROGRAM LEVEL INSTITUTIONAL EFFECTIVENESS REPORTS NORTH CAROLINA AGRICULTURAL AND TECHNICAL STATE UNIVERSITY Institutional Effectiveness Assessment and Improvement Report Department of Curriculum and Instruction MS Instructional Technology Anthony Graham, Ph.D. 1. Expected Outcomes for the Educational Program and Its Student Learning Outcomes Program outcomes and selected student learning outcomes for the MS Instructional Technology program are as follows: a. Program Outcomes (POs) (1) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will achieve excellence and recognition for high quality in teaching and learning, including the achievement of national accreditation by the National Council for the Accreditation of Teacher Education (NCATE). (2) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will meet or exceed the University s goals for enrollment, retention, and degree completion, and will achieve a placement rate of at least 90% of its graduates in jobs. (3) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will contribute appropriately to intellectual climate and research/creative productivity, community engagement, and respect for diversity.

19 b. Program Student Learning Outcomes (SLOs) (1) Students in the MS Instructional Technology program will exhibit effective communication skills (written, oral, graphic, and interpersonal) appropriate for instructional technologists with advanced certification. (2) Students in the MS Instructional Technology program will use quantitative and qualitative problem solving skills appropriate for instructional technologists with advanced certification. (3) Students in the MS Instructional Technology program will demonstrate a level of expertise (knowledge, skills, dispositions) appropriate for instructional technologists with advanced certification. (4) Students in the MS Instructional Technology program will demonstrate the ability to engage productively in the review and conduct of disciplinary research and creative professional activity appropriate for instructional technologists with advanced certification. 2. Analysis of the Extent to Which each Outcome is Achieved a. Program Outcomes (1) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will achieve excellence and recognition for high quality in teaching and learning, including the achievement of national accreditation by the National Council for the Accreditation of Teacher Education (NCATE). The program was last accredited by the National Council for the Accreditation of Teacher Education (NCATE) in 2007, following its site visit in April The accreditation included no recommendations that required a response. The next scheduled accreditation visit in the spring (2) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will meet or exceed the University s goals for enrollment, retention, and degree completion, and will achieve a placement rate of at least 90% of its graduates in jobs or graduate education. The University Fact Book provided the data shown in Tables 1 to 3. (Please remember to introduce each table and discuss the main point(s).) Table 1: Enrollment Data Enrollment Academic New Continuing Total Year Candidates Candidates Enrollment Growth/Decline from Previous Year Table 2: Retention Rate Cohort Entering Year % % Page 2 of 7

20 Table 3: Graduation Data Cohort Entering Graduation Rates 3 Years 4 Years 5 Years 6 Years % N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Table 4: Placement Data Placement Rates Graduating Year Number of Graduates Job Placement Graduate Education % Uncertain % Uncertain % Uncertain (3) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will contribute appropriately to intellectual climate and research/creative productivity, community engagement, and respect for diversity. The data presented in Tables 5 to 6 are drawn from faculty annual reports, largely maintained using Digital Measures. Faculty Intellectual Activities and Community Service Table 5: Intellectual and Service Activities Academic Years Refereed Publications Scholarly Presentations Funding Proposals Submitted Funded Awards from Submissions Respect for Diversity Candidates in the MS Instructional Technology Program engage in multiple, embedded opportunities to explore, engage, and integrate diversity in their professional lives and settings. Specifically candidates in the program: a) complete learning objectives related to diversity and equity related research in a designated course entitled INST 735 Assistive Technology and Diverse Learners, b) identify and respond to the diversity in their employment sites through contextual exercises in a designated course entitled INST 798 Internship in Instructional Technology, Page 3 of 7

21 c) design and redesign instructional units to respond to the diverse learners in their employment settings as a part of their programmatic exiting requirements and state accreditation standards. b. Student Learning Outcomes (1) Students in the MS Instructional Technology program will exhibit effective communication skills (written, oral, graphic, and interpersonal) appropriate for instructional technologists with advanced certification. Candidates in the MS Instructional Technology program must illustrate their communication skills at various benchmarks. Candidates must submit electronic evidences throughout the program that illustrate their abilities to communicate orally and in writing. One electronic evidence is the MS Instructional Technology Assistive Technology Project for Diverse Learners where candidates design an electronic communication video presentation using 21 st Century technology grounded in appropriate theories of digital learning and researchverified strategies for children and adult learners. This evidence has four indicators that focus explicitly on candidate written communication skills. University faculty utilize a rubric to assess the components of this key assessment using the following scale: 1.00=Unacceptable, 2.00=Acceptable, and 3.00=Target, and 4.00=Accomplished. University faculty who assess candidate performance with this rubric receive training prior to its use. Given this evidence, the mean score for candidates (N=9) on the Assistive Technology Project for Diverse learners electronic evidence was 3.80 out of 4.00 points. One hundred percent of the candidates achieved Met status on this evidence, suggesting they have the ability to communicate effectively through responsive technological tools. Candidates in the MS Instructional Technology program must also illustrate their communication skills during their practicum experience. A rubric with rating scales of 1.00=Strongly Disagree, 2.00=Neither Disagree or Agree, 3.00=Agree, and 4.00=Strongly Agree is used during this experience to assess candidate skills across various dimensions. Several indicators on the rubric specifically address candidates abilities to communicate effectively. Table 6 illustrates the mean ratings for candidates performance on these communication indicators during the academic year. Table 6: Mean Candidate Score on Communication Indicators Academic Year Number of Candidates Language appropriate to the position Clear and understandable oral communication Clear and understandable written communication Communicates well with members of a team (2) Students in the MS Instructional Technology program will use quantitative and qualitative problem solving skills appropriate for instructional technologists with advanced certification. Candidates in the MS Instructional Technology program must demonstrate their ability to utilize quantitative and qualitative problem solving skills on key assessments. Candidates illustrate their ability to use quantitative and qualitative data to problem solve on their Leadership and Management Project in INST 703. A rubric with rating scales of 1.00=Developing, 2.00=Acceptable, 3.00=Effective, and 4.00=Exceptional is used during this Page 4 of 7

22 experience to assess candidate skills across dimensions related to the use of quantitative and qualitative data analysis. Several indicators on the rubric specifically address candidates abilities to collect and analyze data to make data-driven decisions. Table 8 illustrates the mean ratings for candidates performance on these indicators during the academic year. Table 8: Mean Candidate Score on Leadership Project Rubric Academic Year Number of Candidates Criterion 3. Discussion of Problems Criterion 4. Initiative development Criterion 7. Improving participation plan (3) Students in the MS Instructional Technology program will demonstrate a level of expertise (knowledge, skills, dispositions) appropriate for instructional technologists with advanced certification. Candidates in the MS Instructional Technology program must demonstrate their expertise of appropriate knowledge, skills, and dispositions on key assessments. One key assessment is the comprehensive examination, which illustrates content knowledge and skills. The comprehensive examination requires candidates to write well-reasoned essay responses with scholarly literature support that addresses the areas of general teaching and assessment, research and inquiry, instructional design, and learning theories. During the academic year, faculty evaluated 20 candidates comprehensive examinations and 100% achieved the Met status. These data indicate candidates performed at a rate of 70% or higher on their written comprehensive examination. Candidates in the Instructional Technology program must also illustrate their content knowledge and professional skills by conducting and uploading a Microteaching video for analysis and evaluation by faculty. This evidence requires candidates to articulate an explicit lesson objective, utilize sound instructional technology pedagogy appropriate for the lesson objective, record the microteaching video, and write a reflection on the experience. Faculty use a rubric to assess and evaluate the microteaching video, relying on a final evaluation or Met or Not Met. During the academic year, faculty evaluated 16 candidates Microteaching artifact, and 100% achieved the Met status. (4) Students in the MS Instructional Technology program will demonstrate the ability to engage productively in the review and conduct of disciplinary research and creative professional activity appropriate for instructional technologists with advanced certification. Candidates in the MS Instructional Technology program must demonstrate their ability to engage in the review and conduct of disciplinary research. Candidates in the MS Instructional Technology program must compose a research literature review in the CUIN 711 Research and Inquiry course. Candidates construct literature reviews based on relevant, significant, and pertinent topics in the instructional technology discipline. Faculty use a 3-point rubric (i.e., Exemplary=2.00, Acceptable=1.00, Failing=0.00) that focuses on the dimensions of the introduction, body, conclusion, organization, in-text citations, APA references, grammar/mechanics, and appropriate sources. Faculty consider a score of 70% Page 5 of 7

23 or higher a passing score, resulting in a Met on the activity. Any score below 70% elicits a Not Met status on this assignment. During the academic year, faculty evaluated five candidates literature review assignments, and 100% achieved Met on this document. Candidates in the MS Instructional Technology program must demonstrate their ability to engage in the review and conduct of disciplinary research by completing a thesis or research project as their final culminating activity. Candidates in the MS Instructional Technology program must compose a written research project or thesis in INST 790. Candidates construct this document, which is broken into five chapters: Introduction, Literature Review, Research Design, Findings, Conclusion and Recommendations. Candidates who compose a thesis must defend the document before his or her committee. Faculty rate these projects using a Met or Not Met ranking. During the academic year, faculty evaluated seven candidates thesis or research projects, and 100% achieved Met status. 3. Evidence of PO and SLO Improvements Using the Results of the Assessments (Closing the Loop) Summarize and present evidence showing that achievement of each of the Program Outcomes and Student Learning Outcomes has been continuously improved over the years as a function of acting on specific assessment results. (What specific program and learning improvements were made during academic year? What assessment data led to the improvements? Provide anecdotes and examples to help the reader understand how the improvements are continually reviewed and further enhanced each year.) a. Program Outcomes (1) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will achieve excellence and recognition for high quality in teaching and learning, including the achievement of national accreditation by the National Council for the Accreditation of Teacher Education (NCATE). (2) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will meet or exceed the University s goals for enrollment, retention, and degree completion, and will achieve a placement rate of at least 90% of its graduates in jobs. (3) The MS Instructional Technology degree program in the Department of Curriculum and Instruction will contribute appropriately to intellectual climate and research/creative productivity, community engagement, and respect for diversity. (4) Student Learning Outcomes (1) Students in the MS Instructional Technology program will exhibit effective communication skills (written, oral, graphic, and interpersonal) appropriate for instructional technologists with advanced certification. No significant changes have occurred on the communication skills student learning outcome in the Instructional Technology program based on data. Prior programmatic changes during the academic year are still under review as data are collected and analyzed for future considerations. (2) Students in the MS Instructional Technology program will use quantitative and qualitative problem solving skills appropriate for instructional technologists with advanced certification. Page 6 of 7