AREAS OF EVALUATION. State the most recent mission, goals, and objectives approved by the department/program.

Transcription

1 Certification stipulates that every academic program of the UPR system is to be evaluated every five years. As of 2014, UPRM follows a calendar that specifies when each program should be evaluated. The document provided by the Office of the UPR Vice President, Guide to Evaluate Academic Programs at the UPR provides the data gathering criteria for each required area in the report. I. Introduction Please include the following in this area: AREAS OF EVALUATION a. Program s title b. Degree(s) conferred c. Date and duration of created program d. Accreditation(s), if applicable e. Authorizations and licenses (CEPR) f. Program Administration - describe all personnel; include organizational chart (organigram) of the institution ( pdf) and the Department/program organizational chart. g. Any extra pertinent information II. MISSION, Goals, and Objectives (keep in mind, you are evaluating five (5) years of the program). State the most recent mission, goals, and objectives approved by the department/program. 1. Were the program s goals and objectives aligned with the UPR and the UPRM s goals? Provide a table showing how these are aligned: For the UPR Plan visit: For the UPRM Strategic Plan (English version) visit: OIIP will provide the previous UPRM Strategic Plan 2. (This question is composed of questions 2 and 3 in the original document) Demonstrate that the goals and objectives focus on student learning and program improvement. Provide evidence of these achievements within the goals and objectives of the program. Use syllabi of all courses. Your program should also have an Assessment Plan which includes this information. Include letters or results of exit surveys by alumni. Incorporate graduation and placement rates (OIIP). 3. To what extent has the program reached its expected scope and purposes? Compare goals over time and assessment report(s). Identify gaps 4. What changes transpired and in what year were the program s mission, goals, and objectives updated? Use cycles from the Assessment Report. Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 1

2 III. Needs and Justification of the Program Please describe the scope of the program in relation to the needs of the population. 1. Provide data that demonstrates that the program adequately satisfied the identified needs and opportunities (as per the last evaluation or when the program was created). Use admission information, degrees conferred, placement rates, and retention rates (OIIP). 2. What are the needs and expectations that justify the continuation of the program? Number of applicants, transfers, and the number of students served every year.(oiip) Any information on jobs, skills, and/or market tendencies and needs. IV. Program Relevance This section highlights the program s unique characteristics, the existence of other similar programs, its relationship with other programs, program demand, and other relevant features. 1. What characteristics make your program a distinguishable and valuable study alternative within the UPRM system, education system in general, and compared to other institutions? To answer, use accreditations, if applicable; associations within the program, faculty-degree distribution, and any significant experiences and opportunities offered to the students (undergraduate research, COOP initiatives, internships, etc.). Compare and contrast with similar programs in other campuses and universities. 2. What is the program s impact among other programs and departments within UPRM and the UPR system? Include information about shared teaching among campuses, research, program articulations, interdisciplinary activities, and others. 3. If programs at a different level are offered within the same discipline or professional area, what is its relationship with the program under evaluation? This applies to different tracks within your program as well as different degrees (e.g. Master s, Ph.D.). How does it compare to the program being assessed? Highlight differences. 4. What information is available to show that different groups, the industry, research centers, and other educational institutions or agencies have a vested interest in this program? Information from the last five years is needed as well as five year projections. Also, include any agreements, collaborations, project funds, alliances, job fair recruiting information and, if available, letters from companies and industries. 5. What tendencies are observed in terms of number of students accepted in the program per year, applicants, and percentages of students in the program? Use registration information from the last five years (OIIP). Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 2

3 V. Curriculum This section guides in the analysis of the different components of the program curriculum in regards to its effectiveness in achieving the goals, objectives, and profile of the graduating student. 1. How adequate is the curriculum in achieving the developmental competencies included in the graduating student profile? Include assessment report(s), exit surveys, data from surveys of graduated students who are in the workforce. 2. What data demonstrates that the curriculum is relevant in terms of amplitude, depth, and level in the discipline? Compare curricula of other countries and within Puerto Rico. 3. Provide evidence that the syllabi of the courses are up to date and fully meet the requirements of Certification 130 ( ) of the Board of Trustees, Manual para el registro y codificación uniforme de cursos. Print copies of master syllabi as well as individual professor s syllabi. 4. What courses have been added, modified, or deleted in the past five years? Show table with this information. Academic Affairs can provide the information for the five years. 5. Is the content of the program courses aligned with the graduating student profile? Similar to question 1. Assessment report(s). Exit surveys or search for those students who graduated and are in the workforce and administer a survey to gather data. In other words, reach for alumni and employers to gather data through surveys or questionnaires. 6. How do course offerings meet the needs of the student? A general report will be generated by OIIIP including: The number of sections offered per semester, the number of spaces available, occupation rate, the yearly distribution of credits taken by the students in the program. 7. Provide evidence that students completed the program in the established time. On time graduation rate for the program exit cohort will be provided by OIIP. 8. What sequence courses have not been offered in the past five years and why? Each program responds if applicable. 9. How accurate is the course description information among the diverse sources of dissemination? Verify syllabi, catalogue, and Internet information. Provide copies as evidence. 10. What percentage of students complete the sequence of courses in the time allotted by the program? 100% and 150% Graduation rates as well as graduating from the program and graduating from any other program will be provided by OIIP. 11. How effective is the teaching methodology used in delivering the program curriculum? Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 3

4 Distribution of grades per course, percentage of passing grades (OIIP), portfolios per course. 12. What additional activities were carried out during the past five years to strengthen the program s curriculum? Major and minor curriculum changes (Academic Affairs will have this information). 13. What curriculum changes, if any, would you include into the program? Based on all the data about the program, what changes would be effective. VI. Assessment Results 1 In this section, the assessment plan and assessment techniques used by the program are examined to determine the success in achieving the mission, goals and objectives (institutional effectiveness) and student learning outcomes (assessment of learning). All three questions rely on the program s assessment plan. The use of portfolios can also serve as evidence. 1. What structures, processes and strategies were used to assess the effectiveness of the program and of the student learning process. How do these relate to the program s assessment plan and the UPRM s assessment plan? 2. What do the results on student learning and the effectiveness of the program demonstrate? 3. Considering the results from program effectiveness and student learning, what curricular and instructional changes were made? VII. Students This section examines the scope of the policies and practices of recruitment, admission and retention, trends and projections of enrollment, amongst other accomplishment indicators of the program in achieving its mission, goals, and objectives. 1. How effective was the program in recruiting, attracting, and retaining a diverse and qualified student population? Admissions data, retention and graduation rates, average IGS-All provided by OIIP 2. What is the socio-demographic and academic profile of students enrolled in the program and how do these relate to academic success? It is the intent of OIIP to provide some pre-evaluated and analyzed data for this element, but we should be clear on the issue that the available data related to the student s socio-demographic reality, is not totally reliable.. 3. How does the number of students enrolled annually in the program compare to the projected amount of students? Number of students in the program at the beginning of each semester- OIIP. 4. How does the demand for the program (new admissions, internal and external transfers) compare with the established student quota established for the program? Admissions report, internal/external transfers data- OIIP 1 Avalúo: Proceso mediante el cual, sistemáticamente y haciendo uso de una variedad de métodos o estrategias, se recopilan y se analizan datos sobre el desempeño de los estudiantes y de la institución con el propósito de mejorarlo y fortalecerlo. Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 4

5 5. What was the passing rate of the core and concentration courses within the program? Grade distribution Report- OIIP 6. How does the number of degrees conferred compare to those projected? Degrees conferred/projected when program was created- OIIP 7. What was the graduation rate of high school students admitted to the program and, if it was not satisfactory, what strategies were established to improve it? Graduation rates- OIIP graduation rates. Evidence of actions taken- use assessment plan. 8. What was the rate of persistence (continued enrollment) of the students in the program and, if it was not satisfactory, what strategies were established to improve it? Persistence rates- OIIP Evidence of actions taken-use assessment plan. 9. What data was collected to demonstrate the success of graduates and how is the information used to strengthen the program? Assessment plan. Exit surveys. Provide evidence of actions taken as a result of the assessment plan. VIII. Faculty In this section the quality, diversity, and competence of faculty is assessed as well as the faculty s contribution to the mission, goals and objectives of the program. 1. Does the program have sufficient qualified faculty with the academic preparation and experience necessary to respond to the knowledge base and the critical issues of the discipline or program? Curriculum vitaes, contract letters, part time/temporary positions. Table of core courses with professors who have preparation to teach these courses. Also, relate program skills with professor expertise. 2. Show evidence to demonstrate that the faculty contributes to the achievement of the program goals. External funds, projects, associations, symposiums (attended and created), thesis committee participation, publications, undergraduate and graduate research. 3. How do the expectations of the institution and the program compare to the productivity achieved by the faculty in the areas of research, scholarly and creative work, as well as community service? Professor contract letters which include expectations vs. plan docente and semi-annual reports on special activities-appendix A. 4. What trends exist between creative work and research as evidenced in referred publications? Appendix A and COE; number of publications per professor per year. 5. What impact do the assessment practices show in the faculty and the quality of teaching and learning? Changes and tendencies as a result of the COE Personnel committee reports 6. How have the faculty assessment results been used to strengthen the program? Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 5

6 Appendix A and COE (similar to question number 4). 7. Based on the conditions and market trends of the discipline, how capable was the program in attracting and retaining diverse personnel adequate in number and qualification for the program s success? Curriculum vitaes, recruitment reports (personnel committees). 8. What changes or revisions were made to the program s recruitment and retention plan to respond to the conditions and tendencies of the market and the discipline? Show evidence of changes in the recruitment plan in response to the assessment, market conditions, or economic realities. Given the current situation, compare and delineate what is done to recruit at UPRM. 9. How does the program encourage and support faculty and researchers to obtain terminal degrees in their respective disciplines, and if worthy, for post-doctoral experiences? Any plan or program in place during the last five years. Number of professors who have been sponsored to study or work on research projects. 10. What does the data show in terms of the effectivity of the faculty improvement plan and its contribution to the mission, goals, and objectives of the program? The question assumes that your program has a faculty improvement plan. How is the faculty assessed within the program? Response should be based on Personnel Committee criteria, tools, and protocols. 11. How does the program guarantee that the faculty is up-to-date in terms of knowledge, emerging teaching practices, assessment, diversity, integration of teaching technology, and any other areas within its mission? Workshops professors attended workshops (report), symposiums, conferences, etc. IX. Administrative and Support Personnel and Academic Advising This section examines the program s efforts, from student recruitment to graduation, in providing quality services that are relevant and necessary for their integral development and constancy in the program. 1. What evidence exists to show that the support services meet the needs of the students? Counseling workshops and/or orientation meetings. An exit survey that includes services. Collect students opinions on services received at the Department. 2. For the past five years, what does the data demonstrate regarding the levels of efficiency, effectiveness, and satisfaction with student support services? Student surveys. Collect opinions of the served students. 3. What changes or revisions were made in terms of the services offered by the program? Per program- explain the services offered in general and include the academic guidance system within your program and changes implemented in: procedures, politics, protocol, personnel; any change made to improve such services. 4. How does the program guarantee that the students had access to academic and professional counseling, and that these were adequate and of value? Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 6

7 Explain the academic, professional guidance system within the program. Include details on service availability (service hours) and the process to access the services. 5. How do the skills and amount of supporting staff compare to the type and demand of services? Provide a table that lists all services and student needs and the personnel who offer the services. 6. What information was used to establish an improvement plan for the supporting staff, and how did this impact the quality of service? A satisfaction survey is appropriate to determine the quality of service and measure improvement- now or in the future. If personnel attended workshops by the department or the institution, this information should also be included. X. Learning and Information Resources This section examines the instructional and information resources (bibliographic and electronic) available and accessible to faculty and students. 1. What evidence is there to show that the instructional resources and information were effective in achieving the mission and goals of the program? Exit surveys. Use of Library databases, evidenced in projects, thesis, and paper references. 2. What evidence is there to show that the faculty and students had access to and utilized information in various formats, including electronic sources of information? Library workshops. Open access to the portal. Ask CTI to provide the number of hits into the system. 3. How does the program evidence its progress and accomplishment in the use and integration of technology? Capability of faculty in utilizing technology Integration of information skills in the curriculum, including computer skills and computer literacy Assessment of the effectiveness of the learning experience and the program Institutional technological support Strengthening of research Strengthening of services CEP hours. Classrooms with technology (explain why these spaces were created), courses taught in these classrooms. 4. What changes or revisions were made to ensure that the incorporation of learning resources in the academic and administrative components of the program were effective? Hybrid courses, workshops, courses using Moodle. XI. Outreach and Service This section describes the achievements of the program to connect with the community. Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 7

8 1. What evidence is there to show that the program has integrated topics, research, service, and cultural matters of the communities in their curriculum? Professors syllabi, INTD community course agreements, capstone course/projects. 2. How correct and current are the academic course catalog, syllabi, publications, and other electronic media for program outreach with the community? How was the scope and impact of these determined? Verify revision dates of the syllabi. Who updates the program s webpage and how frequently is it updated. Explain that the catalogue is revised during the summer, every year. 3. What evidence is there to show that the academic community and the external community use the abovementioned media and strategies? Be creative! Suggestions are welcome! 4. Show how the program included students and administrative personnel in offering public service opportunities, prosocial alternatives, community projects, and internships? Same as Number 1- Professors syllabi, INTD community course agreements, capstone course/projects. 5. Show evidence of program linkage and contribution to the needs and expectations of the community. Internal community- mention all activities/services to community per program. External community- projects, community initiatives. 6. What changes and revisions were made to the program to blend the curriculum and educational experiences with the institutional goals and the needs and expectations of the community? Added INTD courses, new courses, and any changes to the syllabi. XII. Program Operation and Effectiveness This section examines the structure as well as the impact of the policies, procedures, and administrative and management practices in the successful operation of the program. 1. What is the profile of the administrative and support personnel? How has this contributed to the program s effectiveness? Create table- employees and their roles in the program/department. 2. How does the administrative and support personnel s profile respond to the policies, procedures and administrative practices? Same as Number 1- Employees and their role in the program/department. Also, add director, associate director, guidance counselor, etc. 3. How adequate is the coordination of program operation between management and administrators? Answer what these are in paragraph form and state their importance in the continuity of the program which includes students, etc. Also, the frequency of staff meetings at the different levels can be included. 4. What activities and strategies did the program develop in the last five years to promote the professional development of the administrative personnel? Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 8

9 Mention all professional developing activities offered to or attended by the administrative personnel: institutional and at the program level. 5. How involved were the faculty, supporting personnel, and students in the decision- making processes? Provide details on the decision-making process, including who is involved at the different levels. Use minutes and attendance sheets from meetings. 6. What changes and/or revisions were made to the administrative and management components to ensure the program s effectiveness? Include hiring/loss of personnel. Also include use of spaces and how these allowed for the program to be effective. XIII. Fiscal Aspects This section focuses on the relationship between the budget and the mission, goals, and objectives of the program. 1. Describe the program s operation in relation to the available fiscal resources. This is individual to each program. Explain the use of the budget. 2. How sufficient were the fiscal resources to support the developmental initiatives that needed to be institutionalized? Individual to each program. Explain how sufficient were the funds and if not, what would be needed. 3. What external resources did the program receive for its operation and continuity? What impact did these resources offer the program? External funds and what these were used for in the program. What projects need resources in order to operate and/or continue? XIV. Facilities, Laboratories, and Ancillary Teaching Equipment This section examines the suitability and adequacy of the facilities, laboratories, and additional teaching equipment necessary to the program. 1. Did the program receive the facilities, laboratories, equipment, instruments, and any equipment projected five years ago? Table comparing five years and includes all facilities, laboratories, etc. 2. Have the facilities, laboratories, and ancillary equipment been modernized to attend to the progress of the program and its academic discipline? Describe the program s plan and show (use same table as question number 1) what the program has, received, will receive, etc. 3. What do the assessment findings show about the facilities in the last five years? Assessment plan. Also include the space conditions and resources to attend to students. If the program attends to the entire university, this should be included and the need should be explained. Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 9

10 XV. Strengths and Limitations Based on the analysis of the assessment findings developed in the previous sections, identify the strengths and limitations of the program. ALL THESE QUESTIONS ARE SPECIFIC TO EACH PROGRAM. WRITE A PARAGRAPH OR MORE FOR EACH. USE ASSESSMENT TOOLS WHEN AVAILABLE. 1. What strengths are evidenced in the last five years? What is needed to maintain and expand these strengths? 2. What areas need improvement? 3. What is needed to overcome the limitations? 4. What challenges or obstacles hinder addressing or overcoming these limitations? XVI. Development Plan This section requires the development of a specific plan to work with the assessment findings. SPECIFIC TO EACH PROGRAM. THE PLAN SHOULD BE ALIGNED WITH THE STRATEGIC PLANS (INSTITUTIONAL, FACULTY, PROGRAM LEVELS) AS WELL AS THE ASSESSMENT PLAN. The plan should include, at a minimum, the following sections: Areas to be addressed Program goals for identified areas Strategies and activities to conduct Resources Date(s) when these actions are completed Measurements of achievements Professors Morales and Ferrer- UPRM Offices- OMCA and OIIP respectively/ 10