Case of the Department of Biomedical Engineering at the Lebanese. International University

Journal of Modern Education Review, ISSN 2155-7993, USA July 2014, Volume 4, No. 7, pp. 555 563 Doi: 10.15341/jmer(2155-7993)/07.04.2014/008 Academic Star Publishing Company, 2014 http://www.academicstar.us A Sustainable Approach for Program Students Outcomes Assessment: The Case of the Department of Biomedical Engineering at the Lebanese International University Mohamad Hajj-Hassan, Bassam Hussein, Khaled Chahine, Amin Haj-Ali (Lebanese International University, Beirut, Lebanon) Abstract: Since its inception in 2004, the Department of Biomedical Engineering at LIU has been actively engaged in a process of constant evolution of its program. The objective has always been to align with a sustainable assessment approach that is aimed at determining how well graduating students achieve intended learning outcomes. This is all done with the intention of having the program fully compatible with the Accreditation Board for Engineering and Technology (ABET) and in anticipation of future accreditation. This paper highlights the Biomedical Engineering program students learning outcomes assessment approach that has emerged, its indices and the results from the preliminary outcomes reporting that involves the collection of data in line with best practices, well established key performance indicators and standardized benchmarking in the biomedical engineering education community. All of which has been done as a part of education quality management and continual improvement purposes. Key words: ABET accreditation, PSO, sustainability, academic assessment, reporting tools 1. Background and Overview Since its establishment in 2002, the Lebanese International University (LIU) recognized the importance of students learning outcomes and their impact on the quality of education. LIU s mission states: LIU endeavors to align its values and commitments to student learning, support, and communication and continually reviews curricula to introduce innovative outcomes (LIU, 2013). Learning outcomes are statements of what a learner is expected to know, understand and/or be able to demonstrate after completion of a process of learning (Donnelly & Fitzmaurice, 2005). According to a report prepared by the World Bank in 2012, LIU is considered as the largest and fastest growing private university in Lebanon. It has 13% of the overall number of students enrolled in private higher education institutes in Lebanon (World Bank, 2012). LIU s School of Engineering (SoE) has around 4000 fulltime students enrolled in the following programs: surveying, mechanical, biomedical, electrical, electronics, computer, communications and computer, communications and industrial. Students usually complete a three-year degree of Bachelor of Science (BS) in engineering followed by a two-year program of Master of Science (MS) in Mohamad Hajj-Hassan, Ph.D., Assistant Professor, Lebanese International University; research areas/interests: education, biological micro and nanosystems. E-mail: mohamad.hajjhassan@liu.edu.lb. 555

engineering. According to the Lebanese laws and the Order of Engineers and Architects of Lebanon regulations, students are only allowed to practice the engineering profession if they complete the five years combined program successfully (OEA, 2011). The Department of Biomedical Engineering at LIU is an academic unit of the School of Engineering. It serves to administer biomedical engineering degree programs at the Bachelor s and Master s levels. The programs consist of 108 and 52 credits, respectively. Both B.S and M.S educational programs provide a core of engineering, physical and biomedical sciences, mathematics and advanced courses that apply these concepts to various biomedical problems. To enhance learning, many of the required undergraduate courses have a laboratory component. This way, students can quickly see the application of the concepts they learn in class. Advanced classes deal with current research and design topics and students are exposed to the latest developments in the field. The department s faculty work extensively to build interdisciplinary research and education environment that trains students to have a positive impact in industry and medicine. 2. The Learning Outcomes Assessment Process Assessment is a cycle that consists of the following: learning outcomes definition, measurement of outcomes, analysis of results, and recommending changes to improve programs and activities. Being focused particularly on improving student learning, academic assessment addresses student-learning goals and conducts measurements of learning. Assessment is best thought of as a continuous and dynamic process, as it adapts and responds to changing local and global dimensions of knowing. Figure 1 depicts the cyclical nature of assessment, as faculty members are always investigating, intervening, interpreting, and improving. Figure 1 The Assessment Cycle 2.1 Definition of Learning Outcomes Step 1 of the assessment cycle is defining learning outcomes which includes the following elements: review of the program mission and goals, defining learning outcomes that are consistent with the goals, and finally course mapping which is basically saying which courses cover what outcomes (Kennedy et al., 2006). 556

2.1.1 Mission Statement At this point in the assessment cycle, the mission statement has either to be created, if it does not exist, or reviewed to cope with changes in the program. The mission statement represents the purpose that achieving the program outcomes is meant to serve. Reaching a general agreement in its development affects the degree to which it has value and significance among faculty members. It should reflect their points of view toward the program, inform the students what the program is to accomplish, and provide raison d être of the program. For instance, engineering students have the right to know the purpose an introductory course in linear algebra serves. In this respect, a clear mission statement not only facilitates reaching a consensus among faculty members on the program goals and outcomes, but also aligns the efforts of students and faculty members with the purpose of the program. The mission of the Biomedical Engineering Department at LIU is captured in the following paragraph: The mission of the Department of Biomedical Engineering is to educate students and provide them with multi-disciplinary training for productive careers in the health-related areas of industry. To achieve our educational mission, a curriculum that integrates engineering sciences, life sciences, clinical medicine, research and engineering design is offered. Cultivating our students problem-solving and communication skills, promoting their ability to think critically and independently, and helping them to understand scientific and engineering approaches through a five-year sequence of courses represent the scope of the Biomedical Engineering program. 2.1.2 Goals Goals of an academic program are statements of what the program is to achieve or become in the long run. The statements can be general but should give direction to the program through affecting decisions related to its scope, requirements, and priorities and scope. A successful academic program relies, among other factors, on agreement on its goals, comprehending what the program is to achieve and establishing a link between the goals and the curriculum (Bloom et al., 1964). The program goals of the Biomedical Engineering Department are given in the following statement: The B.S. in Biomedical Engineering is a three year program which ensures a solid foundation in mathematics, life sciences, electric and electronic circuits and systems, microcontrollers, and biosensors and acquisition systems. The program strives to produce graduates who are expected to demonstrate an ability to: Integrate and apply basic principles of mathematics, life sciences, and engineering fundamentals to identify and solve problems in the field of biomedical engineering. Use modern technology. Build their professional identity and develop in their jobs. Undergo intellectual growth and engage in life-long learning. Communicate effectively in both written reports and oral presentations. Work effectively within multidisciplinary teams. The M.S. program in Biomedical Engineering (MBENG) is a two year intensive program which includes course work as well as a research project work. It is based on a solid foundation of science and mathematics coursework. The M.S. program in Biomedical Engineering is designed to produce highly motivated and trained healthcare specialists capable of using traditional engineering expertise to analyze and solve problems in biology and medicine, providing an overall enhancement of health care. The program provides excellent and exhaustive education in medical imaging, bioinstrumentation, biomaterials, clinical engineering, and signal and image processing. The program covers advanced and emerging topics such as bionanotechnology. 557

Graduates in this program will be able to: Analyze, design, and implement solutions, systems, and devices related to biomedical problems. Design and conduct experiments as well as to measure, analyze, and interpret experimental data from living systems. Use their multidisciplinary background to foster vital link and communication across professional and disciplinary boundaries with the highest professional and ethical standards. Demonstrate how ethical, social, and professional responsibilities impact the practice of biomedical engineering. Describe emerging and contemporary issues and challenges facing biomedical engineers. Lead in research and development in various biomedical industries and can be called upon in a wide range of capacities: to design instruments, devices, and software, to bring together knowledge from many technical sources to develop new procedures, or to conduct research needed to solve clinical problems. 2.1.3 Learning Outcomes Upon reaching a consensus on the program mission and goals, faculty members proceed with the definition of specific learning outcomes. These outcomes should be consistent with goal statements, which in their turn have to be aligned with the mission statement of the program (Biggs, 2003). In contrast to goals, which are broad statements, learning outcomes present a clear, accurate and specific account of the required level of learning to be achieved in the process of earning a degree and meeting program goals. Table 1 shows how learning outcomes can be derived from program goals. Program Goals Table 1 Learning Outcomes Derived From Program Goals Student Learning Outcomes Outcome 1 Outcome 2 Outcome 3 Outcome 4 Goal 1 Goal 2 Goal 3 It is important that faculty members make sure that learning outcomes provide answers to the following three questions: (1) What knowledge and information should students in a given major acquire? (2) What skills and competencies should students gain? (3) What values, attitudes or qualities should be instilled in students? It is worth mentioning that the discussion about learning outcomes should not be restricted to faculty members. Other stakeholders with different perspectives such as students, employers, and alumni can provide valuable input. In discussing learning outcomes, faculty members usually start with reviewing the program mission statement and goals. Additional suggestions and ideas can be obtained by checking learning outcomes of departments offering similar programs. The objective is to come up with an expansive list of relevant learning outcomes that can be later narrowed to a number between three and five. The selected outcomes should meet the SMART criteria: Specific as to what the learner will be able to do. Measurable can be observed by the end of the program or module. Attainable can be achieved within scheduled time and given resources. 558

Relevant oriented to the needs of the learner and the institution. Time-bound can be completed by the end of the program or module. Narrowing the list the outcomes is challenging task that is usually tackled using the Delphi technique. This technique consists of the following. First, each faculty member anonymously ranks the outcomes in the list by assigning a numerical value to each of them. Then, an impartial facilitator computes the scores, ranks the outcomes, and announces the rankings. This process is repeated until faculty members reach an agreement and a minimum number is retained. Intended learning outcomes are concerned with what students in the major should know and are capable of doing upon completion of the program (student-centered) rather than with what faculty members teach (teacher-centered) (Kennedy et al., 2006). Finally, making learning outcomes public is important as it helps students in the program become aware of the where their direction is heading and be more involved in the learning and assessment process. 2.1.4 Course-Outcome Mapping Departments typically use course mapping to determine how learning outcomes are addressed in the course offering, i.e., each course covers what outcomes. In general, a table, with one axis containing the program learning outcomes and the other containing courses, is used to represent a course map. The cells of Table 2 show the learning outcomes covered by each course. Table 2 Mapping Courses to Learning Outcomes Program Learning Outcomes Course 1 Course 2 Course 3 Course 4 Course 5 Outcome 1 Outcome 2 Outcome 3 Outcome 4 Not only does course mapping provide a view of how each course is mapped to the program learning outcomes, but also shows the weight or emphasis assigned to each outcome. For instance, Outcome 1 in the Table 2 is mapped to five courses and is therefore given greater significance than Outcomes 3 and 4. Another advantage of a map is that it identifies redundancies or gaps in the map. The above example shows that only one course currently covers Outcome 4. This may be appropriate if a single course is strongly oriented toward the intended outcome. For instance, development of research skills might be one of the intended learning outcomes for an academic program, and it might occur that only one course covers this outcome. In reviewing such a course map, faculty members have to decide whether the outcome is addressed less prominently in other courses (for instance, research skills may be taught directly in only one course, but may be required by students in other courses). If it addressed by a single course, faculty members have to decide whether it is sufficient to have this single course covering a high-priority learning outcome. In summary, a course map can determine the extent to which the program currently addresses the list of intended learning outcomes. Moreover, it can display the degree to which a course stresses a specific outcome. Furthermore, it may be useful to show the time allocated to each outcome in each course. For developmental outcomes, descriptors such as low, medium, and high can be used to indicate the level of achievement expected in each course and how student achievement is building up during progress through the major. 559

2.2 Assessment of Learning Outcomes Assessment of learning outcomes is usually tackled using two primary approaches, known as summative assessment and formative assessment. Formative assessment, referred to as assessment for learning, involves collecting information prior to or during instruction, thus providing immediate evidence of student learning and allowing instructors to take appropriate instructional decisions and make timely adjustments. A common formative assessment technique is student-self assessment. The purpose of this technique is to have students assess their own progress towards the intended learning outcomes. For this technique to be effectively used, students must understand the learning outcomes of the course, their importance and what needs to be done to achieve them. The reason is that students tend to overestimate their own abilities and understanding of the course. Summative assessment, referred to as assessment of learning, involves collecting information at the end of a course or program. It is comprehensive in nature, in the sense that it determines if the overall student learning outcomes have been achieved (at the course or program level). Summative assessments of individual students may be used for promotion, certification or admission to higher levels of education. Formative assessment, by contrast, draws on information gathered in the assessment process to identify learning needs and adjust teaching (Looney, 2011). Summative assessment should always be a direct method of assessment, whereas formative assessment could be either a direct or an indirect method of assessment. Direct measures of assessment are measures in which the products of student work are evaluated given the learning outcomes for the program. Activities from coursework such as projects or specialized tests of knowledge or skills are examples of direct measures. In all cases, they involve the evaluation of student learning demonstrations. The assessment of a given learning outcome should use at least one direct measure. Indirect measures of assessment are measures in which students judge their own ability to achieve the learning outcomes. Indirect measures are called so since they are not based directly on student academic work but rather on how students perceive their own learning. Alumni may also be asked how and to what extent the program prepared them to achieve learning outcomes. In addition, people in contact with the students, such as supervisors and employers, may be asked to provide feedback on the effectiveness of program graduates. For indirect measures, the assessment is based on perception rather than direct demonstration. 2.3 Analysis of the Assessment Results Step 3 of the assessment cycle involves analyzing the results of Step 2. Faculty members interpret the assessment results in light of the intended learning outcomes. Understanding of the results must then become part of a broader faculty conversation across the academic program. In interpreting assessment results, the first element is to compare the actual outcomes to the benchmark or the outcomes that were intended (as identified in Step 1). The comparison determines whether the programs learning outcomes were achieved and students are learning what was intended, and whether there is room for improvement on any of the intended outcomes that were the subject of the assessment. In case of anomalous results, the comparison indicates if a problem exists with the assessment instrument itself or with the student learning outcomes. Faculty members must consider these points in order to begin to understand the assessment results. Once the faculty members conducting the assessment are confident of their findings, they should disseminate them to all program faculty members for discussion and interpretation. This can be achieved through faculty meetings, committee discussions, e-mail, etc. Assessment results almost always indicate some room for 560

improvement in the program. Assessment results should never be framed in such a way so as to unveil a particular person s shortcoming. Shortcomings should be considered as opportunities for improvement and faculty members are urged to ask how and what can we do better? Viewed from this perspective, assessment becomes opportunity to improve and do a better job for the students. 2.4 Recommending Changes Based on the Results The fourth and final step of the cycle is to use the analyzed results of the assessment to improve the academic program through recommending changes. These program changes must be directly related to the assessment results. If the original assessment has been well designed, it will likely identify one or more general areas where the program could be improved. Either the original assessment or possible follow-up studies will suggest the specific concerns that need to be addressed. Despite the fact that assessment results can point the way and suggest specific improvements, it is the duty of faculty members and administrators to reach a decision that balances the original intended learning outcomes, the available resources, and the other competing priorities of the program and the school. Using assessment results to make improvements in programs initiates a new cycle of assessment. After the program change has had a chance to take full effect, a new assessment is conducted, beginning as before with confirming the original mission statement and intended learning outcomes, and proceeding through evidence collection, interpretation of results, and any further modifications of the program that are indicated by the second round of assessment. Depending on the nature of any program changes, the time elapsed between the completion of the first assessment cycle and the beginning of the second may be as short as one semester or as long as a few years. 3. Approach Description Biomedical engineering course EENG574 (Medical Instrumentation II) was selected as an application of the assessment cycle illustrated in Figure 1. The course learning outcomes, listed in Table 3, were explained to students in details at three different stages (1) first class of the semester (2) beginning of each chapter to which relevant CLOs belong (3) last class of the semester. The assessment was conducted in the last class of the semester. Students were asked to carefully read each course learning outcome and check the appropriate box that corresponds to the extent they feel the class has helped them to achieve this course outcome. The results of the assessment are shown in Table 3. Obviously, the first five course learning outcomes were satisfying. However, 35% of the students assessed the last course learning outcome, CLO-6, as neutral. This means that this percentage of students is unsure about being able to remember the required knowledge to Identify the elements of risk for different instrumentation methods and basic electrical safety. The students evaluation reveals an important gap. In order to better understand the reasons behind this gap, the department conducted a root cause analysis of the matter. One possible reason lying behind the relatively below target average percentage is that this part of the course is only covered theoretically in class. It is not covered in EENG574L, the laboratory of the EENG574 course, which represents the hands-on application of the learned concepts. One possible solution can be to include hospital visits to observe biomedical engineering in general and electrical safety in particular in practice. The course coordinator in consultation with the department management and the course instructors recommended on-site hospital visits to be conducted in the future. Considering the notion of continuous on-going improvement, there is still room for changes to make the course 561

more attractive and self-supporting. Even though the process of assessing program students outcomes in the Department of Biomedical Engineering at LIU is still in its infancy, the Department strives to continuously apply and implement the closed loop assessment cycle of Figure 1. This is achieved by frequently asking students to provide useful feedback on what, how much, and how well they are learning. Faculty can then use this information to refocus their teaching to help students make their learning more efficient and more effective. Table 3 Course Learning Outcomes Assessment EENG574 - Section A - Beirut Please check the appropriate box that corresponds to the extent you feel the class has helped you to achieve the course outcomes below CLO-1 CLO-2 CLO-3 CLO-4 CLO-5 CLO-6 Describe different diagnostic measurement methods for different humane variables and their necessary instrumentation Explain different therapeutic methods of treatment where electrical medical equipments are a vital part of the method and their necessary instrumentation. Analyze the effect of different diagnostic and therapeutic methods, their risk potential, physical principles, opportunities and possibilities for different medical procedures. Demonstrate a basic understanding of medical terminology, relevant for biomedical instrumentation. Describe the physical and medical principles used as a basis for biomedical instrumentation. Identify the elements of risk for different instrumentation methods and basic electrical safety. 4. Conclusion, Observations and Future Work Strongly Agree Agree Neutral 11 5 1 8 8 1 Disagree 5 10 1 1 7 10 9 8 4 7 6 Strongly Disagree Developing learning outcomes and assessments, creating links to graduate attributes and mapping the curricula to them has provided LIU s Department of Biomedical Engineering with the opportunity to reflect on best practices in teaching and learning. As faculty come together to describe what students will know and be able to do upon the successful completion of a course; instructing and measuring learning become an integral part of the process. Management, staff, faculty and students in Biomedical Engineering have been working together to develop new and creative ways of instructing and assessing. This model is also being adopted across all engineering departments at LIU. Using the outcomes based approach to align with ABET accreditation provides an opportunity to enhance understanding of teaching and learning (ABET, 2011). This is crucial to institute a process that will facilitate continuous improvement. Based on best practices in higher education (Ramsden, 2003; Toohey, 1999), the Department of Biomedical Engineering has been implementing a strategy of learning outcomes setting and assessing. A sample course was assessed and it seems that the course in its current form can be considered relatively successful despite the fact that there was one critical course outcome that requires attention to be attained. In response to that gap, the department has put together an action plan at the end of the semester. The plan is slated for execution and implementation in the next semester. To increase the course efficiency, the course coordinator has added a new activity which includes on-site hospital visits. There will always be space for improvement and it is fully understood that assessment is an ongoing process with significant work remaining before the vision is fully realized. Despite the long road ahead, it is clear that the ABET s outcomes based approach to accreditation has created an opportunity for LIU School of Engineering 562

departments to make significant and positive changes to teaching and learning based on best practices in higher education. References ABET (2011). Accreditation board for engineering and technology guide, available online at: http://www.abet.org. Biggs J. (2003). Aligning Teaching snd Assessing to Course Objectives Teaching and Learning in Higher Education: New Trends and Innovations, University of Aveiro. Bloom B. S., Masia B. B. and Krathwohl D. R. (1964). Taxonomy of Educational Objectives, Volume II: The Affective Domain, New York, New York: McKay Publishing. Donnelly R. and Fitzmaurice M. (2005). Designing modules for learning, in: O Neill G. et al., Emerging Issues in the Practice of University Learning and Teaching, Dublin, Ireland: AISHE Publishing. Kennedy D., Hyland A. and Ryan N. (2006). Writing and Using Learning Outcomes, Bologna Handbook, Implementing Bologna in Your Institution, Berlin, Germany: Raabe Verlag Publishing. Looney J. W. (2011). Integrating formative and summative assessment: Progress toward a seamless system?, OECD Education Working Papers, No. 58, OECD Publishing, available online at: http://dx.doi.org/10.1787/5kghx3kbl734-en. LIU (2013). Lebanese international university website, available online at: http://www.liu.edu.lb. OEA (2011). Order of engineers and architects regulations, available online at: http://www.oea.org.lb. Ramsden P. (2003). Learning to Teach in Higher Education, London, UK: Routledge Publishing. Toohey S. (1999). Designing Courses for Higher Education, Buckingham, UK: SRHE and OU Press. World Bank (2012). Middle East and North Africa Region University Governance Benchmarking Report, Center for Mediterranean Integration (CMI), World Bank, Marseille: France. 563