Hands-On CFD Educational Interface for Engineering Courses and Laboratories

Hands-On CFD Educational Interface for Engineering Courses and Laboratories FREDERICK STERN IIHR-Hydroscience & Engineering The University of Iowa TAO XING IIHR-Hydroscience & Engineering The University of Iowa DONALD B. YARBROUGH Center for Evaluation and Assessment The University of Iowa ALRIC ROTHMAYER Aerospace Engineering Iowa State University GANESH RAJAGOPALAN Aerospace Engineering Iowa State University SHOURYA PRAKASH OTTA Aerospace Engineering Iowa State University DAVID CAUGHEY Mechanical and Aerospace Engineering Cornell University RAJESH BHASKARAN Mechanical and Aerospace Engineering Cornell University SONYA SMITH Mechanical Engineering Howard University BARBARA HUTCHINGS Fluent Inc. SHANE MOEYKENS Fluent Inc. ABSTRACT This study describes the development, implementation, and evaluation of an effective curriculum for students to learn computational fluid dynamics (CFD) in introductory and intermediate undergraduate and introductory graduate level courses/ laboratories. The curriculum is designed for use at different universities with different courses/laboratories, learning objectives, applications, conditions, and exercise notes. The common objective is to teach students from novice to expert users who are well prepared for engineering practice. The study describes a CFD Educational Interface for hands-on student experience, which mirrors actual engineering practice. The Educational Interface teaches CFD methodology and procedures through a step-bystep interactive implementation automating the CFD process. A hierarchical system of predefined active options facilitates use at introductory and intermediate levels, encouraging self-learning, and eases transition to using industrial CFD codes. An independent evaluation documents successful learning outcomes and confirms the effectiveness of the interface for students in introductory and intermediate fluid mechanics courses. Keywords: computer-assisted learning, hands-on CFD Educational Interface, simulation technology I. INTRODUCTION There is no question of the need and importance of integrating computer-assisted learning and simulation technology into undergraduate engineering courses and laboratories, as simulation-based design, and ultimately virtual reality, become increasingly important in engineering practice. The scope of simulation technology is broad covering computerized systems and computerized solutions of engineering problem formulations using mathematical physics modeling, numerical methods, and high performance computing; all of which broadly influence all engineering disciplines. Recent research has shown the effectiveness of computer-assisted learning for accounting tutorials [1], food process design projects [2], electrical machines laboratories [3], the use of multi-media courseware for bicycle dissection [4] and scrapers [5], and an on-line internal combustion engine research facility using both computations and experiments [6]. Systemsbased simulation technology has also shown to be effective for chemical plant design [7], electronics laboratories [8], and chemical processes [9], including the use of commercial software for chemical processes [10] and educational computer programs for mechanical systems [11] and neural networks [12]. Methods for assessing the effectiveness of using simulation technology in engineering education include student presentations, surveys, and interviews; student performance, including pre- and post-tests both with and without intervention; statistical analysis; and faculty perception. January 2006 Journal of Engineering Education 63

With respect to employing simulation technology in the curriculum, consideration must be given to issues of: learning vs. research objectives; usability vs. predetermined objectives; and student demographics. Previous studies focusing on use of simulation technology in education have shown enhancement of the curriculum [1 12]; increased learning efficiency and understanding [7, 8, 10]; effectiveness of novel and hands-on learning methods [4]; efficacy of combined physical and simulation laboratories [8]; importance of user-friendly interfaces [5, 11]; and positive student responses [7, 10]. Curricula must be developed for physics-based simulation technology, such as computational fluid dynamics (CFD), which is of present interest, but diverse learning objectives and limited research both are complicating factors for successfully incorporating CFD into the curriculum. CFD is a widely used tool in fluids engineering, with many specialty and commercial CFD codes in use through out the world, covering many application areas. The lack of trained users is a major obstacle to the greater use of CFD. In parallel with the use of CFD for research and development activities over the past 35 years, graduate student level CFD courses have become well developed and common in most engineering discipline graduate programs. Intermediate and advanced level CFD courses teach modeling and numerical methods using textbooks, computer-programming assignments, and specialty [13 15] or commercial software [16 18]. These courses have a common objective of learning CFD for code development and applications in support of M.S. and Ph.D. thesis research. More recently, as CFD becomes pervasive in engineering practice and engineers are expected to use it without post-graduate education, educators have additionally focused on teaching CFD at the undergraduate level. Various curricula have been developed, including CFD courses, laboratories, and/or projects and multi-media [19, 20], studio models [21, 16 18], and computerized textbooks [22]. These curricula use both specialty [23, 24] and commercial [16 18, 25 27] software, which is sometimes combined with experiments [16, 27]. Additionally, the curricula frequently cover a diverse range of learning objectives. A graduate student intermediate level CFD course is generally also open as a technical elective to undergraduate students, while the curriculum is optimized separately for the graduate or undergraduate groups. Integrating specialty or commercial CFD software for the non-expert user into lecture and/or laboratory courses can facilitate comparisons with experiments and analytical methods. The objective is to enhance the curriculum through use of interactive CFD exercises, multi-media, and studio models for teaching fluid mechanics, including heat transfer and aerodynamics. A limited evaluation following the aforementioned methods shows promise, with achievements as noted in the previously mentioned studies at both graduate and undergraduate levels. However, there remain many unresolved issues. For example: 1. When is the hands-on and discovery-oriented approach to be preferred over demonstration? 2. When does CFD detract from, rather than aid, the development of deeper knowledge of fundamental fluid mechanics concepts? 3. How can student perception of CFD as a black box be avoided, and understanding of detailed CFD methodology and procedures be promoted? 4. Should specialized educational software replace the use of commercial software? 5. How can the steep learning curve required for practical engineering applications be mitigated? 6. What are the best approaches for introductory vs. intermediate undergraduate and intermediate vs. advanced graduate level courses? 7. When is lecture and laboratory course teaching more appropriate than the studio and multi-media models? 8. What is the best curriculum content for teaching code developers vs. expert users? The most effective curricula to achieve optimal CFD education remain unspecified, partly due to the limited evaluation and assessment performed to date. This research focused on the development, implementation, and evaluation of an effective curriculum for students to learn CFD in introductory and intermediate undergraduate and introductory graduate level courses/laboratories. The curriculum is designed for use at different universities with different courses/ laboratories, learning objectives, applications, conditions, and exercise notes. The common objective is to teach students from novice to expert users who are well prepared for engineering practice. This also accommodates all previously mentioned learning objectives, except for computer programming. Here, an expert user is a person well qualified to enter engineering practice as a CFD engineer. Incorporation of commercial industrial software such as CFX, FLUENT, and StarCD can expose students to the same or similar software they may use as professionals in industry. To allow students early hands-on experience, while avoiding the steep learning curve typically associated with any sophisticated software system, and to avoid having students treat the software as a black box required development of an Educational Interface. Here, hands-on is the use of a CFD engineering tool to achieve a meaningful learning experience that mirrors real-life engineering practice. The CFD Educational Interface developed in this study teaches CFD methodology and procedures through the step-bystep interactive implementation that automates the CFD process. A hierarchical system of predefined active options, which facilitates the use of the Educational Interface at introductory and intermediate levels, encourages students self-learning, and eases the transition to using industrial CFD codes. A later section (IV) reports the independent evaluation of these educational tools conducted through collaboration with the University of Iowa, Center for Evaluation and Assessment (CEA). Our industrial partner, Fluent Inc [28], is disseminating the CFD Educational Interface and associated exercise notes. Section II explains the concept of the CFD Educational Interface, including the development process that lead from using FLU- ENT and unmodified FlowLab directly to the development of the CFD Educational Interface; design specifications; detailed features and differences compared to FLUENT and unmodified FlowLab; and prototype capabilities. Section III describes the implementation and refinement of the CFD Educational Interface at partner sites. The different universities, course/laboratories, learning objectives, applications, conditions and exercise notes are described, which provides evidence of the versatility of the CFD Educational Interface. More details are given for one of the sites since it was at this site where both formative and summative evaluation occurred. Section IV presents the evaluation design and results, including formative evaluation at all sites, summative evaluation and outcomes assessment at one site, and overall conclusions and discussion. Limitation 64 Journal of Engineering Education January 2006

of resources precluded collecting pre- and post-test achievement data at all sites; however, based on summative survey results (skill and efficacy ratings) and their convergence with the knowledge test data from Site 1 (The University of Iowa), it is reasonable to assume similar outcomes at the partner sites. Lastly, Section V provides conclusions and future work, including discussion of previously listed unresolved issues regarding those addressed by the CFD Educational Interface and those remaining to be addressed. While the research is focused on a subject of special interest to some but not all engineering disciplines (CFD), it is offered as a case study of the application of similar industrial software in other engineering fields. II. CFD EDUCATIONAL INTERFACE The concept of a CFD Educational Interface resulted from the authors collaboration on the development, site testing, and evaluation of teaching modules (TMs) for complementary CFD, experimental fluid dynamics (EFD), and uncertainty analysis (UA). The project, entitled Integration of Simulation Technology in Undergraduate Engineering (ISTUE), was sponsored by the National Science Foundation from March 2002 through February 2005. An earlier proof of concept study (1999 2002) used FLU- ENT directly. Introducing FLUENT to novice users required lengthy detailed instructions. Some users were confused by the many parameters that were required to be set, many of which were often unrelated to the particular student application of interest and difficult to explain. Because experienced users can perform such tasks manually, the FLUENT interface did not provide automated options for modeling, numerical methods, and verification and validation studies, as desired for learners in the current study. Such automation could have been developed for FLUENT directly. However, because pre-processing is handled by a separate application, GAMBIT, other avenues for developing an Educational Interface were considered. The initiation of the collaboration coincided with Fluent s release of FlowLab version 1.0 (2002). FlowLab is designed as a general-purpose CFD template, which allows students to define a geometry, specify physics, mesh the domain, and solve CFD models using predefined exercises. During the first year of the project, faculty partners collaborated with Fluent on setting up CFD templates for their respective learning objectives, courses, and/or laboratories with an agreed focus on introductory undergraduate level pipe flow and airfoil exercises. This initial work was performed using unmodified FlowLab version 1.0 (the 2002 release version). After completing a capabilities review of these exercises, and upon coming to an agreement regarding a systematic CFD process in the context of the requirements of the present initiative, faculty partners and Fluent implemented modifications to the FlowLab operations menu to conform with the agreed upon requirements of the CFD process. This work also included verifying the accuracy of results from the templates for specific applications, including making comparisons with analytical and experimental validation data. Interim evaluations in 2002 2003 confirmed that the implementation was worthwhile and promising, but also identified opportunities for improvement. The use of different specialized CFD templates for each exercise did not directly facilitate adherence to the previously agreed upon CFD process, and these differences further complicated site testing. Additionally, these initial exercises lacked options and depth, and were overly automated in some instances, giving students a black box impression of CFD. Some aspects of the interface were not very user-friendly, and solution accuracy and the quality of flow visualization were substandard relative to the requirements of the present initiative. Anonymous student responses suggested that the EFD, CFD, and UA labs were helpful to their learning of fluid mechanics and provided practice with important tools that they may need to use as professional engineers. However, they also reported that they wanted the learning experience to be more hands-on and tailored to their personal learning needs. This formative feedback led to development of the CFD Educational Interface, which among other objectives, provided a vehicle for more close adherence to the agreed upon CFD process. This activity was of direct benefit to Fluent in that the operations menu for unmodified FlowLab exercises was updated to conform with the CFD Process, requiring physics options to be specified prior to developing the computational mesh. Selected formative evaluation comments from students based on their use of FLUENT, unmodified FlowLab, and the CFD Educational Interface are listed in Table 1. A. Design Specifications The CFD Educational Interface is designed to teach students systematic CFD methodology (modeling and numerical methods) and procedures through hands-on, user-friendly, interactive implementation of practical engineering applications, while not requiring computer programming. The CFD process is automated, following a step-by-step approach which leads students seamlessly through setup and solution of the initial boundary value problem (IBVP) appropriate for the application at hand. The CFD process mirrors actual engineering practice: geometry (solid and other fluid boundaries), physics (compressible/incompressible, with/without heat transfer, fluid properties, modeling, initial and boundary conditions), mesh specification (structured/unstructured, manual/ automatic meshing), solution procedure (numerical parameters, solution convergence monitoring, different numerical schemes), and reports/post-processing (flow visualization, analysis, verification, validation using imported EFD data and uncertainties). A hierarchical system of predefined active options facilitates the use of exercises at both introductory and intermediate levels, and encourages students self-learning. Enough information is provided to ease the student transition from this intermediate level to using the full FLUENT (or any other industrial CFD) code directly. A static sketch window is used to illustrate the flow problem currently being investigated. Generalization of internal and external flow templates to inter and multi disciplinary applications facilitates their use at different universities having different objectives, applications, conditions, and exercise notes. B. Features The hands-on CFD Educational Interface has the following features: 1) User-friendly and interactive interface: The interface design is in the objective-oriented mode with interactive interfaces and smooth data transformations. 2) Follows exactly the CFD Process : The software orients students to setup, solve and analyze CFD problems step-by-step, while conforming to the CFD Process as defined by collaborating faculty partners. January 2006 Journal of Engineering Education 65

Table 1. Selected student comments on using FLUENT, unmodified FlowLab, and the CFD Educational Interface. 3) No requirement for advanced computer language skills: The interface is designed to help students focus on CFD methodology and procedures following the CFD process. 4) Stand-alone application: Unlike most CFD commercial software that requires different software applications to perform grid generation, solving, and post-processing, this Educational Interface combines all of the necessary steps to define and solve an Initial Boundary Value Problem (IBVP). 5) Compatible with Microsoft Operating Systems: Student familiarity with Microsoft Operating Systems facilitates the learning and the use of this interface. The interface allows a user to copy, paste, and import or export data. Figures/Data can be edited in popular Microsoft software, such as WORD, EXCEL, and NOTEPAD. 6) Different depths of CFD templates: Options for CFD templates are designed in such a way that they can be used at both introductory and intermediate levels. 7) Hands-on: Students interact with the software using mouse and keyboard input. Students use CFD, EFD and UA engineering tools in a meaningful learning experience, which mirrors as closely as possible a real-life engineering practice. 8) Self-guided studies: The teaching modules are designed to meet students requirements on self-learning. 9) Powerful and accurate solvers: The interface was built on top of GAMBIT and the solvers applied are the same as the solvers used in the commercial software FLUENT. 10) Powerful virtualization tools: Virtual reality tools enhance students understanding of fluid physics. The CFD Educational Interface uses GUI tools to plot contours, vectors, streamlines and make animations. 11) CFD uncertainty analysis: For the first time, CFD verification and validation tools are incorporated into an Educational Interface to enable students to learn the basic theory of CFD Uncertainty Analysis. 12) Sketch window: This feature illustrates the geometry and boundaries with all of the nomenclature that will be used in the simulation. The primary differentiators between FLUENT, unmodified FlowLab and the CFD Educational Interface are illustrated in Table 2 for the twelve features listed above. C. Prototype The prototype for the CFD Educational Interface was constructed using FlowLab versions 1.1 (2003) and 1.2 (2004) to create common CFD templates for flow in pipes, with and without 66 Journal of Engineering Education January 2006

Table 2. Different features for FLUENT, unmodified FlowLab, and the CFD Educational Interface. heat transfer; for compressible flow in a nozzle with shock waves; flow in a diffuser; flow past a circular cylinder; flow past an airfoil; and the Ahmed car with unsteady separation. Students interact with the software using mouse and keyboard input following the systematic CFD process. The CFD Educational Interface combines software tools for grid generation, flow solving, and postprocessing to establish and solve an IBVP. The student s familiarity with menu-driven software systems facilitates the easy use and learning of the interface. All functions of the interface, such as copy and paste of the figures and import and export data, are conducted using commonly available office software (WORD, EXCEL, and NOTEPAD). The FlowLab interface was built on top of the Fluent grid generation software, GAMBIT, and the solvers are the same as those used in the commercial version of FLUENT. The interface uses GUI tools to plot contours, vectors, streamlines and to make animations. Verification and validation tools are included for teaching CFD uncertainty analysis. Figure 1 shows a screen image of the pipe flow template at a specific step of the CFD process. Figure 2 is a flow chart showing the combined capabilities of the current CFD templates, as are described next. 1) Geometry: Students can create different geometries and domains, including: (a) pipe, (b) nozzle, (c) airfoil (Clark Y, NACA 0012, LS(1)0417, or import geometry data), (d) diffuser (asymmetric or axisymmetric), and (e) 2D Ahmed car body. Students need to input different parameters for the particular class of geometry they have selected, such as pipe (radius and length), nozzle (inlet/outlet/throat radius, converging/diverging/outlet length, plenum length/radius), airfoil ( O / C mesh topology, chord length, angle of attack), diffuser (inlet/outlet dimension/ length, diffuser angle), and Ahmed car (slant angle, upstream/ downstream length, domain height, gap). All geometry and domain parameters are illustrated in the sketch window (the pipe example is shown in Figure 1). 2) Physics: Students need to choose whether to model the flow as compressible/incompressible, with/without heat transfer, as inviscid/viscous, and as laminar/turbulent; set up the fluid properties January 2006 Journal of Engineering Education 67

Figure 1. Screen image for the pipe flow CFD template. Figure 2. Flow chart for the combined CFD template. (density, viscosity, specific heat, thermal conductivity); select appropriate turbulence models, if appropriate (S-A, k-epsilon, k- omega, V2F); and define boundary conditions (inlet, outlet, symmetry, wall, axis) and initial conditions. Students are required to specify all the variables (velocities, pressure, temperature, heat flux, turbulent quantities) on all boundaries using constant values, zero gradient, or specified distributions in order to emphasize and investigate the role of boundary conditions in well-posed IBVPs. 3) Mesh: Both structured and unstructured meshes are available. When using structured meshes the student either automatically or manually generates the desired meshes. Automatic meshing is designed for novice/introductory level students, who lack the basic knowledge of the methodology and procedures of mesh generation. By specifying coarse, medium, or fine meshes, the Educational Interface will automatically generate a mesh of the corresponding density using parameters hard coded in the software. Manual meshing is designed for professional/intermediate level students. To use this feature, students need to define the boundary grid by specifying the number of grid points, the grid spacing, and grid distribution functions for each boundary. This procedure is consistent with the steps and methodology applied in most commercial CFD software. 68 Journal of Engineering Education January 2006

4) Solve: Students need to specify appropriate solution parameters. These include whether the flow is to be treated as steady or unsteady, maximum iteration count, convergence limit, numerical precision (single/double), spatial difference scheme (1st order, 2nd order, QUICK scheme), and axial output locations (for output variables to compare with EFD). 5) Reports: After the iterative solution process converges, all the integral parameters of the solution, such as total forces and lift/drag coefficients, are reported. Various XY plots and verification and validation functions are also available for students to validate their simulations using benchmark, or their own, EFD data, and to conduct CFD uncertainty analysis. Available XY plots include axial velocity profiles, pressure coefficient distributions, centerline pressure/velocity distribution, shear stress, Y plus, wall friction factor, and wall temperature/nusselt number. The total reduction in magnitude of solution residual and final level of solution residual are used to determine stopping criteria for the iterative solution process. For unsteady flows, the time history of integral variables (e.g., drag force) is used to determine the degree of convergence of the iterative solution. At the introductory level, grid uncertainty is analyzed using only two meshes generated by the automatic function of the interface (coarse and medium, or coarse and fine). At the intermediate level, at least three meshes (generated either automatically or manually ) are used to quantitatively calculate grid uncertainties using Richardson Extrapolation. Grid refinement ratio can also be used to create different sets of meshes. In the future, reports will be combined with postprocessing. 6) Post-processing: Powerful tools can be used to visualize and examine the flow field, such as contours (total/static pressure, velocities, Turbulent Kinetic Energy, temperature, Mach number), vectors, streamlines, and animations. Animations can be used only for unsteady separated flows at the intermediate level course. III. IMPLEMENTATION AND REFINEMENT AT PARTNER SITES The CFD Educational Interface has been implemented at different universities with different courses/laboratories, learning objectives, applications, conditions, and exercise notes for introductory and intermediate undergraduate, and introductory graduate level courses and laboratories over the past three years in conjunction the development of TMs for the ISTUE project. Teaching Modules have three parts: (1) lectures on CFD, EFD, and UA methodology and procedures; (2) hands-on CFD Educational Interface for academic use of commercial industrial CFD software; and (3) exercise notes for use of CFD Educational Interface and complementary EFD and UA. Faculty partners are from colleges of engineering at large public, small private, and small historically minority private universities in departments of mechanical and industrial, aerospace, mechanical and aerospace, and mechanical engineering. Faculty partners developed TMs for their respective courses/laboratories using the same CFD Educational Interface. Courses/laboratories include introductory (all three years) and intermediate (the Fall of 2004) level fluid mechanics at The University of Iowa, introductory gas-dynamics laboratory and introductory aerodynamics laboratory at Iowa State University, intermediate fluids mechanics and heat transfer laboratory at Cornell University, and intermediate fluid mechanics at Howard University. Upon initiation of the ISTUE project, the faculty partners primary learning objectives were to integrate commercial CFD for non-expert users into lecture and/or laboratory courses, including comparisons with experiments and analytical methods, and to enhance the curriculum with CFD as an instructional tool for increased knowledge. Over the course of the project, the objective shifted to teaching CFD from novice to expert users well prepared for engineering practice using CFD Educational Interface, which accommodates the former objectives. Although all the faculty partners used the same CFD Educational Interfaces, the actual implementation varied considerably depending upon the course at hand and the faculty member s preferred teaching approach. The following will present an overview of the courses at all partner universities, but with The University of Iowa as a more detailed example. The introductory level fluid dynamics course at The University of Iowa is a four-semester hour junior level course, required of all students in mechanical and civil and environmental engineering and frequently elected by biomedical engineering students. Traditionally, the course used four lectures per week for analytical fluid dynamics (AFD) with a few additional EFD labs for highlighting fundamental principles. The course was restructured to consist of three-semester hours of AFD (3 lectures per week) and onesemester hour (one laboratory meeting per week) of complementary EFD, CFD, and UA laboratories, with detailed course, EFD and CFD lab learning objectives (Appendix A). The course is offered in both fall and spring semesters with about 65 and 15 students, respectively, with different professors in spring and fall, and four and two teaching assistants, respectively. The pipe and airfoil flow CFD Educational Interfaces were used. Three lectures were used to prepare the students for the complementary laboratories. At the start of course, AFD, EFD, and CFD are introduced as complementary tools of fluids engineering practice. At the start of EFD laboratories, EFD methodology and procedures are presented. At start of CFD laboratories, CFD methodology and procedures are presented. The CFD lectures cover what, why, and where is CFD used; modeling; numerical methods; types of CFD codes; the CFD process; an example; and an introduction to the CFD Educational Interface and student applications. The laboratories for fluid properties and EFD UA (EFD only), pipe flow (EFD and CFD), and airfoil (EFD and CFD) flow were sequential from the beginning to the end of the semester, with increasing depth. Detailed exercise notes guide students step-by-step on how to use the Educational Interface to achieve specific objectives for each lab, including how to input/output data, what figures/data need to be saved for the lab report, and questions that need to be answered in the lab report. CFD lab report instructions (Appendix B) guide students step-by-step through how to present their results and findings in written and graphical form. Lectures and exercise notes are distributed through the class Web site [29]. CFD concepts covered in pipe and airfoil exercise notes were developed to meet the learning objectives of course, EFD and CFD Labs (Appendix A). CFD concepts for pipe flow are definition of CFD process, boundary conditions (inlet, outlet, wall, axis), iterative and grid convergence, developing length and fully developed velocity profiles of laminar and turbulent flow, effect of single/double precision, verification using AFD for laminar flow, and validation using students own EFD data for turbulent flow. January 2006 Journal of Engineering Education 69

The CFD concepts for airfoil flow are boundary conditions (inlet, outlet, symmetry, airfoil), pressure coefficient and lift/drag coefficients, inviscid vs. viscous flow, effects of angle of attack, effects of turbulence models, and validation using students own EFD data. Student performance was evaluated based on their CFD Lab reports and pre-lab and post-lab testing. The CFD Lab report covers the purpose of the experiment and design of the simulation, the CFD process, data analysis and discussion, and conclusion. Pre- and post-tests cover the concepts students are expected to learn in the complementary laboratories (22 AFD, 19 CFD, and 22 EFD questions). All questions provided multiple alternatives of which only one choice was correct. Some questions may ask students to write down their own answer if none of the choices is correct. After choosing the answer for each question, students indicated how confident they were of their answer by circling a number on the confidence scale below that item, i.e., completely confident, somewhat confident, not at all confident, and just guessing. The intermediate level fluid dynamics course at The University of Iowa is a three-semester hour senior undergraduate and firstyear graduate level course elected by mechanical, civil and environmental, and biomedical engineering students. Traditionally, the course used three-lectures per week for AFD. The course was restructured for addition of the CFD lectures and laboratories, which count for one-third of the course grade. Detailed course and CFD lab learning objectives are presented in Appendix C. The course is offered in the fall semester with about 39 students, one professor, and one teaching assistant. The pipe, airfoil, diffuser, and Ahmed car flow CFD Educational Interfaces were used. Four lectures were used to prepare the students for the CFD laboratories. At the start of the course, CFD lecture 1, Introduction to CFD, was presented to prepare students to learn CFD methodology and procedures. The CFD lecture at the intermediate level covers similar topics to those in the introductory level course, but with more details on CFD uncertainty analysis. Three additional CFD lectures were presented to help students learn deeper CFD knowledge, including Numerical Methods for CFD, Turbulence Modeling for CFD, and Grid Generation and Post-processing for CFD. The laboratories for pipe flow, airfoil flow, diffuser flow, and Ahmed car flow were sequential from beginning to end of semester with increasing depth. Unlike the CFD labs at the introductory level, labs at the intermediate level are largely self-guided. However, a short workshop was used to show students the basic procedures and key functions/features of the Educational Interface before CFD Lab 1. Regular office hours were also provided every week to answer students questions. Detailed exercise notes guide students step-by-step on how to use the Educational Interface to achieve specific objectives for each lab, which is similar to those at the introductory level, but were designed to encourage students self-learning. CFD lab report instructions (Appendix D) help students step-by-step how to present their results and findings in written and graphical form. Lectures and exercise notes are distributed through the class Web site [30]. CFD concepts covered in pipe, airfoil, diffuser, and Ahmed car exercise notes were developed to meet the learning objectives of course and CFD Labs (Appendix C). The CFD concepts for the pipe flow are those covered in the introductory level pipe flow lecture, and more on iterative error, verification for friction factor and axial velocity profiles, the effect of grid refinement ratio, and validation using EFD. CFD concepts for the airfoil flow module are boundary conditions (inlet, outlet, symmetry, airfoil), effect of domain size, effect of order of accuracy on verification results, validation of pressure coefficient using EFD, manual definition of grid topology, effect of angle of attack, and inviscid vs. viscous modeling. CFD Concepts for the diffuser flow module are grid and iterative convergence, turbulent flow with/without boundary layer separation, streamlines, effect of turbulence models, effect of expansion angle, and validation using EFD. CFD Concepts for the flow over Ahmed car module are mesh and iterative convergence, effect of slant angle, unsteady boundary layer separation with vortex shedding frequency and Strouhal Number analysis, flow animations, and validation using EFD. Student performance is evaluated based on their CFD reports and pre/post-tests. The CFD report is in a similar format to that used in the introductory level report, but with questions that are more difficult. Types of questions in pre- and post-tests are similar to those used at introductory level, but cover more advanced topics in CFD (31 CFD questions), specially focused on CFD uncertainty analysis (verification and validation). The gas-dynamics and aerodynamics laboratories at Iowa State University are 0.5-semester hour courses required in aerodynamics engineering. Traditionally, laboratories used EFD for highlighting fundamental principles covered in complementary aerodynamics lecture courses for AFD, but were restructured for complementary CFD. The CFD lectures covered theory, Schlieren systems, and CFD methodology and procedures. The nozzle and airfoil flow CFD Educational Interfaces were used. Concepts introduced for the airfoil flow module are streamlines, streaklines, and path lines (AFD) and their connection to flow visualization using CFD and EFD, Bernoulli s equation, and aerodynamic characteristics of an airfoil (lift/drag coefficients vs. angle of attack). Concepts for the gas-dynamics-laboratory course are shock positions within a nozzle, 1st, 2nd and 3rd critical Mach numbers for the nozzle, axisymmetric vs. 2D flows, Mach number, and -shock wave patterns. The senior-level fluid mechanics and heat transfer lab course at Cornell University is required of all students in mechanical and aerospace engineering. Traditionally, the laboratory used EFD only, and was modified to include complementary CFD. The course typically has about 110 students with two professors and six teaching assistants providing instruction. A heated pipeflow experiment is one of six experiments the students perform during the semester, and this course also places emphasis on the ability of students to express themselves clearly in a technical document, so their lab reports are graded for clarity of expression as well as technical content. The pipe flow CFD Educational Interface was used. Concepts covered are: the CFD process, basic CFD strategy, turbulence modeling, and operating details of the Educational Interface. The Cornell experiment is unique among the pipe-flow experiments described here, in that it studies the effect of heat transfer from the pipe to the flowing gas and CFD is used to predict the development of the thermal boundary layer, as well as velocity profiles, inside the pipe. Comparisons are made with classical correlations and measured values of Nusselt number, as well as the effect of heating on pipe friction factor. Students find the added CFD component to be especially enlightening for this experiment, as they can see visualizations of the velocity and temperature fields 70 Journal of Engineering Education January 2006

inside the pipe details they cannot observe directly in the experiment, as the pipe wall is solid brass. The required, junior-level fluids mechanics course in the Mechanical Engineering department at Howard University does not have a formal laboratory component. The thermal-fluids laboratory is required in the second-semester as part of the Applied Thermodynamics course, which is also required. The fluid mechanics course had 25 students and one TA. The CFD section of the course consisted of two lectures devoted to basic CFD concepts and uncertainty analysis, two additional lectures covering the use of FlowLab, and one lecture/demonstration of each template from The University of Iowa. The students were required to do an internal and an external flow computational project that was an expansion of a textbook exercise using the FlowLab templates. The students used the two pipe flow templates and the airfoil templates for these assignments. Their performance was evaluated using their laboratory reports for the CFD analysis and by their performance on exams. The exams also involved a CFD exercise. IV. EVALUATION Over the three-year period of the ISTUE project, the third party evaluator implemented separate evaluation subprojects for each course at each university. The evaluation design for this project included both formative and summative focuses. In years 1 and 2, formative purposes were most important, i.e., the primary use of the evaluation information was to investigate ways that the educational components could be improved. For example, at three of the implementation sites (Cornell University, Iowa State University, and University of Iowa), students responded to objective Likert type and supply type items that allowed them to report their ability to learn basic concepts and problem solving skills, the strengths of the teaching and labs as they experienced them, and suggestions for improvement. A number of changes were implemented in response to these suggestions, including more hands-on activities, improved laboratory notes, more tailored and effective assignments, improved teaching modules, and the improved CFD template presented in Figure 2. At two annual meetings of project staff (June 2003 and June 2004), the evaluation team presented detailed reports of this formative evaluation, including analyses of students scaled and open-ended responses. What follows is a summary of these more detailed formative reports and how they resulted in improvements in the Educational Interface, teaching modules and instructional practices. A. The University of Iowa Prior to initiation of the ISTUE project, course evaluation were consistent with the ABET engineering criteria using course outcomes worksheets and assessment reports based on a broad, but limited, number of course objectives (about ten). Assessment techniques used student performance, student surveys administered by the College of Engineering, and faculty observation. As part of the ISTUE project, an additionally detailed evaluation was performed for the introductory level fluid mechanics course for all three years and for the intermediate level course for the last year. For the introductory level course, during the first year, detailed objectives and survey items were developed for lectures (12 objectives and 32 items), problem solving (7 objectives and 27 items), and EFD and CFD (5 objectives and 12 items each) laboratories. The student self-report surveys included demographics, allowed for student comments and suggestions, and were administered by the CEA with independent and anonymous student responses. The complete survey was used during the first and second years. Student survey data during the first two years indicated that students took the formative evaluation task seriously and that they could contribute good suggestions for improvements. Laboratory reports at both introductory and intermediate levels indicated that students learned the purposes of CFD and design simulation, CFD processes, and data analyses, including verification and validation, and that they developed a deeper knowledge of fundamental CFD concepts. Students comments in year 1 at the introductory level indicated that they needed hands-on as much as possible, that it was difficult to import EFD data and compare with CFD results, and that could benefit from more flexibility on specifying software functions, such as change of background color in XY plot and easier way on gathering and saving figures. Students comments in year 3 at the same level indicated that they thought, hands-on part is interesting and helpful, and data comparison between CFD and EFD is effective for more understanding of fluid mechanics. Intermediate level students comments in year 3 also provided very positive evaluations, such as I learned a lot from the labs, The design of this interface eases the CFD learning, I am satisfied with the hands-on part, and The interface will better prepare for industry and my future career. Overall student comments during the past three years indicated that they liked the hands-on, step-by-step approach, appreciated the features of the CFD Educational Interface that allowed flow visualization and comparisons with AFD and EFD features of the CFD Educational Interface and considered valuable the opportunity to learn CFD, which they may use in their future careers. However, they also felt that clearer instructions and a more userfriendly, in-depth, robust, faster interface should be developed, with broader internet accessibility. Students at introductory levels preferred to work in groups, whereas at the intermediate level, they preferred to work individually (one-person one-computer). Faculty opinions based on their observations were consistent with interpretations of students lab performance and their survey data. B. Iowa State University Formative evaluation at Iowa State also relied on students responding to end of course surveys, as presented in Tables 3 and 4. Responses to CFD-related questions indicated that students benefited from the use of FlowLab. Though most students assessed the volume of material covered to be approximately correct, a few students felt that the FlowLab exercises took too long to complete. However, most of the students appreciated having the CFD component in the course and felt that having all three components of fluid flow analysis, i.e., EFD, AFD and CFD, led to better understanding of the course material. C. Cornell University Formative evaluation using surveys at Cornell indicated that small groups were well suited to introducing students to CFD basics through the Educational Interface. The FlowLab experience improved students understanding of the lab through contour and January 2006 Journal of Engineering Education 71

Table 3. Key results from AERO E. 243L survey, Iowa State University. Response scale: Strongly Agree (SA), Moderately Agree (A), Slightly Agree (a), Slightly Disagree (d), Moderately Disagree (D), Strongly Disagree (SD), and No Opinion (nop). vector plots, and resulted in many students showing an enthusiasm for learning more about CFD. D. Howard University An informal self-evaluation at Howard received generally positive responses from students. The most frequent comment was the desire for more time to use the CFD software during the semester. E. Summative Evaluation and Outcomes Assessment: Overview By the third year of the project (2004-2005), the formative phase of the evaluation design was completed. The year 3 evaluation focused on documenting student outcomes for the revised and improved implementation of the CFD components, including the Educational Interface. In order to concentrate on documenting outcomes with objective achievement tests, the year 3 evaluation relied on multiple choice and supply-type objective tests of students knowledge of basic facts, skills, problems and applications related to CFD. The summative evaluation and outcomes assessment took place at The University of Iowa in both introductory and intermediate fluid mechanics courses. Implementing this summative evaluation required an objective measure of student outcomes in at least one curricular area, and the project staff and evaluation team chose the CFD Educational Interface as applied to undergraduate fluid dynamics curricula. First, the instructional staff wrote and revised a pool of items for each of the two fluid dynamics courses (introductory and intermediate level undergraduates) as described in the previous sections. Representative items from the test for each course are presented in Table 5 and Table 6. Then staff reviewed the items and paired them for equivalency following a Table of Specifications [31]. Items in each pair were randomly assigned to either an A or a B version of the pre-test. The same procedure was followed for both courses using the corresponding item pools. At the time of testing during the first week of classes, students in both classes were randomly assigned to either the A or the B version of the pre-test, which they completed. Later in the semester, after completing the appropriate study of CFD and supporting technology, the students then completed the posttests, which consisted of both the A and B versions. Thus, all students completed only an A or a B version for the pre-test, but took both the A and the B at post-test. F. Outcome Data for the Introductory Students 1) Pre/Post Knowledge and Skill Test Outcomes: The most intuitively appealing test of students knowledge and skill outcomes is whether their post-test scores were significantly higher than their pre-test scores on either the A or the B versions of the pre-tests and post-tests, depending on their random assignment. Table 7 presents the results for the introductory course. Both A and B versions of the tests each contained 10 items. As can be seen from Table 7, students who took the A version of the pre-test scored on average only about 30 percent correct, 72 Journal of Engineering Education January 2006

Table 4. Key results from AERE 311 survey, Iowa State University. Response scale: Strongly Agree (SA), Moderately Agree (A), Slightly Agree (a), Slightly Disagree (d), Moderately Disagree (D), Strongly Disagree (SD), and No Opinion (nop). probably close to what they would have scored by chance alone. This finding suggests that students did not have the skills or knowledge tested by the quiz prior to instruction. Students responding to the B version scored a little better, approximately 40 percent correct at pre-test, but still not well enough to indicate that they knew much of the information contained on the B version of the pre-test. The scores on the post-tests were also low and did not indicate much knowledge and skill growth on the tested content. Students who took the B version presented virtually flat performance, correctly answering on average only 38.4 percent of the items on the post-test. Students responding to the A version demonstrated some modest improvement on their post-test, going from about 30 percent correct on the pre-test to 46 percent correct on average on the post-test. However, this growth for the A version students, while statistically significant ( t (df 1, 32) 4.67, p 0.0001), can not be viewed as much of a success. The expectation was that all students would demonstrate substantial gains and move to 70 75 percent mastery on the post-test items for both A and B versions. It is also worth noting that all students took both A and B versions at post-test. Students post-test scores on the not pre-tested items (post-test B for those who had pre-test A and post-test A for those who had pre-test B) are similar to their other scores. Pre-test A students had a mean of 3.94 on the B items at post test, which is not significantly different from the B version pre-test students, whose mean on the post-test was 3.84 (see Table 7). Similarly, pre-test B students had a mean of 4.24 on the A items at post-test, which is only marginally different from the A version pre-test students, whose mean at post-test was 4.61. Taken by themselves, the post-test items indicate that these introductory students did not demonstrate satisfactory knowledge of this content as sampled by the post-test after instruction in CFD. These items were clearly too difficult for students at both pre-test and post-test. 2) Introductory Students Self Reported Knowledge and Skills: In the current evaluation, introductory students also selfassessed their growth in conceptual knowledge about fluid dynamics, experimental fluid dynamics and computational fluid dynamics on pre and post surveys. Specific items on the survey clustered into factors related to knowledge and skill in each of three clusters. Table 8 lists the items that constituted the CFD cluster. Students filled out the self-evaluation of their abilities at the beginning of the course and then again at the end, after they had taken the post-test. As can be seen from the items in Table 8, on the self-reported survey items, students were evaluating their knowledge and skill at a more general level than that measured by the test items. Table 9 January 2006 Journal of Engineering Education 73