SYLLABUS Fundamentals of Evaluation CPH 544 Spring 2013 Time: TBA Location: This is an on-line course Instructor: Ralph Renger, Ph.D. Contact Information: Address: 1295 N. Martin Ave. Room 225 Phone: (520) 626-0726 E-mail: renger@email.arizona.edu Teaching Assistant: Jirina Kitka Foltysova, foltysov@email.arizona.edu Office Hours: Because the course is on-line, students often believe the instructor is available 24 hours a day, seven days a week. This is not the case. Questions regarding course content should be posted to the discussion board for other students to assist in answering. The instructor will monitor the discussion board daily. If the answers being posted are incorrect or require additional clarification, then the instructor will make a note in the discussion to refer to the FAQ for a complete answer. Requests for face-to-face appointments are always welcome. Course Description: Evaluation is essential to all research- and service-based programs. The course provides students interested in pursuing an advanced public health degree with the fundamentals of planning and evaluation. Students are required to complete seventeen lessons. The lessons emphasize public health practice and are designed to assist students in understanding how to do evaluation. Students are introduced to different evaluation theories, including the integrated theory of evaluation (ITOE). ITOE provides the student with a framework for bridging theory into practice. Students are introduced to the logic model, a critical tool in guiding agencies interested in demonstrating the impact of the strategies they implement, through a planning and evaluation process. 1
Course Prerequisites: None Course Objectives: To gain a better understanding of the field of evaluation, the purposes of evaluation, and methodology that can assist in planning, implementation, and evaluation of service and research programs. MPH Competencies: 1. Leading and participating in groups to address specific issues, including ability to work in teams, span organizational boundaries, and cross systems 2. Translates policy into organizational plans, structures, and programs 3. Developing mechanisms to monitor and evaluate programs for their effectiveness and quality 4. Defining, assessing, and understanding the health status of population, determinants of health and illness, factors contributing to health promotion and disease prevention, and factors influencing the use of health services 5. Conduct projects that address one or more of the 3 core functions and 10 essential services of public health. 6. Understanding the theory of organizational structure and its relation to professional practice Course Notes: Available on-line Recommended Texts/Readings: Provided by Instructor and made available on-line Course Requirements: As this is an on-line course students are required to have access to the internet. It is not the responsibility of the instructor to provide access to the internet. Real Player and Windows Media Player are needed to view some of the course content (free download of Real Player available at www.real.com). Students having trouble viewing and downloading material are directed to CCIT and/or Htay Hla at the College of Public Health for assistance. It is the student s responsibility to resolve any technical difficulties. To access your course on D2L you must have a UA NetID and be officially enrolled in the course for at least 24 hours. Your browser and its settings must be compatible with D2L. 1. To begin, go to www.d2l.arizona.edu. 2. Click on the "Check that your browser is compatible" hyperlink. (This is step Number 1 in the student login instructions.) 3. Click on the UA NetID Login button on the top left side of the screen. 4. Enter your UA NetID and password (This is the same information you provide to access your UA Webmail) 5. Once you are inside scroll down to My Courses. If you cannot see your courses click on the down arrow on the right of the title bar. Then click on the hyperlinked Course Name. 6. On the Course Home page, take a look at blue and red navigational bars along the top of your screen. Click 'Content' on the blue bar to get to the Syllabus, etc. for your course. Each student is expected to independently complete all on-line assignments by the due date. Assignments are to be placed in the appropriate D2L drop box. The drop box will be set to reject assignments submitted after each assignment s corresponding deadline. 2
Grading/Student Evaluation: (i) Exams: There will be two on-line exams: The midterm exam worth 30% and a final exam worth 40%. The exams will be essay style with a word limit. Questions exceeding the word limit will not be graded. The exams are cumulative; however emphasis for each subsequent exam will be on new material covered. The date and time for each exam is set in advance (see course schedule below). The exam times have been selected to a) accommodate working professionals, and b) not conflict with other MEZCOPH course offerings. Setting the exam dates in advance provides students with conflicts ample opportunity to make alternative arrangements. Students who cannot make these exam times are encouraged to withdraw from the course immediately as there are no exceptions for rescheduling. You are required to agree to these terms by SUCCESSFULLY COMPLETING THE SYLLABUS AGREEMENT (located under the Content Section on D2L) before you will be able to continue with the course. Exams are released on the date and time as shown in the course schedule below. Students must submit their exams to the appropriate dropbox. The dropbox remains open for 15 minutes after the exam. Exams not submitted before the dropbox closes will not be graded. (ii) Course Improvement Feedback: As this class is about evaluation, incentives will be provided for students to provide feedback for improving the course content and delivery. Thus 5% of the student s final grade will be awarded for suggestions concerning course improvement. Students are asked provide specific feedback regarding course improvement on two occasions: 1. On or before February 23, 2013 - lessons 1 through 8) 2. On or before April 20, 2013 - lessons 9 through 17). Late feedback will not be accepted. Each feedback must be 250 to 400 words and include at least three ideas for additional exercises to enhance learning. The exercise ideas must be fully developed, with suggested instructions for students, links, reference materials needed, and so forth. Improvements in technical aspects and course content are also encouraged and appreciated. (iii) Debates: Four debates will be held during the course. Each student is required to participate in only one debate. On January 9, 2013 the debate chronology, propositions, rules, format, and grading criteria will be posted in the D2L content area in a file called Debates. On February 6, 2013 students will be randomly assigned by the instructor to a debate and to a group (affirmative or negative). The teams will be posted in the D2L content area in the files called Debate 1 Groups, Debate 2 Groups, Debate 3 Groups, and Debate 4 Groups. The debates One and Two will start on February 25, 2013 and the propositions will be based on the material covered in lessons 1-8. The debates Three and Four will start on April 1, 2013 and the propositions will be based on the material covered in lessons 1-13. Students are strongly encouraged to o Read all the materials to have all the information for the debatej. o Monitor all debates because a) they will be asked for feedback as to which side won the debates (required exercise), and b) the exam may contain questions about the debates. 3
(iv) Required Exercises: Most lessons have at least one and often several exercises designed to assist the student with learning concepts. Students are encouraged to complete all the exercises, as this material can be tested on the exams. However, only a few of the exercises are required and must be submitted for course credit. These are clearly marked. Required exercises for each section must be submitted on or before the due dates. Required Exercises are worth 10% of a student s final grade. Required Exercises: Lesson Number Required Exercise Name Due Date Dropbox Title Exercise Number 1 1 CST Project (Part 1) 1/19/2013 CST Project (Part 1) 3 2 HUD 1/26/2013 HUD 6 3 How do you know 2/9/2013 How do you know 8 4 CST Project (Part 2) 2/16/2013 CST Project (Part 2) 11 5 Logic Map 3/9/2013 Logic Map 11 6 Member Check 3/23/2013 Member Check 12 7 Yuma Logic Map 4/6/2013 Yuma Logic Map 13 8 Alaska Logic Map 4/13/2013 Alaska Logic Map 9 Debate Comments #1 3/30/2013 10 Debate Comments # 2 4/27/2013 11 Debate Comments # 3 Debate 1 Class Comments Debate 2 Class Comments Debate 3 Class Comments Debate 4 Class Comments Grading Scale Requirements Weighting A = 90% - 100% Midterm Exam 30% of final grade B = 80% - 89% Final Exam 40% of final grade C = 70% - 79% Debate 15% of final Grade D = 60% - 69% Required Exercises 10% of final Grade F = 59% and below Course Improvement 5% of final grade 1 Students not assigned to a particular debate will be asked for feedback as to which side won the debates. For example, student assigned to Debate 1, have to submit comments about Debate 2, Debate 3 and Debate 4. Student assigned to Debate 2, have to submit comments about Debate 1, Debate 3 and Debate 4. Student assigned to Debate 3, have to submit comments about Debate 1, Debate 2 and Debate 4. Student assigned to Debate 4, have to submit comments about Debate 1, Debate 2 and Debate 3. 4
Course Schedule: Date Topic January 9, 2013 Classes Begin. Complete Syllabus Agreement. Start working on lessons 1 and 2. The debate chronology, propositions, rules, format, and grading criteria posted. January 19, 2013 Required exercise for lesson 1 (Part 1) due by 23:59 MST. January 26, 2013 Required exercise for lesson 3 (HUD) due by 23:59 MST. February 6, 2013 Students agreeing to syllabus randomly assigned to a debate. February 9, 2013 Required exercise for lesson 6 (How do you know) due by 23:59 MST. February 16, 2013 Required exercise for lesson 8 (Part 2) due by 23:59 MST. February 18, 2013 Midterm Exam 30% 19:30-21:30 MST. February 23, 2013 Recommendations for course improvement #1 due by 23:59 MST. February 25, 2013 Debates One and Two begin at 19:00 MST. March 9, 2013 Required exercise #1 for lesson 11 (Logic Map) due by 23:59 MST. March 23, 2013 Required exercise #2 for lesson 11 (Member check) due by 23:59. March 28, 2013 Debates One and Two end at 19:00 MST. March 30, 2013 Class comments for Debates One and Two are due by 23:59 MST. April 1, 2013 Debates Three and Four start at 19:00 MST. April 6, 2013 Required exercise for lesson 12 (Yuma logic map) due by 23:59. April 13, 2013 Required exercise for lesson 13 (Alaska logic map) due by 23:59. April 20, 2013 Recommendations for course improvement # 2 are due by 23:59 MST. April 25, 2013 Debates Three and Four end at 19:00 MST. April 27, 2013 Class comments for Debates Three and Four are due by 23:59 MST. April 29, 2013 Final Exam 40% 19:30-21:30 MST. 5
Lessons 2 : January 9 - February 18, 2013: Theory in Evaluation Lesson 1 The Importance of Theory in Evaluation Lesson 2 Integrated Theory of Evaluation Lesson 3 Evaluation of Oversight and Compliance Lesson 4 Program Improvement Lesson 5 Knowledge Development Lesson 6 Evaluation of Merit and Worth Lesson 7 The difference between research and evaluation Lesson 8 Applying the Integrated Theory of Evaluation February 19 April 29, 2013: Logic Modeling Lesson 9 Introduction to Logic Modeling Lesson 10 Logic Modeling The ATM Approach, Step 1 Lesson 11 Logic Modeling The ATM Approach, Step 2 Lesson 12 Logic Modeling The ATM Approach, Step 3 Lesson 13 Logic Modeling Evaluation Standards and Reporting Lesson 14 Unanticipated Benefits of the ATM Approach Lesson 15 Using Results of the ATM Approach to Construct the Logic Model Lesson 16 Using Source Documentation to Reconstruct Program Theory Lesson 17 Other Approaches to Outcome Evaluation Supplemental Materials 3 Lesson 18 Introduction to the Field of Evaluation Lesson 19 Employment and Evaluation Lesson 20 Factors Affecting an Evaluator s Credibility Lesson 21 Impact of Professional Standards on Credibility Lesson 22 Building Trust Lesson 23 The Guiding Principles and Program Evaluation Standards Lesson 24 The Certification Debate Lesson 25 Empowerment Evaluation - The Fetterman-Scriven Debate Lesson 26 Mainstreaming Lesson 27 Theories in Evaluation Lesson 28 The Causal Attribution Debate Lesson 29 Reporting Evaluation Results 2 Lessons do not correspond to calendar weeks! These lessons are also subject to change as they are updated throughout the semester. 3 Supplemental materials focus on overview of the field of evaluation and current issues and debate within evaluation discipline. These materials are optional and will not be tested but it is highly recommended for those wishing to pursue an evaluation career to review this material on an on-going basis throughout the course. 6
APA Style: All documents submitted for this course must be in APA style. To familiarize yourself with APA style, please go to: University of Arizona library APA Style Guide http://www.library.arizona.edu/search/reference/citation-apa.html American Psychological Association (APA) (6th Edition, 2009) http://umclibrary.crk.umn.edu/apa6thedition.pdf Purdue Online APA Style http://owl.english.purdue.edu/owl/resource/560/03/ Plagiarism: What counts as plagiarism? Copying and pasting information from a web site or another source, and then revising it so it sounds like your original idea. Doing an assignment/essay/take home test with a friend and then handing in separate assignments that contain the same ideas, language, phrases, etc. Quoting a passage without quotation marks or citations, so it looks like your own. Paraphrasing a passage without citing it, so it looks like your own. Hiring another person to do your work for you, or purchasing a paper through any of the onor off-line sources. For more information, please go to Indiana (training/certificate) https://www.indiana.edu/~istd/plagiarism_test.html Plagiarism detection (Turnitin.com) All documents shall be submitted to D2L Dropbox. Upon students submitting their papers to D2L, instructor and TA will receive a report identifying unoriginal phrases or paragraphs within a student paper. Video instructions: Instructions for Windows users on how to unblock firewall can be found at http://windows.microsoft.com/en-us/windows7/allow-a-program-to-communicate-through-windows- Firewall Mac Users can download Windows Media Components for QuickTime at: http://www.microsoft.com/downloads/en/details.aspx?familyid=915d874d-d747-4180-a400-5f06b1b5e559&displaylang=en. The Components will give you the ability to watch videos that are compatible with Windows Media. These are files that end in extensions.wma and.wmv. 7
Communications: You are responsible for reading emails sent to your UA account from your professor and the announcements placed on the course web site. Information about readings, news events, your grades, assignments and other course related topics will be communicated to you via these electronic methods. The official policy can be found at: http://www.registrar.arizona.edu/emailpolicy.htm. Disability Accommodation: If you anticipate issues related to the format or requirements of this course, please meet with the instructor. He would like us to discuss ways to ensure your full participation in the course. If you determine formal, disability-related accommodations are necessary, it is very important to register with Disability Resources (621-3268; drc.arizona.edu) and notify instructor of your eligibility for reasonable accommodations. We can then plan how best to coordinate your accommodations. The official policy can be found at: http://catalog.arizona.edu/2012%2d13/policies/disability.htm. Academic Integrity: All UA students are responsible for upholding the University of Arizona Code of Academic Integrity, available through the office of the Dean of Students and online: The official policy found at: http://deanofstudents.arizona.edu/codeofacademicintegrity. Classroom/Online Behavior: Students are expected to be familiar with the UA Policy on Disruptive Behavior in an Instructional Setting found at http://policy.arizona.edu/disruptive-behavior-instructional and the Policy on Threatening Behavior by Students found at http://www.policy.arizona.edu/threatening-behavior-students. Grievance Policy: Should a student feel he or she has been treated unfairly, there are a number of resources available. With few exceptions, students should first attempt to resolve difficulties informally by bringing those concerns directly to the person responsible for the action, or with the student's graduate advisor, Assistant Dean for Student and Alumni Affairs, department head, or the immediate supervisor of the person responsible for the action. If the problem cannot be resolved informally, the student may file a formal grievance using the Graduate College Grievance Policy found at http://grad.arizona.edu/academics/policies/academicpolicies/grievance-policy. Grade Appeal Policy: http://catalog.arizona.edu/2012%2d13/policies/gradappeal.htm Syllabus Changes: Information contained in the course syllabus, other than the grade and absence policies, may be subject to change with reasonable advance notice, as deemed appropriate. 8