1 Research Methods in Applied Psychology: Psych 508 Spring, 2014 Monday and Wednesday 10 a.m. to 11:15 a.m. 2305 Gilman Instructor: Lisa Larson, Ph.D., 216 W Lagomarcino 294-1487 Office hours: Mondays 1:30 3:00 p.m. lmlarson@iastate.edu Course overview: A central purpose of this course is for students to demonstrate knowledge about the fundamentals of research design, specifically as they pertain to Counseling Psychology. In addition, students are expected to demonstrate the ability to use the scientific method to think about counseling. Students will be required to develop a research paper that will be critiqued by the instructor and students in the class. I hope this course will desensitize you to research. It is also hoped that you will gain a greater understanding of and appreciation for the role of research in our profession. Prerequisites: The prerequisites for the course include a research design course in undergraduate psychology and a basic graduate and/or undergraduate statistics course. In addition, students will have taken or are taking two graduate level statistics courses: Psych 501, 402, or 404. Required readings: 1. Research Design in Counseling, by Heppner, Kivlighan, & Wampold, 2008. 2. Course packet of articles and lecture notes on Blackboard Recommended: Research Design, Qualitative, Quantitative and Mixed Methods Approaches, by John Creswell, 4 th edition, 2014 The Publication Manual of the American Psychological Association, 6th edition. Objectives: 1. Knowledge of research methodologies, statistical techniques, and research paradigms used in Counseling Psychology. 2. Ability to critique data-based articles. 3. Ability to generate and interpret statistical analyses 4. Ability to conduct an experimental study, produce an APA paper, and present it in class. Grading: Your grade will be based on one exam (20%), Critiquing data based articles (10%), generating SPSS printouts (10%), Assignments 1 and 2 (5% each), critiquing each other s papers (5%), a class presentation (20%), and a research paper (25%). Class Participation: a. You will critique articles during class by answering the questions on the last page of the syllabus. You will sign up for one of the article critiques listed on the syllabus. It will be your responsibility to find an article for your critique. Everyone is responsible for answering the critique questions on the syllabus each week prior to class and
2 turning it in. The article must be data-based empirical, pertain to the day's topic (e.g., if we are discussing ANOVA then the article needs to include ANOVA as the analysis), and be emailed to the class and myself one week ahead of time. The dates are marked in the syllabus. (10%) b. Each of you will be responsible for running and interpreting statistical analyses that match with the statistical analyses being discussed that day. In class we will discuss the results and conclusions drawn from the analysis. The dates are marked in the calendar. (10%) c. Completion of assignment 1 and assignment 2. (10%) Both of these pertain to the abbreviated research paper d. Critique of colleagues' research papers (5%). Exam: The exam, Feb. 17 th, will be essay and short answer. I will give you a study guide on Blackboard. Abbreviated Research Paper. The research paper should be no more than 20 pages of text and will be worth 25% of your grade. It will include an abbreviated introduction (the purpose of the study, the hypotheses in the study, and the rationale for the hypotheses), the methods, the results, and the discussion. My rubric for grading will be on Blackboard. The paper will be an experimental study in which the minimum sample size requirement will be 10 people per cell and at least 3 cells. This means if you chose to do a within subjects design, then 10 participants will be sufficient. If you choose to do a between groups design with one independent variable with three levels, then 30 participants would be sufficient. If you choose more independent variables or more levels, then just remember you need at least 10 participants per cell. You may choose to collect data online. Data collection will occur outside of class. A rough draft of your paper is required April 16 th. You will distribute copies of the rough draft to your colleagues who will serve as reviewers of your paper. The rough draft will not be graded but it is an essential requirement of the class. The research paper may relate to your thesis idea but cannot be used for a conference presentation and research article since we are not going to be using the ISU IRB approval process. This is ethical because we will collect the data for educational purposes only. The final paper is due April 30th. THE PAPERS WILL BE DUE BY 5 P.M. ON THE DAY THEY ARE ASSIGNED. Class Presentation: Near the end of the semester you will present your research paper to the class allowing some time for oral discussion and feedback of your paper. The length of your presentation will be 15-20 minutes. Your presentation will be worth 20% of your grade. I will place the rubric for my grading of the presentation on Blackboard. Your colleagues will also give you written feedback. Critique of Quantitative Research Article 1. What are the underlying assumptions concerning this research concerning the nature of science and how we know what we know? 2. What are the research questions? 3. What is the research design?
3 4. If the study is experimental, it is making a causal rather than relational claim. Have the authors shown: a. that the iv and dv co occur, b. that the iv came first, c. that alternative explanations for iv causing dv have been ruled out? 5. How are the variables operationalized or measured? 6. Are the responses to the measures reliable and valid? (Construct validity) 7. Is the sample representative of the population of interest? (External validity) 8. Is there sufficient power to obtain significant results? (Statistical conclusion validity) 9. What are the statistical analyses? Does it match the research question? 10. What are the results? Match the results with your hypotheses that you identified above. Don t worry about additional results for now. If the results are significant, can you determine the effect size? Did the authors have sufficient power to find a significant difference if it was there? 11. Is the discussion of the results in line with the results? Critique of Qualitative Research Article 1. What type of qualitative study is it? Is it an ethnography, case study, biography, portrait, phenomenology, or grounded theory? (see figure on BLACKBOARD) 2. What are the underlying assumptions concerning this research concerning the nature of science and how we know what we know? 3. Is the purpose of the study to explore the unique or the particular in depth? 4. To what extent did the findings match reality or are they credible or adequate. That is, did they use triangulation, member checks, peer examination, involving participants in all phases of the research process, clarifying researcher bias at outset, prolonged engagement in the field)? (translation of internal validity) (see page on BLACKBOARD) 5. How transferable or applicable or comparable is the information as judged by use of thick, rich description, examination of the typical or modal category, or use of cross-site analysis? (translation of external validity) (see page on BLACKBOARD) 6. How dependable or consistent are the results as judged by the extent to which the authors used purposive sampling, triangulation, clear audit trail? (translation of reliability) 7. Is the discussion in line with the results? (see page on BLACKBOARD
4 Week Day Topic Article Book Chapter Homework PART I: QUANTITATIVE PHILOSOPHICAL FOUNDATIONS/RESEARCH TRAINING/ETHICS 1 2 Jan. 13 Overview/introduction Jan. 15 Training Scientists in the Scientist-Practitioner Model Philosophy of Science 1, 2 Jan. 20 HOLIDAY no class Jan. 22 Choosing Research Topics 3, 4 Design Relation of the research question to the research design Relation of the research design to the statistical analyses Jan. 27 Ethics 6 3 4 Jan. 29 Feb. 3 Feb. 5 Asking different types of questions and validity related to each Asking Frequency and Relational Questions: Construct Validity Asking Causal Questions: Statistical conclusion validity, external validity, and internal validity Experimental Between group and within group designs Chapter 3 by Beth Morling Relational study Larson & Borgen, 2006 *: Experimental between groups 5 5 7
5 5 Feb. 10 Feb. 12 Contrasting Quantitative Philosophical Assumptions and Qualitative Philosophical Assumptions Qualitative Research continued Conceptual/methodological issues in multicultural research 6 Feb. 17 EXAM ONE study Experimental within groups design Cresswell Chapter 9 Qualitative Cresswell Chapter 9 11 PART 2: DESIGNING EXPERIMENTAL STUDY AND STATISTICAL ANALYSES 6 Feb. 19 Hypothesis Generation 12 Assignment 1 assigned Feb. 24 Working out logistics of 13 study: Construct validity Measurement 7 Feb. 26 Working out logistics of study: Internal validity 14 Assignment 1 due Assignment 2 assigned 8 Mar. 3 Working out logistics of research study when examining multicultural research: Multicultural research article with ethnicity as mediator or moderator 15
6 Conceptual/methodological issues in multicultural research 9 9 10 11 Mar. 5 Mar. 10 Meeting individually with the instructor Going over Data Set Statistical Conclusion Validity Power Analysis Mar. 12 Causal questions: Answering whether or not one group s proportion or mean is significantly different from another group s proportion or mean Chi Square / Dependent t test and independent t test Mar. 17 Spring Break Mar. 19 Mar. 24 Expanding to include comparisons of group means beyond 2 groups ANOVA Mar. 26 ANOVA with interaction (moderation) Larson et al. (1999) Cohen (1992) Chi square, dependent t-test, independent t-test ANOVA Bushman study 3 Baron & Kenny (1986) Assignment 2 due Assigned: SPSS printouts: COSE Scale reversals, factors, totals, and descriptives SPSS printouts: COSE Scale reversals, factors, totals, and descriptives due SPSS printout for Chi square, dependent t-test, and independent t-test assigned SPSS printout for Chi square, dependent t- test, and independent t-test due Assigned: SPSS printout ANOVA no interaction SPSS printouts for ANOVA no interaction due SPSS Printout: ANOVA with
7 12 13 Mar. 31 Extension of ANOVA to add power by removing within group error due to a continuous variable ANCOVA Apr. 2 Answering Simple and More complex Relational Questions when the dependent variable is continuous: Correlation and Multiple Regression Apr. 7 Apr. 9 Answering More complex Relational Questions when the dependent variable is categorical: Binary logistic regression Data analysis How to write/suggestions for publishing/going over paper Data analysis ANCOVA article JCP: 383-392 Critique: Staggs et al. (2003) JCP (372-382) Larson, Stephen, et al. 2013 India Study interaction assigned 11(244 255) SPSS Printout: ANOVA with interaction due SPSS Printout with ANCOVA assigned SPSS Printout with ANCOVA due SPSS Printout SPSS Printout: Correlation and Multiple regression assigned SPSS Printout: Multiple regression analysis due Data collection completed 14 Apr. 14 Apr. 16 Peers reviewing papers Rough drafts due 15 Apr. 21 Analyzing example data sets Apr. 23 Presentations Apr. 28 Presentations 16 Apr. 30 Presentations Final Paper Due Course evaluations *For article critiques: Get article approved by Dr. Larson at least 1 week before leading the critique in class, and distribute it to your classmates once it is approved.