Strategies for Educational Inquiry Y520 - Section 6342 Spring Semester 2004 Wright Education Bldg. 2277 A sense of humor is the oil of life s engine. Anonymous Robert S. Michael, PhD Center for Evaluation & Education Policy 855-1240 rsmichae@indiana.edu http://www.indiana.edu/~educy520/index.html Office Hours: By appointment Text Johnson, Burke, & Christensen, Larry. (2004). Educational Research: Quantitative, Qualitative, & Mixed Approaches, 2nd Edition. Prentice Hall. 1 Additional Readings: Assigned for some sessions. Some readings will be required. Others will be suggested. Computers Each student should have an IU-Bloomington username and password, an e-mail account, an Internet connection and browser, access to either a statistical program or (as a last resort) a spreadsheet. You also might find software such as Citation, Orbis (part of Nota Bene), Procite, Endnote useful. Course Description The catalogue description for Y520: Introductory course intended to orient beginning graduate students to the conduct of social science inquiry in general and educational inquiry in particular and to acquaint them with key terms and generally accepted procedures in qualitative and quantitative inquiry. 2 Throughout your career you will encounter empirical research reports that claim to describe a true state of affairs about a particular aspect of reality. The report may (a) describe a phenomenon of interest to educators, or (b) identify causal relationships between certain outcomes of interest and antecedent conditions and/or characteristics. In Y520 students learn to analyze and evaluate the adequacy of these descriptive studies and claims of causation by focusing on the theory and specific methods of empirical social science research used to conduct inquiry about education related topics. No pre-requisites are listed for this course. However, some statistical and quantitative analytical techniques are, of necessity, used. Students should be able to perform basic mathematical operations. Completion of an undergraduate course in statistics is helpful, but not required. 1. You should recognize that this reference does not conform to APA standards for citing references. In what way(s) is it incorrect? 2. Bulletin 2003-2005: Indiana University School of Education Graduate Program, p. 89. 1
Goals: 1. Become familiar with the major philosophical approaches that underlie research in education. 2. Become familiar with the ideas and concepts underlying empirical, analytical investigation. 3. Become familiar with the major research designs and control techniques and how to apply them. 4. Improve your ability to read, understand, interpret, evaluate and use empirical research. 5. Learn to discriminate between good research and other (presumably, not-so-good) research. 6. Appreciate the role of judgment when drawing inferences from data and analysis. 7. Learn how to discriminate among and interpret commonly used statistical tests, and to select the appropriate statistical tests for a given research problem. 8. Improve your analytical and writing skills. Topics A. Philosophical Context 1. What is ontology, worldview, epistemology, knowledge, truth, belief, justification/warrant? How is each related to a researcher s methods? 2. What is positivism, postpositivism, subjectivism, interpertive perspective, naturalism, relativism, critical theory, marxism, postmodernism, modernism? In what way(s) are each of these terms related to research methods? B. Research Proposal 1. List and describe the chapters and subsections of a research proposal and a research report and their proper order. 2. Describe the characteristics of an appropriate proposal title. 3. Compare and contrast the styles appropriate for (a) a dissertations, (b) a research proposal, (c) a research report, (d) a professional paper, and (e) a journal article. C. Obtaining background information (including reporting related literature and research) 1. Describe a good note taking procedure. 2. Define a theory, a concept and a construct. 3. Describe the function of theories. 4. Compare facts and theory. 5. Use the IU library system. 6. Identify sources of related literature. 7. Make appropriate citations and bibliographic entries in a research study (APA style). D. Selecting and Defining the Problem 1. Describe sources of research problems. 2. Describe factors to consider in choosing a problem. 3. Describe the process of developing a problem statement. 4. List the characteristics of properly stated problems. 5. Define related research terminology. 6. List and define the types of independent variables. 7. Select independent and dependent variables from problem statements. 8. Write problem statements possessing the necessary characteristics. E. Data-Gathering Instruments 1. Construct the questionnaire 2. Pretest the questionnaire 3. Determine validity and reliability of the questionnaire. 4. Construct a cover letter. 5. Prepare first, second and third follow-up communications. 2
6. Differentiate between qualitative and quantitative approach to research data collection. F. Research Designs 1. List and define the types of research; identify studies by purpose 2. Describe characteristics of descriptive research. 3. Outline the steps in descriptive research. 4. Identify, describe and compare/contrast the pre-experimental, true experimental, and quasi-experimental designs. 5. Define internal and external validity. 6. For each threat to the internal and external validity of a study: a. Define the threat and give an example of how it could occur. b. Describe how to control each threat. c. Identify threats in studies. 7. Identify and describe correlational research 8. Identify and describe causal-comparative research 9. Identify and describe qualitative and historical research. 10. Select the most appropriate design for a study. G. Objectives and Hypotheses 1. Write research objectives for survey, ex post facto, and experimental research. 2. Evaluate objectives for appropriate content criteria. 3. Develop a hypothesis. 4. Describe the relationship of theory to hypothesis development. 5. Discuss the testing of hypotheses. 6. Write null and alternative hypotheses for statistical testing. H. Sampling 1. Describe the meaning, rationale, and steps involved in sampling. 2. Distinguish between probability and non-probability sampling. 3. List the characteristics, uses, and limitations of probability and non-probability sampling. 4. Determine sample size and select sample for their research. I. Descriptive Statistics 1. Identify the characteristics and limitations of four types of measurement scales--nominal, ordinal, interval, and ratio. 2. Organize research data into frequency distributions 3. Understand the measures of central tendency and calculate the mean, the median, the mode for any given data. 4. Understand the concepts of variability and compute variance and standard deviation. 5. Explain the concept of correlation coefficient between two sets of data. J. Data Analysis: 1. Enter and analyze quantitative data using SPSS. 2. Interpret findings using basic descriptive statistics. 3. Interpret basic relationships and differences. K. Inferential Statistics: 1. Distinguish between a sample and a population. 2. Explain the steps in hypothesis testing. 3. Describe a sampling distribution. 4. Explain the difference between point and interval estimation. 5. Explain the purpose of including effect size. 3
L. Reporting 1. Demonstrate the procedure to follow when preparing a research article for presentation at a defense, meeting or professional conference. 2. State the procedure to follow when preparing an article for a report or professional journal. Course Organization This is a traditional, face-to-face, classroom-based course that include lectures, classroom discussions, assigned readings, homework, tests, group interaction, student presentations. Required Readings. Most topics have one or more required readings, either a textbook chapter and/or article(s). By the end of the course you will have read most chapters of the text. Other required and suggested readings will be announced as we progress throughout the semester. Suggested Readings. Sometimes required readings appear (a) impenetrable, (b) too simplistic, or (c) fluff. Some of the required reading may provoke one of these reactions from you and if so, turn to the suggested readings for similar materials by different authors. Critiques. Students will critique several empirical research articles. This provides opportunity for students to demonstrate their mastery of analytical skills in speaking and/or writing. Homework. The number of homework assignments varies each semester, and this semester, will be limited primarily to statistical problems. Course Requirements and Grading Grades will be determined by performance on critique of empirical research article(s) (10 percent), homework assignment(s) (5 percent), Individual research porject (40 percent), and Quiz/test-like event occuring about three-fourths of the way through the semester: multiple choice, true/false, short answer/ short essay, matching test (30 percent). Letter grades will be assigned as follows: 100 90 percent = A 89 80 percent = B 79 70 percent = C Students are responsible for their individual preparation and participation. Students are expected to read articles prior to the class period (except for the first two classes) for which the pages are listed and be prepared to discuss. Preparation consists of reading the assigned material (and note-taking) and completing assignments prior to the class period for which they are listed. (The first week of class is the only exception). Participation is defined as contributing relevant comments to group and class discussions (i.e., answering questions posed either by other students or the instructor and withholding irrelevant comments), working on individual and group assignments, engaging in constructive criticism, assisting other class members, and demonstrating solutions to problems. Critiques may be formal, written products that demonstrate a thorough understanding of an assigned research report and its limitations. Other critiques may require you to complete an article 4
critique form, which, in effect, encourages a similar degree of analysis. Students may present critiques in class, with supporting overheads/handouts. Homework exercises are opportunities for you to demonstrate your understanding of certain concepts and mastery of data analytic techniques. Students conduct a small scale research project. The research project requires you to undertake a small research project first hand and develop a research report. The following steps indicate the general scope of the project: Feb 6: Submit via e-mail topic and hypothesis for instructor approval before undertaking subsequent steps. Feb 28: Conduct library research (literature review). Summarize and synthesize articles, pointing out the logical connections to your research topic, and analyze the quality of the studies. (Note: A useful method of presenting your analysis of the quality of studies is a table in which columns are attributes of comparison (e.g., purpose of study, sample size, sampling type, randomization, measurement method, etc), and each study is a row entry. Your paper should report on a minium of seven primary (i.e., empirical, firsthand) research reports). Mar 12: Collect data first hand data collection. You might observe, for example, the incidence of attention-seeking behavior in a kindergarten classroom and teacher/ other student responses; or develop a questionnaire to measure attitudes towards school choice; or observe the activities in a higher education unit; or conduct a content analysis of documents; or develop a unique method of measuring a phenomenon related to education. This short list is not exhaustive. Mar 19: Data analysis. Analysis of your data essential. How appropriate are the analytical techniques you have chosen? Results. What did you find? Does it confirm or fail to confirm your hypothesis? Conclusions. This is the so what? portion. What is the significance of your findings? What do your results suggest? You should also indicate an awareness of the limitations of your study and be aware that every study has limitations. April 2: (Optional) Send initial draft to instructor for feedback. The content of the paper will be graded according to the guidelines in Grading of written assignments. You will present your results to the class at one of the last two class sessions. Class members will offer comments on the content and presentation of the project. Grading of written assignments The grade is based on organization and completeness of the document; quality and originality of ideas; use of proper grammar, spelling, and syntax; ability to choose the relevant concepts or techniques and apply them correctly; and proper citation of the literature. Written assignments should be in courier 11 or 12 points, double spaced, and follow the conventional rules of grammar, punctuation, spelling, and notation of references. Use the Publication Manual of the American Psychological Association as your style guide. Be sure to use the margins and type size as specified. Always keep a copy of your written assignments for yourself. Assignments turned in late will lose points. For critiques, you will lose 2 points for each day late. Please note that all University policies regarding cheating and plagiarism will be strictly followed. Cheating and plagiarism are subject to grade lowering and/or other sanctions. This also applies to the preparation of research papers and projects. You may not submit simultaneously a research or term paper for credit in more than one class. Additionally, all papers must be your own (or your group s) original work. You may not use reproductions, work completed by someone else, or purchased work. 5
Table 1. Schedule Original Date Topics Textbook Readings 1 Jan 17 Introduction to educational research: Sources of knowledge. What is inquiry? What is empirical research? Scientific method. Research content and quality. Selecting a research topic. Literature search techniques 2 Jan 18 Writing a hypothesis (and proposal). Developing a research plan. Ethics in research. Theory, variables, hypotheses. Measuring variables. Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 18 3 Jan 31 Measurement. Reliability & validity. Other data collection techniques Chapter 5 Chapter 6 4 Feb 1 Sampling Threats to Internal and External Validity 5 Feb 14 Experimental Designs Quasi-Experimental Designs 6 Feb 15 Descriptive, Correlational, Causal-Comparative designs Qualitative Research 7 Feb 28 Historical Research / Content Analysis Mixed Methods Research Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Chapter 12 Chapter 13 Chapter 14 8 Feb 29 Descripitive Statistics Chapter 15 9 Mar 27 Inferential Statistics Chapter 16 10 Mar 28 Interential Statistics (continuted) 11 Apr 17 Qualitative data analysis Chapter 17 12 Apr 18 Quiz/test-like event 13 May 1 Evaluating a research report 14 May 2 Student Presentations of Research Additional instructions for the project and due dates for components of the project are forthcoming. 6
Table 2. Schedule Updated 04 Jan 27 Date Topics Textbook Readings 1 Jan 17 Introduction to educational research: Sources of knowledge. What is inquiry? What is empirical research? Scientific method. Research content and quality. Selecting a research topic. Literature search techniques 2 Jan 18 Writing a hypothesis (and proposal). Developing a research plan. Ethics in research. Theory, variables, hypotheses. Measuring variables. The Feb 29 meeting occurs Saturday evening, Feb 28, 6:00 to 8:00 pm. The Mar 28 meeting occurs Saturday evening, Mar 27, 6:00 to 8:00 pm. The May 2 meeting occurs Saturday evening, May 1, 6:00 to 8:00 pm. Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 18 3 Jan 31 Measurement. Reliability & validity. Other data collection techniques Chapter 5 Chapter 6 4 Feb 1 Sampling Threats to Internal and External Validity 5 Feb 14 Experimental Designs Quasi-Experimental Designs Critiques: Bruce, Casey, Mindy 6 Feb 15 Descriptive, Correlational, Causal-Comparative designs Qualitative Research 7 Feb 28 1-4:30 & 6-8 Historical Research / Content Analysis Mixed Methods Research Critiques: Jeff, Mark, Mark, Norah Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Chapter 12 Chapter 13 Chapter 14 8 Feb 29 Descripitive Statistics Chapter 15 (Feb 28) 9 Mar 27 Inferential Statistics Chapter 16 Critiques: Jeff, Jennifer, Mary, Shelly 10 Mar 28 Interential Statistics (continuted) (Mar 27) 11 Apr 17 Qualitative data analysis Chapter 17 12 Apr 18 Quiz/test-like event Critique: Rick 13 May 1 Evaluating a research report 14 May 2 (May 1) Student Presentations of Research 7