1 J381.1 Content Analysis Dr. Tom Johnson Spring 2011 Friday CMA 3.108 9-12 Office: CMA 5155 e- mail: tom.johnson@austin.utexas.edu Office hours: TTH 1:30-3, by appointment or when you least expect it REQUIRED READINGS Kimberly A. Neuendorf The Content Analysis Guidebook Thousand Oaks: Sage Publications 2002 The Content Analysis Guidebook Online http://academic.csuohio.edu/kneuendorf/content COURSE DESCRIPTION Content analysis is the systematic, objective and quantitative analysis of message characteristics. Content analysis is one of the fastest growing methods used in communication research. Content analysis is a systematic way to analyze the content of documented communications, whether they are written, audio/visual or digital. The emphasis of the course will be to explain the methodological steps involved in conducting content analysis so that you will be able to design and execute content analysis studies, whether they be traditional manual approaches or more recent computer- based techniques. GRADING The majority of the grades from the class will come directly or indirectly from group projects that we will conduct on Sarah Palin on Facebook and Twitter. The class will vote on specific topics and I anticipate that there will be two or three groups examining each topic. You will be creating the project from topic selection to coding the content. In order to systematically develop the project I will ask you to submit seven different memos (explained on a different sheet). The paper will also be submitted in five states 1) topic statement 2) introduction and problem statement 3) literature review and methods 4)
2 rough draft (optional) and 5) final paper. The paper is explained in more depth in a separate sheet. Research memos..100 points (33 percent) Introduction/problem statement.15 points Literature review/methods.35 points Final Draft of Term paper 100 points (50 percent total paper) Participation 50 points (17 percent) Participation will be based on how active you are in discussions, how active you have been in your group project as well as your participation on individual in class exercises. In addition, I have created a Facebook group, J381 Content Analysis: Count on it! as a place to post notices and material from class. It is also a place where you are free to post notices, or to use to get your research groups organized ORGANIZATION OF THE CLASS Listed below are the tentative topics to be discussed in the course. The order of discussion might change, as might due dates, but I will give you at least a week notice about due date changes. January 21 Introduction to the course and content analysis January 28 An overview of content analysis and its major decision points (Decision point 1: Topic selection) Readings: Neuendorf, Chapts. 1-3 Assignment: Research Memo 1: topic for class research project February 4 Message units and sampling Readings: Neuendorf, Chapt. 4 Assignment: Research Memo 2: Topic for your research paper
3 February 11 Message Units and Sampling (cont) Decision point 2: Selection of messages Assignment: Research Memo 3: What issues will be analyzed over what period of time? Laboratory: discuss what issues will be measured, over what period of time, and whether to do census or sample? February 18 Variables and Predictions Decision point 3: Defining content categories Readings: Neuendorf, Chapt. 5 Assignment: Submit the introduction and problem statement for your term paper Laboratory: Conceptualize variables and develop hypotheses and research questions February 25 Variables and Predictions Assignment: Research Memo 4: Conceptualize key variables from your study and develop hypotheses and research questions March 4 Decision point 5: Operationalizing content categories Measurement Techniques Readings: Neuendorf, Chapt. 6 March 11 Measurement Techniques Assignment: Research Memo 5: Operationalize key variables from your study and revisit hypotheses and research questions Laboratory: Develop codebook for individual studies. March 18 SPRING BREAK!!
4 March 25 Decision Point 5: Establishing Reliability Readings: Neuendorf, Chapt. 7 Assignment: Research Memo 6: Pilot study where you check agreement on key variables Laboratory: Revise codebook based on pilot study April 1 Establishing reliability Laboratory: coding content and checking reliability of document April 8 Results and Reporting Decision Point 6: Data analysis Readings: Neuendorf, Chapt. 8 Assignment: Literature Review and methods sections due Laboratory: Data collection and coding April 15 Laboratory: Data Analysis Assignment: Research Memo 7 Checking Intercoder reliability on measures April 22 Laboratory: Data analysis Assignment: Rough Draft of paper due (optional) April 29 Laboratory: Data analysis May 6 Poster Session previewing final research paper Final Exam: Wednesday, May 11, 9:00- noon Final Paper due by noon, May 11
Attendance This is a course where you will learn about how to conduct content analysis. Also quite a bit of class time will be devoted to working on the class research project as well as in- class assignments. Attendance, therefore, is vital. If you cannot make it to class, please contact me in advance to get an excused absence. If you have more than two unexcused absences, I will reduce your score by half a letter grade. Religious holy days A student who misses classes or other required activities, including examinations, for the observance of a religious holy day should inform the instructor as far in advance of the absence as possible, so that arrangements can be made to complete an assignment within a reasonable time after the absence. The Texas Education Code specifies that an institution of higher education shall excuse a student from attending classes or other required activities, including examinations, for the observance of a religious holy day, including travel for that purpose. A student whose absence is excused under this subsection may not be penalized for that absence and shall be allowed to take an examination or complete an assignment from which the student is excused within a reasonable time after the absence. Absence for military service In accordance with section 51.9111 of the Texas Education Code, a student is excused from attending classes or engaging in other required activities, including exams, if he or she is called to active military service of a reasonably brief duration. The maximum time for which the student may be excused has been defined by the Texas Higher Education Coordinating Board as "no more than 25 percent of the total number of class meetings or the contact hour equivalent (not including the final examination period) for the specific course or courses in which the student is currently enrolled at the beginning of the period of active military service." The student will be allowed a reasonable time after the absence to complete assignments and take exams. Policies 5
6 affecting students who withdraw for military service are given below. <http://www.utexas.edu/student/registrar/catalogs/gi03-04/ch4/ch4g.html#attendance> Students with Disabilities Please notify your instructor of any modification/adaptation you may require to accommodate a disability- related need. You will be requested to provide documentation to the Dean of Student's Office in order that the most appropriate accommodations can be determined. Specialized services are available on campus through Services for Students with Disabilities: http://www.utexas.edu/diversity/ddce/ssd/ University of Texas Honor Code The core values of The University of Texas at Austin are learning, discovery, freedom, leadership, individual opportunity, and responsibility. Each member of the university is expected to uphold these values through integrity, honesty, trust, fairness, and respect toward peers and community. Policy on Scholastic Dishonesty The University defines academic dishonesty as cheating, plagiarism, unauthorized collaboration, falsifying academic records, and any act designed to avoid participating honestly in the learning process. Scholastic dishonesty also includes, but is not limited to, providing false or misleading information to receive a postponement or an extension on a test, quiz, or other assignment, and submission of essentially the same written assignment for two courses without the prior permission of the instructor. By accepting this syllabus, you have agreed to these guidelines and must adhere to them. Scholastic dishonesty damages both the student's learning experience and readiness for the future demands of a work- career. Students who violate University rules on scholastic dishonesty are subject to disciplinary penalties,
7 including the possibility of failure in the course and/or dismissal from the University. http://deanofstudents.utexas.edu/sjs/acint_student.php. For the University's official definition of scholastic dishonesty, see Section 11-802, Institutional Rules on Student Services and Activities. http://registrar.utexas.edu/catalogs/gi08-09/app/gi08.appc03.html#sec- 11-802- scholastic- dishonesty19 Additional recommended resources: Gottschalk, L. A., & Bechtel, R. J. (Eds.). (2008). Computerized content analysis of speech and verbal texts and its many applications. New York: Nova Science Publishers, Inc. Hak, T., & Bernts. (1996). Coder training: Theoretical training or practical socialization? Qualitative Sociology, 19, 235-257. Hester, J.B., & Dougall, E. (2007). The efficiency of constructed week sampling for content analysis of online news. Journalism and Mass Communication Quarterly, 84, 811-824 Janis, I. (1965). The problem of validating content analysis. In H.D. Lasswell, N. Leites, & Associates (Eds.), Language of politics (pp. 55-82). Cambridge: MIT Press Jordan, A. B., Kunkel, D., Manganello, J., & Fishbein, M. (Eds.). (2009). Media messages and public health: A decisions approach to content analysis. New York: Routledge. Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage. Krippendorff, K. (2004). Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research, 30, 411-433. Krippendorff, K., & Bock, M. A. (Eds.). (2009). The content analysis reader. Thousand Oaks, CA: Sage.
8 McMillan, S.J. (2000). The microscope and the moving target: The challenge of applying content analysis to the World Wide Web. Journalism & Mass Communication Quarterly, 77, 80-98. Matthes, J. (2009). What s in a frame? A content analysis of media framing studies in the world s leading communication journals, 1990-2005. Journalism & Mass Communication Quarterly, 86, 349-367. Matthes, J., & Kohring, M. (2008). The content analysis of media frames: Toward improving reliability and validity. Journal of Communication, 58, 258-27. Neuendorf, K. A. (2009). Reliability for content analysis. In A. B. Jordan, D. Kunkel, J. Manganello, & M. Fishbein (Eds.), Media messages and public health: A decisions approach to content analyis (pp. 67-87). New York: Routledge. Neuendorf, K. A., & Skalski, P. D. (2010, May). Extending the utility of content analysis via the scientific method. Paper presented to the Social Science and Social Computing Workshop, University of Hawaii, Honolulu, HI. Riffe, D., Aust, C.F., & Lacy, S.R. (1993). The effectiveness of random, consecutive day and constructed week sampling in newspaper content analysis. Journalism & Mass Communication Quarterly, 70, 133-139. Riffe, D., Lacy, S., & Fico, F. G. (2005). Analyzing media messages: Using quantitative content analysis in research (2nd ed.). Mahwah, NJ: Lawrence Erlbaum. Roberts, C. W. (Ed.). (1997). Text analysis for the social sciences: Methods for drawing statistical inferences from texts and transcripts. Mahwah, NJ: Lawrence Erlbaum. Shapiro, G. (1997). The future of coders: Human Judgments in a world of sophisticated software. In C.W. Roberts (Ed.), Text analysis for the social science: Methods for drawing statistical inferences from texts and transcripts (pp. 225-235). Mahwah, NJ: Lawrence Erlbaum. Smith, C. P. (Ed.). (1992). Motivation and personality: Handbook of thematic content analysis. New York: Cambridge University Press. Weber, R.P. (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage. West, M. D. (Ed.). (2001). Applications of computer content analysis. Westport, CT: Ablex.
9 Research Memos Our general research topic for this semester will be Sarah Palin on Facebook and Twitter. This is obviously too broad of a topic. Through the research memos and lab sessions in class (and outside) we will develop a more specific research topic or topics (we will likely have two or three research groups) and you will decide exactly what you would like to examine. Everyone will help decide what media we will look at over what period of time and conceptualize and operationalize key variables. To ensure that we have a reliable measure, everyone will help with a pilot study and check intercoder reliability. All the research memos will be due the day before class by 4:30. Here is a description of the memos: Research Memo 1 (due Jan 27) 10 points: Everyone will submit a topic idea for our class study dealing with the general topic of Sarah Palin and Facebook and Twitter. Be as specific as possible. For instance Sarah Palin using Facebook and Twitter in the health care debate is pretty vague. But the degree to which leading liberal and conservative blogs discuss Palin s claims that health care will lead to death panels, how the blogs framed it and whether coverage supported the death panel claim or challenged it is a more specific topic that can be examined. The topic (or likely topics) voted on by the class will receive 5 bonus points. Research Memo 2 (due Feb. 3) 10 points: Each of you is expected to do a portion of the content analysis for this topic that will lead to a paper that you can present at a conference and get published. For instance, if we were looking at how Sarah Palin promoted her daughter Bristol on Dancing with the Stars, you might decide to look at photos and videos. Or if we were looking at blog and media reaction to a Palin Facebook update or updates you could choose specific blogs to analyze. Be as specific as you can about what your paper will examine. Give the specific topic, variables you might address, media you might want to look at. Research Memo 3 (due Feb. 10) 10 points: This is where you will flesh out your memo a bit from last week discussing specific issues you will examine in what media and over what period of time. In class we will discuss more broadly the issues the study will entail, the time period, media used, and sampling decisions (census or sampling, hand coding vs. computer coding).
10 Research Memo 4 (due Feb 24) 10 points: This is where you conceptualize key variables (three to five) in your study. It is best if you rely on past studies to come up with measures to improve validity of the measures. Also develop preliminary research questions and/or hypotheses. As a group we will discuss how to conceptualize the variables and specific questions we will address. If we adopt your conceptualization scheme you will get 5 bonus points. Research Memo 5 (Due March 10) 10 points: This is where you operationalize key variables (three to five) in your study. It is best if you rely on past studies to come up with specific measures to improve validity of the measures. Also develop refined research questions and/or hypotheses. As a group we will discuss how to operationalize the variables. If we adopt your operational scheme you will get 5 bonus points Research Memo 6 (March 24) 25 points: This is where we will test the coding books through a pilot study. We will look at a selection of content and each person will code variables of interest from someone from the other study. The pilot study will help us determine if the coding book are clear enough in explaining how to code variables. Also, we can tell if the content categories are mutually exclusive and exhaustive and if there are problems distinguishing between values in the variables. You will actually be doing two codings this week. You will code three to five of your own variables and three to five of the variables in the other studies. Research Memo 7 (April 21) 25 points: This is actually an extension of the pilot study, coding 5 to 10 percent of the content for three to five variables of interest to you and to someone else in the other group. We will check intercoder reliability. If it is not adequate we can hopefully rectify it by say combining categories. But if not, we may have to go back and recode the content.
11 J 381.1 Content analysis Term-paper assignment Spring 2011 Each of you will write a 15-25 page research paper based on your portion of the content analysis project. The paper will be submitted in five stages 1) topic selection 2) introduction and problem statement 3) literature review and methods 4) rough draft (optional) 5) final draft 1. Topic: This is actually research memo 2. Please give me a brief (one page or less) description of what the focus of your term paper will be. Be as specific as you can about what your paper will examine. Give the specific topic, variables you might address, media you might want to look at. The topic is due February 3th by 4:30 p.m. 2. Introduction and problem statement: The introduction typically is 1-3 paragraphs that introduces the topic you will explore and tries to do so in an engaging manner. The problem statement explains what is missing from the literature, and how your work will address that hole in the literature. Then you explain what your paper will examine. This is like a thesis statement that clearly sets out what you plan to do. The introduction and problem statement is due Feb. 18th by 4:30. 3. Literature Review and methods section: The literature review, as the name implies, reviews the relevant literature linked to your specific topic. The literature review ends with hypotheses or research questions. I think it is often better to begin with the hypotheses or research questions, then write the literature review based on the idea of what does the reader need to understand to put the hypotheses and research questions in context. The literature review and methods is due Friday, April 8 by 4:30. 5. The rough draft of your paper with results and discussion (and ideally bibliography) is not required. However, if you are thinking of submitting your paper to ICA, you are STRONGLY URGED to do a rough draft. The rough draft is due Friday, April 22 by 4:30 p.m.
6. Your final journal- length term paper. The paper should be typed and double spaced. It should also include footnotes/endnotes and a bibliography. Any of the standard footnote and bibliographical styles is acceptable. My suggestion is that you look at the journal you hope to send it to and adopt that style The final draft of the paper is due Friday, May 11 by noon 12