2 Ethnography in Design Qualitative Research in Interaction Design and UX http://vimeo.com/6038262 Mattias Arvola Department of Computer and Information Science 3 4 Research for design Good design requires insight Insight requires research Empathy The ability to take someone else's perspective Get an empirical benchmark that everybody agree on, so that you after the design can show impact Do it yourself as a designer By Bill Verplank. 5 6 Kinds of Scientific Research Basic research Applied research Contract research Scientific Research Systematic search for knowledge The choice of method depends on what the research question is New knowledge Replication Systematic and critical review of previous research
Scientific Research vs. Research for Design 7 8 Scientific research is: Statement by statement, verified by carefully described observations, or references to such Criticisable and verifiable Reproducible aimed to distinguish more likely conclusions from less so As objective and without local and temporary prejudices as possible Cumulative For common good Research for design is: Good enough based on resources Check that you are on the right track Find a baseline to show that you have made improvements Inspiration Find out what you need to know to move on or get unstuck Quality and trustworthiness Adds new original knowledge Clear conditions and starting point Purpose & questions Methods explained in relation to questions Consideration of method alternatives Systematic and critical analysis of carefully gathered data Possible sources of errors and limitations identified and discussed Arguments clearly articulated, and relevant for conclusions Not over-generalise 9 10 Ethic considerations at the start of a project Choose the method with the least harmful consequences for people, if otherwise equal Expected benefits and potential scientific value of the research should should be weighed against harmful consequences Benefits and harm for whom? Openness is the lodestar Make dependencies and interests clear Ethics Completely voluntary participation Informed consent: verbal or written Explain risks and possibilities of participating Respect participants integrity and privacy Confidentiality: coding and masking What do participants get out of it? Let the participants read/see your results 11 12 Informed Consent Awareness of researcher s identity and purpose In general announcements inform about purpose, research topic, and data gathering methods Seek permission from gatekeepers Make sure that there is no pressure to participate If possible, get informed consent from every individual Video recording Can intrude on private life and integrity, since people can be identified Use only when the same data cannot be obtained by other methods (eg. masked still images) Respectful and responsible recording If minors (<15) are recorded: Consent from both legal guardians Informed so that the child understands Inform: What will be analysed in the video?
13 14 Ethics in Participant Observation Long term immersion in everyday activities of the people being studied to record conduct under the widest range of possible settings Information is the result of dialogical interaction between the researcher and the informants: any interaction may constitute some form of data gathering Informed consent process should be dynamic and continuous Unfamiliar settings can be sensitive, and have clashes in ethical principles Data collection Take notes in a hardback notebook Preferably two persons Specifically if you are a novice Sound recording is important Look out for tech trouble Take photos Memory aid and details about the context of use 15 16 In practice Overarching process 1. Kickoff meeting 2. Stakeholder interviews 3. Competitive analysis and literature study 4. User interviews and observations 17 18 Stakeholders People in the organisation that: Sponsor, build, test, market, sell, and support the product Plus: Other people that affect the design of the product Kickoff meeting Gather project members and stakeholders Discuss What is the product? Who will use it? What do the users need? What customers and users are the most important? What challenges do we face?
19 20 Stakeholder interviews General questions What is your role in relation to this product? What did you do before you had this role? What should this product or service be? Whom is it for? When should the version we are designing for be released? Why is it important that it is released then? What do you worry about in this project? What is the worst thing that could happen? Stakeholder interviews Cont. What is effect should this project have one the business or organisation? How would you, personally, define success for this project? Is there anyone we should talk to that isn t on this list? Who are they? How would you like to be involved in this project, and how should we contact you? 21 22 Stakeholder interviews Marketing: Who are your customers and users today, and how would you like it to be in five years? How does this product fit into the overarching product strategy? Who are you biggest competitors and what is it about them that worry you? Stakeholder interviews Cont. What 3-4 qualities would you like for people to associate with your company and your product? What is the current status of your identity? Could we have a copy of your graphical profile if there is one and examples of how it has been applied? 23 24 User interviews and observations Engineering questions What technology decisions have already been made? What are the reasons for them? How many engineers and other tech staff are allocated to this project, and what are their competencies? Can you draw a diagram and broadly explain how the existing system works? Try to understand what is done, why, what goals there are, what mental models people have, how they feel about things Search for similarities and differences between people, what different goals they have and if (how) they solve the same task in different ways Interview people in different roles Make sure you have a protocol
25 Ask the users what they want? 30 Ask the users what they want? Compare to the physician who asks: What medicin would you like to have? Or the architect that just make the finished drawings from the clients sketches?
General questions Product or service Business-oriented email system Digital camera for hobby use Web site for camera business Service for helping students find relevant summer jobs Bad general questions What email system do you use? What kind of camera do you have? Good general questions We are interested in what kind of communication you have with other people during a typical day at work. Could you start by going through your work day yesterday? What role does photography play in your life? What do you usually take photos of and how often? How do you use our web site? 1. Tell me about the last time you bought a camera. 2. Tell me about some occasion when you after after buying a camera felt that you have needed extra info or help. Have you had a job this summer? What kind of job have you had? this summer? 31 Interview themes Objects Find important nouns Objects from the mental model I have seen you work with three types of forms so far: travel order, travel expenses, and purchase order. What other kinds of forms do you use and why? Attributes of the objects What information does a travel order contain? How many fields are there in a travel order? Relations between objects What is the difference between a travel order and a travel expense form? Quantity How many travel orders do you usually handle every week? 32 33 34 Actions Find verbs So every time you get a travel order, you first take a copy for archiving before you contact the owner for to verify. The reason for an action Why should the travel order be archived? How is an action performed and described? The administrator looks at a post-it note with codes for accounts when entering the travel order into the accounting system. Frequency and priority Do you ever wait with archiving a travel order? The photocopier is not warm early in the mornings, so it takes forever to take a copy for archiving, so I usually wait with all the copying until later in the day. There s usually no hurry anyway. The role of the present product I have to enter all information manually into the the accounting system. since travel orders are handed in on paper. Relations to other people: The cost owner The one who hands in the travel order Archivist Workarounds Do you ever not archive a travel order? There is this professor who always changes his mind in the last second. I usually don t archive his travel orders until he actually have left. I would otherwise have to retrieve them every time he changes his mind. 35 36 Frustrations Sighs, facial expressions, body language, gestures. etc. It s a drag with travel orders since you have to feed them into a PDF, but you cannot save the file, just print it, so if you afterwards realise that you ve done something wrong you have to start all over. Skills and experience How long have you used this product? You seem to know quite a bit about this technology. What did you do before starting here? Study how the interviewee expresses things. Choice of terminology, etc. Goals What characterises a good experience at work? When someone recognises that you ve done a good job. Why is someone doing an action? I take photos since she grows up so fast. Now that she is two, I can hardly remember what she was like when she was one. Track changes I want to take a photo that communicates how amazing it is when I see this really striking landscape. Capture an experience of something When I design a garden, I take photos of what it looks like for future reference for the design work. When the new design is done I take photos of the result for my portfolio to show people that I can do a good work. Support work and sales
37 Ethnographic Observations
FreeImages.com/kuki rosen 43 UX Research Involves Interpretation 44 Understanding UX Requires Qualitative Research FreeImages.com/Julia Freeman-Woolpert The Hermeneutical Circle 45 46 Characteristics of Qualitative Research Whole No sharp boundaries of study objects Careful contextualisation of the instance Transferability rather than generalisability Cooperation between researcher and participants Purposeful sampling Alternative interpretations Suspicion of interpretations Going beyond what is said and done to what is intended and motivated FreeImages.com/francesco fattore Parts 47 48 Measures for Trustworthyness in Qualitative Research Prolonged engagement and observation Building trust Learning the culture Checking misinformation and distortions Finding the focus Triangulation Researcher Theory Method Peer review, debriefing and external audits Clarifying what subjective things that have likely shaped the the interpretation past experiences, biases prejudices orientations Member checking Rich and thick description Revise working hypothesis Negative cases with disconfirming evidence Data Analysis by Affinity Diagramming A special case of thematic analysis conducted in group 1.All workshop participants read the notes, mark important passages, and notes codes in the margin 2.Every person reads their markings aloud and write them down on a sticky note (code notes on origin and person) 3.Put notes on wall or table and move them around so that related notes are placed together 4.Discuss what binds the groups of notes together 5.Name the groups: you now have themes/categories
52 A Qualitative and Interpretative Stance What is surprising here? What did we expect to see that did not appear? What does the data tell us that we didn t already know? What in the data makes us see things differently? Why does a pattern or anomaly arise? What theoretical concepts can we use to make sense of our data? 53 54 Develop UX Themes Form Statements about: What do people do? What do people know? How do they do it? What do their actions lead to? What is their frame of reference? What habits of mind does it lead to? What do people feel? How are things experienced? What do their experiences mean for them? Order them according to their affinity to form themes What is similar across participants? What is different? Why? By Bill Verplank.
55 In summary There are differences between scientific research and research for design Ethics including informed consent is critical Understanding UX requires qualitative research into the interactions, mental models, and experiences people have Qualitative research methods includes both observations and interviews Analysis and interpretation into UX themes can be made with affinitiy diagrams @mattiasarvola www.liu.se