Usability Testing Plan

Transcription

1 Usability Testing Plan Client: Gramercy Consultants Product: Art My Kid Made (ios App) Product Description: Art My Kid Made is an ios application currently available at no cost at the App Store. This application is designed to allow parents to photograph, archive and share their children s artwork. Kid Art STUDY OBJECTIVE The purpose of this test is to determine the overall usability of the app, as users complete the following core tasks: 1. Login, Logout, and Initial Account Set Up. 2. Taking photos and editing them. 3. Saving/archiving photos. 4. Sharing photos. Additionally, the client would like to determine if a similar but paid, professional service offering professional photography service and additional archiving options would be attractive to users. Main Questions: 1. Login/Setup: a. Is the login and initial account/child set up considered to be fast and easy to accomplish? b. Are there any confusing aspects to the initial account set up? c. Is the navigation system understandable and logical to the user (incl. clarity and hierarchy)? d. Is the logout process easy to find, and can the user easily log back into the system? 2. Photography: a. Can users take or access previous photographs easily? b. Are the editing tools sufficient to the user? c. Are any of the editing tools unnecessary or unattractive to users? d. How does the user feel about editing their child s original work? (ie. necessity, ethics) 3. Storage: a. Is the process of saving/archiving images clear to the user? b. Once saved, can the user determine how to view them / easily locate them? c. Are the storage location options sufficient for the user? d. Are there any additional storage locations that the user would prefer? KWS Market Research, Inc. Page 1

2 4. Sharing: a. Is the process of how to share images clear to the user? b. Are the sharing options sufficient and appropriate for the target audience? (If not, can the user suggest other desired alternatives?) c. Do users experience any confusion, error, or difficulty with the sharing process? 5. General: (Only to be studied if time permits) a. Are the secondary features useful and attractive to the user? (ie. favorites, rss, fridge) b. Is the design of the site (imagery, colors) appealing to the user? c. Can they easily recommend this site to a friend? Note: Additional questions regarding the paid service can be found in the post-test questionnaire. Participant Requirements: A total of 3 participants will be recruited from amongst friends and family and must meet the following characteristics: Female parent of at least one elementary school-aged child Between the ages of yrs. old Owns an ios device (Apple iphone or ipad) Uses native apps on their phone at least once per day Have never used this app before (but may have used a similar app before) Member of one or more Social Networking KWS sites Market (ie. Facebook, Research, Inc. Twitter, Instagram, etc.) Currently displays and/or stores their children s artwork for any amount of time Note: Questionnaire will not include demographic information since the participants are personally known to the moderator and were chosen based on the requirements. Location & Pre-Test Requirements: All tests will be conducted in the individual residences of the participants as this will best simulate the actual use of the product, and will also be the most logical way to access the artwork to be photographed (in the event a piece is difficult to transport due to size, shape or construction). Prior to the test, participants will be asked to provide a quiet environment - free of distractions, and a well-lit table, clear of debris on which to conduct the test. In addition, they will be asked to collect 3 pieces of artwork from each of their children in order to expedite the test. Tests will be conducted using the participant s own ios device and the observer will request the participant to download the app at the beginning of the test, or with permission, will do so for the participant as outlined in the consent form. Observer will sit next to the participant, directing, asking questions and taking occasional notes. Photographs or videos may also be taken by the obsersver during the test if deemed useful or necessary but will be minimal. KWS Market Research, Inc. Page 2

3 USABILITY TASKS: Task Ratings: 3 = User successfully completes task with no errors. 2 = User completes task with minimal error. 1 = User completes task with higher than expected amount of error. 0 = User fails to complete task. Task 1: Ask user to open the app, create a login/password and then login. Q. Does the login appear be fast and easy to accomplish? Circle ( ) Task 2: Ask user to set up account preferences to provide locations for the images for at least 2 children (either fictional or real). Q. Can user determine how to add a child? ( ) Q. Add addtional children? ( ) Task 3: Ask user to take photographs (2 or more) of artwork for Child 1, edit as desired and SAVE. Q. Can user determine how to take a photograph? ( ) Q. Can user add image details and edit them without error? ( ) Q. Can user SAVE image without sharing? ( ) Q. What improvements, if any, would the user recommend for this feature? Task 4: Ask user to take photographs (2 or more) of artwork for Child 2, edit as desired and SHARE one (or, if user is uncomfortable posting to their personal social media account, just pretend to share without actually completing the post). Q. Can user SHARE image without error or confusion? ( ) Q. What improvements, if any, would the user recommend for this feature? Q. Is the selection of sharing options sufficient? Can the user suggest others they would desire? Task 5: Ask user to view ALL of the images taken thus far of BOTH children s artwork. Then, ask them to view only the artwork for Child #1, followed by Child #2. Q. Does user know how to view a gallery? ( ) Q. Does the user understand how to switch between views? ( ) Q. Are there any download or other issues with this process? KWS Market Research, Inc. Page 3

4 Task 6: Ask user to to delete one of the images they have taken. Q. Does user know how to remove an image? ( ) Q. What steps does the user take in order to locate this feature? Q. What improvements, if any, would the user recommend for this feature? Task 7: Ask user to logout of the application, then log back in, pretending they have forgotten their password. Q. Can user find logout easily? ( ) Q. Can user recover a lost password easily? ( ) Task 8: Ask user to pretend that they are recommending this app to a friend. Ask them to explain how they would go about helping this friend find the app. Q. Does the user encounter any problems or errors with this task? ( ) Q. How does user undertake this task? General Exploration: Q. Does the navigation appear to be logical? (ie, Know what represents the main page)? Q. What features do users gravitate towards? Q. What issues or problems do they consistently encounter if any? Q. Are the secondary features attractive and deemed useful to the user (fridge, artist of the day, rss)? Q. Is the visual design of the site (imagery, colors) appealling to and appropriate for the user? Q. Does the user understand the implications of each sharing method? (ie. privacy) KWS Market Research, Inc. Page 4

5 Schedule: An informal pilot test was conducted on 06/12/13 with a highly structured, and ordered set of tasks for the user. This test revealed that a user s natural exploration was stifled by the directed tasks making it difficult to determine organic pathways, therefore the test was reformatted to allow for a more natural task flow. Tests will be conducted during the week of 06/17/13 on days and times most convenient for the individual participants. Each testing session is expected to take minutes. No formal compensation will be presented as the participants will be drawn from a pool of willing friends and family (though the observer may offer to buy the participant a cup of coffee at a later date and time, or bring one to the test). Agenda: A. Greeting - ensure pre-test requirements and conditions have been met at the location. B. Have consent form signed while setting up equipment. C. Moderator Guide Introduction. D. Begin app download. E. Begin usability tasks. F. Post-test questionnaire. G. Q&A / Debrief RESULTS Final Report: A Usability Test Report will be provided at the conclusion of the usability test. It will consist of a report and/or a presentation of the results and will attempt to answer the questions posed at the beginning of the study. Details of the partipant s subjective findings will be included as needed to demonstrate specific usability problems, and/or recommendations for improvements to the software. The anticipated delivery date for the report is 06/27/13 by 5:00 pm. KWS Market Research, Inc. Page 5

6 Post-Test Interview I would like to ask you some questions about the app that you just used. 1. Was the application something that you think you would use? Not at all Not very likely Unsure Probably would Definitely would 2. Do you feel that you would have time to use this app on a regular basis? Not at all Not very likely Unsure Probably would Definitely would 3. On a scale of 1-10, how likely would you be to recommend this application to a friend? Not at all likely Neutral Extremely likely 4. What are some things that you liked about the app in terms of features, design or function? 5. What are some features or functions that you feel need improvement? 6. Are there any features that you wish the app had? Now I would like to ask you some questions about your children s artwork. 7. Once your children s artwork has been displayed for a short time, do you tend to...? O Recycle every piece and store nothing. O Store only the favorites and recycle the rest. O Store every single piece. 8. Please briefly describe the type of storage container you use for your children s artwork, if any. KWS Market Research, Inc. Post-Test Interview Page 1 of 2

7 9. Do you currently keep any kind of digital archive of your children s artwork? O I take occasional photos of a few pieces. O I take regular photos of every piece. O I do not photograph my children s artwork at all. 10. How interested are you in creating a permanent archive of your children s artwork? Not at all Not very Unsure Somewhat Very interested interested interested interested 11. How likely are you to share images of your child s artwork with friends or family? Not at all likely Not very likely Unsure Somewhat likely Extremely likely 12. If you were NOT using this app and wanted to capture images of your children s artwork, would you be more likely to take quick photos of it on a regular basis, or would you rather take all the photos at one time, say, at the end of the school year? O Regular basis O All at once O Unsure 13. How interested would you be in a paid service that would take all of your children s art for an entire school year and photograph it for you? Not at all Not very Unsure Somewhat Very interested interested interested interested 14. If you were to use the paid service, what type of format would you like to receive the final images in? (Select all that apply) O Compact Disc of raw images O Digital Download (retrieve images from a server) O Digital Slideshow O Traditional Prints O Printed Photo Album O A Portfolio Website for your child(ren) 15. How do you currently store or archive personal digital images? For office use only: Session: No.: Date: KWS Market Research, Inc. Post-Test Interview Page 2 of 2

8 Usability Test Consent Form Please read this form carefully and in it s entirety, and sign it prior to beginning the test. In this usability test: You will be asked to download a free app onto your personal ios device which may be deleted or kept once the test is complete. will You be asked to perform certain tasks on the app using your personal ios device. will You be asked to share images that you take using your social network account. will You be verbally directed, and an interview will be conducted with you. will You also be asked to in a questionnaire. Your hands and device may be photographed or video recorded during the evaluation but no login or password information will be captured. Participation in this usability study is voluntary. All information will remain strictly The descriptions and may be used to help improve the application. However, at no time will your name or any other be used. You can withdraw your consent to the experi - ment and stop participation at any time. If you have any questions after today, please contact Kate Scott at I agree to participate in the study conducted and videotaped by Kate Scott. I have read and under - stood the information on this form and had all of my questions answered. I understand and consent to the use and release of the video and images by Kate Scott. I under - stand that the information, imagery and videot is for research purposes only and that my name and image will not be used for any other purpose. I relinquish any rights to the images and video footage and understand they may be copied and used by Kate Scott without further permission. I understand that I can leave at any time. I agree to immediately raise any concerns or areas of discomfort with the study administrator. Participant s Signature Date Usability Consultant Date For office use only: Session: No.: Date: KWS Market Research, Inc.

9 Moderator Notes Agenda: A. Greeting - ensure pre-test requirements and conditions have been met at the location. B. Have consent form signed while setting up equipment. C. Moderator Guide Introduction. D. Request app download. E. Begin usability tasks. F. Post-test questionnaire. G. Q&A / Debrief Introduction: (participant will already know that it s an art related app + it s estimated time) A. First, I want to thank you for agreeing to assist me with this evaluation. I really appreciate your time and your opinions. The app you will be evaluating today is an app for parents that allows you to photograph your children s artwork, store and share it. It s called Art My Kid Made. B. This app was not designed by me, so please be completely honest about your experiences with it s use. You will not hurt my feelings in any way. C. Even though this is formally known as a usability test, you are not actually being tested - the product is. You cannot do anything wrong, and there are no incorrect actions that you can take. If something goes wrong - it is not your fault. D. During this test, I will be verbally giving you some basic tasks that I want you to carry out with the app. As you carry out the actions that I request, I would like you to think out loud in a play-byplay type of format, verbally describing your actions as you go along. Let me know what you plan to do (ex. I am going to attempt to login now... ), Let me know what you are selecting (ex. I am going to try and click on the home button. ), Let me know what results you expect to see when you make your selections (ex. I think this will take me to the home page. ). If you see what you expected, let me know. If you have an unexpected result, let me know that as well. If there is something that you like about the design or functionality, please point it out. And conversely, if there is something that you dislike about the way the app looks or functions, please indicate that as well. After you complete each task, wait for further instruction before you make another selection. E. If you have any questions during your assessment, please feel free to pose them, but just know that I may not be able to answer them. (Though I will be recording them as these inquiries are vital to uncovering potential usability issues.) F. During the test, I may jot down occasional notes, take photos or videos (which will never include your face). Please know that these will be used for educational purposes only. G. Before we begin the download and test, do you have any questions? (Search for the app for the participant, but then allow them privacy to enter their password. Then go back to app store to delete name that will still be saved in memory.) KWS Market Research, Inc. Page 1

10 USABILITY TASKS: (Begin after the app has been successfully downloaded.) Task Ratings: 3 = User successfully completes task with no errors. 2 = User completes task with minimal error. 1 = User completes task with higher than expected amount of error. 0 = User fails to complete task. Note: Record quotations from participants for use in final report! Task 1: Ask user to open the app, create a login/password and then login. Q. Does the login appear be fast and easy to accomplish? Circle ( ) Task 2: Ask user to set up account preferences to provide locations for the images for at least 2 children (either fictional or real). Q. Can user determine how to add a child? ( ) Q. Add addtional children? ( ) Task 3: Ask user to take photographs (2 or more) of artwork for Child 1, edit as desired and SAVE. Q. Can user determine how to take a photograph? ( ) Q. Can user add image details and edit them without error? ( ) Q. Can user SAVE image without sharing? ( ) Q. What improvements, if any, would the user recommend for this feature? Task 4: Ask user to take photographs (2 or more) of artwork for Child 2, edit as desired and SHARE one (or, if user is uncomfortable posting to their personal social media account, just pretend to share without actually completing the post). Q. Can user SHARE image without error or confusion? ( ) KWS Market Research, Inc. Page 2

11 Q. What improvements, if any, would the user recommend for this feature? Q. Is the selection of sharing options sufficient? Can the user suggest others they would desire? Task 5: Ask user to view ALL of the images taken thus far of BOTH children s artwork. Then, ask them to view only the artwork for Child #1, followed by Child #2. Q. Does user know how to view a gallery? ( ) Q. Does the user understand how to switch between views? ( ) Q. Are there any download or other issues with this process? Task 6: Ask user to to delete one of the images they have taken. Q. Does user know how to remove an image? ( ) Q. What steps does the user take in order to locate this feature? Q. What improvements, if any, would the user recommend for this feature? Task 7: Ask user to logout of the application, then log back in, pretending they have forgotten their password. Q. Can user find logout easily? ( ) Q. Can user recover a lost password easily? ( ) KWS Market Research, Inc. Page 3

12 Task 8: Ask user to pretend that they are recommending this app to a friend. Ask them to explain how they would go about helping this friend find the app. Q. Does the user encounter any problems or errors with this task? ( ) Q. How does user undertake this task? General Exploration: Q. Does the navigation appear to be logical? (ie, Know what represents the main page)? Q. What features do users gravitate towards? Q. What issues or problems do they consistently encounter if any? Q. Are the secondary features attractive and deemed useful to the user (fridge, artist of the day, rss)? Q. Is the visual design of the site (imagery, colors) appealling to and appropriate for the user? Q. Does the user understand the implications of each sharing method? (ie. privacy) KWS Market Research, Inc. Page 4

13 That is all the tasks I have for you at this time. I would now like to ask you a few questions about the application you just used. Once again, please be candid with your responses and provide as much detail as you can. (After interview is complete...) This concludes our test for today. Before I leave, do you have any additional questions or comments about either the app, or about the testing procedure itself? BRIEFLY REVIEW ANSWERS Thank you for your time and opinions today. Your feeback is very valuable to us and will be of great assistance to us in improving our products. For office use only: Session: No.: Date: KWS Market Research, Inc. Page 5

14 MODERATOR TIPS: Shut up and listen! Replies... What would YOU do? What do YOU think? What words are going through your mind? What are you looking for? How would YOU go about accomplishing that? Do YOU think it was the correct selection? What about that makes you unsure it was the correct selection? When errors occur... Wait, then ask... Can you tell me about why you approached it that way? What were you hoping it would to? What were you expecting to see/happen? What made you realize it didn t work? Remember to STOP them mid-action to gain insight... Before you click on that, what do you think will happen? Is that what you expected? KWS Market Research, Inc. Page 6