Tips for Creating and Delivering Effective Mobile Assessments

Transcription

1 Tips for Creating and Delivering Effective Mobile Assessments Ivan Forward, Questionmark Antonio Herrera, Inlea Copyright Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark

2 Percentage of World s population covered by a mobile signal Slide 2

3 Smartphone usage continues to grow Slide 3

4 2011 Census of South Africa Mobile Phone/Device Usage: Decade of Staggering Growth Rate 84.70% 90% 50% 57% 58.20% 32% % 11.80% % of homes with flush toilets % of homes with electricity % of Higher Education completers % mobile phone ownership Slide 4

5 % of overall website traffic from mobile devices Slide 5

6 Browser Support more important than ever October 2008 October 2012 Source: StatCounter.com Slide 6

7 What we will cover Example applications of mobile assessment Tips from our experience Questionmark support of mobile assessment A real-life example

8 Mobile delivery enables new possibilities Slide 8

9 Inexpensive, very portable Internet-connected mobile devices

10 What are the potential applications? Observational assessments Formative assessment Surveys & Course Evaluation

11 Observational Assessments Slide 11

12 Traditional vs. Observational Assessment Traditional assessment Participant is presented with questions for direct response Observational assessment Observer is presented with questions to rate participant to measure behaviors, skills and abilities Example Applications Certification Process Observational assessments can be used as critical components of the certification process in some cases are required by regulatory authorities. Medical and Dental School exams: Medical and dental schools make extensive use of observational assessments such as the Objective Structured Clinical Examination (OSCE). Equipment operation: Observational assessment can be used to record and rate operators' skills, abilities, safety practices and adherence to required procedures

13 Typical workflow for assessor Assessor logs in Selects assessment Selects participant to be observed Answers/submits responses to questions Results/feedback reported to stakeholders Slide 13

14 Slide 14

15 Slide 15

16 Back Slide 16

17 Formative Assessment Slide 17

18 Quizzes on mobile devices Low stakes and openly available Often appropriate for mobile devices Enables retrieval practice to reduce forgetting Provides the participant with corrective feedback for revision and learning Encourages self-directed and independent learners

19 Surveys and course evaluations Slide 19

20 QR Codes for Launching Assessments Course evaluation example: Embed demographic data in the URL to be saved to special fields in the assessment results, e.g. Course name Instructor name Location If you are delivering a survey (or course evaluation), QR Codes can help by: Increasing response rates because Quick and easy to access the right URL (no typing required) Increasing data captured allows Data easily captured (no typing required) Easier analysis of survey results by demographic is a registered trademark of Questionmark Computing Limited. Slide All 20other trademarks are acknowledged.

21 QR codes enable quick launching of quizzes and course evaluations Books PowerPoint Signage Slide 21 Benefits Ease of access for increased response rates Demographic data easily captured Better data accuracy Easier analysis of survey results by demographic

22 Apps to Scan QR Codes iphone Redlaser Neoreader ScanLife Android Google Goggles QR Droid Scanlife Windows Mobile QR Reader BeeTag Quickmark Blackberry ScanLife BeeTag QR Code Scanner is a registered trademark of Questionmark Computing Limited. Slide All 22other trademarks are acknowledged.

23 The Big Switch Survey

24 Mobile Test Centres Use notebooks/3g laptops/ipads to create mobile test centres Enables greater flexibility in where you deliver assessments Assessments can be conducted in on location environments is a registered trademark of Questionmark Computing Limited. Slide All 24other trademarks are acknowledged.

25 Securing an ipad An app should prevent candidates from easily getting to other URLs But access to home button allows task-switching Hardware solutions are available to enable use of ipad in a kiosk mode is a registered trademark of Questionmark Computing Limited. Slide All 25other trademarks are acknowledged.

26 Secure Browser One Setting Multiple Platforms Prevents. Printing Screen capture Task switching Control/system keys Right-click / context menu options View source Accidentally exiting exam Questionmark Secure for Windows Questionmark Secure for Mac Questionmark Secure for ipad Slide 26

27 Kiosk Mode Apps Padlock App Mobile Kiosk App Kiosk Pro App Lockdown to a website Allow linked websites Custom message if linked website is not allowed Back to home timer Browsing limit timer Custom message when browsing time limit is reached Show address bar Show forward / back page navigation arrows Digital signage mode (non touch) Scale pages to fit Clear local HTML presentation files

28 Tablet Enclosures Slide 28

29 Considerations: The higher the stakes, the higher the propensity to cheat still require human monitoring! Bandwidth Be sure to test the wifi and/or 3G signal strength prior to event is a registered trademark of Questionmark Computing Limited. Slide All 29other trademarks are acknowledged.

30 Tips for Creating and Delivering Effective Mobile Assessments Surveys and course evaluations: Sometimes less is more Think small: Screens, input devices and bandwidth To Flash, or not to Flash? Be aware of what technologies your participants devices support Use feedback carefully and make it short enough to read Test and test again: Try before you deliver! Slide 30

31 Mode of Deployment Mobile (phone) short 1-5 minute modules o Updates on new information o Review of previous curriculum o Learning Check to enhance retention Video (ipad) minute modules o Content involves more explanation o Larger volume of content required to present elearning (computer) minute modules o Difficult or complex content o Requires interaction and instruction o Higher level of security needed Classroom 1hr+

32 True for most surveys; Especially true for mobile surveys Consider your objectives: Keep your surveys brief to help increase response rate Put your most important questions at the start of the survey Include only one concept per question Group related questions together Avoid relying on open-ended questions (e.g. essay ) Slide 32

33 Think small: Screens, input devices and bandwidth

34 Develop for the Device Browser screen 480 pixels Graphics 200 pixels wide Video H.264 Audio mp3 at 22 khz

35 Lay-Out Most important goes high and left (4-5 inch square) Finger size matters (44 x 44) Place videos last Avoid Flash Beware of the Fold Line Font size or 2 columns of content/page

36 Screen Furniture Ensure your form buttons and other navigation controls (a/k/a screen furniture ) work with low screen resolutions Slide 36

37 Images Optimise size and compression of larger image files to minimise bandwidth requirements Detail on images should be kept to a minimum to ensure readability on small screens

38 Be cautious in use of Flash Be aware of what technologies your participants devices support Slide 38

39 Be cautious in use of Flash Helps minimise bandwidth requirements Strengthens cross-device compatibility Flash will not work on Apple iphones, ipads or ipod touch devices Flash is compatible with some mobile platforms such as Android phones (and tablets) and Windows Mobile Slide 39

40 When is feedback useful? Correct faulty recollection Helps participant with faulty recall Correcting misconceptions Provide guidance when evidence of fundamental misunderstanding of a topic Feedback only useful if it is read keep short Slide 40

41 Try before you deliver! Slide 41

42 Test your content! Test your assessment content With different browsers On as many different devices as practical Emulators can help understand how content will appear on the small screen -- but don t always give a true user experience Setting up a mobile test centre? Test delivery environments when possible particularly for signal/wifi strength Many free online bandwidth tests are available

43 Questionmark s support for Mobile Delivery Slide 43

44 Levels of Monitoring Blended Delivery Diagnostic Tests Low Stakes Tests Medium Stakes Exams High Stakes Exams Formative Quizzes Course Evaluations Stakes of Assessment

45 Access to your Assessments Mobile delivery makes it easier for participants to access assessments: Use mobile Apps Place links in content (PDFs, HTML Web Pages, etc) Embed within a Portal For employees, partners and customers

46 Android Questionmark Mobile Apps iphone, ipad and ipod touch

47 Participant Devices Responsive Design 4. On-the-fly auto-sizing of content and controls 1. Senses device/browser 3. Sends assessment to browser 2. Device drivers used to format assessment for device/browser Slide 47

48 Responsive Design Will resize the assessment tools/content for optimum fit Consider if your assessment content will be affective on smaller screens

49 Benefits Author once Deliver to any device or context Schedule once Single results set Slide 49

50 Case study Antonio Herrera, Inlea

51 Mobile Delivery Real Case Test of Entrepreneurial Profile Belongs to Inlea s CSR activity to help Startup Entrepreneurs to design and launch their business projects (Fundación Inlea, linktostart)

52 Mobile Delivery Real Case Context, about Inlea: Inlea is an international group of companies dedicated to transfer knowledge between Industry and the Education Sector. We support education programs for companies such as Cisco, Oracle, Corning, and Televes Inlea is Questionmark s distributor for Spain

53 Mobile Delivery Real Case entrepreneur, i.e. one who "undertakes an enterprise". The term puts emphasis on the risk and effort of individuals who own and manage a business, and on the innovations that result from their pursuit of economic success

54 Mobile Delivery Real Case

55 Mobile Delivery Real Case

56 Mobile Delivery Real Case Why Mobile? This Self-Test will be delivered to large communities of Startup Entrepreneurs, intra-entrepreneurs, and University students. Many of them will be able to take the test in mobile devices and will prefer such option. Mobile delivery will allow a wider deployment to service more users with their device of choice.

57 Mobile Delivery Real Case Issues? Is a mobile device the best tool for an entrepreneurial profile self-test? What size of mobile device is sufficient? What happens on a crowded context? Is it possible to brand and customize the mobile template?

58 Screen shot: Laptop Mobile Delivery Real Case

59 Screen shot: Samsung Galaxy S4 Mobile Delivery Real Case

60

61 Thank you for attending! Ivan Forward Antonio Herrera Copyright Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark