Introduction to Questionmark s Assessment Platform

Transcription

1 Introduction to Questionmark s Assessment Platform Brian McNamara Marketing Director, Questionmark

2 Agenda Key Drivers for Assessment Assessment Platform Overview Collaborative Authoring Demonstration Blended & Secure Delivery Demonstration Reporting and Analytics Demonstration Integration Q&A Slide 2

3 To ask questions, use the Q&A feature Watch for an after the webinar: Download slides (PDF) View a recording Answer a survey Slide 3

4 About Questionmark Background Founded in 1988 Assessment solutions that enable organizations to measure knowledge, skills, abilities and attitudes securely for certification, regulatory compliance and improved learning outcomes ISO/IEC Certified (Learn more: Customers Enterprise customers worldwide Business, Government; Higher Education; Certification Providers; Awarding Bodies

5 Assessment Platform Reporting to analyze results and inform stakeholders Collaborative, secure authoring of surveys, quizzes, tests, and exams Multilingual: 36 Languages Blended delivery to support a broad range of devices and security needs Global 24/7 Support

6 Deployment Options OnDemand OnDemand for Government OnPremise

7 Key Drivers for using Assessments Measure and Enhance Learning Measure knowledge, skills and attitudes before, during and after learning Correct misconceptions and increase knowledge retention Qualification and Certification Deliver reliable, defensible tests to certify employees Ensure sound personnel decisions Regulatory Compliance Ensure knowledge of essential laws, regulations and health/safety guidelines Demonstrate compliance and reduce risk Sales/Technical Channel Competence/product expertise of sales and technical professionals Increase revenues, customer satisfaction Reduce costs Credentialing Efficiently create and maintain valid, reliable tests Generate revenue from exams and practice tests Higher Education Higher Education Increase student satisfaction Reduce cheating Improve learning

8 Questionmark Conference 2019 Empowering Results Still time to register! February 26th-March 1st Hard Rock Hotel San Diego "My team always walks away with new knowledge - and we have been using the product for over 12 years!" Slide 9

9 Authoring and Item Banking Slide 10

10 Authoring and Item Banking Browser-based Role-based access control Secure, collaborative authoring Author items (questions) Author assessments Publish assessments for delivery Manage item bank By topic, subtopic and metatag Item version control Import items & assessments Manage translations Slide 11

11 Many item types to choose from Multiple choice Multiple Response Matching Ranking Pull-down List Select-a-blank Matching Job Task Analysis Knowledge Matrix Survey Matrix Likert scale True/False Yes/No Text Match Drag-and-Drop Hotspot Numeric Essay Fill-in-the-blanks Explanation File Upload Slide 12

12 Demonstration Slide 13

13 Question-level feedback Question with participant s answer Corrective feedback Slide 20

14 Feedback and Scoring at Topic Level Assessment Electrical Skills Topic AC Motors Synchronous Motors Switchgear Question Item Item Item Item Item Item Topics Topic Score Topic Feedback Slide 21

15 Roles & Permission Management Provide users role-based access to content

16 Item version history and rollback Audit trail of edits and option to rollback Compare revisions side-by-side Slide 23

17 Item Search Search item bank to find items that contain certain words and/or tags Search description, stimulus, choices, status, meta tags, dates and more

18 Translation Management for Multilingual Assessments

19 Blended, Secure Delivery Slide 30

20 Author Once Deliver Anywhere Author once Deliver to any device in context Available Securely Single results set Slide 31

21 Responsive Design, Blended Delivery One Assessment many devices and browsers

22 Demonstration Slide 34

23 Printing and Scanning: Deliver assessments online or on paper Print your own bubble sheets Participants enter responses Scan and upload for scoring Analyze and report on results Deliver assessments on paper or online Print your own question and bubble sheets Use your own scanner to read responses Upload scanned bubble sheets for scoring Scanned assessment results available for reporting Slide 36

24 S t a k e s o f A s s e s s m e n t Delivering high stakes assessments Diagnostic Tests Low Stakes Tests Medium Stakes Tests High Stakes Exams Quizzes & Surveys S e c u r i t y R e q u i r e m e n t s

25 Manage and Enforce Strong Password Policies Set password expiration periods Force password changes Set password requirements/constraints by Length Complexity Alphanumeric Punctuation Upper/lower case Digits/Digit placement Exclude previous passwords Password qwerty Football abc123 Slide 40

26 Set Scheduling Restrictions to limit access Set date ranges for availability of assessment(s) Set maximum number of attempts Slide 42

27 Questionmark Secure One Setting Multiple Platforms Secure browser for high-stakes assessments Reduce content theft, cheating; improve exam integrity Prevents. Windows Mac ios Android Printing Screen capture Task switching Control/system keys Right-click / context menu options View source Accidentally exiting exam Slide 43

28 Require Proctoring Assessment Schedules Assessment Settings Slide 44

29 OnSite Proctoring Proctors can control and intervene in exam delivery: Generate unlock code for test Start, pause, un-pause and stop Dashboard to monitor exam status of each test taker Works with Questionmark Secure

30 Live Online Proctoring Convenience of online exam delivery at home or workplace under the watchful eye of a proctor Works with most standard webcams/audio Live proctor can start, pause, and stop exam More info: Website Your account manager Powered by Slide 46

31 Accessibility Also support for - Non-mouse navigation; alternate pointing devices - Screen readers

32 Integration Single-Sign-On SAML LDAP Launch and Track via AICC or SCORM SuccessFactors, Cornerstone, etc. Learning Tools for Interoperability (LTI) Blackboard, Canvas, D2L, Moodle, etc. Web services API QMWISe OData APIs Results API Slide 48

33 Reporting and Analytics Slide 50

34 Reporting and Analytics What can we learn from assessments? Did they like it? Did they learn it? Can they do it? Can we prove it? Slide 51

35 What you can learn from Questionmark reporting and analytics How are our Items and Assessments doing? How are our Participants doing? How did one group of test takers compare to another? Did the learning investment impact performance? Where are the skills and knowledge gaps in my organization? How do I coach participants to improve performance? Are my questions and tests valid and reliable? What did participants learn from the training or course? What did participants think of the learning experience?

36 Types of reports available Item and Test Analysis Item Analysis report Test Analysis report Score list report Test center analysis report Assessment completion time report Question frequency analysis report Demographic report Question type report Assessment Overview report Assessment results over time report Participant-focused reports Score list report Coaching Report Transcript Report Grade Book Report Survey Reports Survey Frequency Analysis Job-task-analysis (JTA) JTA Summary JTA dimension by demographic Course Evaluation Course Summary Report Instructor Summary Report Class Summary Report Class Detail Report Slide 53

37 Score List Report Demonstration Slide 54

38 Item Analysis: Reliability of Question Results Data Difficulty index (aka P value ) Item correlation discrimination: What it measures: ~ % answered correctly How it is expressed: Number from 0.0 to 1.0 How to interpret: Too high: Question is too easy. Too low: Might be Too difficult or confusing, ambiguous wording or more than one possible correct answer or miscoded What it measures: Correlation between item and assessment scores How it is expressed: Number between -1.0 and How to interpret: Generally < 0.2 low > 0.3 good

39 Correlation - Discrimination Difficulty/ P-Value Slide 56

40 Difficulty (p-value) Discrimination Option Analysis Slide 57

41 Coaching Report Compare against benchmarks or other participants Participant Topic level Overall Slide 58

42 Job Task Analysis (JTA) Reports Slide 59

43 Four Course Evaluation Reports: Course Summary Instructor Summary Class Summary Class Detail Slide 60

44 Big Data Applications: Results API Built-in Reports Custom Reports and Dashboards Microsoft Excel PowerPivot BI Tools R e s u l t s W a r e h o u s e Slide 61

45 Consulting / Solution Services Deployment Planning Installation Integration Authoring Assessments Importing items Templates / Look & feel Reporting Item reports Other custom reports Solutions & Integration Testing and Assessment Portals People Data syncing Results Data syncing Self Registration Auto Scheduling Assessment Clustering Certificate Generation Distributed Administration Assessment Catalogs w w w. q u e s t i o n m a r k. c o m / c o n s u l t i n g Slide 62

46 Summary of key benefits Collaborative Authoring Secure Browser-based Available globally Blended Delivery Responsive Design Browser support Tablets/smartphones Accessibility Secure delivery for high stakes exams Reporting and analytics How are my participants doing? How are my tests doing? Results API Multilingual Translation Management Screens/Controls in 36 languages Right-to-left languages Integration Supports industry standards APIs Connectors

47 What questions do you have?

48 How to Evaluate Request a one-on-one demo The Questionmark team will contact you to arrange a demonstration tailored to your needs and questions Request a 30-day trial of Questionmark OnDemand Complimentary technical support during trial Slide 67

49 ebooks, White Papers, Infographics and more! w w w.questionmark.com/learningresources Slide 68

50 Thank you for attending! We hope to see you at a future webinar Keep up to date at blog.questionmark.com! Connect!