Introduction to Questionmark s Assessment Platform Brian McNamara Marketing Director, Questionmark
Agenda Key Drivers for Assessment Assessment Platform Overview Collaborative Authoring Demonstration Blended & Secure Delivery Demonstration Reporting and Analytics Demonstration Integration Q&A Slide 2
To ask questions, use the Q&A feature Watch for an email after the webinar: Download slides (PDF) View a recording Answer a survey Slide 3
About Questionmark Background Founded in 1988 Assessment solutions that enable organizations to measure knowledge, skills, abilities and attitudes securely for certification, regulatory compliance and improved learning outcomes ISO/IEC 27001 Certified (Learn more: www.questionmark.com/trust) Customers Enterprise customers worldwide Business, Government; Higher Education; Certification Providers; Awarding Bodies
Assessment Platform Reporting to analyze results and inform stakeholders Collaborative, secure authoring of surveys, quizzes, tests, and exams Multilingual: 36 Languages Blended delivery to support a broad range of devices and security needs Global 24/7 Support
Deployment Options OnDemand OnDemand for Government OnPremise
Key Drivers for using Assessments Measure and Enhance Learning Measure knowledge, skills and attitudes before, during and after learning Correct misconceptions and increase knowledge retention Qualification and Certification Deliver reliable, defensible tests to certify employees Ensure sound personnel decisions Regulatory Compliance Ensure knowledge of essential laws, regulations and health/safety guidelines Demonstrate compliance and reduce risk Sales/Technical Channel Competence/product expertise of sales and technical professionals Increase revenues, customer satisfaction Reduce costs Credentialing Efficiently create and maintain valid, reliable tests Generate revenue from exams and practice tests Higher Education Higher Education Increase student satisfaction Reduce cheating Improve learning
Questionmark Conference 2019 Empowering Results Still time to register! February 26th-March 1st Hard Rock Hotel San Diego "My team always walks away with new knowledge - and we have been using the product for over 12 years!" www.questionmark.com/go/conference Slide 9
Authoring and Item Banking Slide 10
Authoring and Item Banking Browser-based Role-based access control Secure, collaborative authoring Author items (questions) Author assessments Publish assessments for delivery Manage item bank By topic, subtopic and metatag Item version control Import items & assessments Manage translations Slide 11
Many item types to choose from Multiple choice Multiple Response Matching Ranking Pull-down List Select-a-blank Matching Job Task Analysis Knowledge Matrix Survey Matrix Likert scale True/False Yes/No Text Match Drag-and-Drop Hotspot Numeric Essay Fill-in-the-blanks Explanation File Upload Slide 12
Demonstration Slide 13
Question-level feedback Question with participant s answer Corrective feedback Slide 20
Feedback and Scoring at Topic Level Assessment Electrical Skills Topic AC Motors Synchronous Motors Switchgear Question Item Item Item Item Item Item Topics Topic Score Topic Feedback Slide 21
Roles & Permission Management Provide users role-based access to content
Item version history and rollback Audit trail of edits and option to rollback Compare revisions side-by-side Slide 23
Item Search Search item bank to find items that contain certain words and/or tags Search description, stimulus, choices, status, meta tags, dates and more
Translation Management for Multilingual Assessments
Blended, Secure Delivery Slide 30
Author Once Deliver Anywhere Author once Deliver to any device in context Available Securely Single results set Slide 31
Responsive Design, Blended Delivery One Assessment many devices and browsers
Demonstration Slide 34
Printing and Scanning: Deliver assessments online or on paper Print your own bubble sheets Participants enter responses Scan and upload for scoring Analyze and report on results Deliver assessments on paper or online Print your own question and bubble sheets Use your own scanner to read responses Upload scanned bubble sheets for scoring Scanned assessment results available for reporting Slide 36
S t a k e s o f A s s e s s m e n t Delivering high stakes assessments Diagnostic Tests Low Stakes Tests Medium Stakes Tests High Stakes Exams Quizzes & Surveys S e c u r i t y R e q u i r e m e n t s
Manage and Enforce Strong Password Policies Set password expiration periods Force password changes Set password requirements/constraints by Length Complexity Alphanumeric Punctuation Upper/lower case Digits/Digit placement Exclude previous passwords Password1 12345678 qwerty Football abc123 Slide 40
Set Scheduling Restrictions to limit access Set date ranges for availability of assessment(s) Set maximum number of attempts Slide 42
Questionmark Secure One Setting Multiple Platforms Secure browser for high-stakes assessments Reduce content theft, cheating; improve exam integrity Prevents. Windows Mac ios Android Printing Screen capture Task switching Control/system keys Right-click / context menu options View source Accidentally exiting exam Slide 43
Require Proctoring Assessment Schedules Assessment Settings Slide 44
OnSite Proctoring Proctors can control and intervene in exam delivery: Generate unlock code for test Start, pause, un-pause and stop Dashboard to monitor exam status of each test taker Works with Questionmark Secure
Live Online Proctoring Convenience of online exam delivery at home or workplace under the watchful eye of a proctor Works with most standard webcams/audio Live proctor can start, pause, and stop exam More info: Website Your account manager Powered by www.questionmark.com/proctoring Slide 46
Accessibility Also support for - Non-mouse navigation; alternate pointing devices - Screen readers
Integration Single-Sign-On SAML LDAP Launch and Track via AICC or SCORM SuccessFactors, Cornerstone, etc. Learning Tools for Interoperability (LTI) Blackboard, Canvas, D2L, Moodle, etc. Web services API QMWISe OData APIs Results API Slide 48
Reporting and Analytics Slide 50
Reporting and Analytics What can we learn from assessments? Did they like it? Did they learn it? Can they do it? Can we prove it? Slide 51
What you can learn from Questionmark reporting and analytics How are our Items and Assessments doing? How are our Participants doing? How did one group of test takers compare to another? Did the learning investment impact performance? Where are the skills and knowledge gaps in my organization? How do I coach participants to improve performance? Are my questions and tests valid and reliable? What did participants learn from the training or course? What did participants think of the learning experience?
Types of reports available Item and Test Analysis Item Analysis report Test Analysis report Score list report Test center analysis report Assessment completion time report Question frequency analysis report Demographic report Question type report Assessment Overview report Assessment results over time report Participant-focused reports Score list report Coaching Report Transcript Report Grade Book Report Survey Reports Survey Frequency Analysis Job-task-analysis (JTA) JTA Summary JTA dimension by demographic Course Evaluation Course Summary Report Instructor Summary Report Class Summary Report Class Detail Report Slide 53
Score List Report Demonstration Slide 54
Item Analysis: Reliability of Question Results Data Difficulty index (aka P value ) Item correlation discrimination: What it measures: ~ % answered correctly How it is expressed: Number from 0.0 to 1.0 How to interpret: Too high: Question is too easy. Too low: Might be Too difficult or confusing, ambiguous wording or more than one possible correct answer or miscoded What it measures: Correlation between item and assessment scores How it is expressed: Number between -1.0 and +1.0. How to interpret: Generally < 0.2 low > 0.3 good
Correlation - Discrimination Difficulty/ P-Value Slide 56
Difficulty (p-value) Discrimination Option Analysis Slide 57
Coaching Report Compare against benchmarks or other participants Participant Topic level Overall Slide 58
Job Task Analysis (JTA) Reports Slide 59
Four Course Evaluation Reports: Course Summary Instructor Summary Class Summary Class Detail Slide 60
Big Data Applications: Results API Built-in Reports Custom Reports and Dashboards Microsoft Excel PowerPivot BI Tools R e s u l t s W a r e h o u s e Slide 61
Consulting / Solution Services Deployment Planning Installation Integration Authoring Assessments Importing items Templates / Look & feel Reporting Item reports Other custom reports Solutions & Integration Testing and Assessment Portals People Data syncing Results Data syncing Self Registration Auto Scheduling Assessment Clustering Certificate Generation Distributed Administration Assessment Catalogs w w w. q u e s t i o n m a r k. c o m / c o n s u l t i n g Slide 62
Summary of key benefits Collaborative Authoring Secure Browser-based Available globally Blended Delivery Responsive Design Browser support Tablets/smartphones Accessibility Secure delivery for high stakes exams Reporting and analytics How are my participants doing? How are my tests doing? Results API Multilingual Translation Management Screens/Controls in 36 languages Right-to-left languages Integration Supports industry standards APIs Connectors
What questions do you have?
How to Evaluate Request a one-on-one demo The Questionmark team will contact you to arrange a demonstration tailored to your needs and questions www.questionmark.com/go/request-demo-enus Request a 30-day trial of Questionmark OnDemand Complimentary technical support during trial www.questionmark.com/go/od30us Slide 67
ebooks, White Papers, Infographics and more! w w w.questionmark.com/learningresources Slide 68
Thank you for attending! We hope to see you at a future webinar Keep up to date at blog.questionmark.com! Connect!