The Closed Runway Operation Prevention Device: Applying Automatic Speech Recognition Technology for Aviation Safety

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "The Closed Runway Operation Prevention Device: Applying Automatic Speech Recognition Technology for Aviation Safety"

Transcription

1 MITRE CAASD The Closed Runway Operation Prevention Device: Applying Automatic Speech Recognition Technology for Aviation Safety Shuo Chen Hunter Kopald June 25, 2015 Approved for Public Release; Distribution Unlimited The MITRE Corporation. All rights reserved.

2 2 Outline Overview of closed runway operations Closed Runway Operation Prevention Device (CROPD) Role of speech recognition Automatic speech recognition, in general and in ATC Tuning techniques to tune performance for the ATC domain Language modeling Acoustic model adaptation Semantic interpretation and confidence thresholding Evaluating the CROPD speech recognition performance through field demonstration Implications and future work

3 Overview of Closed Runway Operations 3 Local Controller Transmission American one twenty-three, wind three five zero at eight, runway one left, cleared for takeoff Prevention Mechanisms Standard Operating Procedures Basic memory aids: flight strip placards Additional automation systems that provide alerts Pilot Readback Runway one left, cleared for takeoff, american one twenty-three, X X Operations no longer allowed on runways designated as closed

4 Closed Runway Operation Prevention Device (CROPD) 4 FAA-proposed concept to prevent operations on closed runways by using automatic speech recognition to monitor controller clearances CROPD alerts if both: 1. Clearance detected cleared to land cleared for takeoff line up and wait Other clearances to be added, e.g., cleared touch and go 2. Runway associated with clearance is designated as closed Additional details Only listens to local controller transmissions Not integrated with other tower systems (e.g., ASDE-X) Intended for use at airports both with and without advanced runway safety systems Air Traffic Control Tower! Alert Controller Closed runway status Graphical User Interface Transmission Audio Closed Runway Status Alert Trigger CROPD in Equipment Room Speech recognition and understanding Identify spoken intent: 1. Runway 2. Clearance Alert logic Compare: 1. Runway and clearance detected 2. Closed runway status Trigger alert

5 5 The Role of Speech Recognition Objective of the speech recognition is to detect intent to use a runway United one twenty-three, wind three three zero at six, runway one center cleared to land, traffic departs runway three zero. Recognized numbers One twenty-three Three three zero Six One (center) Logical concept ACID Wind direction Wind speed Runway clearance For CROPD, intent can be considered the presence of a clearance and the runway associated with that clearance Simplifies performance measure to specific phrase-level accuracy Requires disambiguation of phrases that could be confused with runways Requires accurate association of the correct runway with the clearance post-recognition Three zero Runway Not all words in the transmission are equally important to the detection of intent Word Error Rate (WER) is not appropriate Need an application-specific metric Composite measure of both runway and clearance Considers correct association of runway with clearance Use intent as metric, with 5 outcome types CROPD Result Intent: RWY 30, CFT Intent: RWY 19L, CTL No Intent Truth Actual Transmission Intent: RWY30, CFT Correct Intent Incorrect Intent Missed Intent No Intent False Intent False Intent Correct Rejection Speech recognition performance is different from system performance Incorrect speech recognition results do not always translate to incorrect system performance

6 Automatic Speech Recognition 6 In General Automatic speech recognition performance depends not only on the speech input at the time of application, but also on what is already known about the application prior to use In some cases, meaning deduction is more important than word recognition Need to tailor system to the application Need to define what recognition is needed for success With Air Traffic Control ATC characteristics that make automatic speech recognition challenging Acoustic characteristics of audio equipment Acoustic characteristics of the speaker set Rate of speech Pronunciation Accents Deviations from standard phraseology ATC characteristics that facilitate automatic speech recognition Standard phraseology, when followed Acoustic modeling custom pronunciation adaptation Context information: what the system should expect to hear KIAD Airport Diagram 12 19R 1L 19C 1C 30 19L 1R

7 Tuning Recognition Performance: Language Model Creation and Adaptation 7 Language models define the universe of words and word sequences that the speech recognition system is designed to recognize 1. Finite-state grammars Manually defined vocabulary and word sequences Can yield near-perfect recognition in applications where speakers adhere to defined sequences Poor tolerance for phrase deviations 2. Statistical language models (SLMs) Machine-generated vocabulary and probability model of word sequences based on transcription data from target environment Robust to speech variations and disfluencies Can yield improbable word sequences CROPD used both types of models Initial A grammar benchmark containing of speech only runway recognition and performance on local controller audio using clearance the two phrases, CROPD with language common models yielded substitutions, 70% true intent additions, detection and omissions A 50% statistical false intent language detection model trained on 10,000+ transcriptions from the local controller position

8 Tuning Recognition Performance: Acoustic Model Adaptation 8 Acoustic models define the statistical signatures of speech sounds, i.e. phonemes, and specify the sound sequences that form words Default acoustic model usually provided out-of-the-box Methods of adaptation: 1. Pronunciation dictionaries CROPD used both types Acoustic of acoustic Model tuning = techniques = cleared-to-land Manually defined word pronunciations Accounts for common pronunciation variation and non-standard words, e.g. airline names, fix names 2. Automatic acoustic model adaptation Automatic supervised training of existing acoustic model with audio and transcriptions from target speaker set and environment Accounts for channel characteristics and consistent speech patterns, i.e. accents, within a speaker set Custom ATC dictionary Non-standard words, e.g. = niner niner = one Composite = words where coarticulation = and assimilation were frequently observed, e.g. clearedto-land clea da lan = Custom acoustic model adapted with audio and transcriptions from local controller position Second benchmark of speech recognition performance showed improvement increased true intent detection to 89% reduced false intent detection to 18%

9 Tuning Recognition Performance: Post- Recognition Processing 9 Semantic interpretation performs text processing to derive logical concepts from potentially error-filled raw text Apply application-specific reasoning to disambiguate confusable phrases Introduce robustness to recognition error by selectively using only parts of the recognized text For CROPD, a semantic interpretation algorithm was developed Rescores hypotheses based on systemgenerated confidence scores Deduces the most probable combination of clearance and runway Third benchmark of speech recognition performance showed improvement increased true intent detection to 95% reduced false intent detection to 17% Confidence Thresholding uses system-generated scores to bias final result to a specific balance of missed and false detections In CROPD, confidence thresholding is implemented System users must determine what confidence threshold is operationally acceptable Can vary by facility Does not affect speech recognition system Advises the application using the speech recognition whether to accept a recognition result based on its own certainty of accuracy

10 Evaluating Performance through Field Demonstration 10 In summer of 2014, the CROPD was field tested at KIAD as a part of the NAS Passively monitored local controller channel by connecting to the facility voice switch A subset of field audio was selected for performance evaluation The effect of adjusting the confidence threshold was also studied True Intent Transmissions with Intent Incorrect Intent Missed Intent Transmissions without Intent Correct Non-Intent False Intent 92.70% 4.77% 2.54% 90.33% 9.67% Analysis of some of the false and incorrect intent errors showed that the semantic interpretation algorithm could be further improved Erroneously discarding correctly recognized runway phrases Some ambiguous phrases still erroneously tagged as runways

11 11 Implications and Future Work A combination of various tuning techniques can yield significant performance improvements CROPD is a simple, isolated application that demonstrates the feasibility of applying automatic speech recognition on live ATC transmissions Other applications could make use of the same detected clearances for other purposes or detect new instructions for other controller positions More complex applications could also be feasible with the addition of context information from tower automation systems MITRE is currently investigating the benefits of integrating speech recognition with Tower/Surface surveillance data Leverage speech-derived intent to improve safety alert performance Use aircraft and airport state information to improve speech recognition performance

12 2015 The MITRE Corporation. All rights reserved. Questions?

13 Transmission without Intent Transmission with Intent A Closer Look at Recognition vs. System Performance 13 Spoken Intent: 19L, cleared to land Closure: none Detected: none Correct non-alert Spoken Intent: 19L, cleared to land Closure: 19L/1R Detected: none Missed Alert Correct Alert Behavior Incorrect Alert Behavior Spoken runway not closed, no missed or false alert Spoken runway closed, missed alert Missed Intent Detection

14 Transmission without Intent Transmission with Intent A Closer Look at Recognition vs. System Performance 14 Correct Alert Behavior Incorrect Alert Behavior False Intent Detection Detected runway not closed, no false alert Detected runway closed, false alert Spoken Intent: none Closure: none Detected: 19L, cleared to land Correct non-alert Spoken Intent: none Closure: 19L/1R Detected: 19L, cleared to land False Alert

15 Transmission without Intent Transmission with Intent A Closer Look at Recognition vs. System Performance 15 Spoken Intent: 19L, cleared to land Closure: none Detected: 1L, cleared to land Correct non-alert Spoken Intent: 19C, cleared to land Closure: 19C/1C Detected: 1C, cleared to land Correct non-alert Correct Alert Behavior Spoken runway not closed, detected runway not closed, no missed or false alert Spoken runway closed, detected runway is opposite end of spoken runway, alert correctly generated Incorrect Alert Behavior Spoken runway closed, detected runway not closed, missed alert Spoken Intent: 19L, cleared to land Closure: 19L/1R Detected: 1L, cleared to land Missed Alert Detected runway is not closed, no false alert Detected runway is closed, false alert Spoken Intent: 19L, cleared to land Closure: none Detected: 1L, cleared to land Correct non-alert Incorrect Intent Detection Spoken Intent: 19L, cleared to land Closure: 1L/19R Detected: 1L, cleared to land False Alert

The Closed Runway Operation Prevention Device: Applying Automatic Speech Recognition Technology for Aviation Safety

The Closed Runway Operation Prevention Device: Applying Automatic Speech Recognition Technology for Aviation Safety Eleventh USA/Europe Air Traffic Management Research and Development Seminar (ATM2015) The Closed Runway Operation Prevention Device: Applying Automatic Speech Recognition Technology for Aviation Safety

More information

Aviation English Solutions

Aviation English Solutions Aviation English Solutions DynEd's Aviation English solutions develop a level of oral English proficiency that can be relied on in times of stress and unpredictability so that concerns for accurate communication

More information

The use of speech recognition confidence scores in dialogue systems

The use of speech recognition confidence scores in dialogue systems The use of speech recognition confidence scores in dialogue systems GABRIEL SKANTZE gabriel@speech.kth.se Department of Speech, Music and Hearing, KTH This paper discusses the interpretation of speech

More information

Increased Acceptance of Controller Assistance by Automatic Speech Recognition

Increased Acceptance of Controller Assistance by Automatic Speech Recognition www.dlr.de/fl Chart 1 Increased Acceptance of Controller Assistance by Automatic Speech Recognition Project partners Supported by DLR Technology Marketing and Helmholtz Validation Fund Hartmut Helmke Heiko

More information

ESCALE Enhanced Speech Tracking of Air Traffic Control Communications

ESCALE Enhanced Speech Tracking of Air Traffic Control Communications ESCALE Enhanced Speech Tracking of Air Traffic Control Communications IntuiLab Institut de Recherche en Informatique de Toulouse Technical University of Crete THALES Avionics THALES Research & Technology

More information

CIVIL AVIATION REQUIREMENT Section 7-Flight Crew Standards Training and Licensing Series G Part IV

CIVIL AVIATION REQUIREMENT Section 7-Flight Crew Standards Training and Licensing Series G Part IV Government of India OFFICE OF THE DIRECTOR GENERAL OF CIVIL AVIATION Opposite Safdarjung Airport, New Delhi-110003 CIVIL AVIATION REQUIREMENT Section 7-Flight Crew Standards Training and Licensing Series

More information

Speech Recognition at ICSI: Broadcast News and beyond

Speech Recognition at ICSI: Broadcast News and beyond Speech Recognition at ICSI: Broadcast News and beyond Dan Ellis International Computer Science Institute, Berkeley CA Outline 1 2 3 The DARPA Broadcast News task Aspects of ICSI

More information

Human Factors Computer Based Training in Air Traffic Control

Human Factors Computer Based Training in Air Traffic Control Paper presented at Ninth International Symposium on Aviation Psychology, Columbus, Ohio, USA, April 28th to May 1st 1997. Human Factors Computer Based Training in Air Traffic Control A. Bellorini 1, P.

More information

Initial English Language Training for Controllers and Pilots. Mr. John Kennedy École Nationale de L Aviation Civile (ENAC) Toulouse, France.

Initial English Language Training for Controllers and Pilots. Mr. John Kennedy École Nationale de L Aviation Civile (ENAC) Toulouse, France. Initial English Language Training for Controllers and Pilots Mr. John Kennedy École Nationale de L Aviation Civile (ENAC) Toulouse, France Summary All French trainee controllers and some French pilots

More information

Test of English for Aviation. Guide for: Test-takers & Teachers

Test of English for Aviation. Guide for: Test-takers & Teachers Guide for: Test-takers & Teachers What is the TEA test? TEA is a test of your ability to communicate in English it is not a test of your operational knowledge. TEA is a test of plain English in an aviation

More information

Statistical Modeling of Pronunciation Variation by Hierarchical Grouping Rule Inference

Statistical Modeling of Pronunciation Variation by Hierarchical Grouping Rule Inference Statistical Modeling of Pronunciation Variation by Hierarchical Grouping Rule Inference Mónica Caballero, Asunción Moreno Talp Research Center Department of Signal Theory and Communications Universitat

More information

Lecture 16 Speaker Recognition

Lecture 16 Speaker Recognition Lecture 16 Speaker Recognition Information College, Shandong University @ Weihai Definition Method of recognizing a Person form his/her voice. Depends on Speaker Specific Characteristics To determine whether

More information

Decision Making. Dr Tony Head College of Aeronautics Cranfield University

Decision Making. Dr Tony Head College of Aeronautics Cranfield University Decision Making Dr Tony Head College of Aeronautics Cranfield University How many psychologists to change a light bulb? First, the light bulb has to want to be changed Land or Go Around? 1000ft 500ft 100ft

More information

EUROCONTROL Explanatory Factors

EUROCONTROL Explanatory Factors 1 Safety Learning Cards EUROCONTROL Explanatory Factors Safety Learning Cards Safety Learning Cards Copyright This document is published by EUROCONTROL for information purposes. The text in this document

More information

DIALOGUES IN AIR TRAFFIC CONTROL

DIALOGUES IN AIR TRAFFIC CONTROL DIALOGUES IN AIR TRAFFIC CONTROL Gavin E Churcher, Clive Souter & Eric S Atwell School of Computer Studies, Leeds Univeristy, Leeds LS2 9JT UK Tel: +44 113 2336827 (Email: gavin@scs.leeds.ac.uk) ABSTRACT

More information

Analysis about English Audiovisual and Speaking Teaching of Students Majoring in Flight Technology Based on ICAO Standards Juan Huang

Analysis about English Audiovisual and Speaking Teaching of Students Majoring in Flight Technology Based on ICAO Standards Juan Huang International Conference on Education, Management and Computing Technology (ICEMCT 2014) Analysis about English Audiovisual and Speaking Teaching of Students Majoring in Flight Technology Based on ICAO

More information

Chapter 2 Keyword Spotting Methods

Chapter 2 Keyword Spotting Methods Chapter 2 Spotting Methods This chapter will review in detail the three KWS methods, LVCSR KWS, KWS and Phonetic Search KWS, followed by a discussion and comparison of the methods. 2.1 LVCSR-Based KWS

More information

Edition 1.1 Edition date 01 Dec 2016 ELPAC. English Language Proficiency for Aeronautical Communication. ELPAC Pilot - Test taker preparation

Edition 1.1 Edition date 01 Dec 2016 ELPAC. English Language Proficiency for Aeronautical Communication. ELPAC Pilot - Test taker preparation Edition 1.1 Edition date 01 Dec 2016 ELPAC English Language Proficiency for Aeronautical Communication ELPAC Pilot - Test taker preparation ELPAC is developed in cooperation with: ENOVATE is responsible

More information

Informacije o predmetu PHONETICS OF ENGLISH. Bodovanje. Branches of Linguistics. Phonetics

Informacije o predmetu PHONETICS OF ENGLISH. Bodovanje. Branches of Linguistics. Phonetics Informacije o predmetu PHONETICS OF ENGLISH Lecture 1 Introduction to Phonetics Fond: 2P+2V Slajdovi sa predavanja Knjiga: Fonetika i fonologija engleskog jezika (Igor Lakić) English Pronunciation in Use

More information

An Adaptive and Intelligent Tutoring System with Fuzzy Reasoning Capabilities

An Adaptive and Intelligent Tutoring System with Fuzzy Reasoning Capabilities An Adaptive and Intelligent Tutoring System with Fuzzy Reasoning Capabilities dli2@students.towson.edu Hzhou@towson.edu Abstract The intelligence of E-learning system has become one of regarded topic to

More information

* Corresponding Author. Watthana Suksiripakonchai* Suan Dusit Rajabhat University

* Corresponding Author. Watthana Suksiripakonchai* Suan Dusit Rajabhat University Watthana Suksiripakonchai* Suan Dusit Rajabhat University Abstract The English language has been considered a contributory factor causing confusion or misunderstanding in aviation communication; therefore,

More information

AL THE. The breakthrough machine learning platform for global speech recognition

AL THE. The breakthrough machine learning platform for global speech recognition AL THE The breakthrough machine learning platform for global speech recognition SEPTEMBER 2017 Introducing Speechmatics Automatic Linguist (AL) Automatic Speech Recognition (ASR) software has come a long

More information

COMS W4705x: Natural Language Processing FINAL EXAM December 18th, 2008

COMS W4705x: Natural Language Processing FINAL EXAM December 18th, 2008 COMS W4705x: Natural Language Processing FINAL EXAM December 18th, 2008 DIRECTIONS This exam is closed book and closed notes. It consists of four parts. Each part is labeled with the amount of time you

More information

Teaching SFAR 93. Stuart W. Goering FAA Aviation Safety Counselor

Teaching SFAR 93. Stuart W. Goering FAA Aviation Safety Counselor Teaching SFAR 93 Stuart W. Goering FAA Aviation Safety Counselor When to Teach SFAR 93 Student Pilots Flight Reviews Area Checkouts 3/27/2003 17:48 Copyright 2003 Stuart W. Goering 2 Student Pilots Based

More information

Discovering Negative Categories by Clustering Drifted Terms. Discovering Negative Categories to Improve Semantic Lexicon Induction

Discovering Negative Categories by Clustering Drifted Terms. Discovering Negative Categories to Improve Semantic Lexicon Induction Discovering Negative to Improve Semantic Lexicon Induction Learning multiple semantic categories simultaneously improves bootstrapping because the categories constrain each other. Nevertheless, bootstrappers

More information

Designing an Interruption Management Experiment: Evaluating the Working Awareness Interruption Tool (WAIT) for Air Traffic Controllers

Designing an Interruption Management Experiment: Evaluating the Working Awareness Interruption Tool (WAIT) for Air Traffic Controllers Designing an Interruption Management Experiment: Evaluating the Working Awareness Interruption Tool (WAIT) for Air Traffic Controllers Meshael Alqahtani, Dr. Jonathan Histon, Dr. Stacey Scott May 1 st,

More information

L12: Template matching

L12: Template matching Introduction to ASR Pattern matching Dynamic time warping Refinements to DTW L12: Template matching This lecture is based on [Holmes, 2001, ch. 8] Introduction to Speech Processing Ricardo Gutierrez-Osuna

More information

Chapter 1. Using Resources for Vocabulary Development 1. Chapter 2. Writing about Increases and Decreases 24

Chapter 1. Using Resources for Vocabulary Development 1. Chapter 2. Writing about Increases and Decreases 24 Introduction x Chapter 1. Using Resources for Vocabulary Development 1 1.1 Characteristics of a Word 2 1.2 Resource: Learner Dictionaries 9 1.3 Resource: Collocations Dictionaries 14 1.4 Resource: Online

More information

Introduction to Classification, aka Machine Learning

Introduction to Classification, aka Machine Learning Introduction to Classification, aka Machine Learning Classification: Definition Given a collection of examples (training set ) Each example is represented by a set of features, sometimes called attributes

More information

Centro Evaluador de Competencia Lingüística para. Controladores de Tránsito Aéreo, Operadores ARO-AIS y Especialistas NOTAM

Centro Evaluador de Competencia Lingüística para. Controladores de Tránsito Aéreo, Operadores ARO-AIS y Especialistas NOTAM EXAMEN DE CERTIFICACIÓN EN COMPETENCIA LINGÜÍSTICA EN IDIOMA INGLÉS PARA CONTROLADORES DE TRÁNSITO AÉREO EXAM FORMAT PART A ONE-TO-ONE INTERACTION SECTION 1: Personal Interview (approximately 3 minutes)

More information

Adaptive Monitoring: Risk-Based Monitoring and Beyond

Adaptive Monitoring: Risk-Based Monitoring and Beyond Vol. 9, No. 9, September 2013 Happy Trials to You Adaptive Monitoring: Risk-Based Monitoring and Beyond By Michael Rosenberg In August 2011, the FDA and EMA issued a guidance and reflection paper, respectively,

More information

U-NORM Likelihood Normalization in PIN-Based Speaker Verification Systems

U-NORM Likelihood Normalization in PIN-Based Speaker Verification Systems U-NORM Likelihood Normalization in PIN-Based Speaker Verification Systems D. Garcia-Romero, J. Gonzalez-Rodriguez, J. Fierrez-Aguilar, and J. Ortega-Garcia Speech and Signal Processing Group (ATVS) Universidad

More information

CS474 Natural Language Processing. Word sense disambiguation. Machine learning approaches. Dictionary-based approaches

CS474 Natural Language Processing. Word sense disambiguation. Machine learning approaches. Dictionary-based approaches CS474 Natural Language Processing! Today Lexical semantic resources: WordNet» Dictionary-based approaches» Supervised machine learning methods» Issues for WSD evaluation Word sense disambiguation! Given

More information

LISTENING Grades 7 9 DATE LEVEL 3 Expanding

LISTENING Grades 7 9 DATE LEVEL 3 Expanding Alberta Education ESL Proficiency Benchmarks Tracking Sheets: Grades 7 9 Student Name: Grade 7 Year: Teacher: Grade 8 Year: Teacher: Grade 9 Year: Teacher: Note: LEVEL 1 LEVEL 2 LISTENING Grades 7 9 LEVEL

More information

LANGUAGE IN INDIA Strength for Today and Bright Hope for Tomorrow Volume 8 : 3 March 2008

LANGUAGE IN INDIA Strength for Today and Bright Hope for Tomorrow Volume 8 : 3 March 2008 LANGUAGE IN INDIA Strength for Today and Bright Hope for Tomorrow Volume 8 : 3 March 2008 Managing Editor: Editors: B. Mallikarjun, Ph.D. Sam Mohanlal, Ph.D. B. A. Sharada, Ph.D. A. R. Fatihi, Ph.D. Lakhan

More information

A TRANSITIVITY ANALYSIS OF RADIOTELEPHONY AT AHMAD YANI INTERNASIONAL AIRPORT

A TRANSITIVITY ANALYSIS OF RADIOTELEPHONY AT AHMAD YANI INTERNASIONAL AIRPORT A TRANSITIVITY ANALYSIS OF RADIOTELEPHONY AT AHMAD YANI INTERNASIONAL AIRPORT JOURNAL ARTICLE Submitted in Partial Fulfillment of the Requirements for the Degree of Sarjana Sastra (S.S) By: Rani Ardi Pradipta

More information

CSCI-GA Compiler Construction Lecture 6: Syntax Analysis. Mohamed Zahran (aka Z)

CSCI-GA Compiler Construction Lecture 6: Syntax Analysis. Mohamed Zahran (aka Z) CSCI-GA.2130-001 Compiler Construction Lecture 6: Syntax Analysis Mohamed Zahran (aka Z) mzahran@cs.nyu.edu Context-Free Grammars Precise syntactic specifications of a programming language For some classes,

More information

Robust DNN-based VAD augmented with phone entropy based rejection of background speech

Robust DNN-based VAD augmented with phone entropy based rejection of background speech INTERSPEECH 2016 September 8 12, 2016, San Francisco, USA Robust DNN-based VAD augmented with phone entropy based rejection of background speech Yuya Fujita 1, Ken-ichi Iso 1 1 Yahoo Japan Corporation

More information

Analyzing Human and Machine Performance In Resolving Ambiguous Spoken Sentences

Analyzing Human and Machine Performance In Resolving Ambiguous Spoken Sentences Analyzing Human and Machine Performance In Resolving Ambiguous Spoken Sentences Hussein Ghaly 1 and Michael I Mandel 1,2 1 City University of New York, Graduate Center, Linguistics Program 2 City University

More information

STUDY PLAN PhD. in Linguistics

STUDY PLAN PhD. in Linguistics STUDY PLAN PhD. in Linguistics I. GENERAL RULES CONDITIONS: Plan Number 1. This plan conforms to the valid regulations of the programs of graduate studies. 2. Areas of specialty of admission in this program:

More information

PRELIMINARY EVALUATION OF THE VOYAGER SPOKEN LANGUAGE SYSTEM*

PRELIMINARY EVALUATION OF THE VOYAGER SPOKEN LANGUAGE SYSTEM* PRELIMINARY EVALUATION OF THE VOYAGER SPOKEN LANGUAGE SYSTEM* Victor Zue, James Glass, David Goodine, Hong Leung, Michael Phillips, Joseph Polifroni, and Stephanie Seneff Spoken Language Systems Group

More information

Identifying Localization in Peer Reviews of Argument Diagrams

Identifying Localization in Peer Reviews of Argument Diagrams Identifying Localization in Peer Reviews of Argument Diagrams Huy V. Nguyen and Diane J. Litman University of Pittsburgh, Pittsburgh, PA, 15260 {huynv,litman}@cs.pitt.edu Abstract. Peer-review systems

More information

Introduction Department/Program: Aviation/ Professional Pilot Degree program(s): PPILB Person(s) preparing report: Barney King

Introduction Department/Program: Aviation/ Professional Pilot Degree program(s): PPILB Person(s) preparing report: Barney King Annual Progress Report on Assessment of Student Learning for Undergraduate Programs (Rev 04/11) The purpose of the Annual Progress Report (APR) of Student Learning Assessment is to provide a continuous

More information

DETECT AND AVOID FOR REMOTELY PILOTED AIRCRAFT IN AIR TRAFFIC CONTROL SIMULATIONS

DETECT AND AVOID FOR REMOTELY PILOTED AIRCRAFT IN AIR TRAFFIC CONTROL SIMULATIONS DETECT AND AVOID FOR REMOTELY PILOTED AIRCRAFT IN AIR TRAFFIC CONTROL SIMULATIONS Jens Alfredson*+**, Petra Hagström** *Linköping University, **Saab Aeronautics Jens.Alfredson@Saabgroup.com Keywords: Detect

More information

Smart Skies Distance-Rate-Time Mathematics In Air Traffic Control. NASA: With You When You Fly. NASA and the NextGen ATS. Agenda

Smart Skies Distance-Rate-Time Mathematics In Air Traffic Control. NASA: With You When You Fly. NASA and the NextGen ATS. Agenda National Aeronautics and Space Administration Distance-Rate-Time Mathematics In Air Traffic Control Grades 6-7 Standards- Based Free! Rebecca Green NCTM Regional Conference - Houston, T -Nov-14 1 NASA:

More information

Sustaining Six Sigma

Sustaining Six Sigma Sustaining Six Sigma September 27 th, 2006 Denver, Colorado Neeraj Katare Master Black Belt Sanjay Joshi Adaptive Manufacturing Consultant Business Agility & Six Sigma What is Six Sigma? Why Six Sigma?

More information

AUTOMATIC CHINESE PRONUNCIATION ERROR DETECTION USING SVM TRAINED WITH STRUCTURAL FEATURES

AUTOMATIC CHINESE PRONUNCIATION ERROR DETECTION USING SVM TRAINED WITH STRUCTURAL FEATURES AUTOMATIC CHINESE PRONUNCIATION ERROR DETECTION USING SVM TRAINED WITH STRUCTURAL FEATURES Tongmu Zhao 1, Akemi Hoshino 2, Masayuki Suzuki 1, Nobuaki Minematsu 1, Keikichi Hirose 1 1 University of Tokyo,

More information

Sample Exam Syllabus

Sample Exam Syllabus ISTQB Foundation Level 2011 Syllabus Version 2.9 Release Date: December 16th, 2017. Version.2.9 Page 1 of 26 Dec 16th, 2017 Copyright 2017 (hereinafter called ISTQB ). All rights reserved. The authors

More information

Measurement & Analysis in the Real World

Measurement & Analysis in the Real World Measurement & Analysis in the Real World Tools for Cleaning Messy Data Will Hayes SEI Robert Stoddard SEI Rhonda Brown SEI Software Solutions Conference 2015 November 16 18, 2015 Copyright 2015 Carnegie

More information

PUBLIC ΦΕΒΡΟΥΑΡΙΟΣ ELPCE English Language Proficiency Certification Examination for ATCOs ΥΠΑ/ΓΔΦΠΥΑ/Δ16

PUBLIC ΦΕΒΡΟΥΑΡΙΟΣ ELPCE English Language Proficiency Certification Examination for ATCOs ΥΠΑ/ΓΔΦΠΥΑ/Δ16 ΕΠΑΓΕ ELPCE ELPCE ENGLISH LANGUAGE PROFICIENCY CERTIFICATION EXAMINATION FOR AIR TRAFFIC CONTROLLER LICENCE (ATCO) and STUDENT AIR TRAFFIC CONTROLLER LICENCE ADV, ADI, APP, APS, ACP, ACS ΕΠAΓΕ ΕΞΕΤΑΣΗ

More information

NEWS ENGLISH LESSONS.com

NEWS ENGLISH LESSONS.com NEWS ENGLISH LESSONS.com U.S. air traffic controllers asleep on the job MANY FLASH AND ONLINE ACTIVITIES FOR THIS LESSON, PLUS A LISTENING, AT: http://www.newsenglishlessons.com/1104/110417-air_traffic_controllers.html

More information

Academic Vocabulary CONTENT BUILDER FOR THE PLC READING GRADE 7

Academic Vocabulary CONTENT BUILDER FOR THE PLC READING GRADE 7 Academic Vocabulary CONTENT BUILDER FOR THE PLC READING GRADE 7 : academic vocabulary directly taken from the standard : academic vocabulary used on the assessment (source released tests) STANDARD 7.6

More information

Learning user behaviours in real mobile domains

Learning user behaviours in real mobile domains Learning user behaviours in real mobile domains Andreas Markitanis, Domenico Corapi, Alessandra Russo, and Emil C. Lupu Department of Computing, Imperial College London {andreas.markitanis07,d.corapi,a.russo,e.c.lupu}@imperial.ac.uk

More information

Extending Sparse Classification Knowledge via NLP Analysis of Classification Descriptions

Extending Sparse Classification Knowledge via NLP Analysis of Classification Descriptions Extending Sparse Classification Knowledge via NLP Analysis of Classification Descriptions Attila Ondi 1, Jacob Staples 1, and Tony Stirtzinger 1 1 Securboration, Inc. 1050 W. NASA Blvd, Melbourne, FL,

More information

A Speech Act Model of Air Traffic Control Dialogue. by Karen Ward B. Sci., University of Oregon, 1978

A Speech Act Model of Air Traffic Control Dialogue. by Karen Ward B. Sci., University of Oregon, 1978 A Speech Act Model of Air Traffic Control Dialogue by Karen Ward B. Sci., University of Oregon, 1978 A thesis presented to the faculty of the Oregon Graduate Institute of Science & Technology in partial

More information

Introduction to Classification

Introduction to Classification Introduction to Classification Classification: Definition Given a collection of examples (training set ) Each example is represented by a set of features, sometimes called attributes Each example is to

More information

The cognitive domain deals with the student s recall or recognition of knowledge, in other words, the intellectual development of the learner.

The cognitive domain deals with the student s recall or recognition of knowledge, in other words, the intellectual development of the learner. Domains of Learning Beginning in 1948, a group of educators began the task of classifying a taxonomy of education goals and objectives. Their intention was to develop a classification system for educators

More information

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 19, NO. 4, MAY

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 19, NO. 4, MAY IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 19, NO. 4, MAY 2011 1015 Automatic Prediction of Children s Reading Ability for High-Level Literacy Assessment Matthew P. Black, Student

More information

AUTOMATIC LEARNING OBJECT CATEGORIZATION FOR INSTRUCTION USING AN ENHANCED LINEAR TEXT CLASSIFIER

AUTOMATIC LEARNING OBJECT CATEGORIZATION FOR INSTRUCTION USING AN ENHANCED LINEAR TEXT CLASSIFIER AUTOMATIC LEARNING OBJECT CATEGORIZATION FOR INSTRUCTION USING AN ENHANCED LINEAR TEXT CLASSIFIER THOMAS GEORGE KANNAMPALLIL School of Information Sciences and Technology, Pennsylvania State University,

More information

Classification of News Articles Using Named Entities with Named Entity Recognition by Neural Network

Classification of News Articles Using Named Entities with Named Entity Recognition by Neural Network Classification of News Articles Using Named Entities with Named Entity Recognition by Neural Network Nick Latourette and Hugh Cunningham 1. Introduction Our paper investigates the use of named entities

More information

Recognition Confidence Scoring for Use in Speech Understanding Systems Λ

Recognition Confidence Scoring for Use in Speech Understanding Systems Λ Recognition Confidence Scoring for Use in Speech Understanding Systems Λ Timothy J. Hazen, Theresa Burianek, Joseph Polifroni and Stephanie Seneff Spoken Language Systems Group MIT Laboratory for Computer

More information

Source Language Diagnostics for MT

Source Language Diagnostics for MT Source Language Diagnostics for MT Teruko Mitamura, Kathryn Baker, David Svoboda, and Eric Nyberg Language Technologies Institute Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 teruko,klb,svoboda,ehn

More information

Score Guide Version 8/ October 2017

Score Guide Version 8/ October 2017 Score Guide Version 8/ October 2017 Copyright Pearson Education Ltd 2017. All rights reserved; no part of this publication may be reproduced without the prior written permission of Pearson Education Ltd.

More information

Transportation of All Kinds In the Air, On the Land, or In the Water My Truck Is Stuck

Transportation of All Kinds In the Air, On the Land, or In the Water My Truck Is Stuck of All Kinds In the Air, On the Land, or In the Water My Truck Is Stuck Domain: Language, Cognitive, Socialization Goals: Child will increase the number of spoken or signed words in his/her vocabulary.

More information

Mark Scheme (Results) Summer Set B. Pearson Edexcel International GCSE in English as a Second Language (4ES0/01R)

Mark Scheme (Results) Summer Set B. Pearson Edexcel International GCSE in English as a Second Language (4ES0/01R) Scheme (Results) Summer 2015 Pearson Edexcel International GCSE in English as a Second Language (4ES0/01R) Set B Edexcel and BTEC Qualifications Edexcel and BTEC qualifications are awarded by Pearson,

More information

1 The project contains a clear statement of the task. For this level to be achieved, the task should be stated explicitly.

1 The project contains a clear statement of the task. For this level to be achieved, the task should be stated explicitly. Criterion A: Introduction In this context, the word task is defined as what the student is going to do ; the word plan is defined as how the student is going to do it. A statement of the task should appear

More information

Comparison of Voice and Text ATC Communications in the Cockpit for ESL Pilots

Comparison of Voice and Text ATC Communications in the Cockpit for ESL Pilots Dissertations and Theses Spring 2013 Comparison of Voice and Text ATC Communications in the Cockpit for ESL Pilots Shannon Marie Cummings Embry-Riddle Aeronautical University - Daytona Beach Follow this

More information

The Critical Components of Aviation English. Marjo Mitsutomi. University of Redlands. Kathleen O Brien

The Critical Components of Aviation English. Marjo Mitsutomi. University of Redlands. Kathleen O Brien Aviation English 1 Running head: AVIATION ENGLISH MODEL The Critical Components of Aviation English Marjo Mitsutomi University of Redlands Kathleen O Brien Federal Aviation Administration Flight Standards

More information

PALS 1-3 Frequently Asked Questions. Word Recognition

PALS 1-3 Frequently Asked Questions. Word Recognition PALS 1-3 Frequently Asked Questions Word Recognition What do I do if the list is too difficult for the student? If the student misses three out of the first five words, you may open up the list and ask

More information

ETS Automated Scoring and NLP Technologies

ETS Automated Scoring and NLP Technologies ETS Automated Scoring and NLP Technologies Using natural language processing (NLP) and psychometric methods to develop innovative scoring technologies The growth of the use of constructed-response tasks

More information

EF Language Learning Solutions for Schools. Diagnostic test guide

EF Language Learning Solutions for Schools. Diagnostic test guide EF Language Learning Solutions for Schools class.ef.com Diagnostic test guide DIAGNOSTIC TEST GUIDE, ABOUT About diagnostic tests. EF Class diagnostic tests can help you plan for the new school year by

More information

DECLARATIVE AND PROCEDURAL KNOWLEDGE IN TEACHING GRAMMAR. Bambang Suroso Muhammadiyah University of Purwokerto. Abstract

DECLARATIVE AND PROCEDURAL KNOWLEDGE IN TEACHING GRAMMAR. Bambang Suroso Muhammadiyah University of Purwokerto. Abstract 56 DECLARATIVE AND PROCEDURAL KNOWLEDGE IN TEACHING GRAMMAR Bambang Suroso Muhammadiyah University of Purwokerto Abstract When one has to teach a language, he should possess a vlew of what languages. One

More information

Segmentation evaluation metrics, a comparison grounded on prosodic and discourse units

Segmentation evaluation metrics, a comparison grounded on prosodic and discourse units Segmentation evaluation metrics, a comparison grounded on prosodic and discourse units Klim Peshkov, Laurent Prévot Aix Marseille Université & CNRS, Laboratoire Parole et Langage, Aix-En-Provence, France

More information

SHEEP, GOATS, LAMBS and WOLVES A Statistical Analysis of Speaker Performance in the NIST 1998 Speaker Recognition Evaluation

SHEEP, GOATS, LAMBS and WOLVES A Statistical Analysis of Speaker Performance in the NIST 1998 Speaker Recognition Evaluation SHEEP, GOATS, LAMBS and WOLVES A Statistical Analysis of Speaker Performance in the NIST 998 Speaker Recognition Evaluation George Doddington,2,3,5, Walter Liggett, Alvin Martin, Mark Przybocki, Douglas

More information

INTERNATIONAL CIVIL AVIATION ORGANIZATION EIGHTH MEETING OF CIVIL AVIATION AUTHORITIES OF THE SAM REGION (RAAC/8)

INTERNATIONAL CIVIL AVIATION ORGANIZATION EIGHTH MEETING OF CIVIL AVIATION AUTHORITIES OF THE SAM REGION (RAAC/8) RAAC/8-IP/18 INTERNATIONAL CIVIL AVIATION ORGANIZATION EIGHTH MEETING OF CIVIL AVIATION AUTHORITIES OF THE SAM REGION (RAAC/8) Agenda Item 8: Other Matters Buenos Aires, Argentina, 23 25 April, 2003 Status

More information

Florida Center for Reading Research Guidelines to Review Reading Intervention Programs

Florida Center for Reading Research Guidelines to Review Reading Intervention Programs A Correlation of to the Florida Center for Reading Research Guidelines to Review Reading Intervention Programs Kindergarten T/R-86B Introduction This document demonstrates how and where Scott Foresman

More information

Online Resources for Spanish Pronunciation

Online Resources for Spanish Pronunciation 1 Online Resources for Spanish Pronunciation WordReference, Forvo, UCLA, Linguee, Acapela Group WordReference http://www.wordreference.com/es/ While primarily known as a multilingual dictionary, this ever-updating

More information

Natural Language Processing. COMP-599 Sept 5, 2017

Natural Language Processing. COMP-599 Sept 5, 2017 Natural Language Processing COMP-599 Sept 5, 2017 Preliminaries Instructor: Jackie Chi Kit Cheung Time and Loc.: TR 16:05-17:25 in MAASS 217 Office hours: TAs: T 14:30-15:45 or by appointment in MC108N

More information

Negative News No More: Classifying News Article Headlines

Negative News No More: Classifying News Article Headlines Negative News No More: Classifying News Article Headlines Karianne Bergen and Leilani Gilpin kbergen@stanford.edu lgilpin@stanford.edu December 14, 2012 1 Introduction The goal of this project is to develop

More information

Bachelor in Business Administration - BBA The ESADE Admission Test

Bachelor in Business Administration - BBA The ESADE Admission Test Admission Test Example Bachelor in Business Administration - BBA The ESADE Admission Test The Admission Test consists of the following parts: Multiple choice test in three sections: 1. Sufficiency of information

More information

Language Testing. Types of Tests

Language Testing. Types of Tests Language Testing Types of Tests Why do we want to test? Assess Evaluate Test Assessment of people processes products environments Types of tests (test techniques): What comes to mind first? Multiple choice

More information

REVIEWERS GUIDE. For more information, contact: Michael Freedman /20/2005

REVIEWERS GUIDE. For more information, contact: Michael Freedman /20/2005 1/20/2005 REVIEWERS GUIDE For more information, contact: Michael Freedman mfreedman@thinkmap.com 212-381-0556 THINKMAP, INC. 155 SPRING STREET SUITE 3A. NEW YORK NY 10012 T. 212.285.8600 F. 212.285.8999.

More information

TExES English Language Arts and Reading 4 8 (117) Test at a Glance

TExES English Language Arts and Reading 4 8 (117) Test at a Glance TExES English Language Arts and Reading 4 8 (117) Test at a Glance See the test preparation manual for complete information about the test along with sample questions, study tips and preparation resources.

More information

Learning Bayesian Network Structures to Augment Aircraft Diagnostic Reference Models

Learning Bayesian Network Structures to Augment Aircraft Diagnostic Reference Models IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 1 Learning Bayesian Network Structures to Augment Aircraft Diagnostic Reference Models Daniel L. C. Mack, Gautam Biswas, Fellow, IEEE, Xenofon D.

More information

Using Linguistic Knowledge for Improving Automatic Speech Recognition Accuracy in Air Traffic Control

Using Linguistic Knowledge for Improving Automatic Speech Recognition Accuracy in Air Traffic Control Using Linguistic Knowledge for Improving Automatic Speech Recognition Accuracy in Air Traffic Control Master s Thesis in Computer Science Van Nhan Nguyen May 18, 2016 Halden, Norway www.hiof.no Abstract

More information

Test Information Booklet V1 09/06/2016

Test Information Booklet V1 09/06/2016 Test Information Booklet V1 09/06/2016 Contents Introduction 3 Test Validation 4 Test Coverage 7 Test Questions 8 Score Reporting 12 2 Introduction Placement: An Overview Placement is a fully online adaptive

More information

L16: Speaker recognition

L16: Speaker recognition L16: Speaker recognition Introduction Measurement of speaker characteristics Construction of speaker models Decision and performance Applications [This lecture is based on Rosenberg et al., 2008, in Benesty

More information

I. PREREQUISITE For information regarding prerequisites for this course, please refer to the Academic Course Catalog.

I. PREREQUISITE For information regarding prerequisites for this course, please refer to the Academic Course Catalog. Note: Course content may be changed, term to term, without notice. The information below is provided as a guide for course selection and is not binding in any form, and should not be used to purchase course

More information

On-Line Data Analytics

On-Line Data Analytics International Journal of Computer Applications in Engineering Sciences [VOL I, ISSUE III, SEPTEMBER 2011] [ISSN: 2231-4946] On-Line Data Analytics Yugandhar Vemulapalli #, Devarapalli Raghu *, Raja Jacob

More information

Mapping Transcripts to Handwritten Text

Mapping Transcripts to Handwritten Text Mapping Transcripts to Handwritten Text Chen Huang and Sargur N. Srihari CEDAR, Department of Computer Science and Engineering State University of New York at Buffalo E-Mail: {chuang5, srihari}@cedar.buffalo.edu

More information

Systems simulation with digital computers

Systems simulation with digital computers The general nature of digital sirnulation of a system is discussed. A machine-independent examination of the associated programming problem is conducted and illustrated by means of an example. Finally,

More information

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should

More information

Augmented Cognition. Bringing the Brain on Task

Augmented Cognition. Bringing the Brain on Task Augmented Cognition Dylan Schmorrow, Ph.D. LCDR, MSC, US Navy Program Manager, ITO more of Bringing the Brain on Task Objective: 1, 2, or 3 Order of Magnitude Improvement in Net Human- Machine Information

More information

INF5820/INF9820: Examples of exam questions

INF5820/INF9820: Examples of exam questions INF5820/INF9820: Examples of exam questions 1 Spoken dialogue 1. Turn-taking is a crucial part of conversational competence. What linguistic and extra-linguistic factors can influence how people take and

More information

Disclaimer. Copyright. Machine Learning Mastery With Weka

Disclaimer. Copyright. Machine Learning Mastery With Weka i Disclaimer The information contained within this ebook is strictly for educational purposes. If you wish to apply ideas contained in this ebook, you are taking full responsibility for your actions. The

More information

Combined systems for automatic phonetic transcription of proper nouns

Combined systems for automatic phonetic transcription of proper nouns Combined systems for automatic phonetic transcription of proper nouns A. Laurent 1,2, T. Merlin 1, S. Meignier 1, Y. Estève 1, P. Deléglise 1 1 Laboratoire d Informatique de l Université du Maine Le Mans,

More information

PRIVATE SCHOOL APPLICATION TO ADMINISTER FLORIDA STATEWIDE ASSESSMENTS IN

PRIVATE SCHOOL APPLICATION TO ADMINISTER FLORIDA STATEWIDE ASSESSMENTS IN PRIVATE SCHOOL APPLICATION TO ADMINISTER FLORIDA STATEWIDE ASSESSMENTS IN 2015-16 Attachment H: Appendix A (Accommodations) of the FSA Writing Component FT Test Administration Manual See question #19 in

More information

Syllabus for PORT 2301 Advanced Portuguese One Dr. Debora Zamorano

Syllabus for PORT 2301 Advanced Portuguese One Dr. Debora Zamorano Syllabus for PORT 2301 Advanced Portuguese One Dr. Debora Zamorano Spring 2011 Days/times: M/M/W 11:30-12:20 Office Hours: M/W: 9:00-10:20 1:30-4:20 & T: 1:10-2:50 Class: LART 122 Dr. Debora Zamorano Office:

More information

COURSE: ADVANCED SPANISH SPAN 350AN. 80 clock hours 3 weekly meetings 2 hours each

COURSE: ADVANCED SPANISH SPAN 350AN. 80 clock hours 3 weekly meetings 2 hours each COURSE: ADVANCED SPANISH SPAN 350AN 80 clock hours 3 weekly meetings 2 hours each Goals By the end of the course, students are expected to be able to: 1. Understand a wide variety of long texts with a

More information

Using Word Confusion Networks for Slot Filling in Spoken Language Understanding

Using Word Confusion Networks for Slot Filling in Spoken Language Understanding INTERSPEECH 2015 Using Word Confusion Networks for Slot Filling in Spoken Language Understanding Xiaohao Yang, Jia Liu Tsinghua National Laboratory for Information Science and Technology Department of

More information