# Word Recognition with Conditional Random Fields

Size: px
Start display at page:

## Transcription

1 Outline ord Recognition with Conditional Random Fields Jeremy Morris 2/05/2010 ord Recognition CRF Pilot System - TIDIGITS Larger Vocabulary - SJ Future ork 1 2 Conditional Random Fields (CRFs) Discriminative probabilistic sequence model Directly defines a posterior probability Y X) of a label sequence Y given a set of observations X exp ( isi ( x, yk ) jt j ( x, yk, yk 1)) k i j Y X ) Z( x) The form of the CRF model includes weighted state feature functions and weighted transition feature functions Both types of functions can be defined to incorporate observed inputs 3 4 Our previous work compared CRF models for phone recognition to HMM models *Signficantly (p<0.05) better than comparable Tandem 16mix triphone system (Morris & Fosler-Lussier 08) Accuracy CRF (phone classes) 69.92%* HMM Tandem16mix (phone classes) 69.34% CRF (phone classes +phonological features) 70.63%* HMM Tandem16mix (phone classes+ phonological features) 69.40% Problem: How do we make use of CRF classification for word recognition? Attempt to fit CRFs into current state-of-the-art models for speech recognition? Attempt to use CRFs directly? Each approach has its benefits Fitting CRFs into a standard framework lets us reuse existing code and ideas (Crandem system) A model that uses CRFs directly opens up new directions for investigation Requires some rethinking of the standard model for ASR 5 6 1

2 Review - ord Recognition X ) Problem: For a given input signal X, find the word string that maximizes X) Review - ord Recognition X ) ) arg max X ) arg max X ) Problem: For a given input signal X, find the word string that maximizes X) In an HMM, we would make this a generative problem 7 8 Review - ord Recognition X ) X ) ) Review - ord Recognition X ) X ) ) Problem: For a given input signal X, find the word string that maximizes X) In an HMM, we would make this a generative problem e can drop the X) because it does not affect the choice of e want to build phone models, not whole word models 9 10 Review - ord Recognition X ) X ) ) e want to build phone models, not whole word models so we marginalize over the phones X ) ) ) 11 Review - ord Recognition X ) X ) ) e want to build phone models, not whole word models so we marginalize over the phones X ) ) ) arg max X ) ) ), and look for the best sequence that fits these constraints 12 2

3 Review - ord Recognition X ) ) ) ord Recognition X ) ) ) Acoustic Acoustic Lexicon Language However - our CRFs model Φ X) rather than X Φ) This makes the formulation of the problem somewhat different ord Recognition X ) ord Recognition X), X), X ) X ) e want a formulation that makes use of Φ X) e want a formulation that makes use of Φ X) e can get that by marginalizing over the phone strings But the CRF as we formulate it doesn t give Φ X) directly ord Recognition, X ) X ) Φ here is a phone level assignment of phone labels CRF gives related quantity X) where is the frame level assignment of phone labels ord Recognition Frame level vs. Phone level Mapping from frame level to phone level may not be deterministic Example: The word OH with pronunciation /ow/ Consider this sequence of frame labels: ow ow ow ow ow ow ow This sequence can possibly be expanded many different ways for the word OH ( OH, OH OH, etc.)

4 ord Recognition Frame level vs. Phone segment level This problem occurs because we re using a single state to represent the phone /ow/ Phone either transitions to itself or transitions out to another phone e can change our model to a multi-state model and make this decision deterministic This brings us closer to a standard ASR HMM topology ow1 ow2 ow2 ow2 ow2 ow3 ow3 Now we can see a single OH in this utterance ord Recognition X ) P (, P (, X ) ( P ) Multi-state model gives us a deterministic mapping of -> Φ Each frame-level assignment has exactly one segment level assignment associated with it Potential pitfalls if the multi-state model is inappropriate for the features we are using ord Recognition X ), X ) X ),,, X ) ) ) ) Replacing Φ X) we now have a model with our CRF in it hat about Φ,X)? Conditional independence assumption gives Φ) ord Recognition X ), ) ) hat about Φ)? Non-deterministic across sequences of words Φ = / ah f eh r / =? a fair? affair? The more words in the string, the more possible combinations can arise ord Recognition ) ) ) ) Bayes Rule ) language model Φ ) dictionary model Φ) prior probability of phone sequences ord Recognition hat is Φ)? Prior probability over possible phone sequences Essentially acts as a phone fertility/penalty term lower probability sequences get a larger boost in weight than higher probability sequences Approximate this with a standard n-gram model Seed it with phone-level statistics drawn from the same corpus used for our language model

5 ord Recognition Pilot Experiment: TIDIGITS ) ) X ) arg max ),, ) Our final model incorporates all of these pieces together Benefit of this approach reuse of standard models Each element can be built as a finite state machine (FSM) Evaluation can be performed via FSM composition and best path evaluation as for HMM-based systems (Mohri & Riley, 2002) 25 First word recognition experiment TIDIGITS recognition Both isolated and strings of spoken digits, ZERO (or OH) to NINE Male and female speakers Training set 112 speakers total Random selection of 11 speakers held out as development set Remaining 101 speakers used for training as needed 26 Pilot Experiment: TIDIGITS X ) Important characteristics of the DIGITS problem: ) ) arg max ),, ) A given phone sequence maps to a single word sequence A uniform distribution over the words is assumed Φ) easy to implement directly as FSM Pilot Experiment: TIDIGITS Implementation Created a composed dictionary and language model FST No probabilistic weights applied to these FSTs assumption of uniform probability of any digit sequence Modified CRF code to allow composition of above FST with phone lattice Results scored using standard HTK tools Compared to a baseline HMM system trained on the same features Pilot Experiment: TIDIGITS Labels Unlike TIMIT, TIDIGITS is only labeled at the word level Phone labels were generated by force aligning the word labels using an HMM-trained, MFCC based system Features TIMIT-trained MLPs applied to TIDIGITS to create features for CRF and HMM training Pilot Experiment: Results ER HMM (triphone, 1 Gaussinan, ~4500 parameters) 1.26% HMM (triphone, 16 Gaussians ~120,000 paramters) 0.57% CRF (monophone, ~4200 parameters) 1.11% CRF (monophone, windowed, ~37000 parameters) 0.57% HMM (triphone, 16 Gaussians, MFCCs) 0.25% Basic CRF performance falls in line with HMM performance for a single Gaussian model Adding more parameters to the CRF enables the CRF to perform as well as the HMM on the same features

6 Larger Vocabulary all Street Journal 5K word vocabulary task Bigram language model MLPs trained on 75 speakers, 6488 utterances Cross-validated on 8 speakers, 650 utterances Development set of 10 speakers, 368 utterances for tuning purposes Results compared to HMM-Tandem baseline and HMM-MFCC baseline Larger Vocabulary Phone penalty model Φ) Constructed using the transcripts and the lexicon Currently implemented as a phone pair (bigram) model More complex model might lead to better estimates Larger Vocabulary Direct finite-state composition not feasible for this task State space grows too large too quickly Instead Viterbi decoding performed using the weighted finite-state models as constraints Time-synchronous beam pruning used to keep time and space usage reasonable Larger Vocabulary Initial Results ER HMM MFCC Baseline 9.3% HMM PLP Baseline 9.7% HMM Tandem MLP 9.1% CRF (phone) 11.3% CRF (phone windowed) 11.7% CRF (phone + phonological) 10.9% CRF (3state phone inputs) 12.4% CRF (3state phone + phono) 11.7% HMM PLP (monophone labels) 17.5% Preliminary numbers reported on development set only Next Steps Discussion Context Exploring ways to put more context into the CRF, either at the label level or at the feature level Feature selection Examine what features will help this model, especially features that may be useful for the CRF that are not useful for HMMs Phone penalty model Results reported with just a bigram phone model A more interesting model leads to more complexity but may lead to better results Currently examining trigram phone model to test the impact

7 References J. Lafferty et al, Conditional Random Fields: Probabilistic models for segmenting and labeling sequence data, Proc. ICML, 2001 A. Gunawardana et al, Hidden Conditional Random Fields for phone classification, Proc. Interspeech, 2005 J. Morris and E. Fosler-Lussier. Conditional Random Fields for Integrating Local Discriminative Classifiers, IEEE Transactions on Audio, Speech and Language Processing, 2008 M. Mohri et al, eighted finite-state transducers in speech recognition, Computer Speech and Language, 2002 Tandem HMM Generative probabilistic sequence model Uses outputs of a discriminative model (e.g. ANN MLPs) as input feature vectors for a standard HMM Tandem HMM ANN MLP classifiers are trained on labeled speech data Classifiers can be phone classifiers, phonological feature classifiers Classifiers output posterior probabilities for each frame of data E.g. X), where is the phone class label and X is the input speech feature vector Tandem HMM Posterior feature vectors are used by an HMM as inputs In practice, posteriors are not used directly Log posterior outputs or linear outputs are more frequently used linear here means outputs of the MLP with no application of a softmax function Since HMMs model phones as Gaussian mixtures, the goal is to make these outputs look more Gaussian Additionally, Principle Components Analysis (PCA) is applied to features to decorrelate features for diagonal covariance matrices Idea: Crandem Use a CRF model to create inputs to a Tandem-style HMM CRF labels provide a better per-frame accuracy than input MLPs e ve shown CRFs to provide better phone recognition than a Tandem system with the same inputs This suggests that we may get some gain from using CRF features in an HMM Idea: Crandem Problem: CRF output doesn t match MLP output MLP output is a per-frame vector of posteriors CRF outputs a probability across the entire sequence Solution: Use Forward-Backward algorithm to generate a vector of posterior probabilities

8 Forward-Backward Algorithm Similar to HMM forward-backward algorithm Used during CRF training Forward pass collects feature functions for the timesteps prior to the current timestep Backward pass collects feature functions for the timesteps following the current timestep Information from both passes are combined together to determine the probability of being in a given state at a particular timestep Forward-Backward Algorithm i, ti, t yi, t X ) Z( x) This form allows us to use the CRF to compute a vector of local posteriors y at any timestep t. e use this to generate features for a Tandem-style system Take log features, decorrelate with PCA Phone Recognition Pilot task phone recognition on TIMIT 61 feature MLPs trained on TIMIT, mapped down to 39 features for evaluation Crandem compared to Tandem and a standard PLP HMM baseline model As with previous CRF work, we use the outputs of an ANN MLP as inputs to our CRF Phone class attributes Detector outputs describe the phone label associated with a portion of the speech signal /t/, /d/, /aa/, etc. 45 Results (Fosler-Lussier & Morris 08) Phone Accuracy PLP HMM reference 68.1% Tandem 70.8% CRF 69.9% Crandem log 71.1% * Significantly (p<0.05) improvement at 0.6% difference between models 46 ord Recognition Second task ord recognition on SJ0 Dictionary for word recognition has 54 distinct phones instead of 48 New CRFs and MLPs trained to provide input features MLPs and CRFs trained on SJ0 corpus of read speech No phone level assignments, only word transcripts Initial alignments from HMM forced alignment of MFCC features Compare Crandem baseline to Tandem and original MFCC baselines Initial Results ER MFCC HMM reference 9.12% Tandem MLP (39) 8.95% Crandem (19) (1 epoch) 8.85% Crandem (19) (10 epochs) 9.57% Crandem (19) (20 epochs) 9.98% 47 * Significant (p 0.05) improvement at roughly 1% difference between models 48 8

9 ord Recognition CRF performs about the same as the baseline systems But further training of the CRF tends to degrade the result of the Crandem system hy? First thought maybe the phone recognition results are deteriorating (overtraining) Initial Results Phone Accuracy MFCC HMM reference 70.09% Tandem MLP (39) 75.58% Crandem (19) (1 epoch) 72.77% Crandem (19) (10 epochs) 72.81% Crandem (19) (20 epochs) 72.93% 49 * Significant (p 0.05) improvement at roughly 0.07% difference between models 50 ord Recognition Further training of the CRF tends to degrade the result of the Crandem system hy? First thought maybe the phone recognition results are deteriorating (overtraining) Not the case Next thought examine the pattern of errors between iterations Initial Results Crandem (1 epoch) Crandem (10 epochs) Shared Errors New Errors (1->10) Total Errors Insertions Deletions Subs * (102) 261** (211) * 29 deletions are substitutions in one model and deletions in the other **50 of these subs are different words between the epoch 1 and epoch 10 models 52 ord Recognition Training the CRF tends to degrade the result of the Crandem system hy? First thought maybe the phone recognition results are deteriorating (overtraining) Not the case Next thought examine the pattern of errors between iterations There doesn t seem to be much of a pattern here, other than a jump in substitutions ord identity doesn t give a clue similar words wrong in both lists ord Recognition Further training of the CRF tends to degrade the result of the Crandem system hy? Current thought perhaps the reduction in scores of the correct result is impacting the overall score This appears to be happening in at least some cases, though it is not sufficient to explain everything

10 ord Recognition ord Recognition MARCH vs. LARGE Iteration m l en em m em l en m em l hh m em l w m aa em l m aa em w l Iteration m em en n l m em l en m em l en m em l aa m aa em l m aa em l MARCH vs. LARGE - logspace Iteration m l en em m em l en m em l hh m em l w m aa em l m aa em w l Iteration m em en n l m em l en m em l en m em l aa m aa em l m aa em l ord Recognition Additional issues Crandem results sensitive to format of input data Posterior probability inputs to the CRF give very poor results on word recognition. I suspect is related to the same issues described previously Crandem results also require a much smaller vector after PCA MLP uses 39 features Crandem only does well once we reduce to 19 features However, phone recognition results improve if we use 39 features in the Crandem system (72.77% -> 74.22%) 57 10

### Segmental Conditional Random Fields with Deep Neural Networks as Acoustic Models for First-Pass Word Recognition

Segmental Conditional Random Fields with Deep Neural Networks as Acoustic Models for First-Pass Word Recognition Yanzhang He, Eric Fosler-Lussier Department of Computer Science and Engineering The hio

### Learning Methods in Multilingual Speech Recognition

Learning Methods in Multilingual Speech Recognition Hui Lin Department of Electrical Engineering University of Washington Seattle, WA 98125 linhui@u.washington.edu Li Deng, Jasha Droppo, Dong Yu, and Alex

### BAUM-WELCH TRAINING FOR SEGMENT-BASED SPEECH RECOGNITION. Han Shu, I. Lee Hetherington, and James Glass

BAUM-WELCH TRAINING FOR SEGMENT-BASED SPEECH RECOGNITION Han Shu, I. Lee Hetherington, and James Glass Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology Cambridge,

### A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation

A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation SLSP-2016 October 11-12 Natalia Tomashenko 1,2,3 natalia.tomashenko@univ-lemans.fr Yuri Khokhlov 3 khokhlov@speechpro.com Yannick

### Speech Segmentation Using Probabilistic Phonetic Feature Hierarchy and Support Vector Machines

Speech Segmentation Using Probabilistic Phonetic Feature Hierarchy and Support Vector Machines Amit Juneja and Carol Espy-Wilson Department of Electrical and Computer Engineering University of Maryland,

### Modeling function word errors in DNN-HMM based LVCSR systems

Modeling function word errors in DNN-HMM based LVCSR systems Melvin Jose Johnson Premkumar, Ankur Bapna and Sree Avinash Parchuri Department of Computer Science Department of Electrical Engineering Stanford

### BUILDING CONTEXT-DEPENDENT DNN ACOUSTIC MODELS USING KULLBACK-LEIBLER DIVERGENCE-BASED STATE TYING

BUILDING CONTEXT-DEPENDENT DNN ACOUSTIC MODELS USING KULLBACK-LEIBLER DIVERGENCE-BASED STATE TYING Gábor Gosztolya 1, Tamás Grósz 1, László Tóth 1, David Imseng 2 1 MTA-SZTE Research Group on Artificial

### Phonetic- and Speaker-Discriminant Features for Speaker Recognition. Research Project

Phonetic- and Speaker-Discriminant Features for Speaker Recognition by Lara Stoll Research Project Submitted to the Department of Electrical Engineering and Computer Sciences, University of California

### STUDIES WITH FABRICATED SWITCHBOARD DATA: EXPLORING SOURCES OF MODEL-DATA MISMATCH

STUDIES WITH FABRICATED SWITCHBOARD DATA: EXPLORING SOURCES OF MODEL-DATA MISMATCH Don McAllaster, Larry Gillick, Francesco Scattone, Mike Newman Dragon Systems, Inc. 320 Nevada Street Newton, MA 02160

### Modeling function word errors in DNN-HMM based LVCSR systems

Modeling function word errors in DNN-HMM based LVCSR systems Melvin Jose Johnson Premkumar, Ankur Bapna and Sree Avinash Parchuri Department of Computer Science Department of Electrical Engineering Stanford

### Speech Recognition at ICSI: Broadcast News and beyond

Speech Recognition at ICSI: Broadcast News and beyond Dan Ellis International Computer Science Institute, Berkeley CA Outline 1 2 3 The DARPA Broadcast News task Aspects of ICSI

### Autoregressive product of multi-frame predictions can improve the accuracy of hybrid models

Autoregressive product of multi-frame predictions can improve the accuracy of hybrid models Navdeep Jaitly 1, Vincent Vanhoucke 2, Geoffrey Hinton 1,2 1 University of Toronto 2 Google Inc. ndjaitly@cs.toronto.edu,

### Calibration of Confidence Measures in Speech Recognition

Submitted to IEEE Trans on Audio, Speech, and Language, July 2010 1 Calibration of Confidence Measures in Speech Recognition Dong Yu, Senior Member, IEEE, Jinyu Li, Member, IEEE, Li Deng, Fellow, IEEE

### ADVANCES IN DEEP NEURAL NETWORK APPROACHES TO SPEAKER RECOGNITION

ADVANCES IN DEEP NEURAL NETWORK APPROACHES TO SPEAKER RECOGNITION Mitchell McLaren 1, Yun Lei 1, Luciana Ferrer 2 1 Speech Technology and Research Laboratory, SRI International, California, USA 2 Departamento

### A NOVEL SCHEME FOR SPEAKER RECOGNITION USING A PHONETICALLY-AWARE DEEP NEURAL NETWORK. Yun Lei Nicolas Scheffer Luciana Ferrer Mitchell McLaren

A NOVEL SCHEME FOR SPEAKER RECOGNITION USING A PHONETICALLY-AWARE DEEP NEURAL NETWORK Yun Lei Nicolas Scheffer Luciana Ferrer Mitchell McLaren Speech Technology and Research Laboratory, SRI International,

### Speech Recognition using Acoustic Landmarks and Binary Phonetic Feature Classifiers

Speech Recognition using Acoustic Landmarks and Binary Phonetic Feature Classifiers October 31, 2003 Amit Juneja Department of Electrical and Computer Engineering University of Maryland, College Park,

### INVESTIGATION OF UNSUPERVISED ADAPTATION OF DNN ACOUSTIC MODELS WITH FILTER BANK INPUT

INVESTIGATION OF UNSUPERVISED ADAPTATION OF DNN ACOUSTIC MODELS WITH FILTER BANK INPUT Takuya Yoshioka,, Anton Ragni, Mark J. F. Gales Cambridge University Engineering Department, Cambridge, UK NTT Communication

### On the Formation of Phoneme Categories in DNN Acoustic Models

On the Formation of Phoneme Categories in DNN Acoustic Models Tasha Nagamine Department of Electrical Engineering, Columbia University T. Nagamine Motivation Large performance gap between humans and state-

### Semi-Supervised GMM and DNN Acoustic Model Training with Multi-system Combination and Confidence Re-calibration

INTERSPEECH 2013 Semi-Supervised GMM and DNN Acoustic Model Training with Multi-system Combination and Confidence Re-calibration Yan Huang, Dong Yu, Yifan Gong, and Chaojun Liu Microsoft Corporation, One

### Lecture 9: Speech Recognition

EE E6820: Speech & Audio Processing & Recognition Lecture 9: Speech Recognition 1 Recognizing speech 2 Feature calculation Dan Ellis Michael Mandel 3 Sequence

### Improvements to the Pruning Behavior of DNN Acoustic Models

Improvements to the Pruning Behavior of DNN Acoustic Models Matthias Paulik Apple Inc., Infinite Loop, Cupertino, CA 954 mpaulik@apple.com Abstract This paper examines two strategies that positively influence

### Artificial Neural Networks written examination

1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box 337 751 05 Uppsala Artificial Neural Networks written examination Monday, May 15, 2006 9 00-14

### IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 17, NO. 3, MARCH

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 17, NO. 3, MARCH 2009 423 Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition George

### Investigation on Mandarin Broadcast News Speech Recognition

Investigation on Mandarin Broadcast News Speech Recognition Mei-Yuh Hwang 1, Xin Lei 1, Wen Wang 2, Takahiro Shinozaki 1 1 Univ. of Washington, Dept. of Electrical Engineering, Seattle, WA 98195 USA 2

### Eli Yamamoto, Satoshi Nakamura, Kiyohiro Shikano. Graduate School of Information Science, Nara Institute of Science & Technology

ISCA Archive SUBJECTIVE EVALUATION FOR HMM-BASED SPEECH-TO-LIP MOVEMENT SYNTHESIS Eli Yamamoto, Satoshi Nakamura, Kiyohiro Shikano Graduate School of Information Science, Nara Institute of Science & Technology

### Lecture 1: Machine Learning Basics

1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

### Probabilistic Latent Semantic Analysis

Probabilistic Latent Semantic Analysis Thomas Hofmann Presentation by Ioannis Pavlopoulos & Andreas Damianou for the course of Data Mining & Exploration 1 Outline Latent Semantic Analysis o Need o Overview

### A study of speaker adaptation for DNN-based speech synthesis

A study of speaker adaptation for DNN-based speech synthesis Zhizheng Wu, Pawel Swietojanski, Christophe Veaux, Steve Renals, Simon King The Centre for Speech Technology Research (CSTR) University of Edinburgh,

### PREDICTING SPEECH RECOGNITION CONFIDENCE USING DEEP LEARNING WITH WORD IDENTITY AND SCORE FEATURES

PREDICTING SPEECH RECOGNITION CONFIDENCE USING DEEP LEARNING WITH WORD IDENTITY AND SCORE FEATURES Po-Sen Huang, Kshitiz Kumar, Chaojun Liu, Yifan Gong, Li Deng Department of Electrical and Computer Engineering,

### WHEN THERE IS A mismatch between the acoustic

808 IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 14, NO. 3, MAY 2006 Optimization of Temporal Filters for Constructing Robust Features in Speech Recognition Jeih-Weih Hung, Member,

### Generative models and adversarial training

Day 4 Lecture 1 Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research Fellow Insight Centre for Data Analytics Dublin City University What is a generative model?

### Speech Emotion Recognition Using Support Vector Machine

Speech Emotion Recognition Using Support Vector Machine Yixiong Pan, Peipei Shen and Liping Shen Department of Computer Technology Shanghai JiaoTong University, Shanghai, China panyixiong@sjtu.edu.cn,

### Switchboard Language Model Improvement with Conversational Data from Gigaword

Katholieke Universiteit Leuven Faculty of Engineering Master in Artificial Intelligence (MAI) Speech and Language Technology (SLT) Switchboard Language Model Improvement with Conversational Data from Gigaword

### Experiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling

Experiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling Notebook for PAN at CLEF 2013 Andrés Alfonso Caurcel Díaz 1 and José María Gómez Hidalgo 2 1 Universidad

### Role of Pausing in Text-to-Speech Synthesis for Simultaneous Interpretation

Role of Pausing in Text-to-Speech Synthesis for Simultaneous Interpretation Vivek Kumar Rangarajan Sridhar, John Chen, Srinivas Bangalore, Alistair Conkie AT&T abs - Research 180 Park Avenue, Florham Park,

### 2/15/13. POS Tagging Problem. Part-of-Speech Tagging. Example English Part-of-Speech Tagsets. More Details of the Problem. Typical Problem Cases

POS Tagging Problem Part-of-Speech Tagging L545 Spring 203 Given a sentence W Wn and a tagset of lexical categories, find the most likely tag T..Tn for each word in the sentence Example Secretariat/P is/vbz

### Unsupervised Acoustic Model Training for Simultaneous Lecture Translation in Incremental and Batch Mode

Unsupervised Acoustic Model Training for Simultaneous Lecture Translation in Incremental and Batch Mode Diploma Thesis of Michael Heck At the Department of Informatics Karlsruhe Institute of Technology

### Analysis of Emotion Recognition System through Speech Signal Using KNN & GMM Classifier

IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) e-issn: 2278-2834,p- ISSN: 2278-8735.Volume 10, Issue 2, Ver.1 (Mar - Apr.2015), PP 55-61 www.iosrjournals.org Analysis of Emotion

### Distributed Learning of Multilingual DNN Feature Extractors using GPUs

Distributed Learning of Multilingual DNN Feature Extractors using GPUs Yajie Miao, Hao Zhang, Florian Metze Language Technologies Institute, School of Computer Science, Carnegie Mellon University Pittsburgh,

### Corrective Feedback and Persistent Learning for Information Extraction

Corrective Feedback and Persistent Learning for Information Extraction Aron Culotta a, Trausti Kristjansson b, Andrew McCallum a, Paul Viola c a Dept. of Computer Science, University of Massachusetts,

### Unvoiced Landmark Detection for Segment-based Mandarin Continuous Speech Recognition

Unvoiced Landmark Detection for Segment-based Mandarin Continuous Speech Recognition Hua Zhang, Yun Tang, Wenju Liu and Bo Xu National Laboratory of Pattern Recognition Institute of Automation, Chinese

### A Neural Network GUI Tested on Text-To-Phoneme Mapping

A Neural Network GUI Tested on Text-To-Phoneme Mapping MAARTEN TROMPPER Universiteit Utrecht m.f.a.trompper@students.uu.nl Abstract Text-to-phoneme (T2P) mapping is a necessary step in any speech synthesis

### Human Emotion Recognition From Speech

RESEARCH ARTICLE OPEN ACCESS Human Emotion Recognition From Speech Miss. Aparna P. Wanare*, Prof. Shankar N. Dandare *(Department of Electronics & Telecommunication Engineering, Sant Gadge Baba Amravati

### DIRECT ADAPTATION OF HYBRID DNN/HMM MODEL FOR FAST SPEAKER ADAPTATION IN LVCSR BASED ON SPEAKER CODE

2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) DIRECT ADAPTATION OF HYBRID DNN/HMM MODEL FOR FAST SPEAKER ADAPTATION IN LVCSR BASED ON SPEAKER CODE Shaofei Xue 1

### The NICT/ATR speech synthesis system for the Blizzard Challenge 2008

The NICT/ATR speech synthesis system for the Blizzard Challenge 2008 Ranniery Maia 1,2, Jinfu Ni 1,2, Shinsuke Sakai 1,2, Tomoki Toda 1,3, Keiichi Tokuda 1,4 Tohru Shimizu 1,2, Satoshi Nakamura 1,2 1 National

### Softprop: Softmax Neural Network Backpropagation Learning

Softprop: Softmax Neural Networ Bacpropagation Learning Michael Rimer Computer Science Department Brigham Young University Provo, UT 84602, USA E-mail: mrimer@axon.cs.byu.edu Tony Martinez Computer Science

### Deep Neural Network Language Models

Deep Neural Network Language Models Ebru Arısoy, Tara N. Sainath, Brian Kingsbury, Bhuvana Ramabhadran IBM T.J. Watson Research Center Yorktown Heights, NY, 10598, USA {earisoy, tsainath, bedk, bhuvana}@us.ibm.com

### DOMAIN MISMATCH COMPENSATION FOR SPEAKER RECOGNITION USING A LIBRARY OF WHITENERS. Elliot Singer and Douglas Reynolds

DOMAIN MISMATCH COMPENSATION FOR SPEAKER RECOGNITION USING A LIBRARY OF WHITENERS Elliot Singer and Douglas Reynolds Massachusetts Institute of Technology Lincoln Laboratory {es,dar}@ll.mit.edu ABSTRACT

### Learning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models

Learning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models Stephan Gouws and GJ van Rooyen MIH Medialab, Stellenbosch University SOUTH AFRICA {stephan,gvrooyen}@ml.sun.ac.za

### Likelihood-Maximizing Beamforming for Robust Hands-Free Speech Recognition

MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Likelihood-Maximizing Beamforming for Robust Hands-Free Speech Recognition Seltzer, M.L.; Raj, B.; Stern, R.M. TR2004-088 December 2004 Abstract

### Automatic Pronunciation Checker

Institut für Technische Informatik und Kommunikationsnetze Eidgenössische Technische Hochschule Zürich Swiss Federal Institute of Technology Zurich Ecole polytechnique fédérale de Zurich Politecnico federale

### OCR for Arabic using SIFT Descriptors With Online Failure Prediction

OCR for Arabic using SIFT Descriptors With Online Failure Prediction Andrey Stolyarenko, Nachum Dershowitz The Blavatnik School of Computer Science Tel Aviv University Tel Aviv, Israel Email: stloyare@tau.ac.il,

### Robust Speech Recognition using DNN-HMM Acoustic Model Combining Noise-aware training with Spectral Subtraction

INTERSPEECH 2015 Robust Speech Recognition using DNN-HMM Acoustic Model Combining Noise-aware training with Spectral Subtraction Akihiro Abe, Kazumasa Yamamoto, Seiichi Nakagawa Department of Computer

### Analysis of Speech Recognition Models for Real Time Captioning and Post Lecture Transcription

Analysis of Speech Recognition Models for Real Time Captioning and Post Lecture Transcription Wilny Wilson.P M.Tech Computer Science Student Thejus Engineering College Thrissur, India. Sindhu.S Computer

### PHONETIC DISTANCE BASED ACCENT CLASSIFIER TO IDENTIFY PRONUNCIATION VARIANTS AND OOV WORDS

PHONETIC DISTANCE BASED ACCENT CLASSIFIER TO IDENTIFY PRONUNCIATION VARIANTS AND OOV WORDS Akella Amarendra Babu 1 *, Ramadevi Yellasiri 2 and Akepogu Ananda Rao 3 1 JNIAS, JNT University Anantapur, Ananthapuramu,

### CS Machine Learning

CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing

### Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur

Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should

### LEARNING A SEMANTIC PARSER FROM SPOKEN UTTERANCES. Judith Gaspers and Philipp Cimiano

LEARNING A SEMANTIC PARSER FROM SPOKEN UTTERANCES Judith Gaspers and Philipp Cimiano Semantic Computing Group, CITEC, Bielefeld University {jgaspers cimiano}@cit-ec.uni-bielefeld.de ABSTRACT Semantic parsers

### SEMI-SUPERVISED ENSEMBLE DNN ACOUSTIC MODEL TRAINING

SEMI-SUPERVISED ENSEMBLE DNN ACOUSTIC MODEL TRAINING Sheng Li 1, Xugang Lu 2, Shinsuke Sakai 1, Masato Mimura 1 and Tatsuya Kawahara 1 1 School of Informatics, Kyoto University, Sakyo-ku, Kyoto 606-8501,

### Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures

Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures Alex Graves and Jürgen Schmidhuber IDSIA, Galleria 2, 6928 Manno-Lugano, Switzerland TU Munich, Boltzmannstr.

### International Journal of Advanced Networking Applications (IJANA) ISSN No. :

International Journal of Advanced Networking Applications (IJANA) ISSN No. : 0975-0290 34 A Review on Dysarthric Speech Recognition Megha Rughani Department of Electronics and Communication, Marwadi Educational

### Rule Learning With Negation: Issues Regarding Effectiveness

Rule Learning With Negation: Issues Regarding Effectiveness S. Chua, F. Coenen, G. Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX Liverpool, United

### AUTOMATIC DETECTION OF PROLONGED FRICATIVE PHONEMES WITH THE HIDDEN MARKOV MODELS APPROACH 1. INTRODUCTION

JOURNAL OF MEDICAL INFORMATICS & TECHNOLOGIES Vol. 11/2007, ISSN 1642-6037 Marek WIŚNIEWSKI *, Wiesława KUNISZYK-JÓŹKOWIAK *, Elżbieta SMOŁKA *, Waldemar SUSZYŃSKI * HMM, recognition, speech, disorders

### Using Articulatory Features and Inferred Phonological Segments in Zero Resource Speech Processing

Using Articulatory Features and Inferred Phonological Segments in Zero Resource Speech Processing Pallavi Baljekar, Sunayana Sitaram, Prasanna Kumar Muthukumar, and Alan W Black Carnegie Mellon University,

### International Journal of Computational Intelligence and Informatics, Vol. 1 : No. 4, January - March 2012

Text-independent Mono and Cross-lingual Speaker Identification with the Constraint of Limited Data Nagaraja B G and H S Jayanna Department of Information Science and Engineering Siddaganga Institute of

### NCU IISR English-Korean and English-Chinese Named Entity Transliteration Using Different Grapheme Segmentation Approaches

NCU IISR English-Korean and English-Chinese Named Entity Transliteration Using Different Grapheme Segmentation Approaches Yu-Chun Wang Chun-Kai Wu Richard Tzong-Han Tsai Department of Computer Science

### Unsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model

Unsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model Xinying Song, Xiaodong He, Jianfeng Gao, Li Deng Microsoft Research, One Microsoft Way, Redmond, WA 98052, U.S.A.

### Word Segmentation of Off-line Handwritten Documents

Word Segmentation of Off-line Handwritten Documents Chen Huang and Sargur N. Srihari {chuang5, srihari}@cedar.buffalo.edu Center of Excellence for Document Analysis and Recognition (CEDAR), Department

### Extracting Opinion Expressions and Their Polarities Exploration of Pipelines and Joint Models

Extracting Opinion Expressions and Their Polarities Exploration of Pipelines and Joint Models Richard Johansson and Alessandro Moschitti DISI, University of Trento Via Sommarive 14, 38123 Trento (TN),

### SARDNET: A Self-Organizing Feature Map for Sequences

SARDNET: A Self-Organizing Feature Map for Sequences Daniel L. James and Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 dljames,risto~cs.utexas.edu

### Detecting English-French Cognates Using Orthographic Edit Distance

Detecting English-French Cognates Using Orthographic Edit Distance Qiongkai Xu 1,2, Albert Chen 1, Chang i 1 1 The Australian National University, College of Engineering and Computer Science 2 National

### Semi-Supervised Face Detection

Semi-Supervised Face Detection Nicu Sebe, Ira Cohen 2, Thomas S. Huang 3, Theo Gevers Faculty of Science, University of Amsterdam, The Netherlands 2 HP Research Labs, USA 3 Beckman Institute, University

### Non intrusive multi-biometrics on a mobile device: a comparison of fusion techniques

Non intrusive multi-biometrics on a mobile device: a comparison of fusion techniques Lorene Allano 1*1, Andrew C. Morris 2, Harin Sellahewa 3, Sonia Garcia-Salicetti 1, Jacques Koreman 2, Sabah Jassim

### ACOUSTIC EVENT DETECTION IN REAL LIFE RECORDINGS

ACOUSTIC EVENT DETECTION IN REAL LIFE RECORDINGS Annamaria Mesaros 1, Toni Heittola 1, Antti Eronen 2, Tuomas Virtanen 1 1 Department of Signal Processing Tampere University of Technology Korkeakoulunkatu

### Semi-supervised methods of text processing, and an application to medical concept extraction. Yacine Jernite Text-as-Data series September 17.

Semi-supervised methods of text processing, and an application to medical concept extraction Yacine Jernite Text-as-Data series September 17. 2015 What do we want from text? 1. Extract information 2. Link

### Chinese Language Parsing with Maximum-Entropy-Inspired Parser

Chinese Language Parsing with Maximum-Entropy-Inspired Parser Heng Lian Brown University Abstract The Chinese language has many special characteristics that make parsing difficult. The performance of state-of-the-art

### Vimala.C Project Fellow, Department of Computer Science Avinashilingam Institute for Home Science and Higher Education and Women Coimbatore, India

World of Computer Science and Information Technology Journal (WCSIT) ISSN: 2221-0741 Vol. 2, No. 1, 1-7, 2012 A Review on Challenges and Approaches Vimala.C Project Fellow, Department of Computer Science

### Discriminative Learning of Beam-Search Heuristics for Planning

Discriminative Learning of Beam-Search Heuristics for Planning Yuehua Xu School of EECS Oregon State University Corvallis,OR 97331 xuyu@eecs.oregonstate.edu Alan Fern School of EECS Oregon State University

### Knowledge Transfer in Deep Convolutional Neural Nets

Knowledge Transfer in Deep Convolutional Neural Nets Steven Gutstein, Olac Fuentes and Eric Freudenthal Computer Science Department University of Texas at El Paso El Paso, Texas, 79968, U.S.A. Abstract

### Large vocabulary off-line handwriting recognition: A survey

Pattern Anal Applic (2003) 6: 97 121 DOI 10.1007/s10044-002-0169-3 ORIGINAL ARTICLE A. L. Koerich, R. Sabourin, C. Y. Suen Large vocabulary off-line handwriting recognition: A survey Received: 24/09/01

### Language Model and Grammar Extraction Variation in Machine Translation

Language Model and Grammar Extraction Variation in Machine Translation Vladimir Eidelman, Chris Dyer, and Philip Resnik UMIACS Laboratory for Computational Linguistics and Information Processing Department

### WE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT

WE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT PRACTICAL APPLICATIONS OF RANDOM SAMPLING IN ediscovery By Matthew Verga, J.D. INTRODUCTION Anyone who spends ample time working

### Characterizing and Processing Robot-Directed Speech

Characterizing and Processing Robot-Directed Speech Paulina Varchavskaia, Paul Fitzpatrick, Cynthia Breazeal AI Lab, MIT, Cambridge, USA [paulina,paulfitz,cynthia]@ai.mit.edu Abstract. Speech directed

### Learning From the Past with Experiment Databases

Learning From the Past with Experiment Databases Joaquin Vanschoren 1, Bernhard Pfahringer 2, and Geoff Holmes 2 1 Computer Science Dept., K.U.Leuven, Leuven, Belgium 2 Computer Science Dept., University

### Lecture 10: Reinforcement Learning

Lecture 1: Reinforcement Learning Cognitive Systems II - Machine Learning SS 25 Part III: Learning Programs and Strategies Q Learning, Dynamic Programming Lecture 1: Reinforcement Learning p. Motivation

### Speaker recognition using universal background model on YOHO database

Aalborg University Master Thesis project Speaker recognition using universal background model on YOHO database Author: Alexandre Majetniak Supervisor: Zheng-Hua Tan May 31, 2011 The Faculties of Engineering,

### Disambiguation of Thai Personal Name from Online News Articles

Disambiguation of Thai Personal Name from Online News Articles Phaisarn Sutheebanjard Graduate School of Information Technology Siam University Bangkok, Thailand mr.phaisarn@gmail.com Abstract Since online

### Twitter Sentiment Classification on Sanders Data using Hybrid Approach

IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 4, Ver. I (July Aug. 2015), PP 118-123 www.iosrjournals.org Twitter Sentiment Classification on Sanders

### An Online Handwriting Recognition System For Turkish

An Online Handwriting Recognition System For Turkish Esra Vural, Hakan Erdogan, Kemal Oflazer, Berrin Yanikoglu Sabanci University, Tuzla, Istanbul, Turkey 34956 ABSTRACT Despite recent developments in

### Edinburgh Research Explorer

Edinburgh Research Explorer Personalising speech-to-speech translation Citation for published version: Dines, J, Liang, H, Saheer, L, Gibson, M, Byrne, W, Oura, K, Tokuda, K, Yamagishi, J, King, S, Wester,

### Python Machine Learning

Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

### arxiv: v1 [cs.cl] 2 Apr 2017

Word-Alignment-Based Segment-Level Machine Translation Evaluation using Word Embeddings Junki Matsuo and Mamoru Komachi Graduate School of System Design, Tokyo Metropolitan University, Japan matsuo-junki@ed.tmu.ac.jp,

### Specification and Evaluation of Machine Translation Toy Systems - Criteria for laboratory assignments

Specification and Evaluation of Machine Translation Toy Systems - Criteria for laboratory assignments Cristina Vertan, Walther v. Hahn University of Hamburg, Natural Language Systems Division Hamburg,

### Class-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification

Class-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification Tomi Kinnunen and Ismo Kärkkäinen University of Joensuu, Department of Computer Science, P.O. Box 111, 80101 JOENSUU,

### Firms and Markets Saturdays Summer I 2014

PRELIMINARY DRAFT VERSION. SUBJECT TO CHANGE. Firms and Markets Saturdays Summer I 2014 Professor Thomas Pugel Office: Room 11-53 KMC E-mail: tpugel@stern.nyu.edu Tel: 212-998-0918 Fax: 212-995-4212 This

### Speech Recognition by Indexing and Sequencing

International Journal of Computer Information Systems and Industrial Management Applications. ISSN 215-7988 Volume 4 (212) pp. 358 365 c MIR Labs, www.mirlabs.net/ijcisim/index.html Speech Recognition

### Proceedings of Meetings on Acoustics

Proceedings of Meetings on Acoustics Volume 19, 2013 http://acousticalsociety.org/ ICA 2013 Montreal Montreal, Canada 2-7 June 2013 Speech Communication Session 2aSC: Linking Perception and Production

### Course Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE

EE-589 Introduction to Neural Assistant Prof. Dr. Turgay IBRIKCI Room # 305 (322) 338 6868 / 139 Wensdays 9:00-12:00 Course Outline The course is divided in two parts: theory and practice. 1. Theory covers