The CSTR entry to the Blizzard Challenge 2017

Similar documents
arxiv: v1 [cs.dl] 22 Dec 2016

Natural language processing implementation on Romanian ChatBot

E-LEARNING USABILITY: A LEARNER-ADAPTED APPROACH BASED ON THE EVALUATION OF LEANER S PREFERENCES. Valentina Terzieva, Yuri Pavlov, Rumen Andreev

Management Science Letters

Consortium: North Carolina Community Colleges

A study of speaker adaptation for DNN-based speech synthesis

'Norwegian University of Science and Technology, Department of Computer and Information Science

Fuzzy Reference Gain-Scheduling Approach as Intelligent Agents: FRGS Agent

CONSTITUENT VOICE TECHNICAL NOTE 1 INTRODUCING Version 1.1, September 2014

HANDBOOK. Career Center Handbook. Tools & Tips for Career Search Success CALIFORNIA STATE UNIVERSITY, SACR AMENTO

UNIDIRECTIONAL LONG SHORT-TERM MEMORY RECURRENT NEURAL NETWORK WITH RECURRENT OUTPUT LAYER FOR LOW-LATENCY SPEECH SYNTHESIS. Heiga Zen, Haşim Sak

part2 Participatory Processes

Application for Admission

The NICT/ATR speech synthesis system for the Blizzard Challenge 2008

VISION, MISSION, VALUES, AND GOALS

Modeling function word errors in DNN-HMM based LVCSR systems

BUILDING CONTEXT-DEPENDENT DNN ACOUSTIC MODELS USING KULLBACK-LEIBLER DIVERGENCE-BASED STATE TYING

Robust Speech Recognition using DNN-HMM Acoustic Model Combining Noise-aware training with Spectral Subtraction

Learning Methods in Multilingual Speech Recognition

Speech Emotion Recognition Using Support Vector Machine

The IRISA Text-To-Speech System for the Blizzard Challenge 2017

Using Articulatory Features and Inferred Phonological Segments in Zero Resource Speech Processing

Modeling function word errors in DNN-HMM based LVCSR systems

Human Emotion Recognition From Speech

Speech Recognition at ICSI: Broadcast News and beyond

AUTOMATIC DETECTION OF PROLONGED FRICATIVE PHONEMES WITH THE HIDDEN MARKOV MODELS APPROACH 1. INTRODUCTION

ADVANCES IN DEEP NEURAL NETWORK APPROACHES TO SPEAKER RECOGNITION

A Comparison of DHMM and DTW for Isolated Digits Recognition System of Arabic Language

On the Formation of Phoneme Categories in DNN Acoustic Models

also inside Continuing Education Alumni Authors College Events

Edinburgh Research Explorer

WHEN THERE IS A mismatch between the acoustic

Design Of An Automatic Speaker Recognition System Using MFCC, Vector Quantization And LBG Algorithm

Phonetic- and Speaker-Discriminant Features for Speaker Recognition. Research Project

Autoregressive product of multi-frame predictions can improve the accuracy of hybrid models

Analysis of Emotion Recognition System through Speech Signal Using KNN & GMM Classifier

International Journal of Computational Intelligence and Informatics, Vol. 1 : No. 4, January - March 2012

Statistical Parametric Speech Synthesis

A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation

Letter-based speech synthesis

Likelihood-Maximizing Beamforming for Robust Hands-Free Speech Recognition

Speech Synthesis in Noisy Environment by Enhancing Strength of Excitation and Formant Prominence

2014 Gold Award Winner SpecialParent

Expressive speech synthesis: a review

Segmental Conditional Random Fields with Deep Neural Networks as Acoustic Models for First-Pass Word Recognition

Word Segmentation of Off-line Handwritten Documents

Semi-Supervised GMM and DNN Acoustic Model Training with Multi-system Combination and Confidence Re-calibration

Affective Classification of Generic Audio Clips using Regression Models

Class-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification

Improvements to the Pruning Behavior of DNN Acoustic Models

International Journal of Advanced Networking Applications (IJANA) ISSN No. :

Probabilistic Latent Semantic Analysis

INVESTIGATION OF UNSUPERVISED ADAPTATION OF DNN ACOUSTIC MODELS WITH FILTER BANK INPUT

Digital Signal Processing: Speaker Recognition Final Report (Complete Version)

Spoofing and countermeasures for automatic speaker verification

Lecture 9: Speech Recognition

SEMI-SUPERVISED ENSEMBLE DNN ACOUSTIC MODEL TRAINING

BAUM-WELCH TRAINING FOR SEGMENT-BASED SPEECH RECOGNITION. Han Shu, I. Lee Hetherington, and James Glass

Speaker recognition using universal background model on YOHO database

On March 15, 2016, Governor Rick Snyder. Continuing Medical Education Becomes Mandatory in Michigan. in this issue... 3 Great Lakes Veterinary

PREDICTING SPEECH RECOGNITION CONFIDENCE USING DEEP LEARNING WITH WORD IDENTITY AND SCORE FEATURES

Unvoiced Landmark Detection for Segment-based Mandarin Continuous Speech Recognition

Eli Yamamoto, Satoshi Nakamura, Kiyohiro Shikano. Graduate School of Information Science, Nara Institute of Science & Technology

Speaker Identification by Comparison of Smart Methods. Abstract

Unsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model

Artificial Neural Networks written examination

Speech Recognition by Indexing and Sequencing

A Case Study: News Classification Based on Term Frequency

Speech Segmentation Using Probabilistic Phonetic Feature Hierarchy and Support Vector Machines

Unit Selection Synthesis Using Long Non-Uniform Units and Phonemic Identity Matching

Speech Recognition using Acoustic Landmarks and Binary Phonetic Feature Classifiers

Analysis of Speech Recognition Models for Real Time Captioning and Post Lecture Transcription

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 17, NO. 3, MARCH

Calibration of Confidence Measures in Speech Recognition

Web as Corpus. Corpus Linguistics. Web as Corpus 1 / 1. Corpus Linguistics. Web as Corpus. web.pl 3 / 1. Sketch Engine. Corpus Linguistics

A NOVEL SCHEME FOR SPEAKER RECOGNITION USING A PHONETICALLY-AWARE DEEP NEURAL NETWORK. Yun Lei Nicolas Scheffer Luciana Ferrer Mitchell McLaren

Body-Conducted Speech Recognition and its Application to Speech Support System

Mandarin Lexical Tone Recognition: The Gating Paradigm

/$ IEEE

Unsupervised Acoustic Model Training for Simultaneous Lecture Translation in Incremental and Batch Mode

A Hybrid Text-To-Speech system for Afrikaans

SARDNET: A Self-Organizing Feature Map for Sequences

TRANSFER LEARNING OF WEAKLY LABELLED AUDIO. Aleksandr Diment, Tuomas Virtanen

Procedia - Social and Behavioral Sciences 141 ( 2014 ) WCLTA Using Corpus Linguistics in the Development of Writing

Segregation of Unvoiced Speech from Nonspeech Interference

Automatic Speaker Recognition: Modelling, Feature Extraction and Effects of Clinical Environment

arxiv: v1 [math.at] 10 Jan 2016

Rule Learning With Negation: Issues Regarding Effectiveness

Support Vector Machines for Speaker and Language Recognition

Chapter 10 APPLYING TOPIC MODELING TO FORENSIC DATA. 1. Introduction. Alta de Waal, Jacobus Venter and Etienne Barnard

DIRECT ADAPTATION OF HYBRID DNN/HMM MODEL FOR FAST SPEAKER ADAPTATION IN LVCSR BASED ON SPEAKER CODE

Learning Methods for Fuzzy Systems

Reducing Features to Improve Bug Prediction

PRAAT ON THE WEB AN UPGRADE OF PRAAT FOR SEMI-AUTOMATIC SPEECH ANNOTATION

Using dialogue context to improve parsing performance in dialogue systems

DOMAIN MISMATCH COMPENSATION FOR SPEAKER RECOGNITION USING A LIBRARY OF WHITENERS. Elliot Singer and Douglas Reynolds

Seminar - Organic Computing

Experiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling

Generative models and adversarial training

A Privacy-Sensitive Approach to Modeling Multi-Person Conversations

Transcription:

The CSTR etry to the Blizzard Challege 2017 Srikath Roaki, Mauel Sam Ribeiro, Felipe Espic, Oliver Watts The Cetre for Speech Techology Research (CSTR), Uiversity of Ediburgh, UK srikath.roaki@ed.ac.uk Abstract The aual Blizzard Challege coducts side-by-side testig of a umber of speech sythesis systems traied o a commo set of speech data. Similar to 2016 Blizzard challege, the task for this year is to trai o expressively-read childre s story-books, ad to sythesise speech i the same domai. The Challege therefore presets a opportuity to ivestigate the effectiveess of several techiques we have developed whe applied to expressive ad prosodically-varied audiobook data. This paper describes the text-to-speech system etered by The Cetre for Speech Techology Research ito the 2017 Blizzard Challege. The curret system is a hybrid sythesis system which drives a uit selectio sythesiser usig the output from a eural etwork based acoustic ad duratio model. We assess the performace of our system by reportig the results from formal listeig tests provided by the challege. Idex Terms: Merli, hybrid speech sythesis, uit selectio, deep eural etworks. 1. Itroductio The CSTR etry to this year s Blizzard Challege builds o the hybrid Multisy [1, 2] system submitted for last year [3]. Hybrid sythesis systems o the basis of target cost fuctio [4, 5, 6, for example] employ statistical models to predict acoustic properties of speech thereby brigs the beefits of extremely atural-soudig uit selectio (which is uaffected by the degradatios itroduced by vocodig [7, 8]). Similar to last year, the data used for this year s Challege was obtaied from professioally-read child-directed audio books ad is therefore much more prosodically rich tha the more stadard prompt-based speech data. The experimet preseted i [5, 6] established that improvig the uderlyig SPSS of a hybrid sythesiser results i improvemets to the cocateated output speech. Therefore, for our previous etry, we have icorporated two major improvemets to the uderlyig SPSS model compared to the system preseted i [6]: the decisio tree duratio model was replaced with a bi-directioal log short-term memory (LSTM) recurret eural etwork, ad the feed-forward DNN acoustic model was replaced with a LSTM etwork. Compared to our previous year etry, the text processig ad speech parameterizatio steps are largely uchaged. Acoustic model predictio is slightly optimised for better predictio of fudametal frequecy by addig supra-segmetal features based o acoustic couts ad is explaied i 2.3.2. A otable exceptio is our attempt to smooth the jois: this ew developmet is described i sectio 2.6 below. The eural etworks used i this etry were traied usig our ope-source Merli speech sythesis toolkit [9]. 2.1. Data 2. Descriptio The database provided to the Challege by Usbore Publishig Ltd. cosists of the speech ad text of 56 childre s audiobooks spoke by a British female speaker. We made use of a segmetatio of the audiobooks carried out by two other Challege participats 12 ad kidly made available to other participats. The total duratio of the audio is approximately 6 hours after segmetatio. Three audiobooks from the give corpus were held out to act as a iteral developmet set to gauge system performace before geeratig the fial test data. The held-out data cosists of three full short stories: Goldilocks ad the Three Bears, The Boy Who Cried Wolf ad The Eormous Turip, havig a total combied duratio of approximately 10 miutes. 2.1.1. Setece selectio For setece selectio, we have followed the same approach as last year. For clarity, we repeat the procedure followed from our previous year etry [3]. Haressig the variety of speakig styles preset i expressively-read audiobooks might eable us to produce less robotic-soudig TTS systems. However, iitial experimets showed that the extreme variatio i parts of the traiig data for the Challege resultig i poor uit selectio. We therefore filtered the data usig the active learig approach described i [10]: 198 utterace-level acoustic features are extracted, ad 15 seteces iitially labelled as keep or too expressive by a expert listeer. Ucertaity samplig [11] usig a esemble of decisio trees was the used to select a further iformative sample to be had-labelled; this process cotiued for 20 miutes (real time). A classifier built o the etire set of hadlabelled data was the used to determie the subset of available seteces to be used for traiig. 20% of the traiig seteces were discarded i this way; iformal compariso suggested this resulted i more stable sythesis with fewer uwarrated prosodic excursios. 2.2. Text processig We have used Festival s Eglish frot-ed with the British Received Prouciatio versio of the Combilex lexico [12]. 163 items were added to cover words appearig i the traiig data but otherwise abset from the dictioary. There were slight differeces i the lexico-lookup procedures used i preparig the aotatio for traiig the SPSS model ad those employed by the Festival frot-ed used for Multisy. The resultig icosistecies were dealt with by aligig the DNN s phoe sequeces to those expected by Multisy i a ad hoc fashio ad is similar to our previous year etry. 1 Ioetics: https://www.ioetics.com 2 IIIT-H: http://speech.iiit.ac.i

Word ad syllable level vector represetatios were icluded, accordig to the method described i [13]. These were leared by takig couts of acoustic evets of f0 ad eergy stylized by clustered vectors ad mea values defied over syllables or words. The traiig data available for the Challege was used to lear these matrices. Experimets usig vectors represetatios leared over a larger database of a differet speaker, but we have observed that results were comparable with speaker-depedet vectors leared o a smaller database. 2.3. Parametric system The parametric system was implemeted usig DNNs i a covetioal two-stage approach. I the first stage, a duratio model is used to predict phoe duratios to form frame-level liguistic features. I the secod stage, a acoustic model is used to geerate parameters from those liguistic features. 2.3.1. Duratio model The duratio model traied for our etry to the challege made use of a simple ad straightforward approach with feed-forward eural etworks (DNNs) as demostrated i [14, 15]. The duratio model is traied o the aliged data ad geerates statelevel duratios give phoe-level liguistic features. The described approach was used oly to geerated duratios, which were the used to form frame-level liguistic features used as iput i the geeratio of acoustic parameters. The hybrid Multisy uit-selectio system, however, does ot make use of ay duratio-related features i its target cost fuctio. Icludig such features i the uit selectio process is left for future work. 2.3.2. Acoustic model The liguistic features extracted from the frot-ed were coverted to umerical vectors usig a set of cotiuous ad biary questios [9]. To these, we appeded the syllable ad word level vector represetatios based o acoustic couts [13]. The duratios geerated by the duratio model described above were used to propagate all feature to frame-level. These frame-level feature vectors were the used as iput to a acoustic model. A feedforward eural etwork was traied at the framelevel to map liguistic iputs to vocoder parameters cosistig of static ad dyamic (delta ad delta-delta) features. These acoustic parameters iclude 60 mel-cepstra coefficiets, 25 bad aperiodicities, log-f0, ad a biary voicig decisio. Maximum likelihood parameter geeratio (MLPG) ad postfilterig are the applied to the geerated acoustic parameters. I SPSS these parameter trajectories would the be passed through the vocoder to sythesize a speech waveform. Istead, we use them as targets for selectig waveform uits. Withi each phoe uit, geerated parameters are split uiformly across time ito 4 sectios. A Gaussia distributio is the fitted for each sub-phoe sectio of acoustic parameters. The variaces of these Gaussia distributios are floored at 1% of the global variace per parameter [6]. The distributios associated with each of the 4 sub-phoe sectios are used to costruct a diphoe represetatio for the target utterace. To costruct a diphoe represetatio, we take the first or last 2 sectios associated with its correspodig phoes. Comparable distributios were geerated for the diphoe cadidates i the uit database, based o vocoder parameters extracted from the traiig data ad atural duratios obtaied by forced aligmet. 2.3.3. Feature extractio Phoe sequeces were obtaied from the text usig Festival [16]. Festvox s ehmm method [17] was used to modify the phoe sequeces by the isertio of acoustically-motivated pauses; A state-level forced aligmet of these phoe sequeces with the setece-segmeted audio was the obtaied usig cotext-idepedet HMMs, similar to [18]. Each phoe was the characterised by a vector of 481 text-derived biary ad umerical features a subset of the features used as decisiotree clusterig questios i the HTS demo [19], adapted for our phoeset. These questios icluded liguistic cotexts such as quiphoe idetity which are added at phoe-level, ad part-ofspeech, positioal iformatio relatig to syllables, words, phrases, etc. All umerical features are give as iput (after appropriate ormalisatio) directly to the etwork, ad ot ecoded as (for example) 1-of-K. For duratio modellig, all these features were used as iput ad ormalised to the rage of [0.01, 0.99]. The output for traiig is a five-dimesioal vector of duratios for every phoe, comprisig five sub-state duratios. For acoustic modellig, the iput uses the same features as duratio predictio, to which 9 umerical features were appeded. These capture frame positio i the HMM state ad phoeme, state positio i phoeme, ad state ad phoeme duratio, similar to [18]. The speech data was aalysed with STRAIGHT [20], ad each 5ms frame was represeted usig 60 mel cepstral coefficiets (MCC), measures of aperiodicity i 25 frequecy bads (BAP), logarithmic F 0 iterpolated through uvoiced regios, ad a biary voicig feature. These 87 static features were supplemeted with delta ad delta-delta features, ad for both the duratio ad acoustic data, a per-compoet mea ad variace ormalisatio was applied prior to model traiig, with the trasformatio reversed as part of sythesis. 2.3.4. Duratio ad acoustic model traiig For the duratio model, we have used 481-dimesioal biary ad cotiuously valued feature vectors as iput. Its output was a 5-dimesioal feature vector represetig state duratios i terms of frames. The model was defied to be 6 feedforward hidde layers, each with 1024 odes, usig the tah activatio fuctio. Mii batch size was set to 64 ad learig rate was set to 0.002, beig was reduced by 50% with each epoch after the first 10 traiig epochs. For the acoustic model, we have used the same 481- dimesioal feature vector represetig liguistic features. To these, we added syllable ad word level vector represetatios spaig a widow of 3 uits. Nie frame-level features were icluded accordig to [18] ad available from [9]. The iput vector to the acoustic models cosisted of a total of 1900 dimesios. The model cosisted of 6 feedforward hidde layers, each with 1024 odes, usig the tah activatio fuctio. Miibatch was set to 256 ad remaiig parameters were idetical to the duratio model. 2.4. Uit selectio waveform rederer For uit selectio, we have followed the same approach as last year. For clarity, we repeat the procedure followed from our previous year etry [3]. A modified form of Festival s Multisy egie [2] was used for the uit selectio stage of our system. To compare the suit-

ability of a give cadidate diphoe i the uit database with the 4 distributios represetig a sythesised diphoe, the symmetrised Kullback Leibler divergece (KLD) [21] is used. The KLD is computed betwee each of the 4 cadidate uit s distributios ad the correspodig target uit distributios idividually. The resultig 4 scores are the summed to produce the fial target score. The stadard Multisy joi cost (sum of distaces betwee 12 MFCCs, f 0 ad eergy from the frame either side of the joi) is retaied, as well as the stadard pre-selectio criterio of cadidate uits (by matchig diphoe idetity). The stadard Multisy Viterbi search (with pruig to reduce the search time) is performed i order to optimise target cost ad joi cost. Also the stadard Multisy back-off rules are used where the target diphoe to be sythesised is ot preset i the traiig data. 2.5. Speech sythesis At sythesis time, duratio is predicted first, ad is used as a iput to the acoustic model to predict the speech parameters. Maximum likelihood parameter geeratio (MLPG) [22] usig variaces computed from the traiig data was applied to the output features for sythesis, ad spectral ehacemet postfilterig was applied to the resultig MCC trajectories. These parameter trajectories are the used to produce diphoe coefficiets. The Festival Multisy egie was used to compute the target ad joit cost betwee target uit ad pre-selected cadidate uits to select the fial cadidate, as explaied above. The fial waveform sythesis was doe by joiig the selected uits. A additioal smoothig ad post-modificatio of prosody was performed durig joiig the uits ad is explaied i below sectio. 2.6. Cocateatio ad joi smoothig The selected waveform uits are parameterised by usig the method proposed i [23]. It extracts pitch sychroous speech features i a frame-by-frame basis, describig the complex spectra ad F0 cotour. The correctio/smoothig operatios are performed over these features to produce seamless cocateatio of uits. 2.6.1. Cocateatio ad correctio of F0 cotours The F 0 mid poit (F 0 m) betwee two cosecutive uits is give by F 0 m = (F 0 p[n p 1] + F 0 c[0])/2, where p meas precedig uit, c curret uit, ad N is the uit legth i frames. The, the slope of the F0 cotours of both uits are adjusted to reach the F 0 m just i the joi locatio. The corrected F 0 cotours are computed by the Equatios 1 ad 2. ( ) F 0 c c[ c] = F 0 c[ c] + (F 0 m F 0 c[0]) + 1 (1) 1 N c F 0 p[ p] = F 0 p[ p] + (F 0 m F 0 p[n p 1]) p N p 1 (2) Where F 0 is the corrected F 0, ad is the frame idex withi each uit. After havig all the corrected F 0 cotours for all the uits, these are appeded buildig a sigle F 0 cotour for the whole setece. 2.6.2. Spectral cocateatio ad smoothig Basically, it is doe by overlappig ad crossfadig the complex FFT spectra of two cosecutive uits. Some extra frames are extracted from the sources, so the uits ca be overlapped without 1 2 3 4 5 Mea Opiio s (aturaless) All listeers 518 516 518 519 518 517 516 517 518 517 516 518 517 516 513 517 519 Figure 1: Our system(e): Mea opiio score for aturaless of the sythesized speech with ratigs from all listeers. affectig their expected locatios i the sythesised waveform. Three extra frames o each side of the uits are extracted from the sources, thus a overlap of seve frames aroud the jois is produced. The FFT complex spectrum S is derived from the parameters proposed i [23], M, R, ad I, by S = M (R + Ij). The crossfade is liearly applied to mix the FFT complex spectra of two cosecutive uits, progressively. It is seve frames legth, ad i case that a uit is too short, the crossfade is shorteed accordigly. After performig this operatio o every joi, the FFT complex spectra of all the uits are cocateated producig a sigle complex spectra stream, that describes the whole utterace. Fially, the sigal is sythesised by covertig the FFT complex spectra to time domai, ad applyig Pitch Sychroous Overlap-Add as explaied i [23], usig the corrected F 0 cotour. 2.7. Paragraph-level sythesis From the seteces sythesised i this way, files were made cotaiig whole paragraphs, chapters ad books as required by the Challege by simply cocateatig the waveforms. While proper exploitatio of log-distace cotexts ought to improve sythesis quality, o cotexts outside the curret setece were used for the preset submissio. 3. Results The idetifier for our system i the published results is E. 3.1. Naturaless Mea opiio scores for aturaless from all listeers o book seteces are show i Figure 1. I our discussio, we make use of the published statistical aalysis of the results at 1% level with Boferoi corrected alpha) [24]. Our system outperformed all three baselies (systems B, C ad D). Amog the 12 other challege participats, our system is outperformed oly by a sigle system (I). The same tred ca be see across the scores

Mea Opiio s (similarity to origial speaker) All listeers Mea Opiio s (audiobook paragraphs overall impressio) All listeers 1 2 3 4 5 257 258 257 257 257 257 257 257 257 257 257 257 257 257 257 257 257 0 10 20 30 40 50 60 597 590 597 596 597 595 592 593 595 592 595 593 596 592 594 596 593 Figure 2: Our system(e): Mea opiio score for speaker similarity with ratigs from all listeers. Figure 3: Our system(e): Mea opiio score for overall impressio with ratigs from all listeers. made by paid listeers, o-lie voluteers ad cosiderig ratigs oly from speech experts, o other system was sigificatly better tha ours. Overall, our system outperformed 11 out of 15 other systems (icludig three baselies) evaluated for listeig test. 3.2. Speaker similarity The mea opiio scores for speaker similarity from all listeers o book seteces are show i Figure 2. Cosiderig ratigs from all listeers (or ay other listeer group), o other system was sigificatly better tha ours ad our system was i tur sigificatly better tha 13 other systems. These results show the effectiveess of waveform cocateatio systems for speaker similarity. 3.3. Evaluatio of audiobook paragraphs We ow cosider the results for evaluatio of audiobook paragraphs that have bee evaluated o several other factors like stress, itoatio, emotio, pleasatess, listeig effort, speech pauses ad overall impressio. Cosiderig ratigs from all listeers o overall impressio, our system showed similar performace as i the case of the isolated setece evaluatio of aturaless ad speaker similarity. Oly oe system (I) outperformed us ad our system was sigificatly better i tur tha 11 other systems (cf. Figure 3). A similar tred ca be see across the scores made by speech experts, olie voluteers ad paid listeers. Cosiderig ratigs for other idividual factors (e.g., itoatio, emotio ad pleasatess) from all listeers, agai oly system I cosistetly outperformed ours. Overall, our system outperforms betwee 7 ad 11 other systems i evaluatio of each of these factors, performig best i emotio ad pleasatess. 3.4. Itelligibility (SUS) We ow cosider the results for itelligibility of sematically upredictable seteces (makig use of the published statisti- cal aalysis of sigificat differece betwee word error rates of the systems). Takig ito accout ratigs from all listeers, there are oly three other systems out of 15 (D, L, ad M) sigificatly better tha ours. Cosiderig oly paid listeers, there are oly two other systems (D ad L) sigificatly better tha ours. Out of 15 other systems evaluated by paid listeers, 10 were ot sigificatly more or less itelligible tha ours, 3 were sigificatly less itelligible, ad oly 2 sigificatly more itelligible. The results show that our system is quite effective o itelligibility as well. Overall, our system has show cosistet performace (stadig i the top four) i all the factors evaluated for the Challege. 4. Coclusios ad future work For this year s CSTR Blizzard Challege etry, the hybrid system submitted for last year [3] was slightly optimized i acoustic modelig for better predictio of F0 ad performed smoothig betwee jois. The results of the evaluatio are o the whole very positive, but there are still a umber of potetial future improvemets which could be made to the hybrid sythesis system described here. These iclude adoptig cosistet lexico-lookup for both the SPSS ad uit selectio systems, makig use of same acoustic features for both joi ad target cost, predictio of phrase breaks, ad the explicit iclusio of predicted duratio i the uit selectio sythesis target cost. Reproducibility: We used the Ope Source Merli toolkit 3 for parameter predictio ad Festival Multisy 4 for uitselectio. Ackowledgemet: Watts was supported i this research by EPSRC Stadard Research Grat EP/P011586/1, Speech Sythesis for Spoke Cotet Productio (SCRIPT). 3 https://github.com/cstr-ediburgh/merli 4 http://www.cstr.ed.ac.uk/projects/festival

5. Refereces [1] R. A. Clark, K. Richmod, ad S. Kig, Festival 2 build your ow geeral purpose uit selectio speech sythesiser, i Proc. SSW, 2004. [2] R. A. Clark, k. Richmod, ad S. Kig, Multisy: Ope-domai uit selectio for the Festival speech sythesis system, Speech Commuicatio, vol. 49, o. 4, pp. 317 330, 2007. [3] T. Merritt, S. Roaki, Z. Wu, ad O. Watts, The CSTR etry to the Blizzard Challege 2016, i Proc. Blizzard Challege workshop, 2016. [4] Z.-J. Ya, Y. Qia, ad F. K. Soog, Rich-cotext uit selectio (rus) approach to high quality tts, i Proc. ICASSP, 2010, pp. 4798 4801. [5] Y. Qia, F. K. Soog, ad Z.-J. Ya, A uified trajectory tilig approach to high quality speech rederig, IEEE Audio, Speech, Laguage Process., vol. 21, o. 2, pp. 280 290, 2013. [6] T. Merritt, R. A. J. Clark, Z. Wu, J. Yamagishi, ad S. Kig, Deep eural etwork-guided uit selectio sythesis, i Proc. ICASSP, 2016. [7] H. Ze, A. Seior, ad M. Schuster, Statistical Parametric Speech Sythesis Usig Deep Neural Networks, i Proc. ICASSP, 2013, pp. 7962 7966. [8] G. E. Heter, T. Merritt, M. Shao, C. Mayo, ad S. Kig, Measurig the perceptual effects of modellig assumptios i speech sythesis usig stimuli costructed from repeated atural speech, i Proc. Iterspeech, 2014, pp. 1504 1508. [9] Z. Wu, O. Watts, ad S. Kig, Merli: A Ope Source Neural Network Speech Sythesis, i Proc. SSW, Suyvale, USA, 2016. [10] O. Watts, A. Sta, R. Clark, Y. Mamiya, M. Giurgiu, J. Yamagishi, ad S. Kig, Usupervised ad lightly-supervised learig for rapid costructio of TTS systems i multiple laguages from foud data: evaluatio ad aalysis, i 8th ISCA Workshop o Speech Sythesis, Barceloa, Spai, Aug. 2013, pp. 121 126. [11] D. D. Lewis ad W. A. Gale, A sequetial algorithm for traiig text classifiers, i Proceedigs of the 17th aual iteratioal ACM SIGIR coferece o Research ad developmet i iformatio retrieval, 1994, pp. 3 12. [12] S. Fitt ad K. Richmod, Redudacy ad productivity i the speech techology lexico - ca we do better? i Proc. Iterspeech 2006, Sep. 2006. [13] M. S. Ribeiro, O. Watts, ad J. Yamagishi, Learig word vector represetatios based o acoustic couts, i Proc. Iterspeech, Stockholm, Swede, August 2017. [14] S. Roaki, Z. Wu, O. Watts, ad S. Kig, A Demostratio of the Merli Ope Source Neural Network Speech Sythesis, i Proc. SSW, Suyvale, USA, 2016. [15] S. Roaki, O. Watts, S. Kig, ad G. E. Heter, Media-based geeratio of sythetic speech duratios usig a o-parametric approach, i IEEE workshop o Spoke Laguage Techology, Sa Diego, Califoria, 2016. [16] A. Black, P. Taylor, R. Caley, R. Clark, K. Richmod, S. Kig, V. Strom, ad H. Ze, The festival speech sythesis system, versio 1.4. 2, Upublished documet available via http://www. cstr. ed. ac. uk/projects/festival. html, 2001. [17] K. Prahallad, A. W. Black, ad R. Mosur, Sub-phoetic modelig for capturig prouciatio variatios for coversatioal speech sythesis, i Proc. ICASSP, 2006, pp. I 853 I 856. [18] Z. Wu, C. Valetii-Botihao, O. Watts, ad S. Kig, Deep eural etworks employig multi-task learig ad stacked bottleeck features for speech sythesis, i Proc. ICASSP, 2015, pp. 4460 4464. [19] H. Ze, T. Nose, J. Yamagishi, S. Sako, T. Masuko, A. Black, ad K. Tokuda, The HMM-based speech sythesis system (HTS) versio 2.0, i Proc. SSW, vol. 6, 2007, pp. 294 299. [20] H. Kawahara, STRAIGHT, exploitatio of the other aspect of VOCODER: Perceptually isomorphic decompositio of speech souds, Acoust. Sci. Techol., vol. 27, o. 6, pp. 349 353, 2006. [21] J. R. Hershey ad P. A. Olse, Approximatig the Kullback- Leibler divergece betwee Gaussia mixture models, i Proc. ICASSP, 2007. [22] K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, ad T. Kitamura, Speech parameter geeratio algorithms for HMM-based speech sythesis, i Proc. ICASSP, vol. 3, 2000, pp. 1315 1318. [23] F. Espic, C. Valetii-Botihao, ad S. Kig, Direct modellig of magitude ad phase spectra for statistical parametric speech sythesis, i Proc. Iterspeech, Stochohlm, Swede, August 2017. [24] R. A. Clark, M. Podsiadlo, M. Fraser, C. Mayo, ad S. Kig, Statistical aalysis of the Blizzard Challege 2007 listeig test results, Proc. Blizzard Challege Workshop, 2007.