Syllabus for the course «Digital signal processing» (Цифровая обработка сигналов)

Similar documents
Lecture 1: Machine Learning Basics

Lahore University of Management Sciences. FINN 321 Econometrics Fall Semester 2017

MASTER OF PHILOSOPHY IN STATISTICS

ACTL5103 Stochastic Modelling For Actuaries. Course Outline Semester 2, 2014

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur

STA 225: Introductory Statistics (CT)

Detailed course syllabus

Lecture 10: Reinforcement Learning

Probability and Statistics Curriculum Pacing Guide

Mathematics. Mathematics

Human Emotion Recognition From Speech

Speaker Identification by Comparison of Smart Methods. Abstract

WHEN THERE IS A mismatch between the acoustic

Evolutive Neural Net Fuzzy Filtering: Basic Description

AUTOMATIC DETECTION OF PROLONGED FRICATIVE PHONEMES WITH THE HIDDEN MARKOV MODELS APPROACH 1. INTRODUCTION

Python Machine Learning

A Model to Predict 24-Hour Urinary Creatinine Level Using Repeated Measurements

A study of speaker adaptation for DNN-based speech synthesis

Stochastic Calculus for Finance I (46-944) Spring 2008 Syllabus

Probabilistic Latent Semantic Analysis

Lecture 1: Basic Concepts of Machine Learning

Statewide Framework Document for:

Algebra 1, Quarter 3, Unit 3.1. Line of Best Fit. Overview

AGS THE GREAT REVIEW GAME FOR PRE-ALGEBRA (CD) CORRELATED TO CALIFORNIA CONTENT STANDARDS

Class-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification

Reinforcement Learning by Comparing Immediate Reward

Math 96: Intermediate Algebra in Context

Analysis of Emotion Recognition System through Speech Signal Using KNN & GMM Classifier

Course Syllabus for Math

(Sub)Gradient Descent

Office Hours: Mon & Fri 10:00-12:00. Course Description

Speech Emotion Recognition Using Support Vector Machine

Analysis of Enzyme Kinetic Data

UNIT ONE Tools of Algebra

The 9 th International Scientific Conference elearning and software for Education Bucharest, April 25-26, / X

JONATHAN H. WRIGHT Department of Economics, Johns Hopkins University, 3400 N. Charles St., Baltimore MD (410)

4 th year course description

Mathematics Program Assessment Plan

Design Of An Automatic Speaker Recognition System Using MFCC, Vector Quantization And LBG Algorithm

ENME 605 Advanced Control Systems, Fall 2015 Department of Mechanical Engineering

EGRHS Course Fair. Science & Math AP & IB Courses

S T A T 251 C o u r s e S y l l a b u s I n t r o d u c t i o n t o p r o b a b i l i t y

Speaker recognition using universal background model on YOHO database

Math 121 Fundamentals of Mathematics I

Generative models and adversarial training

Class Meeting Time and Place: Section 3: MTWF10:00-10:50 TILT 221

Math 098 Intermediate Algebra Spring 2018

AP Calculus AB. Nevada Academic Standards that are assessable at the local level only.

MGT/MGP/MGB 261: Investment Analysis

OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS

Spring 2016 Stony Brook University Instructor: Dr. Paul Fodor

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 17, NO. 3, MARCH

PROGRAM REVIEW CALCULUS TRACK MATH COURSES (MATH 170, 180, 190, 191, 210, 220, 270) May 1st, 2012

Artificial Neural Networks written examination

Answer Key Applied Calculus 4

Honors Mathematics. Introduction and Definition of Honors Mathematics

MTH 215: Introduction to Linear Algebra

Extending Place Value with Whole Numbers to 1,000,000


Instructor: Mario D. Garrett, Ph.D. Phone: Office: Hepner Hall (HH) 100

CS/SE 3341 Spring 2012

Penn State University - University Park MATH 140 Instructor Syllabus, Calculus with Analytic Geometry I Fall 2010

Learning Methods in Multilingual Speech Recognition

Mathematics subject curriculum

ITSC 1301 Introduction to Computers Course Syllabus

Circuit Simulators: A Revolutionary E-Learning Platform

Phonetic- and Speaker-Discriminant Features for Speaker Recognition. Research Project

We are strong in research and particularly noted in software engineering, information security and privacy, and humane gaming.

Calibration of Confidence Measures in Speech Recognition

Grade 6: Correlated to AGS Basic Math Skills

Julia Smith. Effective Classroom Approaches to.

State University of New York at Buffalo INTRODUCTION TO STATISTICS PSC 408 Fall 2015 M,W,F 1-1:50 NSC 210

ME 4495 Computational Heat Transfer and Fluid Flow M,W 4:00 5:15 (Eng 177)

Modeling function word errors in DNN-HMM based LVCSR systems

Machine Learning and Development Policy

Foothill College Summer 2016

Probability and Game Theory Course Syllabus

Speech Synthesis in Noisy Environment by Enhancing Strength of Excitation and Formant Prominence

A Comparison of DHMM and DTW for Isolated Digits Recognition System of Arabic Language

Speech Recognition at ICSI: Broadcast News and beyond

University of Groningen. Systemen, planning, netwerken Bosman, Aart

CAAP. Content Analysis Report. Sample College. Institution Code: 9011 Institution Type: 4-Year Subgroup: none Test Date: Spring 2011

Universityy. The content of

Learning Disability Functional Capacity Evaluation. Dear Doctor,

Undergraduate Program Guide. Bachelor of Science. Computer Science DEPARTMENT OF COMPUTER SCIENCE and ENGINEERING

Modeling function word errors in DNN-HMM based LVCSR systems

EECS 700: Computer Modeling, Simulation, and Visualization Fall 2014

Language Acquisition Fall 2010/Winter Lexical Categories. Afra Alishahi, Heiner Drenhaus

A Neural Network GUI Tested on Text-To-Phoneme Mapping

Likelihood-Maximizing Beamforming for Robust Hands-Free Speech Recognition

OFFICE SUPPORT SPECIALIST Technical Diploma

The NICT/ATR speech synthesis system for the Blizzard Challenge 2008

SAT MATH PREP:

B.S/M.A in Mathematics

Syllabus ENGR 190 Introductory Calculus (QR)

Course Specifications

COMPUTER INTERFACES FOR TEACHING THE NINTENDO GENERATION

Rule Learning With Negation: Issues Regarding Effectiveness

Learning Methods for Fuzzy Systems

Massachusetts Institute of Technology Tel: Massachusetts Avenue Room 32-D558 MA 02139

Transcription:

Government of Russian Federation Federal State Autonomous Educational Institution of High Professional Education «National Research University Higher School of Economics» National Research University High School of Economics Faculty of Psychology Syllabus for the course «Digital signal processing» (Цифровая обработка сигналов) 030300.68 «Cognitive sciences and technologies: from neuron to cognition», Master of Science Author: Alexei Ossadtchi, professor, aossadtchi@hse.ru Approved by: Department of Data Analysis and Artificial Intelligence Recommended by: Moscow, 2014

1. Teachers Author, lecturer: Alexei Ossadtchi, Faculty of Computer Science, Department of Data Analysis and Artificial Intelligence, professor; Faculty of Social Sciences, Department of Psychology, Center for cognition and decision making, senior scientist. Tutor: Nikita Novikov, Faculty of Social Sciences, Department of Psychology, Center for cognition and decision making, research scientist. Teaching assistant: Zafer Iscan (to be confirmed) 2. Scope of Use The present program establishes minimum demands of students knowledge and skills, and determines content of the course. The present syllabus is aimed at department teaching the course, their teaching assistants, and students of the Master of Science program 030300.68 «Cognitive sciences and technologies: from neuron to cognition». This syllabus meets the standards required by: Educational standards of National Research University Higher School of Economics; Educational program «Psychology» of Federal Master s Degree Program 030300.68, 2014; University curriculum of the Master s program in psychology (030300.68) for 2014. 3. Summary In modern science system's based approach is frequently exercised as it allows to formalize the problems encountered in the real world by representing them in the well studied framework that provides for efficient analysis and solution. In order to get a full grip of his powerful machinery researchers need to understand the fundamentals principles of the theory. The existing classes are either too specialized and mathematically detailed or too much of a cook-book nature. The goal of this class is to provide master students of non-mathematical background with a unifying view on the theory behind the system-oriented approach. The main goal is to develop the intuition behind the complex concepts. We will attempt to do it by showing the similarity of discrete and continuous treatments of the system's theory and interpret the results using the natural concepts of linear algebra. The second part of the class is primarily dedicated to the fundamentals of estimation theory. We start from the basic concepts and estimation of model parameters in the white and colored noise cases. We will then introduce the Maximum Likelihood and Bayesian approaches to the problem of parameter estimation. We will illustrate the Bayesian methodology via examples of solving the inverse problem in neuroimaging and spectroscopy. We will use the Bayesian concept to develop Kalman filters - advanced model-based estimators taking into account the dynamical properties of the signals.

4. Learning Objectives National Research University Higher School of Economics Learning objectives of the Digital signal processing class are to provide students with the unifying view on the theory behind system oriented approach in the modern science and to demonstrate the equivalence of continuous and discrete approaches. The following grand topics will be covered and supported by an extensive weekly homework assignments and practical sessions. 1 Signals 2 Transforms and decompositions 3 Statistical description of signals in original and transformed spaces 4 Signal models 5 Systems 6 Fundamentals of estimation theory 5. Learning outcomes After completing the study of the Digital signal processing the student should: Know the definition of signal, basic model signal types, understand the difference between the continuous and discrete signals, be able to formally write an expression of an arbitrary signal, be able to show the link between the continuous and discrete treatments, be able to explain the role linear algebra plays in signal processing. Understand the notion of a transform, be able to interpret a transform in the linear algebra framework, be familiar with basic transforms used (Fourier transform, Wavelet transform), be able to formulate one s own transform and write expressions for the coefficients and synthesis equation, be able to connect statistical properties of signals on both sides of the transform operation. Know basic signal models (AR, ARMA, Markov model), list the parameters, understand how parameters affect the nature of the resulting process and its statistical properties. Understand the notion of system, understand why LTI systems is an important class of systems, know what convolution operation is, be able to calculate the output of an arbitrary LTI system, be able to relate statistical properties of the output signal to those of the input Understand system parameters estimation problem, know basic criteria used to judge the quality of an estimator, understand the notion of the lower bound on the variance of the estimate, know basic estimation approaches (ML, Bayesian, Least squares) and understand the interrelation between those After completing the study of the discipline «Digital signal processing» the student should have the following competences: Competence Code Code (UC) Descriptors (indicators of achievement of the result) Educative forms and methods aimed at generation and development of the competence The ability to reflect developed methods of activity. SC-1 SC-М1 The student is able to reflect developed mathematical methods to psychological fields and problems. Lectures and tutorials, group discussions, presentations, paper reviews.

Competence Code Code (UC) The ability to propose a model to invent and test methods and tools of professional activity Descriptors (indicators of achievement of the result) SC-2 SC-М2 The student is able to improve and develop research methods of optimization, approximation and computational problem solving. Educative forms and methods aimed at generation and development of the competence Classes, homeworks. Capability of development of new research methods, change of scientific and industrial profile of self-activities The ability to describe problems and situations of professional activity in terms of humanitarian, economic and social sciences to solve problems which occur across sciences, in allied professional fields. The ability to detect, transmit common goals in the professional and social activities SC-3 SC-М3 The student obtain necessary knowledge in signals and systems theory and basics of estimation theory PC-5 IC- M5.3_5. 4_5.6_2. 4.1 The student is able to describe psychological problems in terms of computational mathematics. PC-8 SPC-M3 The student is able to identify systems theory aspects in psychological and neurobiological research tasks. Suggest a method to tackle the problem and rank several available techniques in the order of applicability in the current situation Homeworks, paper reviews, additional topics Lectures and tutorials, group discussions, presentations, paper reviews. Understanding of signals and systems fundamentals. Ability to formalize real-life problems Understanding of the estimation theory fundamentals. Maximum likelihood vs Maximum aposteriori probability estimation. Understanding of the uses in real-life examples. Discussion of paper reviews; cross discipline lectures

6. Place of the discipline in the Master s program structure The Digital signal processing course is an attempt to give master students with nonmathematical background a unifying view on the system-oriented approach used in the modern science. The class is a minor-type course offered by the Data analysis and artificial intelligence department of HSE to the first year of the Master s program «Cognitive sciences and technologies». It is recommended for students of the Master s program who are planning to exploit quantitative methods in the their research and want to build to an intuition to tackle the advanced topics in signals and data analysis Prerequisites The course is based on the basic knowledge of mathematical analysis and linear algebra concepts obtained by the students during their first semester at the program. The course is oriented to students with humanitarian background and will maximize the use of plots and pictures rather than equations and formal proofs to introduce the main concepts. The following knowledge and competence are needed to study the discipline: A good command of the English language. A basic knowledge in mathematics. 1. Mathematical analysis basics 2. Fundamentals of linear algebra Main competences developed after completing the study this discipline can be used to learn the following disciplines: Advanced topics in estimation theory. Adaptive signal processing Advanced multivariate data analysis and statistical inference Comparison with the other courses at HSE This class is unique in both the scope and the level of delivery that assumes only very basic mathematical background. The main stress is on building the intuition and teaching students to formalize the real-life problems and tackle them from the systems theory point of view. The scope of the class is very broad and ranges from basic signal types to the advanced topics of estimation theory. The unifying idea is the notion of the system. Signals, their properties and how they change when go through the system. Is time domain analysis the most convenient way for making sense of what an LTI systems does with the signal? This question leads to the introduction of transforms and the links to the state-space representation and linear algebra basics are shown illustrating that a transform is a simple change of basis operation. Which transforms are helpful? What do we learn when viewing signals in the transformed domain? Can we use transforms to analyze not only signals but systems? Great, we know how an LTI system works but what if I want to find its parameters based on the observed input/output data. This leads to the estimation theory part of the class where we will consider the fundamentals of this beautiful theory and exercise its uses in everyday practice of a scientist from estimation of a sample mean to solving the inverse problems in neuroimaging or spectroscopy. Importantly, the class reflects the author's point of view on the described topics that was formed in the mind of the author of this class who first obtained an engineering degree and then had a chance to go over the extensive and quite a broad training in signal processing. The author is also a practicing signal processing engineer and a scientist whose research is focused on development of practical algorithms for analysis of multivariate data in the field of neuroimaging.

7. Schedule National Research University Higher School of Economics One pair consists of 2 academic hours for lecture or 2 academic hours of practical session (seminar) Please, see the Course description section for assignment of lectures. Topic Total Contact hours hours Lectures Seminars Self-study 1. Signals 20 4 4 12 2. Transforms and decompositions 26 4 4 18 3. Statistical description of signals in the original and transformed spaces 26 4 4 18 4. Signal models 20 4 4 12 5. Systems 22 4 4 14 6. Estimation theory 22 4 4 14 7. Practical examples and Summary 16 4 4 8 Total: 152 28 28 96 8. Requirements and Grading Type of grading Final 9. Assessment Type of work Characteristics # Homework(HW) 20 Solving homework tasks and examples, weekly Written exam, solving Midterm exam 2 problems similar to those in the (MTE) HWs Written exam, 120 min, Final exam (FE) 1 solving problems similar to those in the HWs Grade formula 0.5*HW+0.15*MTE1+0.15*MTE2 +0.2*FE The assessment consists of classwork and homework, assigned after each lecture. Students have to demonstrate their knowledge in each lecture topic concerning both theoretical facts, and practical tasks solving. All tasks are connected through the discipline and have increasing complexity. Intermediate assessment is in the form of two midterm exams. Students have to demonstrate knowledge of theory facts, but the most of tasks would evaluate their ability to solve practical examples, present straight operation, and recognition skills to solve them. Final assessment is the final exam. Students have to demonstrate knowledge of theory facts, but the most of tasks would evaluate their ability to solve practical examples, present straight operation, and recognition skills to solve them. While comprehensive will primarily focus on the second part of the class (Estimation theory) The grade formula: Each exam (2 midterms and the final) will consist of 5 problems, giving two marks each Final course mark is obtained from the following formula: О final = 0,5*О hw+0,15*о mte1 +0,15*О mte2 + 0.2*O FE

The grades are rounded in favour of examiner/lecturer with respect to regularity of class and home works. All grades, having a fractional part greater than 0.5, are rounded up. Table of Grade Accordance 1 - very bad 2 bad 3 no pass Ten-point Grading Scale Five-point Grading Scale Unsatisfactory - 2 4 pass 5 highly pass Satisfactory 3 6 good 7 very good Good 4 8 almost excellent 9 excellent 10 perfect Excellent 5 FAIL PASS 10. Course Description The following list describes main topics in the order as they will be presented. Lecture numbers are given in square brackets Signals [1,2] 1.1 Deterministic Signals (discrete, continuous) 1.2 Random signals, random processes Mean, std, autocorrelation function, distribution function Properties of random signals (stationarity, ergodicity) Scalar and vector random processes 1.3 Signals as vectors 1.4 Hilbert space, infinite dim Eucledian space, notion of distance 1.5 Demonstrated equivalence of discrete and continuous forms of treatment Transforms and decompositions [3,4,5] 2.1 General idea of a transform 2.2 A transform as a change of basis 2.3 Calculating transform coefficients or why do we need an orthogonal basis? 2.4 Fourier transform Periodic signals Aperiodic signals Discrete Fourier transform 2.5 State-space representation and discrete transforms as a matrix-vector multiplication 2.6 Time-frequency representation of signals 2.7 Short-time Fourier transform 2.8 Wavelet transform Statistical description of signals in the original and transformed spaces [6,7,8] 3.1 Second order statistical description of random signals revisited 3.2 Periodogram, Power spectral density(psd) 3.3 Wiener-Hinchin theorem 3.4 Methods to compute PSD Signal models [9]

4.1 Autoregressive model 4.2 Autoregressive moving average model 4.3 Parameter estimation: problem statement Systems [10,11,12,13] 5.1 Input-output relation, a system as an operator 5.2 Calculation of an output of a system for an arbitrary input, pulse response 5.3 Linear time invariant (LTI) systems 5.4 Transient processes 5.5 Passing random process through an LTI system 5.6 Time domain 5.7 Frequency domain, Transfer function or why a linear system cannot bring in new frequencies 5.8 Z- transform as a method for analysis of discrete-time LTIs 5.9 Digital Temporal filters as most frequently used LTIs 5.10 FIR, IIR filters, 5.11 Stability, implementation details 5.12 Digital filters as shaping filters for ARMA processes 5.13 Synthesis of signals with desired second order statistical properties Estimation theory [14-20] 6.1 Properties of an estimate, criteria overview 6.2 Notion of an optimal estimator 6.3 Cramer-Rao lower bound 6.4 The concept of a general minimum variance unbiased (MVU) estimator 6.5 Sufficient statistics, finding an MVU estimator 6.6 Best linear unbiased estimator (BLUE) Scalar case Vector case 6.7 Maximum Likelihood (ML) estimation Gaussian mixture model, Expectation-Maximization algorithm, clustering 6.8 The Bayesian philosophy 6.9 Maximum aposteriori probability (MAP) estimators 6.10 Linear vector observation model Estimation of parameters in white Gaussian noise (WGN) case Estimation of parameters in colored Gaussian noise (CGN) case Wiener filter( time and frequency domain) 6.11 Process model 6.12 Kalman filter (smoother, predictor) 6.13 Wiener filter vs Kalman filter Examples and Summary [21, 22] 11. Term Educational Technology The following educational technologies are used in the study process: discussion and analysis of the results of the home task in the group; individual education methods, which depend on the progress of each student; analysis of skills to formulate common problem in terms of mathematics and solve it; 12. Recommendations for course lecturer

Course lecturer is advised to use interactive learning methods, which allow participation of the majority of students, such as slide presentations, combined with writing materials on board, and usage of interdisciplinary papers to present connections between mathematics and psychology. The course is intended to be adaptive, but it is normal to differentiate tasks in a group if necessary, and direct fast learners to solve more complicated tasks. 13. Recommendations for students The course is interactive. Lectures are combined with classes. Students are invited to ask questions and actively participate in group discussions. There will be special office hours for students, which would like to get more precise understanding of each topic. Teaching assistant will also help you. All tutors are ready to answer your questions online by official e-mails that you can find in the contacts section. 14. Final exam questions 1. Deterministic Signals(discrete, continuous) 2. Random signals, random processes 3. Mean, std, autocorrelation function, distribution function 4. Properties of random signals (stationarity, ergodicity) 5. Scalar and vector random processes 6. Signals as vectors, Hilbert space, infinite dim Eucledian space, notion of distance 7. Demonstrate equivalence of discrete and continuous forms of treatment 8. General idea of a transform 9. A transform as a change of basis 10. Calculating transform coefficients or why do we need an orthogonal basis? 11. Fourier transform, periodic signals 12. Fourier transform, Aperiodic signals 13. Discrete Fourier transform 14. State-space representation and discrete transforms as a matrix-vector multiplication 15. Time-frequency representation of signals, Short-time Fourier transform 16. Time-frequency representation of signals, Wavelet transform 17. Second order statistical description of random signals( Periodogram, Power spectral density(psd)) 18. Wiener-Hinchin theorem, methods to compute PSD 19. Autoregressive model 20. Autoregressive moving average model 21. Parameter estimation: problem statement 22. Input-output relation, a system as an operator 23. Calculation of an output of a system for an arbitrary input, pulse response, Linear time invariant (LTI) systems 24. Transient processes 25. Passing random process through an LTI system (time domain analysis) 26. Passing random process through an LTI system (frequency domain, transfer function) 27. Z- transform as a method for analysis of discrete-time LTIs 28. Digital Temporal filters as most frequently used LTIs 29. FIR, IIR filters, stability, implementation details

30. Digital filters as shaping filters for ARMA processes 31. Synthesis of signals with desired second order statistical properties 32. Parameter estimation, problem statement, examples 33. Notion of an optimal estimator 34. Cramer-Rao lower bound 35. The concept of a general minimum variance unbiased (MVU) estimator 36. Sufficient statistics, finding an MVU estimator 37. Best linear unbiased estimator (BLUE), scalar case 38. Best linear unbiased estimator (BLUE), vector case 39. Maximum Likelihood (ML) estimation 40. Gaussian mixture model, Expectation-Maximization algorithm, clustering 41. Maximum aposteriori probability (MAP) estimators 42. Linear vector observation model : estimation of parameters in white Gaussian noise (WGN) case 43. Linear vector observation model : estimation of parameters in colored Gaussian noise (CGN) case 44. Wiener filter( time and frequency domain) 45. Kalman filter (smoother, predictor) 46. Wiener filter vs. Kalman filter 15. Reading and Materials This course is intended to provide students with the unfying view on system's theory nd help them to develop the corresponding intuition. Unfortunately, there is know single book and we will draw from a range of books listed below. 16. Required Reading Selected chapters from Oppenheim A. and Schafer R. (1989 or newer) Discrete time signal processing, Prentice Hall signal processing series, ISBN 0-13-216292-X. Sharma S. (1996) Applied multivariate techniques, John Wiley & Sons, Inc Kay S. (1993) Fundamentals of statistical signal processing, Prentice Hall signal processing series, ISBN 0-13-345711-7. Casella G., Berger R.(1990) Statistical inference, Wadsworth and Brooks/Cole Advanced Books & Software, Pacific Grove, California, ISBN 0-534-11958-1 Strang, G. (2009). Introduction to Linear Algebra, 4 th edition, Wellesley-Cambridge Press. 17. Recommended Reading Mark Wickert, Signals and systems for dummies, paperback, ISBN-13: 978-1118475812, 1 st edition, S.L. Marple Jr, Digital Spectral Analysis with Applications: Second Edition, 2014, Prentice Hall signal processing series 18. List of papers for review Will be provided as the class progresses

19. Course telemaintenance National Research University Higher School of Economics All materials of the discipline are posted in informational educational site at NRU HSE portal www.hse.ru. Students are provided with links on relevant papers, tests, electronic books, articles, etc. 15. Equipment The course requires a laptop, projector, and acoustic systems. Lecture materials, course structure and the syllabus are prepared by Alexei Ossadtchi, Ph.D.