ECE592-064 Digital Image Processing and Introduction to Computer Vision Spring 2017 Time: Tue. Thu. 1:30pm-2:45pm, Location: 01228 EBII Course Description: Visual information plays important roles in our daily life. Nowadays, much of this information is recorded by digital images. Digital images are also the main inputs when we teach a computer to see like us humans (i.e., Computer Vision, CV, or so-called Machine Vision). In this course, digital image processing (DIP) focuses on general principles of image processing, rather than specific applications. This course will also cover some basic topics in computer vision. It treats DIP as a process from images to images (i.e., image in and image out), and thus a low-level vision module. DIP and CV is ubiquitous, with applications including television, tomography, photography, printing, robot perception, medical imaging and remote sensing, to name a few. Topics includes, but not limited to: image acquisition, color representation, image sampling and quantization, image transforms, image filtering in spatial and frequency domains, image restoration and reconstruction from projections, multiresolution processing, local features and edge/line-segment detection, image segmentation and basics in object recognition. Course website: http://www4.ncsu.edu/~twu19/teaching_posts/ece592-064-dip-introcv/ Learning Objectives: The objectives of this course are to: Cover the basic theory and algorithms that are widely used in digital image processing and CV Expose students to current methods and issues that are specific to DIP / CV systems Develop hands-on experience in implementing algorithms to process images Familiarize with a popular toolbox (e.g., MATLAB or Python Image Processing / CV Toolbox) Develop critical thinking about limitations of state-of-the-art in image processing / CV methods Develop solid backgrounds for advance topics in computer vision (e.g., ECE763) Instructor: Tianfu (Matt) Wu 530-24 Venture II (https://maps.ncsu.edu/#/buildings/vc2) North Carolina State University Phone: 919-515-4361 Email: tianfu_wu@ncsu.edu Office Hours: Tue, 3:00pm 5:00pm (Other times by appointment, not guaranteed always)
Texts: (DIP) Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing, 3 rd Edition, Prentice Hall, 2008 (http://www.imageprocessingplace.com/dip-3e/dip3e_main_page.htm) (CV) Richard Szeliski, Computer Vision: Algorithms and Applications, Springer, 2011 (a free version is available for personal use at http://szeliski.org/book/ ) Additional References: Anil K. Jain, Fundamentals of Digital Image Processing, Prentice Hall, 1989. David A. Forsyth and Jean Ponce, Computer Vision: A Modern Approach, 2 nd Edition, Prentice Hall, 2003 Optional Readings: There will be weekly suggested readings. Grading: Homework (25%), Project (20%), Midterm (25%), Final (30%) Attendance Policy: Students are required to attend 85% of the classes and have an average grade of 80% on the homework and project assignments. For complete attendance and excused absence policies, please see http://policies.ncsu.edu/regulation/reg-02-20-03 Requirements for Auditors (AU): Information about and requirements for auditing a course can be found at http://policies.ncsu.edu/regulation/reg-02-20-04 Prerequisites: Students will be expected to be familiar with linear algebra, calculus, probability theory and basic statistics, as well as a decent amount of programming skills. Linear algebra: We will use matrix transpose, inverse, and other operations to do algebra with matrix expressions. We ll use transformation matrices to rotate/transform points, and we ll use Singular Value Decomposition. (If you are a quick learner you should be able to learn them during the class, and in case you haven t yet, we will provide review materials.) o Review material: http://cseweb.ucsd.edu/classes/wi05/cse252a/linear_algebra_review.pdf Calculus: You are expected to be able to take a derivative, and maximize/minimize a function by finding where the derivative equals 0. Basic probability and statistics: You should understand random variables, conditional probability, mean, and variance, etc. o Review material: http://cseweb.ucsd.edu/classes/wi05/cse252a/random_var_review.pdf Programming skills (e.g., Matlab, Python or C/C++): You can use either Matlab or Python in assignments (check the course website for updates on coding resources). Matlab Short Tutorial [A hello-matlab tutorial] (https://learnxinyminutes.com/docs/matlab/)
Official Documents [Overall Reference](https://www.mathworks.com/help/matlab/ ) [Image Processing Toolbox](https://www.mathworks.com/help/images/index.html ) [Computer Vision Toolbox](https://www.mathworks.com/products/computervision.html ) Coding Tips Python [Writing fast matlab code] (http://www.mathworks.com/matlabcentral/fileexchange/5685-writing-fast-matlabcode ) Useful Third-party Toolbox [Piotr Dollar s Matlab Toolbox](https://pdollar.github.io/toolbox/ ) [VLFeat Toolbox](http://www.vlfeat.org/index.html ) Short Tutorial [A hello-python2 tutorial] (https://learnxinyminutes.com/docs/python/ ) [Python Numpy] (https://github.com/kuleshov/cs228- material/blob/master/tutorials/python/cs228-python-tutorial.ipynb ) [Comparision between Matlab matrix and Python Numpy] (https://docs.scipy.org/doc/numpy-dev/user/numpy-for-matlab-users.html ) Official Documents [Python 2](https://docs.python.org/2/ ) Integrated Development Environment (IDE) [Pycharm free community version](https://www.jetbrains.com/pycharm/ ) IPython Notebook [Official documents](http://ipython.org/ ) [Tutorial](http://cs231n.github.io/ipython-tutorial/ ) Interactive Tool for Visualizing Code [Python Tutor](http://pythontutor.com/ ) Useful DIP and CV Toolbox [Scikit-Image](http://scikit-image.org/ ) [Matplot Lib](http://matplotlib.org/ ) [SimpleCV](http://tutorial.simplecv.org/en/latest/index.html# ) [OpenCV Python Wrapper](http://docs.opencv.org/3.0- beta/doc/py_tutorials/py_tutorials.html ) Course Outline: Date Topics Reading Assignments 1/10 Introduction to DIP and CV DIP-Ch01; CV-Ch01 HW0 out
1/12 Image Formation: Cameras and Optics DIP-Ch2.1/.3; CV-Ch2.1 1/17 Image Formation: Light and Color DIP-Ch2.2,6.1/.2; CV-Ch2.2/.3 1/19 Image Sampling and Quantization DIP-Ch2.4/.5; CV-Ch2.3 1/24 Point Operators (transformation and histogram) 1/26 Filtering in the Spatial Domain: Fundamentals 1/31 Filtering in the Spatial Domain: Smoothing, Sharpening and Combined Methods 2/2 Filtering in the Frequency Domain: Background and Continuous Fourier Transform 2/7 Filtering in the Frequency Domain: Discrete Fourier Transform (DFT), 1-D and 2-D 2/9 Filtering in the Frequency Domain: Properties of DFT and Fundamentals 2/14 Filtering in the Frequency Domain: Smoothing and Sharpening 2/16 Image Restoration: Background and Noise Models DIP-Ch3.1/.2/.3, 6.5; CV-Ch3.1 HW0 due. HW1 out DIP-Ch3.4; CV-Ch3.2 DIP-Ch3.5/.6, 6.6 DIP-Ch4.1/.2./3; CV-Ch3.4 DIP-Ch4.4/.5; CV-Ch3.4 DIP-Ch4.6/.7 DIP-Ch4.8/.9, 6.6 DIP-Ch5.1/.2, 6.8 HW1 due HW2 out HW2 due Project1 out 2/21 Image Restoration by Spatial Filtering DIP-Ch5.3 HW3 out 2/23 Image Restoration by Frequency Domain Filtering 2/28 Image Reconstruction: Computed Tomography, Projections and Radon Transform 3/2 Image Reconstruction: Fourier-Slice Theorem and Beam Filtered Backprojections 3/7 Spring break; no class 3/9 Spring break; no class DIP-Ch5.4 DIP-Ch5.11.1/.2/.3 DIP-Ch5.11.4/.5/.6 3/14 Midterm Exam Cover all previous lectures Project1 due Project2 out 3/16 Pyramids and Haar transform DIP-Ch7.1 HW3 due 3/21 Multiresolution Expansions DIP-Ch7.2 3/23 Wavelet Transformations, 1-D and 2-D DIP-Ch7.3/.4 3/28 Morphological Image Processing: operators and transformation DIP-Ch9.1/.2/.3 HW4 out
3/30 Morphological Image Processing: basic algorithms DIP-Ch9.5/.6 4/4 Local Features: Interest Points and Corners DIP-Ch10.2; CV-Ch4.1 Project2 due Project3 out 4/6 Edge Detection and Linking, and Lines DIP-Ch10.2; CV-Ch4.2/.3 4/11 Segmentation by Thresholding DIP-Ch10.3 HW4 due HW5 out 4/13 Segmentation by Split-and-Merge DIP-Ch10.4/.5; CV-Ch5.2 4/18 Segmentation by Active Contours CV-Ch5.1 4/20 Segmentation by Mean shift and model finding 4/25 Basic Concepts in Recognition, Nearest Neighbor Match 4/27 Exam Overview 5/1 Final Exam CV-Ch5.3 DIP-Ch12 HW5 due Project3 due Projects: Project I: Filtering related small project Project II: Image restoration related small project Project III: Image segmentation related small project Late Assignments and Missed Quizzes: Only the University approved reasons will be accepted for missing a quiz (See http://www.ncsu.edu/policies/academic_affairs/pols_regs/reg205.00.4.php ). There are no make-up tests. With proper documentation, the missing test grade will be the weighted average of other assignments. In all cases, signed documentation must be provided to the instructor in order to obtain credit. Policies on Incomplete Grades: If an extended deadline is not authorized by the Graduate School, an unfinished incomplete grade will automatically change to an F after either (a) the end of the next regular semester in which the student is enrolled (not including summer sessions), or (b) by the end of 12 months if the student is not enrolled, whichever is shorter. Incompletes that change to F will count as an attempted course on transcripts. The burden of fulfilling an incomplete grade is the responsibility of the student. The university policy on incomplete grades is located at http://policies.ncsu.edu/regulation/reg-02-50-03. Additional information relative to incomplete grades for graduate students can be found in the Graduate Administrative Handbook in Section 3.18.F at http://www.fis.ncsu.edu/grad_publicns/handbook/ Academic Integrity: Students are required to comply with the university policy on academic integrity found in the Code of Student Conduct found at http://policies.ncsu.edu/policy/pol-11-35-01
Academic Honesty: See http://policies.ncsu.edu/policy/pol-11-35-01 for a detailed explanation of academic honesty. Accommodations for Disabilities: Reasonable accommodations will be made for students with verifiable disabilities. In order to take advantage of available accommodations, student must register with the Disability Services Office (http://www.ncsu.edu/dso), 919-515-7653. For more information on NC State's policy on working with students with disabilities, please see the Academic Accommodations for Students with Disabilities Regulation at http://policies.ncsu.edu/regulation/reg- 02-20-01. Non-Discrimination Policy: NC State University provides equality of opportunity in education and employment for all students and employees. Accordingly, NC State affirms its commitment to maintain a work environment for all employees and an academic environment for all students that is free from all forms of discrimination. Discrimination based on race, color, religion, creed, sex, national origin, age, disability, veteran status, or sexual orientation is a violation of state and federal law and/or NC State University policy and will not be tolerated. Harassment of any person (either in the form of quid pro quo or creation of a hostile environment) based on race, color, religion, creed, sex, national origin, age, disability, veteran status, or sexual orientation also is a violation of state and federal law and/or NC State University policy and will not be tolerated. Retaliation against any person who complains about discrimination is also prohibited. NC State's policies and regulations covering discrimination, harassment, and retaliation may be accessed at http://policies.ncsu.edu/policy/pol-04-25-05 or http://www.ncsu.edu/equal_op/. Any person who feels that he or she has been the subject of prohibited discrimination, harassment, or retaliation should contact the Office for Equal Opportunity (OEO) at 919-515-3148.