One of the fastest growing application areas for computers is the processing of multimedia — sound, images, and video. Multimedia places great demands on processing power, network bandwidth, storage capacity, I/O speed, and software design. In this course, you will learn how multimedia information is captured, represented, processed, communicated, and stored in computers. The specific topics we will cover include: physical properties of multimedia source information (sound, images), devices for information capture (microphones, cameras), digitization, compression, digital media representation (JPEG, MPEG), digital signal processing, and network communication. By the end of this course, you should understand the problems and solutions facing multi/hypermedia systems development in the areas of user interfaces, information retrieval, data structures and algorithms, and communications. As a result, you should be well-prepared to work with electrical engineers in the design of advanced signal processing systems (e.g., wireless communication devices) and multimedia computing systems.

Course Objectives

The goals of this course are for you to learn:

• What signals are like in the “real” world and how the properties of multimedia signals (sounds, images, video) affect how we perceive them.
• How to use mathematics as a tool to make problem solving simpler, for example, converting laborious trigonometric computations to straightforward algebra with polynomials.
• How these signals get into the computer, how they are represented within the computer, and the tradeoffs among sampling speed, levels of quantization, and file size.
• What are the basic algorithms that perform simple signal processing to remove noise, emphasize important features, etc. You should be well-prepared to work with electrical engineers in the design of more advanced signal processing systems.
• How multimedia file sizes can be reduced by compression, and the tradeoffs among compression, processing overhead, and media quality.
• How these concepts are applied in multimedia applications and standards.

Prerequisites

This course covers much of the mathematical foundations for understanding signals and signal processing, however, it is assumed that you are familiar with topics such as complex numbers, trigonometry, derivatives, vectors, the basic idea of integrals, infinite series, and basic physics (mass, acceleration, force). CSS 342 and lower division math courses are the only formal prerequisites. While we may do some programming, this is not a programming course.

Instructor

Michael Stiber stiber@u.washington.edu, room UW1-341, phone (425) 352-5280, office hours Mondays 1–2PM or by appointment.

Lectures

Mondays and Wednesdays, 3:30-5:35PM, UW1-050.

Textbook

Michael Stiber & Bilin Stiber, Signal Computing: Digital Signals in the Software Domain, manuscript copies available in the bookstore.

On reserve

A hyperlinked PDF version of the course textbook is available via e-reserve. The following books are also on reserve in the library for further reading:

• J. Crowcroft, M. Handley, & I. Wakeman, Internetworking Multimedia, Morgan Kaufmann, 1999, chapter 4 (§ 4.1–4.5).
• Donald Hearn & M. Pauline Baker, Computer Graphics, Second Edition, Prentice Hall, 1997, chapter 2 (§ 2.1–2.4).
• Martin D. Levine, Vision in Man and Machine, McGraw-Hill, 1985, chapter 1, chapter 2 (§ 2.1, 2.2).
• James H. McClellan, Ronald W. Schafer, and Mark A. Yoder, DSP First: A Multimedia Approach, Prentice Hall, Upper Saddle River, NJ, 1999.
• Alistair Moffat and Andrew Turpin, Compression and Coding Algorithms, Kluwer Academic Publishers, Boston, 2002.
• Mark Nelson and Jean-Loup Gailly, The Data Compression Book, 2nd edition, M&T Books, New York, 1995.
• Ken Pohlman, The Compact Disc Handbook, A-R Editions, 1992.
• K.R. Rao & J.J. Hwang, Techniques & Standards for Image, Video & Audio Coding, Prentice Hall, 1996, chapters 4 & 5.
• Robert S. Tannenbaum, Theoretical Foundations of Multimedia, Computer Science Press, 1998, chapters 1 & 2.
• A. Murat Tekalp, Digital Video Processing, Prentice Hall, 1995, chapters 1, 2, 18, 19, 21.
• Ian H. Witten, Alistair Moffat, and Timothy C. Bell, Managing Gigabytes: Compressing and Indexing Documents and Images, Morgan Kaufmann, San Francisco, 1999.

Software

We will be using J-DSP for the bulk of our computing laboratories. J-DSP is a Java applet that lets you build signal processing systems by assembling “block diagrams”. The initial laboratory assignment will be an orientation to J-DSP.

We are still in the process of customizing and extending J-DSP to suit this class. As a result, a minority of labs will use MATLAB software. MATLAB is available on the CSS Windows and Linux computers, and is accessible remotely via the CSS Linux servers. We will not be using MATLAB enough to justify you purchasing your own copy.

Grading

35% laboratories + 30% midterm + 35% final

Laboratories

You will be completing laboratories for the homework portion of this course. For the most part, each laboratory will involve a PDF file describing what you are to do using J-DSP or possibly MATLAB. You are asked to write up laboratory reports and submit them as hard copy on the due date. No particular format is prescribed; it is your responsibility to ensure that your report clearly shows that you have followed the stated procedures (at a minimum) and unambiguously documents your results. This may require you to include screen captures of J-DSP block diagrams, windows, or graphs. Parts of some labs will also include written (i.e., pencil and paper) portions.

Special needs

If you believe that you have a disability and would like academic accommodations, please contact Disability Support Services at (425) 352-5307, TDD (425) 352-5303, FAX (425) 352-5455, or at rlundborg@uwb.edu. In most cases, you will need to provide documentation of your disability as part of the review process.

Problems

If you have problems with anything in the course, please come and see me during office hours, make an appointment to see me at some other time, or send email. I want to make you a success in this course. Laboratory reports/deliverables represent hard deadlines; this is to prevent your schedule from slipping so much that you won’t be able to complete the class. I will not give out grades of “incomplete” except in extreme circumstances.

Course Outline