CSS 434: Parallel and Distributed Computing - Fall 2024

CSS 434: Parallel and Distributed Computing
Fall 2024
MW 330-530pm at UW2 131
Prof. Munehiro Fukuda

Professor:

Munehiro Fukuda <mfukuda@uw.edu>, UW1-136, phone 352-3459, office hours: MW 2:30-3:15pm

Course Description:

This course introduces the concepts and design of distributed computing systems. Topics covered include message passing, remote procedure calls, process management, migration, mobile agents, distributed coordination, distributed shared memory, distributed file systems, fault tolerance, and grid computing.

The first five weeks focus on the basic mechanism and the Java programming techniques for message passing, process management, and migration. We will use Java sockets, MPI: Message Passing Interface, Java RMI, and UWAgent: a Java-based mobile agent system the professor and his former CSS students have designed. The last five weeks discuss advanced topics, where the instructor will overview each topic; each student will review a topic-related research paper; and all of us will be involved in group discussions on those papers.

Four programming assignments are given: (1) A chat server design with Java sockets, (2) A parallel-computing application programming with MPI Java, (3) A functional/performance comaprison between Java RMI and UWAgent, and (4) A distributed file system implementation using Java RMI. Two lab sessions are planned for each of the first three assignments to get familiar with sockets, MPI, RMI, and mobile agents. Additionally, three more lab sessions are scheduled for the last three weeks to understand how to use distributed-computing tools: Spark, Hazelcast, and Zookeeper.

Prerequisites:

CSS343.

Work Load and Grading:

Course Work Percentage Achievements Approximately Corresponding Numeric Grade
Program 1 10% 90s 3.5 - 4.0
Program 2 10% 80s 2.5 - 3.4
Program 3 10% 70s 1.5 - 2.4
Program 4 10% 60s 0.7 - 1.4
Lab. Work 0.5% x 9 = 4.5%
Weekly surveys 2% x 4 = 8%
Presentation 2.5%
Discussions 0.5% x 10 = 5%
Midterm Exam 20%
Final Exam 20%

Textbooks:

The lecture covers the following two textbooks, (i.e., 70% weight on van Steen's and 20% weight on Coulouris') , while the rest focuses on some advanced topics such as MPI, mobile agents, and some research-oriented topics. The textbooks are not mandatory.
  1. Distributed Systems, 4th Edition, Maaten van Steen and Andrew S. Tanenbaum, 2023
  2. Distributed Systems: Concepts and Design, 5th Edition, George Coulouris, Jean Dollimore, Tim Kindberg, and Gordon Blair, Addison-Wesley Publishers, 2012

Some Programming Textbooks:

The following books and manuals are useful for system, network, and MPI programming.

Policies:

All programming assignments are to be done independently. Any collaboration of work will result in severe penalty. You may discuss the problem statement and any clarification with each other, but any actual work to be turned in, must be done without collaboration.

Any programming homework is due at 11:59pm on its given due date and must be submitted to Canvas. Once the submission window gets closed, no late submission is accepted even through email (firm). The submission may be postponed only in emergencies such as accidents, sickness, sudden business trips, and family emergencies, in which case you may turn in yor homework late with a written proof. No make-up exams will be given except under exceptional circumstances. Barring emergencies, I must be informed before the exam.

Each weekly survey report is due at the beginning of the class hours on its given due date and must be submitted to Canvas in a group. Once the submission window gets closed, no late submission is accepted even through email (firm). The submission may be done by each group representative. The submission cannot be postponed at all, as it is a group submission.

To request academic accommodations due to a disability, please contact Disability Resources for Stuents (DRS) in UW1-080, (email: uwbdrs@uwb.edu, Phone: 425-352-5307, and FAX: 425-352-5114). If you have a documented disability on file with the DSS office, please have your DSS counselor contact me and we can discuss accommodations. For assistance from Veteran & Military Community Programs, please email Linh Thai, Program Manager for Veterans & Military Community Programs, at leon2004@uw.edu or call 425.352.3943.

The class follows the policies below. Please check:

Course Goals:

The overall goal of CSS 434, "Parallel and Distributed Computing" includes: To strengthen your understanding of fundamental concepts, you are recommended to solve the problems that are given on the final page of each lecture slide, (while some of them will be solved in class discussions.). To survey esearch/commertial systems, you must read research papers every week, summarize your work holistically in your own words (rather than cut and paste phrases from what you read), respond to given questions, and prepare for an in-class presentation. Finally, you need to use Linux machines (cssmpi1h.uwb.edu - cssmpi12h.uwb.edu) for testing and evaluating the performance of your distributed program. Therefore, as with most technical courses, besides ability and motivation, it takes time to learn and master the subject. Expect to spend an additional 10 to 15 hours a week outside of class time on the average.

Assignments:

This is a research-flavored course. Each assignment specificaiton only gives you a topic and a guideline in order to work on the assignment. The answer and the quality of assignment work just depend on your enthusiuasm for assignment work. Therefore, there are no specific key answers.
  1. Program 1 (pdf, word): learns how to use Java-based TCP communication through a design of chat client and server program.
  2. Program 2 (pdf, word): exercises how to write a parallel-computing application using MPI Java.
  3. Program 3 (pdf, word): compares RMI and mobile agenets in terms of programmability and performance. Click here to download the UWAgent manual.
  4. Program 4 (pdf, word): designs and implements a very simple distributed file system.
  5. Survey 1-4: gives each group a couple of research papers every week. Each group will summarize their understanding in a weekly report. Furthermore, they will give a 30-minute presentation in turn (just one time through the quarter).
Please read assignment.html to understand the environment you use for assignments and the submission/grading procedures.

Laboratory Work:

We have nine laboratory assignments in that we will understand the usage of and programming with Java sockets, MPI, mobile agents, Spark, Hazelcast, and Zookeeper. They are intended to mitigate your learning curve of new programming paradigms and ease your programming work. The grading is based on your active participation.
  1. Lab Work 1a: is a hands-on exercise for Java sockets. (0.5% of an entire grade)
  2. Lab Work 1b: is a hands-on exercise for Java object serialization and deserialization. (0.5% of an entire grade)
  3. Lab Work 2a: is a hands-on exercise for setting up and running MPI programs. (0.5% of an entire grade)
  4. Lab Work 2b: is a hands-on exercise for MPI programming. (0.5% of an entire grade)
  5. Lab Work 3a: is a hands-on exercise for RMI. (0.5% of an entire grade)
  6. Lab Work 3b: is a hands-on exercise for distributed computing with mobile agents. (0.5% of an entire grade)
  7. Lab Work 4: is a hands-on exercise for distributed shared hashmap with Hazelcast. (0.5% of an entire grade)
  8. Lab Work 5: is a hands-on exercise for data streaming with Spark. (0.5% of an entire grade)
  9. Lab Work 6: is a hands-on exercise for job coordination with Zookeeper. (0.5% of an entire grade)

Topics covered and tentative 434 spring schedule:

Note that this is an approximate ordering of topics. Chapters will take about the allotted time and not all sections in all chapters are covered.

Week Date Topics Lab and Discussions van Steen's Chapters Coulouris' Chapters Assignment
0 Sep 25 1. Introduction   1 1  
1 Sep 30 2. Architecture   2.1 2  
  Oct 2 3. Foundations of Communication Lab Work 1a (Sockets) 4.1, 4.3 3 Program 1 assigned
2 Oct 7 4. Message-Oriented Communication Lab Work 1b (Sockets) 4.3 4.1 - 4.3  
  Oct 9 5. Reliable Multicast Lab Work 2a (MPI) 8.4.3 4.4  
3 Oct 14 6. MPI and Other Multicast Communication Lab Work 2b (MPI) 4.3 4.6
  Oct 16 7. Remote Procedure Call Lab Work 3a (RMI) 4.2 5 Program 2 assigned
Program 1 due at 11:59pm
4 Oct 21 8. Processes   3.1 - 3.2 7
  Oct 23 9. Code Migration   3.5 7.4.2  
5 Oct 28 10. Mobile Agents Lab Work 3b (Agents) 3.5 7.4.2  
  Oct 30 Midterm exam in class   1 - 4.3, and 8.4.3 1 - 5, 7  
6 Nov 4 11. Clock Synchronization and Global States 5.1 - 5.2 14 Program 3 assigned
Program 2 due at 11:59pm
  Nov 6 12. Mutual Exclusion and Distributed Election Survey 1 due at 3:30pm 5.3 - 5.4 15 Talk by Groups 1 and 5
Program 2 due at 11:59pm
7 Nov 11 Holiday - No Class        
  Nov 13 13. Consitency in Distributed Memory and Cache   7.1 - 7.2 6.5  
8 Nov 18   Survey 2 due at 3:30pm
Lab Work 4a (Hazelcast)		     Talk by Groups 2 and 6
  Nov 20 14. Consistency in File/DB Systems   7.3 12,
16.2.1 - 16.2.2		 Program 4 assigned
Program 3 due at 11:59pm
9 Nov 25   Survey 3 due at 3:30pm
Lab Work 4b (Spark)		     Talk by Groups 3 and 7
  Nov 27 15. Replica Management   7.4 - 7.5 17.1 - 17.3,
18		  
10 Dec 2 16. Fault Tolerance   8 15.4 - 15.5, 17.6  
  Dec 4   Survey 4 due at 3:30pm
Lab Work 4c (ZooKeeper)		     Talk by Groups 4 and 8
11 Dec 9 Final exam in class   5, 7, and 8 12, 14, 15, 16.2.1-16.2.2, 17, and 18  
  Dec 11 Program 4 Deadline       Program 4 due on at 11:59pm