CS 677 Distributed Operating Systems (2008)

CS 677 Distributed Operating Systems

Spring 2008

Programming Assignment 3
Turning the Pygmy into an Amazon: Replication, Caching and Consistency

Due: May 13, 2008

You may work in groups of two for this lab assignment.

Purpose of this project is to familiarize you with Replication, Caching and Consistency. We will first use repliction for performance/scalability and then use it for high-availability in the presence of faults.

You can be creative with this project. You are free to use any programming languages (C, C++, Java, etc) and any abstractions such as sockets, RPCs, RMIs, threads, events, etc. that might be needed. You have considerable flexibility to make appropriate design decisions and implement them in your program.

The Problem:

First, Pygmy.com has three NEW books on catalog:

How to finish Project 3 on time
Why theory classes are so hard.
Spring in the Pioneer Valley

The spring break sale turned out to be a big success. However, with the growing popularity of the book store, buyers begin to complain about the time that the system needs to process their query requests. Pygmy.com decides to solve the problem.

This project is based on project 2. This assignment has two parts.

Part I. Replication and Caching

Create a Dispatcher (A), which forwards the requests from the clients to ONE of the three Front end servers.

Each front end server caches results of recently accessed lookup queries (i.e. book id, number of items in stock, and cost) locally. When a new query request comes in, the Front end server checks the cache first before it forwards the request to the catalog/database server.

Cache consistency needs to be addressed whenever a database entry is updated by buy requests or arrival of new stock of books. You have two ways to address the problem:

'Client Pull': Each client check with the server periodically(TTL) about the update.
'Server Push': Every time the database is updated, the database server sends 'invalid' messages to all the front end servers.

A pictorial representation of the system is shown in the figure below:

Part II. Byzantine Fault Tolerance

ALL

Assume one of the three Catalog servers is not functioning correctly, which returns the Dispatcher with faulty information for catalog queries; the other two are reliable. So the Dispatcher needs to compare the information from all three Catalog servers before it returns the correct information back to the Front end servers.

Which Catalog server is not working correctly is randomly selected. You can be creative on how to simulate this in your system.

A pictorial representation of the system is shown in the figure below:

Requirements:

You need to implement BOTH Part 1 and Part 2.

All the requirements of Project 2 still apply to Project 3.

B. Evaluation and Measurement

Compute the average response time (query/buy) of your new systems as before.
Design a test to show whether your dispatcher(A) balance the workload among the three front end servers.
Design a test to show how the cache updating operation affect the performance.
Design a test to show your fault tolerance module does the work as you expect.

Make necessary plots to support your conclusions.

C. What you will submit

When you have finished implementing the complete assignment as described above, you will submit your solution in the form of printouts.

Each program must work correctly and be documented. You should hand in:

A copy of the output generated by running your program. When it receives a book, have your program print a message "bought book book_name ". When a client issues a query (lookup/search), the cache updating begins/ends, the Dispatcher (B) catches a Catalog server not functioning correctly, having your program print the returned results in a nicely formatted manner.
A seperate (typed) document of approximately two pages describing the overall program design, a description of "how it works", and design tradeoffs considered and made. Also describe possible improvements and extensions to your program (and sketch how they might be made).
A program listing containing in-line documentation.
A seperate description of the tests you ran on your program to convince yourself that it is indeed correct. Also describe any cases for which your program is known not to work correctly.
Performance results.
A readme file about how to run your code on EDLAB machines.
After you turn in all the stuff mentioned above, please also send TA an email telling the location of your files .

Let us not waste a lot of trees. So, if any of the above turn out to be large, just save the relevant information in a file, leave it on your EDLAB account and submit the name of the file.

D. Grading policy for all programming assignments

Program Listing
Design Document
Thoroughness of test cases ---------- 10%
Grades for late programs will be lowered 12 points per day late.