Typhoon

Team Members: Hakim Weatherspoon_______________________________________________

Shelley Zhuang___________________________________________________

Matthew Delco________________________________________________

Professor Name: Brewer and Kubi__________________________________________________

Course: CS 262 and CS 252_______________________________________________________

Course Title: Advance Topics in Systems and Advanced Topics in Computer Architecture______

Term: Fall 1999________________________________________________________________

Project Title: Typhoon: An OceanStore - Replication Mechanism for High Availability and Automatic Disaster Recovery using Tornado Codes_____________________________________________________

Current Date: October 1, 1999______________________________________________________.

Typhoon Project Description

- Goal: Create a Replication Mechanism for High Availability and Automatic Disaster Recovery of data using Tornado Codes

Our project goal is to design and create a simple data storage mechanism (support for a file system) that uses a combination of multiple servers and erasure codes (specifically tornado codes) to provide a high level of availability such that nearly half of the constituents of the system could fail without affecting the overall accessibility of data. This system would use a similar amount of aggregate disk space as a mirrored file system, so a definite aspect of the project would be to compare and contrast the performance and features of these types of systems. In order to make the analysis more complete, a comparison to a RAID data storage mechanism (support for a file system) could also be performed. In addition, a comparison would be made between scalar processor implementation vs. a vector processor implementation.

Typhoon Assumptions

- A “file” is a generic term for what the Replication Mechanism manipulates.

- Basic Interface calls are Read and Write file.

- Ocean store is divides into a series of “pools”. Where a pool can be a single node, an SMP, or a cluster.

- A pool is a unit of operation the Naming and Location mechanism, Replication mechanism, and Introspective mechanism.

- And the above mention components reside on a high bandwidth connection to the Network.

- At least one Naming and Location mechanism resides in a pool, and one Replication mechanism resides in a pool.

- A pool in which a file resides, has the entire files “data set”. Where a “data set” is the encoded file using Tornado codes. (i.e. “data set” of a “file” is encoded into 2n pieces for a file of n partitions.

- Another system or mechanism is responsible for copying of data to other pools (replication the data in other pools).

- Exact copies of a “data set” of a “file” can reside in more than “pool”.

- Most of the time the Naming and Location Mechanism will tell us to write to the local pool.

Hypothesis:

Compared to conventional network data delivery systems, such as NFS, a system that utilizes Tornado Codes has the potential to provide a much higher degree of reliability of data. Furthermore, this new system should be able to provide this benefit while providing a comparable level of performance in terms of latency and throughput.

Project Key Deliverables:

Implement a system and perform comparisons to determine how well the system performs to other network file system(s) such as NFS or WinNT. Depending on those results, comparisons could be performed between variants that use different network communicationbv methods, such as UDP, TCP, and/or multicast.

Project Requirements:

- Understand Erasure Codes (Tornado codes).

- Understand Digital Fountain.

- Measure time to encode and distribute file.

- Measure time retrieve and decode(reconstruct) file.

-.

Example of Read and Write Functionality

A. READ

Suppose a user request file X, then:

1) User request file from closest oceanstore component.

2) Naming and Location Mechanism gives back

a) Closest pool which file resides (location).

b) Means to identify file (name).

3) Contact server in pool l and ask for file. Where pool l is the a) above the file is b)

(i.e. ask for file “foobar65” in pool 86).

a) If file not in pool l, then tell “oceanstore component that file does not exist in pool l.

b) Then go back to step to and repeat until find file

(i.e Oceansotre component” will make another request to the Naming and Location Mechanism)

4) Assuming pool l has the specified file (i.e. “data set” of the file), then Replication Mechanism gets back “data set” and reassemble file (decode).

5) send to cache server.

6) send to user.

B. WRITE

Suppose user writes file X, then:

Summary) First, dump/flush file X to local pool, then encode and distribute file X. PERIOD.

-or- below is what closest “oceanstore component” does:

1) Cache server “wants” to flush file.

(i.e. file X is closed -or- 30 second dirty data time limit -or- Seismic Earthquake warning).

2) Ask Naming and Location Mechanism to “give us a name” AND what pool (location) to place file.

3) Assuming cache server and pool server are separate, then send file in its entirety to pool server.

4) Then server in pool will encode and distribute file in the local pool with the specified name.

Expected Outcomes/Critical Success Indicators to measure success:

Indicators used to evaluate the internship/performance review will be completion of milestones as the summer and projects progess. Listed below

Key milestone	Expected completion
1. Form team. Define project. Finish Project Proposal. Define member roles.	Week 0
2.	Week 1
3.	Week 2
4.	Week 3
5.	Week 4
6.	Week 5
7.	Week 6
8.	Week 6

Member Signature: Date:

_______________________________________________ ______________________

Member Signature: Date:

_______________________________________________ ______________________

Member Signature: Date:

_______________________________________________ ______________________