Time, Clocks, and the Ordering of Events in a Distributed System

Notes by Jia Wang (3/3/98)

• distributed system - a collection of spatially separated distinct processes which message transmission delay is not negligible compared to the time between events in a single process
• the relation "happen before" - a partial ordering on the events
• extend to total ordering and use it to solve synchronization problems  

• a collection of distinct processes
  • processes are spatially separated
  • each process consist of a sequence of events (automatic)
  • communication between processes via message transmission
  • message transmission delay is not negligible compared to the time between events in a single process

• the relation "® " on the set of events of a system is the smallest relation satisfying the following three conditions
  • if a and b are events in the same process, and a comes before b, then a® b
  • if a is the sending of a message by one process and b is the receipt of the same message by another process, then a® b
  • if a® b and b® c then a® c
• if a® b then a can affect b
• two distinct events a and b are concurrent if a¤ ® b and b¤ ® a (a¤ ® a)
• the relation "® " is an irreflexive partial ordering on the set of all events in the system

• logical clock C

• clock condition: for any events a, b, if a® b then C<a> < C<b>.
• implementation rules
  • IR1: each process P_i increments C_i between any two successive events
  • IR2: (a) if event a is the sending of a message m by process P_i, then the message m contains a timestamp T_m = C_i<a>; (b) upon receiving a message m, process P_j sets C_j greater than or equal to its present value and greater than T_m.
• Ordering the events totally
  • Use any arbitrary total ordering p of the process
  • Relation Þ : if a is an event in process P_i and b is an event in process P_j, then a Þ b iff either (i) C_i<a> < C_j<b> or (ii) C_i<a> = C_j<b> and P_i p P_j
  • Þ defines a total ordering

• mutual exclusion problem
  • a fixed collection of processes share a single resource, only one process can use the resource at a time
  • conditions
    • a process which has been granted the resource must release it before it can be granted to another process
    • different requests for the resource must be granted in the order in which they are made
    • if every process which is granted the resource eventually releases it, then every request is eventually granted
  • assume that resource is initially granted to exactly one process
• distributed algorithm - no central synchronizing process or central storage
  • some assumptions
    • for any two processes P_i and P_j, the message sent from P_i to P_j are received in the same order as they are sent
    • every message is eventually received
    • a process can send message directly to every other process
  • algorithm: implement a system of logical clocks with rule IR1 and IR2, and use them to define a total ordering Þ of all events
  • requires the active participation of all the processes
  • causes anomalous behavior if casual relationships happen outside of the system:
  • avoid anomalous behavior by using properly synchronized physical clocks which run at approximately equal rate and have some bounds on transmission time of messages

• To give a distributed algorithm to solve the mutual exclusion problem, some assumptions are made as stated above. Are those assumptions reasonable in the real distributed system?
• Author illustrated the use of total ordering of events by solving mutual exclusion problem. Do you think any other usage of total ordering of events?