Lecture Summary

Additional references

Concurrency

Consider the OCaml function input_line (in the Pervasives module). You might use it to read the first line of a file:

let file = open_in "foo.txt" in
let line = input_line file   in
print_endline line

The type of input_lines is

input_lines : in_channel -> string

It reads a line from the given file and returns it.

But what /really/ happens? For full details, you should take architecture and OS courses, but we can give a caricature. The processor sends a read command over the bus to the hard drive. The hard drive physically moves a mechanical arm to the physical location of the data on the disk, and then it sends a message back to the CPU containing the data. The operating system then notices that your input is ready, and resumes the execution of your program.

This process can take on the order of 10ms (enough to sort an array of 5000 elements). During this time, the operating system schedules other programs to run on the CPU; your program can do nothing else because the call to input_line has not returned.

This is time that your program could be using to do other things. For example, if your program is a server, it might be servicing other clients during that time. If it is a game, it may be drawing graphics or performing a physics calculation. If it is a word processor or web browser, it might be updating the user interface in response to the user scrolling or pressing a button.

Threading

One approach to letting programs take advantage of this time is to use threads. Threads are an abstraction provided by the operating system that allow a program to seem to do multiple things at the same time. In reality, the processor rapidly switches between the threads of various running processes, providing the illusion of simultaneous execution.

If one of the threads of a program is blocked waiting for input, the operating system simply doesn't schedule it, but it does not block the other threads of the same program. This allows the program to continue to make forward progress.

In a sense, using threads to wait for I/O uses one illusion to cancel out the bad effects of another. The first illusion is that while I/O is happening, nothing else can (which is not true --- the OS just runs other programs while yours waits). The second is that if a program has multiple threads then it can do two things at the same time (which is also not true --- the OS just rapidly switches between multiple tasks).

While they make programming easier, these illusions come at a price. For programs that are doing lots of I/O (such as servers and desktop applications), creating lots of threads to have them sit around waiting for input can be prohibitively expensive. Because of this, an alternative approach is becoming popular for many applications.

Asynchronous (event based) programming

In the next few lectures (and PS5) we will explore asynchronous programming.

In an asynchronous model, we forgo both of the above illusions. Instead, we allow the programmer to start an input operation without immediately waiting for the input to become available. The program can then do other useful things while waiting for the input.

In order to actually deal with the input when it does become available, we keep a list of callback functions that should be called sometime after their input arrives. A function called the scheduler, when run, selects one of the callbacks whose input has arrived, and calls it. When the callback returns, the scheduler the selects another callback, and so on.

There are two important rules to keep in mind:

Rule 1: Asynchronous functions are not allowed to block (i.e. wait for external events)

Rule 2: In the land of asynchrony, scheduler calls YOU. It only gets to run when your function returns. In particular, you cannot be interrupted. (and you block everyone else if you violate rule 1)

The Async library

Jane Street's Async library enables asynchronous programming in OCaml. It provides useful abstractions for building asynchronous programs, an interface to various I/O devices (to asynchronously read files, wait for timeouts, communicate on the network, etc), and a scheduler.

Deferred.t and upon

The central type in Async is 'a Deferred.t. An 'a Deferred.t is a box that starts out empty (or undetermined) and can be filled with an 'a (at which point we say it is determined). Once determined, the value cannot be changed in the future.

Let's look at some code to see how this works. Let's start by rewriting the input_line code from above:

let run file =
  open Async.Std

  (** read from file *)
  upon (Reader.read_line file)
       (fun line -> print_endline line))

Let's unpack this a bit. Reader.read_line has type

val Reader.read_line : Reader.t -> string Deferred.t

When executed, it causes the read command to be sent to the disk, and then immediately returns an undetermined string Deferred.t.

upon has the type

val upon : 'a Deferred.t -> ('a -> unit) -> unit

It causes the function given by its second argument to be scheduled to run when the deferred given by its first argument becomes determined. In this case, the function (fun line -> print_endline line) is registered with the scheduler.

When run () is called by the scheduler, the read request will be issued, and the callback function will be scheduled. run will then return to the scheduler. The scheduler will choose another task to run (if one is available, it will wait if there is nothing left to do). At some point in the future, the read request will return a line of the file, and the corresponding Deferred.t will become determined. Sometime after that happens, the callback function will be called with the line passed in as an argument.

Sequencing

Here is another example. Consider the following code:

let run file =
  upon (Reader.read_line file)    (fun _ -> print_endline "line 1");
  upon (after (Time.Span.sec 60)) (fun _ -> print_endline "line 2");
  upon (after (Time.Span.sec 30)) (fun _ -> print_endline "line 3");
  print_endline "line 4";
  ()

There is a new function here: after has the type

val after : Core.Std.Time.Span.t -> unit Deferred.t

It returns a unit deferred that becomes determined after the given time span has elapsed.

What does run () do? The first thing it does is issue a read request to the given file, and associates a callback that prints "line 1" with it. It then starts a timer that will run for 60 seconds, and registers the function that prints "line 2" to be run when the timer determines its deferred. It calls after again to create a deferred that will become determined after 30 seconds, and registers the "line 3" callback to run when that deferred becomes determined. Finally it prints "line 4" and returns.

We are guaranteed that "line 4" will be printed first. The only way for the other three messages to be printed is if the scheduler invokes the corresponding callbacks, but once the scheduler starts running the "run" function, it will not run again until "run" returns.

Lines 1, 2, and 3 may be printed in any order. The scheduler only guarantees that the corresponding callbacks will be executed some time after the deferreds become determined; it gives no guarantees about when or in what order. In class, as we ran through the example, all three of our deferreds happened to be determined before we returned to the scheduler at all; at that point the scheduler is free to choose any of the three runnable callbacks to execute next.

bind

To sequence together multiple operations, we can use the Deferred.bind function. Bind is similar to upon:

val upon : 'a Deferred.t -> ('a -> unit) -> unit val bind : 'a Deferred.t -> ('a -> 'b Deferred.t) -> 'b Deferred.t

[bind d f] runs f after d is determined; f should return a deferred; the deferred returned by bind becomes determined when the deferred returned by f is determined.

For example, suppose we wanted to write a function that begins the following sequence of operations and returns a deferred that becomes determined when they are all done:

we could write it as follows:

let first_line_of_file name =
  bind (Reader.open_file name) (fun file ->
    bind (Reader.read_line file) (fun line ->
      return line
    )
  )

return simply takes a normal value x of type 'a and creates a deferred that is immediately determined with the value x. We use it here because the second argument to bind needs to be a function that returns a deferred, not a normal value.

The infix operator (>>=) is shorthand for bind:

let (>>=) d f = bind d f

it makes writing asynchronous functions very natural. You should read

d >>= fun x ->
e

(which is parenthesized implicitly as (d) >>= (fun x -> e)) as

let x = d in
e

or alternatively

> evaluate d (which may start a computation
> when d completes, run e

For example, we can rewrite the above example using this notation:

let first_line_of_file name =
  Reader.open_file name >>= fun file ->
  Reader.read_line file >>= fun line ->
  return line

Compare this to the non-asynchronous version:

let first_line_of_file name =
  let file = open_in name in
  let line = input_line file in
  line