# Streams and Laziness
* * *
*
Topics:
* infinite data structures
* streams
* thunks
* lazy evaluation
*
* * *
## Infinite data structures
We already know that OCaml allows us to create recursive functions—that is,
functions defined in terms of themselves. It turns out we can define other values
in terms of themselves, too.
```
# let rec ones = 1::ones;;
val ones : int list = [1; ]
# let rec a = 0::b and b = 1::a;;
val a : int list = [0; 1; ]
val b : int list = [1; 0; ]
```
The expressions above create *recursive values*. The list `ones` contains an
infinite sequence of `1`, and the lists `a` and `b` alternate infinitely between
`0` and `1`. As the lists are infinite, the toplevel cannot print them in their
entirety. Instead, it indicates a *cycle*: the list cycles back to its beginning.
Even though these lists represent an infinite sequence of values, their representation
in memory is finite: they are linked lists with back pointers that create those cycles.
There are other kinds of infinite mathematical objects we might want to represent
with finite data structures:
* Infinite sequences, such as the sequence of all natural numbers, or the sequence of
all primes, or the sequence of all Fibonacci numbers.
* A stream of inputs read from a file, a network socket, or a user. All of these are
unbounded in length, hence we can think of them as being infinite in length. In fact,
many I/O libraries treat reaching the end of an I/O stream as an unexpected situation
and raise an exception.
* A *game tree* is a tree in which the positions of a game (e.g., chess or tic-tac-toe)_
are the nodes and the edges are possible moves. For some games this tree is
in fact infinite (imagine, e.g., that the pieces on the board could chase each other
around forever), and for other games, it's so deep that we would never want to
manifest the entire tree, hence it is effectively infinite.
Suppose we wanted to represent the first of those examples: the sequence of all
natural numbers. Some of the obvious things we might try simply don't work:
```
# let rec from n = n :: from (n+1);;
# let nats = from 0;;
Stack overflow during evaluation (looping recursion?).
# let rec nats = 0 :: List.map (fun x -> x+1) nats;;
Error: This kind of expression is not allowed as right-hand side of let rec
```
The problem with the first attempt is that `nats` attempts to compute the entire
infinite sequence of natural numbers. Because the function isn't tail recursive,
it quickly overflows the stack. If it were tail recursive, it would go into an
infinite loop.
The second attempt doesn't work for a more subtle reason. In the definition of a
recursive value, we are not permitted to use a value before it is finished being
defined. The problem is that `List.map` is applied to `nats`, and therefore
pattern matches to extract the head and tail of `nats`, but we are in the middle
of defining `nats`, so that use of `nats` is not permitted.
Let's find another way.
## Streams
A *stream* is an infinite list. Sometimes these are also called sequences, delayed lists,
or lazy lists. We can try to define a stream by analogy to how we can define (finite)
lists. Recall that definition:
```
type 'a mylist =
| Nil
| Cons of 'a * 'a mylist
```
We could try to convert that into a definition for streams:
```
(* doesn't actually work *)
type 'a stream =
| Cons of 'a * 'a stream
```
Note that we got rid of the `Nil` constructor, because the empty list is finite,
but we want only infinite lists.
The problem with that definition is that it's really no better than the built-in
list in OCaml, in that we still can't define `nats`:
```
# let rec from n = Cons (n, from (n+1));;
# let nats = from 0;;
Stack overflow during evaluation (looping recursion?).
```
As before, that definition attempts to go off and compute the entire infinite
sequence of naturals.
What we need is a way to *pause* evaluation, so that at any point in time,
only a finite approximation to the infinite sequence has been computed. Fortunately,
we already know how to do that!
Consider the following definitions:
```
# let f1 = failwith "oops";;
Exception: Failure "oops".
# let f2 = fun x -> failwith "oops";;
val f2 : 'a -> 'b =
# f2 ();;
Exception: Failure "oops".
```
The definition of `f1` immediately raises an exception, whereas the definition of `f2`
does not. Why? Because `f2` wraps the `failwith` inside an anonymous function.
Recall that, according to the dynamic semantics of OCaml, **functions are already
values**. So no computation is done inside the body of the function until it is applied.
That's why `f2 ()` raises an exception.
We can use this property of evaluation—that functions delay evaluation—to
our advantage in defining streams: let's wrap the tail of a stream inside a function.
Since it doesn't really matter what argument that function takes, we might as well
let it be unit. A function that is used just to delay computation, and in
particular one that takes unit as input, is called a *thunk*.
```
(* An ['a stream] is an infinite list of values of type ['a].
* AF: [Cons (x, f)] is the stream whose head is [x] and tail is [f()].
* RI: none.
*)
type 'a stream =
Cons of 'a * (unit -> 'a stream)
```
This definition turns out to work quite well. We can define `nats`, at last:
```
# let rec from n = Cons (n, fun () -> from (n+1));;
val from : int -> int stream =
# let nats = from 0;;
val nats : int stream = Cons (0, )
```
We do not get an infinite loop or a stack overflow. The evaluation of `nats` has
paused. Only the first element of it, `0`, has been computed. The remaining elements
will not be computed until they are requested. To do that, we can define functions
to access parts of a stream, similarly to how we can access parts of a list:
```
(* [hd s] is the head of [s] *)
let hd (Cons (h, _)) = h
(* [tl s] is the tail of [s] *)
let tl (Cons (_, tf)) = tf ()
(* [take n s] is the list of the first [n] elements of [s] *)
let rec take n s =
if n=0 then []
else hd s :: take (n-1) (tl s)
(* [drop n s] is all but the first [n] elements of [s] *)
let rec drop n s =
if n = 0 then s
else drop (n-1) (tl s)
```
It is informative to observe the types of those functions:
```
val hd : 'a stream -> 'a
val tl : 'a stream -> 'a stream
val take : int -> 'a stream -> 'a list
val drop : int -> 'a stream -> 'a stream
```
Note how, in the definition of `tl`, we must apply the function `tf` to `()` to obtain
the tail of the stream. That is, we must *force* the thunk to evaluate at that point,
rather than continue to delay its computation.
We can use `take` to observe a finite prefix of a stream. For example:
```
# take 10 nats;;
- : int list = [0; 1; 2; 3; 4; 5; 6; 7; 8; 9]
```
## Programming with streams
Let's write some functions that manipulate streams. It will help to have
a notation for streams to use as part of documentation. Let's use
`` to denote the stream that has elements `a`, `b`, and `c`
at its head, followed by infinitely many other elements.
Here are functions to square a stream, and to sum two streams:
```
(* [square ] is [ square (tf ()))
(* [sum ] is
* [] *)
let rec sum (Cons (h1, tf1)) (Cons (h2, tf2)) =
Cons (h1+h2, fun () -> sum (tf1 ()) (tf2 ()))
```
Their types are:
```
val square : int stream -> int stream
val sum : int stream -> int stream -> int stream
```
Note how the basic template for defining both functions is the same:
* Pattern match against the input stream(s), which must be `Cons`
of a head and a tail function (a thunk).
* Construct a stream as the output, which must be `Cons` of a new
head and a new tail function (a thunk).
* In constructing the new tail function, delay the evaluation of the
tail by immediately starting with `fun () -> ...`.
* Inside the body of that thunk, recursively apply the function
being defined (square or sum) to the result of forcing a thunk (or
thunks) to evaluate.
Of course, squaring and summing are just two possible ways of mapping
a function across a stream or streams. That suggests we could write
a higher-order map function, much like for lists:
```
(* [map f ] is [] *)
let rec map f (Cons (h, tf)) =
Cons (f h, fun () -> map f (tf ()))
(* [map2 f ] is
* [] *)
let rec map2 f (Cons (h1, tf1)) (Cons (h2, tf2)) =
Cons (f h1 h2, fun () -> map2 f (tf1 ()) (tf2 ()))
let square' = map (fun n -> n*n)
let sum' = map2 (+)
```
And their types are as we would expect:
```
val map : ('a -> 'b) -> 'a stream -> 'b stream
val map2 : ('a -> 'b -> 'c) -> 'a stream -> 'b stream -> 'c stream
val square' : int stream -> int stream
val sum' : int stream -> int stream -> int stream
```
Now that we have a map function for streams, we can successfully define `nats`
in one of the clever ways we originally attempted:
```
# let rec nats = Cons(0, fun () -> map (fun x -> x+1) nats);;
val nats : int stream = Cons (0, )
# take 10 nats;;
- : int list = [0; 1; 2; 3; 4; 5; 6; 7; 8; 9]
```
Why does this work? Intuitively, `nats` is `<0; 1; 2; 3; ...>`, so
mapping the increment function over `nats` is `<1; 2; 3; 4; ...>`.
If we cons `0` onto the beginning of `<1; 2; 3; 4; ...>`, we get
`<0; 1; 2; 3; ...>`, as desired. The recursive value definition is
permitted, because we never attempt to use `nats` until after its definition
is finished. In particular, the thunk delays `nats` from being
evaluated on the right-hand side of the definition.
Here's another clever definition. Consider the Fibonacci sequence
`<1; 1; 2; 3; 5; 8; ...>`. If we take the tail of it, we get
`<1; 2; 3; 5; 8; 13; ...>`. If we sum those two streams, we get
`<2; 3; 5; 8; 13; 21; ...>`. That's nothing other than the tail
of the tail of the Fibonacci sequence. So if we were to prepend
`[1; 1]` to it, we'd have the actual Fibonacci sequence. That's
the intuition behind this definition:
```
let rec fibs =
Cons(1, fun () ->
Cons(1, fun () ->
sum fibs (tl fibs)))
```
And it works!
```
# take 10 fibs;;
- : int list = [1; 1; 2; 3; 5; 8; 13; 21; 34; 55]
```
Unfortunately, it's highly inefficient. Every time we force the computation
of the next element, it required recomputing all the previous elements, twice:
once for `fibs` and once for `tl fibs` in the last line of the definition.
By the time we get up to the 30th number, the computation is noticeably slow;
by the time of the 100th, it seems to last forever.
Could we do better? Yes, with a little help from a new language feature.
## Laziness
The example above with the Fibonacci sequence demonstrates that it would
be useful if the computation of a thunk happened only once: when it is
forced, the resulting value could be remembered, and if the thunk is ever
forced again, that value could immediately be returned instead of
recomputing it. That's the idea behind the OCaml `Lazy` module:
```
module Lazy :
sig
type 'a t = 'a lazy_t
val force : 'a t -> 'a
end
```
A value of type `'a Lazy.t` is a value of type `'a` whose computation
has been delayed. Intuitively, the language is being *lazy* about
evaluating it: it won't be computed until specifically demanded. The
way that demand is expressed with by *forcing* the evaluation with
`Lazy.force`, which takes the `'a Lazy.t` and causes the `'a` inside it
to finally be produced. The first time a lazy value is forced, the
computation might take a long time. But the result is *cached*
aka *memoized*, and any subsequent time that lazy value is forced,
the memoized result will be returned immediately.
(By the way, "memoized" really is the correct spelling of this term.
We didn't misspell "memorized", though it might look that way.)
The `Lazy` module doesn't contain a function that produces a
`'a Lazy.t`. Instead, there is a keyword built-in to the OCaml
syntax that does it: `lazy e`.
* **Syntax:** `lazy e`
* **Static semantics:** If `e:u`, then `lazy e : u Lazy.t`.
* **Dynamic semantics:** `lazy e` does not evaluate `e` to a value.
Instead it produced a *delayed value* aka *lazy value* that,
when later forced, will evaluate `e` to a value `v` and return `v`.
Moreover, that delayed value remembers that `v` is its forced
value. And if the delayed value is ever forced again, it immediately
returns `v` instead of recomputing it.
To illustrate the use of lazy values, let's try computing the 30th
Fibonacci number using the definition of `fibs`, which we repeat
here for convenience:
```
let rec fibs =
Cons(1, fun () ->
Cons(1, fun () ->
sum fibs (tl fibs)))
```
If we try to get the 30th Fibonacci number, it will take a long
time to compute:
```
let fib30long = take 30 fibs |> List.rev |> List.hd
```
But if we wrap evaluation of that with `lazy`, it will return
immediately, because the evaluation of that number has been
delayed:
```
let fib30lazy = lazy (take 30 fibs |> List.rev |> List.hd)
```
Later on we could force the evaluation of that lazy value,
and that will take a long time to compute, as did `fib30long`:
```
let fib30 = Lazy.force fib30lazy
```
But if we ever try to recompute that same lazy value, it will
return immediately, because the result has been memoized:
```
let fib30fast = Lazy.force fib30lazy
```
(The above examples will make much more sense if you try them
in utop rather than just reading these notes.)
Nonetheless, we still haven't totally succeeded. That particular
computation of the 30th Fibonacci number has been memoized,
but if we later define some other computation of another
it won't be sped up the first time it's computed:
```
(* slow, even if [fib30lazy] was already forced *)
let fib29 = take 29 fibs |> List.rev |> List.hd
```
What we really want is to change the representation of streams itself
to make use of lazy values.
**Lazy lists.** Here's a representation for infinite lists using
lazy values:
```
type 'a lazylist =
Cons of 'a * 'a lazylist Lazy.t
```
We've gotten rid of the thunk, and instead are using a lazy value
as the tail of the lazy list. If we ever want that tail to be computed,
we force it.
Now, assuming appropriate definitions for `hd`, `tl`, `sum`, and `take`
(left as an exercise for the reader),
we can define the Fibonacci sequence as a lazy list:
```
let rec fibs =
Cons(1, lazy (
Cons(1, lazy (
sum (tl fibs) fibs))))
(* both fast *)
let fib30lazyfast = take 30 fibs
let fib29lazyfast = take 29 fibs
```
**Lazy vs. eager.**
OCaml's usual evaluation strategy is *eager* aka *strict*:
it always evaluate an argument before function application.
If you want a value to be computed lazily, you must specifically
request that with the `lazy` keyword. Other function languages,
notably Haskell, are lazy by default. Laziness can be
pleasant when programming with infinite data structures.
But lazy evaluation makes it harder to reason about space and time,
and it has bad interactions with side effects. That's one reason
we use OCaml rather than Haskell in this course.
## Summary
The stream data structure can be used to represent an infinite
mathematical sequence, but with only a finite amount of memory.
That's because the values of the sequence are not produced
until they are specifically requested. The thunks used in streams
are used to pause evaluation until such a request is made.
Thunks are a way of implementing lazy evaluation, which OCaml
also has available. The advantage of OCaml's built-in implementation
is that it can memoize results, avoiding the need for recomputation.
## Terms and concepts
* caching
* cycle
* delayed evaluation
* eager
* force
* infinite data structure
* lazy
* memoization
* thunk
* recursive values
* stream
* strict
## Further reading
* *More OCaml: Algorithms, Methods, and Diversions*, chapter 2, by
John Whitington. This book is a sequel to *OCaml from the Very Beginning*.