Loop invariants

A loop invariant is a condition that is true at the beginning and end of every loop iteration, analogously to the way that a class invariant is true at the beginning and end of every public method. When you write a loop that works correctly, you are at least implicitly relying on a loop invariant. Knowing what a loop invariant is and thinking explicitly about loop invariants will help you write correct, efficient code that implements tricky algorithms.

Binary search via iteration

Suppose we want to find an element in a sorted array. We can do much better than scanning from left to right: we can use binary search. Here is the binary search algorithm, written as a loop.

binary_search.java

Conceptually, this algorithm is simple. But it is deceptively tricky to get exactly right. How do we know we got the computation of m right? Why is it k <= a[m] and not k < a[m]? Why m and m+1 in the two updates to r and l respectively? If we change any of these decisions, the algorithm can fail to find the correct element.

Binary search loop invariant

To convince ourselves that we wrote the correct code, we need a loop invariant with three clauses:

1. a is sorted in ascending order
2. l≤r
3. k ∈ a[l..r]

Note that we use the notation i..j to denote the set {x | i ≤ x ≤ j} = {i,i+1,...,j-1,j}. We use the notation a[i..j] to indicate the subsequence of the array a starting from a[i] and continuing up to and including a[j].

If we know what the loop invariant is for a loop, it is often a good idea to document it. In fact, we can document it in a checkable way by using an assert statement that is executed on every loop iteration.

Using loop invariants to show code is correct

Loop invariants can help us convince ourselves that our code, especially tricky code, is correct. They also help us develop code to be correct in the first place, and they help us write efficient code.

To use a loop invariant to argue that code does what we want, we use the following steps:

1. Establishment. Show that the loop invariant is true at the very beginning of the loop. (also known as Initialization)
2. Preservation. Show that if we assume the loop invariant is true at the beginning of the loop, it is also true at the end of the loop. (Other than coming up with the loop invariant in the first place, this step is typically the most challenging.) Also known as Maintenance.
3. Postcondition. Show that if the loop guard is false (so the loop exits) and the loop invariant still holds, the loop must have achieved what is desired. This is a crucial step too. If the chosen loop invariant is too weak, this step will not be possible.

These three steps allow us to conclude that the loop satisfies partial correctness, which means that if the loop terminates, it will succeed. To show total correctness, meaning that the loop will always terminate, there is a fourth step:

4. Termination. Assuming the loop invariant holds at the start of each iteration, show that some quantity strictly decreases, and that it cannot decrease indefinitely without making either the loop guard or the loop invariant false. This quantity is called the decrementing function or loop variant.

Let's try these four steps on the binary search algorithm.

1. Establishment. The loop invariant has three parts:
   1. The array is sorted because that's a precondition of the method.
   2. Since a.length is at least 1, l≤r.
   3. k is in a[l..r] because that's the whole array and the precondition guarantees k is there.
2. Preservation. First, notice that the array is never changed in the loop, so part (1) of the invariant is preserved.

   We use l', r' to represent the values of l and r at the end of the loop. Then part (2) requires l'≤r' and part (3) requires k∈a[l'..r']. Notice that m is the average of l and r, rounded down. So we know that l≤m≤r. We know that either k∈a[l..m] or k∈a[m+1..r]. We analyze the two cases separately.

   • Case k∈a[l..m]:
     

     In this case we must have k≤a[m], so the if guard is true and r' = m and l' = l. We have l'≤r' as required, since l≤m. Since k∈a[l..m] by assumption, k∈a[l'..r']. No changes were made to the array, so the array is still sorted

   • Case k∉a[l..m]:
     

     In this case we must have r>m≥l and k∈a[m+1..r]. Since k is not in the required range of the array and the array is sorted, the if condition must be false. Therefore, we have l'= m+1 and r' = r. Since r>m, l'>r' as required. We know No changes were made to the array, so the array is still sorted.

3. Postcondition. For the algorithm to be correct, we need a[l] = k. If the loop guard is false, we know l≥r. But the invariant (2) guarantees l≤r, so this can happen only if l=r. We know from the invariant (3) that k∈a[l..r], which has been reduced to a single element that must be where k is.
4. Termination. The value r−l is guaranteed by the invariant (2) to be non-negative. In the case where k∈a[l..m], we know m<r, so l'−r' < l−r. In the other case, we know l<m+1, so again, l'−r' < l−r. Because integer division rounds down, it gets smaller on every loop iteration. Therefore the loop eventually terminates.

This loop invariant has three clauses, but it's easy to leave things out of the loop invariant. If clauses are omitted from the loop invariant, it makes Establishment easy to argue, but often it becomes impossible to show Preservation or Postcondition. (This is the usual error.) If the loop invariant has extra things in it that aren't really true during the whole loop execution, Establishment or Preservation become impossible to show.

Let's consider what would have happened had we omitted any of the three clauses from the binary search loop invariant:

1. a is sorted in ascending order.

   Without this clause, we can't show Preservation, because there is no guarantee that the updated range a[l'..r'] contains the desired element.

2. l≤r

   Without this clause, we don't know that we are going to the correct side when we split on m. The Termination argument also fails because the decrementing function is no longer guaranteed to be nonnegative.

3. k ∈ a[l..r]

   Without this clause, we don't know that the loop has found anything when it terminates, so Postcondition fails.

Example: Exponentiation by squaring and multiplication

Here is an implementation of exponentiation that is efficient but whose correctness is not instantly apparent.

Pow.java

Intuitively, what this algorithm does is to convert the exponent e into a binary representation, which we can think of as a sum of powers of 2: e = (2^k₁ + 2^k₂ + ...). So x^e = x^(2k₁)·x^(2k₂)· ... . By repeatedly dividing y and inspecting the resulting parity, The algorithm finds each of the “1 digits” in the binary representation, corresponding to the terms 2^k_i, and for such a digit at position k, multiplies into r the appropriate factor x^(2k). However, the loop invariant will help convince us that it really does work. The loop invariant captures that part of the final result has been transferred into r and what remains is b^y.

Let's consider the four steps outlined above.

1. Establishment. Initially, r=1, b=x and y=e, so trivially we have r·b^y = x^e.
2. Preservation.Let us use y', b', and r' to refer to the values of these variables at the end of the loop. We need to show that if r·b^y = x^e at the beginning, then r'·b'^y' = x^e at the end. There are two cases to consider:
   • Case: y is even. In this case, r' = r, y' = y/2, and b'= b². Therefore, r'·b'^y' = r·(b²)^y/2 = r·b^y, as desired.
   • Case: y is odd. Here we have r'=r·b, y' = (y-1)/2, and b' = b². Therefore, r'·b'^y' = r·b·(b²)^(y-1)/2 = r·b·(b)^(y-1) = r·b^y, again.
3. Postcondition. If the loop guard is false, then y = 0, because y can never become negative by dividing it by 2. If y = 0, then r·b^y = r, so r must be equal to x^e.
4. Termination. Dividing by two makes the quantity y smaller on every loop iteration, because it is always nonnegative (nonnegativity is actually a second clause in the loop invariant). It can never become negative, so eventually it will become zero and the loop will terminate.

Therefore, the loop terminates and computes the correct value for x^e in variable r.

Example: Insertion sort

insertion_sort.java

There are two loops, hence two loop invariants. These loop invariants can be visualized with the following diagram:

Notice that the loop invariant holds in for loops at the point when the loop guard (i.e., i < a.length) is evaluated, and not necessarily at the point when the for statement starts executing. That is, the initialization expression in the for statement can help establish the loop invariant.

Loop invariants in software engineering

Loop invariants capture key facts that explain why code works. This means that if you write code in which the loop invariant is not obvious, you should add a comment that gives the loop invariant. This helps other programmers understand the code, and helps keep them (or you!) from accidentally breaking the invariant with future changes.

If you have figured out (even part of) the loop invariant, it also makes sense to add an assertion that checks the loop invariant on every iteration. Such assertions will tend to quickly expose problems with your understanding of why the code works, and coding errors when implementing the loop.