A loop invariant is a condition that is true at the beginning and end of every loop iteration, analogously to the way that a class invariant is true at the beginning and end of every public method. When you write a loop that works correctly, you are at least implicitly relying on a loop invariant. Knowing what a loop invariant is and thinking explicitly about loop invariants will help you write correct, efficient code that implements tricky algorithms.
Suppose we want to find an element in a sorted array. We can do much better than scanning from left to right: we can use binary search. Here is the binary search algorithm, written as a loop.
binary_search.java
Conceptually, this algorithm is simple. But it is deceptively tricky to get
exactly right. How do we know we got the computation of m
right?
Why is it k <= a[m] and not k < a[m]? Why
m
and m+1
in the two updates to r
and
l
respectively? If we change any of these decisions, the algorithm can
fail to find the correct element.
To convince ourselves that we wrote the correct code, we need a loop invariant with three clauses:
a
is sorted in ascending orderl
≤r
a[l..r]
Note that we use the notation i..j
to denote the set {x | i ≤ x ≤
j} = {i,i+1,...,j-1,j}. We use the notation a[i..j]
to indicate
the subsequence of the array a
starting from a[i]
and continuing
up to and including a[j]
.
If we know what the loop invariant is for a loop, it is often a good
idea to document it. In fact, we can document it in a checkable way
by using an assert
statement that is executed on every
loop iteration.
Loop invariants can help us convince ourselves that our code, especially tricky code, is correct. They also help us develop code to be correct in the first place, and they help us write efficient code.
To use a loop invariant to argue that code does what we want, we use the following steps:
These three steps allow us to conclude that the loop satisfies partial correctness, which means that if the loop terminates, it will succeed. To show total correctness, meaning that the loop will always terminate, there is a fourth step:
Let's try these four steps on the binary search algorithm.
We use l', r' to represent the values of l and r at the end of the loop. Then part (2) requires l'≤r' and part (3) requires k∈a[l'..r']. Notice that m is the average of l and r, rounded down. So we know that l≤m≤r. We know that either k∈a[l..m] or k∈a[m+1..r]. We analyze the two cases separately.
In this case we must have k≤a[m], so the if
guard is
true and r' = m and l' = l. We have l'≤r' as required, since
l≤m. Since k∈a[l..m] by assumption, k∈a[l'..r']. No changes were
made to the array, so the array is still sorted
In this case we must have r>m≥l and k∈a[m+1..r]. Since k is not
in the required range of the array and the array is sorted, the
if
condition must be false. Therefore, we have l'= m+1
and r' = r. Since r>m, l'>r' as required. We know
No changes were made to the array, so the array is still sorted.
This loop invariant has three clauses, but it's easy to leave things out of the loop invariant. If clauses are omitted from the loop invariant, it makes Establishment easy to argue, but often it becomes impossible to show Preservation or Postcondition. (This is the usual error.) If the loop invariant has extra things in it that aren't really true during the whole loop execution, Establishment or Preservation become impossible to show.
Let's consider what would have happened had we omitted any of the three clauses from the binary search loop invariant:
a
is sorted in ascending order.
Without this clause, we can't show Preservation, because there is no guarantee that the updated range a[l'..r'] contains the desired element.
l
≤r
Without this clause, we don't know that we are going to the correct side when we split
on m
. The Termination argument also fails because the decrementing
function is no longer guaranteed to be nonnegative.
a[l..r]
Without this clause, we don't know that the loop has found anything when it terminates, so Postcondition fails.
Here is an implementation of exponentiation that is efficient but whose correctness is not instantly apparent.
Pow.java
Intuitively, what this algorithm does is to convert the exponent e into a binary representation, which we can think of as a sum of powers of 2: e = (2^{k1} + 2^{k2} + ...). So x^{e} = x^{(2k1)}·x^{(2k2)}· ... . By repeatedly dividing y and inspecting the resulting parity, The algorithm finds each of the “1 digits” in the binary representation, corresponding to the terms 2^{ki}, and for such a digit at position k, multiplies into r the appropriate factor x^{(2k)}. However, the loop invariant will help convince us that it really does work. The loop invariant captures that part of the final result has been transferred into r and what remains is b^{y}.
Let's consider the four steps outlined above.
Therefore, the loop terminates and computes the correct value for x^{e} in variable r.
insertion_sort.java
There are two loops, hence two loop invariants. These loop invariants can be visualized with the following diagram:
Notice that the loop invariant holds in for
loops at the point
when the loop guard (i.e., i < a.length) is evaluated, and
not necessarily at the point when the for
statement starts
executing. That is, the initialization expression in the for
statement can help establish the loop invariant.
Loop invariants capture key facts that explain why code works. This means that if you write code in which the loop invariant is not obvious, you should add a comment that gives the loop invariant. This helps other programmers understand the code, and helps keep them (or you!) from accidentally breaking the invariant with future changes.
If you have figured out (even part of) the loop invariant, it also makes sense to add an assertion that checks the loop invariant on every iteration. Such assertions will tend to quickly expose problems with your understanding of why the code works, and coding errors when implementing the loop.