A loop invariant is a condition that is true at the beginning and end of every iteration of a loop. The concept is similar to a class invariant, which must be true at the beginning and end of every public method. When you write a loop that works correctly, you are at least implicitly relying on a loop invariant. Knowing what a loop invariant is and thinking explicitly about loop invariants will help you write correct, efficient code that implements tricky algorithms.
Suppose we want to find an element in a sorted array. We can do much better than scanning from left to right: we can use binary search. Here is the binary search algorithm, written as a loop.
binary_search.java
Conceptually, this algorithm is simple. But it is deceptively tricky to get
exactly right. How do we know we got the computation of m right?
Why is it k <= a[m] and not k < a[m]? Why
m and m+1 in the two updates to r and
l respectively? If we change any of these decisions, the algorithm is
incorrect and will sometimes fail to find the correct element.
To convince ourselves that we wrote the correct code, we need a loop invariant that describes the conditions that we want the loop body to preserve. For this example, our loop invariant has three clauses:
a is sorted in ascending order0≤l≤r≤a.length-1a[l..r]
Note that we use the notation i..j to denote the set {x | i ≤ x ≤
j} = {i,i+1,...,j-1,j}. We use the notation a[i..j] to indicate
the subsequence of the array a starting from a[i] and continuing
up to and including a[j].
If we have a loop and we know what the loop invariant is for that loop, it is often a good
idea to document it. In fact, we can document it in a checkable way
by using an assert statement that is executed on every loop iteration.
Loop invariants can help us convince ourselves that our code, especially tricky code, is correct. They also help us develop code that is correct in the first place, and they help us write efficient code.
To use a loop invariant to argue that code does what we want, we use the following steps:
while (guard) { body }, this means just before entering the loop. For for loops for (init; guard; incr) { body }, this means just after executing init.guard are true
just before executing the loop body, then the loop invariant
is true just after executing the loop body. The loop invariant may fail to be true at intermediate
steps during the execution of the loop body, as long as it is reestablished by the end.
(For for loops, the loop body also includes the increment incr.)
guard is false (so the loop exits)
and the loop invariant holds, then the desired result of the loop has been achieved.Other than coming up with the loop invariant in the first place, the Preservation step is typically the most challenging. The Postcondition step is a crucial step too. If the chosen loop invariant is too weak, this step will not be possible.
These three steps allow us to conclude that the loop satisfies partial correctness, which means that if the loop terminates, it will succeed. To show total correctness, we must show in addition that the loop eventually terminates. To show this, there is a fourth step:
Let's try these four steps on the binary search algorithm.
The loop invariant has three parts:
a.length is at least 1 by the precondition that k is in the array, and since initially l=0 and r=a.length-1, we have 0≤l≤r≤a.length-1.k is in a[l..r] because that is the whole array and the precondition
guarantees that k is there.First, note that the array is never changed in the loop, so part (1) of the invariant is preserved.
We use l', r' to represent the values of l and r at the end of the loop. Then
part (2) requires l'≤r' and part (3) requires k∈a[l'..r'].
Notice that m is the average of l and r, rounded down. So we
know that l≤m<r. (We cannot have m=r by the assumption that the guard is true.)
We know that either k∈a[l..m] or k∈a[m+1..r].
We analyze the two cases separately.
Case k∈a[l..m]:
In this case we must have k≤a[m], so the if guard is
true and r' = m and l' = l. We have l'≤r' as required, since
l≤m. Since k∈a[l..m] by assumption, k∈a[l'..r'].
Case k∉a[l..m]:
In this case we must have r>m≥l and k∈a[m+1..r]. Since k is not
in the required range of the array and the array is sorted, the
if condition must be false. Therefore, we have l'= m+1
and r' = r. Since r>m, l'>r' as required.
For the algorithm to be correct, we need a[l] = k. If the loop guard is false, we know l≥r. But the invariant (2) guarantees l≤r, so this can happen only if l=r. We know from the invariant (3) that k∈a[l..r], which has been reduced to a single element that must be where k is.
The value r−l is guaranteed by the invariant (2) to be nonnegative. In the case where k∈a[l..m], we know m<r, so l'−r' < l−r. In the other case, we know l<m+1, so again, l'−r' < l−r. Because integer division rounds down, it gets smaller on every loop iteration. Therefore the loop eventually terminates.
In this case, the loop invariant has three clauses, but it's easy to leave things out of the loop invariant. If clauses are omitted, the invariant may be too weak: Initialization is easier to argue, but it becomes impossible to show Preservation or Postcondition. On the other hand, if the loop invariant is too strong because it contains clauses that shouldn't be there, then Initialization or Preservation become impossible to show.
Let's consider what would have happened had we omitted any of the three clauses from the binary search loop invariant:
a is sorted in ascending order.
Without this clause, we can't show Preservation, because there is no guarantee
that the updated range a[l'..r'] contains the desired element.
l≤r
Without this clause, we don't know that we are going to the correct side when we split
on m. The Termination argument also fails because the decrementing
function is no longer guaranteed to be nonnegative.
k ∈ a[l..r]
Without this clause, we don't know that the loop has found anything when it terminates,
so Postcondition fails.
Here is an implementation of exponentiation that is efficient but whose correctness is not immediately apparent.
Pow.java
Intuitively, this algorithm converts the exponent e into a binary representation, which we can think of as a sum of powers of 2. So e = 2k1 + 2k2 + ··· and xe = x2k1+2k2+··· = x2k1·x2k2···. (Note that x2k always means x(2k), never (x2)k.) By repeatedly dividing y and inspecting the resulting parity, the algorithm finds each of the “1 digits” in the binary representation, corresponding to the terms 2ki, and for such a digit at position k, multiplies into r the appropriate factor x2k. However, the loop invariant will help convince us that it really does work. The loop invariant captures that part of the final result has been transferred into r and what remains is by.
Let's consider the four steps outlined above.
Initially, r=1, b=x and y=e, so trivially we have r·by = xe.
Let us use y', b', and r' to refer to the values of these variables at the end of the loop. We need to show that if r·by = xe at the beginning, then r'·b'y' = xe at the end. There are two cases to consider:
Case: y is even. In this case, r' = r, y' = y/2, and b'= b2. Therefore, r'·b'y' = r·(b2)y/2 = r·by, as desired.
Case: y is odd. Here we have r'=r·b, y' = (y-1)/2, and b' = b2. Therefore, r'·b'y' = r·b·(b2)(y-1)/2 = r·b·by-1 = r·by, again.
If the loop guard is false, then y = 0, because y can never become negative by dividing it by 2. If y = 0, then r·by = r, so r must be equal to xe.
Dividing by two makes the quantity y smaller on every loop iteration, because it is always nonnegative (nonnegativity is actually a second clause in the loop invariant). It can never become negative, so eventually it will become zero and the loop will terminate.
Therefore, the loop terminates and computes the correct value for xe in variable r.
insertion_sort.java
There are two loops, hence two loop invariants. These loop invariants can be visualized with the following diagram:
Notice that the loop invariant holds in for loops at the point
when the loop guard (i.e., i < a.length) is evaluated, and
not necessarily at the point when the for statement starts
executing. That is, the initialization expression in the for
statement can help establish the loop invariant.
Loop invariants capture key facts that explain why code works. If you write code in which the loop invariant is not obvious, you should add a comment that gives the loop invariant. This helps other programmers understand the code, and helps keep them (or you!) from accidentally breaking the invariant with future changes.
If you have figured out (even part of) the loop invariant, it also makes sense
to add an assert statement that checks the loop invariant on every iteration. Such
assertions will catch errors early and expose problems with your understanding of why
the code works.