TRANSITIVE CLOSURE AND ALL-PAIRS SHORTEST PATHS Suppose we want the shortest path from every vertex to every other. We could do Dijkstra's algorithm n times, giving a running time of O(n^3 + mn) or O(nmlog n) or O(nm + n^2log n) depending on the implementation of extract-min. However there are clever direct ways that use the adjacency matrix representation of directed graphs. Simpler problem: reflexive transitive closure of a directed graph. Input: a graph G. Output: a new graph G' that has an edge (i,j) iff there is a path of any length (including 0) from i to j in G. Thus G' short-circuits all the paths in G with a single edge. Note that since there is a path from any vertex to itself of length 0, the graph G' will also have self loops on every vertex. This problem can be solved efficiently using the adjacency matrix representation. Recall that the adjacency matrix of a graph with n vertices is an n x n Boolean matrix with 1 (true) in location i,j if there is an edge from i to j, and 0 (false) if not. With this representation, we can restate the problem: Input: the adjacency matrix M of a directed graph G Output: a Boolean matrix with a 1 in location i,j if there is a path from i to j, 0 if not. Let us build up the solution iteratively. This (and all the other algorithms below) are examples of dynamic programming which is discussed more generally in CS482. Let M^k be a Boolean matrix with M^k(i,j) = 1 if there is a path of length exactly k from i to j, 0 otherwise. Note that M = M^1. Let M* be the matrix M* = M^0 + M^1 + M^2 + M^3 + ... where + denotes Boolean or, done componentwise on matrices. Thus the i,j-th element of M* is 1 if the i,j-th element of M^0 is 1, or the i,j-th element of M^1 is 1, or the i,j-th element of M^2 is 1, or the i,j-th element of M^3 is 1, or ... iff there is a path from i to j of length 0, or there is a path from i to j of length 1, or there is a path from i to j of length 2, or there is a path from i to j of length 3, or ... iff there is a path from i to j of any length. Now we can restate our problem as: Input: M Output: M* Insight 1: There is a path from i to j iff there is a path of length n-1 or less from i to j. This is true because if there is a path of any length >= n, then there must be a repeated vertex, and we can cut out a loop to obtain a shorter path. If the resulting path is still of length >= n, we can cut out another loop, and so on, until we get down to length < n. What we just argued was: M* = M^0 + M^1 + M^2 + ... + M^(n-1). Insight 2: For any k, there is a path of length k+1 from i to j iff there is a path of length k from i to some vertex v and an edge from v to j. i - - - - - - - - - > v ---> j Let's use Insight 2 to compute M^(k+1)from M^k. Restating Insight 2 in terms of matrices, For any k, M^(k+1)(i,j) = 1 iff there is a v such that M^k(i,v) = 1 and M(v,j) = 1. In other words, M^(k+1)(i,j) = 1 iff (M^k(i,0) = 1 and M(0,j) = 1) or (M^k(i,1) = 1 and M(1,j) = 1) or (M^k(i,2) = 1 and M(2,j) = 1) or ... (M^k(i,n-1) = 1 and M(n-1,j) = 1) iff (M^k(i,0) * M(0,j)) + (M^k(i,1) * M(1,j)) + (M^k(i,2) * M(2,j)) + ... (M^k(i,n-1) * M(n-1,j)) = 1, where we are using + to represent Boolean "or" and * to represent Boolean "and". Therefore M^(k+1)(i,j) = (M^k(i,0) * M(0,j)) + (M^k(i,1) * M(1,j)) + (M^k(i,2) * M(2,j)) + ... + (M^k(i,n-1) * M(n-1,j)). = OR (M^k(i,v) * M(v,j)) v=0 to n-1 Note that this is just Boolean matrix multiplication! You calculate the matrix product of M^k and M, except you use Boolean operations instead of arithmetic ones. Thus we can write simply M^(k+1) = M^k * M, where we interpret * as Boolean matrix multiplication. Once we have observed this, we see that M^k is the k-th Boolean power of M, i.e., the Boolean matrix product of k copies of M. So we can interpret the superscript ^k as an exponent. Now we know how to calculate M*: calculate all the powers of M up to n-1, then take the Boolean sum of all these matrices The Boolean sum of matrices is computed by taking the componentwise "or" of their elements; that is, the i,j-th element of the sum is the Boolean "or" of the i,j-th elements of the matrices being summed. In our notation, (M + M')(i,j) = M(i,j) + M'(i,j) for all i,j. This is great news--we can use an off-the-shelf matrix multiplication algorithm (provided we can switch + to OR and * to and) to solve the problem. Everybody knows how to do matrix multiplication in O(n^3). So we're automatically done in O(n^4). Better yet, other people can write great matrix multiplication algorithms and we can steal them. So let's say we did O(n*(time to do n x n matrix multiplication)). By using a little algebra, we can do even better... Note that M^0 = I (the Boolean identity matrix) M^1 = M, so I + M has a 1 in location i,j if there is a path of length 0 or 1 from i to j. Insight 3: There is path of length k or less from i to j iff there is a path of length k/2 or less from i to some v and a path of length k/2 or less from v to j. That is, rather than adding a single edge at each step, we double the path length. So (I + M)^2 gives paths of length 0, 1, or 2, (I + M)^4 gives paths of length up to 4, etc. We can actually prove this algebraically: first verify that a few basic algebraic laws hold for Boolean matrix multiplication: (A + B) + C = A + (B + C) associativity of + A + B = B + A commutativity of + (AB)C = A(BC) associativity of * A(B + C) = AB + AC left distributivity (A + B)C = AC + BC right distributivity IA = AI = A multiplicative identity A + A = A idempotency Using these laws we can show that (I + M)^k = I + M + M^2 + ... + M^k: Basis: (I + M)^1 = I + M Induction step: (I + M)^(k+1) = (I + M)^k (I + M) = (I + M + M^2 + ... + M^k)(I + M) by induction hypothesis = II + MI + M^2I + ... + M^kI + IM + MM + M^2M + ... + M^kM by distributivity = I + M + M^2 + ... + M^k + M + M^2 + M^3 + ... + M^(k+1) = I + M + M^2 + ... + M^(k+1) by idempotency. Thus (I + M)^(n-1) = M*. We can thus calculate M* by forming I + M and squaring log n times. This takes time O(log n * (time to multiply 2 n x n Boolean matrices), which is no worse than O(n^3 log n). Extending transitive closure to all pairs shortest paths is straightforward. We put the weights in the matrix, use min instead of (Boolean ) +, and use (arithmetic) + instead of *! I.e, define M as follows: M(i,j) = 0 if i=j w(i,j) if (i,j) is an edge with weight w(i,j) infinity otherwise This is just the adjacency matrix representation of a weighted graph. The insight (completely analagous) is that the shortest path from i to j that has at most k edges is given by the matrix M^k. So this is just matrix multiplication with min instead + and + instead of times. Running time is O(log n * (time to do n by n matrix multiplication)). Floyd-Warshall There is another all pairs shortest paths algorithm which uses a different insight and runs in time O(n^3). It is called the Floyd-Warshall algorithm. It still uses matrices, but the insight is very different: Instead of shortest path of length at most k, we do shortest path with no interior vertices numbered higher than k. (The /interior vertices/ of a simple path from i to j are all vertices on the path except i and j.) Let D^k(i,j) be the length of the shortest (weighted) path from i to j with no interior vertices numbered higher than k. Then D^k(i,j) = * w(i,j) if k=0 * min(D^(k-1)(i,j), D^(k-1)(i,k) + D^(k-1)(k,j)) if k > 0 The second line says the shortest path with no interior vertex higher than k is whichever is the shorter of the following: * The shortest path without using vertex k * The shortest path from i to k together with the shortest path from k to j. Just compute D^k(i,j) in order or increasing k. This is O(1) for each i,j at each stage, or O(n^3) total. For simplicity, we have ignored the problem of keeping track of the paths -- we have just calculated the distances. For single-source, we had a predecessor array. We can do the same thing here, maintaining an n x n matrix of shortest known paths from i to j. The details of maintaining this while doing the algorithm are not difficult.