CS410, Summer 1998 Lecture 27 Outline Dan Grossman Goals: * Running time of Prim's algorithm and Dijkstra's algorithm * Transitive Closure and All pairs shortest paths Let G have n vertices and m edges. Recall Prim's algorithm for MST: Prim: while more edges to add u = extract-min add (pred[u], u) to A for each edge (u,v) if v is not in A and weight(u,v) < v.key then set pred[v] to u and decrease-key(v, weight(u,v)) So we have n extract-mins <= m decrease-keys Recall Dijkstra's algorithm for SSSP (positive edge weights): Dijkstra: while more shortest paths to find u = extract-min add u to A for each edge (u,v) relax (u,v) So we have n extract-mins m relaxations for both: if we use a binary heap with keys being shortest distance to A (for Prim) or shortest path from s (for Dijkstra), we have O(n) build-heap (trivial actually -- they're all infinity except r is 0). O(nlog n) delete-mins O(mlog n) decrease-keys -------- O(m log n) This is the same as Kruskal b/c O(m log n) == O(mlog m). I _think_ in practice Kruskal is faster b/c sorting can have low constant factors. Actually though, we can do better in asymptopia using fibonacci heaps (CS482). They do delete-min in O(log n) but decrease-key in O(1). The constant factors are huge -- "use them in a proof but never implement them". In this case, it makes Prim's and Dijkstra's O(m + nlogn) which is an improvement, but nobody does it in practice. If the graph is dense (m on the order of n^2) we can do better with something more simple: instead of a heap, just look through the dist array every time and find the minimum: O(n^2) delete-mins (n of them in O(n) each) O(m) decrease-keys (m of them in O(1) each) ------ O(n^2 + m) == O(n^2) So the question is mlog m vs. n^2. Depends on your graph. Your homework explores using something besides a heap in order to get O(Wn + m), assuming all edge weights are integers between 0 and W. TRANSITIVE CLOSURE AND ALL PAIRS SHORTEST PATHS Suppose we want the shortest path from every vertex to every other. We could do Dijkstra's algorithm n times, giving a running time of O(n^3 + mn) or O(nmlog n) or O(nm + n^2log n) depending on the implementation of extract-min (see above). There are clever direct ways that use the adjacency matrix representation of directed graphs... Simpler problem: transitive closure of a directed graph. Input: a matrix with a 1 iff there is a path with 0 or 1 edges from i to j. Output: a matrix with a 1 iff there is a path of any length from i to j. iff there is a path of length n or less from i to j. That is, given a graph G, we want a graph G' that has an _edge_ (i,j) iff there is a _path_ from i to j in G. Let us build up the solution iteratively. This (and all the other algorithms below) are examples of dynamic programming which is discussed more generally in CS482. Insight 1: There is a path from i to j iff there is a path of length n or less from i to j. So let M^k be the matrix of paths length k or less. We can restate our problem as: Input: M^1 Output: M^n Insight 2: There is a path of length k or less from i to j iff: * there is a path of length k-1 or less from i to j OR * there is a path of length k-1 or less from i to some v and an edge from v to j. Let's use Insight 2 to write a function that takes M^k to M^(k+1). Then we're done by taking M^1 and running the funciton n times. Iteratively take M^k --> M^(k+1): What is m^(k+1)(i,j)? It is (m^k(i,1) and m^1(1,j)) or (m^k(i,2) and m^1(2,j)) or ... (m^k(i,n) and m^1(n,j)) OR (m^k(i,j)) This is just insight 2 re-written in matrix notation. Furthermore, the second term (m^k(i,j)) is unnecessary because m^1(j,j) is 1 so it is already covered by the first term. Rewriting without ... notation, we have: m^(k+1)(i,j) = OR (m^k(i,v) and m^k(v,j)) v=1 to n Now here is an eerie analogy: Replace the OR with + and then and with * and we have the definition of the matrix multiplication of M^k and M^1 !!! This is great news -- we can use an off-the-shelf matrix multiplication algorithm (provided we can switch + to OR and * to and) to solve transitive closure. Everybody knows how to do matrix multiplication in O(n^3). So we're "automatically" done in O(n^4). Better yet, other people can write great matrix multiplication algorithms and we can steal them. So let's say we did O(n*(time to do n by n matrix multiplication)). We can do even better... Insight 3: There is path of length k or less from i to j iff: * There is a path of length k/2 or less from i to some v and a path of length k/2 or less from v to j That is, rather than adding one edge at each step, let's add another path. In matrix terms: M^2k = (M^k)^2 So instead of "multiplying" M^k by M^1, just multiply it by M^k again! We'll have M^(n') where n' > n in log n steps. This is called "repeated squaring" and for no more work reduces the running time to O(log n * (time to do n by n matrix multiplication)) or at worst O(n^3log n). Extending transitive closure to all pairs shortest paths is straightforward. Now define M^1 as follows: m^1(i,j) = 0 if i=j w(i,j) if (i,j) is an edge with weight w(i,j) infinity otherwise This is just the adjacency matrix representation of a weighted graph. The insight (completely analagous) is that the shortest path from i to j that has less than or equal to 2k edges is m^2k(i,j) = min (m^k(i,v) + m^k(v,j)) v=1to n So this is just matrix multiplication with min instead of + and + instead of times. Running time is O(log n * (time to do n by n matrix multiplication)). Floyd-Warshall There is another all pairs shortest paths algorithm which uses a different insight and runs in time O(n^3). It is called the Floyd-Warshall algorithm: It still uses matrices, but the insight is very different: Instead of shortest path of length at most k we do shortest path with no interior vertices numbered higher than k. (The interior vertices of a simple path from i to j is all vertices on the path except i and j.) shortest path with no interior vertices numbered higher than k = * w(i,j) if k=0 * min(d^(k-1)(i,j), d^(k-1)(i,k) + d^(k-1)(k,j)) if k > 0 (The second line says the shortest path with no interior vertex higher than k is whichever is shorter of the following: * The shortest path without using vertex k * The shortest path from i to k together with the shortest path from k to j.) Just solve the n^3 problems in order or increasing k. This is O(1) each since all the subproblems are already solved! So it's O(n^3) total. For simplicity, we have ignored the problem of keeping track of the paths -- we have just calculated the distances. For single-source, we had a predecessor array. We do the same thing here, but we need a predecessor matrix. We need something to the effect of predecessor matrix: (i,j) vertex before j on path from i to j The details of maintaining this while doing the algorithm are not difficult.