CS 312 Lecture 25
Priority Queues, Heaps, Shortest Paths

(We started this lecture by finishing the analysis of splay trees)

Priority Queues

Priority queues are a kind of queue in which the elements are dequeued in priority order.

src/imp_prioq.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27		  
signature IMP_PRIOQ =
  sig
    (* A 'a prioq is a mutable priority queue.
     * Abstractly, it is a possibly empty sequence of
     * elements [a1,...,an] sorted in priority order.
     * The operations destructively update the data
     * structure. *)
    type 'a prioq

    (* Create a new, empty priority queue *)
    val create : ('a * 'a -> order) -> 'a prioq

    (* insert(q,a) inserts a into q in priority order. *)
    val insert : 'a prioq -> 'a -> unit

    (* extract_min(q) removes and returns the first
     * element in the queue. Checks whether the
     * queue is nonempty. *)
    val extract_min : 'a prioq -> 'a

    val empty : 'a prioq -> bool

    val set_location: 'a * location -> unit
    val get_location: 'a -> location
    val incr_priority: 'a prioq * 'a *

  end
 

There are many ways to implement this signature. For example, we could implement it as a linked list where the cells of the list are connected through refs so it can be updated imperatively:

src/list_prioq.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23		  
structure ListPrioq : IMP_PRIOQ =
  (* Represents the priority queue as a list ordered by key,
   * and min element at head. *)
  struct
    type 'a prioq = {compare: 'a * 'a -> order,
                     elements: 'a list ref}

    fun create (c:'a*'a->order) = {compare=c, elements=ref []}
    fun empty({compare, elements}: 'a prioq) = null(!elements)
    fun insert ({compare,elements}: 'a prioq) (x:'a): unit =
      let fun ins [] = [x]
            | ins (hd::tl) =
        (case compare(hd,x) of
           LESS => hd::(ins tl) | _ => x::(hd::tl))
      in
        elements := ins(!elements)
      end
    exception EmptyQueue
    fun extract_min ({compare,elements}:'a prioq):'a =
      case (!elements) of
        [] => raise EmptyQueue
      | hd::tl => (elements := tl; hd)
  end
 

What is the asymptotic performance of this implementation?

Another alternative implementation is to use red-black trees or another of the balanced search trees. For example, in red-black trees we can find the minimum element by simply walking down the left children all the way from the root. Extracting the minimum element requires deleting it from the tree; we haven't seen how to do this, but it's about twice as complicated as the insertion we've already seen. This implementation has better performance for many applications:

In fact, we can tell that this is the best we do in terms of asymptotic performance, because we can implement sorting using O(n) priority queue operations, and we know that sorting takes O(n lg n) time in general. The idea is simply to insert all the elements to be sorted into the priority queue, and then use extract_min to pull them out in the right order:

src/heapsort.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54		  
(* sort(lst) contains the elements of lst in sorted order
 * according to cmp *)
fun sort(lst: 'a list, cmp: 'a*'a -> order):'a list =
  let
    val pq = create(cmp)
    val t1 = Time.now()
    val _ = foldl (fn(a:'a, b:unit) =>
                  insert pq a) () lst
    val t2 = Time.now()
    fun loop(): 'a list =
      if empty(pq) then []
      else let val a = extract_min(pq) in
        a::loop()
      end
    val result = loop()
    val t3 = Time.now()
  in
    print ("Insertion: " ^ Time.toString(Time.-(t2,t1)) ^ " sec\n");
    print ("Extraction: " ^ Time.toString(Time.-(t3,t2)) ^ " sec\n");
    print ("Total: " ^ Time.toString(Time.-(t3,t1)) ^ " sec\n");
    result
  end

(* The list [1,...,n] *)
fun one_to_n(n: int) =
  let fun m_to_n(m,n) =
    if n = m then [m]
      else m::(m_to_n(m+1,n))
  in
    m_to_n(1,n)
  end

fun heap_to_n(n) =
  let val h = create(Int.compare) in
    foldl (fn(a,b) => insert h a) () (one_to_n(n));
    h
  end

fun timeit(f: unit->'a):unit = let
  val time1 = Time.now()
  val _ = f()
  val time2 = Time.now()


in
  print ("\nTotal time = " ^ (Time.toString(Time.-(time2,time1)))
         ^ " seconds\n")
end

fun sorti(lst: int list) = sort(lst, Int.compare)

val _ = SMLofNJ.Internals.GC.messages false
 

Heaps

Although they have good asymptotic performance, it turns out that red-black trees are overkill for implementing priority queues: they are more complicated and slower than necessary. There is a simple, fast way to implement priority queues.

A heap is a special kind of balanced binary tree. Sometimes it is called a binary heap to distinguish it from a memory heap. The tree satisfies two invariants:

Suppose the priorities are just numbers. Here is a possible heap:

              3
             / \
            /   \
           5     9
          / \   /
         12  6 10

Obviously we can find the minimum element in O(1) time. Extracting it while maintaining the heap invariant will take O(lg n) time. Inserting a new element and establishing the heap invariant will also take O(lg n) time. So asymptotic performance is the same as for red-black trees but constant factors are better for heaps.

The key observation is that we can represent a heaps as an array. 

The root of the tree is at location 0 in the array and the children of the node stored at position i are at locations 2i+1 and 2i+2. This means that the array corresponding to the tree contains all the elements of tree, read across row by row. The representation of the tree above is:

[3 5 9 12 6 10]

Given an element at index i, we can compute where the children are stored, and conversely we can go from a child at index j to its parent at index floor((j-1)/2).

The rep invariant for heaps in this representation is actually simpler than when in tree form:

Rep invariant for heap a (the partial ordering property):

a[i] ≤ a[2i+1] and a[i] ≤ a[2i+2]
for 1 ≤ i ≤ floor((n-1)/2)

Now let's see how to implement the priority queue operations: 

insert

  1. Put the element at first missing leaf. (Extend array by one element.)

  2. Switch it with its parent if its parent is larger: "bubble up"

  3. Repeat #2 as necessary.

Example: inserting 4 into previous tree.

              3
             / \
            /   \
           5     9        [3 5 9 12 6 10 4]
          / \   / \
         12  6 10  4

              3
             / \
            /   \
           5     4        [3 5 4 12 6 10 9]
          / \   / \
         12  6 10  9

This operation requires only O(lg n) time -- the tree is depth
ceil(lg n) , and we do a bounded amount of work on each level.

extract_min

extract_min works by returning the element at the root.

The trick is this:

Original heap to delete top element from (leaves two subheaps)

              3
             / \
            /   \
           5     4        [3 5 4 12 6 10 9]
          / \   / \
         12  6 10  9

copy last leaf to root

              9
             / \
            /   \
           5     4        [9 5 4 12 6 10]
          / \   /
         12  6 10

"push down"

              4
             / \
            /   \
           5     9        [9 5 4 12 6 10]
          / \   /
         12  6 10


Again an O(lg n) operation.

We can sort using this implementation of priority queues.
How expensive is the sorting function built from this?

  n insertions, at O(lg n) cost, for O(n lg n) total
  n deletions, at O(lg n) cost, for O(n lg n)  total.

  Thus, O(n lg n) total cost.

It's called heapsort and it's a standard, reliable sorting algorithm.

If you have to sort by doing comparisons only, this is as fast as possible (up to a constant factor). There are plenty of other O(n lg n) algorithms with better properties in some cases, for example:

One last comment -- you might be worried about the fixed size for the array of values. The solution is just to use a resizable array abstraction (like Java Vectors), which you should be able to figure out how to build.

src/heap.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127		  
structure Heap : IMP_PRIOQ =
  struct
    type 'a heap = {compare : 'a*'a->order,
                    next_avail: int ref,
                    values : 'a option Array.array
                    }
    type 'a prioq = 'a heap

(* We embed a binary tree in the array 'values', where the
 * left child of value i is at position 2*i+1 and the right
 * child of value i is at position 2*i+2.
 *
 * Invariants:
 *
 * (1) !next_avail is the next available position in the array
 * of values.
 * (2) values[i] is SOME(v) (i.e., not NONE) for 0<=iorder) *)

(* get_elt(p) is the pth element of a. Checks
 * that the value there is not NONE. *)
fun get_elt(values:'a option Array.array, p:int):'a =
  valOf(Array.sub(values,p))

val max_size = 500000
fun create(cmp: 'a*'a -> order):'a heap =
  {compare = cmp,
   next_avail = ref 0,
   values = Array.array(max_size,NONE)}
fun empty({compare,next_avail,values}:'a heap) = (!next_avail) = 0

exception FullHeap
exception InternalError
exception EmptyQueue

fun parent(n) = (n-1) div 2
fun left_child(n) = 2*n + 1
fun right_child(n) = 2*n + 2

(* Insert a new element "me" in the heap.  We do so by placing me
 * at a "leaf" (i.e., the first available slot) and then to
 * maintain the invariants, bubble me up until I'm <= all of my
 * parent(s).  If there's no room left in the heap, then we raise
 * the exception FullHeap.
 *)
fun insert({compare,next_avail,values}:'a heap) (me:'a): unit =
  if (!next_avail) >= Array.length(values) then
    raise FullHeap
  else
    let fun bubble_up(my_pos:int):unit =
      (* no parent if position is 0 -- we're done *)
      if my_pos = 0 then ()
      else
        let (* else get the parent *)
          val parent_pos = parent(my_pos);
          val parent = get_elt(values, parent_pos)
        in
          (* compare my parent to me *)
          case compare(parent, me) of
            GREATER =>
              (* swap if me <= parent and continue *)
              (Array.update(values,my_pos,SOME parent);
               Array.update(values,parent_pos,SOME me);
               bubble_up(parent_pos))
          | _ => () (* otherwise we're done *)
        end
        (* start off at the next available position *)
        val my_pos = !next_avail
    in
      next_avail := my_pos + 1;
      Array.update(values,my_pos,SOME me);
      (* and then bubble me up *)
      bubble_up(my_pos)
    end

exception EmptyQueue
(* Remove the least element in the heap and return it, raising
 * the exception EmptyQueue if the heap is empty.  To maintain
 * the invariants, we move a leaf to the root and then start
 * pushing it down, swapping with the lesser of its children.
 *)
fun extract_min({compare,next_avail,values}:'a heap):'a =
  if (!next_avail) = 0 then raise EmptyQueue
  else (* first element in values is always the least *)
    let val result = get_elt(values,0)
      (* get the last element so that we can put it at position 0 *)
      val last_index = (!next_avail) - 1
      val last_elt = get_elt(values, last_index)
      (* min_child(p) is (c,v) where c is the child of p at which
       * the minimum element is stored), and v is the value
       * at that position. Requires p has a child. *)
      fun min_child(my_pos): int*'a =
        let
          val left_pos = left_child(my_pos)
          val right_pos = right_child(my_pos)
          val left_val = get_elt(values, left_pos)
        in
          if right_pos >= last_index then (left_pos, left_val)
          else
            let val right_val = get_elt(values, right_pos) in
              case compare(left_val, right_val)
                of GREATER => (right_pos, right_val)
                 | _ => (left_pos, left_val)
            end
        end
      (* Push "me" down until I'm no longer greater than my
       * children. When swapping with a child, choose the
       * smaller of the two.
       * Requires: get_elt(values, my_pos) = my_val
       *)
      fun bubble_down(my_pos:int, my_val: 'a):unit =
        if left_child(my_pos) >= last_index then () (* done *)
        else let val (swap_pos, swap_val) = min_child(my_pos) in
          case compare(my_val, swap_val)
            of GREATER =>
              (Array.update(values,my_pos,SOME swap_val);
               Array.update(values,swap_pos,SOME my_val);
               bubble_down(swap_pos, my_val))
             | _ => () (* no swap needed *)
        end
    in
      Array.update(values,0,SOME last_elt);
      Array.update(values,last_index,NONE);
      next_avail := last_index;
      bubble_down(0, last_elt);
      result
    end
  end
 

Finding Shortest Paths

In recitation we talked a bit about graphs: how to represent them and how to traverse them. Today we will discuss one of the most important graph algorithms: Dijkstra's shortest path algorithm, a greedy algorithm that efficiently finds shortest paths in a graph. (About pronunciation: "Dijkstra" is Dutch and starts out like "dike").

Many more problems than you might at first think can be cast as shortest path problems, making this algorithm a powerful and general tool. For example, Dijkstra's algorithm is a good way to implement a service like MapQuest that finds the shortest way to drive between two points on the map. It can also be used to solve problems like network routing, where the goal is to find the shortest path for data packets to take through a switching network. It is also used in more general search algorithms for a variety of problems ranging from automated circuit layout to speech recognition.

Let's start by defining a data abstraction for weighted, directed graphs so we can express algorithms independently of the implementation of graphs themselves. In a weighted graph, each of its edges has a nonnegative weight that we can think of as the distance one must travel when going along that edge.

src/wgraph.sig
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23		  
(* A signature for directed graphs. The signature is
 * simplified by not explicitly representing edges as
 * type. *)
signature WGRAPH = sig
  type graph  (* A directed graph comprising a set of
               * vertices V and directed edges E with nonnegative
               * weights. *)
  type vertex (* A vertex, or node, of the graph *)
  type edge (* A edge of the graph *)

  (* eq(v1,v2) Whether v1 and v2 are the same vertex. *)
  val eq: vertex*vertex->bool
  (* All vertices in the graph, without any duplicates.
   * Run time: O(|V|). *)
  val vertices: graph->vertex list
  (* outgoing(v) is a list of the edges leaving the vertex v.
   * Run time: linear in the length of the result. *)
  val outgoing: vertex->edge list
  (* edgeinfo(e) is (src,dst,w) where src is the source of
   * the edge e, dst is its destination, and w is its weight *)
  val edgeinfo: edge->vertex*vertex*int
end
 

There are some constraints on the running time of certain operations in this specification. Importantly, we assume that given a vertex, we can traverse the outgoing edges in constant time per edge. Some graph implementations do not have these properties, but we can easily write an almost trivial implementation that does:

src/wgraph.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31		  
structure Graph : WGRAPH = struct
  (* Note: vertex must contain a ref to allow graphs
   * containing cycles to be built and to give vertices
   * a notion of unique identity (ref identity).
   * The type vertex must be a datatype to permit it to
   * be defined recursively. *)
  datatype vertex = V of (vertex*int) list ref
  type edge = vertex*vertex*int
  type graph = vertex list

  fun eq(V(v1), V(v2)) = (v1 = v2)
  fun vertices(g) = g
  fun edgeinfo(e) = e
  fun outgoing(V(lr)) = map (fn(dst,w) => (V(lr), dst, w)) (!lr)

  (* add_vertex(g) is (g',v) where g' contains the same
   * vertices and edges as g, plus a new vertex v with no outgoing
   * edges. *)
  fun add_vertex(g: graph):graph*vertex = let
    val v = V(ref [])
  in
    (v::g, v)
  end

  (* Effects: add_edge(src,dst,w) adds an edge from src to dst,
   * with weight w. *)
  fun add_edge(src: vertex, dst: vertex, weight:int) =
    case src of
      V(lr) =>
        lr := (dst,weight)::(!lr)
end
 

A path through the graph is a sequence (v1, ..., vn) such that the graph contains an edge e1 going from v1 to v2, an edge e2 going from v2 to v3, and so on. That is, all the edges must be traversed in the forward direction. The length of a path in a weighted graph is the sum of the weights along these edges e1,..., en-1. We call this property "length" even though for some graphs it may represent some other quantity: for example, money or time.

To implement MapQuest, we need to solve the following shortest-path problem:

Given two vertices v and v', what is the shortest path through the graph that goes from v to v' ? That is, the path for which summing up the weights along all the edges from v to v' results in the smallest sum possible.

It turns out that we can solve this problem efficiently by solving a more general problem, the single-source shortest-path problem:

Given a vertex v, what is the length of the shortest path from v to every vertex v' in the graph?

It is this problem that we will now investigate.

The single-source shortest path problem can also be formulated on an undirected graph; however, it is most easily solved by converting the undirected graph into a directed graph with twice as many edges, and then running the algorithm for directed graphs. There are other shortest-path problems of interest, such as the all-pairs shortest-path problem: find the lengths of shortest paths between all possible source–destination pairs. The Floyd-Warshall algorithm is a good way to solve that problem efficiently.  

Single-source shortest path on unweighted graphs

Let's consider a simpler problem: solving the single-source shortest path problem for an unweighted directed graph. In this case we are trying to find the smallest number of edges that must be traversed in order to get to every vertex in the graph. This is the same problem as solving the weighted version where all the weights happen to be 1.

Do we know an algorithm for determining this? Yes: breadth-first search. The running time of that algorithm is O(V+E) where V is the number of vertices and E is the number of edges, because it pushes each reachable vertex onto the queue and considers each outgoing edge from it once. There can't be any faster algorithm for solving this problem, because in general the algorithm must at least look at the entire graph, which has size O(V+E).

We saw in recitation that we could express both breadth-first and depth-first search with the same simple algorithm that varied just in the order in which vertices are removed from the queue. We just need an efficient implementation of sets to keep track of the vertices we have visited already. A hash table fits the bill perfectly with its O(1) amortized run time for all operations. Here is an imperative graph search algorithm that takes a source vertex v0 and performs graph search outward from it:

src/traversal1.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41		  
(* Simple graph traversal (BFS or DFS) *)
let val q: queue = new_queue()
    (* visited is the set of vertices that have been visited. *)
    val visited: vertexSet = create_vertexSet()

    (* Expand the visited set to contain everything v goes to,
     * and add newly seen vertices to the queue. *)
    fun expand(v: vertex) =
      let fun handle_edge(e: edge): unit =
        let val (v, v') = Graph.edgeInfo(e) in
          if not (member(visited,v'))
            then ( add(visited, v');
                   enqueue(q, v') )
            else ()
        end
      in
        app handle_edge (Graph.outgoing(v))
      end
in
  add(visited, v0);
  enqueue(q, v0);
  while (not (empty_queue(q))) do expand(dequeue(q))
end

/*----- Build your own control structures! -----*/

val a = ref 0

fun while' (condition: unit->bool, body: unit->unit) =
  if condition() then
    (body(); while'(condition,body))
  else ()

fun w1() =
  while !a < 10 do
    (print (Int.toString(!a)); a := !a + 1)

fun w2() =
  while' (fn()=> !a < 10,
          fn()=> (print (Int.toString(!a)); a := !a + 1))
 

This code implicitly divides the set of vertices into three sets:

  1. The completed vertices: visited vertices that have already been removed from the queue.
  2. The frontier: visited vertices on the queue
  3. The unvisited vertices: everything else

Except for the initial vertex v0, the vertices in set 2 are always neighbors of vertices in set 1. Thus, the queued vertices form a frontier in the graph, separating sets 1 and 3. The expand function moves a frontier vertex into the completed set and then expands the frontier to include any previously unseen neighbors of the new frontier vertex.

The kind of search we get from this algorithm is determined by the dequeue function, which selects a vertex from a queue. If q is a FIFO queue, we do a breadth-first search of the graph. If q is a LIFO queue, we do a depth-first search.

 If the graph is unweighted, we can use a FIFO queue and keep track of the number of edges taken to get to a particular node. We augment the visited set to keep track of the number of edges traversed from v0; it becomes a map dist from vertices to edge counts (ints), possibly implemented as a hash table (an even more imperative alternative is to store distances directly in the vertices). The only algorithmic modification needed is in expand, which adds to the frontier a newly found vertex at a distance one greater than that of its neighbor already in the frontier:

src/traversal2.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24		  
(* unweighted single-source shortest path *)
let val q = new_queue()
    (* a map from visited vertices to their distances *)
    val dist: vertexMap = create_vertexMap()

    (* expand the visited set to contain everything v goes to,
     * and add newly seen vertices to the queue. *)
    fun expand(v: vertex) =
      let val d: int = valOf(get(dist, v))
          fun handle_edge(e: edge) =
            let val (v, v') = Graph.edgeinfo(e) in
              case get(dist, v') of
                SOME(d') => ()  (* note: d' <= d+1 *)
              | NONE => ( add(dist, v', d+1);
                         enqueue(q, v') )
            end
      in
        app handle_edge (Graph.outgoing(v))
      end
in
  add(visited, v0, 0);
  enqueue(q, v0);
  while (not (empty_queue(q))) do expand(dequeue(q))
end
 

Single-Source Shortest Path on Weighted Graphs

Now we can generalize to the problem of computing the shortest path between two vertices in a weighted graph. We can solve this problem by making minor modifications to the BFS algorithm for shortest paths in unweighted graphs. As in that algorithm, we keep a visited map that maps vertices to their distances from the source vertex v0. We change expand so that instead of adding 1 to the distance, its adds the weight of the edge traversed. Furthermore, when we find an already visited vertex, we update its distance only if the new distance is less than the old distance. Here is a first cut at an algorithm:

src/traversal3.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21		  
(* weighted single-source shortest path (broken!) *)
let val q = new_queue()
    val dist: VertexMap.map = create_vertexMap()
    fun expand(v: vertex) =
      let
        val d: int = valOf(get(visited, v))
        fun handle_edge(e: edge) =
          let val (_, v', w) = Graph.edgeinfo(e) in
            case get(visited, v') of
              SOME(d') => ()
            | NONE => ( add(dist, v', d+w);
                        enqueue(q, v') )
          end
      in
        app handle_edge (Graph.outgoing(v))
      end
in
  add(visited, v0, 0);
  enqueue(q, v0);
  while (not (empty_queue(q))) do expand(dequeue(q))
end
 

This is nearly Dijkstra's algorithm, but it doesn't work. To see why, consider the following graph, where the source vertex is v0 = A.

The first pass of the algorithm will add vertices B and D to the map visited, with distances 1 and 5 respectively. D will then become part of the completed set with distance 5. Yet there is a path from A to D with the shorter length 3. We need two fixes to the algorithm just presented:

  1. In the SOME case a check is needed to see whether the path just discovered to the vertex v' is an improvement on the previously discovered path (which had length d)
  2. The queue q should not be a FIFO queue. Instead, it should be a priority queue where the priorities of the vertices in the queue are their distances recorded in visited. That is, dequeue(q) should be a priority queue extract_min operation that removes the vertex with the smallest distance.

    The priority queue must also support a new operation incr_priority(q,v) that increases the priority of an element v already in the queue q. This new operation is easily implemented for heaps using the same bubbling-up algorithm used when performing heap insertions.

With these two modifications, we have Dijkstra's single-source shortest path algorithm:

src/traversal5.sml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25		  
(* Dijkstra's Algorithm *)
let
  fun ordering((v1,d1), (v2,d2)) = Int.compare(d1,d2)
  val q: (vertex*int) Heap.heap = Heap.create(ordering)
  val dist: VertexMap.map = VertexMap.create()
  fun expand(v: vertex, d: int) =
    let fun handle_edge(e: edge) =
      let val (_, v', w) = Graph.edgeinfo(e) in
        case get(dist, v') of
          SOME(d') =>
            if d+w < d'
              then (add(dist, v', dist+w);
                    incr_priority(q, v', d+w) )
            else ()
        | NONE => (add(dist, v', d+w);
                   insert(q, v', d+w) )
      end
    in
      app handle_edge (Graph.outgoing(v))
    end
in
  add(visited, v0, 0);
  insert(q, (v0, 0));
  while (not (empty_queue(q))) do expand(extract_min(q))
end
 

There are two natural questions to ask at this point: Does it work? How fast is it?

Run time of Dijkstra's algorithm

Every time the main loop executes, one vertex is extracted from the queue. Assuming that there are V vertices in the graph, the queue may contain O(V) vertices. Each extract_min operation takes O(lg V) time assuming the heap implementation of priority queues. So the total time required to execute the main loop itself is O(V lg V). In addition, we must consider the time spent in the function expand, which applies the function handle_edge to each outgoing edge. Because expand is only called once per vertex, handle_edge is only called once per edge. It might call insert(v'), but there can be at most V such calls during the entire execution, so the total cost of that case arm is at most O(V lg V). The other case arm may be called O(E) times, however, and each call to increase_priority takes O(lg V) time with the heap implementation. Therefore the total run time is O(V lg V + E lg V), which is O(E lg V) because V is O(E) assuming a connected graph.

There is a more complicated priority-queue implementation called a Fibonacci heap that implements incr_priority in O(1) time, so that the asymptotic complexity of Dijkstra's algorithm becomes O(V lg V + E). However, this bound is largely of theoretical interest because large constant factors make Fibonacci heaps impractical for most uses.

Correctness of Dijkstra's algorithm

Each time that expand is called, a vertex is moved from the frontier set to the completed set. Dijkstra's algorithm is an example of a greedy algorithm, because it just chooses the closest frontier vertex at every step. A locally optimal, "greedy" step turns out to produce the global optimal solution. We can see that this algorithm finds the shortest-path distances in the graph example above, because it will successively move B and C into the completed set, before D, and thus D's recorded distance has been correctly set to 3 before it is selected by the priority queue.

The algorithm works because it maintains a loop invariant that holds every time the while loop is executed:

For every visited vertex v, the recorded distance (in visited) is the shortest-path distance to that vertex from v0, considering just the paths that traverse only completed vertices and the vertex v itself. We will call these paths internal paths.

This invariant obviously holds when the main loop starts, because the only visited vertex is v0 itself, at recorded distance 0.  If the invariant holds when the algorithm terminates, the algorithm works correctly, because all vertices are completed and all paths are internal paths. To show that the algorithm works, we need to show that the algorithm terminates and that each iteration of the main loop preserves the invariant.

Clearly the algorithm terminates, because a vertex can only be inserted once and every loop removes some vertex from the frontier. Showing the invariant is preserved is more interesting. 

Each step of the main loop takes the closest frontier vertex v and promotes it to the completed set. For the invariant to be maintained, it must be the case that the recorded distance for the closest frontier vertex is also the shortest internal-path distance to that vertex. However, adding v to the completed set creates no new internal paths to v, so the existing distance must still be the best internal-path distance.

We need to show that the invariant holds on all the other visited vertices too. To show this we begin by observing that the recorded distance to the vertex that is removed from the priority queue can never decrease from one loop iteration to the next; that is, executing the loop can only increase the priority of the minimum-priority element in the queue. Consider what happens during one execution of the loop. Extracting the minimum element clearly leaves behind elements that have distances at least as large. One iteration may add some new elements to the queue, but at distances no less than that of the vertex just removed. It may also reduce the distance estimate to some elements already in the queue, but the new distance estimate will also be no less than that of the vertex just removed.

For vertices that are not successors to v, adding v can only create new internal paths if those paths go through another completed vertex v′. But by the monotonicity argument in the previous paragraph, the distance to v' must be less than the distance to v, so any such new internal path will be at least as long as the already known path.

A vertex v′ that is a successor to v is either a completed vertex, a newly visited vertex, or on  the frontier:

  1.  Completed.  The vertex v′ must already be at a distance no greater than that of v, so its distance estimate is not updated.
  2. Newly visited. The vertex v′ is only reachable though v, so the shortest internal path to it will be the shortest path through v, followed by the edge leading from v. Therefore d+w is the shortest internal-path distance of v′.
  3. Frontier. Vertices v′ on the frontier are reachable directly via v with distance d+w, and this path might be better than the previously known paths. Adding v might also create other new internal paths that visits some completed vertex (or vertices) v′′  after v and before v′. However, the other completed vertices are closer than v, so for any such path, there is another shorter path to v′′ (and hence v′ ) that doesn't go through v. The existing distance estimate for v′ must already include this estimate.

We might also be concerned that incr_priority could be called on a vertex that is not in the priority queue at all. But this can't happen because incr_priority is only called if a shorter path has been found to a completed vertex v′. Because of the monotonic increase in distances, this shorter path cannot exist.

The invariant implies that we can use Dijkstra's algorithm a little more efficiently to solve the simple shortest-path problem in which we're interested only in a particular destination vertex. Once that vertex is dequeued from the priority queue, the traversal can be halted because its recorded distance is correct. Thus, to find the distance to a vertex v the traversal only needs to visit the graph vertices that are at least as close to the source as v is.

References

Cormen, Leiserson, and Rivest. Introduction to Algorithms.
Aho, Hopcroft, Ullman. Data Structures and Algorithms.