Priority Queue & Heaps

PriorityQueue ADT (PQ)

Level - Order Traversal of ordered binary trees.

Diagrams

Order of traversal Diagram 1:

Order of traversal diagram 2:

* in some tests, it is bottom-up, not top-down.

Complete Binary Tree

A complete binary tree of height h is:

  1. A binary tree of height h;
  2. with 2d2^{d} nodes at depth d, for every 0≤d<h0 \leq d < h
  3. level order traversal visits every internal node before any leaf
  4. every internal node is proper*, except perhaps the last**, which may have just a left child.

Diagrams

Example 1: X (4)

Example 2: checkmark

Example 3: X (3)

Example 4: X (4)

Example 5: checkmark

Example 6: checkmark

Example 7: checkmark

Example 8: X (5)

Example 9: X (4)

Example 10: checkmark

Example 11: checkmark

Unlabled tree on next slide:

Binary Heap Data Structure

Example (checkmark:

Example (X):

This is the basic DS for implementing PQs (binary min-heap).

Notice: there no choice about how the shape changes:

Example 1:

Example 2:

Example 3:

Heap Insert

To insert an item with key k:

  1. add a new leaf v with key(v)=k, so as to maintain the shape invariant.
  2. re-establish the order invariant by executing percolate_up(v).

Code:

percolate_up(v){
  while(v is not root and key(v) < key(parent(v))) {
    swap positions of v and parent(v) in the tree
  }
}

Insert 2, then 4, then 3 into:

Original:

Insert 2:

insert 4:

Insert 3:

Becomes:

Heap Extract-Min:

Consider (need result of dot dot dots):

We must replace the root with the smaller of its children:

Diagram labled “OK”:

Diagram labled “NOT OK”:

Heap Extract-Min

To remove the (item with the) smalled key form the heap:

  1. rename the root
  2. replace the root with the “last leaf”, so as to maintain the shape invariant.
  3. restore the order invariant by calling percolate_down(root)

Percolate_down is more work than percolate_up, because it must look at both children to see what to do (and the children may or may not exist)

Code:

percolate_down(v){
  while(v has a child c with key(c) < key(v)){
    c <- child of v with the smallest key among the children of v.
    swap v and c in the tree
  }
}

Notice that:

Do extract-min 3 times

Original:

First extract-min:

Second extract-min:

Third extract-min:

Final form:

Complexity of Heap Insert & Extract-min

Size bounds on complete binary trees

2h≤nlog⁡22h≤log⁡2nh≤log⁡2nh=O(log⁡n) \begin{aligned} 2^{h} & \leq n\\ \log_{2} 2^{h} & \leq \log_{2} n\\ h & \leq \log_{2} n\\ h & = O(\log n) \end{aligned}

Heap insert & extract min take time O(log n)

Linked Implementation of Heap

Node:

Array-Based Binary Heap Implementation

Uses this embedding of a complete binary tree of size n in a size-n array:

Tree version:

Becomes, array version:

ith ndoe in level-order traversal becomes ith array element.

* growing and shrinking the tree is easy in the array embedding.

Partially-filled Array Implementation of Binary Heap: Insert

Original:

equals:

  1. 2 left, right
  2. 7 left, right
  3. 6
  4. 8
  5. 10
  6. 4

Insert 1:

array implementation:

  1. 2 (left, right)
  2. 7 (left, right)
  3. 6 (left)
  4. 8
  5. 10
  6. 9
  7. (inserted) 1

Becomes:

Array implementation:

  1. 1
  2. 7
  3. 2
  4. 8
  5. 10
  6. 9
  7. 6

Additional diagram:

In array form:

  1. 1 (2 is crossed out) (left, right)
  2. 7 (left, right)
  3. 2 (6, 1 are crossed out)
  4. 8
  5. 10
  6. 9
  7. 6 (1 is crossed out)

Insert for Array-based Heap

insert(k){
  A[size] <- k; // Add k to the new 'last leaf'
  v <- size
  p <- floor((v-1)/2) // p <- parent(v); percolate_up
  while(v>0 and A[v]<A[p]){
    swap A[v] and A[p]
    v <- p
    p <- floor((v-1)/2)
  }// end of percolate_up
  size <- size + 1;
}

Partially-filled Array Implementation of Binary Heap: Extract-min

Original tree:

Array implemntation:

  1. 2 (left, right)
  2. 7 (left, right)
  3. 6
  4. 8
  5. 10
  6. 9
  7. 11

After extract-min, tree:

Array implementation:

  1. 6
  2. 7
  3. 9
  4. 8
  5. 10
  6. 11

After another extract-min, tree:

Extract_min for Array-based Heap

Code:

extract_min(){
  temp <- A[0] // record value to return
  size <- size-1
  A[0] <- A[size] // move *old* last leaf to root
  i <- 0 // percolate down
  while(2i+1<size){// while i not a leaf
    child <- 2i+1 // the left child of i
    if(2i+2<size and A[2i+2] < A[2i+1]){
      child <- 2i+2 // use the right child if it exists and a smaller key
    }
    if(A[child]<A[i]){ // if order violated,
      swap A[child] and A[i] // swap parent+child
    } else {
      return temp
    }
  } // percolate-down
  return temp.
}

A small space-for-time trade-off in Extract-min

Extract-min variant

Code:

extract_min(){
  temp <- A[0] // record value to return
  size <- size-1
  A[0] <- A[size] // move *old* last leaf to root
  A[size] <- inf // **
  i <- 0 // percolate down
  while(A[2i+1]+A[i] *or* A[2i+2]+A[i]){ // i has a child that is out of order
    if(A[2i+1]<A[2i+2]){ //if is a left child
      swap A[2i+1] and A[i]
      i <- 2i+1
    } else { //it is a right child
      swap A[2i+2] and A[i]
      i <- 2i+2
    }
  }
  return temp
}

Making a Heap from a Set

make_heap(T){
  //T is a complete b.t. with n keys.
  for(i=floor(n/2)-1 down to 0){
    call percolate_down on node i
  }
}

How does make-heap work?

Tree diagram:

Last internal node equation: ⌊n2⌋−1=⌊212⌋−1=9\lfloor \frac{n}{2} \rfloor -1 = \lfloor \frac{21}{2} \rfloor -1 = 9

Make heap example

Note: n=10n=10; ⌊n/2⌋−1=4\lfloor n/2 \rfloor -1 =4

Notice: The exact order of visitng nodes does not matter – as long as we visit children before parents. [It follows that it is easy to do a recursive make-heap]

Make heap Example

Note: n=10n=10; ⌊n/2⌋−1=4\lfloor n/2 \rfloor -1 = 4

Notice: The exact order of visitng nodes does not matter – as long as we visit children before parents. [It follows that it is easy to do a recursive make-heap]

Make-heap Complexity

Diagram of a perfect binary tree with h=5. It is missing the rightmost 4 at h=5. Easier than using a tree.

Time Complexity of Make-heap

S(n)≤∑d=0h−12d(h−d) S(n) \leq \sum_{d=0}^{h-1} 2^{d} (h-d)

Explanation:

Part Note
∑d=0h−1\sum_{d=0}^{h-1} percolate_down is called, at most on each node at each depth d from 0 to h-1
2d2^{d} there are 2d2^{d} nodes at depth d
(h−d)(h-d) The max # of swaps for a call to percolate-down on a node at depth d is h-d

S(n)≤∑d=0h−12d(h−d)=20(h−0)+21(h−1)+⋯+2h−2(h(h−2))+hh−1(h−(h−1)) \begin{aligned} S(n) & \leq \sum_{d=0}^{h-1} 2^{d} (h-d)\\ & = 2^{0} (h-0) + 2^1 (h-1) + \cdots + 2^{h-2} (h(h-2)) + h^{h-1} (h-(h-1)) \end{aligned}

Set i=h=di=h=d, d=h=id=h=i and while d ranges over 0,1,⋯ ,h−10,1,\cdots,h-1, i will range over h−0,h−1,⋯ ,h−(h−1)h-0,h-1,\cdots,h-(h-1)

Now:

S(n)≤∑i=1h2h−i(i)=∑i=1h2h2ii≤∑i=1hn2ii=n∑i=1hi2i≤n∑i=0hi2i≤2n \begin{aligned} S(n) \leq \sum_{i=1}^{h} 2^{h-i} (i) & = \sum_{i=1}^{h} \frac{2^{h}}{2^i} i \leq \sum_{i=1}^{h} \frac{n}{2^i} i\\ & = n \sum_{i=1}^{h} \frac{i}{2^i} \leq n \sum_{i=0}^{h} \frac{i}{2^i} \leq 2n \end{aligned}

(∑i=0hi2i=020+121+222+323+⋯=12+12+38+14+532+⋯) \Bigg ( \sum_{i=0}^{h} \frac{i}{2^i} = \frac{0}{2^0} + \frac{1}{2^1} + \frac{2}{2^2} + \frac{3}{2^3} + \cdots = \frac{1}{2} + \frac{1}{2} + \frac{3}{8} + \frac{1}{4} + \frac{5}{32} + \cdots \Bigg )

Everything after 38\frac{3}{8} is less than or equal to 1.

Complexity of Make-heap

Work done by make-heap is bounded by a constant times the number of swaps so is O(n).

Updating Priorities

Tree 1:

Tree 2:

End (transcriber’s note: not the end)

Correctness of swapping in percolate down

Now:

Correctness of swapping in percolate_up

Now: