CSS 343: Notes from Lecture 6 (DRAFT)

Administrivia

• midterm: rescheduled to Oct. 29
  • no timelimit (within reason)
  • you may bring single 8.5x11" cheat sheet (double-sided)
  • no calculators
• assignment 1: revised due date Oct. 22
• usual office hours Sunday

Priority Queues

• dictionary: random insert, random lookup/delete
• priority queue: random insert, lookup/delete smallest
  • smallest could be larger if comparing by > instead of <
  • may use any comparison relation that gives "total order"
• heap is a concrete implementation of the priority queue abstract data type
  • terms may be used interchangably when we're being sloppy
  • different kinds of heap: binary, binomial, Fibonacci
    • we'll only talking about binary heaps in this class
• "heap" is an overloaded term
  • unrelated use: memory pool used for dynamic allocation
• some texts talk about min heap vs. max heap
  • differ in the comparison relationship so one selects smallest, the other sellects largest
  • otherwise identical

(More) Properties of Binary Trees

• full binary tree: every non-leaf (interior) node has exactly two children
  • example: red-black tree is full (because we add the synthetic black leaf nodes)
• complete binary tree: all levels except last are completely filled; last level is filled left-to-right
  • ambiguity in the literature: may also be called almost or nearly complete if last level is not full
• perfect binary tree: complete binary tree with last level full

Complete/Perfect Binary Trees

• nodes at level 0: 1 (root)
• nodes at level 1: 2 (both children of root)
• nodes at level d: twice the nodes at level (d-1) (each node at level (d-1) has two children)
• total number of nodes from root to level d: 2(d+1) - 1
  • 1 + 2 + 4 + 8 + 16 + ... is the same as 1 + 10 + 100 + 1000 + 10000 + ... = 11111... in binary
• if we enumerate the nodes in level order (top-to-bottom, left-to-right), starting with root = 1
  • left_child(n) = 2 * n
  • right_child(n) = left_child(n) + 1 = 2 * n + 1
  • parent(n) = floor(n / 2)

We can represent a complete binary tree in an array.

• no pointers! (pointers are implicit)
• abstract view: we can still draw tree nodes and connecting lines on the white board
• jumping between two levels of abstraction (again)
• we're numbering the nodes from 1, so we either need to do a small transformation (knowing exactly when to add one and when to subtract one—which is error-prone), or we can simply "waste" a single cell in the array

Binary Heaps

A binary heap (or just heap) is a complete binary tree such that the for each node, its priority is higher that the priority of its left and right child (in my universe, higher numbers give lower priority, unless I decide otherwise).

pri(n) ≤ pri(left(n))
pri(n) ≤ pri(right(n))
  

Implications:

• the root node is the overall minimum element
  • contrast to BST: minimum element is left-most branch
• there is absolutely no relationship between the elements of the left and right subtrees
  • contrast to BST: every element of the left subtree is smaller that every element of the right subtree
• any subtree of a heap is also a heap
• contrast to BST: any subtree of a BST is also a BST
• the rules put the tree into a partial ordering (http://en.wikipedia.org/wiki/Partially_ordered_set#Formal_definition) even though we use a total order (http://en.wikipedia.org/wiki/Total_order) predicate (as in any comparison we can use for sorting)

Do not confuse the binary heap and the binary search tree.

Constructing a Heap

The heap is constructued recursively. Each new element is inserted at the to the next available leaf position (which is the end of the array, if implemented using an array). The new element is then sifted or bubbled up by swapping with its parent as long as it is less than its parent

Insertion requires O(log N) time, so overall construction of the tree is (N log N) time.

Min Element

The minimum element is the root, so it can be found in O(1) time.

Removing the minimum element destroys the tree structure, so it must be restored. We restore the heap invariant by taking the rightmost leaf on the bottom row (last element in array, if using array implmentation) and moving it to the root position. The heap property is then restored by swaping the node with the smaller of its left and right children, repeating the sift/bubble down procedure until it reaches a leaf position or is less than its children.

Sift down takes O(log N) time.

Reduce Key

If we know the tree position of a node, we can raise its priority (or lower its key value) and apply bubble up from its current position.

Animation

Heapsort Algorithm

• selection sort: scan list, removing smallest element
  • scan requires O(N) time
  • overall performance: O(N2)
• with heap, selecting minimum element requires O(log N) time, giving overall O(N log N) performance

Removing the minimum element frees up the last element of the array. The minimum elment can be placed in the position vacated by the last element.

• this will result in the elements sorted in reverse
  • fix this by a final O(N) pass to reverse the elements, or use > for comparison instead of <

Optimality of the Heap

The sorting problem is known to have an O(N log N) lower bound. Suppose there existed an algorithm that performed better than O(log N) for remove-min. Such an algorithm would yield a faster-than-O(N log N) sorting algorithm. By contradiction, a faster priority queue cannot exist.