Nirmalya Roy
School of Electrical Engineering and Computer Science
Washington State University
Cpt S 223 – Advanced Data Structures Priority Queues
(Heaps)
Motivation
Queues are a standard mechanism for orderingtasks on a first-come, first-served basis
However, some tasks may be more important ortimely than others (higher priority)
Priority queues
Store tasks using a partial ordering based on priority
Ensure highest priority task at head of queue
Heaps are the underlying data structure of priorityqueues
Priority Queues
Main operations
insert (i.e., enqueue)
Dynamic insert
Specification of a priority level (0 (high), 1, 2, …, low)
deleteMin (i.e., dequeue)
Finds the current minimum element (element with highestpriority) in the queue, deletes it from the queue, and returns it
Performance goal is for operations to be “fast”
Priority Queue Implementations
Unordered list
O(1) for insert
O(N) for deleteMin
Ordered list
O(N) for insert
O(1) for deleteMin
Balanced BST
O(log2 N) for insert and deleteMin
Can it be done faster?
Binary Heap
A priority queue data structure
Binary Heap
A binary heap is a binary tree with two properties
Structure property
Heap-order property
Structure Property of Binary Heap
A binary heap is a complete binary tree
Each level is completely filled
Bottom level may be partially filled from left to right
Height of a complete binary tree with N elements islog2 N
Binary Heap Example
N = 10
Array representation of the binary heap above:
Every level
(except last)
completely filled
Not a Binary Heap
A
B
C
D
E
F
H
G
Why is the above cannot be a binary heap?
Heap-Order Property of Binary Heap
Heap-order property (for a “min binary heap”)
For every node X, key(parent(X)) key(X)
Except root node, which has no parent
Thus, minimum key always at the root
Alternatively, for a “max binary heap”, always keep themaximum key at the root
insert and deleteMin must always maintainheap-order property
Heap-Order Property Example
13
14
16
19
21
68
26
65
19
31
32
An example of a min binary heap
Duplicates are allowed
No order implied for elements at the same tree level or among siblings
Not a Min Binary Heap
17
14
16
19
21
68
26
65
11
31
32
The following is NOT a min binary heap
Why is the above NOT a min binary heap?
Heap-Order Property Example
30
28
27
20
11
2
15
13
1
2
9
An example of a max binary heap
Duplicates are allowed
No order implied for elements at the same tree level or among siblings
Not a Max Binary Heap
The following is NOT a max binary heap
Why is the above NOT a max binary heap?
30
32
27
20
11
2
15
13
1
2
12
Implementing Complete Binary Trees as Arrays
Given an element at position i in the array
i’s left child is at position 2i
i’s right child is at position 2i + 1
i’s parent is at position i / 2
Heap data structures
Draw the heap as trees with the implication that an actualimplementation will use simple arrays
An estimate of the maximum heap size is required in advance
Fix heap after deleteMin
An array (of Comparableobjects) and an integerfor heap size
Heap Insert
Insert new element into the heap at the nextavailable slot (“hole”)
Needed to maintain a complete binary tree
Then, “percolate” (or “trickle”) the element up theheap while min heap-order property is not satisfied
This strategy is know as percolate up
Heap Insert: Example
Insert 14:
14 vs. 31
14 vs. 13
14 vs. 21
Percolating up
Heap-order okay; structure okay
Heap Insert: Implementation
O(log N) time
Insert 14
Heap DeleteMin
Minimum element is always at the root
A hole is created at the root when minimum element isremoved
Heap decreases by one in number of elements (or size)
Last element X should be moved somewhere in theheap
If it can be placed in the hole, then we are done
Push smaller of the hole’s children into the hole, thuspushing the hole down one level
Repeat this step until X can be placed in the hole
Percolate down until the heap-order property is not restored
Heap DeleteMin: Example
Remove value at the root
Move 31 (last element) to the root
Is 31 > min(14, 16)?
If yes, swap 31 with min(14, 16)
If no, leave 31 in hole
Heap DeleteMin: Example (cont’d)
Is 31 > min(65, 26)?
If yes, swap 31 with min(65, 26)
If no, leave 31 in hole
Is 31 > min(19, 21)?
If yes, swap 31 with min(19, 21)
If no, leave 31 in hole
Percolating down…
Heap DeleteMin: Example (cont’d)
Percolating down…
Heap-order property okay;
Structure okay;
Done.
Heap DeleteMin: Implementation
O(log N) time
Heap DeleteMin: Implementation (cont’d)
Percolating down…
Left child
Right child
Pick child to swap with
Other Heap Operations
decreaseKey(p, v)
Lowers value of item p to v
Need to percolate up
E.g., change job priority
increaseKey(p, v)
Increases value of item p to v
Need to percolate down
remove(p)
First, decreaseKey(p, -)
Then, deleteMin
E.g., terminate job
Assume: We already know the
location of node containing p
Run-times for all 3 functions?
Building a Heap
What if all N elements are all available upfront tobuild the heap?
To build a heap with N elements
Default method takes O(N log N) time.
New method will take O(N) time – i.e., optimal
Building a Heap (cont’d)
Construct a heap from an initial set of N items
Solution 1
Perform N successive inserts
O(1) constant time on average and O(log N) worst-case
O(N) average case, but O(N log2 N) worst-case
Solution 2
Randomly populate initial heap with structure property
Perform a percolate-down from each internal node (i.e., H[size/2] toH[1])
To take care of heap-order property
Insert 33 or
Insert 14
Building a Heap: Example
Input: 150, 80, 40, 30, 10, 70, 110, 100, 20, 90, 60, 50, 120, 140, 130
•Randomly initialize heap
•Structure property satisfied
•Heap-order property NOT satisfied
•Leaves are all valid heaps (implicit)
So, let us look at each internal
node, and fix if necessary
Building a Heap: Example (cont’d)
Building a Heap: Example (cont’d)
Dotted lines show the path of percolating down
Building a Heap: Example (cont’d)
Dotted lines show the path of percolating down
Building a Heap: Example (cont’d)
Dotted lines show the path of percolating down
Building a Heap: Analysis
Running time = ?
O(sum of the heights of all internal nodes)
Theorem 6.1
For the perfect binary tree of height h containing 2h+1 – 1 nodes,the sum of heights of the nodes is 2h+1 – 1 – (h + 1)
Proof. Tree consists of 1 node at height h, 2 nodes at height (h-1), 22 nodes at height (h-2) and in general 2i nodes at height (h-i)
The sum of the heights of all the nodes
Since h = log N, then the sum of heights is O(N)
Will be slightly better in practice
Implication: Each insertion costs O(1) amortized time
Binary Heap Operations Worst-Case Analysis
Height of heap is log2 N
insert: O(log2 N)
2.607 comparisons on average, i.e., O(1)
deleteMin: O(log2 N)
decreaseKey: O(log2 N)
increaseKey: O(log2 N)
remove: O(log2 N)
buildHeap: O(N)
Building a Heap: Implementation
Start with the lowest, rightmost internal node
Applications of Heaps
Operating system scheduling
Process jobs by priority
Graph algorithms
Find shortest path
Event simulation
Instead of checking for events at each time click, look upnext event to happen
An Application: The Selection Problem
Given a list of n elements, find the kth smallestelement
Solution 1: Read the elements into an array andsort them, returning the appropriate element
Running time is O(n2)
Solution 2: Using simple sorting algorithm
Sort the list (O(n log n))
Pick the kth element
A better approach
Use a MinHeap
Selection Using a MinHeap
Input: n elements
Algorithm:
buildHeap(n elements) ==> O(n)
Perform k deleteMin() operations ==> O(k lg n)
Report the kth deleteMin() output
Total run-time = O(n + k lg n)
If k = O(n / lg n) then the run-time becomes O(n)
Other Types of Heaps
d-Heaps
Generalization of binary heaps (i.e., 2-heaps)
All nodes have d children
Insert in O(logd N) time; deleteMin in O(d logd N) time
Leftist heaps
Like a binary heap, it has both a structural and heap orderproperty
Binary tree, but not perfectly balanced
Supports merging of two heaps in O(m + n) time (i.e., sub-linear)
Skew heaps
Binary tress with heap order but without any structuralconstraints
O(log n) amortized run-time
Mergeable Heaps
Heap merge operation
Useful for many applications
Merge two (or more) heaps into one
Identify new minimum element
Maintain heap-order property
Merge in O(log N) time
Still support insert and deleteMin in O(log N) time
Insert = merge existing heap with one-element heap
d-Heaps require O(N) time to merge
Leftist Heaps
Null path length npl(X) of node X
Length of the shortest path from X to a node withouttwo children
Leftist heap property
For every node X in heap, npl(leftChild(X)) ≥npl(rightChild(X))
Leftist heaps have deep left subtrees and shallowright subtrees
Thus if operations reside in right subtree, they will befaster
Leftist Heaps
Leftist heap
Not a leftist heap
npl(X) shown in nodes
Leftist Heaps
Merge heaps H1 and H2
Assume root(H1) > root(H2)
Recursively merge H1 with right subheap of H2
If result is not leftist, then swap the left and rightsubheaps
Running time O(log N)
DeleteMin
Delete root and merge children
Leftist Heaps: Example
Right subheap
Root 6 of H2 > Root 3 of H1
Merge H2 (larger root) with rightsub-heap of H1 (smaller root)
Leftist Heaps: Example
Merge H2 (larger root) with rightsub-heap of H1 (smaller root)
Leftist Heaps: Example
Attach previous heap as H1’s right child.
Leftist heap?
Leftist Heaps: Example
Swap root’s children to make leftist heap.
Skew Heaps
Self-adjusting version of leftist heap
The relationship of Skew heaps are analogous toleftist heaps as splay trees are to AVL trees
Skew merge same as leftist merge, except wealways swap left and right subheaps
No need to maintain or test NPL of nodes
Worst case is O(N)
Amortized cost of M operations is O(M log N)
Binomial Heaps
Binomial Heap
A forest of heap-ordered binomial trees
Structure property
Heap-order property
The structural property of each heap-ordered treein binomial heap is different from binary heap
The heap-order property applies to each binomialtree
is the form of the coefficients in binomial theorem
Definition: A “Binomial Tree” Bk
A binomial tree of height k is called Bk
It has 2k nodes
The number of nodes at depth d is the binomial co-efficient
A binomial tree need not to be a binary tree
B3
Depth
d = 0
d = 1
d = 2
d = 3
# of nodes
Binomial Heap with 31 Nodes
We know:
Binomial heap should be a forest of binomial trees
Each binomial tree has power of 2 elements
How many binomial trees do we need?
n = 31 = 1 1 1 1 12
B0
B1
B2
B3
B4
Binomial Heap with 31 Nodes (cont’d)
B0
B1
B2
B3
B4
B2
B2
B3
B3
Bk == Bk-1 + Bk-1
A binomial tree Bk consists of a root with children B0, B1, B2 ….Bk-1
How many trees are there in this forest if the # of nodes are N?
Binomial Heap Property
Lemma: There exists a binomial heap for everypositive value of n
Proof:
Any arbitrary n can be represented in powers of two (binaryrepresentation)
Have one binomial tree for each power of two with co-efficient of 1
e.g., if n = 10
Binary Representation ==> (1010)2 ==> forest contains {B3, B1}
Binomial Heaps: Heap-Order Property
Each binomial tree should contain the minimumelement at the root of every subtree
Just like binary heap, except that the tree here is abinomial tree structure (and not a complete binary tree)
The order across binomial trees is irrelevant
Definition of Binomial Heaps
A binomial heap of n nodes is:
A forest of binomial trees as dictated by the binaryrepresentation of n (this is the structure property)
Each binomial tree is a min-heap or a max-heap (this isthe heap-order property)
Binomial Heaps: Example
Key Properties
Could there be multiple trees of the same height in abinomial heap? No
How many trees could there be, in the worst-case, ina binomial heap of n nodes? log n
Given n, can we tell (for sure) how many trees willthere be of a given height k? Check if bit is 1 in thekth LSB that means Bk exists if and only if the kth leastsignificant bit is 1 in the binary representation of n
Binomial Heap: Operations
insert(x)
merge(H1, H2)
x = deleteMin()
insert(x) in Binomial Heap
Goal
To insert a new element x into a binomial heap H
Observation
Element x can be viewed as a single element binomialheap
insert(x) == merge(H, {x})
So, if we decide how to do merge we will automatically figure out how toimplement both insert() and deleteMin().
merge(H1, H2)
Let n1 be the number of nodes in H1
Let n2 be the number of nodes in H2
Therefore, the new heap is going to have n1 + n2nodes
Logic:
Merge trees of same height, starting from lowest heighttrees
If only one tree of a given height, then just copy that
May need carryover (just like adding two binary numbers)
merge(H1, H2): Example
B0
B1
B2
+
13
14
26
16
26
?
carryover
B2
merge(H1, H2): Example Result
Merge takes O(log n) time (i.e. comparisons).
Merging Two Binomial Trees of theSame Height
=
B2
B2
B3
Simply compare the roots.
Merge is defined only for binomial trees with the same height
+
Merging Two Binomial Trees of the SameHeight (cont’d)
Merging More Than Two Trees
Merging three or more binomial trees of the sameheight could generate carry-overs
12
21
24
65
23
51
24
65
7
10
8
11
+
+
= ?
Merge any two and leave the third as carry-over.
findMin()
Goal: Given a binomial heap, H, find the minimumand delete it
Observation: The root of each binomial tree in Hcontains its minimum element
Approach: Therefore, return the minimum of allthe roots (minimums)
Complexity: O(log n) comparisons
findMin() and deleteMin() Example
B0
B2
B3
B0’
B1’
B2’
For deleteMin(): After delete, how to adjust the heap?
New heap: Merge {B0, B2} and {B0’, B1’, B2’}.
Find the binomial tree with smallest root say Bk and original priority queue let be H
Remove Bk from the forest of trees in H, forming the new binomial queue H’
Remove the root of Bk creating binomial trees B0, B1, B2 ….Bk-1 which collectively
forms H’’
Merge H’ and H’’
Binomial Queue Run-time Summary
insert
O(log n) worst-case
O(1) amortized time if insertion is done in a sequence
deleteMin, findMin
O(log n) worst-case
merge
O(log n) worst-case
An Implementation of a Binomial Heap
B0
B1
B2
B3
B4
B5
B6
B7
Array of binomial trees, children of each node are kept in a linked list
Trees use first-child, right-sibling representation
Analogous to a bit-based representation of a binary number n
Maintain a linked list oftree pointers (for theforest)
What is the value of n?
Priority Queues in STL
Binary Heap is implemented bythe class template namedpriority_queue found instandard header file queue
Default is max-heap
Methods
Push, pop, top, empty,clear
Duplicates allowed, only onelargest is removed if several
#include <queue>
#include <iostream>
using namespace std;
int main(void) {
priority_queue<int> Q;
for (int i = 0; i < 100; i++)
Q.push(i);
while (!Q.empty()) {
cout << Q.top() << endl;
Q.pop();
}
}
Calls deleteMax()
For min-heap, declare priority queue as:
priority_queue<int, vector<int>, greater<int> > Q;
Run-time Per Operation
Types of Heap
Insert
DeleteMin
Merge (=H1+H2)
Binary Heap
O(log n) worst-case
O(1) amortized forbuildHeap (constanttime on average)
O(log n)
O(n)
Leftist Heap
O(log n)
O(log n)
O(log n)
Skew Heap
O(log n)
O(log n)
O(log n)
BinomialHeap
O(log n) worst case
O(1) amortized forsequence of n inserts(constant time onaverage)
O(log n)
O(log n)
Summary
Priority queues maintain the minimum or maximumelement of a set
Standard binary heap implementation is elegant for itssimplicity and speed
It requires no links and only a constant amount of space
Leftist heap is a wonderful example of power of recursion
Binomial queue is a simple idea to achieve good time bound
Support O(log N) operations in worst-case
insert, deleteMin, merge
Many applications of priority queues ranging fromoperating systems scheduling to simulation