The Greedy Method

  1. Greedy Template

  2. First Application: Selection Sort

  3. Second Application: Optimal Merge Patterns

  4. Third Application: The Knapsack Problem

  5. Fourth Application: Minimum Spanning Trees

  6. Fifth Application: Single-Source Shortest Paths Problem



I. Greedy Template:

 


Greedy(input I) begin while (solution is not complete) do Select the best element x in the remaining input I; Put x next in the output; Remove x from the remaining input; endwhile end



Back to Top

II. First Application: Selection Sort



Back to Top

III. Second Application: Optimal Merge Patterns



Back to Top

IV. Third Application: The Knapsack Problem



Back to Top

V. Fourth Application: Minimum Spanning Tree (MST)

  • Definition of a spanning tree: A spanning tree of a graph G of n nodes is a tree that has all the n nodes of the graph such that every edge of the tree is an edge in the graph.

  • Definition: if the edges have weights, then the weight of a tree is the sum of the weights of the edges of the tree.

  • Statement of the MST problem:
    • Input : a weighted connected graph G=(V,E). The weights are represented by the 2D array (matrix) W[1:n,1:n], where W[i,j] is the weight of edge (i,j).

    • Problem: Find a minimum-weight spanning tree of G.

  • The greedy method for this problem works on the basis of this slection policy: choose the minimum-weight remaining edge. If that edge does not create a cycle in the evolving tree, add it to the tree.

  • The greedy MST algorithm:



     
    	
    Procedure ComputeMST(in:G, W[1:n,1:n];out:T)
    begin
       Put in T the n nodes and no edges;
         while T has less than n-1 edges do
             Choose a remaining  edge e of 
    	   minimum weight;
    	 Delete e from the graph;
    	 if (e does not create a cycle in T) then 
    	   Add e to T;
    	 endif
         endwhile
    end ComputeMST
     

  • the main implementation questions are:

    1. how to find and delete the min-weight edges

    2. How to tell if an edge creates a cycle in T

  • For finding and deleting the min-weight edge, use a minheap where its nodes are the labels+weights of the graph edges.

  • For cycle detection, note that

    • T is a forest at any given time,

    • adding an edge eliminates two trees from the forest and replaces them by a new tree containg the union of the nodes of the two old trees, and

    • and edge e=(x,y) creates a cycle if both x and y belong to the same tree in the forest.

  • Therefore, a Union-Find data structure is perfect to tell if an edge creates a cycle, and to keep track of what nodes belong to each tree in the forest.

  • A detailed implementation code of ComputeMST:
     
    
    
    Procedure ComputeMST(input:G,W[1:n,1:n];output:T) begin integer PARENT[1:n]; initialize PARENT to -1 in each entry; Build a minheap H[1:|E|] for all the |E| edges; Put in T the n nodes and no edges; while T has less than n-1 edges do e=delete-min(H); /* assume e=(x,y)*/ r1 := F(x); r2 := F(y); if (r1 != r2) then Add e to T; U(r1,r2); endif endwhile end ComputeMST

  • Time complexity of the MST algorithm:

    • O(|E|) to build the heap

    • up to |E| calls to U and F, taking
      O(|E|log n) time

    • up to |E| calls to delete-min, taking
      O(|E|log |E|) time.

    • therefore, the total time is O(|E|log |E|).

  • Theorem: the ComputeMST algorithms computes a mininum spanning tree.
  • Proof:

    1. Let T be the tree generated by the algorithm

    2. Let T' be a minimum spanning tree

    3. If T=T', done. So assume that T != T'

    4. Strategy: T' will be transformed to T without a change of weight

    5. Let e be a min-eight edge in T-T'

    6. All edges in T that are < W(e) are thus in T' as well.

    7. Adding e to T' creates a cycle e1 ,e2 ,...ek

    8. This cycle must have an edge ej that is not in T because T has no cycles. ej is in T'.

    9. Claim: W(ej) >= W(e).
      We prove the claim by contradiction.

      • Assume W(ej) < W(e)

      • ComputeMST would process ej before e

      • All the edges processed before e and are of weight < W(e), and which are entered into T, are also in T' (by item 6)

      • Therefore, when ej is processed, it would be found not to create a cycle because ej and all the edges that preceded it are all in T', and T' does not have a cycle.

      • Thus, ej would have to be added by the algorithm to T, contradicting the fact that ej is not in T.

    10. Replace ej by e in T', resulting in a new tree T".

    11. W(T")=W(T')+W(e)-W(ej) <= W(T').

    12. Since T' is a minimum spanning tree, W(T") can't be < W(T').

    13. Therefore, W(T")=W(T').

    14. That is, T" is an MST and T" differs less from T that T' did.

    15. This kind of edge replacement operation, which make T' resemble T more and more without a change of weight, can be repeated a finite number of times until T' becomes identical to T.

    16. Thus, T and T' have the same weight, making T a minimum spanning tree.
      Q.E.D.



Back to Top

VI. Fifth Application: Single-Source Shortest Path problem