Gaussian Elimination and Heaps: Algorithm Design Techniques by B.B. Karki - Prof. B. Karki, Study notes of Computer Science

Download Gaussian Elimination and Heaps: Algorithm Design Techniques by B.B. Karki - Prof. B. Karki and more Study notes Computer Science in PDF only on Docsity!

Algorithm Design Techniques

 Brute force

 Divide-and-conquer

 Decrease-and-conquer

 Transform-and-conquer

 Space-and-time tradeoffs

 Dynamic programming

 Greedy techniques

Transform-and-Conquer

Gaussian Elimination

 Solving a system of n linear equations in n unknowns

a 11 x 1 + a 12 x 2 + ......+ a 1 n xn = b 1

a 21 x 1 + a 22 x 2 + ......+ a 2 n xn = b 2

a

n 1

x

1

+ a

n 2

x

2

+ ......+ a

nn

x

n

= b

n  Substitution method  Not suitable when n is large  Gaussian elimination  Transform a given system of n equations to an equivalent system with an upper triangular coefficient matrix € Ax = b In matrix notation

A =

a 11 a 12 ...... a 1 n

a 21 a 22 ... a 2 n

an 1 an 2 ... ann

, b =

b 1

b 2

bn

Ax = b ⇒ A ′ x = b ′

A^ ′ =

a^ ′

11

a ′

12 ......^

a ′

1 n

0 a ′ 22 ... a ′ 2 n

0 0 ... a ′

nn

, b ′ =

b^ ′

1

b^ ′

2

b ′

n

Upper Triangular Matrix Coefficient

 A series of elementary operations  Exchanging two equations of the system  Replacing an equation with its nonzero multiple  Replacing an equation with a sum or difference of this equation and some multiple of another equation  Elementary operation does not change a solution to a system  Normal pivoting  Use aii as a pivot to make all xi coefficients zeros in the equations below i th one.  Partial pivoting  Choose a row with the largest absolute value of the coefficient in the i th column

Heaps

 A heap is a binary tree satisfying two conditions  Shape requirement: The tree is complete so that all its levels are full except that only some rightmost leaves may be missing  The height of the tree is h =  log 2 n   Parental dominance: The key at each node is greater than or equal to the keys at its children  The root always contains the largest element  Keys are ordered top down  A node of a heap considered with all its descendants is also a heap.  Implementation as an array H [1…. n ]: H [ i ] ≥ max{ H [2 i ], H [2 i + 1]} for i = 1, …  n /2  Parental node keys: first  n /2 positions  Leaf keys: last  n /2 positions **5 7 4 2 10 1 Index 0 1 2 3 4 5 6

---

Values 10 5 7 4 2 1 parents leaves**

CSC 3102 0.8 B.B. Karki, LSU

Constructing a Heap: Bottom-Up Algorithm

 Two-steps heap construction  Initializes a complete binary tree with n nodes by placing keys in the order given  Heapifies the tree  Establish parental dominance starting with the last parental node  See the text for the pseudocode (page

 Time efficiency (the worst case)  Each key on level i of the tree travel to the leaf level during heapification 9 7 6 5 2 8 6 8 2 5 9 7 After heapification A heap for the list 2, 9, 7, 6, 5, 8 Initial complete binary tree € Cworst ( n ) = 2 ( h − i ) = keys ∑ i = 0 h − 1 ∑ 2 ( h^ −^ i ) i i = 0 h − 1 ∑

= 2 ( n − log 2 ( n + 1 )) ∈ O ( n )

Key Deletion from a Heap

 Deleting the root’s key: maximum key deletion from a heap  Exchange the root’s key with the last key K of the heap  Decrease the heap’s size by 1  Heapify the smaller tree by establishing the parental dominance for K  Time efficiency is in O (log n )  Deleting an arbitrary key v  Search for v by sequential search in H [1… n ]. Matching is found in position i so v = H [ i ].  Delete the element H [ i ] by using three-step process as before.  Time efficiency is in [ O ( n ) + O (log n )] 8 6 2 5 9 1 5 6 2 1 Deleting the root’s key 8

Heapsort

 A two-stage sorting algorithm  Stage 1 Heap construction: Construct a heap for a given array  Stage 2 Maximum deletions: Apply the root-deletion operation n -1 times to the remaining heap  The array elements are eliminated in decreasing order and are placed in a new array in ascending order  Sort the array 2, 9, 7, 6, 5, 8 by heapsort Stage 1 Stage 2 2 9 7 6 5 8 9 6 8 2 5 7 2 9 8 6 5 7 7 6 8 2 5 | 9 2 9 8 6 5 7 8 6 7 2 5 9 2 8 6 5 7 5 6 7 2 | 8 9 6 8 2 5 7 ……. 5 2 2 | 5 2

Problem Reduction

 Problem reduction strategy  Reducing geometric problems to algebraic ones  Solve the question about relative positions of three arbitrary points in the plane by finding the sign of a determinant  The sign determines whether the point P 3 ( x 3 , y 3 ) lies to the left or right of the directed line through points P 1 ( x 1 , y 1 ) and P 2 ( x 2 , y 2 )

x

1

y

1

x

2

y

2

x

3

y

3

= x

1

y

2

+ x

3

y

1

+ x

2

y

3

− x

3

y

2

− x

2

y

1

− x

1

y

3 Problem (^1) Problem 2 solution^ to Problem 2 Reduction Algorithm A (to be solved) (solvable by algorithm A) (original problem solved) P 3 P 2 P 1

Computing lcm

 Compute the least common multiple ( lcm ) of two integers, m and n , by solving the problem gcd (greatest common divisor)  Efficient algorithm (Euclid’s algorithm) exists for finding gcd  Apply repeatedly the equality gcd( m,n ) = gcd( n,m mod n ) until m mod n is 0. The last value of m is the gcd.

lcm ( m , n ) =

m • n

gcd( m , n )

Reduction of Optimization Problems

 Reduce a minimization problem to a maximization problem  To minimize a function, we can maximize its negative instead, and change the sign of the answer:  Opposite is also true:  Can be generalized to the case of function optimization. € min f ( x ) = −max[− f ( x )]

max f ( x ) = −min[− f ( x )]

-f( x) f( x) x* f( x) -f( x)

CSC 3102 0.17 B.B. Karki, LSU

Linear Programming

 A problem of optimizing a linear function of several variables subject to constraints in the form of linear equations and linear inequalities.  Solve the knapsack problem by reducing it to an instance of linear programming problem € v (^) j x (^) j j = 1 n ∑ w (^) j x (^) j ≤ W j = 1 n ∑

x j ∈ { 0 , 1 }

maximize subject to for j = 1, …. n