Insertion Sort, Selection Sort and Bubble Sort - Lecture Notes | EEL 4851, Study notes of Electrical and Electronics Engineering

Download Insertion Sort, Selection Sort and Bubble Sort - Lecture Notes | EEL 4851 and more Study notes Electrical and Electronics Engineering in PDF only on Docsity!

Sorting - 2

Insertion sort

Selection sort

Bubble sort

The efficiency of handling data can be substantially improved if the data is sorted according to some criteria of order. In a telephone directory we are able to locate a phone number, only because the names are alphabetically ordered. Same thing holds true for listing of directories created by us on the computer. Retrieval of a data item would be very time consuming if we don’t follow some order to store book indexes, payrolls, bank accounts, customer records, items inventory records, especially when the number of records is pretty large.

We want to keep information in a sensible order. It could be one of the following schemes: − alphabetical order − ascending/descending order − order according to name, ID, year, department etc.

The aim of sorting algorithms is to organize the available information in an ordered form.

There are dozens of sorting algorithms. The more popular ones are listed below:

− Selection Sort − Bubble Sort − Insertion Sort − Merge Sort − Quick Sort

As we have been doing throughout the course, we are interested in finding out as to which algorithms are best suited for a particular situation. The efficiency of a sorting algorithm can be worked out by counting the number of comparisons and the number of data movements involved in each of the algorithms. The order of magnitude can vary depending on the initial ordering of data.

How much time does a computer spend on data ordering if the data is already ordered? We often try to compute the data movements, and comparisons for the following three cases:

best case ( often, data is already in order), worst case( sometimes, the data is in reverse order), and average case( data in random order).

Some sorting methods perform the same operations regardless of the initial ordering of data. Why should we consider both comparisons and data movements?

Selection Sort Example

Sorted Unsorted

Original List

After pass 1

After pass 2

After pass 3

After pass 4

After pass 5

Selection Sort Algorithm

/ Sorts by selecting smallest element in unsorted portion of array and exchanging it with element at the beginning of the unsorted list. /

void selectionSort(int list[], int n) { int cur, j, smallest, tempData;

for (cur = 0; cur <= n; cur ++){ smallest = cur;

for ( j = cur +1; j <= n ; j++) if(list[ j ] < list[smallest]) smallest = j ;

// Smallest selected; swap with current element

tempData = list[cur]; list[cur] = list[smallest]; list[smallest] = tempData; } }

The outer loop is executed n times, and the inner loop iterates j = (n –1 – cur) times.

So it is a n times summation of j. Thus the order turns out to be O(n 2 ).

The number of comparisons remain the same in all cases.

There may be some saving in terms of data movements (swappings).

Insertion Sort Example

Sorted Unsorted

Original List

After pass 1

After pass 2

After pass 3

After pass 4

After pass 5

Insertion Sort Algorithm

/ With each pass, first element in unsorted sublist is inserted into sorted sublist.*

*/

void insertionSort(int list[], int n) { int cur, located, temp, j;

for (cur = 1; cur <= n ; cur++){ located = 0; temp = list[cur]; for ( j = cur - 1; j >= 0 && !located;) if(temp < list[ j ]){ list[j + 1]= list[ j ]; j--; } else located = 1; list[j + 1] = temp; } return; }

An advantage of this method is that it sorts the array only when it is really necessary.

If the array is already in order, no moves are performed. However, it overlooks the fact that the elements may already be in their proper positions

. When an item is to be inserted, all elements greater than this have to be moved. There may be large number of redundant moves, as an element ( properly located) may be moved , but later brought back to its position. The best case is when the data are already in order. Only one comparison is made for each position, so the comparison complexity is O(n), and the data movement is 2n – 1 , i.e. it is O(n).

The worst case is when the data are in reverse order. Each data element is to be moved to new position and for that each of the other elements have to be shifted. This works out to be complexity of O(n^2 ).

What happens when the elements are in random order? Does the over complexity is nearer to the best case , or nearer to the worst case? It turns out that both number of comparisons and movements turn out to be closer to the worst case.

Bubble Sort Example

Trace of 1st^ pass of Bubble Sort Example:

23 78 45 8 32 56

23 78 45 8 32 56

Sorted Unsorted

Original List

After pass 1

After pass 2

After pass 3

After pass 4

Sorted!

Bubble Sort Algorithm

/ Sorts list using bubble sort. Adjacent elements are compared and exchanged until list is completely ordered.*

*/ void bubbleSort(int list[], int n) { int cur, j, temp; for (cur = 0; cur <= n; cur++){ for ( j = n; j > cur; j--) if(list[j ] < list[ j - 1]){ temp = list[ j ]; list[ j ] = list[ j – 1]; list[ j -1] = temp; } } return; }

What is the time complexity of the Bubble sort algorithm?

The number of comparisons for the worst case ( when the array is in the reverse order sorted ) equals the number of iterations for the inner loop ,i.e. O(n 2 ). How many comparisons are to be made for the best case? Well, it turns out that the number of comparisons remain the same.

What about the number of swaps? In the worst case, the number of movements is 3 n(n – 1 )/2. In the best case, when all elements are ordered, no swaps are necessary. If it is a randomly ordered array, then number of movements are around half of the worst case figure.

Compare it with insertion sort. Here each element has to be bubbled one step at a time. It does not go directly to its proper place as in insertion sort. It could be said that insertion sort is twice as fast as the bubble sort , for the average case., because bubble sort makes approximately twice as many comparisons.

In summary, we can say that bubble sort, insertion sort and selection sort are not very efficient methods, as the complexity is O(n 2 ).

if A[actr] < B[bctr] C[cctr] = A[actr]; cctr++; actr++; } else { C[cctr] = B[bctr]; cctr++; bctr++; }

Let us take an example. Say at some point in the sorting process we have to merge two lists 1, 3, 4, 7 and 2, 5, 9, 11

We store the first list in Array A and the second list in Array B. The merging goes in following fashion:

Example: Linear Merge

A B C

actr bctr cctr

To sort an array A of size n, the call from the main program would be mergesort(A, n). Here is a typical run of the algorithm, for an array of size 8. Unsorted array A = [31 45 24 15 23 92 30 77 ]

The original array is kept on splitting in two halves till it reduces to array of one element. Then these are merged.

After merging [31 ] &[45 ] merged array is [31 45 ]

After merging [24 ] & [15 ] merged array is [15 24 ]

After merging [31 45 ] & [15 24 ] merged array is [15 24 31 45 ]

After merging [23 ] & [92 ] merged array is [23 92 ]

After merging [30 ] & [77 ] merged array is [30 77 ]

After merging [23 92 ] & [30 77 ] merged array is [23 30 77 92 ]

After merging [15 24 31 45 ] & [23 30 77 92 ]

merged array is [15 23 24 30 31 45 77 92 ]

Computational Complexity:

Intuitively we can see that as the mergesort routine reduces the problem to half its size every time, (done twice), it can be viewed as creating a tree of calls, where each level of recursion is a level in the tree. Effectively, all n elements are processed by the merge routine the same number of times as there are levels in the recursion tree. Since the number of elements is divided in half each time, the tree is a balanced binary tee. The height of such a tree tends to be log n.

The merge routine steps along the elements in both halves, comparing the elements. For n elements, this operation performs n assignments, using at most n – 1 comparisons, and hence it is O(n). So we may conclude that [ log n. O(n) ] time merges are performed by the algorithm.

The same conclusion can be drawn more formally using the method of recurrence relations. Let us assume that n is a power of 2, so that we always split into even halves. The time to mergesort n numbers is equal to the time to do two recursive mergesorts of size n/2, plus the time to merge , which is linear.

For n = 1 , the time to mergesort is constant.

We can express the number of operations involved using the following recurrence relations:

T(1) = 1 T(n) = 2 T(n/2) + n

Using same logic and going further down

T(n/2) = 2 T(n/4) + n/

Substituting for T(n/2) in the equation for T(n),we get

T(n) = 2[ 2 T(n/4) + n/2 ] + n

= 4 T(n/4) + 2n

Again by rewriting T(n/4) in terms of T(n/8), we have

T(n) = 4 [ 2 T(n/8) + n/4 ] + 2n

= 8 T(n/8) + 3 n

= 2^3 T( n/ 2^3 ) + 3 n

The next substitution would lead us to

T(n) = 2^4 T(n/ 2^4 ) + 4 n

Continuing in this manner, we can write for any k,

T(n) = 2k^ T(n/ 2k^ ) + k n

This should be valid for any value of k. Suppose we choose k = log n, i.e. 2k^ = n. Then we get a very neat solution:

T(n) = n T(1) + n log n

= n log n + n

Thus T(n) = O( n log n)

This analysis can be refined to handle cases when n is not a power of 2. The answer turns out to be almost identical.

Quicksort

As the name implies the quicksort is the fastest known sorting algorithm in practice. It has the best average time performance. Like merge sort, Quicksort is also based on the divide-and-conquer paradigm.

• But it uses this technique in a somewhat opposite manner, as all the hard work is done before the recursive calls.
• It works by partitioning an array into two parts, then sorting the parts independently, and finally combining the sorted subsequences by a simple concatenation.

In particular, the quick-sort algorithm consists of the following three steps:

1. Choosing a pivot: :. To partition the list, we first choose some element from the list which is expected to divide the list evenly in two sublists. This element is called a pivot.
2. Partitioning: Then we partition the elements so that all those with values less than the pivot are placed in one sublist and all those with greater values are placed in the other sublist. 3. Recur : Recursively sort the sublists separately. Repeat the partition process for both the sublists. Choose again two pivots for the two sublists and make 4 sublists now. Keep on partitioning till there are only one cell arrays that do not need to be sorted at all. By dividing the task of sorting a large array into two simpler tasks and then dividing those tasks into even simpler task, it turns out that in the process of getting prepared to sort , the data have already been sorted. This is the core part of the quicksort. The steps involved in the quicksort algorithm are explained through an example.

Example array:

[ 56 25 37 58 95 19 73 30 ] lh rh

1. Choose first element 56 as pivot.
2. Move rh index to left until it coincides with lh or points to value smaller than pivot. In this example it already points to value smaller than the pivot.
3. Move lh index to right until it coincides with rh or points to value equal to or greater than pivot.

[ 56 25 37 58 95 19 73 30 ]

lh rh

4. If lh and rh not pointing to same element , exchange the elements.

[56 25 37 30 95 19 73 58]

lh rh

5. Repeat steps 2 to 4 until lh and rh coincide.

[56 25 37 30 95 19 73 58] lh rh

move lh to right till it finds an element larger than 56

[56 25 37 30 95 19 73 58] lh rh

Now exchange

[56 25 37 30 19 95 73 58] lh rh

move rh to left

[56 25 37 30 19 95 73 58] lh rh No exchange now, as both lh and rh point to the same element

6. This will be the smallest value. Exchange this with the pivot.

[19 25 37 30 56 95 73 58]

7. This will result in two sub arrays, one to left of pivot and one to right of pivot.

[19 25 37 30 ] 56 [ 95 73 58 ]

8. Repeat steps 1 to 6 for the two sub arrays recursively.