Understanding Algorithm Efficiency: Correctness and Efficiency Properties, Study notes of Computer Science

The importance of correctness and efficiency in algorithm design. How the correctness of an algorithm ensures it solves the problem and produces a correct answer, while efficiency focuses on how quickly the problem can be solved. The document also introduces the concept of big-o notation and its role in describing algorithmic efficiency. Students of computer science, particularly those in algorithms or data structures courses, will find this document useful for understanding the fundamental concepts of algorithm design.

Typology: Study notes

Pre 2010

Uploaded on 08/18/2009

koofers-user-02a
koofers-user-02a 🇺🇸

10 documents

1 / 4

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
On the Board
Homework due today
Reading — Chapters 5 & 6
Important Properties of Algorithms
In Chapter 4, we saw a number of approaches to algorithm design — graph
traversals, divide and conquer, greedy, dynamic planning. Speaking in
broad terms, two properties of an algorithm should be critical concerns (in
this order)
Correctness — The algorithm should solve the problem and produce a
correct answer. Correctness is the focus of Chapter 5.
For some problems, such as phonebook lookup, the correctness criterion
is simple and well defined: “Joe” is in the phonebook or he is not.
Sorting (as in bubblesort, mergesort, and insertion sort) is equally well
defined. For other problems, the criterion may not be well defined:
navigating from Rice to the Galleria, the greedy algorithm will make
locally optimal decisions that may lead to globally suboptimal solutions.
Is the suboptimal solution wrong? It leads to the Galleria. It is faster than
some of the paths. The greedy algorithm approximates the minimal-cost
path with a lower cost algorithm.
With an algorithm that produces approximate solutions, it is critical that
the designer and user both understand what results the algorithm
produces—in particular, whether the solution is exact or approximate. (In
some cases, we can bound the distance between the approximation and
the optimal solution!)
Efficiency — How quickly can we solve the problem? Efficiency is the
focus of Chapter 6.
Going back to the first lecture, the parallel sort by first name was faster
(in clock time) than the insertion sort. To achieve that speed, it used all of
your brains in parallel. (Looking at total resource utilization, it may have
required more resources than the insertion sort.)
COMP 200: Elements of Computer Science
Fall 2004
Lecture 20: October 14, 2004
Introduction to Algorithmic Complexity
pf3
pf4

Partial preview of the text

Download Understanding Algorithm Efficiency: Correctness and Efficiency Properties and more Study notes Computer Science in PDF only on Docsity!

On the Board Homework due today Reading — Chapters 5 & 6 Important Properties of Algorithms In Chapter 4, we saw a number of approaches to algorithm design — graph traversals, divide and conquer, greedy, dynamic planning. Speaking in broad terms, two properties of an algorithm should be critical concerns (in this order)

  • Correctness — The algorithm should solve the problem and produce a correct answer. Correctness is the focus of Chapter 5. For some problems, such as phonebook lookup, the correctness criterion is simple and well defined: “Joe” is in the phonebook or he is not. Sorting (as in bubblesort, mergesort, and insertion sort) is equally well defined. For other problems, the criterion may not be well defined: navigating from Rice to the Galleria, the greedy algorithm will make locally optimal decisions that may lead to globally suboptimal solutions. Is the suboptimal solution wrong? It leads to the Galleria. It is faster than some of the paths. The greedy algorithm approximates the minimal-cost path with a lower cost algorithm. With an algorithm that produces approximate solutions, it is critical that the designer and user both understand what results the algorithm produces—in particular, whether the solution is exact or approximate. (In some cases, we can bound the distance between the approximation and the optimal solution!)
  • Efficiency — How quickly can we solve the problem? Efficiency is the focus of Chapter 6. Going back to the first lecture, the parallel sort by first name was faster (in clock time) than the insertion sort. To achieve that speed, it used all of your brains in parallel. (Looking at total resource utilization, it may have required more resources than the insertion sort.) COMP 200: Elements of Computer Science Fall 2004 Lecture 20: October 14, 2004 Introduction to Algorithmic Complexity

Both the bubblesort algorithm and the mergesort algorithm that we derived in class perform use a number of comparisons that is roughly equivalent to the square of the number of elements being sorted. The mergesort uses about half as many comparisons as bubblesort, but both are still proportional to N^2 for a set of N names. A good queue implementation (remember the homework that is due today) has the property that Add , Remove , and Empty all perform a constant amount of work each time that they are used. Efficiency: What’s the Big Deal? Computer scientists are always concerned about the efficiency of algorithms. Why? Computers have finite resources—including memory, disk, screen resolution, and computational capacity. (What’s the difference between a 1 GHZ Pentium and a 2.8GHZ Pentium? Why would you want one over the other? Same question about a 20GB iPod versus a 40GB iPod.) In choosing between two algorithms, you might well consider their relative efficiency.

  • How does the cost of the algorithm (number of comparisons, total number of operations) increase as the size of the input grows?
  • How much memory does the algorithm require? And how does the memory requirement grow with input size? These considerations have practical impact. The antilock brakes in your car are run by software. If the algorithm to detect a skid is too slow, you lose control of the car. If the algorithm to find the cheapest flight between Houston and Boston has a complexity that rises too quickly with the number of available flights, Travelocity goes out of business. If the algorithm that digitizes your voice in a cellphone (or that maintains the phone’s connection with the network) uses too many operations, your battery life suffers. How efficient should an algorithm be?
  • Finding the largest number in a list of numbers, you would reasonably expect that we must look at each number. An algorithm that uses roughly N comparisons is about as good as we can do — common sense. In fact, N- 1 comparisons is a lower bound for this problem.
  • Finding the minimal length path from Rice to the Galleria, the approximate greedy algorithm will make a choice at each intersection. The number of comparisons (and decisions) should not be more than the number of intersections — in the worst case. In the expected case, you hope to avoid visiting every intersection (a complete tour).

operations. That notwithstanding, its worst case behavior is O (number of intersections). If the function, f(N) , in the inequality can be expressed as a polynomial, we say that the algorithm has polynomial complexity. In general, problems that have polynomial complexities are considered tractable problems. Many important problems have this property — in fact, most of the problems that we solve with computers are tractable. Low-order polynomial algorithms are considered to have efficient solutions. Multiplying two matrices — a common subtask in many scientific applications — takes O ( N^3 ) time.