Download Machine Learning Essentials and more Cheat Sheet Machine Learning in PDF only on Docsity!
Algorithms: A Brief Introduction
Slides by Christopher M. Bourke Instructor: Berthe Y. Choueiry
Spring 2006
Computer Science & Engineering 235 Introduction to Discrete Mathematics Section 2.1 of Rosen [email protected]
Notes
Algorithms
Brief Introduction
Real World Computing World Objects Data Structures, ADTs, Classes Relations Relations and functions Actions Operations
Problems are instances of objects and relations between them. Algorithms^1 are methods or procedures that solve instances of problems
(^1) ”Algorithm” is a distortion of al-Khwarizmi, a Persian mathematician
Notes
Algorithms
Formal Definition
Definition
An algorithm is a sequences of unambiguous instructions for solving a problem. Algorithms must be
I (^) Finite – must eventually terminate. I (^) Complete – always gives a solution when there is one. I Correct (sound) – always gives a “correct” solution.
For an algorithm to be a feasible solution to a problem, it must also be effective. That is, it must give a solution in a “reasonable” amount of time. There can be many algorithms for the same problem.
Notes
Algorithms
General Techniques
There are many broad categories of Algorithms: Randomized algorithms, Monte-Carlo algorithms, Approximation algorithms, Parallel algorithms, et al. Usually, algorithms are studied corresponding to relevant data structures. Some general styles of algorithms include
- Brute Force (enumerative techniques, exhaustive search)
- Divide & Conquer
- Transform & Conquer (reformulation)
- Greedy Techniques
Notes
Pseudo-code
Algorithms are usually presented using some form of pseudo-code. Good pseudo-code is a balance between clarity and detail. Bad pseudo-code gives too many details or is too implementation specific (i.e. actual C++ or Java code or giving every step of a sub-process). Good pseudo-code abstracts the algorithm, makes good use of mathematical notation and is easy to read.
Notes
Good Pseudo-code
Example
Intersection
Input : Two sets of integers, A and B Output : A set of integers C such that C = A ∩ B 1 C ← ∅ 2 if |A| > |B| then 3 swap(A, B) 4 end 5 for every x ∈ A do 6 if x ∈ B then 7 C ← C ∪ {x} 8 end 9 end 10 output C
Latex notation: \leftarrow.
Notes
MAX
Analysis
This is a simple enough algorithm that you should be able to:
I (^) Prove it correct I (^) Verify that it has the properties of an algorithm. I Have some intuition as to its cost.
That is, how many “steps” would it take for this algorithm to complete its run? What constitutes a step? How do we measure the complexity of the step? These questions will be answered in the next few lectures, for now let us just take a look at a couple more examples.
Notes
Other examples
Check Bubble Sort and Insertion Sort in your textbooks, which you have seen ad nauseum, in CSE155, CSE156, and will see again in CSE310. I will be glad to discuss them with any of you if you have not seen them yet.
Notes
Greedy algorithm
Optimization
In many problems, we wish to not only find a solution, but to find the best or optimal solution. A simple technique that works for some optimization problems is called the greedy technique. As the name suggests, we solve a problem by being greedy—that is, choosing the best, most immediate solution (i.e. a local solution). However, for some problems, this technique is not guaranteed to produce the best globally optimal solution.
Notes
Example
Change-Making Problem
For anyone who’s had to work a service job, this is a familiar problem: we want to give change to a customer, but we want to minimize the number of total coins we give them. Problem Given An integer n and a set of coin denominations (c 1 , c 2 ,... , cr) with c 1 > c 2 > · · · > cr Output A set of coins d 1 , d 2 , · · · , dk such that ∑k i=1 di^ =^ n^ and^ k is minimized.
Notes
Example
Change-Making Algorithm
Change
Input : An integer n and a set of coin denominations (c 1 , c 2 ,... , cr ) with c 1 > c 2 > · · · > cr. Output : A set of coins d 1 , d 2 , · · · , dk such that Pki=1 di = n and k is minimized. 1 C ← ∅ 2 for i = 1,... , r do 3 while n ≥ ci do 4 C ← C ∪ {ci} 5 n ← n − ci 6 end 7 end 8 output C
Notes
Change-Making Algorithm
Analysis
Will this algorithm always produce an optimal answer? Consider a coinage system:
I where c 1 = 20, c 2 = 15, c 3 = 7, c 4 = 1 I (^) and we want to give 22 “cents” in change.
What will this algorithm produce? Is it optimal? It is not optimal since it would give us one c 4 and two c 1 , for three coins, while the optimal is one c 2 and one c 3 for two coins.
Notes
