Computers and Computer Science - Lecture Slides | CS 325, Study notes of Algorithms and Programming

Download Computers and Computer Science - Lecture Slides | CS 325 and more Study notes Algorithms and Programming in PDF only on Docsity!

Algorithms

Prasad Tadepalli

Computers and Computer Science

“Computer Science is nomore about computers thanAstronomy is abouttelescopes” – E.W. Dijkstra(1930-2002)

The field of Algorithmswhich is at the heart ofComputer Science is mucholder than Computers andComputer Science

Algorithms – Pre-history

325- 265 BC, Euclid of Alexandria –Sometimes called the “grandfatherof Algorithms” invented Geometryand number theory.

~600 AD – Decimal system wasinvented in India

~780-850 AD – Al Khwarizmi

~1170-1250 AD – LeonardoFibonacci – popularized algorithmsin Europe

1448 – Guttenberg invented theprinting press

Fibonacci

Euclid

Turing Machine and Effective

Procedures

Alan Turing (1912-1954) isconsidered the Father ofComputer Science.

In 1936, he formalized thenotion of an

effective procedure

as a set of rules that can bemechanically followed by amachine.

The rules must be precise andunambiguous

For a procedure to be an algorithm

, the machine must

eventually stop with the correctanswer.

Some High-level Programming Languages (Turing equivalent)

Fortran, John Backus, 1957

Lisp, John McCarthy, 1959

Cobol, Grace Hopper et al, 1959

Algol 60, John Backus, 1960

Pascal, Niklaus Wirth, 1970

C, Dennis Ritchie, 1972

Prolog, Alain Colmerauer and PhilippeRoussel, 1972

C++, Bjarne Stroustrup, 1986

Java, Sun Microsystems, 1995

What are algorithms?

Precise mechanical procedure to computecorrect answers to all instances of a problem.(Must terminate on all possible inputs.)

Are described in

pseudocode

that can be

translated to any programming language.

Three questions we ask about every algorithm

Is it correct?

How fast does it run?

Can we do better?

Computational Problems

Computational problems are mathematical functionsfrom inputs to outputs:

Shortest Path Problem Input:

Map, starting address, destination

Output

: Directions for

shortest route

Sorting numbers Input

: A sequence of numbers, e.g., [5,8,1,2]

Output:

Sorted sequence of numbers, e.g., [1,2,5,8]

Problem instance is one particular possible input, e.g.,[5, 8, 1, 2]

The size of the instance is the length of its encoding,e.g., the number of numbers to be sorted. We use

n

to

denote problem size.

Do all problems have algorithms?

How to measure the time?

In general, the bigger the problem, the longer it takes tosolve it. So we make the time complexity

t(n),

a function

of the problem size

n

.

Even for the same size, some problems are more difficultthan others. Analyze the hardest problem of that size(

worst-case

complexity).

The exact time taken depends on

The programming language used to implement the algorithm

The compiler

The computer hardware

What other jobs are running in the machine

How much memory does one have

Network speed, etc.

O

-Notation

Problem: How do we compare times in away that does not depend on the details ofimplementation?

Insight: Implementation differences onlymake a constant factor difference in time.

We say a function

f = O(g)

if there is a

constant

c

such that

f(n)

c g(n)

for all

n

A function

f =

(g)

if

g = O(f)

A function

f =

(g)

if f

(g)

and

f=O(g)

Three functions

0

900080007000600050004000300020001000

0

2

4

6

8

10

12

14

y=x^2y=x^3y=2^x

Exponentials vs. Polynomials

0

800000600000400000200000

12000001000000

0

5

10

15

20

25

y=x^2y=x^3y=2^x

Efficient usually means polynomial-time, although for someapplications, O(n^5) or even O(n^3) is not considered efficient;and for some applications, exponential worst-case is stillacceptable.

Three Questions

Is Fib1 correct? Yes – it directly translates the definition into

pseudocode.

How long does it take? Suppose it takes t(n) on n. We have the

following

recurrence relation

t(0) = 1; t(1)=2; t(n)= t(n-1)+t(n-2)+3.

So t(n) > F(n). This is bad because F(n)=1.

n

Why is Fib1 so bad?

Fib1(4) = Fib1(3) + Fib1(2)

= Fib1(2)+Fib1(1)+Fib1(1)+Fib1(0)

= Fib1(1)+Fib1(0) + 1 + 1 + 0 = 1 + 0 + 1 + 1 + 0 = 3

Fib1(2) is calculated twice! To calculate Fib1(5), Fib1(3) will be called twice,

and Fib1(2) will be called 3 times, Fib1(1) will be called 4 times!

The number of redundant calculations increase

exponentially with n!