Download Algorithm Analysis: Comparing Time Efficiency and Space Complexity of Algorithms A and B and more Exams Data Structures and Algorithms in PDF only on Docsity!
02-0: Algorithm Analysis
When is algorithm A better than algorithm B? 02-1: Algorithm Analysis When is algorithm A better than algorithm B?
02-2: Algorithm Analysis
When is algorithm A better than algorithm B?
- Algorithm A runs faster
- Algorithm A requires less space to run
02-3: Algorithm Analysis
When is algorithm A better than algorithm B?
- Algorithm A runs faster
- Algorithm A requires less space to run
Space / Time Trade-off
- Can often create an algorithm that runs faster, by using more space
For now, we will concentrate on time efficiency 02-4: Best Case vs. Worst Case How long does the following function take to run:
boolean find(int A[], int element) { for (i=0; i Worst Case performance is almost always important. Usually , Best Case performance is unimportant (why?) Usually , Average Case = Worst Case (but not always!) 02-7: Measuring Time Efficiency How long does an algorithm take to run? 02-8: Measuring Time Efficiency
How long does an algorithm take to run?
- Implement on a computer, time using a stopwatch.
02-9: Measuring Time Efficiency
How long does an algorithm take to run?
- Implement on a computer, time using a stopwatch. Problems: - Not just testing algorithm – testing implementation of algorithm - Implementation details (cache performance, other programs running in the background, etc) can affect results - Hard to compare algorithms that are not tested under exactly the same conditions
02-10: Measuring Time Efficiency
How long does an algorithm take to run?
- Implement on a computer, time using a stopwatch. Problems: - Not just testing algorithm – testing implementation of algorithm - Implementation details (cache performance, other programs running in the background, etc) can affect results - Hard to compare algorithms that are not tested under exactly the same conditions
- Better Method: Build a mathematical model of the running time, use model to compare algorithms
02-11: Competing Algorithms
for (i=low; i <= high; i++) if (A[i] == elem) return true; return false;
int BinarySearch(int low, int high, elem) { if (low > high) return false; mid = (high + low) / 2; if (A[mid] == elem) return true; if (A[mid] < elem) return BinarySearch(mid+1, high, elem); else return BinarySearch(low, mid-1, elem); }
Time Required, for a problem of size n (worst case): c 2 ∗ lg(n) for some constant c 2
02-16: Do Constants Matter?
- Linear Search requires time c 1 ∗ n, for some c 1
- Binary Search requires time c 2 ∗ lg(n), for some c 2
What if there is a very high overhead cost for function calls?
What if c 2 is 1000 times larger than c 1? 02-17: Constants Do Not Matter! Length of list Time Required for Time Required for Linear Search Binary Search 10 0.001 seconds 0.3 seconds 100 0.01 seconds 0.66 seconds 1000 0.1 seconds 1.0 seconds 10000 1 second 1.3 seconds 100000 10 seconds 1.7 seconds 1000000 2 minutes 2.0 seconds 10000000 17 minutes 2.3 seconds 1010 11 days 3.3 seconds 1015 30 centuries 5.0 seconds 1020 300 million years 6.6 seconds 02-18: Growth Rate We care about the Growth Rate of a function – how much more we can do if we add more processing power
Faster Computers 6 = Solving Problems Faster Faster Computers = Solving Larger Problems
- Modeling more variables
- Handling bigger databases
- Pushing more polygons
02-19: Growth Rate Examples Size of problem that can be solved time 10 n 5 n n lg n n^2 n^3 2 n 1 s 1000 2000 1003 100 21 13 2 s 2000 4000 1843 141 27 14 20 s 20000 40000 14470 447 58 17 1 m 60000 120000 39311 774 84 19 1 hr 3600000 7200000 1736782 18973 331 25 02-20: Constants and Running Times
- When calculating a formula for the running time of an algorithm:
- Constants aren’t as important as the growth rate of the function
- Lower order terms don’t have much of an impact on the growth rate
- We’d like a formal method for describing what is important when analyzing running time, and what is not.
02-21: Big-Oh Notation
O(f (n)) is the set of all functions that are bound from above by f (n)
T (n) ∈ O(f (n)) if
∃c, n 0 such that T (n) ≤ c ∗ f (n) when n > n 0 02-22: Big-Oh Examples n ∈ O(n)? 10 n ∈ O(n)? n ∈ O(10n)? n ∈ O(n^2 )? n^2 ∈ O(n)? 10 n^2 ∈ O(n^2 )? n lg n ∈ O(n^2 )? ln n ∈ O(2n)? lg n ∈ O(n)? 3 n + 4 ∈ O(n)? 5 n^2 + 10n − 2 ∈ O(n^3 )? O(n^2 )? O(n)? 02-23: Big-Oh Examples n ∈ O(n) 10 n ∈ O(n) n ∈ O(10n) n ∈ O(n^2 ) n^2 6 ∈ O(n) 10 n^2 ∈ O(n^2 ) n lg n ∈ O(n^2 ) ln n ∈ O(2n) lg n ∈ O(n) 3 n + 4 ∈ O(n) 5 n^2 + 10n − 2 ∈ O(n^3 ),∈ O(n^2 ), 6 ∈ O(n)? 02-24: Big-Oh Examples II √ n ∈ O(n)? lg n ∈ O(2n)? lg n ∈ O(n)? n lg n ∈ O(n)? n lg√ n ∈ O(n^2 )? n ∈ O(lg n)? lg n ∈ O(
n)? n lg n ∈ O(n (^32) )? n^3 + n lg n + n
n ∈ O(n lg n)? n^3 + n lg n + n
n ∈ O(n^3 )? n^3 + n lg n + n
n ∈ O(n^4 )? 02-25: Big-Oh Examples II
T (n) ∈ O(f (n)) and T (n) ∈ Ω(f (n)) 02-31: Big-Oh Rules
- If f (n) ∈ O(g(n)) and g(n) ∈ O(h(n)), then f (n) ∈ O(h(n))
- If f (n) ∈ O(kg(n) for any constant k > 0 , then f (n) ∈ O(g(n))
- If f 1 (n) ∈ O(g 1 (n)) and f 2 (n) ∈ O(g 2 (n)), then f 1 (n) + f 2 (n) ∈ O(max(g 1 (n), g 2 (n)))
- If f 1 (n) ∈ O(g 1 (n)) and f 2 (n) ∈ O(g 2 (n)), then f 1 (n) ∗ f 2 (n) ∈ O(g 1 (n) ∗ g 2 (n))
(Also work for Ω, and hence Θ) 02-32: Big-Oh Guidelines
- Don’t include constants/low order terms in Big-Oh
- Simple statements: Θ(1)
- Loops: Θ(inside) * # of iterations
- Nested loops work the same way
- Consecutive statements: Longest Statement
- Conditional (if) statements: O(Test + longest branch)
02-33: Calculating Big-Oh
for (i=1; i 02-38: Calculating Big-Oh
for (i=1; i sum = 0; for (i=1; i 02-56: Calculating Big-Oh
Of course, a little change can mess things up a bit ...
sum = 0; for (i=1; i<=n; i=i+1) Executed n times for (j=1; j<=i; j=j*2) Executed <= lg n times sum++; O(1)
So, this is code is O(n lg n) – but is it also Ω(n lg n)? 02-57: Calculating Big-Oh
Of course, a little change can mess things up a bit ...
sum = 0; for (i=1; i<=n; i=i+1) Executed n times for (j=1; j<=i; j=j*2) Executed <= lg n times sum++; O(1)
Total time sum++ is executed:
∑^ n
i− 1
lg i
This can be tricky to evaluate, but we only need a bound ... 02-58: Calculating Big-Oh Total # of times sum++ is executed:
∑^ n
i=
lg i =
n/ ∑ 2 − 1
i=
lg i +
∑^ n
i=n/ 2
lg i
∑^ n
i=n/ 2
lg i
∑^ n
i=n/ 2
lg n/ 2
= n/2 lg n/ 2 ∈ Ω(n lg n)
