Big O Notation and Time Complexity in Discrete Mathematics, Slides of Discrete Mathematics

Download Big O Notation and Time Complexity in Discrete Mathematics and more Slides Discrete Mathematics in PDF only on Docsity! Introduction to Discrete Mathe ma tic s Docsity.com This is The Big Oh! Docsity.com * * * * * * * * * * * * * * * * * * * * * * * * * * + How to a dd 2 n-b it numbe rs Docsity.com * * * * * * * * * * * * * * * * * * * * * * * * * * * * + How to a dd 2 n-b it numbe rs Docsity.com * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * + How to a dd 2 n-b it numbe rs Docsity.com Our Goal We wa nt to de fine “time ” in a wa y tha t tra ns c e nd s imp le me nta tion de ta ils a nd a llows us to ma ke a s s e r tions a bout g ra de s c hool a d d ition in a ve ry ge ne ra l ye t us e ful wa y. Docsity.com • A given algorithm will take different amounts of time on the same inputs depending on such factors as: » Processor speed » Instruction set » Disk speed » Brand of compiler Roadblock ??? Docsity.com On any reasonable computer, adding 3 b its a nd writing down the two b it a ns we r c a n b e done in c ons ta nt time Pick a ny pa r tic ula r c ompute r M a nd de fine c to be the time it ta ke s to pe rform on tha t c ompute r. Tota l time to a dd two n-b it numbe rs us ing gra de s chool a dd ition: c n [c time for e a c h of n c olumns ] Docsity.com Thus we arr ive at an implementation inde pe nde nt ins ight: Gra de Sc hool Add ition is a line a r time a lgor ithm This p roc e s s of a b s tra c ting a wa y de ta ils a nd de te rmining the ra te of re s ourc e us a ge in te rms of the p roble m s ize n is one of the fund a me nta l ide a s in c ompute r s c ie nc e. Docsity.com Time vs Input Size For a ny a lgor ithm, de fine Input S ize = # of b its to s pe c ify its inputs . De fine TIMEn = the wors t-c a s e a mount of time us e d on inputs of s ize n We ofte n a s k: Wha t is the growth ra te of Time n ? Docsity.com X * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * n2 How to multip ly 2 n-b it numbe rs . Docsity.com How much time does it take to square the numbe r N us ing gra de s chool multip lic a tion? Docsity.com Grade School Multiplication: Quadratic time # of bits in numbers t i m e Input s ize is me a s ure d in b its , unle s s we s a y othe rwis e. c (log N)2 time to s qua re the numbe r N Docsity.com How much time does it take? Nursery School Addition Input: Two n-b it numbe rs , a a nd b Output: a + b S ta r t a t a a nd inc re me nt (by 1) b time s T(n) = ? If b = 000…0000, the n NSA ta ke s a lmos t no time If b = 1111…11111, the n NSA ta ke s c n2n time Docsity.com Thus, Nursery School adding and Kinde rga r te n multip lic a tion a re e xpone ntia l time. The y s c a le HORRIBLY a s input s ize grows. Gra de s c hool me thods s c a le polynomia lly: jus t line a r a nd qua d ra tic . Thus , we c a n a dd a nd multip ly fa ir ly la rge numbe rs . Docsity.com If T(n) is not polynomia l, the a lgor ithm is not e ffic ie nt: the run time s c a le s too poorly with the input s ize . This will be the ya rds tick with whic h we will me a s ure “e ffic ie nc y”. Docsity.com Multiplication is effic ient, what about “re ve rs e multip lic a tion”? Le t’s d e fine FACTORING(N) to b e a ny me thod to p roduc e a non-tr ivia l fa c tor of N, or to a s s e r t tha t N is p r ime. Docsity.com Notation to Discuss Growth Rates For any monotonic function f from the pos itive in te ge rs to the pos itive in te ge rs , we s a y “f = O(n)” or “f is O(n)” If s ome c ons ta nt time s n e ve ntua lly domina te s f [Forma lly: the re e xis ts a c ons ta nt c s uc h tha t for a ll s uffic ie ntly la rge n: f(n) ≤ c n ] Docsity.com # of bits in numbers t i m e f = O(n) me a ns tha t the re is a line tha t c a n be d ra wn tha t s ta ys a bove f from s ome point on Docsity.com For any monotonic function f from the pos itive in te ge rs to the pos itive in te ge rs , we s a y “f = Ω(n)” or “f is Ω(n)” If f e ve ntua lly domina te s s ome c ons ta nt time s n [Forma lly: the re e xis ts a c ons ta nt c s uc h tha t for a ll s uffic ie ntly la rge n: f(n) ≥ c n ] Other Useful Notation: Ω Docsity.com # of bits in numbers t i m e f = Θ(n) me a ns tha t f c a n be s a ndwic he d be twe e n two line s from s ome point on. Docsity.com Notation to Discuss Growth Rates For any two monotonic functions f a nd g from the pos itive in te ge rs to the pos itive in te ge rs , we s a y “f = O(g )” or “f is O(g )” If s ome c ons ta nt time s g e ve ntua lly domina te s f [Forma lly: the re e xis ts a c ons ta nt c s uc h tha t for a ll s uffic ie ntly la rge n: f(n) ≤ c g (n) ] Docsity.com # of bits in numbers t i m e f = O(g ) me a ns tha t the re is s ome c ons ta nt c s uc h tha t c g (n) s ta ys a bove f(n) from s ome point on. f g 1.5g Docsity.com – n = O(n2) ? Take c = 1 For all n ≥ 1 , it hold s tha t n ≤ c n 2 Ye s ! Docsity.com – n = O(n2) ? – n = O(√n) ? Suppose it were true that n ≤ c √n for s ome c ons ta nt c a nd la rge e nough n Ca nc e lling, we would ge t √n ≤ c . Whic h is fa ls e for n > c 2 Ye s ! No Docsity.com – n = O(n2) ? – n = O(√n) ? – 3n2 + 4n + 4 = O(n2) ? – 3n2 + 4n + 4 = Ω(n2) ? – n2 = Ω(n log n) ? – n2 log n = Θ(n2) ? Yes! No Yes! Yes! Yes! No Docsity.com Small Growth Rates Logar ithmic Time: T(n) = O(logn) Exa mple : T(n) = 15log 2(n) Polyloga r ithmic Time : for s ome c ons ta nt k, T(n) = O(log k(n)) Note : The s e kind of a lgor ithms c a n’t pos s ib ly re a d a ll of the ir inputs . A ve ry c ommon e xa mple of loga r ithmic time is looking up a word in a s or te d d ic tiona ry (b ina ry s e a rch) Docsity.com Some Big Ones T ( n ) = 2 2 k n T ( n ) = 2 2 2 k n Doubly Exponential Time means that for some constant k Tr iply Exponential Docsity.com 2 2 2 2 : : : 2 “tower of n 2’s” 2STACK(n) = Faster and Faster: 2STACK 2STACK(0) = 1 2STACK(n) = 22STACK(n-1) 2STACK(1) = 2 2STACK(2) = 4 2STACK(3) = 16 2STACK(4) = 65536 2STACK(5) ≥ 1080 = a toms in unive rs e Docsity.com Ackermann’s Function A (0, n) = n + 1 for n ≥ 0 A(m, 0) = A(m - 1, 1) for m ≥ 1 A(m, n) = A(m - 1, A(m, n - 1)) for m, n ≥ 1 A(4,2) > # of pa r tic le s in unive rs e A(5,2) c a n’t be writte n out a s de c ima l in th is unive rs e Docsity.com Ackermann’s Function A (0, n) = n + 1 for n ≥ 0 A(m, 0) = A(m - 1, 1) for m ≥ 1 A(m, n) = A(m - 1, A(m, n - 1)) for m, n ≥ 1 De fine : A’(k) = A(k,k) Inve rs e Acke rma n α(n) is the inve rs e of A’ Pra c tic a lly s pe a king : n × α(n) ≤ 4n Docsity.com The inverse Ackermann function – in fa c t, Θ(n α(n)) a r is e s in the s e mina l pa pe r of: D. D. S le a tor a nd R. E. Ta rja n. A data structure for dynamic trees. Journal of Computer and System Sciences, 26(3):362-391, 1983. Docsity.com