Download Asymptotic Notations: Big O, Vinogradov Notation, and Asymptotic Series and more Study notes Number Theory in PDF only on Docsity!
Chapter 2
Asymptotic notations
2.1 The “oh” notations
Terminology Notation Definition Big oh notation f (s) = O(g(s)) (s ∈ S) There exists a constant c such that |f (s)| ≤ c|g(s)| for all s ∈ S Vinogradov nota- tion
f (s) g(s) (s ∈ S) Equivalent to “f (s) = O(g(s)) (s ∈ S)” Order of magnitude estimate
f (s) g(s) (s ∈ S) Equivalent to “f (s) g(s) and g(s) f (s) (s ∈ S)”. Small oh notation f (s) = o(g(s)) (s → s 0 ) lims→s 0 f (s)/g(s) = 0 Asymptotic equiva- lence
f (s) ∼ g(s) (s → s 0 ) lims→s 0 f (s)/g(s) = 1
Omega estimate f (s) = Ω(g(s)) (s → s 0 ) lim sups→s 0 |f (s)/g(s)| >
Table 2.1: Overview of asymptotic terminology and notation. In these defi- nitions S denotes a set of real or complex numbers contained in the domain of the functions f and g, and s 0 denotes a (finite) real or complex number or ±∞.
A very convenient set of notations in asymptotic analysis are the so-
10 CHAPTER 2. ASYMPTOTIC NOTATIONS
called “big oh” (O) and “small-oh” (o) notations, and their variants. These notations are in widespread use and are often used without further explana- tion. However, in order to properly apply these notations and avoid mistakes resulting from careless use, it is important to be aware of their precise defi- nitions. In this section we give formal definitions of the “oh” notations and their variants, show how to work with these notations, and illustrate their use with a number of examples. Tables 2.1 and 2.2 give an overview of these notations.
Short-hand form Full form f (s) = O(g(s)) (s → s 0 ) There exists a constant δ > 0 such that f (s) = O((g(s)) (|s − s 0 | ≤ δ). f (x) = O(g(x)) There exists a constant x 0 such that f (x) = O((g(x)) (x ≥ x 0 ). f (x) = o(g(x)) f (x) = o(g(x)) (x → ∞).
Table 2.2: Notational conventions and shortcuts for commonly occurring asymptotic expressions.
2.1.1 Definition of “big oh”, special case
We consider first the simplest and most common case encountered in asymp- totics, namely the behavior of functions of a real variable x as x → ∞. Given two such functions f (x) and g(x), defined for all sufficiently large real num- bers x, we write f (x) = O(g(x))
as short-hand for the following statement: There exist constants x 0 and c such that |f (x)| ≤ c|g(x)| (x ≥ x 0 ).
If this holds, we say that f (x) is of order O(g(x)), and we call the above estimate a O-estimate (“big oh estimate”) for f (x). The constant c called the O-constant, and the range x ≥ x 0 the range of validity of the O- estimate. In exactly the same way we define the relation “f (n) = O(g(n))” if f and g are functions of an integer variable n.
12 CHAPTER 2. ASYMPTOTIC NOTATIONS
O-notation allows us to ignore these complications: all we need to know is the existence of a constant, and this, as we have seen, is easy to establish with general continuity or compactness arguments.
Example 2.3. If P (x) =
∑n k=0 akx
k (^) is a polynomial of degree n, then
P (x) = O(xn).
Proof. For x ≥ 1 we have
|P (x)| ≤
∑^ n
i=
|ai|xi^ ≤
( (^) n ∑
i=
|ai|
xn,
so the required inequality holds with x 0 = 1 and c =
∑n i=0 |ai|.
Example 2.4. The relation
f (x) = O(1)
simply means that f (x) is bounded as x → ∞.
2.1.2 Dependence on parameters
In many cases, the functions involved in an O-estimate depend on one or more parameters. It may then be important to know whether the O-constant depends on these parameters or can be chosen independently of the param- eters. If the constant (possibly) depends on one or more parameters, it is customary to indicate this dependence by placing the parameters as sub- scripts to the O-symbol and writing, for example, Oλ, Ok, or Ok,. The same convention applies, if the constant depends on a parameter arising in the range of an estimate (rather than the functions to be estimated). To avoid mistakes, it is a good practice to explicitly indicate the depen- dence of O-estimates on any parameters by using the subscript notation, and we will generally adhere to this practice. If it is possible to choose the constant in an O-estimate independent of some parameter occurring in the definition of the function or the range of the estimate, we say that the estimate is uniform (or holds uniformly) with respect to the given parameter. Uniform estimates are more informative and more useful than nonuniform estimates, and obtaining uniform estimates or making non-uniform estimates uniform (e.g., by making the dependence on parameters explicit) is a desirable goal.
2.1. THE “OH” NOTATIONS 13
Example 2.5. In Example 2.2 we showed, by a simple continuity argument, that, for any positive constants A and , we have (x+1)A^ = O(exp((log x)1+)) in the range x ≥ 1. While, in this case, the range x ≥ 1 could be chosen independently of the constants A and , this is not true for the O-constant c. Thus, to indicate the (possible) dependence of the O-constant on A and , we should write this O-estimate more precisely as
(x + 1)A^ = OA,
exp
(log x)1+
(x ≥ 1).
In general, the subscript notation simply says that the constant may depend on the indicated parameters, not that it is not possible (for example, through a more clever argument) to find a constant independent of the parameters. However, in this particular example, it is easy to see that the constant necessarily has to depend on both parameters A and .
2.1.3 Definition of “big oh”, general case
If f (s) and g(s) are functions of a real or complex variable s and S is an arbitrary set of (real or complex) numbers s (belonging to the domains of f and g), we write f (s) = O(g(s)) (s ∈ S),
if there exists a constant c such that
|f (s)| ≤ c|g(s)| (s ∈ S).
To be consistent with our earlier definition of “big oh” we make the following convention: If a range is not explicitly given, then the estimate is assumed to hold for all sufficiently large values of the variable involved, i.e., in a range of the form x ≥ x 0 , for a suitable constant x 0.
Example 2.6. Given any positive constant r < 1, we have
log(1 + z) = Or(|z|) (|z| < r).
Proof. Note that the function log(1 + z) is analytic in the open unit disk |z| < 1 and has power series expansion
log(1 + z) =
∑^ ∞
n=
(−1)n+ n zn^ (|z| < 1).
2.1. THE “OH” NOTATIONS 15
more naturally as
log(1 + x) = x + O(x^2 ) (|x| ≤ 1 /2).
The latter can be thought of as a succinct form of the following rather unwieldy statement. “ log(1+x) is equal to x plus a function that, in absolute value, is bounded by a constant times x^2 in the range |x| ≤ 1 / 2 .”
Example 2.8. Power series expansions naturally lead to O-estimates in the above more generalized sense. In particular, if f (z) is a function analytic in some disk |z| < R, then for any r < R and any fixed positive integer n, we have, by Taylor’s theorem,
f (z) =
∑^ n
k=
akzk^ + Or,n(|z|n+1) (|z| < r),
where the ak are the Taylor coefficients of f (z).
Example 2.9. A term O(1) simply stands for a bounded function. For example, the “floor function” [x] satisfies
[x] = x + O(1),
since |[x] − x| ≤ 1.
2.1.5 The Vinogradov “ ” notation
This notation was introduced by the Russian number theorist I.M. Vino- gradov as an alternative to the O-notation. Along with the closely related notations “ ” and “”, it has all become standard in number theory, though it is less common in other areas of mathematics. In the case of functions of a real variable x and (implicit) ranges of the form x ≥ x 0 , these three notations are defined as follows:
- “f (x) g(x)” is equivalent to “f (x) = O(g(x))”.
- “f (x) g(x)” is equivalent to “g(x) f (x)”.
- “f (x) g(x)” means that both “f (x) g(x)” and “g(x) f (x)” hold.
These definitions generalize in an obvious manner to more general func- tions and ranges.
16 CHAPTER 2. ASYMPTOTIC NOTATIONS
If f (x) g(x), we say that f (x) and g(x) have the same order of magnitude. From the definition it is easy to see that “f (x) g(x)” holds if and only if there exist positive constants c 1 and c 2 and a constant x 0 such that
(2.1) c 1 |g(x)| ≤ |f (x)| ≤ c 2 |g(x)| (x ≥ x 0 ).
As with the O-notation, dependence on parameters may be indicated by putting the parameters as subscripts to the “ ” or “ ” symbols. For example, the estimate (x + 1)A^ = OA (exp((log x)1+)), which we considered in Example 2.2, could have been written in the equivalent form
(x + 1)A^ A, exp
(log x)1+
The primary advantage of the Vinogradov notation over the O-notation is a typographical one: If the function g(x) is a complicated expression (for example, a sum of several integrals), then f (x) g(x) looks much cleaner than f (x) = O(g(x)) (which would require an oversized set of parentheses). In addition, the Vinogradov notation provides an easy way to express lower bounds by using the symbol “ ” instead of “ ”, and the “” symbol allows one to express two O-estimates in a single statement. The Vinogradov notation has the drawback that, unlike the O-notation, it does not extend to terms in arithmetic expressions. Thus, for example, while one can rewrite the estimate
π(x) −
x log x
= O
x log x)^2
in an equivalent manner as
π(x) =
x log x
1 + O
log x
only the first version can be stated using the Vinogradov “ ” notation:
π(x) − x log x
x (log x)^2
Thus, depending on the situation, one or the other of these two notations may be more convenient to use, and we will use both notations interchange- ably throughout this course, rather than settle on one particular type of notation.
18 CHAPTER 2. ASYMPTOTIC NOTATIONS
2.1.6 Other variants of the O-notation
Some other notations that are equivalent to or related to the O-notation and which are occasionally used are the following. All of these notations are non-standard and do not have a generally accepted meaning, so they should be avoided, or at least precisely defined before use.
- In some areas of analysis (especially harmonic analysis), the symbol “.” is used with the same meaning as “ ”.
- The symbol “≪” is sometimes used to indicate that one function is “of smaller order of magnitude” than another function, usually in the sense that the ratio between the two functions tends to 0 (i.e., the equivalent of the o-notation defined below). In their book “Concrete Mathematics”, Graham, Knuth, and Patashnik use the symbol “≺” in the same sense. However, neither of these notation is very widespread.
- In numerical applications the value of an O-constant is important. One notation that refines the O-notation by keeping track of constants is the θ-notation, which means the same as the O-notation with constant c = 1. For example, since | log(1 + z)| ≤
n=1 |z|
n/n ≤ |z|/(1 − |z|) ≤ 2 |z| for |z| ≤ 1 /2, we have, using the θ-notation, log(1 + z) = θ(2|z|) for |z| ≤ 1 /2.
- The symbol “≈” is sometimes used with the same meaning as . However, more commonly, this symbol is used in an informal manner (e.g., in heuristic arguments) to indicate that one quantity is “approx- imately” equal to another quantity.
2.1.7 The “small oh” notation and asymptotic equivalence
The notation f (x) = o(g(x)) (x → ∞)
means that g(x) 6 = 0 for sufficiently large x and limx→∞ f (x)/g(x) = 0. If this holds, we say that f (x) is of smaller order than g(x). This is equivalent to having an O-estimate f (x) = O(g(x)) with a constant c that can be chosen arbitrarily small (but positive) and a range x ≥ x 0 (c) depending on c. Thus, an o-estimate is stronger than the corresponding O-estimate. A closely related notation is that of asymptotic equivalence:
f (x) ∼ g(x) (x → ∞)
2.1. THE “OH” NOTATIONS 19
means that g(x) 6 = 0 for sufficiently large x and limx→∞ f (x)/g(x) = 1. If this holds, we say that f (x) is asymptotic (or “asymptotically equiva- lent”) to g(x) as x → ∞. Just as an o-estimate refines the O-estimate, the asymptotic equivalence relation f (x) ∼ g(x) refines the order of magnitude estimate f (x) g(x). By an asymptotic formula for a function f (x) we mean a relation of the form f (x) ∼ g(x), where g(x) is a “simple” function. In much the same way as the O-notation, the o-notation can be general- ized to functions for complex variables, and to more general limits: If f (s) and g(s) are functions of a real or complex variable s and s 0 is a real or complex number or infinity, we write
f (s) = o(g(s)) (s → s 0 ),
if the limit lims→s 0 f (s)/g(s) exists and is equal to 0, Asymptotic formulas with respect to the limit s → s 0 are defined analogously. It is important to keep in mind that the o-notation is always with respect to a given limiting process. If a limiting process is not explicitly given (in a form like “x → x 0 ”), the limit is usually understood to be taken as the variable tends to infinity. In the same way as we have done with the O-notation, we allow o-terms to appear inside arithmetic expressions: a term o(g(x)) stands for a function f (x) that satisfies limx→∞ f (x)/g(x) = 0 (but on which we have no further information). With this convention the asymptotic formula f (x) ∼ g(x) is easily seen to be equivalent to either of the relations
f (x) = g(x) + o(g(x))
or f (x) = g(x)(1 + o(1)). Another related notation that is used, for example, in number theory, is the Ω-notation. This notation simply means the opposite of “small oh”: Namely, we write f (x) = Ω(g(x)) (x → ∞),
if the relation f (x) = o(g(x)) is false, i.e., if lim supx→∞ |f (x)/g(x)| > 0. Analogous definitions apply for the case of more general functions or limits. For example, we have sin x = Ω(1) as x → ∞, and sin x = Ω(x) as x → 0. Note that the relation f (x) = Ω(g(x)) is not equivalent to f (x) f (x). Indeed, the latter means that |f (x)| > c|g(x)| holds, with some positive constant c, for all sufficiently large x, whereas f (x) = Ω(g(x)) only requires this inequality to hold for arbitrarily large values of x.
2.1. THE “OH” NOTATIONS 21
PNT with modest error term: A more precise version of the above form of the PNT shows that the relative error in the above asymptotic formula is of order O(1/ log x):
π(x) = x log x
1 + O
log x
(x ≥ 2).
This version, while far from the best-known version of the PNT, is sharp enough for many applications.
PNT with “classical” error term: To be able to state more precise versions of the PNT, the function x/ log x as approximation to π(x) is too crude; a better approximation is provided by the “logarithmic integral”,
Li(x) =
∫ (^) x
2
dt log t
(x ≥ 2).
With Li(x) as main term in the approximation to π(x), the relative error in the approximation can be shown to be much smaller than any negative power of log x. Indeed, the analytic method introduced by Hadamard and de la Vall´ee Poussin in their proof of the PNT yields the estimate
π(x) = Li(x)
1 + O
exp(−c
log x)
(x ≥ 3),
where c is a positive constant. This result, which is now more than 100 years old, can be considered the “classical” version of the PNT with error term.
PNT with Vinogradov-Korobov error term: The only significant im- provement in the error term for the PNT obtained during the past 100 years is due to I.M. Vinogradov and A. Korobov, who improved the above classical estimate to
π(x) = Li(x)
1 + O
exp(−(log x)^3 /^5 −
(x ≥ 3),
for any given > 0. The Vinogradov-Korobov result is some 50 years old, but it still represents essentially the sharpest known form of the PNT.
PNT with conjectured error term: A widely believed conjecture is that the “correct” relative error in the PNT should be about 1/
x. More precisely, the conjecture states that
π(x) = Li(x)
1 + O
x−^1 /2+
(x ≥ 3)
22 CHAPTER 2. ASYMPTOTIC NOTATIONS
holds for any given > 0. This conjecture is known to be equivalent to the Riemann Hypothesis. It is interesting to compare the size of the (relative) error term in this conjectured form of the PNT with that in the sharpest known form of the PNT, i.e., the Vinogradov-Korobov estimate cited above: To this end, note that
exp
−(log x)^3 /^5 −
≥ exp
−(log x)^3 /^5
x−^ (x ≥ 3)
for any > 0. Thus, while the conjectured form of the PNT involves a relative error of size Oα(x−α) for any fixed exponent α < 1 /2, our present knowledge does not even give such an estimate for some positive value of α.
Omega estimate: It is known that the relative error in the PNT cannot be of order O(x−α) with an exponent α > 1 /2. Using the “Omega” notation introduced above, this can be expressed as follows: For any α > 1 /2, we have π(x) − Li(x) = Ω
Li(x)x−α
(x → ∞).
2.2 Working with the “oh” notations
Recall that an O-term in an arithmetic expression or an equation represents a function that satisfies the inequality implicit in the definition of an O- estimate. With this convention, expressions involving several O-terms have a well-defined meaning. However, we have to be careful when working with such terms as these are not ordinary arithmetic expressions and cannot be manipulated in the same way. Fortunately, most arithmetic operations are permissible with O-terms.
2.2.1 Rules for “big oh” and “small oh” estimates
We now list some basic rules for manipulating O-terms. For simplicity, we state these only for functions of a real variable x and do not explicitly indicate the range (which thus, by our convention, is of the form x ≥ x 0 ). However, the same rules hold in the more general context of functions of a complex variable s and O-estimates valid in a general range s ∈ S.
- Constants in O-terms: If C is a positive constant, then the estimate f (x) = O(Cg(x)) is equivalent to f (x) = O(g(x)). In particular, the estimate f (x) = O(C) is equivalent to f (x) = O(1).
24 CHAPTER 2. ASYMPTOTIC NOTATIONS
Hence (^) ∫ (^) x
x 0
f (y)dy = O
(∫ (^) x
x 0
|g(y)|dy
(x ≥ x 0 ),
as desired.
Rules for o-estimates. Some, but not all, of the above rules for O- estimates carry over to o-estimates. For example, the first four rules also hold for o-estimates. On the other hand, this is not the case for the last two rules. For instance, if f (x) = e−x^ and g(x) = 1/x^2 , then f (x) = o(g(x)) as x → ∞. On the other hand, the integrals F (x) =
∫ (^) x 1 f^ (x)dy^ and G(x) =
∫ (^) x 1 g(y) are equal to^ e
− (^1) − e−x (^) and 1 − 1 /x, respectively, and satisfy
limx→∞ F (x)/G(x) = e−^1 , so the relation F (x) = o(G(x)) does not hold. This example illustrates the difficulties and pitfalls that one may encounter when trying to manipulate o-terms. To avoid these problems, it is advisable to work with O-estimates rather than o-estimates, whenever possible.
2.2.2 Equations involving O-terms
In all examples we considered so far, all O-terms occurred on the right-hand side of the equation. It is useful to further extend the usage of the O-notation by allowing equations in which O-terms arise on both sides, provided one takes care in properly interpreting such an equation. In particular, equa- tions in which there are O-terms on both sides are not symmetric and should be read left to right. For example, the relation
O(
x) = O(x) (x ≥ 1),
is to be understood in the sense that any function f (x) satisfying f (x) = O(
x) for x ≥ 1 also satisfies f (x) = O(x) for x ≥ 1, a statement that is obviously true. On the other hand, if we interchange the left- and right-hand sides of the above equation, we get
O(x) = O(
x) (x ≥ 1),
which, when interpreted in the same way (i.e., read left to right), is patently false. For similarly obvious reasons, O-terms in equations cannot be cancelled; after all, each O-term stands for a function satisfying the appropriate O- estimate, and multiple instances of the same O-term (say, multiple terms O(x)) in general it will represent different functions. For example, from f (x) = log x + O(1/x) and g(x) = log x + O(1/x) we can only conclude that
2.2. WORKING WITH THE “OH” NOTATIONS 25
f (x) = g(x) + O(1/x), i.e., that f (x) and g(x) differ (at most) by a term of order O(1/x), but not that f (x) and g(x) are equal.
2.2.3 Simplifying O-expressions
The following are some transformation rules which often allow one to dra- matically simplify messy expressions involving O-terms.
1 + O(φ(x))
= 1 + O(φ(x)),
(1 + O(φ(x)))p^ = 1 + Op(φ(x)), log(1 + O(φ(x))) = O(φ(x)), exp(O(φ(x))) = 1 + O(φ(x)).
Table 2.3: Some common transformations of O-expressions, valid when φ(x) → 0. Here p is any real or complex parameter.
These relations are to be interpreted from left to right as described in the preceding subsection. For example, the first estimate means that any function f (x) satisfying f (x) = 1/(1 + O(φ(x))) also satisfies f (x) = 1 + O(φ(x)). The above relations follow immediately from the following basic O- estimates, which are easily proved (e.g., via the first-order Taylor formula):
1 + z = 1 + O(|z|),
(1 + z)p^ = 1 + Op(|z|), log(1 + z) = O(|z|), ez^ = 1 + O(|z|),
Table 2.4: Some basic O-estimates, valid for z → 0, i.e., with a range |z| ≤ δ, for a suitable constant δ > 0. Here p is any real or complex parameter.
2.2.4 Some asymptotic tricks
Factoring out dominant terms. A simple, but very effective technique in asymptotic analysis is to identify a dominant term in an estimate and
2.2. WORKING WITH THE “OH” NOTATIONS 27
Taking logarithms. Another sometimes very useful technique in asymp- totic analysis is to take logarithms in order to transform products to sums and exponentials to products.
Example 2.15. Consider the function
f (x) = (log x + log log x)^1 /
√log log x .
This is a rather fierce looking function, and its behavior as x → ∞ (for example, the question whether it is bounded) is anything but obvious. Taking logarithms, we can answer such questions. We have
log f (x) = log(log x + log log x) √ log log x
and we recognize the numerator as the expression estimated in the above example. Using the notation and result of this example, we get
log f (x) =
L 2
L 2 +
L 2
L
+ O
L^22
L^2
L 2 +
L 2
L
+ O
L^32 /^2
L^2
To get back to f (x), we exponentiate, using the estimate ez^ = 1 + O(|z|), valid for |z| ≤ 1, say. Thus,
f (x) = exp
log log x +
log log x log x
1 + O
(log log x)^3 /^2 (log x)^2
In particular, we now see that f (x) tends to infinity as x → ∞.
Swapping main and error terms in convergent series and integrals. A common problem in asymptotic analysis is that of estimating partial sums S(x) =
n≤x an^ of an infinite series^
n=1 an.^ While the sums^ S(x) can rarely be evaluated in closed form, it is usually easy to get estimates for the summands of the form an = O(φ(n)). Applying such an estimate directly to the summands in S(x) would lead to an error term of size O(
n≤x |φ(n)|), which is at best∑ O(1) (unless φ(n) = 0 for all n). However, if the series ∞ n=1 |φ(n)|^ (and hence also^
n=1) converges, we can use the following trick to obtain an estimate for S(x) with error term tending to zero as x → ∞. Namely, we extend the range of summation in S(x) =
n≤x an^ to infinity
28 CHAPTER 2. ASYMPTOTIC NOTATIONS
and write S(x) = S − R(x), where S =
n=1 an^ and^ R(x) =^
n>x an. Applying now the estimate an = O(φ(n)) to the tails R(x) of the series then leads to an estimate with error term O(
n>x |φ(n)|).^ The convergence of the series
n=1 |φ(n)|^ implies that this error term tends to zero, and usually it is easy to obtain more precise estimates for this error term.
Example 2.16. Consider the sum
S(x) =
≤x
n − log
n
n≤x
an.
The terms in this series satisfy an = O(1/n^2 ) for all n, since x − x^2 / 2 ≤ log(1 + x) ≤ x for 0 ≤ x ≤ 1 (which can be seen, for example, from the fact that log(1 + x) = x − x^2 /2 + x^3 / 3 −... is an alternating series with decreasing terms). Substituting this estimate directly into the terms in S(x) would only give the estimate
S(x) = O
n≤x
n^2
= O(1).
However, the trick of extending the summation to infinity leads to an esti- mate with error term O(1/x),
S(x) = S + O
n>x
n^2
= S + O
x
where S =
n=1(1/n^ −^ log(1 + 1/n)) is some (finite) constant. Note that the method does not give a value for this constant. This is an intrinsic limitation of the method, but in most cases the series simply do not have an evaluation in “closed form” and trying to find such a evaluation would be futile. One can, of course, estimate this constant numerically by computing the partial sums of the series.
Extending the range of an O-estimate. According to our convention, an asymptotic estimate for a function of x without an explicitly given range is understood to hold for x ≥ x 0 for a suitable x 0. This is convenient as many estimates (e.g., log log x = O(
log x)), do not hold, or do not make sense, for small values of x, and the convention allows one to just ignore those issues. However, there are applications in which it is desirable to have an estimate involving a simple explicit range for x, such as x ≥ 1, instead
