Download Reductions and Complexity: Lecture Notes on Algorithms and Data Structures and more Slides Computer Science in PDF only on Docsity!
ReductionsSome of these lecture slides are adaptedfrom CLRS Chapter 31.5 and Kozen Chapter 30.
Contents
Contents.^ n
"Linear-time reductions." n^
Undirected and directed shortest path. n^
Matrix inversion and multiplication. n^
Integer division and multiplication. n^
Sorting and convex hull.
3
Reduction
Intuitively, decision problem X reduces to problem Y if:^ n
Any instance of X can be "rephrased" as an instance of Y. n^
The solution to instance of Y provides solution to instance of X. Consequences:^ n
Used to establish relative difficulty between two problems. n^
Given algorithm for Y, we can also solve X. (design algorithms) n^
If X is hard, then so is Y. (prove intractability)
Reduction
Problem X linearly reduces to problem Y if, given a black box thatsolves Y in O(f(N)) time, we can devise an O(f(N)) algorithm for X.Ex 1. X = PRIME linearly reduces to Y = COMPOSITE.^ n
PRIME(x): Is x prime? n^
COMPOSITE(x): Is x composite? n^
To compute PRIME(x), call COMPOSITE(x) and return oppositeanswer.
Docsity.com
5
Reduction: Undirected to Directed Shortest PathEx 2. Undirected shortest path (with nonnegative weights) linearlyreduces to directed shortest path.^ n
Replace each directed arc by two undirected arcs. n^
Shortest directed path will use each arc at most once.
s
2 3
5 6
t
10 5
(^91512)
10 12
15
4
s
2 3
5 6
t
10 5
(^91512)
(^1012)
4
9
10
4
15 12
12
10
15 15
Reduction: Undirected to Directed Shortest PathEx 2. Undirected shortest path (with nonnegative weights) linearlyreduces to directed shortest path.^ n
Replace each directed arc by two undirected arcs. n^
Shortest directed path will use each arc at most once. n^
Note: reduction invalid in networks with negative cost arcs, even ifno negative cycles.
t
2
s^
7
t
2
s^
7
7
7
Network Flow Running Times and Linear Time Reductions
undirected shortest pathnonnegative weights
O(m)
shortest path nonnegative weights
O(m + n log n)
undirected shortest path
no negative cyclesO(mn + n
2 log n)
shortest path no negative cycles
O(mn)
assignment
(weighted bipartite matching)
O(mn + n
2 log n)
weighted non-bipartite matchingO(mn + n
2 log n)
directed MSTO(m + n log n)
MST undirected O(m
α(m,n) log
α(m,n))
non-bipartitematching O(mn
1/2 )
bipartite matching
O(mn
1/2 )
max flow bipartite DAGO(mn log(m/ n
2 ))
max flow O(mn log(m/ n
2 ))
min cut O(mn log(m/ n
2 ))
max flowundirected
min cutundirected
min cost flow O(m
2 log n + mn log
2 n)
transportation O(m
2 log n + mn log
2 n)
min vertex cover
bipartiteO(mn
1/2 )
Matrix Inversion
Fundamental problem in numerical analysis.^ n
Intimately tied to solving system of linear equations. n^
Note: avoid explicitly taking inverses in practice.
1 2 3
x x x
x
b
A
3
2
1
3
2
1
3
2
1
3
2
1
x
x
x
x
x
x
x
x
x
x
x
x
=^
−
−
1
1
b
A
x
A
Docsity.com
13
Integer Arithmetic
Fundamental questions.^ n
Is integer addition easier than integer multiplication? n^
Is integer multiplication easier than integer division? n^
Is integer division easier than integer multiplication?
OperationAddition
O(N)
Upper Bound
(N)
Lower Bound
Multiplication
O(N log N log log N)
(N)
Division
O(N log N log log N)
(N)
Warmup: Squaring vs. Multiplication
Integer multiplication: given two N-digit integer s and t, compute st.Integer squaring: given an N-digit integer s, compute s
Theorem. Integer squaring and integer multiplication have the sameasymptotic complexity.Proof.^ n
Squaring linearly reduces to multiplication.^ –
trivial: multiply s and s n^
Multiplication linearly reduces to squaring.^ –
regularity assumption: S(N+1) = O(S(N))
((^
2 2 2
12
t s t s
st
15
Integer Division (See Kozen, Chapter 30)
Integer division: given two integers s and t of at most N digits each,compute the quotient q and remainder r:^ n
q =
s / t
^ , r = s mod t.
n^
Alternatively, s = qt +r, 0
r < t.
Example.^ n
s = 1000, t = 110
q = 9, r = 10.
n^
s = 4905648605986590685, t = 100
r = 85.
We show integer division linearly reduces to integer multiplication.
Integer Division: "Grade-School"
Divide two integers, eachis N bits or less.^ n
q =
s / t
n^
r = s mod t. Running time. O(N
n^
O(N) per iteration + recursive calls. n^
Denominator increases by factor of 2 each iteration.^ –
s < 2
N^ and does not change
-^
≤^
t^ ≤
s throughout
⇒
O(N) recursive calls
IF (s < t)
RETURN (0, t) (q’, r’)
IntegerDivision(s, 2t)
IF (r’ < t)
RETURN(2q’, r’) ELSE
RETURN (2q’ + 1, r’ - t)
(q, r) = IntegerDivision (s, t)
Docsity.com
17
Integer Division: "Grade-School"
The algorithm correctly compute q =
s / t
^ , r = s mod t.
Proof by reverse induction.^ n
Base case: t > s. n^
Inductive step: algorithm computes q’, r’ such that^ –
q’ =
s / 2t
^ , r’ = s mod 2t.
-^ s = q’ (2t) + r’, 0
r’ < 2t.
n^
Goal: show
otherwise 1
2
if
2 q
t r
q
s t
^
^
^
r t
q
t
r t q
s t
Newton’s Method
Given a differentiable function f(x), find a value x* such that f(x*) = 0.Newton’s method.^ n
Start with initial guess x
n^
Compute a sequence of approximations: n^
Equivalent to finding line of tangent to curve y = f(x) at x
andi^
taking x
i+
to be point where line crosses x-axis.
1
i^ i
i
i^
x
f
x
f
x
x^
xi
xi+
19
Newton’s Method
Convergence of Newton’s method.^ n
Not guaranteed to converge to a root x*. n^
If function is well-behaved, and x
0 sufficiently close to x* then
Newton’s method converges quadratically.^ –
number of bits of accuracy doubles at each iteration
Applications.^ n
Computing square roots: n^
Finding min / max of function.
Extends to multivariate case.
n^
Cornerstone problem in continuous optimization. n^
Interior point methods for linear programming.
(^12)
1
2
txi
i
i
x
x
x t
x f
Integer Division: Newton’s Method
Our application of Newton’s method.^ n
We will use exact binary arithmetic and obtain exact solution. n^
Approximately compute x = 1 / t using Newton’s method. n^
We’ll show exact answer is either
s x
^
or
s x
Theorem: given a O(M(N)) algorithm for multiplying two N-digitintegers, there exists an O(M(N)) algorithm for dividing two integers,each of which is at most N-digits.
2
1
i
i
i^
tx
x
x
x
t
x
f
Docsity.com
25
Analysis
L3: Algorithm terminates after O(log N) steps.^ n
By L2, after k =
log
log 2
(s / t) 2
steps, we have:
Note: 2
k^ = O(N), k = O(log N).
L4: Algorithm returns correct answer.^ n
By L1, x
k^
≤^
1 / t.
n^
Combining with proof of L3: n^
This implies,
s / t
^
is either
s x
k or
s x
k
the remainder can be found by subtraction.
≤^
k
sx
s t
2
t s
tx
k
k^
Analysis
Theorem: Newton’s method does integer division in O(M(N)) time,where M(N) is the time to do multiply two N-digit integers.^ n
By L3, 2
k^ = O(N), and the number of iterations is O(log N).
n^
Each Newton iteration involves two multiplications, one addition,and one subtraction. n^
Technical fact (not proved here): algorithm still works if we onlykeep track of 2
i^ significant digits in iteration i.
Bottleneck operation = multiplications. 3
2M(1) + 2M(2) + 2M(4) +... + 2M(
k) = O(M(N)).
2
1
i
i
i^
tx
x
x
x
t
x
f
27
Integer Arithmetic
Theorem: The following integer operations have the same asymptoticbit complexity.^ n
Multiplication. n^
Squaring. n^
Division. n^
Reciprocal: N-significant bit approximation of 1/s.
^
− N^ s
1 (^22)
Sorting and Convex Hull
Sorting.^ n
Given N distinct integers, rearrange in increasing order. Convex hull.^ n
Given N points in the plane, find their convex hull in counter-clockwise order.
Find shortest fence enclosing N points.
Docsity.com
29
Sorting and Convex Hull
Sorting.^ n
Given N distinct integers, rearrange in increasing order. Convex hull.^ n
Given N points in the plane, find their convex hull in counter-clockwise order. Lower bounds.^ n
Recall, under comparison-based model of computation, sorting Nitems requires
(N log N) comparisons.
n^
We show sorting linearly reduces to convex hull. n^
Hence, finding convex hull of N points requires
(N log N)
comparisons.
Sorting Reduces to Convex Hull
Sorting instance:Convex hull instance.Key observation.^ n
Region {x : x
2
≥^
x} is convex
all points are on hull. n^
Counter-clockwise order ofconvex hull (starting at pointwith most negative x) yieldsitems in sorted order.
(^
2
22 2
(^21) 1
N N^
x x
x x
x x^
u
x^ N
x x^
,^
2 1
u
f(x) = x
(^2) ) 2 , (
xi xi^2 ) , (
xj xj
Docsity.com
