Download Algorithmic Correctness, Lecture Slides - Computer Science and more Slides Introduction to Computers in PDF only on Docsity!
COMS21103: Summary of Lecture 7
Algorithm correctness
Clearly, correctness is an issue of primary concern when designing algorithms. An efficient algorithm that is not correct is pretty much useless. We did not go much into details of what correctness is, since there was clear consensus as to the intuitive meaning. Roughly, an algorithm is correct if when given a certain type of input, the output of the algorithm satisfies some desired relation with the input. For example, a sorting algorithm takes as input an array A of n elements and outputs an array B. The desired relation is that B is the sorted version of A. We can specify this relation formally as follows:
((∃f ∈ Bij(n)) : A[i] = B[f (i)]) ∧ ((∀i ∈ { 1 , 2 ,... , n − 1 }) : B[i] ≤ B[i + 1])
where Bij(n) represents the set of all bijections on the set { 1 , 2 ,... , n}. The formula above may look somewhat complex, but it really says something simple. The first part says that all of the elements of A appear in B (more precisely, the element on position i in A appears on position f (i) in B), and the second part of the formula says that the elements of B are in sorted order. In principle, one could specify any relation between the input and the desired output of al- gorithms. It is important however to understand that to prove correctness of an algorithm fully formally precise logical formulations like above cannot be avoided.
Elements of the Floyd-Hoare logic
There has been a huge amount of work on rigorous ways of proving the correctness of algorithms. The Floyd-Hoare logic is one influential example of such work which we have briefly discussed in class. Here is the main idea behind this approach. Consider a generic algorithm (which is nothing but a sequence of instructions):
instruction 1 instruction 2 instruction 3 .............. instruction n
Under this approach the correctness of the algorithm is expressed in the following terms: if the input satisfies some property P then the output satisfies some property Q. As an example consider a “swapping” algorithm. Such an algorithm should take as input two variables X and Y and swap
their value. That is, we would want that if before the execution of the algorithm it holds that X = a and Y = b (for some values a and b), then after the execution of the algorithm X = b and Y = a. To prove the correctness of an algorithm one annotates the algorithm with “assertions”: think of these assertions as statements about the state of the memory of the algorithm.
{Assertion 1} instruction 1 {Assertion 2} instruction 2 {Assertion 3} instruction 3 {Assertion 4} .............. {Assertion n} instruction n {Assertion n+1}
These assertions should be such that for any i, if Assertion i is true before instruction instruction i is executed, then after this instruction is executed, Assertion i + 1 is true. Finally, we would like that any valid input for the algorithm satisfies Assertion 1, and Assertion n+1 implies the desired property on the output. How should one come up with these assertions? How would you know which assertions are ok to use (in that they do satisfy the property above)? The Floyd-Hoare logic comes with axioms that explain how to construct/use assertions for each instruction in the language (notice that we also need to know the precise semantics for the instructions). We did not introduce the logic formally, and we didn’t speak about these instructions. We only sketched one axiom for the assignment instruction. Consider variables X 1 ,X 2 ,...,Xn and consider the assignment instruction:
X 1 ← Exp(X 1 , X 2 , ..., Xn),
where Exp is a mathematical expression over variables X 1 ,X 2 ,...,Xn. For example
Exp(X 1 , X 2 , ..., Xn) = X 1 + X 2 + ... + Xn.
If {X 1 =a 1 ,X 2 =a 2 ,...,Xn=an} is an assertion that holds before the above instruction is exe- cuted, then assertion {X 1 =Exp(a 1 ,a 2 ,...,an), X 2 =a 2 ,...,Xn=an } holds after the execution of the instruction. This property follows from the semantics of the assignment instruction. The above axiom is sufficient for proving correctness of programs that only use assignment instructions. We have done so for two algorithms for swapping the value of two variables. The algorithms, together with the corresponding assertions are bellow. The first algorithm swaps the values of variables X and Y, using a third variable Z and is the following:
To prove that the algorithm is correct we use the following loop invariant: variable S holds the sum A[1] + A[2] +... + A[i − 1]. We need to show that this invariant satisfies the desired properties.
- Initialization Before the loop starts being executed we have that S = 0 and i = 1. Clearly S = A[1] +... + A[0] (this is the empty sum).
- Maintenance We need to show that if the property is true before some loop iteration then the property is true right before the next iteration. Here one difficulty that we typically encounter is that we may mix the variables i and S with their values. To avoid this, lets assume that the value of before going into the loop is i = i 0 , and thus that the value S 0 of S is S 0 = A[1] +... + A[i 0 ]. When executing the loop once, the value of S becomes S 0 + A[i 0 ] and the value of i becomes i 0 + 1. It is easy to see that the loop invariant is still satisfied.
- Termination The loop terminates when the condition i ≤ n is not satisfied, i.e. i = n + 1. In this case we therefore have that S = A[1] +... + A[n + 1 − 1], as desired.
In class we have also looked at the correctness of a simple merge procedure. The procedure takes as input two sorted arrays A and B of size n, respectively m and produces an array C that contains all of the elements of A and B in sorted order. For simplicity we have assumed that A[n + 1] = B[m + 1] = ∞ (as in the merge procedure in [CLRS]). The algorithm is the following:
i ← 1; j ← 1; k ← 1 while i ≤ n or j ≤ m do if A[i] ≤ B[j] then C[k] ← A[i]; i ← i + 1 else C[k] ← B[j]; j ← j + 1; k ← k + 1
To prove the correctness of the above algorithm we consider the following properties which we’ll show form a loop invariant:
a) k = i + j − 1
b) the entries C[1], C[2]... , C[k−1] hold the entries A[1],... , A[i−1],B[1],... , B[j −1] in sorted order.
c) A[i] and B[j] are greater than C[1], C[2]... , C[k − 1]
d) i ≤ n + 1 and j ≤ m + 1.
The properties we are after are the following:
- Initialization. Before the loop starts being executed i = j = k = 1 and a) clearly holds. The requirements b), c), and d) also trivially hold.
- Maintenance. Assume that the properties are true before one iteration of the loop. Let i 0 , j 0 , k 0 be the values for the variables i, j, k respectively before the iteration^1. Maintenance (^1) It is often quite helpful to explicitly consider the (abstract) values that the variables have before one loop iteraction so that it’s easier to distinguish between the variable and its value.
requires to show that if properties a)-d) hold for these particular values, meaning that we assume that: k 0 = i 0 + j 0 − 1 (3) C[1] ≤ C[2] ≤ C[k 0 − 1] (4)
then they hold for the values of the variable after one iteration. a) is clearly still satisfied as k gets increased in the loop by 1 to k 0 + 1, and precisely one of the variables i and j gets increased. So either i will be i 0 + 1 and j remains unchanged, or the other way around. In any case the relation k = i + j − 1 is preserved. b) let’s consider the case when A[i 0 ] ≤ B[j 0 ] (the case when the oposite is true is similar). Then, after the loop is executed C[1],... , C[k 0 − 1] are unchanged (so they hold the values of A[1],... , A[i 0 − 1], B[1],... , B[j 0 − 1] in sorted order, C[k 0 ] is A[i 0 ], i is i 0 + 1, j = j 0 and k = k 0 + 1. Then b) is still true since C[k 0 ] = A[i 0 ] is greater than C[1],... , C[k 0 − 1]. Also, since arrays A and B are ordered, we have that A[i 0 + 1] ≥ A[i 0 ] ≥ C[1]... , C[k 0 ], so c) is also satisfied. Finally, notice that if i 0 ≤ n + 1 and j 0 ≤ m + 1 then we have the folowing cases: 1) i 0 ≤ n and j 0 ≤ m from which i 0 + 1 ≤ n + 1 and j 0 ≤ m + 1, 2) i 0 = n + 1 and j 0 ≤ m and in this case j gets increased so i 0 = n + 1 and j 0 ≤ m + 1 or 3) i 0 ≤ n and j 0 = m + 1 which is treated similarly.
- Termination. The loop terminates when i ≥ n and j ≥ m. From condition d) of the loop invariant we know that i ≤ n + 1 and j ≤ m + 1 so when the loop terminates it must be the case that i = n + 1 and j = m + 1. We have that k is n + m + 1 (from condition a) of the invariant). Condition b) of the invariant then says that C[1],... , C[n + m] hold A[1],... , A[n], B[1],... , B[m] in sorted order. We’re done.
Further reading
- Read the part on loop invariants for insertion sort from page 17 and for merge sort on page 30 of [CLRS]
