Linear Algebra for Sciences, Skripte von Lineare Algebra

Nur auf Docsity: Lade Linear Algebra for Sciences und mehr Skripte als PDF für Lineare Algebra herunter!

Linear Algebra for the Sciences

Thomas Kappeler, Riccardo Montalto

4 CONTENTS

Chapter 1

Systems of linear equations

At the core of linear algebra is the solving of systems of linear equations. A linear system

(S) with n unknowns and m equations has the form

a 11 x 1

• a 12 x 2
• · · · + a 1 n x n = b 1 (Eq 1

a 21 x 1 + a 22 x 2 + · · · + a 2 nxn = b 2 (Eq 2 )

a m 1 x 1

• a m 2 x 2
• · · · + a mn x n = b m (Eq m

The numbers a 11 ,... , a 1 n , a 21 ,... , a 2 n ,.. ., a m 1 ,... , a mn are called the coecients of the

system. The system is said to be real if all coecients a ij , i = 1,... , n, j = 1,... , m

as well as the numbers b 1 ,... , bm are real. In this chapter we will only consider real

systems of linear equations, but the techniques we develop to solve them also apply to

complex systems, i.e., systems where aij and bi are complex numbers which will only be

introduced in a later chapter. Given the real system (S), a solution of (S) is a set of n

real numbers x 1 ,... , x n so that equations (Eq 1 ) (Eq m ) are satisfied simultaneously. The

basic questions with regard to linear systems of the form (S) are the following ones:

(1) Does (S) have a solution? (Existence)

(2) Does (S) have at most 1 solution? (Uniqueness)

Or formulated in more general terms:

(3) What are the properties of the set of solutions

L :=

n

(x 1 ,... , xn) : x 1 2 R,... , xn 2 R; (x 1 ,... , xn) satisfies (S 1 ) (Sm)

o

Here and in the sequel, R denotes the set of real numbers. Note that in this terminology

questions (1) and (2) can be reformulated as follows:

(1’) Is L a nonempty set?

(2’) Does L have at most one element?

In practical applications, systems of linear equations can be very large. One therefore

needs theoretical concepts and numerical algorithms to investigate respectively solve such

systems.

1.1. LINEAR SYSTEMS WITH TWO EQUATIONS AND TWO UNKNOWNS 7

Now let us go back to the linear system of two equations with two unknowns,

ax + by = e (1.1.3)

cx + dy = f (1.1.4)

The set of solutions L of (1.1.3), (1.1.4) is then given by the intersection of the set of

solutions of (1.1.3) with the set of solutions of (1.1.3), L = L 1

\ L

2 , where

L

1

n

(x, y) 2 R

2 : ax + by = e

o

, L

2

n

(x, y) 2 R

2 : cx + dy = f

o

In case (a, b) 6 = (0, 0) and (c, d) 6 = (0, 0), L 1 and L 2 are lines in R

2 and L is the intersection

of them. Thus L can be a one point set (L 1 and L 2 are not parallel), or a line (L 1

= L

2

or the emptyset (L 1 and L 2 are parallel but do not coincide).

Let us now describe an algorithm how to determine the set of solutions of (1.1.3), (1.1.4)

in a systematic way. You know this algorithm already from high school. To simplify the

algorithm we assume that

a 6 = 0. (1.1.5)

Step 1: eliminate x from equation (1.1.4) by replacing (1.1.4) by (1.1.4)

c

a

(1.1.3) It

means that the left hand side of (1.1.4) is replaced by

cx + dy

c

a

ax + by

(use that a 6 = 0!)

whereas the right hand side of (1.1.4) is replaced by

f

c

a

e.

The new system of equations then reads as follows

ax + by = e (1.1.6)

d

c

a

b

y = f

c

a

e. (1.1.7)

It is straightforward to see that under the assumption (1.1.5), the set solutions of (1.1.3),

(1.1.4) coincides with the set of solutions of (1.1.6), (1.1.7). In such a case we say that

the two systems are equivalent.

Step 2: The system (1.1.6), (1.1.7) is solved by first solving (1.1.7) for y and then use

(1.1.6) to determine x:

Case d

c

a

b 6 = 0: then (1.1.7) has the unique solution

y =

f

c

a

e

d

c

a

b

af ce

ad bc

and when substituted into (1.1.6) one obtains

ax = e b

af ce

ad bc

8 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

or

x =

de bf

ad bc

Hence the set of solutions L consists of one element

L =

n⇣ de bf

ad bc

af ce

ad bc

⌘o

Case d

c

a

b = 0 , f

c

a

e 6 = 0: equation (1.1.7) has no solutions and hence L = ;.

Case d

c

a

b = 0 , f

c

a

e = 0: then any y 2 R is a solution of (1.1.7) and solutions of

(1.1.6) are given by x =

e

a

b

a

y. Hence the set of solutions L is given by

L =

n⇣ e

a

b

a

y, y

: y 2 R

o

Motivated by formula (1.1.8) for the solutions of (1.1.6), (1.1.7) in the case where d

c

a

b 6 =

0 we make the following definitions:

Definition 1.1.1. (i) A real 2 ⇥ 2 matrix (plural: matrices) is an array A of real numbers

of the form

A :=

a b

c d

, a, b, c, d 2 R ;

(ii) the determinant of a 2 ⇥ 2 matrix A is defined as

det(A) := ad bc.

The notion of the determinant can be used to characterize the solvability of the system

(1.1.6), (1.1.7) and to obtain formulas for its solutions. We state without proof the

following

Theorem 1.1.1. (i) The system of linear equations (1.1.6), (1.1.7) has a unique solution

if and only if

det

a b

c d

(ii) If det

a b

c d

6 = 0, then the unique solution of (1.1.6), (1.1.7) is given by the following

formulas (Cramer’s rule)

x =

det

e b

f d

det

a b

c d

◆ (^) , y =

det

a e

c f

det

a b

c d

10 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

Definition 1.2.1. We say that two systems of linear equations

n X

j=

a ij x j = b i for any 1  i  m

and

n X

j=

c kj x j = d k for any 1  k  p

are equivalent if their sets of solutions coincide.

An example of two equivalent systems with three unknowns is the following one:

4 x 1

• 3x 2
• 2x 3

x 1

• x 2
• x 3

and 8

4 x 1

• 3x 2
• 2x 3

x 1

• x 2
• x 3

2 x 1

• 2x 2
• 2x 3

since the latter equation is obtained from the equation x 1

• x 2
• x 3 = 4 by multiplying

left and right hand side by the factor 2.

The idea of Gaussian elimination is to replace a given system of linear equations in a

systematic way by an equivalent one which is easy to solve. In Subsection 1.1 we have

demonstrated this method in the case of two equations (m = 2) and two unknowns

(n = 2). Gaussian elimination uses the following basic operations, referred to as row

operations, which leave the set of solutions of a given system of linear equations invariant:

(R1) Exchange of two equations (rows) of a system of linear equations.

Example:

5 x 2 + 15x 3 = 10

4 x 1

• 3x 2
• x 3

4 x 1 + 3x 2 + x 3 = 1

5 x 2

• 15x 3

It means that the equations get listed in a di↵erent order.

(R2) Multiplication of an equation (row) by a real number ↵ 6 = 0.

Example:

4 x 1

• 3x 2
• x 3

5 x 2 + 15x 3 = 10

4 x 1

• 3x 2
• x 3

x 2 + 3x 3 = 2

We have multiplied the left and right hand side of the second equation by the factor

1.2. GAUSSIAN ELIMINATION 11

(R3) An equation (row) gets replaced by the equation obtained by adding to it the mul-

tiple of another equation. More formally, this can be expressed as follows: the kth

equation

P

n

j=

akj xj = bk is replaced by the equation

n X

j=

akj xj + ↵

n X

j=

aj xj = bk + ↵b

for some where 1   m with 6 = k – or more explicitly,

(a k 1

• ↵a ` 1 )x 1
• · · · + (a kn
• ↵a `n )x n = b k
• ↵b `

Example:

x 1 + x 2 = 5 (Eq1)

4 x 1

• 2x 2 = 3 (Eq2)

(Eq2) (Eq2)4(Eq1)

x 1 + x 2 = 5

2 x 2

It is not dicult to verify that these basic row operations lead to equivalent linear systems.

We state without proof the following

Theorem 1.2.1. The basic row operations lead to equivalent systems of linear equations.

We now show how these basic row operations can be used to solve a system (S) of linear

equations of the form

a 11 x 1

• a 12 x 2
• · · · + a 1 n x n = b 1

a 21 x 1

• a 22 x 2
• · · · + a 2 n x n = b 2

a m 1 x 1

• a m 2 x 2
• · · · + a mn x n = b m

where a ij (1  i  m , 1  j  n) and b i (1  i  m) are real numbers. To (S) we

associate its coecient matrix

A =

B

a 11 · · · a 1 n

a m 1 · · · a mn

C

A.

A is an array of real numbers with m rows and n columns. Such an array of real numbers

is called an m ⇥ n matrix and written in a compact form as

A = (a ij ) 1 im

1 jn

The augmented coecient matrix of (S) is the following array of real numbers

B

a 11 · · · a 1 n b 1

a m 1 · · · a mn b m

C

A

written in a compact form as (A | b).

1.2. GAUSSIAN ELIMINATION 13

2 x 1

• x 2
• x 3

3 x 3

is in row echelon form. Solve the second equation for x 3 and then the first equation

for x 1

3 x 3 = 6 x 3 = 2 ; x 2 is a free variable;

2 x 1 = 2 x 2 x 3 x 1

x 2

and hence

L =

n

x 2 , x 2 , 2) : x 2

2 R

o

which is a straight line in R

3 trough (0, 0 , 2) in direction ( 1 , 2 , 0).

(3) The augmented coecient matrix

A

is in row echelon form. The corre-

sponding system of linear equations can be solved as follows: since

0 · x 1

• 0 · x 2

has no solutions, one concludes that L = ;.

(4) The augmented coecient matrix

A

is in row echelon form. The corre-

sponding system of linear equations can be solved as follows:

0 · x 1

• 0 · x 2

is satisfied for any x 1 , x 2

2 R;

3 x 2 = 6 x 2 = 2

2 x 1 = 1 x 2 x 1

Hence L = {( 1 , 2)}.

(5) The augmented coecient matrix

is in row echelon form.

x 3 = 6 , x 2 = 1 2 x 3 = 11 , x 1 free variable ,

hence

L =

n

(x 1 , 11 , 6) : x 1

2 R

o

which is a straight line in R

3 trough the point (0, 11 , 6) in direction (1, 0 , 0).

14 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

(6) The augmented coecient matrix

is in row echelon form.

3 x 4 = 6 x 4 = 2 ; x 3 and x 4 are a free variables ;

x 1 = 2 x 2 x 4 = 2 x 2

hence

L =

n

( 2 x 2 2 , x 2 , x 3 , 2) : x 2 , x 3 2 R

o

which is a plane in R

4 containing the point ( 2 , 0 , 0 , 2) and spanned by the vectors

(0, 0 , 1 , 0) and ( 2 , 1 , 0 , 0).

(7) The augmented coecient matrix

is in row echelon form.

x 4 is a free variable ; 3 x 3 = 6 x 3

x 1 = 2 x 2 x 3 = 2 x 2 2.

Hence

L =

n

( 2 x 2 2 , x 2 , 2 , x 4 ) : x 2 , x 4

2 R

o

which is a plane in R

4 containing the point ( 2 , 0 , 2 , 0) and spanned by the vectors

(0, 0 , 0 , 1) and ( 2 , 1 , 0 , 0).

Gaussian elimination is a method of transforming a given augmented coecient matrix

with the help of basic row operations into row echelon form. How can this be achieved?

We explain the procedure with a few examples. It is convenient to introduce for the three

basic row operations the following notations:

• R

i$k : exchange rows i and k

• R

k

↵R

k : replace k-th row R k by ↵R k

• R

k

R

k

+ ↵R

: replace k-th row by adding to it ↵R where ` 6 = k and ↵ 2 R.

Examples

(1) The augmented coecient matrix

is not in row echelon form. Apply

R 1 $ 2 to get ✓

(2) The augmented coecient matrix

is not in row echelon form. The

first row is ok; in the second row we have to eliminate 4; hence

R

2

R

2

2 R

1

yielding ✓

16 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

Now R 1

, R

2

, R

3 are ok, but we need to eliminate the last coecients in R 4 and R 5

i.e.,

R 4 R 4 R 3 , R 5 R 5 R 3 ,

leading to 0 B B B B @

1 C C C C A

which is in row echelon form.

(5) The augmented coecient matrix

A

is not in row echelon form.

R

1 is ok, but we need to eliminate the first coecients in R 2

, R

3 , i.e.,

R

2

R

2

+ R

1

, R

3

R

3

+ 2R

1

yielding 0

A

To bring the lattest augmented coecient matrix in row echelon form we need to

exchange the second and the third row, R 2 $ 3 , leading to

A

which is in row echelon form.

We now want to go one step further and describe the set of solutions of a system of

linear equations in more detail. We begin by making some preliminary considerations.

Consider the system (S) (

x 1 + 4x 2 = b 1

5 x 1

• 2x 2 = b 2

The corresponding augmented coecient matrix is given by

(A | b) =

1 4 | b 1

5 2 | b 2

Let us compare it with the system (

e S), obtained by exchanging the two columns of A.

Introducing as new unknowns y 1 , y 2 this system reads

4 y 1

• y 2 = b 1

2 y 1

• 5y 2 = b 2

1.2. GAUSSIAN ELIMINATION 17

and the corresponding augmented coecient matrix is given by (

e A | b) where

e A =

Denote by L and

e L the set of solutions of (S) respectively (

e S). It is easy to see that the

map (x 1 , x 2 ) 7! (y 1 , y 2 ) := (x 2 , x 1 ) induces a bijection between L and

e L. It means that

any solution (x 1 , x 2 ) of (S) leads to the solution y 1 := x 2 , y 2 := x 1 of (

e S) and conversely,

any solution (y 1 , y 2 ) of (

e S) leads to a solution x 1 := y 2 , x 2 := y 1 of (S), or said di↵erently,

by renumerating the unknowns x 1 , x 2 , we can read o↵ the set of solutions of (

e S) from the

one of (S). This procedure can be used to bring an augmented coecient matrix (A | b) in

row echelon form into an even simpler form: assume that A has m rows and n columns.

By renumerating the unknowns, which corresponds to a permutation of the columns of

A, (A | b) can be brought into the echelon form (

e A | b),

e A =

0 B B B B B B B B B B B @

ea 11

0 |ea 22

0 0 |ea 33

0 · · · |ea kk | b

1 C C C C C C C C C C C A

where k is an integer with 0  k  min(m, n) and ea 11

= 0, ea 22

= 0,... , ea kk

= 0. If

k = 0, then

e A is the matrix whose entries are all zero. Note that now all echelons have

height and length equal to ’one’. Using the row operations (R2) and (R3), (

e A | b) can be

simplified. First we apply (R2) to the rows R i , 1  i  k, (R i

1

ea ii

(R

i ) yielding

e e A =

0 B B B B B B B B B B B B B B @

e ea 12

e ea 13

e ea 1 k |

e ea 23

e ea(k1)k |

e e b

1 C C C C C C C C C C C C C C A

1.2. GAUSSIAN ELIMINATION 19

Hence the system (1.2.2) and therefore also the original system with augmented

coecient matrix (A | b) has infinitely many solutions. The map R

nk ! R

n , given

by

B

t k+

t n

C

A 7!

0 B B B B B B B B B B @

b b 1

b b k

1 C C C C C C C C C C A

• t k+

0 B B B B B B B B B @

ba1(k+1)

ba k(k+1)

1 C C C C C C C C C A

• t k+

0 B B B B B B B B B @

ba1(k+2)

ba k(k+2)

1 C C C C C C C C C A

• · · · + t n

0 B B B B B B B B B @

ba 1 n

ba kn

1 C C C C C C C C C A

is a parameter representation of

e L.

Let us now illustrate the discussed procedure with a few examples:

(1) Consider the case of one equation and two unknowns,

a 11 x 1

• a 21 x 2 = b 1 , a 11

Then

x 1

• ba 21 x 2

b b 1 , ba 21

a 21

a 11

b b 1

b 1

a 11

and we are in the CASE 2B with k = 1, m = 1, n = 2. The set of solutions

L =

e L (no renumeration of unknowns were necessary) has the following parameter

representation

R! R

2 , t 2

b b 1

• t 2

ba 21

It is a straight line in R

2 , passing through the point (

b b 1 , 0) and having the direction

(ba 21 , 1).

(2) Consider the case of one equation and three unknowns,

a 11 x 1

• a 12 x 2
• a 13 x 3 = b 1 , a 11

We divide the equation by a 11 and obtain

x 1

• ba 12 x 2
• ba 13 x 3

b b 1

where

ba 12

a 12

a 11

, ba 13

a 13

a 11

b b 1

b 1

a 11

and we are again in the case 2-B with k = 1, m = 1, n = 3. The set of solutions

L =

e L has the following parameter representation

R

2 ! R

3 ,

t 2

t 3

b b 1

A

• t 2

ba 12

A

• t 3

ba 13

A

20 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

It is a plane in R

3 passing through the point (

b b 1 , 0 , 0) and spanned by the vectors

(ba 12 , 1 , 0) and (ba 13

(3) Consider the following system

x 1

• x 2
• x 3 = 4 , x 1 x 2 2 x 3

The corresponding augmented coecient matrix (A|b) is given by

Apply row operation (R3) and replace R 2 by R 2

R

1 to get

Apply row operation (R2), R 2

1

2

R

2 to get

3

2

and finally apply again row operation (R3), R 1

R

1

R

2 , yielding

1

2

3

2

which is in refined echelon form. We are in the CASE 2B with k = 2, m = 2, n = 3.

Since we have not permuted the unknowns,

L = L and a parameter representation

of L is given by

R! R

3 , t 3 7!

A

• t 3

A

which is a straight line in R

3 , passing through the point (2, 2 , 0) and having the

direction (1/ 2 , 3 / 2 , 1).

(4) Consider 8

x 1

• 2x 2 x 3

2 x 1

• x 2
• x 3

3 x 1 + 0 · x 2 + 3x 3 = 1.

Apply row operation (R3) by replacing R 2

R

2

2 R

1

, R

3

R

3

3 R

1 , yielding

A