Linear Algebra and Probability Review for Machine Learning and Data Science, Lecture notes of Mathematical Methods

Download Linear Algebra and Probability Review for Machine Learning and Data Science and more Lecture notes Mathematical Methods in PDF only on Docsity!

ENCS

Machine Learning and Data Science

Linear Algebra and Probability Review

Yazan Abu Farha - Birzeit University Slides are based on Stanford CS229 course

Linear Algebra

The Identity Matrix

• The identity matrix, denoted 𝐼 ∈ ℝ !×! , is a square matrix with ones on the diagonal and zeros everywhere else. That is, 𝐼'( = -

• It has the property that for all 𝐴 ∈ ℝ $×! , 𝐴𝐼 = 𝐴
• Ex: 𝐼# =

Diagonal matrices

• A diagonal matrix is a matrix where all non-diagonal elements are 0. This is typically denoted 𝐷 = 𝑑𝑖𝑎𝑔(𝑑", 𝑑#, … , 𝑑!), with 𝐷'( = -

• For example the identity matrix 𝐼 = 𝑑𝑖𝑎𝑔 ( 1 , 1 , … , 1 )

Matrix-Vector Product

• If we write A by rows, then we can express Ax as,

Matrix-Vector Product

• If we write A by columns, then we have: y is a linear combination of the columns of A.

Matrix-Vector Product

• It is also possible to multiply on the left by a row vector.
  • expressing A in terms of rows we have y T is a linear combination of the rows of A.

Matrix-Matrix Multiplication (different views)

1. As a set of vector-vector products (dot product)
2. As a sum of outer products

Matrix-Matrix Multiplication (properties)

• Associative: (AB)C = A(BC).
• Distributive: A(B + C) = AB + AC.
• In general, not commutative; that is, it can be the case that AB ≠ BA. (For example, if 𝐴 ∈ ℝ $×! and B ∈ ℝ !×* , the matrix product BA does not even exist if m and q are not equal!)

The Transpose

• The transpose of a matrix results from “flipping” the rows and columns. Given a matrix 𝐴 ∈ ℝ $×! , its transpose, written 𝐴 & ∈ ℝ !×$ , is the n × m matrix whose entries are given by
• The following properties of transposes are easily verified:

Examples of Norms

The Inverse of a Square Matrix

Probability Theory

Definitions, Axioms, and Corollaries