Fisher Linear Discriminant: Feature Selection in Computational Functional Genomics - Prof., Study notes of Biology

Download Fisher Linear Discriminant: Feature Selection in Computational Functional Genomics - Prof. and more Study notes Biology in PDF only on Docsity!

• Computational Functional GenomicsLecture 17Genomic data-mining method 5 – feature selection Yang DaiBioE

Prof. Yang Dai BioE 594 Computational Functional Genomics

Fisher Linear Discriminant -1 † Fisher Linear Discriminant finds projection to a linewhich preserves direction useful for data classification † FLD seeks projection that best separates the data in aleast-square sense. Consider the problem of projectingdata (x,...,

x^ ) in^ d-dimensional space onto a line.n n: number of the points in subset

D, label +;

n: number of the points in subset

D, label -;

Let^ kwk^ = 1, consider the inner products of

w^ and^ x^ i

ziis the projection of^

x^ onto a line in the directioni^

w

Divide^ z^ into two classes according to the labels ofi^

x^ i

Prof. Yang Dai BioE 594 Computational Functional Genomics

Fisher Linear Discriminant - 3 † Find the best direction

w „^ A measure of the separation between the projected points is thedifference of the sample means. „^ Let Then the sample mean for projected points of each class is

Prof. Yang Dai BioE 594 Computational Functional Genomics

Fisher Linear Discriminant - 4^ †^ The distance between the projected means is^ †^ We want this difference as large as possible. Large relative tosome measure of the standard deviation for each class.DefineThis is called scatter for projected samples in class

Dk

„^ scatter measures the same thing as variance, the spread ofdata around the mean. It is just on different scale thanvariance † : the total within-class scatter of the projected samples † The Fisher linear discriminant employs that linear functions

T wx

for which

is maximized

Prof. Yang Dai BioE 594 Computational Functional Genomics

Fisher Linear Discriminant - 6^ †^ The object function is^ †^ Let

Prof. Yang Dai BioE 594 Computational Functional Genomics

Fisher Linear Discriminant - 7 †^ But from †^ We can simply take

-1 w=S (μW

• μ), since the scale of w 12

does not matter. This

w^ is the direction determined by Fisher's linear discriminant. † Thus the classification has been converted from a

d-

dimensional problem to more manageable one-dimensional one. † All that remains is to find the threshold,

i.e. , the point

along the one-dimensional subspace separating theprojected points.

Prof. Yang Dai BioE 594 Computational Functional Genomics

Fisher Linear Discriminant - example † First compute the mean for each class † Compute scatter matrices

Sand^ S^

for each class

†^ Within class scatter matrix †^ The inverse of

Sw^ is †^ Therefore the optimal line direction

w

Prof. Yang Dai BioE 594 Computational Functional Genomics

Fisher Linear Discriminant - example^ †^ As long as the line hasthe right direction, itsexact position does notmatter^ †^ Finally we compute theactual

1D^ vector

y^ for each class

Prof. Yang Dai BioE 594 Computational Functional Genomics

Feature selection -2^ †^ Feature vector selection : Consider all possible combination ofs features out of original d ones. For each combination,measure the separability :Project points to an s-dimensional space.the higher this value is the better the discriminative power.|A| is the determinant of a matrix A. The determinant is theproduct of the eigenvalues, and hence is the product of the``variances'' in the principal directions.^ †^ Choosing the combination with the best C(s) value will give thebest classification, however, it is impractical for large d and s.Need to develop heuristic methods.

s.projection for thematricesscatter areand, | || SB (^) )( SSSC = BW | SW

Prof. Yang Dai BioE 594 Computational Functional Genomics

Feature selection -3^ †^ Sequential forward selection1.^ Computer the criterion value for each of the feature; Selectthe feature with the best value; s:=1;2.^ For the current s-dimensional feature vector, form allpossible (s+1)-dimensional vectors by adding one featurefrom the remaining features d-s. (Therefore obtain d-sdifferent ways of select the feature vector)(if 1 is selected at step (1) then we have (1,2) (1,3) (1,4) (1,5)combinations of features if d=5)3.^ Choose the feature with the best criterion value; s:=s+1;4.^ If s=s

(a previously determined value) then stop, otherwise (^0) go to 2.