Learning with Weighted Examples - Machine Learning | CSCI 5622, Study notes of Computer Science

Download Learning with Weighted Examples - Machine Learning | CSCI 5622 and more Study notes Computer Science in PDF only on Docsity! 1 Learning With Weighted Examples Greg Grudic Weighted Training Examples • Training Examples • Associate a weight • The greater the weight, the greater the importance of the training example – Might come from prior knowledge – Or a boosting algorithm…. ( ) ( ){ }1 1, ,..., ,N ND y y= x x { }1 2 3, ,...,w w w 0iw > 2 Weighted Sum of Least Squares • The Loss Function being minimize: • The matrixes and outputs become • The Solution is 2 0 1 1 weighted RSS N d i i j ij i j w y xβ β = =   = − −    ∑ ∑ 1 1 11 1 1 1 1 w 1 , d N N N N Nd N N w w x w x w y X w w x w x w y           = =            Y ( ) 1ˆ T T w − =β X X X Y Weighted Ridge Regression • The Loss Function being minimize: • The matrixes and outputs become • The Solution is 2 2 0 1 1 1 weighted Ridge Loss N d d i i j ij j i j j w y xβ β λ β = = =      = − − +        ∑ ∑ ∑ ( ) 10ˆ T T wIλ − = +β X X X Y 1 1 11 1 1 1 1 w 1 , d N N N N Nd N N w w x w x w y X w w x w x w y           = =            Y 0I is a (d+1) by (d+1) identity matrix with the first entry set to zero