CS229 Lecture notes
Andrew Ng
The k-means clustering algorithm
In the clustering problem, we are given a training set {x(1),...,x(m)}, and
want to group the data into a few cohesive “clusters.” Here, x(i)∈Rn
as usual; but no labels y(i)are given. So, this is an unsupervised learning
problem.
The k-means clustering algorithm is as follows:
1. Initialize cluster centroids µ1, µ2,...,µk∈Rnrandomly.
2. Repeat until convergence: {
For every i, set
c(i):= arg min
j||x(i)−µj||2.
For each j, set
µj:= Pm
i=1 1{c(i)=j}x(i)
Pm
i=1 1{c(i)=j}.
}
In the algorithm above, k(a parameter of the algorithm) is the number
of clusters we want to find; and the cluster centroids µjrepresent our current
guesses for the positions of the centers of the clusters. To initialize the cluster
centroids (in step 1 of the algorithm above), we could choose ktraining
examples randomly, and set the cluster centroids to be equal to the values of
these kexamples. (Other initialization methods are also possible.)
The inner-loop of the algorithm repeatedly carries out two steps: (i)
“Assigning” each training example x(i)to the closest cluster centroid µj, and
(ii) Moving each cluster centroid µjto the mean of the points assigned to it.
Figure 1 shows an illustration of running k-means.
1
