Download Normalized Cuts Algorithm for Image Segmentation by Sumengen and Tesic and more Papers Political Science in PDF only on Docsity!
ECE 278 Final Report
Normalized Cuts and Image Segmentation
Baris Sumengen, Jelena Tešić
Abstract
Our project work includes the embedding of a normalized cuts algorithm and testing its application for image segmentation. Proposed solution minimizes a graph bi partitioning based on a modified cost function. Final result has a form of partition candidates based on a low-level image description. To increase the stability of the proposed method, algorithm has an iterative form. Computational complexity issues remain unsolved, even though the method performs well for a preprocessing.
1. Introduction
The importance and difficulties of perceptual grouping have been very well studied in vision research and image processing literature. How to partition image to subsets, when there are many possible ways of doing it? The intuitive answer would be a partitioning based on the available prior knowledge. The difficulty is specifying the knowledge in a compact and useful way i.e. defining the descriptors of an image. It is clear that the use of low-level descriptors could not lead to a complete and correct final segmentation. Instead, image attributes should be used for obtaining candidate partitions. Higher-level descriptors can be used in decision process and “finer tuning” of the proposed segments. Over the decades, different scientific communities proposed various approaches to generalized image segmentation problem. Tree-based hierarchical clustering structures, region-based merge and split image processing algorithms and statistical approach to image segmentation introduced additional constraints on a possible solution. Before any algorithm implementation we have to determine what criterion we want to optimize and if there is an efficient algorithm for implementation method. We separate the proposed method from the ones mentioned above because it has a novel approach to an image. It is taking a global image feature descriptor as a weighted graph and reduces segmentation to an optimal graph partitioning. New optimization criterion - normalized cut - is used for finding the optimal partitioning. The graph partitioning approach and related work is explained in the section 2. Normalized cuts criterion and its justification are presented in the section 3. The algorithm and its properties are presented in the section 4. Conducted experiments and results are outlined in the section 5. Section 6 discusses the proposed approach and derives the project conclusion.
2. Graph Partitioning
The set of points in an arbitrary feature space is presented as a weighted undirected graph G=(V,E). Nodes of the graph are the points in the feature space. An edge is formed between every pair of nodes and the weight on each edge W(i,j) is a function of the similarity between nodes i and j. A graph G=(V,E) is partitioned into two disjoint complementary sets A and B, B=V-A , by removing the edges connecting two parts. The degree of dissimilarity between two sets can be computed as a total weight of removed edges. That closely relates to a mathematical formulation of a cut (1):
cut A B ( , ) = ∑ u ∈ A v , ∈ Bw u v ( , ) (1)
2.1. Related Approaches
Problem formulation is to find an optimal bi partitioning of a graph G. The problem reduces to finding a cost function that will give us perceptually good results (since we’re dealing with feature descriptors of an image). Global optimum is achieved if we minimize the cut value. If N is cardinality of V , N =|V |, there is an exponential number of possible partitions 2N.^ This optimization is well studies and there is an efficient algorithm for finding a minimum cut [ 3 ]. Wu and Leahy proposed a clustering method based on the minimum cut criterion [ 5 ]. They are trying to partition the graph into k sub graphs so that the maximum cut across the partitions is minimized. They introduced an efficient way to obtain the solution by recursively finding the minimum cut that bisects the existing elements. This method produced some good segmentation results. Authors noticed that the performance is highly influenced by the graph outliers. From the definition of global optimum (1) , this is an expected outcome. The following example illustrates the bad partitioning case:
Cox [r] proposed a modified cost function (2), where weight(A) is some function of set A, i.e. sum of elements in A.
( ) (^ ,^ ) (2)
WCut A cut A V^ A
weight A
=^ −
3. The grouping algorithm
The eigenvector corresponding to the second smallest eigenvalue is the real-valued solution that optimally sub-partitions the entire graph, the third smallest value is a solution that optimally partitions the first part into two, etc. One can subdivide the existing graphs each time using the eigenvector with the next smallest eigenvalue. However, since the approximation error accumulates with every eigenvector taken and we want eigenvectors with discrete elements to satisfy the orthogonal property, the solution based higher eigenvector become unreliable. We solve the partitioning problem for every sub graph separately and this is implemented as a recursive grouping algorithm. Eigenvalue computation is expensive. We are using the approximate Lanczos method for computation the eigenvector of a very sparse matrix where we need only couple of eigenvectors. Computational cost is typically less than O(n3/2) where n is the number of nodes in the graph. Graph is partitioned using the second smallest eigenvector. Eigenvector elements take on continuous values, so we need to define a splitting point for eigenvector. Authors adopt a heuristic method for splitting point search. Number of evenly spaced splitting points in the vector range is checked, and normalized cuts are computed for each point. The point that gives the minimum Ncut value is chosen as a splitting point. The other approach could be to take 0 or a median value as a splitting point, but it wasn’t really reliable because of the approximation error. Both methods are good enough if values are very well separated. We recursively apply the algorithm to every sub graph until the Ncut exceeds certain limit. Output of the one algorithm pass gives us partition candidates. Cut is unstable if Ncut value does not change much when varying the set of graph edges forming the cut. Summarized algorithm flow looks like:
- Define the feature description matrix for a given image and a weighting function
- Set up a weighted graph G=(V,E), compute the edge weights and summarize information into W and D
3. Solve ( D − W x ) = λ Dx for eigenvectors with the smallest eigenvalues.
- Use the eigenvector with second smallest eigenvalue to bipartition the graph by finding the splitting points such that Ncu t is minimized.
- Decide if the current partition is stable and check the value of the resulting Ncut
- Recursively repartition the segmented parts (go to step2)
- Exit if Ncut for every segment is over some specified value – maximum allowed Ncut 4. Experiments and Results
We applied the partitioning algorithm to gray-scaled and color images. We constructed the graph G=(V,E) by taking each pixel as a node and defining the edge weight function as following:
2 2
i j i j I X
F F X X
W i j e^ σ e σ X i X j r
− − − −
1
( ), intensity
( ) , sin( ), cos( ), color
* ,..., * n , texture
I i
F i v v s h v s h i
I f I f i
i i i i
4.1. Texture Images - descriptors formed based on intensities
The weight wij =0 for any pair of nodes i and j that are more than r pixels apart. σ= 0.07, r = 3.0, σ I = 0.02, σ X =3.
2 4 6 8 10 12 14 16 18 20
2 4 6 8 10 12 14 16 18 (^202 4 6 8 10 12 14 16 18 )
2 4 6 8 10 12 14 16 18 (^202 4 6 8 10 12 14 16 18 )
2 4 6 8 10 12 14 16 18 20
σ= 0.1, r = 5.0, σ I = 0.01, σ X =4.
5 10 15 20 25 30
5
10
15
20
25
(^305 10 15 20 25 )
5 10 15 20 25 (^305 10 15 20 25 )
5 10 15 20 25 30
Normalized cuts algorithm is slow. Main reason is that, if the image dimension is nxn, we have to keep n2x n2 matrix in the main memory. Based on that reasoning, we have decided to use 20x 20 gray scaled images. If our original image is larger, we subsampled it. Since this algorithm is based on low-level descriptors, use of a coarse image representation is justified. We’re looking for partition candidates, not trying to produce a fine segmentation. The algorithm is implemented in MATLAB. Few eigenvector are found using approximate iterative MATLAB function EIGS. Split point is found among 128 equally spaced points in the vector element range, as the one that gives the smallest Ncut value. Gray-scaled image segmentation is an example of image enhancement. We introduced a Gaussian noise to the image. Segmentation results highly depend on a chosen parameters. These simple examples were the best starting point since the format of the weighted function gives the best results for Gaussian noise.
For color based segmentation, feature vector is defined as F i ( ) = , v v s i i sin( ), h v s i i cos( ) h i , where
h,s,v are the HSV values. Again, depending on the parameters, for the teeth image we get small number of well-defined segments or a big number of small, not perceptually different segments.
5. Conclusion
We implemented a normalized cut algorithm and used it for the intensity and color segmentation. Even though the approximate eighenvalue method and the algorithm construction optimize implementation, computational complexity remains unsolved for the full-scale image. The performance and stability of the partitioning highly depends on the choice of parameters. We have noticed that the choice of parameters is data-dependent in some sense, but authors don’t offer the solution and do not justify their choice of parameters. Heuristic method for finding the splitting point is questionable. Smallest, non-zero eigenvalues, for given eigensystem with sparse matrices, often have really small magnitude and the choice of splitting point is influenced by computation precision. Stability is controlled by many factors: choice of splitting point, eigenvalue computations and relative segment position. Authors [2] didn’t suggest how to deal with border segments. We modify matrix D and W depending on the relative segment position. Otherwise, algorithm becomes unstable and starts partitioning homogeneous regions (since neighboring regions are not taken into account). The other approach is to choose maximum Ncut for every segment but that would be difficult to implement, if not impossible. Consequently, Normalized cut application is highly constrained by the previous facts.
6. References
[1] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," Int. Conf. Computer Vision and Pattern Recognition , San Juan, Puerto Rico, June 1997. [2] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," Technical Report CSD-97-940, UC Berkley, 1997. [3] A. Pothen and H. Simon, “Partitioning Sparse Matrices with Eigenvectors of Graphs”, IBM Journal of Research and Development , pp. 420-425, 1973. [4] M. Fiedler, "A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory", Czech. Math. J. 25, pp.619--637, 1975. [5] Wu, Z., and Leahy, R., “An Optimal Graph Theoretic Approach to Data Clustering: Theory and Its Application to Image Segmentation”, PAM I (15) , No. 11, pp. 1101-1113, 1993.
