Download Road Sign Recognition System-Computer Sciences-Project Report and more Study Guides, Projects, Research Applications of Computer Sciences in PDF only on Docsity!
i
Table of Contents
Table of Contents ....................................................................................................................... i Abstract: ................................................................................................................................... iv
iv
Abstract:
As the title suggests Road Sign Recognition System‟s main objective locate road signs from a give image and then interpret their meanings to driver. The project can be thus divided into two main phases first being the one in which road sign detection is carried out. This phase will include deciding the nature of input the system shall be capable of taking and then processing using image processing techniques to locate the road sign. The net phase i.e. that of the interpretation of the signs will consist of extracting the information within the road sign, processing it possibly using a matching algorithm and then telling the driver what the sign stands for. This second phase involves development of a database that shall consist of the classes of road signs our system shall be capable of interpreting.
Timeline
The time line has been changed slightly. A phase called the Generate Control Signs has been removed and the thesis writing work has been pushed back. Presently the Image Enhancement phase is not being dealt with instead the whole system has been iterated once updating it. Mainly work on template matching through correlation and neural network is being done. Shown below is the time line, Fig 0-1;
Figure 0-1: Time line
1. Introduction
Road Signs Detection has become a major part of autonomous driving in the last few years. The basic aim behind this project is to be able to control car speed using traffic sign recognition. And prevent any illegal maneuvers by warning the driver. This system can do so by recognizing a basic set of traffic signs which consist mainly of warning road signs. Such as speed limit road signs, left and right lane road signs and Stop road signs.
The report consists of two parts; previous work and present work. Previous work, fig 0-1, comprises the system developed till seventh semester, giving an overview of the algorithms and problems with the previous system. Present work, starting from Section 9-15 gives an overview of the suggested improvements.
Previous Work
Figure 0-1: Pictorial example of the system previously
Red Regions
Black Regions Image after filtration
Image from shape detector
“No Right Turn”
Template
Labeling
RGB Image
Pictogram Image
4. Image Segmentation (Color Segmentation)
Segmentation is an important part of practically any automated image recognition system, since it separates the region of our interest from the image. Two kinds of color segmentation were applied; Red Color Segmentation and Black Color Segmentation.
4 .1 Red Color Segmentation:
Most road signs containing red color boundary are the ones following in the category of regulatory, mandatory and warning road signs. So color segmentation on the basis of red was applied to highlight areas in the form of blobs areas of concern. The information is then forwarded to the labeling unit.
4 .2 Black Color Segmentation:
Most road signs having red color boundary contain information inside them which is of black color. Segmenting the picture with black color highlights areas where possibly the pictogram or the information of concern may exist. Here the information is used in pictogram extraction used in template matching by correlation.
4 .3 Assumptions:
Images are noise free, noises such as salt and pepper etc. Images have a steady contrast.
4. 4 Common Problems:
The road sign in the image has some shadow. Depending on the darkness of the shadow, the shadowed regions of the road sign get ignored in the segmentation process. At times the road sign with a building in the background or foliage on top of it was detected which resulted in either merged or incomplete segmentation respectively.
5. Labeling
The objective of this module is to count the umber of objects in the image. For further processing an image needs to have at least one object. All connected white pixels in the segmented image are dealt under one label i.e. treat it as one object. As a pre processing step the objects are filled before being labeled as area within a closed boundary are also to be considered as part of that object.
5 .1 Advantages:
Number of objects are counted Acts as a stopping condition for further processing, i.e. if no objects are found current image is dismissed. A lot of effort is saved.
6. Filtration
Filtration was used as a tool to remove extra blobs from the image leaving optimal size objects that may be road signs. This is done by setting thresholds on the no. of pixels a road sign will accumulate. As a preprocess blobs from the segmentation process were labeled.
6 .1 Advantages:
Extra blobs are removed. A minimum and maximum size of road sign that may be processed further is set.
7. Shape Detection
Fuzzy Shape Descriptor uses the region properties of objects in an image e.g. area and center of mass to recognize their shapes. These properties which called “shape measures” would be calculated for each object and they will become the input to the Fuzzy Shape Recognizer. Analyzing the values of the inputs, the object would be categorized in one of the descriptions of geometrical shape types. An figurative overview is given in fig 0-1.
8 .2.1 Pictogram Extraction
a) Remove red border region from the shape by using Erosion. Structuring element is the quarter of the full size of the sign board. (Already done as a post processing step in Shape detection Module) b) Find all points (locations) where the pictogram may exist. c) Extract the pictogram at the same locations from the intensity picture i.e. V from HSV (from Colour Segmentation Module) d) Filter out any blobs other than the pictogram. e) Label the pictogram picture and get only the region containing the pictogram. Crop off the extra area
8 .2.2 Resizing according to Template in Database
The extracted pictogram (cropped) is resized using bi-cubic interpolation to a standard size e.g. to the size of template signs in the database.
8 .2.3 Matching Algorithm
8.2.3.1 Template Matching Template matching techniques compare portions of images against one another. Sample image may be used to recognize similar objects in source image. If standard deviation of the template image compared to the source image is small enough, template matching may be used. Since most of the systems operations involve binary images so Bi-Level Image Template Matching was used.
8.2.3.2 Template Matching Database In order to handle rotation the image of the same sign must be taken from many angles. The database will be categorized according to shapes output by the shape detection module. The Circular Signs will be directed to circular sign templates and matching will be done until all resources are not exhausted. The same goes for Triangular Signs.
8 .3 Process
The process for pictogram matching is as follows: a. Take the binary templates for the circular and triangular signs from the database Apply algorithm for normalized correlation. It would return a matrix with coefficients of correlation. Here correlation is a measure of the degree to which two variables agree, not necessary in actual value but in general behavior. The two variables are the corresponding pixel values in two images, template and source. b.Apply the matching threshold of point 0.7 to get a matrix showing points of match. c. If matching occurs the image of the threshold matrix would show a single bright spot which can be indicated by presence of „1‟ in the image. d.Interpret the sign by its name.
8 .4 Output:
A resulting matrix showing matched values. To the resulting matrix is applied a threshold if matching occurs a single bright spot is shown in the image of the threshold matrix.
Present Work
9. Color Segmentation
The problem with previous colour segmentation module was that it was not invariant to shadows and highlights. The new algorithm overcomes the former weaknesses as Hue image alone is used for segmentation. Hue is invariant to changing light intensity as it is shadow- invariant, multiplicative/ scale invariant, additive/shift invariant and is invariant under saturation changes [1]. Hue is unaffected by shadows and highlights on the object when illumination is white [2].
Vitabile et al. [3] defined three different areas in the HSV colour space. The achromatic area: s <= 0.25 or v <=0.2 or v>=0.
(a)
(b)
(c) Figure 0-1: (a) Ellipticity (b) triangularity (c) shape output
The rules followed by these variables are:
- If E is medium high (MH) and Triangularity is medium high (MH) Then Shape is Triangle
- If E is high (H) and triangularity is medium (M) Then Shape is Circle
- If E is medium high (MH) and triangularity is high (H) Then Shape is Triangle
11. Solution for Incomplete Red Segmentation
For fuzzy shape recognizer the contour has to be complete for classification of shapes. A method maybe devised, in which we may still be able to use the fuzzy system for classification of shapes. Instead of using the red segmentation solely, we may use white segmentation alongside. The white regions may compensate the break in the red boundary. Secondly, using white region along with red would ensure the probably of road sign with red boundary and white inner region.
The algorithm is as follows:
- Find red regions in the image, as in fig. 0-1 (b).
- For each red region that satisfies the threshold size, white regions within the red are found, fig. 0-1 (c).
- Result of both images i.e. red and white, are OR-ed, fig. 0-1 (d).
- The enclosed regions are filled and the solid shapes formed are sent to the Fuzzy Shape recognizer, fig. 0-1 (e).
(a) (b) (c) (d) (e) Figure 0-1: Solution for incomplete red segmentation, (a) original image (b) red regions (c) white regions (d) red and white regions (e) enclosed regions are filled
12. Pictogram Extraction
Pictogram maybe extracted in two ways; either the white regions in a road sign may be sent to the recognition module, or the black symbols maybe recognized.
12.1 White Pictograms
12.2 Black Pictograms
By properly locate the road sign region, characters and symbols maybe extracted by a simple threshold using intensity values. This extraction method, however, sometimes, fails due to bad lighting condition such as large intensity gradation over the sign. Now as the grayscale threshold is sensitive to lighting conditions, an intensity transformation based on the intensity histogram of the image maybe performed.
Black pictograms can be isolated by subtracting the red regions from the black in the end.
12.2.1 Histogram Equalization
Due to changing illumination levels in images the same threshold do not give successful results for black colour. A value set for a darker image would not work for an image which is well illuminated. The objective is to lighten the dark images, and slightly darken the light images.
Two images from the database were used fig 0-2, according to which the histograms of all the remaining images were equalized. These images make black region extraction from images possible for both well and dimly lighted images using the same threshold value.
Figure 0-2: Reference image histograms Since, histogram equalization for images of size 480 x 640 is computationally expensive; the hunt for a relatively cheaper method was conducted.
Figure 0-3: White color segmentation using histogram equalization
12.2.3 Otsu’s Method
Histogram Equalization works on the entire image and the normalization of intensity levels of the entire image increases the computation cost considerably. So a method maybe opted for, in which the threshold is found for only those portions of the image where road sign maybe present, e.g. within the red segmented objects.
Choosing a universal threshold for black regions is a difficult task. If there is just a road sign in the image, e.g. a sub image containing only the area probable of road sign, then Otsu‟s method, which chooses a threshold to minimize the intra-class variance, works very well. This is essentially because in this case, the image histogram will be strongly bimodal between white regions and black pictogram regions, and this method will easily find the dividing point. However, in an image that has a false road sign; Otsu‟s method will return terrible results, as it will cause a segmentation of the non uniform intensity background.
One way to solve this problem is simply to set a fixed parameter that represents a maximum reasonable black intensity, and to take the threshold to be the minimum of this intensity and the threshold computed by Otsu‟s method. But computing a global threshold is not the aim here. Minimizing area of performing operations would result in increased efficiency.
(a) Original image (b) White segmented Figure 0-5 White Segmentation using Adaptive method
Otsu‟s method fails in images with backgrounds darker than the road signs inner pictogram, as shown in figure 0-6 (d). So images where road sign is at some distance from the camera, Otsu‟s method might fail. Even though histogram equalization gives better results in similar situations, its results are slower as compared to Adaptive thresholding.
(a) (b)
(c) (d) Figure 0-6: (a) Original image (b) white segmentation using Adaptive method (c) white segmentation using histogram equalization (d) failure of Otsu method
13. Recognition Using Rotation Invariant Template
Matching
Instead of keeping numerous templates for rotated and affine transformed signs in the database, we may make template matching invariant to rotation by rotating each template in the database to 26 degree in clockwise and anti-clockwise direction with a 6 degree interval. Since the size of templates is 16x16, so the task would not impose too much of computation cost. We may also keep already rotated images in the database. However, in the former case, the rotation for a particular sign will stop as soon as matching occurs.
The problem with the rotation of templates from the database is that, due to the small size of templates, rotation distorts the template to an extent that matching becomes impossible. The information is distorted will rotating a sign with a total of 16x16 pixel information, as shown n fig. 0-1.
Figure 0-7: 16x16 sized template rotated 6, 12, 18, 24 degrees
Solution to overcome this distortion might be to adopt a template matching system, in which the rotated templates from an at least 40x40 template image be already kept in the database, as shown in fig. 0-2. Instead of rotating online, an already rotated image can be selected.
Figure 0-8: 40x40 sized template rotated 6, 12, 18, 24 degrees
We may also inculcate a condition for rotation that if the coefficient values for template matching are going towards the threshold value, we may continue rotating in that direction,
Pictogram f Original Imagerom, resized to 16x
Template resized to 16x
Template rotated 6 degree and resized
Pictogram from resized to^ Original Image 16x16,^ Template rotated^ 12 degree and resized
Template rotated 18 degree and resized
Template resized to 16x
Template rotated 6 degree and resized
Template rotated 12 degree and resized
Template rotated 18 degree and resized
Template rotated 24 degree and resized
