Docsity
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Docsity AI
Docsity AI
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Object Detection and Tracking In Video Thesis-Computer Sciences-Project Report, Study Guides, Projects, Research of Applications of Computer Sciences

Birla Institute of Technology and ScienceApplications of Computer Sciences

This report is for final year project to complete degree in Computer Science. It emphasis on Applications of Computer Sciences. It was supervised by Dr. Abhisri Yashwant at Bengal Engineering and Science University. Its main points are: Block, Diagram, Hamburg, Taxi, Sequence, Visual, Surveillance, Gradient, Video, Flow, Chart

Typology: Study Guides, Projects, Research

2011/2012

Uploaded on 07/18/2012

padmini
padmini 🇮🇳

4.4

(207)

175 documents

1 / 83

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
vii
List of Figures
Figure 1
-
1 Block Diagram of system
................................
................................
.............
3
Figure 2
-
1:
Application in Vis
ual Surveillance: Optical Flow computed on Carthe
Hamburg taxi sequence
................................
................................
..........................
8
Figure 2
-
2 : Shows the color confidence maps for a certain frame in the sequence used
................................
................................
................................................................
9
Figure 2
-
3: Gradient based background subtraction in a certain frame
.......................
10
Figure 2
-
4: The video results showing background subtraction in Frame 10 and 15
..11
Figure 2
-
5: The video results showing background subtraction in Frame 42 and 45
..11
Figure 2
-
6: The video results showing background subtraction in Frame 86 and 92
..12
Figure 2
-
7: Video Results showing high illumination in the background in frame 56
and 58
................................
................................................................
...................
12
F
igure 4
-
1: Flow chart representation of Template based Target Tracking
.................
22
Figure 4
-
2: Video results showing Target tracking using template matching in frame 7
and 11
................................
................................................................
...................
23
Figure 4
-
3: Video results showing Target tracking using template matching in frame
52 and 75
................................................................
................................
..............23
Figure4
-
4: Change in orientation of the object
................................
............................
24
Figure 4
-
5: Template selected for Tracking
................................
................................
.
24
Figure 4
-
6: Tracker losing the object
................................
................................
...........25
Figure 4
-
7: Template selected for tracking
................................
................................
..26
Figure 4
-
8: Integral Image
................................
................................
............................
31
Figure 4
-
9: Flow Chart representat
ion of Fast Mean Shift Algorithm
.........................
34
Figure 4
-
10 : Flow Chart representation of Fast Mean Shift Algorithm (cont
-
I)
........35
Figure 4
-
11
: Flow Chart representation of Fast Mean Shift Algorithm (cont
-
II)
........36
Figure 4
-
12: Results of Fast Mean Shift based tracking in Frame number 45 and 56
.
37
Figure 4
-
13 : Results of Fast Mean Shift based tracking in Frame number 67 and 113
................................
................................
................................
..............................
37
Figure 4
-
14: Results of Fast Mean Shift based tracking in frame number 13 and 25
..38
Figure 4
-
15: Results of Fast Mean Shift based tracking in frame number 69 and 94
..38
Figure 4
-
16: Flow Chart representation o
f Fast Mean Shift Algorithm
.......................
39
Figure 4
-
17: Flow Chart representation of Fast Mean Shift Algorithm (cont
-
I)
.........40
Figure 4
-
18: Flow
Chart representation of Fast Mean Shift Algorithm (cont
-
II)
........41
Figure 5
-
1Constant Velocity Model
................................
................................
.............46
Figure 5
-
2 Random Walk Mode
l
................................................................
.................
47
Figure 5
-
3 Constant Acceleration Model
................................
................................
.....49
Figure 5
-
4 Constant Acceleration Model with Random Walk
................................
.....50
Figure 5
-
5 Random Velocity Random Acceleration Model
................................
........51
Figure 5
-
6 Discrete Kalman Filter
................................................................
...............
52
Figure 5
-
7 Flow Chart representation of Discrete Kalman Filter
................................
60
Figure 5
-
8 Tracking Results using Kalman Filter Showing frame number 10 and 20
61
Figure 5
-
9 Tracking Results using Kalman Filter Showing frame number 42 and 43
62
Figure 5
-
10 Tracking Results using Kalman Filter Showing frame number 45 and 52
................................
................................
................................
..............................
63
Figure 5
-11 Orientation changes and Kalman Filter s Tracking ability shown in frame
number 143 and 150
................................
................................
..........................
64
Figure 5
-
12 Orienta
tion changes in frame number 179
................................
...............
65
Figure 5
-
13 Tracking Results using Kalman Filter Showing frame number 155
........65
docsity.com
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16
pf17
pf18
pf19
pf1a
pf1b
pf1c
pf1d
pf1e
pf1f
pf20
pf21
pf22
pf23
pf24
pf25
pf26
pf27
pf28
pf29
pf2a
pf2b
pf2c
pf2d
pf2e
pf2f
pf30
pf31
pf32
pf33
pf34
pf35
pf36
pf37
pf38
pf39
pf3a
pf3b
pf3c
pf3d
pf3e
pf3f
pf40
pf41
pf42
pf43
pf44
pf45
pf46
pf47
pf48
pf49
pf4a
pf4b
pf4c
pf4d
pf4e
pf4f
pf50
pf51
pf52
pf53

Partial preview of the text

Download Object Detection and Tracking In Video Thesis-Computer Sciences-Project Report and more Study Guides, Projects, Research Applications of Computer Sciences in PDF only on Docsity!

vii

Figure 2 - 2 : Shows the color confidence maps for a certain frame in the sequence used

  • Figure 1 - 1 Block Diagram of system List of Figures
    • Hamburg taxi sequence Figure 2 - 1: Application in Vis ual Surveillance: Optical Flow computed on Carthe
  • Figure 2 - 3: Gradient based background subtraction in a certain frame
  • Figure 2 - 4: The video results showing background subtraction in Frame 10 and 15 ..
  • Figure 2 - 5: The video results showing background subtraction in Frame 42 and 45 ..
  • Figure 2 - 6: The video results showing background subtraction in Frame 86 and 92 ..
  • Figure 2 - 7: Video Results showing high illumination in the background in frame
    • and
  • F igure 4 - 1: Flow chart representation of Template based Target Tracking
  • Figure 4 - 2: Video results showing Target tracking using template matching in frame
    • and
    • 52 and 75 .............. Figure 4 - 3: Video results showing Target tracking using template matching in frame
  • Figure4 - 4: Change in orientation of the object
  • Figure 4 - 5: Template selected for Tracking
  • Figure 4 - 6: Tracker losing the object ...........
  • Figure 4 - 7: Template selected for tracking ..
  • Figure 4 - 8: Integral Image
  • Figure 4 - 9: Flow Chart representation of Fast Mean Shift Algorithm
  • Figure 4 - 10 : Flow Chart representation of Fast Mean Shift Algorithm (cont- I) ........
  • Figure 4 - 11 : Flow Chart representation of Fast Mean Shift Algorithm (cont- II) ........
  • Figure 4 - 12: Results of Fast Mean Shift based tracking in Frame number 45 and
  • Figure 4 - 13 : Results of Fast Mean Shift based tracking in Frame number 67 and
  • Figure 4 - 14: Results of Fast Mean Shift based tracking in frame number 13 and 25 ..
  • Figure 4 - 15: Results of Fast Mean Shift based tracking in frame number 69 and 94 ..
  • Figure 4 - 16: Flow Chart representation o f Fast Mean Shift Algorithm
  • Figure 4 - 17: Flow Chart representation of Fast Mean Shift Algorithm (cont- I) .........
  • Figure 4 - 18: Flow Chart representation of Fast Mean Shift Algorithm (cont- II) ........
  • Figure 5 - 1Constant Velocity Model .............
  • Figure 5 - 2 Random Walk Mode l
  • Figure 5 - 3 Constant Acceleration Model .....
  • Figure 5 - 4 Constant Acceleration Model with Random Walk .....
  • Figure 5 - 5 Random Velocity Random Acceleration Model ........
  • Figure 5 - 6 Discrete Kalman Filter
  • Figure 5 - 7 Flow Chart representation of Discrete Kalman Filter
  • Figure 5 - 8 Tracking Results using Kalman Filter Showing frame number 10 and
  • Figure 5 - 9 Tracking Results using Kalman Filter Showing frame number 42 and
  • Figure 5 - 10 Tracking Results using Kalman Filter Showing frame number 45 and
    • number 143 and Figure 5 -11 Orientation changes and Kalman Filter s Tracking ability shown in frame
  • Figure 5 - 12 Orienta tion changes in frame number
  • Figure 5 - 13 Tracking Results using Kalman Filter Showing frame number 155 ........
  • Figure 6 - 1 Grap hical User Interface ............. viii
  • Figure 6 - 2 Path selection for video input .....
  • Figure 6 - 3 Input video is selected
  • Figure 6 - 4 Object to be tracked is identified
  • Figure 6 - 5 Track button starts Tracking .......
  • Figure 6 - 6 Drop down list for tracking methods
  • Figure 6 - 7 Tracking video window ..............

x

Abstract

The techniques studied, implemented and presented are all premeditated in detail and then put into p ractice in this thesis. The techniques deliberated and implemented so far are Template Matching, Fast Mean Shift and Kalman Filter where as for object detection background subtraction through frame differencing is implemented. Still there is room for improvement and future enhancements which can improve the efficiency and increase performance level. Object tracking through Kalman filter is known for accuracy and precision so as Fast Mean Shift is also a very receptive method to be applied in practical implementations where as it is found after implementation and study that Template Matching is a lengthy procedure due to its correlation operations but withstands occlusion to some extent. So this thesis is an inclusive introduction to object tracking hierarchy and object tracking techniques with their implementation complications and experimental results.

xi

events, and recognition of objects. Video tracking is a vital and active research area in computer vision.

In its simplest form, tracking can be defined as the problem of estimating the trajectory of an object in the image plane as it moves around a scene. In other words, a tracker assigns consistent labels to the objects in question in different frames. Object tracking no doubt is an exigent problem. Impenetrability in tracking can arise due to abrupt motion of object, changing directions and appearances of the object and the scene, non rigid or articulated object structures as birds and human beings, object- to -object and object- to -scene occlusions, and camera motion [ 5]. Tracking is generally performed in the milieu of higher-level applications that require the location and/or shape of the object in every frame. Normally, assumptions are made to constrict and lessen the tracking problem s.

1.2 Scope and Objective

Moving object detection is important in many real-time image processing applications such as autonomous robotics, traffic control, and driver assistance and surveillance systems. Usually high resolution gray-scale images must be processed; since each image pixel may belong to a moving object, pixel- wise processing is required. Now days, video surveillance is an important and challenging field in computer vision for both indoor and outdo or environments. Organizations which need a surveillance system can easily get low priced surveillance cameras but they still need many security agents to keep a constant watch on all monitors. This approach is not efficient, and in fact most of the time video tapes or files are replayed a number of times to check on a particular event after it has happened thus the automation of this system is highly desired.

The project Moving object tracking from video sequences is an attempt to study some algorithms, which are robust for the tracking of mobile but non rigid objects from the image sequences precisely called video. The Figure 1-1 shows the basic working grid of the processes involved in the system as Figure 1-1 also includes pre and post processing sequences as well which includes noise removal, image enhancement issues, organization and classification etc. but these issues are not of primary

importance as far as this project is concerned so are not addressed in detail in this project.

Figure 1-1 Block Diagram of system

1.4 Thesis Outline

Chapter 1 includes the introduction of the problem with its objective and scope along with applications. The concept of object detection and its methods along with the method studied and implemented which is background subtraction is all engrossed in Chapter 2. Object tracking in detail and its techniques are described in chapter 3. Similarly, chapter 4 depicts the modus operandi used for object tracking during this project along with their experimental results including Template matching and Fast Mean Shift. The Kalman filter and it necessary details along with implementation complexities, algorithm and results are described in chapter 5 .It also includes the comprehensive comparison of techniques used. In the end whole of the work is concluded and few future work aspects are mentioned in chapter 6.

Chapter 2

2 Object Detection

One of the most difficult problems in image processing is detecting particular objects in an image. For a human observer it is very easy to identify any object, however it is far more difficult for a machine. Numerous methods exist for detecting objects of known type in a particular environment or image. However, in many cases, the visual characteristics of the objects are unknown, or it is necessary to detect objects that are very different from each other. This kind of method has applications in the domain of robotics, particularly for robots that are designed to operate in a hostile or unknown environment.

2.1 Introduction of Object Detection

Videos are the sequences of frames that run fast enough to give an effect of continuity as human eye perceives the frame sequences moving with a particular speed as a video. As far as object detection is concerned techniques of image processing are applied to the frames in order to identify any change so as to state the motion of object detected through the change observed in the two consecutive frames after attaining the results of the image processing techniques used for identification purpose. A surveillance system can be implemented in three steps. The first step consists of detecting the objects in motion .Then tracking them and finally High- level interpretation of the ongoing events. First step of detecting the object in question is described in detail with the experimental results in order to elaborate the mechanism in Chapter 2. Object detection is the first step in the motion tracking phenomena. The object detection is performed through background subtraction algorithm in this project though many other detection methods have been already developed and are widely in use.

2.2.2 Optical Flow

The optical flow is the disarticulation field allied to each of the pixels in a sequence. Such displacement field results from the apparent motion of the image brightness in time. For the computation of optical flow it is assumed that image brightness is continuous and differentiable as many times as needed in both the spatial and temporal domain. Estimating the optical flow is fundamental problem in low- level vision, and can be undoubtedly serve for many applications in image sequence processing. Most of the algorithms for the estimation of the optical flow concentrate on the goal of estimating the motion field between succeeding images in a sequence, disregarding the estimates obtained for the previous image pair.

If the apparent brightness of moving objects remain constant then the image brightness E over time is given by

dE dt (^) 0 (2.2)

Figure 2-1: Application in Visual Surveillance: Optical Flow computed on Carthe Hamburg taxi sequence

Since image brightness E is regarded as a function of both spatial coordinates of the image plane, x and y , and of time, that is,

E E x , y , t (2.3)

Via a chain rule of differentiation, the total temporal derivative reads as

dE xtdt , yt , t Exdxdt Eydydt E t 0 (2. 4)

2.2.3 Color based Method

This method is based on background modeling and subtraction using both color and edge information. Both the color and edge models and subtraction are computed separately [10]. For storage of results confidence maps are used rep resent ing how confident the method is in recognizing that a pixel is a foreground obje ct. Color and intensities of the previous image are compared with every new coming image and this difference signifies the motion. Background subtraction is done by performing the color -based subtraction and the edge-based subtraction separately and then combining the re sults [10]. Color -based subtraction is performed by subtracting the current image from the mean image in each color channel. For each pixel, the confidence is computed as

c

c cc c c

c M D M

MD mm m D M

D m C 100

A significant change in any color channel indicates motion and thus a foreground region is detected.

50 100 150 200 250 300

50 100 150 200

Figure 2-2 : Shows the color confidence maps for a certain frame in the sequence used

2.2.6 Experimental Results

The results achieved after applying the background subtraction algorithm using Frame Differencing technique are illustrated as under.

Figure 2-4 : The video results showing background subtraction in Frame 10 and 15

Figure 2-5 : The video results showing background subtraction in Frame 42 and 45

Figure 2-6: The video results showing background subtraction in Frame 86 and 92

In few cases where the background illumination or brightness becomes higher up to a certain level that it becomes detectable after subtraction like the object it also appears as shown in figure 2 - 4.

Figure 2-7 : Video Results showing high illumination in the background in frame 56 and 58

So, the experimental results shown in Figure 2- 1,2 - 2,2 -3 show the object in question which is a plane clearly detected from the background .Similarly, Figure 2-4 shows another dimension of the background subtraction algorithm that it has to be robust enough in order to overcome the raising illuminat ion and brightness factors.

2.3 Conclusion

Object detection is the lying at the start in the hierarchy of motion based tracking mechanism. It basically deducts the background and focuses the object of interest from the scene. Many methods can be used for background subtraction depending on color, edge and such kind of features .Some of the methods are described in brief in the preceding chapter .It is effectively used in real time applications as well .It is also feasible in correcting images deserts caused by inappropriate illumination and brightness effects .Thus background subtraction is one of the widely used and important technique as far as object detection in concerned.

out in automatic mode. Once the target is confirmed the control of the system is transferred to tracking state.

3.2.2 Tracking state

This stage should use computationally inexpensive techniques. Current location extracted by locking state is used for processing. Next position of the target is identified, and that positional information is stored in history database. If the target does not exist in the predicted window area, then the system control is transferred to recovery state.

3.2.3 Recovery state

Quite often the moving object of interest may be lost temporarily or permanently. In this state if the target is lost, the system will try to recover the target from low- resolution image. If the target is recovered in a few frames, then the system will transfer control to tracking state; otherwise it remains in recovery state till its predefined time expires. After the time is elapsed, control transfers to locking state.

3.3 Different Approaches for Object Tracking

Different methods have been used for moving object tracking given as under.

  1. Correlation based method
  2. Feature based method
  3. Histograms method
  4. Gradient based method
  5. Contour based method
  6. Kernel based method
  7. Kalman Filter
  8. Extended Kalman Filter
  9. Particle Filter

3.3.1 Correlation based method

The correlation based method simply operates on the principal of intriguing correspondence between the previous frame and every up coming frame. This mechanism finally results in the correlation vector of the previous and new frame which in fact gives the measure of relationship between both frames. Thus the difference indicates the motion and determines the new direction for the object of interest. Relatively similar approach is followed in template matching which is described in detail in chapter 3.In template matching the correlation is taken between the template and every up coming frame in the sequence which gives the similarity ratio in the form of a single point which is then resolved by plummeting the components around it to make it template for the next frame. Correlation value differs between -1 to 1 so whenever correlation methods is used in tracking some threshold is set by the .This threshold servers as a reference as how much the images of previous ad current frame match each other or differ from each other which finally determines the motion of the object if interest.

3.3.2 Feature - based Method s

In feature-based object detection, standardization of image features and registration (alignment) of reference points are important. The images may need to be transformed to another space for handling changes in illumination, orientation and size. One or more features are extracted and the objects of interest are modeled in terms of these features. Object detection and recognition then can be transformed in to a graph - matching problem. All the methods we have already presented were using features, but they can be qualified of trivial features. This method considers more sophisticated ones. We consider: The moments computed on the image The contour li nes extracted from the image Color transition