ECE 395 - Advanced Digital Projects Lab | ECE 395, Study Guides, Projects, Research of Electrical and Electronics Engineering

Download ECE 395 - Advanced Digital Projects Lab | ECE 395 and more Study Guides, Projects, Research Electrical and Electronics Engineering in PDF only on Docsity!

ECE 395 Project: Vision Bill Healey Jon Johnson Ziyi Zhang Introduction

Our project for ECE 395 was in the field of computer vision or sending information for a computer to analyze and return with results. Most of the project was done with C++ which allowed us to make the filter file “filter.ax” and a color palette to select the range of colors to highlight and allow a vehicle to follow. The purpose of this project is to make a vehicle with a mounted video camera follow a red ball. The first step requires the user to pick a color, which was originally red but later added to be any color, and the vehicle will keep turning until it sees that color. Once the computer decides that there is significant number of pixels of that certain color, it tries to find the center of that block of color as well as a radius that would best fit the image. Next the computer will decide where the vehicle should move to close the distance between it and the target. The computer does this by sending code to another computer that uses its parallel port to control the remote control. A major component we used in this project was DirectDraw, or more specifically DirectShow, which deals with the actual filter and calculation with the pixels. We programmed our filter using Visual Studio that communicates with various functions of DirectShow. Our filter computed bit-wise operations such as checking the thresholds, assigning groups, and blurring at some point. Another program that we used to actually connect our filter into use was GraphEdit. The GraphEdit program allowed us to implement various filters visually. It allows us to choose a source whether it be a video file or capturing streaming video, puts it through any filters that we would want and exports it to screen. This way we can see the original video that the camera sees as well as the filtered video that has the color highlights. Our filter was called Vision Filter, and the program has an interface as such:

To simply specifying the range of colors we would like to use as highlight and basically set the threshold, we converted the normal RGB axis into hue, saturation, and luminosity. That way we can adjust to the lighting and environment much easier and also change our target color just by adjusting the hue. However, since the functions in C only used RGB values, we had to convert back after specifying the parameters to make the proper calculations. Eventually we programmed a palette that would allow the user to select a range of hues, saturations, and luminosities. The hue and saturation combine into a circle the ring of the circle is the hue and the saturation increases as it gets closer to the center of the circle. We also have a side bar for the range of luminosity selection. So this transformed our original idea of following a red ball into following any high saturation colored ball the user desires. Another obstacle that we faced was to determine the distance the vehicle is from the target. We thought of using two cameras to perceive depth by comparing the inputs of the cameras. Implementation of such a comparison would be very complicated since we would have to do shape recognition on each image and map it onto the other image to define an object. We thought of an easier way to judge distance; since all we care about is the distance from the target ball, we can just calculate the radius of the target color region after we specify a center. Finding the center was also not an easy task because there are numerous ways of estimating the center. One way we thought of doing was to add weight onto each pixel. If at first we see a red pixel, we assign it a 1, if the pixel adjacent to it is also red, assign it a weight of 2, and if the next pixel is red, assign it 4 and so on. By using this method, we would have a massive peak on areas with the most red and noise would seem insignificant. To do this properly, we would have to calculate each pixel from left to right as well as from up to down. But then that would only cause the highest weight at the rightmost and bottommost pixel but not the center. So to find the center, we would have to repeat the process from right to left and down to up. We concluded

that this process would require too many calculations. The idea of simply average all the red pixels’ row and column seemed too crude and inaccurate, however when we tried it out, the process resulted with surprising accuracy. As we established a center for the ball, we had to know the distance, which can be judged by the radius of the image of the ball. The radius is determined by the average number of red pixels in all four directions from the center. We drew the center with a white pixel as well as four points to signify the approximate radius. Making communications work between the computer that processes the images and filtering was a vital process. We had two computers to do this; one had the video capture card and the other has the socket output to parallel port which controls the remote to the vehicle. Both computers are on the LAN and the computer that has to communicate to the parallel port had to have a socket. The connection is opened whenever we start up the filtering program and closes whenever that program closes. That computer sends bits to the parallel port, which sends pulses to the remote control to the vehicle to give it instructions. The bits that the computer sends out have a total of eight bits. The first four bits are speed bits, which control the speed of the vehicle. Since the vehicle doesn’t have variable velocity, we had to implement a pulse every time period. A faster speed means a faster pulse to move the vehicle. The last four bits instructs the vehicle where to go. Bit 3 and bit 2 control the left wheel movements (backwards, forwards). Bit 1 and bit 0 control the right wheel movements. For example, if we wanted the vehicle to move at speed 5, and turn left, we would send 0101 1001, which means that the speed is 5, the left wheels moves backwards while the right wheels move forward. So every time the computer sends an instruction, it controls both the speed and the direction of the vehicle. Our initial vehicle was a plain remote control car. The trouble we had with that was the turning radius with that vehicle was incredibly wide. If our target moved slightly too fast at too sharp of an angle, the car would not be able to make the turn to catch the target. Later

the description of the voltage and current that it needed but somehow it didn’t work unless we plugged it into the wall. Since our project is based on wireless systems, we had to abandon that digital camera. Our second camera was a webcam; it was smaller than the previous camera and there was no power problem. The only tradeoff is that the smaller camera didn’t have the high quality as the original camera had. The images were darker and with less saturation. We had to adjust our range of saturation and luminosity to compensate for the problem. When the camera was on for too long it would heat up and the images sent through would be full of noise. At times the color would be gone, but that is a problem that we cannot fix. So usually we turn off the camera when it’s not being used to prevent any bad signals from transmitted. Next we had the problem of what to do when the ball moves out of the view of the vehicle’s camera. Our solution was to save the last movement command it executed, or rather just the direction. So if the ball was to its left when it last disappeared, the vehicle would turn left and in place to look for the ball. If the ball was last seen right, it would move right. When the ball was last seen straight, we didn’t want the vehicle to just go straight since it’s obviously not in front of it so we assigned it to go right (pick left would not have made a difference). One other major obstacle that we came upon was the trade off of quality signal detection and the processing power for the images. When we first implemented the connected sets for grouping of pixels, the vehicle would first follow the target correctly but as the target turned, it reacted to it much later than it should. Therefore it would over shoot the target and when it tries to compensate, it will do so again in the other direction. The calculation was taking up too much processing power while the frames were being queued to be processed. We came up with several methods to deal with this challenge. One simple option was to drop frames; since the camera was sending so many frames per second, we could afford to skip a few frames and still have the vehicle operate correctly. That reduced the processing

speed dramatically. However the time delay was still a problem when we increase the speed of the vehicle. Therefore we decided to use the connected sets only when the ball is far away or when there’s not too many highlighted pixels. So when the ball is up close, there are more red pixels on the screen and the computer would have to process more pixels. We set it up so that if at least 1/4 of the screen is filled with red, we would not use the connected sets process. It worked because if the ball is close to the camera, noise will matter less because the camera would pick up huge chunks of red from the ball. When the ball is far from the camera, the circle is small and noise contributes to more, that’s when we need to refine the filtering by using connected sets. After implementing the decision of when to use connected sets, the vehicle worked perfectly. Unfortunately the screenshots of the filter in action cannot be made due to complications of the video mode that would leave the video window complete black and not as the frame in video. However we did demonstrate the working model during open house demonstrations, and everything worked well.