Eye Controlled Typing - Senior Design Project Lab | ECE 445, Study Guides, Projects, Research of Electrical and Electronics Engineering

Download Eye Controlled Typing - Senior Design Project Lab | ECE 445 and more Study Guides, Projects, Research Electrical and Electronics Engineering in PDF only on Docsity!

EYE CONTROLLED TYPING

By Xiaoyu Ma Hung-Yuan Wu ECE 445, SENIOR DESIGN PROJECT SPRING 2006 TA: Dwayne Hagerman 5-1- Project 37

ABSTRACT

Our design was to create a typing tool that is controlled by movements of the eyes. This product is intended for use by people who are paralyzed and are unable to communicate otherwise. The basis for this design is the cornea-retinal potential of the eyes. After this potential is sensed by electrodes, the signals are amplified, filtered, shifted, and sent into the PIC microcontroller. The PIC converts the analog signal to digital while interpreting the voltage levels before sending it into the computer through RS232 serial communication. Once inside the computer, keys from a keypad can be selected by looking at different directions. In the end, our design worked but was not as reliable and consistent as we would have liked. Further improvements can be made by using a better method for electrode adhesion and adding in auto-calibration. The cost for this design is fairly low, allowing the product to be very affordable. ii

• 1. INTRODUCTION....................................................................................................................
  • 1.1 Purpose...............................................................................................................................
  • 1.2 Specifications......................................................................................................................
  • 1.3 Subprojects......................................................................................................................... - 1.3.1 High Gain Amplifier..................................................................................................
    • 1.3.2 Low Pass Filter..........................................................................................................
      • 1.3.3 Summing Amplifier...................................................................................................
      • 1.3.4 Calibration Indicator..................................................................................................
    • 1.3.5 Buffer Circuit.............................................................................................................
    • 1.3.6 Power Supply.............................................................................................................
    • 1.3.8 Software User Interface.............................................................................................
• 2. DESIGN PROCEDURE...........................................................................................................
  • 2.1 Design Decisions................................................................................................................
  • 2.2 Tools Used..........................................................................................................................
• 3. DESIGN DETAILS..................................................................................................................
  • 3.1 High Gain Amplifier...........................................................................................................
  • 3.2 Low Pass Filter...................................................................................................................
  • 3.3 Summing Amplifier............................................................................................................
  • 3.4 Calibration Indicator...........................................................................................................
  • 3.5 Power Supply Module........................................................................................................
  • 3.6 PIC Microcontroller............................................................................................................
  • 3.7 Software User Interface......................................................................................................
• 4. DESIGN VERIFICATION.......................................................................................................
  • 4.1 Testing................................................................................................................................ - 4.1.1 High Gain Amplifier.................................................................................................. - 4.1.2 Summing Amplifier................................................................................................... - 4.1.3 Power Supply............................................................................................................. - 4.1.4 PIC Microcontroller................................................................................................... - 4.1.5 Software User Interface.............................................................................................
  • 4.2 Conclusions.........................................................................................................................
• 5. COST......................................................................................................................................
  • 5.1 Parts..................................................................................................................................
  • 5.2 Labor.................................................................................................................................
• 6. CONCLUSIONS....................................................................................................................
  • REFERENCES.......................................................................................................................
• APPENDIX A– BLOCK DIAGRAMS.................................................................................A
• APPENDIX B - SCHEMATICS...........................................................................................A
• APPENDIX C - PICTURES..................................................................................................A
• APPENDIX D – PIC CODE..................................................................................................A
• APPENDIX E – SOFTWARE CODE.................................................................................A

1. INTRODUCTION

1.1 Purpose People suffering from various kinds of disease such as Lou Gehrig's, muscular distrophy, or pseudocoma are unable to communicate because most of their voluntary muscles are inhibited. However, they are still conscious and have clear thought. Often, the only muscles that are still under voluntary control are those responsible for eye movements. The eye has a natural electrical potential difference between the cornea and the retina where the nerve cells are located. The rods and cones in the retina will release negatively charged ions when stimulated by light. This difference can be measured by electrodes placed on opposite sides of the eyes. Depending on the polarity of the voltage measured, the direction in which the eye is looking at can be determined. Although there are technologies out there in the market that provide paralyzed people a method to express their thoughts on the computer through their eyes, it is too expensive to be wide spread. Therefore we would like to develop a cheap and easy to use typing tool that is controlled by movements of the eye. This would provide these paralyzed people a way to communicate with people freely while not spending a fortune at the same time. 1.2 Specifications The weak electro-oculogram(EOG) signals need to be amplified from the 1mV range to 1V range. It also need to be adjusted to the range of 0V to 5V so it can be used as input to the PIC. After input into the PIC, the PIC need to distinguish looking at different directions as well as blinking based on voltage values. A left/down should be interpreted if the voltage level is from 0V to 1V. From 1V to 2V, the user should be looking straight. From 2V to 3V, a right/up signal is interpreted. Beyond 3V from the vertical electrodes input, a blink is interpreted. Finally, the software interface has to provide a non-standard keyboard layout to make selection and typing keys easier and faster. A display window above the keypad will show what has been typed. In addition, there will be an option to convert what is typed into speech. 1.3 Subprojects Both the hardware and software components were divided into many subcomponents. The hardware component includes a high gain amplifier, low pass filter, summing amplifier, calibration indicator, buffer circuit, and a power supply module. The software component includes the PIC, MAX232 circuit, and software user interface. 1.3.1 High Gain Amplifier An amplifier with a gain of 1000 is needed to amplify the low EOG signals from about 1mV to 1V. 1.3.2 Low Pass Filter Since the frequency of the EOG signal is usually below 25Hz, frequencies greater than 25Hz can be filtered out. This will block out noise from the surrounding as well as noise from power cables and instruments in the lab. 1.3.3 Summing Amplifier

2. DESIGN PROCEDURE

2.1 Design Decisions The main amplification was achieved by the AD622 instrumentation amplifier because of its high gain and ease of use. The AD622 chip contains a buffer circuit and a differential amplifier. This provides high input impedance, which ensures safety of the user. The gain is easily controlled by adjusting the value of a gain resistor. The remaining circuit all used the LM741 op-amp, which is cheap and easy to use. Originally a FPGA was under consideration for this project. The FPGA was supposed to take the two input voltages and convert it to useful data for the software interface. The FPGA was eventually replaced by a 16F877A PIC due to the fact that input voltages from the electrode need to be converted to digital signals first before it could be used by computer software. Since 16F877A had built in ADC functions, it became a better choice for this project. Another design decision that was necessary is the language the software user interface will be using. In the end, Visual Basics 6.0 was chosen over all the other languages because it is easy to program, easy to display visual elements, and easy to implement necessary functions such as serial port communication and text to speech. Keyboard layout is also a key decision in the project. A non-conventional keyboard layout was chosen in this project because a regular keyboard layout is not easy to type with by the movement of the eye. Instead, a cell phone keypad was selected to make typing with eye easier to do. 2.2 Tools Used In designing the hardware for this project, Cadence PSD was used to draw the schematics and run simulations. After the circuit was constructed, the function generator and Agilent 54642A oscilloscope were used to conduct testing on all the components. On the software side, PIC-C was used to write the code for the PIC and Visual Basic was used to program the user interface. A function generator and Hyperterminal were used to test the software components.

3. DESIGN DETAILS

3.1 High Gain Amplifier An instrumentation amplifier will be used to amplify the potential difference between two opposing electrodes. The instrumentation amplifier consists of two parts. First is a buffer circuit that provides high input impedance to reduce power from electrodes. The second part is a differential amplifier. Another advantage of using this is its ease of control over amplification. The amplification factor can be selected by adjusting the value of a single resistor. Since the strength of EOG signal is less than 1mV, a high gain amplifier is needed. The AD622 instrumentation amplifier will be used for its maximum gain of 1000. To achieve a certain gain on the AD622, the gain resistor needs to be set such that Rg = 50.5k/(G-1) (3.1.1) If we want to bring the EOG signal up to the range of volts, we need a gain of 1000 and Rg needs to be 51.1 ohms. 3.2 Low Pass Filter A low pass filter will be used to filter out any undesired noise. The filter will ensure that only the signals from the electrodes being measured and passed over to the other components. The typical EOG signal has frequencies ranging from 0Hz to 30Hz. All higher frequencies can be filtered. The cutoff frequency of a simple RC low pass filter is determined by the following equation:

 R C 

fo 2 * * 1   (3.2.1) If we chose to use a 1uF capacitor and a 5k resistor, frequencies higher than 32Hz will be blocked. 3.3 Summing Amplifier The voltage when the user looks straight has an offset that varies each time electrodes are hooked up. In order to pass the signal into the PIC for A/D conversion, the analog input voltage has to be between Vss and Vdd, which is from 0 to 5 volts. Therefore, the signal coming out of the amplifier and filter will need to be shifted into this workable range. The voltage output when looking straight will be shifted to 1.5V. This is to ensure the voltage when looking to the left or down will still be above 0 and the voltage when looking right, up, or blinking will be below 5V. The signal will be shifted by using a summing amplifier built from a LM741 op amp. The amplifier's non-inverting input will be tied to ground. The inverting input will be connected to both the input from electrodes as well as a voltage to be added that is determined using a potentiometer. Assuming V+ and V- of the op amp are equal and that no current goes into the two inputs, an equation relating Vin and Vout can be written: Vin/20k + Vr/33k = -Vout/33k (3.3.1) which rearranges to Vout = - (3.3Vin + Vr) (3.3.2)

impossible to determine how far off in a direction the user was looking at. With the cell phone keypad, there was only one key the user could select in a given direction. The software read from the serial port and used the data read o determine what to update on the user interface. Possible actions include backspace, space, clear text, text to speech, alphabet keypad and number keypad. When user looked in a certain direction, the key in that direction is automatically selected. Blinking is used to specify that the user “pressed” the key in the center on the screen. The text to speech engine required some thoughts. There were two text to speech engines available. One with a graphical display and other did not. The one with no graphical displays was the preferred one, because the graphical display made the user interface looked rather unprofessional. Unfortunately, the one without graphical display required the input text to be lower case in order to read out correctly. Since the user interface design had text display in all capital case, this was not a feasible option. In the end, the text to speech engine that had a graphical display was chosen. While using hyperterminal to simulate serial input to test the software, a flaw in the design was discovered. If there is an undesired spike in the input voltage, the software will not be able to filter it out, and interpret it as user having a movement in his/her eye. Also, when looking diagonal, there is the possibility that a vertical or horizontal movement is detected first, before the actual diagonal is detected. In order to solve both problems, a counter was placed in the program to count the serial inputs. If a certain direction input was not received for more than 50 consecutive times, it would be discard by the program. This method solved both problems mentioned, since neither a spike in the input nor intermediate directions in a diagonal scenario will be transmitted more than 50 consecutive times.

4. DESIGN VERIFICATION

4.1 Testing Individual testing was done on most of the components before integration. All the components must be functioning in order for the final product to be working. This also made debugging the circuit easier. 4.1.1 High Gain Amplifier A gain resistor was chosen so the EOG signal from electrodes will be amplified by a factor of 1000. To test this, a 1mVPP sine wave from a function generator was used as input into the instrumentation amplifier. The output was a 1VPP sine wave. This demonstrated the gain was indeed 1000. Figure 4.1.1 1mVPP sine wave input into amplifier Figure 4.1.2 1VPP sine wave output from amplifier

Figure 4.1.5 +5V output of power supply 4.1.4 PIC Microcontroller Testing of the PIC was fairly straightforward. Two function generators provided a sin wave function with 0.001V peak-to-peak value. The offset of these functions will act like the input voltage value for our PIC. The output of the serial port is connected to a computer with Hyperterminal running. The output of the PIC based on different voltage inputs will appear on the Hyperterminal. The PIC worked on our first try, and required no more testing after that. 4.1.5 Software User Interface The testing of the software interface could be break up into two parts. First part is to test if the software reacted correctly based on the direction the user was looking at. To test this, an extra textbox and command button were added to the software. A direction was manually entered into the textbox, and a push of the command button will make the software execute with the direction in the textbox as input. The testing went pretty smoothly, and not much debugging was required. The second part of the software test is to test serial port communication. Hyperterminal was used again to for this time. This time however, Hyperterminal was used to simulate serial input from the PIC. To do this test, two computers were used, along with a serial cable. One of the computers will have Hyperterminal open and transmit serial data representing direction of the eye to the other computer, which will have the software running. Debugging was performed based on the result of the software and the expected behavior. Since the major functions of the program were already tested in the previous part, once the serial communication started to work, the software was completely ready for integration with other modules. 4.2 Conclusions After the above tests were done, the circuit components were integrated together. The hardware component was then tested as a whole with electrodes sensing EOG signals as the inputs. After the voltage level when looking straight was set to 1.5V, the voltage level increased by 1V when looking to the right or up and decreased by 1V when looking left or down.

Figure 4.2 Response after looking up, right, down, and left. Top signal due to vertical electrodes, bottom signal due to horizontal electrodes. The software components on the other hand, were tested as whole together as well. Function generators were used once again to provide input for the PIC. The serial output of the PIC was connected to a computer with the software user interface running. The hardware components worked as expected. However, testing the design as a whole with all modules connected was a hard task. Although the hardware part worked nicely in the test performed, its performance is not very stable. Sometimes no signal could be read out from the electrodes. Also, since electrodes were not re-useable, and only limited numbers of electrodes were available, chances to test the design as a whole were very limited.

6. CONCLUSIONS

Individually, all the components were tested to function properly. The hardware circuit gave the correct output values. When the function generator was used as input to the PIC, the intended keys were typed into the software. However, when the design was integrated together, we encountered some problems. First, not enough signal was being transferred into the PIC. Therefore, we had to add in a buffer circuit to lower the output impedance. The major problem, however, was that the signals coming from the electrodes are not stable enough. The slightest movement will cause the voltage levels to fluctuate. As the electrodes starts to become loose, the signal also starts to shift up or down. This makes it difficult to use because the circuit is no longer calibrated. However, within a short period of time after the electrodes are placed around the eyes, there is still some degree of control when typing. Typing out words and sentences requires time and patience. As can be seen from the sample screenshot included in Fig. C.8 of the Appendix, ECE444 and ECE414 was typed out when ECE445 was intended. The other random letters displayed were result of small movements during typing. This shows our design is not very reliable but for our intended users who are paralyzed, movements besides eye motions are not possible. Even so, there are improvements we can make to our design to increase its reliability and consistency. Possible improvements that could be done in the future are including the use of goggles. The goggles can push the electrodes closer to our skin to guarantee better connection and signal. Also, the wires to the electrodes could be wired around the goggle to make the design look more clean and portable. Goggles are also a cheaper solution compare to re-useable electrodes for this project. Signal processing could also be done to account for the gradual drift in the input voltage to the PIC. As mentioned above, as electrodes lose its sensitivity, the input voltages tend to decrease slowly, and eventually out of range. Therefore, instead of using preset voltage range to determine the direction of the eye, average of the input voltages could be taken. When a dramatic spike is observed in the input voltage compare to the average, it could be interpreted as a change in eye movement. Finally, a better approach to the whole project would be to use a webcam to detect motion instead of using electrodes. Webcams could be setup to observe the pupil of the eye, and detect difference in the position of the pupil between frames to frames. A difference in the current frame and previous frame could be interpreted as a movement in the eye, and no pupil presented could be interpreted as a blink. This is a much better method compare to using electrodes. The problem with this approach is that it is quite a hard task to detect difference in frames with hardware circuit. FPGA will be a better choice for this task, which means special circuit will be required as well. The cost of the design will increase dramatically as well. But if we were to continue on with this project, we would look further into the FPGA solution compare to the electrode solution.

REFERENCES

[1] ECE 445 Course Website, University of Illinois at Urbana-Champaign, “ECE345PS.sch,” September 2003, http://courses.ece.uiuc.edu/ece445/documents/Power_Supply_Schematic.pdf. [2] ECE 445 Course Website, University of Illinois at Urbana-Champaign, “PIC Tutorial,” September 2003, http://courses.ece.uiuc.edu/ece445/wiki/?n=Topics.PicProgramming. [3] N.Matic, “Programming PIC MCUs in BASIC,” September 2004, http://courses.ece.uiuc.edu/ece445/wiki/?n=Topics.PicProgramming. [4] N. Shanbhag, “ECE 442 (Electronics Circuits)” class notes for ECE 442, Depart. Of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Fall 2005.