Face Emotion detection App, Lab Reports of Artificial Intelligence

Download Face Emotion detection App and more Lab Reports Artificial Intelligence in PDF only on Docsity!

i

Facial Expression Recognition

System

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

BANGLADESH UNIVERSITY OF BUSINESS & TECHNOLOGY (BUBT)

MIRPUR-2. DHAKA

ii

Facial Expression Recognition

System

Submitted to - Department of Computer Science and Engineering Bangladesh University of Business and Technology (BUBT), Dhaka In partial fulfillment of requirements For the Facial Expression App

Submitted By-

Abdur Rahim (ID: 19203203015)

Nur Islam (ID: 19203203017)

Fahmida Akhter (ID:19203203021)

Sumaiya Islam (ID: 19203203044)

Rasel Mia (ID: 19203203046)

Intake – 36 rd^ (sec 1)

Supervision By-

Mr. Md. Shahiduzzaman

Assistant Professor,

Department of Computer Science and Engineering (CSE)

Bangladesh University of Business and Technology (BUBT)

Mirpur-2, Dhaka-1216, Bangladesh

iv Supervisor’s Recommendation

I hereby recommend that this project work report is satisfactory in the partial

fulfillment for the requirement of Bachelor of Science in Computer Science

and Information Technology and be processed for the evaluation.

Mr. Md. Shahiduzzaman Department of Computer Science and Engineering (CSE) Assistant Professor (Supervisor) Date:

v LETTER OF APPROVAL This is to certify that the project prepared by Abdur Rahim (15), Nur Islam (17), Fahmida Akhter (21), Sumaiya Islam (44) and Rasel Mia (46) entitled “FACIAL EXPRESSION RECOGNITION SYSTEM” in partial fulfillment of the requirements for the degree of B.Sc. in Computer Science and Information Technology has been well studied. In our opinion it is satisfactory in the scope and quality as a project for the required degree. ………………............................ ………………........................... Bangladesh University of Business and Technology (BUBT) Mr. Md. Shahiduzzaman and Information Technology Department of Computer Science and Information Technology (Chairman) (Supervisor) ………………............................ ………………............................ (Internal Examiner) (External Examiner)

vii ABSTRACT

Facial Expression conveys non-verbal cues, which plays an important role in

interpersonal relations. The Facial Expression Recognition system is the

process of identifying the emotional state of a person. In this system captured

image is compared with the trained dataset available in database and then

emotional state of the image will be displayed.

This system is based on image processing and machine learning. For

designing a robust facial feature descriptor, we apply the Local Binary

Pattern. Local Binary Pattern (LBP)is a simple yet very efficient texture

operator which labels the pixels of an image by thresholding the neighborhood

of each pixel and considers the result as a binary number. The histogram will

be formed by using the operator label of LBP.

The recognition performance of the proposed method will be evaluated by

using the trained database with the help of Support Vector Machine.

Experimental results with prototypic expressions show the superiority of the

LBP descriptor against some well-known appearance-based feature

representation methods.

We evaluate two Model on the face detection and expression detection

dataset. The Precision, Recall and FscSore from the expression detection

dataset were 83.6142%,95.0822% and 88.9955% respectively and that of face

detection dataset were 91.8986%,98.3649%, 95.0218% respectively.

Experimental results demonstrate the competitive classification accuracy of

our Model.

viii List of Figures: Figure 1: Applying face detection with OpenCV Haar cascades................................................... 4 Figure 2: The seven expressions from one subject ................................................................ 4 Figure 3 : System Diagram ................................................................................................. 6 Figure 4 : Flowchart of Face Emotion ......................................................................................... 7 Figure 5 : Experimental Demonstration from Image File................................................................ 1 0 Figure 6 : Experimental Demonstration from Camera ...............................................................

• CHAPTER Table of Contents
  • 1. INTRODUCTION
       • 1.1.Motivation
       • 1.2.Problem Statement
       • 1.3.Objectives
       • 1.4.Scope and Applications
• CHAPTER
  • 2. REQUIREMENT ANALYSIS.................................................................................
       • 2.1.Planning
       • 2.2.Literature Reviews
       • 2.3.Data collection - 2.3.1. Pre-trained face detection model by Haar Cascade..................................... - 2.3.2 Pre-trained emotion detection model by ……………………………...
  • 2.4. Software Requirement Specification:
  • 2.4.1. Functional requirements:
  • 2.4.2. Non-Functional requirements:
    • 2.5. Software and Hardware Requirement.............................................................. - 2. 5 .1. Software Requirement - 2. 5 .2. Hardware Requirement
• CHAPTER
  • 3. PROJECT METHODOLOGY
  • 3.1.System Design x
  • 3.1.1. System Diagram……………………………………………………….
  • 3.1.2. System Flowchart………………………………………………………
  • 3.2.Phases in Facial Expression Recognition
  • 3.2.1. Image Acquisition
  • 3.2.2. Face Detection…………………………………………………………
  • 3.2.3. Image Pre-Processing……………………………………………………
  • 3.2.4. Feature Extraction…………………………………………………………
  • 3.2.5. Classification……………………………………………………………...
• CHAPTER
  • 4. DEVELOPMENT AND TESTING
  • 4.1.Implementation Tools………………………………………………………………
  • 4.1.1. Programming Language and Coding Tools ………………………………….….
  • 4.1.2. Library …………………………………………………………………………...
• CHAPTER
  • 5. EXPERIMENTATION AND RESULTS
• CHAPTER 6 ……………………………………………………………………………1
  • 6. EXPERIMENTAL DEMONSTRATION …….…………………………………………….
  • 6.1. Experimental Demonstration from Image File …………………………………1
  • 6.2.Experimental Demonstration from Camera ………………………………………
• CHAPTER
  • 7. CONCLUSION AND RECOMMENTATION
  • 7.1. Conclusion …………………………………………………………………….
  • 7.2. Future Scope ……………………………………………………………………...

1

CHAPTER 1

1. INTRODUCTION

A Facial expression is the visible manifestation of the affective state, cognitive activity, intention, personality and psychopathology of a person and plays a communicative role in interpersonal relations. It have been studied for a long period of time and obtaining the progress recent decades. Though much progress has been made, recognizing facial expression with a high accuracy remains to be difficult due to the complexity and varieties of facial expressions. Generally human beings can convey intentions and emotions through nonverbal ways such as gestures, facial expressions and involuntary languages. This system can be significantly useful, nonverbal way for people to communicate with each other. The important thing is how fluently the system detects or extracts the facial expression from image. The system is growing attention because this could be widely used in many fields like lie detection, medical assessment and human computer interface. The Facial Action Coding System (FACS), which was proposed in 1978 by Ekman and refined in 2002, is a very popular facial expression analysis tool. On a day-to-day basics humans commonly recognize emotions by characteristic features, displayed as a part of a facial expression. For instance, happiness is undeniably associated with a smile or an upward movement of the corners of the lips. Similarly other emotions are characterized by other deformations typical to a particular expression. Research into automatic recognition of facial expressions addresses the problems surrounding there presentation and categorization of static or dynamic characteristics of these deformations of face pigmentation. The system classifies facial expression of the same person into the basic emotions namely anger, disgust, fear, happiness, sadness and surprise. The main purpose of this system inefficient interaction between human beings and machines using eye gaze, facial expressions, cognitive modeling etc. Here, detection and classification of facial.

3

CHAPTER 2

2.REQUIREMENT ANALYSIS

2.1. Planning In planning phase study of reliable and effective algorithms is done. On the other hand, data were collected and were preprocessed for more fine and accurate results. Since huge amount of data were needed for better accuracy, we have collected the data surfing the internet. Since, we are new to this project we have decided to use local binary pattern algorithm for feature extraction and support vector machine for training the dataset. We have decided to implement these algorithms by using OpenCV framework. 2.2. Literature Reviews Research in the fields of face detection and tracking has been very active and there is exhaustive literature available on the same. The major challenge that the researchers’ facies the non-availability of spontaneous expression data. Capturing spontaneous expressions on images and video is one of the biggest challenges ahead. Many attempts have been made to recognize facial expressions, facial feature representation and classifier problem. 2.3. Data collection Some of the public databases to evaluate the facial expression recognition algorithms are: 2.3.1. Pre-trained face detection model by Haar Cascade What is Haar Cascade, and how it works? Haar Cascade is a feature-based object detection algorithm to detect objects from images. A cascade function is trained on lots of positive and negative images for detection. The algorithm does not require extensive computation and can run in real-time. We can train our own cascade function for custom objects like animals, cars, bikes, etc. Haar Cascade can’t be used for face recognition since it only identifies the matching shape and size. Haar cascade uses the cascade function and cascading window. It tries to calculate features for every window and classify positive and negative. If the window could be a part of an object, then positive, else, negative. For more in-depth knowledge of its working,

4 Figure 1: Applying face detection with OpenCV Haar cascades. 2.3. 2. Pre-trained face detection model by keras model.h The .h5 file extension typically refers to a file format used to store a Keras model. Keras is a popular open-source deep learning library for Python that provides a high-level interface for building and training neural networks. When you build and train a Keras model, the resulting model is typically saved to a file with the .h5 extension. The .h5 file format is based on the Hierarchical Data Format version 5 (HDF5) and provides a way to store and organize large amounts of numerical data and metadata. The .h5 file contains the architecture of the neural network model as well as the trained weights and biases of the model. This allows the trained model to be easily reloaded and used for inference on new data without having to retrain the model. Figure 2: Applying emotion detection with mode.h

6

CHAPTER 3

3. PROJECT METHODOLOGY

3.1. System Design System design shows the overall design of system. In this section we discuss in detail the design aspects of the system: 3.1.1 System Diagram Figure 3: System Diagram.

7 3.1.2. System Flowchart Figure 4 : Flowchart of Face Emotion. 3.2. Phases in Facial Expression Recognition The facial expression recognition system is trained using supervised learning approach in which it takes images of different facial expressions. The system includes the training and testing phase followed by image acquisition, face detection, image preprocessing, feature extraction and classification. Face detection and feature extraction are carried out from face images and then classified into six classes belonging to six basic expressions which are outlined below: 3.2.1 Image Acquisition Images used for facial expression recognition are static images or image sequences. Images of face can be captured using camera.

9

CHAPTER 4

4. DEVELOPMENT AND TESTING

4.1. Implementation Tools 4.1.1. Programming Language and Coding Tools a) Python Python is a general-purpose, versatile, and powerful programming language. It’s a great first language because Python code is concise and easy to read. Whatever you want to do, python can do it. From web development to machine learning to data science, Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented and functional programming. b) Thonny IDE For Python Easy to get started. Thonny comes with Python 3.10 built in, so just one simple installer is needed and you're ready to learn programming. (You can also use a separate Python installation, if necessary.) The initial user interface is stripped of all features that may distract beginners. If you use small steps, then you can even see how Python evaluates your expressions. You can think of this light-blue box as a piece of paper where Python replaces subexpressions with their values, piece-by-piece. 4.1.2. Library a) OpenCV OpenCV (Open-Source Computer Vision Library) is an open-source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. The library has more than 2500 optimized algorithms, which includes a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3Dmodels of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high-resolution image of an entire scene, find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, recognize scenery. c) Keras Keras is an API designed for human beings, not machines. Keras follows best practices for reducing cognitive load: it offers consistent & simple APIs; it minimizes the number.

10

CHAPTER 5

5. EXPERIMENTATION AND RESULTS

The aim of this project work is to develop a complete facial expression recognition system. Two datasets, haar Cascade Classifier and Model.h5 were used for the experimentations. First of all, system was trained using different random samples in each dataset by supervised learning. In each datasets the data were partitioned into two parts for training and testing. Every dataset have completely different samples which are selected randomly in uniform manner from the pool of given dataset. The confusion and accuracy evaluation results of haar Cascade Classifier and Model.h5 datasets

CHAPTER 6

6 .Experimental Demonstration 6 .1 Experimental Demonstration from Image File Figure 5 : Experimental Demonstration from Image File