Scanner Compiler construction, Schemes and Mind Maps of Computer science

Download Scanner Compiler construction and more Schemes and Mind Maps Computer science in PDF only on Docsity!

Experiment in compiler 1

Experiment in

Compiler Construction

Scanner design

School of Infomation and Communication

Technology

Hanoi University of Science and

Technology

Experiment in compiler 2

What is a scanner?

 The compiler’s

component/module

that perform the job of

lexical analysis

(scanning) is called

scanner.

 Compiler’s first phase

Lexical

Analysis

Syntax

Analysis

Semantic

Analysis

Code

Generation

Experiment in compiler 4

Tasks of a scanner

 Skip meaningless characters: blank, tab, new

line character, comment.

 Recognize illegal character and return error

message

 Recognize different types of token

• identifier
• keyword
• number
• special character
• ...

Experiment in compiler 5

Tasks of a scanner

 Recognize tokens of different types

• identifier
• keyword
• number
• special character
• ...

 Pass recognized tokens to the parser (the

module that perform the job of syntatic

analysis)

Experiment in compiler 7

KPL’s tokens

 Keywords

PROGRAM, CONST, TYPE, VAR, PROCEDURE, FUNCTION, BEGIN, END, ARRAY, OF, INTEGER, CHAR, CALL, IF, THEN, ELSE, WHILE, DO, FOR, TO

 Operators

:= (assign)

• (addition), - (subtraction), * (multiplication), / (division)

= (comparison of equality), != (comparison of difference),

(comparison of greaterness), < (comparison of lessness), >= (comparison of greaterness or equality), <= (comparison of lessness or equality)

Experiment in compiler 8

KPL’s tokens

 Special characters

; (semicolon),. (period), : (colon), , (comma), (

(left parenthesis), ) (right parenthesis), ‘

(singlequote)

 Also

(. and .) to mark the index of an array

element

(* and *) to mark the comment

 Others

identifier, number, illegal charater

Experiment in compiler 10

Recognizing

KPL’s

tokens

 After every token is

recognized, the

scanner starts in

state 0 again

 If an illegal

character is met,

the scanner would

change to the state

-1 which tell the

scanner to stop

scanning and

return error

messages.

Experiment in compiler 11

KPL scanner - organization

# Filename Task

1 Makefile Project

2 scanner.c Main

3 reader.h, reader.c Read the source code

4 charcode.h, charcode.c

Classify character

5 token.h, token.c Classify and recognize token, keywords

6 error.h, error.c Manage error types and messages

Experiment in compiler 13

KPL scanner – charcode

typedef enum { CHAR_SPACE, // space CHAR_LETTER, // character CHAR_DIGIT, // digit CHAR_PLUS, // ‘+’ CHAR_MINUS, // ‘-’ CHAR_TIMES, // ‘*’ CHAR_SLASH, // ‘/’ CHAR_LT, // ‘<‘ CHAR_GT, // ‘>‘ CHAR_EXCLAIMATION, // ‘!’ CHAR_EQ, // ‘=‘ CHAR_COMMA, // ‘,’ CHAR_PERIOD, // ‘.’ CHAR_COLON, // ‘:’ CHAR_SEMICOLON, // ‘;’ CHAR_SINGLEQUOTE, // ‘\’’ CHAR_LPAR, // ‘(‘ CHAR_RPAR, // ‘)’ CHAR_UNKNOWN // invalid character } CharCode;

Experiment in compiler 14

KPL scanner – charcode

 In charcode.c , we define charCodes array that

associates every ASCII character with an

unique predifined CharCode.

 getc () function may return EOF (or -1) which is

not an ASCII character.

Experiment in compiler 16

KPL scanner – token

// Structure of a token typedef struct { char string[MAX_IDENT_LEN + 1]; int lineNo, colNo; TokenType tokenType; int value; } Token;

// Check whether a string is a keyword or not TokenType checkKeyword(char *string);

// Create new token, provided type of token and location Token* makeToken(TokenType tokenType, int lineNo, int colNo);

Experiment in compiler 17

KPL scanner – error management

// List of error may occur in lexical analysis typedef enum { ERR_ENDOFCOMMENT, ERR_IDENTTOOLONG, ERR_INVALIDCHARCONSTANT, ERR_INVALIDSYMBOL } ErrorCode;

// Error message #define ERM_ENDOFCOMMENT "End of comment expected!" #define ERM_IDENTTOOLONG "Identification too long!" #define ERM_INVALIDCHARCONSTANT "Invalid const char!" #define ERM_INVALIDSYMBOL "Invalid symbol!"

// Return error message void error(ErrorCode err, int lineNo, int colNo);

Experiment in compiler 19

Assignment

 Complete following function in scanner.c

• void skipBlank();
• void skipComment();
• Token readIdentKeyword(void);*
• Token readNumber(void);*
• Token readConstChar(void);*
• Token getToken(void);*

getToken() (1)

 Program  getToken()  TokenType: token

|- digit readNumber() |- letter readIdentKeyword() |- blank skipBlank() | getToken(); |- ( |-. SB_LSEL |- * skipComment() | getToken(); |- other SB_LPAR |- ‘ readConstChar() |- < |- = SB_LE |- other SB_LT