Syntax Testing: Understanding Input Validation and BNF for Dependable Software Systems - P, Exams of Computer Science

Download Syntax Testing: Understanding Input Validation and BNF for Dependable Software Systems - P and more Exams Computer Science in PDF only on Docsity!

Dependable Software Systems

Topics in

Syntax Testing

Material drawn from [Beizer]

Syntax Testing

• System inputs must be validated. Internal

and external inputs conform to formats:

• Textual format of data input from users.
• File formats.
• Database schemata.
• Data formats can be mechanically converted

into many input data validation tests.

• Such a conversion is easy when the input is

expressed in a formal notation such as BNF

(Backus-Naur Form).

Million Monkey Phenomenon

• A million monkeys sit at a million

typewriters for a million years and

eventually one of them will type Hamlet!

• Input validation is the first line of defense

against a hostile world.

Input-Tolerance Testing

• Good user interface designers design their

systems so that it just doesn’t accept

garbage.

• Good testers subject systems to the most

creative “garbage” possible.

• Input-tolerance testing is usually done as

part of system testing and usually by

independent testers.

Automation is Necessary

• Test execution automation is essential for

syntax testing because this method produces

a large number of tests.

How to Find the Syntax

• Every input has a syntax.
• The syntax may be:
  • formally specified
  • undocumented
  • just understood
• … but it does exist!
• Testers need a formal specification to test

the syntax and create useful “garbage”.

BNF Example

• Correct phone numbers:
  • 3469900, 9904567, 3300000
• Incorrect phone numbers:
  • 0551212, 123, 8, ABCDEFG special_digit ::= 0 | 1 | 2 | 5 other_digit ::= 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 ordinary_digit ::= special_digit | other_digit exchange_part ::= ordinary_digit number_part ::= phone_number ::= exchange_part number_part 2 other _ digit 4 ordinary _ digit

Why BNF?

• Using a BNF specification is an easy way to

design format-validation test cases.

• It is also an easy way for designers to

organize their work.

• You should not begin to design tests until

you are able to distinguish incorrect data

from correct data.

Testing Strategy

• Create one error at a time, while keeping all

other components of the input string correct.

• Once a complete set of tests has been

specified for single errors, do the same for

double errors, then triple, errors, ...

• Focus on one level at a time and keep the

level above and below as correct as you can.

Example: Telephone Number

(Level 1)

• phone_number ::= exchange_part number_part
  • Empty string.
  • An exchange_part by itself.
  • Two from exchange_part.
  • Two from number_part.
  • One from exchange_part and two from number_part.
  • Two from exchange_part and one from number_part.
  • ...

Example: Telephone Number

(Level 2)

• Bad number_part :
• number_part ::= ordinary_digit
  • Not enough from ordinary_digit.
  • Too many from ordinary_digit.
  • ... 4

Example: Telephone Number

(Level 3)

• ordinary_digit ::= special_digit | other_digit
  • Not a digit - alphabetic.
  • Not a digit - control character.
  • Not a digit - delimiter.
  • ...

Delimiter Errors

• Delimiters are characters or strings placed

between two fields to denote where one

ends and the other begins.

• Delimiter Problems:
  • Missing delimiter. e.g., (x+y
  • Wrong delimiter. e.g., (x+y]
  • Not a delimiter. e.g., (x+y 1
  • Poorly matched delimiters. e.g., (x+y))

Sources of Syntax

• Designer-Tester Cooperation
• Manuals
• Help Screens
• Design Documents
• Prototypes
• Programmer Interviews
• Experimental (hacking)