Testing the Software with Blinders On - Lecture Notes | SE 320, Assignments of Engineering

Download Testing the Software with Blinders On - Lecture Notes | SE 320 and more Assignments Engineering in PDF only on Docsity!

Testing the Software with

Blinders on

[Reading assignment: Chapter 5, pp. 63-79]

Dynamic black-box testing

• Dynamic black-box testing is testing without

having an insight into the details of the

underlying code.

• Dynamic, because the program is running
• Black-box, because testing is done without knowledge of how the program is implemented.
• Sometimes referred to as behavioral testing.
• Requires an executable program and a

specification (or at least a user manual).

• Test cases are formulated as a set of pairs
  • E.g., (input, expected output)

Test-to-pass and test-to-fail

• Test-to-pass:
  • assures that the software minimally works,
  • does not push the capabilities of the software,
  • applies simple and straightforward test cases,
  • does not try to “break” the program.
• Test-to-fail:
  • designing and running test cases with the sole purpose of breaking the software.
  • strategically chosen test cases to probe for common weaknesses in the software.

Discussion …

• Why should a tester always start with a

test-to-pass approach?

• Isn’t this a waste of time?

• What assurance does test-to-pass give

us?

• Shouldn’t the programmers (i.e., not the

testers) do test-to-fail?

Black-box testing

Equivalence

Partitioning

• Equivalence

partitioning is the

process of

methodically

reducing the huge

(or infinite) set of

possible test cases

into a small, but

equally effective,

set of test cases.

• Inputs which conform to the pre-

conditions.

• Inputs where a pre-condition does not

hold.

• Inputs where the key element is a

member of the array.

• Inputs where the key element is not a

member of the array.

Search routine input partitions

Search routine input partitions

Array Element Single value In array Single value Not in array More than 1 value First element in array More than 1 value Last element in array More than 1 value Middle element in array More than 1 value Not in array

Data Testing

• If you think of a program as a function, the

input of the program is its domain.

• Examples of program data are:
  • words typed into MS Word
  • numbers entered into Excel
  • picture displayed in Photoshop
  • the number of shots remaining in an arcade game
  • …

Boundary input data

• Boundary conditions are situations at the

edge of the planned operational limits of the

software.

• E.g., negative to zero to positive numbers, exceeding the input field length of a form, etc.
• Choose input data that lie on the boundary

when formulating equivalence partitions.

• Test the valid data just inside the boundary
• Test the last possible valid data
• Test the invalid data just outside the boundary
• Security flaws such as buffer overflow attacks

exploit boundaries of array buffers.

Garbage-In Garbage-Out

• “Garbage-In equals Garbage-Out” is one of

the worst cop-outs ever invented by the

computer industry.

• GI-GO does not explain anything except our

failure to:

• install good validation checks
• test the system’s tolerance for bad data.
• Systems that interface with the public must be

especially robust and consequently must

have prolific input-validation checks.

Million Monkey Phenomenon

• A million monkeys sit at a million

typewriters for a million years and

eventually one of them will type Hamlet!

• Input validation is the first line of

defense against a hostile world.

Syntax Testing Steps

• Identify the target language or format.
• Define the syntax of the language, formally, in

a notation such as BNF.

• Test and Debug the syntax:
  • Test the “normal” conditions by covering the BNF syntax graph of the input language. (minimum requirement)
  • Test the “garbage” conditions by testing the system against invalid data. (high payoff)

Automation is Necessary

• Test execution automation is essential

for syntax testing because this method

produces a large number of tests.