SEN4013 LECTURE NOTES, Lecture notes of Software Engineering

Download SEN4013 LECTURE NOTES and more Lecture notes Software Engineering in PDF only on Docsity!

SEN

Software Verification and

Validation

Lecture 3 Basic Principles

Learning Objectives

• (^) Understand the basic principles

underlying A&T techniques

• (^) Grasp the motivations and applicability of

the main principles

Sensitivity

• (^) Human developers make errors,

producing faults in software.

• (^) Faults may lead to failures, but faulty

software may not fail on every execution.

• (^) The sensitivity principle states that it is

better to fail every time than sometimes.

Sensitivity

• (^) Consider the cost of detecting and repairing

a software fault.

• (^) If it is detected immediately, then the cost of correction is very small.
• (^) If a fault is detected in inspection or unit testing, the cost is still relatively small.
• (^) If a fault survives initial detection efforts at the unit level, but triggers a failure detected in integration testing, the cost of correction is much greater.
• (^) If the first failure is detected in system or acceptance testing, the cost is very high.
• (^) The most costly faults are those detected by customers in the field.

Sensitivity Example

• (^) Three faulty calls to string copy procedures.
• (^) The call to strcpy, strncpy, and stringCopy all pass a source string “Muddled,” which is too long to fit in the array middle.
• (^) The vulnerability of strcpy is well known, and is the culprit in the by- now-standard buffer overflow attacks on many network services.
• (^) Unfortunately, the fault may or may not cause an observable failure depending on the arrangement of memory (in this case, it depends on what appears in the position that would be middle[7], which will be overwritten with a newline character).

Sensitivity Example

• (^) The standard recommendation is to use strncpy in place of strcpy.
• (^) While strncpy avoids overwriting other memory, it truncates the input without warning, and sometimes without properly null- terminating the output.
• (^) The replacement function stringCopy, on the other hand, uses an assertion to ensure that, if the target string is too long, the program always fails in an observable manner.

Sensitivity Example

• (^) Replacing strcpy and strncpy with

stringCopy in the program is a simple

example of application of the sensitivity

principle in design.

• (^) Run-time array bounds checking in many

programming languages (including Java but

not C or C++) is an example of the

sensitivity principle applied at the language

level.

• (^) A variety of tools and replacements for the

standard memory management library are

available to enhance sensitivity to memory

allocation and reference faults in C and C+

Sensitivity

• (^) Always privilege design and code solutions

that lead to consistent behavior, that is,

such that fault occurrence does not depend

on uncontrolled execution conditions that

may mask faults, thus resulting in random

failures.

• (^) The sensitivity principle can also be applied

to test and analysis techniques.

• (^) In this case, we privilege techniques that

cause faults to consistently manifest in

failures.

Sensitivity

• (^) A deadlock is when two (or more) threads are blocking each other. Usually this has something to do with threads trying to acquire shared resources.
• (^) Race conditions occur when two threads interact in a negative (causing bugs) way depending on the exact order that their different instructions are executed.

Sensitivity

• (^) Testing a concurrent system on a single

configuration may fail to reveal

deadlocks and race conditions.

• (^) Repeating the tests with different

configurations and system loads may

help, but it is difficult to predict or control

the circumstances under which failure

occurs.

Sensitivity

• (^) Test adequacy criteria identify partitions of the input domain of the unit under test that must be sampled by test suites.
• (^) For example, the statement coverage criterion requires each statement to be exercised at least once, that is, it groups inputs according to the statements they execute.
• (^) Reliable criteria require that inputs belonging to the same class produce the same test results: They all fail, or they all succeed.
• (^) When this happens, we can infer the correctness of a program with respect to a whole class of inputs from a single execution.
• (^) Unfortunately, general reliable criteria do not exist.

Sensitivity

• (^) Code inspection can reveal many subtle faults.
• (^) However, inspection teams may produce completely different results depending on the cohesion of the team, the discipline of the inspectors, and their knowledge of the application domain and the design technique.
• (^) The use of detailed checklists and a disciplined review process may reduce the influence of external factors, such as teamwork attitude, inspectors’ discipline, and domain knowledge, thus increasing the predictability of the results of inspection.
• (^) In this case, sensitivity is applied to reduce the influence of external factors.

Sensitivity: Better to fail

every time than

sometimes

• (^) Consistency helps:
  • (^) A test selection criterion works better if every selected test provides the same result, i.e., if the program fails with one of the selected tests, it fails with all of them (reliable criteria)
  • (^) Run time deadlock analysis works better if it is machine independent, i.e., if the program deadlocks when analyzed on one machine, it deadlocks on every machine

Redundancy

• (^) Redundancy is the opposite of

independence.

• (^) If one part of a software artifact

(program, design document, etc.)

constrains the content of another, then

they are not entirely independent, and it

is possible to check them for consistency.