SEN4013 LECTURE NOTES, Lecture notes of Software Engineering

Download SEN4013 LECTURE NOTES and more Lecture notes Software Engineering in PDF only on Docsity!

SEN

Software Verification and

Validation

Lecture 2 A Framework for Test and Analysis

Learning Objectives

• (^) Introduce dimensions and tradeoff

between test and analysis activities

• (^) Distinguish validation from verification

activities

• (^) Understand limitations and possibilities of

test and analysis

Software Test

• (^) No perfect test or analysis techniques.
• (^) Not a single “best” technique for all

circumstances.

• (^) Testing techniques
  • (^) exist in a complex space of trade-offs
  • (^) have complementary strengths and weaknesses.

First Computer Bug

• (^) In 1947 Harvard University was operating a room-sized computer called the Mark II. - (^) Mechanical relays - (^) Glowing vacuum tubes - (^) Technicians program the computer by reconfiguring it - (^) Technicians had to change the occasional vacuum tube.
• (^) A moth flew into the computer and was zapped by the high voltage when it landed on a relay.
• (^) Hence, the first computer bug!

Bugs – Errors – Faults

• AT&T (1990)

• (^) One switching system in New York City experienced an intermittent failure that caused a major service outage
• (^) The first major network problem in AT&T’s 114-year history
• (^) Cause: Wrong BREAK statement in C Code
• (^) Complete code coverage could have revealed this bug during testing

Bugs – Errors – Faults

• (^) Ariane 5 flight 501 (1996)
  • (^) Destroyed 37 seconds after launch (cost: $370M)
• (^) Cause: Arithmetic overflow
  • (^) Data conversion from a 64-bit floating point to 16-bit signed integer value caused an exception
  • (^) The software from Ariane 4 was re-used for Ariane 5 without retesting

Bugs – Errors – Faults

• (^) THERAC-25 Radiation Therapy (1985)
  • (^) 3 cancer patients received fatal overdoses
• (^) Cause:
  • (^) Miss-handling of race condition of the software in the equipment

Software Failure, Fault &

Error

• (^) Fault
  • (^) Incorrect portions of code.
    • (^) may involve missing code as well as incorrect code
  • (^) Necessary (not sufficient) condition for the occurrence of a failure.
• (^) Failure
  • (^) Observable incorrect behavior of a program.
• (^) Error
  • (^) Cause of a fault. something bad a programmer did (conceptual, typo, etc.).
• (^) Bug
  • (^) Informal term for fault/failure.

Approaches to Reduce

Faults

• (^) Manual code review

Approaches to Reduce

Faults

16

Approaches to Reduce

Faults

• (^) Testing: Feed input to software and run it to

see whether its behavior is as expected

• (^) Limitations: (1) Impossible to cover all

possible execution, (2) Need test oracles

“Testing can show the presence of faults, but not their

Approaches to Reduce

Faults

• (^) Formal Verification: Consider all the possible

program executions, and formally prove that

the program is correct or not

• (^) Limitations: (1) Difficult to have a formal

specification, (2) Most real-world programs

are too expensive to prove

Validation

• (^) Software meeting the user’s real needs???
• (^) Fulfilling requirements is not the same as conforming to a requirements specification.
• (^) A specification is a statement about a particular proposed solution to a problem, and that proposed solution may or may not achieve its goals.
• (^) Specifications are written by people, and therefore contain mistakes.
• (^) A system that meets its actual goals is useful, while a system that is consistent with its specification is dependable.

Verification

• (^) Checking the consistency of an

implementation with a specification.

• Verification is a check of consistency

between implementation and

specification, in contrast to validation

which compares a description (whether a

requirements specification, a design, or a

running system) against actual needs.