Software Testing: An Overview of Dependable Software Systems, Exams of Computer Science

Download Software Testing: An Overview of Dependable Software Systems and more Exams Computer Science in PDF only on Docsity!

Dependable Software Systems

Topics in

Software Testing

Material drawn from [Beizer, Sommerville]

Software Testing

• Software testing is a critical element of software quality assurance and represents the ultimate review of: - specification - design - coding
• Software life-cycle models ( e.g., waterfall) frequently include software testing as a separate phase that follows implementation!

Software Testing Terminology

• Error: A measure of the difference between the actual and the ideal.
• Fault: A condition that causes a system to fail in performing its required function.
• Failure: The inability of a system or component to perform a required function according to its specifications.
• Debugging: The activity by which faults are identified and rectified.

Software Testing Myths

• If we were really good at programming, there would be no bugs to catch. There are bugs because we are bad at what we do.
• Testing implies an admission of failure.
• Tedium of testing is a punishment for our mistakes.

Software Testing Reality

• Human beings make mistakes, especially when asked to create complex artifacts such as software systems.
• Studies show that even good programs have 1-3 bugs per 100 lines of code.
• People who claim that they write bug-free software probably haven’t programmed much.

Goals of Testing

• Discover and prevent bugs.
• The act of designing tests is one of the best bug preventers known. (Test, then code philosophy)
• The thinking that must be done to create a useful test can discover and eliminate bugs in all stages of software development.
• However, bugs will always slip by, as even our test designs will sometimes be buggy.

Testing Isn’t Everything

• Other methods for improving software reliability are: - Inspection methods: Walkthroughs, formal inspections, code reading. - Design style : Criteria used by programmers to define what they mean by a “good program”. - Static analysis: Compilers take over mundane tasks such as type checking. - Good Programming Languages and Tools: Can help reduce certain kinds of bugs ( e.g., Lint).

Testing Versus Debugging

• The purpose of testing is to show that a program has bugs.
• The purpose of debugging is to find the faults that led to the program’s failure and to design and implement the program changes that correct the faults.
• Testing is a demonstration of failure or apparent correctness.
• Debugging is a deductive process.

Function Versus Structure

• Functional Testing:
  • Program is treated as a black box.
  • Program is subjected to inputs, and its outputs are verified for conformance to specified behavior.
  • Implementation details do not matter.
  • Takes a user’s point of view.
  • In principle, can detect all bugs in an infinite amount of time.

Function Versus Structure (Cont’d)

• Structural Testing:
  • Aims at exercising the different control and data structures used in the program.
  • Criteria are precise as they are based on program structures ( i.e., are quite precise).
  • Looks at implementation details.
  • Takes a developer’s point of view.
  • Is inherently finite but cannot detect all faults.

Designer Versus Tester (Cont’d)

• The more the tester knows about the design, the more likely he will eliminate useless tests (functional differences handled by the same code).
• Testers that have design knowledge may have the same misconceptions as the designer.

Designer Versus Tester (Cont’d)

• Lack of design knowledge may help the tester to develop test cases that a designer would never have thought of.
• Lack of design knowledge may result in inefficient testing and blindness to missing functions and strange cases.

Programs and their Environment

• A program’s environment is the hardware and systems software required to make it run.
• Programmers should learn early in their careers that it is not smart to blame the environment for bugs.
• Bugs in the environment are rare because most bugs have been found over a long period of usage by a large number of users.

Myths About Bugs

• Benign Bug Hypothesis: Bugs are nice, tame, and logical.
• Bug Locality Hypothesis: A bug discovered within a component affects only that component’s behavior.
• Control Bug Dominance: Most bugs are in the control structure of programs.
• Corrections Abide: A corrected bug remains correct.