Download Slides on Software Testing - Software Engineering | CMSC 435 and more Study notes Software Engineering in PDF only on Docsity!
cmsc435 - 1
Software testing
Objectives
l To discuss the distinctions between validation
testing and defect testing
l To describe the principles of system and
component testing
l To describe strategies for generating system
test cases
l To understand the essential characteristics of
tool used for test automation
cmsc435 - 3
How does software fail?
l Wrong requirement: not what the
customer wants
l Missing requirement
l Requirement impossible to implement
l Faulty design
l Faulty code
l Improperly implemented design
Testing goals
l Fault identification: what fault caused
the failure?
l Fault correction: change the system
l Fault removal: take out the fault
cmsc435 - 7
Levels of testing
Programmers or independent testers
Black box; white box
Changed Any of the above documentation; High level design
Regression
Whole product in Black box Customer environment
Beta Ad hoc
Whole product in Black box Customer environment
Requirements Analysis
Acceptance
Independent tester
Whole product in Black box environment
Requirements Analysis
System
Independent tester
Functional High level design Whole product Black box
White box; black Programmer box
Low level design; Multiple classes High level design
Integration
Low-level design; Classes White box Programmer actual code
Unit
Testing type Specification General scope Opacity Who does it?
Fault type Meaning Function Fault that affects capability, end-user interfaces, product interfaces, interface with hardware architecture, or global data structure Interface Fault in interacting with other components or drivers via calls, macros, control blocks or parameter lists Checking Fault in program logic that fails to validate data and values properly before they are used Assignment Fault in data structure or code block initialization. Timing/serialization Fault that involves timing of shared and real-time resources Build/package/merge Fault that occurs because of problems in repositories, management changes, or version control Documentation Fault that affects publications and maintenance notes Algorithm Fault involving efficiency or correctness of algorithm or data structure but not design
IBM orthogonal defect classification
cmsc435 - 9
Typical inspection preparation
and meeting times
Development artifact Preparation time Meeting time Requirements document 25 pages per hour 12 pages per hour Functional specification 45 pages per hour 15 pages per hour Logic specification 50 pages per hour 20 pages per hour Source code 150 lines of code per hour 75 lines of code per hour User documents 35 pages per hour 20 pages per hour
- Faults found during discovery activities.
Discovery activity Faults found per thousand lines of code Requirements review 2. Design review 5. Code inspection 10. Integration test 3. Acceptance test 2.
Proving code correct
l Formal proof techniques
l Symbolic execution
l Automated theorem-proving
l Will discuss these next lecture
cmsc435 - 13
Test planning
l Establish test objectives
l Design test cases
l Write test cases
l Test test cases
l Execute tests
l Evaluate test results
The testing process
l Component (or unit) testing
Testing of individual program components;
Usually the responsibility of the component
developer (except sometimes for critical
systems);
Tests are derived from the developer’s
experience.
l System testing
Testing of groups of components integrated to
create a system or sub-system;
The responsibility of an independent testing team;
Tests are based on a system specification.
cmsc435 - 15
Testing process goals
l Validation testing
To demonstrate to the developer and the system customer that the software meets its requirements;
A successful test shows that the system operates as intended.
l Defect testing
To discover faults or defects in the software where its behavior is incorrect or not in conformance with its specification;
A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system.
Tests show the presence not the absence of defects
l Only exhaustive testing can show a program is
free from defects. However, exhaustive
testing is impossible
l Testing policies define the approach to be
used in selecting system tests:
All functions accessed through menus should be
tested;
Combinations of functions accessed through the
same menu should be tested;
Where user input is required, all functions must
be tested with correct and incorrect input.
Testing policies
cmsc435 - 19
(Non-)Testing
l Beta-testing – Relied on too heavily by large
vendors, like Microsoft.
Allow early adopters easy access to a new product
on the condition that they report errors to
vendor. A good way to stress test a new system.
Sample system
A
B C D
E F G
cmsc435 - 21
Bottom up testing
Test
E
Test
F
Test
G
Test
B,E,F
Test
C
Test
D,G
Test
A,B,C,D
E,F,G
Top down testing
Test
A
Test
A,B,C,D
Test
A,B,C,D,
E,F,G
cmsc435 - 25
Comparison of integration strategies
Bottom-up Top-down Modified top-down
Big-bang Sandwich Modified sandwich Integration Early Early Early Late Early Early Time to basic working program
Late Early Early Late Early Early
Component drivers needed
Yes No Yes Yes Yes Yes
Stubs needed
No Yes Yes Yes Yes Yes
Work parallelism at beginning
Medium Low Medium High Medium High
Ability to test particular paths
Easy Hard Easy Easy Medium Easy
Ability to plan and control sequence
Easy Hard Hard Easy Hard Hard
Incremental integration testing
Regression tests
cmsc435 - 27
Testing approaches
l Black box testing, also called functional
testing and behavioral testing, focuses on
determining whether or not a program does
what it is supposed to do based on its
functional requirements.
No knowledge of internal structure of code
l White-box testing is testing that takes into
account the internal mechanism of a system or
component. White-box testing is also known as
structural testing, clear box testing, and glass
box testing.
Knowledge of code
Testing guidelines
l Testing guidelines are hints for the testing
team to help them choose tests that will
reveal defects in the system
Choose inputs that force the system to generate
all error messages;
Design inputs that cause buffers to overflow;
Repeat the same input or input series several
times;
Force invalid outputs to be generated;
Force computation results to be too large or too
small.
cmsc435 - 31
Performance testing
l Part of release testing may involve testing the
emergent properties of a system, such as
performance and reliability.
l Performance tests usually involve planning a
series of tests where the load is steadily
increased until the system performance
becomes unacceptable.
Performance tests
l Stress tests
l Volume tests
l Configuration tests
l Compatibility tests
l Regression tests
l Security tests
l Timing tests
l Environmental tests
l Quality tests
l Recovery tests
l Maintenance tests
l Documentation tests
l Human factors
(usability) tests
cmsc435 - 33
Stress testing
l Exercises the system beyond its maximum
design load. Stressing the system often
causes defects to come to light.
l Stressing the system test failure behaviour.
Systems should not fail catastrophically.
Stress testing checks for unacceptable loss of
service or data.
l Stress testing is particularly relevant to
distributed systems that can exhibit severe
degradation as a network becomes overloaded.
l Objectives are to detect faults due to
interface errors or invalid assumptions about
interfaces.
l Particularly important for object-oriented
development as objects are defined by their
interfaces.
Interface testing
cmsc435 - 37
Test case design
l Involves designing the test cases (inputs and
outputs) used to test the system.
l The goal of test case design is to create a set
of tests that are effective in validation and
defect testing.
l Design approaches:
Requirements-based testing;
Partition testing;
Structural testing.
Requirements based testing
l A general principle of requirements
engineering is that requirements should be
testable.
l Requirements-based testing is a validation
testing technique where you consider each
requirement and derive a set of tests for that
requirement.
cmsc435 - 39
Equivalence partitioning
Equivalence partitions
Less than 1 0000 (^) Betw een 10000 and 99999 More than 99999
Input values
Less than 4 Betw een 4 and 10 Mor e than 10
Number of input values
