Download Software Engineering: Validation, Verification, Testing - Case Studies & Best Practices and more Exams Software Engineering in PDF only on Docsity!
CS 451
Software Engineering
Winter 2009
Sunny Wong [email protected]
V&V
Validation: the process of evaluating software during or at the end of the development process to determine whether it satisfies specified requirements Verification: the process of evaluating software to determine whether the products of a given development phase satisfy the conditions imposed at the start of the phase IEEE Standard 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology. 1990.
Software Testing
http://xkcd.com/327/
Some (In)famous Failures
AT&T Long Distance (1990) Nine hour service failure Wrong !"#$% statement in C code MIM-104 Patriot (1991) Missed incoming Iraqi Scud Floating point roundoff error Incomplete refactoring of code
Some (In)famous Failures
Ariane 5 Flight 501 (1996) Self-destruction 37 seconds after launch Data conversion overflow USS Yorktown CG-48 (1998) Three hour propulsion system failure Division by zero from input Northeast US and Canada Blackout (2003) Alarm system failure Race condition
Significance of Testing
Some estimates that testing consumes 50% of development costs NIST Study*: annual national cost of inadequate testing up to $59 billion Testing has limitations Exhaustive testing is impractical/impossible Code changes require re-testing “Testing shows the presence, not the absence of bugs.” — E. W. Dijkstra “NIST Report Takes a Step toward Better Testing.” IEEE Software. Vol. 19, No. 6, Nov/Dec 2002.
Testing Models
Black Box Testing White Box Testing
White Box Testing
Test functionality of methods/procedures Check logic in code (loops, conditions, etc.) Given an input, check path of code def fib(n) return 1 if n <= 1 return fib(n - 1) + fib(n - 2) input (^) end output
ATM Example (2)
Cash withdrawal Preconditions: User session valid (PIN verified) Test data loaded Input: Account number: 09451 Withdrawal amount: $ Expected Result:?
ATM Example (3)
Cash withdrawal invalid account Preconditions: User session valid (PIN verified) Test data loaded (account 09451 not exists) Input: Account number: 09451 Withdrawal amount: $ Expected Result: Reports invalid account to user
ATM Example (4)
Cash withdrawal valid account Preconditions: User session valid (PIN verified) Test data loaded (account 08451 exists) Input: Account number: 08451 Withdrawal amount: $ Expected Result:?
ATM Example (5)
Cash withdrawal valid account, insufficient funds Preconditions: User session valid (PIN verified) Test data loaded (account 08451 exists, balance $10) Input: Account number: 08451 Withdrawal amount: $ Expected Result: Reports insufficient funds to user
ATM Example (6)
Cash withdrawal valid account, sufficient funds Preconditions: User session valid (PIN verified) Test data loaded (account 07350 exists, balance $20) Input: Account number: 07350 Withdrawal amount: $ Expected Result: $20 given to user Account 07350 has balance $
ATM Example (7)
Cash withdrawal invalid account $20 from account 09451 Cash withdrawal valid account, insufficient funds $20 from account 08451, with $ Cash withdrawal valid account, sufficient funds $20 from account 07350, with $
? Cash withdrawal valid account, sufficient funds
$15 from account 09458, with $
Equivalence Classes
Divide input conditions into groups/classes Input in same class should behave similarly Reduces number of test cases to finite (small) number from each equivalence class Test both boundary and mid-range values Test invalid input values
ATM Withdrawal
amount account invalid accounts valid accounts insufficient funds sufficient funds $20 from 09451 $20 from 07350 $20 from 08451 $15 from 09458
