Download Software Engineering Methods and Tools - Worksheet 3 | CSCI 3308 and more Assignments Software Engineering in PDF only on Docsity!
Software Testing Notebook Worksheet # Automating Testing & Fixing Bugs Due: November 12, 2004
Name:
Lab Time:
Grade: /
On my honor, as a University of Colorado at Boulder student, I
have neither given nor received unauthorized assistance on this work.
Signature:
Automating Testing
This week you will be writing three tcsh-based shell scripts to automate testing. You will then use these scripts to help debug the ezpay program until you get it to pass all of its test cases. Place your shell scripts in your architecture- independent bin directory, and make sure that this directory is in your path.
Setting Up
This week’s focus is not on structural testing, so we will not use a jcoverage- instrumented version of ezpay. To switch back to a version of ezpay that has not been modified by jcoverage, perform the following steps.
- In your ezpay directory, type ant clean-all. This will delete jcoverage.ser and the build, dist, and report directories.
- Rename your build.xml file to build-old.xml.
- Copy ezpay3.tar from ~csci3308/src directory into your src directory and unpack it. This tar file is designed to update your existing ezpay installation. All it contains is a new build.xml file.
- Type ant in your ezpay directory to build a non-instrumented version of ezpay.
Now we need to modify the ezpay shell script that currently sits in your src/ezpay directory so that it can be moved to your architecture-independent bin directory.
- Create a copy of your ezpay script and name it ezpay.old. Use this file as a way to revert your changes just in case you run into problems with the instructions below.
- In the ezpay script, change the following line:
set ezpayJars = (ezpay-jcov.jar ezpay.jar)
to look like this:
set ezpayJars = (ezpay.jar)
- Change the following line:
set ezJars = ($ezJars dist/lib/$jar)
to look like this:
set ezJars = ($ezJars $HOME/csci3308/src/ezpay/dist/lib/$jar)
- Move this modified script to $HOME/csci3308/bin
- Type rehash and verify that this new ezpay script is now first in your command path. That is, the output of “which ezpay” should be:
$HOME/csci3308/bin/ezpay
.
- You are now ready to write your new shell scripts.
A Script to Run One Test Case
Create a shell script in $HOME/csci3308/bin called run-test that takes as arguments a test set and a test case, and an optional verbose flag, -v. This script might be invoked like this:
run-test ts1 tc run-test -v ts2 tc
If a test case passes, the run-test script should exit with a status code of 0, otherwise it should exit with a status code of 1. In verbose mode, the script should print the test set and test case number, and either test passed, or test failed, like this:
ts1 tc05: test failed ts2 tc13: test passed
integer arguments. It will take those arguments and divide the first by the second, multiply the result by 100, and produce a formatted percentage string. Thus, the invocation
percentage 4 23
produces the string “17.39%”. The behavior of percentage is undefined if anything but integers are passed as its arguments. This script should allow you to produce the summary line for the run-set script with ease.
A Script to Run All Test Sets
Create a shell script called run-all in $HOME/csci3308/bin that calls run-set on all of the test sets for the program (just ts01 and ts02 in our case.) This script can take an optional verbose flag, -v, which it passes to run-set, if present. Be sure to comment your script.
Script Design
You will have to make some choices about how each script works. If you need to change directories, should you do it in run-test, or in run-set before calling run-test? What error checking should you do? Document your design decisions in a README file. Additionally, be sure to document how to use each script. IMPORTANT: These scripts should assume that they are being invoked in the architecture-independent build directory that we used in worksheet 1. You should create a ts02 directory in $HOME/csci3308/build/ezpay/test so that you can run test cases from both test sets in the $HOME/csci3308/build/ezpay directory. Your test sets still live in your ezpay src directory, but they are “executed” in the builddirectory (again, just like we did in worksheet 1). FURTHERMORE: The -v flag for the run-test, run-set, and run-all scripts has nothing to do with ezpay’s -v flag. Your functional and structural test plans determine when ezpay is invoked with its -v flag.
Debugging
Now, you need to debug the ezpay program. Execute run-all -v on the ezpay program; save the results of this test run to a file called pre-debug.txt. Now, modify the ezpay source code to fix bugs. Note: the source code is contained in $HOME/csci3308/src/ezpay/src/ezpay. After each bug fix, recompile the ezpay program (by invoking “ant” in the ezpay src directory) and then run the run-all -v script in the ezpay build directory to see if more test cases pass. You do not need to save the output of these incremental test runs. Keep debugging the program until all of your test cases pass for both test sets. If you can’t get 100% of your test cases from ts and ts02 to pass, then get as close as you can, and explain why it is impossible to achieve 100%. When you have finished debugging, execute run-all -v one last time and save the results to a file called post-debug.txt.
What to Hand In
This worksheet is worth 75 points. You should hand in the following items:
- a printout of the README file that describes the scripts and your design decisions. (15 points)
- printouts of the three scripts themselves. (30 points)
- printouts of pre-debug.txt and post-debug.txt. Be sure to label them! ( points)
- a description of the debugging process, including a list of bugs fixed (e.g. describe the bug, its location in the source code, and how you fixed it). If you did not succeed in getting 100% of your test cases to pass, include an explanation as to why it was impossible to achieve that goal. (25 points)
