CSE784 Software Engineering Studio
Fall 2003
Test Harness Program
B Level Specification
for
Software Development
version 1.0
15 September, 2003
CSE 784 – Software Engineering Studio
Riddhiman Ghosh
Test Harness B Specification 1
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Test Harness Program: B Level Specification for Software Development | CSE 784, Exams of Software Engineering
Material Type: Exam; Class: Software Engineering Studio; Subject: Computer Engineering; University: Syracuse University; Term: Fall 2003;
Typology: Exams
Pre 2010
1 / 18
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Test Harness Program: B Level Specification for Software Development | CSE 784 and more Exams Software Engineering in PDF only on Docsity!
Fall 2003
Test Harness Program
B Level Specification
for
Software Development
version 1.
15 September, 2003
CSE 784 – Software Engineering Studio
Riddhiman Ghosh
Table of Contents
Fall 2003 1.Architecture The Test Harness program provides a framework using which developers can test application code. Its goal is to offer facilities that make it easy to construct and execute test suites. The code under test can also be instrumented with the help of Test Harness services that keep track of memory allocations and de-allocations in order to detect memory leaks. The results of the various test procedures can be logged to either console output or to a memory stream, or the results can be stored persistently in the file system. Figure 1 – Test Harness Context Diagram As can be seen from the Context Diagram in Figure 1, the major sinks and sources of information which transact with the Test Harness are the application code under test, the file system, the system memory and the console user interface. Application Code Test Harness Console UI File System System Memory execution commands test and heap status, errors 'test vector' data log files memory log function invocation results, memory allocation messages, memory deallocation messages function invocations
Fall 2003 Figure 2 represents the high-level partitioning of the Test Harness. The TestAggregator module oversees the execution of the tests contained in one or more TestModule s (typically one for each module of application code). The TestAggregator determines the order of execution of the test modules, and thus is in control of the overall flow of the testing process. TestModule contains the actual test procedures that exercise various features of the application code and verify their operation. The TestModule uses AllocTracker to monitor memory allocations and de-allocations from the heap in individual modules. The TestAggregator executive uses updates provided by AllocTracker to maintain state information pertaining to the net usage of heap memory. The Logger module is responsible for redirecting the results of tests to either a memory stream or the console output, or for assigning test results to persistent storage. Both TestModule and TestAggregator use Logger to log test results and heap status. Since the Logger deals with different types of data streams (memory, file, console), it uses the services of the StreamHandler helper module, which provides an abstraction of the different data stores to the Logger module. TestAggregator Test Module(s) 'N' Logger AllocTracker StreamHandler
Fall 2003
2. Reference Documents 2.1. Customer A-Specification Project #1 – Test Harness, Monday 25 August, 2003
Fall 2003
3. Requirements Processing for the Test Harness program is composed of 4 main processes: Test Aggregation, Testing, Allocation Tracking and Result Logging. Each of these processes is responsible for a set of requirements, described in this section. Figure 3 - Data Flow Diagram 1 Test Aggregation 1 Testing 2 Result Logging 4 execution commands function invocations function invocation results TestResults Data Structure log files memory log Allocation Tracking 3 memory allocation messages memory deallocation messages 'test vector' data test order test and heap status net bytes not freed errors
Fall 2003 3.1.2. Testing The Testing process starts executing the test procedures contained within the test modules, the order of execution of the modules being specified by the ‘test order’ message. The Testing process also decides whether a test succeeded or failed. 3.1.2.1. inputs The test order message Test vector data is the input information required to execute the various tests contained in a test module. Function invocation results are the values returned by various application code function calls, used to determine test success or failure. 3.1.2.2. processing The Testing process shall execute of all test procedures within a module. Order of execution of the test modules shall be in the order specified by the test order message. It shall use the information in test vector data as the required input for the testing procedures. It shall extract the file path descriptor from the data structure referenced by the test order message and read the test vector data information from this file. If an attempt to read the file or parse the path fails, it shall send an error message to the standard output and continue execution of other test modules. The Testing process shall determine at the end of execution of each test procedure whether the outcome of the test was a success or failure. It shall count the overall number of test successes and failures. 3.1.2.3. outputs The function invocation messages are calls out into the application code made by the test procedures in the Testing process. The TestResults Data Structure message contains a reference to the container(s) used by the Testing process. It holds success and failure information of all tests carried out.
Fall 2003 3.1.3. Allocation Tracking The Allocation Tracking process is responsible for using the memory allocation and memory de-allocation messages to monitor the status of the heap. The output of this process is sent in the form of a message indicating the net allocated bytes not freed from the heap. 3.1.3.1. inputs The memory allocation and memory de-allocation messages are sent in response to the application code allocating and freeing memory from the system heap. 3.1.3.2. processing The Allocation Tracking process shall use the memory allocation and de-allocation messages to keep track of the total number of memory bytes allocated and the net number of all allocated bytes remaining when the program terminates. 3.1.3.3. outputs The net bytes not freed message contains information regarding how many bytes were allocated from the heap, and the net bytes of allocated memory remaining after any de-allocation.
Fall 2003 3.1.4.1. Console Display This process displays on the console the status of the tests and heap, i.e. how many tests failed and how many allocated bytes remain on the heap. 3.1.4.1.1. inputs The TestResults Data Structure message contains a reference to the container(s) used by the Testing process. It holds success and failure information of all tests carried out. The net bytes not freed message contains the number of total allocated bytes on the heap that have not been de-allocated. 3.1.4.1.2. processing The Console Display process shall extract the status of all tests carried out. It shall extract the total number of allocated bytes on the heap that have not been freed. It shall display on the standard output the total number of tests carried out, the number of failures, and the net bytes not de-allocated. 3.1.4.1.3. outputs This process sends all its outputs to the standard output in the test and heap status message. 3.1.4.2. Stream Logging 3.1.4.2.1. inputs The test and heap status message contains the total number of tests carried out, the total number of failures, and net allocated bytes on the heap which have not been freed. 3.1.4.2.2. processing The Stream Logging process shall write the contents of the test and heap message to a log file. It shall write time and date of logging into the file. The format for storing the information in the file is not specified, but it should be clear and readable. It shall write the contents of the test and heap message to a memory stream. The format of writing
Fall 2003 to the stream is not specified, but the format should be such as to make it convenient for other programs/processes to extract and use this status information. 3.1.4.2.3. outputs The log files message The memory log message These 2 messages contain information described in the processing section above. The formats of the two are different and are hence denoted as two separate outputs.
Fall 2003 constructing and executing tests, tracking dynamic memory allocations and logging test results. The Test Harness program shall provide at least one class derived from the tracking interface to implement allocation tracking. It shall implement at least 3 classes derived from the logging interface, one each to implement logging to the console, memory and file streams. The Test Harness program shall demonstrate the fulfillment of the requirements mentioned in Section 3 by testing several components of a software project implemented in CSE 681 or CSE 687. For each module of the application code being tested, the Test Harness program shall provide at least one class derived from the interface to construct and execute tests. 3.2.2. Development Environment The Test Harness program shall compile and link successfully from the ECS clusters using Visual Studio 7.0.
Fall 2003
4. Data Dictionary Message Name Interpretation DFD execution commands commands given by user to start the Test Harness program DFD test order reference to container with ordered test modules and corresponding test vector file paths DFD test vector data input information required for the test procedures DFD function invocations calls made into the application code functions for testing DFD function invocation results return values of application code function invocations DFD errors message describing path error or file not found DFD TestResults Data Structure reference to container containing success/failure results of tests DFD memory allocation messages messages corresponding to allocation of memory on heap in application code DFD memory de-allocation messages messages corresponding to de-allocation of memory on heap in application code DFD net bytes not freed represents total allocated bytes on heap that have not been freed DFD test and heap status message describing status of tests and status of heap on console output DFD log files files with logged information on test and heap status DFD memory log logged memory stream with status of tests and heap DFD Figure 5 - Data Dictionary