Prepare for your exams
Get points
Guidelines and tips
Sell on Docsity
Docsity AI

Prepare for your exams

Study with the several resources on Docsity

Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips

Sell on Docsity

Docsity AI

Log in Sign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents

20 Points

For each uploaded document

Answer questions

5 Points

For each given answer (max 1 per day)

All the ways to get free points

Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Testing - Programming Language Technologies and Paradigms | CMSC 433, Study notes of Programming Languages

University of Maryland Programming Languages

Prof. Atif M. Memon

Material Type: Notes; Professor: Memon; Class: PROG LANG TECH & PDGMS; Subject: Computer Science; University: University of Maryland; Term: Spring 2007;

Typology: Study notes

Pre 2010

Uploaded on 02/13/2009

koofers-user-61t 🇺🇸

10 documents

1 / 8

This page cannot be seen from the preview

Don't miss anything!

CMSC 433 – Programming Language

Technologies and Paradigms

Spring 2007

Testing

Feb. 15, 2007

Some slides adapted from FSE’98 Tutorial by Michal Young and

Mauro Pezze’

2

Testing

• Execute program on sample input data

– Check if output correct (acceptable)

•Goals

– Increase confidence program works correctly

• Acceptance Testing

– Find bugs in program

• Debug Testing

3

Example (Black Box)?

% java TestServlet HelloWorld /FooBar/Test > out

HTTP/1.0 200

Content-Type: text/plain

Hello /FooBar/Test

% diff out expectedOutput

4

Sorting

Program

Sorting

Program

A Real Testing Example

SPECS:

Takes a list

of numbers;

returns a

sorted list.

{1,3,2}

{1,2,3}

{3,2,3}

{}

{-1, -2}

Just a list.

A sorted list.

Repeated entry.

Empty list.

Negative numbers.

Test Cases

{1, 2, 3}

Output

{1, 2, 3}

Output

{2, 3, 3}

Output

{}

Output

{-2, -1}

Output

Discover Study notes of Programming Languages University of Maryland

Partial preview of the text

Download Testing - Programming Language Technologies and Paradigms | CMSC 433 and more Study notes Programming Languages in PDF only on Docsity!

CMSC 433 – Programming Language

Technologies and Paradigms

Spring 2007

Testing

Feb. 15, 2007

Some slides adapted from FSE’98 Tutorial by Michal Young and

Mauro Pezze’

Testing

Execute program on sample input data

– Check if output correct (acceptable)

Goals

– Increase confidence program works correctly

• Acceptance Testing

– Find bugs in program

• Debug Testing

Example (Black Box)?

% java TestServlet HelloWorld /FooBar/Test

out

Hello /FooBar/TestContent-Type: text/plainHTTP/1.0 200

% diff out expectedOutput

Sorting

Program

Sorting

Program

A Real Testing Example

SPECS:

of numbers;Takes a list

returns a

sorted list.

Just a list.

Repeated entry.A sorted list.

Empty list.

Negative numbers.

Test Cases

Output

Limitations of Testing

Program runs on (very small)

subset

of input data

– Exhaustive testing usually impossible

• Too large input space (possibly infinite)

Many situations hard to test

– Inadequate test environment (e.g., lack of hardware)– Hard-to-reach states (e.g., error states)– Parallel code (due to non-determinism)

Testing cannot prove absence of bugs

– Especially a problem in security

Black Box Testing

Pick subcomponent of program

– Internals of component not considered

Give it inputs

Compare against expected outputs

inputs

outputs

being testedsubcomponent

Is it correct?

oracle

Black Box Testing

Pick subcomponent of program

– Internals of component not considered

Give it inputs

Compare against expected outputs

– Depends on the software specification …– But how do I know what the expected outputs are?

The Test Case Generation Problem

What tests will show that my program works?

– Must consider “operational scenarios”

• What is the correct action or output?• What is legitimate input?

– Must consider “abnormal behaviors” as well

behaviors of my program have been tested?How can I make sure that all of the important

– Usually, you can’t!

White/Glass Box Testing

Pick subcomponent of program

Give it inputs

– Based on component code

• If you don’t execute the code, you don’t know

whether or not it works

Compare against correct outputs (properties)

inputs

outputs

being testedsubcomponent

Is it correct?

oracle

Statement Coverage

Faults handling positive values of a[i] not revealed One test case (n=1, a[0]=-7, x=9) covers all statements

{int select(int[] a, int n, int x)

{while (i<n && a[i] <x)int i=0;

if (a[i]<0)

a[i] = - a[i];

i++;

return 1;}

i++

i<n and a[i] <x

a[i]<

a[i] = - a[i];

return 1

true

false

true

false

i=

Branch Coverage

i=

i<n and a[i] <x

a[i]<

a[i] = - a[i];

return 1

true

false

true

false

Faults exiting the loop with a[i] <x not revealedFaults handling positive values of a[i] revealed.Must add test case (n=1, a[0]=7, x=9) to cover false branch of if

{int select(int[] a, int n, int x)

{while (i<n && a[i] <x)int i=0;

if (a[i]<0)

a[i] = - a[i];

i++;

return 1;}

i++

Condition Coverage

i=

i<n and a[i] <x

a[i]<

a[i] = - a[i];

return 1

true

false

true

false

Faults that arise after several loop iterations not revealed.Must add tests that cause loop to exit for a value greater than X.Both i<n and a[i]<x must be false and true for different tests.

{int select(int[] a, int n, int x)

{while (i<n && a[i] <x)int i=0;

if (a[i]<0)

a[i] = - a[i];

i++;

return 1;}

i++

Structural Coverage Testing

Adequacy criteria

– If significant parts of program structure are not tested,

testing is surely inadequate

Control flow coverage criteria

– Condition coverage– Branch (edge) coverage– Statement (node, basic block) coverage

and the inadequateAttempted compromise between the impossible

Granularity of Tests

Unit testing

– Individual components of a program are tested

• Processes of a distributed system• Classes/packages• Methods

Integration testing

– Test case inputs to subsystem, multiple subsystems, or

the whole program, and outputs examined

White/Glass Box vs. Black Box

• Black box

– scales up– depends on spec

• different techniques at

different granularity levels

– cannot reveal code coverage

problems

• same specification

different modulesimplemented with

• White box

– depends on control or data

flow coverage

– does not scale up

• mostly applicable at unit

leveland integration testing

– cannot reveal missing path

errors

• part of the specification

that is not implemented

Testing Activities

Test case execution is only a part of the process

Must also consider

– Test result evaluation– Test case generation

Planning is essential

– To coordinate complementary analysis and testing– To build a testable product– To choose appropriate techniques at each stage– To achieve early and continuous visibility

Example

SXEOLFFODVV0DWK^

SULYDWHSULYDWHSULYDWHSULYDWHLQW

LQWLQWLQW QXPEHU

QXPEHUQXPEHUQXPEHU

SULYDWHSULYDWHSULYDWHSULYDWHLQW

LQWLQWLQW QXPEHU

QXPEHUQXPEHUQXPEHU

SXEOLFSXEOLFSXEOLFSXEOLF0DWKLQW

0DWKLQW

0DWKLQW QXP

QXPQXPQXPLQW

LQWLQWLQW QXP^

QXP^

QXPEHU QXP

SXEOLFSXEOLFSXEOLFSXEOLFLQW````

LQW

LQW DGG^

DGG^DGG^DGG^

LQW

LQW QXPEHU QXPEHUQXPEHU

QXPEHU QXPEHUQXPEHUQXPEHU QXPEHUQXPEHUQXPEHU QXPEHUQXPEHU

UHWXUQQXPEHU

SXEOLFSXEOLFSXEOLFSXEOLFLQW````

LQW

LQW VXEWUDFW^

VXEWUDFW^VXEWUDFW^VXEWUDFW^

LQW

LQW QXPEHU QXPEHU

QXPEHU QXPEHUQXPEHU QXPEHUQXPEHU QXPEHU

QXPEHU

UHWXUQQXPEHU

````

SXEOLFSXEOLFSXEOLFSXEOLFLQW

LQWLQWLQW PXOWLSO\^

PXOWLSO\^PXOWLSO\^PXOWLSO\^

LQW

LQW QXPEHU QXPEHU QXPEHU

QXPEHU QXPEHU QXPEHUQXPEHU QXPEHU QXPEHUQXPEHU QXPEHU QXPEHU

UHWXUQQXPEHU

SXEOLFGRXEOHGLYLGH^SXEOLFGRXEOHGLYLGH^SXEOLFGRXEOHGLYLGH^SXEOLFGRXEOHGLYLGH^````

GRXEOHQXPEHU QXPEHUQXPEHU

UHWXUQQXPEHU

SXEOLFSXEOLFSXEOLFSXEOLFLQW````

LQWLQWLQW PRG^

PRG^PRG^PRG^

LQW

LQW QXPEHU QXPEHU#QXPEHU

QXPEHU QXPEHU#QXPEHUQXPEHU QXPEHU#QXPEHUQXPEHU QXPEHU#QXPEHU

UHWXUQQXPEHU

````

`

Example

LPSRUWLPSRUWLPSRUWLPSRUWMXQLWIUDPHZRUN

MXQLWIUDPHZRUNMXQLWIUDPHZRUNMXQLWIUDPHZRUN

SXEOLFFODVVSXEOLFFODVVSXEOLFFODVVSXEOLFFODVV0DWK7HVW

0DWK7HVW0DWK7HVW0DWK7HVW H[WHQGV

H[WHQGVH[WHQGVH[WHQGV7HVW&DVH

7HVW&DVH

7HVW&DVH ^

^^^

SULYDWH0DWKWHVW9DU

SXEOLFVWDWLFYRLG

SXEOLFVWDWLFYRLGPDLQ6WULQJ

PDLQ6WULQJPDLQ6WULQJPDLQ6WULQJ DUJV

DUJVDUJVDUJV>@^

>@^>@^>@^

MXQLWWH[WXL7HVW5XQQHUUXQ0DWK7HVWMXQLWWH[WXL7HVW5XQQHUUXQ0DWK7HVWMXQLWWH[WXL7HVW5XQQHUUXQ0DWK7HVWMXQLWWH[WXL7HVW5XQQHUUXQ0DWK7HVW

FODVV

`

SURWHFWHGYRLG

SURWHFWHGYRLGVHW8S

VHW8SVHW8SVHW8S^

^

WHVW9DU QHZ0DWKWHVW9DU QHZ0DWKWHVW9DU QHZ0DWKWHVW9DU QHZ0DWKWHVW9DU QHZ0DWKWHVW9DU QHZ0DWKWHVW9DU QHZ0DWKWHVW9DU QHZ0DWK

`

SURWHFWHGYRLG

SURWHFWHGYRLGWHDU'RZQ

WHDU'RZQWHDU'RZQWHDU'RZQ^

^

6\VWHPRXWSULQWOQ57HDU6\VWHPRXWSULQWOQ57HDU6\VWHPRXWSULQWOQ57HDU6\VWHPRXWSULQWOQ57HDU 'RZQ5

'RZQ5'RZQ5'RZQ5

`

SXEOLFYRLGSXEOLFYRLGSXEOLFYRLGSXEOLFYRLGWHVWDGG

WHVWDGG

WHVWDGG^

^^^

^`

LQWLQWLQWLQW H[SHFWHG

H[SHFWHG

DVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGWHVW9DUDGG

WHVW9DUDGG

SXEOLFYRLGSXEOLFYRLGSXEOLFYRLGSXEOLFYRLGWHVWVXEWUDFW````

WHVWVXEWUDFWWHVWVXEWUDFWWHVWVXEWUDFW^

^ ^

^

^^^^

`

^`

`

^`

`

^`

`

^`

LQWLQWLQWLQW H[SHFWHG

H[SHFWHG H[SHFWHG

H[SHFWHG

DVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGWHVW9DUVXEWUDFW

WHVW9DUVXEWUDFWWHVW9DUVXEWUDFWWHVW9DUVXEWUDFW

SXEOLFYRLGSXEOLFYRLGSXEOLFYRLGSXEOLFYRLGWHVWPXOWLSO````

WHVWPXOWLSO\WHVWPXOWLSO\WHVWPXOWLSO\^

^^^

^$`

^`

^$`

^`

^$`

^`

^$`

^`

LQWLQWLQWLQW H[SHFWHG

H[SHFWHG H[SHFWHG

H[SHFWHG

DVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGWHVW9DUPXOWLSO\

WHVW9DUPXOWLSO\WHVW9DUPXOWLSO\WHVW9DUPXOWLSO\

SXEOLFYRLGSXEOLFYRLGSXEOLFYRLGSXEOLFYRLGWHVWGLYLGH````

WHVWGLYLGHWHVWGLYLGHWHVWGLYLGH^

^^^

^`

DVVHUW(TXDOVQHZDVVHUW(TXDOVQHZDVVHUW(TXDOVQHZDVVHUW(TXDOVQHZ 'RXEOHH[SHFWHGGRXEOHH[SHFWHG GRXEOHH[SHFWHG GRXEOHH[SHFWHG GRXEOHH[SHFWHG

'RXEOHH[SHFWHG 'RXEOHH[SHFWHG

'RXEOHH[SHFWHGQHZ

QHZQHZQHZ

SXEOLFYRLGSXEOLFYRLGSXEOLFYRLGSXEOLFYRLGWHVWPRG````'RXEOHWHVW9DUGLYLGH'RXEOHWHVW9DUGLYLGH'RXEOHWHVW9DUGLYLGH'RXEOHWHVW9DUGLYLGH

WHVWPRGWHVWPRGWHVWPRG^

^^^

^(`

^`

^(`

^`

^(`

^`

^(`

^`

LQWLQWLQWLQW H[SHFWHG

H[SHFWHG H[SHFWHG

H[SHFWHG

DVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGDVVHUW(TXDOVH[SHFWHGWHVW9DUPRG

WHVW9DUPRGWHVW9DUPRGWHVW9DUPRG

To Execute Tests within a Class

Pick a Test Runner:

– junit.awtui.TestRunner

– Graphical

– junit.swingui.TestRunner

– Graphical

– junit.textui.TestRunner

– Textual

Invoke on the test case class

OK (2 tests)Time: 0.03.. > java junit.textui.TestRunner ListTest

Junit

Components

Test cases (class

TestCase

– Can reuse test case setup (optional)– Individual tests

Test suites (class

TestSuite

– Test case container

Test runner (various classes)

– Executes test suites and presents results

Each Test Has Three Parts

Code that creates test objects

– Create a subclass of junit.framework.TestCase

Code that executes the test

– Override the method runTest() (which executes the test)

Code that verifies the result

– E.g., use junit.framework.assertTrue() to check results

(throws exception is test fails)

Setup/Teardown

Creating objects for each test insufficient

– Instead, group setup (and teardown) code in one place– Setup overhead grows as number of tests grows

and reuse

junit.framework.TestCase.run() executes test case:

– public void

run

() { setUp(); runTest();

tearDown(); }

• Do not override this method!

– Put cleanup code in tearDown() method– Put setup code in setUp() method

More Asserts

Junit has several different tests

– assertNotSame(o1, o2) -- assert o1 != o2– assertSame(o1, o2) -- assert o1==o2– assertNull(o) -- assert o == null– assertNotNull(o) -- assert o != null– assertEquals(o1, o2) -- assert that o1.equals(o2)– assertFalse(b) -- asserts that b is false– assertTrue(b) -- asserts that b is true

Testing - Programming Language Technologies and Paradigms | CMSC 433, Study notes of Programming Languages

Related documents

Partial preview of the text

Download Testing - Programming Language Technologies and Paradigms | CMSC 433 and more Study notes Programming Languages in PDF only on Docsity!

CMSC 433 – Programming Language

Technologies and Paradigms

Spring 2007

Testing

Feb. 15, 2007

Some slides adapted from FSE’98 Tutorial by Michal Young and

Mauro Pezze’

Testing

Execute program on sample input data

– Check if output correct (acceptable)

Goals

– Increase confidence program works correctly

• Acceptance Testing

– Find bugs in program

• Debug Testing

Example (Black Box)?

% java TestServlet HelloWorld /FooBar/Test

out

Hello /FooBar/TestContent-Type: text/plainHTTP/1.0 200

% diff out expectedOutput

Sorting

Program

Sorting

Program

A Real Testing Example

SPECS:

of numbers;Takes a list

returns a

sorted list.

Just a list.

Repeated entry.A sorted list.

Empty list.

Negative numbers.

Test Cases

Output

Output

Output

Output

Output

Limitations of Testing

Program runs on (very small)

subset

of input data

– Exhaustive testing usually impossible

• Too large input space (possibly infinite)

Many situations hard to test

– Inadequate test environment (e.g., lack of hardware)– Hard-to-reach states (e.g., error states)– Parallel code (due to non-determinism)

Testing cannot prove absence of bugs

– Especially a problem in security

Black Box Testing

Pick subcomponent of program

– Internals of component not considered

Give it inputs

Compare against expected outputs

inputs

outputs

being testedsubcomponent

Is it correct?

oracle

Black Box Testing

Pick subcomponent of program

– Internals of component not considered

Give it inputs

Compare against expected outputs

– Depends on the software specification …– But how do I know what the expected outputs are?

The Test Case Generation Problem

What tests will show that my program works?

– Must consider “operational scenarios”

• What is the correct action or output?• What is legitimate input?

– Must consider “abnormal behaviors” as well

behaviors of my program have been tested?How can I make sure that all of the important

– Usually, you can’t!

White/Glass Box Testing

Pick subcomponent of program

Give it inputs

– Based on component code