Automated Test Input Generations in Software Testing | CSC 712, Study notes of Computer Science

Download Automated Test Input Generations in Software Testing | CSC 712 and more Study notes Computer Science in PDF only on Docsity!

Automated Test-Input Generation

Tao Xie

North Carolina State University Department of Computer Science

Sept 2006

http://www.csc.ncsu.edu/faculty/xie/

Why Automate Testing?

•^ Software testing is important

• Software errors cost the U.S. economy about $59.5 billioneach year (0.6% of the GDP)

[NIST 02]

• Improving testing infrastructure could save 1/3 cost

•^ Software testing is costly

• Account for even half the total cost of softwaredevelopment

[Beizer 90]

•^ Automated testing reduces manual testing effort

• Test generation: Parasoft Jtest, Agitar Agitator, etc. – Test execution: JUnit framework – Test-behavior inspection: Agitar Agitator

Approaches

• Method-sequence exploration

• Parasoft Jtest 4.5 and others

• Concrete-state exploration

• Rostra

[Xie et al. ASE 04]

, NASA JPF

[Visser et al. ISSTA 04]

• Symbolic-state exploration

• Symstra

[Xie et al. TACAS 05]

• Concolic-state exploration

• DART

[Godefroid et al. PLDI 05]

, EGT

[Cadar&Engler SPIN 05]

CUTE

[Sen et al. FSE 05]

Exploring Method Sequences

• Method arguments:

insert(1),

insert(2),

insert(3),

remove(1),

remove(2),

remove(3),

contains(1) new

BST()

Exploring Method Sequences

• Method arguments:

insert(1),

insert(2),

insert(3),

remove(1),

remove(2),

remove(3),

contains(1) new

BST()

insert(1)

insert(2)

remove(2) remove(1) insert(3)

remove(3)

Iteration 2

contains(1)

insert(1)

contains(1)

Generating Tests from Exploration

•^ Collect method sequence along the shortest path

(constructor-call edge

Æ

each method-call edge)^ new

BST()

insert(1)

insert(2)

remove(2) remove(1) insert(3)

remove(3)

Iteration 2

contains(1)

BST^

t^ =^

new^

BST();

t.insert(1);t.insert(1);

insert(1)^ …

contains(1)

Pruning State-Preserving Methods

• Method arguments:

insert(1),

insert(2),

insert(3),

remove(1),

remove(2),

remove(3),

contains(1) new

BST()

insert(1)

insert(2)

remove(2) remove(1) insert(3)

remove(3)

Iteration 2

contains(1)

insert(1)

contains(1)

Observation

new

BST()

insert(1)

insert(2)

remove(2) remove(1) insert(3)

remove(3)

Iteration 2

contains(1)

remove(1)

•^ Some method sequences lead receiver object states back toearlier explored states insert(1)

contains(1)

Exploring Concrete States

• Method arguments:

insert(1),

insert(2),

insert(3),

remove(1),

remove(2),

remove(3)

new

BST()

Exploring Concrete States

• Method arguments:

insert(1),

insert(2),

insert(3),

remove(1),

remove(2),

remove(3)

new

BST()

insert(1)

2

insert(2)

remove(1) remove(2)

3

insert(3)

remove(3)

1

Iteration 1

Generating Tests from Exploration

•^ Collect method sequence along the shortest path

(constructor-call edge

Æ

each method-call edge)^ new

BST()

insert(1)

2

insert(2)

remove(1) remove(2)

3

insert(3)

remove(3)

1

1 2

insert(3) 1 3

remove(2)remove(3)insert(1) insert(2)

remove(1)

BST^

t^ =^

new^

BST();

t.insert(1);t.insert(3);

Issues of Concrete-State Exploration • State explosion– need at least

N^

different

insert

arguments to

reach a BST with size

N

– run out of memory when

N^

reaches 7

• Relevant-argument determination

– assume a set of given relevant arguments

• e.g.,

insert(1),

insert(2),

insert(3),

etc.

State Abstraction: Symbolic States

new

BST()

insert(1)

2 insert(2)

3 insert(3)

1 1 2

x

insert(x

1 3

new

BST()

insert(3)

insert(2)

Iteration 2

State Abstraction: Symbolic States

new

BST()

insert(1)

2 insert(2)

3 insert(3)

1 1 2

insert(3)

insert(2)

x1 x1 < x^2 x1^ x

insert(x

insert(x

1 3

new

BST()

Iteration 2