Evaluation - Software Engineering - Lecture Slides | CS 1530, Study Guides, Projects, Research of Software Engineering

Material Type: Project; Class: SOFTWARE ENGINEERING; Subject: Computer Science; University: University of Pittsburgh; Term: Fall 2004;

Typology: Study Guides, Projects, Research

Pre 2010

Uploaded on 09/02/2009

koofers-user-36f
koofers-user-36f 🇺🇸

10 documents

1 / 9

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
1
CS 1530 Software Engineering Fall
2004
Software Engineering
Evaluation
CS 1530 Software Engineering Fall
2004
Reading
Chapter 12 in SE
Categories of evaluation
Feature analysis: rate and rank
attributes
Survey: document relationships
Case study
Formal experiment
pf3
pf4
pf5
pf8
pf9

Partial preview of the text

Download Evaluation - Software Engineering - Lecture Slides | CS 1530 and more Study Guides, Projects, Research Software Engineering in PDF only on Docsity!

CS 1530 Software 2004 Engineering Fall

Software Engineering

Evaluation

CS 1530 Software Engineering Fall 2004

Reading

■ Chapter 12 in SE

Categories of evaluation

■ Feature analysis: rate and rank

attributes

■ Survey: document relationships

■ Case study

■ Formal experiment

Example feature analysis

Table 12.1. Design tool ratings

Feature Tool 1: t-OO-l ObjecTool Tool 2: EasyDesign Tool 3: Importance

Good user interface Object-oriented design 45 55 45

Consistency checking 5 3 1 3

Use cases Runs on Unix 45 44 45

Score 85 77 73

Case study types

■ Sister projects: each is typical and has

similar values for the independent

variables

■ Baseline: compare single project to

organizational norm

■ Random selection: partition single

project into parts

Formal experiment

■ Controls variables

■ Uses methods to reduce bias and

eliminate confounding factors

■ Often replicated

■ Instances are representative: sample

over the variables (whereas case study

samples from the variables)

Table 12.3. Results of comparing prediction models. (Lanubile 1996) Modeling technique Predictive validity Proportion of false negatives Proportion of false positives Proportion of false classificati ons Completen ess Overall inspection Wasted inspection Discrimin ant analysis p=0.621 28% 26% 54% 42% 46% 56% Principal component analysis plus discrimina nt analysis p=0.408 15% 41% 56% 68% 74% 55% Logistic regression p=0.491 28% 28% 56% 42% 49% 58% Principal component analysis plus logistic regression p=0.184 13% 46% 59% 74% 82% 56% Logical classificati on model p=0.643 26% 21% 46% 47% 44% 47% Layered neural network p=0.421 28% 28% 56% 42% 49% 58% Holograph ic network p=0.634 26% 28% 54% 47% 51% 55% Heads or tails? p=1.000 25% 25% 50% 50% 50% 50% Table 12.4. ISO 9126 quality characteristics. Quality characteristic Functionality Definition A set of attributes that bear on the existence of a set of functions and their specified properties. The functions are Reliability those that satisfy stated or implied needs.A set of attributes that bear on the capability of software to maintain its performance level under stated conditions Usability for a stated period of time.A set of attributes that bear on the effort needed for use and on the individual assessment of such use by a stated or Efficiency implied set of users.A set of attributes that bear on the relationship between the software performance and the amount of resources’s Maintainability used under stated conditions.A set of attributes that bear on the effort needed to make specified modifications (which may include corrections, improvements, or adaptations of software to environmental changes and changes in the requirements and functional specifications). Portability A set of attributes that bear on the ability of software to be transferred from one environment to another (including the organizational, hardware or software environment).

Dromey quality model

■ Identify a set of high-level quality attributes.

■ Identify product components.

■ Identify and classify the most significant,

tangible, quality-carrying properties for each

component.

■ Propose a set of axioms for linking product

properties to quality attributes.

■ Evaluate the model, identify its weaknesses,

and refine or recreate it.

Table 12.5. Quantitative targets for managing US defense projects. (NetFocus 1995)

Item Target Malpractice level Fault removal efficiency > 95% < 70% Original fault density < 4 per function point > 7 per function point Slip or cost overrun in excess of risk reserve 0% > 10% Total requirements creep (function points or equivalent) < 1% per month average > 50% Total program documentation < 3 pages per function point

6 pages per function point Staff turnover 1 to 3% per year > 5% per year

Software reuse

■ Producer reuse: creating components

for someone else to use

■ Consumer reuse: using components

developed for some other product

■ Black-box reuse: using component

without modification

■ Clear- or white-box reuse: modifying

component before reusing it

More on reuse

■ Compositional reuse: uses components as

building blocks; development done from

bottom up

■ Generative reuse: components designed

specifically for a domain; design is top-down

■ Domain analysis: identifies areas of

commonality that make a domain ripe for

reuse

Table 12.9. When post-implementation evaluation is done.

Time period Percentage of respondents (of 92 organizations)

Just before delivery 27.8%

At delivery 4.20%

One month after delivery Two months after delivery 22.20%6.90%

Three months after delivery 18.10%

Four months after delivery Five months after delivery 1.40%1.40%

Six months after delivery Twelve months after delivery 13.90%4.20%

Table 12.10. Required questions for level 1 of process maturity model.

Question number Question 1.1.3 Does management the Software reporting Quality channel Assurance separate function from the have software a development project management? 1.1.6 Is project there thata software involves configuration software development? control function for each 2.1.3 Is software a formal development process used prior in the to managementmaking contractual review of each commitments? 2.1.14 2.1.15 IsIs aa formalformal procedureprocedure usedused toto makeproduce estimates software of development software size? schedules? 2.1.16 Are development formal procedures cost? applied to estimating software 2.2.2 Are configuration profiles of item software over time?size maintained for each software 2.2.4 2.4.1 AreDoes statistics senior managementon software codehave (^) aand mechanism test errors for gathered? the regular 2.4.7 reviewDo software^ of^ the development^ status^ of^ software first-line^ development managers^ projects?sign off on their schedule and cost estimates? 2.4.9 Is requirements? a mechanism used for controlling changes to the software 2.4.17 Is a mechanism used for controlling changes to the code?

Table 12.11. Key process areas in the CMM (Paulk et al. 1993)

CMM level Key process areas

Initial none

Repeatable Requirements management

Software project planning

Software project tracking and oversight

Software subcontract management

Software quality assurance

Software configuration management

Defined Organization process focus

Organization process definition

Training program

Integrated software management

Software product engineering

Intergroup coordination

Peer reviews

Managed Quantitative process management

Software quality management

Optimizing Fault prevention

Technology change management

Process change management

Evaluation of CMM

conformance

■ Commitment to perform

■ Ability to perform

■ Activities performed

■ Measurement and analysis

■ Verifying implementation

SPICE

■ Base practices (essential activities) and

general practices (institutionalize or

implement the process)

■ Ratings in terms of

■ customer-supplied processes

■ engineering processes

■ project processes

■ support processes

■ organizational processes

Table 12.12. ISO 9001 clauses.

Clause number Subject matter

4.1 4.2 Management responsibilityQuality system

4.3 Contract review

4.4 4.5 Design controlDocument and data control

4.6 Purchasing

4.7 4.8 Control of customer-supplied productProduct identification and traceability

4.10^ 4.9^ Process controlInspection and testing

4.11 4.12 Control of inspection, measuring and test equipmentInspection and test status

4.13 4.14 Control of nonconforming productCorrective and preventive action

4.15 Handling, storage, packaging, preservation and delivery

4.16 Control of quality records

4.17 4.18 Internal quality auditsTraining

4.19 Servicing

4.20 Statistical techniques