Software Reliability Engineering: A West Virginia University Perspective - Prof. Bojan Cuk, Study notes of Engineering

Download Software Reliability Engineering: A West Virginia University Perspective - Prof. Bojan Cuk and more Study notes Engineering in PDF only on Docsity!

Software Reliability Engineering:

Introduction^ Bojan Cukic

Lane Department of Computer Science and Electrical Engineering

West Virginia University

Goals

Ê^ Traditional problems with software development

Ê^ (Un)reliability of released products Ê^ Missed schedules Ê^ Cost overruns Ê^ All the attributes of software quality. Ê^ Positive effects on market share and profitability. Ê^ Attention paid on

engineering

aspects of software

engineering.^ Ê^ Resolution of conflicting quality related demands^ Ê^ Quantitative guidance for software developmentprocess.

Why SRE?

Ê^ Proven, standard best practice. Ê^ Additional advantages (time to market, cost). Ê^ Widely applicable. Ê^ Low cost of application within any project.

Ê^ Observed increase in project cost: less than 1%. Ê^ Minor impact on schedule in terms of adding newactivities. Ê^ Can be deployed in stages to maximize adoptionbenefits.

What is SRE

Ê^ The set of best practices that empower testers anddevelopers to

Ê^ Ensure product reliability meets users needs Ê^ Speed the product to market faster Ê^ Reduce product cost Ê^ Improve customer satisfaction (fewer angry users) Ê^ Increase their productivity Ê^ Applicable to all software based systems

More definitions

Ê^ Failure intensity (FI):

The number of failures per

natural or time unit, an alternative way ofexpressing reliability. Ê Availability:

The average (over time) probability that a system or a capability of a system isfunctional in a specified environment.

Fundamental ideas (1)

Ê^ Increase the effectiveness of available resources.^ Ê^ Quantitatively characterize expected use.^ Ê^ Focus resources on the most used/critical functions

Ê^ Critical: having great extra value when successful, or greatimpact if not. Ê Make testing realistically represent field conditions. Ê^ Balance customer needs.

Impacts

Ê^ AT&T International Definity PBX^ Ê^ Reduced customer-reported problems by the factor of^10 Ê^ Reduced system test interval by the factor of 2.^ Ê^ Reduced total development time by 30%.^ Ê^ No serious service outages in 2 years of deployment. Ê^ Widespread practice: Alcatel, AT&T, Telcordia,Lucent, Ericsson, HP, Lokheed-Martin,Microsoft, Mitre, Tandem, Saab Aircraft…

Wide applicability

Ê^ Suitable for any software based product. Ê^ Complete SRE may be impractical for smallcomponents (<2 months of staff months effort)^ Ê^ Unless used in product lines/families.^ Ê^ Could be applied in an abbreviated form. Ê^ Independent of development technology andplatform. Ê^ Requires no changes in architecture, design, code,but may suggest beneficial ones.

SRE Process

Ê^ Tasks frequently iterate. Ê^ Post-delivery and maintenance phase (not shown) Ê^ Testers involved throughout the process^ Ê^ Allows better understanding of user’s perspective^ Ê^ Improvement of system requirements, planning Ê^ Selection of appropriate mix of^ Ê^ fault prevention^ Ê^ fault removal^ Ê^ fault tolerance

Test types

Ê^ Types of tests applicable to SRE (based onobjectives, rather than phases in the life-cycle)^ Ê^ Reliability growth tests (find and remove faults)

Ê^ need a minimum of 10-20 detected faults to achievestatistically meaningful results Ê Feature

(minimize impact of the environment),

load

(maximize environmental impacts),

regression

tests

(following a major change) Ê Certification tests^ Ê^ no debugging, accept or reject software under test^ Ê^ no. observed failures not important

SRE and SW design & test process

Ê^ Use knowledge of operational profile to guideand focus design efforts Ê^ Established failure intensity drives the qualityassurance efforts Ê^ Failure intensity goal determines when to stoptesting Ê^ Measurement throughout the life-cycle helpsidentify better methodologies

Is Reliability Important?

Ê^ It should be, since it is a measurable property^ Ê^ Unlike “software quality” Ê^ Useful, since the software is tested under theconditions of perceived usage.^ Ê^ The number of resident faults, for example, is adeveloper oriented measure. Reliability is a useroriented measure.^ Ê^ The number of faults found has NO correlation toreliability. Neither has program complexity.^ Ê^ Accurate measurements of reliability are feasible.

Common Misconceptions

Ê^ Software reliability is primarily concerned withsoftware reliability models. Ê^ It copies hardware reliability theory.^ Ê^ No, because reliability of software is more likely tochange over time (modifications, upgrades). Ê^ It deals with faults or “bugs”. Ê^ It does not concern itself with requirements basedtesting. Ê^ Testing “ultrareliable” software is hopeless.

Few more definitions

Ê^ Software availability: fraction of time when the system isfunctioning acceptably^ Ê^ uptime/(uptime+downtime) Ê^ Maintainability indicated by the average staff hoursneeded to resolve a failure. Ê^ Reliability measurement^ Ê^ Execution time

(time actually spent by the processor executing instructions of the given program).^ Ê^ Models based on ET are more accurate Ê Wall-clock time^ Ê^ All decisions must be related to WcT to be meaningfull