Download Software Engineering Economics and more Exercises Software Engineering in PDF only on Docsity!
Software Engineering Economics
BARRY W. BOEHM
Manuscript received April 26, 1983 ;revised June 28, 1983. The author is with the Software Information Systems Division, TRW Defense Systems Group, Redondo Beach, CA 90278.
Abstruct-This paper summarizes the current state of the art and recent trends in software engineering economics. It provides an over- view of economic analysis techniques and their applicability to soft- ware engineering and management. It surveys the field of software cost estimation, including the major estimation techniques available, the state of the art in algorithmic cost models, and the outstanding research issues in software cost estimation. Index Terms-Computer programming costs, cost models, manage- ment decision aids, software cost estimation, software economics, software engineering, software management.
Definitions
The dictionary defines "economics" as "a social science concerned chiefly with description and analysis of the produc- tion, distribution, and consumption of goods and services." Here is another defmition of economics which I think is more helpful in explaining how economics relates to software engi- neering. Economics is the study of how people make decisions .in resource-limited situations. This definition of economics fits (^) the major branches of classical economics very well. Macroeconomics is the study of how people make decisions in resource-limited situations on a national or global scale. It deals with the effects of decisions that national leaders make on such issues as tax rates, interest rates, foreign and trade policy.
Microeconomics is the study of how people make decisions in resource-limited situations on a more personal scale. It deals with the decisions that individuals and organizations make on such issues as how much insurance to buy, which word proc- essor to buy, or what prices to charge for their products or services.
Economics and Software Engineering Management
If we look at the discipline of software engineering, we see that the microeconomics branch of economics deals more with the types of decisions we need to make as software engineers or managers. Clearly, we deal with limited resources. There is never
enough time or money to cover all the good features we would
like to put into our software products. And even in these days of cheap hardware and virtual memory, our more significant software products must always operate within a world of lim- ited computer power and main memory. If you have been in the software engineering field for any length of time, I am sure
you can think of a number of decision situations in which you
had to determine some key software product feature as a func- tion of some limiting critical resource. Throughout the software life cycle,' there are many de- cision situations involving limited resources in which software engineering economics techniques provide useful assistance. To provide a feel for the nature of these economic decision issues, an example is given below for each of the major phases in the software life cycle. Feasibiliw Phase: How much should we invest in in- formation system analyses (user questionnaires and in-
1 Economic principles underlie the overall structure of the software Iife cycle, and its primary refinements of prototyping, incremental de- velopment, and advancemanship. The primary economic driver of the life-cycle structure is the significantly increasing cost of making a soft- ware change or fmhg a software problem, as a function o f the phase in which the change or fur is made. See [ 11, ch. 41.
I1 of this paper provides an overview of these techniques and their applicability to software engineering.
One critical problem which underlies all applications of
economic techniques to software engineering is the problem of estimating software costs. Section I11 contains three major sections which summarize this field: 111-A: Major Software Cost Estimation Techniques 111-B: Algorithmic Models for Software Cost Estimation 111-C: Outstanding Research Issues in Software Cost Estima- tion. Section IV concludes by summarizing the major benefits of software engineering economics, and commenting on the major challenges awaiting the field.
Overview of Relevant Techniques
The microeconomics field provides a number of techniques for dealing with software life-cycle decision issues such as the ones given in the previous section. Fig. 1 presents an overall master key to these techniques and when to use them.* As indicated in Fig. 1, standard optimization techniques can be used when we can find a single quantity such as dollars (or pounds, yen, cruzeiros, etc.) to serve as a "universal sol-
vent" into which all of our decision variables can be converted.
Or, if the nondollar objectives can be expressed as constraints (system availability must be at least 98 percent; throughput must be at least 150 transactions per second), then standard constrained optimization techniques can be used. And if cash flows occur at different times, then present-value techniques can be used to normalize them to a common point in time.
2 The chapter numben in Fig. 1 refer to the chapters in [ 11 ] , in which those techniques are discussed in further detail.
MASTER KEY
T O SOFTWARE ENGINEERING ECONOMICS
DECISION ANALYSIS TECHNIQUES
USF STANOARn NE r VALUE PRESENT SJ TFCHNIOUFSICHAPTTRS 10. 131
N t W - S D C r VCS USE ZTbNDAROCONSTRAINF~- EXPUCSSIRLE AS OPTIMIZATION ICHAPTFR 161TkCHNlOUCS
W M - S O C : % USE COSTBENEFIT lCRl^ -
-RF w r I~ - C R I ~ IRION, n c c l s l O N MAKING^1
TF CHNIOUFSICIiAPTFRS 11. 121
USF FIGURE OF - ~(IAMTI~ I A B L C 'WN^ -^ S OCs CB TFCHNIOUES ICHAPTF R IS
O U A N I IFIAHLERECONCll ING NON- DCs ICHAPTER 101
HlGHLV SENSl r l V ETO ASSUMPTIONS? HAPTFR I 7
y
FINO. USE LES.SENSlTlVF
USE PRESENT VALUE r E C H N l W F S TO
CONVERT FUTURF S T 0 PRESENT ICHAPTkR 14) S I I
I (^) W E N SOME 00 I N V O L V E I UNCE QTAINTIES I
Fig. 1. laster key to software engineering economics decision analysis techniques!
The cost-versus-performance curve for these two options are shown in Fig. 2. Here, neither option dominates the other, and various cost-benefit decision-making techniques (maximum profit margin, costlbenefit ratio, return on in- vestments, etc.) must be used to choose between Options A and B. In general, software engineering decision problems are even more complex than Fig. 2, as Options A and B will have several important criteria on which they differ (e.g., robustness, ease of tuning, ease of change, functional capability). If these criteria are quantifiable, then some type of figure of merit can be defined t o support a comparative analysis of the preferability of one option over another. If some of, the criteria are unquantifiable (user goodwill, pro- grammer morale, etc.), then some techniques for comparing unquantifiable criteria need to be used. As indicated in Fig. 1, techniques for each of these situations are available, and discussed in [ 111.
Analyzing Risk, Uncertainty, and the Value o f Information
In software engineering, our decision issues are generally
even more complex than those discussed above. This is be-
cause the outcome of many of our options cannot be deter- mined in advance. For example, building an operating sys- tem with a significantly lower multiprocessor overhead may be achievable, but on the other hand, it may not. In such cir- cumstances, we are faced .with a problem of decision making under uncertainty, with a considerable risk of an undesired outcome. The main economic analysis techniques available to sup- port us in resolving such problems are the following.
- Techniques for decision making under complete un- certainty, such as the maximax rule, the maximin rule, and the Laplace rule [38]. These techniques are generally inade- quate for practical software engineering decisions.
2) Expected-value techniques, in which we estimate the probabilities of occurrence of each outcome (successful or unsuccessful development of the new operating system) and complete the expected payoff of each option:
These techniques are better than decision making under com- plete uncertainty, but they still involve a great deal of risk if the Prob(fai1ure) is considerably higher than our estimate of it. 3) Techniques in which we reduce uncertainty by buying information. For example, prototyping is a way of buying in- formation to reduce our uncertainty about the likely success or failure of a multiprocessor operating system; by developing a rapid prototype of its high-risk elements, we can get a clearer picture of our likelihood of successfully developing the full operating system. In general, prototyping and other options for buying in- formation3 are most valuable aids for software engineering de- cisions. However, they always raise the following question: "how much information-buying is enough?" In principle,. this question can be answered via statistical de- cision theory techniques involving the use of Bayes' Law, which allows us to calculate the expected payoff from a software project as a function of our level of investment in a prototype or other information-buying option. (Some examples, of the use of Bayes' Law to estimate the appropriate level of invest- ment in a prototype are given in [l 1, ch. 201 .) Ln practice, the use of Bayes' Law involves the estimation of a number of conditional probabilities which are not easy to
3 Other examples of options for buying information to support software engineering decisions include feasibility studies, user sur- veys, simulation, testing, and mathematical program verification tech- niques.
Pitfall 1: Always use a simulation to investigate the feasibil- ity of complex realtime software. Simulations are often ex- tremely valuable in such situations. However, there have been a good many simulations developed which were largely an ex- pensive waste of effort, frequently under conditions that would have been picked up by the guidelines above. Some have been relatively useless because, once they were built, nobody could tell whether a given set of inputs was realistic or not (picked up by Condition 3). Some have been taken so long to develop that they produced their first results the week after the pro- posal was sent out, or after the key design review was com- pleted.(picked up by Condition 4). Pitfall 2: Always build the software twice. The guidelines indicate that the prototype (or build-it-twice) approach is often valuable, but not in all situations. Some prototypes have been
built of software whose aspects were all straightforward and
familiar, in which case nothing much was learned by building them (picked up by Conditions 1 and 2). Pitfall 3: Build the sofnvare purely top-down. When inter- preted too literally, the top-down approach does not concern itself with the design of low level modules until the higher levels have been fully developed. If an adverse state of nature makes such a low level module (automatically forecast sales volume, automatically discriminate one type of aircraft from another) impossible to develop, the subsequent redesign will generally require the expensive rework of much of the higher level design and code. Conditions 1 and 2 warn us to temper our top-down approach with a thorough top-to-bottom soft- ware risk analysis during the requirements and product design phases. Pitfall 4: Every piece of code should be proved correct. Correctness proving is still an expensive way to get informa- tion on the fault-freedom of software, although it strongly satisfies Condition 3 by giving a very high assurance of a pro- gram's correctness. Conditions 1 and 2 recommend that proof
techniques be used in situations where the operational cost of a software fault is very large, that is, loss of life, compromised national security, major financial losses. But if the operational cost of a software fault is small, the added information on fault-freedom provided by the proof will not be worth the in- vestment (Condition 4). Pitfall 5: Nominal-case testing is sufficient. This pitfall is just the opposite of Pitfall 4. If the operational cost of poten- tial software faults is large, it is highly imprudent not t o per- form off-nominal testing.
Summary: The Economic Value of lizforrnation ' Let us step back a bit from these guidelines and pitfalls. Put simply, we are saying that, as software engineers: "It is often worth paying for information because it helps us make better decisions." If we look at the statement in a broader context, we can see that it is the primary reason why the software engineering field
exists. It is what practicalIy all of our software customers say
when they decide to acquire one of our products: that i t is worth paying for a management information system, a weather forecasting system, an air traffic control system, an inventory controlsystem, etc., because it helps them make better decisions. Usually, software engineers are producers of management information to be consumed by other people, but during the software life cycle we must also be consumers of management information to support our own decisions. As we come t o ap- preciate the factors which make it attractive for us t o pay for processed information which helps us^ make better decisions^ as software engineers, we will get a better appreciation for what our customers and users are looking for in the information processing systems we develop for them.
6 ) Top-Down: An overall cost estimate for the project is
derived from global properties of the software product. The total cost is then split up among the various components.
- Bott0.m-Up: Each component of the software job is separately estimated, and the resuits aggregated to produce an estimate for the overall job. The main conclusions that we can draw from Table I are the following. None of the alternatives is better than the others from all aspects. The Parkinson and price-to-win methods are unaccept- able and do not produce satisfactory cost estimates.
The strengths and weaknesses of the other techniques are complementary (particularly the algorithmic models versus expert judgment and top-down versus bottom-up). Thus, in practice, we should use combinations of the above techniques, compare their results, and iterate on them where they differ.
TABLE I
STRENGTHS AND WEAKNESSES OF SOFTWARE COST-ESTIMATION METHODS
slbPmJeinp*r Assessment d drawnstancsr Calibrated to prrf not future NO bener ~h.n p r r c ~ ~ t s Biases.- i recall Analogy Based on m t i v e errperierwr, Parkinson Pnce to win Less dewed brPs Less stable
Fundamental Limitations o f Software Cost Estimution Techniques Whatever the strengths of a software cost estimation tech- nique, there is really no way we can expect the technique to compensate for our lack of definition or understanding of the software job to be done. Until a software specification is fully defined, it actually represents a range of software products, and a corresponding range of software development costs. This fundamental limitation of software cost estimation technology is illustrated in Fig. 3, which shows the accuracy within which software cost estimates can be made, as a func- tion of the software lifecycle phase (the horizontal axis), or of the level of knowledge we have of what the software is in- tended to do. This level of uncertainty is illustrated in Fig. 3 with respect to a human-machine interface component of the software.
CarcDt of Product^ Deuiled m i m Requirements~ p c i f i u t i o n s^ rpecifiut~onrd-i?^ ~ ~ a i t ~ u t ~ o r aipn.*^ Acceptedsoftmre 0 A A A A Feas~blhtv (^) requt~rnentsPlans and Productd-19" Dcta~led- 8 9 " Develoommt and test Phases md m i l a t o n s Fig. 3. Software cost estimation accuracy versus^ phase.
used for task scheduling, error handling, abort processing, and the like. These will be resolved by the end of the detailed de- sign phase, but there will still be a residual uncertainty about 10 percent based on how well the programmers really under- stand the specifications to which they are to code. (This factor also includes such consideration as personnel turnover uncer- tainties during the development and test phases.)
B. Algorithmic Models for Software Cost Estimation Algorithmic Cost Models: Early Development Since the earliest days of the software field, people have been trying to develop algorithmic models to estimate soft- ware costs. The earliest attempts were simple rules of thumb, such as: on a large project, each software performer will provide an average of one checked-out instruction per man-hour (or roughly 150 instructions per man-month); each software maintenance person can maintain four boxes of cards (a box of cards held 2000 cards, or roughly 2000 instructions in those days of few comment cards). Somewhat later, some projects began collecting quantita- tive data on the effort involved in developing a software product, and its distribution across the software life cycle. One of the earliest of these analyses was documented in 1956 in [8]. It indicated that, for very large operational software products on the order of 100 000 delivered source instructions (100 KDSI), that the overall productivity was more like 64 DSIJman-month, that another 100 KDSI of support-software would be required; that about 15 000 pages of documentation would be produced and 3000 hours of computer time consumed; and that the dis- tribution of effort would be as follows:
Program Specs: 10 percent Coding Specs: 30 percent
Coding: 10 percent Parameter Testing: 20 percent Assembly Testing: 30 percent
with an additional 30 percent required to produce operational specs for the system. Unfortunately, such data did not become well known, and many subsequent software projects went through a painful process of rediscovering them. During the late 1950's and early 1960's, relatively little progress was made in software cost estimation, while the fre- quency and magnitude of software cost overruns was becom- ing critical t o many large systems employing computers. In 1964, the U.S. Air Force contracted with System Develop- ment Corporation for a landmark project in the software cost estimation field. This project collected 104 attributes of 169 software projects and treated them to extensive statistical anal- ysis. One result was the 1965 SDC cost model [41] which was the best possible statistical 13-parameter linear estimation model for the sample data:
+9.15 (Lack of Requirements) (0-2)
- 10.73 (Stability of Design) (0-3) +0.5 1 (Percent Math Instructions) +0.46 (Percent StoragelRetrieva1Instructions) +0 -40 (Number of Subprograms) +7.28 (Programming Language) (0-1) -2 1.45 (Business Application) (0-1)
- 13.53 (Stand-Alone Program) (0.1)
- 12-35 (First Program on Computer) (0-1)
- 58.82 (Concurrent Hardware Development) (0-1) +30.6 1 (Random Access Device Used) (0-1)
Catcgorles C = Control I = Inputloutput P = Relport processor A = Algor~thm D = Data management T = Time cr~ticalpraessor Sample range excludes upper and lower 20 percentiles
New
Old
0 T (all) Category ( ).
/-
~ i g .4. TRW^ Wolverton model: Cost per object instruction versus rela- tive degree of difficulty.
10
ally. This, a 1000 object-instruction module of new data man- agement software of medium (50 percent) difficulty would be costed at $46/instruction, or $46 000. This model is well-calibrated to a class of near-real-time government command and control projects, but is less ac- curate for some other classes of projects. In addition, the model provides a good breakdown of project effort by phase and activity.
Easy Medium Hard 0 m 40 60 80 l ( Relative degree of d~fficulty:percent of total sample expcrlencmg this rate or less
In the late 1 9 7 0 ' ~ ~several software cost estimation models were developed which established a significant advance in the state of the art. These included the Putnam SLIM Model [44] , the Doty Model [27], the RCA PRICE S model [ 2 2 ] , the COCOMO model [I 11, the IBM-FSD model [53], the Boeing
model [9] , and a series of models developed by GRC [15]. A
summary of these models, and the earlier SDC and Wolverton models, is shown in Table 11, in terms of the size, program, computer, personnel, and project attributes used by each model to determine software costs. The first four of these models are discussed below.
The Pu tnam SLIM Model [44], [45] The Putnam SLIM Model is a commercially available (from Quantitative Software Management, Inc.) software product based on Putnam's analysis of the software life cycle in terms of the Rayleigh distribution of project personnel level versus time. The basic effort macro-estimation model used in SLLM is
where
Ss = number of delivered source instructions K = life-cycle effort in man-years td = development time in years Ck = a "technology constant."
Values of Ck typically range between 6 10 and 57 3 14. The
current version of SLIM allows one to calibrate Ck to past
projects or to past projects'or to estimate it as a function of a project's use of modern programming practices, hardware con- straints, personnel experience, interactive development, and other factors. The required development effort, DE, is esti- mated as roughly 40 percent of the life-cycle effort for large
