Download Statistical Software Quality Assurance-Methods Of Software Engineering-Lecture Notes and more Study notes Software Engineering in PDF only on Docsity!
Lecture No. 30
Statistical Software Quality Assurance
Statistical SQA is a technique that measures the quality in a quantitative fashion. It implies that information about defects is collected and categorized and an attempt is made to trace each defect to underlying cause. It uses Pareto Principle to identify vital causes (80% of defects can be traced to 20% of causes) and moves to correct the problems that have caused the defects.
Example
Let us assume that information about defects is collected for one year and categorized as follows:
- Incomplete or erroneous specifications (IES)
- Misinterpretation of customer communication (MCC)
- Intentional deviation from specifications (IDS)
- Violation of programming standards (VPS)
- Error in data representation (EDR)
- Inconsistent component interface (ICI)
- Error in digital logic (EDL)
- Incomplete or erroneous testing (IET)
- Inaccurate or incomplete documentation (IID)
- Error in programming language translation of design (PLT)
- Ambiguous or inconsistent HCI (HCI)
- Miscellaneous (MIS)
The following data is collected for these categories
Error Category (^) Serious Moderate Minor Sub Total
IES 34 68 103 205
MCC 12 68 76 156
IDS 1 24 23 48
VPS 0 15 10 25
EDR 26 68 36 130
ICI 9 18 31 58
EDL 14 12 19 45
IET 12 35 48 95
IID 2 20 14 36
PLT 15 19 26 60
HCI 3 17 8 28
MIS 0 15 41 56
Total 128 379 435 942 We can easily see the following:
- IES, MCC, and EDR are the vital errors – cause 53% of all errors
- IES, EDR, PLT, and EDL are vital if only serious errors are considered
We now start corrective action focused on vital few. For example for EDR we review the data representation techniques to identify the possible improvement areas and adopt a use case tool for data modeling and perform stringent data design reviews.
Error Index (EI)
Another statistical technique known as Error Index (EI) is used to develop an overall indication of improvement in software quality. The EI is computed as follows:
Let
- Ei – the total number of errors uncovered during the ith step in the SE process
- Si – number of serious errors
- Mi – number of moderate errors
- Ti – number of minor errors
- PS – product size at the ith step
- ws, wm, wt – weighting factors for serious, moderate, and minor errors. Recommended values for these are 10, 3, 1 respectively.
At each step of the software process a Phase Index is computed as:
PIi = ws(Si/Ei) + wm(Mi/Ei) + wt(Ti/Ei)
Now EI is computed as the cumulative effect on each PIi = ∑(I x PIi)/PS
It is important to note that weighting errors encountered in the SE processes more heavily than those encountered earlier. As stated earlier, it can be used to develop an overall indication of improvement in software quality.
Software Reliability
Software reliability is another very important quality factor and is defined as probability of failure free operation of a computer program in a specified environment for a specified time. For example, a program X can be estimated to have a reliability of 0.96 over 8 elapsed hours.
Software reliability can be measured, directed, and estimated using historical and development data. The key to this measurement is the meaning of term failure. Failure is defined as non-conformance to software requirements. It can be graded in many different ways as shown below:
- From annoying to catastrophic
- Time to fix from minutes to months
- Ripples from fixing
Lecture No. 31
Software Safety
Software Safety is a software SQA activity that focuses on identification of potential hazards that may affect software negatively and cause an entire system to fail. Modeling and analysis process is conducted as part of software safety and hazards are identified and categorized by criticality and risk.
Example
Let us assume that the following hazards are associated with a computer-based cruise control for an automobile:
- Causes uncontrolled acceleration that cannot be stopped
- Does not respond to depression of brake pedal
- Does not engage when switch is activated
- Slowly loses or gains speed
Once system-level hazards are identified, analysis techniques are used to assign severity, and probability of occurrence. This technique is similar to risk analysis. To be effective, software must be analyzed in the context of the entire system. Analysis techniques such as fault tree analysis can be used to predict the chain of events that can cause hazards and the probability that each of these events will occur to create the chain. Once hazards are identified and analyzed, safety-related requirements can be specified for the software.
Reliability and safety are closely related. Software reliability uses statistical techniques to determine the likelihood that a software failure will occur. Occurrence of a software failure does not necessarily result in a hazard or mishap. On the other hand, software safety examines the ways in which failures result in conditions that can lead to a mishap.
Lecture No. 32
Poka-Yoke (Mistake-Proofing)
Poka-yoke is a QA technique developed by Shingo at Toyota in 1960‘s. Poka-yoke devices are mechanisms that lead to prevention of potential quality problem before it occurs or the rapid detection of quality problems if they are introduced.
- Examples:
- Light on if the car door is not properly closed
- Warning beep if the engine is turned-off when lights are on
Characteristic of a Poka-yoke device
- It is simple and cheap
- It is part of the process
- It is located near the process task where the mistake occurs
Example of Poka-yoke in software
Let us assume that a\A software products company sells application software to an international market. The pull-down menus and associated mnemonics provided with each application must reflect the local language. For example, the English language menu item for ―Close‖ has the mnemonic ―C‖ associated with it. When the application is sold to Urdu speaking users, the same menu item is ―Band Karen‖ with mnemonic ‖ب‖. To implement the appropriate menu entry for each locale, a ―localizer‖ translates the menus accordingly. The problem is to ensure that each menu entry conforms to appropriate standards and there are no conflicts, regardless of the language used. We can consider a prevention device to write a program to generate mnemonics automatically, given a list of labels in each menu. It would prevent mistakes, but the problem of choosing a good mnemonic is difficult and the effort required would not be justified by the benefit gained. It is therefore not a good Poka-yoke device as it is not simple. We now consider another prevention device. In this case we write a program to prevent the localizer from choosing mnemonics that did not meet the criteria. It would prevent mistakes but the benefit is minimal as incorrect mnemonics are easy enough to detect and correct after they occur. So once again this is not a good choice. We now consider a detection device. In this case we write a program to verify that the chosen menu labels and mnemonics meet the criteria. We run this program to detect errors. Errors are sent back to localizer. Since this device requires a lot of back and forth movement, it violates the principle of co-location. We now modify it to the following detection device. We write a program to verify that the chosen menu labels and mnemonics meet the criteria. The localizer would run this program to detect errors. It would give quick feedback and hence it is a good Poka-yoke device.
o I can‘t figure-out the changes made to the previous version. o I can‘t go back
Then there are software delivery problems.
―Which configuration does this customer have?‖ ―Did we deliver a consistent configuration?‖ ―Did the customer modify the code?‖ ―The customer skipped the previous two releases. What happens if we send him the new one?‖ Shipped the wrong version to the client.
This is not all. There may be more chaos in the following shapes and forms:
The design document is out of sync with programs I don‘t know if all the changes that were suggested have been incorporated Which code was sent to testing?
SCM is a function that, if implemented, will reduce these problems to a minimal level.
Configuration management
As defined by CMM, the purpose of SCM is to establish and maintain the integrity or software products through the project‘s life cycle. Configuration management is concerned with managing evolving software systems. It acknowledges that system change is a team activity and thus it aims to control the costs and effort involved in making changes to a system. SCM involves the development and application of procedures and standards to manage an evolving software product and is part of a more general quality management process. Baseline When released to CM, software systems are called baselines and serve as the starting point for further development.A baseline is a software configuration management concept that helps us to control change without seriously impeding justifiable change. It is defined by IEEE as:
A specification or a product that has been formally reviewed and agreed upon, that thereafter serves as the basis for further development, and that can be changed only through formal change control procedures. Software Configuration Item (SCI) A Software Configuration Item is the information that is created as part of the software engineering process. Typical SCIs include requirement specifications, design specification, source code, test cases and recorded results, user guides and installation manuals, executable programs, and standards and procedures (for example C++ design guidelines).
Software Configuration Management Tasks
Software configuration management tasks include:
Identification Version Control Change Control Configuration Auditing Reporting
Identification addresses how does an organization identify and manage the many existing versions of a program in a manner that will enable changes to be accommodated efficiently?
Version Control talks about how does an organization control changes before and after software is released to a customer? It is actually a combination of procedures and tools to manage different versions of the software configuration.
Clemm states that Configuration management allows the user to specify alternative configurations of the software system through the selection of the appropriate versions. This is supported by associating with each software version, and then allowing configuration to be specified and constructed by describing the set of desired attributes.
A version has many different attributes. In the simplest form a specific version number that is attached to each object and in the complex form it may have a string of Boolean variables (switches) that indicate specific types of functional changes that have been applied to the system.
The Change Control process addresses the important question of who has the responsibility for approving and ranking changes. Configuration Auditing deals with ensuring that the changes have been made properly and finally Reporting talks about the mechanism used to apprise others of changes that are made. Configuration Identification involves identification of a tool for SCM. Then a baseline is established and identified which is then used to identify configurable software items. At the minimum, all deliverables must be identified as configurable items. This includes design, software, test cases, tutorials, and user guides.
Change control
James Back points out the difficulties related to change control as follows:
Change control is vital. But the forces that make it necessary also make it annoying. We worry about change because a tiny perturbation in the code can cause a big failure in the product. But it can also fix a big failure or enable wonderful new capabilities. We worry about change because a single rogue developer could sink the project; yet brilliant ideas originate in the minds of those rogues, and a burdensome change control process could discourage them from doing creative work.
That is, like all engineering activities, change control is the name of a balancing act. Too much or too little change control creates different types of problems as uncontrolled change rapidly leads to chaos.
The change control process is elaborated in the following sub-section.
