Quality Assurance Plan for Data Acquisition and Management System, Assignments of Software Engineering

Download Quality Assurance Plan for Data Acquisition and Management System and more Assignments Software Engineering in PDF only on Docsity!

Quality Assurance Plan for the Data

Acquisition and Management System for

Business

November 24, 2019

DAMS Team

Version 1.

Team Members:

Sidrah Kaleem Amna Adeela Sara Amin 1

REVISION LIST

Revision Date Author Comments 0.1 10/6/2019 Adeela Introduction, background section completed 0.2 11/5/2019 Sara Amin Section 1 edited 0.3 11/10/2019 Amna Section 3 completed 0.4 11/11/2019 Sidrah Kaleem Add Review process 0.5 11/13/2019 Adeela Tool and Techniques section completed. 0.6 11/13/2019 Amna Added some comments. 0.7 11/14/2019 Sidrah Kaleem QA qualities Reviewed. 0.8 11/15/2019 Adeela Check Grammar, Updated Documentation. 0.9 11/16/2019 Sara Amin Tools and Techniques section is modified. 1.0 11/18/2019 Sidrah Kaleem Updated the documents depending on Stuart feedback 1.1 11/21/2019 Amna Updated the test coverage information per Stuart feedback

APPROVAL BLOCK

Version Comments Responsible Party Date 0.9 I have reviewed the QA plan, some comments are embedded in the attached. The parts of the plan I found less convincing were those describing the QA metrics (Goals and process measures). As indicated in my comments it was not really clear what, if anything, one could conclude about the actual quality of the software even if all of the quality goals are satisfied. The Process Metrics did not given enough specifics to be clear how the cited measures will be used. I think these Andleeb Zaib Shah 11/16/

Table of Contents

• 1 INTRODUCTION..................................................................................................................................................................
• 2 REFERENCED DOCUMENTS...........................................................................................................................................
• 3 QUALITY ASSURANCE STRATEGY...............................................................................................................................
• 4 DOCUMENTATION............................................................................................................................................................
• 4.1 PURPOSE................................................................................................................................................................................
• 4.2 MINIMUM DOCUMENTATION REQUIREMENTS........................................................................................................................
  • 4.2.1 Concept of Operations (ConOps).............................................................................................................................
  • 4.2.2 Software Requirements Document (SRS)................................................................................................................
  • 4.2.3 Software Test Reports...............................................................................................................................................
  • 4.2.4 Software Architecture and Design............................................................................................................................
  • 4.2.5 User Documentation.................................................................................................................................................
  • 4.2.6 Other Documents......................................................................................................................................................
• 5 GOALS....................................................................................................................................................................................
• 5.1 QA GOALS OF EACH PHASE...................................................................................................................................................
• 6 REVIEWS AND AUDITS.....................................................................................................................................................
• 6.1 WORK PRODUCT REVIEWS....................................................................................................................................................
• 6.2 QUALITY ASSURANCE PROGRESS REVIEWS........................................................................................................................
• 7 TOOLS AND TECHNIQUES.............................................................................................................................................
• 7.1 TOOLS AND TECHNIQUES FOR ASSURING QUALITY OF FUNCTIONAL REQUIREMENTS.........................................................
• 7.2 TOOLS AND TECHNIQUES FOR ASSURING THE QUALITY ATTRIBUTE REQUIREMENTS..........................................................
• 8 TESTING STRATEGY.......................................................................................................................................................
• 8.1 UNIT TESTING......................................................................................................................................................................
• 8.2 INTEGRATION TESTING........................................................................................................................................................
• 8.3 ACCEPTANCE TESTING.........................................................................................................................................................
• 8.4 REGRESSION TESTING..........................................................................................................................................................
• 8.5 TEST COMPLETION CRITERIA..............................................................................................................................................
• 9 ORGANIZATION................................................................................................................................................................
• 9.1 AVAILABLE RESOURCES THAT TEAM INTENDS TO DEVOTE.................................................................................................
• 9.2 QUALITY ASSURANCE TEAM................................................................................................................................................
• 9.3 MANAGING OF THE QUALITY OF ARTIFACTS......................................................................................................................
• 9.4 PROCESS FOR PRIORITIZING QUALITY ASSURANCE TECHNIQUES.......................................................................................
• 9.5 QA STRATEGY BREAK DOWN INTO TASKS...........................................................................................................................
• 9.6 QUALITY ASSURANCE PROCESS MEASURES.......................................................................................................................
• 10 GLOSSARY........................................................................................................................................................................
• 10.1 DEFINITION........................................................................................................................................................................
• 10.2 ACRONYMS........................................................................................................................................................................

1 INTRODUCTION

Purpose: The purpose of this quality assurance (QA) plan is to establish the QA practices and procedures for the data acquisition and management system that will be used to collect, manage, archive, retrieve, and analyze data from the field instrumentation that will monitor activities at the site. This plan considers all hardware, software, and documentation associated with the monitoring activities, including the design, implementation, testing, installation, and operation of the end-to-end system from a total quality management perspective. It also establishes the roles and responsibilities for configuration management during the development, release, and maintenance of all computer software and supporting documentation. Scope: This QA plan is a comprehensive guide; it consolidates and discusses issues traditionally addressed in separate documents, such as configuration management, validation and verification, and life-cycle support. To the extent possible, the practices and procedures described herein have been streamlined to meet the requirements of the monitoring project. Because of the dynamic nature of this project, this QA plan will be updated as necessary to address new technical, administrative, and programmatic issues as they arise. This SQAP (Software Quality Assurance Plan) covers all important aspects of software development; i.e. requirements analysis, architecture and design, implementation, testing and verification, and user acceptance. Background and Context The concept of data management arose in the 1980s as technology improved to storage. Those suggesting that data management was more important in business process management used arguments such as "a customer's home address is stored in 75 (or some other large number) places in our computer systems." As application software evolved into real-time, interactive usage, it became obvious that both management processes were important. If the data was not well defined, the data would be mis-used in applications. If the process wasn't well defined, it was impossible to meet user needs. The use of collaboration tools such as MariaDB provides a rich database of developer interactions and artifacts. This suggests that it may be possible to monitor progress and compare it to the results of past projects to alert users when signs of trouble are detected. Project Objectives The data acquisition and management system is designed to accurately and reliably collect, manage, archive, retrieve, and analyze data from the field instrumentation that will monitor activities at the site. Information from the sites will also be managed and analyzed by the system. The purpose of the Data acquisition and Management system is to do this by collecting and modeling historical project data to predict, in real time, the health of an in-progress project and to alert the project stakeholders when signs of trouble are detected. Architectural Objectives The data acquisition and management system is composed of three major subsystems, each consisting of hardware and software components. Interface to software to monitoring and logging the products or instruments. Data acquisition computer for uploading data recorded by the data loggers. data analysis computer for archiving data, performing statistical analyses. Thus, the system can work against a different collaboration software. In this regard, different modules of the system are decoupled to achieve this architectural objective.

 We will select the test factors and rank them. The selected test factors such as reliability, maintainability, portability or etc, will be placed in the matrix according to their ranks.  Use of standardized, accepted, and published statistical methods and treatments  The use of tested, peer reviewed, and published statistical analysis software  Another step is to identify the business risks of the software deliverables. The risks will be ranked into three ranks such as high, medium and low.  Use of experienced statisticians to perform the statistical analysis activities.  The last step is to decide the test phase in which risks will be addressed. In this step, we will decide which risks will be placed in each development phase. On the basis of quality requirements, items that do not provide accurate results will be replaced by items that satisfy all requirements. Verification will be based on calculations and visual inspections of equipment and data, as appropriate. Equipment that has failed a performance test will be clearly marked as inoperable and removed from inadvertent use until it has been repaired, replaced, and/or recalibrated. The risks would also be accompanied with their mitigation strategies and in case the risk materialized into a problem, the respective mitigation would be applied. It is for these reasons, that a mention is made about the matrix here in a separate section of the document and not mixed with other sections of the document to avoid repetition.

4 DOCUMENTATION

4.1 PURPOSE

This section shall perform the following functions: a) Identify the documentation governing the development, verification and validation, use, and maintenance of the software. b) List which documents are to be reviewed or audited for adequacy. For each document listed, identify the reviews or audits to be conducted and the criteria by which adequacy is to be confirmed. 4.2 MINIMUM DOCUMENTATION REQUIREMENTS To ensure that the implementation of the software satisfies the technical requirements, the following documentation is required as a minimum. 4.2.1 Concept of Operations (ConOps) The ConOps may be written by the business owner (internal or external), the customer, or by both. It should address the basic expected feature sets and constraints imposed on the system’s operation. Each requirement should be uniquely identified and defined such that its achievement is capable of being objectively measured. An active review process is to be used to ensure suitability and completeness of user requirements.

4.2.2 Software Requirements Document (SRS) Software specification review is to be used to check for adequacy and completeness of this documentation. The Software Requirements Document, which defines all the functional requirements, quality attributes requirements, product descriptions and constraints of the project. 4.2.3 Software Test Reports Software Test Reports are used to communicate the results of the executed test plans. This being the case, a particular report should contain all test information that pertains to the current system aspect being tested. The completeness of reports will be verified in walkthrough sessions. 4.2.4 Software Architecture and Design Software Architecture and Design reviews are to be used for adequacy and completeness of the design documentation. This documentation should depict how the software will be structured to satisfy the requirements. The system should describe the components and subcomponents of the software design, including databases and internal interfaces. 4.2.5 User Documentation User documentation guides the users in installing, operating, managing, and maintaining software products. The user documentation should describe the data control inputs, input sequences, options, program limitations, and all other essential information for the software product. All error messages should be identified and described. All corrective actions to correct the errors causing the error messages shall be described. 4.2.6 Other Documents

1. Software Project Management Plan (SPMP)

5 GOALS

5.1 QA GOALS OF EACH PHASE

Phase Goals Requirement gathering SRS should be approved for no defects by the customer and engineering managers. Architecture It should not have any defects per architectural representation during its formal technical review (FTR). Development Application should not have more than 10 defects per 1 KLOC Testing All tested work products should be checked for having 0 defects in documents, closed defects should be at least 80% of the previous build and new defect should be maximum 20% of the previous build

Quality Assurance (Status or Criteria) (Standards or Method) Requirements (Software Requirements Specification) After a new release or modification The Requirements Specification document is reviewed and approved by the assigned reviewer(s). The reviewed document is presented to the customer for acceptance. The Requirements Specification document forms the baseline for the subsequent design and construction phases. Software Design Review After a new release or modification The Architecture/Design phase is carried out using an appropriate system design methodology, standards and guidelines, taking into account the design experience from past projects. The design output is documented in a design document and is reviewed by the Reviewer to ensure that:  The requirements including the statutory and regulatory requirements as stated in the Requirements Specification document, are satisfied  The acceptance criteria are met  Appropriate information for service provision (in the form of user manuals, operating manuals, as appropriate) is provided. Acceptance for the design document is obtained from the customer. The Design Document forms the baseline for the Construction phase. Construction (Code) After a new release or modification The Project Team constructs the software product to be delivered to meet the design specifications, using:  Suitable techniques, methodology, standards and guidelines  Reusable software components, generative tools, etc. as appropriate  Appropriate validation and verification techniques as identified in the Project Plan. Testing and Inspection After a new release or modification Before delivery of the product, SQA ensures that all tests, reviews, approvals and acceptances as stipulated in the Project Plan have been completed and documented. No product is delivered without these verifications. 6.2 QUALITY ASSURANCE PROGRESS REVIEWS In order to remove defects from the work products early and efficiently and to develop a better understanding of causes of defects so that defects might be prevented, a methodical examination of software work products is conducted in projects in the following framework:

1. Reviews of Project Plans and all deliverables to the customer are carried out as stated in the Quality Plan of the project. A project may identify additional work products for review.
2. The program manager and other line managers will inform personnel about the acceptable bounds of work performance, including the performance objectives for which personnel are held accountable.
3. Reviews emphasize on evaluating the ability of the intended product to meet customer requirements. The reviewer also checks whether the regulatory statutory and unstated requirements, if any, have been addressed.

4. Work process reviews will be held as needed to help workers meet performance requirements and help reduce risks.
5. Personnel independent of the activity being performed carry out the reviews.
6. Reviews focus on the work product being reviewed and not on the developer. The result of the review in no way affects the performance evaluation of the developer.
7. The goal is to give employees regular feedback on the acceptability of their work and opportunities for improvement.
8. The defects identified in the reviews are tracked to closure. If a work product is required to be released without tracking the defects to closure, a risk analysis is carried out to assess the risk of proceeding further.
9. Before undertaking a task, the performer will ensure that preconditions have been met, there are no interferences that would prevent the effective pursuit of the work, and the performance requirements and QA actions are understood.

7 TOOLS AND TECHNIQUES

The project uses the following strategy for selection of the tool on the project:

1. On the basis of quality requirements, items that do not provide accurate results will be replaced by items that satisfy all requirements.
2. The testing tool is selected based on core functionality of the project.
3. Verification will be based on calculations and visual inspections of equipment and data, as appropriate.
4. The usage of the tool is mapped to the life cycle phase in which the tool will be used.
5. Matching the tool selection criteria to the expertise of the QA team.
6. Software written in-house will be tested extensively before being put into use.
7. Selection of the tool not only depends upon affordability but also depends on the quality standard requirement of the project. 7.1 TOOLS AND TECHNIQUES FOR ASSURING QUALITY OF FUNCTIONAL REQUIREMENTS In order to ensure the quality of functional requirements, the team has applied the following techniques:
8. Peer review: A minimum of one Structured will be performed on each lifecycle deliverable. This provides a facility for all team members to review the contents online at the same-time as well as provide comments on each other's work. This review mainly focus on the integrations of the parts that we design (such as interfaces, form, database.) We will ask other team members to do the walkthroughs with the presence of coder.
9. Customer review: After peer review, the team sends requirements and other documentation to the project manager. The manager is requested to review the document with a specific perspective (role such as user) as well as an instructor's viewpoint. The project manager feedback is discussed and included in the document and then sent again for final review.
10. Traceability Checking: Once requirements are documented and reviewed, a requirements traceability matrix is developed. The team intends to use the traceability matrix to trace the source of any

constraints in time and resources. during development and in production. Availability Technological platform or the administrative staff is there when it is needed and whether it delivers the defined services to the agreed service level These commands and system logs will help to find the availability of the server and application system. Usability Whether staff &customers are able to carry out specific tasks effectively, efficiently and with satisfaction. User questionnaire or surveys. These techniques will help to understand the user specific requirements and how the system is user friendly. Concept of Operations document that describes various use cases will be useful to refer while testing usability.

8 TESTING STRATEGY

Testing for the project seeks to accomplish two main goals:

1. Detect failures and defects in the system.
2. Detect storage faults.
3. Detect inconsistency between requirements and implementation. To achieve these goals, the testing strategy for the required system will consist of four testing levels. These are unit testing, integration testing, acceptance testing, and regression testing. The following subsections outline these testing levels, which development team roles are responsible for developing and executing them, and criteria for determining their completeness. 8.1 UNIT TESTING The target of unit tests is a small piece of source code. Unit tests are useful in detecting bugs early and also in validating the system architecture and design. These tests are done one function at a time and written by the developer. Ideally each logic path in the component and every line of code are tested. However, covering every line of code with unit tests is not time or cost effective in most cases. Code coverage goals will be defined to ensure that the most important code is well covered by tests while still making efficient use of developer time. Unit testing will be done by the developers during each of the three development phases outlined in the Project Plan from October. 22 to December. 10. All unit tests must be executed and passing before each check-in to the source control system. Unit tests will also be run automatically as part of the continuous integration process. The results of these test runs will be stored by the continuous integration system and emailed to the development team.

8.2 INTEGRATION TESTING

Integration testing will execute several modules together to evaluate how the system as a whole will function. Integration tests will be written and executed by the testing team. Attempting to integrate and test the entire system all at once will be avoided as it makes finding the root cause of issues more difficult and time consuming. Instead, integration tests will be done at specific points, ideally where one component interacts with another through an interface. Integration tests will focus on these specific points of interaction between two components. This testing of interaction between two modules ultimately leads to an end-to-end system test. Each test is written to verify one or more requirements using the scenarios or use cases specified in the requirements document. Integration tests also include stress or volume testing for large numbers of users. 8.3 ACCEPTANCE TESTING Acceptance testing is functional testing that the customer uses to evaluate the quality of the system and verify that it meets their requirements. The test scripts are typically smaller than integration or unit testing due to the limited time resources of the customer. Acceptance tests cover the system as a whole and are conducted with realistic data using the scenarios or use cases specified in the requirements as a guide. 8.4 REGRESSION TESTING The purpose of regression testing is to catch any new bugs introduced into previously working code due to modifications. As such, the regression test suite will be run every time the system changes. Regression tests will be created and run by the testing team. Regression testing will consist of running previously written automated tests or reviewing previously prepared manual procedures. It is common for bug fixes to introduce new issues and therefore several “test/fix” cycles will be planned and conducted during regression testing. 8.5 TEST COMPLETION CRITERIA In each development phase, tests will be conducted and their completeness will be judged by the following criteria:  Unit Testing : Complete when: o At least 60% of the code lines (including all critical sections) has been tested o All major and minor bugs found have been logged and fixed.  Regression Testing : Complete when: o At least 90% of modules functions have been covered, including all modified modules, o At least two test/fix cycles have been completed. o All issues/defects have been logged and corrected.  Integration Testing : Complete when: o 100% of module interfaces have been tested.  Acceptance Testing : Complete when: o The customer is satisfied that the product has met the agreed upon requirements criteria.

Quality Reviewer Reviews and identifies defects in project artifacts Provides feedback for improved quality in software artifacts SQA Team Member Responsible for providing support during SQA activities by carrying out assigned tasks as they relate to quality of the system Throughout the SQA process each team member is responsible for knowing:  Their Roles and Responsibilities  Goals of each activity with which they are associated  Processes that are to be carried out 9.3 MANAGING OF THE QUALITY OF ARTIFACTS When changes are made to the system, reviews/tests will be conducted on the artifacts affected by those changes. All testing and review activities shall have documentation indicating: Process How a particular method or technique should be carried out Goals This will state the purpose of quality activities associated with the artifacts. Results Outputs of the methods and techniques and Analysis and Conclusions that are formed as a result of them Reviewer Roles and Responsibilities of SQA team members in relation to artifacts Notes Any comments concerning the artifact that will be useful for successfully using the artifact A code/document management system shall be in place, which enables the team to easily revert to a previous version in the event issues are discovered in connection with said changes. 9.4 PROCESS FOR PRIORITIZING QUALITY ASSURANCE TECHNIQUES This section contains a step-by-step outline of the process employed to prioritize the QA techniques used for evaluation of the process artifacts.

1. Create a prioritized checklist of testing characteristics/interests of the system; these will be relative to the requirements and quality attributes.
2. Describes the QC activities to be conducted before, during, and after data collection to verify data are of acceptable quality and are complete and correct.

3. Choose techniques (i.e. design and code reviews) that seem to fit in line with the characteristics identified (i.e. from common knowledge or based on research);
4. SQA team should engage in dialogue and assign weight to each technique for each checklist item in terms of how useful each technique is to serve the purposes of testing relative to the criteria(in the checklist) that are of interest; the rating will be 1-10 with 1 being the weakest and the strongest
5. Outlines the acceptance processes and criteria that will be used to determine if data is fit for use. Includes data sampling, review, and checking processes, and error resolution procedures for data not meeting criteria.
6. SQA team conducts an assessment session of techniques that could be useful for testing purposes; the SQA leader will be in charge of this session
7. Team should come to an agreement about a specific technique and engage in dialogue to address any issues with a particular technique
8. Identifies the quality-related responsibilities of the data collection team, including Agency and Data Collection Contractor members.
9. Weighting and majority team agreement should be deciding factor on a technique
10. Outlines the documentation expected for QM activities, and format for QM logging, tracking and reporting.

and should at least 80% of the number of defect Defect Density should be at most 10 defects per 10 statements and decreasing overtime Type of Errors identified Percentage of type of defects each build should be : 5% critical 20% Major and 75% Minor and critical should tend to be 0 as possible with each final build Follow up and Tracking: When reviews and testing are completed, a measure of success or failure will be assigned. If successful, the process would ensure that the work product is packaged for release or documents are base-lined. If failure occurs, the bugs will be tracked in a defect repository against the artifact in question. Appropriate actions will be carried out to ensure reevaluation and corrections are made. Exit Criteria: The exit criteria as defined in the plan depends upon the goals set for the specific sections of the plan. Thus, whenever the process of review or testing takes place, the goal, specific to a deliverable or work product being tested or reviewed, would serve as the exit criteria for that section.

10 GLOSSARY

10.1 DEFINITION

Audit An independent examination of a software product, software process, or set of software processes to assess conformance with specifications, standards, contractual agreements, or other criteria. Project support functions All project support documents will be completed in applicable phases Inspection A visual examination of a software product to detect and identify software anomalies, including errors and deviation from standards and specifications. Inspections are peer examinations led by impartial facilitators who are trained in inspection technique Review A process or meeting during which a software product is presented to project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval Software documentation Documentation such as project charter, Business Requirement Document, Functional specification document, cost benefit Analysis, Technical Specification

document, detail design document, Test Plan, Implementation Plan, and Benefit Realization document. Walk-through A static analysis technique in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems