Software Quality Assurance Plan of WorkSource Integrated Technology (WIT), Summaries of Quality Management

Download Software Quality Assurance Plan of WorkSource Integrated Technology (WIT) and more Summaries Quality Management in PDF only on Docsity!

WorkSource Integrated Technology (WIT)

Software Quality Assurance Plan

Version 0.

4/13/

WorkSource Integrated Technology (WIT) Date: 4/13/

DOCUMENT CONTROL / REVISION HISTORY

TITLE Quality Assurance Plan RELATED DELIVERABLE Project Management Plan REPOSITORY LOCATION TBD URL TBD

Date Version Description Author

1/27/2015 0.01 Initial Draft Mary Mueller

2/6/2015 0.02 Updated Draft Mike Juhl

2/11/2015 0.03 Updated Draft Mike Juhl

2/12/2015 0.04 Draft review edits Mary Mueller

2/21/2015 0.05 Updated Draft Mike Juhl

2/27/201 5 0 .06 Updated Draft Natajaran Sangeetha

3/3/2015 0 .07 Updated Draft Jennifer Forrest

3/10/2015 0 .08 Updated Draft Jennifer Forrest

4/13/2015 0.08 DRAFT Approved/Final Mike Juhl

• WorkSource Integrated Technology (WIT) Date: 4/13/
• 1. INTRODUCTION __________________________________________________________
     • 1.1 Purpose
     • 1.2 Scope
     • 1.3 Overview
     • 1.4 References
     • 1.5 Definitions
• 2. QUALITY OBJECTIVES _____________________________________________________
     • 2.1 Product Quality
     • 2.2 Process Quality
• 3. MANAGEMENT ____________________________________________________________
     • 3.1 Organization
     • 3.2 Tasks and Responsibilities
• 4. QUALITY RELATED DOCUMENTATION ___________________________________
     • 4.1 Quality records
     • 4.2 Meeting Minutes ............................................................................................................
     • 4.3 Requirements Traceability Documentation ..................................................................
     • 4.4 Formal Deliverable Review and Acceptance .................................................................
     • 4.5 Security Assessment.......................................................................................................
     • 4.6 Configuration and Code Quality Audits (IV&V) ..........................................................
     • 4.7 Defect Status and Severity Summaries ..........................................................................
     • 4.8 Testing Standards, Evaluation Summaries and Reports ..............................................
     • 4.9 QA Reports ....................................................................................................................
• 5. STANDARDS AND GUIDELINES ___________________________________________
• 6. REVIEWS AND AUDITS ____________________________________________________
     • 6.1 Project Reviews ..............................................................................................................
     • 6.2 Formal Reviews..............................................................................................................
     • 6.3 Informal Reviews ...........................................................................................................
• 7. TOOLS, TECHNIQUES, AND METHODOLOGIES ____________________________
     • 7.1 Tools ..............................................................................................................................
     • 7.2 Techniques.....................................................................................................................
     • 7.3 Methodologies ...............................................................................................................
• 8. METRICS _________________________________________________________________
  • WorkSource Integrated Technology (WIT) Date: 4/13/
    • 8.1 Problem and Reporting Metrics ....................................................................................
    • 8.2 Other metrics .................................................................................................................
• 9. EVALUATION AND TEST __________________________________________________
     • 9.1 Unit Testing ...................................................................................................................
     • 9.2 Build and Deployment Testing .....................................................................................
     • 9.3 Integration Testing ........................................................................................................
     • 9.4 Business Functional Testing (BFT) ..............................................................................
     • 9.5 Regression Testing ........................................................................................................
     • 9.6 Technical Testing
     • 9.7 Enterprise (System) Testing ..........................................................................................
     • 9.8 User Acceptance Testing (UAT) ...................................................................................
• 10. PROBLEM RESOLUTION AND CORRECTIVE ACTION ______________________
• 11. CONFIGURATION MANAGEMENT_________________________________________
• 12. TRAINING ________________________________________________________________

WorkSource Integrated Technology (WIT) Date: 4/13/

1. INTRODUCTION

1.1 Purpose

The purpose of this plan is to specify how Software Quality Assurance (SQA) and Project Quality Assurance (PQA) will be performed throughout the WorkSource Integrated Technology (WIT) Project. This plan describes the SQA activities to be performed and designates a set of standardized techniques for performing those activities. SQA and the Project Management Office (PMO) will ensure that project tasks are performed in accordance with this plan and the Project Management plan.

1.2 Scope

This plan applies to all stages of the WIT project. Although the solution to be implemented is a Commercial off the Shelf (COTS) product with limited custom development, proof of adherence to SQA and PQA standards will need to be obtained in order to satisfy state and ESD QA guidelines have been met. Adherence to this plan will show satisfaction of meeting the state and ESD QA guidelines. Maintenance of the Software Quality Assurance Plan will continue throughout the project as needed, based on the results of project execution and in accordance with best-practices. The plan will be reviewed and updated, if needed, on at least a bi-monthly basis by SQA in coordination with the PMO.

1.3 Overview

The Software Quality Assurance Plan is organized into the following sections: Quality Objectives - Describes the objectives of the project with regard to product and process quality. Management - Describes tasks and responsibilities for project participants with regard to quality assurance. Review and Audit Plan - Describes the type of reviews and audits to be conducted to ensure product quality. Quality Related Documentation - Describes the documentation produced to support quality assurance activities. Standards and Guidelines - Identifies the standards and guidelines that must be adhered to by the project. Metrics - Defines high level metrics employed during the project. Evaluation and Test - Describes the type of testing to be performed to evaluate and verify product (system) quality. Problem Resolution and Corrective Actions – Describes the management and resolution of problems and issues. Tools, Techniques and Methodologies – identifies the tools to be used for quality related activities, techniques for quality assurance, and methodologies or procedures to be adhered to when executing and evaluating project activities. Quality Records - Describes the records that will be maintained to track formal Quality Assurance activities performed.

WorkSource Integrated Technology (WIT) Date: 4/13/

Studio. If a defect is encountered, depending on the severity, we either postpone it or request another build. If postponed, the defect will be added to a future release. If another build is accepted, we then start our regression cycle over again. Once regression testing is completed and signed off, code would then be production ready and approved for deployment.  Technical Testing - Technical testing is performed to ensure that the system conforms to certain supplementary specifications. This may include performance, load, and volume testing used to measure the application response, execution-time, performance under stress, and software limits.  Enterprise (System) Testing - Enterprise testing ensures the delivered application performs as expected when fully integrated with other enterprise applications.  User Acceptance Testing (UAT) - User Acceptance testing is performed to ‘accept’ the system prior to moving to production. Acceptance test includes exercise of functional test cases to ensure there are no significant functional problems present in the system.  Independent Verification and Validation (IV&V) - The process of independently checking that a software system meets specifications and that it fulfills its defined requirements and specifications.

2. QUALITY OBJECTIVES

Quality management addresses the two aspects of system development: Product Quality and Process Quality. For this project, product “quality” means “conformance with requirements”, i.e., software and non-software products adhere to the defined business and technical specifications. It is the joint responsibility of Monster Government Solutions (MGS) and the Employment Security Department (ESD) to establish processes which assure that requirements are clearly stated (in requirement specifications) and accepted at each phase of the project life cycle. It is the joint responsibility of the project team to review and contribute to all Quality Assurance activities applied during the project (all phases). It is the responsibility of the PMO to verify agreed upon processes are followed, verify results of testing, create required reports and to ensure quality reviews of all artifacts are held on a timely manner. Adherence to budget, scope and schedule are considered to be Critical Success Factors and are subject to quality review, reporting and controls.

2.1 Product Quality

Ultimately, the product quality goal is to implement a solution that adheres to approved requirements. The software is the primary ‘product’ being provided, however, artifacts needed to develop, implement and maintain the software, are also part of the product. The quality of these artifacts is directly relatable to the quality of project outcomes. Assessment of artifacts will be conducted throughout the project lifecycle. Two primary factors are to be used to assess each artifact:

 How well it meets the defined acceptance criteria  How it measures against established standards or guidelines defined for the project

WorkSource Integrated Technology (WIT) Date: 4/13/

The Product Quality objective is to implement a solution that operates per the agreed upon requirements. This will be achieved through effective requirement management, quality processes, formal and informal reviews, evaluation and testing as well as other monitoring and controlling activities, as defined in this plan. Refer to the Requirement Management Plan for additional information related to requirement tracking, and in particular, requirement traceability and baselines.

2.2 Process Quality

Process Quality refers to the degree to which an accepted process is implemented and followed in order to produce project artifacts and deliverables. Quality products require establishing and following processes that build quality during development. Through the use of quality methods, tools and techniques, development can be planned, measured, and improved as necessary. The success of process quality is ultimately measured both by the degree to which processes are adhered to, as well as the level of quality of the artifacts (and ultimately the product) produced by those processes. The Process Quality objectives for the project are:  Ensure that processes are defined and project participants follow those processes.  Ensure the work performed follows approved project plans.  Pre-define and agree on acceptance criteria for project artifacts.  Ensure that processes defined in this plan adequately measure and track product quality across all project activities.

3. MANAGEMENT

3.1 Organization

ESD standards require a measure of independence for project SQA personnel. The project’s SQA Manager and/or consultant will report directly to the Executive Sponsors, rather than being considered a part of the Project’s PMO. This independence provides a key strength to SQA; that is, SQA has the freedom, if the quality of the product or project is being jeopardized, to report this possibility directly to the Executive Sponsors. While in practice this rarely occurs, the fact that SQA can go above the project level provides the ability to resolve many of these problems at the project level.

Figure 3-1 shows the SQA organization with relation to the project organization.

WorkSource Integrated Technology (WIT) Date: 4/13/

7. Facilitating the identification and successful completion of artifact acceptance criteria c. MGS Technical Resources are responsible for:
8. Reviewing and commenting on the SQA Plan.
9. Utilizing quality processes in accordance with this SQA Plan.
10. Resolving and following-up on any quality issues raised by SQA related to software engineering.
11. Identifying, implementing, and evaluating the quality factors to be implemented in the system (software and hardware). d. MGS and ESD Software Design/Development is responsible for:
12. Reviewing and commenting on the SQA Plan
13. Implementing the quality program in accordance with this SQA Plan
14. Resolving and following-up on any quality issues raised by SQA related to software design and development
15. Identifying, implementing, and evaluating the quality factors to be implemented in the software e. MGS and ESD Software Test are responsible for:
16. Reviewing and commenting on the SQA Plan
17. Implementing the quality program in accordance with this SQA Plan
18. Resolving and following-up on any quality issues raised by SQA related to software test
19. Verifying the quality factors are implemented in the system (software) f. ESD Independent Verification and Validation (IV&V) is responsible for:
20. Reviewing and commenting on the SQA Plan
21. Implementing the quality program in accordance with this SQA Plan
22. Resolving and following-up on any quality issues raised by SQA
23. Verifying the quality factors are implemented in the system (hardware and software)
24. Implementing the practices, processes, and procedures as defined for IV&V in reference (e) and other program/project planning documents

The ESD Project Director is responsible for working with the Project Management Office

(PMO) to ensure that appropriate reviews are conducted throughout the project. These include Formal Reviews, Informal Reviews, Quality Checkpoints, Quality Gates, and walkthroughs as described in the Review and Audit Plan section of this document.

The MGS Project Manager is responsible for planning and coordinating Functional, Technical,

and Enterprise System Testing. They are also responsible for Test Evaluation Summaries and tracking of Test and Defect metrics. Additionally, this position is responsible for coordinating ESD participation in the planning and execution of User Acceptance Test and tracking of UAT execution status.

WIT Functional Managers are responsible for ensuring that defined processes are being

followed for their respective areas of responsibility.

WorkSource Integrated Technology (WIT) Date: 4/13/

Team Members conducting reviews are responsible for ensuring; formal artifacts reviews are

coordinated with the WIT Contract Specialist through the Formal Artifact Review procedure; review materials are distributed in advance of meetings allowing sufficient time for review by participants; Review Records are created for all formal reviews and follow-up actions and responsibilities are communicated and resolved in a timely manner.

WIT Contract Specialist is responsible for coordination and tracking of Formal Artifact

Reviews and review feedback to provide to MGS.

Review Participants (both MGS and ESD) are responsible for participating in assigned reviews

and for being prepared for designated review sessions.

Developers are responsible for Unit and first level of Integration testing which validates that

components work together.

MGS Testers are responsible for Functional, Technical, and Enterprise (System) testing. ESD

software development engineers in test (SDET) will have a limited role in ensuring that interfaces and reports are tested in conjunction with requirements and that system testing is validated. ESD will not have a primary role in Functional and Enterprise Testing. Technical Testing will be performed by MGS technical staff specialized in load and performance testing. Proof of satisfactory completion of Unit, Function, Performance and System testing will be provided to ESD to validate.

Users are responsible for User Acceptance Testing.

Independent Quality Assurance provides independent assessment around critical business

tasks and project milestones (business and technical). They provide independent advice to the WIT Steering Committee and Project Director on quality related aspects of the project.

Independent Verification and Validation is responsible for independent review, verification,

and validation of technical components such as the data model and application structure.

4. QUALITY RELATED DOCUMENTATION

A variety of documentation will be produced during the project to satisfy the requirements of the project. Each of these documents is created to support the goals of the project.

4.1 Quality records

The following quality records, detailed in following sub-sections, will be maintained:  Meeting Minutes  Requirements Traceability Documentation  Security Assessment  Formal Deliverable Review and Acceptance documentation  Configuration and Code Quality Audits (IV&V)*  Defect Status and Severity Summaries  Test Evaluation Progress Status Summaries  QA Reports

*Configuration and Code Quality Audits will be limited to project specific APIs and new interfaces with

WorkSource Integrated Technology (WIT) Date: 4/13/

4.6 Configuration and Code Quality Audits (IV&V)

MGS shall fully participate and comply with configuration and code quality audits for project specific APIs and new interfaces with existing ESD systems and applications. These configuration and code quality audits will be conducted by the project’s Independent Verification and Validation lead. The assessment will ensure that the project specific software and configuration provided by MGS meet all state and agency standards and policies. IV&V audits will be conducted per the agreed upon IV&V plan.

4.7 Defect Status and Severity Summaries

Defects and associated resolution information will be managed by MGS. Regular reports will be provided to the Executive Steering Committee, Executive Advisory Committee, PMO, SQA consultant and IV&V consultant. These reports will include, at minimum, defect counts by severity. Please see WIT Test Plan for additional details.

4.8 Testing Standards, Evaluation Summaries and Reports

MGS shall provide documentation of standards for Unit, Functional, Performance and System Testing to ESD. Testing standards should describe the type of testing, scope and methodology utilized (including any software tools used) in the completion of testing. MGS shall provide test evaluation summaries and defect reports to help ESD determine that Functional, Performance and System Testing have been completed and that identified defects have been addressed. These should identify type of testing conducted and summarized test results including specific efforts to resolve any significant issues identified. The test evaluation summary should also contain traceability information detailing linkage to the requirements delivered. The test evaluation summary should contain a definitive statement of quality and a confirmation that all defects have been resolved to ESD’s satisfaction. Sub-documentation shall be delivered to ESD at the completion of each three-sprint delivery cycles to assist with the effective and efficient user review of the product delivered. Prior to implementation, draft final test evaluation summary shall be delivered for ESD review. Deployment of the solution shall not be initiated until the mutually agreed upon test success criteria have been met. Please see WIT Test Plan for additional details.

4.9 QA Reports

The ESD SQA consultant shall provide monthly QA reports to the WIT Executive Sponsors and Steering Committee. The WIT Project Director and MGS Project Manager are responsible for responding to any and all quality issues identified in the monthly reports. Their responses, containing actions to mitigate issues and the status of the issue resolution, shall be delivered to the SQA consultant for review and then to the Executive Steering Committee by the following month.

WorkSource Integrated Technology (WIT) Date: 4/13/

5. STANDARDS AND GUIDELINES

Project activities will use existing standards and guidelines for developing work products. The establishment and adherence to standards and guidelines strengthens the quality of analysis, design, development and implementation efforts. Standards and guidelines help project team members deliver consistent, high-quality artifacts and provide project reviewers with criteria for assessing those artifacts. At minimum, the following standards and guidelines will be established for the project and followed by the project team:  Documentation Guidelines  Programming Standards and Guidelines  Test Standards and Guidelines  Security Standards and Guidelines

Within each standard or guideline, checklists will be established to verify artifacts meet the appropriate standards and/or guidelines.

6. REVIEWS AND AUDITS

It is a goal of the project to build in quality through multiple types of reviews and audits. There are three types of reviews, and the type used will depend on the artifact.

 A Walkthrough is an informal review process in which the MGS owner of an artifact leads

one or more members of the review team through all or part of the artifact. The other members ask questions and make comments about the artifact technique, style, possible errors, violation of development standards, and other issues or concerns that may affect the quality of the artifact or the process it supports. This is typically the mechanism used to receive comments on components resulting from identified development sprints.

 An Inspection, or Audit, is a formal evaluation technique in which an author of an artifact

leads a group examining the artifact in detail to find errors or necessary additions or changes. Inspections are used to review detailed technical specifications such as a data model. For this project, inspections will typically be limited to areas having significant technical complexity or risk. In addition, the IV&V consultant will perform independent inspections as specified in the IV&V Plan.

 A Review is a meeting at which one or more artifacts is presented to users, customers, or other

interested parties for comments and/or approval. Reviews may be informal as in the case of peer reviews, or formal as in the case of formal artifact reviews. Reviews will be the primary mechanism used prior to and after submission of formal artifacts through the FAR process. In each of the different types of reviews, feedback will be incorporated into the artifacts that are the focus of the review. Significant issues identified will be recorded and managed according to the issue resolution process described in the Issue Management Plan.

WorkSource Integrated Technology (WIT) Date: 4/13/

 All baselined artifacts will be stored in the configuration management tool

7.2 Techniques

 In reviewing documents, a combination of reviews and walkthroughs will be conducted. Prior to submission of document related artifacts for approval and sign-off, authors are required to evaluate documents against corresponding guidelines and associated checklists.  Functional requirement documents will require both Technical and Functional Reviews. Execution of these reviews will be recorded in the revision log.  MGS Developers will use Unit Testing and Integration Testing to prove the initial quality of the application and interfaces.  MGS testers will perform functional testing of developed code with encouraged participation of ESD Business Process Owners, Subject Matter Experts and/or ESD Testers.  Defects will be recorded in ClearQuest for problems identified through any form of testing with the exception of Unit Testing.  Prior to submission for FAR approval, internal walkthroughs or reviews will be conducted with internal MGS staff. In addition, a pre-FAR review or walkthrough should be held with the primary stakeholder and identified participants to ensure that the artifact is ready for formal submission.  For certain types of artifacts, such as detailed requirement specifications, specific processes will be documented describing the processes used to develop the artifacts including roles, activities, inputs, outputs, and review steps to be conducted prior to submission for FAR approval.

7.3 Methodologies

 Source code will undergo technical inspection by peers to ensure compliance with all appropriate standards. Independent assessments by IV&V reviews are also anticipated.  Source code will be built and tested by MGS prior to delivery to ESD staff for user acceptance testing.  Performance testing will be conducted at the hosting site using HP LoadRunner.  MGS will work with the ESD Technical, Business Manager and Contract specialist to plan and conduct all Formal Artifact Review activities (See Deliverable Acceptance Plan ).  Each week, the project schedule is updated to reflect completion status of scheduled events.

8. METRICS

This section deals with the high-level metrics that are employed during the life cycle of this project. It is vital for the success of the project that the important metrics are agreed upon up- front during the planning stage. Failure to do this will result in repetitive discussions and significant lost time while the metrics are debated, particularly in the heat of the moment, as project milestones are attempting to come to closure.

8.1 Problem and Reporting Metrics

 Problems will be logged into ClearQuest MGS defect tracking system to capture issues discovered while checking code or testing. Reports may be produced from the defect tracking

WorkSource Integrated Technology (WIT) Date: 4/13/

system identifying the number of problems, severity of problem, etc.  Project Schedules will be produced (in MS Project) and maintained on SharePoint or Project Server. The Project Schedule will be updated with task status on a weekly basis. The overall percentage complete for ‘in-process’ deliverables will be reported.

8.2 Other metrics

 Adherence to schedule. How close is the actual work delivered as compared to the effort expended? This is tracked based on percentage complete in the project plan. The schedules are updated weekly and available in CM.  Artifacts completed as compared to the plan. Formal artifacts are tracked in the schedules as Milestones and are tracked in an Artifact Tracking spreadsheet. The Artifact Tracking spreadsheet is updated when formal artifacts pass FAR.  Test case/test script status - percentage exercised, percentage passed/failed. This may be tracked in the project schedule or a tracking spreadsheet. This status is updated and monitored by MGS Test Manager during Construction and WIT Business Manager during User Acceptance Test.  Software defects found. This reflects the quality of the code as it is being tested. The goal is to produce code that is reasonably free of defects. Testers will identify defects, classify the severity and log their findings. The measurement is how many are found at each severity level. This is tracked weekly during Construction and UAT. Reports are produced from ClearQuest and reported to the PMO weekly.  Software defect turnaround. This reflects the time it takes to analyze and correct defects. This will be categorized by severity.

9. EVALUATION AND TEST

The objective of the activities in this section is to validate the application meets requirements. The core types of testing to be performed are highlighted here. Detailed descriptions of the testing approach, timing, and plan of execution will be described in Test Plan.

9.1 Unit Testing

9.1.1 Objective

Unit testing is intended to exercise and validate the smallest testable element (units) of the modified software, and involves testing the internal structure such as logic and data flow, and the unit's function and observable behaviors. The goal is to ensure the unit (component) implemented by the developer behaves as designed, and through knowledge of the internal structure of the code, that the developer has validated all decision paths through the code.

9.1.2 Benefits

Unit Testing establishes that the selected code operates as designed.

9.1.3 Responsibility

 Developers are responsible for executing unit tests against their developed components  Unit tests must meet unit testing standards to be developed for the project  Designers may specify additional unit test requirements for the developers to meet

WorkSource Integrated Technology (WIT) Date: 4/13/

integration.  Testing is planned by the appropriate managers or team leads.

9.3.4 Activities

 Plan testing approach.  Schedule delivery of components and timing of test execution.  Ensure integration environments are configured and available.  Execute the defined tests.  Record and work with developers to resolve identified defects.

9.4 Business Functional Testing (BFT)

9.4.1 Objective

Ensure the delivered application release meets all the functional requirements as defined in the requirements specification documents.

9.4.2 Benefits

This is the final testing cycle performed by MGS prior to releasing the code to ESD.

9.4.3 Responsibility

 MGS dedicated Tester resources will execute the tests for each release.  Testers log defects identified during BFT.  Developers analyze and resolve logged defects.  The MGS test manager manages execution of testing and documents results in the test evaluation summary report.

9.4.4 Activities

 Execute the defined tests.  Record and work with developers to resolve identified defects.  Track testing metrics  The MGS Test Manager writes a Test Evaluation report for delivery to ESD.  Deliver to ESD at each Sprint:  Release Notes  Test Evaluation Summary (using Test Evaluation Summary Template)

9.5 Regression Testing

9.5.1 Objective

Ensure the delivered application release meets all the functional requirements as defined in the requirements specification documents..

9.5.2 Benefits

This testing is done to ensure that the application meets the business requirement

WorkSource Integrated Technology (WIT) Date: 4/13/

9.5.3 Responsibility

 MGS dedicated Tester resources will execute the tests for each release.  Testers log defects identified during regression  Developers analyze and resolve logged defects and depending on severity move it future releases.  The MGS test manager manages execution of testing and documents results in the test evaluation summary report.

9.5.4 Activities

 Execute the regression tests.  Record and work with developers to resolve identified defects.  Track testing metrics  The MGS Test Manager writes a Test Evaluation report for delivery to ESD.  Deliver to ESD at the end of testing:  Release Notes  Test Evaluation Summary (using Test Evaluation Summary Template)

9.6 Technical Testing

9.6.1 Objective

Ensure the final delivered application meets established load & performance criteria established in the technical specifications.

9.6.2 Benefits

This will ensure the final application will not produce any surprises, will meet desired performance metrics and will produce the desired results per the specifications.

9.6.3 Responsibility

 MGS and ESD will collaborate to define the load & performance criteria as part of the Requirement Specifications.  MGS Testers execute load & performance tests and log performance issues.  Developers and the DBA analyze and resolve performance issues.  The Test Manager oversees the execution of testing and documents the results in the Test Evaluation Summary report.

9.6.4 Activities

 Build and deploy product from source.  Execute the tests to validate load & performance criteria identified in supplementary specifications.  Record defects found.  Incorporate findings in the Test Evaluation Report.  Deliver to PMO: