Quality Management Plan: Risk Cues & Decision Drivers for Program Management, Exercises of Quality Management

Download Quality Management Plan: Risk Cues & Decision Drivers for Program Management and more Exercises Quality Management in PDF only on Docsity!

Modernization Program

Quality Management Plan

Version: 1.

Revision date: December 2018

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management

Plan V1.0.docx

• Program Overview TABLE OF CONTENTS
• Document Purpose
• Document Audience
• Risks, Assumptions, and Constraints
• Roles and Responsibilities
• Quality Management Overview
• Deliverable Acceptance Criteria....................................................................................................................
• Deliverable Approval Process
• Quality Management Coordination
• Lessons Learned
• Metrics
• Document Maintenance
• Approving Authorities
• Appendix A – Quality Standards
• Appendix B – Quality Checklists
• Appendix C – OSCIO Quarterly Reporting: Project Status Update Report Template
• Appendix D – OSCIO Quarterly Reporting: Project Assessment Report Template
• Appendix E – OSCIO Quarterly Reporting: Project Variance Report Template
• Figure 1 – Roles and Responsibilities TABLE OF FIGURES
• Figure 2 – Significant Processes Subject to Quality Assurance
• Figure 3 – Major Deliverables Subject to Quality Control Review

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx  Solution vendor staff will actively participate in quality assurance, including quality reviews of their work products and processes before submission. CONSTRAINTS  State of Oregon (Policy No. 107- 004 - 030 ) requires projects exceeding $5 million in budget (or meet other criteria set by OSCIO if between $1 million and $5 million) to use professional iQMS.  The iQMS vendor must be selected from the state’s quality management price and services agreement. The State’s contract for iQMS also has defined deliverables that must be met. Roles and Responsibilities Significant roles and responsibilities involved in these processes are described below. Figure 1 – Roles and Responsibilities

Role Responsibilities

Modernization Sponsor (Modernization Director) Accountable for the quality delivered for all modernization projects. Administers the iQMS contract vendor. Modernization Program Manager Develops the quality management strategy, and works with Modernization Quality Analyst to develop sufficient quality management processes. Meets with iQMS vendor to provide information as assessments are conducted. Modernization Quality Analyst Responsible for quality oversight of all modernization projects. Develops and maintains the quality management plan. Facilitates iQMS vendor’s ability to review program and project plans and activities and acts as the point of contact for question raised by the iQMS vendor. Meets with iQMS vendor to provide information as assessments are conducted. Reviews documents and work products to ensure the documents meet agency and project standards for quality and content. Ensures quality processes are incorporated with modernization processes. Monitors and facilitates management of quality related risks. iQMS vendor Conducts independent quality assurance and quality control reviews of project management processes and project deliverables. Conducts product verification and validation activities. Conducts risk assessments and provides recommendations on mitigation strategies.

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx

Role Responsibilities

Solution vendors Responsible for quality oversight of their system and vendor team, ensuring modernization quality standards are considered before submission of deliverables. Office of the State Chief Information Officer (OSCIO) Provides statewide quality oversight of major agency projects. Reviews iQMS vendor deliverables and reports. Quality Management Overview The four main components of quality management are:  Quality planning  Quality assurance  Quality control  Independent verification and validation QUALITY PLANNING Quality planning includes identification and monitoring of quality standards relevant to a project, as well as determining how to satisfy them within the constraints of the project schedule, available resources, and internal policies and procedures. The initial product of quality planning is this quality management plan. The modernization team will work with the iQMS vendor to produce additional artifacts, including all requisite quality standards, checklists, report templates, and processes. Such artifacts will be used in quality reviews of all major documents and processes for the project. This includes reviews of the program and project management plans, schedules, resources, processes, and products. Quality checklists will be developed by the iQMS vendor and will be incorporated in later iterations of this plan in Appendix B. The iQMS vendor will provide any specific quality standards as a deliverable, and will indicate how these standards relate to the quality standards established by OSCIO’s quality management program. OSCIO’s quality standards can be referenced in Appendix A. QUALITY ASSURANCE (QA) Quality assurance includes the periodic review of key project processes, documentation, and interviews with key business and technical staff. Quality assurance also includes evaluating, identifying, and recommending adjustments to the activities or tasks that must be performed to provide confidence that the modernization project will satisfy the relevant quality standards. Quality assurance activities focus on processes used to manage and deliver the program deliverables and objectives. Quality assurance standards require an evaluation of overall project performance on a regular basis. iQMS are required by the State of Oregon for all IT projects exceeding certain thresholds. The Modernization Program will contract services of an iQMS vendor to review project processes and work products, prepare monthly and quarterly reports, and provide updates to OSCIO and the Legislative Fiscal Office (LFO). See Appendices C-E for examples of the OSCIO quality reporting templates.

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx QUALITY CONTROL (QC) Quality control tasks involve monitoring project results to determine if they comply with stated project requirements and foundational strategies. Project results include both work product results (notably deliverables) and project management results (notably schedule, scope, and cost performance). Quality control activities are performed continually to verify that internal project management and vendor deliverables are of high quality, meet State of Oregon quality standards, and meet contractual thresholds for deliverable acceptance. This includes review of completed deliverables to determine whether they conform to project critical success factors, and meet business, functional, or technical requirements. Quality control helps uncover causes of unsatisfactory results, and identifies lessons learned to avoid similar issues in subsequent phases or projects. Quality control includes identifying quality improvements, and recommending and tracking changes that realize those improvements. Quality control for the projects will include the following techniques:  Initial assessment – Review of key project documentation and interviews with key project, business, and technical staff.  Peer review and work product review – Modernization team members will assign peer reviewers for significant work products.  Software testing – Modernization project teams, including the solution vendor, will perform testing activities as detailed in the project’s testing plans.  Independent verification and validation (IV&V) – Modernization project teams, including the solution vendor, will conduct software testing and other quality control activities. These activities are discussed in further detail later within this plan. The iQMS vendor will assist with identification of quality risks and issues relating to project management processes and deliverable work products. The iQMS vendor will perform these functions. Internal reviews will further ensure project and product quality. The major deliverables subject to quality control are listed in the table below. This is not an exhaustive list. Additional quality control activities will take place. Figure 3 – Major Deliverables Subject to Quality Control Review

Process Quality Standard QA Activity Frequency Performer

iQMS deliverables Deliverable expectation document Contractual expectations High professional standards for distribution to external stakeholders including the legislature iQMS vendor internal review Modernization team review Per quality schedule Upon receipt of deliverable expectation document and upon receipt of submitted deliverable (per contract and/or project schedule) iQMS vendor Designated deliverable reviewer per contract

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx

Process Quality Standard QA Activity Frequency Performer

Solution vendor deliverables Deliverable expectation document Contractual expectations Quality checklist (when applicable) Solution vendor internal review Modernization team review iQMS vendor review Two levels of review per implementation schedule Upon receipt of deliverable expectation document and upon receipt of submitted deliverable (per contract and/or project schedule) Per quality schedule (upon receipt of submitted deliverable) Solution vendor Designated deliverable reviewer per contract iQMS vendor Agency internal deliverables and work products High professional standards for distribution to external stakeholders including the legislature Modernization team review iQMS vendor review Per project schedule Appropriate peer; assigning manager or delegate iQMS vendor INDEPENDENT VERIFICATION AND VALIDATION (IV&V) The iQMS vendor will perform independent verification and validation tasks that ensure technical artifacts, the system, and its components, as delivered by the solution vendor, are accurate, functional, stable, and secured as defined by approved requirements of the modernization project. The iQMS vendor will develop an IV&V test plan. This test plan will include the plans, methodologies, and bug tracking that the iQMS vendor will employ. The IV&V test plan will emphasize testing of high risk and new code areas. High risk areas will include, but are not limited to sub-system integration and interfaces to other data systems. At a minimum, the IV&V test plan will include the following elements:  Identification of potential high risk or new code areas  Test definition and test matrix  Detailed test script development procedure  Detailed configuration and build control procedure  Testing procedure  Testing environment  Test scripts  Testing metric and reporting

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx  Meets regularly with iQMS vendor and solution vendor to review progress and updates associated with the project.  Participates in iQMS vendor debriefing sessions.  Reviews iQMS vendor monthly and quarterly reports, provides feedback, and edits or coordinates the collection of feedback from other project team members to include in the response.  Drafts a response to concerns, risks, or issues raised by iQMS vendor in their monthly and quarterly reports, and collects feedback from project team members to include in the response.  Develops action plans as a response to concerns, risks, or issues, and ensures action plans are assigned and carried out.  Compiles and provides quarterly reports to OSCIO.  Reviews iQMS vendor deliverables, and processes invoices once deliverables are approved. Additionally, the Modernization Quality Analyst may perform additional internal quality coordination tasks:  Reviews documents and work products to ensure the documents meet agency and project standards for quality and content. Work products may include meeting minutes, status updates, plans, and future project contracts.  Works with agency subject matter experts involved in review and testing of deliverables. Lessons Learned The modernization project team will conduct a lessons learned exercise at the completion of each project implementation phase. The resulting lessons learned document is developed for an internal audience. Modernization managers, project leads, and members of the program management team will participate in developing the lessons learned document for each project implementation phase. With lengthy phases, there may be focused lessons learned gathered for specific areas, which will be combined into a summary lessons learned document. Program and project management plans are then updated to reflect adjustments to address challenges and improvements identified. The Modernization Program Manager will schedule process reviews (assessments) at regular intervals. Improvements or clarifications identified as a result of these assessments will be incorporated into program and project management plans. A lessons learned deliverable may also be required of the iQMS vendor at the end of each project implementation phase. This lessons learned document is at a higher level and designed for an external audience (i.e. OSCIO, LFO). Metrics In support of quality assurance and quality control activities, metrics will be prepared as needed to ensure project success.

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx BUDGET METRICS The budget management process is described within the program’s budget management plan. Budget reports are included within the modernization monthly status reporting process. The Modernization Budget Analyst prepares a quarterly budget variance report required by OSCIO. CHANGE CONTROL METRICS The change control process is described within the program’s change control process plan. The Modernization Change Analyst will produce monthly change control metrics showing change requests reviewed by the program’s change control board. These metrics are reported monthly within the modernization monthly status reporting process. CHANGE MANAGEMENT METRICS The change management process is described within the program’s change management plan, along with specific project-level change plans. Readiness assessments will be conducted at different intervals throughout the project. The results of these assessments are not intended to be shared broadly, rather are used to help prepare tactics and methods to ensure employees are prepared for changes resulting from modernization projects. COMMUNICATIONS METRICS Communication metrics are described within the program’s communications and outreach plan. DECISION METRICS The program’s decision process, including documentation of decisions, is described within the program’s governance plan. DEFECT METRICS Metrics regarding system defects will be outlined within the project’s testing plan. DELIVERABLE METRICS A deliverables registry is maintained for each vendor contract, which is updated regularly by the Modernization Contract Analyst. Vendor deliverable metrics are described in the program’s contract management plan. OBJECTIVE METRICS Program goals, objectives, and supporting metrics are described within the program’s charter. The objectives and metrics pertaining to the individual modernization projects are described within the project’s charter and detailed in the project’s metric scorecard. These metrics will be gathered and reported at the completion of each project implementation phase. RISK AND ISSUE METRICS The risk and issue management process is described within the program’s risk management plan. Risk and issue metrics are included within the modernization monthly status reporting process.

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx Approving Authorities Eric H. Smith, Modernization Sponsor Date Interim Modernization Director Jennifer Hannan, Modernization Program Manager Date Kelley Duron, Modernization Quality Analyst Date Jennifer Hannan (Dec 18, 2018) Jennifer Hannan Kelley Duron (Dec 18, 2018) Kelley Duron Eric H Smith (Dec 18, 2018)

Appendix A – Quality Standards The following was taken from OSCIO’s recommended iQMS SOW: https://www.oregon.gov/das/OSCIO/Documents/2b--QA_SOW_appendices_a_thru_e_v2.0.doc The following table identifies quality standards that will be for quality management and risk assessment purposes. Additional standards may be added, while standards identified as unnecessary can be deleted. However, there are specific quality standards, identified herein with an asterisk () and/or specified in the QA Contract Statement of Work, that must be reported on. a. The quality standards in the following table are organized with the following headers: b. QS# - A sequentially assigned number for quality standards c. Quality Category – Header that names the category in which the following Quality standards belong d. Quality Standard – Named areas of potential quality standards. “” indicates recommended minimums e. Low Risk Cues – Characteristics of this quality standard when it can be considered low risk to the project f. Medium Risk Cues – Characteristics of this quality standard when it should be considered high risk to the project g. High Risk Cues – Characteristics of this quality standards when it should be considered high risk to the project h. Rating – Level of quality risk you think is true of this project a. Low – This project exhibits the low risk cue, or appears to have no risks in this area b. Medium – This project exhibits the medium risk cue, or something similar in threat c. High – This project exhibits the high risk cue, or something similar in threat d. N / A – This factor is not applicable to this project e. Need Info – The Contractor needs information from someone else (perhaps an expert) to make a judgment f. TBD – The project is not far enough along to make a rating; the Contractor needs to review the quality standard at a later time i. Risk Rank – The numerical rating for risk as it ranks with other identified. For example the quality standard may have high risk cues, but for the project may be of low risk

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx

Quality Standard Low Risk Cues Medium Risk Cues High Risk Cues

8 Attractive Technology technology selected has been in use for some time project is being done in a sub-optimal way, to leverage the purchase or development of new technology project is being done as a way to show a new technology or as an excuse to bring a new technology into the organization 9 Short Term Solution project meets short term need without serious compromise to long term outlook project is focused on short-term solution to a problem, with little understanding of what is needed in the long term project team has been explicitly directed to ignore the long term outlook and focus on completing the short term deliverable Project Management 10 *Definition of the project project is well- defined, with a scope that is manageable by this organization project is well-defined, but unlikely to be handled by this organization project is not well- defined or carries conflicting objectives in the scope 11 *Project Objectives verifiable project objectives, reasonable requirements some project objectives, measures may be questionable no established project objectives or objectives are not measurable 12 *Leadership project has active sponsor project has sponsor responsible for project, but unable to spend enough time to direct effectively project has no sponsor, or project manager concept is not in use 13 *PM Approach product and process planning and controls in place planning and controls need enhancement weak or nonexistent planning and controls 14 PM Communication clearly communicates goals and status between the team and rest of organization communicates some of the information some of the time rarely communicates clearly to the team or to others who need to be informed of team status 15 PM Experience PM very experienced with similar projects PM has moderate experience or has experience with different types of projects PM has no experience with this type of project or is new to project management 16 PM Attitude strongly committed to success willing to do what it takes cares very little about project

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx

Quality Standard Low Risk Cues Medium Risk Cues High Risk Cues

17 *PM Authority has line management or official authority that enables project leadership effectiveness is able to influence those elsewhere in the organization, based on personal relationships has little authority from location in the organization structure and little personal power to influence decision- making and resources 18 Support of the PM complete support by team and of management support by most of team, with some reservations no visible support; manager in name only Project Parameters 19 Project Size small, non-complex, or easily decomposed medium, moderate complexity, decomposable large, highly complex, or not decomposable 20 Hardware Constraints little or no hardware- imposed constraints or single platform some hardware-imposed constraints; several platforms significant hardware- imposed constraints; multiple platforms 21 Reusable Components components available and compatible with approach components available, but need some revision components identified, need serious modification for use 22 Supplied Components components available and directly usable components work under most circumstances components known to fail in certain cases, likely to be late, or incompatible with parts of approach 23 *Budget & Resource Size sufficient budget and resources allocated questionable budget and resources allocated doubtful budget and resources are sufficient 24 Budget Constraints funds allocated without constraints some questions about availability of funds allocation in doubt or subject to change without notice 25 *Cost Controls well established, in place system in place, weak in areas system lacking or nonexistent 26 *Delivery Commitment stable commitment dates some uncertain commitments unstable, fluctuating commitments 27 *Development Schedule team agrees that schedule is acceptable and can be met team finds one phase of the plan to have a schedule that is too aggressive team agrees that two or more phases of schedule are unlikely to be met Project Team

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx

Quality Standard Low Risk Cues Medium Risk Cues High Risk Cues

39 Policies and Standards development policies and standards are defined and carefully followed development policies and standards are in place, but are weak or not carefully followed no policies or standards, or they are ill-defined and unused 40 Management Support strongly committed to success of project some commitment, not total little or no support 41 *Executive Involvement visible and strong support occasional support, provides help on issues when asked no visible support; no help on unresolved issues 42 Resource Conflict projects within the organization share resources without any conflict projects within the organization schedule resources carefully to avoid conflict projects within the organization often need the same resources at the same time (or compete for the same budget) 43 Customer Conflict multiple customers of the project have common needs multiple customers of the project have different needs, but do not conflict multiple customers of the project are trying to drive it in very different directions Customer/User 44 *User Involvement users highly involved with project team, provide significant input users play minor roles, moderate impact on system minimal or no user involvement; little user input 45 User Experience users highly experienced in similar projects; have specific ideas of how needs can be met users have experience with similar projects and have needs in mind users have no previous experience with similar projects; unsure of how needs can be met 46 *User Acceptance users accept concepts and details of system; process is in place for user approvals users accept most of concepts and details of system; process in place for user approvals users do not accept any concepts or design details of system 47 User Training Needs user training needs considered; training in progress or plan in place user training needs considered; no training yet or training plan is in development requirements not identified or not addressed 48 User Justification user justification complete, accurate, sound user justification provided, complete with some questions about applicability no satisfactory justification for system

\Wpoedfill04\ 014 \Shared\00 Program Management\Oversight-OSCIO-LFO-DOL-SOS\Stage Gate 2\Modernization Quality Management Plan V1.0.docx

Quality Standard Low Risk Cues Medium Risk Cues High Risk Cues

Product Standards Product Content 49 Requirements Stability little or no change expected to approved set (baseline) some change expected against approved set rapidly changing or no agreed-upon baseline 50 *Requirements Complete and Clear all completely specified and clearly written some requirements incomplete or unclear some requirements only in the head of the customer 51 *Testability product requirements easy to test, plans underway parts of product hard to test, or minimal planning being done most of product hard to test, or no test plans being made 52 Design Difficulty well defined interfaces; design well understood unclear how to design, or aspects of design yet to be decided interfaces not well defined or controlled; subject to change 53 *Implementation Difficulty algorithms and design are reasonable for this team to implement algorithms and/or design have elements somewhat difficult for this team to implement algorithms and/or design have components this team will find very difficult to implement 54 System Dependencies clearly defined dependencies of the software effort and other parts of system (hardware, process changes, documentation, ...) some elements of the system are well understood and planned; others are not yet comprehended no clear plan or schedule for how the whole system will come together Development Process 55 Alternatives Analysis analysis of alternatives complete, all considered, assumptions verifiable analysis of alternatives complete, some assumptions questionable or alternatives not fully considered analysis not completed, not all alternatives considered, or assumptions faulty 56 Commitment Process changes to commitments in scope, content, schedule are reviewed and approved by all involved changes to commitments are communicated to all involved changes to commitments are made without review or involvement of the team