Test and Verification - Applying Systems Engineering - Lecture Slides, Slides of Systems Engineering

Download Test and Verification - Applying Systems Engineering - Lecture Slides and more Slides Systems Engineering in PDF only on Docsity!

Verification and Validation Testing

Objectives

• Understand Test Verification and Validation

• Define System Verification and where it fits into

the Systems Engineering process

• Understand the Methods of Verification

• Understand Technical Reviews

• Review the Verification Timeline

• Chapter 6 in Text book

The Problem

Correction Costs

System Life Cycle

Req’ts Concept

Design

Title

Design

Test

10

0

10

4

10

2

10

6

Build Production

10

8

SRR

SCDR

CDR

PDR

TRR

ATR

FCA/

PCA

System Verification Planning Resource Allocation

Pre-Conceptual Conceptual Title 1 Design Title 2 Design Construction Acceptance

Testing

Turn-Over

Project Life-Cycle

SE Activities (% of total effort)

Design Engineering

Activities

Requirements/Concepts/

Alternative Designs

Evaluation, Implementation,

Verification and Testing

/

Verification Planning

What is System Verification?

System Verification is a testing

process which ensures that the

selected solution meets specified

requirements and properly integrates

with interfacing products.

— Martin, p. 20

Verification

(INCOSE Handbook)

• Verification is the confirmation that the model does what it was

designed to do.

• A proper development process will have direct correspondence between

the definition/decomposition activities and the integration/verification

activities. For every specification, there should be an independent

integration and verification activity. For every level in the hierarchical

decomposition, there should be an independent level of integration and

verification.

• Verification tasks are test, analysis, inspection, demonstration, or

simulation; also includes: manufacturing process proofing, quality

assurance and acceptance, and development test and evaluation (DT&E).

• Not INCOSE – Plain and simple, if you buy something, you should have a plan

to test that what you receive is what you ordered. I bought 1GB DRR

Memory, I get two strips. I plug it in and start the computer. I open

MyComputer/Properties/General. It says there is 1GB RAM. At least my

computer thinks there is 1GB. I’m happy.

Methods of Verification

• Inspection

• Analysis

• Demonstration (Simulation)

• Testing

Req’t # Test Demo Analyze Inspect

X

X

X

X

X

Requirements Verification Matrix

System Verification Through Testing

Report

Retest

FRACAS

Report

FMA

Analyze

Results

Conduct

Test(s)

Test

Procedures

Test

Plan

TEMP

Verification Matrix

(from specifications)

No Yes Test

Passed

Test

Specifications

FMA – Failure Modes Analysis

FRACAS – Failure Reporting, Analysis and

Corrective Action System

System Verification Through Technical Reviews

• System Requirements Review (SRR)
• System Conceptual Design Review (SCDR)
• Preliminary Design Review (PDR) or Title I (30%)
• Critical Design Review (CDR) or Title II (90%)
• Test Readiness Review (TRR)
• Formal Qualification Review (FQR) or Acceptance Test Review (ATR)
• Operational Readiness Review
• Functional Configuration Audit (FCA)
• Physical Configuration Audit (PCA)

Technical Review Team Members

• Project Management

• Systems Engineering

• Technical Specialists

• Manufacturing

• Quality Assurance

• Finance

• Suppliers

• Customers

How are Technical Reviews

Performed?

• Checklists
• Peer Reviews
• Components
• References
• Slowly and deliberately.
• Avoid mass meeting reviews (the standard

approach.)

• NRC budgets 10 pages per day per man.

Review Techniques

• Review for

– Complete

– Accurate

– Functional

– Traceable

– Verifiable

– Consistent

– Readable

• Pick a key topic and work the problem yourself.

• Check a reference document; find it, study it.

• Pull the thread – pick a promise or conclusion, “This document will prove”, “It is

recommended that” and then try to validate that statement based on what is in the

document.

• If you find a virgule (/), parenthetical expression (), quotation (“”) test it.

• With an electronic document, pick a word, phrase, or acronym, follow it throughout the

document.

• Read a paragraph and try to explain it to a co-worker.

Test Plans, Procedures,

Reports and MTEMP (TEMP)

• See Army TEMP Guide Pamphlet 73-
• 64 page guideline.
• Basically – Plan everything.
• See http://www.acomtech.com/testplan.html for

a set of test plan, test specification and test

report outlines, per IEEE

• See

http://www.southwestern.edu/~bucheles/SE/VV

Plan-outline.html for a partially annotated outline

Types of Verification and Validation or

Testing

• Tests vary from initial theoretical considerations to re-

engineering

• Type of tests depend on customer objectives
• DOE and DOD budget for and manage Basic Science studies,

developmental programs, project development, and then the

gambit of tests and V&V throughout a program or project

• What one does depends on the Plan; again, it begins with a

problem definition, mission needs, requirements, functional

analysis, architecture, etc.

• Discuss various development programs in these terms: ONR,

DARPA (Defense Advanced Research Projects Agency), NASA,

DOE