Wasson's Principles

Question 1

SE Alpha-Omaga Principle

Answer 1

SE begins and ends with the Users of a system, product, or service

Question 2

Content-Grammar Principle

Answer 2

Substantive content must always precede grammar to achieve successful results. Avoid negotiating content for the sake of achieving grammatical elegance and eloquence unless it precluded misinterpretation

Question 3

Intellectual Control Principle

Answer 3

One of the key roles of a SE is to maintain “intellectual control of the problem solution” (means you can’t just check off boxes, you need to understand how/why the requirements are constantly changing)

Question 4

User’s Problem Space Principle

Answer 4

Thoroughly understand the problem or issue form the user’s perspective. Knowing the underlying root cause is not enough

Question 5

Decision Artifacts Principle

Answer 5

If a key decision or event and its contributory inputs, constraints, and their sources are not documented, the decision or event never occurred

Question 6

System Interactions Principle

Answer 6

Many parts of the system will have to interact with people/systems outside of itself

“Systems, products, and services must be capable of encountering, engaging, and responding to external systems and dynamic conditions in the operating environment”

Question 7

Types of Interaction Principle

Answer 7

External system encounters and interactions are characterized as:
1. Cooperative: The external entities work together with the system, sharing information, resources, or functions to achieve common goals.
2. Supportive: The external interactions provide help or reinforcement to the system, such as additional capabilities or backup, even if they are not directly working side-by-side in a fully integrated manner.
3. Defensive: The system encounters external forces that are antagonistic or pose potential risks, so these interactions are managed in a way that protects the system’s integrity and performance.

…or combinations of these

Question 8

System Reactive and Adaptive Behavior Principle

Answer 8

The system needs to be able to actually react (and adapt) to the people/systems that are interacting with it

“Systems/products/services must be capable of responding with reactive and adaptive behavior (non-responses, aggressive actions, protection mechanisms, or defensive countermeasures) to stimuli or cues originating from external systems in their operating environment”

Question 9

System Responses Principle

Answer 9

Systems produce outputs (products, services, behaviors, or combinations of these) necessary for performance and survival

“Systems produce products, by-products, services, behaviors, or combinations of these to accomplish mission outcome-based performance objectives and survive in their operating environment”

Question 10

Law of Unintended Consequences

Answer 10

Sometimes the outputs (products, services, behaviors, or combinations of these) of a system end up preventing it performing and surviving

“Systems, products, by-products, or services responses may result in self-inflicted adverse or catastrophic conditions or effect with negative consequences that impact its performance, mission or survival”

Question 11

Higher Order Systems Principle

Answer 11

There will always be some person/system that provides “command and control” inputs to the system

“Every system serves as the pleasure of or is subject to higher order human systems and natural environment systems that exercise authoritative control over the system, its operation, and the conduct of its missions.”

Question 12

System Existence Principle

Answer 12

Every System exists for its stakeholders based on their perceived operational needs

Question 13

System Equilibrium Principle

Answer 13

Every system, exists in a “state of equilibrium” with its operating environment to ensure survival

Question 14

System Stabilization Principle

Answer 14

“Every System exhibits a level of stability that requires the user and system to monitor, command & control, its performance, to successfully accomplish mission objectives”

Question 15

System Life Cycle Principle

Answer 15

Your product development team needs to consider the entire product lifecycle, from conception to disposal

“Every natural and human system exhibits a system life cycle that characterizes its stage evolution from conception to disposal”

Question 16

User Benefits Principle

Answer 16

A system isn’t just “good” because it works—it must work well, be available, easy to use, effective in achieving mission goals, suitable for its operational context, and efficient in its use of resources

Question 17

Operational Utility Principle

Answer 17

The system must offer the necessary functions and capabilities to directly address the mission’s requirements. Does it do what stakeholders need (with the fewest number of human errors)?

“Every system/product/service must be operationally useful to enable its User to C2 the system and perform situational assessments with the least number of human errors”

Question 18

Operational Suitability Principle

Answer 18

Beyond just having the right functions, the system must be designed appropriately for the mission’s environment (the right tool for the right job). This includes factors like reliability, maintainability, and how well its design fits operational conditions.

“Every system/product/service must be operationally suitable for the user’s mission application (the right tool for the right job)”

Question 19

Operational Usability Principle

Answer 19

The system should minimize “cognitive load” and consistent with the user’s “mental models”. In other words, the system must be designed so that its users can easily and effectively interact with it, reducing the likelihood of errors.

“Every system/product/service must be operationally usable (easy to understand and operate) according to the user’s mental models, knowledge and skill levels with inducing human errors that affect mission or system performance”

Question 20

Operational Availability Principle

Answer 20

The system must be ready and functioning when needed. This means it’s accessible, easy to power on, and have minimal service downtime.

“Every system/product/service must be operationally available on demand to perform missions when required by its user.”

Question 21

Operational Effectiveness Principle

Answer 21

It needs to perform its required tasks under real-world conditions so that its overall contribution to the mission is clear and measurable.

“Every system/product/service must be operationally effective in producing the required mission outcomes”

Question 22

Operationally Efficient Prionciple

Answer 22

The system should do all of the above in a resource- and cost-effective manner. It should make the best use of available resources (time, money, or manpower) ensuring that its benefits justify its operational costs.

“Every system/product/service must be operationally efficient for the user’s mission application”

Question 23

Customer Needs Principle

Answer 23

Success in providing solutions to a highly competitive global marketplace requires two levels of knowledge:
1. Understanding what users/customers need
2. Understanding what users/customers expect

Question 24

System Existence Principle

Answer 24

Every system/product/service has a purpose and exists for the benefit of performing missions for its users and end users.
Failure of either or both represents system obsolescence (ultimately leading to retirement and disposal)

Question 25

The customer and "customer's customer" principle

Answer 25

Understanding the customer isn't enough, you need to capture the needs and expectations of the system's *end users*

Question 26

Dual Producter-Supplier Role Principle

Answer 26

Every system product or service preforms two contextual roles: 1. A mission system (producer) role: In this role, the system is directly responsible for producing the desired output or fulfilling the core mission. It’s the “workhorse” that delivers the primary functions or capabilities required for a mission or task. 2. An enabling system (supplier) role: In this role, the system provides support, resources, or services that help other systems (including mission systems) perform their functions. It acts as a supplier—delivering essential inputs, capabilities, or infrastructure that enable the mission system to operate effectively.

Question 27

System I/O Fitness-for-use principle

Answer 27

You do NOT truly understand the needs/expectations of the stakeholders until you work with them to establish fitness-for-use performance standards and "acceptance criteria" for each input and output of the system

Question 28

Enabling System (supplier role) principle

Answer 28

The system provides support, resources, or services that help both stakeholders and other systems perform their functions. It acts as a supplier—delivering essential inputs, capabilities, or infrastructure that enable the mission system to operate effectively

Question 29

Mission system (producer role) principle

Answer 29

The system is directly responsible for producing the desired output or fulfilling the core mission. It’s the “workhorse” that delivers the primary functions or capabilities required for a mission or task.

Question 30

Mission Outcomes Principle

Answer 30

Mission outcomes identify what has to be accomplished to eliminate, minimize, or control an emerging problem space or exploit an opportunity space.

Question 31

Stakeholder moments of truth principle

Answer 31

Every single time a system stakeholder (user and end user) encounters and interacts with your organization and its solutions it is a "moment of truth" that results in positive or negative experiences, with consequences for your business with that stakeholder in the future

Question 32

Problem, Opportunity, and Solution Spaces Principle

Answer 32

Need to Understand the user's problem/opportunity and solution spaces. A worst-case scenario is writing perfectly stated specification requirements for the *wrong problem*

Question 33

Problem/Opportunity Space Principle

Answer 33

One enterprise's or system's problem space is an opportunity space for a competitor or adversarial system.

Question 34

Operational Needs Principle

Answer 34

System analysis requires recognition and validation of three types of stakeholder (user and end user) operational needs: 1. Real: These are the objective, tangible requirements that are critical for the system to perform its intended functions. They are based on measurable facts and the actual conditions required for effective operation. 2. Perceived: These are the requirements that stakeholders believe or expect to have, which may be influenced by their experiences, opinions, or the information they’ve received. While they might not always match the objective realities, they are important because they shape user satisfaction and acceptance 3. Projected: These are the anticipated or future requirements that may not be immediately evident but are expected to become important as conditions change. They help ensure the system remains relevant and effective over time by considering long-term trends or evolving operational contexts.

Question 35

Problem-Symptom Solving Principle

Answer 35

There are two types of solution development activities: 1. This approach digs deep to identify and address the root cause of an issue. It’s about understanding the fundamental problem and developing a solution that prevents the issue from recurring. This type of activity aims for a long-term, sustainable fix. 2. Symptom solving targets the immediate manifestations or side effects of a problem without necessarily tackling its underlying cause. While it can provide quick relief or improvement, it may not stop the core issue from coming back. You can tell that you are not problem solving if: -You did not do a root cause analysis -The issue recurs -Your solution was quickly implemented (indicating a band-aid solution) Need to Recognize the difference.

Question 36

Eliminate or Control the Problem Space Principle

Answer 36

If you cannot *eliminate* the problem space, try to *control* it until you can resolve or eliminate it.

Question 37

Problem Statement Principle

Answer 37

Every problem or opportunity space should be clearly and *concisely* bounded by a well- articulated problem statement that does not identify causes, assign blame, or propose solutions

Question 38

Contributory Cause Determination Principle

Answer 38

Investigative teams determine probable causes and recommend solution based on analysis of the problem statement.

Question 39

Problem Complexity Reduction Principle

Answer 39

Literally just means you should always *break up* complex problems into smaller, more manageable, parts "Partition-decompose Problem spaces into one or more manageable solution spaces as a means of reducing complexity and managing risk"

Question 40

On-site visits principle

Answer 40

Systems engineers should always be involved in the market research process "Always send a qualified SE to accompany business development personnel during on-site visits to understand analyze, and document the opportunity/problem and solution spaces."

Question 41

Counter Reactions Principle

Answer 41

When you define the boundaries of your solution—what it will include and how it will work—you need to proactively think about how competitors or adversaries might react. "When bounding a solution space, anticipate short-term and long-term competitor or adversarial reactionary responses and countermeasures to the solution space."

Question 42

Mission Success Principle

Answer 42

Mission success requires five key elements: 1. A Purpose 2. Timely, sustainable resources 3. A reasonably achievable outcome-based performance objectives 4. A (mission evet timeline) MET 5. A willingness to perform

Question 43

Mission Statement Principle

Answer 43

A Mission statement should specify one and only one outcome to be achieved and supported by one or more performance based objectives.

Question 44

Mission Objectives Principle

Answer 44

Each mission should be bounded and specified by one or more performance-based objectives

Question 45

Mission Operating Constraints Principle

Answer 45

Each mission should be bounded by operating constraints that limit its acceptable usage: 1. safety - The mission must operate without putting people or assets at undue risk 2. operating range - There are limits on where or how far the mission can function effectively. 3. affordability - The mission’s costs must remain within budget or cost-effectiveness thresholds. 4. environmental conditions - The mission is only suitable under specific environmental or operational conditions (such as weather, terrain, or temperature)

Question 46

Mission Reliability Principle

Answer 46

This means that when planning a mission, you must specify a clear reliability target that the mission must achieve to be considered successful. "Each mission should be bounded in terms of the mission reliability to be achieved."

Question 47

User mission profile principle

Answer 47

This means that any system, product, or service should come with a detailed description of the various ways and contexts in which its users will employ it to achieve their goals. "Every system/product/service should include a characterization of its user mission profiles"

Question 48

Mission phases of operation principle

Answer 48

Human systems (enterprise and engineered) have at least three primary phases of operation: 1. Pre-phase 2. Mission 3. Post-Mission *An interim phase, such as Storage may be required for some systems between missions

Question 49

Mission Efficiency and Effectiveness Principle

Answer 49

Every system/product/service mission should be defined in terms of its efficiency and effectiveness. Practical Steps: Define KPIs: Clearly outline what metrics matter for your mission (e.g., a system might need to be 99.9% reliable and complete tasks 20% faster than its predecessor). Set Benchmarks and Targets: Establish acceptable thresholds based on requirements, industry standards, or past performance. Collect and Analyze Data: Use simulations, prototypes, field tests, or monitoring tools to gather data on both resource usage and mission outcomes. Iterate and Compare: Compare your measured values against your targets. If the solution meets its efficiency targets but falls short on effectiveness (or vice versa), you may need to refine your design or implementation.

Question 50

Mission event timeline (MET) Principle

Answer 50

Every system/product/service should include a mission evet timeline (MET) that identifies time-dependent events that represent the start and completion of mission operations and interim waypoints required to be accomplished to achieve mission success

Question 51

Measures of Effectiveness Principle

Answer 51

Mission and system success requires establishment of one or more MOEs that quantify mission objectives in terms of performance-based outcomes

Question 52

Measures of Success Principle

Answer 52

Mission and system success require establishment of one or more MOSs that quantify how well a system is required to perform its mission

Question 53

USe Case Title Principle

Answer 53

Every Used case consists of a title that expresses an outcome to be accomplished from the perspective of the User (Actor)

Question 54

Use Case Identifier Principle

Answer 54

Every Use Case must be unique and consist of an identifier that: 1. Differentiates it from other Use Cases 2. Facilitates referencing

Question 55

Use Case Description Principle

Answer 55

Every Use Case should consist of a breif synopsis that describes how the US will accomplish the required outcome.

Question 56

Use case Actros Principle

Answer 56

Each use case represents a system/product/service capability required by one or more users in performing their assigned mission or task

Question 57

Use Case Outcome Principle

Answer 57

Every Use case expresses what outcome the users requires the system/product/service to produce, not what the system does to perform the action.

Question 58

Use Case Event Timeline Principle

Answer 58

When applicable, every Use Case should be bounded by and synchronized to an even timeline.

Question 59

Use Case Preconditions Principle

Answer 59

Establish the pre-conditions context for each UC

Question 60

Use Case Trigger Principle

Answer 60

Every use case requires a triggering condition or event to initiate its performance

Question 61

Use Case Completion Principle

Answer 61

Every Use case requires definition of actions to be accomplished by the user or system to finalize the use case.

Question 62

Use case Happy Path Principle

Answer 62

Every use case has a main success scenario that when everything works flawlessly the normal sequence of actions produces the required outcome

Question 63

Use Case Scenarios Principle

Answer 63

Every use case should include considerations for the most likely or probably scenarios when things go wrong that required alternate flows from the main success scenario (happy path) to enable recover and avoid system or product failure

Question 64

Ssytem Operations Model Principle

Answer 64

Human Systems (enterprise and engineered are characterized by a generalized system operations model developed in collaboration with users concerning how a system/product/service will be deployed, operatied , maintainted, sustained, retired, and disposed

Question 65

Operational Capability Principle

Answer 65

Every system operations represents a required operational capability that must produce a specified performance-based outcome while coping with one or more probably or most likely Operating Environment scenarios.

Question 66

System Operations Dictionary

Answer 66

Every project should consist of a system operations directory that clearly defined each system operation, its scope, and activities

Question 67

Value-added Operations Principle

Answer 67

Each system operations model operation or task and its performance by the system elements either add value and contribute to achieving a mission-based performance objective or not; if not, eliminate it!

Question 68

Synchronized Operations Principle

Answer 68

Every system operations model activity should be synchronized with the system/product/service's mission event timeline

Question 69

ConOps Principle

Answer 69

Every project should include a system conops document that describes how the system/product/service being developed is envisioned to be deployed, operated, maintained, sustained, and disposed, via its OCDs

Question 70

Sub-phases of operation Principle

Answer 70

when enterprise or engineerd system/product/service's Mission Life cycle phase of operation become too abstract, partition them into lower level sub-phases of operation, outcomes, and objectives

Question 71

System States Principle

Answer 71

Every Eneterprice and Engineered system, produce, or service is characterized by system states that represent its current logistical employment as an asset

Question 72

Operational States Principle

Answer 72

Every enterprise or engineered system/product/service is characterized by examinational states that represent its current condition, status, readiness, or availability to perform or continue a mission or task

Question 73

Moder of Operation Principle

Answer 73

Every enterprise or engineered system product or service is characterized by user-selectable modes of operation that enable the user to comand and control unique sets of UC-based architectural capabilities to accomplish a specific mission outcome and its objectives

Question 74

Triggering event criteria Principle

Answer 74

Every enterprise or engineered system mode of operation requires a pre-defined set of condition-based triggering event criteria to initiate a transition from one mode of operation to another mode

Question 75

Abstract US Modes Principle

Answer 75

Each enterprise and engineered system mode of operation represents a user selectable option to perform one or more USs that share similar outcomes and objectives

Question 76

Configuration Status Principle

Answer 76

Each enterprise or engineered system/product/service mode of operation commands and controls the configuration state of its architectural configurations

Question 77

Enivronmental States Principle

Answer 77

Every enterprise or engineered system/product/service must be capable of operating in and interacting with various types of time and location-dependent environmental states that exist in its operating environments

Question 78

Dynamic States Principle

Answer 78

Every enterprise or engineered system, product, or service must be capapble of withstanding and surviving dynamic or transitory conditions created by interactions created by interactions with its operating environment

Question 79

Allowable and prohibited actions principle

Answer 79

Every enterprize or engineered system/priduct/service mode of operations is constrainted by allowable and prohibited actions that serve as safeguards to ensure the safety of its users and prevent damage to equipment, the public, and external systems in its operating environment

Question 80

Levels of Abstraction Principle

Answer 80

Every SOI, its mission system enabling systems, and equipment (hardware and software) consists of one or more levels of abstraction with standardized titles

Question 81

Performing Entities Concept

Answer 81

System levels of abstraction (system, product

Question 82

Operational Environment Domains Principle

Answer 82

A system's operational environment consists of two classes of domains: 1. A higher-order system domain 2. A physical environment domain.

Question 83

High-Order systems principle

Answer 83

Every human systme exists and performs mission under the C2 of higher-order systems domains

Question 84

Higher-Order system elements principle

Answer 84

Higher-order systems are composed of four analytical system elements: 1. An organization element 2. Roles and mission elements 3. Operating constraints element 4. a resources element

Question 85

Physical Environment Domain composition principle

Answer 85

A system's phsyical environment domain is composed of three classes of analytical elements: 1. *Natural* system environment 2. *Human* system environment 3. *Induced* system environment

Question 86

Logistical-Physical Interfaces Principle

Answer 86

Logistical interfaces establish associative relationships with perfromance-based outcomes (e.g. what is to be achieved). Physical interfaces stablish *how* a logical interface will be implemented

Question 87

Interface Complexity Reduction Principle

Answer 87

Where practical, minimize the number of interfaces and to reduce complexity, technical risk, increase reliability, and reduce maintainability cost by standardizing on widely accepted and proven protocols and standards

Question 88

Interface Failure Mitigation Principle

Answer 88

Every interface should be specified and designed to : 1) detect failures 2) contain them to their sources 3) mitigate their effects on mission performance

Question 89

System Responsiveness Principle

Answer 89

Every system responds to stimuli, excitations, and cues in its operating environment with behavior actions, products, by-products, services, or combinations thereof

Question 90

System Encounters and Interactions Principle

Answer 90

System encounters and interactions with external systems in its operating environment during an engagement may be: a. cooperative b. friendly c. benign d. competitive e. adversarial f. a combination of these

Question 91

Compatibility and Interoperability Principle

Answer 91

Interfaces must be compatability in terms of *form* and *fit*. Where applicatible, they must be interoperable to be able to intelligently encode/decode, interpret, process, and exchange data

Question 92

System Capability Operations Principle

Answer 92

every operational capability, as an integrated system consists of a minimum of three phases of operations: 1. Pre-capability operations 2. Capability operations 3. post-capability operations

Question 93

Exception Handling and Recovers Operations Principle

Answer 93

Every capability requires a safety or contingency mechanism to detect exception, mitigate, and attempt recovery to return to and restore normal operations without doing any harm

Question 94

Capability Initiation Principle

Answer 94

Every operational capability requires an external trigger such as a stimulus, excitation, or visual cue to initiate its performance-based outcome processing

Question 95

Outcome results Reporting Principle

Answer 95

On completion of its required performance, every capability should report successful completion of the task.

Question 96

Four domain Solutions Principle

Answer 96

A system or entity's design regardless od level of abstraction is composed of four domain solutions sequenced in a logical workflow: 1. Requirements 2. Operations 3. Behavioral 4. Physical *based on decision dependencies to minimize redesign and rework

Question 97

Task-Based Outcome and Rewards Principle

Answer 97

If you reward engineers for activities, you get... activities. If you equip Engineers with the right processes, tools, and methods to accomplish performance-based outcomes, you get *performance-based outcomes*

Question 98

System Design Principle

Answer 98

system design is a highly interative, collaborative and multi-level process with each level of abstraction dependent on the maturations, stability, and integrity of higher-level specification requirements and design decisions

Question 99

Developemental Test & Evaluation Principle

Answer 99

DT&E is performed by the system developer to: 1. Mitigate development configuration technology, design, and 'other' risks 2. Provide insights and assurance that the evolving system design solution will comply with its SPS requirements

Question 100

System Design Solution Completion Principle

Answer 100

Contractually, a system design solution is not considered complete until it has been formally verified as compliant with its System performance specification (SPS) by the systems acquirer or used acceptance.

Question 101

Engineering Truth Principle

Answer 101

If engineering fails to perform its job well, system users (along with personnel, the equipment, the general public, and the environment may be placed at risk)

Question 102

Latent Defects Cost Principle

Answer 102

Identify and correct letent defects immediately. Depending on when a latent defect enters the system development phase, the cost-to-correct may increase as much as 100 times by the time system integration, test & Evaluation (SITE) is performed

Question 103

Verification Principle

Answer 103

Verification answers the stakeholder and developer's question: Are we developing a system, entity, or task in compliance with its specified requirements?

Question 104

Validation Principle

Answer 104

Validation answers the user's question: did we acquire the right system/entity/work product to satisfy our operational needs?

Question 105

Lifecycle V&V principle

Answer 105

Verification and validation (V&V) are performed continuously and relentlessly throughout every System/product life cycle phase, contract/subcontract, project charter, or task.

Question 106

V&V Applicability Principle

Answer 106

Verification and Validation to every stage of system development beginning with the proposal phase and continuing after contract award until system delivery and acceptance completion

Question 107

Defect Elimination Principle

Answer 107

The number of latent defects in the fielded system, product, or service is dependent on: 1. a willingness and commitment or provide resources to avoid and eliminate them 2. personnel competency (experience knowledge, training) 3. employment of the *right* tools, processes, and methods 4. time available to perform the job

Question 108

System Development V&V Strategy Principle

Answer 108

A system development V&V strategy requires three elements: 1. A roadmap consisting of incremental V&V activities to progress from contract award to system, product, or service delivery and acceptance 2. A task-based plan of action for implementing the strategy synchronized to milestone-driven event, accomplishments, and criteria. 3. Documented objective evidence of task completion that is, QRs (that demonstrates that you accomplished the planned results)

Question 109

System Verification Principle

Answer 109

System verification assesses the compliance of a work product's attributes, characteristics and performance-based outcomes to one or more contract, project charter, spec, or task requirements.

Question 110

Verification Methods Principle

Answer 110

Verification methods consist of inspection, examination, analysis, demonstration, and test or combinations of these

Question 111

Verification Method selection principle

Answer 111

Select the least number of verification methods required to prove compliance to a single requirment

Question 112

Compliance Testing Principle

Answer 112

Leverage each test to collect as much data that will enable you to verify compliance to the largest number of requirements

Question 113

System Validation Principle

Answer 113

System validation assesses a stakeholder's satisfaction with a work products performance-based outcomes to their documented operational needs and expectations

Question 114

Operational Test and Evaluation Principle

Answer 114

OT&E enables users to assess how well a system/product/service satisfies their operational needs based on independent field trials with trained user personnel operating the system in a realistic operating environment

Question 115

Validation Independence Principle

Answer 115

To ensure independnece and avoid a conflict of interest, OT&E is typically performed by an ITA with User Personnal the have been trained to operate and maintain the system under actual field operating environment conditions based on scripted scenarios

Question 116

Verified System Modifications Principle

Answer 116

Any uncoordinated, unapproved, and inverified modifications to a verified developmental configuration of a physical system without prior authorization during OT&E effectively invalidate the developmental configuration SVR results

Question 117

Problem-solving-solution development Principle

Answer 117

A quantum leap from requirements to a point design solution does not reflect problem solving or solution development

Question 118

Decision-Making Stability Principle

Answer 118

Mature and stabilize decision-making at higher levels as soon as practical to enable decision-making at each successive lower levels to mature and stabilize

Question 119

Suboptimization Principle

Answer 119

Avoid degrading or suboptimizing overall system or entitiy technical cost or risk performance just to *optimize* a lower level specific aspect.

Question 120

Agile Incremental Product Releases Principle

Answer 120

Incrementally develop and deliver just-in-time product releases using short iterative development cycles called 'Sprints' based on the value and priority of each to the User

Question 121

User Story Authoring Principle

Answer 121

Every User Story should be written by the user in their own words from the perspective of a specific role-based use of a system/product/service

Question 122

USer Story priority Principle

Answer 122

Express each user story in terms of the originator's: 1. Role 2. Need 3. Value 4. Priority to their enterprise or mission

Question 123

User Story composition Principle

Answer 123

Each user story consisits of two parts: 1. Statement of Need 2. Conditions of Satisfaction

Question 124

User Story syntax principle

Answer 124

Format each user story using an "As a , I want so the " syntax

Question 125

User Stories Repository

Answer 125

Create a product backlog managed by a product owner to serve as the single repository for story and managing all user stories

Question 126

USer Story Scope Princple

Answer 126

Scope the selected set of user stories for completion within the time-boxed constrains of a sprint

Question 127

Agile Product Increment Principle

Answer 127

Create optional Product increments based on a grouping of user stories that form a planned software "build"

Question 128

Product backlog completion principle

Answer 128

Track planned versus actual performance of product backlog user stories in terms of those completed, those in-process, and those remaining to be completed

Question 129

Sprint backlog completion Principle

Answer 129

Track planned versus actual performance of sprint backlog user stories to terms of those completed, those in-process, and those remaining to be implemented

Question 130

Daily Scrums Princple

Answer 130

conduct brief daily scrums to assess individual or team accountability for three questions: 1. What did you accomplish yesterday 2. What issues need to be addressed today? 3. What do you plan to accomplish today?

Question 131

Incomplete User Stories Principle

Answer 131

Return user stories that are incomplete at the end of a tim-boxed sprint to the product increment backlog (optional) or product backlog for reprioritization and reassignment to a subsequent sprint

Question 132

Development process model principle

Answer 132

Select a development process model based on requirements, technology, team skills, and schedule risk constraints unique to the product, not the project itself

Question 133

Risk-success expectations principle

Answer 133

Adjust project expectations of success based on the level of risk

Question 134

Non-prescriptive methods principle

Answer 134

As a process, Agile development does not prescribe a specific development method or model to be used to create a system/product/service

Question 135

Minimal-essential SE documentation Principle

Answer 135

Produce only minimal, essential documentation that is necessary and sufficient for the: 1. Developers to specifcy, design, build, integrate, test, train, and maintain the product 2. User's to operate, maintain, and sustain (where applicable) the system, product, or service to perform missions

Question 136

Project Development Models Principle

Answer 136

Select system development process models to uniquely satisfy: 1. system, product, subsystem, assembly, development requirements, 2. technology, risk, team skills, and development constraints [what is he talking about??]

Question 137

Development Model Rationale Principle

Answer 137

Document each development model selection including rationale for: 1) Selection of a model 2) Rejection of other models

Question 138

Configuration Items (CIs) Principle

Answer 138

Every physical component within a system as an item by CM. Some items are designated as CIs for internal development; others as COTS products or NDIs available for procurement from external vendors for procurement

Question 139

CI Boundary Principle

Answer 139

CIs are constrained to the confines of a physical boundary such as a Subsystem, Assembly, Subassembly, HWCI, CSCI, or Part; they do not physically exist beyond that boundary.

Question 140

CI Ownership and Accountability Principle

Answer 140

Every Configuration Item (CI) should be assigned to an owner accountable for its design, implementation, integration & test, verification, and validation

Question 141

Primary Configuration Baselines Principle

Answer 141

CM views every System as having three primary Developmental Configuration Baselines: 1. System Requirements or Functional Baseline 2. Allocated Baseline 3. Product Baseline

Question 142

SE Design Configurations Principle

Answer 142

Every System or Entity at any level of abstraction has eight SE design configurations: As Specified, As Allocated, As Designed, As Built, As Verified, As Validated, As Maintained, and As Produced that must be current and consistent with each other

Question 143

SPS Ownership Principle

Answer 143

As a contract element, the System Acquirer owns and controls the contract SPS; the System Developer maintains the SPS in accordance with the contract direction authorized and provided by the Acquirer Contracting Officer (ACO)

Question 144

Design as a Last Resort Principle

Answer 144

System design of a new CI should be a last resort option after all practical efforts have been exhausted to identify legacy designs that are reusable COTS, NDI, or AFP components to meet entity requirements

Question 145

Caveat Emptor Principle

Answer 145

When procuring COTS/NDI, thoroughly investigate and understand the TCO - development and life cycle - and support; otherwise, caveat emptor —buyer beware *Since vendors will only answer the question you ask, always ask what you need-to-know about the product that you may not have asked. *Finally, caveat emptor —let the buyer beware

Question 146

Task Expectations Principle

Answer 146

A 40-hour task with an 8 hour limitation produces an 8 hour summary of key points. The same task with a 40-hour limitation produces a summary with key points and elaborated levels of details. Recognize the difference

Question 147

Sensitive Information Protection Principle

Answer 147

Establish a sensitive information protection program within your Enterprise, identify a central Point of Contact (POC), develop and disseminate guidelines and protocols; and train personnel to implement them properly

Question 148

Contract Issues Principle

Answer 148

Rule #1: Always read the contract! Rule #2: When contract issues arise, go back to Rule #1

Question 149

Technical Review Scheduling Principle

Answer 149

Schedule technical reviews at critical control or staging points in system development to assess the status, progress, maturity, compliance, and risk of the evolving System Design Solution in meeting its contract and specification requirements; no sooner, no later.

Question 150

Conference Invitations Protocol Principle

Answer 150

Under contract protocol, System Develop- ers invite the System Acquirer to project events; the System Acquirer extends invitations to the User and End Users unless the System Acquirer has authorized the System Developer to send invitations

Question 151

Technical Review Entry/Exit Criteria Principle

Answer 151

Avoid signing contracts with ambiguous technical review entry and exit criteria language that may be open to interpretation, especially when it comes to getting paid.

Question 152

Contract Protocol Principle

Answer 152

Read and thoroughly understand your contract. If unclear, consult with your Enterprise’s Project Management Team (PMT). The PMT, in turn, consults with internal Contract and Legal organizations for specific guidance and expertise in interpreting the contract’s Ts&Cs

Question 153

Conference Documentation Principle

Answer 153

Conference minutes document attendees, agenda, discussion topics, meeting, and action items. Perform the task well and obtain System Acquirer acceptance via the ACO

Question 154

Conference Minutes Principle

Answer 154

Technical review agendas, attendees, discussions, decisions, and action items are documented via conference minutes and reviewed, approved, and released via contract protocol

Question 155

Official Contract Direction Principle

Answer 155

Under contract protocol, only the System Acquirer’s Contracting Officer (ACO) is officially authorized to issue technical direction to the System Developer in response to technical review comments, conference minutes, action items, contract modifications, and acceptance of contract documents

Question 156

Technical Review Entry Criteria Principle

Answer 156

At a minimum, establish explicit entry criteria for technical reviews based on: *Level of maturity of work products to be reviewed. *COIs or CTIs to be resolved. *Closure status of outstanding AIs. *Assessment of System Design Solution supporting data—analyses, trade studies, etc. *Configuration Status Accounting (CSA) of the Development Configuration. *Other criterion, as required

Question 157

Technical Review Exit Criteria Principle

Answer 157

At a minimum, establish explicit exit criteria for technical reviews based on: *Closure of outstanding and new AIs. *Resolution of COIs or CTIs. *Acceptable assessment of System Design Solution maturity and risk. *Approval of conference minutes. *Other criterion, as required

Question 158

Objective Evidence Principle

Answer 158

If an analysis, review, decision, result, or event is not corroborated by a Quality Record (QR) as objective evidence of accomplishment, it is nothing more than hearsay

Question 159

Specification Developer Principle

Answer 159

Every specification requires assignment of a developer accountable for its: development, approval by the next higher-level architecture owner, implementation, verification, compliance, and configuration controlled baseline maintenance.

Question 160

Specification Feasibility and Affordability Principle

Answer 160

Every specification should specify and bound a solution space that is feasible to implement, affordable, and has acceptable risk to the User. A well-defined specification must be feasible and affordable to implement within realistically achievable technologies, skills, processes, tools, and resources with acceptable technical, cost, schedule, and support risk to the System Acquirer and the System Developer or Services Provider.

Question 161

Specification Uniqueness Principle

Answer 161

Each specification documents capability requirements that are unique to one and only one System / Entity with no other instances within the System

Question 162

Solution Independence Principle

Answer 162

A specification specifies what is to be accomplished and how well, not how to design the system, product, or service. Well-defined specifications specify: 1. what capabilities are required to accomplish missions and 2. how well they are to be performed. Specifying how to design the system imposes constraints and reduces the System Developer’s flexibility in achieving at an optimal System Design Solution

Question 163

Essential Requirements Principle

Answer 163

A specification contains only essential requirements that are necessary and sufficient to develop a system, product, or service without constraining the set of viable solution options. Well-defined specifications state only the essential requirements that are necessary and sufficient to design or procure a system, product, or service without the need for additional clarification requirements or language

Question 164

Specification and Requirement Accountability Principle

Answer 164

Assign: (1) specification level and (2) individual requirement level responsibility to an individual who is accountable for its analysis, implementation, verification, traceability, and proposed updates. One of the first steps in achieving System Development success is to ensure that every specification and each of its requirements will be implemented in the System Design Solution. This begins with assignment of responsibility and accountability. Enterprises assign accountability for the specification level; however, individual requirement accountability is often overlooked. Even when accountability is assigned, it is often unclear to the assignee what the scope of their accountability is - analysis, implementation, verification, traceability, and proposed updates

Question 165

Single Interpretation Principle

Answer 165

Every specification requirement must be stated in a brief, clear, and concise manner that results in one and only one interpretation. Well-defined specifications explicitly specify capability requirements using brief, concise, outcome-based, capability requirements statements. One of the challenges of our educational system is the notion that specifications are written like novels. Requirements statements are “open to interpretation.” During System Integration, Test, & Evaluation (SITE), verification becomes conflict in which the System Acquirer has one interpretation of compliance to specification requirements while the System Developer has another interpretation. Well-defined specifications require every requirement statement to be articulated in a brief, clear, and concise manner that results in one and only one interpretation

Question 166

Specification Requirements Coverage Principle

Answer 166

Every specification must specify the set of essential capabilities required to perform User-defined missions with no deficiencies. Well-defined specifications are: *Complete, contain no voids, and require no further clarification. *Do not contain any undefined performance values such as To Be Determined (TBD) or To Be Supplied (TBS) performance values. *Specify one or more verification methods for each requirement statement that will serve as the basis for proving compliance

Question 167

Specification Requirements Consistency Principle

Answer 167

Specification requirements must be consistent in language, meanings, semantics, terminology, and standards of units with all project System/Entity specifications. Well-defined specifications read as if they were written by a single developer. Therefore, to ensure readability and avoid ambiguities, all specifications within a System should be consistent in language, meanings, semantics, terminology, and standards of units

Question 168

Semantics and Terminology Principle

Answer 168

Specifications should employ semantics and terminology that are familiar to its readers. Well-defined specifications are written in a language that uses terms that are easy to understand. In general, many qualified people should independently read any requirement statement and emerge with the same interpretation and understanding of technical performance requirements. For example, the SPS should be written using semantics and terminology that has meaning to the System Acquirer/User and System Developer. Lower level EDS specifications should be written with semantics and terminology that is familiar to the System Developers

Question 169

Requirements Traceability Principle

Answer 169

Every specification and requirement must be traceable to the User’s source or originating requirements. Specifications should not be random, ad hoc “wish lists” of requirements. The validity and integrity of a specification requires that every requirement exists to accomplish a specific outcome that ultimately contributes to achievement of mission objectives. Since every requirement has a cost and risk for implementation (Principle 19.4), any specification requirement that is not traceable to the User’s source or originating requirements should be eliminated as non-value-added waste

Question 170

Requirement MOP Traceability Principle

Answer 170

Where required, specification requirement performance measures must be traceable to a documented analysis that is baselined and controlled. Well-defined specifications are traceable to a structured analysis—approach, methodology, and behavioral model such as Model-Based Systems Engineering (MBSE). One of the challenges in developing specifications is that requirements statement performance measures are often derived informally and discarded. This places the entire project at risk when months later the Project Manager (PM) and Project Engineer (PE) discover that no one can authenticate the validity of a specification performance measure ... since it’s source analysis was discarded

Question 171

Requirement Value-Based Priority Principle

Answer 171

Every specification requirement has a value-based priority to the Stakeholder community. When requirements are elicited, each stakeholder places a value on the importance of the requirement to enable them to achieve their Enterprise missions. We refer to the value as a priority. The realities of system development are that every requirement has a cost to implement and deliver. Given limited resources and stakeholder values, bounding the solution space requires reconciling the cost of the desired requirements with the available resources. As a result, requirements should be prioritized for implementation, especially for Agile Development (Chapter 15) and Commercial Product Development (Figure 5.1). In contrast, Contract System Development (Figure 5.1) typically views all specification requirements as having the same priority

Question 172

Specification Risk Principle

Answer 172

Every specification has a level of cost, technical, technology, and schedule risk to develop and maintain that must be acceptable to the User. Well-defined specifications represent a level of cost, technical, technology, or schedule risk that is acceptable to the stakeholders—System Acquirer or System Developer—and does not impose additional financial or schedule risks

Question 173

Specification Baseline Principle

Answer 173

Every specification should be reviewed, approved, and placed under Configuration Control as soon as it reaches a level of maturity and stability to enable technical decisions at lower levels to be made with acceptable risk. Specifications are baselined at strategic staging or control points when all stakeholders are in mutual agreement with its contents. Once the baseline is established, specifications are up- dated and verified through a Stakeholder review agreement process - Configuration Control (Chapter 16) - that ensures that the document properly reflects the current consensus of stakeholder requirements

Question 174

Specification Tree Principle

Answer 174

Every system, product, or service should include a Specification Tree based on the System Architecture that serves as a hierarchy of linkable requirements. The multi-level allocation and flow down of requirements employ a hierarchical framework that logically links system entities vertically into a structure referred to as the specification tree. The right side of Figure 19.2 provides an example of a Specification Tree

Question 175

Design and Construction Constraints Principle

Answer 175

Every specification should specify design and construction requirements that con- strain the development of System/Entity’s capabilities. Specifications do more than communicate what and how well a System / Entity must accomplish. They communicate the constraints that are levied on the System and System Development decisions related to system operations and capabilities. We refer to these as Design and Construction Constraints. In general, design and construction constraints consist of non-functional requirements such as size, weight, color, mass properties, maintenance, safety restrictions, human factors, and workmanship. Specifications also specify verification methods (Chapter 13) to be used to verify compliance to a requirement

Question 176

Environmental Conditions Principle

Answer 176

Every specification should specify the Environmental Conditions a System/Entity must operate and survive. Every Product, Subsystem, and Assembly must be capable of performing missions in a prescribed Operating Environment at a level of performance that will enable achievement of mission success. Therefore, specifications must define and constrain the Operating Environment conditions that drive and bound entity capabilities and levels of performance

Question 177

SE User–End User Advocacy Principle

Answer 177

One of Systems Engineering’s project roles is to serve as a User and End User advocate to preserve the intent and integrity of their requirements. You will often hear Enterprises and Engineers comment “if a requirement is ‘open to interpretation,’ we will interpret it our way.” First, based on the concepts, principles, and practices discussed in this text, you should avoid getting into a need to “interpret a requirement your way.” Conversely, requesting clarification can also be potentially problematic and not turn out the way you desire; it certainly will not be any different during SITE. The best solution is deal with these matters before the contract is signed

Question 178

Condition-Based Requirements Principle

Answer 178

Every specification must specify require- ments that address three types of Abnor- mal Operations: *External system failures *Degraded operations *Internal system failures such as components and interface Abnormal operations consist of a set of system operations and tasks that focus on detecting, troubleshooting, identifying, isolating, and correcting a physical capability condition. These conditions represent levels of performance that are outside the tolerance band for nominal mission performance but may not be critical to the safety of humans, property, or the environment. Abnormal Operations encompass capabilities required to address External system failures, degraded operations, and internal system failures.

Question 179

Measure of Performance (MOP) Principle

Answer 179

Each specification requirement should specify by one and only one Measure of Performance (MOP) characterized by (1) a Technical Performance Parameter (TPP), (2) a magnitude, and (3) a unit of measure. When a system, product, or service is employed to perform missions, the System Developer needs to understand what constitutes mission success from the perspective of the User.

Question 180

Specification Requirements Deficiencies Principle

Answer 180

Ensure that every specification is free from missing, misplaced, contradictory, and duplicated requirements. Feature-based specifications are essentially ad hoc, brain-stormed requirements that capture the Stakeholders’ - Users or End Users - imagination and attention. Specifications developed in this manner are often just formalized, loosely coupled wish lists. People who lack formal training in specification development commonly use a feature-based approach. Although Feature-Based specifications may use standard specification outlines, they are often poorly organized and prone to missing, misplaced, conflicting, and duplicated requirements

Question 181

Irrational Logic Principle

Answer 181

It is always easy to rationalize erroneous logic that ignores best practices and common sense for the sake of meeting schedule and budget constraints. The Feature-Based Approach enables developers to quickly elicit and collect requirements inputs with a minimal effort. Specification “writers” spend their time “writing” requirements and very little time analyzing and understanding potential implications and impacts of those requirements. When time is limited or a new system is being developed from minor modifications of an existing system, this method may be marginally acceptable. Every system is different and should be evaluated on a case-by-case basis.

Question 182

Specification Writing Versus Development Principle

Answer 182

Anyone with basic grammar skills can “write” a random set of requirement “shall” statements. However, “development” of a specification requires a higher level of Systems Thinking that requires understanding, organizing, modeling, and translating a User’s operational needs and constraints into a

Question 183

Point Design Avoidance Principle

Answer 183

Specifications do not specify a point design solution unless there is a compelling technical reason to do so. For specifications developed for implementation internal to the System Developer’s Enterprise such as Configuration Items (CIs) (Chapter 16), you may decide to include this figure as a convenient reference. However, if you intend to procure the System or one or more Subsystems from external vendors, you are violating this principle by mandating a specific solution. Remember - specifications specify what has to be accomplished and how well, not how to design the System. Think about the implications! What if you architecturally: *Specify the wrong solution or one that does not perform? *Exclude solution options that may prove to have been optimal?

Question 184

Specification Scope Principle

Answer 184

Specifications specify what Systems or Entities are expected to accomplish and how well, not work tasks to be performed and accomplished by the project.

Question 185

Project Work Scope Principle

Answer 185

A project contract’s work scope—Contract Statement of Work (CSOW) or Project Charter—specifies work tasks to be accomplished, not Systems or Entity specification requirements

Question 186

Specification Baseline Principle

Answer 186

Every specification should be formally reviewed, approved, baselined and placed under formal Configuration Management Change Management, released, and communicated for technical decision making

Question 187

User Wants, Needs, Can Afford, and Willingness to Pay Principle

Answer 187

When deriving System/Entity requirements, understand: *What the User wants. *What the User needs. *What the User can afford. *What the User is willing to pay

Question 188

Requirements Allocation and Flow Down Principle

Answer 188

SOI specification requirements should be derived to a level that allows direct assignment, allocation, and flow down of each requirement to a specific Entity within the SOI’s architecture for implementation. Without becoming immersed into whether Requirement A_11 should be allocated to Subsystems A or B, we construct a UC thread at the bottom of the figure. Based on UC thread analysis, SEs derive: * Child Requirements A_111 and A_112 from Parent Requirement A_11. * Child Requirements B_111, B_112, and B_113 from Parent Requirement B_11. Since these requirements can be physically implemented, we allocate them to Subsystems A and B in the System Architecture and flow them down to their respective Subsystems A and B EDSs

Question 189

Vertical and Horizontal Requirements Traceability Principle

Answer 189

Specification requirements allocations and flow down ensure vertical requirements traceability to source or originating requirements. Use Case threads integrate those allocations into horizontal capability threads that produce SYSTEM Level performance-based outcomes traceable to specification requirements

Question 190

Technical Performance Measures (TPMs) Principle

Answer 190

Periodically report the status, progress, and maturity of Technical Performance Measures (TPMs) that trace and contribute to achievement of higher level User KPPs and SPS MOEs, or MOSs

Question 191

Requirement Uniqueness Principle

Answer 191

Each requirement statement shall be unique to one and only one System or Entity with no other instances of overlaps, duplications, or confliction within the System

Question 192

Operating Environment Conditions Principle

Answer 192

Every specification should contain a global requirement specifying that a System/Product comply with the requirements before, during, and after exposure to the specification’s Operating Environment conditions and constraints. Each specification must include at least one requirement statement that establishes, characterizes, and bounds the prescribed Operating Environment conditions that constrain the specified System capabilities. For example: *The System shall comply with the requirements specified herein before, during, and after exposure to the conditions specified in Table 20.1 SPS or EDS Section 3.5, “Operating Environment Conditions.”

Question 193

SMART-VT Criteria Principle

Answer 193

Every specification requirement shall comply with the SMART-VT Criteria- Specific, Measurable, Actionable, Realistic, Testable (Doran, 1981, pp. 35–36) and Verifiable, and Traceable. Requirements statements must be Specific, Measurable, Actionable – Allocatable, Realistic, Testable, Verifiable and Traceable. If a requirement statement fails any one of these criterion, it should be reworked until it complies

Question 194

Requirement Verification Method(s) Principle

Answer 194

Compliance to each requirement statement shall be proven via one or more of the following verification methods: Inspection, Examination, Analysis, Demonstration, Test, or Similarity (if permissible) that represents the least cost and effort. Well-defined requirements statements must be supported by one or more verification methods such as Inspection, Examination, Analysis, Demonstration, and Test

Question 195

Requirement Structure Principle

Answer 195

Every requirement statement requires (1) a subject of the action, (2) “shall” to express a mandatory action for compliance, (3) an action-based verb, (4) a capability-based performance outcome to be provided, and (5) conditions for accomplishment. Every specification requirement statement should consist of the following elements: 1. Source of the Action 2. “Shall” (Mandatory compliance) 3. Action-based Verb 4. Performance-based Outcome to be Accomplished (What) 5. Level of Performance (How Well) *In some cases, the requirement statement structure may require additional constraints such as: 6. Condition-Based Actions (Requirement Dependent) (When) 7. Recipient(s) of the Action (Requirement Dependent

Question 196

Requirement Action Source Principle

Answer 196

Each requirement statement shall identify the source and stimulus, excitation, or cue that initiates a system, product, or service capability

Question 197

Shall-Based Requirements Principle

Answer 197

Specify every requirement statement with the word “shall” to express mandatory actions -outcomes - required for accomplishment to achieve compliance and User acceptance. Each requirement statement uses “shall” to express the mandatory action to be accomplished. When “shall”-based requirements are incorporated into a specification that is an element of an approved contract, the requirement is considered to be "legally binding"

Question 198

Requirement Action Verb Principle

Answer 198

Employ an action-based verb in each requirement statement to express the action to be accomplished. Requirements statements employ action-based verbs to express the value-added action to be performed such as sense pressure, convert sunlight to energy, process sensor data, or store information, energy, data, or materials. For example: * “The Vehicle Management System (VMS) shall detect faults... * The computer shall store the XYZ data

Question 199

Requirement Single Outcome Principle

Answer 199

Each requirement statement shall specify one and only one capability with one and only one performance-based outcome to be accomplished

Question 200

Requirement Action Completion Principle

Answer 200

Where appropriate, each requirement statement shall bound and specify the time allowed for completion of an action. Where applicable, well-defined requirements statements may need to specify the maximum allowable time for responding to a stimulus, excitation, or cue.

Question 201

Requirement Outcome Format Principle

Answer 201

Where appropriate, each requirement statement shall specify the format of an action-based outcome response. When applicable, well-defined requirements statements specify the format of the action response to cooperative, benign, or hostile interactions based on rules of engagement

Question 202

Requirement Accuracy and Precision Principle

Answer 202

Each specification requirement statement shall quantitatively bound the accuracy and precision of the outcome level of performance and its tolerance limit(s). Once a capability-based outcome is identified, the next step is to quantitatively bound its level of performance in terms of accuracy and precision.

Question 203

Requirement Response Time Principle

Answer 203

Where appropriate, each requirement statement shall specify the response time that bounds accomplishment of an outcome. Where applicable, well-defined requirements statements specify time constraints between a triggering event such as a stimulus, excitation, or cue and the required outcome response.

Question 204

Requirement Conditional Actions Principle

Answer 204

Where appropriate, each requirement statement shall specify the trigger-based conditions that initiate the action. Sometimes a requirement statement is dependent on triggering conditions such as a stimulus, excitation, or cue. As a result, the condition has to be embedded within the statement.

Question 205

Requirement Media Principle

Answer 205

Where appropriate, each requirement statement outcome shall specify the media to be used for delivering the outcome. When applicable, well-defined requirements statements specify the format of the action response to cooperative, benign, or hostile interactions based on rules of engagement

Question 206

Requirement Action Recipient Principle

Answer 206

Where appropriate, each requirement statement shall specify the recipient of an action. Where applicable, well-defined requirements statements identify the intended recipient(s) of the outcome(s).

Question 207

Requirement Unacceptable Outcomes Principle

Answer 207

Where appropriate, each requirement statement shall specify one and only one outcome to be avoided when performing an action. Well-defined requirements statements may need to specify outcomes to be avoided.

Question 208

Requirement Identifier and Title Principle

Answer 208

Assign a unique identifier and title to each requirement statement to facilitate searches and readability. Requirements tend to lose their identities as statements within a larger document. This is particularly troublesome when the need arises to rapidly search for an instance of a requirement. Make it easy for reviewers and users of specifications to easily locate requirements. Tag each requirement with a unique identifier and label each with a “bumper sticker” title.

Question 209

External References Principle

Answer 209

Specification references to external specifications and standards must include the following: Document ID, Version, and Date. References to internal sections should include Section Number and Title

Question 210

Compound Requirements Principl

Answer 210

Avoid usage of compound requirements statements written in paragraph style. Paragraph-based declarative statements of fact are acceptable in specifications for Specification Section 1.0 Introduction. The problem is System Developers must spend valuable time separating or parsing compound requirements into singular requirements when they appear in SPS or EDS

Question 211

Endless Requirements Principle

Answer 211

Eliminate usage of endless requirements terms such as all, etc., et al., and and/or, which are impractical to specify, bound, and verify. A note to specification writers: Eliminate every use of the word “all” in every specification! Appreciate the legal significance of this statement “...all instances of an XYZ will be verified . . . .” Contemplate how large in scope the “all” universe is! “All”, a limitless word for practical purposes, is boundless and subject to interpretation. As a specification developer, your mission is to specify and bound the solution space as simple and practical; not for verifying every conceivable scenario or instance of an entity

Question 212

Official Requirement Principle

Answer 212

A requirement is not considered official until the following criteria have been met: *Is accepted by a consensus of its Stakeholders. *Is traceable to source or originating requirements via an RVM. *Satisfies requirements validation criteria *Is assigned one or more verification methods. *Is approved and released for implementation

Question 213

System Applications Principle

Answer 213

Specification requirements should be written to support any one of three types of system applications: single use, reusable, and multi-purpose. User mission applications of a system ultimately drive its specification requirements. Most systems are designed for three primary types of mission applications: single use, reusable, and multi-purpose. Let’s clarify these terms: *Single Use Systems are designed for a one-time use. Examples include: satellites, rockets, missiles, munitions, medical syringes, and batteries. *Reusable Systems are systems configured to perform numerous missions with periodic maintenance. Examples include: automobiles, computers, aircraft, flashlight, rechargeable batteries, and homes. *Multi-Purpose Systems represent Reusable Systems that are reconfigurable to accommodate several types of mission applications. For example: aircraft can be reconfigured to carry passengers or cargo, computers can be reconfigured to execute various types of software applications, and external hardware devices such as printers or modems

Question 214

Wasson’s Task Significance Principle

Answer 214

The time allocated by management for most program and technical decision-making tasks is inversely proportional to the significance of the tasks or decision to the deliverable product, User, or Enterprise Three of the most crucial specification analysis tasks during a proposal effort are understanding: 1. What problem the User is trying to solve. 2. What system, product, or service does the System Acquirer’s formal solicitation’s SRD/S00 specify. 3. What you have committed your Enterprise to via the draft SPS submitted as the proposal response

Question 215

Specification Referenced Documents Principle

Answer 215

Specification Section 2.0 “Referenced Documents” lists only documents explicitly referenced by Section 3.0 “Requirements” and 4.0 “Qualification Provisions”; remove all other non-referenced documents

Question 216

Broad References Principle

Answer 216

Avoid broad, general references to external documents. Always, specify document title, version, release date, section number(s) and section title(s)

Question 217

Decision-Making Reference Principle

Answer 217

Under standard operating practices, project personnel perform to current, baselined work products that have been: (1) approved for release and decision-making and (2) communicated to all stakeholders. Once a specification is approved, baselined, and released, changes must be formally approved and communicated to Stakeholders. Because of a lack of communications, two problems can occur: 1. Program and technical management often lack an appreciation of the need to communicate specification changes to project personnel. Ironically, the technical integrity of the project is at risk, which is the very topic management seems to be averse to. 2. When management does communicate changes, Project personnel often ignore announcements about specification changes such as Specification Change Notices (SCNs) and continue to work with a previous version.

Question 218

Specification Effectivity Principle

Answer 218

When specifications apply to different model numbers, versions, or blocks serial numbers, explicitly designate each requirement with the appropriate effectivity. Some specification requirements may only be applicable to specific configurations and units—that is, configuration effectivity. Where this is the case, label the requirement as only applicable to the Configurations A, B, C, etc., or Serial Numbers XXXX–YYYY. If this occurs, include a statement to the reader on the cover and in the Section 1.0 Introduction that this specification applies to configurations A, B, and C and Serial Number effectivity XXXX -YYYY

Question 219

Requirements Issues Tracking Principle

Answer 219

Track specification requirements issues to closure. When specification requirements issues and clarifications are identified, it is critical to bring all of them to closure quickly. The preferred approach for tracking closure status is to establish a metric that represents the number of actionable COIs, CTIs, TBDs, TBSs, and clarifications remain “open.”

Question 220

User’s Mental Model Principle

Answer 220

To reduce or eliminate human error, thoroughly understand the User’s conceptual mental models for task performance before designing Equipment-Hardware and Software

Question 221

Personnel Dual Accountability Principle

Answer 221

The Personnel Element in each Mission System or Enabling System has two levels of accountability: (1) mission accountability and (2) system accountability. Since the Personnel Element is an operational entity within the SEA, our discussions up to this juncture have treated it architecturally as a peer-level component to the other System Elements. Observe the “architectural” context here. The reality is that Higher-Order Systems— Enterprises—employ each SOI, Mission System, and Enabling Systems to accomplish missions that have outcomes and performance-based objectives. Who does a Higher-Order System hold accountable for mission performance - inanimate objects such as the Equipment Element - Hardware and Software, Procedural Data, or Mission Resources? No, the Mission System or Enabling System operators, maintainer, and trainers that comprise the Personnel Element. As a result, the Personnel Element has two levels of accountability: 1. System accountability as a performing System Element within the SEA (Figure 9.2) for safely and properly controlling Mission System performance. 2. Mission accountability to Higher-Order Systems for accomplishment of mission task order outcomes and performance

Question 222

Engineering Accountability Principle

Answer 222

Engineering is a professional discipline that demands performance and accountability for all aspects of our work, including “Engineering the System”, not just the “Engineering of Boxes”

Question 223

Root Causes Principle

Answer 223

Accidents are typically the result of a chain of weaknesses, not just one, in a system’s barriers or safeguards that are intended to prevent a hazard from escalating into an incident or event with negative consequences

Question 224

Do No Harm Principle

Answer 224

Design Enterprise and Engineered Systems, products, or services to DO NO HARM to their Users, End Users, the public, or the environment

Question 225

Human Systems Integration (HSI) Application Principle

Answer 225

HSI is a continuous, multidisciplined activity that spans the entire System/Product Life Cycle, not just System Integration, Test, and Evaluation (SITE). The concept of HSI has existed for several decades. However, one of the difficulties in understanding HSI begins with its name. The fallacy of the HSI title, which can be taken literally and erroneously, is its ambiguity —ambiguity that infers that a System Developer is going to develop a system, product, or service; obtain User acceptance; and deliver it to the User to integrate their operators, maintainers, and trainers for System Operations, Maintenance, and Sustainment. In this context, the application of the term appears to be an event. HSI, however, is not an event. It represents the integration of HE, HFE, and Ergonomics knowledge with Engineering Design to produce systems, products, or services to that are simply compatible and interoperable with and usable by the User(s). To illustrate this point, consider a couple of perspectives from the USAF and INCOSE

Question 226

Stress Points (SPs) Principle

Answer 226

Optimal System performance is achieved when the User–System Stress Points (SPs) have been avoided, eliminated, or reduced

Question 227

User-Centric Design Principle

Answer 227

Design the Equipment Element and adjust the task workload to fit the User’s HF capabilities and limitations, not vice versa

Question 228

Display Information Principle

Answer 228

Display only essential information based on the User’s need-to-know and priorities; avoid data overload

Question 229

Superfluous Information Principle

Answer 229

Eliminate superfluous, non-value-added information unless the User has consciously requested it and the information would be available

Question 230

Human Factors (HF) Application Principle

Answer 230

“Anywhere you find technology and people interacting, there is a need for human factors engineering”

Question 231

HF Qualified Professional Principle

Answer 231

Employ the services of competent, qualified professionals to perform HFE. When human interactions are a key element of “Engineering the System”, challenging technical decisions have to be made regarding the Personnel and Equipment Elements

Question 232

Optimal UCSD Principle

Answer 232

Optimal System performance is achieved when the Personnel and Equipment Elements are compatible, interoperable, and balanced based on what each does best. The SPS and each EDS should specify and bound System capabilities, interfaces, design, and construction constraints related to Personnel capabilities—skill levels and limitations. You should recall that: * One of the fallacies of the ad hoc SDBTF-DPM Enterprise environments is making a quantum leap from SPS or Entity specification requirements to a Physical Domain Solution. *When specifications are written, they should specify what has to be accomplished and how well, not dictate how to design the system, product, or service—Physical Domain Solution

Question 233

Situational Assessment Principle

Answer 233

Provide real-time Situational Assessment information to the User to evaluate Mission performance and System Operational Health & Status (OH&S)

Question 234

Visual Information Principle

Answer 234

Present visual information to the User in accordance with established Human Engineering (HE) and System Safety design principles.

Question 235

Visual Indicators Principle

Answer 235

Provide visual notifications, alerts, and alarms to inform the User about conditions that may impact System performance, cause damage, or threaten the safety of its User(s)

Question 236

Auditory Cues Principle

Answer 236

Provide User auditory cues for notifications, alerts, and alarms commensurate with the level of significance and their spectral audio range. Audio cues consist of warnings, cautions, and alerts that notify the operator or maintainer of specific equipment health and status conditions that require attention with varying levels of urgency for action. Various tonal frequencies as well as sequences and patterns of tones are employed to symbolize system-operating conditions

Question 237

Command and Control (C2) Devices Principle

Answer 237

Select C2 devices that are appropriate, compatible, and interoperable with User anthropometrics and biomechanics

Question 238

Escape Mechanisms Principle

Answer 238

Provide operational escape mechanisms—safety latches, stop switches or alarms, abort processing, restarts, and fire suppression systems—that allow the User to safely override or exit conditions that require immediate corrective actions without causing panic, injury, damage, or catastrophic results

Question 239

Unwarranted Assumptions Principle

Answer 239

“Unwarranted assumptions in design can produce unintended consequences” -This is also important for requirements development

Question 240

System Usability Principle

Answer 240

System, product, or service usability is key to winning User hearts, minds, confidence, and acceptance—key elements of customer satisfaction. SEs should always collaborate with the Users and factor their reviews comments into decision-making processes

Question 241

Fail Safe Design Principle

Answer 241

Where necessary, every system, product, or service should incorporate fail safe design features that prevent or minimize human error that could result in injury or loss of life or damage to the environment. When humans interact with Equipment, the potential for performing unsafe acts —slips, lapses, errors, mistakes, and violations—increases. Design and implement the Equipment Element to DO NO HARM— prevent or minimize the potential for human errors that may injure the User or the public, damage or pollute the environment, or result in loss of life.

Question 242

Engineering Standards Principle

Answer 242

Engineering standards of units, coordinate systems, and conventions should be specified in one and only one official document that has been reviewed, approved, baselined and placed under formal Configuration Management (CM) control, released, and communicated for decision making

Question 243

Engineering Standards, Coordinate Systems, and Conventions Principle

Answer 243

Engineering standards, coordinate systems, and conventions are the foundational Achilles Heal of Engineered Systems and Enterprise System development. Neglect them and the systems will fail. Engineering standards provide a mechanism for Enterprises and industries to: 1. Establish a consensus of performance requirements for development of systems, products, and services. 2. Audit compliance of those deliverable work products. 3. Provide a framework for targeting improvements related to performance and safety

Question 244

SI Fonts Principle

Answer 244

Document quantity symbols using an italic font and symbols for dimensions in sans serif roman capitals”

Question 245

Observer’s Frame of Reference Eyepoint Principle

Answer 245

Human coordinate systems consist of an Observer’s Frame of Reference structured with three principal axes labeled +X, +Y, and +Z that are orthogonal to each other and intersect at an origin located at the Observer’s eyepoint

Question 246

Coordinate System Principle

Answer 246

The Observer’s eyepoint serves as an origin for an Observer’s of Frame Reference; a coordinate system establishes the spatial location of a point or object in space relative to that origin using x-, y-, and z-axis coordinates. Once the Observer’s Frame of Reference is selected, we need to establish a convention to recognize travel along the axes. This leads to a key question: what is a convention? A convention establishes an orientation for describing actions observed relative to the Observer’s Frame of Reference with the eyepoint located at the origin. Coordinate systems serve as a static Observer’s Frame of Reference for locating an object’s position at a specific instant in time relative to the frame’s origin and axes. People often equate an Observer’s Frame of Reference as a Coordinate Reference System. In fact, the Observer’s Frame of Reference provides the multi-axis framework for assigning direction of travel and units of measure that form the Coordinate Reference System

Question 247

Observer’s Frame of Reference Convention Principle

Answer 247

An Observer’s Frame of Reference is characterized by a Left-Hand (LH) or Right-Hand (RH) orientation. One of the first steps in establishing an Observer’s Frame of Reference is deciding to use a LH or a RH Rule convention

Question 248

Observer’s Multi-Axis Configuration Principle

Answer 248

Pick one and only one multi-axis configuration for a given Observer’s Frame of Reference eyepoint, preferably to serve the whole project. Declaring a LH or RH Rule convention is a necessary but insufficient criterion for characterizing the Observer’s Frame of Reference. The reality is the +X, +Y, and +Z axes can be rotated resulting in new structural reference configurations

Question 249

State Vector Principle

Answer 249

Every free body moving through space at a specific instant in time is represented by a state vector that characterizes the body’s geophysical or geospatial location –position vector - and rate of change - velocity vector - components

Question 250

Rotational Conventions Principle

Answer 250

Rotation conventions about the principal +X, +Y, and +Z axes of an Observer’s Frame of Reference are established by the Right-Hand (RH) Grip Rule. Rotational movements enable us to characterize an object’s dynamic movements in terms of multi-axis rotation. This requires that we establish a uniform rotation convention that can be applied to all principal axes. The solution is the Right-Hand (RH) Grip rule

Question 251

System Stakeholder Concerns and Views Principle

Answer 251

An architecture expresses System Stakeholder concerns via views that comply with viewpoint conventions for constructing the view

Question 252

Architectural Ownership and Accountability Principle

Answer 252

Assign ownership and accountability for the development of each System or Entity architecture. Every System or Entity Architecture is assigned to an Owner accountable for its development and maintenance

Question 253

Architectural Uniqueness Principle

Answer 253

Assign a unique identifier (ID) to each System or Entity architecture. Every System or Entity Architecture document should be assigned a unique document ID that differentiates it from all others

Question 254

Architectural Context Principle

Answer 254

Every System or Entity architecture begins with establishing its context within the User’s Level 0 Higher-Order System

Question 255

Architectural Framework Principle

Answer 255

Every System or Entity architecture requires a Level 1 System architecture that illustrates: 1. Its framework of System Elements—Personnel, Equipment, Mission Resources, Procedural Data, System Responses, and Facilities—their relationships, and interactions. 2. External interfaces and interactions with Human Systems, the Natural Environment, and the Induced Environment Systems that comprise its Operating Environment

Question 256

Architectural Domain Views Principle

Answer 256

Each System or Entity architecture consists of one or more viewpoint models that: 1. Represent views shared by one or more of its Stakeholders. 2. Express a conceptualization to guide Solution Space development. 3. Alleviate any concerns

Question 257

Architectural Validity Principle

Answer 257

The validity of an architecture is determined by its ability to: 1. Configure and control its capabilities in response to UC-based User Command and Control (C2) during Deployment; Operations, Maintenance, and Sustainment (OM&S); and Retirement/Disposal Phases of the System/Product Life Cycle. 2. Deliver the required outcomes and levels of performance for each configuration

Question 258

Architecture Completeness Principle

Answer 258

A System or Entity architecture is complete when it: 1. Complies with Stakeholder specification requirements. 2. Clearly communicates a solution that alleviates Stakeholder concerns and mitigates operational, technical, technology, cost, and schedule risks. 3. Contains essential information that is necessary and sufficient for design. 4. Has been verified, validated, and accepted by its Stakeholders

Question 259

Architectural Consistency Principle

Answer 259

A System or Entity Architectures and Engineering designs must be consistent in nomenclature, mnemonics, and terminology with all other architectures internal and external to the System Multi-Level System Architectures, especially for large, complex systems, are often developed by different people and contractors. One of the complexities of System Design integration is ensuring that the architectures be developed as if a single source created them

Question 260

Architectural Traceability Principle

Answer 260

Every System or Entity Architecture must be traceable to Higher-Order System architectures and to User requirements via the System Performance Specification (SPS). System /Entity Architectures must be traceable to each other and User source or originating requirements via the SPS

Question 261

Measurement and Control Principle

Answer 261

Unless you can measure the performance of a system or process, you cannot control it

Question 262

Measuring the Unmeasurable Principle

Answer 262

Avoid demands to “measure the unmeasurable"

Question 263

Situational Assessment Information Principle

Answer 263

Essential, real-time Situational Assessment information should always be current and readily available for Users to make informed decisions

Question 264

MC2 Capability Principle

Answer 264

MC2 is a capability, not a physical implementation. Our discussions up to this point have highlighted the importance of MC2 in Systems or Products. Remember that MC2 is a capability, not a physical implementation. Some Systems may use a single processor to provide both capabilities within an application. In contrast, complex systems may dedicate processors specifically to performing one of the capabilities such as an automobile’s network

Question 265

Fault Isolation & Containment Principle

Answer 265

When appropriate, every System/Entity architecture should provide the capability to detect, isolate, neutralize, and contain faults to prevent or minimize propagation if their effects to internal components or external systems.

Question 266

Fault-Tolerant Architectures Principle

Answer 266

Fault-tolerant architectures are intended to improve the robustness of a System Design Solution to cope with unanticipated faults and scenarios, not as a substitute for a lack of System Development discipline in eliminating latent defects

Question 267

Network Architecture Principle

Answer 267

Network architectures represent potential solutions to System problems, not necessarily the only solution

Question 268

Architectural Power and Backup Principle

Answer 268

Factor energy and backup power source considerations for normal, abnormal, and emergency operations and conditions into every System Architecture

Question 269

Architectural ES&OH Principle

Answer 269

Factor considerations for Personnel and the public’s Environmental Safety and Occupational Health (ES&OH) into every System Architecture.

Question 270

Fire Detection and Suppression Principle

Answer 270

Where appropriate, every system architecture should incorporate safety features for fire detection and suppression

Question 271

Architectural Vulnerabilities Principle

Answer 271

Assess System vulnerabilities and factor System security and protection considerations into every system architecture. For systems that involve sensitive or classified data, system security should be a key architectural consideration. This includes reasonable measures for physical security, operational security, communications security, personnel security, and data security

Question 272

Architectural Remote Inspection Principle

Answer 272

Every system architecture should include Situational Assessment considerations not only for normal but also abnormal operations via remote inspections

Question 273

Interfaces as System Components Principle

Answer 273

Interfaces, as components of every Engineered System or Product, enable emergent behavior to reveal itself

Question 274

System-Interface Design Priority Principle

Answer 274

In some cases, a System or Entity’s design has priority and drives interface design; in other cases, interface design has priority and drives System or Entity design

Question 275

Interface Ownership and Accountability Principle

Answer 275

Assign ownership and accountability for every interface within the System or Product based on the highest-level architecture that identifies the interface. his includes the specification, design, and control of the interface. Observe that we said assign “ownership and accountability for every interface ... to an individual or team."

Question 276

Single Information Source Principle

Answer 276

Everything a User “needs to know” about a System or Entity such as its specification requirements, UCs and scenarios, operational concepts, design, interfaces, testing, verification, or validation should be encapsulated within a single document unique to that subject. Exceptions include separation of the information into different documents due to: (1) the quantity of pages or (2) Sensitive Information Protection (SIP) containment of proprietary, IP, competition sensitive data, privacy, government security, and data on a justified “need-to-know” basis

Question 277

Interface Disclosure Principle

Answer 277

To prevent disclosure of Intellectual Property (IP) or Competition Sensitive information, isolate requirements for a specific interface into a separate for coordination with external Enterprises that have a justified “need to know.”

Question 278

Interface Identification Principle

Answer 278

Assign a unique Source-Destination mnemonic and title to each System/Entity interface. Once an interface is identified, establish a naming convention that assigns a unique identifier to distinguish the interface from all others within the System. A typical naming convention consists of a Source-to-Destination naming convention with the interfacing System or Entity names represented by short names, acronyms, or abbreviations. Consider the following example

Question 279

Power of Engineering Principle

Answer 279

The power of Engineering resides in its application of “...judgment to develop ways to utilize economically the materials and forces of nature for the benefit of mankind.”

Question 280

Redundancy versus Redundant Design Principle

Answer 280

Redundancy is a design method to eliminate Single Failure Points (SFPs), thereby increasing System or Entity’s reliability. Redundant design is the selection and physical placement and separation of components to ensure survival

Question 281

Failure Modes & and Effects Analysis (FMEA) Principle

Answer 281

Failure Modes & Effects Analysis (FMEA) requires more than simply analyzing failures and effects of components fixed in place or connected. Analyze the direct or indirect collateral effects on adjacent Entities.

Question 282

Interface Fault Containment Principle

Answer 282

When faults lead to failures, isolate and contain them to prevent or minimize their consequential effects on nearby Entities—Equipment; Personnel—operators, maintainers; the public, and the environment

Question 283

System Complexity Principle

Answer 283

The complexity and risk of a System or Product is a function of the quantity of its interfaces

Question 284

Interface Reduction Principle

Answer 284

Apply analytical and risk reduction methods to minimize the quantity of interfaces in a System or Product

Question 285

SITE Incremental Verification Principle

Answer 285

Integrate and incrementally verify one and only one test article and its capabilities at a time into a previously verified configuration. The guiding philosophy of SITE can be summarized in a few words: keep it simple! Simplicity means verify each Entity – Product, Subsystem, Assembly, Subassembly, or Part - to its specification requirements and then incrementally integrate other verified items one at a time.

Question 286

Fault Isolation Principle

Answer 286

To isolate a fault in a serial configuration chain, employ the Half Split Method and iterate until the fault is located

Question 287

Single Test-Multiple Requirements Principle

Answer 287

Strive to verify as many specification re- quirements as practical with a single test, if appropriate.

Question 288

Testing Conflict of Interest Principle

Answer 288

Testing and verification of personal work products, especially contract deliverables, is viewed as a conflict of interest and should be objectively performed by an independent tester and verifier.

Question 289

Discrepancy Reporting Principle

Answer 289

Discrepancy Reports (DRs) document a discrepancy found during test article ver- ification, not the source of the problem

Question 290

Corrective Actions Principle

Answer 290

Disposition Discrepancy Reports (DRs) quickly for analysis to determine root cause and corrective actions ; allocate re- sources based on the level of significance and priority . Where appropriate, collaborate with the Acquirer Technical Repre- sentative (ATR). Inevitably, discrepancies occur between actual test data and expected results specified in the SPS, EDS, or design requirements such as drawings, schematics, and parts lists. We refer to the occurrence of a discrepancy as a test event or incident . When test events such as anomalies or failures occur, a Discrepancy Report (DR) is documented by the Test Operator as a Test Event. The Test Director should es- tablish ground rules for what qualifies as a test event in the System Integration and Verification Plan (SIVP) and event criteria for documenting a DR.

Question 291

Acquirer Technical Representative ATR Principle

Answer 291

Every System Acquirer should appoint one Test Representative who serves with authority as the single Voice of the Customer (VOC) representing the Acquirer-User community concerning SITE decisions. Throughout SITE, test issues emerge that require an Acquirer decision. In addition, some Acquirers represent several Enterprises, many with conflicting opinions and agendas. This presents a challenge for a System Developer. One solution is for the Acquirer Project Manager (PM) to appoint one individual to serve (1) as their on-site Acquirer Technical Representative (ATR) at the System Developer’s facility and (2) single voice representing all Acquirer view- points and decisions. Primary responsibilities of the ATR are to: 1. Serve as the single, technical point of contact for all Acquirer and User technical and contract interests and communications. 2. Collaborate with the Test Director to resolve any SITE-related COIs/CTIs that affect Acquirer-User in- terests. 3. Where applicable by contract, collaborate with the Test Director to assign priorities for DRs resolution and corrective action. 4. Where required by contract, review and coordinate approval of ATPs. 5. Where appropriate, provide a single set of ATP com- ments that represent a consensus of the Acquirer-User Enterprises. 6. Witness and formally approve ATP results

Question 292

Root Cause(s) Principle

Answer 292

Identify the root, most likely, or probable cause(s) of discrepancy, incident, or acci- dent events via a process of elimination of contributing factors. When a test anomaly or failure occurs and a DR is recorded, a determination has to be made concerning the significance of the problem on the test article and test plan as well as isolation of the problem source. While the general tendency is to focus on the test article due to its unproven design, the source of the problem can originate from any one of the Test Environment elements shown in Figure 28.1 such as a Test Operator or Test Procedure error; test configuration or Controlled Operating Environment problem; or combinations of these. From these contributing elements, we can construct a Test Discrepancy Fault Isolation Tre

Question 293

Certified Tools & Equipment Principle

Answer 293

To avoid invalidating verification test re- sults, ensure that all required test tools and equipment are certified to be calibrated and aligned to source standards prior to each test. The credibility and integrity of a Verification and Vali- dation (V&V) effort are often dependent on Enabling System: 1. Test Facilities and Equipment to establish a controlled Operating Environment for System modeling, simulation, and testing. 2. Special tools used to make precision adjustments in System/Entity capabilities and outputs. 3. Instrumentation used to measure and record the System’s environment, inputs, and outputs. 4. Tools used to analyze the System responses. All of these factors: 1. Require calibration, alignment, or certification to national and international standards for weights, measures, and conversion factors. 2. Must be traceable to national/international stan- dards. Therefore, avoid rework by ensuring that V&V activi- ties have technical credibility and integrity . Begin with a firm foundation by ensuring that test equipment and tools are calibrated, certified, and traceable to source standards within a prescribed time period as marked on the item.

Question 294

Test Data Access Principle

Answer 294

Anticipate what types of access ports Hardware test points, and Software mnemonic data will be required from En- tities at each level of integration to collect test data required for specification requirements and design verification

Question 295

Fielded System Performance Principle

Answer 295

Continuously monitor and Assess System or Product performance throughout the System Deployment, OM&S, Phase-Out, and Disposal Stages of the System/Product Life Cycle. Due to the lack of Systems Engineering (SE) “up front” and throughout the System Development, there is a general misperception that SE activities end when an Engineered System or Product has been verified, validated, and ac- cepted. However, SE activities continue into the System Deployment, OM&S, Retirement, and Disposal Phases of the System/Product Life Cycle. The difference is that SE, in general, shifts from System Development of the System or Product to assessing the achievement of the verified performance by the User.

Question 296

System Delivery Method Compatibility Principle

Answer 296

The design of every system, product, or service must be physically compatible with its method of delivery and constraints. In general, most Systems or Products are deployed by one of several approaches: * Approach #1 —Containerized shipping—commercial products, parcel packages, fruits and vegetables, mod- ular containers on ships. * Approach #2 —Combinations of land, rail, sea, air, or space-based transport. * Approach #3 —Deployment/relocation of a System under its own power such as aircraft, ships, spacecraft

Question 297

Fielded Systems Baseline Principle

Answer 297

The As-Maintained Developmental Con- figuration baseline should always be main- tained to identically match the physical configuration of the fielded System or Product

Question 298

Data Precision Principle

Answer 298

Data with two-digit precision that require multiplication do not yield four-digits of precision; the best you can achieve is the source’s two-digits of precision

Question 299

Analysis Validity Principle

Answer 299

Analysis results are only as valid as their underlying assumptions, models, and methodology. Validate and preserve their integrity

Question 300

Performance Budgets and Margins Principle

Answer 300

Every System/Entity design re- quires performance budgets that bound capabilities and provide a margin of safety to accom- modate variability and uncertainty in component and operational performance for a prescribed set of Operating Environment conditions

Question 301

MOP Risk Principle

Answer 301

Every MOP has an element of develop- ment risk. Mitigate the risk by establishing one or more boundary condition thresholds to trigger risk mitigation actions to reduce the risk to accept- able levels

Question 302

Equivalence Allocation Principle

Answer 302

Apply the equivalence allocation method to allocate and flow down non-functional requirement constraints to Products, Subsystems, and Assemblies.

Question 303

Apportionment Allocation Principle

Answer 303

Apply the apportionment method to allocate and flow down quantifiable, MOPs such as electrical power, timing, weight, or size based on informedd decisions, component or design history, boundary conditions, technology constraints, feasi- bility, and professional experience.

Question 304

Synthesis Allocation Principle

Answer 304

Apply the synthesis allocation method to flow down complex , quantifiable System-Level performance require- ments values such as Measures of Effectiveness (MOEs) or Measures of Suitability (MOSs) to Products, Subsystems, Assemblies using analytical models

Question 305

Processing Segments Principle

Answer 305

When planning processing tasks, at a min- imum, include considerations for queue time, processing time, and transport time in all computations. Most tasks, whether performed by Personnel or Equip- ment (Figure 24.14), involve three phases as follows: * Pre-Mission Phase preparation, configuration, or re- configuration. * Mission Phase performance of the task. * Post-Mission Phase delivery of the required results and any residual Post-Capability Housekeeping Operations in preparation for the next tasks

Question 306

System Optimization Principle

Answer 306

Theoretically, a System can only be opti- mized within its specified performance re- quirements and boundary conditions when most, if not all, latent defects such as capability deficiencies, design errors, design flaws, and weak components have been eliminated. When the System enters the System Integration, Test, and Evaluation (SITE) Phase, latent defects such as system deficiencies, design flaws, and design errors often consume a significant amount of the SE’s time. The challenge is getting the System to a state of equilibrium and stability that can best be described as compliant with the SPS

Question 307

Pareto 80–20 Principle of SE

Answer 307

On average, 80% of System/Entity per- formance issues are attributable to 20% of the System tasks

Question 308

Analyses Versus Trade Studies Principle

Answer 308

Analyses investigate, analyze, and docu- ment a specific condition, state, circum- stances, or “cause-and effect” relationships; Trade studies formulate, analyze and evaluate viable candidate alternatives and propose prioritized recommendations for selection

Question 309

Key Decisions Tree Principle

Answer 309

Always establish, communicate, and im- plement the decision tree that expresses the sequences and options for key technical decisions and synchronize it with the project schedule mile- stones

Question 310

Trade Study Teams Principle

Answer 310

A trade study team without an approved charter and methodology is prone to wan- der aimlessly

Question 311

Informed Engineering Judgment Principle

Answer 311

Trade studies provide reasoned evaluation substantiated by seasoned experience and facts. As such, they serve as inputs to support informed deci- sion making and should be evaluated with sound Engineering judgment

Question 312

Undocumented Trade Studies Principle

Answer 312

An undocumented trade study is nothing more than personal opinion. A common question is: why document a trade study ? There are several reasons: * First, the Acquirer’s Contract Data Requirements List (CDRL) (Chapter 17) or your Enterprise’s com- mand media may require that you document trade studies. * Second, trade studies formally document key decision criteria, assumptions, and constraints of the trade study environment for posterity. Since SE, as a highly iter- ative process, requires decision making based on pre- scribed conditions and constraints, those same condi- tions and constraints can change quickly or over time. Therefore, you or others may have to revisit previous trade study decisions to investigate how the changing conditions or constraints have impacted the decision or selected course of action such that corrective actions should be initiated. * Third, as a professional, document key decisions and rationale as a matter of disciplinary practice to preserve the integrity of the decision

Question 313

TSR Recommendations Principle

Answer 313

If you charter a trade study, be prepared to implement its recommendations un- less there is a compelling reason to avoid doing so

Question 314

Trade Study Distribution Protocol Principle

Answer 314

Decision authorities disseminate or dele- gate dissemination of TSR copies. To avoid “rumor mill” damage control, obtain advance approval prior to discussion, dissemination, or presentation to anyone. Remember—the decision authority chartered your work, not third parties

Question 315

Cover Letter Principle

Answer 315

Every work product should be delivered to a decision authority via a cover letter that outlines what has been accomplished; any outstanding issues, risks, or concerns; and task charter compliance statement.

Question 316

TSR Principle

Answer 316

Trade Study Reports (TSRs) present fact-based results and prioritized rec- ommendations, not decisions; decision authorities make decisions based on the recommenda- tions, their validity, and confidence in the objectivity of the TSR

Question 317

Trade Study Methodology Principle

Answer 317

Trade studies are only as valid as their task constraints, underlying assumptions, methodology, and data collection. Docu- ment and preserve the integrity of each

Question 318

Resource Commitments Principle

Answer 318

System Design Solution decisions repre- sent commitments of resources long be- fore the resources are actually expended. Beware of those who whimsically believe those decisions are volatile up until the expenditure of funds

Question 319

Model Fidelity Principle

Answer 319

Explicitly define and delineate levels of fi- delity using established industry standards. Model fidelity resides in the User’s mind. High fidelity to one person may be medium fidelity to a sec- ond person and low fidelity to a third person.

Question 320

Root or Probable Cause Methodology Principle

Answer 320

Root, contributory, or probable cause investigations employ a process of elim- ination methodology to rule out failure causes based on objective assessment of the facts

Question 321

Negative Training Principle

Answer 321

In Simulation and Training (S&T), sim- ulated capabilities presented to the User must identically match the operating con- dition – normal, degraded, or failed – of the actual System and its physical components it represents with no discrepan- cies. Otherwise, negative training occurs. One of the challenges of personnel training is to avoid a condition referred to as negative training . Remember—the objective of simulated training is to equip the trainee with the essential skills to be able to operate a System or Product that results in a positive experience. Simulations that physically deviate or detract from the real world are considered negative training

Question 322

Simulation Validity

Answer 322

Models and simulations are only as good as the human attempts to algorithmically represent and validate the System and its interactions with its Operating Environment based on actual field data

Question 323

Box's Model Validity Principle

Answer 323

“All models are wrong but some are useful.”

Question 324

Conditional Reliability Principle

Answer 324

Reliability is the conditional probability that a system, product, or service with a given physical operating condition will successfully complete a mission of a specified duration in a prescribed Operating Environment without disruption

Question 325

R&M Engineering Principle

Answer 325

The objective of R&M Engineering is to ensure continuity of Mission System op- erations and capabilities to successfully complete a mission without disruption and accomplish its performance-based objectives, not keep the Equipment El- ement components from failing

Question 326

Failure Definition Principle

Answer 326

Explicitly define, scope, establish a con- sensus, and document what constitutes a failure among Stakeholders—Enterprise, project, System Acquirer, and User. Then, communicate and ensure everyone understands

Question 327

Mission Reliability Principle

Answer 327

Mission reliability is like a chain; it is only as reliable as the weakest link in its System Architecture configuration

Question 328

Mission-Safety Critical Components Principle

Answer 328

Identify, analyze, and label mission critical and safety critical Entity or Part Level components and ensure ease of access for their replacement

Question 329

Fault Elimination Principle

Answer 329

Every fault— latent defect —left undiscov- ered in a System Design Solution or its implementation is a potential risk that may jeopardize accomplishment of the mission, injury or loss of life to the User and public, result in damage to the System or Product, or damage to the environment.

Question 330

Lifetime Data Functions Principle

Answer 330

Every system, product, or service is char- acterized by seven Lifetime Data Func- tions: 1. The Probability Density Function (PDF) 2. The Cumulative Distribution Function (CDF), F(t) 3. The Reliability Function: R(t) 4. The Failure Rate Function: λ(t) 5. The Mean Time Function 6. The Median Life Function 7. The Mode Life Function

Question 331

Lifetime Data Profile Principle

Answer 331

Every System or Entity has a unique Lifetime Data Distribution profile that characterizes its failure frequency or density as a function of the continuous variable, time

Question 332

Probability Density Function (PDF) Principle

Answer 332

The Probability Density Function (PDF) serves as the foundation for deriving all other Lifetime Data Distributions—the Cumulative Distribution Function (CDF), the Reliability Function (R(t)), the Failure Rate Function (λ(t)), the Mean Time Function, the Median Life Function, and the Mode Life Function

Question 333

PDF Profiles Principle

Answer 333

The Probability Density Function (PDF) is characterized by five primary Lifetime Data Distributions: 1. The Exponential Distribution 2. The Gaussian (Normal) Distribution 3. The Log-Normal Distribution 4. The Weibull Distribution 5. The Bayesian-Weibull Distribution

Question 334

MTBF-MTTF Principle

Answer 334

MTBF represents the Mean Life (θ) of a statistically valid sample of a population of repairable items, MTTF represents the Mean Life (μ) of a statistically valid sample of a population of non-repairable items

Question 335

RMA Estimates Principle

Answer 335

Reliability, Maintainability, and Avail- ability (RMA) are time-variant ass- essments that produce estimates , not predictions , due to the uncertainty in a System or En- tity’s physical operating condition and mission Operating Environment conditions

Question 336

Hazard Rate Principle

Answer 336

Systems or components that are char- acterized by Negative Exponential PDF Distributions mathematically exhibit a constant Hazard Rate, h ( t ), during its Useful Service Life based on random, chance failures

Question 337

Expected Life Principle

Answer 337

The expected life of a System or En- tity is the arithmetic average of the actual lifetimes of a population of units currently in field use

Question 338

Median Life Principle

Answer 338

The median of a System or Entity’s PDF represents a specific instant of time when 50% of identical units are estimated to have failed and 50% continue to survive and operate

Question 339

Mode Life Principle

Answer 339

The mode of a System or Entity’s PDF occurs at the peak failure density (point of inflection) where the maximum number of failures is expected to occur at a specific instant in time

Question 340

Bathtub Curve Principle

Answer 340

The Bathtub Concept is a notional con- cept based on a piecewise composite, end-to-end structure comprised of a sequence of three different Lifetime Data Distributions: (1) Decreasing Failures Region (DFR), (2) Stabilized Fail- ures Region (SFR)—Useful Service Life, and (3) Increasing Failures Region (IFR)

Question 341

Early Failures Principle

Answer 341

Every system, product, or service when placed in active service exhibits a failure rate following manufacture that is a func- tion of its residual latent design defects, weak components and material properties, quality of workmanship, User learn- ing curve, and Operating Environment

Question 342

Early Failures Elimination Principle

Answer 342

Proactive detection and elimination of latent defects results in a diminishing fail- ure rate up to a point of chance failures due to usage, stress (Figure 34.11), and Operating Envi- ronment conditions.

Question 343

Wear-out Principle

Answer 343

Every system, product, or service in active service exhibits a failure rate that increases with age due to wear-out; degradation of components and their mass properties; and User use, misuse and abuse, and misapplication in its Operating Environment

Question 344

Strength-Stress Principle

Answer 344

Every System or Entity should be se- lected with a strength design margin that exceeds its prescribed Operating Environment conditions and stresses

Question 345

Useful Service Life Extension Principle

Answer 345

The Useful Service Life of a system, product, or service can be extended by proactive, Just-in-Time (JIT) sequences of preventive and corrective maintenance actions and up- grades

Question 346

Shelf-Life Principle

Answer 346

Every System or Entity inherently has a “use or lose” shelf-life that diminishes due to the deterioration of material mass properties over time and exposure to environmental condition extremes—temperature, humidity, and other factors

Question 347

Series Network Reliability Estimates

Answer 347

The reliability of a Series Network Con- figuration, "R-Series", composed of “n” com- ponents is computed as the product of the reliabilities a sequence

Question 348

Parallel Network Reliability Principle

Answer 348

The reliability of a Parallel Network Con- figuration, R Parallel is computed as a math- ematical series (it's the same equation as resistors in parallel)

Question 349

Design Modifications Principle

Answer 349

The System Design Solution is not open to random, ad hoc modifications up until the day the System or Product is ac- cepted for delivery without major cost, schedule, and risk consequences

Question 350

Reliability & Maintainability (R&M) Allocations Principle

Answer 350

Allocation and flow down of System Performance Specification (SPS) R&M requirements to lower levels of Entity Development Speci- fications (EDSs) should be based on: 1. A Reliability Allocations Block Diagram. 2. Supported by a documented R&M network configura- tion analysis including assumptions

Question 351

Parts Count Reliability Estimate Principle

Answer 351

Employ Parts Count Reliability Estimates as a “quick look” assessment of a System or Entity’s reliability based on generalized assumptions about component type categories, failure rates, quanti- ties, and quality factors operating in a Series Network Configuration

Question 352

System Reliability and FMEA-Based Architecting Principle

Answer 352

The robustness of a System or Entity’s Architecture (Principle 26.17) is deter- mined by its ability to detect , tolerate , and contain faults as a result of rigorous Failure Modes & Effects Analysis (FMEA) and corrective actions via its com- pensating and mitigating actions

Question 353

Maintainability versus Maintenance Principle

Answer 353

Maintainability is a system, product, or service quality factor attribute . Mainte- nance is a preventive or corrective action activity

Question 354

Levels of Maintenance Principle

Answer 354

Establish levels of maintenance at geo- graphical locations commensurate with the size of the User community, complex- ity of the repair, response times, and cost effectiveness

Question 355

FRACAS Principle

Answer 355

Institute a Failure Reporting and Correc- tive Action System (FRACAS) to track corrective maintenance actions, and data for analysis, defect elimination, and performance improve- ments

Question 356

Maintainability Estimates Principle

Answer 356

Maintainability estimates should be doc- umented and supported by assumptions and/or actual performance measurement data to substantiate the results; otherwise the estimate is nothing more than personal opinion

Question 357

Types of Maintenance Principle

Answer 357

Maintenance consists of two types of ac- tions: preventive and corrective mainte- nance

Question 358

Preventive Maintenance (PM) Principle

Answer 358

Preventive Maintenance (PM) consists of scheduled maintenance actions at regu- lar intervals to Inspect , replenish , and re- place (all italics) expendables and consumables such as lubri- cants, filters, and components to restore a System or Entity to perform as specified and return to active service

Question 359

Corrective Maintenance Principle

Answer 359

Corrective maintenance consists of scheduled or unscheduled maintenance actions required to restore or retrofit a system, product, or service to a specified condition and return it to active service

Question 360

Availability Principle

Answer 360

Availability is the probability that a sys- tem, product, or service will be available on-demand to perform a mission.

Question 361

Availability Metrics Principle

Answer 361

Availability is characterized by three types of metrics: 1. Operational Availability (A o ) 2. Achieved Availability (A a ) 3. Inherent Availability (A i )

Question 362

RMA Optimization Principle

Answer 362

Optimize RMA to achieve a System De- sign Solution that: 1. Complies with SPS requirements. 2. Minimizes the TCO. 3. Reduces risk to a level acceptable to the User

Question 363

Reliability-Centered Maintenance (RCM) Principle

Answer 363

Then objective of Reliability Centered Maintenance (RCM) is not to preserve the Equipment Element or to keep it from failing; it is to avoid or reduce the effects of failure consequences to ensure continuity of capabilities required to complete the mission without disruption