EXAM2 (Develop the TEMP and Test Planning, Best Manufacturing Practices, Scheduling, Request for Proposal, Work Breakdown Structure, Leadership and Ethics, Systems Engineering in Acquisition, Systems Engineering Process Model, Technical Management Processes, Risk Management) Flashcards
Understand the purpose of T&E and why we test
The purpose of test and evaluation (T&E) in a development and acquisition program is to help reduce or manage risks in:
Development
Acquisition
Utilization
We test to measure specific individual performance factors.
Recognize the customers of T&E
Developers / Acquisition Managers
Designers and Engineers
End-Users
Recognize the various test agencies
Director, Operational Test and Evaluation (DOT&E)
Army Test and Evaluation Command (ATEC)
Army Evaluation Center (AEC)
Operational Test Command (OTC)
Recognize the types of DT&E and OT&E
Types of OT&E (Operational Testing)
Early Operational Assessment (EOA)
Operational Assessment (OA)
Initial Operational Test and Evaluation (IOT&E)
Follow-on Operational Test and Evaluation (FOT&E)
Types of DT&E (Developmental Test and Evaluation)
Research Effort/Test
Technical Feasibility Test (TFT)
Engineering Development Test (EDT)
Production Qualification Test (PQT)
Production Verification Test (PVT)
First Article Test (FAT)
Production Acceptance Test and Evaluation (PAT&E)
Define test and evaluation
Test: Program, procedure or process to obtain, verify, or provide data to determine the degree to which a system or systems meet, exceed or fail to meet stated objectives
Obtains raw data (1’s and 0’s)
Measures specific, individual performance factors
Is resource intensive
Evaluation: Independent process to review, analyze, and assess data obtained from testing or other sources
Produces analyzed information from test data, modeling and simulation, or other sources
Draws conclusions by looking at how the factors interact
Is intellectually intensive
Define operational effectiveness and operational suitability
Operational Effectiveness: How well it does the job?
Operational Suitability: How well it can be supported?
Understand the purpose of Live Fire Testing and when it applies
Purpose of Live Fire Testing:
Determines crew survivability and/or system vulnerability
Confirms lethality of munitions/missiles against intended target set
Mandated by FY87 Defense Authorization Bill
Applies to:
ACAT I and II major vehicles and weapons platforms
Conventional weapons that provide some degree of protection
Major munitions greater than 1M rounds
Note - Can be waived - LFT can be waived with justification
Prior to program initiation
Impractical or too expense
Must still accomplish an alternative
To determine vulnerability
Survivability, and lethality
Define Interoperability and JITC’s (Joint Interoperability Test Command) role
Interoperability - The ability of systems, units, or forces to provide data, information, materiel, and services to and accept the same from other systems, units, or forces and to use the data, information, materiel, and services so exchanged to enable them to operate effectively together.
JITC issues a Interoperability Test Certification indicating: System has successfully passed interoperability testing and has met the NR-KPP
Testing is performed on Low Rate Initial Production (LRIP) production representative systems
Recognize the importance of T&E to milestone decisions
Before Milestone A: Approved T&E Strategy
Before Milestone B: Approved TEMP; Identification of LRIP quantities – Only 10% max without waiver; DT&E Report
Before Milestone C: Updated TEMP; Final DT&E Report; Operational Assessment
Define Verification and Validation
Verification that the system meets the performance specifications accomplished through Developmental Testing. (Did you build it right?)
Validation that the system fulfills the user’s required capabilities accomplished through Operational Testing. (Did you build the right thing?)
Define concurrent testing, combined testing, and integrated testing
Concurrent Testing- Circumstances in which developmental testing and operational testing take place at the same time as two parallel, but separate and distinct, activities.
Combined Testing- Single test program conducted to support developmental test and operational test objectives.
Integrated Testing- The collaborative planning and collaborative execution of test phases and events to provide shared data in support of independent analysis, evaluation, and reporting by all stakeholders, particularly the development (both contractor and government) and operational test and evaluation communities.
Identify the forum in which the operational tester brings together all the representatives of agencies involved in the Operational or Force Development Test to determine overall readiness for test
T&E WIPT
Understand the purpose of the T&E WIPT and when it is formed
The T&E WIPT is a defined forum serving the PM, and PMO, as the test and evaluation subject matter experts responsible for supporting the PM, and other program WIPTs, on all aspects of a program’s test and evaluation effort. This effort includes T&E program strategy, design, development, oversight, and the analysis, assessment and reporting of test results.
The T&E WIPT should be established and chartered as early as possible during the Material Solutions Analysis phase so that it is involved in program strategy discussions and plans.
Recognize why independent agencies conduct and oversee OT&E
Recognize why independent agencies conduct and oversee OT&E
Recognize T&E planning and its function
As the essential feedback mechanism for the Systems Engineering Process
Understand that the Test and Evaluation Master Plan (TEMP)
Must be updated periodically to ensure that it stays current and integrates the various disciplines as the program evolves through its life cycle
Understand that Developmental Test and Evaluation (DT&E) is
Essential in determining a system’s readiness for Initial Operational Test and Evaluation (IOT&E)
Understand that Operational Test and Evaluation (OT&E)
Helps determine system operational suitability and effectiveness, and it addresses Critical Operational Issues (COIs) that are defined by the User
The Logistics Demonstration is normally conducted
On those tasks performed at field level and some sustainment level by User representatives
Understand that the Test and Evaluation Master Plan (TEMP)
Is prepared by the Program Manager (PM) in conjunction with principal Test Integration Working Group (TIWG) members
NDIs (Non-developmental item) are
Any previously developed items regardless of the source of development-used exclusively for governmental purposes by federal, state, local, or allied governments
The DoD Acquisition Guidebook states that
Producibility is the design effort priority.
Manufacturing integrated into the design process results in
Reduced program risk
Factors leading to increased cost and production delays
Changing design requirements
Late design releases
Lack of production planning
Unstable funding
!The three steps in Production and Manufacturing Integration include!
!Influence the Design Process
Prepare for Production
Execute the Manufacturing Plan!
A priority for manufacturing is
To develop a stable production process
Process Proofing involves
Proving that the design can actually produce a product that meets customer functional requirements in a realistic, “factory floor” type situation
!The 5 Ms of Manufacturing!
!Material
Method
Machinery
Manpower
Measurement!
!Five top-level design goals for producible products!
!Design for Ease of Fabrication
Design for Ease of Assemble
Design for Multiuse
Minimize Number of Parts
Maximize Number of Common Parts!
!The cost of quality can be broken down into three separate areas!
!Prevention
Appraisal
Correcting Failures!
Contractor must have
A quality management process, but the contractor defines the process (must meet Government objectives)
Learning curve theory states
That as the production of an item doubles, the man-hours needed to produce that item decrease at a fixed rate
!Two main principles of Lean Manufacturing!
!Minimization of Waste
Responsive to Change!
The five major causes of production problems
Unstable Rates and Quantities
Design Instability
Undue emphasis on Schedule
Inadequate Configuration Management System
Inattention to Environmental Impacts
The Integrated Master Plan (IMP) is
An event-based plan that identifies the work that must be accomplished to complete the key program activities and events.
An effective schedule correctly balances the tradeoffs between cost, schedule and performance while taking into account risk and program impact
The 5 step scheduling process includes:
Identify the tasks
Sequence the tasks
Estimate the task duration
Construct the schedule
Implement the schedule
Scheduling is
A listing of activities and events organized by time.
Milestones
Points in time where decision makers evaluate the status of the program and determine if the program should proceed to the next phase
What phases are?
Phases: logical means of translating broad capability needs into well defined system specific requirement ultimately into operationally effective, suitable, and survivable systems.
Types of Schedules
Gantt
Milestone
Network
Production
Milestone Charts
Depicts information about significant project events, focusing on specific points in time
The critical path is
The longest path through the network (the path with the greatest sum of task time duration). Any delay in tasks along the critical path will result in a delay in the overall project time
Time constraints are
Limitations imposed by the length of time, or dates, associated with the schedule
Lead is
Allows an activity to start before the predecessor activity is complete.
A lead allows an acceleration of a successor activity
Lag is
A lag directs a delay in a successor activity
Intermediate schedules tie (or roll up)
Is the detailed schedules to the key events and milestones in the master schedule(s), and provide traceability between the two levels
Given task durations, minimum durations, and cost to reduce the duration, be able to evaluate which tasks should be reduced in order to crash the schedule
Given task durations, minimum durations, and cost to reduce the duration, be able to evaluate which tasks should be reduced in order to crash the schedule
Recognize the roles of the Program Manager and the Contracting Officer
Program Manager: Responsible for bringing the program in on time and within budget
Contracting Officer: Responsible for advising the PM and ensuring that all acquisition laws and regulations are followed
Understand the difference between fixed-price and cost-reimbursement contracts
Fixed-Price: Contractor promises to deliver on time and to meet contract specifications for a negotiated price
Cost-Reimbursement: Contractor promises to put forth their best effort
!Understand that contracts greater than…!
!$3,000 but less than or equal to $150,000 are set-aside exclusively for small businesses, if at least two capable small businesses can be expected to submit offers!
Understand that DoD guidance allows contractors to..
Use the quality assurance process of their choice, as long as it meets program objectives
Identify the three generic classifications of DoD software-intensive systems
Weapons Systems
Command, Control, Communications, Computers and Intelligence, Surveillance and Reconnaissance (C4ISR)
Defense Business Systems (DBS)
Recognize software intensive system acquisition best practices
Identify and manage risk continuously throughout the life of the system
Estimate cost and schedule empirically
Use metrics to monitor risk, identify problems, and base decisions
Track earned value
Establish quality targets and track defects against those targets
Treat people as your most important resource
Implement a sound configuration management process
Manage and trace requirements to lowest level
Use system-based software design to document/evaluate process
Ensure data and database interoperability
Identify the four categories of data rights
Unlimited Rights
Limited Rights
Government Purposed Rights
Restricted Rights
Define Work Breakdown Structure
Product-oriented family tree
Composed of hardware, software, services, data, and facilities
Created from systems engineering process throughout the R&D efforts
Allows acquisition personnel to manage risk at levels lower than system level
Displays and defines the product to be developed and produced
The WBS is an output of the Systems Engineering Process
Understand that there are 11 WBS categories and 10 common elements (Level 2)
WBS Categories:
Aircraft
Electronic/Automated Software
Missile
Ordnance
Ship
Space
Surface Vehicle
Unmanned Air Vehicle
Unmanned Maritime Systems
Launch Vehicle Systems
Automated Information Systems
Common Elements:
Integration, assembly, test, and checkout efforts
Systems engineering/program management
System test and evaluation
Training
Data
Peculiar support equipment
Common support equipment
Operational/site activation
Industrial facilities (construction, conversion or expansion)
Initial spares and repair parts
Understand that the WBS is used for
Planning logistics
Establishing schedules
Tracking cost
Predicting variations
A Functional WBS answers the question
What it must do?