Test & Evaluation Flashcards
What’s the diffference between testing and evaluation?
Testing : obtains raw data, measures specific, individual performance factors
Evaluation: produces analyzed information, draws conclusions by looking at how the data interact
What types of integrated testing are there?
Developmental T&E
Operational T&E
Live Fire T&E
How many and which senior officials have specific guidance and approval responsibilities for all Major Defense Acquisition Programs (MDAPs)?
2; Director of Operational Test & Evaluation (DOT&E) and Director Development Test, Evaluation & Assessment (DTE&A)
What is the role of the DOT&E?
Issue DoD OT&E policy and procedures, review and analyze the results of OT&E, and provide independent assessments.
Provides budgetary and financial recommendations to the SECDEF
What’s the role of the DTE&A?
uses innovative and efficient DT&E strategies to ensure programs meet Warfighter/User needs.
Improves Defense Acquisition T&E workforce practices
Conducts Independent Technical Review Assessments (ITRA) and Milestone Assessments for major acquisition programs.
Who are the different customer groups that use T&E data to make acquisition decisions?
1) Service Operational Test Agencies (OTAs)
2) Product Support Managers and Logisticians
3) Designers and Engineers
4) Acquisition Managers
5) Director, Developmental Test, Evaluation and Assessment (DDTE&A)
6) End Users
How does Service OTAs use T&E data?
1) Test caps upgrades/alterations that changes sys performa after initial testing
2) Test alterations that pose a susbtantial risk of degrading fielded mil capabilities
3) Review and update planning docs, asses tech performance parameters, op effectiveness, and suitability
4) Construct and validate models and simulations
How does Product Support Managers and Logisticians use T&E data?
1) Understand system reliability
2) Address sustainment and provide feedback on sustainment solutions
3) Review and update planning documents and participate in T&E planning activities
How do Designers and Engineers use T&E data?
1) Evaluate system design and performance to see if design changes are needed
2) Develop, update, and validate models and sims
How do Acquisition Managers use T&E data?
1) Developing, producing, and/or fielding weapons systems and defense business systems
2) Determining if system performance meets thresholds
3) If technical changes needed will keep the program on track.
4) Determining whether the system has the potential to meet objectives
How do DDTE&A use T&E Data?
1) Assess adequacy of planned DT&E for selected MCA problems at Milestone B
2) Assess whether conducted DT&E was sufficient to support production and Initial OT&E at Milestone C for selected MCA programs
3) Participate in T&E planning activities
4) Providing independent program risk assessments to the milestone Decision authority for selected programs
How do DOT&E use T&E Data?
1) Ensuring adequacy of Op test planning of T&E on major weapon systems
2) Independently assessing systems under DOT&E oversight
How do end users use T&E Data?
1) Help us learn how the system performs under operation conditions
2) Use operatonal performance characteristics to improve tactics, techniques, and procedures and warfighting capability
What does DT&E measure?
Technical performance in a controlled environment. Helps to reduce risk throughout the acquisition cycle.
Managed by the PM
Conducted by both Gov’t and contractor
What does OT&E measure
Operational effectiveness, suitability, survivability in an operational environment.
Managed by Operational Test Agency
Conducted by actual gov’t users
What does LFT&E measure?
tests in both DT and OT, assessing the vulnerability/lethality of a system as it goes through its development.
Results reported to Congress prior to Full-Rate Production Decision
What are the 4 phases that have OT&E activities?
TMRR, EMD, and Production and Deployment
What OT&E activities are performed in TMRR?
Early Operational Assessments (EOA) - performed on prototypes/components
Assess progress in identifying operational design constraints, developing systems caps, and reducting program risks.
What OT&E activites are performed in EMD?
Operational Assesments (OAs). They can include dedicated early op testing, and developmental test results IOT assess potential to meet mission requirements, support Milestone C, and support LRIP decision.
What OT&E activities are performed in Production and Deployment?
Initial Operational Test and Evaluation (IOT&E) - performed after Milestone C and LRIP. Conducted by end users in an operational environment to support FRP decision review for entering into a full-rate production and deployment
What OT&E activities are performed in Operation and Sustainment?
Follow-on Operational Test and Evaluation (FOT&E) - conducted after the FRIP and may continue throughout life cycle to confirm correction of deficiencies, refine tactics and doctrine and evaluate major modification and/or changes to the system
What is TEMP?
The Test and Evaluation Master Plan (TEMP) provides an overall test management plan and framework within which detailed T&E plans are contained
Whare the minimum requirements provided in a TEMP
1) Resources and test support requirements
2) Developmental, operational, and life fire test objectives and metrics
3) Program schedule
4) Test phase objectives
5) Program decisions and supporting data requirements
6) Data collection requirements
7) Funding sources for all test resources
When are the four required TEMP tests required to be performed?
MSA, MSB, EMD, and MSC
What TEMP is approved at MSA?
Describes the concept for test and evaluation throughout program life cycle, starting with TMRR and continuing through EMD and PD.
What TEMP is approved at MSB?
Approves acquisition strategy (AS), acquisition program baselines (APB), Low-rate Initial Production (LRIP) quantities, EMD exit criteria, Completed TEMP.
What TEMP is approved at EMD?
T&E activities like integrated testing, robust developmental testing on components, subsystems, and fully integrated articles or systems - including LFT&E.
Lead OTA conducts OA IAW a test plan approved by DOT&E
T&E WIPT updates TEMP and refines test plans and strategies, determines testability of exit criteria, evaluates top level system digital models that will be used and refined throughout system development
What TEMP is approved at MSC?
Approves updated APB, updated TEEMP and exit criteria, confirms AS, reviews OA for acceptable operational effectiveness suitability and supportability.
