All Flashcards
(1) What are the two types of coverage in the white-box test technique and (2) how is coverage represented?
(1) Statement Coverage and Decision Coverage
(2) It is the “percentage” of coverage items determined or expressed in the test suite.
What are the responsibilities of the test manager?
- Develop/review the test policy and test strategy
- Write and update the test plan
- Select test approaches
- Estimate testing time, effort, and cost
- Acquiring resources
- Define test levels and cycles
- Plan for defect management
- Coordinate the test plan with others
- Share testing perspectives with other project activities
- Initiate the analysis, design, implementation, and execution of tests
- Monitor test progress and results
- Check status of exit criteria (definition of done)
- Facilitate test completion activities
- Prepare and deliver test progress reports and test summary reports
- Adapt the plan based on results
- Perform test control
- Support setup of defect management system
- Introduce suitable metrics
- Support selection and implementation of tools
- Promote and advocate testers, test team, and test profession
- Develop skills and careers of testers (training plans, performance evaluations, coaching, etc.)
What are the responsibilities of the tester?
What are the types of “defined behaviors” in the use case test technique?
- Basic (1 test case)
- Exceptional or Alternative (potentially multiple test cases)
- Error handling (potentially multiple test cases)
(1) What are the three experience-based test techniques and (2) which are black-box vs. white box?
(1) Error Guessing, Exploratory, and Checklist-based
(2) They can all be used in black OR white-box testing
What are the three triggers for maintenance testing?
Modification, migration, and retirement.
Note: the decision to maintain a system is not a trigger for maintenance testing.
What is another name for exploratory testing?
Session based testing
Note exploratory tests are “designed, executed, logged, and evaluated dynamically during execution.”
What is a test charter?
Documentation of test activities in session-based exploratory testing. Example: a general plan of approach for test cases not documented in advance.
What are the types of reviewer roles?
- Author
- Management/Review Manager
- Facilitator
- Review Leader
- Reviewer
- Scribe/Recorder
Note: Roles can overlap. For example, the Facilitator might also be the Review Leader.
What are the review types (and sub-types)?
Formal:
Walkthrough (led by author)
Technical Review (technical reviewers)
Inspection (rules and checklists; most formal)
Informal (i.e. undocumented)
Note: A single work product might have more than one type of review performed on it.
What is the key attribute of a formal review type?
Formal review will always have a scribe.
What is the main reason for performing a review?
To uncover defects
What are the review techniques?
- Ad hoc (informal, free-for-all)
- Checklist-based
- Scenario-based
- Perspective-based (e.g. stakeholder viewpoints)
- Role-based (end user types and organization roles)
From section 3.2
What are the seven principles of testing?
- Testing shows the presence of defects, not their absence
- Exhaustive testing is a no no
- Early testing; Start with requirements
- Defects can cluster
- Pesticide Paradox
- Testing is context dependent
- Absence of errors is a fallacy
What is test analysis?
A step in the testing process wherein test conditions are identified in the test basis.
What is test control?
A step in the testing process that deals with creating and applying actions to get a test project back on track.
What is test monitoring?
A step in the testing process that involves…
- Checking the status of testing activities
- Identifying any variances from the plan
- Reporting status to stakeholders
What are the seven steps of the testing process?
Test Planning
Test Monitoring/Control
Test Analysis
Test Design
Test Implementation
Test Execution
Test Completion
In which test levels can functional, non-functional, and white-box testing occur?
In any of them (CISA)
In which test levels should functional testing occur?
In all of them (CISA)
In which test levels should non-functional testing occur?
In all of them (CISA)
What is another name for white-box testing?
Structural Testing
What are the 2 types of change-related testing?
Confirmation Testing
Regression Testing
In which test levels (CISA) should change-related testing be performed?
In all of them.
What is the definition of a test type?
A group of test activities based on specific test objectives aimed at specific characteristics of a component or system.
The test types are: Functional, Non-Functional, White Box, and Changed Related.
What are the 4 test types?
Functional
Non-Functional
White-Box
Change-Related
True or False: Some test levels (CISA) will not have any functional, non-functional, or white-box testing performed in them.
False! All test levels might have some form of functional, non-functional, and white-box testing.
True or False: 100 percent statement coverage guarantees 100 percent decision coverage.
False. 100% decision coverage guarantees 100% statement coverage, but not the other way around.
What are the types of work products?
Testware:
- Test Plan
- Test Progress Report
- Test Summary Report
- Test Charter
- Test Procedures and Cases
- Test Suites
- Test Data
- Test Environment
- Test Execution Schedule
- Defect Reports
- Automated Test Scripts
What is “the work product to be tested”?
It is the “test object” or “test item”.
What is a test condition?
An aspect of the test basis that is relevant to the test objectives (i.e. that needs to be tested)
Give examples of typical exit criteria.
- Completed the planned test(s)
- Defined level of coverage achieved
- # unresolved defects is under agreed limit
- # estimated remaining defects is sufficiently low
- levels of quality (from evaluation of reliability, performance, efficiency, usability, security, etc.) are sufficient.
Give examples of typical entry criteria.
Availability of…
- Requirements, user stories, and/or models
- Tests meeting exit criteria for prior test levels (CISA)
- Test environment
- Necessary test tools
- Test data
From pg. 357 in pdf
What are the 7 types of test strategies?
- Analytical (“analysis”)
- Model-based (“model”)
- Methodical (“systematic”)
- Process or standard-compliant
- Directed or consultative (“experts”)
- Regression Averse
- Reactive (“exploratory”)
Note: You can use more that one strategy in a test approach.
Unofficial:
What is the hierarchy of test documentation within an organization?
- Test Policy (organization)
- Test Strategy (organization -> multiple projects)
- Test Approach (project)
- Test Plan (features)
What is a test technique?
A procedure used to derive/make/select test cases.
What are all the test techniques and their categories?
Categories: Black Box, White Box, and Experience Based
Black Box:
- Equivalence Partitioning -> Boundary Value Analysis
- Decision Tables/Trees
- State Transition Diagrams/Tables
- Use Case
White Box:
- Statement Coverage
- Decision Coverage
Experience Based:
- Error Guessing
- Exploratory
- Checklist-based testing
What is a test level?
An instance of a test process (i.e. the seven steps of the test process).
Each test level uses all of the steps in the testing process.
What are the 4 test levels?
- Component (unit)
- Integration
- System
- Acceptance
What is beta testing?
Operational acceptance testing (simulated or actual) conducted at an external site, by roles outside the development organization (e.g. actual customers).
Note: Alpha testing is the same, but performed inside the developer’s test environment.
What is a test procedure?
A sequence of test cases in execution order, and any associated actions that may be required to set up the initial preconditions and any wrap up activities post execution.
The closest thing we have to this at TAB right now is the Xray Test Execution.
What is a test oracle?
A source to determine expected results to compare with the actual result of the system under test.
In TAB’s case, the test oracle is accomplished by the expected vs. actual results functionality in Xray.
What are the 5 steps in the general review process?
- Planning
- Initiate Review
- Individual Review
- Issue Communication & Analysis
- Fixing and Reporting
What is static analysis?
Evaluation of a component/system, without executing it, based on its form, structure, content, or documentation.
It is performed by developers.
What is Test Design?
One of the 7 steps in the testing process.
In this step, you…
- Design and prioritize test cases
- Identify test data
- Design the test environment.
- Set bi-directional traceability between test basis, conditions, and cases.
What is Test Implementation?
One of the 7 steps in the testing process.
In this step, you…
- Implement and prioritize test cases
- Develop and prioritize test procedures
- Organize test procedures into test suites
- Organize test suites into an execution schedule
- Build and verify that the test environment has been set up correctly
- Update and verify bi-directional traceability
Note: there may be overlap between this step and the Test Design step.
What is Test Execution?
One of the 7 steps in the testing process.
In this step, you…
- Record versions of items, objects, tools, and testware
- Execute manual and automated tests
- Compare actual and expected results
- Analyze causes of anomalies
- Log outcomes (pass, fail, blocked)
- Report defects for observed failures
- Re-execution of failed tests
- Verify/update bi-directional traceability between test basis, conditions, cases, procedures, and results.
What is Test Completion?
One of the 7 steps in the testing process.
In this step, you…
- Analyze lessons learned
- Turn over planned deliverables
- Close defect reports
- Create test summary report
- Finalize/archive testware, environment, and infrastructure for later use
- Retrospect and improve test process maturity
What is Impact Analysis?
The identification of all work products affected by a change, including an estimate of the resources needed to accomplish the change.
It can be done before a change is made, to help decide if the change should be made, or after/concurrent with the change to identify the test cases that need to be written/updated and run.
[Reviews] What is the responsibility of the Author?
- Create the work product under review
- Fix defects in the work product under review
[Reviews] What is the responsibility of the Review Manager?
- Review planning
- Decide on planning and execution of reviews
- Assign budget, staff, and time
- Monitor ongoing cost effectiveness
- Execute control decisions
[Reviews] What is the responsibility of the Facilitator/Moderator?
- Leads the review
- Ensure effective running of the review meetings.
- Mediate between different viewpoints
[Reviews] What is the responsibility of the Review Leader?
- Takes overall responsibility for the review
- Decides who will be involved, when and where
[Reviews] What is the responsibility of the Reviewers?
- Review the work product under review
- Identify potential defects
- Present different perspectives
[Reviews] What is the responsibility of the Scribe/Recorder?
- Collate potential defects found during the “individual review” step
- Record new potential defects, open points, and decisions
What are the contents of a defect report?
- An identifier (e.g. jira key)
- Title/Summary
- Date
- Author
- Test Item/Object
- Test Environment
- SDLC phase the defect was observed in
- Defect description
- Steps to reproduce
- Expected vs. actual results
- Scope or degree of impact (severity) to the stakeholder
- Urgency/priority to fix
- Defect state
- Conclusions, recommendations, and approvals
- References, including the test case that revealed the problem
