Module 5: Reviews Flashcards
Reviews - Intro (5.1)
Test Analysts must be active participants in the review process, providing…
When done properly, reviews can be the…
their unique views.
Single biggest, and most cost-effective, contributor to overall delivered quality.
Using Checklists in Reviews (5.2)
Checklist-based reviewing is the most…
Checklists are used during…
They can also help to…
common technique used by a Test Analyst when reviewing the
test basis.
Reviews to remind the participants to check specific points during the review.
De-personalize the review (e.g., “ This is the same checklist we use for every review. We are not targeting only your work product.”).
Using Checklists in Reviews (5.2)
Checklist-based reviewing can be…
For example, a generic checklist might verify…
A specific checklist for a requirements document might contain…
performed generically for all reviews or can focus on specific quality
characteristics, areas or types of documents.
The general document properties such as having a unique identifier, no references marked “to be determined”, proper formatting and similar conformance items.
Checks for the proper use of the terms “shall” and “should”, checks for the testability of each stated requirement, and so forth.
Using Checklists in Reviews (5.2)
The format of the requirements may also indicate…
A requirements document that is in narrative text format will have…
the type of checklist to be used.
Different review criteria than one that is based on diagrams.
Using Checklists in Reviews (5.2)
Checklists may also be oriented toward a particular aspect, such as:
- A programmer/architect skill set or a tester skill set - in the case of the Test Analyst, the tester skill set checklist would be the most appropriate
- A certain risk level (e.g., in safety-critical systems) - the checklists will typically include the specific information needed for the risk level
- A specific test technique - the checklist will focus on the information needed for a particular technique (e.g., rules to be represented in a decision table)
- A particular specification item, such as a requirement, use case or user story - these are discussed in the following sections and generally have a different focus than those used by a Technical Test Analyst for the review of code or architecture
Using Checklists in Reviews - Requirements Reviews (5.2.1)
The following items are an example of what a requirements-oriented checklist could include:
- Source of the requirement (e.g., person, department)
- Testability of each requirement
- Priority of each requirement
- Acceptance criteria for each requirement
- Availability of a use case calling structure, if applicable
- Unique identification of each requirement/use case/user story
- Versioning of each requirement/use case/user story
- Traceability for each requirement from business/marketing requirements
- Traceability between requirements and/or use cases (if applicable)
- Use of consistent terminology (e.g., uses a glossary)
Using Checklists in Reviews - Requirements Reviews (5.2.1)
It is important to remember that if a requirement is not testable…
For example, a requirement that states “The software should be very user friendly” is…
If, instead, the requirement says…
It is also an overarching requirement because…
In this case, this one requirement could…
Traceability from this requirement, or perhaps from the usability standards document, to the test cases, is also…
meaning that it is defined in such a way that the Test Analyst cannot determine how to test it, then there is a defect in that requirement.
Untestable. How can the Test Analyst determine if the software is user friendly, or even very user-friendly?
“The software must conform to the usability standards stated in the usability standards document, version xxx”, and if the usability standards document exists, then this is a testable requirement.
This one requirement applies to every item in the interface.
Easily spawn many individual test cases in anon-trivial application.
Critical because if the referenced usability specification should change, all the test cases will need to be reviewed and updated as needed.
Using Checklists in Reviews - Requirements Reviews (5.2.1)
A requirement is also untestable if the tester is…
Unable to determine whether the test passed or failed, or is unable to construct a test that can pass or fail. For example, “System shall be available 100% of the time, 24 hours per day, 7 days per week, 365 (or 366) days a year” is untestable.
Using Checklists in Reviews - Requirements Reviews (5.2.1)
A simple checklist for use case reviews may include the following questions:
- Is the basic behavior (path) clearly defined?
- Are all alternative behaviors (paths) identified, complete with error handling?
- Are the user interface messages defined?
- Is there only one basic behavior (there should be, otherwise there are multiple use cases)?
- Is each behavior testable?
Using Checklists in Reviews - User Story Reviews (5.2.2)
In Agile software development, requirements usually take the form of…
These stories represent…
Whereas a use case is…
User stories.
small units of demonstrable functionality.
A user transaction that traverses multiple areas of functionality, a user story is a more isolated feature and is generally scoped by the time it takes to develop it.
Using Checklists in Reviews - User Story Reviews (5.2.2)
A checklist for a user story could include the following:
- Is the story appropriate for the target iteration/sprint?
- Is the story written from the view of the person who is requesting it?
- Are the acceptance criteria defined and testable?
- Is the feature clearly defined and distinct?
- Is the story independent of any others?
- Is the story prioritized?
- Does the story follow the commonly used format: As a < type of user >, I want < some goal > so that < some reason > [Cohn04]
Using Checklists in Reviews - User Story Reviews (5.2.2)
If the story defines a new interface, then using a…
Generic story checklist (such as the one above) and a
detailed user interface checklist would be appropriate.
Using Checklists in Reviews - Tailoring Checklists (5.2.3)
A checklist can be tailored based on the following:
- Organization (e.g., considering company policies, standards, conventions, legal constraints)
- Project/development effort (e.g., focus, technical standards, risks)
- The type of work product being reviewed (e.g., code reviews might be tailored to specific programming languages)
- The risk level of the work product being reviewed
- Test techniques to be used
Using Checklists in Reviews - Tailoring Checklists (5.2.3)
Good checklists will:
Using a combination of checklists is a…
Using checklist-based reviewing with standard checklists such as…
Find problems and will also help to start discussions regarding other items that might not have been specifically referenced in the checklist.
Strong way to ensure a review achieves the highest quality work product.
Those referenced in the Foundation Level syllabus and developing organizationally specific checklists such as the ones shown above will help the Test Analyst be effective in reviews.
