Agile Testing Methods, Techniques, and Tools Flashcards
Test-Driven Development: What is it?
Test-driven development (TDD) is used to develop code guided by automated test cases. The process for test-driven development is:
- Add a test that captures the programmer’s concept of the desired functioning of a small piece of code
- Run the test, which should fail since the code doesn’t exist
- Write the code and run the test in a tight loop until the test passes
- Refactor the code after the test is passed, re-running the test to ensure it continues to pass against the refactored code
- Repeat this process for the next small piece of code, running the previous tests as well as the added tests
Test-Driven Development; Expand?
The tests written are primarily unit level and are code-focused, though tests may also be written at the integration or system levels.
Test-driven development gained its popularity through Extreme Programming [Beck02], but is also used in other Agile methodologies and sometimes in sequential lifecycles.
Test-Driven Development; It helps developers focus on?
- Focus on clearly-defined expected results. The tests are automated and are used in continuous integration.
Acceptance Test-Driven Development: Define?
- Defines acceptance criteria and tests during the creation of user stories.
- Acceptance test-driven development is a collaborative approach that allows every stakeholder to understand how the software component has to behave and what the developers, testers, and business representatives need to ensure this behavior.
Acceptance Test-Driven Development: Expand?
- Acceptance test-driven development creates reusable tests for regression testing.
- Specific tools support creation and execution of such tests, often within the continuous integration process.
- These tools can connect to data and service layers of the application, which allows tests to be executed at the system or acceptance level.
- Acceptance test-driven development allows quick resolution of defects and validation of feature behavior.
- It helps determine if the acceptance criteria are met for the feature.
Behavior-Driven Development: Define?
- Allows a developer to focus on testing the code based on the expected behavior of the software. Because the tests are based on the exhibited behavior of the software, the tests are generally easier for other team members and stakeholders to understand.
Behavior-Driven Development: Behavior-driven development frameworks?
- Can be used to define acceptance criteria based on the given/when/then format:
Given some initial context,
When an event occurs,
Then ensure some outcomes.
Behavior-Driven Development: How does it assist developers?
- The behavior-driven development framework generates code that can be used by developers to create test cases.
- Behavior-driven development helps the developer collaborate with other stakeholders, including testers, to define accurate unit tests focused on business needs.
Behavior-Driven Development: How does it assist developers?
- The behavior-driven development framework generates code that can be used by developers to create test cases.
- Behavior-driven development helps the developer collaborate with other stakeholders, including testers, to define accurate unit tests focused on business needs.
The Test Pyramid: Define?
- A software system may be tested at different levels. Typical test levels are, from the base of the pyramid to the top, unit, integration, system, and acceptance.
- The test pyramid emphasizes having a large number of tests at the lower levels (bottom of the pyramid) and, as development moves to the upper levels, the number of tests decreases (top of the pyramid).
- Usually unit and integration level tests are automated and are created using API-based tools. At the system and acceptance levels, the automated tests are created using GUI-based tools. The test pyramid concept is based on the testing principle of early QA and testing (i.e., eliminating defects as early as possible in the lifecycle).
Testing Quadrants, Test Levels, and Testing Types: Define?
- Testing quadrants align the test levels with the appropriate test types in the Agile methodology.
- The testing quadrants model, and its variants, helps to ensure that all important test types and test levels are included in the development lifecycle.
- This model also provides a way to differentiate and describe the types of tests to all stakeholders, including developers, testers, and business representatives.
Testing Quadrants, Test Levels, and Testing Types: The four quadrants are as follows:
* Quadrant Q1 is unit level, technology facing, and supports the developers. This quadrant contains unit tests. These tests should be automated and included in the continuous integration process.
- Quadrant Q2 is system level, business facing, and confirms product behavior. This quadrant contains functional tests, examples, story tests, user experience prototypes, and simulations. These tests check the acceptance criteria and can be manual or automated. They are often created during the user story development and thus improve the quality of the stories. They are useful when creating automated regression test suites.
- Quadrant Q3 is system or user acceptance level, business facing, and contains tests that critique the product, using realistic scenarios and data. This quadrant contains exploratory testing, scenarios, process flows, usability testing, user acceptance testing, alpha testing, and beta testing. These tests are often manual and are user-oriented.
- Quadrant Q4 is system or operational acceptance level, technology facing, and contains tests that critique the product. This quadrant contains performance, load, stress, and scalability tests, security tests, maintainability, memory management, compatibility and interoperability, data migration, infrastructure, and recovery testing. These tests are often automated.
Testing Quadrants, Test Levels, and Testing Types: The four quadrants are as follows:
* Quadrant Q1 is unit level, technology facing, and supports the developers. This quadrant contains unit tests. These tests should be automated and included in the continuous integration process.
- Quadrant Q2 is system level, business facing, and confirms product behavior. This quadrant contains functional tests, examples, story tests, user experience prototypes, and simulations. These tests check the acceptance criteria and can be manual or automated. They are often created during the user story development and thus improve the quality of the stories. They are useful when creating automated regression test suites.
- Quadrant Q3 is system or user acceptance level, business facing, and contains tests that critique the product, using realistic scenarios and data. This quadrant contains exploratory testing, scenarios, process flows, usability testing, user acceptance testing, alpha testing, and beta testing. These tests are often manual and are user-oriented.
- Quadrant Q4 is system or operational acceptance level, technology facing, and contains tests that critique the product. This quadrant contains performance, load, stress, and scalability tests, security tests, maintainability, memory management, compatibility and interoperability, data migration, infrastructure, and recovery testing. These tests are often automated.
Testing Quadrants, Test Levels, and Testing Types: Apply to which test type, static or dynamic?
The testing quadrants apply to dynamic testing rather than static testing.
The Role of a Tester: Teamwork :The following are organizational and behavioral best practices in Scrum teams:
- Cross-functional: Each team member brings a different set of skills to the team. The team works together on test strategy, test planning, test specification, test execution, test evaluation, and test results reporting.
- Self-organizing: The team may consist only of developers, but, as noted in Section 2.1.5, ideally there would be one or more testers.
- Co-located: Testers sit together with the developers and the product owner.
- Collaborative: Testers collaborate with their team members, other teams, the stakeholders, the product owner, and the Scrum Master.
- Empowered: Technical decisions regarding design and testing are made by the team as a whole (developers, testers, and Scrum Master), in collaboration with the product owner and other teams if needed.
- Committed: The tester is committed to question and evaluate the product’s behavior and characteristics with respect to the expectations and needs of the customers and users.
- Transparent: Development and testing progress is visible on the Agile task board (see Section 2.2.1).
- Credible: The tester must ensure the credibility of the strategy for testing, its implementation, and execution, otherwise the stakeholders will not trust the test results. This is often done by providing information to the stakeholders about the testing process.
- Open to feedback: Feedback is an important aspect of being successful in any project, especially in Agile projects. Retrospectives allow teams to learn from successes and from failures.
- Resilient: Testing must be able to respond to change, like all other activities in Agile projects.
The Role of a Tester: Sprint Zero:
- Sprint zero is the first iteration of the project where many preparation activities take place.
- Sprint zero sets the direction for what testing needs to achieve and how testing needs to achieve it throughout the sprints.
The Role of a Tester: Sprint Zero: The tester collaborates with the team on the following activities during this iteration:
- Identify the scope of the project (i.e., the product backlog)
- Create an initial system architecture and high-level prototypes
- Plan, acquire, and install needed tools (e.g., for test management, defect management, test automation, and continuous integration)
- Create an initial test strategy for all test levels, addressing (among other topics) test scope, technical risks, test types (see Section 3.1.3), and coverage goals
- Perform an initial quality risk analysis (see Section 3.2.1)
- Define test metrics to measure the test process, the progress of testing in the project, and product quality
- Specify the definition of “done”
- Create the task board (see Section 2.2.1)
- Define when to continue or stop testing before delivering the system to the customer
The Role of a Tester: Integration
- In Agile projects, the objective is to deliver customer value on a continuous basis (preferably in every sprint).
- To enable this, the integration strategy should consider both design and testing.
- To enable a continuous testing strategy for the delivered functionality and characteristics, it is important to identify all dependencies between underlying functions and features.
The Role of a Tester: Test Planning
- Since testing is fully integrated into the Agile team, test planning should start during the release planning session and be updated during each sprint.
- Sprint planning results in a set of tasks to put on the task board, where each task should have a length of one or two days of work. In addition, any testing issues should be tracked to keep a steady flow of testing.
The Role of a Tester: Agile Testing Practices: Many practices may be useful for testers in a scrum team, some of which include:
- Pairing: Two team members (e.g., a tester and a developer, two testers, or a tester and a product owner) sit together at one workstation to perform a testing or other sprint task.
- Incremental test design: Test cases and charters are gradually built from user stories and other test bases, starting with simple tests and moving toward more complex ones.
- Mind mapping: Mind mapping is a useful tool when testing [Crispin08]. For example, testers can use mind mapping to identify which test sessions to perform, to show test strategies, and to describe test data.
How do you Assess Quality Risks in Agile Projects?:
- Risk identification, analysis, and risk mitigation strategies can be used by the testers in Agile teams to help determine an acceptable number of test cases to execute
Assessing Quality Risks in Agile Projects: Define risk:
- Risk is the possibility of a negative or undesirable outcome or event.
- The level of risk is found by assessing the likelihood of occurrence of the risk and the impact of the risk.
- When the primary effect of the potential problem is on product quality, potential problems are referred to as quality risks or product risks.
Assessing Quality Risks in Agile Projects: What are potential problems referred to When the primary effect of the potential problem is on project success?
- project risks or planning risks
Assessing Quality Risks in Agile Projects: In Agile projects, quality risk analysis takes place at two places:
- Release planning: business representatives who know the features in the release provide a high-level overview of the risks, and the whole team, including the tester(s), may assist in the risk identification and assessment.
- Iteration planning: the whole team identifies and assesses the quality risks.
Assessing Quality Risks in Agile Projects: What are Higher and Lower Risks?:
Tasks associated with higher risks should start earlier and involve more testing effort. Tasks associated with lower risks should start later and involve less testing effort.