ISTQB Chapter 2 - Testing Throughout the Software Development Lifecycle - Keywords Flashcards
Acceptance Testing
Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.
Acceptance testing focuses on validation and on demonstrating readiness for deployment, which means that the system fulfills the user’s business needs. Ideally, acceptance testing should be performed by the intended users. The main forms of acceptance testing are: user acceptance testing (UAT), operational acceptance testing, contractual and regulatory acceptance testing, alpha testing and beta testing.
Black-Box (Testing / Technique / Test Design Technique / Test Technique)
Procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional, of a component or system without reference to its internal structure.
Testing the software without knowledge of its internal structure or implementation details, focusing on inputs, outputs, and behavior.
It is specification-based and derives tests from documentation external
to the test object. The main objective of black-box testing is checking the system’s behavior against its specifications.
(Also known as specification-based techniques) are based on an analysis of
the specified behavior of the test object without reference to its internal structure. Therefore, the test cases are independent of how the software is implemented. Consequently, if the implementation changes, but the required behavior stays the same, then the test cases are still useful.
Component Integration Testing
Testing performed to expose defects in the interfaces and interactions between integrated components.
Component integration testing (also known as unit integration testing) focuses on testing the interfaces and interactions between components. Component integration testing is heavily dependent on the integration strategy approaches like bottom-up, top-down or big-bang.
Component Testing
The testing of individual software components.
- Component testing (also known as unit testing) focuses on testing components in isolation. It often requires specific support, such as test harnesses or unit test frameworks. Component testing is normally performed by developers in their development environments.
Confirmation Testing
Testing that runs test cases that failed the last time they were run, in order to verify the success of corrective actions.
Confirmation testing confirms that an original defect has been successfully fixed. Depending on the risk, one can test the fixed version of the software in several ways, including:
* executing all test cases that previously have failed due to the defect, or, also by
* adding new tests to cover any changes that were needed to fix the defect
However, when time or money is short when fixing defects, confirmation testing might be restricted to simply exercising the steps that should reproduce the failure caused by the defect and checking that the failure does not occur.
Functional Testing
Testing based on an analysis of the specification of the functionality of a component or system.
Functional testing evaluates the functions that a component or system should perform. The functions are “what” the test object should do. The main objective of functional testing is checking the functional completeness, functional correctness and functional appropriateness.
Integration Testing
Testing performed to expose defects in the interfaces and in the interactions between integrated components or systems.
Maintenance Testing
Testing the changes to an operational system or the impact of a changed environment to an operational system.
There are different categories of maintenance, it can be corrective, adaptive to changes in the environment or improve performance or maintainability (see ISO/IEC 14764 for details), so maintenance can involve planned releases/deployments and unplanned releases/deployments (hot fixes). Impact analysis may be done before a change is made, to help decide if the change should be made, based on the potential consequences in other areas of the system. Testing the changes to a system in production includes both evaluating the success of the implementation of the change and the checking for possible regressions in parts of the system that remain unchanged (which is usually most of the system).
The scope of maintenance testing typically depends on:
* The degree of risk of the change
* The size of the existing system
* The size of the change
The triggers for maintenance and maintenance testing can be classified as follows:
* Modifications, such as planned enhancements (i.e., release-based), corrective changes or hot fixes.
* Upgrades or migrations of the operational environment, such as from one platform to another, which can require tests associated with the new environment as well as of the changed software, or tests of data conversion when data from another application is migrated into the system being maintained.
* Retirement, such as when an application reaches the end of its life. When a system is retired, this can require testing of data archiving if long data-retention periods are required. Testing of restore and retrieval procedures after archiving may also be needed in the event that certain data is required during the archiving period.
Non-Functional Testing
Testing the attributes of a component or system that do not relate to functionality, e.g., reliability, efficiency, usability, maintainability and portability.
Non-functional testing evaluates attributes other than functional characteristics of a component or system. Non-functional testing is the testing of “how well the system behaves”. The main objective of nonfunctional testing is checking the non-functional software quality characteristics. The ISO/IEC 25010 standard provides the following classification of the non-functional software quality characteristics:
* Performance efficiency
* Compatibility
* Usability
* Reliability
* Security
* Maintainability
* Portability
It is sometimes appropriate for non-functional testing to start early in the life cycle (e.g., as part of reviews and component testing or system testing). Many non-functional tests are derived from functional tests as they use the same functional tests, but check that while performing the function, a non-functional constraint is satisfied (e.g., checking that a function performs within a specified time, or a function can be ported to a new platform). The late discovery of non-functional defects can pose a serious threat to the success of a project. Non-functional testing sometimes needs a very specific test environment, such as a usability lab for usability testing.
Regression Testing
Testing of a previously tested program following modification to ensure that defects have not been introduced or uncovered in unchanged areas of the software, as a result of the changes made. It is performed when the software or its environment is changed.
Regression testing confirms that no adverse consequences have been caused by a change, including a fix that has already been confirmation tested. These adverse consequences could affect the same component where the change was made, other components in the same system, or even other connected systems. Regression testing may not be restricted to the test object itself but can also be related to the environment. It is advisable first to perform an impact analysis to optimize the extent of the regression testing. Impact analysis shows which parts of the software could be affected. Regression test suites are run many times and generally the number of regression test cases will increase with each iteration or release, so regression testing is a strong candidate for automation. Automation of these tests should start early in the project. Where CI is used, such as in DevOps (see section 2.1.4), it is good practice to also include automated regression tests. Depending on the situation, this may include regression tests on different levels.
Shift-Left
The principle of early testing (see section 1.3) is sometimes referred to as shift-left because it is an approach where testing is performed earlier in the SDLC. Shift-left normally suggests that testing should be done earlier (e.g., not waiting for code to be implemented or for components to be integrated), but it does not mean that testing later in the SDLC should be neglected.
There are some good practices that illustrate how to achieve a “shift-left” in testing, which include:
* Reviewing the specification from the perspective of testing. These review activities on specifications often find potential defects, such as ambiguities, incompleteness, and inconsistencies
* Writing test cases before the code is written and have the code run in a test harness during code implementation
* Using CI and even better CD as it comes with fast feedback and automated component tests to accompany source code when it is submitted to the code repository
* Completing static analysis of source code prior to dynamic testing, or as part of an automated process
* Performing non-functional testing starting at the component test level, where possible. This is a form of shift-left as these non-functional test types tend to be performed later in the SDLC when acomplete system and a representative test environment are available
A shift-left approach might result in extra training, effort and/or costs earlier in the process but is expected to save efforts and/or costs later in the process.
For the shift-left approach it is important that stakeholders are convinced and bought into this concept
System Integration Testing
Testing the integration of systems and packages; testing interfaces to external organizations (e.g., Electronic Data Interchange, Internet).
System integration testing focuses on testing the interfaces of the system under test and other systems and external services . System integration testing requires suitable test environments preferably similar to the operational environment.
System Testing
Testing an integrated system to verify that it meets specified requirements.
System testing focuses on the overall behavior and capabilities of an entire system or product, often including functional testing of end-to-end tasks and the non-functional testing of quality characteristics. For some non-functional quality characteristics, it is preferable to test them on a complete system in a representative test environment (e.g., usability). Using simulations of sub-systems is also possible. System testing may be performed by an independent test team, and is related to specifications for the system.
Test Level
A group of test activities that are organized and managed together. A test level is linked to the responsibilities in a project. Examples of test levels are component test, integration test, system test and acceptance test.
Test levels are groups of test activities that are organized and managed together. Each test level is an instance of the test process, performed in relation to software at a given stage of development, from individual components to complete systems or, where applicable, systems of systems.
Test levels are related to other activities within the SDLC. In sequential SDLC models, the test levels are often defined such that the exit criteria of one level are part of the entry criteria for the next level. In some iterative models, this may not apply. Development activities may span through multiple test levels. Test levels may overlap in time.
Test types are groups of test activities related to specific quality characteristics and most of those test activities can be performed at every test level.
2.2.1. Test Levels
In this syllabus, the following five test levels are described:
* Component testing (also known as unit testing)
* Component integration testing (also known as unit integration testing)
* System testing
* System integration testing
* Acceptance testing
Test levels are distinguished by the following non-exhaustive list of attributes, to avoid overlapping of test activities:
* Test object
* Test objectives
* Test basis
* Defects and failures
* Approach and responsibilities
Test Object
The component or system to be tested.
Test Type
A group of test activities aimed at testing a component or system focused on a specific test objective, i.e. functional test, usability test, regression test etc. A test type may take place on one or more test levels or test phases.
White-Box (Testing / Technique / Test Design Technique / Test Technique)
Procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system.
It is structure-based and derives tests from the system’s implementation or internal structure (e.g., code, architecture, work flows, and data flows). The main objective of white-box testing is to cover the underlying structure by the tests to the acceptable level.
(Also known as structure-based techniques) it is based on an analysis of the test object’s internal structure and processing. As the test cases are dependent on how the software is designed, they can only be created after the design or implementation of the test object.
