ISTQB-TAE Learning Objectives Flashcards
ALTA-E-1.1.1 (K2) Explain the objectives of test automation.
OBJECTIVES of test automation:
1 - Improving test efficiency
2 - Providing wider function coverage
3 - Reducing total test cost
4 - Performing tests that manual testers cannot
5 - Shortening the test execution period
6 - Increasing the test frequency/reducing time required for test cycles
ALTA-E-1.2.1 (K2) Identify technical success factors of a test automation project.
(1) TEST AUTOMATION ARCHITECTURE (TAA)
- that it’s closely aligned with that of the SUT and is maintainable, performant, and learnable.
(2) SUT TESTABILITY
- SUT should be made to be as testable by automation as possible and the most testable parts of the SUT should be targeted first as they will be more successfully covered.
(3) TEST AUTOMATION STRATEGY
- strategy must address maintainability and consistency of the SUT. Consider costs, benefits, risks of applying automation to different parts of the code (e.g., old vs new modules)
(4) TEST AUTOMATION FRAMEWORK (TAF) Focus on: - (a) Implementing reporting facilities - (b) Enable easy troubleshooting - (c) Address the test environment appropriately - (d) Document the automated test cases - (e) Trace the automated test - (f) Enable easy maintenance - (g) Keep the tests up to date - (h) Plan for deployment - (i) Retire tests as needed - (j) Monitor and restore the SUT
ALTA-E-2.1.1 (K4) Analyze a system under test to determine the appropriate automation solution.
SAMPLE QUESTION:
You are deciding on the functional test automation approach for a highly complex system which is expected to be used in production for many years. You have already conducted a survey with the major tool vendors and you have determined that you will not be able to use any of their tools to create your test automation due to the non-standard interfaces that were required for the software. You and the developers have decided that your best automation approach will be through custom interfaces implemented specifically for testing. What are two concerns you should have with this approach? (Pick
2)
a. If the interfaces are not disabled before release, there could be a security risk
b. The performance of the test interfaces may be much faster than the performance of the real interfaces
c. The effort spent to develop test automation will not be justified by the expected lifespan of the code in production
d. Because there is a high level of intrusion, there may be false alarms generated
e. Because there is a low level of intrusion, the test results will not be representative of the production code
CORRECT ANSWER: A and D
A is noted in section 3.2.3 of the syllabus where it talks about checking for security risks if you leave test interfaces in place.
B is not correct because we are testing the code, not the performance.
C is not correct because it is stated in the question that this is a long-lived, complex product.
D is correct because developing custom test interfaces is considered a high level of intrusion and this may generate false alarms due to the different code being exercised.
E is not correct because the custom interfaces introduce a high level of intrusion.
ALTA-E-2.2.1 (K4) Analyze test automation tools for a given project and report technical findings and recommendations.
SAMPLE QUESTION:
You are evaluating functional test automation tools. The tool that you prefer can provide your organization with a favorable cost-benefit ratio, which appeals to senior management. Additionally, you feel the tool is superior in meeting your other technical criteria. However, the tool has many features most of which will never be used. As a result, you feel this makes the tool overly complex and confusing. What should your next steps be regarding selection of this tool?
a. Consider another tool that is more user friendly
b. Explore the possibility of configuring the tool to turn off unnecessary features
c. Plan to supplement selection of this tool with lengthy and comprehensive training sessions
d. Consider acquiring an additional tool that can provide a more user friendly interface to the preferred tool
CORRECT ANSWER: B
A is incorrect; the preferred tool meets your criteria, finding another tool may be difficult.
B is correct; tools are often configurable and allow functionality to be hidden.
C is incorrect; training sessions take away from testing time and the need for training would continue for the life of the tool, which would be an expensive alternative.
D is incorrect; additional tool costs, configuration, and maintenance would make this an unacceptable alternative.
ALTA-E-2.3.1 (K2) Understand “design for testability” and “design for test automation” methods applicable to the SUT.
COMPONENTS OF DESIGN FOR TESTABILITY
- (1) OBSERVABILITY - SUT needs to provide interfaces that give insight into the system that test cases can observe.
- (2) CONTROL (-ability) - SUT needs to provide interfaces that can be used to perform actions.
- (3) CLEARLY DEFINED ARCHITECTURE - Architecture must provide clear and understandable interfaces giving control and visibility on all test levels.
CONSIDERATIONS OF DESIGN FOR AUTOMATION
- (1) Compatibility with existing test tools should be established early on.
- (2) The issue of test tool compatibility is critical in that it may impact the ability to automate tests of important functionality (e.g., incompatibility with a grid controller prevents all tests using that control).
- (3) Solutions may require development of program code and calls to APIs.
ALTA-E-3.1.1 (K2) Explain the structure of the gTAA.
(1) TEST GENERATION LAYER
- Provides the means to design test cases by manual or automated means.
(2) TEST DEFINITION LAYER
- Supports the definition and implementation of test suites and / or test cases. It contains the means to define high-level and low-level tests, which are handled in the test data, test cases, test procedures, and test library components or combinations thereof.
(3) TEST EXECUTION LAYER
- Provides a test execution tool to execute the selected tests automatically and a logging and reporting component.
(4) TEST ADAPTATION LAYER
- Provides different adaptors for connecting to the SUT via interfaces, APIs, protocols, services, and others.
(5) It will also have interfaces for project management, config management, and test management.
ALTA-E-3.2.1 (K4) Design the appropriate TAA for a given project.
SAMPLE QUESTION:
You are working on the design of a test automation project. The gTAA was set up by a consulting
company prior to your being hired. You are now looking to create the TAA from the gTAA. You have
the following requirements that must be addressed by your TAA:
- The TAA must support independence across technologies. It is likely that this same test suite will be used in different test environments and on different target technologies
- The test artifacts need to be portable
- Vendor-neutrality is preferable
- It is vitally important that the TAA be maintainable and maintenance costs be minimized
- It is preferred that less technical people be able to maintain the system even though highly technical people have built it
- There is a large project budget for this effort for the next two years, but the budget will decrease after that time
Given these requirements, which one of the following should the TAE consider during the implementation of the TAA?
a. The communication protocols used by the TAS to interact with the SUT
b. The number of automated test cases to be maintained by the system
c. The test roles that will be supported by the implementation
d. The use of abstraction in the implementation
CORRECT ANSWER: D
A, B, and C are incorrect; while these are certainly considerations for the implementation of the TAA, they will not help address the stated goals.
D is correct. Abstraction will likely be needed to accomplish the stated goals.
ALTA-E-3.2.2 (K2) Explain the role that layers play within a TAA.
(1) TEST GENERATION LAYER
TOOL SUPPORT FOR:
o Manually design test cases,
o Develop, capture, or derive test data,
o Automatically generate test cases from models that define the SUT and/or its environment (i.e., automated model-based testing).
USED TO:
o Edit and navigate test suite structures
o Relate test cases to test objectives or SUT requirements
o Document the test design
MAY ALSO INCLUDE:
o Ability to model the SUT, its environment, and/or the test system
o Ability to define test directives and to configure/parameterize test generation algorithms
o Ability to trace the generated tests back to the model (elements)
(2) TEST DEFINITION LAYER
TOOL SUPPORT FOR:
o Specifying test cases (high and/or low level)
o Defining test data for low-level test cases
o Specifying test procedures for a test case or a set of test cases
o Defining test scripts for the execution of the test cases
o Providing access to test libraries as needed (e.g., keyword-driven approaches).
USED TO:
o Partition/constrain, parameterize, or instantiate test data
o Specify test sequences of fully-fledged test behaviours (including control statements and expressions), to parameterize and/or to group them
o Document the test data, test cases, and/or test procedures
(3) TEST EXECUTION LAYER
TOOL SUPPORT FOR:
o Executing test cases automatically
o Logging the test case executions
o Reporting the test results
MAY PROVIDE CAPATILITIES TO:
o Setup and tear down the SUT for test execution
o Setup and tear down test suites
o Configure and parameterize the test setup
o Interpret both test data and test cases and transform them into executable scripts.
o Instrument the test system and/or the SUT for (filtered) logging of test execution and/or for fault injection
o Analyze the SUT responses during test execution to steer subsequent runs
o Validate the SUT responses (comparison of expected and actual results) for automated test case execution results.
o Control the automated test execution in time
(4) TEST ADAPTATION LAYER
TOOL SUPPORT FOR:
o Controlling the test harness
o Interacting with the SUT
o Monitoring the SUT
o Simulating or emulating the SUT environment
PROVIDES FUNCTIONALITY FOR:
o Medicating between the technology-neutral test definitions and the specific technology requirements of the SUT and the test devices
o Applying different technology-specific adaptors to interact with the SUT
o Distributing the test execution across multiple test devices / test interfaces or executing tests locally.
ALTA-E-3.2.3 (K2) Understand design considerations for a TAA.
(1) Capture requirements needed to define an appropriate TAA
- Which activity/phase of the process, which test level, which type of test, which test roles, which product lines/families, which SUT tech will be supported by the TAA?
(2) Compare and contrast different design/architecture approaches
- (see notes on this chapter for a list of approaches for the different layers)
(3) Identify areas where abstraction can deliver benefits
- Abstraction makes TAA more flexible but comes at the cost of larger initial investment, which may delay initial breakeven but may pay off big time in the long run.
(4) Understand SUT technologies and how these interconnect with the TAS
- what are the interfaces with the SUT and what is their paradigm (event-driven, client-server, peer-to-peer, etc).
(5) Understand the SUT environment
(6) Time and Complexity estimation for a given testware architecture implementaion
(7) Ease of use for a given testware architecture implementation
ALTA-E-3.2.4 (K4) Analyze factors of implementation, use, and maintenance requirements for a given TAS.
SAMPLE QUESTION:
You are working on a test automation project that will automate business scenarios for the user acceptance testers to use when conducting UAT. The business scenarios are well-defined and frequently repeated during UAT. The goal is to also be able to test these same scenarios as part of regression testing by executing the test automation.
Structured scripting has already been used to develop the library of functions that can be used by the test automation. What scripting technique should be used to build upon the structured scripting that has already been done and to implement this test automation to meet the business goals?
a. Scenario-driven scripting
b. Keyword-driven scripting
c. Process-driven scripting
d. Linear scripting
CORRECT ANSWER: C
A is incorrect; this is not a real scripting type.
B is incorrect; this could feed into the process-driven scripting, but C is more complete.
C is correct. This will allow building on the structured scripting that has already been done and the libraries that have been created while automating the business processes.
D is incorrect; it is referring to a more basic type of scripting and it has already been specified that structured scripting has been done.
ALTA-E-3.3.1 (K3) Apply components of the generic TAA (gTAA) to construct a purpose-built TAA.
SAMPLE QUESTION:
You are using a gTAA to create a TAS for a project. You have reviewed the project and determined that the test automation should be focused at the command line level rather than the user interface level due to the rapid and continual changes occurring in the user interface. The command line interface allows you to access all the functionality and will be a part of the released project. Given this information, what part of the standard gTAA can you leave out of the TAS?
a. The test data component of the test definition layer
b. The GUI component of the test adaptation layer
c. The test model component of the test generation layer
d. The user interface component of the test execution layer
CORRECT ANSWER: B
A is incorrect; you are still likely to need test data for your tests.
B is correct.
C is incorrect; there is nothing in the question that indicates this is not needed;
D is incorrect; it is not a component of the test execution layer.
ALTA-E-3.3.2 (K2) Explain the factors to be considered when identifying reusability of components.
- Reusable artifacts could include:
o (parts of) test models of test goals, scenarios, components, or data
o (parts of) test cases, data, procedures, or libraries themselves
o The test engine and/or test report framework
o The adaptors to the SUT components and/or interfaces - TAA definition will settle reuse aspects, but TAS can help this along by
o Following TAA (triggering revision of the TAA where needed)
o Documenting TAS artifacts to be easily understood and incorporated into new contexts
o Ensuring correctness of artifacts so their high quality supports usage in new contexts
ALTA-E-4.1.1 (K3) Apply guidelines that support effective test tool pilot and deployment activities.
SAMPLE QUESTION:
Senior management wants to implement test automation into your organization and has asked you to lead this initiative. The organization has a number of systems of varying criticality, some of which may
benefit from automation once a TAA is mature. You have been given direction to start with a pilot. How do you select a suitable project for the TAS?
a. A high visibility project in order to highlight the success of the pilot
b. A non-critical project in order to mitigate any delays the TAS may cause
c. A project that is simple and easy to automate
d. A new project that is immature and still in development
CORRECT ANSWER: B
A is incorrect; this may result in great risk to the automation project if it fails.
B is correct; the pilot should have minimal impact upon normal business but should be conducted with a realistic project.
C is incorrect; a trivial project will be of little benefit.
D is incorrect; new projects are not good candidates for automation.
ALTA-E-4.2.1 (K4) Analyze deployment risks and identify technical issues that could lead to failure of the test automation project, and plan mitigation strategies.
SAMPLE QUESTION:
Your team has been working on creating a strong and maintainable TAS. The TAS is expected to be used for at least five years, so good maintainability is critical. The team has done the following:
- Created an impact analysis process for all proposed changes to the system
- Documented the usage for the TAS
- Documented the third party dependencies, including contacts within the third party organization
- Verified that the TAS runs in an environment separate from the SUT environment
Given this information, what is a major factor of maintainability that has not been addressed?
a. The TAS must be modular, so key components can be replaced as needed
b. The TAS must be a copy of the gTAA
c. The SUT must reside in the same environment as the TAS
d. The TAS must unite the test scripts with the TAF
CORRECT ANSWER: A
A is correct. The TAS must be modular for maintainability.
B is incorrect; the TAS is an instantiation of the gTAA.
C is incorrect; the two must reside in separate environments.
D is incorrect; the TAS must separate the scripts from the TAF.
ALTA-E-4.3.1 (K2) Understand which factors support and affect TAS maintainability.
SAMPLE QUESTION:
Why is it important to have standard naming conventions for a TAS?
a. A TAS that uses standard names will enable faster execution of the test automation
b. A TAS that uses standard names will be easier for a new person to learn
c. A TAS that uses standard names will support global substitutions when test automation
standards change
d. A TAS that uses standard names will allow separation of the test scripts from the TAF
CORRECT ANSWER: B
B is correct; standard names will be easier for a new person to learn and will make the code easier to
understand.
A, C and D are incorrect; using a standard naming convention won’t affect any of these areas.
