General Automation Flashcards
What things can you automate?
- Regression test suite
- Smoke / Sanity test suite
- Build deployment
- Test data creation
- Automating behind the GUI like testing of APIs and methods.
When is Automation testing useful?
-
Regression testing: In case of a bug fix or new module implementation, we have to make sure that the already implemented or unchanged functionality is not affected. In this case, we end up running the regression test case multiple times.
For Example: After each change request or bug fix, after each iteration in case of incremental development approach, etc. -
Non-functional Testing: Testing the non-functional aspects of an application.
For Example: Load testing or performance testing, etc are very difficult for humans to track and analyze. - Complex calculation checks or test scenarios that are prone to human errors.
- Repeated execution of the same tests: Sometimes, we have to run the same set of test case for a different set of data or after each build release or on multiple hardware, software or combination of both.
(i) Smoke test cases: Smoke testing is also known as build verification testing. Smoke test cases are run every time when a new build is released to check the health of the build for acceptance to perform testing.
(ii) Regression Test Cases: Regression testing is the testing to ensure that previously developed modules are functioning as expected after a new module is added or a bug is fixed.
Regression test cases are very crucial in incremental software approach where a new functionality is added at each increment phase. In this case, regression testing is performed at each incremental phase.
(iii) Complex Calculation test cases: Test cases which involve some complex calculations to verify a field for an application fall into this category. Complex calculation results are more prone to human errors hence when automated they give accurate results.
(iv) Data-driven test cases: Test cases which have the same set of steps and run multiple times with the change of data are known as data-driven test cases. Automated testing for these kinds of test cases is quick and cost-effective.
(v) Non-functional test cases: Test cases like load tests and performance tests require a simulated environment with multiple users and multiple hardware or software combinations.
What type of test cases you won’t pick up to automate?
Before picking up the test cases to automate, I do check whether the application is stable or not. So based on this, I don’t pickup test cases when the AUT changes frequently and the test cases which I run rarely and run only one time. When I do usability and exploratory testing.
Can you achieve 100% automation?
100% automation would be difficult to achieve because there would be many edge test cases and some cases that are executed seldom. Automating these cases which are not executed that often will not add value to the automated suite.
I don’t pickup test cases when the AUT changes frequently and the test cases which I run rarely and run only one time. When I do usability and exploratory testing.
How to decide the tool that one should use for Automation testing in their projects?
- Understand your project requirements thoroughly and identify the testing scenarios that you want to automate.
- Search for the list of tools that support your project’s requirements.
- Identify your budget for the automation tool. Select the tools within your budget.
- Identify if you already have skilled resources for the tools. If you don’t have the necessary skilled resources then identify the cost for training the existing resources or hiring new resources.
What are the key criteria of tools?
- How easy is it to develop and maintain the scripts for the tool?
- Can a non-technical person also execute the test cases with little training?
- Does the tool support different types of platforms like web, mobile, desktop, etc based on your project requirements?
- Does the tool have a test reporting functionality? If not, is it easily configurable for the tool?
- How is the tool for cross-browser support for web-based applications?
- How many different testing types can this tool support?
- How many languages does the tool support?
Is Automation testing a Black box testing or White-box testing?
Automation testing is mostly a black box testing as we just program the steps that a manual tester performs for application under test without knowing the low-level design or code of the application.
Sometimes, automated test scripts need access to the database details that are used in the application under test or some more coding details and thus can be a type of white-box testing.
Thus automated testing can be both black or white box type of testing depending on the scenarios in which automation is performed.
List some advantages and disadvantages of Manual testing.
Advantages:
No environment setup needed.
Programming knowledge is not required.
Recommended for dynamically changing requirements.
Allow human observation power to detect more bugs.
The cost is less for short-term projects.
Flexibility
Disadvantages:
Difficult to perform complex calculations.
Reusability
Time taking
High risk of human errors or mistakes.
More human resources are required.
List some advantages and disadvantages of Automation testing.
Advantages:
Fewer human resources
Reusability
More Test Coverage in less time
Reliability
Parallel execution of test cases
Fast
Disadvantages:
Development and maintenance time is more.
Tool Cost
Skilled resources are required.
Environment setup
Test Script debugging is an issue.
What are some of the factors that determine the effectiveness of automation testing?
- Time saved by running scripts over the manual execution of test cases.
- Defects found
- Test Coverage or code coverage
- Maintenance time or development time
- Stability of the scripts
- Test Reusability
- Quality of the software under test
