Introduction Flashcards
What is Software Testing?
Software testing is an investigation conducted to provide stakeholders with information about the quality (the degree to which a product meets customer expectations) of the product or service under test.
Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation.
Software testing can be stated as the process of validating and verifying that a computer program/ application/ product: meets the requirements that guided its design and development works as expected can be implemented with the same characteristics satisfies the needs of stakeholders.
What does Poppendieck’s theory tell us about the development cycle and the need for testing…
Poppendieck’s estimate that each handoff leaves
behind approximately 50% of the knowledge we intend to transfer
What is Positive Testing?
Does the software meet the requirements ?
Do the requirements meet the users needs ?
What is Negative Testing?
Finding defects in code and requirements
Trying to break the system
Demonstrating it does not do what its not supposed to do
What two features of a product must be assessed by testing?
Quality
The degree to which a component, system or process meets the specified requirements and/or user/customer needs and expectations.
Risk
A factor that could give negative consequences; usually expressed as impact and likelihood
What is the Test Basis?
All documents from which the requirements of a component or system can beinferred. The documentation on which the test cases are based. If a document can be amended only by way of formal amendment procedure, then the test basis is called a frozentest basis.
What is a Test Condition?
An item or event of a component or system that could be verified by one or more test cases, e.g. a function, transaction, feature, quality attribute, or structural element.
What is a Test Case?
A set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement.
What are the 5 Main activities in the Testing process?
Test Planning and Control Test Analysis and Design Test Implementation and Execution. Evaluating Exit Criteria and Reporting (can loop back to any previous stage and move down through again) Test Closure
What is Test Planning?
The activity of defining the objectives of testing and the testing activities
What is Test Control?
The on-going activity of comparing progress against the plan and reporting status
What tasks are involved in the Test Analysis and Design
phase?
Reviewing the test basis
Evaluating the testability of the test basis
Identifying and prioritising test conditions
Designing and prioritising high level test cases
Identifying necessary test data
Designing the test environment set-up
Identifying any required infrastructure and tools
Creating bi-directional traceability
Test cases to test basis
Test basis to test cases
What tasks are involved in the Test Implementation and Execution phase?
Finalising and prioritising test cases
Finalising the test data
Developing and prioritising test procedures
Creating the test data
Preparing test harnesses
Writing automated test scripts
Creating test suites from the test procedures for efficiency
Verifying the test environment has been set up correctly
Verifying traceability between test basis and test cases
Executing test procedures in planned sequence
Manually or using test execution tools
Logging the outcome of test execution
Comparing actual results with expected results
Recording discrepancies as incidents
Analysing incidents for causes
Repeating test activities (with any necessary regression)
What tasks are involved in the Evaluating Exit Criteria and Reporting phase?
Checking logs against exit criteria specified in Test Planning
Assessing if more tests are required
Assessing if exit criteria need changing
Writing a test summary for stakeholders
What tasks are involved in the Test Closure phase?
Checking planned deliverables have been delivered
Closing incident reports
Raising change records for remaining open incidents
Documenting the acceptance of the system
Finalising and archiving test-ware and test environment
Analysing lessons learned
Using information gathered to improve test maturity (retrospective)
What are the Seven Principles of Testing?
Testing shows the presence of defects Exhaustive testing is impossible Early testing Defect Clustering Pesticide paradox Testing is context dependent Absence-of-errors fallacy
Explain the principle: Testing shows the presence of defects?
Testing can show the defects are present, but cannot prove that there are no defects. Even after testing the application or product thoroughly we cannot say that the product is 100% defect free.
Testing always reduces the number of undiscovered defects remaining in the software but even if no defects are found, it is not a proof of correctness.
Explain the principle: Exhaustive testing is impossible?
Testing everything including all combinations of inputs and preconditions is not possible. So, instead of doing exhaustive testing we can use risks and priorities to focus testing efforts.
For example: In an application in one screen there are 15 input fields, each having 5 possible values, then to test all the valid combinations you would need 30,517,578,125 (515) tests. This is very unlikely that the project timescales would allow for this number of tests. So, accessing and managing risk is one of the most important activities and reason for testing in any project.
Explain the principle: Early testing?
In the software development life cycle testing activities should start as early as possible and should be focused on defined objectives.
Explain the principle: Defect Clustering?
A small number of modules contains most of the defects discovered during pre-release testing or shows the most operational failures.
Explain the principle: Pesticide paradox?
If the same kinds of tests are repeated again and again, eventually the same set of test cases will no longer be able to find any new bugs. To overcome this “Pesticide Paradox”, it is really very important to review the test cases regularly and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.
Explain the principle: Testing is context dependent?
Testing is basically context dependent. Different kinds of sites are tested differently. For example, safety – critical software is tested differently from a free mobile app.
Explain the principle: Absence-of-errors fallacy?
If the system built is unusable and does not fulfil the user’s needs and expectations then finding and fixing defects does not help.
What are the differences between the mindset of a Developer vs a Tester?
Developer:
The software is of acceptable quality and will work
Tend to look for what works
Thinks testers are too negative and pedantic
Tester:
The software contains defects and won’t work
Tend to look for what does not work
Thinks developers are too optimistic
What attributes are present in an effective Testing Team?
Promote collaboration not battles
Remind all of common goal
Communicate failures found in a neutral fact-focused way
Do not criticise the author
Understand how the other person feels
Try to understand why they react the way they do
Confirm the other person understands what you have said and vice-versa
What makes a good Tester?
Curiosity Professional pessimism A critical eye Attention to detail Good communicator Experience on which to base error guessing
What is Verification
Confirmation by examination and through the provision of objective evidence that specified requirements have been fulfilled [ISO 9000]
“Are we building the product right?”
What is Validation (fit for purpose)
Confirmation by examination and through the provision of objective evidence that the requirements for a specific intended use or application have been fulfilled [ISO 9000]
“Are we building the right product?”
Define Component Testing…
Searches for defects in and verifies the functioning of software modules, programs, objects, classes etc. that are separately testable. Usually takes place during the coding phase.
What is TDD, Test Driven Design?
Test-driven design (TDD) is a development technique where you must first write a test that fails before you write new functional code. TDD is being quickly adopted by agile software developers for development of application source code
Define Integration Testing…
Testing performed to expose defects in the interfaces and in the interactions between integrated components or systems
Component Integration Testing -
performed to expose defects in the interfaces and interaction between integrated components.
System Integration Testing -
Testing the integration of systems and packages; testing interfaces to external organisations.
Explain two Integration Testing Strategies
Incremental (recommended)
One interface at a time
Can be Top down, Bottom Up or Functional task
Big Bang (to be avoided) All interfaces at the same time
What is a Driver?
A software component or test tool that replaces a component that takes care of the control and/or the calling of a component or system.
What is a Stub?
A skeletal or special-purpose implementation of a software component, used to develop or test a component that calls or is otherwise dependent on it. It replaces a called component.
What is a Test Harness?
A test environment comprised of stubs and drivers needed to execute a test.
What is System Testing?
Concerned with the behaviour of the whole system/product. The Test Environment should correspond to the final target environment.
System testing should investigate both Functional and Non Functional requirements of the system.
What is Functional System Testing?
Functional Testing – Testing of function, starts with appropriate specification-based (black box) techniques for the aspect of the system to be tested. E.g. Decision table.
Functional Testing also assess Security &Interoperability
(The ability of the software product to interact with one or more specified components or systems)
What s Non-Functional System Testing?
Non functional - Testing of non-functional software characteristics, Structure based techniques (white box) – may then be used to assess the thoroughness of the testing with respect to a structural element e.g. a menu structure or web page navigation.
What is Acceptance Testing?
Users testing the software
Often the responsibility of the users or customers of the system. Builds confidence / assesses readiness.
Acceptance Testing: What is UAT (User Acceptance Testing)?
Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorised entity to determine whether or not to accept the system.
Acceptance Testing: What is OAT : operational acceptance testing?
Operational testing in the acceptance test phase,
typically performed in a (simulated) operational environment by operations and/or systems administration staff focusing on operational aspects, e.g. recoverability, resource-behaviour, and technical compliance.
Acceptance Testing: What is CAT (Contract Acceptance Testing) and Regulation Acceptance Test?
In contract acceptance testing, a system is tested against acceptance criteria as documented in a contract, before the system is accepted.
In regulation acceptance testing, a system is tested to ensure it meets governmental, legal and safety standards.
Acceptance Testing: What is Alpha Testing?
Simulated or actual operational testing by potential users/customers or an independent test team at the developers’ site, but outside the development organisation.
Acceptance Testing: What is Beta Testing?
Operational testing by potential and/or existing users/customers at an external site not otherwise involved with the developers, to determine whether or not a component or system satisfies the user/customer needs and fits within the business processes. Beta testing is often employed as a form of external acceptance testing for off-the-shelf software in order to acquire feedback from the market.
What are the Four Types of Testing?
Functional - Testing of function
Non functional - Testing of non-functional software characteristics
Structure - Testing of Software Structure
Change related - Testing related to changes in software (Re-testing, Regression testing)
What is Black Box Testing?
Functional Testing concerns the external behaviour of the system (Black Box). This method is named because the software program or system, in the eyes of the tester, is like a black box; inside which one cannot see. Black Box Testing method is applicable to all levels of the software testing process:
Non-Functional Testing: Define Performance Testing?
Determine how fast some aspect of a system performs under a particular workload.
Purposes -
- demonstrate that the system meets performance
criteria.
- compare two systems to find which performs
better.
- measure what part of the system or workload
causes the system to perform badly.
Non-Functional Testing: Define Load Testing?
- Conducted to understand the behaviour of the application under a specific expected load.
- Performed to determine a system’s behaviour under both normal and at peak conditions. It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation.
- Involves simulating real-life user load for the
target application. It helps you determine how your application behaves when multiple users hit it simultaneously.
Non-Functional Testing: Define Stress Testing?
Involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results.
It is a form of software testing that is used to determine the stability of a given system.
Non-Functional Testing: Define Usability Testing?
– How easy it is to use the software.
– How easy it is to learn the software.
– How convenient is the software to end user.
Usability testing includes the following components:
Learnability, Efficiency, Memorability, Errors, Satisfaction
Non-Functional Testing: Define Maintainability Testing?
How easy it is to maintain the system. This means that how easy it is to analyse, change and test the application or product.
Specified in terms of the effort required to effect a change in :
Corrective maintenance
Perfective maintenance
Adaptive maintenance
Preventive maintenance
Non-Functional Testing: Define Reliability Testing?
Exercising an application so that failures are discovered and removed before the system is deployed.
Software Reliability is the probability of failure-free software operation for a specified period of time in a specified environment. Includes maturity (robustness), fault-tolerance, recoverability and compliance.
Non-Functional Testing: Define Portability Testing?
The ease to which a software product can be transferred from one hardware or software environment to another.
