SQA Fundamentals Flashcards
1
Q
So what happens to all the traditional software testing methods, types and artifacts? Do we throw them away?
A
Naaah! You will still need all those software testing methods, types and artifacts (but at varying degrees of priority and necessity). You will, however, need to completely throw away that traditional attitude and embrace the agile attitude.
2
Q
Test case
A
A test case is a set of conditions or variables under which a tester will determine whether a system under test satisfies requirements or works correctly.
The process of developing test cases can also help find problems in the requirements or design of an application.
3
Q
Regression testing
A
Regression testing is a type of software testing that intends to ensure that changes (enhancements or defect fixes) to the software have not adversely affected it.
The likelihood of any code change impacting functionalities that are not directly associated with the code is always there and it is essential that regression testing is conducted to make sure that fixing one thing has not broken another thing.
During regression testing, new test cases are not created but previously created test cases are re-executed.
4
Q
Security
A
The extent of protection of software against unauthorized access, invasion of privacy, theft, loss of data, etc.
5
Q
Ad hoc testing
A
Ad hoc Testing, also known as Random Testing or Monkey Testing, is a method of software testing without any planning and documentation. The tests are conducted informally and randomly without any formal expected results.
The tester improvises the steps and arbitrarily executes them (like a monkey typing while dancing). Though defects found using this method are more difficult to reproduce (since there are no written test cases), sometimes very interesting defects are found which would never have been found if written test cases existed and were strictly followed.
6
Q
STLC
A
SOFTWARE TESTING LIFE CYCLE (STLC) defines the steps / stages / phases in testing of software. However, there is no fixed standard STLC in the world and it basically varies as per the following:
- Software Development Life Cycle
- Whims of the Management
- Nevertheless, Software Testing Life Cycle, in general, comprises of the following phases:
- Requirements/design review
- Test planning
- Test designing
- Test environment setup
- Test execution
- Test reporting
Note that the STLC phases mentioned above do not necessarily have to be in the order listed; some phases can sometimes run in parallel. For instance, Test Designing and Test Execution. Interestingly, no matter how well-defined a Software Testing Life Cycle you have in your project or organization, there are chances that you will invariably witness the following widely-popular cycle:
- Testing
- Cursing
7
Q
Literal definition of regression
A
the act of going back to a previous place or state; return or reversion.
8
Q
Disadvantages of functional testing
A
- It has a potential of missing logical errors in software.
- It has a high possibility of redundant testing.
NOTE: Functional testing is more effective when the test conditions are created directly from user/business requirements. When test conditions are created from the system documentation (system requirements/ design documents), the defects in that documentation will not be detected through testing and this may be the cause of end-users’ wrath when they finally use the software.
9
Q
At which level(s) is gray box testing applicable?
A
Though Gray Box Testing method may be used in other levels of testing, it is primarily useful in Integration Testing.
10
Q
When is acceptance testing performed?
A
Acceptance Testing is the fourth and last level of software testing performed after System Testing and before making the system available for actual use.
11
Q
System testing analogy
A
During the process of manufacturing a ballpoint pen, the cap, the body, the tail, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. When the complete pen is integrated, System Testing is performed.
12
Q
How to report a defect effectively
A
It is essential that you report defects effectively so that time and effort is not unnecessarily wasted in trying to understand and reproduce the defect. Here are some guidelines:
-Be specific:
Specify the exact action: Do not say something like ‘Select ButtonB’. Do you mean ‘Click ButtonB’ or ‘Press ALT+B’ or ‘Focus on ButtonB and click ENTER’? Of course, if the defect can be arrived at by using all the three ways, it’s okay to use a generic term as ‘Select’ but bear in mind that you might just get the fix for the ‘Click ButtonB’ scenario. [Note: This might be a highly unlikely example but it is hoped that the message is clear.]
In case of multiple paths, mention the exact path you followed: Do not say something like “If you do ‘A and X’ or ‘B and Y’ or ‘C and Z’, you get D.” Understanding all the paths at once will be difficult. Instead, say “Do ‘A and X’ and you get D.” You can, of course, mention elsewhere in the report that “D can also be got if you do ‘B and Y’ or ‘C and Z’.”
Do not use vague pronouns: Do not say something like “In ApplicationA, open X, Y, and Z, and then close it.” What does the ‘it’ stand for? ‘Z’ or, ‘Y’, or ‘X’ or ‘ApplicationA’?”
-Be detailed:
Provide more information (not less). In other words, do not be lazy. Developers may or may not use all the information you provide but they sure do not want to beg you for any information you have missed.
-Be objective:
Do not make subjective statements like “This is a lousy application” or “You fixed it real bad.”
Stick to the facts and avoid the emotions.
-Reproduce the defect:
Do not be impatient and file a defect report as soon as you uncover a defect. Replicate it at least once more to be sure. (If you cannot replicate it again, try recalling the exact test condition and keep trying. However, if you cannot replicate it again after many trials, finally submit the report for further investigation, stating that you are unable to reproduce the defect anymore and providing any evidence of the defect if you had gathered. )
-Review the report:
Do not hit ‘Submit’ as soon as you write the report. Review it at least once. Remove any typos.
13
Q
Definition of Quality (IEEE)
A
- The degree to which a system, component, or process meets specified requirements.
- The degree to which a system, component, or process meets customer or user needs or expectations.
14
Q
Functionality
A
The ability of software to carry out the functions as specified or desired.
15
Q
Portability
A
The ability of software to be transferred easily from one location to another.
16
Q
White box testing example
A
A tester, usually a developer as well, studies the implementation code of a certain field on a webpage, determines all legal (valid and invalid) AND illegal inputs and verifies the outputs against the expected outcomes, which is also determined by studying the implementation code.
White Box Testing is like the work of a mechanic who examines the engine to see why the car is not moving.
17
Q
Bug classifications
A
- Severity / Impact (See Defect Severity)
- Probability / Visibility (See Defect Probability)
- Priority / Urgency (See Defect Priority)
- Related Dimension of Quality (See Dimensions of Quality)
- Related Module / Component
- Phase Detected
- Phase Injected
NOTE: We prefer the term ‘Defect’ over the term ‘Bug’ because ‘Defect’ is more comprehensive.
18
Q
Cost of Quality
A
Cost of Quality (COQ) is a measure that quantifies the cost of control/conformance and the cost of failure of control/non-conformance. In other words, it sums up the costs related to prevention and detection of defects and the costs due to occurrences of defects.
Definition by ISTQB: cost of quality: The total costs incurred on quality activities and issues and often split into prevention costs, appraisal costs, internal failure costs and external failure costs.
Definition by QAI: Money spent beyond expected production costs (labor, materials, equipment) to ensure that the product the customer receives is a quality (defect free) product. The Cost of Quality includes prevention, appraisal, and correction or repair costs.
19
Q
Integration testing approaches
A
- Big Bang is an approach to Integration Testing where all or most of the units are combined together and tested at one go. This approach is taken when the testing team receives the entire software in a bundle. So what is the difference between Big Bang Integration Testing and System Testing? Well, the former tests only the interactions between the units while the latter tests the entire system.
- Top Down is an approach to Integration Testing where top level units are tested first and lower level units are tested step by step after that. This approach is taken when top down development approach is followed. Test Stubs are needed to simulate lower level units which may not be available during the initial phases.
- Bottom Up is an approach to Integration Testing where bottom level units are tested first and upper level units step by step after that. This approach is taken when bottom up development approach is followed. Test Drivers are needed to simulate higher level units which may not be available during the initial phases.
- Sandwich/Hybrid is an approach to Integration Testing which is a combination of Top Down and Bottom Up approaches.
20
Q
Defect detection efficiency
A
Defect Detection Efficiency (DDE) is the number of defects detected during a phase/stage that are injected during that same phase divided by the total number of defects injected during that phase.
The phase a defect is ‘injected’ in is identified by analyzing the defects [For instance, a defect can be detected in System Testing phase but the cause of the defect can be due to wrong design. Hence, the injected phase for that defect is Design phase.]
21
Q
Security testing
A
Security Testing is a type of software testing that intends to uncover vulnerabilities of the system and determine that its data and resources are protected from possible intruders.
22
Q
Bebugging
A
The process of intentionally injecting bugs in a software program, to estimate test coverage by monitoring the detection of those bugs
23
Q
Gray box testing example
A
An example of Gray Box Testing would be when the codes for two units/ modules are studied (White Box Testing method) for designing test cases and actual tests are conducted using the exposed interfaces (Black Box Testing method).
24
Q
Who performs unit testing?
A
It is normally performed by software developers themselves or their peers. In rare cases it may also be performed by independent software testers.
25
White box testing
(also known as Clear Box Testing, Open Box Testing, Glass Box Testing, Transparent Box Testing, Code-Based Testing or Structural Testing) is a software testing method in which the internal structure/ design/ implementation of the item being tested is known to the tester. The tester chooses inputs to exercise paths through the code and determines the appropriate outputs. Programming know-how and the implementation knowledge is essential. White box testing is testing beyond the user interface and into the nitty-gritty of a system.
This method is named so because the software program, in the eyes of the tester, is like a white/ transparent box; inside which one clearly sees.
Testing based on an analysis of the internal structure of the component or
system.
26
Defect age calculation
Defect Age in Time = Defect Fix Date (OR Current Date) – Defect Detection Date
Normally, average age of all defects is calculated.
27
Uses of defect density
- For comparing the relative number of defects in various software components so that high-risk components can be identified and resources focused towards them.
- For comparing software/products so that quality of each software/product can be quantified and resources focused towards those with low quality.
28
Boundary value analysis
It is a software test design technique that involves determination of boundaries for input values and selecting values that are at the boundaries and just inside/ outside of the boundaries as test data.
29
Acceptance testing
ACCEPTANCE TESTING is a level of software testing where a system is tested for acceptability. The purpose of this test is to evaluate the system’s compliance with the business requirements and assess whether it is acceptable for delivery.
Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.
30
Performance testing
Performance Testing is a type of software testing that intends to determine how a system performs in terms of responsiveness and stability under a certain load.
31
System testing
SYSTEM TESTING is a level of software testing where a complete and integrated software is tested. The purpose of this test is to evaluate the system’s compliance with the specified requirements.
The process of testing an integrated system to verify that it meets specified requirements.
32
Defect age
Defect Age can be measured in terms of any of the following:
Time
Phases
DEFECT AGE (IN TIME)
Definition
Defect Age (in Time) is the difference in time between the date a defect is detected and the current date (if the defect is still open) or the date the defect was fixed (if the defect is already fixed).
Dropped defects are not counted.
'fixed’ means that the defect is verified and closed; not just ‘completed’ by the developer.
33
Unit testing
UNIT TESTING is a level of software testing where individual units/ components of a software are tested. The purpose is to validate that each unit of the software performs as designed. A unit is the smallest testable part of software. It usually has one or a few inputs and usually a single output. In procedural programming a unit may be an individual program, function, procedure, etc. In object-oriented programming, the smallest unit is a method, which may belong to a base/ super class, abstract class or derived/ child class. (Some treat a module of an application as a unit. This is to be discouraged as there will probably be many individual units within that module.) Unit testing frameworks, drivers, stubs, and mock/ fake objects are used to assist in unit testing.
34
Equivalence partitioning
It is a software test design technique that involves dividing input values into valid and invalid partitions and selecting representative values from each partition as test data.
35
Cost of Quality Calucation
Cost of Quality (COQ) = Cost of Control + Cost of Failure of Control
where
Cost of Control = Prevention Cost + Appraisal Cost
and
Cost of Failure of Control = Internal Failure Cost + External Failure Cost
36
Test plan
- test plan: A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst other items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice, and any risks requiring contingency planning. It is a record of the test planning process.
- master test plan: A test plan that typically addresses multiple test levels.
- phase test plan: A test plan that typically addresses one test phase.
37
Different levels of testing from most granular to broadest
Unit -> Integration -> System -> Acceptance
38
Maintainability
The ease with which software can be modified (adding features, enhancing features, fixing bugs, etc)
39
Guidelines for implementing a defect lifecyccle
- Make sure the entire team understands what each defect status exactly means. Also, make sure the defect life cycle is documented.
- Ensure that each individual clearly understands his/her responsibility as regards each defect.
- Ensure that enough detail is entered in each status change. For example, do not simply DROP a defect but provide a reason for doing so.
- If a defect tracking tool is being used, avoid entertaining any ‘defect related requests’ without an appropriate change in the status of the defect in the tool. Do not let anybody take shortcuts. Or else, you will never be able to get up-to-date defect metrics for analysis.
40
Localizability
The ability of software to be used in different languages, time zones etc.
41
Concurrency
The ability of software to service multiple requests to the same resources at the same time.
42
Types of standards for compliance testing
- Internal standards could be standards set by the company itself. For example, a web application development company might set the standard that all webpages must be responsive.
- External standards could be standards set outside of the company. For example, Health Insurance Portability and Accountability Act (HIPAA) has set regulations for the healthcare industry.
- Compliance testing could also be done by an external organization. This normally results in some sort of compliance certification.
- The method and type of testing to be conducted during compliance testing depends on the specific regulation / standard being assessed.
- The depth of compliance testing could range from a high-level audit on a sampling basis to a detailed scrutiny of each specified standard.
43
Scalability
The measure of software’s ability to increase or decrease in performance in response to changes in software’s processing demands.
44
Defect life cycle
DEFECT LIFE CYCLE (Bug Life cycle) is the journey of a defect from its identification to its closure. The Life Cycle varies from organization to organization and is governed by the software testing process the organization or project follows and/or the Defect tracking tool being used.
It could be something like NEW - ASSIGNED - DEFERRED - DROPPED - COMPLETED - REASSIGNED CLOSED
Note on deferred: If a valid NEW or ASSIGNED defect is decided to be fixed in upcoming releases instead of the current release it is DEFERRED. This defect is ASSIGNED when the time comes.
45
Test case format
Normally a test management tool is used by companies and the format is determined by the tool used, but in general it should contain:
- Test suite ID
- Test case ID
- Test case summary
- Related requirement
- Prerequisites
- Test procedure (step by step process)
- Test data (or links to the data) to be used in the test
- Expected result
- Actual result
- Status (Pass or Fail. Other statuses can be ‘Not Executed’ if testing is not performed and ‘Blocked’ if testing is blocked.)
- Comments
- Created by
- Date of creation
- Executed by
- Date of execution
- Test environment (The environment (Hardware/Software/Network) in which the test was executed.)
46
Tasks involved in acceptance testing
Acceptance Test Plan
- Prepare
- Review
- Rework
- Baseline
Acceptance Test Cases/Checklist
- Prepare
- Review
- Rework
- Baseline
Acceptance Test
-Perform
47
Compliance testing
Compliance Testing [also known as conformance testing, regulation testing, standards testing] is a type of testing to determine the compliance of a system with internal or external standards.
48
Accessibility
The degree to which software can be used comfortably by a wide variety of people, including those who require assistive technologies like screen magnifiers or voice recognition.
49
Which level(s) of testing is regression testing applicable to?
Regression testing can be performed during any level of testing (Unit, Integration, System, or Acceptance) but it is mostly relevant during System Testing
50
Defect (bug) report template
In most companies, a defect reporting tool is used and the elements of a report can vary. However, in general, a defect report can consist of the following elements:
```
Unique defect ID
Project name
Product name
Release version
Module
Detected build version
Summary
Description
Steps to replicate
Actual result
Expected results
Attachments (screenshots, logs)
Remarks
Defect severity
Defect priority
Reported by
Assigned to
Status
Fixed build version
```
51
Types of load testing
- Load Testing is a type of performance testing conducted to evaluate the behavior of a system at increasing workload.
- Stress Testing a type of performance testing conducted to evaluate the behavior of a system at or beyond the limits of its anticipated workload.
- Endurance Testing is a type of performance testing conducted to evaluate the behavior of a system when a significant workload is given continuously.
- Spike Testing is a type of performance testing conducted to evaluate the behavior of a system when the load is suddenly and substantially increased.
52
Methods for integration testing
Any of Black Box Testing, White Box Testing, and Gray Box Testing methods can be used. Normally, the method depends on your definition of ‘unit’.
53
Defect density formula
Defect density = Number of defects / Size
54
Test automation frameworks
There are also many Test Automation Tools/Frameworks that generate the test scripts for you; without the need for actual coding. Many of these tools have their own scripting languages (some of them based on a core scripting languages). For example, Sikuli, a GUI automation tool, uses Sikuli Script which is based on Python. A test script can be as simple as the one below:
55
Defect severity classifications
- Critical: The defect affects critical functionality or critical data. It does not have a workaround. Example: Unsuccessful installation, complete failure of a feature.
- Major: The defect affects major functionality or major data. It has a workaround but is not obvious and is difficult. Example: A feature is not functional from one module but the task is doable if 10 complicated indirect steps are followed in another module/s.
- Minor: The defect affects minor functionality or non-critical data. It has an easy workaround. Example: A minor feature that is not functional in one module but the same task is easily doable from another module.
- Trivial: The defect does not affect functionality or data. It does not even need a workaround. It does not impact productivity or efficiency. It is merely an inconvenience. Example: Petty layout discrepancies, spelling/grammatical errors.
```
Severity is also denoted as:
S1 = Critical
S2 = Major
S3 = Minor
S4 = Trivial
```
56
At what levels is white box testing applicable?
- Unit Testing: For testing paths within a unit.
- Integration Testing: For testing paths between units.
- System Testing: For testing paths between subsystems.
57
Advantages of smoke testing
- It exposes integration issues.
- It uncovers problems early.
- It provides some level of confidence that changes to the software have not adversely affected major areas (the areas covered by smoke testing, of course)
58
Target value of defect detection efficiency
The ultimate target value for Defect Detection Efficiency is 100% which means that all defects injected during a phase are detected during that same phase and none are transmitted to subsequent phases. [Note: the cost of fixing a defect at a later phase is higher.]
59
How is defect priority determined?
Priority is quite a subjective decision; do not take the categorizations above as authoritative. However, at a high level, priority is determined by considering the following:
- Business need for fixing the defect
- Severity/Impact
- Probability/Visibility
- Available Resources (Developers to fix and Testers to verify the fixes)
- Available Time (Time for fixing, verifying the fixes and performing regression tests after the verification of the fixes)
Defect Priority needs to be managed carefully in order to avoid product instability, especially when there is a large of number of defects.
60
Advantages of black-box testing
- Tests are done from a user’s point of view and will help in exposing discrepancies in the specifications.
- Tester need not know programming languages or how the software has been implemented.
- Tests can be conducted by a body independent from the developers, allowing for an objective perspective and the avoidance of developer-bias.
- Test cases can be designed as soon as the specifications are complete.
61
Usability testing
Usability Testing is a type of testing done from an end-user’s perspective to determine if the system is easily usable.
The purpose of this event is to review the application user interface and other human factors of the application with the people who will be using the application. This is to ensure that the design (layout and sequence, etc.) enables the business functions to be executed as easily and intuitively as possible.
62
Compatibility
The suitability of software for use in different environments like different Operating Systems, Browsers, etc
63
Who performs integration testing?
Either Developers themselves or independent Testers perform Integration Testing.
64
Goals of black-box testing
- Incorrect or missing functions
- Interface errors
- Errors in data structures or external database access
- Behavior or performance errors
- Initialization and termination errors
65
Uses of defect detection efficiency
For measuring the quality of the processes (process efficiency) within software development life cycle; by evaluating the degree to which defects introduced during that phase/stage are eliminated before they are transmitted into subsequent phases/stages.
For identifying the phases in the software development life cycle that are the weakest in terms of quality control and for focusing on them.
66
Verification
The process of evaluating work-products (not the actual final product) of a development phase to determine whether they meet the specified requirements for that phase.
Objective is to ensure that the product is being built according to the requirements and design specifications. In other words, to ensure that work products meet their specified requirements. "Are we building the product right?"
It's done with:
- Reviews
- Walkthroughs
- Inspections
...which evaluate the Plans, Requirement Specs, Design Specs, Code, and Test Cases
67
Integration testing analogy
During the process of manufacturing a ballpoint pen, the cap, the body, the tail and clip, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. For example, whether the cap fits onto the body or not.
68
Reliability
The ability of software to perform a required function under stated conditions for stated period of time without any errors.
69
At which level(s) is/are usability testing applicable?
Usability Testing is normally performed during System Testing and Acceptance Testing levels.
NOTE: Usability Testing is NOT to be confused with User Acceptance Testing or User Interface / Look and Feel Testing.
70
Method(s) for system testing
Usually, Black Box Testing method is used.
71
Gray box testing
Gray Box Testing is a software testing method which is a combination of Black Box Testing method and White Box Testing method. In Black Box Testing, the internal structure of the item being tested is unknown to the tester and in White Box Testing the internal structure in known. In Gray Box Testing, the internal structure is partially known. This involves having access to internal data structures and algorithms for purposes of designing the test cases, but testing at the user, or black-box level.
Gray Box Testing is named so because the software program, in the eyes of the tester is like a gray/ semi-transparent box; inside which one can partially see.
72
When is unit testing performed?
Unit Testing is the first level of software testing and is performed prior to Integration Testing.
73
Acceptance testing analogy
During the process of manufacturing a ballpoint pen, the cap, the body, the tail and clip, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. When the complete pen is integrated, System Testing is performed. Once System Testing is complete, Acceptance Testing is performed so as to confirm that the ballpoint pen is ready to be made available to the end-users.
74
Languages commonly used for automated testing
```
JavaScript
Perl
Python
Ruby
Tcl
Unix Shell Script
VBScript
```
75
Defect (bug) report
After uncovering a defect (bug), testers generate a formal defect report. The purpose of a defect report is to state the problem as clearly as possible so that developers can replicate the defect easily and fix it.
76
What type of testing is used in functional testing?
During functional testing, Black Box Testing technique is used in which the internal logic of the system being tested is not known to the tester.
77
Defect priority
Defect Priority (Bug Priority) indicates the importance or urgency of fixing a defect. Though priority may be initially set by the Software Tester, it is usually finalized by the Project/Product Manager.
Priority can be categorized into the following levels:
- Urgent: Must be fixed in the next build.
- High: Must be fixed in any of the upcoming builds but should be included in the release.
- Medium: May be fixed after the release / in the next release.
- Low: May or may not be fixed at all.
Priority is also denoted as P1 for Urgent, P2 for High and so on.
78
Which levels of software testing is black-box testing applicable to?
- Integration testing
- System testing
- Acceptance testing
79
Agile manifesto (specifically tailored for agile testing)
We are uncovering better ways of testing
software by doing it and helping others do it.
Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.
80
Characteristics of a good test case
- Accurate: Exacts the purpose.
- Economical: No unnecessary steps or words.
- Traceable: Capable of being traced to requirements.
- Repeatable: Can be used to perform the test over and over.
- Reusable: Can be reused if necessary.
81
SDLC
SOFTWARE DEVELOPMENT LIFE CYCLE [SDLC] or Software Development Process, defines the steps / stages / phases in the building of software. The exact life cycle / process varies from one model to the other and there are various kinds of software development models like:
- Waterfall model
- Spiral model
- Iterative and incremental development (like ‘Unified Process’ and ‘Rational Unified Process’)
- Agile development (like ‘Extreme Programming’ and ‘Scrum’)
Models are evolving with time and it is beyond the scope of this particular article to discuss each model. However, each model comprises of all or some of the following phases/ activities/ tasks.
82
Smoke testing
Smoke Testing, also known as “Build Verification Testing”, is a type of software testing that comprises of a non-exhaustive set of tests that aim at ensuring that the most important functions work. The results of this testing is used to decide if a build is stable enough to proceed with further testing.
The term ‘smoke testing’, it is said, came to software testing from a similar type of hardware testing, in which the device passed the test if it did not catch fire (or smoked) the first time it was turned on.
83
Defect propability
Defect Probability (Defect Visibility or Bug Probability or Bug Visibility) indicates the likelihood of a user encountering the defect / bug.
- High: Encountered by all or almost all the users of the feature
- Medium: Encountered by about 50% of the users of the feature
- Low: Encountered by very few or no users of the feature
Defect Probability can also be denoted in percentage (%).
NOTE: The measure of Probability/Visibility is with respect to the usage of a feature and not the overall software. Hence, a bug in a rarely used feature can have a high probability if the bug is easily encountered by users of the feature. Similarly, a bug in a widely used feature can have a low probability if the users rarely detect it.
84
Method(s) for acceptance testing
Usually, Black Box Testing method is used in Acceptance Testing. Testing does not normally follow a strict procedure and is not scripted but is rather ad-hoc.
85
White box testing advantages
- Testing can be commenced at an earlier stage. One need not wait for the GUI to be available.
- Testing is more thorough, with the possibility of covering most paths.
86
Defect severity
DEFECT SEVERITY or Impact is a classification of software defect (bug) to indicate the degree of negative impact on the quality of software.
87
Which part of the software development lifecycle is SQA involved in?
Software Quality Assurance encompasses the entire software development life cycle and the goal is to ensure that the development and/or maintenance processes are continuously improved to produce products that meet specifications/requirements.
88
Steps involved in functional testing
- Identify functions that the software is expected to perform.
- Create input data based on the function’s specifications.
- Determine the output based on the function’s specifications.
- Execute the test case.
- Compare the actual and expected outputs.
89
When is integration testing performed?
Integration Testing is the second level of testing performed after Unit Testing and before System Testing.
90
Usability
The degree of software’s ease of use.
91
Black box testing techniques
- Equivalence partitioning
- Boundary value analysis
- Cause effect graphing
92
Who performs system testing?
Normally, independent Testers perform System Testing.
93
Disadvantages of black-box testing
- Only a small number of possible inputs can be tested and many program paths will be left untested.
- Without clear specifications, which is the situation in many projects, test cases will be difficult to design.
- Tests can be redundant if the software designer/ developer has already run a test case.
- Ever wondered why a soothsayer closes the eyes when foretelling events? So is almost the case in Black Box Testing.
94
Focus areas of security testing
- Network security: This involves looking for vulnerabilities in the network infrastructure (resources and policies).
- System software security: This involves assessing weaknesses in the various software (operating system, database system, and other software) the application depends on.
- Client-side application security: This deals with ensuring that the client (browser or any such tool) cannot be manipulated.
- Server-side application security: This involves making sure that the server code and its technologies are robust enough to fend off any intrusion.
95
Tips for performance testing
- Establish a test environment as close to the production environment as possible.
- Isolate the test environment even from the QA or UAT environment.
- Though there’s no perfect tool for performance testing, research and decide on the tool that best fits your purpose.
- Do not rely on the results of one test. Conduct multiple tests to arrive at an average number. Be wary of any changes to the test environment from one test to the other.
96
What is SQA?
SOFTWARE QUALITY ASSURANCE (SQA) is a set of activities for ensuring quality in software engineering processes (that ultimately result in quality of software products).
97
Test script
A Test Script is a set of instructions (written using a scripting/programming language) that is performed on a system under test to verify that the system performs as expected. Test scripts are used in automated testing.
Sometimes, a set of instructions (written in a human language), used in manual testing, is also called a Test Script but a better term for that would be a Test Case.
98
Important reason to have unit testing
Lets say you have a program comprising of two units and the only test you perform is system testing. [You skip unit and integration testing.] During testing, you find a bug. Now, how will you determine the cause of the problem?
- Is the bug due to an error in unit 1?
- Is the bug due to an error in unit 2?
- Is the bug due to errors in both units?
- Is the bug due to an error in the interface between the units?
- Is the bug due to an error in the test or test case?
Unit testing is often neglected but it is, in fact, the most important level of testing.
99
Testability
The ability of software to be easily tested.
100
Defect density
Defect Density is the number of confirmed defects detected in software/component during a defined period of development/operation divided by the size of the software/component.
The ‘defects’ are:
- confirmed and agreed upon (not just reported).
- Dropped defects are not counted.
- The period might be for one of the following:
- for a duration (say, the first month, the quarter, or the year).
- for each phase of the software life cycle.
- for the whole of the software life cycle.
The size is measured in one of the following:
- Function Points (FP)
- Source Lines of Code
101
Writing good test cases
- As far as possible, write test cases in such a way that you test only one thing at a time. Do not overlap or complicate test cases. Attempt to make your test cases ‘atomic’.
- Ensure that all positive scenarios and negative scenarios are covered.
- Language:
- Write in simple and easy to understand language.
- Use active voice: Do this, do that.
- Use exact and consistent names (of forms, fields, etc).
102
When is system testing performed?
System Testing is the third level of software testing performed after Integration Testing and before Acceptance Testing.
103
The success of ad hod testing depends on
the creativity and tenacity of the tester (and, of course, luck).
104
Validation
The process of evaluating software during or at the end of the development process to determine whether it satisfies specified business requirements.
Objective is to ensure that the product actually meets the user’s needs, and that the specifications were correct in the first place. In other words, to demonstrate that the product fulfills its intended use when placed in its intended environment.
"Are we building the right product?"
It's done with testing.
It is entirely possible that a product passes when verified but fails when validated. This can happen when, say, a product is built as per the specifications but the specifications themselves fail to address the user’s needs.
- Trust but Verify.
- Verify but also Validate.
105
Bug tracking tool
Applications for tracking bugs
106
SQA vs. Software Quality Control (SQC)
SQC is about detecting faults. SQA is about preventing them
107
Integration testing
INTEGRATION TESTING is a level of software testing where individual units are combined and tested as a group. The purpose of this level of testing is to expose faults in the interaction between integrated units. Test drivers and test stubs are used to assist in Integration Testing.
-component integration testing: Testing performed to expose defects in the interfaces and
interaction between integrated components.
-system integration testing: Testing the integration of systems and packages; testing
interfaces to external organizations (e.g. Electronic Data Interchange, Internet).
108
Cause/effect graphing
It is a software test design technique that involves identifying the cases (input conditions) and effects (output conditions), producing a Cause-Effect Graph, and generating test cases accordingly.
109
White box testing disadvantages
- Since tests can be very complex, highly skilled resources are required, with thorough knowledge of programming and implementation.
- Test script maintenance can be a burden if the implementation changes too frequently.
- Since this method of testing is closely tied with the application being testing, tools to cater to every kind of implementation/platform may not be readily available.
110
Who performs acceptance testing?
Internal Acceptance Testing (Also known as Alpha Testing) is performed by members of the organization that developed the software but who are not directly involved in the project (Development or Testing). Usually, it is the members of Product Management, Sales and/or Customer Support.
External Acceptance Testing is performed by people who are not employees of the organization that developed the software.
Customer Acceptance Testing is performed by the customers of the organization that developed the software. They are the ones who asked the organization to develop the software. [This is in the case of the software not being owned by the organization that developed it.]
User Acceptance Testing (Also known as Beta Testing) is performed by the end users of the software. They can be the customers themselves or the customers’ customers.
111
Caution about defect severity
Defect Severity is one of the most common causes of feuds between Testers and Developers. A typical situation is where a Tester classifies the Severity of Defect as Critical or Major but the Developer refuses to accept that: He/she believes that the defect is of Minor or Trivial severity.
Though we have provided you some guidelines in this article on how to interpret each level of severity, this is still a very subjective matter and chances are high that one will not agree with the definition of the other. You can however lessen the chances of differing opinions in your project by discussing (and documenting, if necessary) what each level of severity means and by agreeing to at least some standards (substantiating with examples, if necessary.)
Normally, Testers have the final say on Defect Severity while the Project Management / Product Management /Client has the final say on Defect Priority.
ADVICE: Go easy on this touchy defect dimension and good luck!
112
Notes on COQ
- In its simplest form, COQ can be calculated in terms of effort (hours/days).
- A better approach will be to calculate COQ in terms of money (converting the effort into money and adding any other tangible costs like test environment setup).
- The best approach will be to calculate COQ as a percentage of total cost. This allows for comparison of COQ across projects or companies.
- To ensure impartiality, it is advised that the Cost of Quality of a project/product be calculated and reported by a person external to the core project/product team (Say, someone from the Accounts Department).
- It is desirable to keep the Cost of Quality as low as possible. However, this requires a fine balancing of costs between Cost of Control and Cost of Failure of Control. In general, a higher Cost of Control results in a lower Cost of Failure of Control. But, the law of diminishing returns holds true here as well.
113
Example of a basic security test
- Log into the web application.
- Log out of the web application.
- Click the BACK button of the browser (Check if you are asked to log in again or if you are provided the logged-in application.)
Most types of security testing involve complex steps and out-of-the-box thinking but, sometimes, it is simple tests like the one above that help expose the most severe security risks.
114
Functional testing
Functional Testing is a type of software testing whereby the system is tested against the functional requirements/specifications.
115
In which levels is functional testing performed?
Functional testing is normally performed during the levels of System Testing and Acceptance Testing.
116
Debugging
The process of finding the cause of bugs
117
Efficiency
The ability of software to perform well or achieve a result without wasted energy, resources, effort, time or money.
118
Installability
The ability of software to be installed in a specified environment.
119
Advantages of functional testing
- It simulates actual system usage.
| - It does not make any system structure assumptions.
120
Which levels of testing is smoke testing applicable to?
Smoke testing is normally used in Integration Testing, System Testing and Acceptance Testing levels.
NOTE: Do not consider smoke testing to be a substitute of functional/regression testing.
121
Performance
The speed at which software performs under a particular load.
122
Impact analysis
In an ideal case, a full regression test is desirable but oftentimes there are time/resource constraints. In such cases, it is essential to do an impact analysis of the changes to identify areas of the software that have the highest probability of being affected by the change and that have the highest impact to users in case of malfunction and focus testing around those areas.
Due to the scale and importance of regression testing, more and more companies and projects are adopting regression test automation tools.
123
Quality and integrity
* Whistleblower for serious/safety-related things
| * If not serious, voice your concern, stand your ground
124
Latest iOS version
11.2.6 (Feb 19th)
125
watchOS
4.2.3 (Feb 19th)
126
tvOS
11.2.6 (Feb 19th)
127
Xcode
9.2 (Dec 4th)
128
macOS
macOS High Sierra 10.13.3 (Feb 19th)
129
How to prioritize
Bin things based on urgency and importance (2 separate characteristics)
* Highly urgent AND important = first
* High importance, low urgency = carve out time for steady progress
* Low importance, high urgency = look for other ways of getting things done (someone else)
* Low/low = ask if it even needs to be done
130
Major goal of OTA/Update QA
Never brick a device.
131
First step of creating a test plan
Understand the scope!
- Process flow/state-diagrams:
* Different states of the system, how does it transition between them?
* Map out the ecosystem. Then come up with a Matrix.
Are the tests quick and dirty or a complex, significant task?
Figure out how many samples you’ll need, how much time, the complexity.
-Sometimes give a date for a date (I’ll know when this test will be done in 3 days)
132
How to be successful at Apple
Success in this role is all about critical thinking, knowing what you know and what you don’t—knowing who the experts are, and diplomatically driving different teams in bringing a product to life. With realms of expertise and complexity involved in any project, a QA Engineer plays a critical role in fostering healthy debate. You must listen to myriad ideas and criticisms, challenge from a place of expertise, and then commit to the best path for the product. You must be “exponential”—by learning and leveraging the skills and passions of the people you work with, and serving as mentor when needed.
133
Association diagrams
* Apple makes things as simple as possible to the end user but it’s actually really complicated. Draw out all the associations, see “what if this connection is broken… etc.”
134
iOS architecture
1. Core OS Layer:
The Core OS layer holds the low level features that most other technologies are built upon.
- Core Bluetooth Framework.
- Accelerate Framework.
- External Accessory Framework.
- Security Services framework.
- Local Authentication framework.
- 64-Bit support from IOS7 supports the 64 bit app development and enables the application to run faster.
2. Core Services Layer:
Some of the Important Frameworks available in the core services layers are detailed:
- Address book framework – Gives programmatic access to a contacts database of user.
- Cloud Kit framework – Gives a medium for moving data between your app and iCloud.
- Core data Framework – Technology for managing the data model of a Model View Controller app.
- Core Foundation framework – Interfaces that gives fundamental data management and service features for iOS apps.
- Core Location framework – Gives location and heading information to apps.
- Core Motion Framework – Access all motion based data available on a device. Using this core motion framework -Accelerometer based information can be accessed.
- Foundation Framework – Objective C covering too many of the features found in the Core Foundation framework
- Healthkit framework – New framework for handling health-related information of user
- Homekit framework – New framework for talking with and controlling connected devices in a user’s home.
- Social framework – Simple interface for accessing the user’s social media accounts.
- StoreKit framework – Gives support for the buying of content and services from inside your iOS apps, a feature known asIn-App Purchase.
3. Media Layer: Graphics, Audio and Video technology is enabled using the Media Layer.
Graphics Framework:
- UIKit Graphics – It describes high level support for designing images and also used for animating the content of your views.
- Core Graphics framework – It is the native drawing engine for iOS apps and gives support for custom 2D vector and image based rendering.
- Core Animation – It is an initial technology that optimizes the animation experience of your apps.
- Core Images – gives advanced support for controling video and motionless images in a nondestructive way
- OpenGl ES and GLKit – manages advanced 2D and 3D rendering by hardware accelerated interfaces
- Metal – It permits very high performance for your sophisticated graphics rendering and computation works. It offers very low overhead access to the A7 GPU.
Audio Framework:
Media Player Framework – It is a high level framework which gives simple use to a user’s iTunes library and support for playing playlists.
AV Foundation – It is an Objective C interface for handling the recording and playback of audio and video.
OpenAL – is an industry standard technology for providing audio.
Video Framework
AV Kit – framework gives a collection of easy to use interfaces for presenting video.
AV Foundation – gives advanced video playback and recording capability.
Core Media – framework describes the low level interfaces and data types for operating media.
4. Cocoa Touch Layer
- EventKit framework – gives view controllers for showing the standard system interfaces for seeing and altering calendar related events
- GameKit Framework – implements support for Game Center which allows users share their game related information online
- iAd Framework – allows you deliver banner-based advertisements from your app.
- MapKit Framework – gives a scrollable map that you can include into your user interface of app.
- PushKitFramework – provides registration support for VoIP apps.
- Twitter Framework – supports a UI for generating tweets and support for creating URLs to access the Twitter service.
- UIKit Framework – gives vital infrastructure for applying graphical, event-driven apps in iOS. Some of the Important functions of UI Kit framework:
- Multitasking support.
– Basic app management and infrastructure.
– User interface management
– Support for Touch and Motion event.
– Cut, copy and paste support and many more.
135
How to update iPhone/iPad
Wirelessly:
1. Plug device into power and connect to Internet with Wi-Fi
2. Settings > General > Software Update
3. Tap Download and Install. If a message asks to temporarily remove apps because iOS needs more space for the update, tap Continue or Cancel. Later, iOS will reinstall apps that it removed. If you tap Cancel
4. Choose to update now (Install), or Later and choose Install Tonight or Remind Me Later. If you choose Install Tonight, just plug in your device before you go to sleep.
Using iTunes:
1. Install the latest version of iTunes on your computer.
2. Connect your device to your computer.
3. Open iTunes and select your device.
4. Click Summary, then click Check for Update.
136
How to update Apple Watch
Before beginning:
1. Update your iPhone to the latest version of iOS.
2. Make sure that your Apple Watch is on its charger and is at least 50 percent charged.
3. Connect your iPhone to Wi-Fi.
4. Keep your iPhone next to your Apple Watch, so that they're in range.
Then:
1. Keep your Apple Watch on its charger until the update completes.
2. On your iPhone, open the Apple Watch app, then tap the My Watch tab.
3. Tap General > Software Update.
4. Download the update. If asked for your iPhone passcode or Apple Watch passcode, enter it.
5. Wait for the progress wheel to appear on your Apple Watch. It could take from several minutes to an hour for the update to complete.
If you installed an iOS or watchOS beta or developer seed:
1. Before you update, remove the beta profile from your device. To view and remove profiles:
- Open the Apple Watch app on your iPhone, tap the My Watch tab, then go to General > Profiles. Tap the beta profile that you want to remove, then tap Delete Profile. Enter your iPhone passcode if requested.
- Open the Settings app on your iPhone, then tap General > Profiles & Device Management. Tap the beta profile that you want to remove, then tap Delete profile. Enter your iPhone passcode if requested.
2. After you remove the profiles, restart both devices and check for the update again.
To continue using the public beta of iOS, you can install the beta profile again.
137
Troubleshooting Apple Watch update issue
1. Make sure that your Apple Watch connects to its charger.
2. Restart your Apple Watch. Press and hold the side button until you see Power Off, then drag the slider. To turn your watch on again, press and hold the side button.
3. Restart your paired iPhone. Press and hold the Sleep/Wake button until the red slider appears, then drag the slider. To turn your iPhone on again, press and hold the Sleep/Wake button.
4. Try to start the update again.
If the update won't start, open the Apple Watch app on your iPhone, tap General > Usage > Software Update, then delete the update file. After you delete the file, try to download and install watchOS again.
138
How to update HomePod
By default, HomePod automatically installs new software updates, but you can manually check for updates in the Home app. Here's how:
1. On your iOS device, open the Home app. Make sure that your iOS device is signed in with the Apple ID that you used to set up HomePod.
2. In the upper-left corner, tap .
3. Tap Software Update.
4. If there's a new update, tap Install. This will install the update on all of your HomePod speakers you have set up in your home.
139
Variables involved in iOS updates
- Device category (iPhone, iPad, Watch, HomePod, AirPods)
- Device model (iPhone 6, 7, etc...)
- Current iOS version on device
- Battery level
- Internet connection/speed
- Power connectino
- Distance between Watch and iOS device
- Whether or not user has a Beta profile on their device
- Method of updating (USB, OTA, OTA through other device like Watch through phone)
- Time of day
- Install Now vs Tonight vs Remind me Later
- Storage space (and then, Cancel? Remove apps? Remove apps manually and retry?)
- Interruptions/negative actions (remove from power, lose internet connection,
- If connected via USB, is it direct to computer's USB port or to an accessory (monitor, USB hub, keyboard...)
140
Program increment for ART
default is 10 weeks, shortest is 8 weeks and never seen shorter than 8 because they come too fast, longer is 12 weeks and longer than 12 have too much variability built in plan may be by the time you synchronize again you might have too much to adapt.
141
How many members in each team in an ART?
5 to 9
142
Scrum roles
- Product owner - Product Owner Defines and accepts stories. He act as an customer for developer questions and works with product management to plan releases.
- Scrum master - Scrum master runs the team meetings to drive the agile behaviour. He removes all the impediments from the team member, protects the team from outside influence and He attends Scrum of Scrum meetings.
- Developers/Testers - Developers and Testers are the Backbones for the Agile Teams. They create and refine user stories and acceptance criteria. They Define, Build, Test and Deliver the user stories. They commit to the PI Objectives and Sprint plans.
143
Release train engineer
The Release Train Engineer(RTE) facilitates the Agile Release Train for process and program execution, He collects all the impediments, manages the Risks and helps for continuous improvements of program-level. He is also responsible to facilitate few events such as Release planning, Inspect & Adapt and Scrum of Scrum.
144
Product manager
Product Manager acts as a Content authority for the Agile Release Train. He is responsible for defining and prioritizing the Program Backlog, To provide the Vision and Roadmap for the Team. He will be working with Product Owners to optimize the Feature delivery for the customers in a series of Releases.
145
Scrum team member essentials
Courage, focus, commitment, respect, openness
146
Naming a problem in Radar
If issue found in REL, start with "REL:". If it's a task, start with "TASK: ".
147
"New Bugs" component in Radar
“New Bugs” is typically used as the catch-all version to submit to for a component or product. It’s then up to that team to disposition
148
States in Radar
- Analyze: anything that’s been created but hasn’t been dispositioned
- Integrate: (SW) I understand what needs to be done to fix this. It’s being worked on. (HW) Creating the CAD and waiting for Proto 2 to happen.
- Verify: Handing off to REL to verify that it is fixed. Added to new firmware build for testing, etc.
149
Who can close the bug in Radar?
either the originator of the bug or the test group/project manager who verify it
150
Can you un-close a bug in Radar?
No
151
Red text at top of Radar
a warning — it means someone outside Apple can view this bug (ODM, contractor, etc.)
152
In order to close a bug in Radar, you must first...
you have to choose classification first
153
Priorities in Radar
1: it absolutely has to get fixed
2: it should get fixed but we understand there are tradeoffs
3: it’s a nice-to-have
154
Milestone in Radar
Next event/fix-by date
155
Adding to diagnosis in Radar
Diagnosis can become very long. Sometimes it’s better to add to description if it’s something you want everyone to see right away when viewing the bug.
156
Radar scripts
Radar web page has a list of scripts people use regularly.
157
Find - program ID (in Radar)
Can paste anything in here. It will parse any relevant numbers from it and give you the associated Radar ID’s
