4452 - Testing Midterm Flashcards
1
Q
Quality is the ability of a product or service to fulfill its:
A
Quality Requirements
2
Q
What are the 3 dimensions of quality?
A
Entity, Viewpoint, Attribute
3
Q
An attribute depends on:
A
the entity and viewpoint
4
Q
Process quality is the quality factors used to:
A
develop, operate and maintain software artifacts
5
Q
What is the purpose of ISO 9126?
A
to eliminate misunderstandings relating to quality between customer and supplier
6
Q
What is the key difference between customer and supplier in ISO 9126
A
customer understands and communicates requirements the supplier does as well but also needs to assess with confidence whether the product can be produced with the right level of quality
7
Q
Who evaluates the software without knowing any internal aspects of it?
A
the user
8
Q
What is the difference between an engineer and manager view of quality?
A
engineers care about the final product as well as all intermediate artifacts, managers care more about overall quality
9
Q
What are the quality criteria for functionality?
A
suitability, accuracy, interoperability, security, compliance
10
Q
The presence and appropriateness for a set of functions for a specified tasks is
A
suitability
11
Q
the provision of right or agreed upon results is
A
accuracy
12
Q
the ability to interact with specified systems is
A
interoperability
13
Q
the ability to prevent unauthorized access is
A
security
14
Q
adhering to application related standards or regulations is
A
compliance
15
Q
What are the quality criteria for reliability?
A
maturity, crash frequency, fault tolerance, recoverability
16
Q
The frequency of failure by faults is?
A
maturity
17
Q
what is crash frequency?
A
number of system crashes per unit of time
18
Q
what is fault tolerance?
A
ability to maintain a specified level of performance in case of software faults or infringements
19
Q
what is recoverability
A
ability to re-establish performance and recover lost data
20
Q
What are the sub-factors to the Reliability quality factor?
A
maturity, crack frequency, fault tolerance, recoverability
21
Q
What is understandability?
A
effort required by user to grasp the logical concept and its applicability
22
Q
what is learnability?
A
ability for a user to learn the software (input, output, operational control)
23
Q
what is operability?
A
ability for user to perform operations
24
Q
What are the sub factors of usability?
A
understandability, learnability, operability
25
what is time behavior?
response, processing time and throughput rates
26
what is resource behavior?
amount of resources used and the duration they are used
27
what are the sub factors of efficiency?
time and resource behaviors
28
what is analyzability?
ability to diagnose failures and identify things to by modified
29
what is changeability?
effort needed to actually modify the software
30
what is stability?
encapsulates the risk of unexpected events occurring when software is being modified
31
what is testability?
the ability and effort relating to verifying the modified software
32
what are the sub factors of maintainability?
analyzability, changeability, stability, testability
33
what is adaptability?
ability to be adapted to other environments
34
what is installability?
ability for the software to be installed on systems
35
what is conformance?
a softwares ability to conform to standards surrounding portability
36
what is replaceability?
effort needed to replace the software in its environment
37
what are the sub factors for portability?
adaptability, installability, conformance, replaceability
38
What are the three product states in the McCall model?
operation, revision, transition
39
What quality factors are grouped under product operation in McCalls model?
reliability, correctness, usability, integrity, efficiency
40
What quality factors are grouped under product revision in McCalls model?
Maintainability, flexibility, testability,
41
What quality factors are grouped under product transition in McCalls model?
portability, interoperability, reusability
42
What is completeness?
degree to which full implementation of requirements has been achieved
43
what is consistency?
uniform design and implementation
44
what is traceability?
ability to link software components to its requirements
45
what is accuracy?
precision of computations of outputs
46
what is communication commonality?
degree to which standard protocols and interfaces are used
47
what is access control?
ability to control and protect the software and data
48
what is conciseness?
compactnss of the source code
49
what is access to audit?
ease in which software can be checked for compliance
50
what is data commonality?
standard representation of data
51
what is error tolerance?
degree to which operations can continue under adverse conditiions
52
what is execution efficiency?
the run time efficiency of the software
53
TF: McCall uses a hierarchical framework?
T
54
TF: ISO 9126 uses a hierarchical framework?
T
55
Which quality model reflects the user view?
ISO 9126
56
Which quality model reflects the product view?
McCall
57
What are the high level characteristics in Boehms's quality model?
portability, as-is utility, maintainability
58
what are the three hierarchical levels to Boehms quality model?
high levle, intermediate and primitive characteristics
59
What are the two difference requirements in Furps+ model?
functional and non functional
60
What does FURPS stand for?
functionality, usability, reliability, performance, supportability
61
what does the + in FURPS+ represent?
design, implementation, interface and physical
62
What are the 4 quality factors in SEI model?
performance, dependability, security , safety
63
a tool that makes a quality factor is called what?
a quality metric
64
what is static software measurement derived from?
examination of software artifacts
65
what is dynamic software measurement derived from?
examination of the execution of software
66
what is the difference between direct and indirect software measurement?
direct is when you look at things like time, effort, size which are factual frontline measures, indirect is when you measure based on things derived from direct measures such as failure intensity (failures/time)
67
what is a commonly used metrics when measuring reliability through maturity?
software maturity index
68
what are the 4 important crash frequency metrics?
expected total failures, mean time to failure, failure intensity, crash free (probability of a failure free operation)
69
what metric is the expected number of failure experienced in a time period?
failure intensity
70
the probability of a failure free operation is what metric?
reliability/crash free
71
what must be considered when measuring fault tolerance?
overall system architecture
72
what is the difference between serial and parallel system architectures in terms of fault tolerance?
serial - fails if one component fails
| parallel - fails if all components fail
73
TF: in a serial system architecture the reliability of one component is higher than the reliability of the overall system
T
74
TF: in a parallel system architecture the reliability of one component is higher than the reliability of the overall system
F
75
What are the recoverability metrics?
system recovery time, service degradation rate, time to switch
76
How can accuracy be measured?
problem reports per phase, priority, category
reported problems in a period of time
rate of error disovery
77
reported problems can be classified into what 3 categories?
open, closed, unevaluated
78
How is security measured?
with the security level metric
79
What are the dimensions used to measure analyzability? Provide some example metrics
complexity and readability
- cyclomatic number for the flow of control
- size and complexity of logical and physical statements
- comment rate on code
80
what dimension is used to measure changeability? what are some example metrics?
modularity
- depth of architecture
- fan in fan out
- number of branches, nested levels, shared variables
81
what dimension is used to measure stability? what are some example metrics?
modularity
- fan in fan out
- number of parameters referenced/changed, global variables, called relationships
- depth of architectue
82
what dimensions are used to measure testability? what are some example metrics?
modularity, complexity
- cyclomatic number
- number of call-paths, non-cyclic paths, nested levels of architecture
83
What metric is used to test portability?
Degree of portability
84
When is portability cost effective
if DP > 0
85
TF: Cpdoc, Cptest > Crdoc, Crtest (generally)
False
86
What does it mean if Cmod > Crdesign + Crcode?
a portable design has not been developed
87
What does it mean if Cmod << Crdesign + Crcode?
an effective portable design exists
88
Quality factors tend to be what since they are related to description related to applications, components, or users
domain specific
89
What are the two main approaches to fault tolerance and what is the difference between them?
single approach
- integrating mechanisms into a single unique piece of software
multi approach
- develop different software versions using different techniques
90
What are some typical exceptions handled through tolerating by acceptance?
deadlocking, corrupting data, failure to provide intended functionality
91
what does redundancy enable the software to do?
detect and recover from failures
92
What are the 3 major parts the fault tolerance model?
fault detection, fault recovery, fault correction
93
What are the sub-aspects of fault correction?
diagnosis and repair
94
How are faults detected?
using embedded checks
95
Fault detection can be carried out in which 2 modes?
concurrently (during service delivery) and preemptively (while service is suspended)
96
What are the two techniques to fault recovery?
backward recovery and forward recovery
97
What is the backward recovery technique?
consists of discarding the current state in favor of an earlier state
98
What is the forward recovery technique?
making use of the current corrupted state to construct a recovery state
99
TF: Forward recovery is typically more difficult than backward recovery?
T
100
What are the 3 mechanisms used to record and store system states in favor of an earlier state?
undoing transactions, checkpoint/rollback, degraded service
101
Why is forward recovery generally very difficult to get right?
It is system specific and depends on accurate predictions of the location and cause of errors
102
What is the core concept of multi-version fault tolerance?
redundancy
103
What is the though process behind multi version fault tolerance?
components that are built differently should fail differently and therefore at least one of the versions should deliver an appropriate output
104
What are the two types of redundancy?
serial and parallel
105
How does multi version fault tolerance work?
design multiple components with the same functionality, implement two distinct versions of the same software executing on the same inputs, if there is a discrepancy in the outputs it triggers an error detection
106
What are the two key architectural features that relevant to security?
immunity and resilience
107
What is immunity?
ability to prevent an attempted attack
108
How does software architecture encourage immunity?
minimizing exploitable security weakness, making sure all security features are included in design
109
What is resilience?
a systems ability to recover
110
How does software architecture encourage resilience?
segmentation of functionality to contain an attack, ability to quickly restore functionality
111
Efficiency describes what 2 system constraints
time and capacity
112
What aspects are included in efficiency
throughput, response time, and load
113
Describe throughput
how many processed can be processed per minute
114
Describe response time
how long does it take to process a request
115
Describe load
how many users can be supported before response time and load suffer?
116
What mechanisms can be used to improve response time?
scheduling and caching
117
what are the different scheduling types
round robin, explicit, fifo, earliest deadline
118
What is load balancing?
a way to improve the distribution of workloads across multiple workstations
119
To achieve the quality actor of maintainability a software must be?
easily changed
120
What two ways can a software unit be affected by a software change? What does each one mean?
directly affected - when the responsibilities of the software unit must change
indirectly affected - responsibilities don't change, but implementation must be revisited
121
What are some tactics for minimizing the impact on changeability (and therefore maintainability), of directly affected software units?
clustering anticipate changes
- identify design decisions that likely to be directly affected
- encapsulate them in software units
- ensure that these software units are highly cohesive
- try to confine them in a few units
122
What is the core strategy for reducing the impact on changeability and therefore maintainability, of indirectly affected software units?
reduce dependencies
123
What are ways to reduce dependencies between software units?
low coupling, and designing to the interface (units interact only through their interfaces)
124
How can the MVC framework be applied to achieve high usability?
understandability --> UX information architecture
learnability --> View
operability --> controller
125
what is a program fault?
an incorrect step, process or data definition in a program
126
What are the two fundamental ideas (criteria) for the defect based quality model and how are they measured?
software should be defect free (measured by fault count) and suitable for use (measured with failure reports)
127
a fault __________ an error, and an error _________ failure
activates, propigates
128
where may a fault originate?
design, specification, implementation (programmer mistake)
129
what are the two states of a fault?
active --> produces and error
| dormant --> exists but is not producing an error
130
what is an error?
part of an artifact state that propagates failure
131
what is failure?
an inability to deliver on quality requirements, occurs when artifact is executing
132
what are the 6 fault categories?
creation phase, system boundaries, domain, phenomenological causes, intent, persistence
133
what are the two creation phase fault sub-cats?
operational, developmental
134
what are the two system boundary fault sub-cats?
internal, external
135
what are the domain fault sub-cats?
hardware, software
136
what are the phenomenological fault sub-cats?
human made, natural
137
what are the intent fault sub-cats
malicious (deliberate), accidental (unintentional)
138
what are the persistence fault sub-cats?
permanent, transient
139
what is always required in order to generate a good product?
a good process
140
what is the principal quality determinant for manufactured goods?
the process
141
in design based activities what must be considered?
the process as well as additional factors, like designer skill
142
What happens to the cost of fixing an error as the lifecycle of a product progresses?
it gets more expensive
143
what is the SQM framework?
software quality management and it is the quantitative planning and guiding of the software development
144
what are the 5 steps to SQM?
1. engineer quality factors
2. develop operational profiles
3. Software quality assurance planning
4. Software quality assurance control
5. Apply failure data to guide decisions
145
what is failure defined as under SQM?
the negative drift on the software quality requirements (things that should not be done to them)
146
How are failure severity classes determined under SQM?
failures with the same degree of per-failure impact on a user are given the same class
147
what are the criteria used to assign the per-failure impact of a failure?
cost, environment, human-life, system capability
148
what factors are considered in the system capability criteria?
loss of data, recovery time, total downtime
149
What does FIO represent?
failure intensity
150
in terms of failure severity classes, what should be resolved before proceeding to the operational profile?
any disagreements between stakeholders regarding them
151
what is an operational profile?
the set of operations and their probability of occurrence
152
what does the operational profile reflect?
how the software will be used in practice
153
what are the uses of an operational profile?
proportionally distribute test cases, accurate measurement of reliability since it models real use, for a competitive release strategy since you can implement most used operations in early releases
154
what are the tiers of the operational profile triangle from the top? (CUSFOT)
customer, user, system, function, operation, test selection
155
what is the customer profile?
the total number of customer groups and the associated probability that they will use the software (number of customer in group / total customers)
156
what is the difference between a user and a customer?
a customer acquires the software a user "uses" the software
157
what is a user profile?
the total number of user groups and their associated probabilities
158
what is a user group?
a set of users that use the software in the same way
159
what is a customer group?
a set of customers that acquire the software in the same way
160
Users of a system, a system own internal controller, and external systems are examples of what?
actors in a use case
161
what is the system mode?
a collection of function grouped in a way that makes it easy to analyze their execution behavior
162
what is the best source for identifying system functions?
functional and business requirements
163
tasks in a workflow are modeled as different function if what?
managing their development has different priorities or resource allocation, or if they differ in frequency of use or criticality
164
what are environmental variables?
they describe conditions that do not relate directly to features (paths taken, data accessed)
165
what is an operation (within the context of operational profiles)
a system task as observed by the end user
166
TF: Operational architecture captures how sub systems and modules combine
False: system architecture captures this, operational architecture only captures how a user will invoke operations to complete functions
167
TF: usually there are more operations than functions
True
168
what is a key input variable?
a variable that is a common input for two or more operations whose value differentiates them
169
what is the difference between implicit and explicit representation of key input variables
implicit splits up key input variables into sub profiles with associated probabilities, explicit represents each possible combination of key input variables desperately and assigns a probability to each one
170
what are occurrence rates based on?
field data, system logs, manually collected data, or experience
171
how are occurrence rates calculated
take the individual occurrence rate and divide by total occurrence rate
172
what are the two ways you can estimate a test case and ultimately which one should you chose?
1. using team capacity and cost
2. using historical data or industry benchmarks
you should go with the lower of the two
173
where should the majority of test cases be allocated to?
developed components
174
if acquired components represent a substantial portion of the software and their reliability is questionable what is the general rule for assigning test cases?
20-30% of test cases to these acquired components
175
what should be utilized for the basis of test case allocation?
the operational profile
176
what is the methodology for allocating test cases to operations?
assign one test case to each infrequent new operation, pre-assign test cases to rarely occurring critical operations, distribute remaining test cases among remaining new operations based on occurrence probability
177
what is used to determine whether or not a new operation is infrequent or not?
an occurrence probability threshold
178
what makes a new operation critical?
its high failure severity
179
why do we initially allocate test cases only to "rarely occurring" critical operations?
because common occurring ones will get enough test cases due to their occurrence probability in the next step
180
what is the V&V perspective?
validation and verification
181
what are the 4 dimension of the V&V perspective (brian marricks quadrants)
business facing vs technology facing
| supporting programming vs critiquing product
182
what are the 4 types of testing that make up the 4 quandrants in V&V perspective (brian marrick)
feature testing, user acceptance testing, functional testing, quality factor testing
183
what are software review and inspections
quality control processes for written material
184
what is the difference between a review and inspection
a review is a process or meeting to examine software artifacts where stakeholder can give their input
an inspection is a formal engineering process for detecting defects in software products
185
what does an inspection confirm?
that software presents overall specifications
186
what are the outputs of an inspection?
detailed defect list
defect summary list
estimated rework effort and completion date
187
TF: inspections and testing are opposing verification techniques
False: they are complimentary
188
TF: inspections cannot check non-functional quality requirements
True
189
who introduced testing and for what reason?
some dude names Fagan because he thought that execution testing was not robust enough
190
what are the inspection roles under fagans model? (MART)
moderator, author, reader, tester
191
what should be used to drive the inspection?
a checklist of common defects
